Skip to content

add incremental rebuild orchestrator (PR-T3)#272

Merged
HumanBean17 merged 1 commit into
masterfrom
feat/tier2-pr3-incremental-orchestrator
Jun 6, 2026
Merged

add incremental rebuild orchestrator (PR-T3)#272
HumanBean17 merged 1 commit into
masterfrom
feat/tier2-pr3-incremental-orchestrator

Conversation

@HumanBean17

Copy link
Copy Markdown
Owner

Summary

  • Implements build_ast_graph_incremental() — the core orchestrator for incremental Kuzu graph rebuild (PR-T3 from plans/active/PLAN-TIER2-INCREMENTAL-REBUILD.md)
  • Takes changed file paths, expands to dependency closure via 8 rules using inverted .deps.json maps, deletes old data via symmetric delete helpers (PR-T2), re-runs pass1-5 on dirty subset only, loads remaining non-dirty data from DB, merges, runs pass6 globally, writes incremental data with duplicate-PK avoidance, rewrites all HTTP_CALLS/ASYNC_CALLS fresh
  • Falls back to full rebuild when .deps.json missing, ontology stale, or dirty set >50%
  • Adds --changed-paths CLI flag and last_rebuild_mode GraphMeta field
  • 22 tests: equivalence (6 corpora), fallback (3), closure (8), deps merge (1), pass6 invariant (1), meta stats (1), CLI (2)

Code review fixes (post-implementation)

  • Fixed .deps.json overwrite bug: Was writing only dirty-file entries (via partial), losing all non-dirty deps. Extracted _write_dependency_index_data() to write the properly merged index.
  • Fixed missing DECLARES_PRODUCER load: _load_remaining_from_db was loading DECLARES_CLIENT but not DECLARES_PRODUCER, creating asymmetry in merged data. Added load + merge.

Manual evidence

.venv/bin/ruff check build_ast_graph.py
# All checks passed!

.venv/bin/python -m pytest tests/test_incremental_equivalence.py -v
# 22 passed in ~5min

Out of scope (do NOT touch)

  • PR-T4 (integrate with CLI increment command) — future PR
  • Existing test suite tests — no modifications
  • server.py, kuzu_queries.py, search_lancedb.py — untouched
  • java_ontology.py — untouched

🤖 Generated with Claude Code

@HumanBean17 HumanBean17 force-pushed the feat/tier2-pr3-incremental-orchestrator branch from 7ad9acc to 29fe5dd Compare June 6, 2026 17:58
Implements build_ast_graph_incremental() which re-processes only changed
files and their dependency closure, falling back to full rebuild when
deps.json is missing, ontology is stale, or dirty set exceeds 50%.

Key additions:
- expand_to_closure: 8-rule fixed-point expansion using inverted .deps.json
- pass1-5 subset variants for partial re-processing
- _load_remaining_from_db + _merge_tables for pass6 global rerun
- _write_nodes_incremental with duplicate-PK avoidance
- _write_meta_incremental querying live DB for global stats
- --changed-paths CLI flag and last_rebuild_mode GraphMeta field
- 22 tests: equivalence, fallback, closure, deps merge, pass6 invariant, meta stats, CLI

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@HumanBean17 HumanBean17 force-pushed the feat/tier2-pr3-incremental-orchestrator branch from 29fe5dd to 87dd1bc Compare June 6, 2026 18:15
@HumanBean17 HumanBean17 merged commit f25f63a into master Jun 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant