Commit Graph
726 Commits
Author SHA1 Message Date
Andrew Hundt cc3794deaa fix(registry): reject unreachable qualified fallbacks
When a dotted or namespace-qualified callee has import evidence but no candidate is import-reachable, stop before weak same-simple-name fallback.

This prevents external calls such as email.message.Message or receiver calls such as response.get from resolving to unrelated project symbols while preserving import_map, import_map_suffix, qualified_suffix, and bare-name fallback behavior.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 02:59:22 -04:00
Andrew Hundt 11aad8657f fix(pipeline): drop weak file dotted calls
Suppress Python file-node dotted call edges when the registry result is only a weak short-name guess and the target is not import-reachable.

Keep route-literal handling, strong registry strategies, field-hint materialization, and import-reachable suffix fallbacks intact. Add canaries for the FastAPI-style request.headers.get false edge and the import-reachable preservation case.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 02:45:33 -04:00
Andrew Hundt c0825bb49b fix(pipeline): drop weak file self calls
Suppress Python file-node self.* CALLS edges when the registry result is only a weak short-name match. This targets exact-scratch cases where a missing enclosing function lets an instance-style call fall back to a file-level suffix edge that fresh full indexing does not emit.

Add a focused pipeline canary for self.add_api_route file-node fallback. The normal FastAPI matrix remains guarded to full reindex, so this is a correctness slice toward scoped exact parity, not a default-performance claim.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 02:30:32 -04:00
Andrew Hundt 098583aa37 fix(pipeline): materialize field hint targets
Exact scratch can now resolve field-type-hint CALLS targets through the existing store-backed node lookup instead of requiring every unchanged target to be preloaded into the scratch graph. Parallel and full gbuf paths keep the gbuf-only lookup behavior.

Add a FastAPI-like exact-scratch regression test for request.body -> GzipRequest.body and keep the scoped_lsp_gap production guard in place. The guarded FastAPI matrix remains canonical-equal but full-reindex, so this is a correctness enabler rather than default-performance closure.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 02:21:34 -04:00
Andrew Hundt 53e2bc24b7 fix(pipeline): suppress weak external lsp fallbacks
When LSP resolves a call to an external or unindexed target, keep strong registry evidence such as same-module/import-map matches but drop weak non-import-reachable short-name fallbacks. This prevents false CALLS edges like scope.get -> local *.get while preserving Python re-export/import behavior such as fastapi.Header.

Add a focused pipeline canary for external LSP fallback suppression and keep the existing Python re-export and super().__init__ canaries green. FastAPI matrix remains correctness-safe through the scoped_lsp_gap full-reindex guard, so this is not a default-performance closure.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 02:05:54 -04:00
Andrew Hundt 1bc2d11151 fix(pipeline): keep scoped lsp exact guarded
Reject the unsafe FastAPI scoped-LSP exact publish path while preserving reusable resolver and frontier improvements.

Move the field-type hint resolver into shared pipeline internals so sequential exact and parallel full paths use the same bounded refinement. Avoid importer-frontier expansion for unchanged exports, keeping changed-export expansion covered by tests.

FastAPI matrix evidence showed forced scoped exact could be faster but failed canonical graph equality, including routing.py call/complexity drift. Keep the production scoped_lsp_gap full-reindex guard and do not expose the ineffective scoped-symbol cap as user config.

Validation: pipeline suite 341 passed; parallel suite 29 passed; store_nodes suite 111 passed; source-safety passed; git diff --check passed; product build passed; FastAPI safe-guard matrix passed with canonical_graph.equal=true and publish_kind=full.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 01:48:39 -04:00
Andrew Hundt 9d6688d8e9 fix(pipeline): refine lsp import symbols
Add a shared cross-LSP import-value refinement helper that upgrades module import QNs to exact imported symbol QNs only when the active LSP def set proves module.local_name exists.

Use the helper in both sequential and parallel fused cross-LSP paths. The parallel path keeps ordinary registry import caches unchanged and bounds refinement through the existing module-def filtered subset when available.

Tighten the store-backed scoped exact oracle back to strict fresh-rebuild equality, closing the prior lsp_method versus suffix_match metadata mismatch for the Python receiver fixture.

Validation: rebuilt build/c/test-runner; focused strict oracle passed; CBM_ONLY_SUITE=pipeline passed 341 tests; CBM_ONLY_SUITE=parallel passed 29 tests; source-safety, diff-check, and product build passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 00:52:24 -04:00
Andrew Hundt 4f2f7d4efa test(pipeline): guard scoped lsp exact oracle
Extend the test-only exact scratch helper so a caller can opt into all-file metadata and the store-backed scoped-LSP cap while preserving the existing wrapper for default-off callers.

Add a Python receiver guardrail that proves the store-backed exact scratch can publish the desired provider.Logger.log edge, while recording the current strict oracle gap between lsp_method metadata and the fresh full suffix_match metadata. This keeps scoped-LSP guard relaxation blocked until the oracle path is improved or resolver metadata equivalence is explicitly designed.

Validation: rebuilt build/c/test-runner; focused guardrail passed; CBM_ONLY_SUITE=pipeline passed 341 tests; source-safety and diff-check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 00:39:35 -04:00
Andrew Hundt f1a15960e7 feat(pipeline): feed scoped lsp from store
Add an opt-in store-backed scoped cross-LSP path that builds bounded persisted defs from import QNs, loads property-bearing nodes through the store batch lookup, and refines import values only when the store proves module.local_name is an unchanged supported symbol.

Keep the path default-off behind explicit ctx metadata and scope cap, preserve the existing scoped-LSP fallback guards, and add a pass-level Python canary for resolving Service.run to provider.Logger.log without selecting OtherLogger.log.

Validation: build/c/test-runner rebuilt; focused scoped-LSP canaries passed; CBM_ONLY_SUITE=pipeline passed 340 tests; CBM_ONLY_SUITE=store_nodes passed 111 tests; source-safety, diff-check, and product build passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 00:30:07 -04:00
Andrew Hundt 0d8b279e4b feat(store): list symbol scope qns
Add a bounded store helper that expands imported symbol QNs to exact and dotted-member QNs in project scope, preserving input order and reporting truncation.

Use it in the scoped cross-LSP diagnostic so persisted Python defs are discovered from the imported Logger scope instead of a hand-built member list, while proving OtherLogger remains excluded.

Validation: make -f Makefile.cbm -j16 build/c/test-runner; CBM_ONLY_SUITE=store_nodes; CBM_ONLY_SUITE=pipeline; focused store/pipeline tests; scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-05 00:09:50 -04:00
Andrew Hundt dd199278dc feat(store): batch load nodes by qualified name
Add a bounded property-bearing batch lookup for candidate qualified names using the existing VALUES CTE pattern from node-id batch lookup.

Update the scoped cross-LSP persisted-def diagnostic to use the batch node loader, and cover ordering, duplicates, missing inputs, project scope, and properties preservation.

This is a resolver-context primitive only; it does not relax scoped-LSP fallback guards or claim default incremental readiness.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 23:56:55 -04:00
Andrew Hundt e728a22c52 test(pipeline): prove persisted defs feed scoped lsp
Add an internal helper that rebuilds CBMLSPDef rows from persisted graph nodes by parsing existing node properties and routing through the existing cross-LSP conversion path.

Cover the Python receiver-method case with a focused pipeline test that combines changed-file defs with persisted provider defs and resolves the call through the existing Python LSP cross resolver.

This is a diagnostic foundation only; it does not relax scoped-LSP fallback guards or claim FastAPI/default incremental readiness.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 23:48:34 -04:00
Andrew Hundt 9fc6cf3a5c test(pipeline): guard python scoped receiver overlay
Add a focused regression for Python scoped overlay/exact behavior when receiver-method calls need broader resolver context. The test keeps overlay_publish=small_deltas on, edits one Python file, and asserts the production scoped_lsp_gap path falls back to full reindex, avoids replacement overlay publication, preserves the expected CALLS target, and matches a fresh FAST rebuild.

Also correct stale cross-LSP comments so the code describes the current split between full FAST fused cross-LSP, sequential changed-file cross-LSP, and parallel scoped no-op cross-LSP.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 23:33:28 -04:00
Andrew Hundt 11a37079ff fix(pipeline): align sequential edge parity
Match sequential CALLS property capacity to the parallel pass for edge JSON emitted through calls_emit_edge so changed-file overlay/exact paths preserve later call arguments.

Require THROWS/RAISES source nodes to be callable scopes through a shared pipeline helper, preventing sequential file/module fallback edges from diverging from parallel full-index behavior.

Add regression coverage for the eighth CALLS arg and top-level raise sequential/parallel parity. Validated with focused canaries, full pipeline and parallel suites, source-safety, product build, guarded FastAPI matrix, and an isolated guard-removal experiment that still failed active equality.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 23:20:34 -04:00
Andrew Hundt 05b24d4062 fix(pipeline): force full rebuild for unsupported inbound edges
Treat exact-delta inbound_edges_require_full as a hard fallback instead of allowing the incremental containment path to publish a graph after exact upsert has already reported that unsupported inbound edges require a full rebuild.

Strengthen the focused incremental regression to assert full fallback behavior and compare the result against a fresh FAST rebuild.

Validation: make -f Makefile.cbm -j16 build/c/test-runner; CBM_ONLY_SUITE=pipeline CBM_ONLY_TEST=incremental_fast_mixed_unowned_edge_frontier_falls_back_to_full_rebuild build/c/test-runner; CBM_ONLY_SUITE=pipeline CBM_ONLY_TEST=inbound build/c/test-runner; CBM_ONLY_SUITE=pipeline build/c/test-runner; bash scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 22:31:39 -04:00
Andrew Hundt 1ac3c136f1 fix(pipeline): preserve websocket route registrations
Share the route path/handler scanner between sequential and parallel call passes, add FastAPI/Starlette websocket route-registration suffixes, and let sequential unresolved route-registration suffix calls emit Route registration facts when the existing route-literal guard accepts the path.

Add a sequential/parallel parity canary for FastAPI websocket decorators. Validation: make -f Makefile.cbm -j16 build/c/test-runner; CBM_ONLY_SUITE=parallel build/c/test-runner; CBM_ONLY_SUITE=pipeline CBM_ONLY_TEST=fast_route build/c/test-runner; make -f Makefile.cbm -j16 cbm; bash scripts/check-source-safety.sh; git diff --check.

FastAPI matrix evidence: guarded production remains safe by full fallback in /private/tmp/cbm-fastapi-matrix-websocket-parity-current-20260704T2215.json. Forced exact cap64 now preserves the previously missing tests/test_ws_router websocket Route nodes and 9 route-registration edges, but still fails canonical edge parity; keep scoped-LSP full fallback until the remaining edge-attribution mismatch is fixed.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 22:22:32 -04:00
Andrew Hundt b684bb1f7d test(pipeline): align exact scratch helper
Mirror the production exact-route path-alias and package-map setup in the test-only scratch builder so lower-level exact-delta tests exercise the same resolver inputs.\n\nAdd a Python package exact-delta regression that applies the lower-level scratch delta and compares the result with a fresh FAST rebuild. This proves the small synthetic package is safe while the larger FastAPI scoped-LSP mismatch remains covered by external benchmark artifacts and the production full-reindex guard.\n\nValidation:\n- CBM_ONLY_SUITE=pipeline CBM_ONLY_TEST=incremental_exact_scratch_python_package_matches_fresh_rebuild build/c/test-runner\n- CBM_ONLY_SUITE=pipeline build/c/test-runner\n- bash scripts/check-source-safety.sh\n- git diff --check

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 21:42:23 -04:00
Andrew Hundt 033b5ba480 fix(pipeline): guard scoped exact scratch routes
Route scoped-LSP exact scratch publishes to the existing full-reindex fallback when the scoped planner cannot prove active/full graph equivalence. FastAPI probes showed the no-importer shortcut was faster but produced divergent IMPORTS/CALLS/THROWS edges, so this commit preserves correctness and records the remaining performance blocker.

Also build scratch package maps for overlay/exact delta routes to match full-pipeline resolver context, add opt-in inbound-edge debug logging through cbm_log_debug, and update Python route-decorator tests to expect the guarded full route while retaining fresh-rebuild equality checks.

Validation: focused pipeline suite 335 passed (/private/tmp/cbm-test-pipeline-scoped-gap-kind-fix-20260705T.log); source-safety passed; git diff --check passed; sandbox full suite reached 6391 passed with 13 known HTTP listen failures plus one stale assertion fixed here; escalated HTTP suite 28 passed (/private/tmp/cbm-test-httpd-scoped-gap-kind-fix-20260705T.log). No tag created.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 21:30:43 -04:00
Andrew Hundt 5d6377fc08 perf(pipeline): bound scoped exact frontiers
Add an internal frontier no-op mask for exact file-delta batches so graph-equal frontier files can prove coverage without recursively expanding or being rewritten. Reuse the existing preserved-inbound-edge helper for scoped changed files and keep the old batch planner/apply APIs as wrappers for non-scoped paths.

Focused validation: ASan/UBSan test-runner build passed, pipeline suite passed with 335 tests, source-safety passed, and product build passed. Synthetic python_reexport exact matrix passed. Real FastAPI routing.py still falls back with inbound_edges_require_full, so default-readiness and the broad FastAPI speed blocker remain open.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 20:35:21 -04:00
Andrew Hundt 95762706ac fix(pipeline): guard scoped lsp frontier fallbacks
Prevent unsafe scoped overlay/containment publication for cross-LSP frontier-too-large edits until scoped parity is proven. FastAPI/Python showed active-overlay and containment mismatches for routing.py; this routes that class through the existing full rebuild path, preserving correctness while leaving performance work open.

Also suppress weak suffix fallback for Python super().__init__ when LSP resolved an external base, and recompute scoped complexity derived fields for changed in-scope nodes instead of preserving stale recursive/transitive loop-depth values.

Validation: git diff --check; source-safety; ASan/UBSan test-runner build; CBM_ONLY_SUITE=pipeline 333 passed; production build/sign; isolated FastAPI oracle passed with full fallback and canonical equality.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 19:44:16 -04:00
Andrew Hundt b62bb54e37 feat(pipeline): report exact frontier cap telemetry
Expose when exact incremental frontier telemetry is capped so index_repository callers can distinguish a known affected-path count from a frontier that hit incremental_exact_max_affected_paths. Preserve existing publish routing and response fields while adding affected_paths_limit and affected_paths_truncated only when available.

Also centralize planner fallback reporting and cover oversized inbound-frontier fallback metadata in the pipeline suite.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 18:52:29 -04:00
Andrew Hundt 5095a6f27e feat(mcp): report overlay compaction status
Expose background overlay compaction lifecycle through index_status so long-lived MCP clients can distinguish idle, running, and finished workers without relying on internal join helpers.

Add focused MCP regression coverage for the finished-worker state after background compaction completes, and tighten local project-name snapshots to use checked snprintf handling.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 18:14:57 -04:00
Andrew Hundt dd89b734bf fix(mcp): reap finished overlay compaction workers
Allow long-lived MCP servers to start a later overlay compaction pass after a prior background worker has finished but has not been explicitly joined yet. The start path now reaps one finished worker through the existing join/reset logic, preserves false while a worker is active, and keeps failed reaps on the existing log path only.

Add a regression that uses the existing two-overlay fixture to prove a finished unjoined worker does not block a second compaction start.

Validation: build/c/test-runner rebuild /private/tmp/cbm-build-overlay-reap-20260704T1818.log; CBM_ONLY_SUITE=mcp /private/tmp/cbm-test-mcp-overlay-reap-20260704T1825.log (171 passed); source safety /private/tmp/cbm-source-safety-overlay-reap-20260704T1826.log; diff check /private/tmp/cbm-diff-check-overlay-reap-20260704T1826.log; production build/sign /private/tmp/cbm-prod-build-overlay-reap-20260704T1828.log.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 18:06:09 -04:00
Andrew Hundt 21e3c59b8c fix(cypher): make overlay id policy explicit
Overlay rows intentionally do not expose stable canonical node or edge ids until compaction. Rename the active-read guard around canonical identity, document the id() boundary, and add a Cypher regression proving active overlay rows are used for supported queries but disabled for id() projections.

Validation: CBM_ONLY_SUITE=cypher ./build/c/test-runner; CBM_ONLY_SUITE=mcp ./build/c/test-runner; bash scripts/check-source-safety.sh; git diff --check; make -j8 -f Makefile.cbm cbm.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 17:30:58 -04:00
Andrew Hundt 08979d0b36 fix(cypher): traverse variable paths both ways
Fix canonical and active-overlay variable-length Cypher traversal for undirected relationship patterns. The store BFS string API now maps historical directions onto the existing CBM_STORE_EDGE_DIR constants, so canonical and overlay BFS share the same inbound/outbound/any decision instead of duplicating string checks.

query_graph can now answer MATCH (a)-[:TYPE*min..max]-(b) from ready overlay edges, while id() Cypher queries remain explicitly canonical-only until overlay id semantics or compaction exists.

Validation: red canonical TDD /private/tmp/cbm-test-cypher-any-variable-relationship-before-20260704T-current.log; red MCP TDD /private/tmp/cbm-test-mcp-any-variable-relationship-before-20260704T-current.log; green consolidated suites /private/tmp/cbm-test-cypher-any-variable-relationship-consolidated-suite-20260704T-current.log, /private/tmp/cbm-test-mcp-any-variable-relationship-consolidated-suite-20260704T-current.log, /private/tmp/cbm-test-store-nodes-any-variable-relationship-consolidated-20260704T-current.log; source-safety /private/tmp/cbm-source-safety-any-variable-relationship-consolidated-20260704T-current.log; production build /private/tmp/cbm-prod-build-any-variable-relationship-consolidated-20260704T-current.log.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 17:16:37 -04:00
Andrew Hundt d71c367add feat(cypher): traverse overlay variable paths
Use the existing qn-keyed overlay BFS helper for directed variable-length Cypher relationship patterns in active query_graph reads.

Keep undirected variable-length traversal and id() canonical-only with explicit warnings until those semantics have a separate tested contract.

TDD: /private/tmp/cbm-test-mcp-active-variable-relationship-before-20260704T-current-2.log failed before the production wiring because FreshVarSource was absent.

Validation: /private/tmp/cbm-build-active-variable-relationship-20260704T-current-2.log; /private/tmp/cbm-test-mcp-active-variable-relationship-focused-20260704T-current-2.log; /private/tmp/cbm-test-mcp-active-variable-relationship-suite-20260704T-current-2.log; /private/tmp/cbm-test-cypher-active-variable-relationship-20260704T-current.log; /private/tmp/cbm-test-store-nodes-active-variable-relationship-20260704T-current.log; /private/tmp/cbm-source-safety-active-variable-relationship-20260704T-current.log; /private/tmp/cbm-prod-build-active-variable-relationship-20260704T-current.log.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 17:01:24 -04:00
Andrew Hundt fbedf35c19 feat(cypher): support active one-hop match chains
Extend active overlay Cypher relationship reads across additional MATCH and OPTIONAL MATCH patterns when every relationship pattern is fixed one-hop.

Reuse the qn-keyed active edge-node store helper for terminal-bound expansion and keep OPTIONAL MATCH no-edge rows on the shared preservation path.

Variable-length relationship traversal and id() projections remain canonical-only with caller-visible warnings.

Validation: /private/tmp/cbm-build-active-multipattern-relationship-20260704T-current-3.log; /private/tmp/cbm-test-mcp-active-multipattern-relationship-focused-20260704T-current-2.log; /private/tmp/cbm-test-mcp-active-multipattern-relationship-suite-20260704T-current-2.log; /private/tmp/cbm-test-cypher-active-multipattern-relationship-20260704T-current-2.log; /private/tmp/cbm-test-store-nodes-active-multipattern-relationship-20260704T-current-2.log; /private/tmp/cbm-source-safety-active-multipattern-relationship-20260704T-current-2.log; /private/tmp/cbm-prod-build-active-multipattern-relationship-20260704T-current-2.log.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 16:46:10 -04:00
Andrew Hundt 2207f0ad8f feat(cypher): read overlay one-hop relationships
Add a store-level qn-keyed active edge-node expansion helper and use it in query_graph's active Cypher path for fixed one-hop relationship patterns.

Variable-length, multi-pattern, and id() Cypher remain canonical-only with explicit warnings. This expands AI-facing freshness without widening unsupported relationship semantics.

Validation: build/c/test-runner rebuild; focused MCP active relationship and variable-length canonical tests; full MCP suite 170 passed; full Cypher suite 144 passed; store_nodes 109 passed; source-safety; git diff --check; product build/sign. Logs: /private/tmp/cbm-build-active-cypher-relationship-20260704T-current-2.log, /private/tmp/cbm-test-mcp-active-relationship-query-20260704T-current-2.log, /private/tmp/cbm-test-mcp-variable-relationship-canonical-20260704T-current-2.log, /private/tmp/cbm-test-mcp-active-relationship-suite-20260704T-current-3.log, /private/tmp/cbm-test-cypher-active-relationship-20260704T-current-2.log, /private/tmp/cbm-test-store-nodes-active-relationship-20260704T-current-2.log, /private/tmp/cbm-source-safety-active-relationship-20260704T-current-2.log, /private/tmp/cbm-diff-check-active-relationship-20260704T-current-2.log, /private/tmp/cbm-prod-build-active-relationship-20260704T-current.log.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 16:32:21 -04:00
Andrew Hundt 5917762a16 feat(cypher): use active overlay edges for node predicates
Teach query_graph's active-overlay Cypher path to answer node-only degree properties and simple EXISTS predicates from ready overlay edges instead of forcing those shapes back to canonical rows.

Relationship-pattern Cypher and id() still remain canonical-only until active relationship binding is available, so the change expands overlay freshness without pretending full active relationship traversal exists.

Validation before commit: build/c/test-runner --suite=mcp --filter active_edges_for_degree_and_exists; build/c/test-runner --suite=mcp --filter id_query_canonical; build/c/test-runner --suite=mcp; build/c/test-runner --suite=cypher; build/c/test-runner --suite=store_nodes; scripts/source_safety_check.sh; git diff --check; make -f Makefile.cbm cbm. Logs: /private/tmp/cbm-test-mcp-active-cypher-edge-derived-20260704T-current-2.log, /private/tmp/cbm-test-mcp-id-canonical-after-active-cypher-20260704T-current-2.log, /private/tmp/cbm-test-mcp-active-cypher-suite-20260704T-current-2.log, /private/tmp/cbm-test-cypher-active-edge-derived-20260704T-current-2.log, /private/tmp/cbm-test-store-nodes-active-edge-derived-20260704T-current-2.log, /private/tmp/cbm-source-safety-active-cypher-edge-derived-20260704T-current-2.log, /private/tmp/cbm-diff-check-active-cypher-edge-derived-20260704T-current-2.log, /private/tmp/cbm-prod-build-active-cypher-edge-derived-20260704T-current.log.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 16:04:02 -04:00
Andrew Hundt 0a739e5091 test(pipeline): cover mixed exact stale refresh
Add an end-to-end pipeline regression for a full-mode incremental run that deletes one file and adds another under incremental_derived_refresh=stale_on_exact.

The test verifies exact publish, stale semantic_edges metadata, deleted file cleanup, added file visibility, and absence of synchronous semantic/similarity recomputation. Validation: focused regression, full pipeline suite, and git diff --check.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 15:31:29 -04:00
Andrew Hundt 4ac8c422c7 perf(pipeline): defer global derived refresh for exact deltas
Add an opt-in incremental_derived_refresh=stale_on_exact policy for full and moderate exact incremental publishes. The default remains eager, while the opt-in path reuses existing derived-view staleness semantics so semantic/similarity queries warn until an eager index run or full reindex rebuilds global edges.

The exact path now skips synchronous SIMILAR_TO/SEMANTICALLY_RELATED recomputation only when file-delta publish marks semantic_edges stale. Broad fallback, overlay, delete, rename, and replacement publish behavior remain conservative.

Validation: CBM_ONLY_SUITE=pipeline build/c/test-runner (330 passed); focused derived-refresh/config tests passed; make -f Makefile.cbm lint-source-safety passed; make -j10 -f Makefile.cbm cbm passed; isolated full-mode benchmark measured 40.153 ms incremental vs 415.058 ms fresh full (10.337x).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 14:50:08 -04:00
Andrew Hundt 0e04d50dd0 perf(overlay): publish proven additive header overlays
Allow all-header C-family incremental batches to publish active overlay rows when a store-level subset proof shows the new delta preserves the file-owned nodes and edges already in the canonical graph. Mixed source/header batches still fall back conservatively with additive_subset_required.

Make MCP read paths treat additive overlay rows as ready even without file tombstones, and fix the BM25 active-overlay query to join active_overlay_files. Keep default non-git search_code scanning on the live root when no file_pattern is provided so active unindexed files remain searchable.

Validation: full ASan/UBSan test runner passed with 6393 passed in /private/tmp/cbm-full-after-live-root-fix-20260704T1340.log; lint-source-safety passed in /private/tmp/cbm-source-safety-after-live-root-fix-20260704T1340.log; production build passed in /private/tmp/cbm-prod-build-after-live-root-fix-20260704T1340.log.

Benchmark: self-dogfood store_pipeline_batch over MCP with overlay_publish=small_deltas and rank_refresh=stale_on_exact passed active overlay graph equality and qualitative oracles; incremental overlay 688 ms vs fresh full rebuild 21676 ms, 31.51x speedup, cleanup true, artifact /private/tmp/cbm-store-pipeline-batch-additive-live-root-fix-20260704T1340.json.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 14:33:46 -04:00
Andrew Hundt 52b017f84b fix(pipeline): block unsafe broad header overlays
High-cap diagnostics showed that publishing non-tombstone additive overlay rows for broad C-family header exact frontiers can leave stale canonical edges visible in the active overlay view and can be slower than a full rebuild.

Keep the single-header overlay path separate, but require broad header batches to publish through the exact route until a strict additive-subset proof exists. Update the pipeline canary to require exact publish and fresh-rebuild equivalence for this case.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 13:06:09 -04:00
Andrew Hundt af4022b91f feat(pipeline): publish additive header overlays
Add a pipeline wrapper for non-tombstone overlay publishes and reuse the store active overlay CTE for file-node reads and summary counts. This lets header/type overlay rows refresh active read views without hiding canonical source-owned rows.

Add focused tests for duplicate-QN additive overlay ranking, single header type/implementation additive overlay behavior, and exact-frontier overlay dirty state.

Validation: bash scripts/check-source-safety.sh; bash scripts/test-source-safety.sh; git diff --check; make -f Makefile.cbm -j8 cbm; CBM_ONLY_SUITE=store_nodes build/c/test-runner; CBM_ONLY_SUITE=pipeline build/c/test-runner.

Scope: this is not the store_pipeline_batch default speed fix. Current benchmark evidence still shows safe full fallback around 2.6x, so narrower dependency-slice or active-read algorithm work remains open.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 12:53:55 -04:00
Andrew Hundt 170aa7d830 feat(store): support additive overlay rows
Split active overlay readability from file replacement tombstones. Replacement overlay publishes still hide canonical file rows, while the new additive publish API layers safe overlay facts without tombstoning the file.

Update the active-overlay CTE, node-view summary, and benchmark mirror so additive overlay rows are visible while canonical rows for the same file remain visible.

Add a store canary proving an additive overlay symbol is readable without creating a file tombstone or hiding existing canonical rows.

Validation: uv py_compile benchmark harness passed; ASan/UBSan test-runner build passed; store_nodes suite passed 108; source-safety passed; diff check clean; production build passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 11:52:11 -04:00
Andrew Hundt 320d63a7d2 fix(pipeline): guard unsafe header overlay pairs
Keep the single-header overlay fast path for safe header-only additions, but fall back when a C-family header delta contains type-like nodes and a same-stem implementation file exists.

This prevents active overlay from selecting header-owned type rows where a fresh full rebuild selects richer source-owned type rows, as seen in the store.h/store.c batch diagnostics.

Adds a pipeline canary for header type/implementation pair fallback while preserving the single-header overlay and multi-file header safety canaries.

Validation: focused header overlay tests passed; pipeline suite passed 326; source safety OK; diff check clean; production build passed; one_source_file benchmark stayed incremental_overlay at 29.677x. Full ASan target was also run and was not clean, with obsolete in-run and environment-sensitive failures recorded in the notes.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 11:42:28 -04:00
Andrew Hundt b6cc825941 perf(pipeline): allow single-header overlay publishes
Permit the early overlay publish route for a single changed C-family header while keeping multi-file header batches on the conservative exact/full path. The route is still gated by the active overlay graph equality checks instead of relying on an unchecked fast path.

Add a pipeline canary that edits a header, verifies incremental_overlay publication, confirms canonical rows stay unchanged, and confirms overlay rows become ready. The existing multi-file header safety canary remains in place.

Validation: pipeline suite 325 passed; source-safety OK; diff check clean; production build signed; self-dogfood one_source_file benchmark passed via incremental_overlay with 20.7767x speedup and cleanup complete.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 11:07:58 -04:00
Andrew Hundt 1406f2ee9b fix(pipeline): suppress unresolved route suffix self-calls
Parallel resolve used an unresolved route-registration suffix fallback for callees such as .get and .post. When no route literal was present, that fallback could fall through to a normal CALLS edge using the source node as the target, which fabricated self-calls for ordinary receiver accessors like ec.get(e.type).

Return without emitting when the suffix-only fallback has no route literal, while preserving resolved route API calls and unresolved route registrations that do have a valid route path. Add a parallel resolver canary that first proves extraction sees ec.get and then asserts no self CALLS edge is emitted.

Validation: build/c/test-runner rebuilt; CBM_ONLY_SUITE=parallel CBM_ONLY_TEST=parallel_unresolved_route_suffix_does_not_emit_self_call passed; CBM_ONLY_SUITE=parallel passed 27 tests; lint-source-safety passed; git diff --check passed; make -f Makefile.cbm cbm built and signed; MCP self-dogfood overlay matrix passed 5/5 with cleanup.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 10:52:51 -04:00
Andrew Hundt 47e9a6c6fc fix(pipeline): preserve scoped recursive complexity
Scoped exact and overlay updates run complexity with an incomplete call graph, so they cannot prove that an existing recursive SCC disappeared. Preserve stored recursive=true in scoped complexity mode while full complexity continues to recompute from the current graph.

Adds a focused in-memory pipeline canary for the persisted recursive property. Validated with the focused canary, complexity subset, full pipeline suite, source-safety, diff check, and a production build.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 10:32:08 -04:00
Andrew Hundt ce009783da fix(store): dedupe active overlay edges
Project active overlay edges as logical graph edges instead of per-file overlay ownership rows. Cross-file edges can be owned by multiple changed files in one overlay generation for cleanup and compaction bookkeeping, but AI-facing active views must not inflate degree or schema counts.

Mirror the production active-edge semantics in the self-dogfood benchmark and add a store regression for duplicate cross-file overlay edge ownership.

Validation: CBM_ONLY_SUITE=store_nodes build/c/test-runner; make -f Makefile.cbm cbm; self-dogfood full matrix at /private/tmp/cbm-overlay-active-edge-dedupe-full-self-dogfood-20260704T.json; source-safety at /private/tmp/cbm-overlay-active-edge-dedupe-source-safety-20260704T.log.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 10:10:42 -04:00
Andrew Hundt 793ede109e fix(pipeline): keep header deltas off active overlay
Guard C-family header changes from active overlay publication until active edge parity is proven against fresh rebuilds. Bounded header edits can still use exact base-delta publication, preserving correctness without forcing every safe case to full rebuild.

Add a focused C-header incremental canary that verifies no overlay publish, exact frontier expansion, exact publish stats, and fresh-rebuild equality. Clean up stale direct-frontier helper code from the failed cap64 overlay experiment.

Validation: CBM_ONLY_SUITE=pipeline build/c/test-runner -> 323 passed; make -f Makefile.cbm lint-source-safety -> OK; self-dogfood cap16/cap64 store_pipeline_batch passed canonical equality but still fell back with frontier_too_large at 2.81x/2.74x.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 09:47:04 -04:00
Andrew Hundt f17f2db19a fix(pipeline): preserve overlay graph parity
Serialize call args consistently for call-derived edges across sequential and parallel indexing, including CONFIGURES edges used by MCP graph responses and data-flow derivation.

Add a generic argument-list fallback for call extraction so grammars exposing arguments as argument_list still populate CBMCall.args.

Guard exact delta and overlay publication against cross-file node-QN collisions, falling back instead of letting a changed header override a canonical source definition in the active overlay view.

Validation: build/c test runner and binary rebuild passed; CBM_ONLY_SUITE=pipeline passed; lint-source-safety passed; full self-dogfood MCP matrix passed 5/5 in /private/tmp/cbm-cross-file-qn-guard-full-self-dogfood-20260704T.json.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 07:30:46 -04:00
Andrew Hundt 0aa4e4062b fix(pipeline): preserve file node extension identity
Keep symbol and module FQNs compatible while making File node qualified names injective for same-stem files such as .c/.h pairs.

Add a regression canary for File node FQNs and update pipeline import-map fixtures to use the extension-preserving file identity. Keep the package-map module lookup compatible by trying the legacy exact QN before a deterministic File-node fallback.

Validation: CBM_ONLY_SUITE=fqn build/c/test-runner; CBM_ONLY_SUITE=registry build/c/test-runner; CBM_ONLY_SUITE=pipeline build/c/test-runner; CBM_ONLY_SUITE=mcp build/c/test-runner; make -f Makefile.cbm lint-source-safety.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 06:47:51 -04:00
Andrew Hundt 4e8ef6018e feat(pipeline): publish small overlays before exact frontier expansion
Add an overlay-first pure-upsert route for overlay_publish=small_deltas so bounded changed-file overlays can publish before recursive exact inbound-frontier expansion falls through to containment.

The route reuses existing scratch seed, extract/resolve, postpass, delta metadata, preserved-inbound, and overlay publish helpers. It is gated to pure upserts in FAST-or-higher modes within the existing exact changed/affected path caps, and falls through to the existing exact/containment paths on failure.

Avoid a duplicate failed overlay generation when overlay-first reaches the publish step and fails; exact upsert skips the second overlay publish attempt and continues to canonical exact fallback.

Add a wide-inbound pipeline canary proving a changed Leaf with four unchanged callers now publishes incremental_overlay before frontier_too_large and active overlay BFS still sees CallerA -> Leaf.

Validation: make -f Makefile.cbm -j8 build/c/test-runner; CBM_ONLY_SUITE=pipeline build/c/test-runner (321 passed); CBM_ONLY_SUITE=mcp build/c/test-runner (166 passed); scripts/check-source-safety.sh on touched files; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 06:15:11 -04:00
Andrew Hundt d752729e56 feat(pipeline): add preserved inbound overlay delta helper
Carry inbound edge properties through cbm_store_list_file_delta_inbound_edges so active overlay rows do not lose relationship metadata.

Promote the recomputed-edge skip policy into a shared pipeline predicate with edge-type constants, and reuse it from incremental replay.

Add cbm_pipeline_file_delta_add_preserved_inbound_edges for overlay-producer support: upsert deltas preserve only non-recomputed one-hop inbound edges whose target QN still exists in the new delta, with duplicate suppression.

Validation: make -f Makefile.cbm -j8 build/c/test-runner; CBM_ONLY_SUITE=store_nodes build/c/test-runner (105 passed); CBM_ONLY_SUITE=pipeline build/c/test-runner (320 passed); CBM_ONLY_SUITE=mcp build/c/test-runner (166 passed); scripts/check-source-safety.sh on touched files; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 06:00:23 -04:00
Andrew Hundt 1f4491b248 test(mcp): cover active inbound overlay edges
Add a focused MCP canary for active overlay graph reads where an unchanged canonical caller points to a changed overlay target. The test proves the old canonical target is hidden by the overlay tombstone while search_graph relationship reads still return the caller and replacement target with overlay_active_graph metadata.

Validation:
- make -f Makefile.cbm -j8 build/c/test-runner
- CBM_ONLY_SUITE=mcp build/c/test-runner
- bash scripts/check-source-safety.sh tests/test_mcp.c
- git diff --check

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 05:45:33 -04:00
Andrew Hundt 0f12f28b78 fix(pipeline): align parallel env access graphing
Reuse the existing sequential EnvVar/CONFIGURES materialization helper from the parallel full-index registry phase so full rebuilds remain a valid oracle for incremental slices.

Add a sequential-vs-parallel pipeline canary for C getenv access that verifies the CONFIGURES edge and canonical graph equality. This fixes the route_handler self-dogfood mismatch where incremental containment had EnvVar data that parallel full rebuild omitted.

Validation: make -f Makefile.cbm -j8 build/c/test-runner; CBM_ONLY_SUITE=pipeline build/c/test-runner; bash scripts/check-source-safety.sh src/pipeline/pass_definitions.c src/pipeline/pass_parallel.c src/pipeline/pipeline_internal.h tests/test_pipeline.c; git diff --check; make -f Makefile.cbm -j8 cbm; route_handler self-dogfood rerun passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 05:25:19 -04:00
Andrew Hundt e41a49d4b7 fix(pipeline): preserve nested folder containment
Create ancestor folder nodes before child folder edges in the full structure pass so first-seen nested directories produce deterministic CONTAINS_FOLDER edges.

Add the matching CBM_PIPELINE_EDGE_CONTAINS_FOLDER constant and reuse it in the exact-delta structural context path. Strengthen the pipeline structure canary to require pkg -> pkg/util containment and keep cleanup ahead of final assertions.

Validation: make -f Makefile.cbm -j8 build/c/test-runner; CBM_ONLY_SUITE=pipeline build/c/test-runner; bash scripts/check-source-safety.sh src/pipeline/pipeline.c src/pipeline/pipeline_delta.c src/pipeline/pipeline_internal.h tests/test_pipeline.c; git diff --check; make -f Makefile.cbm -j8 cbm; refreshed bounded P.A.N8.7 overlay CLI matrix passed 5/5.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 05:01:50 -04:00
Andrew Hundt 0f684ddc08 feat(mcp): trigger overlay compaction after publish
Add opt-in overlay_compaction_policy=after_publish with a bounded overlay_compaction_max_generations setting. index_repository now starts a one-shot compaction worker only after a successful incremental_overlay publish and reports the trigger status in the response.\n\nValidation:\n- make -f Makefile.cbm -j8 build/c/test-runner\n- CBM_ONLY_SUITE=mcp build/c/test-runner\n- CBM_ONLY_SUITE=pipeline build/c/test-runner\n- make -f Makefile.cbm -j8 cbm\n- git diff --check\n- bash scripts/check-source-safety.sh src/mcp/mcp.c src/cli/cli.c src/pipeline/pipeline.h tests/test_mcp.c tests/test_pipeline.c

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 04:45:48 -04:00
Andrew Hundt e227296a5b test(mcp): cover overlay compaction worker failures
Add reusable MCP overlay-compaction fixtures and cleanup helpers, then cover missing project DB no-create behavior, invalid input rejection, and server-free joins of pending compaction workers.\n\nValidation:\n- make -f Makefile.cbm -j8 build/c/test-runner\n- CBM_ONLY_SUITE=mcp build/c/test-runner\n- git diff --check\n- bash scripts/check-source-safety.sh tests/test_mcp.c

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-04 04:36:03 -04:00