Previously, build_project_list_error_srv discarded whether automatic indexing was disabled or stopped by auto_index_limit and always recommended index_repository, even when streamlined tools/list hid that tool. Double-quoted path examples could also invalidate the nested structured JSON response.
Record request-thread-owned block state in maybe_auto_index and cbm_mcp_auto_index_within_limit. mcp_index_recovery_action at src/mcp/mcp.c:3937 now uses the existing profile/reveal authorities: classic or revealed streamlined mode calls index_repository directly, unrevealed streamlined mode calls _hidden_tools and refreshes tools/list first, and profiles without that tool use the CLI. Existing synchronous first-use recovery still runs before any user action is emitted. Centralize streamlined/classic config values in src/cli/cli.h:383.
tests/test_mcp.c:9617 covers auto_index=false, bounded file-limit rejection, classic, and revealed streamlined behavior; each response must parse as JSON with structuredContent and must not claim auto_indexing. Verified: focused ASan/UBSan test 1/1; scripts/check-source-safety.sh; git diff --check; git clang-format --diff; staged private-path scan.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Before this change, src/mcp/mcp.c:3638 trusted a cached SQLite handle until an in-process notification arrived. A sibling CLI or MCP publisher cannot send that notification, so atomic database replacement left a long-lived server reading the old unlinked inode. The search_code handler at src/mcp/mcp.c:13580 also bypassed the canonical startup-index join and could return project not found or not indexed while indexing was active.
Add portable path identity reads in src/foundation/compat_fs.c:74 and :735, record identity only for query-only stores in src/store/store.c:1365, and reopen a replaced generation from resolve_store on the request-owning thread. Existing atomic writer publication remains unchanged; the read path adds one constant-time metadata probe and no database write, graph scan, process-wide lock, or unrelated-project serialization.
Route search_code through resolve_project_store, report its resolved target in JSON and TOON, release syntax-kind thread caches on every auto-index exit, and notify publication only for responses accepted by cbm_mcp_index_response_published.
TDD fixtures: tests/test_mcp.c:6891, :9517, and :11534. Verification: ASan/UBSan MCP 270/270, tool consolidation 113/113, CLI 232/232; focused TSan tests passed; macOS leaks reported 0 leaks/0 bytes for startup indexing and external replacement; source safety, git diff --check, and git clang-format --diff passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Before this change, execute_return_agg and with_agg_find_or_create compared each binding against every existing group. A 600-distinct-group fixture performed 179,700 string comparisons, and production queries over the 20,423-node repository store took 3.373 s for direct qualified-name grouping and 2.793 s through WITH.
Add one aggregate_group_index_t backed by foundation/hash_table.c for both execution paths. Aggregate entries own exact-length keys that remain stable when result arrays grow; the Verstable table borrows those keys and maps them to first-seen array indexes. If index allocation fails, execution retains the exact linear fallback. This changes group lookup from O(rows x groups) to expected O(rows), preserves deterministic emission, and removes the fixed 1 KiB inline key from every group.
tests/test_cypher.c adds a 600-group RETURN/WITH fixture that crosses the 256-entry growth boundary, checks exact ordered results, and caps logical lookups at 1,200. Existing null/empty, node-identity, deadline, JSON/TOON, and MCP compatibility suites remain green.
Verification: Cypher ASan/UBSan 188/188; Cypher TSan 188/188; MCP, tool-consolidation, and CLI suites green; focused macOS leaks 0 leaks/0 bytes; source-safety and git diff checks pass. Five optimized repetitions measured 0.393 s median direct grouping and 0.413 s median WITH grouping on the same store.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previously, tools/list left top-level input objects open and cbm_mcp_handle_tool dispatched raw JSON without checking property names. A search_code call containing repo_path could therefore ignore the caller's intended project and return a plausible empty result. The canonical schemas also omitted the implemented index_repository.format and index_status.verbose inputs.
Add mcp_add_tool_input_schema to close every classic, streamlined pre-reveal, streamlined post-reveal, and _hidden_tools input schema. Add mcp_validate_tool_argument_keys before dispatch so direct, stdio, HTTP, and raw-JSON CLI calls return an isError:true result naming the unknown key and canonical supported keys. Keep the public schema registry as the property authority; output schemas remain open.
Preserve only verified, unambiguous raw-JSON compatibility: string-valued project_name/project_id/projectName map to project; query_graph.cypher maps to query per 32a3820c; search_code search_in=source remains from the 3d133a3c to 39539eff split. Other search_in values name search_graph, and ignored query_graph label input is rejected.
MCP Tool Execution Error behavior follows https://modelcontextprotocol.io/specification/2025-11-25/server/tools. Closed no-argument schemas follow https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1613.
Verification: ASan/UBSan MCP 267/267; TSan MCP 267/267; CLI 232/232; tool consolidation 113/113; HTTP 44/44 with one platform skip; production smoke 35/35; four focused macOS leaks checks each 0 leaks/0 bytes; scripts/check-source-safety.sh; optimized real-store CLI query_graph and typo-rejection dogfood.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior:
- OPTIONAL MATCH null bindings were encoded as empty strings, so count(expr), count(DISTINCT expr), and collect(expr) counted absent values.
- WHERE and coalesce treated a valid JSON empty string as null.
- WITH aggregation grouped distinct same-named nodes together by display name.
- edge_prop used process-global rotating scratch buffers across concurrent requests.
Changes:
- src/cypher/cypher.c adds presence-aware json_extract_prop_ex, node_prop_ex, edge_prop_ex, and binding_get_virtual_ex lookups plus one null bit per bounded binding slot.
- with_agg_accumulate and ret_agg_accumulate skip null expressions while count(*) retains row semantics; WHERE and coalesce preserve valid empty strings.
- group_key_append keys nodes and edges by canonical store ID and length-prefixes scalar values, preventing name and delimiter collisions.
- edge projection scratch storage is thread-local; aggregate null vectors follow existing group cleanup ownership.
- tests/test_cypher.c covers OPTIONAL null aggregates, post-WITH counts, numeric aggregate defaults, empty/null/missing properties, predicates, coalesce, and same-name node grouping.
Complexity and lifecycle:
- Property presence adds no per-row allocation and aggregation retains its existing row/group traversal order.
- Binding storage grows by CYP_MAX_VARS booleans; edge scratch remains fixed-size per thread.
- The one per-group null vector is released with the existing aggregate state.
References:
- https://s3.amazonaws.com/artifacts.opencypher.org/openCypher9.pdf
- https://neo4j.com/docs/cypher-manual/25/functions/aggregating/
- https://neo4j.com/docs/cypher-manual/current/values-and-types/working-with-null/
Verification:
- CBM_ONLY_SUITE=cypher build/c/test-runner: 187 passed under ASan/UBSan.
- CBM_ONLY_SUITE=mcp build/c/test-runner: 262 passed under ASan/UBSan.
- bash scripts/check-source-safety.sh: passed.
- git diff --cached --check: passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
tests/test_cli.c:6554 still required the word 'upstream' after commit aefbec05 changed query_max_rows from an unsafe scan prefix to an exact post-selection result ceiling. That stale wording failed the CLI sanitizer suite even though the registry described the corrected behavior.
Assert the public invariants instead: query_max_rows is a result-row cap, lowering it does not change which rows match, and Cypher LIMIT cannot bypass it. The focused contract passes 1/1 and the complete CLI plus agent-client ASan/UBSan gate passes 232/232; scripts/check-source-safety.sh and git diff --check also pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
src/pipeline/pipeline_incremental.c now detects changed, deleted, and renamed .inc macro context and expands the affected set to discovered ObjectScript files before dirty-ledger and publication routing. This matches cbm_build_macro_table_from_files, whose repository-wide macro table has no per-include consumer map, while leaving other-language increments on the existing O(changed-files) path and reusing exact-delta bounds plus containment fallback.
tests/test_pipeline.c extends the cleanup-safe ObjectScript oracle to prove changed and deleted includes replace or remove stale CALLS edges and remain graph-equal to a fresh rebuild. Verification: pipeline ASan/UBSan 384/384; focused changed/deleted include TSan passed; focused macOS leaks reported 0 leaks for 0 total leaked bytes; scripts/check-source-safety.sh passed; git diff --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previously, src/pipeline/pass_definitions.c populated the sequential result cache without ctx->macro_table, so pass_calls reused unexpanded $$$ calls. Exact incremental runs also extracted only the changed .cls slice without unchanged .inc definitions, dropping the CALLS edge that a clean rebuild produced.
Route sequential extraction through cbm_extract_file_with_options_ex, declare cbm_build_macro_table_from_files once in pipeline_internal.h, and let parallel extraction borrow an immutable caller-owned table. pipeline_incremental.c now builds repository macro context only when the changed slice contains ObjectScript, releases it on every return path, and otherwise preserves changed-file setup complexity.
tests/test_extraction.c verifies the expanded callee remains scoped to MyApp.Caller.Run. tests/test_pipeline.c verifies full resolution, exact incremental publication, CALLS-edge retention, clean-rebuild graph equality, and default fixture cleanup.
Verification: ASan/UBSan pipeline 382/382, extraction 313/313, parallel 36/36; focused TSan passed; macOS leaks reported 0 leaks for 0 total leaked bytes; bash scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previously scripts/run-benchmark-campaign.py required callers to hand-build binaries, author a plan or matrix spec, and choose a campaign root. Timestamp-only artifact names did not distinguish source time from execution time, repeated preparation rebuilt candidates, and absolute checkout paths prevented portable resume.
Add no-argument quick and --full campaign presets, peeled commit/tree identities, detached candidate worktree reuse, canonical Makefile.cbm CFLAGS_PROD capture, streamed production build logs, binary hashing, clean tracked-tree checks, and tamper-aware candidate cache records. Name new campaign evidence with v0001 definition, runset content ID, explicit commit time, and explicit generation time; identity_version=2 normalizes canonical path mappings while legacy cells keep their prior identity document.
scripts/benchmark-incremental-speed.py now uses explicit sortable UTC filenames for retained failure and scaling evidence. tests/test_benchmark_campaign.py covers tag peeling, worktree reuse, stale binary rebuild, quick/full matrix construction, path remapping, bounded names, clean-tree policy, and legacy identity compatibility.
Verification: uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed tests.test_summarize_benchmark_results tests.test_autotune (159 passed); ruff check on all three changed files; git diff --check; bash scripts/check-source-safety.sh.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
scripts/benchmark-incremental-speed.py validates each candidate cache before setting CBM_CACHE_DIR. It rejects the inherited live cache and directories containing an existing project database, preventing cross-version recovery logic from renaming or mutating user data.
scripts/summarize-benchmark-results.py adds separate bold graph, result/quality, and run/lifecycle error columns to the headline table. Requested cleanup failures now produce REJECT: lifecycle cleanup, so historical speedups and Pareto selection remain unavailable to invalid runs.
tests/test_benchmark_incremental_speed.py and tests/test_summarize_benchmark_results.py reproduce both prior contracts. Verification: 149 benchmark/campaign/report/autotune unittests; Ruff format check on four changed files; scripts/check-source-safety.sh.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior: scan_pattern_nodes() limited unlabeled candidates to max_rows * 10 before WHERE evaluation; execute_return_simple() projected max_rows bindings before ORDER BY; and an explicit Cypher LIMIT could bypass max_rows. An exact named-function lookup beyond the seed prefix returned zero rows, while a descending order query ranked only the prefix.
Push mandatory, non-negated file_path CONTAINS conjuncts into SQLite with literal instr() semantics, while retaining the full C evaluator as semantic authority. Restrict bounded initial scans to prefix-preserving query shapes; predicates, relationships, aggregation, DISTINCT, ORDER BY, SKIP, and later stages evaluate the complete eligible set after exact SQL reduction.
Make max_rows and query_max_rows authoritative result ceilings: Cypher LIMIT may lower but not bypass them. Align the shared MCP definition and CLI config registry so streamlined, classic, and CLI surfaces describe the same contract.
TDD covers a 65-node exact lookup beyond the former 50-node prefix, global ORDER BY selection, unsafe OR pushdown exclusion, literal percent/underscore CONTAINS behavior, configured cap enforcement, and streamlined/classic schema parity. Verification: 531 Cypher/companion and 612 MCP/schema/companion ASan/UBSan tests passed.
Retained optimized large-graph validation passed graph and quality gates: query_graph 2.367 ms, exact incremental indexing 363 ms at 71 MB peak, full indexing 6494 ms at 1332 MB peak, and 17.89x full-over-incremental speedup. The benchmark artifact remains untracked.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior: cbm_store_get_overlay_node_view_summary() materialized active_node_candidates and applied ROW_NUMBER ownership ranking for every read, even when overlay_generations contained no overlay_ready row. On the retained 15,664-node graph this added about 120 ms to search_graph, search_code, route, and other MCP reads.
Use the existing indexed cbm_store_count_overlay_generations() authority to gate the active view. When no ready generation exists, reuse cbm_store_count_nodes() for the exact canonical and total counts; projects with ready overlays retain the complete tombstone and ownership CTE.
Add an sqlite3_trace_v2 regression guard in tests/test_store_nodes.c that rejects active_node_candidates on the clean path while requiring the canonical count query. The same test continues to verify exact counts for successive ready overlays.
Verification: CBM_ONLY_TEST=overlay_node_view_summary make -f Makefile.cbm test-focused TEST_SUITES=store_nodes; ASan/UBSan store_nodes, mcp, and tool_consolidation suites; optimized self-dogfood MCP smoke with search_graph median reduced from 120.149 ms to 3.825 ms and search_code from 131.946 ms to 10.436 ms while the exact incremental gate passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior: each query oracle measured one default-format call and one JSON quality call, then reports treated the first call as query latency. One-shot schema context, watcher startup, and cache warming could land on either call, producing apparent 16 ms versus 261 ms regressions from the same underlying path.
Run three canonical JSON trials after the default-format call for both MCP and CLI. Retain every elapsed value and canonical payload hash, report min/median/max plus byte-equality, and preserve the first JSON timing for backwards compatibility. The summarizer now labels one-shot default latency separately and uses the repeated JSON median for cross-candidate query tables.
Add fake-client timing/hash coverage in tests/test_benchmark_incremental_speed.py and cold-versus-repeated aggregation coverage in tests/test_summarize_benchmark_results.py. Verification: 144 benchmark, campaign, and report tests pass; ruff format reports all four changed files formatted; git diff --check passes.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior: a matrix profile could set capabilities such as auto_index_deps=false as descriptive metadata while leaving config_overrides empty. The archived plan then claimed an ablation that the benchmark command never applied; retained worker timings exposed dependency and PageRank phases still running.
Make expand_matrix_spec fail closed when config_profile=default claims a disabled capability without the identical false config override. Named profiles remain compatible because their canonical settings are resolved by benchmark-incremental-speed.py.
Add test_default_profile_rejects_disabled_capability_without_config_override in tests/test_benchmark_campaign.py. Verification: all 26 campaign-runner tests pass and git diff --check passes.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior: self-dogfood graph validation inferred every stale derived view from task-specific MCP warnings. A changed-file query does not mention semantic_edges, so exact incremental runs with intentionally deferred semantic edges were rejected as canonical graph failures despite derived_view_state recording semantic_edges as stale.
Add persisted_stale_views() in scripts/benchmark-incremental-speed.py to read the canonical SQLite freshness ledger through a read-only connection and union it with response-local warnings. Missing legacy derived_view_state tables remain compatible, while core node, edge, and file-hash mismatches are still rejected after excluding only declared SEMANTICALLY_RELATED rows.
Add tests/test_benchmark_incremental_speed.py coverage for project-scoped stale/complete rows and retain tests/test_mcp.c coverage proving moderate incremental_exact publication persists semantic_edges=stale and query_graph reports it for semantic queries.
Verification: 76 benchmark harness tests passed; 262 ASan/UBSan MCP tests passed; focused exact-moderate MCP test passed; git diff --check passed. The repository-wide lint-format target still reports pre-existing formatting violations outside these changes.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add run_mcp_surface_parity() coverage for classic discovery, streamlined discovery before _hidden_tools, and relisting in the same process after reveal. Compare complete tools/list contracts, validation-equivalent get_code aliases, capability outcomes, bounded hidden-handler recognition, and notifications/tools/list_changed without presenting dispatch recognition as behavioral proof.
Make McpClient cleanup record subprocess return codes and require both reader threads and server processes to be reaped. Close SQLite snapshot and cloned-fixture connections explicitly so ResourceWarning-as-error runs detect ownership regressions.
Render capability outcomes before tool counts in scripts/summarize-benchmark-results.py, retain response bytes and single-observation latency with explicit evidence limits, and document the parity mode in docs/BENCHMARK_CAMPAIGN.md.
Verification: 141 warning-clean benchmark/campaign/summarizer tests; optimized build/c/codebase-memory-mcp parity smoke passed for 12 capability groups with full post-reveal classic contract parity and reaped lifecycle; ruff check; source-safety; git diff --cached --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Replace scripts/autotune.py's global config mutation, mutable local-repository lookup, architecture resource request, and source-tree result file with build_matrix_spec() over scripts/run-benchmark-campaign.py.
Bind each plan to the full revision, binary SHA-256, compiler, flags, harness hash, candidate-default/rank-disabled controls, and PageRank parameter profiles. Store content-addressed manifests under the ignored durable campaign root and transfer execution with os.execv so the shared runner owns isolation, resumption, logs, and cleanup.
Add tests/test_autotune.py for shared-runner plan validation, paired profile expansion, and absence of the removed resources/read, atexit, Path.home, and database-deletion paths.
Verification: uv run python -W error::ResourceWarning -m unittest tests.test_autotune (3 tests); ruff check scripts/autotune.py tests/test_autotune.py; git diff --cached --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
tests/test_tool_consolidation.c:cross_project_search_not_confused_by_prefix now describes the path-derived project-name collision without embedding a local username, repository path, or private graph size. The executable myapp/myapp-other-project fixture and prefix-boundary assertions are unchanged.
Verification: cross_project_search_not_confused_by_prefix passed (1 selected, 349 filtered); scripts/check-source-safety.sh passed; git diff --check passed; tracked-source scan found no athundt, justtalk, processtree-rs, or private feedback filenames.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
tests/test_tool_consolidation.c adds query_graph_description_uses_ready_overlay_view, which proves tools/list describes the ready overlay that query_graph executes instead of canonical rows replaced for the same file.
tests/test_mcp.c adds mcp_published_schema_refreshes_description_once and extends mcp_delete_project_sends_list_changed. These protocol fixtures prove publication reopens the store, repeated publication emits one notifications/tools/list_changed event, and project deletion removes cached vocabulary from the next tools/list response.
Verification: 590 MCP/consolidation tests; full ASan/UBSan suite (6867 passed, 1 skipped); TSan suite (449 passed) plus all three exact freshness cases; macOS leaks reports 0 leaks for 0 bytes for each exact case.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior:
- with_add_vbinding_var stored a WITH alias in cbm_node_t.qualified_name to share allocation ownership.
- RETURN g.qualified_name after WITH g, COUNT(*) therefore returned the alias literal g instead of the indexed qualified name.
Changes:
- Track heap-owned variable aliases with binding_t.var_name_owned.
- Copy and free aliases independently from cbm_node_t fields.
- Leave aggregate node stubs eligible for the existing id-based indexed property fetch.
Tests:
- cypher_exec_with_node_groupvar_prop reproduces g versus test.ValidateOrder and now passes.
- build/c/test-runner cypher: 519 passed.
- make -f Makefile.cbm test: 6865 passed, 1 skipped under ASan/UBSan.
- macOS leaks-at-exit exact regression: 0 leaks, 0 leaked bytes.
- Optimized -O2 query_graph returns qualified names in both TOON and JSON.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior:
- src/cypher/cypher.c parsed MATCH after WITH without executing it against projected bindings.
- OPTIONAL MATCH predicates were applied after null extension, multi-key ORDER BY was not represented, and relationship expansion silently capped each seed at ten paths.
Changes:
- Add next_stage ownership and iterative execution/free traversal for MATCH and OPTIONAL MATCH after WITH, including UNION transfer and active-overlay checks.
- Apply OPTIONAL predicates to candidate paths before null extension.
- Represent up to 32 projected ORDER BY keys and use stable O(rows log rows) merge sorts with O(rows) pointer/index scratch.
- Replace fixed-factor node and relationship buffers with ceiling-bounded geometric growth shared by canonical and active-overlay paths.
- Keep classic and streamlined query_graph descriptions aligned with the implemented grammar.
Tests:
- make -f Makefile.cbm test: 6865 passed, 1 skipped under ASan/UBSan.
- build/c/test-runner cypher: 519 passed.
- build/c/test-runner mcp tool_consolidation: 588 passed.
- scripts/check-source-safety.sh and scripts/check-nolint-whitelist.sh pass.
Semantics checked against Neo4j Cypher documentation:
https://neo4j.com/docs/cypher-manual/current/clauses/with/https://neo4j.com/docs/cypher-manual/current/clauses/match/https://neo4j.com/docs/cypher-manual/current/clauses/order-by/
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Response-format consolidation: add cbm_mcp_response_format/
cbm_mcp_invalid_response_format so every response-generating handler
reads the format argument or default_response_format config through one
function instead of duplicating the enum-mapping logic, and migrate
query_graph's legacy-JSON detection to the canonical
cbm_mcp_output_format_t enum. Fix the regression this surfaced: TOON-path
_context/session_project delivery (toon_append_context_once) was
serializing the one-shot context object as a raw escaped JSON string
inside the TOON stream instead of native TOON fields, silently dropping
it for any TOON-consuming client. Add toon_append_context_model plus its
scalar/table helpers to map the same format-neutral context model
inject_context_once already builds into native TOON output.
Self-healing zero-row hints: query_graph_no_rows_hint reparses the
already-executed query on the zero-row path only, walks MATCH/OPTIONAL
MATCH patterns and EXISTS predicates in WHERE and post-WITH-WHERE
clauses (hint_walk_expr_exists_types/hint_walk_where_exists_types), and
probes each referenced label/edge type against the store via
cbm_store_schema_label_observed/cbm_store_schema_type_observed
(schema_probe_one_row: fail-open on any non-SQLITE_DONE outcome, so a
probe error never produces a false accusation). Unobserved names are
named in the response; a bounded hint_seen_t/hint_seen_add set (capped
at CBM_STORE_SCHEMA_HINT_VOCAB_LIMIT) dedups names repeated across
patterns/UNION branches so "Klass, Klass." can't happen and avoids
re-probing a name already resolved. The vocabulary summary and the
generic zero-row fallback no longer claim WHERE-clause predicates were
verified, since only labels/edge types are ever probed — only the
labels/edge types claim is made.
One invalidation authority: every graph-mutation path now calls
the single cbm_mcp_server_notify_index_published (flags-only,
any-thread-safe) instead of ad hoc local staleness bookkeeping —
sync auto-index, in-process index_repository, in-process
autoindex_thread, overlay compaction (gated on compacted>0),
index_dependencies. handle_delete_project's notify is gated on the
mutation actually succeeding (!is_error): a delete of a nonexistent
project is a no-op and must not stale the tools/list description cache
for nothing. The request-thread-only drain
(mcp_drain_tools_list_changed) is factored once and shared by both
transports (line-delimited and Content-Length-framed), gated so a
notification published before any tools/list has ever been served is
suppressed rather than sent for a client that never asked.
Shared overlay-aware schema-view selector (mcp_get_current_schema/
mcp_overlay_view_ready) ends the schema/overlay-readiness dance
previously triplicated across handle_get_graph_schema,
build_resource_schema, and the query path, so query_graph's advertised
vocabulary can never diverge from the view a query actually runs
against. The selector takes an mcp_cypher_vocabulary_t level instead of
a bool: MCP_CYPHER_MATCH_VOCABULARY (labels/edge types/observed
patterns with counts — enough to compose executable MATCH clauses;
never parses property JSON) vs MCP_CYPHER_FULL_QUERY_VOCABULARY (adds
per-label/type property keys for WHERE/RETURN authorship; O(total
property rows) json_each discovery, so only the docstring cache and
get_graph_schema request it).
Route the two remaining canonical-only cbm_store_get_schema callers
through the selector at MATCH_VOCABULARY level: inject_context_once
(first-response _context previously paid the full property-key scan for
counts it never rendered, and could advertise canonical labels a ready
overlay had tombstoned) and build_resource_architecture (rendered only
rel_patterns yet fetched full property keys, canonical-only). The
architecture resource's mixed-read-model disclosure moves
relationship_patterns into the overlay-aware list accordingly.
build_query_graph_tool_description resolves the project's real
store via resolve_store(srv, project) instead of trusting srv->store
as-is: on the very first tools/list of a fresh session for an
already-indexed project, srv->store was still the empty default
in-memory store (maybe_auto_index detects "already indexed" and skips
reindexing without ever opening it), so the schema section rendered
silently empty until some other tool call happened to resolve it first.
The project name is snapshotted into a local buffer before the
resolve_store call rather than passed as a live alias to
srv->current_project, because resolve_store can call
reap_stale_store internally, which frees srv->current_project when a
deferred invalidation is pending — aliasing it produced a
heap-use-after-free (caught by the ASan full-suite run, confirmed fixed
by rerun).
Docstring clarity for newcomers: the live schema section now embeds its
notation legend in the section header ("Labels name{extra property
keys}[count]:") so the compact schema is self-explanatory in every
tools/list response, and the query_graph/get_architecture/search_code
format parameters gain the brief description their three siblings
already had, all naming default_response_format.
Tests: response-format and TOON-context regressions in test_mcp.c/
test_tool_consolidation.c/test_input_validation.c/test_token_reduction.c;
zero-row hint coverage including no-hidden-tool-name, missed-graph shadow-
project probing, empty-project no-crash, TOON/JSON format parity,
UNION/repeated-name dedup, and the previously-untested EXISTS-predicate
walk; notification coverage for every notify call site including the
delete_project no-op negative direction and the "notify before first
tools/list is suppressed" negative direction; a cold-start regression
test seeding a real on-disk project store and calling tools/list as the
literal first request of a fresh session; overlay-consistency tests for
the two rerouted callers (first_response_context_uses_ready_overlay_schema,
resource_arch_rel_patterns_use_ready_overlay) proving _context and the
architecture resource report the active-overlay vocabulary, not
tombstoned canonical labels/types.
Docs: README.md documents the zero-row hint behavior and the
default_response_format config example; docs/CONFIGURATION.md adds the
default_response_format row to the important-keys table.
Verification: make -f Makefile.cbm test (full ASan/UBSan suite) run on
this exact tree after the final revisions. Earlier iterations of this
content additionally passed make -f Makefile.cbm test-leak (0 leaks for
0 total leaked bytes) and manual checks against the live MCP binary
(zero-row hint names real schema on two repositories; the query_graph
tools/list docstring shows the live schema on the very first request of
a fresh session); revisions since those checks are docstring text, the
vocabulary-enum rename, and the two selector reroutes, all covered by
the full-suite rerun.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
get_schema/get_schema_overlay_view previously returned relationship
patterns as preformatted "(A)-[T]->(B) [Nx]" strings via
schema_format_rel_pattern, forcing every consumer to re-parse text to get
at the source label, edge type, target label, or observed count
individually. Replace it with a structured cbm_schema_relationship_t
{source_label, edge_type, target_label, observed_count} array
(schema_collect_rel_patterns_from_stmt, store.h/store.c), and centralize
the previously-duplicated SCHEMA_MAX_JSON_KEYS/SCHEMA_REL_PATTERN_LIMIT
enum constants as CBM_STORE_SCHEMA_PROPERTY_KEY_LIMIT and
CBM_STORE_SCHEMA_RELATIONSHIP_PATTERN_LIMIT beside the existing schema
caps in store.h. Add const-correct cbm_store_schema_node_base_properties/
cbm_store_schema_edge_base_properties accessors so base-column lists have
one owner instead of being redeclared per caller.
Add a declared node/edge property-key registry
(schema_declared_node_property_keys/schema_declared_edge_property_keys,
~107 entries, sorted and duplicate-free) enumerating every JSON property
key any pipeline pass or git-context writer can emit on a node or edge,
exposed via cbm_store_schema_declared_node_property_keys/
cbm_store_schema_declared_edge_property_keys. Each row carries a trailing
comment naming its writer for provenance. The registry is a maintenance
contract, not a runtime dependency: nothing outside the new contract test
reads it. Document the four places that must stay in sync (writer,
store.c table, store.h accessor comment, contract test) in both store.h
and CLAUDE.md.
Add tests/test_schema_declared_property_keys.c: indexes mixed-language
fixtures (Python/TypeScript/Rust) plus a best-effort real git repo
(git init/commit via the portable cbm_git_drain_command, matching
test_git_context.c's tolerance for "git not available") to also
forward-verify the ~10 git_context registry rows, asserts every
discovered non-base property key is in the declared registry for its
entity kind, asserts both tables are sorted/duplicate-free, and reverse-
pins a sample of registry keys to catch dead rows. Cap-blindness guarded:
a label/type that hits the discovery limit fails loudly instead of
passing on unprovable completeness.
Register the new suite in Makefile.cbm (TEST_STORE_SRCS) and
tests/test_main.c.
Verification: make -f Makefile.cbm test — 6850 passed, 1 skipped, 0
failed, ASan/UBSan clean. make -f Makefile.cbm test-leak — 0 leaks for 0
total leaked bytes.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
execute_single previously evaluated the complete WHERE tree while only the first MATCH alias was bound. A target-alias predicate such as NOT callee.file_path CONTAINS "tests/" therefore rejected every seed before CALLS expansion and returned zero rows.
Add allocation-free three-valued partial evaluation in src/cypher/cypher.c: bound leaves prune early, unbound node or edge aliases remain unknown, and AND/OR/NOT/XOR preserve boolean semantics and safe short-circuiting. Register eval_expr_partial in src/foundation/recursion_whitelist.h.
Add four relationship-target and mixed-alias regressions in tests/test_cypher.c. Verified 159/159 Cypher tests under ASan/UBSan, source-safety and whitelist checks, an -O2 production build, and the formerly empty hotspot query returning 20 bounded rows.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
cbm_parse previously returned success when parse_return_item stopped before postfix list indexing or an arbitrary trailing token. A query containing labels(n)[0] could therefore ignore its alias, aggregate, ordering, and limit and execute as an unbounded projection.
Require TOK_EOF after parse_post_where in src/cypher/cypher.c and transfer the parent cursor after a recursively validated UNION tail. Route both list-index rejection paths through one diagnostic that gives supported labels(n) and n.label/count(*) rewrites.
Add parser regressions in tests/test_cypher.c. Verified 155/155 Cypher tests under ASan/UBSan, all three UNION tests, source-safety, an optimized CLI rejection, and the supported five-row node-label aggregation.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
docs/CONFIGURATION.md:82 previously listed auto_index=false even though CBM_CONFIG_REGISTRY in src/cli/cli.c defaults the key to true. This could lead users to assume streamlined first-use indexing was disabled.
tests/test_cli.c:3837 now derives the expected Markdown row from CBM_CONFIG_REGISTRY, so future default changes must keep the configuration table synchronized.
Verification: CBM_ONLY_SUITE=cli build/c/test-runner (151 passed); bash scripts/check-source-safety.sh; git clang-format --diff HEAD -- tests/test_cli.c.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
search_graph previously ran an unfiltered graph search before a JSON semantic-only lookup, so an empty vector result returned unrelated nodes and paid an unnecessary graph-result-page scan. Detect semantic-only requests, skip cbm_store_search and cbm_store_search_overlay_view, and return explicit empty semantic_results with recovery guidance.
Define query_graph as an early compositional tool for effective, computationally efficient custom Cypher. Mark examples, labels, properties, and LIMIT as optional guidance; retain server caps and dependency-ranking instructions; make truncation hints offer narrowing or cap changes without prescribing LIMIT.
Install portable sandbox recovery guidance that distinguishes MCP approval from shell sandbox authorization.
Verification: ASan/UBSan MCP 228 passed; tool_consolidation 103 passed; token_reduction 50 passed; CLI 150 passed; source-safety passed. The repository-wide clang-format target remains red on pre-existing files; changed-line formatting issues were corrected.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
summarize_group now derives a Pareto workload identity from scenario, report mode, index mode, corpus metadata, and task-set hashes. mark_pareto_frontier compares candidates only when that identity matches, preventing small semantic canaries from dominating full-repository measurements.
Add test_pareto_does_not_compare_different_workloads in tests/test_summarize_benchmark_results.py. Verified with 140 benchmark campaign/measurement/report tests, ruff check, and regenerated retained composition output.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
tests/test_pipeline.c lacked persisted CALLS coverage for a Python entry point importing PyO3 functions and lacked incremental-versus-fresh oracles for mixed Python/Rust and Rust/TypeScript/JavaScript repositories.
Add pipeline_python_pyo3_import_resolves_rust_function_calls plus two mixed-language incremental tests. The tests require Python edits to publish as CBM_PIPELINE_PUBLISH_INCREMENTAL_EXACT, require Rust and TypeScript scoped-LSP gaps to publish a full rebuild with reason scoped_lsp_gap, and compare each resulting graph with a fresh fast rebuild.
Route setup_lang_repo and setup_incremental_repo through cbm_resolve_cache_dir, reject truncated paths, and remove partially created fixture trees on setup errors. Assertion-aborted fixtures now remain inside the test runner cache lifecycle and are retained only through CBM_TEST_ARTIFACT_DIR.
Verification: all three focused tests passed with the ASan/UBSan runner and the current-source TSan runner. Apple leaks reported 0 leaks for 0 total leaked bytes for each focused test. A real MCP 2025-11-25 trace_path request against a mixed Python/Rust repository returned cli_main -> _run_cli_command and cli_main -> serve_mcp with isError=false and clean EOF shutdown.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The pipeline had separate coverage for TypeScript re-export IMPORTS edges and direct cross-file calls, but no end-to-end guard proved that their composition resolves a caller through a barrel file to the implementation symbol.
Add pipeline_typescript_barrel_reexport_call_resolves_implementation in tests/test_pipeline.c. Its three synthetic files use nested hyphenated directories, a multiline named import, an async awaited call, and an export-from barrel; cross_file_call_exists must find callerOperation -> targetOperation in the persisted database.
Verification: the focused test passed with the ASan/UBSan runner and the current-source TSan runner. Current-source TSan also passed watcher unwatch, replacement, deferred-free, publication-generation, supervised-response, and pipeline-lock contention guards. git clang-format reported no changed-line edits and git diff --cached --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Incremental behavior previously lacked one table-driven oracle covering every supported parser/LSP language family. That left exact-upsert and scoped-LSP fallback routing vulnerable to language-specific drift even when individual fixtures passed.
Add run_incremental_language_oracle_case in tests/test_pipeline.c. Each synthetic fixture performs a full fast index, edits source to add a callable symbol, runs incremental indexing, asserts the publication kind and fallback reason, and compares the resulting canonical graph with a fresh fast rebuild.
Cover Go, C, C++, CUDA, and Python exact publication plus JavaScript, TypeScript, TSX, PHP, C#, Java, Kotlin, and Rust scoped_lsp_gap full fallback. The helper releases pipelines, configuration, and project storage through one cleanup path; ordinary passes and failures remove fixtures, while CBM_TEST_ARTIFACT_DIR retains failed fixtures inside the runner-owned cache.
Verification: the ASan/UBSan test runner passed incremental_cross_lsp_language_matrix_matches_fresh_rebuild (13 cases) and eight adjacent exact, C-header fallback, Python/JavaScript scoped-LSP, delete, rename, overlay publish-failure, and overlay extraction-failure tests. git clang-format reported no changed-line edits and git diff --cached --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
A supervised watcher reindex could replace a project database while the request thread retained a read-only handle to the unlinked prior generation. The watcher also treated any non-null worker response as success, including status=error, and shutdown freed g_server before an in-flight watcher callback had joined.
Pass the server through cbm_mcp_index_run_supervised_path, defer cached-store retirement through store_stale, classify indexed/degraded responses before committing watcher baselines, and join the watcher before freeing the server while keeping the watcher object alive for autoindex teardown.
Normalize glob-compatible wildcards before POSIX regex compilation so valid-but-ambiguous inputs such as alpha?orker and alternations return the intended symbols across name_pattern, qn_pattern, and pattern. Preserve explicit group, class, escaped, and dot-regex constructs.
Place validation fixtures under the isolated test cache, clean runner-owned caches after both passing and failing runs by default, and reuse CBM_TEST_ARTIFACT_DIR as the explicit failed-run retention path.
Verification: CBM_ONLY_SUITE=input_validation build/c/test-runner (48 passed); CBM_ONLY_SUITE=mcp build/c/test-runner (227 passed); make -f Makefile.cbm cbm (O2 build and ad-hoc signature); MCP initialize plus watcher-active EOF shutdown exited 0; git clang-format reported no changed-line edits; git diff --check passed. The repository-wide lint-format target still reports pre-existing formatting debt outside this diff.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The parallel runner's isolated-process execution exposed three cli tests
that were passing for the wrong reason on Linux: cbm_zed_config_dir (and
the VS Code/vendor path resolvers) prefer $XDG_CONFIG_HOME over
$HOME/.config, CI runners export it, and an isolated `test-runner cli`
process therefore resolved config paths OUTSIDE the test fixture — the
tests only stayed green in the all-suites single process because an
earlier suite's env mutations leaked into them (hidden cross-suite
coupling, deterministic failure in isolation, macOS unaffected since its
resolvers do not consult XDG).
- cli_detect_agents_finds_zed: save/set XDG_CONFIG_HOME to the fixture's
.config for the detection call, restore after.
- cli_vscode_profile_mcp_uninstall: same pin alongside its existing
HOME/PATH/APPDATA handling.
- cli_durable_profiles_follow_current_vendor_paths: XDG_CONFIG_HOME joins
its unset-and-restore env list.
All 222 cli tests pass locally with a hostile XDG_CONFIG_HOME exported.
No production code changed; no assertion weakened.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up correcting d8a78e8, whose message claimed a broken merge cap
crashes "within the first second" so the 10s alarm could not mask it.
Falsification disproved that claim AND found the guard was vacuous on
POSIX from the start:
- With CBM_TS_STACK_MERGE_MAX_DEPTH deleted outright, the ORIGINAL
unbounded test stayed GREEN (220s full grind, no crash) on macOS ASan.
- Small thread stacks (2 MB, then 512 KB) did not change that.
- Instrumenting the vendored recursion showed why: the recursive merge
path is NEVER ENTERED for this input in the observable window — max
depth stays 0. There is nothing for a stack bound to catch here.
So the cap-regression DETECTION lives in the Windows in-process branch
(real ~1 MB native stack, where #913 actually manifested — exercised by
the windows CI leg). The POSIX branch is, and always was, a bounded
no-crash smoke of the pathological parse; the 10s alarm makes it
deterministic (12-15s across three runs, was 2-218s machine-dependent)
without any detection loss, because there was no POSIX detection to
lose. The experimental pthread machinery is dropped; the test comment
now carries the falsification data so no future reader mistakes the
POSIX branch for a cap guard.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
perl_glr_deep_parse_recursion_capped wandered between 2s and 218s for the
IDENTICAL input: past the merge cap the ambiguity-exploded GLR parse grinds
at an environment-dependent rate (allocator/scheduler state in the forked
child), and the test paid for however long that grind ran. It was the
critical path of every parallel run and the dominant chunk of the
sequential one.
The guard's actual signal is the CRASH, not the grind: when
CBM_TS_STACK_MERGE_MAX_DEPTH regresses, the recursive stack-merge
overflows within the first second at this depth (same profile as the
sibling pre-guard probe, rc=139 in under a second). The forked child now
arms a 10s alarm whose handler exits cleanly — a real SIGSEGV/SIGBUS
still terminates the child by signal first and stays visible to
WIFSIGNALED, so the RED path is untouched; only the pointless remainder
of the grind is skipped. Windows (in-process branch) is unchanged.
Suite b: 13s across three consecutive runs (was 2-218s). Full parallel
run: 97s wall, 6,374 passed / 0 failed / 1 skipped, union guard clean —
vs 351s sequential on the same machine (3.6x).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The runner executed all 104 suites sequentially (~10 min locally, the bulk
of every CI test leg). The suites are process-isolated already (per-process
mkdtemp HOME sentinel; two full parallel runs produced zero cross-suite
failures), so the serialization was pure convention.
- test-runner --list-suites: prints every registered suite, one per line,
emitted by the SAME macro table that executes suites — the list cannot
drift from the run set by construction.
- scripts/run-tests-parallel.sh: runs each suite as its own process
(jobs = CPU count; CBM_TEST_PAR_JOBS overrides) under a ZERO-LOSS
CONTRACT: a union guard fails the gate if the set of suites that
produced a result differs from --list-suites (nothing can be silently
dropped, a newly added suite is picked up automatically); per-suite
pass/fail/skip are summed into the sequential runner's exact summary
format; any suite crash, failure, or omission exits nonzero. Per-suite
wall times are printed for balance tracking.
- make test-par: the parallel target. make test (sequential) is unchanged
and remains the escape hatch (CBM_TEST_SEQUENTIAL=1 in test.sh).
- scripts/test.sh: builds, then routes through test-par — every CI test
leg gets the speedup with zero workflow-topology change (same jobs,
same gates, same billing; the legs just finish sooner).
- The stack_overflow suite is split into a/b/c (7+6+7 of its 20 tests,
pure re-registration): as one suite it was the wall-clock critical path
of any parallel run — every other suite finished underneath its ~4
minutes.
Measured locally (Apple Silicon, ASan runner): sequential ~10 min vs
parallel 232 s, totals identical (6,361 passed / 0 failed / 1 skipped),
union guard clean. The TSan leg keeps its dedicated subset runner
(sequential) — out of scope here.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The PR CI caught two consumers the format migration missed — both parsed
tool output by its old shape and failed silently or with a wrong count:
- hook-augment built its PreToolUse additionalContext by reading the
legacy 'results' object-array from search_graph format:json; after the
json-tree reshape that key no longer exists, so the hook emitted nothing
(caught end-to-end by tests/windows/test_hook_augment.py). The parser
now walks groups -> qn_prefix/file + column-ordered rows. A new local
test feeds the parser the LIVE server envelope, so any future drift
between response shape and hook parser fails in the normal suite on
every platform — not only in the Windows CI guard. Reproduced RED
(ctx NULL, the exact CI failure) before the fix, GREEN after.
- smoke-test.sh carried six old-shape parsers (callers[N], rows[N]{,
clusters[N], semantic[N]{, and the assert_toon_table helper); the tree
headers never matched, so trace verification read 0 callers. All six
updated to the tree contract; the full smoke script passes end-to-end
locally.
cli suite 222 passed; lint clean; scripted smoke ALL PASSED.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
handle_search_graph returned completed BM25 JSON before the regular yyjson response builder. Requests with query plus format=json therefore omitted both session_project and the one-shot _context header, while graph-mode JSON and compact TOON included them.
Add json_payload_with_context_once in src/mcp/mcp.c to copy the bounded BM25 object, invoke the shared inject_context_once path, and serialize it before the early return. Extend search_graph_explicit_project_context_uses_resolved_store_project with the exact BM25 JSON branch.
TDD evidence: the new assertion failed with 101 existing consolidation tests passing because session_project was absent. Verified after the fix with ASan/UBSan: tool_consolidation 102/102 and mcp 225/225; scripts/check-source-safety.sh and git diff --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
inject_context_once used srv->session_project and srv->session_root even when search_graph resolved an explicit project in another store. That produced correct result rows with zero node counts, the wrong schema, and the session CWD ecosystem in the same response.
Pass the resolved project through the JSON and TOON context paths, retain session_project as the server CWD identity, resolve ecosystem metadata from the registered target root, and suppress stale PageRank statistics and key functions. Free cbm_project_t fields after the one-shot metadata lookup.
Add search_graph_explicit_project_context_uses_resolved_store_project in tests/test_tool_consolidation.c. Verified with ASan/UBSan suites: tool_consolidation 102/102, mcp 225/225, rust_lsp 506/506, registry 60/60, lang_contract 35/35; scripts/check-source-safety.sh passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Fresh-eyes review of the phase-2/3 work; five defects, each against our own
stated principles (no silent caps, exact counts, same model in both
encodings):
- trace callees_total/callers_total counted test-file rows even when
include_tests=false hides them from the table — the scalar disagreed with
the enumerable set (a field-eval agent read callers_total=175 against a
handful of visible rows and stopped trusting the tool). Totals now apply
the same filter as the emitters; description documents the transitive
(within-depth) semantics. Guard test: filtered=1 vs include_tests=2.
- cycles clamped the SCC list at 100 silently, losing the true count.
cycles_total is now always exact and cycles_omitted is emitted when the
display clamp engages (both encodings).
- detect_changes accepted a 'limit' param (referenced by its own omitted-
hint) that the schema never declared — undiscoverable. Declared.
- detect_changes format:json lacked the impacted_modules rollup the tree
encoding carries — model parity restored via a shared rollup helper.
- tree search extra-field cells were appended raw: a signature with spaces
shifted every following column. Cells now route through the shared
emitters (space-triggered quoting, '-' placeholder); the properties test
pins the quoted form.
Full suite 6,373 passed / 0 failed; lint clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Phase 3 of the graph-analysis overhaul. Two new capabilities on a shared
multi-source traversal substrate, plus a context-bomb guard.
detect_changes — rebuilt into real blast-radius analysis:
The old handler parsed and echoed depth but never traversed: impacted was
just the symbols DEFINED in changed files. It now resolves the git diff to
those symbols, then runs ONE multi-source BFS to the transitive impact set.
- default direction inbound = the blast radius (transitive callers that may
need review); outbound/both available.
- seeds are excluded from the result, and a changed file reached from
another changed file is not counted as extra impact (that is not
downstream fallout).
- merge_base SHA pins the exact diff base for reproducibility.
- impacted_modules: a 2-segment (src/mcp, tests/...) rollup — a quotient
view of where the radius lands, always complete (overflow bucketed, never
silently dropped). Per-symbol rows cap at limit (default 200, nearest
hops first) with impacted_total exact and impacted_omitted surfaced.
On a 7-file core diff: 1469 impacted symbols, 64KB unclipped -> 10.7KB with
the rollup + cap. format:json returns the same model as structured JSON.
cbm_store_bfs_multi (store): one recursive CTE anchored on ALL seeds via a
temp table — never seed_count separate walks over overlapping hubs.
Seed-exclusion NOT IN, MIN(hop) across the set, canonical (hop,id) order,
a memory-safety ceiling that reports *truncated instead of silently
capping. Reproduce-first: seed-exclusion + min-hop, and truncation.
get_architecture aspects:[cycles] — SCC condensation:
Iterative Tarjan (heap stacks, no recursion overflow) over the CALLS graph
reports circular dependencies (strongly-connected components of size > 1).
cbm_store_fetch_call_edges feeds it with a 400k-edge scan budget that
reports partial coverage rather than truncating silently. Opt-in ONLY —
never via a default or 'all' call, since it scans the whole call graph.
On this repo it surfaces 26 real cycles (e.g. the C-LSP type-evaluation
recursion) in ~5KB. Reproduce-first: a 3-node cycle is found, an acyclic
chain is not, and a default call does not run the scan.
get_code_snippet: cap the returned line span at 500 (a structural Module/File
node spans its whole file — an unclipped read returned ~400KB in one call).
source_clipped flags it; start_line/end_line still allow a targeted re-read.
Full suite 6,372 passed / 0 failed; lint clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- cbm_toon_* renamed to cbm_tree_* (the emitters have emitted tree syntax
since the format flip; the name now matches).
- format enums reduced to [tree, json], default tree; dead tree/toon flags
removed; tool RESPONSE descriptions rewritten for the grouped-tree
contract (group prefix printed once, full qn = prefix + '.' + name).
- format:'json' is now the json-stringified tree: the SAME grouped model
serialized as JSON — {total, cols, groups:[{qn_prefix, file,
rows:[[column-ordered cells]]}], has_more} for search, per-leg
{cols, groups} for trace (risk/args columns follow the flags),
query_graph's columns+rows shape already complied. Per-row key envelopes
and the legacy property blob are gone (blob-never-spills pinned by test);
requested add columns in both encodings.
- include_connected keeps the per-node-object shape (nested neighbors) —
the one remaining legacy consumer, conversion tracked.
- legacy verbose emitters deleted (bfs_to_json_array).
Full suite 6,367 passed / 0 failed; lint clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Round-2 overhaul of the tool-output surface, built eval-first (byte harness +
no-context field-eval agents, always vs the v0.9.0 release binary):
Format — tree syntax everywhere:
- compact_out emits tree tables: 'key: N (cols: a b c)' headers, indented
space-delimited rows, '-' placeholders, space-aware quoting. Every table
in every tool converts mechanically.
- Grouped prefix-factored output is the DEFAULT for search_graph and
trace_path: rows grouped under their shared (qn-prefix, file) printed
once, row qn = group prefix + '.' + name (stated in-band). Measured:
192-hit search 2,929B (was 4,811 TOON / 238,466 v0.9.0), 82-row trace
2,256B (was 4,234 / 8,094). Field eval: 5/5 answers correct, QN
reconstruction verified, no comprehension loss.
- detail:'ids' tier: bare-qn enumeration for wide sweeps (-44%).
Pagination — exactly-once, stateless:
- store_meta generation (per-DB random uid + mutation counter bumped in
cbm_store_upsert_project — every index run's choke point): cursors can
never silently validate against a rebuilt DB.
- trace cursors: opaque token (leg, generation, params-hash, (hop,id)
watermark); pages drain callees then callers with exact
callees_total/callers_total on every page; teaching errors for
params-mismatch and staleness. Guard: 12 callees paged 5+5+2, every row
on exactly one page.
- deterministic ordering: BFS ORDER BY hop,id; canonical (hop,id) sort +
min-hop dedup across same-name seeds (hop fed risk_labels wrong before);
search (name,id)/(rank,id) tie-breaks make offset pages stable.
Lean defaults + teaching errors:
- get_graph_schema no longer advertises blocked internal fields (fp/sp/bt).
- index_repository coverage lists: counts + 5 examples + one-line pointer
notes (full record was always in the logfile); 6.0KB -> 1.5KB here,
53KB observed on a large repo. index_status gates the git context block
behind verbose:true.
- trace limit param (<=5000) + truncated:true — never a silent truncation;
invalid direction now errors with the valid values; requesting core
columns via fields hints instead of emitting empty cells; cypher
resolves f.file/f.qn/f.path aliases.
Trace-correctness fixes (reproduce-first, RED verified):
- bfs_collect_edges 4KB id-string overflow: past ~1000 visited nodes the
edge SQL failed to prepare and EVERY trace edge silently vanished; now a
per-connection temp-table join (star guard: 1200/1200 edges).
- bfs_union_same_name: min-hop across seeds + edge dedup.
Guards: gating tool-output regression suite (context-explosion detector:
blocked-field leaks, repeated-key envelopes, oversized notes + calibrated
byte ceilings + semantic floors). Full suite 6,367 passed / 0 failed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/benchmark-incremental-speed.py previously measured Git-history and HTTP-link runtime profiles without task-quality oracles, leaving enabled/disabled cost cells unable to demonstrate user-visible graph evidence.
Add deterministic, network-free fixtures that reuse FILE_CHANGES_WITH and HTTP_CALLS from the production pipeline. The Git fixture records four fixed-date co-change commits. The HTTP fixture places a Python client and source-discovered Ktor route in distinct service namespaces so only the optional HTTP linker produces the direct caller-to-handler edge.
Query existing relationship schemas with bounded results and require the declared paths, counterpart, count, caller, handler, and method/confidence evidence in the quality judgments. Register both cases in --capability-quality and cover fixture identity plus oracle queries in tests/test_benchmark_incremental_speed.py.
Optimized binary SHA-256 2f2ceeb052f78bcdfc2b64b6175cf016315f3c3b9e28af4ec1487624e2523806 passed Git and HTTP enabled smokes; the HTTP disabled control missed the direct fetch_order-to-configureRouting edge as required. Every isolated work root was removed.
Verification: 139 benchmark/campaign/report unittests passed; Ruff, scripts/check-source-safety.sh, git diff --check, and optimized MCP enabled/disabled smoke cases passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
mutation_reindex_details() interpreted candidate-default stale-warning metadata without considering an explicit similarity_enabled=false or semantic_edges_enabled=false override. The semantic-pair table correctly said capability disabled while the incremental mutation table contradicted it with deferred with warning.
Pass disabled_pair_capabilities from summarize_group() into mutation_reindex_details() and give the explicit off state precedence over candidate freshness policy metadata. Extend test_disabled_semantic_pair_control_is_not_described_as_freshness_deferral() to cover the mutation table.
Verification: 135 benchmark/campaign/report unittests passed; Ruff, scripts/check-source-safety.sh, and git diff --check passed. Audit-only regeneration preserved 3/3 semantic-off and 3/3 similarity-off large-corpus cells with zero corrupt, missing, duplicate, or unplanned results, and both reports now render capability disabled in the pair and mutation tables.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
scan_campaign() validated completion and result hashes but trusted the artifact inventory in attempt.json. Changed bytes, a missing worker log, or an unlisted extra file therefore remained status=complete even though phase timing and lifecycle evidence no longer matched the recorded attempt.
Add validate_attempt_artifacts() in scripts/run-benchmark-campaign.py. Completed attempts now require matching cell identity and status, then rebuild the recursive artifact inventory and compare every relative path, byte count, SHA-256, file count, and total byte count. Completions without an attempt field retain historical hand-authored-plan compatibility.
Add changed, missing, and unlisted artifact regressions in tests/test_benchmark_campaign.py and document audit-only re-inventory in docs/BENCHMARK_CAMPAIGN.md. Verification: 135 benchmark/campaign/report unittests passed; Ruff, scripts/check-source-safety.sh, and git diff --check passed. Strict audit-only scans rehashed 39 capability cells, 9 semantic cells, and 9 similarity cells with zero corrupt, missing, duplicate, or unplanned results.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Semantic quality cases store incremental-versus-fresh graph comparisons under pair_lifecycle.canonical_graph. correctness_findings() only read the case-level field, so rejected upstream semantic and similarity rows incorrectly reported that no stage-level witness was recorded even though retained results contained exact counts and first differences.
Add canonical_mismatch_finding() in scripts/summarize-benchmark-results.py and use it for both case-level and pair-lifecycle comparisons. Preserve declared-stale-view context and place the intentional deferred-freshness explanation before its expected mismatch witness.
Add test_pair_lifecycle_canonical_rejection_reports_exact_graph_witness() in tests/test_summarize_benchmark_results.py. Verification: 134 benchmark/campaign/report unittests passed; Ruff, scripts/check-source-safety.sh, and git diff --check passed. Audit-only regeneration completed 39/39 capability cells plus 9/9 semantic and 9/9 similarity cells with zero missing, corrupt, duplicate, or unplanned results.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
evaluate_pair_incremental_policy() treated every unconfigured binary as stale_on_incremental. The pinned upstream binary produced immediate similarity pair freshness without a stale warning, so the evaluator rejected correct observed behavior against a policy that upstream does not expose.
Record candidate_default when incremental_derived_refresh is not explicitly configured. Classify the observed lifecycle as immediate_pair_freshness, immediate_full_freshness, deferred_with_warning, or unreported_stale; retain strict contract checks for explicit eager and deferred overrides. Separate pair freshness from whole-canonical-graph equality so each category remains visible.
Document cross-version default handling in docs/BENCHMARK_CAMPAIGN.md and extend tests/test_benchmark_incremental_speed.py with deferred, immediate-pair, unreported-stale, and explicit-eager cases.
Verification: uv run python -m unittest tests.test_benchmark_incremental_speed tests.test_benchmark_campaign tests.test_summarize_benchmark_results (133 tests); uv run ruff check on benchmark/report files; bash scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Andrew Hundt