Add run_mcp_surface_parity() coverage for classic discovery, streamlined discovery before _hidden_tools, and relisting in the same process after reveal. Compare complete tools/list contracts, validation-equivalent get_code aliases, capability outcomes, bounded hidden-handler recognition, and notifications/tools/list_changed without presenting dispatch recognition as behavioral proof.
Make McpClient cleanup record subprocess return codes and require both reader threads and server processes to be reaped. Close SQLite snapshot and cloned-fixture connections explicitly so ResourceWarning-as-error runs detect ownership regressions.
Render capability outcomes before tool counts in scripts/summarize-benchmark-results.py, retain response bytes and single-observation latency with explicit evidence limits, and document the parity mode in docs/BENCHMARK_CAMPAIGN.md.
Verification: 141 warning-clean benchmark/campaign/summarizer tests; optimized build/c/codebase-memory-mcp parity smoke passed for 12 capability groups with full post-reveal classic contract parity and reaped lifecycle; ruff check; source-safety; git diff --cached --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Response-format consolidation: add cbm_mcp_response_format/
cbm_mcp_invalid_response_format so every response-generating handler
reads the format argument or default_response_format config through one
function instead of duplicating the enum-mapping logic, and migrate
query_graph's legacy-JSON detection to the canonical
cbm_mcp_output_format_t enum. Fix the regression this surfaced: TOON-path
_context/session_project delivery (toon_append_context_once) was
serializing the one-shot context object as a raw escaped JSON string
inside the TOON stream instead of native TOON fields, silently dropping
it for any TOON-consuming client. Add toon_append_context_model plus its
scalar/table helpers to map the same format-neutral context model
inject_context_once already builds into native TOON output.
Self-healing zero-row hints: query_graph_no_rows_hint reparses the
already-executed query on the zero-row path only, walks MATCH/OPTIONAL
MATCH patterns and EXISTS predicates in WHERE and post-WITH-WHERE
clauses (hint_walk_expr_exists_types/hint_walk_where_exists_types), and
probes each referenced label/edge type against the store via
cbm_store_schema_label_observed/cbm_store_schema_type_observed
(schema_probe_one_row: fail-open on any non-SQLITE_DONE outcome, so a
probe error never produces a false accusation). Unobserved names are
named in the response; a bounded hint_seen_t/hint_seen_add set (capped
at CBM_STORE_SCHEMA_HINT_VOCAB_LIMIT) dedups names repeated across
patterns/UNION branches so "Klass, Klass." can't happen and avoids
re-probing a name already resolved. The vocabulary summary and the
generic zero-row fallback no longer claim WHERE-clause predicates were
verified, since only labels/edge types are ever probed — only the
labels/edge types claim is made.
One invalidation authority: every graph-mutation path now calls
the single cbm_mcp_server_notify_index_published (flags-only,
any-thread-safe) instead of ad hoc local staleness bookkeeping —
sync auto-index, in-process index_repository, in-process
autoindex_thread, overlay compaction (gated on compacted>0),
index_dependencies. handle_delete_project's notify is gated on the
mutation actually succeeding (!is_error): a delete of a nonexistent
project is a no-op and must not stale the tools/list description cache
for nothing. The request-thread-only drain
(mcp_drain_tools_list_changed) is factored once and shared by both
transports (line-delimited and Content-Length-framed), gated so a
notification published before any tools/list has ever been served is
suppressed rather than sent for a client that never asked.
Shared overlay-aware schema-view selector (mcp_get_current_schema/
mcp_overlay_view_ready) ends the schema/overlay-readiness dance
previously triplicated across handle_get_graph_schema,
build_resource_schema, and the query path, so query_graph's advertised
vocabulary can never diverge from the view a query actually runs
against. The selector takes an mcp_cypher_vocabulary_t level instead of
a bool: MCP_CYPHER_MATCH_VOCABULARY (labels/edge types/observed
patterns with counts — enough to compose executable MATCH clauses;
never parses property JSON) vs MCP_CYPHER_FULL_QUERY_VOCABULARY (adds
per-label/type property keys for WHERE/RETURN authorship; O(total
property rows) json_each discovery, so only the docstring cache and
get_graph_schema request it).
Route the two remaining canonical-only cbm_store_get_schema callers
through the selector at MATCH_VOCABULARY level: inject_context_once
(first-response _context previously paid the full property-key scan for
counts it never rendered, and could advertise canonical labels a ready
overlay had tombstoned) and build_resource_architecture (rendered only
rel_patterns yet fetched full property keys, canonical-only). The
architecture resource's mixed-read-model disclosure moves
relationship_patterns into the overlay-aware list accordingly.
build_query_graph_tool_description resolves the project's real
store via resolve_store(srv, project) instead of trusting srv->store
as-is: on the very first tools/list of a fresh session for an
already-indexed project, srv->store was still the empty default
in-memory store (maybe_auto_index detects "already indexed" and skips
reindexing without ever opening it), so the schema section rendered
silently empty until some other tool call happened to resolve it first.
The project name is snapshotted into a local buffer before the
resolve_store call rather than passed as a live alias to
srv->current_project, because resolve_store can call
reap_stale_store internally, which frees srv->current_project when a
deferred invalidation is pending — aliasing it produced a
heap-use-after-free (caught by the ASan full-suite run, confirmed fixed
by rerun).
Docstring clarity for newcomers: the live schema section now embeds its
notation legend in the section header ("Labels name{extra property
keys}[count]:") so the compact schema is self-explanatory in every
tools/list response, and the query_graph/get_architecture/search_code
format parameters gain the brief description their three siblings
already had, all naming default_response_format.
Tests: response-format and TOON-context regressions in test_mcp.c/
test_tool_consolidation.c/test_input_validation.c/test_token_reduction.c;
zero-row hint coverage including no-hidden-tool-name, missed-graph shadow-
project probing, empty-project no-crash, TOON/JSON format parity,
UNION/repeated-name dedup, and the previously-untested EXISTS-predicate
walk; notification coverage for every notify call site including the
delete_project no-op negative direction and the "notify before first
tools/list is suppressed" negative direction; a cold-start regression
test seeding a real on-disk project store and calling tools/list as the
literal first request of a fresh session; overlay-consistency tests for
the two rerouted callers (first_response_context_uses_ready_overlay_schema,
resource_arch_rel_patterns_use_ready_overlay) proving _context and the
architecture resource report the active-overlay vocabulary, not
tombstoned canonical labels/types.
Docs: README.md documents the zero-row hint behavior and the
default_response_format config example; docs/CONFIGURATION.md adds the
default_response_format row to the important-keys table.
Verification: make -f Makefile.cbm test (full ASan/UBSan suite) run on
this exact tree after the final revisions. Earlier iterations of this
content additionally passed make -f Makefile.cbm test-leak (0 leaks for
0 total leaked bytes) and manual checks against the live MCP binary
(zero-row hint names real schema on two repositories; the query_graph
tools/list docstring shows the live schema on the very first request of
a fresh session); revisions since those checks are docstring text, the
vocabulary-enum rename, and the two selector reroutes, all covered by
the full-suite rerun.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
docs/CONFIGURATION.md:82 previously listed auto_index=false even though CBM_CONFIG_REGISTRY in src/cli/cli.c defaults the key to true. This could lead users to assume streamlined first-use indexing was disabled.
tests/test_cli.c:3837 now derives the expected Markdown row from CBM_CONFIG_REGISTRY, so future default changes must keep the configuration table synchronized.
Verification: CBM_ONLY_SUITE=cli build/c/test-runner (151 passed); bash scripts/check-source-safety.sh; git clang-format --diff HEAD -- tests/test_cli.c.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
scan_campaign() validated completion and result hashes but trusted the artifact inventory in attempt.json. Changed bytes, a missing worker log, or an unlisted extra file therefore remained status=complete even though phase timing and lifecycle evidence no longer matched the recorded attempt.
Add validate_attempt_artifacts() in scripts/run-benchmark-campaign.py. Completed attempts now require matching cell identity and status, then rebuild the recursive artifact inventory and compare every relative path, byte count, SHA-256, file count, and total byte count. Completions without an attempt field retain historical hand-authored-plan compatibility.
Add changed, missing, and unlisted artifact regressions in tests/test_benchmark_campaign.py and document audit-only re-inventory in docs/BENCHMARK_CAMPAIGN.md. Verification: 135 benchmark/campaign/report unittests passed; Ruff, scripts/check-source-safety.sh, and git diff --check passed. Strict audit-only scans rehashed 39 capability cells, 9 semantic cells, and 9 similarity cells with zero corrupt, missing, duplicate, or unplanned results.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
evaluate_pair_incremental_policy() treated every unconfigured binary as stale_on_incremental. The pinned upstream binary produced immediate similarity pair freshness without a stale warning, so the evaluator rejected correct observed behavior against a policy that upstream does not expose.
Record candidate_default when incremental_derived_refresh is not explicitly configured. Classify the observed lifecycle as immediate_pair_freshness, immediate_full_freshness, deferred_with_warning, or unreported_stale; retain strict contract checks for explicit eager and deferred overrides. Separate pair freshness from whole-canonical-graph equality so each category remains visible.
Document cross-version default handling in docs/BENCHMARK_CAMPAIGN.md and extend tests/test_benchmark_incremental_speed.py with deferred, immediate-pair, unreported-stale, and explicit-eager cases.
Verification: uv run python -m unittest tests.test_benchmark_incremental_speed tests.test_benchmark_campaign tests.test_summarize_benchmark_results (133 tests); uv run ruff check on benchmark/report files; bash scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Large c_new_leaf runs with the default stale_on_incremental policy passed every mutation oracle but differed from fresh rebuilds by 99 SEMANTICALLY_RELATED rows. summarize-benchmark-results.py labeled those disclosed stale views as REJECT: graph correctness, hiding the intended latency/freshness policy.
benchmark-incremental-speed.py now unions structured stale_with_warning views and reruns canonical comparison after excluding only SEMANTICALLY_RELATED rows when semantic_edges is explicitly stale. The declared-stale gate passes only when every remaining node, edge, property, and file hash equals the fresh rebuild; non-semantic or undeclared differences still fail.
summarize-benchmark-results.py reports Core graph separately from Full graph freshness and emits PASS: DECLARED STALE VIEWS for a verified deferred view. docs/BENCHMARK_CAMPAIGN.md documents the gate and tests cover stale-view collection, exact exclusion scope, core mismatches, decision text, and table categories.
Verification: uv run python -m unittest tests.test_benchmark_incremental_speed tests.test_benchmark_campaign tests.test_summarize_benchmark_results (133 tests); uv run ruff check on changed Python files; bash scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
scripts/benchmark-incremental-speed.py previously wrote rank_refresh=stale_on_exact for every matrix cell, including profiles labeled default. That changed candidate behavior and invalidated comparisons against binaries with different compiled policies.
Add RANK_REFRESH_CANDIDATE_DEFAULT and apply_rank_refresh_override() so default runs make no rank_refresh config write. Record the requested policy and whether an override was applied in synthetic, matrix, self-dogfood, and capability-quality reports; retain explicit eager, stale_on_exact, and stale_on_incremental experiments.
Document the comparison contract in docs/BENCHMARK_CAMPAIGN.md and cover CLI default, no-write, and explicit-write behavior in tests/test_benchmark_incremental_speed.py.
Verification: uv run python -m unittest tests.test_benchmark_incremental_speed tests.test_benchmark_campaign (90 tests); uv run ruff check scripts/benchmark-incremental-speed.py tests/test_benchmark_incremental_speed.py; bash scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add self_dogfood workload expansion in scripts/run-benchmark-campaign.py with exact repository revision/tree identities, candidate-scoped profiles, and result rejection on background mismatches.
Make create_self_dogfood_worktree check out the declared commit, retain pre/post mutation SHA-256 values, and add c_new_leaf for a real indexed create-file route alongside high-fanout fallback scenarios.
Document the JSON contract in docs/BENCHMARK_CAMPAIGN.md. Verify with 125 benchmark/report unit tests, Ruff, source-safety, and an optimized c_new_leaf smoke: incremental_exact 367 ms, fresh rebuild 4456 ms, canonical equality 1/1, cleanup complete.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previously, semantic pair fixtures ran only in synthetic repositories. A large-background run could copy ambient dirty files or duplicate this repository's tracked benchmark canaries, making cross-version quality and performance results neither isolated nor commit-auditable.
Add quality_background repo/revision/tree matrix identity and pass the exact commit to benchmark-incremental-speed.py. Materialize only tracked files with git archive, exclude dirty and untracked source state plus the benchmark task-definition subtree before overlaying one canary copy, safely validate archive paths, remove the transient archive, and retain source commit, tree, dirty status, exclusions, and copy policy. Campaign result validation rejects missing or mismatched background commit/tree identity.
The copy uses one Git archive process and linear extraction instead of one Git process per file. Runtime is O(tracked archive bytes + files), memory is bounded by tar metadata, and the source checkout/worktree registry is never modified.
Verification: uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed (76 passed); Ruff checks; git diff --check. The Git fixture proves committed VERSION=1 is copied while dirty VERSION=2, an untracked file, and tracked benchmarks/semantic-pairs-v1 canaries are excluded; campaign tests bind resolved path plus full commit/tree hashes and reject a retained revision mismatch.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previously, compact matrix expansion grouped all repetitions for one candidate/profile together, so thermal or ambient load drift could align with a configuration. Attempt records retained commands and logs but not pre/post host load, disk availability, or physical-memory context.
Add opt-in execution_order=paired_interleaved without changing legacy grouped expansion or archived plan hashes. The new order runs every candidate/profile cell once per repetition block and includes block and absolute position in cell identity. Record pre/post load averages, CPU count, physical memory when available, and campaign-filesystem total/used/free bytes in immutable command and attempt records plus the campaign environment snapshot.
The ordering pass is O(cells log cells) time and O(cells) plan memory; execution remains strictly sequential. Resource collection is constant-space and performs no background sampling or production instrumentation.
Verification: uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed (74 passed); Ruff checks; git diff --check. Tests also prove legacy grouped plans remain the default, paired plans execute repetitions as 1,1,1,1,2,2,2,2, and successful attempts retain both resource snapshots.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previously, semantic capability fixtures measured only the initial full index. They could not prove that a changed source file removed an old relationship, added a new relationship, or matched a fresh rebuild, and expected stale-on-incremental warnings appeared as undifferentiated quality misses.
Add content-addressed post-edit Go and Python sources whose single-file mutations swap the judged SIMILAR_TO and SEMANTICALLY_RELATED pairs. Retain pre/post hashes, changed paths, initial/incremental/fresh index results, pair witnesses, edge-score equality, and whole canonical-graph equality. Add incremental_semantic_freshness_eager as a one-setting profile for incremental_derived_refresh=eager.
Classify default stale_on_incremental separately from eager freshness: deferred publication conforms only when it is immediately canonical or returns the structured semantic_edges stale warning; eager publication requires fresh pair and canonical graph equality without that warning. Pair comparison remains O(observed pairs) in runtime and memory.
Verification: uv run python -m unittest tests.test_benchmark_incremental_speed tests.test_benchmark_campaign (72 passed); Ruff checks; jq manifest parse; gofmt fixture checks; git diff --check. On optimized arm64 binary 36a1297b2571729517b96467965f476f15cd1d60e545454bc0c296f3103b9e17, default similarity/semantic edits conformed with explicit stale warnings at 40/51 ms; eager edits produced TP=1/TN=2/FP=0/FN=0 and canonical fresh equality at 72/71 ms. These are retained single pilot observations, not effect-size claims.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previously, capability-quality campaigns covered only rank and dependency retrieval, so SIMILAR_TO and SEMANTICALLY_RELATED costs had no paired precision/recall evidence. FAST remained the implicit matrix mode even for relationships that require MODERATE or FULL indexing.
Add benchmarks/semantic-pairs-v1/manifest.json with content-addressed Go structural-clone and Python control-flow-variant tasks, explicit lexical negatives, and source hashes. Extend benchmark-incremental-speed.py with unordered pair identities, TP/FP/FN/TN witnesses, precision, recall, F1, false-positive rate, unjudged-background retention, numeric query-score normalization, and response-quality gates. Propagate index_mode through compact campaign cells and document durable task/report semantics.
The bounded scorer is O(judgments + observed pairs) in runtime and memory. Natural repository edges outside the explicit judgment set remain unjudged instead of becoming false positives.
Verification: uv run python -m unittest tests.test_benchmark_incremental_speed tests.test_benchmark_campaign (68 passed); ruff check on changed Python; jq manifest parse; gofmt fixture check; git diff --check. Optimized arm64 binary 36a1297b2571729517b96467965f476f15cd1d60e545454bc0c296f3103b9e17 produced TP=1/TN=2/FP=0/FN=0 for both enabled fixtures; each individually disabled profile removed its judged positive edge.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add cbm_config_apply_preset() in src/cli/cli.c with six named capability sets committed through one SQLite BEGIN IMMEDIATE transaction. Reuse the existing CBM_CONFIG_* keys from cli.h, depindex.h, pagerank.h, and pipeline.h instead of defining parallel settings.
Expose 'config preset list' and 'config preset apply <name>'. Return a nonzero status and identify any environment override whose effective value differs from the stored preset, so comparisons cannot silently run with a mixed configuration.
Document the quality and ablation presets in docs/CONFIGURATION.md. Cover the exact streamlined-quality and minimal-indexing values plus unknown-name rejection in tests/test_cli.c.
Verification: macOS Apple Silicon ASan/UBSan CLI suite 147/147 passed; optimized -O2 binary listed and applied classic-quality in an isolated cache; CBM_TOOL_MODE=streamlined correctly caused classic-quality to exit 1 with the effective-value warning; git diff --cached --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Five-repetition reports showed stable medians alongside occasional system outliers, but the tables exposed only p50/p95 values. That made it difficult to distinguish a typical configuration cost from observed host variability and could invite unsupported confidence claims.
Add query, incremental, and full-index observation counts plus min-max ranges to summarize-benchmark-results.py. The generated section explicitly states that grouped sequential campaigns avoid concurrent contention but do not support paired or randomized effect-size confidence intervals; medians and ratios remain descriptive.
tests/test_summarize_benchmark_results.py verifies exact counts and [min, max] rendering without confidence language. Verified with uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed tests.test_summarize_benchmark_results (93 passed). The ignored 240-input composition report and SHA-256 sidecar were regenerated but not staged.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Separate incremental, rank, and dependency campaigns could not produce categorical overall-quality or Pareto rows because each report lacked the other axes. Hand-merging raw files also risked mixing different config profiles and bypassing completion hashes.
Add --composition-spec to scripts/summarize-benchmark-results.py. Named campaign entries re-expand archived matrix specs, resolve exact cell labels, require every selected completion and binary/result SHA-256 to validate, and materialize derived inputs without changing raw results. Generated Markdown records the composition-spec hash and writes a per-input SHA-256 sidecar.
Preserve execution failures as correctness rejections, but classify a missed rank/dependency target as BELOW QUALITY TARGET when that exact report explicitly disables the corresponding capability. tests/test_summarize_benchmark_results.py covers campaign grouping and composed ablation wording.
Verified with uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed tests.test_summarize_benchmark_results (92 passed). The ignored composition validated 240 retained inputs and identified latest.default.mcp as the only fully passing measured Pareto-front row; generated reports and manifests were not staged.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Rank and dependency fixtures previously required hand-authored campaign cells, which made candidate, profile, transport, and repetition coverage easy to drift and omitted the concise matrix audit trail used by incremental benchmarks.
Add top-level capability_quality expansion in scripts/run-benchmark-campaign.py. Capability cells omit frontier/exact-cap axes, invoke benchmark-incremental-speed.py with --capability-quality and --include-logs, record the fixture in identity parameters, and retain named config profiles for enabled/disabled ablations. Existing incremental expansion remains the default when the field is absent.
tests/test_benchmark_campaign.py verifies a two-transport, two-repetition rank grid has four unique cells, no frontier parameters, the expected command, and accepted exit policy. Verified with uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed tests.test_summarize_benchmark_results (90 passed).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Compact matrix expansion in scripts/run-benchmark-campaign.py hardcoded accepted_exit_codes=[0]. benchmark-incremental-speed.py intentionally exits 1 after writing a valid result when canonical graph or quality gates reject a candidate, so campaigns mislabeled measured upstream failures as missing infrastructure.
Accept a validated top-level matrix-spec exit-code list, copy it into every expanded cell identity, and keep [0] as the default. Result parsing, binary SHA-256 verification, and the structured error check still reject crashes, missing artifacts, hash mismatches, and harness-error documents.
tests/test_benchmark_campaign.py verifies [0,1] propagation. Verified with uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed tests.test_summarize_benchmark_results (89 passed). A retained two-cell upstream probe now reports REJECT: graph correctness with canonical mismatch witnesses instead of missing cells.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Compact matrix specs previously required a positive exact cap and always injected incremental_exact_max_affected_paths. A default-versus-default historical comparison therefore had to overwrite candidate policy with one shared cap.
Allow null in scenarios[].exact_caps to omit the config override, omit the capability identity key, retain exact_cap=null in parameters, and label the cell capdefault. Positive integers keep the existing explicit cap-sweep behavior. Document both forms in docs/BENCHMARK_CAMPAIGN.md.
Tests: uv run python -m unittest tests.test_benchmark_campaign -v (16 passed); uv run ruff check scripts/run-benchmark-campaign.py tests/test_benchmark_campaign.py (passed); git diff --check (passed).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior rendered cleanup.requested=false as Cleanup 0/1, making intentionally retained audit evidence look like a failed lifecycle operation. The campaign guide also directed crash-sensitive plans and results to /tmp even though validate_campaign_root() rejects the operating-system temporary tree by default.
Add evidence_lifecycle() in scripts/summarize-benchmark-results.py to label requested disposal, deliberate retention, missing metadata, and requested cleanup failure separately. Rename the generated column to Evidence lifecycle. Point docs/BENCHMARK_CAMPAIGN.md at the ignored durable .worktrees/benchmark-campaign path. Replace the secondary nDCG reference with Järvelin and Kekäläinen's primary DOI 10.1145/582415.582418 and identify exact pairwise Pareto nondominance with Deb et al. DOI 10.1109/4235.996017.
Tests: uv run python -m unittest tests.test_summarize_benchmark_results -v (26 passed); uv run ruff check scripts/summarize-benchmark-results.py tests/test_summarize_benchmark_results.py (passed); git diff --check (passed).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Previous behavior: install detection and receipts omitted Claude Desktop even when its platform config directory existed, so claude_desktop_config.json required manual registration.
Add cbm_claude_desktop_config_path() at src/cli/cli.c:1267 and route the detected client through the existing mcpServers editor writer at lines 4504-4509. Route uninstall through cbm_remove_editor_mcp() at lines 5232-5237 so only the codebase-memory-mcp entry is removed.
Add plan/install/uninstall ownership coverage at tests/test_cli.c:2157 and platform-directory detection coverage at line 2479. Document the target in README.md, docs/CONFIGURATION.md, and the CLI help output.
Verification: CBM_ONLY_SUITE=cli build/c/test-runner (ASAN+UBSAN, native Apple Silicon macOS): 146 passed. CBM_ONLY_SUITE=cli build/c/test-runner-nosan: 146 passed. Native Windows execution was not performed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Detect the documented Qwen Code, ForgeCode, and Windsurf config roots and register codebase-memory-mcp using each client's native mcpServers JSON path. Add owned QWEN.md and AGENTS.md instruction blocks where the clients load global guidance.
Use the same code paths for install plans and uninstall. Lifecycle tests preserve foreign MCP servers and user-authored instructions while removing only codebase-memory-mcp entries; plan tests verify no files are written.
Document the complete supported-client matrix and correct the Claude hook description. The focused ASan/UBSan CLI suite passes 144 tests.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add --config-profile choices in scripts/benchmark-incremental-speed.py and record the selected profile beside its expanded config_overrides map.
Define minimal_indexing as auto_index_deps=false plus rank_enabled, similarity_enabled, semantic_edges_enabled, githistory_enabled, and httplinks_enabled set to false. Keep repeated --config values higher priority for controlled ablations.
Document the distinction between optional_graph_disabled and minimal_indexing in docs/BENCHMARK_CAMPAIGN.md. Validate with 29 Python unit tests, Ruff, and scripts/check-source-safety.sh.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Set CBM_PROFILE=1 in build_env() for every candidate so index_supervisor.c preserves successful worker logs long enough for build_index_result() to stream mem.phase, pipeline.done, and incremental.done markers. Document that the profiling configuration is consistent across revisions and remains part of the auditable benchmark environment.
Validated with 27 pytest cases across the benchmark, summarizer, and campaign suites; Ruff and git diff --check pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Run each cell from scripts/run-benchmark-campaign.py in an isolated process group. On timeout or SIGINT, signal the group, allow 30 seconds for harness cleanup, force-stop remaining descendants, write attempt.json, and release running.lock. This addresses the detached cbm-self-dogfood worktree left by an interrupted pilot.
Read at most 512 mem.phase, pipeline.done, and incremental.done lines from the response logfile in build_index_result() using a streaming scan. Replace the route_handler comment literal with an executable cbm_http_path_match(path, "/api/pan4-oracle") mutation and target the route oracle with name_pattern=pan4-oracle.
Validated with 26 pytest cases, including descendant timeout cleanup, bounded worker-log metrics, and executable route mutation. Ruff and git diff --check pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Route run_cli_tool_call() and run_mcp_tool_call() through a default-format request for elapsed_ms, response_bytes, and response_token_estimate, then issue format=json only for quality scoring. Record quality_probe_elapsed_ms and quality_response_bytes so the second request cannot inflate the user-facing latency or replace TOON payload sizing.
This fixes the pilot JSONDecodeError: Expecting value at line 1 column 1 when search_graph returned its default TOON content. Document the two-request instrumentation contract in docs/BENCHMARK_CAMPAIGN.md.
Validated with 23 focused pytest cases across tests/test_benchmark_incremental_speed.py, tests/test_summarize_benchmark_results.py, and tests/test_benchmark_campaign.py; Ruff and git diff --check pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add scripts/run-benchmark-campaign.py with SHA-256 cell identities over revisions, binaries, release flags, capabilities, transports, scenarios, repetitions, harnesses, commands, environments, and timeouts.
Retain timestamped command/stdout/stderr/result/attempt artifacts; validate binary and result hashes before atomically writing complete.json. Reject live locks, record stale-lock recovery, enforce disk headroom, archive exact plan bytes and environment snapshots, and classify missing, corrupt, duplicate-attempt, and unplanned run directories in immutable manifests.
Regenerate reports atomically from validated completion records only. docs/BENCHMARK_CAMPAIGN.md documents default, rank_enabled=false, and all-optional-capabilities-off cells plus release-build metadata requirements.
Validated by 19 unittest cases and ruff check across the campaign, measurement, and report scripts; git diff --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Merge upstream/main at 2469ecc3 into integrate-upstream-split-search-tools while retaining branch-specific incremental indexing, MCP API consolidation, and graph extraction behavior.
The staged tree adds upstream's Mojo and Perl grammar support, issue reproduction suites, Graph UI changes, Windows smoke coverage, TOON graph output, subprocess supervision, and Laravel facade route handling. Conflict resolutions in src/pipeline/httplink.c and pass_httplinks.c retain dynamic growth and IMPORTS edge identity; scripts/smoke-test.sh and scripts/soak-test.sh retain hermetic cache isolation.
Validation against the merge plus the follow-up working-tree reconciliation: ASan/UBSan MCP suite 216/216 passed. Follow-up semantic and concurrency fixes remain intentionally unstaged for the next focused commit.
The per-language chapters were LLM-drafted to establish format and
dimension mapping — their symbol targets are not yet ground-truth-derived
and must be regenerated at execution time per §3.1/§12. Make that status
explicit at the top of Appendix A so the drafted symbol names are not
read as validated.
Supersedes #420 (same diff; recreated for DCO author/sign-off).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add CBM_ALLOWED_ROOT to the environment-variable tables in README.md and
docs/CONFIGURATION.md: when set, index_repository refuses a repo_path that
resolves outside it; unset imposes no restriction.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Documents the current behavior as implemented in src/discover/discover.c
and src/discover/gitignore.c: where .cbmignore is read from, the glob
features the parser supports, the layered precedence against built-in
skip lists / .gitignore hierarchy / git global excludes, and what
negation can and cannot override today. Planned negation unification
(un-skipping built-in dirs, non-negatable safety core, shared predicate
for auxiliary walkers) is listed in an explicitly not-yet-implemented
subsection. Linked from the README "Ignoring Files" section.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Replace stale 14-tool and incremental-refresh wording with the current streamlined-plus-classic tool model and configurable reindex policy.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Update README, generated docs, and bundled skills so auto-index and watcher text no longer implies incremental refresh is the default path.
Keep the wording aligned with the current config-backed default: full atomic rebuild unless incremental_reindex is explicitly enabled for benchmarking or canary coverage.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Use trace_path as the single public call-tracing tool across classic and streamlined MCP surfaces, while exposing existing semantic_query and search_code mode parameters in the advertised schemas. Keep the streamlined surface compact, but ensure the hidden-tool reveal covers the full classic capability set without trace aliases.
Persist PageRank, LinkRank, and node_degree rows with direct VALUES inserts instead of per-row SELECT probes. The node/edge project is loaded once with the graph rows so full-scope dependency attribution stays identical to the previous behavior.
Add focused coverage for dependency project attribution, streamlined parameter contracts, and targeted suite allowlist entries so these checks cannot silently run zero tests.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add docs/SECURITY-DISCLOSURE.md describing how vulnerability reports are
handled end to end (private fix, cross-platform validation, reporter
verification, patched release, then a GitHub Security Advisory with a CVE
and credit).
Update SECURITY.md: add GitHub private vulnerability reporting as the
preferred channel, replace the over-tight 48h/7-day commitment with honest
best-effort targets for a solo-maintained project, add a safe-harbor
statement, and refresh the stale supported-versions table (0.5.x -> 0.8.x).
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
Removes the heaviest vendored grammar (66 MB) along with its language
wiring and test fixtures; the supported-language count moves to 158
across README, site, and manifest. The grammar manifest additionally
records the canonical-source decisions for the five
registry-disagreement grammars and the per-directory license files
restored earlier this week.
Describe the Hybrid LSP layer as a lightweight C implementation
structurally inspired by and compatible with major language servers,
rather than a clean-room re-implementation of their algorithms, across
README, site, and llms.txt. Drop intelephense from the reference list.
Java, Kotlin, and Rust join the Hybrid LSP sections: badge and intro counts go 6 -> 9 language families, the capability tables gain the three new resolver rows (README + docs/index.html), the clean-room reference list adds Eclipse JDT and rust-analyzer, llms.txt and the landing-page JSON-LD/FAQ are aligned, the stale test-count badge moves to 5,577, and the sitemap lastmod is refreshed.
The legacy "Deploy from a branch" Pages build ran Jekyll on every push to
main and failed on the planning docs under /docs — Jekyll's Liquid parser
hit `Unknown tag 'data'` in EVALUATION_PLAN.md, so the website build went
red after nearly every commit.
/docs is a hand-written static site (index.html + robots/sitemap/llms.txt),
so it never needed Jekyll. Deploy it with a GitHub Actions workflow that
uploads /docs as-is and only triggers on docs/** changes, and add a
.nojekyll guard. Switch the Pages source to "GitHub Actions" for this to
take over from the legacy build.
Co-authored-by: DeusData <joerge@datadice.io>
Address peer-review feedback on question provenance/external validity.
- Anchor the five dimensions to Sillito, Murphy & De Volder's catalogue of
questions programmers ask during software-evolution tasks (FSE'06/TSE'08),
with LaToza & Myers (2010) and Ko et al. (ICSE'07) as support; add a
D1-D5 -> Sillito-group mapping table
- Adopt published repo-level QA benchmarks for the major languages: reuse and
compare against SWE-QA (primary), with CoReQA, CodeRepoQA and RepoQA as
references; note snippet-level sets (CodeQA/CS1QA/CoSQA) as the contrast
that motivates structural retrieval
- Policy: anchor to Sillito; reuse SWE-QA for major languages; generate the
multilingual remainder from independent LSP/git ground truth, never from
the model under test
Rename TEST_PLAN.md to EVALUATION_PLAN.md and rebuild it as a peer-review
evaluation specification covering all 159 supported languages.
- Graph (MCP) vs Explorer (grep/glob/read) per language, 5 bespoke questions
mapped to 5 universal dimensions (D1-D5) for cross-language aggregation
- Sequential, main-channel execution: one language at a time, Explorer the
only subagent, per-language index lifecycle (cold index -> eval -> delete)
to avoid concurrent-indexing memory blowups; index time is a key metric
- LLM-as-a-Judge grading (single disclosed model, blind A/B, median of passes)
- Per-language node-type and edge-type histograms with every one of the 32
edge types listed, zeros kept (a zero is a real extraction-gap signal)
- Cross-repo intelligence + semantic/similarity deep-dive for the 9 LSP
languages, with recall/precision ground truth
- Single result tree (eval-results/), retiring the versioned-dir scheme
- Master repo assignment for all 159 languages; per-language chapters in
Appendix A
Surface previously undocumented capabilities on the site, README, and
package metadata: semantic vector search (bundled nomic-embed-code
embeddings, fully local), SEMANTICALLY_RELATED / SIMILAR_TO clone
detection, cross-repo intelligence, data-flow tracing, change-impact
analysis, and ADR management. Broaden the cross-service entry to cover
gRPC/GraphQL/tRPC and pub/sub channels. Add a "star on GitHub" nav CTA.
GEO/SEO: add a question-style semantic-search section, extend the
JSON-LD featureList and FAQPage, add semantic keywords, and refresh the
sitemap lastmod.
Correct the language count to 159 (was 155/158) everywhere while keeping
the 157 vendored-grammar figure, update the Linux-kernel index stats to
the measured 4.81M nodes / 7.72M edges, and make trace_path the canonical
tool name (trace_call_path noted as an alias).
Andrew Hundt