Detect the documented Qwen Code, ForgeCode, and Windsurf config roots and register codebase-memory-mcp using each client's native mcpServers JSON path. Add owned QWEN.md and AGENTS.md instruction blocks where the clients load global guidance.
Use the same code paths for install plans and uninstall. Lifecycle tests preserve foreign MCP servers and user-authored instructions while removing only codebase-memory-mcp entries; plan tests verify no files are written.
Document the complete supported-client matrix and correct the Claude hook description. The focused ASan/UBSan CLI suite passes 144 tests.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add --config-profile choices in scripts/benchmark-incremental-speed.py and record the selected profile beside its expanded config_overrides map.
Define minimal_indexing as auto_index_deps=false plus rank_enabled, similarity_enabled, semantic_edges_enabled, githistory_enabled, and httplinks_enabled set to false. Keep repeated --config values higher priority for controlled ablations.
Document the distinction between optional_graph_disabled and minimal_indexing in docs/BENCHMARK_CAMPAIGN.md. Validate with 29 Python unit tests, Ruff, and scripts/check-source-safety.sh.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Set CBM_PROFILE=1 in build_env() for every candidate so index_supervisor.c preserves successful worker logs long enough for build_index_result() to stream mem.phase, pipeline.done, and incremental.done markers. Document that the profiling configuration is consistent across revisions and remains part of the auditable benchmark environment.
Validated with 27 pytest cases across the benchmark, summarizer, and campaign suites; Ruff and git diff --check pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Run each cell from scripts/run-benchmark-campaign.py in an isolated process group. On timeout or SIGINT, signal the group, allow 30 seconds for harness cleanup, force-stop remaining descendants, write attempt.json, and release running.lock. This addresses the detached cbm-self-dogfood worktree left by an interrupted pilot.
Read at most 512 mem.phase, pipeline.done, and incremental.done lines from the response logfile in build_index_result() using a streaming scan. Replace the route_handler comment literal with an executable cbm_http_path_match(path, "/api/pan4-oracle") mutation and target the route oracle with name_pattern=pan4-oracle.
Validated with 26 pytest cases, including descendant timeout cleanup, bounded worker-log metrics, and executable route mutation. Ruff and git diff --check pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Route run_cli_tool_call() and run_mcp_tool_call() through a default-format request for elapsed_ms, response_bytes, and response_token_estimate, then issue format=json only for quality scoring. Record quality_probe_elapsed_ms and quality_response_bytes so the second request cannot inflate the user-facing latency or replace TOON payload sizing.
This fixes the pilot JSONDecodeError: Expecting value at line 1 column 1 when search_graph returned its default TOON content. Document the two-request instrumentation contract in docs/BENCHMARK_CAMPAIGN.md.
Validated with 23 focused pytest cases across tests/test_benchmark_incremental_speed.py, tests/test_summarize_benchmark_results.py, and tests/test_benchmark_campaign.py; Ruff and git diff --check pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add scripts/run-benchmark-campaign.py with SHA-256 cell identities over revisions, binaries, release flags, capabilities, transports, scenarios, repetitions, harnesses, commands, environments, and timeouts.
Retain timestamped command/stdout/stderr/result/attempt artifacts; validate binary and result hashes before atomically writing complete.json. Reject live locks, record stale-lock recovery, enforce disk headroom, archive exact plan bytes and environment snapshots, and classify missing, corrupt, duplicate-attempt, and unplanned run directories in immutable manifests.
Regenerate reports atomically from validated completion records only. docs/BENCHMARK_CAMPAIGN.md documents default, rank_enabled=false, and all-optional-capabilities-off cells plus release-build metadata requirements.
Validated by 19 unittest cases and ruff check across the campaign, measurement, and report scripts; git diff --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Merge upstream/main at 2469ecc3 into integrate-upstream-split-search-tools while retaining branch-specific incremental indexing, MCP API consolidation, and graph extraction behavior.
The staged tree adds upstream's Mojo and Perl grammar support, issue reproduction suites, Graph UI changes, Windows smoke coverage, TOON graph output, subprocess supervision, and Laravel facade route handling. Conflict resolutions in src/pipeline/httplink.c and pass_httplinks.c retain dynamic growth and IMPORTS edge identity; scripts/smoke-test.sh and scripts/soak-test.sh retain hermetic cache isolation.
Validation against the merge plus the follow-up working-tree reconciliation: ASan/UBSan MCP suite 216/216 passed. Follow-up semantic and concurrency fixes remain intentionally unstaged for the next focused commit.
The per-language chapters were LLM-drafted to establish format and
dimension mapping — their symbol targets are not yet ground-truth-derived
and must be regenerated at execution time per §3.1/§12. Make that status
explicit at the top of Appendix A so the drafted symbol names are not
read as validated.
Supersedes #420 (same diff; recreated for DCO author/sign-off).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add CBM_ALLOWED_ROOT to the environment-variable tables in README.md and
docs/CONFIGURATION.md: when set, index_repository refuses a repo_path that
resolves outside it; unset imposes no restriction.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Documents the current behavior as implemented in src/discover/discover.c
and src/discover/gitignore.c: where .cbmignore is read from, the glob
features the parser supports, the layered precedence against built-in
skip lists / .gitignore hierarchy / git global excludes, and what
negation can and cannot override today. Planned negation unification
(un-skipping built-in dirs, non-negatable safety core, shared predicate
for auxiliary walkers) is listed in an explicitly not-yet-implemented
subsection. Linked from the README "Ignoring Files" section.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Replace stale 14-tool and incremental-refresh wording with the current streamlined-plus-classic tool model and configurable reindex policy.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Update README, generated docs, and bundled skills so auto-index and watcher text no longer implies incremental refresh is the default path.
Keep the wording aligned with the current config-backed default: full atomic rebuild unless incremental_reindex is explicitly enabled for benchmarking or canary coverage.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Use trace_path as the single public call-tracing tool across classic and streamlined MCP surfaces, while exposing existing semantic_query and search_code mode parameters in the advertised schemas. Keep the streamlined surface compact, but ensure the hidden-tool reveal covers the full classic capability set without trace aliases.
Persist PageRank, LinkRank, and node_degree rows with direct VALUES inserts instead of per-row SELECT probes. The node/edge project is loaded once with the graph rows so full-scope dependency attribution stays identical to the previous behavior.
Add focused coverage for dependency project attribution, streamlined parameter contracts, and targeted suite allowlist entries so these checks cannot silently run zero tests.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add docs/SECURITY-DISCLOSURE.md describing how vulnerability reports are
handled end to end (private fix, cross-platform validation, reporter
verification, patched release, then a GitHub Security Advisory with a CVE
and credit).
Update SECURITY.md: add GitHub private vulnerability reporting as the
preferred channel, replace the over-tight 48h/7-day commitment with honest
best-effort targets for a solo-maintained project, add a safe-harbor
statement, and refresh the stale supported-versions table (0.5.x -> 0.8.x).
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
Removes the heaviest vendored grammar (66 MB) along with its language
wiring and test fixtures; the supported-language count moves to 158
across README, site, and manifest. The grammar manifest additionally
records the canonical-source decisions for the five
registry-disagreement grammars and the per-directory license files
restored earlier this week.
Describe the Hybrid LSP layer as a lightweight C implementation
structurally inspired by and compatible with major language servers,
rather than a clean-room re-implementation of their algorithms, across
README, site, and llms.txt. Drop intelephense from the reference list.
Java, Kotlin, and Rust join the Hybrid LSP sections: badge and intro counts go 6 -> 9 language families, the capability tables gain the three new resolver rows (README + docs/index.html), the clean-room reference list adds Eclipse JDT and rust-analyzer, llms.txt and the landing-page JSON-LD/FAQ are aligned, the stale test-count badge moves to 5,577, and the sitemap lastmod is refreshed.
The legacy "Deploy from a branch" Pages build ran Jekyll on every push to
main and failed on the planning docs under /docs — Jekyll's Liquid parser
hit `Unknown tag 'data'` in EVALUATION_PLAN.md, so the website build went
red after nearly every commit.
/docs is a hand-written static site (index.html + robots/sitemap/llms.txt),
so it never needed Jekyll. Deploy it with a GitHub Actions workflow that
uploads /docs as-is and only triggers on docs/** changes, and add a
.nojekyll guard. Switch the Pages source to "GitHub Actions" for this to
take over from the legacy build.
Co-authored-by: DeusData <joerge@datadice.io>
Address peer-review feedback on question provenance/external validity.
- Anchor the five dimensions to Sillito, Murphy & De Volder's catalogue of
questions programmers ask during software-evolution tasks (FSE'06/TSE'08),
with LaToza & Myers (2010) and Ko et al. (ICSE'07) as support; add a
D1-D5 -> Sillito-group mapping table
- Adopt published repo-level QA benchmarks for the major languages: reuse and
compare against SWE-QA (primary), with CoReQA, CodeRepoQA and RepoQA as
references; note snippet-level sets (CodeQA/CS1QA/CoSQA) as the contrast
that motivates structural retrieval
- Policy: anchor to Sillito; reuse SWE-QA for major languages; generate the
multilingual remainder from independent LSP/git ground truth, never from
the model under test
Rename TEST_PLAN.md to EVALUATION_PLAN.md and rebuild it as a peer-review
evaluation specification covering all 159 supported languages.
- Graph (MCP) vs Explorer (grep/glob/read) per language, 5 bespoke questions
mapped to 5 universal dimensions (D1-D5) for cross-language aggregation
- Sequential, main-channel execution: one language at a time, Explorer the
only subagent, per-language index lifecycle (cold index -> eval -> delete)
to avoid concurrent-indexing memory blowups; index time is a key metric
- LLM-as-a-Judge grading (single disclosed model, blind A/B, median of passes)
- Per-language node-type and edge-type histograms with every one of the 32
edge types listed, zeros kept (a zero is a real extraction-gap signal)
- Cross-repo intelligence + semantic/similarity deep-dive for the 9 LSP
languages, with recall/precision ground truth
- Single result tree (eval-results/), retiring the versioned-dir scheme
- Master repo assignment for all 159 languages; per-language chapters in
Appendix A
Surface previously undocumented capabilities on the site, README, and
package metadata: semantic vector search (bundled nomic-embed-code
embeddings, fully local), SEMANTICALLY_RELATED / SIMILAR_TO clone
detection, cross-repo intelligence, data-flow tracing, change-impact
analysis, and ADR management. Broaden the cross-service entry to cover
gRPC/GraphQL/tRPC and pub/sub channels. Add a "star on GitHub" nav CTA.
GEO/SEO: add a question-style semantic-search section, extend the
JSON-LD featureList and FAQPage, add semantic keywords, and refresh the
sitemap lastmod.
Correct the language count to 159 (was 155/158) everywhere while keeping
the 157 vendored-grammar figure, update the Linux-kernel index stats to
the measured 4.81M nodes / 7.72M edges, and make trace_path the canonical
tool name (trace_call_path noted as an alias).
Resolved 5 conflicts where the php-lsp branch and main both added LSP
infrastructure but with different APIs:
- Makefile.cbm: kept both TEST_PHP_LSP_SRCS and main's py_lsp/scope/
type_rep/stack_overflow test source variables in ALL_TEST_SRCS.
- internal/cbm/cbm.c: kept both #include lsp/php_lsp.h and lsp/py_lsp.h;
the per-language dispatch already supports stacking PHP after PYTHON.
- internal/cbm/lsp_all.c: kept both lsp/php_lsp.c + php_stdlib_data.c
AND lsp/py_lsp.c + python_stdlib_data.c in the compilation unit.
- src/pipeline/pass_calls.c: took main's version entirely.
find_lsp_resolution + resolve_single_call(ctx, call, lsp_calls, ...)
is language-agnostic — it matches any CBMResolvedCall whose caller_qn
hits the textual call's enclosing function and whose callee_qn
short-name hits the textual callee. PHP LSP populates the same
result->resolved_calls array via cbm_run_php_lsp, so the unified
resolver handles PHP for free. Dropped the php-branch's
qn_short_name + lsp_override_resolution helpers as superseded.
Locked confidence floor at main's CBM_LSP_CONFIDENCE_FLOOR = 0.6f
(was 0.5f on the php branch).
- tests/test_main.c: kept all suite externs and RUN_SUITE invocations
for php_lsp + py_lsp/py_lsp_bench/py_lsp_stress/py_lsp_scale +
scope + type_rep.
Auto-merged: src/pipeline/pass_parallel.c.
Brings in 52 commits from main since the branch was forked at c768b61
(#207 watcher follow-up): #208 through #281, dependabot bumps, Pine
Script, AUR docs, INHERITS edges, get_architecture wiring, codeql
revert, the entire python-lsp subsystem.
Suite: 3249 passed, 0 failed (was 2971 on main; +278 new tests from
php-lsp matches the branch's claimed test count).
BENCHMARK_PYTHON.md updated through Round 10. The stress-test suite
went from 22 patterns (19 hard / 3 KNOWN GAP) to 43 patterns (all
hard-asserted PASS). Bench held at 100% (52/52) across the surface
expansion.
Patterns added in Round 10 cover the framework / real-world cases
that distinguish parity-with-LSP from passing-toy-fixtures:
SQLAlchemy 2.0 Mapped[T], Pydantic BaseModel, async with / async
for, Callable[..., R], next(iter), **kwargs annotation, dict-literal
dispatch tables, lambda call-site inference, recursive types,
classmethod-to-method chains, dataclass default_factory, nested
match patterns.
The original Phase 11 plan called this category "compiler-grade".
The actual line where compiler-rebuild starts is now narrower:
constraint solving for ParamSpec, code-flow narrowing across
basic blocks, custom metaclass __call__, and full descriptor
protocol. None of those are needed for what's resolved on the
bench fixture or the stress suite.
BENCHMARK_PYTHON.md ends the parity push with the in-process bench
results table (81% -> 79% -> 95% across rounds), an explicit
stopping-point list of features that would require building a
constraint solver / code-flow engine / plugin system, and an
"achievable next steps" list for future iterations that stay below
the compiler-rebuild line.
The 81% -> 79% jump going from a 65-line fixture to 136 lines is
deliberate: the smaller fixture was biased toward the resolver's
strengths. The 79% -> 95% jump comes from instance-attribute typing
(Round 4) which closed the largest single gap on real-world code
shapes.
Total: 11 LSP commits on this branch (Phases 0-11 plus Rounds 1-4),
2892 / 0 passing, 95% resolution on the parity fixture.
Edge counts (laravel/framework):
- Total edges: 196,979 (baseline) -> 152,614 (-22%)
- CALLS edges: ~83,000 (baseline) -> 50,150 (-40%)
The drop is from suppressing name-fallback misroutes when the receiver
is statically typed to a vendor class that isn't indexed. The Phase-4
capabilities (generic templates, narrowing, property tracking, full
parent-walk method dispatch, expanded stdlib) cover most of phpactor's
type-resolution surface for in-project receivers.
100 unit tests, all passing. Total project tests: 2913 / 0 failed.
Final phase of Python LSP integration. Adds:
- docs/BENCHMARK_PYTHON.md — methodology document covering target
metrics (per-file LSP overhead < 10 ms, resolution ratio ≥ 40% on
application code, ≥ 70% with stdlib, < 5% spot-check false positive
rate), the declared accepted-failure-class ceiling (metaclass
__call__, __getattr__/__getattribute__, dynamic setattr, runtime
importlib, monkey-patching, eval/exec, descriptors beyond the
standard property/classmethod/staticmethod, dynamic type() classes),
and instructions for running scripts/benchmark-index.sh against the
largest Python repos in datadice (falkemedia: 232 files, api-hub:
188 files).
- tests/test_py_lsp_bench.c — in-process benchmark wired into
scripts/test.sh. Runs cbm_extract_file on a 65-line Python fixture
covering imports, dataclasses, classmethods, single-inheritance with
super(), typed parameters, attribute chains, and stdlib calls.
Asserts soft floor (resolved >= calls/4) and hard time budget
(< 50 ms under sanitizers). Initial run reports:
bench: 65 lines, 16 calls, 13 resolved (81%), 2.45 ms
81% beats the 40% application-code target and the 70%
application-plus-stdlib target. 2.45 ms under ASan + UBSan is well
inside the 10 ms (no-sanitizer) production budget.
This wraps the 11-phase plan documented in PYTHON_LSP_PLAN.md. Total
test count: 2874 / 0 (100% pass). Repository-level benchmarks via
scripts/benchmark-index.sh remain a manual operation against a fresh
production binary; methodology is now in place for when that runs.
PHP score moves from 9/12 (75%) to 10/12 (83%); the two remaining
PARTIALs (Q6 search recall, Q8 trace tool disambiguation) are non-LSP
issues tracked separately per PHP_LSP_PRE_FLIGHT.md §3 and §4.1.
Headline change is a graph-correctness win on laravel/framework: the
collide-set $prompt->value() misroute is gone, and unindexed-vendor
receiver calls no longer produce wrong name-fallback edges (total
edges drop ~14% from 196,979 to 169,268, trading recall for precision).
Pre-flight against laravel/framework establishes that the PHP graph has
systemic call-attribution drift on short names shared between a global
function and a method (e.g. value(), make(), get()). Spot-check shows
~20-30% of typed-receiver method calls misroute to the global helper
because the existing extractor cannot resolve receiver type.
Plan adds internal/cbm/lsp/php_lsp.{c,h} as a third sibling to go_lsp.c
and c_lsp.c. In-process C type resolver, no PHP runtime, no new
dependencies. Uses already-vendored tree-sitter-php grammar and the
existing CBMArena / CBMScope / CBMType / CBMTypeRegistry primitives.
Phase 1 success metric is collide-set attribution correctness, not
benchmark tier alone. Q6 (text search) and Q8 (trace tool
disambiguation) tracked as separate tickets.
- Delete cmd/codebase-memory-mcp/assets/ — skills and codex instructions
are embedded as C string literals in src/cli/cli.c since v0.5.0
- Move BENCHMARK.md to docs/ to reduce root clutter
- Fix setup scripts: replace stale go build with scripts/build.sh
- THIRD_PARTY.md: replace Go module dependencies with vendored C
libraries (sqlite3, mimalloc, mongoose, yyjson, xxhash, tre),
fix CGo reference to static compilation
The snippet handler now uses exact QN + suffix matching only (no fuzzy/auto-resolve).
Updated 3 tests to match: fuzzy_suggestions, fuzzy_last_segment, auto_resolve_enabled.
- Add field('path', ...) to Magma load_statement grammar rule so
parse_generic_imports() finds the import path via field lookup
instead of the broken text fallback (which only extracted 1 per file)
- Fix passImports() to resolve file-path imports (e.g. "utils.mag",
"lib/helpers.h") via fqn.ModuleQN() when raw path doesn't match
any node QN — general fix benefiting any file-path-based import
- Add TestMagmaImport_Regression and TestMagmaCall_Regression
- Update language count 59 → 63 in README, docs/index.html, marketing
- Languages: 35 -> 59
- Add Gemini CLI, VS Code, Zed to supported clients list
- Update benchmark description to match v7 methodology
- Update meta tags and comparison table
Andrew Hundt