Add cbm_str_copy() as the shared allocation-free fixed-buffer copy helper and use it in httplink, envscan, and store file-tree code.
Remove guarded production strncpy() sites, add a source-safety rule and self-test for future strncpy regressions, and tighten the logging thread-safety comment to match the implementation.
Validation: make -f Makefile.cbm lint-source-safety; CBM_ONLY_SUITE=httplink make -f Makefile.cbm test; CBM_ONLY_SUITE=str_util build/c/test-runner; CBM_ONLY_SUITE=pipeline CBM_ONLY_TEST=envscan build/c/test-runner; CBM_ONLY_SUITE=store_arch build/c/test-runner.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Detect exact file deltas whose canonical graph rows already match the persisted store, then refresh only file freshness and incremental metadata under a completed generation. This keeps graph-changing exact deltas on the existing planner/apply path while body-only edits publish as incremental_noop with graph_changed=false.
Also update the incremental benchmark harness so incremental_noop is treated as an explicit incremental route while preserving the existing explicit_exact_or_fallback field for compatibility.
Validation: git diff --check; bash scripts/check-source-safety.sh; bash scripts/test-source-safety.sh; python3 -m py_compile scripts/benchmark-incremental-speed.py; make -j8 -f Makefile.cbm build/c/test-runner; CBM_ONLY_SUITE=store_nodes 81 passed; CBM_ONLY_SUITE=pipeline 295 passed; CBM_ONLY_SUITE=incremental 161 passed; make -j8 -f Makefile.cbm cbm; persistent-MCP affected-frontier matrix 8/8 canonical-equal.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add pipeline-owned publish_reason metadata and surface it in index_repository responses when incremental indexing falls back to containment. The reason is set on existing exact-route skip/fallback paths and cleared on exact publish/full run reset, so MCP and CLI JSON callers can distinguish a safe containment rebuild from an opaque incremental result.
Update the affected-frontier benchmark harness to prefer structured publish_reason while keeping stderr parsing for older binaries. Add focused pipeline assertions and an MCP tool test for incremental_containment with missing_existing_ownership.
Validated with git diff --check, bash scripts/check-source-safety.sh, bash scripts/test-source-safety.sh, python3 -m py_compile scripts/benchmark-incremental-speed.py, escalated make -j8 -f Makefile.cbm build/c/test-runner, escalated CBM_ONLY_SUITE=pipeline build/c/test-runner, escalated CBM_ONLY_SUITE=mcp build/c/test-runner, escalated make -j8 -f Makefile.cbm cbm, and escalated persistent MCP affected-frontier matrix 8/8 pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Extend the existing incremental benchmark with an opt-in --matrix mode for affected-frontier scenarios. The matrix isolates repos and caches per case, records publish_kind and exact/fallback reasons, compares incremental output to a fresh FAST rebuild with canonical SQLite graph rows, and cleans up generated work roots by default.
This is a measurement and regression harness, not a default-policy change. incremental_reindex remains default-off until the broader matrix, freshness semantics, cleanup, and speedup gates are satisfied.
Validated with python3 -m py_compile scripts/benchmark-incremental-speed.py, git diff --check, bash scripts/check-source-safety.sh, and an escalated two-case matrix smoke for go_modify_1/go_modify_2.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add --transport=cli|mcp to the incremental speed benchmark. The MCP path keeps one JSON-RPC server process alive for each measurement phase and detects full versus exact incremental routes from index_repository publish_kind metadata instead of stderr logs.
The harness still preserves the cold CLI path and cleanup behavior, and reports publish_kind alongside existing wall/logged timing fields for current and future performance comparisons.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add indexed_work_elapsed_ms and unlogged_overhead_ms to the incremental speed benchmark output while preserving the existing elapsed_ms and logged_elapsed_ms fields. This separates cold CLI wall time from the logged indexing path so future speedup work can target the correct latency term.
Validation: python3 -m py_compile scripts/benchmark-incremental-speed.py; git diff --check; scripts/benchmark-incremental-speed.py --files 20 --functions-per-file 4 --changed-files 1 --min-speedup 0.5 --timeout 120 --out /private/tmp/cbm-incr-speed-overhead-fields-smoke-pass.json
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add an opt-in benchmark that compares fast-mode exact incremental indexing against a fresh fast full rebuild in an isolated temp repo and CBM_CACHE_DIR. The gate requires the incremental.exact.done marker, records rank-refresh markers and bounded log tails, and cleans only paths it creates.
This does not change defaults. The current implementation is expected to fail the default 10x threshold until exact-delta post-passes are scoped or marked stale safely.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Extend the source-safety self-test to prove fprintf(stdout, ...) is rejected in MCP/pipeline code, matching the existing protocol-output guard.
Reuse store.c's heap_strdup helper for neighbor-name and LIKE-hint allocations instead of direct strdup calls. This keeps ownership behavior unchanged while reducing allocator-wrapper drift in touched store paths.
Validation: make -j8 -f Makefile.cbm build/c/test-runner; CBM_ONLY_SUITE=store_search ./build/c/test-runner; CBM_ONLY_SUITE=mcp ./build/c/test-runner; bash scripts/check-source-safety.sh; bash scripts/test-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Replace raw strdup calls in the incremental inbound-edge snapshot with cbm_strdup so the touched path follows the repository portability wrapper convention. Add a source-safety diff guard and self-test that reject newly introduced raw strdup calls in production code without failing historical existing uses.
Validation: git diff --check; make -j8 -f Makefile.cbm build/c/test-runner; make -f Makefile.cbm lint-source-safety; CBM_ONLY_SUITE=pipeline ./build/c/test-runner; CBM_ONLY_SUITE=incremental ./build/c/test-runner.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add a root-overridable self-test harness for scripts/check-source-safety.sh so the guard can be tested against synthetic trees without scanning the real repository.
Cover clean input, MCP/server stdout writes, unsafe string APIs, and new raw filesystem APIs in production diffs. Add a dedicated lint-source-safety target because the broader lint-no-suppress path is currently blocked by existing NOLINT audit failures before this guard can run.
Validation: bash scripts/test-source-safety.sh; bash scripts/check-source-safety.sh; git diff --check; make -f Makefile.cbm lint-source-safety.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Stabilize several graph-producing paths and add stronger regression coverage before the incremental redesign work continues.
Key changes:
- resolve call import maps from resolved graph-buffer IMPORTS edges so re-exported targets are reused consistently
- make semantic, similarity, complexity, route/section upserts, and resolver scoring more deterministic across full/incremental and worker-order differences
- propagate incremental phase/persist failures instead of continuing toward publication after failed phases
- add canonical graph diff helpers and stricter incremental artifacts for full-vs-incremental comparison
- add source-safety lint coverage for MCP stdout, unsafe string APIs, and new raw env/fs API usage
- replace touched unsafe string copies with bounded or project-wrapper-based forms
Validation:
- make -f Makefile.cbm build/c/test-runner
- CBM_ONLY_SUITE=pipeline ./build/c/test-runner: 217 passed
- CBM_ONLY_SUITE=graph_buffer ./build/c/test-runner: 55 passed
- CBM_ONLY_SUITE=mcp ./build/c/test-runner: 117 passed
- CBM_ONLY_SUITE=simhash ./build/c/test-runner: 24 passed
- CBM_ONLY_SUITE=parallel ./build/c/test-runner: 21 passed
- CBM_ONLY_SUITE=watcher ./build/c/test-runner: 57 passed
- bash scripts/check-source-safety.sh: OK
Known remaining:
- CBM_ONLY_SUITE=incremental still fails incr_accuracy_vs_full: 159 passed, 1 failed. Current artifacts show stale IMPORTS/CALLS/USAGE targets in unchanged importers and downstream SEMANTICALLY_RELATED drift; this is tracked in the forensic notes as a fallback-first incremental redesign task.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Deep --no-ff merge of upstream DeusData/main (597 commits, merge-base 988d975)
into fork api-consolidation (HEAD 3018b22), producing a superset that combines
upstream breadth with the fork's surviving depth.
Upstream additions integrated: semantic/embedding search (vendored/nomic),
minhash/SimHash, cross-repo + cross-file LSP + route + pkgmap passes, artifact
import/export, path_alias, git_context, hook_augment, Windows support
(compat_fs/compat_thread), the UI layer, and ~140 expanded tree-sitter grammars.
Fork depth retained: PageRank/LinkRank ranking (src/pagerank/; sort_by=
relevance/linkrank/degree/calls), dependency indexing (src/depindex/;
CBM_INDEX_MODE=DEP), token reduction, and HTTP-endpoint discovery
(src/pipeline/httplink.c, pass_httplinks.c).
Conflict resolution (27 regions across the 39-file intersection):
- src/mcp/mcp.c: kept the fork superset handler pattern (resolve_project_store +
expand_project_param + inject_context_once) as the single path; UNIONed the
tool dispatch table; emit status:"indexed", excluded[], cross_service trace.
- src/store/store.c: upstream base + fork pagerank/linkrank/node_degree tables
and sort_by plumbing; cbm_store_node_degree now excludes INHERITS
(WHERE type != 'INHERITS'); removed the duplicate cbm_store_get_db.
- src/pipeline/pipeline.c: took upstream run_extraction_phase; grafted the fork
passes (httplinks, normalize, flush_store dump branch); call
cbm_pipeline_set_committed_counts before cbm_gbuf_dump_to_sqlite.
- internal/cbm/lang_specs.c: 158 entries combining the fork section_node_types
with upstream ts_factory/embedded_imports; wired 13 orphaned type-arrays.
- src/depindex/depindex.h: renamed CBM_PATH_MAX -> CBM_DEP_PATH_MAX (it collided
with constants.h enum CBM_PATH_MAX=1024) + 11 call sites in depindex.c.
- Makefile.cbm: upstream grammar/nomic units + fork CGO targets; added
httplink.c + pass_httplinks.c to PIPELINE_SRCS; removed mongoose NOSAN
machinery (mongoose was deleted upstream).
- httplinks restored (the merge silently deleted them because the fork had left
them unchanged since the base): httplink.c/.h, pass_httplinks.c,
tests/test_httplink.c; registered RUN_SUITE(httplink) in tests/test_main.c.
- CBM_INDEX_MODE enum unified to FULL=0, MODERATE=1, FAST=2, DEP=3.
Completeness (dual-parent check): 0 upstream files dropped from the merge tree;
6 fork files dropped, all intentional (upstream-superseded): CHANGELOG.md,
vendored/mimalloc/src/{arena-abandon,segment-map,segment}.c,
vendored/mongoose/{mongoose.c,mongoose.h}. Fork features (pagerank, depindex,
token_reduction, httplink) verified present and registered in tests/test_main.c.
src/pipeline/pass_parallel.c has an identical function set to upstream (49=49,
none dropped); the large line-deltas vs upstream are within-function divergence
of the fork extractor, not dropped functions.
Gate (run manually; hooks skip on merge commits): build green
(make -f Makefile.cbm cbm, exit 0); full C test suite 5980 passed / 2 failed /
0 crashes. The fork's pre-existing ASan crash in build_project_list_error_srv is
fixed by upstream's mcp.c rewrite, so the merge is strictly better than the fork
baseline on the crash axis.
Known issue, tracked separately (NOT a merge defect): 2 nondeterministic
failures in tests/test_incremental.c — incr_db_deleted_recovery:772 (diff_pct)
and incr_accuracy_vs_full:803 (full_nodes vs incr_nodes). Cause: the fork's
extractor (internal/cbm/extract_*.c) and parallel-resolve Route-node creation
are not thread-safe under the concurrent use of upstream's parallel pipeline
(run_parallel_pipeline), producing nondeterministic route/infra HANDLES edges
(~234k edge swing between runs). Sequential extraction (CBM_WORKERS=1) is
deterministic and correct. Root cause, bisect, and repro are documented in
notes/2026-06-24-fork-vs-upstream-divergence-assessment.md (section 8f); the fix
is deferred to a dedicated thread-safety audit (the make test-tsan target is
currently a stub and needs wiring).
The MSYS2 CLANG64 toolchain bakes its package-tracker URL
(https://github.com/msys2/MINGW-packages) into the static Windows .exe via
libc++/compiler-rt. The Layer 2 binary string audit flagged it as an
unauthorized URL, failing smoke-windows (both the standard and ui variants)
while Linux smoke stayed green. Add it to ALLOWED_URLS alongside the other
compiler/linker toolchain URLs (gcc.gnu.org, sourceware.org, launchpad).
Add tests/test_security_strings_allowlist.sh as a reproduce-first regression
guard: it reproduces the BLOCKED failure with a binary fixture and includes a
negative control proving a genuinely unauthorized URL is still rejected. Wired
into scripts/test.sh as Step 6.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The UI license enumeration invoked npm by bare name via subprocess,
which fails on Windows (npm is npm.cmd there — WinError 2 in the
Windows UI-archive packaging step). Resolve it through the shell, which
handles platform shims everywhere; the command is a constant string.
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
It names license identifiers in its verdict maps by design — the same
documented false-positive category as the other license tooling.
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
Every commit must now carry a Signed-off-by trailer matching its
author, certifying the right to submit the change under the project's
MIT license (DCO 1.1, the Linux kernel mechanism). Enforcement is
strict at three layers: a commit-msg hook rejects unsigned commits
locally (scripts/install-git-hooks.sh), and the new DCO workflow
rejects every push and pull request containing one. Merge commits and
bot authors are exempt, matching standard DCO checks.
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
The license gate now proves itself before running: a --selftest mode
plants an unlicensed vendored file and asserts the structural layer
detects it, so a silently broken gate can no longer pass. The
byte-identity provenance audit gains exit-code semantics (any verdict
outside the accepted set fails) and runs as a blocking step in the
security workflow for both dry runs and releases.
A byte-identity audit of all vendored license files against their
upstream repositories found 113 copies differing only by a missing
trailing newline (vendoring artifact) — replaced with the exact
upstream bytes. Five needed real corrections: fennel carried an
unfilled MIT template although its upstream is CC0-1.0 at the pinned
commit; ron now carries the repo's LICENSE-MIT (dual MIT/Apache
upstream); python, wit, and verilog were refreshed to current upstream
bytes. The manifest's first-party table was also corrected: six of the
twelve grammars are self-maintained forks whose retained upstream
licenses are now byte-verified (assembly's upstream has been deleted
from GitHub; the retained MIT copy is the surviving grant). The audit
tool is kept as scripts/audit-license-provenance.py for future sweeps.
Structural layer: drop the hardcoded component list — any directory
containing vendored source or data files must now be covered by a
license file in itself or an ancestor within the vendored tree, so
newly vendored code without a license fails immediately. New third
layer: the graph-UI npm production tree is resolved (postinstall
scripts disabled) and every bundled package must be on the policy
allow-list; unknown licenses fail. Reclassifies @tailwindcss/vite as a
devDependency — it is a build plugin and was inflating the production
tree with native tooling.
The guard now runs a token-level clone detector (jscpd, with an npx
fallback) comparing internal/cbm/lsp/ C sources against clangd C++ — the
one reference close enough to C to tokenize alike — staging both trees as
a common format and reporting only clone pairs that span them. Catches
structural ports that rewrote strings/identifiers, which the verbatim
string/comment scan misses. clangd verified: no structural clones.
Scans internal/cbm/lsp/ for verbatim string/comment overlap against the
reference language servers (pyright, gopls, tsserver, Roslyn, clangd,
Eclipse JDT LS, kotlin-language-server, rust-analyzer, phpactor). Run
before committing new LSP work; add a new language's upstream reference to
the REFS manifest when implementing it. All 9 current languages verified
with no verbatim overlap.
The with-ui archives now append the per-package license texts of the
frontend bundle's production dependency tree to THIRD_PARTY_NOTICES.md
(platform-specific native build tooling is listed but excluded — its
code never reaches the browser bundle). The Glama check image carries
LICENSE and the notices file alongside the binary. Also renames a
TypeScript LSP test to describe its generic ODM-interface fixture.
The graph-UI HTTP server now owns its listening socket directly, so the
Layer 1 static audit's network-call scan would flag its socket()/bind()
calls. Exempt src/ui/httpd.c — it binds 127.0.0.1 only and is audited
separately by security-ui.sh — while still forbidding raw network calls
in every other source file.
Release archives now carry THIRD_PARTY_NOTICES.md, generated by
scripts/gen-third-party-notices.sh from THIRD_PARTY.md, the grammar
manifest, and the per-component license texts; the Homebrew formula
and AUR PKGBUILD install it alongside the binary. The SBOM gains
per-component license metadata, corrected versions, and the previously
missing vendored libraries. The security workflow gains a
vendored-license scan with an explicit allow-list policy, and the
release workflow exposes a skip_perf input for releases that do not
touch pipeline logic.
Removes the heaviest vendored grammar (66 MB) along with its language
wiring and test fixtures; the supported-language count moves to 158
across README, site, and manifest. The grammar manifest additionally
records the canonical-source decisions for the five
registry-disagreement grammars and the per-directory license files
restored earlier this week.
The web server behind the graph UI is now a purpose-built in-house
module (src/ui/httpd.c): localhost-only listener, strict HTTP/1.1
parsing with fixed request caps, a per-connection receive deadline,
and Connection: close semantics. http_server.c keeps the routing and
handlers, rewritten against the new transport API; the public server
API and main.c are unchanged. The previously vendored third-party
server is removed entirely.
Comes with a new 28-test transport + routing suite (tests/test_httpd.c)
covering parsing edge cases (strict CRLF, Content-Length limits, raw
path matching, percent-decode rules) and live-socket behavior (CORS
policy, RPC dispatch, receive deadline, clean shutdown). The security
audit scripts now check the new file layout and treat any network call
in vendored code as a failure.
mimalloc bumped to v3.3.2 (MI_MALLOC_VERSION 30302), which adds static-CRT malloc override on Windows. Makefile: define the real MI_MALLOC_OVERRIDE=1 ONLY on MinGW (enables the static-CRT override entry points — the generic Windows fix for #424); on Unix keep static-link-order override (defining MI_MALLOC_OVERRIDE there compiles alloc-override.c's forwarding defs and crashes macOS with 'mi_free: invalid pointer' since system libs keep the system allocator). The old -DMI_OVERRIDE=1 was never read by mimalloc source — kept as a prod marker. smoke-test.sh: hard-fail on any 'mimalloc: error' in stderr. Regenerated scripts/vendored-checksums.txt for the new tree (security-vendored.sh passes; no dangerous calls in vendored mimalloc).
The standard linux release binary dynamically links glibc 2.38+ and
GLIBCXX_3.4.32, so it fails to start on Debian 11, Ubuntu 20.04/22.04,
RHEL/Rocky 8/9, Amazon Linux 2, etc. — yet install.sh, the npm and PyPI
wrappers, and the binary's own self-update all fetched it by default,
contradicting the "single static binary" promise.
Point every linux install + self-update path at the fully-static
"-portable" asset (gcc -static), which has no glibc floor. macOS/Windows
are unaffected and unchanged.
- install.sh, pkg/npm/install.js, pkg/pypi _cli.py: select -portable on linux
- src/cli/cli.c: self-update download URL AND checksum archive name both use
-portable on linux (they must match or the update fails checksum verify)
- scripts/smoke-test.sh: assert linux self-update targets the -portable asset
- scripts/ci/check-glibc-compat.sh: new guard — runs the binary inside
debian:bullseye (glibc 2.31) and asserts it starts
- _smoke.yml: run the guard on the portable binary in smoke-linux-portable
Reproduced: standard binary -> "GLIBC_2.38 not found" on glibc 2.31;
portable binary runs cleanly.
The C++ heap-corruption crash only manifests in the production binary (MI_OVERRIDE=1), so the smoke job (which runs the prod binary) is where it reproduces — most reliably on Windows. A ~10k-line templated header exercises the parse-churn path; indexing it must succeed.
Phase 16's FIFO writer-close surfaced as POLLHUP rather than a clean POLLIN+EOF, which the server's poll()-based read loop does not treat as shutdown, so the probe left the process alive and the smoke job failed on every platform. Switch to a plain '< /dev/null' immediate EOF — the server exits reliably (~1s) via its normal getline-EOF path, and it is fully portable (POSIX + MSYS2 bash), so no OS gate is needed. Verified end-to-end: bash scripts/smoke-test.sh reaches 'ALL PASSED'.
Phase 16 of smoke-test.sh launches the stdio MCP server with its stdin held open (a running instance), closes stdin to trigger shutdown, and asserts the process terminates within ~6s — guarding the orphaned-background-process failure mode behind #406 from the prod binary on every platform. POSIX-gated (Windows uses job objects; mkfifo is unavailable on MSYS2).
Two reviewed external-fork contributions, re-implemented as clean maintainer code.
#406 / PR #407 (thanks @nvt-pankajsharma): a POSIX parent-death watchdog so the stdio MCP server exits when its launching parent dies instead of lingering on stdin. Refactors signal_handler into an idempotent request_shutdown(); a watchdog thread polls getppid() (500ms) and shuts down + exit(0) once the initial ppid (>1) changes; Windows unaffected (#ifndef _WIN32). The fork's getppid()<=1 startup early-exit was dropped (it could wrongly kill a legitimately-launched server during reparent races / in container launchers; the initial_ppid>1 guard already no-ops safely), and thread-create failure is now non-fatal (matches the watcher/HTTP background-thread policy). Adds tests/test_parent_watchdog.sh + scripts/test.sh wiring.
#413 / PR #414 (thanks @santanusinha): runtime log-level control via CBM_LOG_LEVEL (case-insensitive debug|info|warn|error|none, or numeric 0..4; unset/unknown leaves the level unchanged). getenv() before threads start (no race). Adds tests/test_log.c cases + README env-var row.
Four overload-scoring lookups (cbm_registry_lookup_method_by_types/_by_args, _symbol_by_types/_by_args) linear-scanned the entire project registry on every call site, ignoring the pre-built method_buckets/func_qn_buckets hash index that the by-name lookups already use. On a large C++ header (e.g. 10989 defs) that is O(calls x project_funcs) per file (~34s/file), so big codebases never finish indexing. Add a hashed fast path (O(overloads)) when the registry is finalized; preserve first-match semantics via lowest registration index (equivalence verified over 1606 lookups, 0 divergence); keep the linear fallback for un-finalized single-file registries so normal files are unaffected. Adds tests/test_cpp_index_hang.sh (scale-tier subprocess+timeout repro, opt-in via CBM_RUN_HANG_TEST).
The cypher unsupported-function check captured stdout, but the CLI prints the parse error to stderr (which cli() redirects into $CLI_STDERR) and exits non-zero, leaving stdout empty — so the assertion always failed once the run reached the smoke stage. Verify the exit status and the stderr channel where the binary actually reports the loud failure. Confirmed locally: full smoke-test.sh passes.
cyp_first_cell built the query_graph JSON by interpolation, so a query with a double-quoted string literal (e.g. replace(f.name, "a", "A")) produced invalid JSON and an empty result (FAIL: replace empty). Escape embedded double-quotes before building the JSON.
The update/star notice in src/mcp/mcp.c references the project's own GitHub URL, which scripts/security-audit.sh blocked as not allow-listed. Add it to scripts/security-allowlist.txt.
Add scripts/check-no-test-skips.sh (run from lint) which fails the lint phase on any plain SKIP() or direct tf_skip_count manipulation; only SKIP_PLATFORM() (for genuinely platform-specific tests) is tolerated. Add FAIL() and SKIP_PLATFORM() helpers to the test framework and convert the remaining SKIP()/perf-gated skips across the suite into pass-or-fail assertions, so a suite that cannot meet its preconditions reports a red failure instead of a silent skip.
#272: '&' is neutralised by the single-/double-quoting of the grep/Select-String/
git commands, so it no longer fails validation. A relaxed validate_search_path_arg
(cbm_validate_shell_arg minus '&') is used for search_code + detect_changes
root_path/file_pattern; all other shell metacharacters stay rejected and
base_branch keeps the strict check.
#282: search_code defaulted regex=false, so 'foo|bar' matched the literal pipe
and silently returned 0 results. The result now carries a warnings[] array that
flags a literal '|' under regex=false (with the regex=true hint), so the trap is
visible instead of looking like a legitimate no-match. (The related invalid-regex
silent-empty case was already fixed in #283.)
Also: every search_code result now reports elapsed_ms, and a perf warning fires
when a search exceeds 5s (also logged as search.slow) so slow calls are visible.
Tests for both issues + the multi-word/invalid-regex no-regression; generous
search_code smoke coverage (basic+elapsed_ms, literal-| warning, '&' acceptance).
Closes#272Closes#282
Wire the Leiden detector into get_architecture via a new 'clusters' aspect.
arch_clusters loads Function/Method/Class nodes + CALLS edges (capped for very
large graphs), runs cbm_leiden, and reports the top-N communities compactly —
each with a dominant-package label, member count, cohesion (internal vs boundary
edge ratio), representative top nodes (by degree), and the packages spanned.
Singletons are skipped to avoid noise. The mcp.c serializer already emitted the
clusters array; it is now populated.
Tests: arch_clusters_basic (two cliques + a bridge -> >=2 connected clusters).
Generous smoke coverage added for the full Cypher surface (labels/type/id/keys/
properties/size/reverse/replace/left/coalesce/substring, NOT EXISTS dead-code,
CASE, unsupported-function error) plus the get_architecture clusters aspect.
Add a single-hop, anchored existence predicate:
WHERE NOT EXISTS { (f)<-[:CALLS]-() } -- functions with no caller
WHERE EXISTS { (f)-[:CALLS]->() } -- functions that call something
Parsed via parse_exists_predicate (reusing parse_node/parse_rel) into a leaf
condition (op=EXISTS, anchor variable, edge type, direction); evaluated against
the bound node with cbm_store_find_edges_by_source_type / _by_target_type. This
is edge-type-specific, so it finds true orphans that in_degree/out_degree miss
(e.g. a node with only a DEFINES edge but no CALLS caller). Multi-hop / nested-
WHERE EXISTS is intentionally unsupported and errors clearly.
Tests cover the dead-code (NOT EXISTS) and has-outgoing (EXISTS) cases; smoke
coverage added. ASan-clean.
Refs: Cypher read suite (EXISTS predicate)
Add an args-list (cbm_func_arg_t) to return items and parse comma-separated
arguments (var.prop or string/number literals), enabling multi-argument scalar
functions in projections:
- coalesce(a, b, ...) first non-empty value
- substring(s, start[, len]) 0-indexed substring
- replace(s, from, to) replace all occurrences
- left(s, n) / right(s, n) leading/trailing n chars
Arguments are freed in free_return_clause (ASan-clean). Unit test
cypher_func_multiarg plus smoke coverage for substring + coalesce.
Refs: Cypher read suite (tier 2b)
Add single-argument string functions to projections — size, length, trim,
ltrim, rtrim, reverse — via the generic function recognizer.
More importantly, change the engine to FAIL LOUDLY on unsupported syntax instead
of silently projecting an empty column. An unknown function call (e.g.
split(...), coalesce(...)) or list indexing/slicing in RETURN/WITH now returns a
clear "unsupported function '<name>' (supported: ...)" error rather than a
valid-looking but blank result — the same silent-empty failure mode that hid the
labels() bug. This supersedes the #373 graceful-resync behaviour (its test is
flipped to assert the error; a new test covers an unknown function in RETURN).
Decision: full Cypher Tier 3 (lists/maps/paths/comprehensions/params — a value
data-model rewrite) is intentionally NOT implemented; clear errors on
unsupported features are the higher-value, lower-risk completeness win for the
read subset agents actually use.
Refs: Cypher read suite (tiers 1-2a + unsupported-syntax hardening)
Add single-argument functions to RETURN/WITH projections, dispatched via a
generic IDENT-call recognizer so new functions no longer need a dedicated lexer
keyword:
- labels(n) -> ["<label>"]
- type(r) -> relationship type
- id(n) -> node/edge identity
- keys(n) -> JSON list of the node's non-null property keys
- properties(n) -> the node/edge properties JSON object
- toInteger/toFloat/toBoolean(x) -> numeric/boolean casts (null on non-numeric)
Previously labels()/id()/keys()/etc. fell into the unknown-function path and
silently projected empty (labels(n) returned ""); they now evaluate correctly.
Regex =~ in WHERE was already supported. Unit tests for each function plus
smoke-test coverage via the live query_graph CLI.
Refs: full Cypher read suite (tier 1 of 3)
The Phase 8 agent-install E2E checked the pre-unification Antigravity paths
(~/.gemini/antigravity/), which broke after the config-path fix. Point the
stub setup at ~/.gemini/antigravity-cli/ and assert the MCP server lands in
the shared ~/.gemini/config/mcp_config.json with AGENTS.md under
~/.gemini/antigravity-cli/. Verified end-to-end (smoke ALL PASSED).
count(DISTINCT x) was a parse error ("expected token type 85, got 9").
parse_aggregate_item now consumes an optional DISTINCT inside the call and
records it on the return item. Both aggregation paths track the set of unique
values per group and emit its size for COUNT(DISTINCT): the RETURN path reuses
the existing collect_lists machinery; the WITH path gains distinct_lists/
distinct_n fields. Plain COUNT and the other aggregates are unchanged.
Unit test: COUNT(DISTINCT f.label)=1 (all same label), count(f.label)=4
(non-distinct), COUNT(DISTINCT f.name)=4 (unique names). Smoke suite adds a
count(DISTINCT) query_graph check.
openCypher node-label alternation was unsupported (parse error after the
first label). parse_node now consumes `:A|B|C` into a single "A|B|C" label
string (mirroring the existing relationship-type alternation). The seed path
unions per-label results (scan_alternation_labels), and target/hop label
filters use label_alt_matches, which ORs over the alternatives. Single-label
matching is unchanged.
Unit test (Function|Module → 5 rows; Function|Class → 4) plus a query_graph
alternation check in the smoke suite. Relationship-type alternation tests
remain green.
openCypher label predicates in WHERE clauses were a parse error
("expected token type 67, got 73"). Parse `var:Label` in the WHERE leaf as
a condition with op=HAS_LABEL/value=Label, and evaluate it against the bound
node label in eval_condition (honoring NOT). Composes with AND/OR and works
in count(...) queries.
Unit test covers true/false/negated label predicates; smoke suite adds a
query_graph WHERE-label check.
The DISTINCT keyword on a WITH clause was parsed (r->distinct) but never
applied, so WITH DISTINCT silently returned duplicate rows. Add a dedup pass
in execute_with_clause that drops projected rows whose full value tuple
duplicates an earlier one (first occurrence kept), gated on wc->distinct.
No-op for the aggregation path (which already collapses per group).
Unit test (4 same-label functions collapse to 1 row; control without DISTINCT
keeps 4) plus a query_graph WITH DISTINCT check added to the smoke suite.
Andrew Hundt