Contract test confirming the graph-analytics chain (PageRank + architecture)
runs on a multi-"language" graph (nodes from .py/.ts/.go files with
cross-language call edges). Analytics are language-agnostic (graph-level), so
this complements the language-agnostic pagerank/arch unit tests + the
per-language LSP tests: it asserts the CHAIN runs across the polyglot case
(PageRank ranks all 6 nodes; architecture returns OK). language_count is not
asserted — that depends on language metadata populated at real-index time, not
the analytics. store_arch 53 pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The codebase://architecture resource (key functions = where to start tracing)
is an MCP resource — application-controlled/pull-only by spec, only fetched on
explicit @-mention. So the model never got it automatically. The _context
header already PUSHED schema/status/pagerank (the reliable channel into the
model); now it also pushes a bounded key_functions summary (top by PageRank,
honoring key_functions_exclude), so the model knows where to start without
pulling the resource or enabling the hidden get_architecture tool.
Reuses build_key_functions_sql (the quote-escaped path). Bounded by
CBM_CONTEXT_KEY_FUNCTIONS_LIMIT (10, smaller than get_architecture's 25) to
keep first-response token cost modest. Gated on context_injection (default
true). Verified end-to-end: search_graph's first response now includes
_context.key_functions (10 entries) on the self-indexed repo. mcp 108 pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The post-dump verify (B1 mitigation, 63de254) used the strict
cbm_store_check_integrity, which treats a non-absolute root_path (e.g. a
relative repo_path like ".") as bad_root_path → the verify DELETED a valid DB.
Indexing with a relative path therefore failed (status=error) — caught by
self-indexing the repo (cbm cli index_repository repo_path="."); the same
index with an absolute path succeeded cleanly (14723 nodes).
Switch the verify to cbm_store_check_integrity_full with the path_only
out-param: a path-only defect (cosmetic root_path) is RETAINED, consistent
with #557 resolve_store — only genuine structural corruption (>5 project
rows) is deleted. Verified: relative-path index now returns status=indexed;
absolute-path index unchanged.
test_graph_buffer gbuf_dump_relative_root_path_retained locks this in.
graph_buffer 50 pass.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The architecture file-tree builder concatenated path components into a stack
char dir[CBM_SZ_512] with strcat, bounded only by ST_MAX_PATH_DEPTH (component
COUNT) — not total length. A file_path with >512 chars across components would
overflow the buffer. Replace the strcat loop with length-checked appends that
stop (truncating that dir key) when the next segment wouldn't fit. Normal
paths are unchanged; store_arch 52 pass (exercises the file tree).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
build_key_functions_sql baked each exclude glob (from the get_architecture
`exclude` MCP tool arg AND the key_functions_exclude config) into the SQL
text as `AND n.file_path NOT LIKE '<glob_to_like output>'`. cbm_glob_to_like
copies single quotes verbatim, so an exclude value containing ' broke out of
the string literal — e.g. exclude=["' OR '1'='1"] became
`NOT LIKE '' OR '1'='1'`, an MCP-input-reachable SQL injection.
The search path is NOT affected (it binds LIKE patterns as ?N parameters via
sqlite3_bind_text, which is safe), so the fix is local to key_functions: add
sql_escape_quotes() (doubles ' -> '', the standard SQL string-literal escape)
and apply it to the glob→like output before interpolation. The bound-parameter
callers of cbm_glob_to_like are intentionally untouched (escaping there would
double-escape).
Found by the quality/security sweep agent; verified independently (reachability
via the `exclude` tool arg at mcp.c:3372; cbm_glob_to_like copies ' verbatim at
store.c:2168). The sweep's other 2 findings (cascade_delete_edges NULL-deref,
parse_fp_from_props off-by-one) were verified as FALSE POSITIVES
(remove_edge_from_ptr_array NULL-checks; the hex check uses HEX_LEN not HEX_BUF).
Green: store_arch 52 / mcp 108.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
- store_prefix_ranks_project_above_dep_all_sort_modes: proves dep-last
ranking holds for EVERY sort_by (relevance/name/degree/calls/linkrank),
not just the default. Dep inserted first (lower id) so a missing dep-last
in any mode would let the dep win the id tiebreak — a real per-mode check.
- pagerank_invalid_inputs_clamp_cleanly: negative/>1 damping, negative/zero
epsilon, negative/zero max_iter all clamp to defaults and produce finite
ranks summing to ~1 (guards the #21 config knobs against bad values, and
complements the #44 NaN fix).
Green: tool_consolidation 88 / pagerank 45.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
pass_similarity hardcoded the SIMILAR-edge Jaccard cutoff to
CBM_MINHASH_JACCARD_THRESHOLD (0.95). Now tunable via config.
- pipeline: add similarity_threshold to cbm_pipeline_t + a setter
cbm_pipeline_set_similarity_threshold(); copy it into the pipeline ctx
(alongside mode). <=0 (unset) keeps the built-in default.
- pass_similarity: the sim_query_ctx_t worker struct carries the threshold
(set from ctx); the per-pair check uses sc->threshold when >0, else the
default. No behavior change unless tuned.
- mcp: CBM_CONFIG_SIMILARITY_THRESHOLD ("similarity_threshold") read via
cbm_config_get_double in handle_index_repository (default 0.0 = built-in)
and applied before pipeline_run.
Together with the Leiden resolution knob (prev commit), all graph-analytics
params are now config-tunable: pagerank damping/epsilon/max_iter + edge
weights, Leiden resolution, similarity threshold. mcp 108 / graph_buffer 49 /
full suite clean through the bulk (only known B1 incremental failures).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
#3 — incremental purge was O(C·(N+E)): cbm_gbuf_delete_by_file was called
once per changed/deleted file, each a full nodes+edges scan. Add
cbm_gbuf_delete_by_paths (one pass over nodes matching a path membership
set + one edge cascade = O(N+E) regardless of path count); refactor
cbm_gbuf_delete_by_file to delegate to it (DRY — single implementation).
pipeline_incremental.c now purges all changed+deleted paths in one call
(OOM fallback keeps the old per-file path). graph_buffer 49 + incremental
purge tests pass; only the known B1 full-reindex failures remain.
#8 — cbm_dep_link_cross_edges was N+1: one cbm_store_search per import
(<=500) to find a matching dep Module. Now ONE bulk fetch of all dep Module
nodes + an in-memory name->id hash (first-wins, preserving limit=1
semantics) -> O(1) per import. depindex 32 pass.
#5 verified NOT a bug (avoided a false fix): node_degree IS populated by
cbm_pagerank_compute (pagerank.c:422 INSERT OR REPLACE INTO node_degree);
the correlated-subquery fallback only fires pre-PageRank.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
cbm_store_search had `pr_col = has_degree_filter ? "pr_rank" : "pr_rank"`
— both branches identical because the pagerank JOIN alias is always
pr_rank (unlike name_col/proj_col which legitimately differ under the
degree-filter subquery wrap). cppcheck flagged the duplicate expression.
Simplify to a direct assignment. No behavior change; tool_consolidation 87 passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
- README: "14 MCP tools" -> 15 (classic), in all 4 places; clarify that a
streamlined subset is the default and CBM_TOOL_MODE=classic exposes all 15.
- mcp.c query_graph description: note that dependency sub-project symbols
(proj.dep.*) are tagged source:dependency, and show the ORDER BY pattern to
rank project symbols above them (Cypher is declarative — user controls
ORDER BY, so this is documentation, not auto-reordering).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
cbm_pagerank_compute's clamp used `damping < 0 || damping > 1` and
`epsilon <= 0`, both of which IEEE-754 NaN bypasses (every NaN
comparison is false). Since damping/epsilon became config-tunable and
cbm_config_get_double uses strtod (which parses "nan"), a
`config set pagerank_damping nan` would let NaN propagate through the
power iteration: every rank becomes NaN, the convergence check
`delta < epsilon` is always false (runs full max_iter), and the pagerank
table is silently corrupted, poisoning the search relevance sort.
Switch to the inverted-range form `!(damping >= 0 && damping <= 1)` and
`!(epsilon > 0)`, which reject NaN because the inner comparison is
false. Found by the silent-failure review agent; verified independently
(strtod parses "nan"; NaN bypassed the old clamp).
test_pagerank.c pagerank_nan_damping_is_clamped: a NaN damping must
yield finite ranks summing to ~1. Green: pagerank 44 passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
cbm_pagerank_compute_with_config already read max_iter and the 12
edge-weight keys from config, but damping (0.85) and epsilon (1e-6) were
passed as hard-coded #defines -- the two core convergence params were
not tunable.
- pagerank.h: CBM_CONFIG_PAGERANK_DAMPING / CBM_CONFIG_PAGERANK_EPSILON.
- pagerank.c: read both via cbm_config_get_double in
cbm_pagerank_compute_with_config (defaults = existing #defines, so no
behavior change unless a user tunes them).
- test_pagerank.c: pagerank_damping_epsilon_config_tunable asserts that
tuning damping (0.5 vs 0.99) materially changes the hub rank value and
ranks still sum to ~1 under a custom config.
Green: pagerank 43 passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The dep-last tiebreak in cbm_store_search only fired for glob searches
(params->project_pattern), so the common prefix-match path
(params->project, which includes proj.dep.*) let a dependency symbol
outrank the project's own symbol of the same name -- the "Path" concern
(a stdlib Path fronting the user's own Path).
- store.c: hoist dep-last into one clause applied consistently across
ALL sort modes (relevance/degree/calls/linkrank/name) whenever a
search is scoped to a project; gated on the new param.
- store.h: cbm_search_params_t::disable_dep_ranking (default false =
deps rank last; inverted sense so {0}-init preserves the safe default).
- mcp.c: search_disable_dep_ranking config key (bool, default false)
wired through srv->config and documented in the search_graph schema,
following the existing CBM_CONFIG_* pattern.
- test_tool_consolidation.c: store_prefix_ranks_project_above_dep
(default ranks project above dep) and store_prefix_disable_dep_ranking_lets_dep_win
(pure relevance when disabled).
Green: tool_consolidation 87 passed. Full suite unchanged except the 2
known B1 incremental failures.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Two regression probes that isolate the non-deterministic .db corruption
(B1) to the extraction-population hop, each with a decisive negative:
- test_sqlite_writer.c sw_scale_root_path_integrity: variable-length
properties_json (20-1500B, stressing every page boundary) through
cbm_write_db. Clean 10/10 -> rules out the variable-record
page-boundary hypothesis in the writer.
- test_graph_buffer.c gbuf_dump_pipeline_path_integrity: drives
cbm_gbuf_dump_to_sqlite with a populated + merged gbuf (variable heap
properties_json + cbm_gbuf_merge ID remap, 15k nodes / 50k edges).
Clean 10/10 -> rules out the gbuf->dump handoff (build_dump_nodes /
temp_to_final).
Both confirm the writer and the gbuf->dump handoff are correct on
realistic synthetic data; B1 must originate in real tree-sitter
extraction populating the gbuf (needs real source data to reproduce).
test_main.c: add CBM_ONLY_SUITE fast-filters for graph_buffer + pagerank.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The existing test-leak uses `leaks --atExit`, which finds heap LEAKS — the
wrong tool for non-deterministic corruption (uninit reads, use-after-free,
overruns) that ASan/TSan miss (e.g. the B1 custom-writer bug). Add two
macOS-native targets that run the nosan binary (ASan replaces malloc, which
defeats these libmalloc knobs):
- test-memory: MallocScribble=1 + MallocPreScribble=1 — freed memory -> 0x55
(catches UAF), freshly-allocated -> 0xAA (catches uninitialized reads; the
macOS MSan equivalent, makes uninit deterministic).
- test-gmalloc: DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib (Guard Malloc)
— guard page per allocation, crashes at the exact overrun/UAF with a stack
trace. The decisive memory tool.
Both tee to build/c/mem-report.txt; honor CBM_ONLY_SUITE for targeted runs.
Linux falls back to a pointer to test-leak / -fsanitize=memory. Documented in
CLAUDE.md (Memory-Corruption Debugging section). .PHONY updated.
Verified: `make -n test-memory`/`test-gmalloc` expand correctly; default
`test` target unchanged.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
cbm_gbuf_dump_to_sqlite intermittently emits a structurally-corrupt .db
(B1): on the ~16k-node fastapi repo a full index non-deterministically
persists 0/partial node counts and a garbage project root_path, leaving an
unreadable .db that queries then fail on (PRAGMA integrity_check won't even
prepare). The corruption is ASan/TSan/clang--analyze-invisible in the dump
path; root cause still open.
Add a post-write check in cbm_gbuf_dump_to_sqlite: after cbm_writer_finalize,
open the freshly-written .db and run cbm_store_check_integrity (O(1)
projects-table check, not a full integrity_check). If it fails, unlink the
.db and return GB_ERR so the caller re-indexes instead of serving garbage.
Confirmed firing in the incremental suite (logs dump.verify_corrupt, deletes
the .db).
Makes the system fail-safe against the worst (structurally-unreadable) B1
symptom: no silent corrupt .db is left for queries to read. Does not by
itself fix the underlying non-deterministic write corruption — the 2
pre-existing incremental-accuracy failures (incr_db_deleted_recovery,
incr_accuracy_vs_full) remain; that needs the root-cause fix. Complements
the #557 retain-bad-root_path mitigation in 22e438f.
Verified: store_nodes 56/56, sqlite_writer 7/7, mcp 108/108 (no regression).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Adds sw_scale_root_path_integrity: writes a ~fastapi-scale DB in ISOLATION
(20k nodes / 200k unique edges) via cbm_write_db, then asserts root_path
round-trips exactly, PRAGMA integrity_check == "ok", and node/edge counts.
Purpose: isolate whether the B1 custom-writer corruption reproduces without
the pipeline. Result: PASSES 3/3 — the writer is sound at scale in isolation
(sw_minimal_data covers the tiny case). This rules out the writer's
serialization as the B1 cause and points at the pipeline/parallel handoff
(a data race — to be confirmed with TSan). Also a permanent regression guard
for the custom B-tree writer at scale.
Edges generated with unique (source,target) pairs to respect the table's
UNIQUE(source_id, target_id, type) constraint. Adds sqlite_writer to the
CBM_ONLY_SUITE filter.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
When CBM_ONLY_SUITE is set (e.g. "incremental", "mcp", "store_nodes",
"tool_consolidation"), run only that suite then summarize and exit;
otherwise behavior is unchanged. Zero cost when unset. Speeds the
edit/build/test loop for focused work without spinning up the heavy
LSP/stdlib suites.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
resolve_store (mcp.c) auto-cleaned (deleted .db + WAL + SHM) whenever
cbm_store_check_integrity flagged the projects table. A single malformed
project root_path (e.g. a numeric value) is a cosmetic projects-row
defect — node/edge data stays intact and queries key off project name,
not root_path — yet it triggered a wholesale delete of the freshly-indexed
DB, losing all indexed data (upstream #557).
Add cbm_store_check_integrity_full(store, &path_only_failure) (store.h,
store.c) which classifies a bad-root_path-only defect (rows fine) apart
from genuine >5-row corruption. resolve_store now RETAINS path-only-defect
DBs (logs store.integrity_retain) and only auto-deletes genuine
corruption. The watcher registration is guarded against a bogus
(numeric/empty) root_path from a retained DB. cbm_store_check_integrity()
is kept as a thin wrapper so artifact.c, pipeline.c, and existing tests
are unchanged.
Mitigates #557 data loss and stabilizes incremental node-count reads (the
count no longer sporadically drops to 0 after a query). Does NOT fully
close the 2 pre-existing incremental-accuracy failures
(incr_db_deleted_recovery, incr_accuracy_vs_full), which stem from a
separate full-reindex-after-deletion node-count discrepancy — tracked
separately.
Verified: store_nodes 56/56 incl. new store_integrity_full_path_only_classification;
mcp suite green; incremental suite shows only the 2 pre-existing failures
(no regression; diff_pct=96 identical pre/post).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The full-index hash-persist path (pipeline.c:1127) upserted one row per file under autocommit -- a 10k-file reindex did 10k separate fsyncs under WAL. Wrap the loop in cbm_store_begin/cbm_store_commit (falling back to per-row autocommit if BEGIN fails), matching the incremental path (pipeline_incremental.c:458, persist_hashes). Fork-origin perf finding (#6 from the post-merge latency/complexity audit; git blame = Andrew Hundt). No behavior change: same upserts, single commit.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
persist_hashes (pipeline_incremental.c:452) upserted one file-hash row per file in autocommit -- a 10k-file reindex issued 10k separate COMMITs (each a WAL fsync). Wrap both upsert loops (current discovery + mode-skipped) in cbm_store_begin/commit so N files -> 1 COMMIT. Falls back to per-row autocommit if BEGIN fails (a caller already holds a transaction): 'batched' is gated on BEGIN's return and COMMIT runs only when BEGIN succeeded, so the existing partial-failure policy is unchanged. Fork-origin perf finding #6 (git blame: Andrew Hundt). Compiles clean (make cbm). Other perf findings are upstream-origin (cypher sort/DISTINCT, query_graph JSON, include_connected, properties re-parse -- DeusData #593/#601/#238) or fork-origin but higher-risk (deferred).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The §4b split (39539ef) deleted the search_code_graph mega-tool but left inconsistent references, found by a systematic tool-name + AI-info-string audit: (1) _hidden_tools pseudo-tool JSON (mcp.c:6365) still listed the 3 now-default tools (14 entries) contradicting its own '11 additional' description -- trimmed to the 11 genuinely-hidden tools (15 TOOLS[] minus 4 default-surface-in-TOOLS[]; get_code is a streamlined-only alias); (2) handle_get_code_snippet error hints (4752, 4923) referenced the deleted tool -- search_graph; (3) search_graph mode-error hints (2663, 2670) pointed at the dead search_in='source' param -- the search_code tool; (4) comments (1566, 4725) updated. String/comment-only; no handler logic changed. The two critical hints previously sent clients/AIs to a non-existent tool from default-surface error paths.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Replace the consolidated search_code_graph mega-tool — which merged graph
search + source grep + cypher behind two nearly disjoint parameter
families — with the three existing focused tools as the DEFAULT surface:
search_graph (graph/structural), search_code (source grep), query_graph
(cypher). These are drawn from TOOLS[] (single schema source, DRY);
trace_call_path + get_code remain on the default surface. The
search_code_graph schema and its thin dispatch router are deleted; the
underlying handlers (handle_search_graph/search_code/query_graph) stay.
The _hidden_tools progressive-disclosure hint now lists 11 hidden tools
(was 14). Repointed three user-facing hint strings (handle_trace_call_path
missing-name and function-not-found; two unknown-tool path hints) from the
deleted tool to search_graph.
Tests updated to the new 5-tool surface:
- tests/test_mcp.c: mcp_tools_list + server_handle_tools_list assert the
5 default tools present and search_code_graph absent.
- tests/test_tool_consolidation.c: renamed streamlined_mode_shows_3_tools
-> ..._5_default_tools; repurposed the two search_code_graph dispatch
tests into search_graph_dispatch / query_graph_dispatch (query_graph
handler accepts both cypher= and query=); revised
schema_compact_documented_as_graph_only (the graph-vs-source compact
collision it warned about no longer exists post-split).
- tests/test_input_validation.c: remapped 16 dispatch calls by mode --
cypher -> query_graph, source-grep -> search_code, graph/structural ->
search_graph.
- tests/test_depindex.c: tool_index_dependencies_listed asserts
search_graph (was search_code_graph).
- src/cli/cli.c: six config-description strings (search_limit, tool_mode,
context_injection, compact, default_sort_by, default_include_dependencies).
Back-compat: blast radius is in-repo only (0 references in Go internal/cmd,
docs, or the CGO wrapper). External callers of search_code_graph must
switch to search_graph / search_code / query_graph, which are strictly
more capable. query_graph keeps its query= backward-compat alias.
Context -- of the three section-4 fixes in the merge plan, only section 4b
was a real change: section 4c (function<->class edges) was already
implemented (commit b950bf2 added the missing MEMBER_OF reverse-edge
coverage), and section 4a (passive resources) was already mitigated by
inject_context_once plus the get_graph_schema/get_architecture/index_status
tools -- both verified against code, not assumed.
Gate: 5981 passed / 2 failed (the 2 are the deferred B1 store-layer
incremental failures, unrelated to section 4b).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
contract_edge_defines_method asserted only the forward DEFINES_METHOD
(Class->Method) edge. The fork's reverse MEMBER_OF (Method->Class) edge —
emitted in pass_definitions.c:322 / pass_parallel.c / pass_normalize.c and
consumed by pagerank.c (member_rank_factor) — had ZERO test coverage
anywhere in the suite. Add the MEMBER_OF assertion so the function<->class
tie is locked in in both directions.
§4c investigation finding (evidence-based, refutes the plan's assumption
that a code fix was needed): the function<->class tie was ALREADY
implemented, not a merge regression. The Method-only gate
(pass_definitions.c:318) is identical in fork (3018b22:270), upstream
(34efbc0:315), and HEAD; DEFINES_METHOD is tested+passing across 9
languages (defines_method_go/rust/java/csharp/php/ruby/kotlin/ts/scala in
test_edge_types_probe.c); the documented "Go parent_class never derived"
bug is already fixed (extract_defs.c:2776-2783); and every class-scoped
callable is labeled Method with parent_class populated (push_method_def /
generic extractor / extract_rust_impl), so there is no class-scoped
Function gap to fix. Broadening the gate would change nothing. The only
real gap was the untested reverse edge, closed here.
Gate: 5981 passed / 2 failed (the 2 are the deferred B1 incremental
store-layer failures, unrelated to §4c).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
test-runner-tsan mirrors test-runner but links vendored objects compiled
with -fsanitize=thread and links the mimalloc object with MI_OVERRIDE=0
(no malloc override, so TSan intercepts the real malloc/free), satisfying
the unconditional mi_* calls in src/foundation/mem.c. TSan cannot combine
with ASan/UBSan and uses the system allocator. Replaces the placeholder
test-tsan stub ("TSan not yet wired for full extraction tests").
make -f Makefile.cbm test-tsan # slow; race reports go to stderr
Also add *.plist to .gitignore: `make test-analyze` (clang --analyze)
drops a .plist report per source file in CWD; these regenerable reports
polluted the tree (183 files after the analyzer run).
Verification: built + ran test-runner-tsan over suite_incremental; 0 data
races. The deferred B1 incremental races (HANDLES-edge nondeterminism
under the concurrent parallel pipeline) are timing-nondeterministic and
did not reproduce under TSan's scheduling, so the B1 fix stays a dedicated
follow-up (tracked in notes sec 8f). TSan targets the incremental +
parallel suites that exercise the racy paths.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The merged cbm_mcp_tools_list (src/mcp/mcp.c:868) defaults to streamlined mode
when srv is NULL, so tests/test_mcp.c:mcp_tools_list only exercised the 3
consolidated tools. The classic-mode path (CBM_TOOL_MODE=classic, mcp.c:876)
that emits all 15 TOOLS[] split tools had no test — the MERGE-TODO flagged at
tests/test_mcp.c:172.
Add mcp_tools_list_classic_mode: setenv("CBM_TOOL_MODE","classic"), capture the
list, then unsetenv BEFORE any ASSERT so a failed assertion cannot leak the
classic setting into sibling tests (which expect the streamlined default). Assert
the classic split tools (index_repository, search_graph, query_graph) are present
and the streamlined-only consolidated search_code_graph + the _hidden_tools
progressive-disclosure hint are absent. Registered in suite_mcp.
Full suite: 5981 passed / 2 failed (the 2 are the deferred B1 incremental races,
unrelated to this test).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
tests/test_stack_overflow.c and tests/test_lang_contract.c each have a fork()
helper that runs cbm_extract_file on pathological input in a child process (to
verify no-crash), then bare-waitpids with waitpid(pid, &status, 0). Under
`leaks --atExit` (macOS) the forked child is SIGSTOP'd during heap inspection
and the bare waitpid never returns — the identical hang class already fixed in
b336466 (test_store_bulk.c, src/foundation/compat_fs.c) and 148b4d7
(test_dump_verify_io.c).
Replace the bare waitpid with the WUNTRACED+SIGCONT loop used elsewhere in the
suite (kill(child, SIGCONT) on WIFSTOPPED so leaks can finish and the child
proceeds to _exit), and add #include <signal.h> for kill() in both files. The
helpers keep their existing `return WIFSIGNALED(status)` contract.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
dump_verify_io_fork_crash_uncommitted_discarded used bare
waitpid(pid, &status, 0), which never returns when `leaks --atExit` (macOS)
SIGSTOPs the forked child during heap inspection — the identical hang b336466
fixed in test_store_bulk.c (bulk_crash_recovery) and src/foundation/compat_fs.c
(cbm_exec_no_shell). This test was missed by that earlier fix.
Replace the bare waitpid with the WUNTRACED+SIGCONT loop that mirrors
test_store_bulk.c: on WIFSTOPPED, kill(child, SIGCONT) so leaks can finish
inspecting and the child proceeds to _exit(). Add #include <signal.h> for kill().
Unblocks `make test-leak` past this test (the run previously hung here
indefinitely).
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
The test-runner-nosan target (used by `make test-leak` on macOS for
`leaks --atExit`) failed to link with Undefined symbols: _LZ4_compressBound,
_LZ4_compress_HC, _LZ4_decompress_safe, _suite_zstd, _cbm_artifact_import,
_PRETRAINED_VECTOR_BLOB.
Root cause: OBJS_VENDORED_NOSAN in Makefile.cbm omitted the LZ4/ZSTD/nomic
objects that OBJS_VENDORED_TEST includes, and the test-runner-nosan target
prerequisites/link line missed $(ZSTD_SRCS). Because the LZ4/ZSTD test objects
are compiled with $(SANITIZE) (ASan) and ASan replaces malloc (incompatible with
the `leaks` tool), add sanitizer-free NOSAN variants ($(NOSAN_DIR)/lz4.o,
lz4hc.o, zstd.o) that mirror the existing NOSAN sqlite3/ts_runtime/lsp_all/
preprocessor objects, plus $(UNIXCODER_OBJ) and $(ZSTD_SRCS).
Result: build/c/test-runner-nosan now links; `make test-leak` builds and runs.
(A separate runtime hang at dump_verify_io_fork_crash_uncommitted_discarded
under `leaks --atExit` — a fork+crash+waitpid/leaks interaction — remains and is
tracked separately; it is unrelated to this link fix.)
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Deep --no-ff merge of upstream DeusData/main (597 commits, merge-base 988d975)
into fork api-consolidation (HEAD 3018b22), producing a superset that combines
upstream breadth with the fork's surviving depth.
Upstream additions integrated: semantic/embedding search (vendored/nomic),
minhash/SimHash, cross-repo + cross-file LSP + route + pkgmap passes, artifact
import/export, path_alias, git_context, hook_augment, Windows support
(compat_fs/compat_thread), the UI layer, and ~140 expanded tree-sitter grammars.
Fork depth retained: PageRank/LinkRank ranking (src/pagerank/; sort_by=
relevance/linkrank/degree/calls), dependency indexing (src/depindex/;
CBM_INDEX_MODE=DEP), token reduction, and HTTP-endpoint discovery
(src/pipeline/httplink.c, pass_httplinks.c).
Conflict resolution (27 regions across the 39-file intersection):
- src/mcp/mcp.c: kept the fork superset handler pattern (resolve_project_store +
expand_project_param + inject_context_once) as the single path; UNIONed the
tool dispatch table; emit status:"indexed", excluded[], cross_service trace.
- src/store/store.c: upstream base + fork pagerank/linkrank/node_degree tables
and sort_by plumbing; cbm_store_node_degree now excludes INHERITS
(WHERE type != 'INHERITS'); removed the duplicate cbm_store_get_db.
- src/pipeline/pipeline.c: took upstream run_extraction_phase; grafted the fork
passes (httplinks, normalize, flush_store dump branch); call
cbm_pipeline_set_committed_counts before cbm_gbuf_dump_to_sqlite.
- internal/cbm/lang_specs.c: 158 entries combining the fork section_node_types
with upstream ts_factory/embedded_imports; wired 13 orphaned type-arrays.
- src/depindex/depindex.h: renamed CBM_PATH_MAX -> CBM_DEP_PATH_MAX (it collided
with constants.h enum CBM_PATH_MAX=1024) + 11 call sites in depindex.c.
- Makefile.cbm: upstream grammar/nomic units + fork CGO targets; added
httplink.c + pass_httplinks.c to PIPELINE_SRCS; removed mongoose NOSAN
machinery (mongoose was deleted upstream).
- httplinks restored (the merge silently deleted them because the fork had left
them unchanged since the base): httplink.c/.h, pass_httplinks.c,
tests/test_httplink.c; registered RUN_SUITE(httplink) in tests/test_main.c.
- CBM_INDEX_MODE enum unified to FULL=0, MODERATE=1, FAST=2, DEP=3.
Completeness (dual-parent check): 0 upstream files dropped from the merge tree;
6 fork files dropped, all intentional (upstream-superseded): CHANGELOG.md,
vendored/mimalloc/src/{arena-abandon,segment-map,segment}.c,
vendored/mongoose/{mongoose.c,mongoose.h}. Fork features (pagerank, depindex,
token_reduction, httplink) verified present and registered in tests/test_main.c.
src/pipeline/pass_parallel.c has an identical function set to upstream (49=49,
none dropped); the large line-deltas vs upstream are within-function divergence
of the fork extractor, not dropped functions.
Gate (run manually; hooks skip on merge commits): build green
(make -f Makefile.cbm cbm, exit 0); full C test suite 5980 passed / 2 failed /
0 crashes. The fork's pre-existing ASan crash in build_project_list_error_srv is
fixed by upstream's mcp.c rewrite, so the merge is strictly better than the fork
baseline on the crash axis.
Known issue, tracked separately (NOT a merge defect): 2 nondeterministic
failures in tests/test_incremental.c — incr_db_deleted_recovery:772 (diff_pct)
and incr_accuracy_vs_full:803 (full_nodes vs incr_nodes). Cause: the fork's
extractor (internal/cbm/extract_*.c) and parallel-resolve Route-node creation
are not thread-safe under the concurrent use of upstream's parallel pipeline
(run_parallel_pipeline), producing nondeterministic route/infra HANDLES edges
(~234k edge swing between runs). Sequential extraction (CBM_WORKERS=1) is
deterministic and correct. Root cause, bisect, and repro are documented in
notes/2026-06-24-fork-vs-upstream-divergence-assessment.md (section 8f); the fix
is deferred to a dedicated thread-safety audit (the make test-tsan target is
currently a stub and needs wiring).
The MSYS2 CLANG64 toolchain bakes its package-tracker URL
(https://github.com/msys2/MINGW-packages) into the static Windows .exe via
libc++/compiler-rt. The Layer 2 binary string audit flagged it as an
unauthorized URL, failing smoke-windows (both the standard and ui variants)
while Linux smoke stayed green. Add it to ALLOWED_URLS alongside the other
compiler/linker toolchain URLs (gcc.gnu.org, sourceware.org, launchpad).
Add tests/test_security_strings_allowlist.sh as a reproduce-first regression
guard: it reproduces the BLOCKED failure with a binary fixture and includes a
negative control proving a genuinely unauthorized URL is still rejected. Wired
into scripts/test.sh as Step 6.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three native GitHub Actions to automate the mechanical parts of issue
triage:
- stale.yml: actions/stale scoped to the awaiting-reporter label —
warns at 21 days idle, closes at 35, auto-resets when the reporter
replies. Never touches PRs or unlabeled issues.
- label-actions.yml + .github/label-actions.yml: posts a templated
comment when duplicate or awaiting-reporter is applied (duplicates are
linked, not auto-closed).
- issue-labeler.yml + .github/issue-labeler.yml: adds area labels
(windows, stability/performance, parsing/quality, editor/integration,
ux/behavior, cypher, language-request) from title/body keywords.
Additive only; base bug/enhancement labels still come from the forms.
All third-party actions pinned to full commit SHAs with least-privilege
per-job permissions (issues: write only where required).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- graph-ui: bump vite ^6.4.2 -> ^6.4.3 and add overrides pinning the transitive
form-data >=4.0.6 and @babel/core >=7.29.6 (all dev-scope build/test deps, not
shipped in the binary). Clears the 4 open Dependabot alerts and Scorecard's
VulnerabilitiesID. `npm audit` now reports 0 vulnerabilities.
- codeql.yml: move `security-events: write` from the workflow top level to the
`analyze` job (top level is now `contents: read`), resolving Scorecard's
TokenPermissionsID least-privilege finding. CodeQL still uploads results from the
job-scoped token.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Vendors the two MIT-licensed grammars from intersystems/tree-sitter-objectscript
@a7ffcdf (the language vendor's official grammars, ABI 15) that PR #467 needs:
internal/cbm/vendored/grammars/objectscript_udl/ (.cls)
internal/cbm/vendored/grammars/objectscript_routine/ (.mac/.inc/.rtn/.int)
Each directory carries the verbatim generated parser.c + scanner.c + tree_sitter/
headers + LICENSE. The one adjustment: each scanner.c's upstream
`#include "../../common/scanner.h"` is repointed to a per-directory
`objectscript_common.h` (a verbatim copy of the upstream common/scanner.h), because
this repo's shared vendored/common/scanner.h belongs to the cfml/fsharp grammars and
differs. Updates MANIFEST.md + THIRD_PARTY.md.
These files are dormant until #467 (the grammar shims + extraction) is rebased on
top; main's build is unaffected.
Refs #462, #467.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Andrew Hundt