A node group variable carried through a WITH aggregation
(e.g. `WITH g, count(*) AS c RETURN g.file_path`) returned blank for every
property except its name: the carried virtual binding held only the group
key (the node's name) and lacked a store handle, so node_prop() could
neither read other fields nor compute degrees.
Fix: capture the node id of a bare node group-var in with_agg_find_or_create
and tag the carried virtual binding with it; in node_prop(), when such a stub
(id set, string fields unpopulated) is asked for a missing property, re-fetch
the full node via cbm_store_find_node_by_id and project it. Also propagate
the store onto virtual bindings so node_prop can re-fetch and compute
degrees. The stub gate is heuristic but never yields a wrong value — worst
case is one redundant indexed lookup. Adds regression test
cypher_exec_with_node_groupvar_prop.
Signed-off-by: Kris Kersey <kris@kerseyfabrications.com>
Per-function complexity metadata is now stored on graph nodes and queryable,
alongside several indexing performance and correctness fixes developed and
validated together (3704 tests, ASan/UBSan clean).
Bottleneck metrics (query via query_graph):
- Tier A (in the extraction AST walk): cyclomatic (complexity), cognitive
(nesting-weighted), loop_count, loop_depth (max nested-loop depth),
param_count, max_access_depth.
- Tier B (new pre-dump pass, pass_complexity.c): transitive_loop_depth
propagated along CALLS edges + a recursive flag (direct self-recursion and
mutual-recursion cycles), plus the call-context signals linear_scan_in_loop,
alloc_in_loop, recursion_in_loop and unguarded_recursion.
- query_graph and get_architecture tool descriptions document the metrics and
the Leiden community clusters.
Cypher engine:
- node_prop exposes arbitrary persisted node properties to WHERE/RETURN.
- Fix projection aliasing: multi-property rows shared a single static buffer so
every column returned the last value read; now per-column/rotating buffers.
- Fix a stack-use-after-scope in aggregate RETURN (caller-owned value buffers).
Indexing performance:
- Gate C/C++ #define Macro-node extraction to full mode (it is ~49% of nodes on
the Linux kernel); moderate/fast skip it.
- Emit the complexity property block only for Function/Method nodes so the
millions of Macro/Field/Variable/Class/Enum nodes no longer carry zeroed
fields — large RAM reduction at scale.
- Classify node types via tree-sitter TSSymbol bitsets in cbm_kind_in_set
instead of per-node strcmp scans (thread-local cache, strcmp fallback;
behaviour-identical).
- Subsample frequent (Zipfian) tokens in the semantic co-occurrence finalize;
~14x faster finalize on the kernel, output unchanged.
- pass_lsp_cross: replace O(n^2) linear dedup with hash-set dedup.
Windows:
- Canonicalize drive-letter case during path normalization so "c:/repo" and
"C:/repo" derive the same project key and cache file (#394/#227/#367).
Tests: extraction, pipeline and cypher regressions covering all of the above.
Add a single-hop, anchored existence predicate:
WHERE NOT EXISTS { (f)<-[:CALLS]-() } -- functions with no caller
WHERE EXISTS { (f)-[:CALLS]->() } -- functions that call something
Parsed via parse_exists_predicate (reusing parse_node/parse_rel) into a leaf
condition (op=EXISTS, anchor variable, edge type, direction); evaluated against
the bound node with cbm_store_find_edges_by_source_type / _by_target_type. This
is edge-type-specific, so it finds true orphans that in_degree/out_degree miss
(e.g. a node with only a DEFINES edge but no CALLS caller). Multi-hop / nested-
WHERE EXISTS is intentionally unsupported and errors clearly.
Tests cover the dead-code (NOT EXISTS) and has-outgoing (EXISTS) cases; smoke
coverage added. ASan-clean.
Refs: Cypher read suite (EXISTS predicate)
Add an args-list (cbm_func_arg_t) to return items and parse comma-separated
arguments (var.prop or string/number literals), enabling multi-argument scalar
functions in projections:
- coalesce(a, b, ...) first non-empty value
- substring(s, start[, len]) 0-indexed substring
- replace(s, from, to) replace all occurrences
- left(s, n) / right(s, n) leading/trailing n chars
Arguments are freed in free_return_clause (ASan-clean). Unit test
cypher_func_multiarg plus smoke coverage for substring + coalesce.
Refs: Cypher read suite (tier 2b)
Add single-argument string functions to projections — size, length, trim,
ltrim, rtrim, reverse — via the generic function recognizer.
More importantly, change the engine to FAIL LOUDLY on unsupported syntax instead
of silently projecting an empty column. An unknown function call (e.g.
split(...), coalesce(...)) or list indexing/slicing in RETURN/WITH now returns a
clear "unsupported function '<name>' (supported: ...)" error rather than a
valid-looking but blank result — the same silent-empty failure mode that hid the
labels() bug. This supersedes the #373 graceful-resync behaviour (its test is
flipped to assert the error; a new test covers an unknown function in RETURN).
Decision: full Cypher Tier 3 (lists/maps/paths/comprehensions/params — a value
data-model rewrite) is intentionally NOT implemented; clear errors on
unsupported features are the higher-value, lower-risk completeness win for the
read subset agents actually use.
Refs: Cypher read suite (tiers 1-2a + unsupported-syntax hardening)
Add single-argument functions to RETURN/WITH projections, dispatched via a
generic IDENT-call recognizer so new functions no longer need a dedicated lexer
keyword:
- labels(n) -> ["<label>"]
- type(r) -> relationship type
- id(n) -> node/edge identity
- keys(n) -> JSON list of the node's non-null property keys
- properties(n) -> the node/edge properties JSON object
- toInteger/toFloat/toBoolean(x) -> numeric/boolean casts (null on non-numeric)
Previously labels()/id()/keys()/etc. fell into the unknown-function path and
silently projected empty (labels(n) returned ""); they now evaluate correctly.
Regex =~ in WHERE was already supported. Unit tests for each function plus
smoke-test coverage via the live query_graph CLI.
Refs: full Cypher read suite (tier 1 of 3)
split(f.path,'/')[0] and similar unknown function-call / index
expressions left their trailing tokens unconsumed, desyncing the parser into
a misleading default star projection (columns f.name/f.qualified_name/
f.label, one blank row per node). parse_return_item now consumes the balanced
parentheses/brackets of an unknown call so the parser stays in sync: the
requested RETURN/WITH column shape is preserved (aliases intact), sibling
aggregates like count(*) still compute, and the unsupported expression simply
projects empty instead of corrupting the whole projection. Existing scalar
forms (labels(), toInteger()) are unaffected.
count(DISTINCT x) was a parse error ("expected token type 85, got 9").
parse_aggregate_item now consumes an optional DISTINCT inside the call and
records it on the return item. Both aggregation paths track the set of unique
values per group and emit its size for COUNT(DISTINCT): the RETURN path reuses
the existing collect_lists machinery; the WITH path gains distinct_lists/
distinct_n fields. Plain COUNT and the other aggregates are unchanged.
Unit test: COUNT(DISTINCT f.label)=1 (all same label), count(f.label)=4
(non-distinct), COUNT(DISTINCT f.name)=4 (unique names). Smoke suite adds a
count(DISTINCT) query_graph check.
openCypher node-label alternation was unsupported (parse error after the
first label). parse_node now consumes `:A|B|C` into a single "A|B|C" label
string (mirroring the existing relationship-type alternation). The seed path
unions per-label results (scan_alternation_labels), and target/hop label
filters use label_alt_matches, which ORs over the alternatives. Single-label
matching is unchanged.
Unit test (Function|Module → 5 rows; Function|Class → 4) plus a query_graph
alternation check in the smoke suite. Relationship-type alternation tests
remain green.
openCypher label predicates in WHERE clauses were a parse error
("expected token type 67, got 73"). Parse `var:Label` in the WHERE leaf as
a condition with op=HAS_LABEL/value=Label, and evaluate it against the bound
node label in eval_condition (honoring NOT). Composes with AND/OR and works
in count(...) queries.
Unit test covers true/false/negated label predicates; smoke suite adds a
query_graph WHERE-label check.
The DISTINCT keyword on a WITH clause was parsed (r->distinct) but never
applied, so WITH DISTINCT silently returned duplicate rows. Add a dedup pass
in execute_with_clause that drops projected rows whose full value tuple
duplicates an earlier one (first occurrence kept), gated on wc->distinct.
No-op for the aggregation path (which already collapses per group).
Unit test (4 same-label functions collapse to 1 row; control without DISTINCT
keeps 4) plus a query_graph WITH DISTINCT check added to the smoke suite.
Confirmed working in the current query_graph evaluator and locked in:
- #240 labels() function with AS alias
- #237 DISTINCT applied before ORDER BY + LIMIT
- #252 toInteger()
- #305 count(*) with AS alias
Closes#240. Closes#237. Closes#252. Closes#305.
Adds three changes that landed together in PR #257, with a JSON-escape
hardening on the new File node properties so unusual file extensions
can't produce malformed output.
Mode A — search_graph results expose node properties_json
---------------------------------------------------------
emit_search_results in src/mcp/mcp.c now calls enrich_node_properties
for each result and parks the parsed yyjson_doc* in an out-array. The
two exit paths in handle_search_graph (success and sq_type_error) walk
that array and yyjson_doc_free each entry after yy_doc_to_str runs, so
zero-copy strings stay alive until serialization completes. Callers no
longer have to round-trip through get_code_snippet just to inspect a
node's properties.
Mode B — Cypher bare edge return surfaces full properties JSON
--------------------------------------------------------------
binding_get_virtual in src/cypher/cypher.c returns e->properties_json
(or "{}" if absent) when prop is NULL, instead of "". `RETURN r` on
an edge variable is now actually useful for inspecting timestamps,
weights, etc. without naming each property up front.
Temporal data on FILE_CHANGES_WITH edges and File nodes
-------------------------------------------------------
cbm_change_coupling_t gains last_co_change. cbm_commit_files_t and
the internal commit_t gain a timestamp field. Both git-log paths
(libgit2 and popen --pretty=format:COMMIT:%H:%ct) populate it.
cbm_compute_change_coupling carries a parallel pair_timestamps hash
table and writes the max timestamp to last_co_change. The edge
properties JSON now includes "last_co_change":<unix-epoch>.
cbm_pipeline_githistory_compute additionally aggregates per-file
change_count and last_modified into a new cbm_file_temporal_t array
exposed via result->file_temporal. cbm_pipeline_githistory_apply
upserts those values onto matching File nodes so callers can run
hotspot queries (most-changed, recently-changed) directly. Both the
sequential cbm_pipeline_pass_githistory and the parallel run_githistory
in pipeline.c free file_temporal and trigger apply when either
couplings or temporal entries are present.
The extension stored on the File node properties is now JSON-escaped
via cbm_json_escape before snprintf'ing into the props blob. The
original PR concatenated the raw extension; a hypothetical path with
a quote or backslash in the extension would have produced malformed
JSON. Practical risk was tiny (real extensions never contain those
characters) but the escape is cheap and correct.
Tests
-----
- githistory_coupling_carries_last_co_change pins last_co_change as
the max of the per-pair commit timestamps even when an unrelated
pair sits between them in the input order.
- cypher_bare_edge_return_exposes_properties_json regresses Mode B
by asserting `RETURN r` on an HTTP_CALLS edge surfaces its url_path
in the response.
- tool_search_graph_includes_node_properties regresses Mode A by
asserting search_graph's response contains the signature/return_type/
is_exported keys from a HandleRequest node's properties_json.
Existing githistory tests had their cbm_commit_files_t initializers
extended with a zero timestamp (backward-compatible — last_co_change
falls back to 0 for callers that don't supply timestamps).
Cherry-picked from #257; renamed the timestamp field types to
long long for portability across 32-/64-bit time_t platforms, used
the existing cbm_json_escape utility instead of raw concatenation
for the extension, and added explicit tests for all three changes.
Closes#257.
Co-authored-by: Adam Schulte <adamschulte148@gmail.com>
Enforce WarningsAsErrors: '*' with zero suppressions of entire check categories.
All non-FP warnings fixed at source; true false positives get per-instance NOLINT
with reasoning. Fixes include: braces around statements (1679), isolate declarations
(75), explicit casts for multilevel pointer conversions, missing includes, function
parameter documentation via NOLINT, and proper NOLINTBEGIN/NOLINTEND nesting.
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.
Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
Kris Kersey