Commit Graph
53 Commits
Author SHA1 Message Date
metehanulusoyandClaude Opus 5 3d9a6cf5cb fix(cypher): reject a non-numeric SKIP/LIMIT operand instead of dropping it
expect() fills p->error and returns NULL when the token after SKIP or LIMIT
is not a number, but both call sites only tested `if (num)` and then fell
through to `*out = r; return 0;`. r->limit kept the -1 "unset" initialiser,
which execution reads as "no LIMIT":

    rb_apply_skip_limit(rb, ret->skip, ret->limit >= 0 ? ret->limit : max_rows);

So `RETURN n.name LIMIT $limit` ran unbounded and answered isError:false with
the whole result set. `SKIP $offset LIMIT 10` lost both clauses at once,
because the orphaned operand also blocked the following match(p, TOK_LIMIT).

Propagate the failure the way the ORDER BY branch directly above already
does. This is the failure mode #1334 banned, reached by a different route.

Fixes #1994

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: metehanulusoy <ulusoy.metehan03@gmail.com>
2026-09-02 12:06:45 +03:00
Joshua RichterandClaude Opus 5 afc948bab1 fix(cypher): refuse a name that is not in scope instead of projecting a blank
A RETURN could name a variable the WITH before it dropped. The query ran,
exited clean, printed a column for that name, and filled it with empty
strings. Nothing said anything was wrong.

That silence is the harm. A reader sees a column of nothing and reads it as
"the graph holds no such data". The true answer is "your query named
something that is out of scope". For a code graph, a false "nothing calls X"
is the worst answer it can give.

The check runs on the parsed query rather than on the run-time bindings, and
that split is the whole point. It asks whether the query DECLARED the name,
which is a different question from whether a row happened to bind it. So the
existing convention is untouched: an OPTIONAL MATCH target that matched
nothing is still declared, still legal, and still projects "".

Two placeholders are skipped, because neither names anything the query
declared: count(*) stores "*", and a CASE expression stores "CASE".

A query with more names than the guard can model skips the check rather than
guessing. A wrong refusal costs the caller a working query, which is worse
than the silence this removes.

Same failure shape as #373, and the same answer: say so out loud.

Fixes #1919

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
2026-08-29 19:09:07 -04:00
King Star 7583376fd0 test(cypher): make bounded trails deterministic
Signed-off-by: King Star <mcxin.y@gmail.com>
2026-08-22 03:50:42 +08:00
King Star 12b7886648 test(cypher): avoid duplicate upstream regression
Signed-off-by: King Star <mcxin.y@gmail.com>
2026-08-20 22:42:09 +08:00
King Star cd22487ca6 fix(cypher): surface bounded trail traversal truncation
Signed-off-by: King Star <mcxin.y@gmail.com>
2026-08-20 22:38:57 +08:00
King Star 0d97729c39 fix(cypher): scope trail traversal to path expansion
Signed-off-by: King Star <mcxin.y@gmail.com>
2026-08-20 22:38:57 +08:00
King Star 8546070dc2 fix(cypher): enforce variable-length path semantics
Signed-off-by: King Star <mcxin.y@gmail.com>
2026-08-20 22:38:57 +08:00
Martin Vogel 58ef9f19ac fix(cypher): expansion materializes every matched row — the cap falsified aggregates
Second mechanism of #1196, exposed by the reporter's v0.10.6 retest: every
relationship-expansion site capped its output buffer at bind_cap*10, so edges
past the cap were silently dropped BEFORE WHERE and aggregation ever ran.
count() then reported the scanned prefix as if it were a fact — 9,360 of
13,691 DEFINES field-measured at --max-rows 1000 — and a LABEL on the source
did not protect you (the label workaround only ever fixed source
enumeration, which #1323 already made exact). max_rows is an output-row
limit per the public header; projection already enforces it.

All five capped sites now share one growable append (geometric growth,
size_t sizing): the per-hop expansion, its fixed/variable-length helpers and
process_edges, the bound-terminal driver, and the cross-join outer buffer.
Only allocation failure stops materialisation; match_count stays truthful
either way, so the #627 OPTIONAL contracts hold (a saturated buffer can no
longer exist, and the fallback rows share the same append). The #601
deadline still bounds pathological time, and hop caps keep bounding depth —
this removes only the silent row-dropping.

Regression test: 2 labeled sources x 30 edges with max_rows=2 — the old cap
returned count=20; ground truth 60 now holds, and the list form returns
exactly max_rows rows. Proven RED before and RED again on revert; cypher
181/181 and mcp suites green, including every OPTIONAL/#627 semantics test.

Fixes #1196 (together with #1323, which fixed the unlabeled source scan).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-18 11:19:32 +02:00
Martin Vogel 22d93dd12b Merge pull request #1323 from Enferlain/fix/1196-unlabeled-candidate-limit
fix(cypher): scan all unlabeled query candidates
2026-08-18 00:05:18 +02:00
Martin Vogel e51f7b5f07 Merge pull request #1394 from jstar0/fix/cypher-repeated-node-unification
fix(cypher): unify repeated variable-length nodes
2026-08-15 08:57:18 +02:00
Martin Vogel 75fed228d0 Merge pull request #1378 from SEPURI-SAI-KRISHNA/fix/cypher-bound-terminal-optional-drop
fix(cypher): make expand_from_bound_terminal OPTIONAL fallback lossless
2026-08-13 12:34:22 +02:00
Martin Vogel 886a8856a4 fix: bound response buffers, parse depth and glob matching
A pass over places where a bound was either miscalculated or missing.

UI response builders (src/ui/http_server.c) mixed the clamping helper
http_appendf with raw cursor indexing. Route the per-line separators and
quotes through the helper too, so the cursor cannot leave the buffer once
output saturates it, and terminate explicitly afterwards — the helper pins
the cursor and then writes nothing, which would otherwise leave the "%s"
reply with no terminator in range. The log buffer was budgeted at
LOG_LINE_MAX + 10 per line while JSON escaping can double every byte, so
size it for the escaped worst case. Job status entries now escape their path
and error fields, which were interpolated raw.

Cypher (src/cypher/cypher.c): the WHERE grammar descends once per nested '('
and once per NOT, so parse depth followed the query text rather than
anything bounded. Track depth on parser_t and refuse past 256 levels with a
parse error. Bounding parse depth also bounds the resulting tree, so the
recursive evaluator inherits the limit.

Gitignore (src/discover/gitignore.c): '**' retries the remainder at every
position and consecutive groups multiply, so match cost is exponential in
the number of groups. Thread a step budget through the matcher and give up
past 20000 steps, reporting no-match so a pathological pattern fails to
ignore rather than ignoring the wrong files. glob_match keeps its name so
recursion_whitelist.h still describes the functions that recurse; the new
non-recursive wrapper that seeds the budget is glob_match_bounded.

Call extraction (internal/cbm/extract_calls.c): extract_fp_callee recursed
once per applied argument, so stack use followed the parse-tree depth of the
indexed file. Rewritten as a left-spine loop. Reassigning node/nk before
continuing leaves the fall-through cases where the recursive form left them,
so behaviour is unchanged and the extraction suite is untouched.

CALLS props (src/pipeline/pass_calls.c): `cap - pos - PAIR_LEN` wraps once
pos reaches cap - PAIR_LEN and stops bounding the copy below. Guard
additively, matching the closing write in the same function.

Shell-bound paths (src/foundation/str_util.*): the three git shell-out sites
each wrap a repo path in cmd.exe-compatible double quotes, where %VAR%,
!VAR! and ^ remain active. Two carried private copies of the check that
rejects those three on Windows and the third used the bare validator.
Promote the stricter form to cbm_validate_shell_path_arg and route all three
through it, so the copies cannot drift apart again.

Tests cover the buffer bounds, the parse-depth refusal, matcher termination
and the validator. The bounds tests run in a forked child so a violation
surfaces as a signal instead of silently.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-05 23:27:48 +02:00
Martin Vogel 0a0ef516d6 fix(cypher): parse and apply the full multi-key ORDER BY list, keeping LIMIT (#1334)
The Cypher request model stored a single order_by expression; on
ORDER BY key1 DESC, key2 ASC LIMIT n the parser consumed only key1,
left ', key2 ... LIMIT n' unparsed, and the query silently returned the
entire result set (6326 rows / 117 KB instead of 5 on the reporter's
graph - a token flood straight into agent context).

The return clause now models up to CBM_CYPHER_ORDER_KEYS_MAX (8) sort
keys with per-key direction (Cypher semantics). Both sort sites -
rb_apply_order_by for RETURN and sort_bindings for the WITH pipeline -
compare key-by-key with later keys breaking ties. More keys than the
modeled maximum is a loud parse error, never a silently dropped
remainder. The no-ORDER-BY projection fast path is preserved via
order_key_count == 0.

Tests: parse-level (2 keys, per-key direction, LIMIT consumed;
9-key over-cap rejected), exec-level RETURN (limit kept, tiebreak
ordering, per-key direction), and exec-level WITH pipeline (limit kept).
All RED before the fix (row_count 4 instead of 2), GREEN after.

Recorded, pre-existing and unchanged: an ORDER BY key that is not part
of the projection is silently skipped (single-key main behaves the
same); sorting by unprojected properties needs hidden-column projection
and is a separate issue.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-04 18:09:33 +02:00
King Star ec29cd59a3 fix(cypher): unify repeated variable-length nodes
Signed-off-by: King Star <mcxin.y@gmail.com>
2026-08-01 10:53:01 +08:00
imi 64c5ddadf2 fix(cypher): scan all unlabeled query candidates
Signed-off-by: imi <hoshinoimi@gmail.com>
2026-07-31 18:08:40 +00:00
SEPURI-SAI-KRISHNA 068690cabf fix(cypher): make expand_from_bound_terminal OPTIONAL fallback lossless
Signed-off-by: SEPURI-SAI-KRISHNA <saik20533@gmail.com>
2026-07-31 22:22:09 +05:30
Martin Vogel 27d9831ab8 fix(cypher): never fabricate an OPTIONAL "no match" after the budget fills
process_edges and expand_var_length carried the expansion budget in the LOOP
condition (`ei < edge_count && *new_count < max_new`). Once new_count reached
max_new they stopped iterating entirely, so match_count was never incremented
for the remaining sources -- even though those sources had real neighbours.

expand_pattern_rels' OPTIONAL fallback is ungated (it has a reserved slot per
binding since the #1177 sizing fix), so it saw match_count == 0 and emitted an
unbound row for every such source.

The result is a false negative that reads as a positive assertion:

  MATCH (f:Function) OPTIONAL MATCH (f)-[:CALLS]->(c) WHERE c IS NULL

reports a function as having no callers when it has callers, once an earlier
function in the same scan saturated the budget. That is a dead-code query
telling a user that live code is dead.

The budget caps MATERIALISATION, never detection. Both loops now iterate the
full candidate set, increment match_count whenever a real neighbour passes the
filters, and gate only the write into new_bindings. The bound proof from #1177
is unchanged -- writes are still capped at max_new and fallbacks are still at
most one per binding -- so allocation safety does not depend on this change.

Cost: a saturated source still performs its node lookups and filter checks, the
same ones the unsaturated path performs, and materialises nothing.

Reported by @SEPURI-SAI-KRISHNA while working on the sibling bound-terminal path
in #1378; this is the same defect in expand_pattern_rels, which was already live
on main rather than introduced there.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-31 18:13:35 +02:00
Martin Vogel 95958222c8 Merge pull request #1221 from AmirF194/fix/1111-aggregate-scalar-func-misclassification
fix(cypher): classify aggregation columns by is_aggregate_func, not a bare func check
2026-07-31 09:59:09 +02:00
Amir Fathi 2f522b40cc fix(cypher): exclude entity-introspection funcs from WITH's bare-node id carry
The bare-node-carry check in with_agg_find_or_create only tested
!property && variable, so labels(n)/id(n)/keys(n)/properties(n) aliases
(variable set, property NULL, func set) were also tagged with the source
node's id. A later alias.property then hit node_prop's stub re-fetch
heuristic and silently returned the source node's real property instead
of empty for the non-node alias.

Adds cypher_issue1111_with_scalar_func_alias_no_node_leak, which fails
on the prior code (returns the source node's file_path) and passes with
this fix.

Addresses DeusData's review on #1221.

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
2026-07-31 06:18:52 +00:00
Amir Fathi b43bdddcbe fix(cypher): classify aggregation columns by is_aggregate_func, not a bare func check
RETURN/WITH aggregation classified each projected column with a bare
`if (item->func)` check to decide group-key vs. aggregate value. That
check is true for any function call, so type()/labels()/id()/keys()/
properties() were routed into the aggregate branch alongside real
aggregates and formatted via format_agg_value's default case, which
emits the row count instead of the function's actual value.

Swap the five call sites (ret_agg_build_key, ret_agg_emit_row,
with_agg_build_key, with_agg_find_or_create, with_agg_accumulate) to
is_aggregate_func(), the predicate already used correctly one level up
to decide whether a query needs aggregation at all. Group-key columns
carrying a non-aggregate function now project through the existing
project_item() helper instead of binding_get_virtual(), since they
need to evaluate the function rather than read a raw property.

Fixes #1111 (the type(r)/count(*) half; the inline-property-map half
of that issue does not reproduce on current HEAD)

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
2026-07-31 06:18:52 +00:00
SEPURI-SAI-KRISHNA c2a19b8979 fix(cypher): size expand_pattern_rels output for OPTIONAL fallback rows (heap OOB)
Signed-off-by: SEPURI-SAI-KRISHNA <saik20533@gmail.com>
2026-07-31 09:37:27 +05:30
Martin Vogel 436e4afc10 Merge pull request #1176 from SEPURI-SAI-KRISHNA/fix/cypher-cross-join-overflow
fix(cypher): bounds-check and error-propagate cross_join_nodes allocation
2026-07-31 01:52:29 +02:00
Martin Vogel 2f51672b26 Merge pull request #1173 from SEPURI-SAI-KRISHNA/fix/cypher-unwind-stack-overflow
fix(cypher): bound UNWIND literal-list buffer to prevent stack overflow
2026-07-31 01:51:45 +02:00
SEPURI-SAI-KRISHNA b07dbe67ad fix(cypher): bounds-check and error-propagate cross_join_nodes allocation
Signed-off-by: SEPURI-SAI-KRISHNA <saik20533@gmail.com>
2026-07-22 23:11:34 +05:30
SEPURI-SAI-KRISHNA d0457f2ba9 fix(cypher): bound UNWIND literal-list assembly buffer to prevent stack overflow
Signed-off-by: SEPURI-SAI-KRISHNA <saik20533@gmail.com>
2026-07-19 11:57:38 +05:30
Martin Vogel 0e00ef5702 feat: coordinate concurrent CBM sessions
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-16 19:20:46 +02:00
Martin Vogel 5d150ea051 Merge pull request #1109 from DeusData/fix/601-cypher-exec-deadline
fix(cypher): add a wall-clock execution deadline to abort runaway queries (#601)
2026-07-16 04:56:50 +02:00
Martin Vogel 6bd11ffb58 fix(cypher): add a wall-clock execution deadline to abort runaway queries
An unbounded whole-graph OPTIONAL MATCH or a GROUP BY that produces roughly one
group per node does O(bindings x groups) work in execute_return_agg and can run
for minutes on a large graph before emitting a single row. The 100k row ceiling
never fires (no rows are produced), so query_graph just hangs with no partial
result and no error (#601).

Arm a monotonic wall-clock budget (default 30s) at query entry and check it
(throttled, every 1024 iterations) in the scan, relationship-expansion and
aggregation hot loops. On expiry the query aborts with a clear, actionable error
instead of hanging. A thread-local test hook forces the budget so the guard is
covered by a deterministic reproduce-first test; a companion test confirms the
default budget does not false-positive on a normal query.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-16 03:09:27 +02:00
Martin Vogel 848375912d Merge pull request #1098 from JhohanBustamante/fix/cypher-composite-props
fix(cypher): return whole value for composite and escaped properties
2026-07-16 02:47:38 +02:00
KolisCode 48fa9cbc03 fix(cypher): return whole value for composite and escaped properties
json_extract_prop() scanned a non-string property up to the first ',' and a
string property up to the first '"', ignoring nesting and backslash escapes.
Any array/object property was therefore truncated at its first INTERNAL comma,
and any string containing an escaped quote was cut short.

    decorators: ["@Roles('OWNER', 'ADMIN')","@Get()"]
    projected as: ["@Roles('OWNER'

This makes decorator/route/authz queries unusable on frameworks whose
decorators take multiple arguments (NestJS, Angular, Spring). Values are now
copied as balanced constructs, honoring string state and escape pairs; scalar
and plain-string paths are unchanged.

Signed-off-by: KolisCode <jhohantma@gmail.com>
2026-07-14 22:31:44 -05:00
Harshita Joshi 755ffe57a5 fix(cypher): support coalesce() and other multi-arg scalar functions in WHERE (#874)
Signed-off-by: Harshita Joshi <j.harshitaa06@gmail.com>
2026-07-14 17:27:00 -05:00
Martin Vogel 73a3c347d9 fix(cypher): shortest-path var-length semantics + advertised depth clamp
Three of #797's four findings fixed (the fourth — repeated-variable
unification — already holds on main and is now pinned by the test):

- Walk padding: the BFS recursive CTE deduped (node, hop) PAIRS, so a
  single self-loop minted every hop level for every node it could
  reach — *k..k matched fabricated walks of arbitrary length (a
  100-hop 'longest chain' artifact) and the row set exploded to
  nodes x depth (the reported crash). The traversal now returns each
  node once at its MINIMAL hop (MIN/GROUP BY): variable-length matches
  nodes whose shortest path lies in [min,max]. This is deliberately
  shortest-path semantics, not openCypher trail enumeration — right
  for reachability/depth audits on call graphs and linear instead of
  exponential; trace_path shares the traversal and now also reports
  each node once on cyclic graphs.
- Silent 100-hop cap: a clamped hop range now surfaces as
  result->warning ('clamped ... an empty result may mean clamped, not
  no such path'), emitted by query_graph in both TOON and JSON forms,
  on top of the existing log WARN.

Reproduce-first: cypher_exec_varlength_path_semantics_issue797 —
self-loop + real 2-chain fixture; RED on the walk semantics (*2..2
returned 3 endpoints instead of 1), GREEN now, incl. variable
unification, zero rows for nonexistent path lengths, and the clamp
warning assertion.

Closes #797

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 20:58:28 +02:00
Martin Vogel 4382b97908 feat(cypher): coalesce(var.prop, literal) in WHERE clauses
The WHERE parser rejected coalesce() outright ('unexpected operator'),
blocking null-safe numeric filters over optional graph properties —
the exact audit-query shape from the report:
  MATCH (f:Function) WHERE coalesce(f.transitive_loop_depth, 0) >= 2
RETURN-side coalesce already worked; only the WHERE leaf lacked it.

Implementation: the condition leaf gains a coalesce_default literal.
The operator/value tail of condition parsing is factored into a shared
parse_condition_op so both subjects (var[.prop] and
coalesce(var.prop, literal)) accept the full operator set (comparisons,
IS [NOT] NULL, IN). At eval time a missing/empty property value is
substituted with the default before the operator runs. Supported form
is coalesce(var.prop, literal) — matching openCypher's two-argument
null-substitution for this subset.

Reproduce-first: cypher_exec_where_coalesce_issue874 runs the
reporter's exact query — RED (rc=-1 parse error) on the old parser —
and pins both directions: a failing default excludes property-less
nodes, a passing default includes them.

Closes #874

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 19:10:07 +02:00
Pcristin 6c7fe78e3b fix(cypher): apply distinct before return limits
Signed-off-by: Pcristin <xxxokzxxx@protonmail.com>
2026-07-09 09:40:44 +03:00
Martin Vogel e69e0844db cypher: bound RETURN/WITH projection width to the result column limit
The result projection is materialized per row into fixed-width stack
arrays (CBM_SZ_32 columns) in execute_return_simple and its siblings,
indexed by the parsed RETURN item count — which the parser left
unbounded. A RETURN/WITH clause with more than 32 projection items
wrote past those arrays. Reject an over-wide projection at parse time,
with a fork-isolated regression test driving a 48-column RETURN.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-07 20:21:48 +02:00
Martin Vogel 98e98c3208 fix: cap client-driven traversal depth (var-length Cypher + MCP tools)
Explicit variable-length Cypher bounds ([:CALLS*1..N]) and the client
`depth` on trace_call_path / detect_changes were passed straight into
cbm_store_bfs, so an arbitrarily large value drove an unbounded traversal
(a DoS on cyclic graphs). The MCP_MAX_DEPTH ceiling was defined but never
applied, and CYP_MAX_DEPTH capped only the unbounded var-length forms.

Add env-configurable ceilings cbm_cypher_max_depth() (CBM_CYPHER_MAX_DEPTH,
default 10) and cbm_mcp_max_depth() (CBM_MCP_MAX_DEPTH, default 15), clamp
both entry points to them, and WARN on clamp -- never a silent truncation.
Remove the now-dead CYP_MAX_DEPTH / MCP_MAX_DEPTH constants.

Reproduce-first: cypher_exec_var_length_explicit_bound_capped (12 -> 10 rows)
and tool_trace_call_path_depth_clamped (depth 1000 -> hop 15).

Closes #887.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-05 16:13:00 +02:00
Kris Kersey 8b03974f1b fix(cypher): preserve node properties through WITH aggregation
A node group variable carried through a WITH aggregation
(e.g. `WITH g, count(*) AS c RETURN g.file_path`) returned blank for every
property except its name: the carried virtual binding held only the group
key (the node's name) and lacked a store handle, so node_prop() could
neither read other fields nor compute degrees.
Fix: capture the node id of a bare node group-var in with_agg_find_or_create
and tag the carried virtual binding with it; in node_prop(), when such a stub
(id set, string fields unpopulated) is asked for a missing property, re-fetch
the full node via cbm_store_find_node_by_id and project it. Also propagate
the store onto virtual bindings so node_prop can re-fetch and compute
degrees. The stub gate is heuristic but never yields a wrong value — worst
case is one redundant indexed lookup. Adds regression test
cypher_exec_with_node_groupvar_prop.

Signed-off-by: Kris Kersey <kris@kerseyfabrications.com>
2026-06-14 20:12:04 +00:00
Martin Vogel 12683b84a9 feat(graph): queryable computation-bottleneck metrics + indexing perf & fixes
Per-function complexity metadata is now stored on graph nodes and queryable,
alongside several indexing performance and correctness fixes developed and
validated together (3704 tests, ASan/UBSan clean).

Bottleneck metrics (query via query_graph):
- Tier A (in the extraction AST walk): cyclomatic (complexity), cognitive
  (nesting-weighted), loop_count, loop_depth (max nested-loop depth),
  param_count, max_access_depth.
- Tier B (new pre-dump pass, pass_complexity.c): transitive_loop_depth
  propagated along CALLS edges + a recursive flag (direct self-recursion and
  mutual-recursion cycles), plus the call-context signals linear_scan_in_loop,
  alloc_in_loop, recursion_in_loop and unguarded_recursion.
- query_graph and get_architecture tool descriptions document the metrics and
  the Leiden community clusters.

Cypher engine:
- node_prop exposes arbitrary persisted node properties to WHERE/RETURN.
- Fix projection aliasing: multi-property rows shared a single static buffer so
  every column returned the last value read; now per-column/rotating buffers.
- Fix a stack-use-after-scope in aggregate RETURN (caller-owned value buffers).

Indexing performance:
- Gate C/C++ #define Macro-node extraction to full mode (it is ~49% of nodes on
  the Linux kernel); moderate/fast skip it.
- Emit the complexity property block only for Function/Method nodes so the
  millions of Macro/Field/Variable/Class/Enum nodes no longer carry zeroed
  fields — large RAM reduction at scale.
- Classify node types via tree-sitter TSSymbol bitsets in cbm_kind_in_set
  instead of per-node strcmp scans (thread-local cache, strcmp fallback;
  behaviour-identical).
- Subsample frequent (Zipfian) tokens in the semantic co-occurrence finalize;
  ~14x faster finalize on the kernel, output unchanged.
- pass_lsp_cross: replace O(n^2) linear dedup with hash-set dedup.

Windows:
- Canonicalize drive-letter case during path normalization so "c:/repo" and
  "C:/repo" derive the same project key and cache file (#394/#227/#367).

Tests: extraction, pipeline and cypher regressions covering all of the above.
2026-06-02 18:50:55 +02:00
Martin Vogel 08b62f03b9 feat(cypher): bounded EXISTS { } pattern predicate in WHERE
Add a single-hop, anchored existence predicate:
  WHERE NOT EXISTS { (f)<-[:CALLS]-() }   -- functions with no caller
  WHERE EXISTS { (f)-[:CALLS]->() }       -- functions that call something

Parsed via parse_exists_predicate (reusing parse_node/parse_rel) into a leaf
condition (op=EXISTS, anchor variable, edge type, direction); evaluated against
the bound node with cbm_store_find_edges_by_source_type / _by_target_type. This
is edge-type-specific, so it finds true orphans that in_degree/out_degree miss
(e.g. a node with only a DEFINES edge but no CALLS caller). Multi-hop / nested-
WHERE EXISTS is intentionally unsupported and errors clearly.

Tests cover the dead-code (NOT EXISTS) and has-outgoing (EXISTS) cases; smoke
coverage added. ASan-clean.

Refs: Cypher read suite (EXISTS predicate)
2026-06-01 21:17:56 +02:00
Martin Vogel c160465118 feat(cypher): multi-argument scalar functions (suite tier 2b)
Add an args-list (cbm_func_arg_t) to return items and parse comma-separated
arguments (var.prop or string/number literals), enabling multi-argument scalar
functions in projections:

- coalesce(a, b, ...)        first non-empty value
- substring(s, start[, len]) 0-indexed substring
- replace(s, from, to)       replace all occurrences
- left(s, n) / right(s, n)   leading/trailing n chars

Arguments are freed in free_return_clause (ASan-clean). Unit test
cypher_func_multiarg plus smoke coverage for substring + coalesce.

Refs: Cypher read suite (tier 2b)
2026-06-01 21:04:25 +02:00
Martin Vogel 67a7334e60 feat(cypher): string functions + fail loudly on unsupported syntax
Add single-argument string functions to projections — size, length, trim,
ltrim, rtrim, reverse — via the generic function recognizer.

More importantly, change the engine to FAIL LOUDLY on unsupported syntax instead
of silently projecting an empty column. An unknown function call (e.g.
split(...), coalesce(...)) or list indexing/slicing in RETURN/WITH now returns a
clear "unsupported function '<name>' (supported: ...)" error rather than a
valid-looking but blank result — the same silent-empty failure mode that hid the
labels() bug. This supersedes the #373 graceful-resync behaviour (its test is
flipped to assert the error; a new test covers an unknown function in RETURN).

Decision: full Cypher Tier 3 (lists/maps/paths/comprehensions/params — a value
data-model rewrite) is intentionally NOT implemented; clear errors on
unsupported features are the higher-value, lower-risk completeness win for the
read subset agents actually use.

Refs: Cypher read suite (tiers 1-2a + unsupported-syntax hardening)
2026-06-01 20:47:06 +02:00
Martin Vogel 71a6c5714e feat(cypher): scalar + entity-introspection functions (suite tier 1)
Add single-argument functions to RETURN/WITH projections, dispatched via a
generic IDENT-call recognizer so new functions no longer need a dedicated lexer
keyword:

- labels(n)      -> ["<label>"]
- type(r)        -> relationship type
- id(n)          -> node/edge identity
- keys(n)        -> JSON list of the node's non-null property keys
- properties(n)  -> the node/edge properties JSON object
- toInteger/toFloat/toBoolean(x) -> numeric/boolean casts (null on non-numeric)

Previously labels()/id()/keys()/etc. fell into the unknown-function path and
silently projected empty (labels(n) returned ""); they now evaluate correctly.
Regex =~ in WHERE was already supported. Unit tests for each function plus
smoke-test coverage via the live query_graph CLI.

Refs: full Cypher read suite (tier 1 of 3)
2026-06-01 20:28:53 +02:00
Martin Vogel a2cd1e7c07 fix(cypher): resync on unsupported WITH/RETURN expression (#373)
split(f.path,'/')[0] and similar unknown function-call / index
expressions left their trailing tokens unconsumed, desyncing the parser into
a misleading default star projection (columns f.name/f.qualified_name/
f.label, one blank row per node). parse_return_item now consumes the balanced
parentheses/brackets of an unknown call so the parser stays in sync: the
requested RETURN/WITH column shape is preserved (aliases intact), sibling
aggregates like count(*) still compute, and the unsupported expression simply
projects empty instead of corrupting the whole projection. Existing scalar
forms (labels(), toInteger()) are unaffected.
2026-05-31 20:40:55 +02:00
Martin Vogel 869341c709 feat(cypher): support COUNT(DISTINCT x) (#239)
count(DISTINCT x) was a parse error ("expected token type 85, got 9").
parse_aggregate_item now consumes an optional DISTINCT inside the call and
records it on the return item. Both aggregation paths track the set of unique
values per group and emit its size for COUNT(DISTINCT): the RETURN path reuses
the existing collect_lists machinery; the WITH path gains distinct_lists/
distinct_n fields. Plain COUNT and the other aggregates are unchanged.

Unit test: COUNT(DISTINCT f.label)=1 (all same label), count(f.label)=4
(non-distinct), COUNT(DISTINCT f.name)=4 (unique names). Smoke suite adds a
count(DISTINCT) query_graph check.
2026-05-31 20:22:55 +02:00
Martin Vogel 2a5515e586 feat(cypher): support label alternation MATCH (n:A|B) (#242)
openCypher node-label alternation was unsupported (parse error after the
first label). parse_node now consumes `:A|B|C` into a single "A|B|C" label
string (mirroring the existing relationship-type alternation). The seed path
unions per-label results (scan_alternation_labels), and target/hop label
filters use label_alt_matches, which ORs over the alternatives. Single-label
matching is unchanged.

Unit test (Function|Module → 5 rows; Function|Class → 4) plus a query_graph
alternation check in the smoke suite. Relationship-type alternation tests
remain green.
2026-05-31 20:11:12 +02:00
Martin Vogel b30e6d6443 feat(cypher): support label tests in WHERE — WHERE n:Label (#241)
openCypher label predicates in WHERE clauses were a parse error
("expected token type 67, got 73"). Parse `var:Label` in the WHERE leaf as
a condition with op=HAS_LABEL/value=Label, and evaluate it against the bound
node label in eval_condition (honoring NOT). Composes with AND/OR and works
in count(...) queries.

Unit test covers true/false/negated label predicates; smoke suite adds a
query_graph WHERE-label check.
2026-05-31 20:02:23 +02:00
Martin Vogel 3014867cb3 feat(cypher): apply WITH DISTINCT deduplication (#238)
The DISTINCT keyword on a WITH clause was parsed (r->distinct) but never
applied, so WITH DISTINCT silently returned duplicate rows. Add a dedup pass
in execute_with_clause that drops projected rows whose full value tuple
duplicates an earlier one (first occurrence kept), gated on wc->distinct.
No-op for the aggregation path (which already collapses per group).

Unit test (4 same-label functions collapse to 1 row; control without DISTINCT
keeps 4) plus a query_graph WITH DISTINCT check added to the smoke suite.
2026-05-31 19:53:25 +02:00
Martin Vogel 684045708f test(cypher): regression guards for #240/#237/#252/#305 (already supported)
Confirmed working in the current query_graph evaluator and locked in:
- #240 labels() function with AS alias
- #237 DISTINCT applied before ORDER BY + LIMIT
- #252 toInteger()
- #305 count(*) with AS alias

Closes #240. Closes #237. Closes #252. Closes #305.
2026-05-31 12:38:05 +02:00
Adam Schulte 4e80d2fff7 feat(graph): temporal properties on FILE_CHANGES_WITH edges and File nodes (#257)
Adds three changes that landed together in PR #257, with a JSON-escape
hardening on the new File node properties so unusual file extensions
can't produce malformed output.

Mode A — search_graph results expose node properties_json
---------------------------------------------------------

emit_search_results in src/mcp/mcp.c now calls enrich_node_properties
for each result and parks the parsed yyjson_doc* in an out-array. The
two exit paths in handle_search_graph (success and sq_type_error) walk
that array and yyjson_doc_free each entry after yy_doc_to_str runs, so
zero-copy strings stay alive until serialization completes. Callers no
longer have to round-trip through get_code_snippet just to inspect a
node's properties.

Mode B — Cypher bare edge return surfaces full properties JSON
--------------------------------------------------------------

binding_get_virtual in src/cypher/cypher.c returns e->properties_json
(or "{}" if absent) when prop is NULL, instead of "". `RETURN r` on
an edge variable is now actually useful for inspecting timestamps,
weights, etc. without naming each property up front.

Temporal data on FILE_CHANGES_WITH edges and File nodes
-------------------------------------------------------

cbm_change_coupling_t gains last_co_change. cbm_commit_files_t and
the internal commit_t gain a timestamp field. Both git-log paths
(libgit2 and popen --pretty=format:COMMIT:%H:%ct) populate it.
cbm_compute_change_coupling carries a parallel pair_timestamps hash
table and writes the max timestamp to last_co_change. The edge
properties JSON now includes "last_co_change":<unix-epoch>.

cbm_pipeline_githistory_compute additionally aggregates per-file
change_count and last_modified into a new cbm_file_temporal_t array
exposed via result->file_temporal. cbm_pipeline_githistory_apply
upserts those values onto matching File nodes so callers can run
hotspot queries (most-changed, recently-changed) directly. Both the
sequential cbm_pipeline_pass_githistory and the parallel run_githistory
in pipeline.c free file_temporal and trigger apply when either
couplings or temporal entries are present.

The extension stored on the File node properties is now JSON-escaped
via cbm_json_escape before snprintf'ing into the props blob. The
original PR concatenated the raw extension; a hypothetical path with
a quote or backslash in the extension would have produced malformed
JSON. Practical risk was tiny (real extensions never contain those
characters) but the escape is cheap and correct.

Tests
-----

- githistory_coupling_carries_last_co_change pins last_co_change as
  the max of the per-pair commit timestamps even when an unrelated
  pair sits between them in the input order.
- cypher_bare_edge_return_exposes_properties_json regresses Mode B
  by asserting `RETURN r` on an HTTP_CALLS edge surfaces its url_path
  in the response.
- tool_search_graph_includes_node_properties regresses Mode A by
  asserting search_graph's response contains the signature/return_type/
  is_exported keys from a HandleRequest node's properties_json.

Existing githistory tests had their cbm_commit_files_t initializers
extended with a zero timestamp (backward-compatible — last_co_change
falls back to 0 for callers that don't supply timestamps).

Cherry-picked from #257; renamed the timestamp field types to
long long for portability across 32-/64-bit time_t platforms, used
the existing cbm_json_escape utility instead of raw concatenation
for the extension, and added explicit tests for all three changes.

Closes #257.

Co-authored-by: Adam Schulte <adamschulte148@gmail.com>
2026-05-09 01:15:03 +02:00
Matthew Prock 1bbba506b5 fix(cypher,store): prevent crashes from buffer overflow, OOM, and NULL stmts (#206)
- cypher: bounds check in lex_string_literal (4096-byte stack buffer, escapes still consumed past truncation)
- cypher: NULL checks + non-destructive realloc with cleanup in parse_props, parse_rel_types, parse_in_list, parse_case_expr
- store: sqlite3_prepare_v2 return-code checks in cbm_store_get_schema and collect_pkg_names; partial-output cleanup
- tests: regression test for >4096-byte string literal truncation
2026-05-08 20:10:51 +02:00