The lint chain stops at its first red target, so a clang-format failure hid the memory-core linter
until PR CI: the coverage line-offset table, the Cypher direction swap and the search label each
added a raw allocator site. The table now comes from cbm_alloc/cbm_free (extract class), the swap
frees through safe_str_free, and the BM25 row array moves to cbm_calloc/cbm_free so mcp.c's raw-site
count goes down (baseline 814 -> 813) rather than up.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Nine findings from the 2026-09-16 graph-quality probe over the eleven bench corpora, each with a
test that fails without its fix and a before/after measurement on the real corpus.
Deterministic resolution (O9). Two indexes of one kernel tree differed in 5,821 IMPORTS and 632
CALLS edges. Three order/load dependencies carried it: the `#include` target was the first by-name
hit whose path ends with the include path — a bucket in worker merge order (include/linux/device.h
vs tools/virtio/linux/device.h); the registry broke suffix_match score ties by bucket position; the
LSP budget-share rule measured wall time. Include targets now prefer the File node, the least nested
path, then the lexicographic path; registry ties prefer the least nested QN, then the smaller QN; the
budget share is measured in thread CPU time like the budget itself. Two kernel indexes are now
identical (8,529,723 nodes, 1,974,940 CALLS, zero churn), as are two elasticsearch indexes.
Findability. search_graph's BM25 path honours the `label` filter and ranks an exact-name definition
above test methods that repeat the token (15/15 probe queries).
Coverage. The parse-coverage gap tree subtracts definitions the walker recovered inside an error
region (multi-line only); kernel/sched/core.c reports 476 flagged lines instead of 7,674.
Precision. Config-language callers (Makefile, CMake, YAML) no longer bind unique_name into code
(kernel Makefile -> C CALLS 195 -> 0). JS-family token names ({}, 1, (x: T) => U) are not
definitions; computed, quoted and escaped member keys stay (TypeScript 204 junk -> 0, 1,782 real
names kept). Names are cut at their first line (elasticsearch YAML 8,306 -> 0); a comment inside an
argument list is not an argument, so no Route node is named by a Java block comment; the wildcard
route `/*` is still a route. HTML/Vue/Svelte/Astro embedded member calls stay unbound.
Recall. A Python call through a self- or cls-rooted attribute keeps a specific unique_name member
target (Django +751, elasticsearch +3,288 CALLS, no new callee names).
Robustness. Edge properties never carry a split UTF-8 sequence (the 120-byte expression cut and the
JSON escaper both trim a partial code point). CLI array flags accept a JSON-array literal. Cypher
seeds a single-hop pattern from its selective end (kernel `MATCH (a)-[:CALLS]->(b) WHERE b.name =
'_printk'`: execution limit -> 7.9 s).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The head of the first pattern is bound whether the query names it or not:
execute_single falls back to the synthetic name "_n0". The capacity check
counted only names, so an unnamed head plus 16 named nodes passed it and
binding_set then dropped the 16th name without a word — the same silence
this guard exists to remove.
Count that slot, and name the synthetic variable once instead of twice.
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
check_projection_scope models declared names in a fixed 32-entry array
and skipped the check entirely when a query declared more. Nothing
bounded how many variables a pattern could declare, so the out-of-scope
refusal added by #1922 switched itself off on a wide query: the same
undeclared name was refused at 10 declared names and quietly accepted at
35, answering a column of empty strings.
Skipping was deliberate — a wrong refusal costs the caller a working
query, which is worse than the silence. The premise was wrong. A binding
holds CYP_MAX_VARS node variables and CYP_MAX_EDGE_VARS edge variables in
plain arrays, and binding_set and binding_set_edge drop anything past
those without a word, so a query naming more cannot be answered at all.
Its extra names bind to nothing and project as blanks, which reads as
"the graph holds no such data". The choice was never refuse-or-stay-quiet;
it was refuse, or answer wrong.
So bound the input. check_pattern_var_capacity counts distinct node and
edge variables across every pattern — they share one binding, and an
OPTIONAL MATCH pattern sits in the same array — and refuses beyond what a
binding holds, naming the limit and how to get under it. Leaving the name
off a node frees its slot, because an unnamed node takes none.
Bounding the input rather than each consumer also makes three other silent
drops unreachable: binding_set past 16, binding_set_edge past 8, and the
column collection in execute_default_projection. Those would otherwise
need an error path threaded up through every caller, since both binding
setters return void.
The two `< 0` branches in check_projection_scope stay, with comments
saying they can no longer fire. They keep the guard standing if either
bound ever moves.
Fixes#1995
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
Fix-forward after rebasing onto main (545 commits, 26 conflicted-file
resolutions). Where main had shipped a behaviour in the meantime, main
wins; the PR keeps its output contract on top of it:
- search_code: the scan runs under main's supervised, deadline- and
output-bounded runner and feeds the branch's streaming classifier from
the runner's output file; the POSIX wrappers map grep's no-match to 0,
so every non-zero exit fails closed (no exit-1 special case).
- trace_path: strategy/confidence precede args in header and rows (#1542),
in both the tree table and the json legs.
- cypher: relationship expansion is never capped (a cap falsified
aggregates) and unlabeled MATCH scans every candidate; `truncated` is
raised only by a real ceiling — a clamped hop range is probed one hop
beyond the cap and reports truncated only when a candidate exists there.
The plain BFS and the trail BFS both probe one row past max_results so
saturation is observable instead of silent.
- search_code full mode and the final json use main's U+FFFD sanitizer
instead of '?' replacement.
- search_graph json keeps an empty `groups` array for semantic-only calls;
list_projects json echoes offset/limit next to next_offset and accepts
main's include_details spelling for the stats projection.
- discovery budget: main added get_file_outline, compare_graphs, manage_adr
set_sections and the debug/include_details parameters; their descriptions
are trimmed to the lean style and the ceiling moves from 15 to 18 KiB.
Tests that asserted a JSON default now request format:"json"; the
fail-closed search test uses an unreadable regular file (a missing or
non-regular indexed path is deliberately not a scan operand on main); the
non-regular fixture gives its real sources File nodes, as every indexed
project has; list_projects total counts listable projects (the 0-byte
ghost is never a row, so it is not a page either).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
execute_return_star_after_with sized its column arrays at CYP_MAX_VARS
while the parser allowed up to CBM_SZ_32 items. A WITH naming 17 to 32
aliases parsed cleanly, then answered 16 columns and dropped the rest
with no error. The caller could not tell that from a query that genuinely
had 16 columns.
Widening those arrays to 32 is the wrong fix and would make the result
worse. A binding holds exactly CYP_MAX_VARS variables, and
with_add_vbinding_var drops any alias past the 16th on the way in. The
projection would then report 20 columns of which 4 are always blank —
silent wrong data in place of a silent short answer. The 16 was not an
arbitrary undersize; it matched what the binding can carry.
So bound the WITH where the RETURN is already bounded, in
parse_return_or_with, and refuse the query instead of answering it
short. The clamp in execute_return_star_after_with stays: it can no
longer fire, and it is what keeps that function correct if the bound
ever moves.
The test asserts both halves — a 20-alias WITH is refused, and a
16-alias WITH still succeeds with 16 columns — so the guard rejects only
what the binding genuinely cannot hold.
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
RETURN * built its columns from the variables the query pattern named,
never from the bindings it was about to project. One line caused two
separate wrong answers, and neither one reported an error.
After a WITH, the pattern's variables are out of scope — the WITH
replaced them with the names it made. The old code still asked for the
old names, found none of them, and answered a full result of empty
strings. This query used to print twelve columns of nothing:
MATCH (f:Function) OPTIONAL MATCH (f)-[:CALLS]->(g)
WITH f.name AS caller, g.name AS callee RETURN *
It now prints two columns, caller and callee, holding their values. A
name the WITH made holds one value rather than a node, so it gets one
column, not the four a node variable gets.
Separately, collect_pattern_vars appended every pattern's variables with
no repeat check. A variable named in two patterns got its four columns
twice, which the OPTIONAL MATCH above does with f.
Two tests cover both faults and fail against the old code:
cypher_return_star_dedups_repeated_pattern_var col_count 12, want 8
cypher_return_star_after_with_names_aliases col_count 8, want 2
Cypher suite: 185 passed, 0 failed. clang-format clean on both files.
Reported alongside a second fault this does NOT fix: a variable the WITH
dropped is still accepted afterwards and renders empty, because nothing
checks a projected name against the live scope. See
.agents/research/2026-08-29-cypher-return-star-and-with-scope.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
expect() fills p->error and returns NULL when the token after SKIP or LIMIT
is not a number, but both call sites only tested `if (num)` and then fell
through to `*out = r; return 0;`. r->limit kept the -1 "unset" initialiser,
which execution reads as "no LIMIT":
rb_apply_skip_limit(rb, ret->skip, ret->limit >= 0 ? ret->limit : max_rows);
So `RETURN n.name LIMIT $limit` ran unbounded and answered isError:false with
the whole result set. `SKIP $offset LIMIT 10` lost both clauses at once,
because the orphaned operand also blocked the following match(p, TOK_LIMIT).
Propagate the failure the way the ORDER BY branch directly above already
does. This is the failure mode #1334 banned, reached by a different route.
Fixes#1994
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: metehanulusoy <ulusoy.metehan03@gmail.com>
cbm_parse built a query and returned success without checking that it
had read every token. The grammar accepts at most one WITH and treats
RETURN as optional, so the parser stopped at the first thing it did not
understand and reported success anyway.
The dropped tail took the filter and the RETURN with it. The engine then
answered from the fragment it had parsed, using its default projection.
It reported success and returned wrong rows, which is worse than a
refusal, because nothing tells the caller to look.
Two shapes hit this:
MATCH (f:Function) WHERE f.name = 'x' RETURN f.name AS n BANANA SPLIT 99
-> one row, no error, the trailing words silently dropped
MATCH (f:Function)
OPTIONAL MATCH (a)-[:CALLS]->(f)
WITH f, count(a) AS calls
OPTIONAL MATCH (b)-[:USAGE]->(f)
WITH f, calls, count(b) AS usages
WHERE calls = 0 AND usages = 0
RETURN f.name AS n
-> every function, unfiltered, under the default column names
cbm_parse now checks that the cursor sits on the end-of-input token
before it returns success, and names the leftover token when it does not.
The message adds a note about the one-WITH limit, but only when a
standalone WITH really sits in the part that went unread. Appending it
every time pointed the reader at WITH when the problem was a typo -- the
BANANA query above holds no WITH anywhere. The test cannot simply look at
the token the parse stopped on either: on the second-WITH query above it
stops at OPTIONAL rather than at WITH, because parse_post_where consumes
the first WITH and then has no way to take another MATCH stage. So it
scans the unread tail, and skips a WITH straight after STARTS, which is
the STARTS WITH operator rather than a clause.
The check exposed a second defect. parse_post_where parses the branch
after UNION by calling cbm_parse on a slice of the same tokens, using a
separate parser. The outer parser's cursor never moved past the UNION
keyword, so a valid UNION query looked unfinished. That cursor was
already wrong; nothing caught it, because nothing checked where the
cursor ended up. The UNION branch now moves the cursor to the end after
the sub-parse succeeds, which is sound because that sub-parse no longer
returns success with tokens left over.
Three tests cover both directions. Removing only the guard turns the two
rejection tests red with rc == 0 -- the parser reporting success on input
it never finished reading -- while the acceptance test stays green, so
the guard is load-bearing for exactly these two behaviours.
The two rejection tests also pin the message. The BANANA case must name
BANANA and must not mention WITH; that assertion was seen red first, at
tests/test_cypher.c:226. The second-WITH case must still carry the note;
that one passed before and after, and is the control that stops the note
being suppressed everywhere instead of only where it misleads.
Verified on macOS with Apple clang:
make -f Makefile.cbm test-focused TEST_SUITES=cypher -> 186 passed
guard removed, tests kept -> 184 passed, 2 failed
make -f Makefile.cbm cbm -> exit 0, no warnings
The full suite reports 7623 passed, 2 failed. Both failures are in
tests/test_cli.c (lines 1748 and 6723) and reproduce on a clean tree with
this change stashed out. They depend on the coding agents installed on
the machine, not on this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
A RETURN could name a variable the WITH before it dropped. The query ran,
exited clean, printed a column for that name, and filled it with empty
strings. Nothing said anything was wrong.
That silence is the harm. A reader sees a column of nothing and reads it as
"the graph holds no such data". The true answer is "your query named
something that is out of scope". For a code graph, a false "nothing calls X"
is the worst answer it can give.
The check runs on the parsed query rather than on the run-time bindings, and
that split is the whole point. It asks whether the query DECLARED the name,
which is a different question from whether a row happened to bind it. So the
existing convention is untouched: an OPTIONAL MATCH target that matched
nothing is still declared, still legal, and still projects "".
Two placeholders are skipped, because neither names anything the query
declared: count(*) stores "*", and a CASE expression stores "CASE".
A query with more names than the guard can model skips the check rather than
guessing. A wrong refusal costs the caller a working query, which is worse
than the silence this removes.
Same failure shape as #373, and the same answer: say so out loud.
Fixes#1919
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
Second mechanism of #1196, exposed by the reporter's v0.10.6 retest: every
relationship-expansion site capped its output buffer at bind_cap*10, so edges
past the cap were silently dropped BEFORE WHERE and aggregation ever ran.
count() then reported the scanned prefix as if it were a fact — 9,360 of
13,691 DEFINES field-measured at --max-rows 1000 — and a LABEL on the source
did not protect you (the label workaround only ever fixed source
enumeration, which #1323 already made exact). max_rows is an output-row
limit per the public header; projection already enforces it.
All five capped sites now share one growable append (geometric growth,
size_t sizing): the per-hop expansion, its fixed/variable-length helpers and
process_edges, the bound-terminal driver, and the cross-join outer buffer.
Only allocation failure stops materialisation; match_count stays truthful
either way, so the #627 OPTIONAL contracts hold (a saturated buffer can no
longer exist, and the fallback rows share the same append). The #601
deadline still bounds pathological time, and hop caps keep bounding depth —
this removes only the silent row-dropping.
Regression test: 2 labeled sources x 30 edges with max_rows=2 — the old cap
returned count=20; ground truth 60 now holds, and the list form returns
exactly max_rows rows. Proven RED before and RED again on revert; cypher
181/181 and mcp suites green, including every OPTIONAL/#627 semantics test.
Fixes#1196 (together with #1323, which fixed the unlabeled source scan).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A pass over places where a bound was either miscalculated or missing.
UI response builders (src/ui/http_server.c) mixed the clamping helper
http_appendf with raw cursor indexing. Route the per-line separators and
quotes through the helper too, so the cursor cannot leave the buffer once
output saturates it, and terminate explicitly afterwards — the helper pins
the cursor and then writes nothing, which would otherwise leave the "%s"
reply with no terminator in range. The log buffer was budgeted at
LOG_LINE_MAX + 10 per line while JSON escaping can double every byte, so
size it for the escaped worst case. Job status entries now escape their path
and error fields, which were interpolated raw.
Cypher (src/cypher/cypher.c): the WHERE grammar descends once per nested '('
and once per NOT, so parse depth followed the query text rather than
anything bounded. Track depth on parser_t and refuse past 256 levels with a
parse error. Bounding parse depth also bounds the resulting tree, so the
recursive evaluator inherits the limit.
Gitignore (src/discover/gitignore.c): '**' retries the remainder at every
position and consecutive groups multiply, so match cost is exponential in
the number of groups. Thread a step budget through the matcher and give up
past 20000 steps, reporting no-match so a pathological pattern fails to
ignore rather than ignoring the wrong files. glob_match keeps its name so
recursion_whitelist.h still describes the functions that recurse; the new
non-recursive wrapper that seeds the budget is glob_match_bounded.
Call extraction (internal/cbm/extract_calls.c): extract_fp_callee recursed
once per applied argument, so stack use followed the parse-tree depth of the
indexed file. Rewritten as a left-spine loop. Reassigning node/nk before
continuing leaves the fall-through cases where the recursive form left them,
so behaviour is unchanged and the extraction suite is untouched.
CALLS props (src/pipeline/pass_calls.c): `cap - pos - PAIR_LEN` wraps once
pos reaches cap - PAIR_LEN and stops bounding the copy below. Guard
additively, matching the closing write in the same function.
Shell-bound paths (src/foundation/str_util.*): the three git shell-out sites
each wrap a repo path in cmd.exe-compatible double quotes, where %VAR%,
!VAR! and ^ remain active. Two carried private copies of the check that
rejects those three on Windows and the third used the bare validator.
Promote the stricter form to cbm_validate_shell_path_arg and route all three
through it, so the copies cannot drift apart again.
Tests cover the buffer bounds, the parse-depth refusal, matcher termination
and the validator. The bounds tests run in a forked child so a violation
surfaces as a signal instead of silently.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The over-cap ORDER BY rejection path (and the two sibling error paths
in parse_return_or_with) freed the items array but not the strings
inside it - ASan flagged 7 leaked bytes on the diag and Linux legs the
moment the new over-cap test exercised the path. Route all three error
paths through free_return_clause(), which already releases item
strings, CASE expressions, args and the ORDER BY key list.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Cypher request model stored a single order_by expression; on
ORDER BY key1 DESC, key2 ASC LIMIT n the parser consumed only key1,
left ', key2 ... LIMIT n' unparsed, and the query silently returned the
entire result set (6326 rows / 117 KB instead of 5 on the reporter's
graph - a token flood straight into agent context).
The return clause now models up to CBM_CYPHER_ORDER_KEYS_MAX (8) sort
keys with per-key direction (Cypher semantics). Both sort sites -
rb_apply_order_by for RETURN and sort_bindings for the WITH pipeline -
compare key-by-key with later keys breaking ties. More keys than the
modeled maximum is a loud parse error, never a silently dropped
remainder. The no-ORDER-BY projection fast path is preserved via
order_key_count == 0.
Tests: parse-level (2 keys, per-key direction, LIMIT consumed;
9-key over-cap rejected), exec-level RETURN (limit kept, tiebreak
ordering, per-key direction), and exec-level WITH pipeline (limit kept).
All RED before the fix (row_count 4 instead of 2), GREEN after.
Recorded, pre-existing and unchanged: an ORDER BY key that is not part
of the projection is silently skipped (single-key main behaves the
same); sorting by unprojected properties needs hidden-column projection
and is a separate issue.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The memory-diagnostics report's priority-4 lane (path-sensitive clang-analyzer,
memory checks only) run over all 111 production files. 21 findings triaged;
the real ones, all cold-path (none can explain #581's per-query residual):
LEAKS
- mcp get_architecture: scope_path leaked on the missing-store early return
(REQUIRE_STORE frees only `project`); allocate after the gate.
- pass_definitions: cancellation mid-extraction leaked the pass-owned result
cache including already-extracted entries; mirror the end-of-pass cleanup.
- store package-boundary scan: the row-scan abort path freed the node arrays
but not the boundary accumulators or their duplicated package strings.
- cbm quarantine set: a duplicate path line leaked the replaced value (and a
fresh key copy -- the table borrows key pointers); a partial strdup failure
leaked the surviving half. Reuse the stored key for duplicates.
- pass_githistory: unchecked malloc/strdup -- an OOM dereferenced NULL and a
failed strdup leaked the index cell. Allocate before claiming the slot.
NULL/UB
- cli config subcommand: NULL argv with nonzero argc slipped the guard (the
inner `argv &&` shielded only the help comparison) into argv[0].
- store bfs_multi: a negative max_results broke out before any row was
written, then freed fields of an unwritten negative-index slot. Clamp.
- pass_calls emit_http_async_edge: the service-pattern call sites pass a NULL
target behind a hand-duplicated URL predicate; a drift between the copies
turned target->id into a null deref. The callee is now total.
- sqlite_writer: both leaf-array OOM paths left leaf_count stale with a NULL
array, walking pb_finalize_* into leaves[0]; consistent empty state routes
them to the existing root=0 failure return.
HARDENED (invariants true but invisible to path-sensitive analysis)
- Leiden CSR + aggregate arrays, SCC adjacency: calloc + endpoint guards, so
a future degree/collection miscount degrades benignly instead of UB.
- SCC cycle fill: the ncyc==0 no-slot invariant made local.
RECORDED FALSE POSITIVES (no code change)
- yaml sequence starts (loop bound == alloc bound), cypher agg arrays (same
count both sides), mcp read_message ch (assigned by fgetc each iteration),
pkgmap clean buffer, mcp csize (Tarjan: ncomp>=1 when nverts>=1), vendored
verstable x2.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
GATE + LANES (user decision: runner cost accepted)
- make lint-mem (local triage) and lint-mem-ci (gating: vendored-filtered,
any remaining finding fails). The gate is green because every false
positive above was restructured for provability -- calloc'd fill-cursor
arrays, explicit Tarjan invariant, zeroed buffer tails, min-1-element
allocations -- never suppressed.
- make diag: pinned newest-LLVM ASan/UBSan lane with straighter stacks.
- CI: lint-mem job (_lint.yml) and test-diag job (_test.yml), both on the
pinned LLVM 22 apt toolchain. Cost disclosure: roughly +25-40 min and
+25-60 min (ccache-warm) per push respectively.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
process_edges and expand_var_length carried the expansion budget in the LOOP
condition (`ei < edge_count && *new_count < max_new`). Once new_count reached
max_new they stopped iterating entirely, so match_count was never incremented
for the remaining sources -- even though those sources had real neighbours.
expand_pattern_rels' OPTIONAL fallback is ungated (it has a reserved slot per
binding since the #1177 sizing fix), so it saw match_count == 0 and emitted an
unbound row for every such source.
The result is a false negative that reads as a positive assertion:
MATCH (f:Function) OPTIONAL MATCH (f)-[:CALLS]->(c) WHERE c IS NULL
reports a function as having no callers when it has callers, once an earlier
function in the same scan saturated the budget. That is a dead-code query
telling a user that live code is dead.
The budget caps MATERIALISATION, never detection. Both loops now iterate the
full candidate set, increment match_count whenever a real neighbour passes the
filters, and gate only the write into new_bindings. The bound proof from #1177
is unchanged -- writes are still capped at max_new and fallbacks are still at
most one per binding -- so allocation safety does not depend on this change.
Cost: a saturated source still performs its node lookups and filter checks, the
same ones the unsaturated path performs, and materialises nothing.
Reported by @SEPURI-SAI-KRISHNA while working on the sibling bound-terminal path
in #1378; this is the same defect in expand_pattern_rels, which was already live
on main rather than introduced there.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The bare-node-carry check in with_agg_find_or_create only tested
!property && variable, so labels(n)/id(n)/keys(n)/properties(n) aliases
(variable set, property NULL, func set) were also tagged with the source
node's id. A later alias.property then hit node_prop's stub re-fetch
heuristic and silently returned the source node's real property instead
of empty for the non-node alias.
Adds cypher_issue1111_with_scalar_func_alias_no_node_leak, which fails
on the prior code (returns the source node's file_path) and passes with
this fix.
Addresses DeusData's review on #1221.
Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
RETURN/WITH aggregation classified each projected column with a bare
`if (item->func)` check to decide group-key vs. aggregate value. That
check is true for any function call, so type()/labels()/id()/keys()/
properties() were routed into the aggregate branch alongside real
aggregates and formatted via format_agg_value's default case, which
emits the row count instead of the function's actual value.
Swap the five call sites (ret_agg_build_key, ret_agg_emit_row,
with_agg_build_key, with_agg_find_or_create, with_agg_accumulate) to
is_aggregate_func(), the predicate already used correctly one level up
to decide whether a query needs aggregation at all. Group-key columns
carrying a non-aggregate function now project through the existing
project_item() helper instead of binding_get_virtual(), since they
need to evaluate the function rather than read a raw property.
Fixes#1111 (the type(r)/count(*) half; the inline-property-map half
of that issue does not reproduce on current HEAD)
Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
Round-2 overhaul of the tool-output surface, built eval-first (byte harness +
no-context field-eval agents, always vs the v0.9.0 release binary):
Format — tree syntax everywhere:
- compact_out emits tree tables: 'key: N (cols: a b c)' headers, indented
space-delimited rows, '-' placeholders, space-aware quoting. Every table
in every tool converts mechanically.
- Grouped prefix-factored output is the DEFAULT for search_graph and
trace_path: rows grouped under their shared (qn-prefix, file) printed
once, row qn = group prefix + '.' + name (stated in-band). Measured:
192-hit search 2,929B (was 4,811 TOON / 238,466 v0.9.0), 82-row trace
2,256B (was 4,234 / 8,094). Field eval: 5/5 answers correct, QN
reconstruction verified, no comprehension loss.
- detail:'ids' tier: bare-qn enumeration for wide sweeps (-44%).
Pagination — exactly-once, stateless:
- store_meta generation (per-DB random uid + mutation counter bumped in
cbm_store_upsert_project — every index run's choke point): cursors can
never silently validate against a rebuilt DB.
- trace cursors: opaque token (leg, generation, params-hash, (hop,id)
watermark); pages drain callees then callers with exact
callees_total/callers_total on every page; teaching errors for
params-mismatch and staleness. Guard: 12 callees paged 5+5+2, every row
on exactly one page.
- deterministic ordering: BFS ORDER BY hop,id; canonical (hop,id) sort +
min-hop dedup across same-name seeds (hop fed risk_labels wrong before);
search (name,id)/(rank,id) tie-breaks make offset pages stable.
Lean defaults + teaching errors:
- get_graph_schema no longer advertises blocked internal fields (fp/sp/bt).
- index_repository coverage lists: counts + 5 examples + one-line pointer
notes (full record was always in the logfile); 6.0KB -> 1.5KB here,
53KB observed on a large repo. index_status gates the git context block
behind verbose:true.
- trace limit param (<=5000) + truncated:true — never a silent truncation;
invalid direction now errors with the valid values; requesting core
columns via fields hints instead of emitting empty cells; cypher
resolves f.file/f.qn/f.path aliases.
Trace-correctness fixes (reproduce-first, RED verified):
- bfs_collect_edges 4KB id-string overflow: past ~1000 visited nodes the
edge SQL failed to prepare and EVERY trace edge silently vanished; now a
per-connection temp-table join (star guard: 1200/1200 edges).
- bfs_union_same_name: min-hop across seeds + edge dedup.
Guards: gating tool-output regression suite (context-explosion detector:
blocked-field leaks, repeated-key envelopes, oversized notes + calibrated
byte ceilings + semantic floors). Full suite 6,367 passed / 0 failed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
An unbounded whole-graph OPTIONAL MATCH or a GROUP BY that produces roughly one
group per node does O(bindings x groups) work in execute_return_agg and can run
for minutes on a large graph before emitting a single row. The 100k row ceiling
never fires (no rows are produced), so query_graph just hangs with no partial
result and no error (#601).
Arm a monotonic wall-clock budget (default 30s) at query entry and check it
(throttled, every 1024 iterations) in the scan, relationship-expansion and
aggregation hot loops. On expiry the query aborts with a clear, actionable error
instead of hanging. A thread-local test hook forces the budget so the guard is
covered by a deterministic reproduce-first test; a companion test confirms the
default budget does not false-positive on a normal query.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
json_extract_prop() scanned a non-string property up to the first ',' and a
string property up to the first '"', ignoring nesting and backslash escapes.
Any array/object property was therefore truncated at its first INTERNAL comma,
and any string containing an escaped quote was cut short.
decorators: ["@Roles('OWNER', 'ADMIN')","@Get()"]
projected as: ["@Roles('OWNER'
This makes decorator/route/authz queries unusable on frameworks whose
decorators take multiple arguments (NestJS, Angular, Spring). Values are now
copied as balanced constructs, honoring string state and escape pairs; scalar
and plain-string paths are unchanged.
Signed-off-by: KolisCode <jhohantma@gmail.com>
The compat.h include added for CBM_TLS pulled windows.h into cypher.c,
whose legacy 'far' macro (windef.h defines it empty) broke the
existing EXISTS-pattern local of that name on every Windows leg. Use
C11 _Thread_local for the depth-clamp flag instead; cypher.c stays
free of windows.h.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three of #797's four findings fixed (the fourth — repeated-variable
unification — already holds on main and is now pinned by the test):
- Walk padding: the BFS recursive CTE deduped (node, hop) PAIRS, so a
single self-loop minted every hop level for every node it could
reach — *k..k matched fabricated walks of arbitrary length (a
100-hop 'longest chain' artifact) and the row set exploded to
nodes x depth (the reported crash). The traversal now returns each
node once at its MINIMAL hop (MIN/GROUP BY): variable-length matches
nodes whose shortest path lies in [min,max]. This is deliberately
shortest-path semantics, not openCypher trail enumeration — right
for reachability/depth audits on call graphs and linear instead of
exponential; trace_path shares the traversal and now also reports
each node once on cyclic graphs.
- Silent 100-hop cap: a clamped hop range now surfaces as
result->warning ('clamped ... an empty result may mean clamped, not
no such path'), emitted by query_graph in both TOON and JSON forms,
on top of the existing log WARN.
Reproduce-first: cypher_exec_varlength_path_semantics_issue797 —
self-loop + real 2-chain fixture; RED on the walk semantics (*2..2
returned 3 endpoints instead of 1), GREEN now, incl. variable
unification, zero rows for nonexistent path lengths, and the clamp
warning assertion.
Closes#797
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The WHERE parser rejected coalesce() outright ('unexpected operator'),
blocking null-safe numeric filters over optional graph properties —
the exact audit-query shape from the report:
MATCH (f:Function) WHERE coalesce(f.transitive_loop_depth, 0) >= 2
RETURN-side coalesce already worked; only the WHERE leaf lacked it.
Implementation: the condition leaf gains a coalesce_default literal.
The operator/value tail of condition parsing is factored into a shared
parse_condition_op so both subjects (var[.prop] and
coalesce(var.prop, literal)) accept the full operator set (comparisons,
IS [NOT] NULL, IN). At eval time a missing/empty property value is
substituted with the default before the operator runs. Supported form
is coalesce(var.prop, literal) — matching openCypher's two-argument
null-substitution for this subset.
Reproduce-first: cypher_exec_where_coalesce_issue874 runs the
reporter's exact query — RED (rc=-1 parse error) on the old parser —
and pins both directions: a failing default excludes property-less
nodes, a passing default includes them.
Closes#874
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The result projection is materialized per row into fixed-width stack
arrays (CBM_SZ_32 columns) in execute_return_simple and its siblings,
indexed by the parsed RETURN item count — which the parser left
unbounded. A RETURN/WITH clause with more than 32 projection items
wrote past those arrays. Reject an over-wide projection at parse time,
with a fork-isolated regression test driving a 48-column RETURN.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel