Commit Graph
1334 Commits
Author SHA1 Message Date
Martin Vogel 44fa3aeb21 Merge pull request #2026 from DeusData/fix/windows-guard-daemon-isolation
test(windows): isolate the daemon guards' rendezvous and stop hiding their failures
2026-09-04 12:25:34 +02:00
Martin Vogel 3bbabf187c Merge pull request #1811 from pcristin/fix/issue-1682-dotted-ts-imports
fix(pipeline): preserve dotted TypeScript import basenames
2026-09-04 09:45:57 +02:00
Martin Vogel 9ed01a478c Merge pull request #2024 from DeusData/feat/cshtml-razor-pages
feat(extract): discover Razor Pages / MVC views (.cshtml) and route their @page
2026-09-04 09:45:53 +02:00
Martin Vogel 91a1723175 Merge pull request #1897 from CaptainMittens/fix/resolve-receiver-chain-mismatch
fix(registry): refuse a name-only match the receiver chain contradicts
2026-09-04 09:45:49 +02:00
Martin Vogel f47cd47afe Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-09-03 21:58:07 +02:00
Martin Vogel 380b1c5af1 Merge branch 'main' into feat/cshtml-razor-pages 2026-09-03 21:58:04 +02:00
Martin Vogel 429141261d Merge branch 'main' into fix/resolve-receiver-chain-mismatch 2026-09-03 21:57:58 +02:00
Martin Vogel 4d4af28672 Merge branch 'main' into fix/span-duration-unreadable-timestamps 2026-09-03 21:57:55 +02:00
Martin Vogel 5802ccdd77 Merge pull request #1896 from CaptainMittens/fix/swift-call-arguments
fix(extraction): reach Swift call arguments through call_suffix
2026-09-03 15:12:27 +02:00
Martin Vogel 73f4a432e4 test(windows): isolate the daemon guards' rendezvous and stop hiding their failures
Every Windows guard shared ONE daemon. test_daemon_stability.py said each
section "runs under its OWN cache directory: daemon coordination is
cache-scoped" -- it is not. The rendezvous key is FNV-1a over a compile-time
domain string (service.c: cbm_daemon_rendezvous_key) under a machine-global
parent (win_default_runtime_parent -> SHGetFolderPathW(CSIDL_LOCAL_APPDATA),
which ignores the LOCALAPPDATA variable). So the guard's own `daemon stop`
and kill -9 sections, the sibling guards, and anything else on the runner
all met the same daemon, and a daemon wedged anywhere failed every check
after it. That is the shape of this week's test-windows-guards red on
#1874: lifecycle's `daemon start` connected, the UI-config handshake never
answered, and stability / hook_augment / ui_drive / cli_non_ascii went red
in a row with an orphan daemon killed at cleanup -- on a diff and a base that
were each green elsewhere.

CBM_RUNTIME_DIR is the only relocation hook (bootstrap.c, resolved in
cbm_daemon_bootstrap_endpoint_new for every product endpoint). Each stability
section now gets a private runtime parent next to its cache, created before
the binary runs -- the endpoint validates the parent and never creates it --
and the daemon inherits it from the client that spawns it. Proven on macOS:
a daemon on the default rendezvous keeps its pid across the whole guard
(three stops and a kill -9 inside it).

The reporting was hiding the cause as well:
* every SETUP FAIL printed nothing at all, and every RED printed a 300-400
  character HEAD slice, which is the allocator preamble, never the error.
  Excerpts now show the tail (1500 chars); setup paths print the daemon's
  full output.
* a subprocess timeout raised on a parallel-client worker thread was lost
  and the client reported as "rc=none" with no output; it is now kept as a
  failed result with its partial output. A section-level timeout is a
  named RED with the command and output, not a bare traceback.
* the MCP session's stderr went to DEVNULL; it now goes to a file that is
  printed when initialize never answers -- and that case is a RED against a
  running daemon, not a "setup" skip.
* exit 2 was documented and never produced. Only `daemon start` failing
  raises it now (test_daemon_lifecycle.py owns that surface as a
  regression, so the board loses nothing); main() runs every section and
  names all of the failing ones.
* test_hook_augment.py / test_cli_non_ascii_arg.py print full stdout AND
  stderr on their index-setup failures instead of 300 chars of stdout.

On POSIX (local iteration only) the namespace lives under /tmp: sun_path
caps the socket at 104 bytes on macOS and TMPDIR there is ~50 bytes deep.

Out of scope, noted: test_daemon_lifecycle.py, test_non_ascii_path.py and
test_ui_drive_listing.py still use the machine-global rendezvous; a too-long
CBM_RUNTIME_DIR on POSIX is refused (ipc.c sun_path check) with no
validation detail.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-03 14:56:42 +02:00
Pcristin b64e217000 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-09-03 14:52:02 +03:00
Martin Vogel bac8edc58d Merge pull request #2019 from CaptainMittens/test/cli-registry-plan-machine-independent
test(cli): stop the registry plan check depending on the developer's machine
2026-09-03 13:51:07 +02:00
Pcristin bbecc9b9e9 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-09-03 13:44:15 +03:00
Martin Vogel f37a47eb01 Merge pull request #1871 from LazyXuan/fix/query-tools-readonly-annotations
fix(mcp): non-mutating query-only store resolve; annotate the ten read-only tools
2026-09-03 11:46:06 +02:00
Martin Vogel 1770d68a2c feat(extract): discover Razor Pages / MVC views (.cshtml) and route their @page
Follow-up to #1824, which mapped Blazor `.razor` to the C# host and extracted
`@page` routes from it. `.cshtml` is the same Razor syntax with the same C#
host and was equally unmapped: an ASP.NET Core MVC or Razor Pages application
produced no nodes at all for its entire view tree.

`@page` matters more here than it does for components. On a Blazor component
the directive is an optional annotation; in Razor Pages it is the directive
that *defines* a page, so the routable surface of a Razor Pages app lives
exclusively in this file type.

That is why the extension mapping alone is not enough. `cbm_path_is_razor()`
hardcoded a single `.razor` suffix, so a `.cshtml` file would have been
discovered and parsed but skipped by the directive scan -- present in the
graph, with none of its routes. Widened to a suffix table covering both.

Scope is deliberately the Razor family and nothing else:

  - `.vbhtml` is excluded because there is no VB.NET language in this repo,
    so there is no host to map it to.
  - `.aspx` / `.ascx` are excluded because Web Forms is a different templating
    syntax (`<%@ %>`, `runat="server"`) with no `@page` directive, so neither
    the C# error recovery nor the directive scan applies.

Best-effort on the same terms as `.razor`: the C# grammar recovers the `@{ }`
and `@functions` blocks, and the surrounding markup lands in ERROR regions
reported through parse_partial.

Tests: the extension mapping; a Razor Page asserting the extracted route path
and method; and a negative case for a layout with no `@page`, since most
`.cshtml` files are layouts and partials and a spurious Route node per view
would be worse than the missing ones this fixes.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-03 11:42:52 +02:00
Pcristin e79a7d73b4 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-09-03 12:36:08 +03:00
Martin Vogel 6cca768816 Merge pull request #1773 from rarepops/fix/dockerfile-trailing-blank-eof-1746
fix(extraction): treat trailing blanks at EOF as an absent newline, not a miss (#1746)
2026-09-03 11:26:47 +02:00
Pcristin da09b73bc0 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-09-03 11:20:48 +03:00
Martin Vogel cc3e2638da Merge pull request #1912 from DeusData/feat/python-bare-local-binding
fix(python): suppress bare calls shadowed by an enclosing parameter
2026-09-03 09:39:46 +02:00
周文瑄 adb49b3a28 test(mcp): snapshot the query-branch DB state before suite teardown
CI's tsan and lsan-macos runs failed the mutation-guard suite on
ASSERT(cbm_file_exists(db_path)) at the guard-balance test, while the
behavior suite passed everywhere. The assertion ran after the test's
own teardown: cleanup_project_db() unlinks the same db path before the
assertion block, and the manage_adr branch above replants and
quarantines that very path, so the late file check observed the
test's cleanup, not the query-only resolve it was meant to pin.
Whether it tripped depended on unlink/visibility timing, which is why
it surfaced only under some schedulers and never in the single-suite
local runs.

Capture cbm_file_exists() into query_db_left_in_place right after the
query branch completes, before anything replants or removes the file,
and assert on that snapshot.

Signed-off-by: 周文瑄 <zhouwx1997@126.com>
2026-09-03 15:33:57 +08:00
周文瑄 93d2ee59ea fix(mcp): non-mutating query-only store resolve; annotate the ten read-only tools
search_graph, query_graph, trace_path, get_code_snippet, get_graph_schema,
get_architecture, search_code, index_status, check_index_coverage and
detect_changes were annotated readOnlyHint=false / destructiveHint=true
because resolve_store()'s corrupt-store recovery quarantines database
files - a mutation. That conservatism locked every plan-mode client out
of the read-only analysis surface (#1100 calls the same tools "eleven
read-only analysis tools"), since spec-compliant plan gates refuse tools
that are not declared read-only.

This closes the gap the annotation comment pointed at. The query-only
resolve now classifies a failed integrity check without mutating: no
mutation lease, no quarantine. A confirmed-corrupt database is reported
(a new STORE_RECOVERY_CORRUPT status; the tool reply names the corruption
and points at index_repository instead of a misleading "project not
found") and left in place for a write-side open to quarantine and
rebuild, exactly as before. Transient verdicts stay retryable. Write-side
opens (index_repository, manage_adr updates) keep the existing blocking
recovery, quarantine, and generation re-check semantics unchanged.

The ten query tools' annotations flip to readOnlyHint=true /
destructiveHint=false, making them callable under plan mode in Claude
Code, Codex, ZCode, and every other client that gates on the hints.

Tests: the annotations expectation table is flipped; the query branch of
the guard-balance and guard-denial tests now assert zero leases taken and
byte-identical files; the ghost-db test (#704) expects the corruption
message; the four write-side recovery tests (generation re-check after
lease wait, unique backup naming, publish failure, published-WAL
completeness) are driven through manage_adr update - the write path where
that recovery now lives.

Signed-off-by: 周文瑄 <zhouwx1997@126.com>
2026-09-03 15:33:57 +08:00
Martin Vogel db2f104a1f Merge pull request #2013 from shafty023/fix/1997-extract-scratch-arena
fix(extract): AST traversal stacks no longer outlive the file they walk
2026-09-03 08:14:16 +02:00
Martin Vogel 0e9ed75991 Merge pull request #2016 from rarepops/feat/ui-server-version
feat(ui): show serving binary version (#1820)
2026-09-03 08:13:57 +02:00
Joshua RichterandClaude Opus 5 ad04339030 test(cli): stop the registry plan check depending on the developer's machine
The test built a fixture with its own HOME and PATH, then asserted that
the whole install plan contained no path with "/plugins/" in it. Agent
detection does not stay inside HOME and PATH: cbm_find_cli also looks in
/usr/local/bin and, on macOS, /opt/homebrew/bin. A developer with OpenCode
installed there has it detected inside the fixture, and OpenCode ships a
real plugin file, so the plan legitimately carried a "/plugins/" path and
the assertion failed. On a machine without OpenCode the same assertion
passed, which is why CI never saw it.

Name the two directories the check actually meant — the fixture's Qoder
and Pi directories, neither of which has a plugin directory, so a planned
path under one would be invented. The check now tests the same thing
regardless of what the developer has installed.

No production behaviour changed. Searching well-known install directories
is deliberate, so that install finds an agent whose command is not on the
current PATH.

Fixes the failure at tests/test_cli.c:6859.
Local cli suite: 293 passed, 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
2026-09-02 22:20:54 -04:00
Joshua Richter 2c76563a13 fix(registry): refuse a name-only match the receiver chain contradicts
A dotted callee whose first segment starts upper-case names a type. That
receiver chain is evidence, and the two name-only strategies threw it away:
they matched on the final segment alone, so Foundation's
URLSession.shared.data bound to a project's own PickedFile.data — at 0.75
confidence, with nothing in the graph to show the answer was wrong.

receiver_chain_admits() now requires the candidate's own parent segment to
appear somewhere in that chain. Calendar.utcGregorian.startOfDayUTC still
resolves to AuthDTOs.Calendar.startOfDayUTC, because the project really does
extend Calendar and Calendar is in the chain.

Three shapes pass through untouched:

  - a callee with no separator, which has no chain to judge;
  - a lower-case root, which names a value whose declared type the chain does
    not show (vm.load, http.Get, os.path.join);
  - a name in capitals with underscores, which is a constant holding a value
    rather than a type. Measured: without this carve-out the gate refused
    ISO_4217_URL.lower -> builtins.str.lower, which is correct. JSON and URL
    carry no underscore and stay guarded.

The gate applies only at the two name-only exits of resolve_name_lookup.
import_map, same_module and qualified_suffix already carry real evidence and
are left alone.

Language agnostic by design: the registry holds no language, and every
language that writes receiver chains gains the same protection.

Fixes #1893

Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
2026-09-02 21:55:31 -04:00
Joshua RichterandClaude Opus 5 57ef0a6d86 fix(traces): stop an unreadable span timestamp from becoming a real duration
strtoll answers 0 for text it cannot read, and cbm_parse_duration handed that
0 straight into its subtraction. The report of this said the result was a
false duration of zero. It is worse than that. An unreadable START time reads
as 0, so the span reports the whole END time as its duration:

  FAIL tests/test_traces.c:340: info.duration_ns == 1050000000,
    expected CBM_DURATION_UNKNOWN == -1

That is 1.05 seconds of measured time for a span whose start was never read.
Nothing downstream can tell that number from a real one.

The fix keeps cbm_parse_duration exactly as it is. It is public, it is
declared in traces.h, and eight tests pin its answers -- 0 for a NULL
argument, 0 for an end at or before the start. None of that moves.

A companion carries the extra answer instead:

  int64_t cbm_parse_duration_checked(const char *start, const char *end, bool *ok);

Both reads go through one small reader that follows src/main.c:1104 -- an end
pointer, errno, and a check that nothing was left over -- plus a refusal of a
leading blank, which strtoll would otherwise step over.

cbm_extract_http_info now uses the companion and writes CBM_DURATION_UNKNOWN
(-1) when the timestamps do not read. It still returns true, because the
method and path on that span are still good data and dropping them would
punish them for a fault they had no part in. -1 as "not recorded" is the
sentinel this codebase already uses for the same question, in
CBM_EDGE_CONF_ABSENT. The duration_ns field comment in traces.h now says so,
so a reader who meets -1 has something to read.

Scope note: cbm_extract_http_info has no production caller on this tree. rg
finds it only in src/traces/traces.c, src/traces/traces.h and tests, and
handle_ingest_traces in src/mcp/mcp.c never touches a timestamp. So this is a
defect in a public, tested function rather than one putting bad rows in a
graph today.

Two tests come with the change. The one that pins the behaviour was seen
failing first -- the FAIL line above is from that run. After the fix,
TEST_SUITES="traces" reports 32 passed, 0 failed, exit 0.

The full suite reports 7633 passed, 2 failed. Both failures are in
tests/test_cli.c (lines 1749 and 6725), print "error: one or more agent
cleanup operations failed", and reproduce on a clean tree without this
change. make -f Makefile.cbm lint-ci passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
2026-09-02 21:55:30 -04:00
Joshua Richter c4e902849f fix(extraction): reach Swift call arguments through call_suffix
handle_calls reads a call's arguments through one tree-sitter field
lookup, ts_node_child_by_field_name(node, "arguments"). The vendored
Swift grammar declares no "arguments" field at all -- its own
ts_field_names[] table has zero occurrences, against one in Go and two
in TypeScript. Swift models a call as a target expression plus a
call_suffix, and the arguments hang off the suffix as value_arguments.

So args was null for every Swift call ever parsed, first_string_arg was
never populated, and no Swift HTTP call could raise an HTTP_CALLS edge
or a Route node. Alamofire, Moya and URLSession have been in the
service-pattern table the whole time and match the callee text
correctly; the URL simply never arrived.

Three changes, all in extract_calls.c:

- swift_call_args() reaches the argument list through call_suffix, in
  the same shape as the existing objectscript_call_args() fallback and
  used from the same place. A trailing closure has no value_arguments,
  so it returns a null node and that call behaves as before.
- extract_url_or_topic_arg() unwraps Swift's per-argument
  value_argument node, stepping past a leading value_argument_label.
  Without it, dataTask(with: "/api/v1/widgets") yields the label "with"
  rather than the path. PHP and C# already had the same unwrap for
  their own "argument" node.
- is_string_like() gains line_string_literal, which is what Swift calls
  an ordinary "..." literal. The list already held raw_string_literal,
  so only Swift's common case was missing.

This is the layer underneath #1892 rather than the whole of it. A
literal URL argument now arrives; a literal nested inside a
constructor, as in URLSession.shared.data(from: URL(string: "...")!),
still does not, because extract_positional_url reads a literal, a
template string, a concatenation or a named constant and that shape is
none of them.

Reproduce-first: all three tests fail without the fix, two on a null
first_string_arg and one on HTTP_CALLS being 0.

Fixes #1892

Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
2026-09-02 20:26:17 -04:00
Martin Vogel c38aa35c27 Merge pull request #1777 from ocochard/feature/freebsd-pkg-managed-detection
fix(freebsd): treat ${LOCALBASE}/bin as pkg-managed; resolve self via sysctl
2026-09-03 00:07:06 +02:00
Martin Vogel 192f29609a Merge pull request #2006 from ilyabrykau-orca/fix/go-field-ref-member-signal
fix(pipeline): key the Go Field guard on the recorded selector shape, not reference text
2026-09-02 23:59:55 +02:00
Martin Vogel 01bf5f23e3 Merge pull request #2009 from CaptainMittens/fix/route-handler-last-argument
fix(routes): take a route's handler from the last argument, not the first
2026-09-02 23:53:32 +02:00
Martin Vogel e6a34fcea0 Merge pull request #1824 from The1nk/feat/blazor-razor-routes
feat(extract): discover Blazor .razor components and emit Route nodes for @page
2026-09-02 23:48:14 +02:00
Martin Vogel 2a0e9b9b80 Merge pull request #1876 from ertankucukoglu/fix/windows-unicode-session-context
Fix Unicode daemon session roots on Windows
2026-09-02 23:06:32 +02:00
Martin Vogel 7910de591b Merge pull request #1877 from CaptainMittens/fix/confidence-parse-sentinel
fix(confidence): stop an unreadable confidence from reading as 0.0
2026-09-02 23:04:20 +02:00
Martin Vogel 4bae9bd67f Merge pull request #1875 from CaptainMittens/fix/cypher-reject-trailing-tokens
fix(cypher): refuse a query the parser did not fully read
2026-09-02 23:04:04 +02:00
Martin Vogel 9639803175 Merge pull request #1836 from ertankucukoglu/perf/search-code-scope-suffix
perf(mcp): filter suffix globs while building search scope
2026-09-02 23:03:46 +02:00
Martin Vogel 620613ede6 Merge pull request #1986 from CaptainMittens/fix/swift-scanner-try-bang-shift
fix(grammar): stop the Swift try-bang entry shifting past its own type
2026-09-02 22:51:46 +02:00
Martin Vogel aefd2a1838 Merge pull request #2008 from CaptainMittens/fix/decorator-arg-scan-limit
fix(routes): find a route path wherever it sits in a decorator's arguments
2026-09-02 22:19:03 +02:00
Rares Popa e0785b353a feat(ui): show serving binary version
Expose the compiled server version through the existing UI config endpoint and render it beside the product name. Keep older or unavailable config responses compatible by hiding the label, with focused frontend and native endpoint coverage.

Fixes #1820

Signed-off-by: Rares Popa <2606875+rarepops@users.noreply.github.com>
2026-09-02 20:50:31 +02:00
Martin Vogel 4d8ac9e1ce Merge pull request #1802 from rarepops/fix/install-clients-registry-1798
fix(cli): apply client selector to registry integrations (#1798)
2026-09-02 19:22:45 +02:00
Martin Vogel 1424934980 Merge pull request #1799 from rarepops/fix/nanosleep-eintr-1616
fix(foundation): resume spawn backoff after EINTR (#1616)
2026-09-02 19:12:10 +02:00
Pcristin 3f08fac513 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-09-02 20:04:12 +03:00
Rares Popa 572725e61a fix(extraction): treat trailing blanks at EOF as an absent newline, not a miss
A file whose final line lacks its newline leaves the grammar's mandatory
terminator MISSING at EOF. #1610 suppressed that phantom parse_partial on the
grounds that the node is ZERO-WIDTH: the parser consumed no source for it, so
by construction nothing was dropped.

That suppression tested `end == source_len` exactly. Trailing blanks are extras
owned by no node, so `ENTRYPOINT ["a"] ` + EOF parks the zero-width terminator
one byte short of source_len and the check missed it. This is why neither a
trailing blank nor an absent final newline flagged on its own -- only the pair
did, exactly as the reporter's byte-exact control matrix showed.

Treat "at EOF" as EOF modulo a trailing blank run. Every blank except newline
qualifies, deliberately not a hand-picked subset: a form feed is no more
content than a space, and #1610 exists precisely because whether a file got
flagged used to hinge on such incidentals. Newline stays excluded because a
terminated final line produces no MISSING terminator at all.

The rule remains ZERO-WIDTH ONLY. A width-bearing MISSING/ERROR at EOF is a
genuine loss and is still flagged (a Makefile's unterminated final recipe
really does vanish), as is any failure earlier in the file; both are pinned by
guard tests alongside the existing #1610 guards.

Verified end to end: the issue's own `cli index_repository` reproduction goes
from parse_partial_count 1 to 0 with node count unchanged, so the phantom flag
is gone without anything dropping out of the graph.

Fixes #1746

Signed-off-by: Rares Popa <2606875+rarepops@users.noreply.github.com>
2026-09-02 18:42:40 +02:00
Daniel Ochoa 40f2722d61 fix(extract): AST traversal stacks no longer outlive the file they walk
Fixes #2010. Refs #1997.

Indexing a 14k-file TypeScript repository holds 13.3 GB resident at
post_extract on Linux aarch64, and about 3.1 GB of it is AST traversal
scratch that no CBMFileResult points at.

ts_nstack_init cut its stacks from ctx->arena, which is result->arena. In
the parallel path that arena is stored into ec->result_cache[file_idx]
(pass_parallel.c:963) and freed only after parallel_resolve and the infra
passes (pipeline.c:1324), so a structure written for a one-file lifetime is
in fact held for the whole index, once per file, concurrently. The header
claimed otherwise and was wrong: "freed when the arena is destroyed at end
of file extraction".

The largest instance is channels. CHAN_STACK_CAP is 4096 and sizeof(TSNode)
is 32, so scan_string_consts_js (extract_channels.c:104) and
extract_channels_js (:376) take 128 KB each, per JavaScript or TypeScript
file. Across 12,673 such files that is 3,168 MB, and it produced 23 Channel
nodes out of 164,684.

CBMExtractCtx gains a scratch arena, created and destroyed by the
cbm_extract_file_ex call that builds the context. Nothing else changes about
where anything lives: ctx->arena still owns every string a CBMFileResult
points at, and ctx->scratch owns only traversal stacks, which nothing points
at.

Nothing a CBMFileResult points at moved. A context built without a scratch,
and a file whose scratch arena fails to allocate, both fall back to
ctx->arena, which is exactly the behaviour that shipped before this change,
so no path can be handed a NULL arena and silently lose nodes.

The signature change is what makes the mistake unrepresentable.
ts_nstack_init takes the context rather than an arena, so handing it
ctx->arena is an incompatible pointer type; push and push_children read the
arena the stack recorded, so a stale argument is an arity error. Both fail
under -Werror rather than silently retaining memory.

The 55 init sites are NOT one uniform sed. 29 pass ctx->arena and 26 pass a
local alias, and that alias is shared with result allocations: in
parse_zig_imports (extract_imports.c:1093) the same `a` feeds cbm_node_text,
strip_quotes, path_last and cbm_imports_push (extract_imports.c:1101, 1106,
1108, 1109). Rebinding it would have moved import names and module paths
into memory that dies with the call while the result kept pointing at them.
Each site was edited on its own and no alias was rebound. Re-derive the set
with:

  grep -rn 'ts_nstack_init' internal/cbm/extract_*.c
  grep -rn 'ts_nstack_push' internal/cbm/extract_*.c

parse_lisp_imports used its alias only for the stack, so the alias goes too.
push_nested_class_nodes and push_class_body_children took an arena purely to
reach the stack, so they take the context now; the one call site passes ctx.

Two measurements, from two different builds, and they are not the same
number.

A macOS census build, which instruments every cbm_arena_alloc and sums the
blocks behind every live CBMFileResult arena, reports the retained per-file
arena set dropping from 11,226 MB mapped to 5,680 MB, and blocks from 48,045
to 34,186. Those are mapped bytes, not resident: that build logs
mem.allocator.bound_populations_only, so untouched block tails never commit.

The resident figures come from Linux aarch64, gcc 13.3, static build, where
the build logs mem.allocator.owned classes=all and mapped equals resident.
Five runs per side on the full 13,999-file tree: mem.collect post_extract
rss_mb 13293 -> 6752 (-49.2%), mem.phase peak_mb 14840 -> 8565 (-42.3%).
nodes 164684, parse_partial 280 and registry entries/defines/imports
787954/804999/100027 identical on all ten runs; edges 835869-835874 against
835867-835881, overlapping ranges, so run to run noise. Wall clock means
35651 ms against 30502 ms, but the ranges overlap and the host had other
load, so the only supportable claim is no regression. ObjectScript Studio
Export on a small fixture: nodes 18, edges 25, both sides.

Scratch block size is 512 KB, chosen from measurement. arena_grow fires on
one file in 12,000 at both 512 KB and 1 MB, and constantly at 256 KB, where
the two channel stacks alone are exactly 262144 bytes. 512 KB therefore
costs half the resident block per worker for the same growth behaviour, and
stays under MI_LARGE_MAX_OBJ_SIZE in the vendored mimalloc, which is
MI_LARGE_PAGE_SIZE/8, exactly 524288 bytes (types.h:426, with
MI_ENABLE_LARGE_PAGES defaulting to 1 at :115 and not overridden by this
build), so a 512 KB block is still bin-allocated from a 4 MiB large page.
Growth is not free at that size for the same reason: arena_grow doubles to 1
MiB, above that bound and therefore a singleton OS allocation. One file in
twelve thousand pays it, which is why it is accepted. The largest single
file's scratch high water mark on this corpus is 33.5 MB, now transient
rather than retained.

Tests. Nothing in the suite asserted on result->arena size, block count or
total_alloc after cbm_extract_file, which is why this went unnoticed for the
life of extract_node_stack.h. traversal_stack_not_in_result_arena_issue2010
pins the byte budget: extracting "export const x = 1;" as TypeScript charged
365984 bytes to the result arena before and charges 87456 after, a
difference of 278528, exactly the two 4096-entry channel walks plus the
512-entry ES import walk at 32 bytes per TSNode. Of the 87456 that remain,
7680 is the defs item array at GROW_ARRAY's starting capacity of 32 times
sizeof(CBMDefinition) 240 and the other 79776 is everything else the
extraction interns, none of it traversal scratch; the 128 KB bound is
therefore above the new figure with room and a factor of four below the old
one. extract_traversal_stacks_come_from_ctx_scratch_issue2010 in test_mem.c
pins the lifetime, since a byte budget alone would also be satisfied by
shrinking CHAN_STACK_CAP: it builds the extraction context directly over two
arenas it owns, runs cbm_extract_channels, and asserts the scratch took at
least the 262144 bytes of the two walks while the result arena did not.
extract_c_macro_hidden_call_survives_preprocessed_pass_issue2010 is not a
scratch test. pp_ctx carries ctx->scratch so every context in the file is
uniform, but nothing reads it there: pp_ctx reaches only cbm_extract_unified
and cbm_run_c_lsp, and neither extract_unified.c nor anything under
internal/cbm/lsp/ includes extract_node_stack.h, so no traversal stack is
built on that path today. The test guards the macro-expansion path itself,
which had no assertion on a call that exists only after expansion.
ObjectScript composite extraction is already covered by
pipeline_objectscript_export_preserves_calls_sequential_parallel and
iris_export_xml_multi_class, so no new test is added there; a per-call arena
is correct on that path by construction, since each call owns and destroys
its own.

Local: scripts/test.sh, 7558 passed, 0 failed, 8 skipped, 139 suites.

Alternatives considered

Thread-local scratch arena, reset per file, with a depth guard for nesting.
Rejected on three counts. This codebase never uses thread-local storage for
owned bulk memory; every existing _Thread_local here is a small cache, a
guard or a log buffer, and every per-worker resource is a field in a context
struct. It would have needed cbm_arena_reset, which has no production caller
and a documented subtlety with grown block sizes (arena.c:213-217). And
extraction threads are created per cbm_parallel_for call and joined
(worker_pool.c:71,88) with no thread-exit hook in this project's own code,
so the arena would have leaked once per thread per dispatch and turned the
leak lane red.

Save and restore mark and release on the arena. Rejected because a per-call
arena already makes nesting correct by construction, and a mark type plus
two functions would have to be added to both arena headers.

Per-walk arenas, one per traversal rather than one per file. Rejected on
cost: 55 sites would each need an explicit span, for the same bound.

Right-sizing the initial capacities alone. CHAN_STACK_CAP at 256 rather than
4096 would cut those two sites from 3,168 MB to about 198 MB, which is real,
but it tunes the constant while the lifetime defect remains at all 55 sites,
and growth is not free: 256 doubling to 4096 abandons 122,880 bytes, and one
file in this corpus took 33.5 MB of stacks. The cap and the scratch block
size are coupled, which is a further reason to change one at a time. Worth
doing separately.

Routing to a scratch inside extract_node_stack.h while keeping the arena
parameters. Rejected because a parameter that is ignored lies about the API,
and the alias hazard above would stay a review obligation instead of a
compile error.

Freeing the per-file result arena earlier. Rejected: the results are read by
parallel_resolve and the infra passes. That is the retention half #1925
addresses, and this change is independent of it.

Signed-off-by: Daniel Ochoa <daniel.ochoa@closedloop.ai>
2026-09-02 09:22:37 -05:00
Martin Vogel 7fedaba331 Merge branch 'main' into feature/freebsd-pkg-managed-detection 2026-09-02 11:45:45 +02:00
Martin Vogel 97517a4652 fix(python): carry bare-call parameter bindings in the walk
Replaces the cursor ascent from 0b0d143c. That commit fixed a real hang --
ts_node_parent() is O(depth), so a parent chain is O(depth^2) per call and
O(depth^3) across a file where every level of f(f(f(...))) is itself a bare
call, which hung stack_overflow_b's 30,000-deep fixture at 900s. But it fixed
it by making each hop cheap rather than by removing the per-call walk, so the
cost stayed O(depth) per call and needed a 64-ancestor cap to stay bounded.

That cap FAILS OPEN. Past 64 ancestors the guard stopped suppressing, so
deep-but-ordinary code silently kept the fabricated edges the guard exists to
remove -- and nothing failed when it did.

Carry the answer instead. The unified walk binds a def's or lambda's parameters
when it opens that scope and unwinds them when it closes, so the guard is an
O(1) map lookup. This is the lesson CBMWalkScope already records one struct
away: carry walk state, never recompute it per node.

  - CBMParamSlot is a name -> active-count map, not a set. `def outer(run):
    def inner(run):` binds one name twice and leaving the inner scope must not
    unbind the outer.
  - Bindings are pushed at the end of push_boundary_scopes, after every scope
    push for that node, so they land in the frame just pushed -- covering
    function_definition (push_function_scope) and lambda (push_lexical_boundary)
    alike -- and pop_expired_scopes unwinds to the frame's entry height.
  - Allocation failure sets py_param_tracking_failed and the lookup answers
    false forever after: a lost suppression, never a lost edge.

The repo's other binding table cannot serve this. CBMLexicalBinding is qsort-ed
in cbm_finalize_lexical_usages AFTER the walk and its active_end stays 0 until
then, so its binary search is invalid from handle_calls, which runs mid-walk
and before handle_usages.

extract_python_bare_call_scope_walk_is_bounded pinned the cap's fail-open
contract, which no longer exists, and is replaced by
extract_python_bare_call_flag_is_depth_independent. It keeps the shallow and
200-ancestor cases -- the deep one now asserts the call IS flagged, so
reintroducing a cap fails it -- and adds an unwind case: `handler` as both a
parameter and a module-level def, asserting the two shadowed calls are flagged
and the sibling call after that scope closed is NOT. A binding outliving its
frame would destroy that true edge, the one direction this guard must never
fail in. It passed on the first run.

extraction 327 passed; pipeline registry parallel complexity 407 passed;
stack_overflow_b 6 passed in 38s (46s with the cursor ascent, 900s rc=124
before it), lsp_python_deep_nesting_no_crash PASS.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-02 10:45:37 +02:00
Pcristin cd8a5ed0d7 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-09-02 11:41:12 +03:00
Ilya Brykau 349aecba95 fix(pipeline): key the Go Field guard on the recorded selector shape, not reference text
cbm_go_suppress_bare_field_ref dropped a Field-targeted reference when
strchr(ref_name, '.') == NULL — but the extractor strips the receiver on
every path that reaches the resolver (resolve_lhs_write_name records the
trailing field name of a selector LHS; is_reference_node records the
inner field_identifier), so the dot test could never be false for Go.
The guard was a blanket veto: all ~4588 Go Field nodes recovered by
one 61de19bb were unreachable by USAGE/READS/WRITES, including genuine
selector references (issue #1962, found in maintainer post-merge review
of #1944).

The selector-vs-bare distinction exists at extraction time and was
discarded; record it and let the resolver consume it:

- CBMUsage/CBMReadWrite gain is_member_access. The usage recorder sets
  it when the reference node is a field_identifier (the member half of a
  selector); resolve_lhs_write_name reports it through an out-param when
  it takes the field/member LHS branch.
- cbm_go_suppress_bare_field_ref(is_go, is_member_access, target_label)
  refuses a Field bind only for references that were never the member
  half of a selector. All four resolver sites (sequential and parallel,
  USAGE and READS/WRITES) pass the recorded flag.

Reproduce-first: with the veto restored, the extended #1942 fixtures go
RED on both resolver paths at the new asserts (t.err = nil must WRITE
the field; t.n must produce USAGE); with the fix both twins are GREEN
and the original bare-local negatives still hold. The dead
ASSERT_FALSE(..., "t.err", ...) unit case — an input Go's extractor
cannot produce — is replaced by flag-based cases.

Fixes #1962.

Signed-off-by: Ilya Brykau <ilya.brykau@orca.security>
2026-09-02 10:39:03 +02:00
Martin Vogel 98e386e58b Merge branch 'main' into fix/autoindex-skip-limit-value-1466 2026-09-02 07:49:40 +02:00
Ertan 2f37f61e88 perf(mcp): filter suffix globs while building search scope
Signed-off-by: Ertan <ertan.kucukoglu@gmail.com>
2026-09-02 06:09:34 +03:00
Joshua Richter 592894a438 fix(routes): take a route's handler from the last argument, not the first
Express, Fastify, gin and Laravel all put middleware between the route
path and the handler, and the handler comes last:

    routerGet("/users", requireAuth, rateLimit, listUsers);

extract_handler_arg returned the FIRST argument that looked like a
function reference, so requireAuth won and the HANDLES edge pointed at
the middleware instead of listUsers. That is a wrong edge, not a
missing one, and it misleads anyone tracing a request through the
graph.

The scan bound hid a second fault. An arrow function matches none of
the accepted node kinds, so three inline middlewares pushed the real
handler past MAX_HANDLER_SCAN and no handler came back at all.

The loop now examines every argument and keeps the last eligible one.
Nothing that follows a handler matches the accepted kinds either, since
an options argument is an object node, so the last match is the
handler. HANDLER_START_IDX stays, because argument 0 really is the
path. MAX_HANDLER_SCAN had no other use and is gone.

A two-argument route has one eligible argument, so first and last agree
and the existing Express, Fastify, gin and Laravel route tests are
unchanged.

Both tests fail without the change, the first reporting "requireAuth"
where "listUsers" belongs and the second reporting no handler at all.

Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
2026-09-01 22:31:03 -04:00