mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-24 06:53:52 +08:00
383c2cd9e3dc2bec9ff3c266d26ee530064b085e
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Colby MchenryandClaude Opus 5.5
|
83f54a1853 |
fix(dart): a library links the files its part directives name (#2391)
A Dart `part 'x.g.dart';` now mints an `import` node (name = the URI,
signature = the directive) and an `imports` ref from the library's file
node, in both extractors. The ref resolves through `dartDirectiveFile` by
the rules an import or export follows, so the library's file links the
part's file. A library and its parts are one library, so a change to a
part, generated or hand-written, now reaches the library and what imports
it in `codegraph affected` and the files listed as using the part. Before,
neither `part` nor `part of` left a trace, and the two files were joined
only by whatever symbol edges happened to cross between them: riverpod's
framework.dart, which holds nothing but `part` lines, by none.
`part of` still records nothing: the library's edge already joins the two
files, and a part -> library edge would list every pair on both import
rails of both files and make each a file 2-cycle.
The kernel's directive arm now goes on into the directive's children, as
the TS ladder does. Its `import_or_export` arm used to return first, so an
annotated directive's arguments (`@Tag(f) import 'a.dart';`) were fn-ref
candidates on the wasm path only.
Validation on rrousselGit/riverpod @4ba1be2 and felangel/bloc @b9be1e2
with the kernel loaded, base
|
||
|
Colby MchenryandClaude Opus 5.5
|
d2fb323e51 |
fix(dart): an annotation links the constant or constructor it names, even behind a comment (#2390)
A Dart annotation is a constant expression: a const variable (`@riverpod`, `@meta.immutable`, a type's static constant or enum value) or a call of a const constructor (`@Riverpod(keepAlive: true)`, `@Foo.named(…)`). It was ranked like a Python decorator, which favours a function or method of that name over a class and scores a constant not at all, so riverpod's 506 `@riverpod` annotations went to riverpod_analyzer_utils' extension getter `riverpod` (0.4) instead of riverpod_annotation's `const riverpod = Riverpod();`. matchDartAnnotation (name-matcher.ts) now takes every Dart `decorates` ref before any other strategy, and nothing falls through. It reads the annotation as written from its `@` and links a constant or (called) a class the file's library can see, through the import prefix it is written with (isDartImportPrefix, dart-libraries.ts); a type's own static constant, enum value or (called) named constructor (isDartConstructor); or, on a member, a static constant of the type around it. Never a method, getter or function. The decorator scan also stopped at the first sibling that was not an annotation, so a comment between an annotation and its declaration (`@override` `// ignore: must_call_super` `void f()`, riverpod_lint's `@riverpod` `// expect_lint: …` fixtures) hid the annotation. Dart's decoratorStepOverTypes lists its two comment kinds, which the scan now steps over, and the kernel's extract_decorators_for steps over the same. Validated on rrousselGit/riverpod @4ba1be2 and felangel/bloc @b9be1e2 with the kernel loaded: riverpod's 506 `@riverpod` edges move to `const riverpod` and 16 annotations behind a comment add an edge each, with nothing removed; bloc gains 7 `@override` refs and no edge changes. Kernel and wasm full-index dumps are byte-identical, and the parity sweeps show 0 diffs. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
9924684899 |
fix(dart): a member keeps the dartdoc written above its annotations (#2387)
A Dart member's annotations are siblings that stand between it and the
dartdoc written above them:
/// Builds the widget.
@override
Widget build(BuildContext context) { ... }
Both extractors read a docstring from the comments directly before the
declaration (or its `declaration` wrapper) and stopped at the first node
that was not a comment, so the annotation ended the run before the `///`
was reached. Members behind `@override`, `@protected`, `@mustCallSuper`,
`@pragma(...)`, `@internal` and the like lost their documentation, and
searching for words in it didn't find them.
Extraction (TS + kernel, at parity):
- getPrecedingDocstring takes an optional list of sibling types to step
over, and Dart sets `docstringStepOverTypes: ['annotation']`. Stacked,
multi-line and inline annotations are passed, and comments on either
side of one still join (`/// a` `@x` `// b` gives "a" and "b" on two
lines), as adjacent comments already did. Any other node still ends
the walk: the previous member's body, a field, a top-level variable,
an import or another declaration.
- The kernel's shared docstring.rs gains preceding_docstring_stepping_over.
preceding_docstring passes an empty list, so every other language is
unchanged; only Dart's docstring_of passes ["annotation"].
- Class-like declarations needed nothing: a class, mixin, extension,
extension type, enum or typedef node starts at its first annotation,
so the dartdoc above it is already the node's previous sibling.
Validated on felangel/bloc @b9be1e2 and rrousselGit/riverpod @4ba1be2 at
main
|
||
|
Colby MchenryandClaude Opus 5.5
|
249d9a860e |
fix(dart): a comment inside a member chain no longer loses a call or a read (#2385)
tree-sitter-dart keeps a comment as a named node wherever it is written, so in a member chain broken over lines (`tester //`, then `.state(…)` on the next line: the empty `//` keeps dart format from joining the lines) the comment, not `tester`, is the sibling right before the `.state` selector. Both extractors (the TS walker and the native kernel) read that sibling as the receiver or the callee: - A call with a comment between the member and its arguments (`box.grow /* by */ (3)`) was lost, and recorded as a read of `box.grow`. - A getter read (`box //` + `.area`) or a static access (`Config //` + `.instance`) written after a comment was lost. - A call after a comment kept only its bare name. Since #2376 the resolver reads such a chain link back past comments, so those calls already linked correctly; they now carry the receiver in the name. Every sibling step along a member chain now skips `comment` and `documentation_comment` nodes: the callee before an argument part, dartReceiverOf / receiver_of, dartCalleeOfArgPart, the type in front of a constructor's type arguments, and the member read's receiver and next selector. The misparsed-generic-call gate keeps reading raw siblings, and now rejects a comment against the `<` (`ref.read /* c */<Repo>(p)`), so that shape stays a comparison instead of losing its read. Validated with the kernel loaded on felangel/bloc @b9be1e2, rrousselGit/riverpod @4ba1be2 and flutter/samples @63411527. No edge is added or removed: the 14 comment-split chains there are all in riverpod, and its 5 calls among them gain their receiver and stay unlinked. Kernel/wasm parity has 0 diffs on all three. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
d500e58a52 |
fix(dart): link a call chained on a generic lookup like BlocProvider.of<T>(…) (#750) (#2383)
When a generic call parses as a call (a named argument after a positional
one, as in `(context, listen: false)`, or a type argument like `<A?>`,
`<A, B>` or `<List<A>>`), the Dart extractor records a chain on it as
`BlocProvider.of().increment`. The resolver typed that receiver by what
`BlocProvider.of` is declared to return: `T` for `static T of<T>(…)`,
which names no type, and nothing at all when the lookup comes from a
package, as flutter_bloc and provider do in an app. So
`BlocProvider.of<CounterCubit>(context, listen: false).increment()` and
provider's `Provider.of<Cart>(context, listen: false).add(item)` linked
nothing.
The declared return type is still tried first, so `Foo.create().bar()`
with a concrete return type resolves as before. When it gives no edge,
the chain is read at the call site the way #2376 reads a later link of a
chain: the lookup's own type parameter is the type argument the call
gives, and an outside `of<T>(…)` lookup hands back its T. The edge stays
on the encoded ref, as in Go's fallback (the gin runaway).
The extractors are unchanged. A chain on a call parsed as two
comparisons keeps its bare name (#2375), and now reaches the same
method; the comments that gave the old reason are corrected.
Validated before/after with the native kernel at
|
||
|
Colby MchenryandClaude Opus 5.5
|
1b752de2c0 |
fix(dart): a member with no body keeps its dartdoc and annotations (#2382)
A Dart member written without a body - a constructor like `Foo._();` or
`Point.origin() : x = 0;`, a `const` constructor or redirecting factory
such as flutter_bloc's `const BlocProvider.value(...)`, an abstract or
`external` method - parses as a `declaration` wrapping its signature,
and its `///` dartdoc and `@annotation`s are siblings of that wrapper.
Both extractors looked for them before the signature, inside the
wrapper, and found nothing: the docstring was lost, and
`@visibleForTesting Foo._();` emitted no `decorates` reference.
Extraction (TS + kernel, at parity):
- A Dart-only `getDeclarationWrapper` hook returns the `declaration`
when the signature is its first named child. `docstringFor` and
extractDecoratorsFor's preceding-sibling scan start from it; the
kernel's `declaration_wrapper` does the same for every docstring
lookup and for the annotation scan.
- The shared DOCSTRING_WRAPPER_TYPES / `is_wrapper` list is untouched:
C/C++ share it, and their `declaration` wraps declarators. All 35
non-Dart parity fixtures extract byte-identically before and after.
Validated on felangel/bloc @b9be1e2 and rrousselGit/riverpod @4ba1be2
at main
|
||
|
Colby MchenryandClaude Opus 5.5
|
3f96f80fcb |
fix(dart): link generic calls the parser reads as two comparisons (#2375)
tree-sitter-dart parses a generic call with arguments as two comparisons about as often as it parses it as a call, depending on the code around it: `ref.read<Repo>(p)` comes out as `relational_expression(relational_expression(ref.read < Repo) > (p))`. Generic constructors (`Provider<int>((ref) => 0)`), named arguments (which then parse as a `record_literal`) and calls after `await` or a prefix operator are hit too. Neither extractor recorded a call for that shape, only a member read of `ref.read` that linked nothing, so providers, repository reads and bloc lookups written with a type argument had no callers. Across felangel/bloc, rrousselGit/riverpod and flutter/samples there are 2,853 such calls, against 2,836 generic calls with arguments that parse as calls. - Both extractors (the TS walker and the native kernel) recover the call at its `<` when the type argument names a type and the code is laid out as a call: `<` against the callee and `(` against the `>`. A comparison such as `a < b` is left alone. The recovered call gets what a parsed one does: the call at the `<`, a reference to the type argument, and no member read of the callee or of a prefixed type argument. - A call chained on a recovered call keeps its bare name. The `BlocProvider.of().increment` encoding a parsed chain gets resolves through what `of` returns, which for a generic factory is its type parameter, so it would drop the edge the bare name finds. - When the grammar ends an arrow function in front of a generic call (`(ref) => ref.watch<int>(p)` as `((ref) => ref).watch<int>(p)`), the receiver is read from the end of the arrow's body, so the call is `ref.watch` rather than a bare `watch`. Validated with the kernel loaded: bloc +85, riverpod +1,580 and flutter/samples +54 edges, none lost; kernel/wasm parity has 0 diffs on all three repos. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
652c36b72a |
fix(dart): index const constructors and redirecting factories; a call through a type stays on that type (#2380)
* fix(dart): index const constructors and redirecting factories; a call through a type stays on that type Both Dart extractors (the TS walker and the native kernel) skipped `const` constructors and redirecting factories (`const Foo.bar(...)`, `const factory Foo.bar() = _Bar;`), which parse as signature kinds of their own. A call such as flutter_bloc's `BlocProvider.value(...)` then found no method on BlocProvider, and the resolver guessed another type's `value`: on felangel/bloc, 89 calls linked to `RepositoryProvider::value`. Extraction (TS + kernel, at parity): - Named const constructors and redirecting factories are methods named by the constructor, returning their class, with a parameters-only signature; the unnamed ones stay unindexed. The name is read only from before the parameter list, so a redirect target (`= _Impl.named`) never names it. - A redirecting factory references the class it redirects to, not the target constructor's name or an import prefix. - A constructor called with type arguments keeps its type: `BlocProvider<CounterCubit>.value(...)` is `BlocProvider.value`, as a statement and as an expression (`constructor_invocation`, which produced no call at all before). Resolver: - A Dart call through a type's name reaches only that type's own constructor, static member or static constant, or nothing: no guess by name, no inherited lookup, no same-file look-alike, no instance member of the same name. SDK calls like `Uri.parse` no longer land on a project `parse` past an `extension on Uri`. - A named constructor is never reached by a bare name, the last link of a chain, a function value or an annotation (riverpod's 210 `@internal` annotations). Validated on felangel/bloc @b9be1e2 and rrousselGit/riverpod @4ba1be2 with the kernel loaded: bloc 18,338 -> 18,495 edges, riverpod 85,606 -> 85,854, every removed edge triaged. Afterwards every edge into a named constructor is written `Owner.ctor`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dart): read a type's static members one way for calls and chains - A call through a type's name now finds its member with dartStaticMember, the lookup a chain's static link uses since #2376, so both pick the same member: the call site's own file first, then the nearest copy. - isDartStaticMember's constructor test is isDartConstructor, anchored at the declaration: an instance getter declared just above a same-named factory (`Object? get error` over `const factory AsyncValue.error(...)`) read the factory from the next lines and counted as static. - clearNameMatcherMemos drops DART_CONSTRUCTORS with the other source-derived Dart memos. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
6421644ef2 |
fix(dart): link calls in top-level and field initializers (#2367)
Both Dart extractors (the TS walker and the native kernel) skipped the code in initializers. A `final`/`const` declaration minted its constant and stopped, and a field's or `var`'s initializer only had its types recorded (#2360). In Riverpod apps, where most wiring lives in provider closures (`final repoProvider = Provider((ref) => Repository(ref.watch(dioProvider)));`), what a provider builds with had no callers, and impact and explore flows stopped at the provider. - A `final`/`const` initializer is walked like a body, for its constant: calls, instantiations, static and member reads, types, closures with block bodies and local functions, function values. - A field's (instance, `static var`, typed, `late`) or a top-level `var`/`late final` initializer is walked for its class or the file, since neither mints a node. A local declaration's second variable (`for (var i = 0, j = n(); ...)`) stays its function's. - Each initializer is walked once: a new `walkInitializer` context method marks it, and the dispatcher's function-as-value scan skips it. Other languages' hooks are unchanged. - Resolver: a Dart call through a type's static constant (`FutureProvider.autoDispose(...)` on `static const autoDispose = ...Builder();`) links that constant, not another type's same-named method. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
9f69e47836 |
fix(dart): link getter reads, enum-extension members and more type positions (#2338, #2327) (#2360)
Issue #2338: a Dart getter read (`x.area`, `s.label`) linked nothing, so `codegraph callers area` was empty while `x.grow()` worked, and a member an extension adds to an ENUM (`s.shout()` through `extension ShapeInfo on Shape`) never resolved. Issue #2327: `callers Report` saw only parameter and return types; the type an extension is `on`, field types, and types named in bodies and initializers (`Future<Report?>.value(null)`, riverpod's `final reportProvider = Family<Report?, String>()`) recorded nothing. Causes: - The Dart body walker only emitted calls (a selector with an argument list); a member read is a selector without one, so it produced no ref. - For a typed receiver, resolveMethodOnType looks for `Type::member` and the type's supertype edges; an extension's member is `ShapeInfo::shout`. On a class receiver the call fell through to name matching, which found a unique `twice` by luck; on an enum the "type the project doesn't declare" guard (it only counts classes) returned null. - Type references were taken from function and method signatures only. Fix (TS extractor and the Rust kernel, mirrored): - A member read `x.area` / `x?.area` / `Type.getter` whose receiver is a plain name, and that is not a call, is sent as a `references` ref named `x.area` on the member. The resolver (matchDartMemberRead, ahead of every other strategy) types the receiver — a static read through a project type, a parameter or local in scope, else a field of the enclosing class — and links the getter that type reaches: declared on it or a type it extends, mixes in or implements, else added by an extension on one of those (an unnamed extension only in its own library). The edge is `calls`: a Dart getter is a method node, reading it runs it, and `calls` is what callers, impact and explore's flow follow (as Ruby's paren-less `x.area` and VB.NET's paren-less calls through #2355 already are). A field read, a getter of a type outside the project and an untyped receiver link nothing; there is no name-only fallback. Kotlin/Swift/C#/TS link no instance property reads today. - resolveMethodOnType, for Dart, falls back to a member an extension on the type (or a supertype) adds when no declaration of the type has it. - `references` refs for the type an extension is `on`, a field's declared type and initializer (from its class — Dart fields mint no nodes), an enum field, a top-level variable's type (from the file), a constant's initializer (from the constant), and every type a function body names: locals, generic arguments, casts, type tests, catch clauses. Skipped: a `new`/`const` constructor's class (already instantiates/calls), an import prefix (`p` in `p.Foo`), built-ins, and lowercase names, which in Dart are built-ins (`num`, `dynamic`) or error-recovery artifacts around syntax the grammar predates (dot shorthands). A class whose field holds its own type does not get a self-edge. Verification: __tests__/dart-getter-reads.test.ts (7 tests) and __tests__/dart-type-positions.test.ts (3 tests) fail 8/10 on main (both the TS path and main's kernel) and pass on both arms here; the 2 that pass on main are the controls. New kernel parity fixture TortureReadsTypes.dart (+CRLF); kernel-dart-parity passes with CODEGRAPH_KERNEL_EXPECT=1, and scripts/kernel-parity.mjs over bloc/riverpod/flutter-samples is 0 diffs (2,103 files byte-parity, 300 parse-error files defer as before). tsc and build clean; the issues' repros now list c/a/b for area/label/shout and Holder, ReportX, reportProvider and run for Report. Validation (kernel loaded, main vs this branch): nodes unchanged, 0 edges removed, no new self-loops. - felangel/bloc: 16,372 -> 18,276 edges (+550 getter-read calls, +1,354 type references) - rrousselGit/riverpod: 76,432 -> 81,909 (+598 getter reads, +13 extension-method calls, +4,866 type references) - flutter/samples: 26,133 -> 26,943 (+142 getter reads, +668 type refs) Spot-checked 16 getter reads and 16 type references against source. Index time within noise on riverpod (median of 5 interleaved runs); reads that link nothing are kept for re-resolution like failed calls (riverpod db 92.9 -> 103.2 MB). No EXTRACTION_VERSION bump: it already moved to 28 this release. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
0ff11b36e7 |
fix(dart): a bare call stays in the class hierarchy (#2145)
A receiver-less Dart call now reaches only a method of the enclosing class, something it extends / mixes in / implements, or an extension on one of those types, and the nearest one wins. The hierarchy is read from each declaration's head (comments and type arguments dropped), since supertype edges don't exist on the first pass. Body-less abstract members (extracted as owned functions) are scoped too. Mixin-application classes (`class A = B with C;`) were extracted as `<anonymous>` with no supertypes; both the wasm extractor and the Rust kernel now name them and emit their extends/implements refs (kernel parity verified on bloc, riverpod, shelf, dio, getx). A/B (edges removed / added): riverpod -1945/+517, shelf -785, getx -855/+18, bloc -178/+95, dio -78. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
86708bf519 |
fix(extraction): preserve symbols with colliding same-line IDs (#1349) (#2030)
Same-kind, same-name declarations on one line shared an ID and silently overwrote each other in SQLite. Use per-extraction collision tracking to append a UTF-16 column only to later distinct declarations. Apply the same rule across wasm, Rust language walkers, Liquid, and CFML while preserving file IDs and edge endpoints. Add persistence, call-edge, identity-vector, and Unicode parity regressions. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
yucong-dong
|
6f4ec88350 |
fix(dart): index a Dart 3 extension type as the type it is (#1784) (#1865)
`extension_type_declaration` appears in none of the three places Dart's other type-like declarations do. `extension_declaration` — the older `extension` — is in all three, and the two names are near neighbours. That is why #1780's `isInsideClassLikeNode()` gate dropped these members and no others: the gate asks whether a class-like node is on the stack, and an `extension type` never put one there, while `extension`, `mixin` and `class` all did. The gate surfaced the omission rather than causing it. Before it, the members were still reached — but as top-level `function:km` rather than `method:MetersT::km`, indexed and attributed to nothing. The kernel omits the same node type, so it walks the members through its own fallback and mints them as top-level functions with no `MetersT` to belong to. `kernel-dart-parity.test.ts` fails four cases on `main` because of it; fixing only the wasm side would leave those red. Listing it alongside the others is the whole fix — `extraClassNodeTypes` and `dartEnclosingTypeName` on the wasm side, the two `matches!` arms on the kernel's. Both paths now give: class:MetersT · method:MetersT::km · method:MetersT::report with `report`'s span running to its closing brace rather than stopping at the signature line, and an ordinary class untouched. |
||
|
Colby Mchenry
|
838006c947 |
fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) (#1600)
Fixes #1581.
## What was wrong
`codegraph init` / `codegraph index` died with `Segmentation fault` — the whole CLI
process, not a parse worker — on a C/C++ file with very deep brace nesting (llvm's
`clang/test/Parser/parser_overflow.c`, 16,384 nested `{`). The reporter's diagnosis is
exactly right: tree-sitter's parser is iterative, so the file parses fine, and then the
native kernel's **recursive walker** (`visit_node` → `visit_for_calls_and_structure` → …,
one frame per AST level) overflowed the thread's stack. A native overflow can't be caught
the way a wasm abort can, and a parse worker is a thread of the `codegraph` process, so
the SIGSEGV took the entire indexer down — no message, no per-file fallback, no partial
index.
Two things made "just give the worker a bigger stack" the wrong fix:
- it only moves the cliff — reproduced here: the reporter's 16,384-deep file kills a
default 4 MiB worker (rc=132 on macOS / 139 on Linux), and a 100k-deep file kills the
8 MiB **main** thread too;
- the walkers are shared by every kernel-routed language (20 of them), and each has
several recursion points with different frame sizes, so no single stack size is a
provable bound.
Meanwhile the wasm path already handles this shape gracefully: its JS walker catches its
own `RangeError` per file and stores a partial result with a `parse_error`. The kernel
just needed a way to get there instead of dying.
## What this does
**The kernel guards its own recursion against the calling thread's real stack bounds and
defers a too-deep file to wasm** — the same `defer:` routing signal it already uses for
files with parse errors, which `src/extraction/kernel/index.ts` treats as "take the wasm
path for this file", silently.
- `codegraph-kernel/src/stack.rs`: per-thread stack bounds from the OS, computed once per
thread and cached — glibc/musl `pthread_getattr_np` + `pthread_attr_getstack`, macOS
`pthread_get_stackaddr_np` + `pthread_get_stacksize_np`, Win32
`GetCurrentThreadStackLimits` (a hand-declared `kernel32` extern; no `windows-sys`).
`exhausted()` is one thread-local load and one compare: true once the stack pointer is
within a 256 KiB red zone of the limit, and it latches a flag. Where the OS can't report
bounds it falls back to a fixed 1 MiB descent budget measured from the entry stack
pointer — safe on anything from Node's 4 MiB worker default up. So the guard is exact on
the 4 MiB worker, the 8 MiB main thread, and any `resourceLimits.stackSizeMb` alike.
- `stack_guard!()` (defined in `lib.rs`) is the first statement of every recursive walker
function — all **150** self-recursive or on-cycle functions across the 15 walker modules,
found by script (every cycle in the call graph, not just direct self-calls). It returns
`Default::default()` (`()`, `false`, `None`, `""`) so an exhausted walk simply stops
descending; a hook returning `false` sends its caller down the generic child walk, whose
own guard returns at once.
- `extract_file` runs the whole walk under `stack::run_guarded`: if the flag is set
afterwards the (truncated) result is discarded and replaced by
`defer: nesting too deep for the native walker — wasm recovery handles it`.
- `parse-pool.ts`: a comment at `new Worker(scriptPath)` records why there is deliberately
no `resourceLimits.stackSizeMb` bump.
- No new crates beyond `libc` as a direct unix dependency (already in `Cargo.lock`
transitively). No wire/ABI change.
Net effect for the reporter's repo: `deep.c` goes to the wasm path, lands as
`function foo` plus a recorded parse warning, and the other 31,607 files index normally.
`CODEGRAPH_KERNEL=0` and the `exclude` workaround are no longer needed.
## Tests
**Rust unit tests** (`cargo test`, 21 passed — 7 new in `stack.rs`): the walkers for
C, C++, Rust, TypeScript and Python are driven on a **1 MiB** thread (a quarter of Node's
worker default) with 30k-deep nesting and must return `defer:` instead of crashing;
shallow files are untouched; the latch resets between runs; the OS bounds are sane on the
main thread and describe a small thread's own stack.
**`__tests__/kernel-deep-nesting.test.ts`** (new, 8 tests — skips without a staged `.node`,
fails under `CODEGRAPH_KERNEL_EXPECT=1` if the kernel is missing, like the other kernel
suites):
- every default-routed language (all 20) survives a 60k-deep expression on the main thread
— clean result or the wasm fallback's partial result, never a crash;
- the reporter's exact 16,384-brace C file is indexed (partial) on the main thread;
- 200-deep expressions in every language still take the kernel path clean (the guard never
trips on normal code);
- inside a **default-sized 4 MiB `worker_threads` Worker** through `dist/`: the reporter's
`deep.c` and a 60k-deep expression in every language come back `deferred` with exit 0,
and a normal file still extracts natively;
- end-to-end through the built CLI: `codegraph init` on a repo holding `deep.c` + `ok.c`
exits 0 and records both files, with both functions.
**Existing kernel suites**: all 15 (`kernel-*-parity`, `kernel-scaffold`,
`kernel-retry-materialize`, `kernel-grammar-parity`) pass unchanged, 147 tests — the guard
never fires on the parity fixtures.
**Reporter's probes** (`one.js` from the issue, default 4 MiB worker, this build):
`deep.c` → `deferred`, exitCode=0 (was rc=132/139); `deep100k.c` → `deferred`, exitCode=0.
Main thread: `deep.c` / `deep100k.c` → wasm partial with
`Parse error: Maximum call stack size exceeded`; a 6,000-term binary expression and a
3,000-branch `else if` chain stay on the kernel path with clean results.
**Perf** (same `dist/`, only the `.node` swapped via `CODEGRAPH_KERNEL_PATH`; interleaved
main/new ×3, `codegraph init`, macOS arm64):
| repo | main (median) | guarded (median) | nodes / edges |
|---|---|---|---|
| express (141 files) | 0.60 s (0.58–0.65) | 0.61 s (0.58–0.61) | 1,084 / identical |
| redis (786 C/H files) | 4.44 s (4.39–4.66) | 4.49 s (4.41–4.70) | 19,942 / 76,446 identical |
Within run-to-run noise, as expected for one TLS load + compare per recursion entry.
**Linux (Docker, `node:22-bookworm`, kernel built in-container, `docker run --rm --init`)** —
the reporter's platform and the glibc `pthread_getattr_np` bounds path:
```
=== platform ===
Linux efe3cc86947b 6.12.54-linuxkit #1 SMP Tue Nov 4 21:21:47 UTC 2025 aarch64 GNU/Linux
v22.22.3
-rwxr-xr-x 1 root root 35332288 Aug 22 18:02 codegraph-kernel/prebuilds/linux-arm64/codegraph-kernel.node
=== reporter repro (issue #1581): 16,384-brace deep.c, codegraph init ===
│
└ Done
init exit code: 0
file: deep.c
file: deep100k.c
file: ok.c
function: add
function: bar
function: foo
=== worker probe: kernel raw extract in a default 4 MiB worker ===
deep.c: deferred
deep.c: worker exitCode=0
deep100k.c: deferred
deep100k.c: worker exitCode=0
ok.c: kernel nodes=2
ok.c: worker exitCode=0
=== cargo test stack:: (glibc pthread_getattr_np bounds path) ===
test stack::tests::os_bounds_are_sane_on_this_platform ... ok
test stack::tests::small_stack_reports_its_own_bounds ... ok
test stack::tests::normal_files_are_untouched_by_the_guard ... ok
test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok
test stack::tests::latch_resets_between_runs ... ok
test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.23s
=== vitest: kernel-deep-nesting + kernel-scaffold ===
✓ __tests__/kernel-scaffold.test.ts (10 tests) 30ms
✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 36989ms
Test Files 2 passed (2)
Tests 18 passed (18)
```
(The pre-fix crash was reproduced on macOS — rc=132 in a default worker, rc=139 on the main thread at 100k depth — not re-run inside this container; the reporter's Linux x86_64 trace is the SIGSEGV form of the same overflow.)
**Windows (Parallels ARM64 VM, MSVC 14.44, `cargo 1.97`, kernel built on the VM,
`GetCurrentThreadStackLimits` path)**:
```
head:
|
||
|
Colby MchenryandClaude Fable 5
|
d1b75a1a27 |
feat(kernel): R7b Dart walker — dart module, vendored-grammar-C d4d8f3e + wasm byte-copy vendor, dart default-routed (#1386)
R7b batch 4 #4 — the FINAL R7b language (docs/design/dart-kernel-port-checklist.md is the authoritative quirk list). The fourth vendored-grammar-C language, with a twist: production dart resolved its wasm from tree-sitter-wasms, whose dart dependency is an UNPINNED github:UserNobody14/tree-sitter-dart — a routine dependency update would have silently changed dart's grammar. This PR byte-copies the shipping 0.1.13 artifact into src/extraction/wasm/ (VENDORED_WASM_LANGS += dart) and compiles the same-commit (d4d8f3e337d8) parser.c/scanner.c in the kernel — table identity proven by the kernel-grammar-parity row. crates.io tree-sitter-dart is the nielsenko fork (different lineage) — rejected. The center of gravity is THE SIBLING-BODY DOUBLE-WALK, reproduced bug-for-bug: dart attaches every function/method body as a NEXT SIBLING of its signature, and the TS walkers consume each body TWICE — once via resolveBody (attributed to the function/method) and once via the enclosing generic walk (attributed to the file/class). Duplicate local-function nodes with the SAME id under different parents, duplicated calls/instantiates refs, and file/class-attributed fn-ref twins all emit in the exact observed interleave (a dedicated fixture pins the duplicate-id rows; the bloc kind-census spot-check pins the counts). Also preserved (probe-pinned): the extractBareCall selector matrix (the first callTypes=[] language — cascades completely invisible, `?.` encodes like `.`, the `ConfigT.load()` calls+references double emission with no callee-of-call skip, capitalized-chain `Foo.create().run` re-encode, const-object callee names); the constructor hooks (unnamed ctor skipped, named ctors/factories renamed to the CTOR name with the class as returnType, `@override (T) m()` record-misparse rescued by class-name validation); operator methods minting `method "<anonymous>"`; static_final_declaration constants via the visitNode hook while instance fields mint NOTHING; the prefixed-return-type prefix bug (`other.OtherClass f()` → returnType `other`); enum `with` mixins silent vs `implements` working; anonymous extensions named after the ON type; deferred imports invisible; named-argument callbacks NOT fn-ref-captured (the Flutter `onPressed:` idiom — future accuracy PR, TS-side first); `async*`/`sync*` NOT async; value-refs with the LIVE dart sibling-body pull and the `$X`-vs-`${X}` interpolation asymmetry; dartdoc kept in all three comment forms with the annotation-broken chain. Gates: parity sweeps first-run 0-diff on shelf/bloc/flutter — 5,815 clean files byte-parity, deferrals 10/21/1341 ≈ the survey's 10/21/~1340 (both-arm grammar reality: empty object patterns — the sealed-class idiom — and unnamed `library;` dominate; --max-deferral 0.3); full-init dumps byte-identical ×3 (shelf 7,959 / bloc 40,026 / flutter 1,855,319 dump lines); bloc per-kind node census identical across arms (the double-walk duplicate rows survive the store identically); kernel-dart-parity suite (7 fixtures + in-memory CRLF variants + double-walk duplicate-id pin + generated-file skip pin + two defer pins); full suite 2,688 green ×2 with CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += dart (20 langs — R7b COMPLETE). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |