mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-22 06:40:16 +08:00
355afc70b14c22713fee491eafb881b577d34ef1
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Colby MchenryandClaude Opus 5.5
|
355afc70b1 |
fix(csharp): walk field and property initializers; a constant never names a type (#2337) (#2348)
* fix(csharp): walk field and property initializers as the member Field declarators and property `= initializer` values were skipped by both extractors, so `private readonly ILogger _log = LogManager.GetLogger(typeof(X));` linked nothing, and a method or class used only from an initializer looked unused. - tree-sitter.ts / csharp.rs: walk each C# field declarator whole with its field on the stack, and every property `value:` (not only `=> expr`) with the property on the stack. Calls, instantiations, static reads, lambda bodies and fn-ref candidates (varinit included) belong to the member; the fn-ref scan skips the walked subtrees, so each candidate is captured once. Attributes stay unwalked. - A target-typed `new()` that is the initializer instantiates the declared type (`List<Foo> _items = new();` -> List). In bodies it stays invisible, as before. - name-matcher.ts: a .NET type position never names a `constant` (the kind `const` / `static readonly` fields get). Walking initializers exposed `new Version(5, 18)` binding to a `const string Version` and `static readonly Meter Meter = new(...)` instantiating itself. Kernel/wasm parity: 0 diffs on serilog, Newtonsoft.Json and jellyfin, and full-index dumps byte-identical between the two arms. No EXTRACTION_VERSION bump: #2345 already moved it to 28 this release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): credit the report and contributor behind the constant change The type-position change here is what issue #2337 reports (`new Station { … }` unresolved beside a `private const string Station`) and the same one-line change as @drakeo338's #2339. Verified on the issue's own three-file repro: the `instantiates` edge to `Demo.Core::Station`, missing on main, is present with this branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
489b4749c7 |
fix(vbnet,csharp): index every Structure member; walk property bodies and VB initializers (#2345)
VB.NET's grammar tags each member of a `Structure` as its own `body`
field, so extractAggregate's field lookup saw only the first member.
SCrawler's UserMedia indexed none of its members, and a structure that
opened with a nested enum indexed only the enum. Aggregates now resolve
their body the way classes and enums do (resolveBody, which VB maps to
the declaration itself). Objective-C, the only other struct language
with that hook, is unchanged.
Property bodies were never walked: the propertyTypes branch set
skipChildren after extractProperty, so calls, instantiations and reads
in a VB `Get`/`Set` block, `= initializer` or `As New`, and in a C#
accessor body or `=> expr`, were lost.
- propertyBodies() names the parts of a property that run code (VB: the
whole declaration, as its methods are walked; C#: each accessor's body
and an arrow value), walked with the property on the stack. The
candidates-only fn-ref scan skips the walked subtrees by node id, so a
function value is captured once, from the property. C# `= initializer`s
stay unwalked, as C# field initializers are.
- The C# kernel mirrors it (property_bodies, scan skip list). The parity
suite and a new micro pass; a serilog sweep is 209/214 byte-identical
with 0 diffs (5 deferred for parse errors).
- VB field declarators (`= expr`, `As New T`) and Custom Event
AddHandler/RemoveHandler/RaiseEvent blocks are walked as their field or
event.
Before/after (nodes / edges): SCrawler 9,950 -> 10,427 / 14,336 -> 15,092;
staxrip 13,440 -> 13,644 / 24,120 -> 28,002, most of it its
`Property X As New NumParam With {...}` settings; serilog edges
6,642 -> 6,654. All 23 removed edges are the same call sites re-resolved
by VB's name-guessing resolver now that struct members are candidates:
10 wrong edges dropped, 9 correct ones now guessed wrong, 4 wrong either
way. Designer -> Add/Size edge counts are unchanged.
EXTRACTION_VERSION 27 -> 28.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Colby MchenryandClaude Opus 5.5
|
ff75a6ca5f |
fix(csharp): namespaces, usings and nested types follow the language's scopes (#2220)
A block namespace qualifies only its own body (a nested one is Outer.Inner), mirrored in the kernel; a using names the project's namespace, never another file's using; a global using is its own project's; a nested type is reached by bare name only from inside its owner or a type deriving from it, through any partial part; visible C# types are filtered before ranking. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
f9530d7d5f |
fix(resolution): a call through a declared member calls that member's type (#2171)
C# / Java / Kotlin receivers that are fields, properties or parameters were typed only when declared in the calling method without generics, so a call through a field fell to the receiver-word guess: Newtonsoft's _innerWriter.WriteValue() landed on the wrapper's own WriteValue, a TextWriter's Write() on a test writer's, commons-lang's DateFormat field's parse() on DateParser 284 times. - read class-level member declarations (brace depth 1, comments skipped), walking base classes with their type arguments substituted, C# using aliases, type-parameter bounds, generic and for-each declarations - C# interface members default public and namespace-level types internal (TS extractor + kernel, parity 0 diffs); interface calls resolve at all - the receiver-word guess weighs the owner type's name only, the receiver's last link (a constant's type), and its head noun; .NET std names need a receiver named after the owner; production code never lands in a test suite's mock Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
86708bf519 |
fix(extraction): preserve symbols with colliding same-line IDs (#1349) (#2030)
Same-kind, same-name declarations on one line shared an ID and silently overwrote each other in SQLite. Use per-extraction collision tracking to append a UTF-16 column only to later distinct declarations. Apply the same rule across wasm, Rust language walkers, Liquid, and CFML while preserving file IDs and edge endpoints. Add persistence, call-edge, identity-vector, and Unicode parity regressions. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby Mchenry
|
838006c947 |
fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) (#1600)
Fixes #1581.
## What was wrong
`codegraph init` / `codegraph index` died with `Segmentation fault` — the whole CLI
process, not a parse worker — on a C/C++ file with very deep brace nesting (llvm's
`clang/test/Parser/parser_overflow.c`, 16,384 nested `{`). The reporter's diagnosis is
exactly right: tree-sitter's parser is iterative, so the file parses fine, and then the
native kernel's **recursive walker** (`visit_node` → `visit_for_calls_and_structure` → …,
one frame per AST level) overflowed the thread's stack. A native overflow can't be caught
the way a wasm abort can, and a parse worker is a thread of the `codegraph` process, so
the SIGSEGV took the entire indexer down — no message, no per-file fallback, no partial
index.
Two things made "just give the worker a bigger stack" the wrong fix:
- it only moves the cliff — reproduced here: the reporter's 16,384-deep file kills a
default 4 MiB worker (rc=132 on macOS / 139 on Linux), and a 100k-deep file kills the
8 MiB **main** thread too;
- the walkers are shared by every kernel-routed language (20 of them), and each has
several recursion points with different frame sizes, so no single stack size is a
provable bound.
Meanwhile the wasm path already handles this shape gracefully: its JS walker catches its
own `RangeError` per file and stores a partial result with a `parse_error`. The kernel
just needed a way to get there instead of dying.
## What this does
**The kernel guards its own recursion against the calling thread's real stack bounds and
defers a too-deep file to wasm** — the same `defer:` routing signal it already uses for
files with parse errors, which `src/extraction/kernel/index.ts` treats as "take the wasm
path for this file", silently.
- `codegraph-kernel/src/stack.rs`: per-thread stack bounds from the OS, computed once per
thread and cached — glibc/musl `pthread_getattr_np` + `pthread_attr_getstack`, macOS
`pthread_get_stackaddr_np` + `pthread_get_stacksize_np`, Win32
`GetCurrentThreadStackLimits` (a hand-declared `kernel32` extern; no `windows-sys`).
`exhausted()` is one thread-local load and one compare: true once the stack pointer is
within a 256 KiB red zone of the limit, and it latches a flag. Where the OS can't report
bounds it falls back to a fixed 1 MiB descent budget measured from the entry stack
pointer — safe on anything from Node's 4 MiB worker default up. So the guard is exact on
the 4 MiB worker, the 8 MiB main thread, and any `resourceLimits.stackSizeMb` alike.
- `stack_guard!()` (defined in `lib.rs`) is the first statement of every recursive walker
function — all **150** self-recursive or on-cycle functions across the 15 walker modules,
found by script (every cycle in the call graph, not just direct self-calls). It returns
`Default::default()` (`()`, `false`, `None`, `""`) so an exhausted walk simply stops
descending; a hook returning `false` sends its caller down the generic child walk, whose
own guard returns at once.
- `extract_file` runs the whole walk under `stack::run_guarded`: if the flag is set
afterwards the (truncated) result is discarded and replaced by
`defer: nesting too deep for the native walker — wasm recovery handles it`.
- `parse-pool.ts`: a comment at `new Worker(scriptPath)` records why there is deliberately
no `resourceLimits.stackSizeMb` bump.
- No new crates beyond `libc` as a direct unix dependency (already in `Cargo.lock`
transitively). No wire/ABI change.
Net effect for the reporter's repo: `deep.c` goes to the wasm path, lands as
`function foo` plus a recorded parse warning, and the other 31,607 files index normally.
`CODEGRAPH_KERNEL=0` and the `exclude` workaround are no longer needed.
## Tests
**Rust unit tests** (`cargo test`, 21 passed — 7 new in `stack.rs`): the walkers for
C, C++, Rust, TypeScript and Python are driven on a **1 MiB** thread (a quarter of Node's
worker default) with 30k-deep nesting and must return `defer:` instead of crashing;
shallow files are untouched; the latch resets between runs; the OS bounds are sane on the
main thread and describe a small thread's own stack.
**`__tests__/kernel-deep-nesting.test.ts`** (new, 8 tests — skips without a staged `.node`,
fails under `CODEGRAPH_KERNEL_EXPECT=1` if the kernel is missing, like the other kernel
suites):
- every default-routed language (all 20) survives a 60k-deep expression on the main thread
— clean result or the wasm fallback's partial result, never a crash;
- the reporter's exact 16,384-brace C file is indexed (partial) on the main thread;
- 200-deep expressions in every language still take the kernel path clean (the guard never
trips on normal code);
- inside a **default-sized 4 MiB `worker_threads` Worker** through `dist/`: the reporter's
`deep.c` and a 60k-deep expression in every language come back `deferred` with exit 0,
and a normal file still extracts natively;
- end-to-end through the built CLI: `codegraph init` on a repo holding `deep.c` + `ok.c`
exits 0 and records both files, with both functions.
**Existing kernel suites**: all 15 (`kernel-*-parity`, `kernel-scaffold`,
`kernel-retry-materialize`, `kernel-grammar-parity`) pass unchanged, 147 tests — the guard
never fires on the parity fixtures.
**Reporter's probes** (`one.js` from the issue, default 4 MiB worker, this build):
`deep.c` → `deferred`, exitCode=0 (was rc=132/139); `deep100k.c` → `deferred`, exitCode=0.
Main thread: `deep.c` / `deep100k.c` → wasm partial with
`Parse error: Maximum call stack size exceeded`; a 6,000-term binary expression and a
3,000-branch `else if` chain stay on the kernel path with clean results.
**Perf** (same `dist/`, only the `.node` swapped via `CODEGRAPH_KERNEL_PATH`; interleaved
main/new ×3, `codegraph init`, macOS arm64):
| repo | main (median) | guarded (median) | nodes / edges |
|---|---|---|---|
| express (141 files) | 0.60 s (0.58–0.65) | 0.61 s (0.58–0.61) | 1,084 / identical |
| redis (786 C/H files) | 4.44 s (4.39–4.66) | 4.49 s (4.41–4.70) | 19,942 / 76,446 identical |
Within run-to-run noise, as expected for one TLS load + compare per recursion entry.
**Linux (Docker, `node:22-bookworm`, kernel built in-container, `docker run --rm --init`)** —
the reporter's platform and the glibc `pthread_getattr_np` bounds path:
```
=== platform ===
Linux efe3cc86947b 6.12.54-linuxkit #1 SMP Tue Nov 4 21:21:47 UTC 2025 aarch64 GNU/Linux
v22.22.3
-rwxr-xr-x 1 root root 35332288 Aug 22 18:02 codegraph-kernel/prebuilds/linux-arm64/codegraph-kernel.node
=== reporter repro (issue #1581): 16,384-brace deep.c, codegraph init ===
│
└ Done
init exit code: 0
file: deep.c
file: deep100k.c
file: ok.c
function: add
function: bar
function: foo
=== worker probe: kernel raw extract in a default 4 MiB worker ===
deep.c: deferred
deep.c: worker exitCode=0
deep100k.c: deferred
deep100k.c: worker exitCode=0
ok.c: kernel nodes=2
ok.c: worker exitCode=0
=== cargo test stack:: (glibc pthread_getattr_np bounds path) ===
test stack::tests::os_bounds_are_sane_on_this_platform ... ok
test stack::tests::small_stack_reports_its_own_bounds ... ok
test stack::tests::normal_files_are_untouched_by_the_guard ... ok
test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok
test stack::tests::latch_resets_between_runs ... ok
test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.23s
=== vitest: kernel-deep-nesting + kernel-scaffold ===
✓ __tests__/kernel-scaffold.test.ts (10 tests) 30ms
✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 36989ms
Test Files 2 passed (2)
Tests 18 passed (18)
```
(The pre-fix crash was reproduced on macOS — rc=132 in a default worker, rc=139 on the main thread at 100k depth — not re-run inside this container; the reporter's Linux x86_64 trace is the SIGSEGV form of the same overflow.)
**Windows (Parallels ARM64 VM, MSVC 14.44, `cargo 1.97`, kernel built on the VM,
`GetCurrentThreadStackLimits` path)**:
```
head:
|
||
|
Colby MchenryandClaude Fable 5
|
286e9ccc2d |
feat(kernel): R7b C# walker — csharp module, tree-sitter-c-sharp 0.23.5 pin, csharp default-routed (#1378)
Second R7b port, checklist-first recipe (docs/design/csharp-kernel-port-checklist.md; parity passed FIRST RUN again). No grammar bump — the #717 vendored wasm verified table-identical to crate 0.23.5 (ABI 15, STATE_COUNT 8053, node-kind + field tables); first port with no grammar-prep step. The #237 #if-blanking preParse stays TS-side via the existing route-point hoist. Walker preserves bug-for-bug: the single-namespace-node quirks (second namespace nests under the first, nested namespaces leave no trace, import refs hang off the namespace node), raw member-access callee texts (this./base./literal receivers, multi-line fluent chains) with unconditional chain re-encode, deliberate emission holes (property accessor/expression bodies, ctor initializers, delegates/events/ operators/indexers/local functions, top-level locals), garbage extends refs ((repo) primary-ctor args, BaseDto(Name) record bases, enum : byte), the alias- import moduleName quirks, nameof-as-call, CSHARP fn-ref spec (+= subscription, this.X bare-name form, argument layer, initializer lists), C# type-ref engine (nested-generic returnType failure included), and value-ref shadow pruning. Gates: sweeps 0-diff serilog 211/216 / Newtonsoft.Json 914/945 / jellyfin 2104/2105 (deferrals match the survey's per-repo predictions — both-arm #if damage; default --max-deferral 0.1 holds, no c/cpp exemption); full-init dumps byte-identical ×3 (14.0k/109.1k/210.8k lines); kernel-csharp-parity suite (torture ×3 + CRLF variants + 8 micro-pins + defer) + csharp grammar-parity row; full suite 2,608 ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += csharp (11 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |