mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-16 05:39:41 +08:00
de9f346ceb6f53df5a214cd5563cb2ed907574ff
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
fa25883ab0 |
fix(cpp): resolve local constructors and macro visibility (#1839) (#2035)
* fix(cpp): macro invocations are not calls; local objects call their constructor A function-like macro invocation (`TRACE_POINT(1)`) parses as a call, and the resolver bound it to a same-named free function in another translation unit — a caller and a callee that never existed (#1838). And local object initialization (`Widget w;` / `Widget w(1);` / `Widget w{1};`) reached the class node instead of a constructor, or nothing at all for the default form (#1839). Extraction (wasm walker and the Rust kernel, in parity): - every `preproc_function_def` becomes a `constant` node whose signature is the directive, so the index knows which names are macros; - a C++ declaration emits one `calls` ref per constructed object, shaped `ns::T::T/<arity>`; pointer, reference and function declarators, `extern`, and array element braces construct nothing; the #1035 `instantiates` ref is unchanged; - constructors carry their parameter list as the signature. Resolution: - `cpp-macro-visibility.ts` walks the translation unit (root file plus in-repo includes, each file once) over per-file summaries of function-like defines, undefs and includes, evaluating what each file can decide itself (literals, its own defines, the include-guard idiom). A call whose name is definitely a macro at that point, or a name that only macros bear, resolves to nothing. A macro that exists in one build configuration only (CMSIS's `__DSB()` under the ARM compilers, a target's HAL compat header) keeps the call to the function the other configuration compiles; a wrapper macro that calls its own name is how that function gets called and hides nothing. - `cpp-constructor.ts` resolves the constructor refs: the type in the lexical namespaces of the call site, then the single overload whose parameter count admits the arguments; two admitting overloads, an initializer_list overload, or an aggregate yield no edge. - a `#define` constant is never a `calls` target and does not count toward the same-name ceiling. Measured on a betaflight tree (4,017 files, wasm arm): 153 `calls` edges lost — all fabricated (`MIN`/`MAX` bound to a CLI enum member, macros misparsed as functions, a cross-vendor `SWAPBYTE`) — and 34 constructor edges gained; indexing 34s vs 30s. Fixes #1838 Fixes #1839 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(cpp): resolve local constructors and macro visibility (#1839) Local construction lacked constructor calls, while macro invocations could bind unrelated functions. Resolve constructors using lexical types and overload signatures, preserving separately declared defaults and array element construction. Evaluate macro directives in translation-unit order, respecting reinclusion, guards and pragma once. Mirror extraction in Rust and WASM, preserve type dependencies, and distinguish them in CLI output. Co-authored-by: danusha2345 <danusha2345@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: danusha2345 <danusha2345@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
86708bf519 |
fix(extraction): preserve symbols with colliding same-line IDs (#1349) (#2030)
Same-kind, same-name declarations on one line shared an ID and silently overwrote each other in SQLite. Use per-extraction collision tracking to append a UTF-16 column only to later distinct declarations. Apply the same rule across wasm, Rust language walkers, Liquid, and CFML while preserving file IDs and edge endpoints. Add persistence, call-edge, identity-vector, and Unicode parity regressions. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandClaude Opus 5.5
|
9b5c8e49dd |
fix(extraction): recover single-argument C/C++ function macros (#1373) (#2017)
Single-argument function macros were indexed under parentheses in C or the macro name in C++, breaking caller resolution. Recover both parser shapes only when a preceding local macro definition establishes the function name, and mirror the recovery in the Rust kernel. Preserve existing multi-argument recovery and ordinary declarations; add extraction, resolution, explore-flow, and parity regressions. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Colby MchenryandColby McHenry
|
2f1a99d34c |
fix(extraction): index C++ pure virtual methods as nodes (#1727) (#1758)
Pure-virtual declarations (`virtual int read(int key) = 0;`) parse as field_declaration, not function_definition, so they minted no method node — calls through an abstract base and cpp-override synthesis had nothing to attach to. Mirror Java interface methods: mint the node (TS + kernel), mark isAbstract, and cover with extraction/e2e/parity fixtures. Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com> |
||
|
Colby Mchenry
|
838006c947 |
fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) (#1600)
Fixes #1581.
## What was wrong
`codegraph init` / `codegraph index` died with `Segmentation fault` — the whole CLI
process, not a parse worker — on a C/C++ file with very deep brace nesting (llvm's
`clang/test/Parser/parser_overflow.c`, 16,384 nested `{`). The reporter's diagnosis is
exactly right: tree-sitter's parser is iterative, so the file parses fine, and then the
native kernel's **recursive walker** (`visit_node` → `visit_for_calls_and_structure` → …,
one frame per AST level) overflowed the thread's stack. A native overflow can't be caught
the way a wasm abort can, and a parse worker is a thread of the `codegraph` process, so
the SIGSEGV took the entire indexer down — no message, no per-file fallback, no partial
index.
Two things made "just give the worker a bigger stack" the wrong fix:
- it only moves the cliff — reproduced here: the reporter's 16,384-deep file kills a
default 4 MiB worker (rc=132 on macOS / 139 on Linux), and a 100k-deep file kills the
8 MiB **main** thread too;
- the walkers are shared by every kernel-routed language (20 of them), and each has
several recursion points with different frame sizes, so no single stack size is a
provable bound.
Meanwhile the wasm path already handles this shape gracefully: its JS walker catches its
own `RangeError` per file and stores a partial result with a `parse_error`. The kernel
just needed a way to get there instead of dying.
## What this does
**The kernel guards its own recursion against the calling thread's real stack bounds and
defers a too-deep file to wasm** — the same `defer:` routing signal it already uses for
files with parse errors, which `src/extraction/kernel/index.ts` treats as "take the wasm
path for this file", silently.
- `codegraph-kernel/src/stack.rs`: per-thread stack bounds from the OS, computed once per
thread and cached — glibc/musl `pthread_getattr_np` + `pthread_attr_getstack`, macOS
`pthread_get_stackaddr_np` + `pthread_get_stacksize_np`, Win32
`GetCurrentThreadStackLimits` (a hand-declared `kernel32` extern; no `windows-sys`).
`exhausted()` is one thread-local load and one compare: true once the stack pointer is
within a 256 KiB red zone of the limit, and it latches a flag. Where the OS can't report
bounds it falls back to a fixed 1 MiB descent budget measured from the entry stack
pointer — safe on anything from Node's 4 MiB worker default up. So the guard is exact on
the 4 MiB worker, the 8 MiB main thread, and any `resourceLimits.stackSizeMb` alike.
- `stack_guard!()` (defined in `lib.rs`) is the first statement of every recursive walker
function — all **150** self-recursive or on-cycle functions across the 15 walker modules,
found by script (every cycle in the call graph, not just direct self-calls). It returns
`Default::default()` (`()`, `false`, `None`, `""`) so an exhausted walk simply stops
descending; a hook returning `false` sends its caller down the generic child walk, whose
own guard returns at once.
- `extract_file` runs the whole walk under `stack::run_guarded`: if the flag is set
afterwards the (truncated) result is discarded and replaced by
`defer: nesting too deep for the native walker — wasm recovery handles it`.
- `parse-pool.ts`: a comment at `new Worker(scriptPath)` records why there is deliberately
no `resourceLimits.stackSizeMb` bump.
- No new crates beyond `libc` as a direct unix dependency (already in `Cargo.lock`
transitively). No wire/ABI change.
Net effect for the reporter's repo: `deep.c` goes to the wasm path, lands as
`function foo` plus a recorded parse warning, and the other 31,607 files index normally.
`CODEGRAPH_KERNEL=0` and the `exclude` workaround are no longer needed.
## Tests
**Rust unit tests** (`cargo test`, 21 passed — 7 new in `stack.rs`): the walkers for
C, C++, Rust, TypeScript and Python are driven on a **1 MiB** thread (a quarter of Node's
worker default) with 30k-deep nesting and must return `defer:` instead of crashing;
shallow files are untouched; the latch resets between runs; the OS bounds are sane on the
main thread and describe a small thread's own stack.
**`__tests__/kernel-deep-nesting.test.ts`** (new, 8 tests — skips without a staged `.node`,
fails under `CODEGRAPH_KERNEL_EXPECT=1` if the kernel is missing, like the other kernel
suites):
- every default-routed language (all 20) survives a 60k-deep expression on the main thread
— clean result or the wasm fallback's partial result, never a crash;
- the reporter's exact 16,384-brace C file is indexed (partial) on the main thread;
- 200-deep expressions in every language still take the kernel path clean (the guard never
trips on normal code);
- inside a **default-sized 4 MiB `worker_threads` Worker** through `dist/`: the reporter's
`deep.c` and a 60k-deep expression in every language come back `deferred` with exit 0,
and a normal file still extracts natively;
- end-to-end through the built CLI: `codegraph init` on a repo holding `deep.c` + `ok.c`
exits 0 and records both files, with both functions.
**Existing kernel suites**: all 15 (`kernel-*-parity`, `kernel-scaffold`,
`kernel-retry-materialize`, `kernel-grammar-parity`) pass unchanged, 147 tests — the guard
never fires on the parity fixtures.
**Reporter's probes** (`one.js` from the issue, default 4 MiB worker, this build):
`deep.c` → `deferred`, exitCode=0 (was rc=132/139); `deep100k.c` → `deferred`, exitCode=0.
Main thread: `deep.c` / `deep100k.c` → wasm partial with
`Parse error: Maximum call stack size exceeded`; a 6,000-term binary expression and a
3,000-branch `else if` chain stay on the kernel path with clean results.
**Perf** (same `dist/`, only the `.node` swapped via `CODEGRAPH_KERNEL_PATH`; interleaved
main/new ×3, `codegraph init`, macOS arm64):
| repo | main (median) | guarded (median) | nodes / edges |
|---|---|---|---|
| express (141 files) | 0.60 s (0.58–0.65) | 0.61 s (0.58–0.61) | 1,084 / identical |
| redis (786 C/H files) | 4.44 s (4.39–4.66) | 4.49 s (4.41–4.70) | 19,942 / 76,446 identical |
Within run-to-run noise, as expected for one TLS load + compare per recursion entry.
**Linux (Docker, `node:22-bookworm`, kernel built in-container, `docker run --rm --init`)** —
the reporter's platform and the glibc `pthread_getattr_np` bounds path:
```
=== platform ===
Linux efe3cc86947b 6.12.54-linuxkit #1 SMP Tue Nov 4 21:21:47 UTC 2025 aarch64 GNU/Linux
v22.22.3
-rwxr-xr-x 1 root root 35332288 Aug 22 18:02 codegraph-kernel/prebuilds/linux-arm64/codegraph-kernel.node
=== reporter repro (issue #1581): 16,384-brace deep.c, codegraph init ===
│
└ Done
init exit code: 0
file: deep.c
file: deep100k.c
file: ok.c
function: add
function: bar
function: foo
=== worker probe: kernel raw extract in a default 4 MiB worker ===
deep.c: deferred
deep.c: worker exitCode=0
deep100k.c: deferred
deep100k.c: worker exitCode=0
ok.c: kernel nodes=2
ok.c: worker exitCode=0
=== cargo test stack:: (glibc pthread_getattr_np bounds path) ===
test stack::tests::os_bounds_are_sane_on_this_platform ... ok
test stack::tests::small_stack_reports_its_own_bounds ... ok
test stack::tests::normal_files_are_untouched_by_the_guard ... ok
test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok
test stack::tests::latch_resets_between_runs ... ok
test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.23s
=== vitest: kernel-deep-nesting + kernel-scaffold ===
✓ __tests__/kernel-scaffold.test.ts (10 tests) 30ms
✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 36989ms
Test Files 2 passed (2)
Tests 18 passed (18)
```
(The pre-fix crash was reproduced on macOS — rc=132 in a default worker, rc=139 on the main thread at 100k depth — not re-run inside this container; the reporter's Linux x86_64 trace is the SIGSEGV form of the same overflow.)
**Windows (Parallels ARM64 VM, MSVC 14.44, `cargo 1.97`, kernel built on the VM,
`GetCurrentThreadStackLimits` path)**:
```
head:
|
||
|
ctype_lab
|
8e3cde6994 | feat(kernel): preserve union nodes across extraction | ||
|
ctype_labandClaude Opus 5
|
85e9ac6762 |
fix(c,cpp,objc,rust): index union declarations
A `union` declaration produced no symbol at all in any of the four
languages that have one. The type never entered the graph, and neither
did anything attached to it — in Rust every `impl Trait for MyUnion`
lost its edge, and the impl's methods were left with a qualifiedName
pointing at a type the graph did not contain.
`union_specifier` / `union_item` were absent from the extraction layer
entirely: no `<x>Types` list on the TS side, no dispatch branch in
either kernel walker.
They join `structTypes` (kind `struct` — NodeKind has no `union`), which
is the extension point the table-driven extractors already provide. The
body guard in extractStruct is untouched, so a bodiless `union U;` stays
a forward declaration and is still skipped, exactly like `struct U;`.
`resolveTypeAliasKind` accepts `union_specifier` too, so
`typedef union { … } N;` takes the typedef's name the way
`typedef struct { … } N;` already did. Without it the anonymous union
body would mint a second `<anonymous>` node beside the alias.
Both walkers change together so kernel<->wasm parity holds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Colby MchenryandClaude Fable 5
|
2d72891b59 |
feat(kernel): R7a C/C++ walker — dual-lang ccpp module, preParse hoist, 7 new blanks, c/cpp default-routed (#1346)
Parity: 0 diffs on redis/git/fmt/protobuf/ALS sweeps; full-init dumps byte-identical on all five + linux at kernel scale (10.4M dump lines, same sha256 both arms). Linux 2c/6GB envelope: kernel-arm 19.1min vs wasm-arm 22.9min (parse 356s vs 435s) on a much richer graph (the new blanks recover error-swallowed code: git 2x nodes, linux kernel/+mm/ 3x). Deferral guard corrected by measurement (C/C++ error incidence 9-42%; --max-deferral flag); defer-reuse memo kills the 3x re-blank/re-parse cost deferred files paid. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |