The per-language "resolve calls in AST node" walkers recurse once per nesting
level. Four of them — Python (py_resolve_calls_in), Go (resolve_calls_in_node),
PHP (php_resolve_calls_in_node), Kotlin (kt_resolve_calls_in_node) — had no depth
guard, so a deeply-nested or cyclic file could drive them into a native stack
overflow (SIGSEGV) that takes down the whole index run. C and Java already guard
this (C_LSP_MAX_WALK_DEPTH / JAVA_LSP_MAX_WALK_DEPTH).
Add the same wrapper/inner depth guard to all four, backed by a shared,
env-configurable cap: CBM_LSP_MAX_WALK_DEPTH (default 512) in scope.h. Past the
cap the wrapper skips the subtree — those calls stay unresolved (graceful
degradation, not a crash). The walk_depth-- runs after the inner body returns, so
early returns can't leak the counter.
Reproduce-first: lsp_{python,go,php,kotlin}_deep_nesting_no_crash (30000-deep
nested calls, run under a forked child + WIFSIGNALED). RED without the cap (the
child SIGSEGVs — verified by raising CBM_LSP_MAX_WALK_DEPTH to 100M), GREEN with
it. Full suite 5752/0, no ASan/leak.
Part of the resilient-indexing effort (Track B prevention layer). Refs #668.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
bind_lambda_args and the method-reference arity path indexed the resolved method signature's NULL-terminated param_types array by the CALL-SITE argument index. A call with more arguments than the resolved overload declares (overload mismatch, varargs) read past the terminator and dereferenced whatever followed in the arena as a CBMType* - SIGSEGV while indexing elasticsearch (same out-of-bounds family as #427 and the bitcoin targs fix). Both sites now bound the index by the walked param count. Reproduce-first: 6-line fixture - run(Runnable) invoked with six lambdas - crashed rc=134 pre-fix; fork+exit-signal regression test lsp_java_lambda_args_exceed_params_no_crash added; suite 5574/0.
Indexing large OSS during the perf sweep crashed three LSP resolvers, all by stack overflow in their recursive resolve walks (macOS crash reports): elasticsearch — SIGSEGV in bind_lambda_args under deep recursive java_resolve_calls_in_node frames; bitcoin — SIGSEGV in cbm_type_substitute via c_adl_resolve under deep c_resolve_calls_in_node frames; microsoft/TypeScript — SIGBUS under an unbounded lookup_member_type cycle (cyclic type graph in reallyLargeFile.ts). The expression-eval guards (eval_depth) did not cover these walks. Each resolver now has a depth-guarded entry (walk_depth cap 512 for the Java/C AST walks, member_depth cap 64 for TS member lookup); past the cap the subtree resolves as unknown — graceful degradation instead of a crash. Reproduce-first: prod CLI probes crashed pre-fix (nested-call Java rc=139, nested-call C++ rc=139); fork+exit-signal regression tests added (TS cyclic shape needs a real cross-file registry, so it is verified at the real-repo tier — microsoft/TypeScript now indexes clean). Side-finding documented in the sweep report: Java BLOCK nesting >=3000 is minutes-slow (separate pathology, not this fix).
Root cause (corrected from the report): not a tree-sitter null-subtree sentinel — 0xffffffffffffffff is a freed/heap-corrupted pointer. The vendored ts runtime allocates via its overridable ts_current_malloc/free; under the prod build's MI_OVERRIDE=1, the Windows static-MinGW link (--allow-multiple-definition) resolves ts_malloc to mimalloc but ts_free can resolve to the CRT free (or vice-versa), corrupting the heap freelist and crashing mid-parse on large templated C++ headers (scales with parse churn, not syntax). cbm_init now calls ts_set_allocator(mi_malloc, mi_calloc, mi_realloc, mi_free) so ts allocate+free go through one allocator on every platform. Guarded to CFLAGS_PROD (MI_OVERRIDE=1); the test build stays CRT+ASan to avoid an alloc/free mismatch there. Tests in test_stack_overflow.c: allocator-binding mechanism + large-templated-C++ extraction guard (the C++ gap noted in #424).
AST traversal functions used fixed-size TSNode stack[] arrays. When the
DFS stack filled up, the child-push loop exited silently, dropping
entire subtrees without warning (e.g. 600 TS imports → 511, 130 Express
routes capped at 512). Reported in #199; also addresses #213 (large
TypeScript files producing zero nodes from stack exhaustion) and #215
(SEGV in template calls from stack overflow).
Adds ts_node_stack.h — a growable stack backed by the existing arena
allocator. Initial capacity matches previous fixed caps so small files
allocate no extra memory; doubles on overflow instead of truncating.
Old blocks are abandoned in the arena and freed on arena_destroy at
the end of file extraction (no realloc, no mixed lifetimes).
Applied to all 14 TSNode fixed stacks across 9 extraction files
(extract_calls, extract_channels, extract_defs, extract_env_accesses,
extract_imports, extract_semantic, extract_type_assigns, extract_type_refs,
extract_usages). walk_defs (uses walk_defs_frame_t, different struct
type) deliberately left as-is.
7 regression tests in tests/test_stack_overflow.c cover TS imports >512,
JS/Python calls >512, Go calls >1024, Express routes >150, deeply
nested calls, and YAML vars >256.
Cherry-picked from #217 with merge-conflict resolution against post-#206/#207
main:
- Makefile.cbm: keep TEST_STACK_OVERFLOW_SRCS plus main's new
TEST_ZSTD_SRCS / TEST_ARTIFACT_SRCS.
- extract_channels.c: keep main's scan_string_consts_js name (main also
added scan_string_consts_python after the PR was opened); declare
CHAN_STACK_CAP in the enum block.
- test_stack_overflow.c: replace sprintf with snprintf using the actual
remaining buffer size, since macOS clang rejects sprintf and naive
snprintf with sizeof(p) on a char* fails -Wformat-truncation.
Closes#199, #213, #215. Closes#217.
Martin Vogel