walk_defs used a fixed `walk_defs_frame_t stack[4096]` — a single ~160 KB frame on
the C stack. Two problems:
- On small thread stacks (notably the pre-2026-03 Windows 1 MB main thread) that
huge frame overflowed the stack during the definitions pass — the 0xC00000FD
crash a reporter hit indexing a large SQL file (#668). Reproduced locally by
running extraction under a constrained stack (~208 KB) + ASan.
- The `top < 4096` push guards SILENTLY DROPPED every top-level definition past
4096, so a file with more than 4096 top-level defs was only partially indexed.
Replace the fixed array with a growable heap stack (wd_stack_t + wd_push): a
256-frame initial buffer that doubles on demand, bounded by a generous,
env-configurable ceiling (CBM_WALK_DEFS_MAX, default 8M frames) that WARNs once
instead of dropping, and returns cleanly on realloc OOM (no NULL deref). A normal
file now uses ~10 KB of heap instead of 160 KB of stack, and files with thousands
of top-level defs are fully extracted. The frame stack is a pure index-addressed
LIFO (frames copied by value on pop), so heap-backing is semantically identical.
Reproduce-first: walk_defs_no_truncation_over_4096_issue668 — 5000 top-level defs
must all be extracted (RED on the old 4096 cap, GREEN after). Full suite 5748/0,
no ASan/UBSan reports on the new alloc.
Root cause was pinned by reproduction (constrained-stack + ASan), refuting earlier
grammar/scanner hypotheses — extraction and the vendored tree-sitter tree ops are
otherwise iterative. The separate error-recovery assertion mode
(symbol<token_count) was not locally reproducible and is addressed by the
forthcoming general crash-containment layer.
Refs #668
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel
174e56b447