scripts/add-lang/verify-extraction.mjs ran execFileSync('codegraph', ...).
On Windows, npm installs codegraph as a .cmd shim (so do npm link and
local-install.sh), and Node neither finds a .cmd by bare name nor runs
one without a shell. The check always died with "spawnSync codegraph
ENOENT" (exit 2), and /add-lang's bench.sh read that as broken
extraction and skipped its A/B.
The script now asks, in order:
- CODEGRAPH_BIN, when set. A .js, .mjs or .cjs path runs with the
current node; anything else runs directly (a .cmd or .bat through
cmd.exe).
- On Windows, the codegraph Windows itself would pick: the first PATH
folder holding codegraph under one of PATHEXT's runnable extensions.
A .cmd or .bat runs through cmd.exe (shell: true) as one command
string. Each token is quoted for the C runtime and caret-escaped for
cmd, so spaces, &, parentheses, a %NAME% and a trailing backslash in
the repo path arrive intact. An .exe or .com needs no shell.
- Otherwise codegraph by bare name, exactly as before. On macOS and
Linux without CODEGRAPH_BIN, the call and the output are unchanged.
bench.sh takes CODEGRAPH_BIN too. It makes a path absolute (indexing
runs from inside the repo), exports it so verify-extraction.mjs reads
the index with the same build, and runs its own --version and init calls
with it. Without it, bench.sh behaves exactly as before. The A/B's MCP
server stays run-all.sh's own CG_BIN.
Validation (Windows 11, Node 22.22.0):
- New __tests__/add-lang-verify-extraction.test.ts covers CODEGRAPH_BIN
with a .js stand-in (all platforms), a .cmd shim on PATH with an
awkward repo path (Windows only), and the bare-name call (macOS and
Linux only). With main's script both tests that run here fail ("could
not read codegraph status"); with the fix, 2 pass and 1 is skipped.
- The fixture from PR 1246's review (a codegraph.cmd on PATH): main
exits 2 with ENOENT; the fix reads the status.
- 13 scenarios with npm's own generated shims, and a real index in a
folder named "lua repo & co (x86) 100%": main fails every one that
needs the shim or CODEGRAPH_BIN, and the fix passes them with the
exact argv reaching codegraph. With no codegraph anywhere, both exit 2
the same way.
- bench.sh with the A/B stubbed: with npm link's shims on PATH, main's
copy skips the A/B (verify exit 2) and the fix reaches it (PASS).
CODEGRAPH_BIN as an absolute or a relative path reaches it too, and a
missing one stops with a clear message.
- With process.platform reported as linux or darwin, main's script and
the new one make the same execFileSync call and print the same output.
Raised by PR 1246 (@bslatner).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Root cause: scripts/add-lang/check-grammar.mjs ran the parse loop in its own
process and printed "RESULT: PASS" as soon as the loop finished. A grammar
can finish every parse and still take the process down after that. With the
issue's vim grammar (tree-sitter-wasm 2.0.4) on Node 24, V8's optimizing tier
aborts the process from a background compile job ("Fatal process out of
memory: Zone") after PASS is printed, so the verdict and the exit code
disagree. The check also ran without the V8 flag indexing always uses
(--liftoff-only), so it judged grammars in a setup codegraph never runs them
in, and an iteration count of 0 or a word passed without a single parse.
Fix: the script runs the grammar in a child process, the way indexing does,
and prints the verdict only after that child has exited.
- The child gets indexing's V8 flags (WASM_RUNTIME_FLAGS) and loads every
vendored grammar beside the one under test, like a polyglot index.
- It parses the sample with one reused parser, walks every node of each tree
and frees it, as extraction does. Every parse of the same sample must give
the same tree.
- It then parses the sample repeated to 1 MiB, the most indexing reads from
one file, within the 10 s indexing gives one file. Only a crash or a hang
counts there.
- The parent fails the grammar when the child dies (any exit code or
signal), runs past a timeout (CODEGRAPH_CHECK_GRAMMAR_TIMEOUT_MS, 120 s by
default), or reports ERROR trees or differing trees. An iteration count that
is not a whole number above 0 is refused (exit 2).
The add-lang skill now says what counts as a FAIL.
Validation:
- New __tests__/add-lang-check-grammar.test.ts. A preload marks the process
that runs WebAssembly, records its flags, and on request kills it or makes
it hang at exit, after the parses. With main's script 4 of 5 tests fail:
PASS printed for a process that then died, a hang until the test's 60 s
limit, no --liftoff-only in the parsing process, and "0" passing. With the
fix all 5 pass. The fifth (the tree-sitter-wasms Lua ABI-13 grammar fails
with ERROR trees) passes on both.
- The issue's vim grammar (sha256 df3906a0, the issue's file) on Node 24.21.0:
main's script printed PASS and then died with the Zone OOM, exit 3, in 3 of
3 runs. The new check runs it with --liftoff-only, as indexing does, where
it parses cleanly, 1 MiB included, and passes in 3 of 3. A copy of the new
script with the flag withheld fails it in 3 of 3: "the parses finished, but
the process running them then died (exit code 2147483651, 0x80000003)".
- Old vs new on the 19 vendored grammars with real samples, plus the
tree-sitter-wasms Lua and the vim grammar, on Node 22.22.0 and 24.21.0
(84 runs): the same verdict every time, except vim on Node 24, where main's
PASS came with exit 3. No grammar's parses diverged.
Fixes#2464
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
