macOS answers ESRCH to setpgid and getpgid for our own unreaped child from
the moment it starts to exit, before waitid can report it. a5c7ae97 accepted
a child that had already exited, probed with waitid(WNOHANG|WNOWAIT); in the
window before the child is reapable that probe said "not exited", and the
spawn was reported as SPAWN_FAILED errno 3. macos-15-intel CI kept hitting it
in a different Git-backed fixture on each run; with the new diagnostics it
named itself: `git config --local core.protectNTFS false`, outcome=5 errno=3.
A child of ours that setpgid/getpgid cannot find is not running: it has
exited or is exiting. The parent now waits until it is reapable (WNOWAIT
leaves it for the supervision, which reaps it with its real exit status);
the wait ends with the child's exit. EACCES/EPERM keep the group check.
A test seam makes the parent observe the exiting state (ESRCH, not yet
reapable): subprocess_run_child_exiting_at_group_check is SPAWN_FAILED
without this change and exit status 3 with it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Seeding drops a hunk whose lines are all comments. The check took any line
starting with `*` for a block-comment line and any line ending in a block
comment's close for a comment, so a dereference (`*out = f(a, b, true);`)
and code with a trailing comment counted as comments and their edits seeded
nothing.
On the oracle, PR #2158 changed two `*out_* = extract_route_path_from_args(...)`
lines in scan_route_annotations. Both hunks were dropped, the function was
never a seed, and its coverage id never reached the changed set. The six
artifact_reconcile_* tests that execute it, recorded as complete coverage rows
of the merge-base map, were missed (6 per job, two jobs: the only misses
among 191 jobs). With the check fixed, all six are selected for COVERAGE
again; until now the new graph walk had reached them as STATIC only by
chance.
A line now counts as a comment only when it is nothing but comment: blank,
`//`, a block opened with nothing after its close, `*` followed by
whitespace or the end, a lone close, or the closing line of a block opened
earlier. `*name`, `**`, `*(` and code before a trailing comment are code.
When unsure the check says code, which only seeds more.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/security-audit.sh blocks any `[^a-z_]system(` in a source file that
is not on the allow-list, comments included. A comment in pass_spawns.c
spelled a C call of `system(...)`, so security-static failed on the SPAWNS
pass, which never calls it. The comment now names the library function
without the call syntax; no allow-list entry, since there is no call.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A tree with a file `A` and a directory `a` cannot be materialized on a
case-insensitive volume. On Windows, creating the file over the existing
directory answers ERROR_ACCESS_DENIED, not ERROR_FILE_EXISTS, so the pinned
tree reported an I/O failure where tree_d_native_collision_and_exact_spelling
expects CBM_PINNED_TREE_COLLISION (observed: error 5 on the create).
An access-denied create whose attempted name is present is now the same
collision; one whose name is absent stays an I/O failure.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The pinned tree requires its private parent to be owner-only: a protected
DACL, the current user as owner, one full-access grant to that user. It
accepted only the single-entry encoding (OI|CI). cbm_mkdir_p stamps a fresh
directory with exactly that grant, but written with generic rights, and
Windows stores it as an effective entry plus an inherit-only one
(icacls: user:(F) and user:(OI)(CI)(IO)(F)). The work directory production
creates was therefore refused ("pinning HEAD failed: protected owner-only
DACL policy mismatch"), and every Windows selection ran everything.
Both encodings are now accepted: one or two entries, each an allow entry for
the current user with full access and only inheritance flags, at least one
applying to the object itself. Any other principal, deny entry, partial
mask or flag is still refused.
tree_e_windows_split_owner_dacl_is_owner_only creates the parent with
cbm_mkdir_p and pins on it (red before this change), pins on the split form
written explicitly, and is refused once the inherit-only entry names Everyone.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Git facts ask `git rev-parse --path-format=absolute` for the top level, the
git directory and the common directory, and use them for native file checks
(grafts, shallow). MSYS2 git -- the git the Windows legs install -- answers in
its own namespace (/c/Users/...), which is not a Windows absolute path, so
every git-facts open failed with "Git metadata path is not absolute" and the
engine ran everything (41 test_impact_git failures).
The drive form maps one to one onto C:/Users/... (Cygwin: /cygdrive/c/...),
and is translated. Any other path of that namespace (/tmp, /usr) needs the
runtime's mount table and is still refused.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
After posix_spawn the parent re-asserts the child's process group. A short
command can exit before the parent gets there; macOS then answers ESRCH for
the exited, unreaped child to both setpgid and getpgid, and the run was
reported as SPAWN_FAILED (errno 3). macOS CI read `git write-tree` that way
under load (macos-15-intel and the LeakSanitizer leg).
The group was already set before exec, by the spawn attributes or by the
child itself, so an exited child is accepted when waitid(WNOWAIT) shows it
waiting to be reaped; the supervision then collects its real exit status.
A test seam holds the parent until the child has exited, which makes that
order deterministic: subprocess_run_child_exited_before_group_check is red
without the fix (SPAWN_FAILED) and green with it, on macOS and Linux.
The test file is also reflowed to the project's clang-format, as on main.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The test-impact engine selects tests by walking the graph, so it can only
be as right as the graph. Three gaps made it wrong:
One identity for a definition's mutually exclusive variants
- A definition written once per #if/#else branch, per #[cfg], per
platform file (foo_linux.c / foo_win.c, platform directories) or per Go
build tag is ONE node. The graph buffer keeps every variant's span in a
`variants` node property (definition labels only), so a change in any
variant maps to the definition.
- Extraction now sees definitions inside conditional blocks where it
skipped them: C++/C#/ObjC class members under #if (one cursor walk,
conditional block types per language in lang_specs), Elixir defs under
if/unless, Fortran .F90/.F95/.F03/.F08, Rust #[cfg] twins (no more
`name#cfg(...)` QNs that lost every call), C platform files and
directories (non-static file-scope functions).
- Haskell: the vendored scanner reads only an #if group's first branch.
The grammar stays upstream; every further branch is parsed as its own
same-length view (cpp_branch_views.c) and adds what the raw parse
missed. haskell/bytestring: no new partial parses, +27 functions,
IMPORTS 322 -> 329, 131 definitions with variant spans.
- Incremental closure repair re-parses every file of such a definition
when one of them changes (variant partners through DEFINES), so a
repair equals a full index.
No cross-language name guesses
- A name-only resolution (unique_name, suffix_match, qualified_suffix,
field_type_hint, and same_module across files that only share a path
stem) never binds a symbol another language defines.
- Bash imports are the files a script sources: every top-level command
was read as an import of its name (`set -e` imported "set", bound to a
TSX method in 137 scripts here).
- This repository: cross-language CALLS 5,916 -> 5,091; the remainder are
C/C++ pairs, builtins and same-stem module collisions (recorded).
SPAWNS edges (per language, like the HTTP client patterns)
- A call that starts a program (subprocess.run, exec.Command,
child_process.spawn, posix_spawn, system, Process.Start, Command::new,
... about 30 languages, through the file's own imports) gets a SPAWNS
edge to a Process node named after the program (literal first word, or
<dynamic>) instead of a CALLS guess. No target resolution. Both
resolvers ask at the same point (parity test).
The engine follows the graph
- Seeds map changed lines through every variant span; a test written
once per #if branch is one test; when a change reaches a non-test
`main`, the tests that spawn processes are reached too (the process
runs main, and main reaches everything).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
PR CI on the first push of these commits found what no local leg had
built yet:
- test_impact_result_validate.c: initialize the count GCC's
-Wmaybe-uninitialized flags (-Werror on every Linux build: smoke,
shadow, analyze).
- tests: split one-line `if` chains GCC reports as misleading
indentation, and drop a redundant `= {0}` it reads as missing braces
(the GCC test runner did not compile).
- tests/test_test_impact_engine.c: realpath() into a PATH_MAX buffer;
glibc's _FORTIFY_SOURCE aborts on anything smaller, as
test_index_resilience.c already notes.
- test_impact_git.c: pass --no-lazy-fetch only when the git executable
takes it (git 2.44+, probed once per session with the option on
--version). Older git rejected it and every engine command failed;
Ubuntu 24.04 ships git 2.43. `-c protocol.allow=never` already turns a
lazy fetch into an error there, which is what the option asks for.
- test_impact_engine.c: the selection and publish job threads release
their thread-local node-type bitset cache before they end, as the
parallel workers and the MCP index thread do. LeakSanitizer reported it
(488 bytes in 13 allocations, test-diag); the Linux arm64 ASan leg
reproduced exactly that and is clean with the fix.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Ten files changed on both sides; every conflict was a union of independent
additions:
- workflows (_lint.yml, _test.yml): keep the branch's lane `if:` and main's
pinned ubuntu-24.04 runner with its reason.
- internal/cbm/cbm.h: main's callable-identity and module-doc fields plus the
branch's configured-test fields.
- Makefile.cbm: both new test files; prod_lsp_all.o keeps main's
.build-config dependency and the branch's coverage flags.
- scripts/test.sh: main's setup-scripts contract becomes Step 0y3, ahead of
the branch's 0z series and the contracts-only exit.
- src/foundation/subprocess.c: main's cbm_posix_close_fds_from (it closes the
error descriptor too).
- src/mcp/mcp.c: main's repository default base for detect_changes, with the
early returns released through the branch's detect_release.
- src/pipeline/lsp_resolve.h, tests/test_main.c: both includes, suites and
start-up steps (main's git environment reset first).
- scripts/memory-core-baseline.txt: recounted on the merged tree and lowered
(mcp.c 773, config_toml_edit.c 73).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A main commit can now publish its test-impact bundle — the commit's frozen
graph, its per-test coverage map and receipt.json binding both — and a PR
selection given the bundle of its merge base, verified by the caller,
builds its graph incrementally from the bundle's graph and narrows suites
by the admitted map. Without a verified bundle nothing changes: the graph
is built from scratch and every reached suite runs whole.
Producer and consumer:
- `codebase-memory-mcp test-impact publish|select` (in process, no
daemon): publish builds the bundle of the checked-out commit, from the
previous bundle's graph when that bundle verifies; select is the
selection with optional --artifact/--artifact-commit/--artifact-verified.
The MCP surface is unchanged.
- receipt.json binds the exact imported graph (graph digest), the graph's
content and topology, the map's three files and a compatibility digest
(platform label + the bytes of test_impact.coverage.compatibility_paths).
- A bundle is used only when the caller verified its source, it is the
merge base's own, and its imported graph is the one the receipt names;
the map is admitted by the existing receipt policy (source, ancestry,
content, age, compatibility, metadata). Refusals become rejection
reasons in the answer's receipt.
- C_eff: a changed function the map knows contributes its own id; one it
does not know (new, platform-only, a type or macro) contributes its
nearest mapped callers on the static inbound closure; deleted names
contribute every mapped function of that name.
Graph:
- the frozen build accepts a base candidate and repairs it through the
incremental closure route; any doubt declines to the full build. Pinned
to one worker like the full frozen build.
- incremental closure repair skipped the interprocedural complexity pass:
re-parsed functions lost transitive_loop_depth/recursive and callers kept
stale values. One traversal now serves the in-memory pass and a new
store-level recompute that runs before the importance rescore.
- test-impact's frozen builds use a fixed display Git context: the per-run
pin path and the commit ids made two builds of one tree differ and made
every commit a semantic-input change, so repair always declined.
- graph content and topology digests (store_content_digest.c): the exact
graph digest binds one database (ids, db_uid, generation), never equal
across builds; the content digest keys rows by what they say, the
topology digest by what selection reads.
Measured on this repository (frozen publish, macOS arm64): a two-file C
change repairs in 18-20 s instead of 75-80 s with the topology of the full
build; a docs-only change repairs with identical content; selections with
and without the bundle are identical on all seven replayed pairs
(two repaired incrementally, five declined to the full build). Recorded, not
fixed: closure repair still differs from a full build in edge PROPERTIES
(argument quoting and key order, CONFIGURES args) and in the corpus-wide
SIMILAR_TO/SEMANTICALLY_RELATED edges; selection reads neither, so the
topology digest is the incremental-equals-full check and the receipt
carries both.
Tooling: scripts/test-impact/coverage-merge.py builds a push's incremental
coverage map (re-run suites verbatim, the rest carried with ids remapped
onto the new image's function table), with its contract as test.sh Step
0z7.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
One request pins the change (merge base .. HEAD), builds the graph and the
runnable test registry of exactly the HEAD snapshot, seeds the graph from
the change, walks what depends on it and answers which tests to run
(cbm.test_impact). Missing or doubtful evidence never narrows: it becomes a
run-all answer that names why.
Snapshot (classified ledger -> frozen graph -> registry):
- one shared language classifier for discovery and pinned blobs (raw bytes
on every platform); the classified ledger feeds the frozen build, which
indexes exactly the listed files and records partial parses as per-file
PARSE_GAP instead of refusing the graph;
- the pinned tree skips symlinks and submodule links (counted), as
discovery never indexes them;
- the test model scopes uncertainty to the suites of its file (they run
whole), reads line splices where they occur, treats a mid-file
`#ifndef X` + `#define X` as an ordinary condition, and no longer reads a
plain identifier named like a macro as an invocation. On this repository
the registry is now narrowable and matches a text census exactly
(8,365 cases, 141 runner suites).
Seeding and reach (port of the measured replay, rule B):
- hunk -> overlapping definitions, comment-only hunks dropped, pure
deletions -> enclosing definition, no-definition hunks -> whole file,
deleted names found by text, header changes propagated through the
include closure and the headers' macros (pastes included), file-level
escalation for unclassifiable header lines;
- inbound fixpoint walk over CALLS, ASYNC_CALLS, USAGE, CALL_REFERENCE,
READS (+ routes); cases, suites and the runner's entry points are sinks;
a reached runner main runs everything (RUNNER_REACHED);
- a parse gap whose text names a reached definition reaches the
definition around it (or its file);
- macro-registered tests run their suites whole; cases and suites map to
graph nodes by their start line.
The line patterns reproduce the reference regexes exactly on 1,383,850
lines of this repository and 37,167,926 kernel lines; on 24 replayed
changes the engine selects at least what the reference selected (13
identical, the rest only conservative supersets on the same graph).
Also:
- configured test declarations that cannot be mapped degrade their file
(no configured roles, a test_declarations diagnostic) instead of
aborting the whole index;
- the selector keeps other runners' suites out of the selection, and the
result codec judges the model by narrowability like the selector;
- a change to .codebase-memory.json runs everything (built-in rule);
- .codebase-memory.json with the c-cbm preset and this repository's rules.
Gate fixes (suites, lint-ci, Windows cross-compile):
- the request runs on its own thread with a default-size stack (daemon
tool calls run on 256 KiB threads, where the in-process build died);
- a request-supplied config_path is no longer followed by an unlink of an
uninitialized path buffer;
- the inventory's frame-local reader no longer stays reachable from the
caller's source after the filter call;
- FILE_ID_INFO needs _WIN32_WINNT >= 0x0602; the pinned-tree header pins
it before any system header (toolchains defaulting lower failed);
- memory-core ratchet: the win_utf8 converters gain core-backed
`_in(cls, ...)` forms over one implementation (the legacy names keep
the C library for unmigrated callers), the Git facts module uses them,
and the Windows spawn path releases everything through one failure
exit; pipeline, extraction and config allocations that this work
touched moved to the core in matched pairs; baselines tightened;
- the branch's C files formatted with the project's clang-format.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two pairs of pull requests merged cleanly but left main unbuildable:
- #2447 added a route_mount parameter to emit_route_registration and
handle_route_registration, while #2464 added a new call to each with
the old argument list. The cross-language refused-route helper from
#2464 now takes route_mount and passes it on, and the pass_calls.c
call passes it too, as every other call site does. A route minted for
a refused binding therefore gets the same Laravel api mount as any
other route from that file.
- #2446 added a caller of is_project_db_file, which #2526 replaced with
cbm_is_project_index_db. The new caller uses the replacement, so the
owning-project lookup accepts the same file names as every other
project scan.
Verified: build/c/test-runner and the server binary build again; suites
edge_types_probe, mcp, cli and pipeline: 1081 passed, 4 platform skips.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Preserve the current selector, frozen-input inventory, result codec,
coverage collection, runner filtering, and isolated Windows venue work.
Component-level regression proofs are recorded in the PR handover.
The live classified provider, public execution integration, artifact
admission, fresh oracle/shadow evidence, and full platform gates remain
incomplete. This checkpoint does not enable or claim verified narrowing.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Preserve the original staged graph and LSP work separately from the
smart-CI continuation. This is an unfinished checkpoint; final
integration and cross-platform verification remain outstanding.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Use the shared exact-name database predicate for all five MCP cache
scans, including reindex root ownership. Preserve exclusions for the
two internal stores and cover listing, lookup, hints, and root reuse.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Describe the current filters together when a structural search returns
no results, instead of blaming the label or name pattern.
Refs #1366
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Let each test server use a private scratch parent so cleanup checks cannot
count other processes' temporary files. Observe live scratch in the
hook-rejection fixture, preserve deadline assertions, and cover scratch
creation failure with a valid path filter for leak-sanitizer validation.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Require --delete-indexes for automatic or noninteractive index removal.
Keep the separate terminal prompt with a default of retaining indexes,
and preserve dry-run and guarded activation ordering.
Refs #2200
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel