Three changes to how PR CI consumes the test-impact selection (user
decisions R150 and R151, 2026-11-11).
Fork PRs. select-tests ran only when vars.TEST_IMPACT_MODE was `gate`, and
GitHub passes no repository variables to a pull_request run from a fork:
all 24 fork runs in the last 40 PR runs skipped the job, maintainer-started
ones included. The condition is now (vars.TEST_IMPACT_MODE || 'gate') ==
'gate'. Setting the variable to anything else (e.g. `shadow`) still turns
the gate off for same-repository and Dependabot PRs; for fork PRs the off
switch is the default in the expression on main. No pull_request_target,
no new permissions, no secrets.
Base tooling. select-tests built and ran the PR checkout's own engine,
test-impact-select.sh and selection.py, so a change to them could narrow
its own tests. The job now checks out the merge base (HEAD^1 of the merge
ref) into a worktree, builds that engine and runs that copy of the script
with --repo pointing at the PR checkout. test-impact-select.sh separates
the tooling (its own checkout) from the repository under test (--repo).
Until this lands, a PR's merge base has no --repo: the job then fails and,
as before, every test runs.
All legs. Only the Linux x86_64 test leg consumed a selection. A narrowed
answer now also produces portable/ (selection.py --whole-suites with the
platform-specific suites of the base and HEAD), and the arm64 and macOS
test-unix legs and the Windows leg run that view: platform-neutral suites
as selected, every platform-specific suite whole. No portable view, no
narrowing there. Sanitizer, TSan, MSan, diag and LSan lanes still run
every test. The shard harness already slices the selected list, so the
sharded arm64 leg and the shard-completeness proof need no change.
docs/SMART_CI.md and README.md follow; tests/test_lane_wiring_contract.sh
pins the new wiring.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
selection.py learns two things the CI needs to narrow test legs other than
Linux x86_64, where the coverage map is recorded:
- `platform-suites --repo DIR --rev REV...` lists the platform-specific
runner suites: those whose defining test file holds a SKIP_PLATFORM( call
or an #if/#ifdef on an OS or architecture macro (the two markers
scripts/check-no-test-skips.sh names for a genuinely platform-specific
test), and those tests/test_main.c registers under such a conditional.
Every revision given is read and the results are united, so a change
cannot declassify a suite by dropping its conditionals. Any git failure
exits 1.
- `filter --whole-suites FILE` writes the view for such a leg: every listed
suite runs whole there, its per-test tokens collapsed to the suite; a
listed suite the answer did not select is added and marked optional (a
build that does not register it drops it with a notice, as a full run
never runs it); the answer's own required suites stay required. An
unusable list runs everything.
Nothing consumes the new view yet. Contract cases in
tests/test_test_impact_selection.sh.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Adds a native Windows guard for #654: a successful MCP initialize +
tools/list exchange, run against an isolated cache and runtime directory,
must leave nothing on stderr but the server's own structured log lines.
- tests/windows/test_mcp_stdio.py: the guard. The English and Spanish
path-not-found messages reported in #654 are named cases; any other
non-empty stderr line that is not a structured "level=<name> msg=<event>"
record fails as well, so the same leak on a machine in another language
is caught without listing its translation. The log format is pinned to
text so an inherited CBM_LOG_FORMAT=json cannot change the verdict. Cache
and runtime live under one temp root, so both are removed with it, also
on failure, after the daemon is stopped.
- tests/windows/mcp_stdio.py: close() joins the stderr reader, and the new
stderr_complete() reports whether the reader reached EOF. The guard reads
stderr only after it has; a reader still running after the server exited
is a SETUP failure (exit 2), never a verdict on a partial capture.
- scripts/test-windows.ps1: registers the guard in the green guard list
and documents it in the header.
#2213 was an earlier duplicate of the same check.
Refs #654.
Distilled from #2217.
Co-authored-by: lorenzozanee <wyz0707@proton.me>
Co-authored-by: Shinken009 <182390694+Shinken009@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The vendored vbnet scanner lexes a whole VB XML literal as one external
token, and the parser offers it a '<' wherever a declaration or member
starts. There, in valid VB, '<' opens an attribute block
(`<Serializable()> _`), never an XML literal. The scanner nevertheless
took `<Serializable` for an element start and read the rest of the file
looking for a closing tag before giving up: O(rest of file) per
attribute, O(n^2) per file. Generated proxies and designer code put an
attribute on every member: 2,500 attributes in a 77 KB file advanced
38.5 M characters, and a 1.35 MB synthetic file took 21 s to parse.
The scan now fails at the first character well-formed XML cannot
contain: inside a tag, outside a quoted value, anything other than name
characters, blanks, '=', quotes, '/>', '>' and '<%='; in element text, a
'<' that starts no tag, closing tag, comment, PI or embedded expression,
and an '&' that starts no reference. No valid XML literal lexes
differently: parse trees are byte-identical before and after on all 25
sample and synthetic files tried, XML literals of every shape included.
The 1.35 MB file now parses in 0.08 s.
Residual, recorded in MANIFEST.md: an attribute without an argument list
(`<Fact>`) is a well-formed start tag, so its scan still runs until a
character that cannot be XML, or the end of the file.
The change is kept as tools/tree-sitter-vbnet/patches/
xml-literal-tag-syntax.patch (CodeGraph's scanner 8334fdb -> b3473d1)
and in MANIFEST.md's local-patch table. The new
complexity_vbnet_attribute_lexing_is_linear counts the bytes the lexer
pulls through a chunked TSInput for k and 2k attribute-decorated
classes: ratio 3.99 before the fix, 2.0 after.
Refs #1178, #1179
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Vendors govindbanura/tree-sitter-vbnet (MIT, (c) 2025 Govind Banura) at
538b7087bf80 (tag v0.1-beta, upstream HEAD) with CodeGraph's real-world
VB.NET patch (colbymchenry/codegraph docs/grammars/tree-sitter-vbnet.patch
at fe7542b6f6a4, MIT, (c) 2026 Colby Mchenry), as a self-maintained fork
in tools/tree-sitter-vbnet/. This is the precondition for VB.NET
extraction (#1179, folding in #2885); it registers no language yet.
Why this grammar: it is the licensed one. CodeAnt-AI/tree-sitter-vb-dotnet,
which #1179 was built on, carries no license and cannot be vendored.
Why the patch: upstream checks in grammar.js only and does not parse the
code VB.NET projects are made of. Unpatched, the Single literals in every
WinForms .Designer.vb (New SizeF(6.0!, 13.0!)) and body-less MustOverride
members are ERROR nodes. The 2026-10-06 revision of the patch is pinned
because it also keeps names that begin with a keyword whole: the earlier
revision (the one #2885 carried) turns `Sub NewItem()` into an ERROR and
`Public SharedCache As Integer` into a Shared field named Cache.
Provenance by hash: the patch file is byte-identical to CodeGraph's (blob
37918d81); upstream grammar.js at the pin is the patch's pre-image
(21c8523), and tools/tree-sitter-vbnet/grammar.js minus its 41-line
provenance header is the post-image (c9c1d85), as is scanner.c (8334fdb).
parser.c is generated here with tree-sitter-cli 0.25.10 (ABI 15); two
independent generations are byte-identical. No upstream script, package
hook or binary was run or vendored.
- internal/cbm/vendored/grammars/vbnet/: the generated parser.c,
CodeGraph's external scanner.c (two stateless tokens), the tree_sitter/
headers, upstream's MIT LICENSE byte-identical, and CodeGraph's MIT text
as LICENSE-codegraph: the scanner and the patched rules are CodeGraph's
work, so its notice has to ship with the binary too.
- internal/cbm/grammar_vbnet.c: compiles and links tree_sitter_vbnet. No
CBMLanguage, extension or alias routes to it yet; extraction adds them,
with `vb` and `visualbasic` meaning VB.NET (VB6 keeps its own name).
- MANIFEST.md (fork note, forks table, 163 grammars, ABI recount),
THIRD_PARTY.md (fork list and a vbnet section; the stale "160
pre-generated parsers" now reads 163), scripts/vendored-checksums.txt,
and the FORKS entry in scripts/audit-license-provenance.py.
- README.md and the Chocolatey nuspec say 163 languages: the
language-count contract defines the count as the vendored grammar
directories.
- tests/test_parse_coverage.c: real VB.NET (a WinForms Designer partial
class, Handles, Inherits/Implements, MustOverride, #Region, Property,
keyword-prefixed names) parses with no ERROR node. The case fails on
unpatched upstream (8 ERROR nodes) and on the earlier patch revision (1).
Refs #1178, #1179, #2885
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add .github/workflows/freebsd.yml, a standalone workflow triggered only
by workflow_dispatch. It builds the UI-embedded binary inside FreeBSD
15.1 and 14.4 x86_64 VMs (vmactions/freebsd-vm, full-SHA pin to v1.5.9)
with the canonical scripts/build.sh and runs the binary with --version.
It declares read-only permissions, uploads and publishes nothing, is not
part of ci-ok and is not called by release.yml or dry-run.yml, so a
FreeBSD red never gates a pull request or a release.
scripts/env.sh now detects FreeBSD and selects gmake there (BSD make
cannot read the GNU make syntax of Makefile.cbm) and make everywhere
else; the environment summary line reports the choice. scripts/build.sh
runs $MAKE at all three build call sites. macOS, Linux and Windows still
run make, so their builds are unchanged.
Distilled from #1422 by PR9000: the release matrix, the x86_64-host
choice, the aarch64 exclusion, the vmactions VM approach with a SHA pin,
fail-fast: false and the gmake detection all come from that pull
request, which built on the FreeBSD groundwork of #1093, #1423 and #1467.
Co-authored-by: Pedro Ramos <131530838+pr9000@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Bring the branch up to main e0c7a1da6c
(#2395), which includes #1724 (index resource worker and publication
fence, 008b6c3585).
Conflict resolved (1 file):
- docs/CONFIGURATION.md: both sides appended rows to the "Current keys"
table at the same spot. Resolved as main rows index_max_rss_mb and
index_max_duration_seconds first, then the branch row install_hooks
and its paragraph that closes the table. No conflict side was
concatenated as-is; the branch paragraph must follow the last table
row.
Auto-merged without conflict: README.md, src/cli/cli.c (CONFIG_KEYS
now holds the new index_max_rss_mb / index_max_duration_seconds keys
and install_hooks), src/main.c (worker publish-fence argv next to the
--hooks/--no-hooks help line).
Check: for all seven branch files, the zero-context patch-id of the
merged tree against main equals the branch own change. Semantic check:
the branch re-signs no function; main re-signed the daemon worker start
callback and the supervisor result struct, which the branch does not
use. No call site adapted.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main 9c19b5dd.
Two same-spot insertion conflicts, each resolved as the main version
of the file plus the own diff of this branch:
- src/daemon/application.c: the job struct gained watch_owners on main
and request_started_ms plus the request facts here; both kept.
- tests/test_daemon_application.c: main added the #1818
autoindex-skipped helpers where this branch added the worker resource
failure test; both kept, the test placed after the doc_candidates
test.
Everything else merged cleanly. src/main.c changed on both sides: main
(#2838) lowers the worker CPU priority inside the worker role block,
and this branch strips the publication fence flag before that block
and installs the fence after the worker role options; the hunks are
disjoint and both are kept. tests/test_pipeline.c gained the Next.js
route tests (#2747) on main and the publication gate test here, also
disjoint. No call site needed adapting: no new main code calls a
function this branch re-signed, and main re-signed none that this
branch calls.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer refresh of #1862 (persistent install-hook opt-out, direction
decided in #753) onto main a5b248f4. The first refresh is 2577a86a and
carries the semantic fixes for shell-free Claude hooks.
Textual conflicts, all resolved by keeping both sides:
- src/cli/cli.h: main added CBM_CONFIG_DOC_CANDIDATES on the line where
the PR added CBM_CONFIG_INSTALL_HOOKS. Both defines are kept, in
alphabetical order.
- src/cli/cli.c CONFIG_KEYS[]: main added the watch_non_git row where the
PR added install_hooks. Main row stays first, followed by the
install_hooks row.
- src/cli/cli.c Codex hook install: main (#2719) declared pretool_written
in front of the hooks.json branch the PR prefixed with its --no-hooks
removal branch. The declaration stays first, then the PR removal
branch, then main hooks.json and config.toml upsert branches unchanged.
Check: for every file the PR touches, the added and removed lines of the
merged tree against main equal those of the PR against its old base.
No compile-level semantic conflict: no function the PR calls changed its
signature, and the PR re-signs no function main calls. The new Codex
PreToolUse hook from #2719 is gated by the opt-out in the next commit.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to date with main (a5b248f4). No conflicts.
scripts/test.sh keeps Step 0e4 next to Step 0e3; the gate script and
its test are unchanged by main.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/test.sh lists every shell contract test explicitly and does
not glob tests/, so tests/test_lint_mem_gate.sh never ran on any leg.
Register it as Step 0e4, next to the other gate contract (Step 0e3),
in the same two-line shape as its neighbours. Step 0 runs in every
default scripts/test.sh leg (macOS, Linux, Windows) and in the PR
contracts-only job. The wiring, Windows included, is a maintainer
decision (R1).
Make the test portable to the Windows leg. There python3 is a native
MSYS2 build: MSYS converts path arguments for it, but never the
diagnostic text on stdin, so the absolute fixture path inside the
finding does not resolve there, and a C:/ path would not parse (the
file field of FINDING_RE ends at the first colon). The gate now runs
from inside the fixture directory, in a subshell, with relative paths.
CRLF from a native python3 is stripped from the --hash output, the
whitelist is written once the hash is known (no placeholder rewrite),
the gate exit status is asserted, and every failed assertion is
reported together with the gate output.
Refs #2442
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings #2198 (benchmark harness runtime isolation, #1696) up to date
with main a5b248f4.
No conflicts. Of the files this branch touches, main changed only
scripts/test.sh since the merge base 8530c8cd: it adds Step 5-alloc
(production allocator contract), far from this branch's Step 0i3,
which is unchanged. scripts/test-runtime.sh, which the harnesses and
their contract test source, is unchanged on main. No caller of either
harness outside docs/EVALUATION_PLAN.md exists on main.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
With CBM_BENCH_KEEP_RUNTIME=1, bench_finish ran `daemon stop` with its
output and exit status discarded and wrote runtime-root.txt regardless.
`daemon stop` refuses while a committed client still uses the daemon and
fails when the daemon does not answer; the handoff then promised a
stopped daemon that was still running.
A refused stop now prints the explanation of the product, writes no
handoff, keeps the root without cleaning it up (the daemon may still be
using it) and exits non-zero. A stop with nothing to stop still exits 0,
so a clean run is unaffected.
Section 7 step 8 of the evaluation plan follows the same rule: the kept
root is removed only if its daemon stopped, and is kept with a message
otherwise.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both harnesses discarded the output of `daemon start`, so a refused
start was reported only as "private daemon did not start", the symptom
without its cause. The output now goes to a run-private file next to the
index stderr, the shape both scripts already use for the index call, and
is printed under "--- daemon start output ---" when the start fails.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
benchmark-index.sh removed the runtime-root.txt of an earlier run only
after the repository path, the private runtime and the temp directory had
been set up. A run that failed in one of those steps exited non-zero but left
the previous handoff in the reused results directory, and section 7 of
the evaluation plan sources that file unconditionally, so the graph
session would be pointed at a root that step 8 has already removed.
The results directory is now created and the stale handoff removed right
after argument parsing, before any setup step that can fail.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to current main (it was 2061 commits behind) so
the new Step 0x2 runs inside the current scripts/test.sh. No conflicts.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/test.sh lists every shell contract test explicitly and does
not glob tests/, so an unlisted test never ran on any leg. Register
tests/test_install_android_guard.sh as Step 0x2, right after Step 0x,
in the same two-line shape as its neighbours.
Give the two PATH stubs (uname, curl) the same shebang every other
stub under tests/ uses, so they are proper executables on every leg
instead of relying on the shell fallback for files without one.
Refs #1202
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
tests/test_pipeline.c: main and this branch added tests at the same spot;
both are kept unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Smoke check 8d required Claude's PostToolUse matcher to be exactly Read. The
drift note widens it on purpose to Read|mcp__.*codebase-memory-mcp__.*, so a
graph tool call can reset the search count, and the macOS PR smoke failed on
it. 8d now requires exactly the new matcher, and besides a Search matcher it
also refuses a catch-all ("*" or empty) matcher, as its comment promised.
Checked against real installs: the previous binary's Read-only settings fail
8d, the new install passes, and the new install plus a catch-all PostToolUse
entry fails.
Refs #509
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
bench_finish keeps the private root on a zero exit status, but the index
call swallowed its failure (`|| echo '{"error":"index failed"}'`), so a
refused index exited 0 and, with CBM_BENCH_KEEP_RUNTIME set, kept an
empty root and wrote runtime-root.txt for an index that does not exist.
benchmark-index.sh now records the CLI's exit status, keeps 00-index.json
valid JSON on failure ({"error":"index failed","exit":N}), and exits 1
once every per-run file — the three timing files included — is written,
when the index failed or reported no project (the search-graph twin's own
test). bench_finish then cleans up as its comment always promised, and
the caller's loop notices. The index call keeps its stderr in a
run-private temp file, the twin's shape, and the failure prints it under
`--- index stderr ---` with the first 500 bytes of the response: for a
one-shot CLI, stderr is the only channel a refusal is reported on. A
stale runtime-root.txt from an earlier run in the same results directory
is removed at start, so a failed run leaves no handoff at all.
The contract test's fixture answers `cli index_repository` with a
minimal envelope when CBM_BENCH_PROBE_INDEX_OK is set, so the keep run
covers the successful-index handoff and asserts project.txt; the
unflagged run asserts a non-zero exit and the surfaced cause after a
refused index; a new keep run with a refused index asserts a non-zero
exit, the timing files, no surviving root and no handoff, a pre-seeded
stale one included. Against the previous head the test fails on
"benchmark-index exited 0 after a refused index".
Signed-off-by: Anton Standrik <astandrik@yandex-team.ru>
The evaluation plan (docs/EVALUATION_PLAN.md §7) indexes a language with
benchmark-index.sh and then answers graph questions against that index
from its own MCP session. With a run-private runtime (#1696) the index
was gone before that session could start, so the handoff is now explicit
and opt-in.
With CBM_BENCH_KEEP_RUNTIME set, a SUCCESSFUL run stops its daemon,
leaves the private root in place and records the paths that reach it in
<results>/<lang>/runtime-root.txt (sourceable: CBM_BENCH_RUNTIME_ROOT,
CBM_RUNTIME_DIR, CBM_CACHE_DIR). Ownership of that root, including its
removal, passes to the caller. A failed run cleans up regardless: there
is no index worth keeping and nothing may leak. Nothing changes in the
default path or in the three timing files.
§7's skeleton sets the flag, sources the file before the graph session
and removes the root in step 8 instead of the live-store *.db files.
The contract test drives a kept run and requires the root to survive,
the handoff file to be sourceable, and its paths to be the ones the
product processes actually used; the unflagged run still asserts the
root is gone. The kept-run assertions fail against the previous head.
Signed-off-by: Anton Standrik <astandrik@yandex-team.ru>
Martin Vogel