The full Windows leg on the VM stopped at the coverage-map contract and was
reported green. Two defects, one hiding the other:
- The VM verdict counted the completion marker anywhere in the log. The
isolated-venue contract's stubbed entry prints that marker, and a capacity
case prints "expected=0 passed=True", so a leg that stopped at a later
contract, with no suite run, read as complete. The marker now counts only as
the log's last step header, which is where scripts/test.sh prints it; the
verdict contract pins the observed shape (14 cases).
- Under the native python3 the Windows legs install, the test-impact
contracts failed: the coverage-map builder cannot start the leg's bash
fakes, the scripts wrote CRLF, and cmp (diffutils) is not installed there.
selection.py, coverage-merge.py and coverage-map.py now write LF on every
host, so their outputs are byte-identical across platforms; the two cmp
checks compare bytes in the shell; the builder's functional leg is skipped
under a native Windows python with the reason and what was tried (the map
is built only by the ubuntu publish job), while both pure-python parser
tests still run there.
Production was not affected by the CRLF: run-tests-parallel.sh strips CR from
the selection and the runner trims its tokens.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Producer. test-impact-artifact.yml publishes, on every push to main, the
commit's team artifact bundle: its frozen graph, built incrementally from the
previous bundle, and its per-test coverage map, re-run only for the suites
the push selects (scripts/ci/test-impact-publish.sh). A weekly run rebuilds
both from scratch and fails when the incremental graph's topology differs
from the full one. Bundles are Actions artifacts kept 30 days, never
committed.
Shadow. The report-only test-impact-shadow job also builds the checkout and
runs `test-impact select` against the base side of the PR merge commit
(scripts/ci/test-impact-select.sh). A bundle is used only when it comes from
a successful main-branch producer run for exactly that commit
(scripts/ci/test-impact-bundle.sh). The job stays continue-on-error and
outside ci-ok.
Narrowing switch, PR CI only. While the repository variable
TEST_IMPACT_MODE is `gate`, a select-tests job turns the engine's answer into
the tests the test-unix and test-windows legs run. Unset, nothing changes.
It fails open by construction:
- scripts/test-impact/selection.py is the only place an answer becomes a
runner filter. Only a well-formed selected lane whose filter matches its
own suite list narrows; run-all answers, a lane that selects nothing,
unknown schemas or tokens, and any crash run everything.
- `test` passes a selection only from a successful, narrowed select-tests
run; ci-ok does not need select-tests. A broken selection costs minutes,
never coverage.
- scripts/run-tests-parallel.sh launches only the selected suites and keeps
every guard on the selected list. A selected suite the build does not
register fails the run, unless the engine saw its registration as
conditional; then it is dropped with a notice, exactly as a full run of
that build never runs it. Each runner judges its own tokens, so a selected
test that does not exist fails its suite.
- The shard manifest records the selection's digest, and
verify-shard-union.sh requires every shard of a leg to agree on it.
- Dry runs and releases never pass a selection (lane-wiring contract).
Also fixed on the way:
- publish crashed on any run-all push: a run-all lane carries
"suites": null.
- the git-facts hostile-diff probe re-ran the runner with the parent's
CBM_TEST_ONLY_FILE still set; under a narrowed run naming its suite it
re-ran the spawning test, which spawned again. The child now clears it
(a new test fails without that).
- the select summary crashed on the same null suites.
Contracts: tests/test_test_impact_selection.sh (Step 0z8),
tests/test_harness_selection.sh (Step 2c, the real runner through the real
harness), and new cases in the lane-gate and lane-wiring contracts.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel