mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-10-26 06:43:15 +08:00
Martin Vogel
2f33999044
On 2026-09-18 a Windows leg printed "7925 passed, 0 failed" and "=== All tests passed ===" and still exited 1, while the identical work run through win.sh's other entry exited 0. The status travels ssh -> cmd.exe -> msys2_shell.cmd and that chain loses it. A channel that can turn 0 into 1 can turn 1 into 0, and that direction is a false green: a red Windows leg reported as passing, which is the one thing the local ladder exists to prevent. So the two channels are now split by what each can actually know. The log decides the test outcome — it is written by the runner on the VM and cannot be mangled in transit — and rc only decides what the log cannot see. Green needs zero reported failures AND the completion marker scripts/test.sh prints last, so a leg that stopped early has a summary but no marker and is red. A lost exit status is reported, not obeyed. The logic is a function, vm_verdict(), reachable as `--verdict <log> <rc>`, so tests/test_vm_verdict_contract.sh drives the REAL code with synthetic logs instead of a copy that could drift from it: lost exit status, failures under rc=0, a marker that must not launder failures, failures in a later suite, partial runs, and no summary at all (which keeps its own exit code 90, so "never ran" stays distinguishable from "ran and failed"). Eight cases, wired into scripts/test.sh as step 0y. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
test-infrastructure/ — the local venues
The local half of the 3-OS CI ladder. Every leg here runs the SAME canonical
scripts CI runs (scripts/test.sh, build.sh, smoke-local.sh,
soak-legs.sh, vm-smoke.sh — see scripts/README.md); this directory only
provisions containers and the Windows VM. The venue-parity contract
(tests/test_venue_parity_contract.sh, Step 0j of every test leg) fails the
build if a venue grows its own harness logic — here or in the workflows.
Entry points
| venue | entry | notes |
|---|---|---|
| Linux (arm64 + amd64) + cross-compile | ./run.sh <leg> (--help for the leg list) |
Colima only — Docker Desktop is broken on this machine. A clean-disk preflight (scripts/ci/preflight-docker.sh) runs before every leg: GitHub runners are ephemeral with a known-free 14 GB disk, so the long-lived VM is swept back to that shape first. |
| Real Windows (UTM ARM64 VM) | vm/win.sh <command> (win.sh help) |
Same preflight idea (scripts/ci/clean-test-residue.ps1), plus CI's protected per-user TEMP root. Wine (run.sh windows) is a compile check only — never a substitute for the VM. |
| macOS | run the canonical scripts natively | scripts/test.sh, scripts/build.sh, scripts/smoke-local.sh … |
The ladder (before any push)
scripts/test.sh— macOS native (or--suites …while iterating)./test-infrastructure/run.sh full— Linux arm64 test/build/TSan/smoke/portable + mingw cross-compile (+amd64,soak-linuxas needed)vm/win.sh test-par·guards·smoke-install·soak— the real-Windows legs
Infra unavailable = a run blocker to escalate — never a silent bypass.
Fidelity guarantees (what makes local predictive of CI)
- Same scripts, same sequence, same flags — platform specifics (e.g. the CLANGARM64 trap-UBSan default) live inside the canonical entries, applied identically everywhere.
- Same environment shape — protected TEMP roots, sandboxed smoke, clean builds with the content-verified ccache (a hit is byte-identical to a cold compile; caches only accelerate, never change results).
- Same starting disk — preflights sweep residue and BLOCK below the runner's 14 GB floor (a quietly full disk fails inside install paths and masquerades as a product bug).
- Known, deliberate residuals — runner physics (4 vCPU vs local cores, shared tenancy), Defender ON in the VM vs OFF on GitHub runners, no TSan on Windows anywhere, no ASan runtime on native ARM64 Windows (trap-UBSan + PageHeap stand in).
See vm/README.md for VM provisioning and day-to-day VM mechanics.