From 16e273281955abea911969c94de138414c66b92a Mon Sep 17 00:00:00 2001 From: Martin Vogel Date: Wed, 30 Sep 2026 23:48:22 +0200 Subject: [PATCH] ci: non-product PRs (T0b) run the contract steps too Under R1, a T0b push (test-infrastructure/, lint config, graph-ui, memwaste tooling) selected lint and no test leg, so the static contract steps that police exactly those files -- the VM scripts, the compose and run.sh wiring, the venue-parity rules -- ran nowhere. Replayed over September, 13 of the 18 T0b pushes ran no contract step at all. The R1 table now gives T0b lint + contracts, so those PRs get the contracts job (scripts/test.sh --contracts-only) as T0a PRs do. The fixture table's six T0b cases expect it, and the history replay asserts that every push runs the contract steps (the contracts lane or any test leg). Comments that described the job as docs-only are updated. Signed-off-by: Martin Vogel --- .github/workflows/pr.yml | 8 ++++---- scripts/README.md | 2 +- scripts/ci/select-lanes.sh | 5 +++-- scripts/test-impact/replay-selector.sh | 19 ++++++++++++++++--- scripts/test.sh | 2 +- tests/fixtures/select-lanes/cases.tsv | 12 ++++++------ tests/test_lane_wiring_contract.sh | 2 +- 7 files changed, 32 insertions(+), 18 deletions(-) diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index dbaca5098..da13aea51 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -261,10 +261,10 @@ jobs: ${{ runner.temp }}/test-impact/changed-files.txt if-no-files-found: warn - # ── Docs-only PRs (tier T0a) select no test leg, so this runs the static - # contract steps of the canonical test leg on their own (they police - # docs surfaces too: language counts, version metadata). Every test leg - # runs the same steps, so the job exists only when no leg does. ── + # ── Docs-only and non-product PRs (tiers T0a, T0b) run the static contract + # steps of the canonical test leg on their own: they police docs + # surfaces (language counts, version metadata), test-infrastructure/ and + # lint config too. Every test leg runs the same steps. ── contracts: needs: [changes] if: ${{ !cancelled() && needs.changes.result == 'success' && contains(fromJSON(needs.changes.outputs.lanes), 'contracts') }} diff --git a/scripts/README.md b/scripts/README.md index 93e65b507..519196c48 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -17,7 +17,7 @@ codes) and rejects unknown flags with exit 2 + `Please consult --help.` | leg | entry | what a run gives you | |---|---|---| -| **test** | `test.sh` | DEFAULT = the venue leg: static contracts (Step 0a–0r) + CLEAN ASan+UBSan build + all suites via the parallel harness + prod-binary guards. `--suites a,b` = iteration mode (incremental rebuild, subset, seconds). `--tsan` = the ThreadSanitizer leg. `--contracts-only` = the static contract steps alone (PR CI's docs-only job). CLANGARM64 gets CI's trap-UBSan flags automatically. | +| **test** | `test.sh` | DEFAULT = the venue leg: static contracts (Step 0a–0r) + CLEAN ASan+UBSan build + all suites via the parallel harness + prod-binary guards. `--suites a,b` = iteration mode (incremental rebuild, subset, seconds). `--tsan` = the ThreadSanitizer leg. `--contracts-only` = the static contract steps alone (PR CI's contracts job for docs-only and non-product changes). CLANGARM64 gets CI's trap-UBSan flags automatically. | | **package wrappers** | `ci/test-package-wrappers.sh` | Go, npm, and PyPI runtime-set publication/lock suites on the current host. CI runs the same entry on Linux and Windows so both platform lock implementations gate. | | **build** | `build.sh` | CLEAN production runtime set (native executable + authenticated integration asset; `--with-ui` adds one content-addressed UI pack). ccache via `env.sh` makes repeats fast; `CCACHE_COMPILERCHECK=content` guarantees a hit is byte-identical to a cold compile — never stale. `--version`, `STATIC=1`, `BUILD_DIR=`. | | **lint** | `lint.sh` | clang-tidy + cppcheck + clang-format (+ no-skips policy). `--ci` = the CI gate set (no clang-tidy). Drives the same make targets as `make lint`/`lint-ci`. | diff --git a/scripts/ci/select-lanes.sh b/scripts/ci/select-lanes.sh index 6d618bcb7..b21083082 100755 --- a/scripts/ci/select-lanes.sh +++ b/scripts/ci/select-lanes.sh @@ -251,14 +251,15 @@ FULL = {"codeql-gate", "diag", "license-gate", "lint", "lint-mem", "lsan-macos", "smoke-mac", "smoke-ubuntu", "smoke-win", "tsan-arm", "tsan-mac", "tsan-x86", "unix-arm64", "unix-macos-intel", "unix-macos14", "unix-x86", "windows", "windows-guards"} -UNIVERSE = FULL | {"contracts"} # contracts: the docs-only job; every test leg runs them too +UNIVERSE = FULL | {"contracts"} # contracts: the T0a/T0b job; every test leg runs them too LINT = {"lint", "lint-mem"} CORE_LEGS = {"unix-x86", "unix-macos14", "windows", "shard-completeness"} TEST_LEGS = {"unix-x86", "unix-arm64", "unix-macos14", "unix-macos-intel", "windows"} ALWAYS = {"security-static"} R1 = { "T0a-docs": {"contracts"}, - "T0b-nonproduct": LINT, + # The contract steps police test-infrastructure/ and lint config too. + "T0b-nonproduct": LINT | {"contracts"}, "T1-tests": LINT | CORE_LEGS, "T2-extract": LINT | CORE_LEGS | {"codeql-gate", "tsan-x86", "smoke-ubuntu"}, "T3-mid": LINT | CORE_LEGS | {"codeql-gate", "tsan-x86", "smoke-ubuntu"}, diff --git a/scripts/test-impact/replay-selector.sh b/scripts/test-impact/replay-selector.sh index 5433c588d..f88237fdc 100755 --- a/scripts/test-impact/replay-selector.sh +++ b/scripts/test-impact/replay-selector.sh @@ -9,7 +9,9 @@ # its documented deviation, and a deviation may only ever raise the tier # (the fail-safe direction) -- never quietly select less than was measured; # 2. every real failure of the month (22, fixtures/2026-09-real-failures.tsv) -# had its catching lane selected for its push. +# had its catching lane selected for its push; +# 3. every push runs the static contract steps (the contracts lane or any +# test leg) -- they police docs, test-infrastructure/ and lint config. # Pure data, one selector process (--batch over the distinct file sets). # # Usage: scripts/test-impact/replay-selector.sh [repo-root] @@ -17,7 +19,7 @@ set -euo pipefail case "${1:-}" in -h | --help) - sed -n '2,15p' "$0" | sed 's/^# \{0,1\}//' + sed -n '2,17p' "$0" | sed 's/^# \{0,1\}//' exit 0 ;; -*) @@ -111,6 +113,16 @@ for run_id, pr, sha, fileset, analysis, expected, note in rows("2026-09-pushes.t else: deviated += 1 +# The static contract steps police docs, test-infrastructure/ and lint config, +# so every push runs them: in the contracts job or inside any test leg. +CONTRACT_CARRIERS = {"contracts", "unix-x86", "unix-arm64", "unix-macos14", + "unix-macos-intel", "windows"} +uncovered = collections.Counter(d["tier"] for d in pushes.values() + if not set(d["lanes"]) & CONTRACT_CARRIERS) +for tier, count in sorted(uncovered.items()): + failures.append(f"{count} {tier} push(es) run no contract step (neither the " + f"contracts lane nor a test leg)") + caught = 0 for run_id, attempt, pr, sha, lane, why in rows("2026-09-real-failures.tsv", 6): decision = pushes.get(run_id) @@ -129,7 +141,8 @@ if failures: sys.exit(1) print(f"selector history replay OK: {len(pushes)} pushes, tier as expected for all " f"({matched} equal to the analysis, {deviated} documented fail-safe deviations); " - f"{caught} real failures, every catching lane selected") + f"{caught} real failures, every catching lane selected; every push runs the " + f"contract steps") print(" pushes per tier: " + ", ".join(f"{t} {tiers[t]}" for t in TIERS)) print(f" full runs: {sum(1 for d in pushes.values() if d['full'])}") PY diff --git a/scripts/test.sh b/scripts/test.sh index bfc299b15..eae756138 100755 --- a/scripts/test.sh +++ b/scripts/test.sh @@ -32,7 +32,7 @@ Modes: --contracts-only Only the static contract steps (Step 0*) of the default leg: no compiler, no build, no suites. PR CI runs this for - docs-only changes, where no test leg is selected. + docs-only and non-product changes (tiers T0a, T0b). Options: --arch ARCH Force target arch (arm64 | x86_64), e.g. under Rosetta. diff --git a/tests/fixtures/select-lanes/cases.tsv b/tests/fixtures/select-lanes/cases.tsv index cfcad1257..15ce05df5 100644 --- a/tests/fixtures/select-lanes/cases.tsv +++ b/tests/fixtures/select-lanes/cases.tsv @@ -5,12 +5,12 @@ # "FULL -a -b" = every PR lane except a and b). docs-only README.md docs/BENCHMARK.md T0a-docs false contracts security-static issue-template-is-docs .github/ISSUE_TEMPLATE/bug.yml T0a-docs false contracts security-static -local-infra test-infrastructure/run.sh T0b-nonproduct false lint lint-mem security-static -lint-config scripts/memory-core-baseline.txt T0b-nonproduct false lint lint-mem security-static -graph-ui-adds-guards graph-ui/src/App.tsx T0b-nonproduct false lint lint-mem security-static smoke-ubuntu windows-guards -memwaste-tools-add-memwaste scripts/memwaste.sh T0b-nonproduct false lint lint-mem memwaste security-static -non-gating-script-runs-contracts-leg scripts/benchmark-index.sh T0b-nonproduct false lint lint-mem security-static shard-completeness unix-x86 -test-impact-tooling scripts/test-impact/replay-selector.sh T0b-nonproduct false lint lint-mem security-static shard-completeness unix-x86 +local-infra test-infrastructure/run.sh T0b-nonproduct false contracts lint lint-mem security-static +lint-config scripts/memory-core-baseline.txt T0b-nonproduct false contracts lint lint-mem security-static +graph-ui-adds-guards graph-ui/src/App.tsx T0b-nonproduct false contracts lint lint-mem security-static smoke-ubuntu windows-guards +memwaste-tools-add-memwaste scripts/memwaste.sh T0b-nonproduct false contracts lint lint-mem memwaste security-static +non-gating-script-runs-contracts-leg scripts/benchmark-index.sh T0b-nonproduct false contracts lint lint-mem security-static shard-completeness unix-x86 +test-impact-tooling scripts/test-impact/replay-selector.sh T0b-nonproduct false contracts lint lint-mem security-static shard-completeness unix-x86 tests-only tests/test_store.c T1-tests false lint lint-mem security-static shard-completeness unix-macos14 unix-x86 windows test-fixture tests/fixtures/cpp_include/a.h T1-tests false lint lint-mem security-static shard-completeness unix-macos14 unix-x86 windows contract-test tests/test_version_metadata_contract.sh T1-tests false lint lint-mem security-static shard-completeness unix-macos14 unix-x86 windows diff --git a/tests/test_lane_wiring_contract.sh b/tests/test_lane_wiring_contract.sh index a176079a7..31a2b00b8 100755 --- a/tests/test_lane_wiring_contract.sh +++ b/tests/test_lane_wiring_contract.sh @@ -7,7 +7,7 @@ # (and "standard"): those runs execute every job, sharded, exactly as # before lane selection. Only pr.yml asks for the PR shard profile. # 2. pr.yml feeds the selector's lanes to every reusable workflow, runs the -# docs-only contracts job, and hands ci-ok both the selection and the +# T0a/T0b contracts job, and hands ci-ok both the selection and the # run's job list (actions: read) -- ci-ok needs `contracts` but never the # report-only memwaste / shadow jobs, and the shadow is continue-on-error. # 3. every lane of scripts/ci/select-lanes.sh --list-lanes is served by a