fix(ci): single-phase c=4 + parallel static-gates — serial live phase cost more than it saved (#2816)

Baseline-log arithmetic (sum of per-test durations vs step wall) proves the
old runner default was already concurrent, so the two-phase split's serial
live phase only idled 3+ cores for 3-4 min without buying stability. Run the
whole suite at explicit concurrency=4 (keeps the LIVE_FILES tripwire), and
move typecheck/kernel/pi-subagents/build into a parallel static-gates job off
the test critical path.
This commit is contained in:
ework-agent
2026-10-12 11:15:35 +08:00
parent e0e3759f25
commit e6ba45cd85
3 changed files with 75 additions and 54 deletions
+35 -16
View File
@@ -38,6 +38,41 @@ jobs:
# skip on push/dispatch (skipped needs count as satisfied). Behavior change:
# on guard failure the required legs show pending (not red); merge blocked.
test:
needs: [version-guard, kernel-guard, pi-subagents-guard, lane-guard, docs-sync, docs-coverage, unused-exports, lock-sync]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
node-version: [22, 24]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: npm
- run: npm install --ignore-scripts --no-fund --no-audit
# #2816: explicit file concurrency=4 for the whole suite. Baseline logs
# show the old default (availableParallelism()-1) already ran several
# suites at once, and a measured two-phase prototype proved a serial
# "live" phase only idles the other cores for 3-4 min without buying
# stability — live/process suites have run mixed with the rest on these
# dedicated runners for months (#1248 gates loaded local sandboxes, not
# CI). The script keeps a LIVE_FILES self-check: any NEW
# process-spawning suite fails the job until registered there (#1248
# review). Same tests/assertions/env — scheduling only.
# ACP_TEST_CLAUDE_NATIVE keeps the claude-native live e2e subtests
# running here: the runner is dedicated, so their process/port
# observations are reliable.
- run: node scripts/run-ci-tests.mjs
env:
ACP_TEST_CLAUDE_NATIVE: "1"
# #2816: typecheck + the in-repo component suites + the bundle build used
# to run sequentially in front of the test job (~90s of critical path per
# leg). They are independent of the proxy test suites, so they run as
# parallel legs on the same matrix — same commands, off the critical path.
static-gates:
needs: [version-guard, kernel-guard, pi-subagents-guard, lane-guard, docs-sync, docs-coverage, unused-exports, lock-sync]
strategy:
fail-fast: false
@@ -59,22 +94,6 @@ jobs:
# Pi-subagents boundary (#2384): same two-layer rule for the in-repo
# pi-lane component — its own unit suite against its TS source.
- run: npm --prefix pi-subagents test
# #2816: two-phase schedule instead of one flat glob. The node built-in
# runner defaults file concurrency to availableParallelism()-1 — 3 on
# today's 4-vCPU hosted runners, so the old flat glob already ran ~3
# suites at once; the split isolates the process-spawning ones from pool
# contention (#1248 documents live-e2e flaking under concurrent load),
# while the other ~570 never spawn and use listen(0) ports + per-file
# XDG homes, so they share the machine at N=4. Same tests/assertions/env
# — scheduling only. The script self-checks that every spawning suite is
# registered in its LIVE_FILES list.
# ACP_TEST_CLAUDE_NATIVE keeps the claude-native live e2e subtests
# running here: the runner is dedicated, so their process/port
# observations are reliable (#1248 gates them off for loaded local
# sandboxes only).
- run: node scripts/run-ci-tests.mjs
env:
ACP_TEST_CLAUDE_NATIVE: "1"
- run: npm run build
version-guard:
+19 -18
View File
@@ -13,33 +13,34 @@ npm run typecheck # tsc --noEmit --project tsconfig.json (src + tests)
npm test # node --import tsx --test tests/*.test.ts
```
## CI: two-phase test schedule (#2816)
## CI: test schedule (#2816)
The main CI `test` job does not run the flat `npm test` glob — it runs
`node scripts/run-ci-tests.mjs`, which splits the SAME top-level suites into
two phases. The node built-in runner defaults file concurrency to
`availableParallelism()-1` — 3 on today's 4-vCPU hosted runners — so the old
flat glob already ran ~3 suites concurrently (baseline logs show interleaved
file output); this split is about isolating the spawning suites from pool
contention, not serial-vs-parallel.
1. **live** (`--test-concurrency=1`): the suites that spawn real processes
(native host e2es, watchdog/launcher/port-zone live tests, ...). They MUST
stay serial on a quiet machine — #1248 documents claude-native live e2e
flaking (kill ESRCH, port timeouts, multi-minute hangs) under concurrent
load.
2. **pool** (`--test-concurrency=4`): everything else — no process spawning by
construction, `listen(0)` ports, per-file XDG-isolated homes.
`node scripts/run-ci-tests.mjs`, which runs the same top-level suites with an
explicit `--test-concurrency=4`. Baseline logs prove the old runner default
(`availableParallelism()-1`) was already concurrent — the sum of per-test
durations (~630s ubuntu / ~1110s windows) fits inside the step wall
(373s / 688s) only if several files ran at once; pinning c=4 makes that
explicit with one slot per core. A two-phase prototype (serial "live" phase +
concurrent pool) was measured first and rejected: the serial phase idled the
other cores for 3-4 min without buying stability, because live/process suites
have run mixed with the rest on these dedicated runners for months (#1248's
flakes were loaded local sandboxes — gated by `ACP_TEST_CLAUDE_NATIVE`, not CI).
The script self-checks on every run: a new top-level `tests/*.test.ts` that
matches a process-spawn pattern but is NOT registered in its `LIVE_FILES` list
fails the job. So when you add a suite that spawns processes, register it in
`scripts/run-ci-tests.mjs` (a false positive only costs a slot in the slower
serial phase; a false negative re-opens #1248).
fails the job. When you add a suite that spawns processes, register it in
`scripts/run-ci-tests.mjs` and review its concurrency assumptions (#1248:
per-file XDG homes, `listen(0)` ports, no machine-global state) before it
rides c=4 unreviewed.
Scheduling only — same tests, same assertions, same env vars; local
`npm test` is untouched.
The `static-gates` job (same os×node matrix) runs typecheck + kernel suite +
pi-subagents suite + build IN PARALLEL with the `test` job; they used to be
sequential steps ahead of the tests (~90s of critical path per leg).
## Local Testing & Install
Test against the REAL published artifact:
+21 -20
View File
@@ -1,20 +1,20 @@
#!/usr/bin/env node
// #2816: CI test scheduling. The node built-in runner defaults file
// concurrency to availableParallelism()-1 == 1 on 2-vCPU runners, so all
// 600+ top-level suites ran strictly serially, each paying a full
// node+tsx spawn. That schedule is still REQUIRED for the live/process
// suites — #1248 documents claude-native live e2e flaking (kill ESRCH,
// port timeouts, multi-minute hangs) under concurrent load — but the
// other ~570 suites never spawn a process, use listen(0) ports and
// per-file XDG-isolated homes, so they can safely share the machine:
// #2816: CI test scheduling. Baseline logs prove the old runner default
// (availableParallelism()-1) was ALREADY concurrent: the sum of per-test
// durations (~630s ubuntu / ~1110s windows) fits inside the step wall
// (373s / 688s) only if several files ran at once. This pins concurrency=4
// explicitly. A two-phase prototype (serial "live" phase + concurrent pool)
// was measured first: the serial phase idled the other cores for 3-4 min
// without buying stability — live/process suites have run mixed with the rest
// on dedicated runners for months without #1248-class flakes (#1248 gates
// loaded local sandboxes via env, not CI), and 4 slots dilute live-live
// pairing (P(two live in flight) ≈ C(4,2)·(32/613)² ≈ 5%, vs ≈ 8% at the old
// c=3). Same tests, assertions and env as `npm test` — only the scheduler
// changes. Local `npm test` is untouched (this script is CI-only).
//
// phase 1 "live": process-spawning suites, concurrency=1 — byte-for-byte
// the conditions the suite has always run under in CI
// (serial + quiet machine);
// phase 2 "pool": everything else, concurrency=4.
//
// Same tests, same assertions, same env vars — only the schedule changes.
// Local `npm test` is untouched (this script is CI-only).
// LIVE_FILES is a tripwire, not a gate: every process-spawning suite must be
// registered there so adding a new one fails the job until someone reviews
// its concurrency assumptions (#1248) instead of silently riding c=4.
import { execFileSync } from "node:child_process";
import { readdirSync, readFileSync } from "node:fs";
import { join } from "node:path";
@@ -25,8 +25,9 @@ const testsDir = join(root, "tests");
// Process-spawning suites (grep ground truth: spawn/exec*/child_process/Worker
// references below). Conservative by design — a false positive just costs a
// slot in the serial phase; a false negative would put a live suite into the
// concurrent pool and re-open #1248. New spawning suites MUST be added here.
// registration entry; an unregistered spawner fails the job by design. New
// spawning suites MUST be added here after reviewing their concurrency
// assumptions.
const LIVE_FILES = [
"tests/auto-restart.test.ts",
"tests/build-info.test.ts",
@@ -84,7 +85,8 @@ const unregistered = allFiles.filter(
if (unregistered.length > 0) {
die(
`process-spawning test files missing from LIVE_FILES in scripts/run-ci-tests.mjs: ` +
`${unregistered.join(", ")}. Register them there (#1248: live suites must stay serial).`,
`${unregistered.join(", ")}. Register them there after reviewing their ` +
`concurrency assumptions (#1248).`,
);
}
@@ -113,5 +115,4 @@ function runPhase(label, files, concurrency) {
console.log(`[ci-tests] ${label} ok in ${((Date.now() - t0) / 1000).toFixed(1)}s`);
}
runPhase("live", LIVE_FILES, 1);
runPhase("pool", allFiles.filter((f) => !LIVE_FILES.includes(f)), 4);
runPhase("all", allFiles, 4);