mirror of
https://github.com/ranxianglei/billion-context.git
synced 2026-11-23 01:58:16 +08:00
fix(ci): single-phase c=4 + parallel static-gates — serial live phase cost more than it saved (#2816)
Baseline-log arithmetic (sum of per-test durations vs step wall) proves the old runner default was already concurrent, so the two-phase split's serial live phase only idled 3+ cores for 3-4 min without buying stability. Run the whole suite at explicit concurrency=4 (keeps the LIVE_FILES tripwire), and move typecheck/kernel/pi-subagents/build into a parallel static-gates job off the test critical path.
This commit is contained in:
+35
-16
@@ -38,6 +38,41 @@ jobs:
|
||||
# skip on push/dispatch (skipped needs count as satisfied). Behavior change:
|
||||
# on guard failure the required legs show pending (not red); merge blocked.
|
||||
test:
|
||||
needs: [version-guard, kernel-guard, pi-subagents-guard, lane-guard, docs-sync, docs-coverage, unused-exports, lock-sync]
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest, windows-latest]
|
||||
node-version: [22, 24]
|
||||
runs-on: ${{ matrix.os }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ matrix.node-version }}
|
||||
cache: npm
|
||||
- run: npm install --ignore-scripts --no-fund --no-audit
|
||||
# #2816: explicit file concurrency=4 for the whole suite. Baseline logs
|
||||
# show the old default (availableParallelism()-1) already ran several
|
||||
# suites at once, and a measured two-phase prototype proved a serial
|
||||
# "live" phase only idles the other cores for 3-4 min without buying
|
||||
# stability — live/process suites have run mixed with the rest on these
|
||||
# dedicated runners for months (#1248 gates loaded local sandboxes, not
|
||||
# CI). The script keeps a LIVE_FILES self-check: any NEW
|
||||
# process-spawning suite fails the job until registered there (#1248
|
||||
# review). Same tests/assertions/env — scheduling only.
|
||||
# ACP_TEST_CLAUDE_NATIVE keeps the claude-native live e2e subtests
|
||||
# running here: the runner is dedicated, so their process/port
|
||||
# observations are reliable.
|
||||
- run: node scripts/run-ci-tests.mjs
|
||||
env:
|
||||
ACP_TEST_CLAUDE_NATIVE: "1"
|
||||
|
||||
# #2816: typecheck + the in-repo component suites + the bundle build used
|
||||
# to run sequentially in front of the test job (~90s of critical path per
|
||||
# leg). They are independent of the proxy test suites, so they run as
|
||||
# parallel legs on the same matrix — same commands, off the critical path.
|
||||
static-gates:
|
||||
needs: [version-guard, kernel-guard, pi-subagents-guard, lane-guard, docs-sync, docs-coverage, unused-exports, lock-sync]
|
||||
strategy:
|
||||
fail-fast: false
|
||||
@@ -59,22 +94,6 @@ jobs:
|
||||
# Pi-subagents boundary (#2384): same two-layer rule for the in-repo
|
||||
# pi-lane component — its own unit suite against its TS source.
|
||||
- run: npm --prefix pi-subagents test
|
||||
# #2816: two-phase schedule instead of one flat glob. The node built-in
|
||||
# runner defaults file concurrency to availableParallelism()-1 — 3 on
|
||||
# today's 4-vCPU hosted runners, so the old flat glob already ran ~3
|
||||
# suites at once; the split isolates the process-spawning ones from pool
|
||||
# contention (#1248 documents live-e2e flaking under concurrent load),
|
||||
# while the other ~570 never spawn and use listen(0) ports + per-file
|
||||
# XDG homes, so they share the machine at N=4. Same tests/assertions/env
|
||||
# — scheduling only. The script self-checks that every spawning suite is
|
||||
# registered in its LIVE_FILES list.
|
||||
# ACP_TEST_CLAUDE_NATIVE keeps the claude-native live e2e subtests
|
||||
# running here: the runner is dedicated, so their process/port
|
||||
# observations are reliable (#1248 gates them off for loaded local
|
||||
# sandboxes only).
|
||||
- run: node scripts/run-ci-tests.mjs
|
||||
env:
|
||||
ACP_TEST_CLAUDE_NATIVE: "1"
|
||||
- run: npm run build
|
||||
|
||||
version-guard:
|
||||
|
||||
+19
-18
@@ -13,33 +13,34 @@ npm run typecheck # tsc --noEmit --project tsconfig.json (src + tests)
|
||||
npm test # node --import tsx --test tests/*.test.ts
|
||||
```
|
||||
|
||||
## CI: two-phase test schedule (#2816)
|
||||
## CI: test schedule (#2816)
|
||||
|
||||
The main CI `test` job does not run the flat `npm test` glob — it runs
|
||||
`node scripts/run-ci-tests.mjs`, which splits the SAME top-level suites into
|
||||
two phases. The node built-in runner defaults file concurrency to
|
||||
`availableParallelism()-1` — 3 on today's 4-vCPU hosted runners — so the old
|
||||
flat glob already ran ~3 suites concurrently (baseline logs show interleaved
|
||||
file output); this split is about isolating the spawning suites from pool
|
||||
contention, not serial-vs-parallel.
|
||||
|
||||
1. **live** (`--test-concurrency=1`): the suites that spawn real processes
|
||||
(native host e2es, watchdog/launcher/port-zone live tests, ...). They MUST
|
||||
stay serial on a quiet machine — #1248 documents claude-native live e2e
|
||||
flaking (kill ESRCH, port timeouts, multi-minute hangs) under concurrent
|
||||
load.
|
||||
2. **pool** (`--test-concurrency=4`): everything else — no process spawning by
|
||||
construction, `listen(0)` ports, per-file XDG-isolated homes.
|
||||
`node scripts/run-ci-tests.mjs`, which runs the same top-level suites with an
|
||||
explicit `--test-concurrency=4`. Baseline logs prove the old runner default
|
||||
(`availableParallelism()-1`) was already concurrent — the sum of per-test
|
||||
durations (~630s ubuntu / ~1110s windows) fits inside the step wall
|
||||
(373s / 688s) only if several files ran at once; pinning c=4 makes that
|
||||
explicit with one slot per core. A two-phase prototype (serial "live" phase +
|
||||
concurrent pool) was measured first and rejected: the serial phase idled the
|
||||
other cores for 3-4 min without buying stability, because live/process suites
|
||||
have run mixed with the rest on these dedicated runners for months (#1248's
|
||||
flakes were loaded local sandboxes — gated by `ACP_TEST_CLAUDE_NATIVE`, not CI).
|
||||
|
||||
The script self-checks on every run: a new top-level `tests/*.test.ts` that
|
||||
matches a process-spawn pattern but is NOT registered in its `LIVE_FILES` list
|
||||
fails the job. So when you add a suite that spawns processes, register it in
|
||||
`scripts/run-ci-tests.mjs` (a false positive only costs a slot in the slower
|
||||
serial phase; a false negative re-opens #1248).
|
||||
fails the job. When you add a suite that spawns processes, register it in
|
||||
`scripts/run-ci-tests.mjs` and review its concurrency assumptions (#1248:
|
||||
per-file XDG homes, `listen(0)` ports, no machine-global state) before it
|
||||
rides c=4 unreviewed.
|
||||
|
||||
Scheduling only — same tests, same assertions, same env vars; local
|
||||
`npm test` is untouched.
|
||||
|
||||
The `static-gates` job (same os×node matrix) runs typecheck + kernel suite +
|
||||
pi-subagents suite + build IN PARALLEL with the `test` job; they used to be
|
||||
sequential steps ahead of the tests (~90s of critical path per leg).
|
||||
|
||||
## Local Testing & Install
|
||||
|
||||
Test against the REAL published artifact:
|
||||
|
||||
+21
-20
@@ -1,20 +1,20 @@
|
||||
#!/usr/bin/env node
|
||||
// #2816: CI test scheduling. The node built-in runner defaults file
|
||||
// concurrency to availableParallelism()-1 == 1 on 2-vCPU runners, so all
|
||||
// 600+ top-level suites ran strictly serially, each paying a full
|
||||
// node+tsx spawn. That schedule is still REQUIRED for the live/process
|
||||
// suites — #1248 documents claude-native live e2e flaking (kill ESRCH,
|
||||
// port timeouts, multi-minute hangs) under concurrent load — but the
|
||||
// other ~570 suites never spawn a process, use listen(0) ports and
|
||||
// per-file XDG-isolated homes, so they can safely share the machine:
|
||||
// #2816: CI test scheduling. Baseline logs prove the old runner default
|
||||
// (availableParallelism()-1) was ALREADY concurrent: the sum of per-test
|
||||
// durations (~630s ubuntu / ~1110s windows) fits inside the step wall
|
||||
// (373s / 688s) only if several files ran at once. This pins concurrency=4
|
||||
// explicitly. A two-phase prototype (serial "live" phase + concurrent pool)
|
||||
// was measured first: the serial phase idled the other cores for 3-4 min
|
||||
// without buying stability — live/process suites have run mixed with the rest
|
||||
// on dedicated runners for months without #1248-class flakes (#1248 gates
|
||||
// loaded local sandboxes via env, not CI), and 4 slots dilute live-live
|
||||
// pairing (P(two live in flight) ≈ C(4,2)·(32/613)² ≈ 5%, vs ≈ 8% at the old
|
||||
// c=3). Same tests, assertions and env as `npm test` — only the scheduler
|
||||
// changes. Local `npm test` is untouched (this script is CI-only).
|
||||
//
|
||||
// phase 1 "live": process-spawning suites, concurrency=1 — byte-for-byte
|
||||
// the conditions the suite has always run under in CI
|
||||
// (serial + quiet machine);
|
||||
// phase 2 "pool": everything else, concurrency=4.
|
||||
//
|
||||
// Same tests, same assertions, same env vars — only the schedule changes.
|
||||
// Local `npm test` is untouched (this script is CI-only).
|
||||
// LIVE_FILES is a tripwire, not a gate: every process-spawning suite must be
|
||||
// registered there so adding a new one fails the job until someone reviews
|
||||
// its concurrency assumptions (#1248) instead of silently riding c=4.
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { readdirSync, readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
@@ -25,8 +25,9 @@ const testsDir = join(root, "tests");
|
||||
|
||||
// Process-spawning suites (grep ground truth: spawn/exec*/child_process/Worker
|
||||
// references below). Conservative by design — a false positive just costs a
|
||||
// slot in the serial phase; a false negative would put a live suite into the
|
||||
// concurrent pool and re-open #1248. New spawning suites MUST be added here.
|
||||
// registration entry; an unregistered spawner fails the job by design. New
|
||||
// spawning suites MUST be added here after reviewing their concurrency
|
||||
// assumptions.
|
||||
const LIVE_FILES = [
|
||||
"tests/auto-restart.test.ts",
|
||||
"tests/build-info.test.ts",
|
||||
@@ -84,7 +85,8 @@ const unregistered = allFiles.filter(
|
||||
if (unregistered.length > 0) {
|
||||
die(
|
||||
`process-spawning test files missing from LIVE_FILES in scripts/run-ci-tests.mjs: ` +
|
||||
`${unregistered.join(", ")}. Register them there (#1248: live suites must stay serial).`,
|
||||
`${unregistered.join(", ")}. Register them there after reviewing their ` +
|
||||
`concurrency assumptions (#1248).`,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -113,5 +115,4 @@ function runPhase(label, files, concurrency) {
|
||||
console.log(`[ci-tests] ${label} ok in ${((Date.now() - t0) / 1000).toFixed(1)}s`);
|
||||
}
|
||||
|
||||
runPhase("live", LIVE_FILES, 1);
|
||||
runPhase("pool", allFiles.filter((f) => !LIVE_FILES.includes(f)), 4);
|
||||
runPhase("all", allFiles, 4);
|
||||
|
||||
Reference in New Issue
Block a user