The suite outgrew one runner: ~656 top-level files (~190k LOC) serialized
through --test-concurrency=4 on a single 4-core leg took ~17-20 min (windows
slowest; measured 1225s on the last full green run). Add a `shard` matrix
dimension to the test job so each leg runs a balanced slice; run-ci-tests.mjs
now partitions deterministically + totally (LPT bin-packing over file size)
so the union of all shards == allFiles and no test is skipped. Same tests,
assertions, env, OS and node versions — zero coverage loss. Wall drops ~3x
(~20 -> ~7 min).
Two concurrent full `npm test` runs on one machine corrupted each other:
cores-1 concurrency doubled the spawn/attach load so every 15s readiness
watchdog timed out (mass false-red), and daemons without a per-test
XDG_STATE_HOME shared ~/.local/state/billion-context (port-zone drift,
multi-instance alarms) — with ZERO detection of the sibling run.
Add scripts/test-suite-lock.mjs: an O_EXCL ("wx") lock at
$XDG_CACHE_HOME/billion-context/test-suite.lock holding {pid,ts,cmdline,cwd},
with liveness-only stale steal (kill(pid,0); never steal a live suite; no age
timeout — a local full run can exceed any fixed budget; a corrupt/truncated
file left by a SIGKILLed writer is reclaimable, not a permanent wedge).
`pretest` now prepends a non-owning `gate` so a second run refuses within
milliseconds BEFORE the build; `test` becomes an owning `run` wrapper that
holds the lock across the real suite and releases it on exit (plus a sync
backstop in the exit handler). BILI_TEST_FORCE=1 is the documented escape hatch.
Single-run behavior is unchanged: the wrapper reproduces the exact old
`node --import … --test tests/*.test.ts` invocation (same file set, no
concurrency cap added). Regression suite tests/test-suite-lock.test.ts pins
free / live-block / dead-steal / corrupt-reclaim, the <2s fail-fast acceptance,
force bypass, and release-on-exit.
Freezes the manual v0.1.196 review pass (PR #2962) into one command for
operator machines with live models — the regression lane GitHub CI can
never run. Executed before each release (reference/release.md step 5.5).
Lanes (hermetic scratch: own DSH_HOME/BILI_CONFIG_FILE/sessions/XDG/ports;
never touches ~/.dsh or running proxies; missing prerequisites skip with
a warning, GATE_* knobs in the script header):
- A compression: real dsh session reads a marker corpus at the real
window, then the route window shrinks mid-session (the field-report
"switched to a smaller-window model" lane) -> nudge INJECT -> preflight
fold -> SAME session answers from folded context, cache recovers.
A fresh oversized read tool-result is NOT expected to fold (protected
recent zone, #2847 fail-fast) — asserted as such.
- B model switch: same session resumed on a different model/upstream;
same lineage across switch + proxy restart (dsh resume-replay fork
WITH fork-adoption of the parent's folded blocks passes; an orphaned
pfa fails), cross-upstream usage-baseline demoted to untrusted when
cited (#2283/#2867), cache cold -> hot on the new upstream.
- U --with-update: dsh-managed install detection (no in-place self
update) + latest read from the configured update registry.
Green on 0.1.196: A 8/8, B 6/6, U 2/2 (~4 min, sglang + z.ai relay).
Windows-only update lanes (global installer, desktop-profile in-place
swap + spec converge, doctor) stay on the Win10 VM checklist appended
to reference/release.md.
- post-bind self-probe: where socket semantics tolerate a concurrent bind onto an actively-listening port, the late binder yields to the EADDRINUSE ladder (or dies under strict-port) instead of co-serving one port from two state machines
- health endpoint reports 503 not-ready until startup registration completes: closes the bind->registration window in which a concurrent same-lane launcher saw 'healthy' for an undiscoverable instance and spawned a duplicate
- proxy-origin pointer write refused while a different live instance health-verifies as serving the same address (registry entry still written; draining-predecessor takeover #1723 unaffected)
- attemptListen clears stale once('listening') callbacks so a successful bind after failed attempts announces exactly once
- four real-server test files poll for health readiness after 'listening' to match the new contract
The feature commit added src/host-plugin-notice.ts without regenerating the
module map (repo spec requires the map be regenerated when a source file is
added). Adds entries for host-plugin-notice.ts (#2861) and the previously
missing plugin-advisory.ts (#2462); refreshes the stale top-level test-file
count (426 -> 614).
#2853 made the carve-out channel PR-gated with bugfix-release/* as the marker prefix, but ci.yml version-guard still only whitelisted *_release-v*. Every carve-out release PR (e.g. #2857) went red, and release-bugfix.yml's fail-closed full-CI gate then blocked the merge-time publish.
Baseline-log arithmetic (sum of per-test durations vs step wall) proves the
old runner default was already concurrent, so the two-phase split's serial
live phase only idled 3+ cores for 3-4 min without buying stability. Run the
whole suite at explicit concurrency=4 (keeps the LIVE_FILES tripwire), and
move typecheck/kernel/pi-subagents/build into a parallel static-gates job off
the test critical path.
Baseline logs from the pre-PR flow show interleaved output from multiple
test files in the same seconds on ubuntu-latest: current hosted runners are
4-vCPU, so availableParallelism()-1 == 3 and the old flat glob was NOT
serial. The two-phase split isolates spawning suites from pool contention;
it does not turn serial into parallel. Comment-only.
The old channel auto-published on push of a bugfix-release/** branch, with no
human review gate. Now the carve-out branch goes through a PR against master;
release-bugfix.yml fires when that PR is MERGED and publishes the PRE-MERGE
PR-head tree (last tag + fix + version bump), strictly excluding master's other
unreleased work. All fail-closed guards, the pre-flight and full-CI gates are
unchanged; manual workflow_dispatch on an explicitly chosen ref remains the
emergency fallback. Pushing a bugfix-release/** branch triggers nothing anymore.
The node built-in runner defaults file concurrency to availableParallelism()-1 == 1 on 2-vCPU runners, so the flat tests/*.test.ts glob ran all 600+ suites strictly serially, each paying a node+tsx spawn in turn. scripts/run-ci-tests.mjs splits the same suites into: live (process-spawning, concurrency=1 — #1248 requires quiet-machine conditions) and pool (everything else, concurrency=4 — no spawners by construction, listen(0) ports, XDG-isolated homes), with a self-check that fails the job when a new spawning suite lands unregistered. Same tests/assertions/env; local npm test untouched. Plus two targeted suite speedups: image-request-budget hoists the shared ~100MB body (5 builds -> 3) and claude-native pre-warms the Windows CA (existing #1807 pattern) in the three remaining fresh-home hook e2e tests.
reference/release.md: step 3 (bili) and the kernel-release flow now spell out regenerating package-lock.json after a version bump, since editing package.json alone leaves the lock stale (#2704); points at the new lock-sync guard.
Supplement to 990cbe86f (which fixed only the tsc errors): under
post-#2688 semantics there is no wire-side image reduction left, so
the two tests that asserted the removed stripImages recovery path
(200 + reduced rebuilt body) fail at runtime with the forward-stage
413. Adapt them to assert the actual contract: over-budget rebuilt
bodies stop at stage=forward / request_too_large without touching
upstream, whether images sit in messages or tool outputs; the
admission-slot flow ends at the same forward guard once a slot is
released. Also fix reference/request-body-budget.md (en+zh), which
still told users to configure the removed compress.stripImages.
Verified locally: tsc --noEmit clean; node --import tsx --test
--test-concurrency=1 tests/image-request-budget.test.ts
tests/request-body-budget.test.ts -> 10/10 pass.
A transient runner-to-registry ECONNRESET during setup-phase `npm i -g` /
`npm ci` reds an otherwise-green PR with no test code ever running (#2541 run
38032380794; rerun-failed-jobs green on the same head). Root cause: npm's
built-in fetch retry defaults to only 2 tries. Raise it to 5 via a job-level
env block on every job that touches the registry, so a ~10s jitter rides out
instead of aborting setup. Job-level placement covers the global CLI install
AND `npm ci` AND any build-time fetch in that job. Bounded: worst case ~+2 min
of backoff before a genuine sustained outage still fails fast (jobs carry their
own timeout-minutes).
Applied to the 9 e2e/setup workflows with executable registry installs (12
jobs total). Deliberately NOT applied around `npm publish` (artifact/release
lanes): a publish is non-idempotent, so a fetch-retry re-POST can trip E409 /
double-publish, which is worse than the original failure mode. Documents the
required pattern in reference/testing.md so new lanes inherit it.
ACP-native agents (pi) POST their configured providers to the proxy
(POST /__bili/agent-providers); recipes merge UNDER file recipes at
request-apply time (file wins on name collision). Inline keys ride the
request rail (target.apiKey XOR credentialRef, file world still rejects
plaintext). Web GET exposes the registry without secrets; per-field
PUT now validates chain expansion too (gap: it never did).
compress.externalSummary.targets become "provider/model" references into
named dial recipes in the providers table (baseUrl/api/credential/models);
endpoints, protocol and credentials are derived at apply time. Execution
rail unchanged (expanded settings); unresolvable refs warn + disable the
chain, Web saves reject them with 400.
Strict pure move (#1440 P2 cut 1): zero renames, zero logic edits, zero behavior
change. The SECURITY gate block at the top of handle() (self-probe url
normalization, isAdminPath computation, tunnel-header 403, loopback 403,
trusted-origin 403/404, the whole inline GET/PUT/POST dispatch for
/__bili/* + /__acp/* incl. config/plugin/watcher/upstream routes, and the
unknown-admin-path 404 fallback) moved verbatim into the single entry point
handleAdminRoute(req, res, ctx) in src/server/admin.ts. The admin-only helpers
(handleConfigReload, sendCacheReport, handleConflictsClear, sendStats,
currentAdvisoryPayload, sendStatus, sendResignStatus, sendOverview,
sendWebSessions, sendWebLogs, sendWebSessionDetail), the admin-origin trio
(normalizeAdminHostname/isTrustedAdminOrigin/adminTrustedHostnames, used ONLY
by the gate) and the splitWarnedBases canary set (used ONLY by sendStatus)
traveled with their doc comments intact; reapOrphansLogged (non-admin, called
from four request-pipeline sites) stayed behind. Byte-identity: every deleted
line from server.ts appears verbatim in admin.ts (multiset diff empty except
two blank-line normalizations where block removals had created double blanks).
Deliberately stays in server.ts: the top-of-file proxy-bypass /__bili URL
special case (~line 222), destAdminPath in forward() (~line 5254), the
__biliKeepaliveBoundary / __bili_inplace_sysdev wire markers, selfAdminProbePath
(test-imported export, now also consumed by admin.ts), headerValue (dual-use
with non-admin pipeline code → newly exported), readBody/BodyTooLargeError
(already exported). Log strings mentioning /__bili URLs are unchanged.
AdminCtx fields (all passed BY REFERENCE so mutations stay visible to
server.ts): opts (ProxyOptions — config PUT/reload mutate it in place), core
(CompressionCore — plugin status/tool handlers), config (Config — plugin
manifest resolution via applyCompressSettings), log (tee logger fn), instanceId
+ instanceStartedAt (health endpoint identity), proxyWatchers (Set<number>,
watcher POST mutates; health reads), initialWatcherPid (watchdog arming state).
adminCtx is assembled once per startServer() call because these are per-server
instance values (per-#7 watchdog sets are intentionally per-server, like
instanceId) and threaded through handle() as its final parameter — this keeps
admin.ts exporting exactly ONE symbol (handleAdminRoute; AdminCtx stays
module-private and server.ts types the param as Parameters<typeof
handleAdminRoute>[2]).
Call-site control flow: the original zone FELL THROUGH to the proxy pipeline
for every non-admin URL (it only short-circuits when isAdminPath matched,
after which every path ends the response), so the replacement is
`await handleAdminRoute(req, res, adminCtx); if (res.writableEnded) return;`
— response-ended ⟺ the admin surface answered; non-admin traffic never has a
byte written by the moved code, so fall-through is byte-for-byte the old path.
Verification: tsc --noEmit clean; tsup build OK (dist-import-annotations OK);
18-file behavioral matrix (web-routing, web-sessions, advisory-endpoint-parity,
admin-origin, admin-probe, web-conflict-banner, web-logs-view,
watcher-registration, apig-resign, watchdog-health, watchdog-health-daemon,
conflict-lifecycle, plugin-status-provenance, plugin-fork, runtime-info,
issue833-panel-effective-config, issue532-panel-system-tokens, wire-contract)
= 191 pass / 0 fail. reference/architecture.md: one line added for admin.ts
(docs-sync file map).
Closes part of #1440 (P2 cut 1 of the server.ts god-file decomposition).
Owner decision (option A): the chain must live inside the existing
three-level compress system with no exception fields, and the bespoke
side-file reader goes away.
- mergeCompress: externalSummary is a whole-chain pick (provider/model
level replaces the chain, like tiers) and rides ResolvedKernelConfig
alongside preserveRecentTools; parseRouteEntry no longer rejects
route-level chains.
- external-summary-config.ts deleted: the plan is resolved from the
request config rail (ctx.config / effectiveConfig, #833) — the same
cascade that produces the wire — instead of re-reading the config
file. ConfiguredSummaryPlan re-parses its raw chain so invalid
settings still refuse loudly instead of silently disabling.
- P1: an invalid chain in parseCompressSettings now logs the specific
reason before rejecting the compress section (contract unchanged).
- P2: a disabled chain is inert — targets are not strictly validated
until the chain is enabled, so a stale typo cannot brick compression.
- Injection surfaces (tool schema adaptation, summary notes, plugin
manifest, preflight) read the rail instead of the file.
- Tests moved to the rail; new ladder/inert/route-level coverage.
- Docs: three-level whole-chain semantics in CONFIGURATION en/zh and
reference/external-summary en/zh.
The built-in pi-lane acp_delegate surface reads its config from bili's own
config file (pi.subagents section) instead of pi's acp.json. The four
acp.json keys (delegate/delegatePrompt/displayUsage/debug) become a
deprecated fallback: still read while the section is absent (one-time
warning), ignored once it exists. prompt replaces delegatePrompt; debug is
scoped to the sub-agent subsystem. The standalone package reads the same
section through its own loader (billion-context-pi-subagents#5) — the file
format is the contract, no shared code.
Co-Authored-By: Claude <noreply@anthropic.com>
A broken acp.json previously fell back to defaults with zero diagnostics;
the standalone package logs this via logThrow but bili's lane has no such
channel — console.error matches the agent-module house style.
Wire the acp_delegate sub-agent surface (tools + /acp-fleet command +
fleet shortcut + status widget + read tracking + prompt append) into the
native pi extension via the package's embedder contract: markEmbedded()
claim, manual building-block wiring, tsup noExternal inlining, and an
esbuild alias stub for @earendil-works/pi-coding-agent so the package's
only value import (CONFIG_DIR_NAME) falls back to the documented ".pi".
- agent-gated (pi only; omp lane untouched), single-writer vs a
standalone billion-context-pi-subagents install
- devDeps pinned exact: billion-context-pi-subagents 0.1.1,
@earendil-works/{pi-coding-agent,pi-tui} 0.83.0 (typecheck-only)
- dist/index.js + dist/mcp.js import graph unchanged (zero-dep contract)
New master (#2030) requires every behavior knob to resolve env > config
file > default through src/knobs.ts. The snapshot cap was a bare
process.env read in plugin.ts; it now resolves BILI_PUBLIC_SNAPSHOT_CAP_
BYTES > plugin.snapshotCapBytes > 16 MiB (0 disables) with the env tier
byte-exact to the pre-migration parser (Number(env): set-but-garbage
falls back to default, "" resolves to 0). CONFIGURATION.md en/zh gain
the plugin block, the mapping-table row and the semantics row;
reference/env-vars.md lists plugin.* in the tunables home. Precedence
pinned in tests/knobs-config-precedence.test.ts (default / file tier /
garbage env / empty-string quirk / garbage file).
Closes the three verification gaps left after the #1628 triage: the
unwritable-target wall had no real-chain e2e (only mock-logger unit
tests), the no-op protocol had no local form (a self-built fake baseline
cannot validate the LIVE old updater), and cache-lane doctor ran only
by hand.
- unwritable install dir (posix): chmod 0o555 on the real extracted
install, assert the loud failure + one-per-process location
diagnostic + zero disk flip + no leaked update lock
- real published baseline (BILI_E2E_REAL_BASELINE): download one real
published tarball, prove the live old updater hops onto this dist;
CI resolves the baseline via npm view before the run
- cache-lane doctor: opencode lane layout, doctor --json must resolve
the update target to the running copy