Commit Graph
52 Commits
Author SHA1 Message Date
ranxianglei 92fcd60424 Merge branch 'master' into 2026-10-14_test-suite-lock 2026-10-14 18:43:51 +08:00
ework-agent b8cad881be ci: shard the test suite across the matrix to cut wall time ~3x (#3143)
The suite outgrew one runner: ~656 top-level files (~190k LOC) serialized
through --test-concurrency=4 on a single 4-core leg took ~17-20 min (windows
slowest; measured 1225s on the last full green run). Add a `shard` matrix
dimension to the test job so each leg runs a balanced slice; run-ci-tests.mjs
now partitions deterministically + totally (LPT bin-packing over file size)
so the union of all shards == allFiles and no test is skipped. Same tests,
assertions, env, OS and node versions — zero coverage loss. Wall drops ~3x
(~20 -> ~7 min).
2026-10-14 14:57:38 +08:00
ework-agent c9a204e358 feat(test): per-machine test-suite mutex + fail-fast (#3103)
Two concurrent full `npm test` runs on one machine corrupted each other:
cores-1 concurrency doubled the spawn/attach load so every 15s readiness
watchdog timed out (mass false-red), and daemons without a per-test
XDG_STATE_HOME shared ~/.local/state/billion-context (port-zone drift,
multi-instance alarms) — with ZERO detection of the sibling run.

Add scripts/test-suite-lock.mjs: an O_EXCL ("wx") lock at
$XDG_CACHE_HOME/billion-context/test-suite.lock holding {pid,ts,cmdline,cwd},
with liveness-only stale steal (kill(pid,0); never steal a live suite; no age
timeout — a local full run can exceed any fixed budget; a corrupt/truncated
file left by a SIGKILLed writer is reclaimable, not a permanent wedge).
`pretest` now prepends a non-owning `gate` so a second run refuses within
milliseconds BEFORE the build; `test` becomes an owning `run` wrapper that
holds the lock across the real suite and releases it on exit (plus a sync
backstop in the exit handler). BILI_TEST_FORCE=1 is the documented escape hatch.

Single-run behavior is unchanged: the wrapper reproduces the exact old
`node --import … --test tests/*.test.ts` invocation (same file set, no
concurrency cap added). Regression suite tests/test-suite-lock.test.ts pins
free / live-block / dead-steal / corrupt-reclaim, the <2s fail-fast acceptance,
force bypass, and release-on-exit.
2026-10-14 10:59:25 +08:00
ranxianglei 097e76c9f4 Merge pull request #2947 from ranxianglei/2026-10-13_port-owner-guard
fix: verify sole port ownership after bind; guard proxy-origin against clobbering a live holder (#2941)
2026-10-13 21:25:23 +08:00
ranxianglei 2cc37e2924 scrub private infra details from gate docs
- verdaccio LAN IP -> <verdaccio-LAN-IP> placeholder (release.md checklist)
- private VM runbook path -> generic wording
- relay knob docs: drop provider identity + auth notes, keep semantics
2026-10-13 12:10:47 +08:00
ranxianglei bd930b44e4 add local real-model release gate (npm run gate:local)
Freezes the manual v0.1.196 review pass (PR #2962) into one command for
operator machines with live models — the regression lane GitHub CI can
never run. Executed before each release (reference/release.md step 5.5).

Lanes (hermetic scratch: own DSH_HOME/BILI_CONFIG_FILE/sessions/XDG/ports;
never touches ~/.dsh or running proxies; missing prerequisites skip with
a warning, GATE_* knobs in the script header):

- A compression: real dsh session reads a marker corpus at the real
  window, then the route window shrinks mid-session (the field-report
  "switched to a smaller-window model" lane) -> nudge INJECT -> preflight
  fold -> SAME session answers from folded context, cache recovers.
  A fresh oversized read tool-result is NOT expected to fold (protected
  recent zone, #2847 fail-fast) — asserted as such.
- B model switch: same session resumed on a different model/upstream;
  same lineage across switch + proxy restart (dsh resume-replay fork
  WITH fork-adoption of the parent's folded blocks passes; an orphaned
  pfa fails), cross-upstream usage-baseline demoted to untrusted when
  cited (#2283/#2867), cache cold -> hot on the new upstream.
- U --with-update: dsh-managed install detection (no in-place self
  update) + latest read from the configured update registry.

Green on 0.1.196: A 8/8, B 6/6, U 2/2 (~4 min, sglang + z.ai relay).
Windows-only update lanes (global installer, desktop-profile in-place
swap + spec converge, doctor) stay on the Win10 VM checklist appended
to reference/release.md.
2026-10-13 12:02:28 +08:00
ework-agent 290b2e54e4 fix: verify sole port ownership after bind; guard proxy-origin against clobbering a live holder (#2941)
- post-bind self-probe: where socket semantics tolerate a concurrent bind onto an actively-listening port, the late binder yields to the EADDRINUSE ladder (or dies under strict-port) instead of co-serving one port from two state machines
- health endpoint reports 503 not-ready until startup registration completes: closes the bind->registration window in which a concurrent same-lane launcher saw 'healthy' for an undiscoverable instance and spawned a duplicate
- proxy-origin pointer write refused while a different live instance health-verifies as serving the same address (registry entry still written; draining-predecessor takeover #1723 unaffected)
- attemptListen clears stale once('listening') callbacks so a successful bind after failed attempts announces exactly once
- four real-server test files poll for health readiness after 'listening' to match the new contract
2026-10-13 09:04:39 +08:00
overFloweee 3823e827e7 fix: 将派生缓存纳入会话生命周期并释放请求底层字符串 2026-10-12 19:07:00 +08:00
ranxianglei 565501c261 Merge pull request #2848 from ranxianglei/2026-10-12_ci-test-scheduling
fix(ci): explicit concurrency=4 + parallel static-gates — CI total 16min→11m43s (monitored) (#2816)
2026-10-12 15:35:32 +08:00
ework-agent d1f1e55053 docs: sync reference/architecture.md with advisory/notice modules (#2861)
The feature commit added src/host-plugin-notice.ts without regenerating the
module map (repo spec requires the map be regenerated when a source file is
added). Adds entries for host-plugin-notice.ts (#2861) and the previously
missing plugin-advisory.ts (#2462); refreshes the stale top-level test-file
count (426 -> 614).
2026-10-12 12:50:37 +08:00
ework-agent 60538d1508 fix: allow bugfix-release/* carve-out branches in ci.yml version-guard
#2853 made the carve-out channel PR-gated with bugfix-release/* as the marker prefix, but ci.yml version-guard still only whitelisted *_release-v*. Every carve-out release PR (e.g. #2857) went red, and release-bugfix.yml's fail-closed full-CI gate then blocked the merge-time publish.
2026-10-12 11:21:09 +08:00
ework-agent e6ba45cd85 fix(ci): single-phase c=4 + parallel static-gates — serial live phase cost more than it saved (#2816)
Baseline-log arithmetic (sum of per-test durations vs step wall) proves the
old runner default was already concurrent, so the two-phase split's serial
live phase only idled 3+ cores for 3-4 min without buying stability. Run the
whole suite at explicit concurrency=4 (keeps the LIVE_FILES tripwire), and
move typecheck/kernel/pi-subagents/build into a parallel static-gates job off
the test critical path.
2026-10-12 11:15:35 +08:00
ework-agent e0e3759f25 docs(ci): correct the stale 2-vCPU/serial premise in the schedule rationale (#2816)
Baseline logs from the pre-PR flow show interleaved output from multiple
test files in the same seconds on ubuntu-latest: current hosted runners are
4-vCPU, so availableParallelism()-1 == 3 and the old flat glob was NOT
serial. The two-phase split isolates spawning suites from pool contention;
it does not turn serial into parallel. Comment-only.
2026-10-12 11:00:52 +08:00
ework-agent 810e081821 feat: route bugfix carve-out releases through a PR like normal releases (#2851)
The old channel auto-published on push of a bugfix-release/** branch, with no
human review gate. Now the carve-out branch goes through a PR against master;
release-bugfix.yml fires when that PR is MERGED and publishes the PRE-MERGE
PR-head tree (last tag + fix + version bump), strictly excluding master's other
unreleased work. All fail-closed guards, the pre-flight and full-CI gates are
unchanged; manual workflow_dispatch on an explicitly chosen ref remains the
emergency fallback. Pushing a bugfix-release/** branch triggers nothing anymore.
2026-10-12 10:50:13 +08:00
ework-agent c2914c34ab test(ci): two-phase schedule for the main CI test job (#2816)
The node built-in runner defaults file concurrency to availableParallelism()-1 == 1 on 2-vCPU runners, so the flat tests/*.test.ts glob ran all 600+ suites strictly serially, each paying a node+tsx spawn in turn. scripts/run-ci-tests.mjs splits the same suites into: live (process-spawning, concurrency=1 — #1248 requires quiet-machine conditions) and pool (everything else, concurrency=4 — no spawners by construction, listen(0) ports, XDG-isolated homes), with a self-check that fails the job when a new spawning suite lands unregistered. Same tests/assertions/env; local npm test untouched. Plus two targeted suite speedups: image-request-budget hoists the shared ~100MB body (5 builds -> 3) and claude-native pre-warms the Windows CA (existing #1807 pattern) in the three remaining fresh-home hook e2e tests.
2026-10-12 10:14:26 +08:00
ework-agent a3e1fec6f3 docs: document lock regeneration in the release procedure
reference/release.md: step 3 (bili) and the kernel-release flow now spell out regenerating package-lock.json after a version bump, since editing package.json alone leaves the lock stale (#2704); points at the new lock-sync guard.
2026-10-11 08:29:31 +08:00
ework-agent b0d5119874 fix(#2698): adapt the two runtime-red budget tests + doc to post-#2688 semantics
Supplement to 990cbe86f (which fixed only the tsc errors): under
post-#2688 semantics there is no wire-side image reduction left, so
the two tests that asserted the removed stripImages recovery path
(200 + reduced rebuilt body) fail at runtime with the forward-stage
413. Adapt them to assert the actual contract: over-budget rebuilt
bodies stop at stage=forward / request_too_large without touching
upstream, whether images sit in messages or tool outputs; the
admission-slot flow ends at the same forward guard once a slot is
released. Also fix reference/request-body-budget.md (en+zh), which
still told users to configure the removed compress.stripImages.

Verified locally: tsc --noEmit clean; node --import tsx --test
--test-concurrency=1 tests/image-request-budget.test.ts
tests/request-body-budget.test.ts -> 10/10 pass.
2026-10-11 01:18:32 +08:00
ranxianglei ae7d9c0b49 Merge pull request #2613 from overFloweee/2026-10-10_image-request-budget
fix: 让压缩图片历史先减量,再检查发送字节预算
2026-10-11 00:10:33 +08:00
ework-agent 23d67bef33 fix(#2611): retry npm registry fetches in e2e CI setup to ride out transient blips
A transient runner-to-registry ECONNRESET during setup-phase `npm i -g` /
`npm ci` reds an otherwise-green PR with no test code ever running (#2541 run
38032380794; rerun-failed-jobs green on the same head). Root cause: npm's
built-in fetch retry defaults to only 2 tries. Raise it to 5 via a job-level
env block on every job that touches the registry, so a ~10s jitter rides out
instead of aborting setup. Job-level placement covers the global CLI install
AND `npm ci` AND any build-time fetch in that job. Bounded: worst case ~+2 min
of backoff before a genuine sustained outage still fails fast (jobs carry their
own timeout-minutes).

Applied to the 9 e2e/setup workflows with executable registry installs (12
jobs total). Deliberately NOT applied around `npm publish` (artifact/release
lanes): a publish is non-idempotent, so a fetch-retry re-POST can trip E409 /
double-publish, which is worse than the original failure mode. Documents the
required pattern in reference/testing.md so new lanes inherit it.
2026-10-10 15:22:49 +08:00
overFloweee 92fb3d08c4 fix: 分离图片历史接收解压与转发字节预算 2026-10-10 15:03:32 +08:00
ranxianglei d2a6191ab1 Merge pull request #2435 from xiaozhu1337/2026-10-09_web-access-fetch-coexistence
fix: preserve pi-web-access proxy fetch coexistence
2026-10-09 21:38:14 +08:00
ranxianglei 1f3b145589 refactor(server): extract forward() relay zone into src/server/relay module (#1440 P2 cut 4) 2026-10-09 09:14:49 +08:00
xiaozhu 6db4b84cff fix: preserve pi-web-access proxy fetch coexistence
Fixes #2434
2026-10-09 00:23:53 +08:00
ranxianglei e7af746913 refactor(server): extract proxy request pipeline handle() into src/server/handle.ts (#1440 P2 cut 3) 2026-10-08 22:22:56 +08:00
awork d233af649b feat: vendor billion-context-pi-subagents in-repo under pi-subagents/ (#2384) 2026-10-08 13:09:51 +08:00
ranxianglei a233b6efec refactor(server): extract protocol preparation + tool-injection zone into src/server/{prepare-*,inject} modules (#1440 P2 cut 2) 2026-10-08 09:19:56 +08:00
ranxianglei 63595d8cef feat: agent-reported provider registry fallback (#2336)
ACP-native agents (pi) POST their configured providers to the proxy
(POST /__bili/agent-providers); recipes merge UNDER file recipes at
request-apply time (file wins on name collision). Inline keys ride the
request rail (target.apiKey XOR credentialRef, file world still rejects
plaintext). Web GET exposes the registry without secrets; per-field
PUT now validates chain expansion too (gap: it never did).
2026-10-07 21:34:13 +08:00
ranxianglei 2095cc8b3f Merge origin/master into pr-2167-rework (admin-route extract + dead-code cleanup) 2026-10-07 12:18:21 +08:00
ranxianglei 0b283fb6e2 feat: externalSummary chain references named provider recipes (#2163)
compress.externalSummary.targets become "provider/model" references into
named dial recipes in the providers table (baseUrl/api/credential/models);
endpoints, protocol and credentials are derived at apply time. Execution
rail unchanged (expanded settings); unresolvable refs warn + disable the
chain, Web saves reject them with 400.
2026-10-07 11:43:33 +08:00
ranxianglei 273c83945b refactor(server): extract the /__bili + /__acp admin route surface into src/server/admin.ts (#1440 P2 cut 1)
Strict pure move (#1440 P2 cut 1): zero renames, zero logic edits, zero behavior
change. The SECURITY gate block at the top of handle() (self-probe url
normalization, isAdminPath computation, tunnel-header 403, loopback 403,
trusted-origin 403/404, the whole inline GET/PUT/POST dispatch for
/__bili/* + /__acp/* incl. config/plugin/watcher/upstream routes, and the
unknown-admin-path 404 fallback) moved verbatim into the single entry point
handleAdminRoute(req, res, ctx) in src/server/admin.ts. The admin-only helpers
(handleConfigReload, sendCacheReport, handleConflictsClear, sendStats,
currentAdvisoryPayload, sendStatus, sendResignStatus, sendOverview,
sendWebSessions, sendWebLogs, sendWebSessionDetail), the admin-origin trio
(normalizeAdminHostname/isTrustedAdminOrigin/adminTrustedHostnames, used ONLY
by the gate) and the splitWarnedBases canary set (used ONLY by sendStatus)
traveled with their doc comments intact; reapOrphansLogged (non-admin, called
from four request-pipeline sites) stayed behind. Byte-identity: every deleted
line from server.ts appears verbatim in admin.ts (multiset diff empty except
two blank-line normalizations where block removals had created double blanks).

Deliberately stays in server.ts: the top-of-file proxy-bypass /__bili URL
special case (~line 222), destAdminPath in forward() (~line 5254), the
__biliKeepaliveBoundary / __bili_inplace_sysdev wire markers, selfAdminProbePath
(test-imported export, now also consumed by admin.ts), headerValue (dual-use
with non-admin pipeline code → newly exported), readBody/BodyTooLargeError
(already exported). Log strings mentioning /__bili URLs are unchanged.

AdminCtx fields (all passed BY REFERENCE so mutations stay visible to
server.ts): opts (ProxyOptions — config PUT/reload mutate it in place), core
(CompressionCore — plugin status/tool handlers), config (Config — plugin
manifest resolution via applyCompressSettings), log (tee logger fn), instanceId
+ instanceStartedAt (health endpoint identity), proxyWatchers (Set<number>,
watcher POST mutates; health reads), initialWatcherPid (watchdog arming state).
adminCtx is assembled once per startServer() call because these are per-server
instance values (per-#7 watchdog sets are intentionally per-server, like
instanceId) and threaded through handle() as its final parameter — this keeps
admin.ts exporting exactly ONE symbol (handleAdminRoute; AdminCtx stays
module-private and server.ts types the param as Parameters<typeof
handleAdminRoute>[2]).

Call-site control flow: the original zone FELL THROUGH to the proxy pipeline
for every non-admin URL (it only short-circuits when isAdminPath matched,
after which every path ends the response), so the replacement is
`await handleAdminRoute(req, res, adminCtx); if (res.writableEnded) return;`
— response-ended ⟺ the admin surface answered; non-admin traffic never has a
byte written by the moved code, so fall-through is byte-for-byte the old path.

Verification: tsc --noEmit clean; tsup build OK (dist-import-annotations OK);
18-file behavioral matrix (web-routing, web-sessions, advisory-endpoint-parity,
admin-origin, admin-probe, web-conflict-banner, web-logs-view,
watcher-registration, apig-resign, watchdog-health, watchdog-health-daemon,
conflict-lifecycle, plugin-status-provenance, plugin-fork, runtime-info,
issue833-panel-effective-config, issue532-panel-system-tokens, wire-contract)
= 191 pass / 0 fail. reference/architecture.md: one line added for admin.ts
(docs-sync file map).

Closes part of #1440 (P2 cut 1 of the server.ts god-file decomposition).
2026-10-07 00:49:33 +08:00
ranxianglei 99a435023d refactor: externalSummary joins the three-level compress ladder (#2163)
Owner decision (option A): the chain must live inside the existing
three-level compress system with no exception fields, and the bespoke
side-file reader goes away.

- mergeCompress: externalSummary is a whole-chain pick (provider/model
  level replaces the chain, like tiers) and rides ResolvedKernelConfig
  alongside preserveRecentTools; parseRouteEntry no longer rejects
  route-level chains.
- external-summary-config.ts deleted: the plan is resolved from the
  request config rail (ctx.config / effectiveConfig, #833) — the same
  cascade that produces the wire — instead of re-reading the config
  file. ConfiguredSummaryPlan re-parses its raw chain so invalid
  settings still refuse loudly instead of silently disabling.
- P1: an invalid chain in parseCompressSettings now logs the specific
  reason before rejecting the compress section (contract unchanged).
- P2: a disabled chain is inert — targets are not strictly validated
  until the chain is enabled, so a stale typo cannot brick compression.
- Injection surfaces (tool schema adaptation, summary notes, plugin
  manifest, preflight) read the rail instead of the file.
- Tests moved to the rail; new ladder/inert/route-level coverage.
- Docs: three-level whole-chain semantics in CONFIGURATION en/zh and
  reference/external-summary en/zh.
2026-10-07 00:12:59 +08:00
YZSQ 2e4860d635 Merge remote-tracking branch 'origin/master' into 2026-10-05_external-summary-executor
# Conflicts:
#	src/preflight.ts
#	src/server.ts
2026-10-06 21:17:48 +08:00
ranxiangleiandClaude 6d458a387b feat: pi.subagents config home in billion-context.json (#2230)
The built-in pi-lane acp_delegate surface reads its config from bili's own
config file (pi.subagents section) instead of pi's acp.json. The four
acp.json keys (delegate/delegatePrompt/displayUsage/debug) become a
deprecated fallback: still read while the section is absent (one-time
warning), ignored once it exists. prompt replaces delegatePrompt; debug is
scoped to the sub-agent subsystem. The standalone package reads the same
section through its own loader (billion-context-pi-subagents#5) — the file
format is the contract, no shared code.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-06 13:49:57 +08:00
ework-agent 78c5d65381 fix(pi-subagents): log config-load failure instead of swallowing it silently (#2186)
A broken acp.json previously fell back to defaults with zero diagnostics;
the standalone package logs this via logThrow but bili's lane has no such
channel — console.error matches the agent-module house style.
2026-10-06 11:02:34 +08:00
ranxianglei 8e41ed91e0 feat: inline billion-context-pi-subagents into the pi lane (#2186)
Wire the acp_delegate sub-agent surface (tools + /acp-fleet command +
fleet shortcut + status widget + read tracking + prompt append) into the
native pi extension via the package's embedder contract: markEmbedded()
claim, manual building-block wiring, tsup noExternal inlining, and an
esbuild alias stub for @earendil-works/pi-coding-agent so the package's
only value import (CONFIG_DIR_NAME) falls back to the documented ".pi".

- agent-gated (pi only; omp lane untouched), single-writer vs a
  standalone billion-context-pi-subagents install
- devDeps pinned exact: billion-context-pi-subagents 0.1.1,
  @earendil-works/{pi-coding-agent,pi-tui} 0.83.0 (typecheck-only)
- dist/index.js + dist/mcp.js import graph unchanged (zero-dep contract)
2026-10-06 10:57:34 +08:00
YZSQ 8f6159b502 docs: describe complete external summary review implementation 2026-10-06 00:03:15 +08:00
YZSQ bcf40118d8 feat: add configurable external summary service 2026-10-05 22:40:11 +08:00
YZSQ 6aa00d02ff fix: share external summary deadline across ranges 2026-10-05 19:06:52 +08:00
YZSQ aa8b81f8bb feat: add isolated external summary HTTP candidates 2026-10-05 18:47:06 +08:00
YZSQ 55771690aa feat: add isolated external-summary execution foundation (#2163) 2026-10-05 18:16:54 +08:00
ework-agent 9b9de987c5 docs: fix leading-space defect in architecture.md tree entries (#2121)
The two agent-tree lines added by this PR started with a stray leading space, breaking the ASCII tree alignment. Strip it (no content change).
2026-10-05 02:09:49 +08:00
ework-agent e17504bcaf docs: pi codex WS transports covered natively; Bedrock stays tracked separately (#2111) 2026-10-05 01:51:59 +08:00
ework-agent 6e5007bb46 docs: document in-repo kernel boundary and standalone kernel releases (#2092) 2026-10-04 21:06:18 +08:00
ranxianglei 590331a288 docs(release): release-notes entries are opt-in — only severe releases need one (#1870 policy revision) 2026-10-04 16:25:06 +08:00
ranxianglei 830da41e0c fix(#2017): register BILI_PUBLIC_SNAPSHOT_CAP_BYTES through knobs.ts (#2030 discipline)
New master (#2030) requires every behavior knob to resolve env > config
file > default through src/knobs.ts. The snapshot cap was a bare
process.env read in plugin.ts; it now resolves BILI_PUBLIC_SNAPSHOT_CAP_
BYTES > plugin.snapshotCapBytes > 16 MiB (0 disables) with the env tier
byte-exact to the pre-migration parser (Number(env): set-but-garbage
falls back to default, "" resolves to 0). CONFIGURATION.md en/zh gain
the plugin block, the mapping-table row and the semantics row;
reference/env-vars.md lists plugin.* in the tunables home. Precedence
pinned in tests/knobs-config-precedence.test.ts (default / file tier /
garbage env / empty-string quirk / garbage file).
2026-10-04 15:40:06 +08:00
ranxianglei 45d822eb73 Merge branch 'master' into 2026-10-04_env-knob-config-migration 2026-10-04 15:05:27 +08:00
ework-agent bfbd9330ca docs: README config-file sections, env discipline moved to reference/env-vars.md, AGENTS.md staleness fixes (#2037) 2026-10-04 14:33:45 +08:00
ranxianglei 4f42990828 test(e2e): drive the #1628 failure class end-to-end — unwritable target, real-baseline hop, cache-lane doctor
Closes the three verification gaps left after the #1628 triage: the
unwritable-target wall had no real-chain e2e (only mock-logger unit
tests), the no-op protocol had no local form (a self-built fake baseline
cannot validate the LIVE old updater), and cache-lane doctor ran only
by hand.

- unwritable install dir (posix): chmod 0o555 on the real extracted
  install, assert the loud failure + one-per-process location
  diagnostic + zero disk flip + no leaked update lock
- real published baseline (BILI_E2E_REAL_BASELINE): download one real
  published tarball, prove the live old updater hops onto this dist;
  CI resolves the baseline via npm view before the run
- cache-lane doctor: opencode lane layout, doctor --json must resolve
  the update target to the running copy
2026-10-04 14:15:10 +08:00
ework-agent c2e4ec887f docs(release): document the bugfix carve-out channel in release reference 2026-10-04 08:43:36 +08:00
ework-agent 0be2a7a016 docs: align typecheck script references with the src+tests gate (#1970) 2026-10-03 20:20:44 +08:00