mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-02 02:07:25 +08:00
> Follow-up to #11006 (merged): rebased onto master and ready for review. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release subsystem now publishes canary (every master push), nightly (scheduled, smoke-gated, added in #11006), and stable (manual) > - There is still no human-approved release-candidate lane between nightly and stable, and nothing enforces that a stable actually soaked anywhere before shipping > - Betas need a real approval gate, and stables need a soak policy that is data, not prose > - This pull request adds the beta channel: a manual promotion of a chosen nightly behind the `npm-beta` environment gate, re-smoked after publish, plus a stable preflight that enforces a 3-day beta soak with a written-justification bypass > - The benefit is a complete canary → nightly → beta → stable train where every stable shipped as a beta first, and emergencies leave a written trace ## Linked Issues or Issue Description **Subsystem affected** Release automation: `scripts/release.sh`, `scripts/release-lib.sh`, `.github/workflows/release.yml`, `.github/workflows/docker.yml`, `.github/workflows/release-smoke.yml`. **Problem or motivation** After #11006 the project has canary and nightly prerelease lanes, but no release-candidate lane. Stable promotion has no enforced soak: any ref can ship as stable directly. There is no approval boundary for a broader-audience prerelease, and no structured way to record why an emergency release skipped validation. **Proposed solution** Add a `beta` channel: a manual dispatch that promotes a chosen nightly's source commit, publishes behind the `npm-beta` GitHub environment (required reviewers are the gate), re-smokes the published beta, and tags `beta/vX`. Enforce in the stable path that the source commit shipped as a beta at least 3 days earlier (measured from the beta's npm publish time), with a `skip_soak_justification` input as the recorded emergency bypass. **Alternatives considered** Codifying the soak policy in docs only. Rejected: an unenforced policy decays; the preflight makes the policy executable while the justification input keeps the emergency path usable and auditable. ## What Changed - `scripts/release.sh` + `scripts/release-lib.sh`: `beta` channel — requires HEAD to carry a `nightly/v*` tag, publishes the package set as `YYYY.MDD.P-beta.N` under dist-tag `beta`, tags `beta/vYYYY.MDD.P-beta.N` - `.github/workflows/release.yml`: - `channel: beta` dispatch path: `select_beta` resolves the newest (or an explicit `source_version`) nightly and fails loudly on selection problems; `publish_beta` runs behind the `npm-beta` environment, pushes the tag, and dispatches `docker.yml`; `smoke_beta` re-runs the release smoke suite against the exact published beta version - stable path: new `preflight_stable` job enforces the 3-day beta soak from the beta's npm publish time; `skip_soak_justification` bypasses with the reason echoed into the job summary; dry runs report without blocking - `.github/workflows/docker.yml`: `beta/v*` tags publish `:beta` on both images, with exact version stamping - `.github/workflows/release-smoke.yml`: `beta` added to the dispatch choice list - Docs: `CHANNELS.md` beta entries; `RELEASING.md` beta lane, soak gate, and failure playbook; `RELEASE-AUTOMATION-SETUP.md` `npm-beta` environment setup, including the warning to create the environment before the first beta dispatch (GitHub auto-creates unprotected environments on first reference) - Tests: beta version-counting coverage in `scripts/release-registry-versions.test.mjs`; beta identity and nightly-tag guard coverage in `scripts/__tests__/release-dry-run-notes.test.mjs` ## Verification - `node --test` on the two touched suites: 17 pass, including the 3 new beta tests - `bash -n` on both shell scripts and YAML parse of all three workflows - After merge, in order: create the `npm-beta` environment, dispatch `channel: beta` with `dry_run: true` to preview, then a real promotion of a published nightly through the approval gate, then a stable dry-run against a young beta to see the soak gate report ## Risks - If the `npm-beta` environment does not exist when the first beta dispatch runs, GitHub creates it with no protection rules and the beta publishes without approval. Mitigated by documentation and by creating the environment before merge (operator step) - Until the first beta exists, every stable dispatch requires `skip_soak_justification`. This is deliberate — the first beta ships immediately after this merges — but it is a behavior change to the stable dispatch - The soak clock reads the beta's npm publish time from the registry; a registry outage makes the preflight fall back to requiring justification (fail-closed) ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with extended thinking and full tool use (repository exploration, local test execution, live registry and git verification). All code, tests, and docs in this PR were model-authored under human direction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending — will confirm before merge) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending — will confirm before merge) - [x] I will address all Greptile and reviewer comments before requesting merge
402 lines
10 KiB
JavaScript
402 lines
10 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import test from "node:test";
|
|
|
|
import {
|
|
collectInternalDependencyProblems,
|
|
createManifestLookupKey,
|
|
fetchRegistryJson,
|
|
isCanaryVersion,
|
|
isPrereleaseVersion,
|
|
verifyPackageRegistryProblems,
|
|
verifyPackageRegistryState,
|
|
} from "./verify-release-registry-state.mjs";
|
|
|
|
test("isCanaryVersion matches release canaries", () => {
|
|
assert.equal(isCanaryVersion("2026.427.0-canary.3"), true);
|
|
assert.equal(isCanaryVersion("2026.427.0"), false);
|
|
});
|
|
|
|
test("isPrereleaseVersion matches canary and nightly versions", () => {
|
|
assert.equal(isPrereleaseVersion("2026.427.0-canary.3"), true);
|
|
assert.equal(isPrereleaseVersion("2026.427.0-nightly.0"), true);
|
|
assert.equal(isPrereleaseVersion("2026.427.0-beta.2"), true);
|
|
assert.equal(isPrereleaseVersion("2026.427.0"), false);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems flags missing internal versions", () => {
|
|
const manifest = {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.425.0-canary.5",
|
|
e2b: "^2.19.0",
|
|
},
|
|
};
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, packageDocsByName),
|
|
["dependencies requires @paperclipai/plugin-sdk@2026.425.0-canary.5, but npm does not expose that version"],
|
|
);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems accepts version-specific manifests when the root document is stale", () => {
|
|
const manifest = {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.425.0-canary.5",
|
|
},
|
|
};
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
versions: {},
|
|
},
|
|
],
|
|
]);
|
|
const packageManifestsByKey = new Map([
|
|
[
|
|
createManifestLookupKey("@paperclipai/plugin-sdk", "2026.425.0-canary.5"),
|
|
{ name: "@paperclipai/plugin-sdk", version: "2026.425.0-canary.5" },
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, packageDocsByName, packageManifestsByKey),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems ignores peer dependency range specifiers", () => {
|
|
const manifest = {
|
|
peerDependencies: {
|
|
"@paperclipai/server": "^2026.430.0-canary.0",
|
|
},
|
|
};
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, new Map()),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems reports unfetched transitive dependency metadata neutrally", () => {
|
|
const manifest = {
|
|
optionalDependencies: {
|
|
"@paperclipai/browser": "2026.430.0-canary.0",
|
|
},
|
|
};
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, new Map()),
|
|
[
|
|
"optionalDependencies requires @paperclipai/browser@2026.430.0-canary.0, but npm publication metadata was not fetched for that dependency",
|
|
],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState tolerates a stale root versions map when dist-tags and direct manifests are correct", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/ui",
|
|
{
|
|
"dist-tags": {
|
|
canary: "2026.430.0-canary.0",
|
|
latest: "2026.430.0",
|
|
},
|
|
versions: {},
|
|
},
|
|
],
|
|
[
|
|
"@paperclipai/shared",
|
|
{
|
|
versions: {},
|
|
},
|
|
],
|
|
]);
|
|
const packageManifestsByKey = new Map([
|
|
[
|
|
createManifestLookupKey("@paperclipai/ui", "2026.430.0-canary.0"),
|
|
{
|
|
name: "@paperclipai/ui",
|
|
version: "2026.430.0-canary.0",
|
|
dependencies: {
|
|
"@paperclipai/shared": "2026.430.0-canary.0",
|
|
},
|
|
},
|
|
],
|
|
[
|
|
createManifestLookupKey("@paperclipai/shared", "2026.430.0-canary.0"),
|
|
{
|
|
name: "@paperclipai/shared",
|
|
version: "2026.430.0-canary.0",
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/ui",
|
|
packageDoc: packageDocsByName.get("@paperclipai/ui"),
|
|
packageDocsByName,
|
|
packageManifestsByKey,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.430.0-canary.0",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState fails when canary latest is left in place by default", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-e2b",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.425.0-canary.5",
|
|
canary: "2026.427.0-canary.3",
|
|
},
|
|
versions: {
|
|
"2026.425.0-canary.5": {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.425.0-canary.5",
|
|
},
|
|
},
|
|
"2026.427.0-canary.3": {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.427.0-canary.3",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
],
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/plugin-e2b",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-e2b"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.427.0-canary.3",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
[
|
|
"@paperclipai/plugin-e2b: latest dist-tag still resolves to prerelease 2026.425.0-canary.5; if that state is intentional, rerun the verification script directly with --allow-canary-latest",
|
|
"@paperclipai/plugin-e2b@2026.425.0-canary.5 via latest: dependencies requires @paperclipai/plugin-sdk@2026.425.0-canary.5, but npm does not expose that version",
|
|
],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryProblems marks canary latest drift as non-retriable", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-e2b",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.425.0-canary.5",
|
|
canary: "2026.427.0-canary.3",
|
|
},
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
const problems = verifyPackageRegistryProblems({
|
|
packageName: "@paperclipai/plugin-e2b",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-e2b"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.427.0-canary.3",
|
|
allowCanaryLatest: false,
|
|
});
|
|
|
|
assert.equal(problems[0]?.retriable, false);
|
|
assert.match(problems[0]?.message ?? "", /latest dist-tag still resolves to prerelease/);
|
|
});
|
|
|
|
test("verifyPackageRegistryProblems accepts the nightly channel and flags nightly latest drift", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-e2b",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.425.0-nightly.1",
|
|
nightly: "2026.427.0-nightly.0",
|
|
},
|
|
versions: {
|
|
"2026.427.0-nightly.0": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
const problems = verifyPackageRegistryProblems({
|
|
packageName: "@paperclipai/plugin-e2b",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-e2b"),
|
|
packageDocsByName,
|
|
channel: "nightly",
|
|
distTag: "nightly",
|
|
targetVersion: "2026.427.0-nightly.0",
|
|
allowCanaryLatest: false,
|
|
});
|
|
|
|
assert.equal(problems[0]?.retriable, false);
|
|
assert.match(problems[0]?.message ?? "", /latest dist-tag still resolves to prerelease 2026\.425\.0-nightly\.1/);
|
|
});
|
|
|
|
test("verifyPackageRegistryState allows intentional canary latest but still checks dependencies", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"paperclipai",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.427.0-canary.3",
|
|
canary: "2026.427.0-canary.3",
|
|
},
|
|
versions: {
|
|
"2026.427.0-canary.3": {
|
|
dependencies: {
|
|
"@paperclipai/server": "2026.427.0-canary.3",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
],
|
|
[
|
|
"@paperclipai/server",
|
|
{
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "paperclipai",
|
|
packageDoc: packageDocsByName.get("paperclipai"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.427.0-canary.3",
|
|
allowCanaryLatest: true,
|
|
}),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState still fails when the dist-tag is stale", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/ui",
|
|
{
|
|
"dist-tags": {
|
|
canary: "2026.429.0-canary.2",
|
|
},
|
|
versions: {},
|
|
},
|
|
],
|
|
]);
|
|
const packageManifestsByKey = new Map([
|
|
[
|
|
createManifestLookupKey("@paperclipai/ui", "2026.430.0-canary.0"),
|
|
{
|
|
name: "@paperclipai/ui",
|
|
version: "2026.430.0-canary.0",
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/ui",
|
|
packageDoc: packageDocsByName.get("@paperclipai/ui"),
|
|
packageDocsByName,
|
|
packageManifestsByKey,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.430.0-canary.0",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
["@paperclipai/ui: dist-tag canary resolves to 2026.429.0-canary.2, expected 2026.430.0-canary.0"],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState ignores internal peer dependency ranges", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
"dist-tags": {
|
|
canary: "2026.430.0-canary.0",
|
|
},
|
|
versions: {
|
|
"2026.430.0-canary.0": {
|
|
peerDependencies: {
|
|
"@paperclipai/server": "^2026.430.0-canary.0",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/plugin-sdk",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-sdk"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.430.0-canary.0",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("fetchRegistryJson times out hung requests", async () => {
|
|
const originalFetch = globalThis.fetch;
|
|
|
|
globalThis.fetch = (_url, { signal }) =>
|
|
new Promise((_resolve, reject) => {
|
|
signal.addEventListener(
|
|
"abort",
|
|
() => reject(new DOMException("The operation was aborted.", "AbortError")),
|
|
{ once: true },
|
|
);
|
|
});
|
|
|
|
try {
|
|
await assert.rejects(
|
|
fetchRegistryJson(new URL("https://registry.npmjs.org/@paperclipai%2Fui"), { timeoutMs: 1 }),
|
|
/timed out/,
|
|
);
|
|
} finally {
|
|
globalThis.fetch = originalFetch;
|
|
}
|
|
});
|