mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-02 02:07:25 +08:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The release subsystem publishes the public workspace packages and also powers release-related CI validation. > - The release flow currently asks npm for package versions one package at a time in multiple places. > - That serial registry latency slows the PR Canary Dry Run path and real release invocations even though the checks are independent. > - This pull request batches npm registry version lookups with bounded concurrency and reuses the result for version calculation. > - The benefit is shorter non-build release-script time while preserving the fresh target-version existence check before publishing. ## Linked Issues or Issue Description - No public GitHub issue exists for this release-script performance cleanup. ### Problem or motivation Release validation spends avoidable time on repeated serial `npm view` calls across the public package set. The slow path affects PR release validation and real release invocations because version discovery waits on independent registry reads one at a time. ### Proposed solution Fetch package version maps concurrently with bounded parallelism, reuse that map for stable/canary version calculation, and keep a fresh parallel absence check for the target publish version. ### Alternatives considered Keeping the existing serial shell loop is simpler, but it preserves the CI latency cost. Caching the final target-version existence check was rejected because release publish safety should still query npm freshly before publishing. ### Roadmap alignment This is a small release-tooling performance improvement. It does not duplicate any planned core product work found in `ROADMAP.md`. ## What Changed - Added `scripts/release-registry-versions.mjs` to fetch npm package version maps and assert target-version absence with bounded parallelism. - Updated `scripts/release.sh` to prefetch package versions once and to batch the final target-version absence check. - Updated `next_stable_version` and `next_canary_version` to use the prefetched version map when present, with the existing per-package npm fallback preserved. - Added release-registry helper coverage and included it in `pnpm run test:release-registry`. - Hardened the release publish helper tests so their fake `pnpm`/`npm` fixture PATH is preserved under non-login shell execution. ## Verification - `node --test scripts/release-registry-versions.test.mjs` - `pnpm run test:release-registry` - `bash -n scripts/release.sh scripts/release-lib.sh` - `git diff --check` - Safety scan before push: searched changed files for common key/token/password patterns and PII markers; only benign script-name text matched (`secrets:migrate-inline-env`). - Remote PR checks on the latest head passed, including `Typecheck + Release Registry`, `Canary Dry Run`, build, tests, e2e, policy, security scans, and commitperclip review. - Greptile reviewed the latest head with Confidence Score 5/5 and no blocking issues. ## Risks - Low risk. The release version helpers keep their original npm fallback when no prefetched version map is supplied. - The existence check remains fresh and uncached before publish, but now reports all matching package/version pairs from a parallel check. - If npm has transient failures during the prefetch step, missing or failed packages still map to an empty version list, matching the old helper behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent using GPT-5, with shell/tool execution in the local repository. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude <noreply@paperclip.ing>
115 lines
3.6 KiB
JavaScript
115 lines
3.6 KiB
JavaScript
#!/usr/bin/env node
|
|
// Batched npm registry version queries for the release tooling.
|
|
//
|
|
// The release flow needs published-version data for every public workspace
|
|
// package. Querying them one `npm view` at a time is serial network latency
|
|
// that dominates the non-build time of `release.sh` (and the PR workflow's
|
|
// Canary Dry Run job). This helper runs the same `npm view` queries with
|
|
// bounded concurrency instead.
|
|
//
|
|
// Usage:
|
|
// node scripts/release-registry-versions.mjs fetch <pkg...>
|
|
// Prints a JSON object mapping each package name to its published
|
|
// versions array. Packages that are missing from the registry (or fail
|
|
// to resolve) map to [].
|
|
//
|
|
// node scripts/release-registry-versions.mjs assert-absent <version> <pkg...>
|
|
// Freshly checks that <version> is not published for any <pkg>. Exits 0
|
|
// when absent everywhere; prints the offending package@version pairs to
|
|
// stderr and exits 1 otherwise.
|
|
|
|
import { execFile } from "node:child_process";
|
|
|
|
const CONCURRENCY = Number(process.env.RELEASE_REGISTRY_CONCURRENCY || 10);
|
|
if (!Number.isInteger(CONCURRENCY) || CONCURRENCY < 1) {
|
|
console.error("RELEASE_REGISTRY_CONCURRENCY must be a positive integer.");
|
|
process.exit(2);
|
|
}
|
|
|
|
function npmView(args) {
|
|
return new Promise((resolve) => {
|
|
execFile("npm", ["view", ...args], { encoding: "utf8" }, (error, stdout) => {
|
|
if (error) {
|
|
resolve(null);
|
|
return;
|
|
}
|
|
resolve(stdout.trim());
|
|
});
|
|
});
|
|
}
|
|
|
|
async function mapWithConcurrency(items, limit, fn) {
|
|
const results = new Array(items.length);
|
|
let next = 0;
|
|
|
|
async function worker() {
|
|
while (next < items.length) {
|
|
const index = next;
|
|
next += 1;
|
|
results[index] = await fn(items[index]);
|
|
}
|
|
}
|
|
|
|
const workers = [];
|
|
for (let i = 0; i < Math.min(limit, items.length); i += 1) {
|
|
workers.push(worker());
|
|
}
|
|
await Promise.all(workers);
|
|
return results;
|
|
}
|
|
|
|
async function fetchVersions(packageNames) {
|
|
const versionLists = await mapWithConcurrency(packageNames, CONCURRENCY, async (packageName) => {
|
|
const raw = await npmView([packageName, "versions", "--json"]);
|
|
if (!raw) return [];
|
|
try {
|
|
const parsed = JSON.parse(raw);
|
|
return Array.isArray(parsed) ? parsed : [parsed];
|
|
} catch {
|
|
return [];
|
|
}
|
|
});
|
|
|
|
const map = {};
|
|
packageNames.forEach((packageName, index) => {
|
|
map[packageName] = versionLists[index];
|
|
});
|
|
return map;
|
|
}
|
|
|
|
async function assertAbsent(version, packageNames) {
|
|
const resolved = await mapWithConcurrency(packageNames, CONCURRENCY, async (packageName) => {
|
|
const raw = await npmView([`${packageName}@${version}`, "version"]);
|
|
return raw === version ? packageName : null;
|
|
});
|
|
|
|
return resolved.filter((packageName) => packageName !== null);
|
|
}
|
|
|
|
const [mode, ...rest] = process.argv.slice(2);
|
|
|
|
if (mode === "fetch") {
|
|
if (rest.length === 0) {
|
|
console.error("usage: release-registry-versions.mjs fetch <pkg...>");
|
|
process.exit(2);
|
|
}
|
|
const map = await fetchVersions(rest);
|
|
process.stdout.write(`${JSON.stringify(map)}\n`);
|
|
} else if (mode === "assert-absent") {
|
|
const [version, ...packageNames] = rest;
|
|
if (!version || packageNames.length === 0) {
|
|
console.error("usage: release-registry-versions.mjs assert-absent <version> <pkg...>");
|
|
process.exit(2);
|
|
}
|
|
const existing = await assertAbsent(version, packageNames);
|
|
if (existing.length > 0) {
|
|
for (const packageName of existing) {
|
|
console.error(`npm version ${packageName}@${version} already exists.`);
|
|
}
|
|
process.exit(1);
|
|
}
|
|
} else {
|
|
console.error("usage: release-registry-versions.mjs <fetch|assert-absent> ...");
|
|
process.exit(2);
|
|
}
|