mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-02 02:07:25 +08:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Cloud needs a verified image for each merged source commit. > - Fresh builders restore compiled native dependencies from registry caches. > - The current workflow imports up to eleven historical cache manifests at once. > - Live builds missed native layers that a fresh builder reused from one manifest. > - This PR selects the nearest available cache and tests reuse across fresh builders. ## Linked Issues or Issue Description Refs #13329 and #13330. A search of open cache PRs found no duplicate of this change. **What existing behavior does this improve?** Remote Docker cache reuse on fresh Cloud image builders. **Current behavior** [Cloud run 34714483272](https://github.com/paperclipai/paperclip/actions/runs/34714483272/job/103609096836) imported the previous cache manifest successfully but rebuilt `cargo-chef` and Rust dependencies. The dependency compile took 3m43s. The preceding image build had already exported those layers. A controlled [fresh-builder diagnostic](https://github.com/paperclipai/paperclip/actions/runs/34715336530) used the same source and registry cache. The single-manifest job reused both layers immediately. The multiple-manifest job rebuilt them and failed the cache assertion. Both jobs used GitHub-hosted runners with read-only access. **Proposed behavior** Inspect cache manifests in first-parent order and import only the nearest available one. Keep full-SHA cache exports, the ten-commit search bound, and the legacy fallback. If caches cannot be read, permit a cold build. **Reason and benefit** Avoid the observed cache misses without changing image contents or builder sizes. Expected savings include about four minutes of native tool/dependency compilation when those inputs are unchanged. The final merge-to-deployable gain still needs a post-merge measurement. **Breaking changes** No image, artifact, deployment, or runner-routing contract changes. ## What Changed - Select one available ancestor cache after Docker login and Buildx setup. - Preserve separate writable cache tags for each full source SHA. - Test cache ordering, missing caches, registry errors, and workflow integration. - Add the selector tests to the existing release-registry suite. - Export a local test cache, remove the first builder, and verify a source rebuild on a fresh builder. - Document cache selection and the stronger Docker check. ## Verification - Passed 456 focused workflow, routing, readiness, preview-artifact, and cache-selector tests. - Passed shell syntax, ShellCheck for the changed probe, actionlint workflow validation, and `git diff --check`. actionlint's shell checks were disabled for the workflow validation because unchanged migration-label commands trigger existing SC2012 notes. - The fresh-builder registry diagnostic proves the single-cache behavior. The [permanent two-builder probe passed](https://github.com/paperclipai/paperclip/actions/runs/34715771048/job/103612624090), including a changed real binary and dependency-declaration invalidation. - Passed all 35 latest-head checks (green or intentionally skipped), including full typecheck, test, build, and browser suites in [PR CI run 34715771217](https://github.com/paperclipai/paperclip/actions/runs/34715771217). - The real selector CLI inspected registry metadata and chose the nearest available ancestor cache. - Fresh Greptile review is 5/5 with no open findings. The PR title was corrected to meet the source-change naming rule; the review check passed after that correction. - Local full-suite runs and Docker builds are unavailable because the local Docker daemon is unresponsive after disk exhaustion. CI provides the Linux verification. ## Risks - Missing or unreadable caches cause a slower cold build. The selector logs that condition and preserves image publication. - Inspecting several missing ancestors adds lookup time. Each lookup has a ten-second timeout and the search is bounded. - The Docker test now exports a local cache. It removes the first builder before starting the second to release disk space, then cleans up its builders and files. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The exact serving model ID and context window are not exposed by this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
66 lines
2.8 KiB
JavaScript
66 lines
2.8 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import test from "node:test";
|
|
import { cloudCacheCandidates, selectCloudCache } from "./select-cloud-cache.mjs";
|
|
|
|
const image = "ghcr.io/paperclipai/paperclip";
|
|
const commits = ["a".repeat(40), "b".repeat(40), "c".repeat(40)];
|
|
const candidates = cloudCacheCandidates(image, commits);
|
|
|
|
test("a same-SHA rerun imports only its existing cache", async () => {
|
|
const inspected = [];
|
|
const source = await selectCloudCache(image, commits, {
|
|
exists: async (ref) => { inspected.push(ref); return true; }, log() {},
|
|
});
|
|
assert.equal(source, `type=registry,ref=${candidates[0]}`);
|
|
assert.deepEqual(inspected, candidates.slice(0, 1));
|
|
});
|
|
|
|
test("a new merge imports only its nearest available ancestor", async () => {
|
|
const inspected = [];
|
|
const source = await selectCloudCache(image, commits, {
|
|
exists: async (ref) => { inspected.push(ref); return ref === candidates[1]; }, log() {},
|
|
});
|
|
assert.equal(source, `type=registry,ref=${candidates[1]}`);
|
|
assert.deepEqual(inspected, candidates.slice(0, 2));
|
|
assert.equal(source.includes("\n"), false);
|
|
});
|
|
|
|
test("a still-building parent falls back to an older completed cache", async () => {
|
|
assert.equal(await selectCloudCache(image, commits, {
|
|
exists: async (ref) => ref === candidates[2], log() {},
|
|
}), `type=registry,ref=${candidates[2]}`);
|
|
});
|
|
|
|
test("the legacy cache is used only if no SHA cache exists", async () => {
|
|
const inspected = [];
|
|
assert.equal(await selectCloudCache(image, commits, {
|
|
exists: async (ref) => { inspected.push(ref); return ref === candidates.at(-1); }, log() {},
|
|
}), `type=registry,ref=${candidates.at(-1)}`);
|
|
assert.deepEqual(inspected, candidates);
|
|
});
|
|
|
|
test("missing caches permit a cold build", async () => {
|
|
assert.equal(await selectCloudCache(image, commits, { exists: async () => false, log() {} }), "");
|
|
});
|
|
|
|
test("a failed lookup can fall back without failing image publication", async () => {
|
|
const messages = [];
|
|
assert.equal(await selectCloudCache(image, commits, {
|
|
exists: async (ref) => {
|
|
if (ref === candidates[0]) throw new Error("registry temporarily unavailable");
|
|
return true;
|
|
},
|
|
log: (message) => messages.push(message),
|
|
}), `type=registry,ref=${candidates[1]}`);
|
|
assert.match(messages[0], /Could not inspect cloud cache/);
|
|
});
|
|
|
|
test("ancestry is bounded, deduplicated, and rejects output injection", () => {
|
|
const many = Array.from({ length: 20 }, (_, i) => i.toString(16).padStart(40, "0"));
|
|
assert.equal(cloudCacheCandidates(image, many).length, 11);
|
|
assert.deepEqual(cloudCacheCandidates(image, [commits[0], commits[0]]), [candidates[0], candidates.at(-1)]);
|
|
assert.throws(() => cloudCacheCandidates(`${image}\nsource=untrusted`, commits));
|
|
assert.throws(() => cloudCacheCandidates(image, ["master"]));
|
|
assert.throws(() => cloudCacheCandidates(image, []));
|
|
});
|