mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-02 02:07:25 +08:00
ci: halve the isolated native Runner check by seeding it from the public master build cache (#13736)
## What Recurring CI health check (PAP-31): on the most recent fully-green PR run (`cb703ac`, run [35519542997](https://github.com/paperclipai/paperclip/actions/runs/35519542997)), the slowest check was **Compile isolated native Runner** at **452s** — ahead of the largest test shards (379s). Every run recompiled the full Rust dependency tree from zero, even though `docker.yml` already refreshes a **public** `mode=max` BuildKit cache (`ghcr.io/paperclipai/paperclip:buildcache-{amd64,arm64}`) on every master push, containing exactly these layers. This PR seeds **only the baseline build** with that registry cache, via anonymous pull. **Measured on this PR's own CI (which exercises the seeded path): the check completed in 123s, down from 452s — a 73% reduction, ~5.5 minutes saved per run.** ## Thinking Path Cost breakdown of the 452s from the job log: `cargo chef cook` dependency compile 214.7s, local cache export 43.1s, runner-core build 37.0s, metadata proof layer 36.8s, `cargo install cargo-chef` 36.4s, rebuild-verification build ~65s, setup/teardown ~20s. The dependency compile and toolchain layers are identical to what the production `docker.yml` build already caches publicly on every master push, so recompiling them here bought no signal — the check's real assertions live in the *verification* build, not the baseline. A first attempt used `actions/cache` plus a master `push` trigger, but the CI bot's GitHub App lacks `workflows` permission; the registry-cache approach is strictly better anyway (shared across PRs immediately, no 10GB Actions-cache quota pressure, no workflow change). ## What Changed - `scripts/check-docker-runner-cache.sh`: the baseline build now adds `--cache-from type=registry,ref=ghcr.io/paperclipai/paperclip:buildcache-{amd64|arm64}` (selected by host arch). `RUNNER_CHECK_SEED_CACHE` overrides the ref, or set it empty to force the old cold path. The script header documents the anonymous external read. - `.github/workflows/docker-runner-check.yml` (comment-only): the stale "no external cache" note now describes the anonymous GHCR seed and the verification build's local-cache-only isolation. This was pushed in a follow-up commit with workflow-edit permissions; the original CI-bot token could not touch workflow files. No Dockerfile stages or verification assertions changed. ## Verification - This PR's own `Compile isolated native Runner` check runs the seeded path (the script is in the workflow's trigger paths): **passed in 123s** vs the 452s baseline. - The rebuild-verification semantics are untouched: it still runs on a **fresh builder** importing **only the local cache exported by this run's baseline**, so it proves exactly what it proved before — that the runner image rebuilds reproducibly from this run's own exported layers. - Verified `ghcr.io/paperclipai/paperclip:buildcache-amd64` is anonymously readable (unauthenticated manifest pull succeeds), so the check gains no credential or secret dependency. ## Risks - **Stale or missing seed cache:** if the GHCR ref is unreachable, private, or garbage-collected, BuildKit logs a warning and falls back to the pre-PR cold compile — the check gets slower, never wrong. `RUNNER_CHECK_SEED_CACHE=""` restores the cold path explicitly. - **Cache trust:** the seed only accelerates the *baseline* build; the verification build still runs on a fresh builder against only this run's locally exported cache, so a stale or poisoned registry cache cannot make verification pass spuriously. The ref lives under `ghcr.io/paperclipai/*`, written only by repo CI on master pushes. ## Model Used Claude Fable 5 (`claude-fable-5`) via Paperclip agent **Bender (Fable)**, issue PAP-31. --------- Co-authored-by: Bender (Fable) <bender-fable@paperclip.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Bender
Claude Fable 5
parent
ded156a904
commit
3c2bc4f546
@@ -33,6 +33,9 @@ jobs:
|
||||
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
|
||||
# Compile the real target, then change source in a disposable context.
|
||||
# A fresh builder must import dependencies and produce changed binary metadata.
|
||||
# No registry credentials, external cache, or image publication.
|
||||
# The baseline build anonymously seeds from the public BuildKit cache at
|
||||
# ghcr.io/paperclipai/paperclip:buildcache-{amd64,arm64}; the verification
|
||||
# build imports only this run's locally exported cache on a fresh builder.
|
||||
# No registry credentials or image publication.
|
||||
- name: Verify native build and dependency cache reuse
|
||||
run: bash scripts/check-docker-runner-cache.sh
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build the real Docker target on two fresh builders using an exported cache.
|
||||
# Export only metadata, avoiding a multi-gigabyte test image in the daemon.
|
||||
# External access: the baseline build anonymously reads the public BuildKit
|
||||
# cache at ghcr.io/paperclipai/paperclip:buildcache-{amd64,arm64} (see
|
||||
# RUNNER_CHECK_SEED_CACHE below). No credentials are used or required, and
|
||||
# nothing is pushed.
|
||||
set -euo pipefail
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
probe_dir="$(mktemp -d "${TMPDIR:-/tmp}/paperclip-runner-cache.XXXXXX")"
|
||||
@@ -34,7 +38,27 @@ build_proof() {
|
||||
docker buildx build --builder "$builder" --file "$probe_dir/cache-probe.Dockerfile" --target cache-proof-export --output "type=local,dest=$probe_dir/$result" --progress plain "$@" . 2>&1 | tee "$probe_dir/$result.log"
|
||||
}
|
||||
docker buildx create --name "$baseline_builder" --driver docker-container
|
||||
build_proof baseline "$baseline_builder" --cache-to "type=local,dest=$probe_dir/cache,mode=max"
|
||||
# Seed only the baseline with the public BuildKit cache that docker.yml
|
||||
# refreshes on every master push. The rust stages consume none of that
|
||||
# build's args, so their layer keys match, and mode=max re-exports the
|
||||
# imported layers into $probe_dir/cache — the verification build below
|
||||
# still proves what it always proved from this run's exported cache
|
||||
# alone, on a fresh builder. Anonymous pull only, nothing is pushed; a
|
||||
# missing or unreachable ref is a BuildKit warning and the baseline
|
||||
# degrades to the previous cold compile. Set RUNNER_CHECK_SEED_CACHE to
|
||||
# another ref, or to the empty string to force the cold path.
|
||||
if [[ -z "${RUNNER_CHECK_SEED_CACHE+x}" ]]; then
|
||||
case "$(uname -m)" in
|
||||
x86_64) RUNNER_CHECK_SEED_CACHE="ghcr.io/paperclipai/paperclip:buildcache-amd64" ;;
|
||||
aarch64 | arm64) RUNNER_CHECK_SEED_CACHE="ghcr.io/paperclipai/paperclip:buildcache-arm64" ;;
|
||||
*) RUNNER_CHECK_SEED_CACHE="" ;;
|
||||
esac
|
||||
fi
|
||||
seed_args=()
|
||||
if [[ -n "$RUNNER_CHECK_SEED_CACHE" ]]; then
|
||||
seed_args=(--cache-from "type=registry,ref=${RUNNER_CHECK_SEED_CACHE}")
|
||||
fi
|
||||
build_proof baseline "$baseline_builder" ${seed_args[@]+"${seed_args[@]}"} --cache-to "type=local,dest=$probe_dir/cache,mode=max"
|
||||
# Removing the first builder proves the second build cannot use daemon-local
|
||||
# state, and releases its disk space before importing the exported cache.
|
||||
docker buildx rm "$baseline_builder"
|
||||
|
||||
Reference in New Issue
Block a user