Files
paperclip/doc/preview-release-artifacts.md
Devin FoleyandPaperclip 8ee8f1fd6e ci: retire recurring public cloud image builds (#13827)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core publishes standard images and source verification for
downstream services.
> - Managed services can now compose private images from the signed
standard image.
> - Core still builds a second public cloud image on every master push
and release.
> - That duplicate producer consumes build capacity and retains an
obsolete readiness contract.
> - This pull request retires recurring cloud publication while
preserving the standard producer and rollback artifacts.

## Linked Issues or Issue Description

Refs #13797 and #13789. Related: #12856 changes image dependency
packaging; it does not retire this producer.

**What existing behavior does this improve?**

Core's recurring Docker publication and Cloud readiness workflow.

**Current behavior**

Master pushes call the legacy cloud publisher from Cloud readiness.
Release tags and manual Docker runs call it too. Canary promotion also
requires the legacy image.

**Proposed behavior**

Publish standard Core images and retain `Cloud source verified v1`. Let
downstream services build their managed image. Keep explicit commit
previews and existing images available.

## What Changed

- Remove `docker-cloud.yml`, its master and release callers, and its
unused cache selector.
- Remove the legacy image/migrator wait and `Cloud deployable v1` job.
Keep the full source verification workflow and exact source-proof name.
- Make canary promotion inspect and promote the standard image only.
- Preserve signed standard-image publication, direct migrator
publication, and explicit `release.yml` previews. The preview path still
uses the Dockerfile `cloud` target.
- Update workflow, preview, build-stamp, and packaging tests. Exercise
the promotion shell with mocked registry commands, including
missing-image and missing-tag cases.
- Document frozen legacy aliases, consumer requirements, preview
compatibility, and rollback retention.

## Verification

- All 377 workflow tests pass: `node --test
.github/scripts/tests/*.test.mjs`.
- All 129 release-registry tests pass: `pnpm test:release-registry`.
- Focused source-proof, standard-image, preview, and workflow tests
pass: 256 tests.
- Focused image packaging/build-stamp tests pass: 16 tests.
- Actionlint passes on all three changed workflow files. `git diff
--check` passes.
- Full local `pnpm build` and `pnpm -r typecheck` pass.
- The policy follow-up updates an old assertion that required the
removed readiness job. All 37 source-proof/release-workflow tests pass
locally.
- Full local `pnpm test:run` did not complete successfully while the Mac
ran out of disk space. No full-suite pass is claimed. Removed 1.2 GiB of
generated Cargo output from this isolated worktree with `cargo clean`.
GitHub CI passed on the final head: 52 successful checks and 2 optional
skips.
- Fresh Greptile review for `4f5fe1951f0bd7f7739cf6655d395ff78f1ed944`:
**5/5**, successful current-head check, zero review threads.
- September 23 refresh: the unchanged PR head merges cleanly with
current master `db8f8fe5b73a2697684a30261b0d306a9c631aba`. In an
isolated temporary worktree, all 377 workflow tests and 29
release/preview tests pass on the combined tree. `git diff --cached
--check` passes.
- Refreshed Actionlint workflow validation passes with ShellCheck
disabled. Full Actionlint reports the same 10 existing ShellCheck
diagnostics as master, with no added diagnostics. No source changes or
new PR commits were needed.
- The full local build/typecheck and current-head Linux CI results above
remain the verification for the unchanged PR head. They were not rerun
for this metadata-only refresh. No image publication or tenant
deployment was initiated for this refresh.

## Risks

**Deployment prerequisite satisfied (September 23):** The combined
cleanup release is deployed to staging and production, and production
Support is verified. Active managed-fleet automation uses standard-image
composition. Explicit immutable previews remain supported by the
retained preview publisher. This PR is ready for maintainer review; keep
auto-merge disabled and wait for explicit merge authorization.

- A consumer still selecting `Cloud deployable v1` will stop advancing
at the last legacy-ready commit. Confirm active automatic consumers use
the standard-image composition contract before merge.
- Legacy cloud release-channel aliases stop advancing. Standard
self-hosted aliases continue.
- This PR deletes no registry images, cache tags, migrators,
credentials, or runner infrastructure. Existing immutable releases
remain usable for rollback.
- Explicit legacy previews remain for commit-specific operator
deployments. Retiring that compatibility path requires a separate
consumer migration.
- These changes affect CI publication, not database schema or
application behavior.

## Model Used

OpenAI Codex, GPT-6. The runtime does not expose a more specific model
identifier or context-window size. Used repository inspection,
reasoning, code editing, shell tools, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-23 19:35:19 -07:00

9.5 KiB

Preview deployment artifacts

The preview channel in .github/workflows/release.yml builds deployment artifacts for one immutable source commit. It does not create a GitHub release, move a source branch, or advance any stable, beta, nightly, or canary alias.

Dispatch release.yml on master with these inputs:

Input Value
channel preview
source_ref Full lowercase 40-character commit SHA in this repository
request_id UUID v4 identifying the operator's deployment request
preview_migrator true when exact-source DB/shared packages are needed
dry_run false

The workflow title is Stack deploy <request_id> build. Consumers must find a run by this identity, not by the latest run. Preview builds reject workflow definitions that do not run from master.

Outputs and reuse

The image uses ghcr.io/paperclipai/paperclip:sha-<FULL_SHA>-cloud. This explicit operator path is retained after retirement of the recurring public -cloud publisher. Existing images remain reusable; missing images still build the cloud Dockerfile target. It is separate from private image composition. Full-SHA tags keep separate commits with the same short prefix isolated. Normal release images retain their existing short-tag convention. Build arguments carry the full commit SHA. Preview builds do not import or overwrite the shared release cache or release aliases. Missing images are built for Linux amd64, matching managed deployments.

When requested, both @paperclipai/shared and @paperclipai/db use 0.0.0-preview.g<FULL_SHA>. Workspace dependencies are pinned to exact versions. Packages carry gitHead and paperclipPreviewCommit source identity. npm publishes them under the preview dist-tag only. Normal consumers of latest or canary continue to select normal releases.

Registry 404 responses mean missing artifacts. Authentication errors, outages, or existing package identity mismatches fail the workflow. Retries reuse matching published artifacts, including a shared package published before a DB publish failure. Allow npm's visibility polling to finish before retrying.

The publisher validates both package archives first, then submits each missing package without waiting for the other to become visible. One visibility poll checks both accepted packages, so their registry propagation delays overlap. The publisher succeeds only after both packages pass the identity and distribution-pin checks. A visibility timeout names the package still missing.

The final stack-deploy-result artifact contains result.json with contract version 1, request ID, SHA, stage build, and status ready. It expires after 30 days. This confirms artifact availability; it does not certify a tenant deploy.

Publishing configuration and isolation

Migrator publication on merge

cloud-migrator-artifacts.yml publishes a signed exact-source DB/shared bundle on each canonical master push, independently of image publication and npm. See Direct cloud migrator artifacts below. Downstream deployment tooling must verify source, image, and migrator separately.

The manual npm compatibility path remains available: dispatch release.yml on master with channel=cloud-migrator, the full SHA as source_ref, a new UUID v4 as request_id, and dry_run=false. preview_migrator is not required for this channel. Existing packages are verified and reused. Preview and migrator-only runs use separate workflow concurrency groups. Only their npm publication jobs share a group for the same SHA. This prevents duplicate publication without making the migrator wait for a preview image. Different SHAs remain independent.

Publisher identity

Configure npm trusted publishing for both packages with repository paperclipai/paperclip, workflow release.yml, and environment npm-canary. The image publisher uses the same environment, whose deployment branch policy permits only master. Both publishers also check the workflow ref before running. This uses the existing publisher identity rather than requiring another workflow registration. The job uses npm with OIDC trusted publishing support and provenance. The environment's existing protections still apply.

Package compilation runs in a separate job with read-only repository access and no npm, cloud-admin, or provider credentials. Trusted tooling from master packs the requested source checkout. The existing bundled-package helper takes patch configuration from that source checkout. Build artifacts contain only the two fixed package tarballs.

Publishing runs on a fresh runner with trusted tooling, without a checkout of the requested branch. It checks package identity and exact dependencies, rejects archive path aliases, and publishes with lifecycle scripts disabled and an explicit registry and dist-tag. Image builds also run without registry write access and export a Docker archive. A separate trusted publisher loads that archive as data, verifies its full revision, platform, and image ID, then pushes only the SHA tag. It never runs the image. Dependency resolution disables scripts and pnpmfile hooks. Preview actions are pinned to full commit SHAs.

The deploying control plane must independently verify package integrity, source identity, SQL and migration journal contents, and schema compatibility. Publish this workflow support before enabling an operator CLI that depends on it. Test normal release selection and a new migration-bearing branch in staging before allowing production use.

Local checks

node --test scripts/preview-artifacts.test.mjs
pnpm test:release-registry
pnpm -r typecheck
pnpm test:run
pnpm build

For a package build without publishing, install dependencies in a disposable checkout at an exact commit, then use the trusted helper:

node scripts/preview-artifacts.mjs pack /path/to/source /path/to/output FULL_SHA

This executes source build scripts. Keep output outside the repository and use an environment without publishing or cloud-admin credentials.

Direct cloud migrator artifacts

cloud-migrator-artifacts.yml builds the DB and shared preview packages on each canonical master push. A manual run also requires master and uses its exact commit. This workflow runs on GitHub-hosted runners. It has no PR trigger.

The build resolves a complete npm lockfile from the two local archives. It then pins their download URLs to immutable, content-addressed objects. The cloud migration runner can use npm ci with this lockfile before either new package version is available on npm. Existing external dependencies still come from npm and carry SHA-512 integrity pins. Package lifecycle scripts remain disabled. Before upload, the build job smoke-installs the real archives and their complete external and bundled dependency graph with an empty npm cache. It imports both installed packages. This check uses local archive URLs because public objects do not exist yet; all versions and integrity pins remain unchanged.

Artifacts use the existing runner-history S3 bucket and CloudFront distribution, under the separate cloud-migrators/v1/ prefix. The manifest at https://d1p6rlowie26tp.cloudfront.net/cloud-migrators/v1/<full-sha>/manifest.json records the full source SHA, exact preview version, and the size, URL, and SHA-512 hash of each archive and the lockfile. Blob URLs include the content hash. The publisher validates the complete bundle before any write, writes all blobs before the manifest, and verifies downloads through the public endpoint. A retry reuses a complete existing manifest after verification. The publisher also creates a GitHub/Sigstore build-provenance attestation for the manifest before upload. This independently binds all package and lockfile content hashes to the canonical workflow, master ref, repository identity, and source commit. Cloud must verify this signature and its certificate claims before accepting the executable archives; hashes served by the artifact store alone are not sufficient provenance.

The build job has no AWS credential. The publish job downloads only the four fixed files, validates them, and uploads them without executing their code. The dedicated paperclip-cloud-migrator-github OIDC role trusts only repo:paperclipai/paperclip:ref:refs/heads/master. Its policy permits prefix listing and conditional PutObject calls in this one prefix. It permits no object deletion or overwrite. PRs, including allowlisted PRs, cannot assume it.

The deploy policies are checked in under .github/cloud-migrator-deploy/:

  • trust-policy.json: the role trust policy.
  • upload-policy.json: the role's inline permission policy.
  • cloudfront-read-statement.json: append this statement to the existing bucket policy, preserving its other statements and public-access blocks.

There is no lifecycle expiry on this prefix. Keep referenced artifacts for rollback; deleting them can prevent a fresh migrator install for an old release. Downstream deployment consumers verify the signed direct bundle after its exact-source publisher succeeds. Source verification, image identity, archive integrity and migration compatibility remain required. The retired public Cloud deployable v1 gate no longer waits for this bundle. Explicit npm previews and manual migrator runs remain available for compatible consumers and rollback. See doc/cloud-build-readiness.md for the source proof and retirement boundary.

Local verification:

node --test scripts/cloud-migrator-artifacts.test.mjs
node scripts/cloud-migrator-artifacts.mjs verify <full-sha>