## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core publishes standard images and source verification for downstream services. > - Managed services can now compose private images from the signed standard image. > - Core still builds a second public cloud image on every master push and release. > - That duplicate producer consumes build capacity and retains an obsolete readiness contract. > - This pull request retires recurring cloud publication while preserving the standard producer and rollback artifacts. ## Linked Issues or Issue Description Refs #13797 and #13789. Related: #12856 changes image dependency packaging; it does not retire this producer. **What existing behavior does this improve?** Core's recurring Docker publication and Cloud readiness workflow. **Current behavior** Master pushes call the legacy cloud publisher from Cloud readiness. Release tags and manual Docker runs call it too. Canary promotion also requires the legacy image. **Proposed behavior** Publish standard Core images and retain `Cloud source verified v1`. Let downstream services build their managed image. Keep explicit commit previews and existing images available. ## What Changed - Remove `docker-cloud.yml`, its master and release callers, and its unused cache selector. - Remove the legacy image/migrator wait and `Cloud deployable v1` job. Keep the full source verification workflow and exact source-proof name. - Make canary promotion inspect and promote the standard image only. - Preserve signed standard-image publication, direct migrator publication, and explicit `release.yml` previews. The preview path still uses the Dockerfile `cloud` target. - Update workflow, preview, build-stamp, and packaging tests. Exercise the promotion shell with mocked registry commands, including missing-image and missing-tag cases. - Document frozen legacy aliases, consumer requirements, preview compatibility, and rollback retention. ## Verification - All 377 workflow tests pass: `node --test .github/scripts/tests/*.test.mjs`. - All 129 release-registry tests pass: `pnpm test:release-registry`. - Focused source-proof, standard-image, preview, and workflow tests pass: 256 tests. - Focused image packaging/build-stamp tests pass: 16 tests. - Actionlint passes on all three changed workflow files. `git diff --check` passes. - Full local `pnpm build` and `pnpm -r typecheck` pass. - The policy follow-up updates an old assertion that required the removed readiness job. All 37 source-proof/release-workflow tests pass locally. - Full local `pnpm test:run` did not complete successfully while the Mac ran out of disk space. No full-suite pass is claimed. Removed 1.2 GiB of generated Cargo output from this isolated worktree with `cargo clean`. GitHub CI passed on the final head: 52 successful checks and 2 optional skips. - Fresh Greptile review for `4f5fe1951f0bd7f7739cf6655d395ff78f1ed944`: **5/5**, successful current-head check, zero review threads. - September 23 refresh: the unchanged PR head merges cleanly with current master `db8f8fe5b73a2697684a30261b0d306a9c631aba`. In an isolated temporary worktree, all 377 workflow tests and 29 release/preview tests pass on the combined tree. `git diff --cached --check` passes. - Refreshed Actionlint workflow validation passes with ShellCheck disabled. Full Actionlint reports the same 10 existing ShellCheck diagnostics as master, with no added diagnostics. No source changes or new PR commits were needed. - The full local build/typecheck and current-head Linux CI results above remain the verification for the unchanged PR head. They were not rerun for this metadata-only refresh. No image publication or tenant deployment was initiated for this refresh. ## Risks **Deployment prerequisite satisfied (September 23):** The combined cleanup release is deployed to staging and production, and production Support is verified. Active managed-fleet automation uses standard-image composition. Explicit immutable previews remain supported by the retained preview publisher. This PR is ready for maintainer review; keep auto-merge disabled and wait for explicit merge authorization. - A consumer still selecting `Cloud deployable v1` will stop advancing at the last legacy-ready commit. Confirm active automatic consumers use the standard-image composition contract before merge. - Legacy cloud release-channel aliases stop advancing. Standard self-hosted aliases continue. - This PR deletes no registry images, cache tags, migrators, credentials, or runner infrastructure. Existing immutable releases remain usable for rollback. - Explicit legacy previews remain for commit-specific operator deployments. Retiring that compatibility path requires a separate consumer migration. - These changes affect CI publication, not database schema or application behavior. ## Model Used OpenAI Codex, GPT-6. The runtime does not expose a more specific model identifier or context-window size. Used repository inspection, reasoning, code editing, shell tools, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
9.5 KiB
Preview deployment artifacts
The preview channel in .github/workflows/release.yml builds deployment
artifacts for one immutable source commit. It does not create a GitHub release,
move a source branch, or advance any stable, beta, nightly, or canary alias.
Dispatch release.yml on master with these inputs:
| Input | Value |
|---|---|
channel |
preview |
source_ref |
Full lowercase 40-character commit SHA in this repository |
request_id |
UUID v4 identifying the operator's deployment request |
preview_migrator |
true when exact-source DB/shared packages are needed |
dry_run |
false |
The workflow title is Stack deploy <request_id> build. Consumers must find a
run by this identity, not by the latest run. Preview builds reject workflow
definitions that do not run from master.
Outputs and reuse
The image uses ghcr.io/paperclipai/paperclip:sha-<FULL_SHA>-cloud.
This explicit operator path is retained after retirement of the recurring public
-cloud publisher. Existing images remain reusable; missing images still build
the cloud Dockerfile target. It is separate from private image composition.
Full-SHA tags keep separate commits with the same short prefix isolated. Normal
release images retain their existing short-tag convention. Build arguments carry the full commit SHA.
Preview builds do not import or overwrite the shared release cache or release
aliases. Missing images are built for Linux amd64, matching managed deployments.
When requested, both @paperclipai/shared and @paperclipai/db use
0.0.0-preview.g<FULL_SHA>. Workspace dependencies are pinned to exact versions.
Packages carry gitHead and paperclipPreviewCommit source identity. npm publishes
them under the preview dist-tag only. Normal consumers of latest or canary
continue to select normal releases.
Registry 404 responses mean missing artifacts. Authentication errors, outages, or existing package identity mismatches fail the workflow. Retries reuse matching published artifacts, including a shared package published before a DB publish failure. Allow npm's visibility polling to finish before retrying.
The publisher validates both package archives first, then submits each missing package without waiting for the other to become visible. One visibility poll checks both accepted packages, so their registry propagation delays overlap. The publisher succeeds only after both packages pass the identity and distribution-pin checks. A visibility timeout names the package still missing.
The final stack-deploy-result artifact contains result.json with contract
version 1, request ID, SHA, stage build, and status ready. It expires after
30 days. This confirms artifact availability; it does not certify a tenant deploy.
Publishing configuration and isolation
Migrator publication on merge
cloud-migrator-artifacts.yml publishes a signed exact-source DB/shared bundle
on each canonical master push, independently of image publication and npm.
See Direct cloud migrator artifacts below.
Downstream deployment tooling must verify source, image, and migrator separately.
The manual npm compatibility path remains available: dispatch release.yml on
master with channel=cloud-migrator, the full SHA as source_ref, a new UUID v4
as request_id, and dry_run=false. preview_migrator is not required for this
channel. Existing packages are verified and reused. Preview and migrator-only
runs use separate workflow concurrency groups. Only their npm publication jobs
share a group for the same SHA. This prevents duplicate publication without
making the migrator wait for a preview image. Different SHAs remain independent.
Publisher identity
Configure npm trusted publishing for both packages with repository
paperclipai/paperclip, workflow release.yml, and environment npm-canary.
The image publisher uses the same environment, whose deployment branch policy
permits only master. Both publishers also check the workflow ref before running.
This uses the existing publisher identity rather than requiring another workflow
registration. The job uses npm with OIDC trusted publishing support and provenance.
The environment's existing protections still apply.
Package compilation runs in a separate job with read-only repository access and
no npm, cloud-admin, or provider credentials. Trusted tooling from master packs
the requested source checkout. The existing bundled-package helper takes patch
configuration from that source checkout. Build artifacts contain only the two
fixed package tarballs.
Publishing runs on a fresh runner with trusted tooling, without a checkout of the requested branch. It checks package identity and exact dependencies, rejects archive path aliases, and publishes with lifecycle scripts disabled and an explicit registry and dist-tag. Image builds also run without registry write access and export a Docker archive. A separate trusted publisher loads that archive as data, verifies its full revision, platform, and image ID, then pushes only the SHA tag. It never runs the image. Dependency resolution disables scripts and pnpmfile hooks. Preview actions are pinned to full commit SHAs.
The deploying control plane must independently verify package integrity, source identity, SQL and migration journal contents, and schema compatibility. Publish this workflow support before enabling an operator CLI that depends on it. Test normal release selection and a new migration-bearing branch in staging before allowing production use.
Local checks
node --test scripts/preview-artifacts.test.mjs
pnpm test:release-registry
pnpm -r typecheck
pnpm test:run
pnpm build
For a package build without publishing, install dependencies in a disposable checkout at an exact commit, then use the trusted helper:
node scripts/preview-artifacts.mjs pack /path/to/source /path/to/output FULL_SHA
This executes source build scripts. Keep output outside the repository and use an environment without publishing or cloud-admin credentials.
Direct cloud migrator artifacts
cloud-migrator-artifacts.yml builds the DB and shared preview packages on each
canonical master push. A manual run also requires master and uses its exact
commit. This workflow runs on GitHub-hosted runners. It has no PR trigger.
The build resolves a complete npm lockfile from the two local archives. It then
pins their download URLs to immutable, content-addressed objects. The cloud
migration runner can use npm ci with this lockfile before either new package
version is available on npm. Existing external dependencies still come from npm
and carry SHA-512 integrity pins. Package lifecycle scripts remain disabled.
Before upload, the build job smoke-installs the real archives and their complete
external and bundled dependency graph with an empty npm cache. It imports both
installed packages. This check uses local archive URLs because public objects
do not exist yet; all versions and integrity pins remain unchanged.
Artifacts use the existing runner-history S3 bucket and CloudFront distribution,
under the separate cloud-migrators/v1/ prefix. The manifest at
https://d1p6rlowie26tp.cloudfront.net/cloud-migrators/v1/<full-sha>/manifest.json
records the full source SHA, exact preview version, and the size, URL, and SHA-512
hash of each archive and the lockfile. Blob URLs include the content hash.
The publisher validates the complete bundle before any write, writes all blobs
before the manifest, and verifies downloads through the public endpoint.
A retry reuses a complete existing manifest after verification. The publisher
also creates a GitHub/Sigstore build-provenance attestation for the manifest
before upload. This independently binds all package and lockfile content hashes
to the canonical workflow, master ref, repository identity, and source commit.
Cloud must verify this signature and its certificate claims before accepting
the executable archives; hashes served by the artifact store alone are not
sufficient provenance.
The build job has no AWS credential. The publish job downloads only the four
fixed files, validates them, and uploads them without executing their code.
The dedicated paperclip-cloud-migrator-github OIDC role trusts only
repo:paperclipai/paperclip:ref:refs/heads/master. Its policy permits prefix
listing and conditional PutObject calls in this one prefix. It permits no
object deletion or overwrite. PRs, including allowlisted PRs, cannot assume it.
The deploy policies are checked in under .github/cloud-migrator-deploy/:
trust-policy.json: the role trust policy.upload-policy.json: the role's inline permission policy.cloudfront-read-statement.json: append this statement to the existing bucket policy, preserving its other statements and public-access blocks.
There is no lifecycle expiry on this prefix. Keep referenced artifacts for
rollback; deleting them can prevent a fresh migrator install for an old release.
Downstream deployment consumers verify the signed direct bundle after its
exact-source publisher succeeds. Source verification, image identity, archive
integrity and migration compatibility remain required. The retired public
Cloud deployable v1 gate no longer waits for this bundle. Explicit npm previews
and manual migrator runs remain available for compatible consumers and rollback.
See doc/cloud-build-readiness.md for the source proof and retirement boundary.
Local verification:
node --test scripts/cloud-migrator-artifacts.test.mjs
node scripts/cloud-migrator-artifacts.mjs verify <full-sha>