Files
codebase-memory-mcp/.github/workflows/release.yml
T
Martin VogelandAndrew Hundt 70b2994425 ci(release): verify every platform from one pinned checksums file
The previous commit pinned the linux/amd64 asset hash directly, which
verified the asset this job needs but hard-coded the platform: the
uname-based selection was replaced by a fixed filename, so moving the job
to another runner OS or architecture would have needed a code change and
a second pinned hash.

Pin the SHA-256 of the release's own checksums file instead, and verify
whichever asset the runner selects against it. Upstream publishes that
file for the whole release, covering linux, darwin and windows on both
amd64 and arm64, so one pinned value now covers every platform and the
uname-based selection is restored.

Match the asset by exact filename, since a substring match would also
accept the .sbom.json and .sigstore.json lines for the same asset, and
fail closed when an asset is absent from the checksums file rather than
installing it unverified.

Verified end to end against the pinned release: the checksums file
matches its pinned hash; asset selection resolves to a listed asset for
linux/amd64, linux/arm64, darwin/amd64 and darwin/arm64; the linux/amd64
asset verifies and extracts to a statically linked x86-64 ELF; and
appending one byte to the downloaded asset makes verification FAIL, so
the gate binds rather than passing vacuously.

Co-Authored-By: Andrew Hundt <ATHundt@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-13 15:59:41 +02:00

525 lines
23 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Release pipeline: lint → test → build → smoke/soak → draft → verify → publish
#
# Security (security-static + codeql-gate) starts immediately and runs in
# parallel with lint/test/build/smoke/soak. It only blocks the final verify
# step — everything else proceeds independently.
name: Release
on:
workflow_dispatch:
inputs:
version:
description: "Release version (e.g. v0.8.0)"
required: true
type: string
release_notes:
description: "Release notes (optional — auto-generated from commits if empty)"
required: false
type: string
replace:
description: "Replace existing release if it exists"
required: false
type: boolean
default: false
soak_level:
description: 'Soak: full (quick+asan), quick (10min), none'
type: choice
options: ['full', 'quick', 'none']
default: 'quick'
skip_perf:
description: "Skip performance tests (use when no pipeline logic changed)"
required: false
type: boolean
default: false
skip_tests:
description: "Skip the test phase entirely (re-release of an already test-green tree; build/smoke/soak/verify still run)"
required: false
type: boolean
default: false
permissions:
contents: read
jobs:
# ── Security (starts immediately, blocks only verify) ───────────
security:
uses: ./.github/workflows/_security.yml
secrets: inherit
# ── 1. Lint (cppcheck + clang-format) ───────────────────────────
lint:
uses: ./.github/workflows/_lint.yml
# ── 2. Tests (all platforms, full suite for release) ────────────
# skip_tests exists for RE-releases of a tree whose tests are already green:
# when only packaging/gating changed and the previous run proved the suites,
# re-running ~2h of tests adds no information. It skips ONLY this phase —
# build, smoke, soak and verify always run against the fresh artifacts.
test:
if: ${{ !inputs.skip_tests }}
needs: [lint]
uses: ./.github/workflows/_test.yml
with:
skip_perf: ${{ inputs.skip_perf }}
broad_platforms: true
shard_suites: true
# ── 3. Build all platforms ──────────────────────────────────────
# !cancelled() && !failure(): run when `test` is deliberately skipped, but
# never when lint or test actually failed.
build:
if: ${{ !cancelled() && !failure() }}
needs: [test]
permissions:
contents: read
id-token: write
attestations: write
uses: ./.github/workflows/_build.yml
with:
version: ${{ inputs.version }}
attest: true
# ── 4. Smoke test every binary ──────────────────────────────────
# !cancelled() && !failure(): GitHub propagates "skipped" TRANSITIVELY down the
# needs graph, so with skip_tests=true a skipped `test` skipped smoke and soak
# too — even though `build` overrode the same condition and succeeded. Nothing
# failed and nothing said so; the pipeline simply carried on toward publishing
# artifacts that had never been smoke-tested or soaked. Every job downstream of
# an optional phase needs this override, or the phase being optional silently
# makes the phases after it optional as well.
smoke:
if: ${{ !cancelled() && !failure() }}
needs: [build]
uses: ./.github/workflows/_smoke.yml
with:
broad_platforms: true
# ── 5. Soak tests ──────────────────────────────────────────────
soak:
if: ${{ !cancelled() && !failure() && inputs.soak_level != 'none' }}
needs: [build]
uses: ./.github/workflows/_soak.yml
with:
duration_minutes: 10
run_asan: ${{ inputs.soak_level == 'full' }}
version: ${{ inputs.version }}
# ── 6. Create DRAFT release ────────────────────────────────────
# Requires smoke to have actually SUCCEEDED, not merely "not failed". The bare
# !cancelled() && !failure() form is fail-OPEN: a skipped smoke is neither
# cancelled nor failed, so a draft was one condition away from being cut from
# binaries nobody had run. soak may legitimately be 'skipped' (soak_level=none),
# so that one is enumerated explicitly rather than lumped in.
release-draft:
needs: [smoke, soak]
if: >-
${{ !cancelled() && !failure()
&& needs.smoke.result == 'success'
&& (needs.soak.result == 'success' || needs.soak.result == 'skipped') }}
runs-on: ubuntu-latest
permissions:
contents: write
id-token: write
attestations: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
merge-multiple: true
- name: List artifacts
run: ls -la *.tar.gz *.zip *.mcpb
- name: Generate checksums
run: sha256sum *.tar.gz *.zip *.mcpb > checksums.txt
- name: Attest checksum provenance
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
with:
subject-path: checksums.txt
# publish-mcp-registry reads the *.mcpb sha256 lines from here — a
# same-run workflow artifact, not a draft-release download, so that
# job keeps contents: read and no gh dependency.
- name: Preserve checksums for the registry job
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-checksums
path: checksums.txt
# SBOM content lives in the canonical scripts/ci entry, not inline YAML
# (venue-parity contract): vendored versions are reviewable there.
- name: Generate SBOM
run: python3 scripts/ci/generate-sbom.py "${{ inputs.version }}"
- name: Attest SBOM
uses: actions/attest-sbom@c604332985a26aa8cf1bdc465b92731239ec6b9e # v4.1.0
with:
subject-path: '*.tar.gz'
sbom-path: 'sbom.json'
- name: Install cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
- name: Sign artifacts
run: |
for f in *.tar.gz *.zip *.mcpb checksums.txt; do
cosign sign-blob --yes --bundle "${f}.bundle" "$f"
done
- name: Delete existing release
if: ${{ inputs.replace }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ inputs.version }}
run: gh release delete "$VERSION" --yes --cleanup-tag || true
- name: Create tag
env:
VERSION: ${{ inputs.version }}
# Tag the DISPATCHED sha, not the checkout HEAD: the artifacts were
# built from github.sha, and a commit pushed to main mid-run must not
# move the release tag. (A floating HEAD also broke the 0.8.0 release:
# HEAD had become a commit touching .github/workflows/, and the App
# token may not create refs pointing at workflow-modifying commits.)
run: |
git tag -f "$VERSION" "$GITHUB_SHA"
git push origin "$VERSION" --force
- uses: softprops/action-gh-release@3d0d9888cb7fd7b750713d6e236d1fcb99157228 # v2
with:
tag_name: ${{ inputs.version }}
draft: true
# Semver prerelease (v0.9.1-rc.1): never the "latest" release on GitHub.
prerelease: ${{ contains(inputs.version, '-') }}
files: |
*.tar.gz
*.zip
*.mcpb
checksums.txt
sbom.json
*.bundle
body: ${{ inputs.release_notes || '' }}
generate_release_notes: ${{ inputs.release_notes == '' }}
# ── 7. Verify + Publish (requires security gate) ───────────────
verify:
needs: [release-draft, security]
# Explicit condition: a skipped ancestor (soak with soak_level=none) must
# not skip the publish chain — only real failures/cancellations block it.
if: ${{ !cancelled() && !failure() && needs.release-draft.result == 'success' }}
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Download and extract release binaries
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ inputs.version }}
run: |
ARCHIVE_DIR="$RUNNER_TEMP/release-archives"
mkdir -p "$ARCHIVE_DIR" binaries
gh release download "$VERSION" --dir "$ARCHIVE_DIR" --repo "$GITHUB_REPOSITORY" --pattern '*.tar.gz' --pattern '*.zip' --pattern '*.mcpb'
# 14 = 8 archives + 6 MCPB bundles; runtime 42 = 8×3 archive
# sidecars + 6×3 bundle members (manifest.json, LICENSE, notices).
# The bundle binaries dedupe to the archive objects byte-for-byte.
scripts/ci/extract-release-archives.sh "$ARCHIVE_DIR" binaries \
--expect-archives=14 \
--expect-binaries=14 \
--expect-runtime-files=42
- name: Security audits on all unique extracted release objects
run: |
# Audit every distinct extracted member/UI-asset byte sequence. The
# associations manifest retains each source archive's SHA-256.
# security-strings.sh detects file type and applies binary-only
# rules (URL allowlist, dangerous-command detection) only to real
# binaries; for shell scripts it still runs credential and base64
# pattern audits.
for f in binaries/objects/*; do
[ -f "$f" ] || continue
echo "--- Auditing: $(basename "$f") ---"
scripts/security-strings.sh "$f"
done
- name: VirusTotal scan
uses: crazy-max/ghaction-virustotal@936d8c5c00afe97d3d9a1af26d017cfdf26800a2 # v5.0.0
id: virustotal
with:
vt_api_key: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
files: binaries/objects/*
request_rate: 4
# ZERO tolerance: every extracted object must arrive naturally clean. Any
# malicious or suspicious verdict blocks the release with no exception.
- name: Wait for VirusTotal results
env:
VT_API_KEY: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
VT_ANALYSIS: ${{ steps.virustotal.outputs.analysis }}
VT_EXPECTED_SCAN_SET: binaries/scan-set.tsv
VT_ASSOCIATIONS: binaries/associations.tsv
VT_RESULTS_PATH: binaries/vt-results.tsv
MIN_ENGINES: 50
run: scripts/ci/check-virustotal.sh
- name: Preserve VirusTotal scope and result evidence
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: virustotal-evidence-release-${{ github.run_id }}
path: |
binaries/associations.tsv
binaries/scan-set.tsv
binaries/vt-results.tsv
if-no-files-found: warn
- name: Publish durable public VirusTotal evidence
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ inputs.version }}
VT_EXPECTED_SCAN_SET: binaries/scan-set.tsv
VT_ASSOCIATIONS: binaries/associations.tsv
VT_RESULTS_PATH: binaries/vt-results.tsv
run: bash scripts/ci/publish-vt-evidence.sh
# Renders extracted-file scan results plus archive SHA-256 provenance. It
# runs only after the exact-set zero-tolerance gate passed.
- name: Append VirusTotal scan links to release notes
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ inputs.version }}
VT_EXPECTED_SCAN_SET: binaries/scan-set.tsv
VT_ASSOCIATIONS: binaries/associations.tsv
VT_RESULTS_PATH: binaries/vt-results.tsv
run: bash scripts/ci/append-vt-notes.sh
# ── 8. Publish package wrappers (npm + PyPI) ──────────────────
# Wrappers in pkg/npm and pkg/pypi download the released binary at
# install time, so they only need a version bump (already in the repo
# at this point) — no per-release sha256 substitution.
#
# Runs against the still-DRAFT GitHub release. If publish fails, the
# release stays in draft so we can re-run without a half-shipped state.
publish-registries:
needs: [verify]
if: ${{ !cancelled() && !failure() && needs.verify.result == 'success' }}
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # for npm provenance
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# The packaged versions MUST match the dispatched release version —
# the 0.8.0 release failed here because pkg/npm still carried the
# previous release's hand-pinned version and npm refuses to publish
# over an existing version. Inject the dispatch input so a forgotten
# manual bump can never fail the pipeline again. (server.json needs
# no injection: publish-mcp-registry syncs it from pkg/npm.)
- name: Sync packaging versions to the release version
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
V="${RELEASE_VERSION#v}"
jq --arg v "$V" '.version = $v' pkg/npm/package.json > pkg/npm/package.tmp
mv pkg/npm/package.tmp pkg/npm/package.json
sed -i "s/^version = \".*\"/version = \"$V\"/" pkg/pypi/pyproject.toml
grep -q "\"version\": \"$V\"" pkg/npm/package.json
grep -q "^version = \"$V\"" pkg/pypi/pyproject.toml
echo "packaging synced to $V"
# ── npm ──────────────────────────────────────────────────
- name: Setup Node
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: '22'
registry-url: 'https://registry.npmjs.org'
- name: Publish to npm
working-directory: pkg/npm
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
RELEASE_VERSION: ${{ inputs.version }}
# A prerelease must never become npm `latest` — plain `npm install`
# would serve the RC to everyone. Publish prereleases under the
# `next` dist-tag; testers opt in via `npm i codebase-memory-mcp@next`.
#
# Idempotency: npm versions are immutable, so a re-run of this job
# after a partial success (v0.10.1: npm published, then the twine
# step died) can never republish — it 403s on its own earlier
# success and the release wedges in draft forever. If the exact
# version already exists on the registry, publishing is DONE; skip.
run: |
V="${RELEASE_VERSION#v}"
if npm view "codebase-memory-mcp@$V" version >/dev/null 2>&1; then
echo "npm already has codebase-memory-mcp@$V — skipping publish (idempotent re-run)"
elif [[ "$RELEASE_VERSION" == *-* ]]; then
npm publish --access public --provenance --tag next
else
npm publish --access public --provenance
fi
# ── PyPI ─────────────────────────────────────────────────
- name: Setup Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- name: Build PyPI distribution
working-directory: pkg/pypi
run: |
# Every byte of the publish toolchain is bound by hash: version pins
# alone still let a compromised index serve different content, and a
# bare `pip install` resolves the whole transitive graph at run time
# (OSSF Scorecard pinned-dependencies). requirements-publish.txt is
# generated on a linux/amd64 python:3.12 image — see its header for
# the exact regeneration command.
#
# twine must stay >= 7 and hatchling is pinned in pyproject.toml:
# `python -m build` resolves its backend fresh in an isolated env, and
# a hatchling emitting Metadata-Version 2.5 against twine 6.2.0 is
# what broke the v0.10.1 publish.
python -m pip install --require-hashes -r requirements-publish.txt
python -m build
twine check dist/*
- name: Publish to PyPI
working-directory: pkg/pypi
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
# --skip-existing: PyPI files are immutable like npm versions; a
# re-run after partial success must treat already-uploaded files
# as done, not as a fatal collision.
run: twine upload --non-interactive --skip-existing dist/*
# ── 8b. Publish server.json to the official MCP Registry ──────
# Runs after npm + PyPI so the registry can verify package ownership
# (npm: `mcpName` field in package.json; PyPI: `mcp-name:` marker in
# the package README). Authenticates with GitHub Actions OIDC — no
# token, no interactive device flow. server.json's version is synced
# from the just-published npm package so it always matches the release.
#
# Intentionally does NOT gate publish-final: the binary release is the
# product, the registry entry is metadata. A registry-preview outage
# must never block shipping. Re-run this job alone to retry — it does
# not touch npm/PyPI, so retries are safe.
#
# It DOES depend on publish-final, which is the job that un-drafts the
# release. Both used to need only publish-registries, so they raced: the
# registry validates every package URL it is given by fetching it, and a
# DRAFT release's assets are not publicly readable. In v0.10.3 the registry
# ran 5s before the un-draft and was told its own .mcpb URL was a 404:
# MCPB package '...codebase-memory-mcp-darwin-amd64.mcpb' is not publicly
# accessible (status: 404)
# The same URL served 200 once the release went public, and the job passed on
# a plain re-run. Depending on publish-final keeps the documented intent
# exactly — the registry still cannot block the release, it just runs after
# the assets it validates actually exist to the outside world.
publish-mcp-registry:
needs: [publish-registries, publish-final]
# Skipped for prereleases: the MCP Registry has no channel concept, so an
# RC would present itself as the current version to every registry consumer.
if: ${{ !cancelled() && !failure() && needs.publish-registries.result == 'success' && needs.publish-final.result == 'success' && !contains(inputs.version, '-') }}
runs-on: ubuntu-latest
permissions:
id-token: write # GitHub OIDC auth to the MCP Registry
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Sync server.json version to the released package
env:
# The dispatch input is the authoritative release version —
# pkg/npm/package.json can lag behind it in the dispatched commit.
RELEASE_VERSION: ${{ inputs.version }}
run: |
VERSION="${RELEASE_VERSION#v}"
jq --arg v "$VERSION" '.version = $v | (.packages[].version) = $v' \
server.json > server.tmp && mv server.tmp server.json
echo "server.json pinned to $VERSION"
# The MCPB entries need each bundle's sha256; checksums.txt comes from
# the release-draft job as a same-run artifact (not a draft-release
# download — that would need contents beyond read plus gh).
- name: Fetch release checksums
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: release-checksums
- name: Append MCPB package entries to server.json
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
scripts/ci/gen-mcpb-registry-entries.sh server.json checksums.txt "$RELEASE_VERSION"
cat server.json
# Pinned by version AND content hash: `latest` would let an upstream
# release (or a compromised one) change what runs in the job that holds
# the registry publish credential, and piping curl straight into tar
# executes the payload before anything can verify it.
#
# One pin covers every platform. Upstream ships a checksums file for the
# whole release, so we pin that file's own SHA-256 and verify whichever
# asset this runner needs against it. Moving the job to a different OS or
# architecture therefore needs no change here, and an unlisted asset
# fails closed rather than installing unverified.
- name: Install mcp-publisher
env:
MCP_PUBLISHER_VERSION: v1.8.1
MCP_PUBLISHER_CHECKSUMS_SHA256: f7937a7908096f63147658e13f0f63a393a1c9fc722a2d10017593940d36e59e
run: |
base="https://github.com/modelcontextprotocol/registry/releases/download/$MCP_PUBLISHER_VERSION"
sums="registry_${MCP_PUBLISHER_VERSION#v}_checksums.txt"
asset="mcp-publisher_$(uname -s | tr '[:upper:]' '[:lower:]')_$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/').tar.gz"
curl -fsSLo "$sums" "$base/$sums"
echo "$MCP_PUBLISHER_CHECKSUMS_SHA256 $sums" | sha256sum -c -
curl -fsSLo "$asset" "$base/$asset"
# Exact filename match: a substring match would also accept the
# .sbom.json / .sigstore.json lines for the same asset.
awk -v a="$asset" '$2 == a' "$sums" > expected.sha256
test -s expected.sha256 || { echo "no checksum listed for $asset" >&2; exit 1; }
sha256sum -c expected.sha256
tar xzf "$asset" mcp-publisher
- name: Authenticate to MCP Registry (GitHub OIDC)
run: ./mcp-publisher login github-oidc
- name: Publish server.json to MCP Registry
run: ./mcp-publisher publish
# ── 9. Atomic un-draft (only after npm + PyPI succeed) ────────
# The GitHub release stays in DRAFT until both registries publish.
# If anything upstream fails, the draft can be deleted and the run
# re-tried with replace=true — no half-shipped state visible to users.
publish-final:
needs: [verify, publish-registries]
if: ${{ !cancelled() && !failure() && needs.verify.result == 'success' && needs.publish-registries.result == 'success' }}
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# Legacy ui-* aliases so 0.9.x updaters stop 404ing (#1538). Runs after
# verify: same bytes as archives the VirusTotal gate already cleared, and
# publishing earlier would duplicate the scan set and provenance manifest.
- name: Publish legacy ui-* archive aliases
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: scripts/ci/publish-legacy-aliases.sh "${{ inputs.version }}" "$GITHUB_REPOSITORY"
- name: Un-draft GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ inputs.version }}
run: gh release edit "$VERSION" --draft=false --repo "$GITHUB_REPOSITORY"