mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
Merge upstream main into fix-managed-block-anchor
This commit is contained in:
@@ -0,0 +1,40 @@
|
||||
# ADR 0001: Main and release branch semantics
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-13
|
||||
|
||||
## Context
|
||||
|
||||
Headroom currently develops and releases from `main`. The target lifecycle needs
|
||||
`main` to continue moving while an exact source revision is qualified. A branch
|
||||
name alone is not evidence, and creating a `release` branch before qualification
|
||||
is authoritative would make unqualified code appear publishable.
|
||||
|
||||
## Decision
|
||||
|
||||
`main` remains the default integration branch and the target for ordinary pull
|
||||
requests. The future protected `release` branch represents qualified source. It
|
||||
may receive only an ancestry-preserving promotion of an exact Gate-A-qualified
|
||||
`main` SHA, a verified metadata-only Release Please change, an authorized hotfix,
|
||||
or release-control maintenance.
|
||||
|
||||
Promotion must leave the qualified candidate SHA reachable from `release`.
|
||||
Squash and rebase merges are therefore forbidden for promotion pull requests.
|
||||
Normal pull requests into `main` may retain the repository's usual merge policy.
|
||||
|
||||
The first `release` branch must be created from an exact SHA with a passing,
|
||||
unrevoked qualification manifest. Branch creation, protection, lifecycle
|
||||
activation, and publication credentials remain explicit administrator actions.
|
||||
They are not performed by this contract-only change.
|
||||
|
||||
Gate A admits source to `release`. Gate B independently authorizes publication
|
||||
of final distribution bytes. Neither branch membership nor a tag or GitHub
|
||||
Release substitutes for either gate.
|
||||
|
||||
## Consequences
|
||||
|
||||
Release Please will eventually watch `release`, but only after the qualification
|
||||
controller and migration guard are operational. Promotion automation must use an
|
||||
ancestry-preserving merge. A hotfix made on `release` requires qualification and
|
||||
forward synchronization to `main`; `release` must not become a second development
|
||||
branch.
|
||||
@@ -0,0 +1,35 @@
|
||||
# ADR 0002: Release artifact identity and provenance
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-13
|
||||
|
||||
## Context
|
||||
|
||||
Rebuilding a moving source reference can produce different bytes. Source identity
|
||||
alone cannot prove what qualification tested or what publication shipped.
|
||||
|
||||
## Decision
|
||||
|
||||
Every candidate is bound to an exact full Git commit SHA and an immutable artifact
|
||||
identified by SHA-256, filename, package name, version, size, and build-run
|
||||
identity. When runtime behavior comes from a payload that can differ from package
|
||||
bytes, its SHA-256 is recorded separately.
|
||||
|
||||
Evidence documents are themselves immutable bytes with a SHA-256 and an opaque
|
||||
URI. A producer identity records repository, workflow, run ID, attempt, and the
|
||||
producer commit. Cross-repository consumers verify the digest before processing
|
||||
the artifact or evidence. Secrets and credentials are never evidence fields.
|
||||
|
||||
The candidate artifact is built once. Downstream deterministic, integration, and
|
||||
benchmark lanes consume those exact bytes; they do not rebuild `main` and reuse
|
||||
the candidate identity. Final Gate B compares published bytes with the qualified
|
||||
artifact or applies an explicitly approved, policy-versioned runtime-payload
|
||||
equivalence rule.
|
||||
|
||||
## Consequences
|
||||
|
||||
Artifact stores and workflow runs require immutable retention sufficient for
|
||||
audit. A digest, source SHA, schema version, or producer mismatch makes evidence
|
||||
`inconclusive` or `fail` according to policy; it can never pass silently. Future
|
||||
controllers must enforce equality across documents in addition to validating
|
||||
each document's JSON Schema.
|
||||
@@ -0,0 +1,37 @@
|
||||
# ADR 0003: A1 versus B is the release benchmark claim
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-13
|
||||
|
||||
## Context
|
||||
|
||||
Comparing Headroom against a direct provider path changes more than compression
|
||||
and cannot isolate Headroom's effect. Release evidence needs a paired comparison
|
||||
whose intended difference is only optimization state.
|
||||
|
||||
## Decision
|
||||
|
||||
The headline release comparison is:
|
||||
|
||||
- A1 (`a1_passthrough`): the Headroom proxy and candidate artifact with
|
||||
optimization disabled.
|
||||
- B (`b_headroom`): the same Headroom proxy and candidate artifact with
|
||||
optimization enabled.
|
||||
|
||||
A1 and B must have equal candidate source SHA, artifact digest, rollout registry
|
||||
and snapshot digests, model, agent, taskset, environment image, and resource
|
||||
limits. Tasks are paired, arm order is randomized, and the versioned policy sets
|
||||
repeat count and statistical decision rules. A direct-provider A0 arm may be
|
||||
recorded for diagnosis but is not the headline comparator.
|
||||
|
||||
The benchmark producer reports an explicit identity-verification verdict. Any
|
||||
identity mismatch makes the experiment and release evidence inconclusive. JSON
|
||||
Schema constrains the arm shapes and intended optimization states; the benchmark
|
||||
worker and qualification controller must enforce equality between arms.
|
||||
|
||||
## Consequences
|
||||
|
||||
A single noisy run is not hard release evidence. Missing telemetry, provider
|
||||
outages, unequal environments, or identity drift remain visible and cannot be
|
||||
coerced into `pass`. Accuracy, savings, latency, reliability, and missing-data
|
||||
rules live in versioned policy rather than workflow shell code.
|
||||
@@ -0,0 +1,38 @@
|
||||
# ADR 0004: Black-box release benchmark boundary
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-13
|
||||
|
||||
## Context
|
||||
|
||||
Internal unit and compression tests are necessary but do not prove that the
|
||||
packaged proxy preserves real agent, streaming, and tool-use behavior. Conversely,
|
||||
benchmark infrastructure must not gain release credentials or execute arbitrary
|
||||
release commands.
|
||||
|
||||
## Decision
|
||||
|
||||
Release benchmarks exercise Headroom through its supported public protocol and
|
||||
observable provenance surfaces, using the immutable candidate artifact. They
|
||||
record taskset revision and digest, model and agent identities, environment,
|
||||
resource limits, rollout snapshot, artifact digest, attempts, telemetry
|
||||
completeness, and raw-result identity.
|
||||
|
||||
The benchmark repository receives only the minimum candidate and experiment
|
||||
identities needed to run. It verifies artifact digests before installation and
|
||||
returns an immutable result reference with authenticated producer identity. It
|
||||
does not receive provider secrets through release evidence, package-publish
|
||||
credentials, or permission to mutate the Headroom repository.
|
||||
|
||||
Failures and cancellations preserve terminal evidence. A provider or
|
||||
infrastructure failure is `inconclusive`, not a candidate pass and not necessarily
|
||||
a candidate-caused failure. The qualification controller applies the registered
|
||||
policy to the returned reference; callback identity alone does not establish a
|
||||
passing result.
|
||||
|
||||
## Consequences
|
||||
|
||||
Hermetic PR simulators, scheduled live smoke tests, and release-candidate lanes
|
||||
can share one result shape while retaining their execution depth. Cross-repository
|
||||
authentication, artifact transport, callback authorization, and retention need
|
||||
separate implementation and security review in later PRs.
|
||||
@@ -0,0 +1,40 @@
|
||||
# ADR 0005: Release overrides and fail-closed policy
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-13
|
||||
|
||||
## Context
|
||||
|
||||
Headroom product transforms may fail open to preserve requests, but release
|
||||
control makes a different safety decision: publication must not proceed without
|
||||
complete, matching, current evidence. Operational exceptions still need an
|
||||
auditable representation.
|
||||
|
||||
## Decision
|
||||
|
||||
Release evidence uses exactly four policy statuses: `pass`, `fail`,
|
||||
`inconclusive`, and `skipped_by_policy`. Missing, stale, invalid, mismatched,
|
||||
revoked, unauthorized, or schema-incompatible required evidence cannot produce
|
||||
`pass`. `skipped_by_policy` is allowed only when the selected policy version does
|
||||
not require that evidence.
|
||||
|
||||
Policy is versioned data. It maps risk classes to required Gate A and Gate B
|
||||
evidence and deterministic rules. Missing-data behavior is `inconclusive`.
|
||||
Thresholds and exceptions are not embedded as ad hoc workflow conditions.
|
||||
|
||||
An override records a unique ID, narrow scope, reason, human approver, approval
|
||||
time, expiry, and optional ticket URI. It never contains credentials. Controllers
|
||||
authenticate the approver, reject expired or unauthorized overrides, and include
|
||||
accepted overrides in the gate result. An unsafe runtime rollout override makes
|
||||
the rollout qualification-ineligible and cannot be overridden into a release
|
||||
pass by this mechanism.
|
||||
|
||||
Qualification revocation is explicit, immutable evidence linked to the original
|
||||
qualification. A revoked qualification cannot authorize promotion or publication.
|
||||
|
||||
## Consequences
|
||||
|
||||
The stable GitHub check will be named `release-policy` even when risk-specific
|
||||
evidence varies. Workflow code remains a thin adapter around a tested policy
|
||||
evaluator. Operational urgency stays visible to reviewers instead of silently
|
||||
weakening the evidence chain.
|
||||
@@ -0,0 +1,283 @@
|
||||
# HEAOSS-6 Pull Request Split Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace PR #3382 with six focused draft pull requests whose cumulative tip preserves its reviewed HEAOSS-6 behavior.
|
||||
|
||||
**Architecture:** Build a linear Git branch stack from current upstream `main`. Each branch introduces one independently useful layer: decisions, policy contracts, evidence contracts, manifest tooling, reusable release builds, then candidate orchestration.
|
||||
|
||||
**Tech Stack:** Git, GitHub CLI, Python 3.13, pytest, JSON Schema 2020-12, GitHub Actions, Ruff, mypy, actionlint
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-09-17-heaoss-6-pr-split-design.md`
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- The first branch is based on current upstream `main`; each later branch is based on its immediate predecessor.
|
||||
- Every pull request is opened as a draft and targets the preceding branch.
|
||||
- The original PR #3382 stays unchanged until all six drafts and the cumulative-equivalence audit exist.
|
||||
- No unrelated merge-commit changes from PR #3382 may enter the new stack.
|
||||
- Release publishing behavior remains intact after every executable slice.
|
||||
- Every PR body states `Part of HEAOSS-6; split from #3382.` and identifies its stack position.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Release Qualification Decisions
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/adr/0001-main-release-branch-semantics.md`
|
||||
- Create: `docs/adr/0002-release-artifact-identity.md`
|
||||
- Create: `docs/adr/0003-a1-b-release-benchmark.md`
|
||||
- Create: `docs/adr/0004-black-box-release-benchmarks.md`
|
||||
- Create: `docs/adr/0005-release-overrides-and-fail-closed-policy.md`
|
||||
- Existing: `docs/superpowers/specs/2026-09-17-heaoss-6-pr-split-design.md`
|
||||
- Existing: `docs/superpowers/plans/2026-09-17-heaoss-6-pr-split.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: HEAOSS-6 decisions preserved on `upstream/pr-3382`.
|
||||
- Produces: Stable terminology for source, artifact, rollout, policy, benchmark, and override identity used by later PRs.
|
||||
|
||||
- [ ] **Step 1: Restore the five reviewed ADRs**
|
||||
|
||||
```powershell
|
||||
git checkout upstream/pr-3382 -- docs/adr/0001-main-release-branch-semantics.md docs/adr/0002-release-artifact-identity.md docs/adr/0003-a1-b-release-benchmark.md docs/adr/0004-black-box-release-benchmarks.md docs/adr/0005-release-overrides-and-fail-closed-policy.md
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify documentation integrity**
|
||||
|
||||
Run both `git diff --check` and `git diff --cached --check`; stop if either exits nonzero. The restore command stages the ADRs, so the cached check is required to inspect those files.
|
||||
Run `rg -n 'TB[D]|TO[D]O'` against the five restored ADRs and this split's specification and plan. Exit 1 means no placeholders; exit 0 reports unresolved markers, and exit 2 is a scan error.
|
||||
Expected: both whitespace checks exit 0 and no unresolved placeholder is present in these seven documents. Unrelated plans and ADRs are outside this slice's placeholder check.
|
||||
|
||||
- [ ] **Step 3: Commit the documentation slice**
|
||||
|
||||
```powershell
|
||||
git add docs/adr docs/superpowers
|
||||
git commit -m "docs(release): define qualification decisions"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Push and open draft PR 1**
|
||||
|
||||
```powershell
|
||||
git push -u origin jd/heaoss-6-01-release-qualification-decisions
|
||||
gh pr create --repo headroomlabs-ai/headroom --draft --base main --head JerrettDavis:jd/heaoss-6-01-release-qualification-decisions --title "docs(release): define qualification decisions" --body-file <body-file>
|
||||
```
|
||||
|
||||
### Task 2: Shared Policy Contracts
|
||||
|
||||
**Files:**
|
||||
- Create: `release/contracts/common.schema.json`
|
||||
- Create: `release/contracts/policy.schema.json`
|
||||
- Create: `release/contracts/examples/policy.valid.json`
|
||||
- Create: `release/contracts/README.md`
|
||||
- Modify: `pyproject.toml`
|
||||
- Modify: `uv.lock`
|
||||
- Create: `tests/test_release_contracts.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: ADR terminology from Task 1.
|
||||
- Produces: JSON Schema definitions referenced as `common.schema.json#/$defs/...` and a complete `policy.schema.json` consumer.
|
||||
|
||||
- [ ] **Step 1: Create the stacked branch and restore contract sources**
|
||||
|
||||
```powershell
|
||||
git switch -c jd/heaoss-6-02-release-policy-contracts
|
||||
git checkout upstream/pr-3382 -- release/contracts/common.schema.json release/contracts/policy.schema.json release/contracts/examples/policy.valid.json release/contracts/README.md pyproject.toml uv.lock
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add focused policy contract tests**
|
||||
|
||||
Extract the schema-registry helpers and policy-specific cases from PR #3382's
|
||||
`tests/test_release_contracts.py`. The resulting test file must load local
|
||||
schemas through `referencing.Registry`, validate `policy.valid.json`, reject an
|
||||
unknown schema version, and enforce policy pass/fail invariants.
|
||||
|
||||
- [ ] **Step 3: Validate and commit**
|
||||
|
||||
Run: `uv lock --check`
|
||||
|
||||
Run: `uv run --extra dev pytest tests/test_release_contracts.py -q`
|
||||
|
||||
Run: `uv run --extra dev ruff check tests/test_release_contracts.py`
|
||||
|
||||
Expected: all commands exit 0.
|
||||
|
||||
```powershell
|
||||
git add release/contracts tests/test_release_contracts.py pyproject.toml uv.lock
|
||||
git commit -m "feat(release): define versioned policy contracts"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Push and open draft PR 2 targeting PR 1's branch**
|
||||
|
||||
Use title `feat(release): define versioned policy contracts` and base
|
||||
`jd/heaoss-6-01-release-qualification-decisions`.
|
||||
|
||||
### Task 3: Qualification Evidence Contracts
|
||||
|
||||
**Files:**
|
||||
- Create: `release/contracts/benchmark-result-ref.schema.json`
|
||||
- Create: `release/contracts/candidate-manifest.schema.json`
|
||||
- Create: `release/contracts/gate-result.schema.json`
|
||||
- Create: `release/contracts/integration-result.schema.json`
|
||||
- Create: `release/contracts/qualification-manifest.schema.json`
|
||||
- Create: matching files under `release/contracts/examples/`
|
||||
- Modify: `tests/test_release_contracts.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `common.schema.json` definitions and the release policy contract.
|
||||
- Produces: Complete evidence formats used by candidate tooling and future qualification lanes.
|
||||
|
||||
- [ ] **Step 1: Create the branch and restore all remaining schemas/examples**
|
||||
|
||||
Create `jd/heaoss-6-03-qualification-evidence-contracts` from Task 2, then
|
||||
restore the declared files from `upstream/pr-3382`.
|
||||
|
||||
- [ ] **Step 2: Restore the complete reviewed contract test suite**
|
||||
|
||||
```powershell
|
||||
git checkout upstream/pr-3382 -- tests/test_release_contracts.py
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Validate and commit**
|
||||
|
||||
Run: `uv run --extra dev pytest tests/test_release_contracts.py -q`
|
||||
|
||||
Run: `uv run --extra dev ruff check tests/test_release_contracts.py`
|
||||
|
||||
Expected: all contract examples and negative invariants pass.
|
||||
|
||||
Commit as `feat(release): add qualification evidence contracts`.
|
||||
|
||||
- [ ] **Step 4: Push and open draft PR 3 targeting PR 2's branch**
|
||||
|
||||
### Task 4: Candidate Manifest CLI
|
||||
|
||||
**Files:**
|
||||
- Create: `scripts/candidate_manifest.py`
|
||||
- Create: `tests/test_candidate_manifest.py`
|
||||
- Modify: `.gitignore`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `release/contracts/candidate-manifest.schema.json` and common definitions.
|
||||
- Produces: `inventory`, `create`, and `verify` CLI commands used by Task 6.
|
||||
|
||||
- [ ] **Step 1: Create the branch and restore reviewed CLI files**
|
||||
|
||||
Create `jd/heaoss-6-04-candidate-manifest-cli`, then restore the three declared
|
||||
paths from `upstream/pr-3382`.
|
||||
|
||||
- [ ] **Step 2: Validate behavior and types**
|
||||
|
||||
Run: `uv run --extra dev pytest tests/test_candidate_manifest.py tests/test_release_contracts.py -q`
|
||||
|
||||
Run: `uv run --extra dev ruff check scripts/candidate_manifest.py tests/test_candidate_manifest.py`
|
||||
|
||||
Run: `uv run --extra dev mypy scripts/candidate_manifest.py`
|
||||
|
||||
Expected: all commands exit 0.
|
||||
|
||||
- [ ] **Step 3: Commit, push, and open draft PR 4 targeting PR 3's branch**
|
||||
|
||||
Commit as `feat(release): add immutable candidate manifest verifier`.
|
||||
|
||||
### Task 5: Read-Only Release Build Workflow
|
||||
|
||||
**Files:**
|
||||
- Create: `.github/workflows/release-build.yml`
|
||||
- Modify: `.github/workflows/release.yml`
|
||||
- Modify: `tests/test_release_workflows.py`
|
||||
- Modify: `docs/content/docs/releases.mdx`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Existing release build and smoke jobs.
|
||||
- Produces: Reusable `workflow_call` accepting `source_sha`, `manual_version`, and `release_tag`, with version/npm/canonical outputs and retained artifacts.
|
||||
|
||||
- [ ] **Step 1: Create the branch and restore the reviewed extraction**
|
||||
|
||||
Create `jd/heaoss-6-05-read-only-release-build`, then restore the four declared
|
||||
paths from `upstream/pr-3382`.
|
||||
|
||||
- [ ] **Step 2: Validate workflow structure and release invariants**
|
||||
|
||||
Run: `uv run --extra dev pytest tests/test_release_workflows.py -q`
|
||||
|
||||
Run: `actionlint .github/workflows/release.yml .github/workflows/release-build.yml`
|
||||
|
||||
Expected: 50 or more release workflow tests pass and actionlint emits no findings.
|
||||
|
||||
- [ ] **Step 3: Commit, push, and open draft PR 5 targeting PR 4's branch**
|
||||
|
||||
Commit as `refactor(release): extract read-only build workflow`.
|
||||
|
||||
### Task 6: Immutable Candidate Workflow
|
||||
|
||||
**Files:**
|
||||
- Create: `.github/workflows/candidate-artifact.yml`
|
||||
- Create: `tests/test_candidate_workflow.py`
|
||||
- Modify: `docs/content/docs/releases.mdx`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Task 4 CLI and Task 5 reusable workflow.
|
||||
- Produces: A manual exact-main-SHA candidate artifact and independently verified retained bytes.
|
||||
|
||||
- [ ] **Step 1: Create the branch and restore candidate orchestration**
|
||||
|
||||
Create `jd/heaoss-6-06-immutable-candidate-workflow`, then restore the workflow
|
||||
and test from `upstream/pr-3382`. Reconcile the release documentation so only
|
||||
candidate-specific additions remain in this branch.
|
||||
|
||||
- [ ] **Step 2: Run cumulative focused validation**
|
||||
|
||||
Run: `uv run --extra dev pytest tests/test_candidate_workflow.py tests/test_candidate_manifest.py tests/test_release_contracts.py tests/test_release_workflows.py -q`
|
||||
|
||||
Run: `uv run --extra dev ruff check scripts/candidate_manifest.py tests/test_candidate_manifest.py tests/test_candidate_workflow.py tests/test_release_contracts.py tests/test_release_workflows.py`
|
||||
|
||||
Run: `uv run --extra dev mypy scripts/candidate_manifest.py`
|
||||
|
||||
Run: `actionlint .github/workflows/candidate-artifact.yml .github/workflows/release-build.yml .github/workflows/release.yml`
|
||||
|
||||
Expected: 96 focused tests or the current-main-adjusted equivalent pass; lint,
|
||||
types, and workflow validation exit 0.
|
||||
|
||||
- [ ] **Step 3: Audit cumulative equivalence**
|
||||
|
||||
```powershell
|
||||
git diff --stat upstream/pr-3382...HEAD
|
||||
git diff upstream/pr-3382 HEAD -- .github docs/adr docs/content/docs/releases.mdx release scripts tests/test_candidate_manifest.py tests/test_candidate_workflow.py tests/test_release_contracts.py tests/test_release_workflows.py pyproject.toml uv.lock .gitignore
|
||||
```
|
||||
|
||||
Expected: only the approved split documents and intentional current-main
|
||||
reconciliation differ. Investigate and document every other difference.
|
||||
|
||||
- [ ] **Step 4: Commit, push, and open draft PR 6 targeting PR 5's branch**
|
||||
|
||||
Commit as `feat(release): build immutable exact-SHA candidates`.
|
||||
|
||||
### Task 7: Link the Stack and Supersede PR #3382
|
||||
|
||||
**Files:**
|
||||
- Modify: GitHub draft PR bodies only.
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: URLs and exact head/base SHAs for all six draft PRs.
|
||||
- Produces: Navigable stack metadata and a non-destructive supersession notice on #3382.
|
||||
|
||||
- [ ] **Step 1: Update each draft body with complete previous/next links**
|
||||
|
||||
Use `gh pr edit` and verify the rendered bodies with `gh pr view --json body`.
|
||||
|
||||
- [ ] **Step 2: Convert #3382 to draft and prepend the supersession notice**
|
||||
|
||||
The notice must link all six drafts and retain the prior validation history
|
||||
below it. Do not close #3382.
|
||||
|
||||
- [ ] **Step 3: Verify authoritative GitHub state**
|
||||
|
||||
Run `gh pr view` for all seven PRs and verify draft status, base branch, head
|
||||
branch, mergeability, and URLs. Confirm every new PR head SHA matches the pushed
|
||||
local branch.
|
||||
|
||||
- [ ] **Step 4: Report hosted-check status without claiming unobserved success**
|
||||
|
||||
List terminal, pending, or failing checks for each exact head. Hosted checks may
|
||||
continue after handoff; local validation and PR creation are required before
|
||||
the task is reported complete.
|
||||
@@ -0,0 +1,243 @@
|
||||
# HEAOSS-6 Pull Request Split Design
|
||||
|
||||
## Purpose
|
||||
|
||||
Replace the 3,279-line change in PR #3382 with a reviewable stack of six draft
|
||||
pull requests. Every pull request must have one coherent deliverable, pass its
|
||||
own focused validation, and leave the repository in a useful state if the stack
|
||||
stops there.
|
||||
|
||||
The final stack must preserve the behavior and safety properties already
|
||||
reviewed on PR #3382: versioned qualification evidence contracts, fail-closed
|
||||
candidate manifest verification, exact-main-SHA builds, a read-only build call
|
||||
tree, deterministic artifact inventory, and independent verification of the
|
||||
retained candidate bytes.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Base the first branch on the current upstream `main`.
|
||||
- Stack every later branch on its immediate predecessor.
|
||||
- Keep each pull request draft until the whole stack is reconstructed and its
|
||||
cumulative tip is equivalent to PR #3382, except for deliberate documentation
|
||||
and history changes described here.
|
||||
- Do not modify or delete the original PR #3382 branch while constructing the
|
||||
stack.
|
||||
- Do not include unrelated changes from merge commits or the dirty primary
|
||||
checkout.
|
||||
- Preserve the release workflow's existing publish behavior at every executable
|
||||
step of the stack.
|
||||
- Use conventional commit messages and include the HEAOSS-6 context and stack
|
||||
position in every pull request body.
|
||||
|
||||
## Stack Topology
|
||||
|
||||
| Position | Branch | Base | Deliverable |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | `jd/heaoss-6-01-release-qualification-decisions` | `main` | Release qualification ADRs and this split specification |
|
||||
| 2 | `jd/heaoss-6-02-release-policy-contracts` | PR 1 branch | Shared evidence vocabulary and versioned release policy contract |
|
||||
| 3 | `jd/heaoss-6-03-qualification-evidence-contracts` | PR 2 branch | Candidate, benchmark, integration, gate, and qualification evidence schemas |
|
||||
| 4 | `jd/heaoss-6-04-candidate-manifest-cli` | PR 3 branch | Local candidate inventory, manifest creation, and fail-closed verification CLI |
|
||||
| 5 | `jd/heaoss-6-05-read-only-release-build` | PR 4 branch | Reusable read-only build/smoke workflow used by the existing release pipeline |
|
||||
| 6 | `jd/heaoss-6-06-immutable-candidate-workflow` | PR 5 branch | Manual exact-SHA candidate build, bundling, retention, and independent verification |
|
||||
|
||||
GitHub draft PRs use the preceding branch as their base. This makes each diff
|
||||
show only its own deliverable while retaining an executable cumulative tip.
|
||||
|
||||
## PR 1: Release Qualification Decisions
|
||||
|
||||
### Scope
|
||||
|
||||
- Add ADR 0001 defining authoritative-main release semantics.
|
||||
- Add ADR 0002 defining artifact and runtime-payload identity.
|
||||
- Add ADR 0003 defining the A1/B benchmark comparison.
|
||||
- Add ADR 0004 defining black-box release benchmarks.
|
||||
- Add ADR 0005 defining overrides and fail-closed policy.
|
||||
- Add this stack design document.
|
||||
|
||||
### Value at Merge
|
||||
|
||||
The repository gains an explicit, reviewable release-qualification model before
|
||||
automation encodes it. No runtime or workflow behavior changes.
|
||||
|
||||
### Validation
|
||||
|
||||
- Markdown and formatting checks applicable to documentation.
|
||||
- Confirm the ADRs contain no unresolved placeholders and agree on identity,
|
||||
policy, and override terminology.
|
||||
|
||||
## PR 2: Shared Policy Contracts
|
||||
|
||||
### Scope
|
||||
|
||||
- Add `release/contracts/common.schema.json` with shared identity, status,
|
||||
rollout, producer, environment, override, and equivalence definitions.
|
||||
- Add `release/contracts/policy.schema.json` and its valid example.
|
||||
- Add `release/contracts/README.md` covering schema versioning and validation.
|
||||
- Add `jsonschema>=4.23.0,<5.0` to the development dependencies and refresh
|
||||
`uv.lock`.
|
||||
- Introduce the reusable schema loading and validation test helpers in
|
||||
`tests/test_release_contracts.py`, with positive and negative policy tests.
|
||||
|
||||
### Value at Merge
|
||||
|
||||
Policy authors can publish and validate a complete versioned release policy.
|
||||
The shared vocabulary is exercised by a real consumer in the same pull request;
|
||||
it is not unused scaffolding.
|
||||
|
||||
### Validation
|
||||
|
||||
- `uv lock --check`
|
||||
- `uv run --extra dev pytest tests/test_release_contracts.py -q`
|
||||
- `uv run --extra dev ruff check tests/test_release_contracts.py`
|
||||
|
||||
## PR 3: Qualification Evidence Contracts
|
||||
|
||||
### Scope
|
||||
|
||||
- Add candidate-manifest, benchmark-result-reference, integration-result,
|
||||
gate-result, and qualification-manifest schemas.
|
||||
- Add one valid example for every schema.
|
||||
- Extend contract tests to validate every example, reject malformed identities,
|
||||
enforce pass/fail invariants, enforce Gate B runtime-payload equivalence, and
|
||||
reject unsupported schema versions.
|
||||
|
||||
### Value at Merge
|
||||
|
||||
Qualification producers and consumers have a complete, machine-validated
|
||||
evidence interchange format. The contracts are independently useful without
|
||||
the candidate workflow and can be adopted by later qualification lanes.
|
||||
|
||||
### Validation
|
||||
|
||||
- `uv run --extra dev pytest tests/test_release_contracts.py -q`
|
||||
- `uv run --extra dev ruff check tests/test_release_contracts.py`
|
||||
- Validate that all relative `$ref` targets resolve from the checked-in schema
|
||||
registry.
|
||||
|
||||
## PR 4: Candidate Manifest CLI
|
||||
|
||||
### Scope
|
||||
|
||||
- Add `scripts/candidate_manifest.py` with `inventory`, `create`, and `verify`
|
||||
commands.
|
||||
- Add `.candidate/` to `.gitignore` for local command output.
|
||||
- Add focused tests for deterministic ordering, hashes, sizes, package/version
|
||||
identity, source SHA, producer repository/workflow/run identity, rollout
|
||||
identity, safe qualification eligibility, schema versions, missing files,
|
||||
changed bytes, and unsupported embedded schemas.
|
||||
|
||||
### Value at Merge
|
||||
|
||||
Maintainers can create and independently verify immutable candidate manifests
|
||||
locally or from any later CI workflow. The CLI is fully functional before it is
|
||||
wired into GitHub Actions.
|
||||
|
||||
### Validation
|
||||
|
||||
- `uv run --extra dev pytest tests/test_candidate_manifest.py tests/test_release_contracts.py -q`
|
||||
- `uv run --extra dev ruff check scripts/candidate_manifest.py tests/test_candidate_manifest.py`
|
||||
- `uv run --extra dev mypy scripts/candidate_manifest.py`
|
||||
|
||||
## PR 5: Read-Only Release Build Workflow
|
||||
|
||||
### Scope
|
||||
|
||||
- Extract version resolution, package construction, the cross-platform wheel
|
||||
matrix, distribution collection, and smoke imports from `release.yml` into
|
||||
the reusable `release-build.yml` workflow.
|
||||
- Make `release.yml` call the reusable workflow, then retain all publishing and
|
||||
release creation in `release.yml`.
|
||||
- Ensure every publisher waits for the complete build/smoke group.
|
||||
- Preserve the release workflow entry points and published outputs.
|
||||
- Update release workflow tests and release documentation to describe the new
|
||||
boundary.
|
||||
|
||||
### Value at Merge
|
||||
|
||||
The production release uses a reusable, contents-read-only build boundary with
|
||||
no publishing job in its call tree. This is immediately useful even without the
|
||||
candidate workflow and removes duplicated future build logic.
|
||||
|
||||
### Validation
|
||||
|
||||
- `uv run --extra dev pytest tests/test_release_workflows.py -q`
|
||||
- `actionlint .github/workflows/release.yml .github/workflows/release-build.yml`
|
||||
- Hosted PR dry-run must complete all platform builds and smoke-import jobs;
|
||||
publishing jobs must remain skipped.
|
||||
|
||||
## PR 6: Immutable Candidate Workflow
|
||||
|
||||
### Scope
|
||||
|
||||
- Add `candidate-artifact.yml` as a manual workflow accepting only a full SHA
|
||||
reachable from authoritative `main`.
|
||||
- Call `release-build.yml` directly under `contents: read` permissions.
|
||||
- Produce a deterministic artifact inventory and X-001A candidate manifest.
|
||||
- Verify the manifest before upload, download the retained artifact in a
|
||||
separate job, and verify the exact bytes again.
|
||||
- Reject wrong source, repository, producer, workflow, run, attempt,
|
||||
package/version, filename, size, digest, rollout identity, unsafe rollout,
|
||||
qualification-ineligible rollout, or unsupported schema.
|
||||
- Add focused structural workflow tests and document the candidate workflow.
|
||||
|
||||
### Value at Merge
|
||||
|
||||
Maintainers can build and retain an immutable release candidate from an exact
|
||||
commit on authoritative `main` without exposing publishing permissions. The
|
||||
same bytes are verified independently after artifact retention.
|
||||
|
||||
### Validation
|
||||
|
||||
- `uv run --extra dev pytest tests/test_candidate_workflow.py tests/test_candidate_manifest.py tests/test_release_contracts.py tests/test_release_workflows.py -q`
|
||||
- `uv run --extra dev ruff check scripts/candidate_manifest.py tests/test_candidate_manifest.py tests/test_candidate_workflow.py tests/test_release_contracts.py tests/test_release_workflows.py`
|
||||
- `uv run --extra dev mypy scripts/candidate_manifest.py`
|
||||
- `actionlint .github/workflows/candidate-artifact.yml .github/workflows/release-build.yml .github/workflows/release.yml`
|
||||
- Hosted fork validation must build all five platform targets, run all smoke
|
||||
jobs, emit the candidate, download it independently, and verify its bytes.
|
||||
No publishing job may run.
|
||||
|
||||
## Reconstruction Method
|
||||
|
||||
For each branch, restore only the files or coherent hunks assigned to that pull
|
||||
request from PR #3382's reviewed tip. Where one original test file spans two
|
||||
stack positions, split its tests by the behavior introduced at that position.
|
||||
Commit and validate each branch before creating the next branch from it.
|
||||
|
||||
After PR 6, compare its cumulative tree to PR #3382 with `git diff`, allowing
|
||||
only this specification and any necessary stack metadata. Review every
|
||||
remaining difference rather than assuming equivalence from file counts.
|
||||
|
||||
## Draft Pull Request Format
|
||||
|
||||
Each draft body must include:
|
||||
|
||||
- `Stack: N of 6` and links to adjacent draft PRs when available.
|
||||
- The branch/base relationship.
|
||||
- A concise statement of the independently deliverable value.
|
||||
- Exact local validation results for that branch.
|
||||
- A note that hosted validation is pending or a link to its exact run.
|
||||
- `Part of HEAOSS-6; split from #3382.`
|
||||
- Runtime rollout safety and rollback notes appropriate to the slice.
|
||||
|
||||
## Original PR Transition
|
||||
|
||||
Keep #3382 unchanged until all six draft heads exist and the cumulative tip has
|
||||
passed local equivalence and validation checks. Then convert #3382 to draft and
|
||||
replace its leading status section with a supersession notice linking the six
|
||||
drafts. Do not close it until the stack is confirmed visible and complete; its
|
||||
existing discussion and hosted validation remain useful review evidence.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
The split is complete only when:
|
||||
|
||||
1. Six draft PRs exist with the topology above.
|
||||
2. Every PR diff contains only its declared slice and has passing focused local
|
||||
validation.
|
||||
3. PR 5 demonstrates unchanged release build/smoke behavior with publication
|
||||
skipped on PR validation.
|
||||
4. PR 6 demonstrates candidate creation and independent byte verification with
|
||||
no publishing permissions or publishing jobs in its call tree.
|
||||
5. The cumulative PR 6 tree is reconciled against PR #3382 and every difference
|
||||
is documented and intentional.
|
||||
6. PR #3382 is a draft with a supersession notice linking the complete stack.
|
||||
Reference in New Issue
Block a user