Files
headroom/scripts/tests/test_pr_health_workflow.py
55ce93f3e8 ci(pr-health): check out the commit the workflow runs from (#3887)
## Description

The `PR Governance / label` job fails with `pr-health-labels.py: error:
unrecognized arguments: --field drift --behind-by ... --base-files ...`
on any pull request whose recorded base commit predates #3229.

`pull_request_target` always runs this workflow file from the default
branch, so the job invokes `.github/scripts/pr-health-labels.py` with
the flags #3229 added. But both jobs check out
`github.event.pull_request.base.sha`, which is the base commit recorded
when the PR was last synced, not the commit the workflow came from. On
an older PR that checkout carries the pre-#3229 script, which does not
know those flags. Editing the description of any such PR (an `edited`
event) produces a red run and a failure email, even though nothing about
the PR changed. Today this hit #2626, #2627, #2942 and #3127, whose base
SHAs are all older than c22cd1f1; #3617 and #3108, synced today, pass.

This checks out `github.sha` instead. For `pull_request_target` that is
the last commit on the base branch, the same commit the workflow file is
read from, so the workflow and its helper scripts always match. It is
still trusted base-branch code: the PR head is never checked out, so the
`pull_request_target` safety property is unchanged.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `.github/workflows/pr-health.yml`: both `actions/checkout` steps
(governance and label jobs) use `ref: ${{ github.sha }}` instead of
`ref: ${{ github.event.pull_request.base.sha }}`.
- `scripts/tests/test_pr_health_workflow.py`: a contract test that no
checkout uses `pull_request.base.sha` and every checkout pins
`github.sha`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ uv run --frozen --extra dev pytest scripts/tests/test_pr_health_workflow.py scripts/tests/test_pr_health_labels.py -q
21 passed in 0.08s

# new test against main's pr-health.yml (before this change)
$ uv run --frozen --extra dev pytest scripts/tests/test_pr_health_workflow.py -q -k checkouts_use
1 failed, 7 deselected in 0.04s

$ uvx ruff check scripts/tests/test_pr_health_workflow.py && uvx ruff format --check scripts/tests/test_pr_health_workflow.py
All checks passed!
1 file already formatted
```

## Real Behavior Proof

- Environment: GitHub Actions on headroomlabs-ai/headroom, PR Governance
runs 36719570172, 36719565247, 36719562482 and 36719559715 (2026-09-30,
`pull_request_target` `edited` events).
- Exact command / steps: edited the PR body of #2626, #2627, #2942 and
#3127 (all with `base.sha` older than c22cd1f1), then read the failed
`label` job logs and compared each PR's `base.sha` against c22cd1f1 with
`git merge-base --is-ancestor`.
- Observed result: all four `label` jobs failed with
`pr-health-labels.py: error: unrecognized arguments: --field drift
--behind-by 150 --base-files [...]`; all four base SHAs lack c22cd1f1,
while #3617 and #3108 (base ffc6edb4, which contains it) ran green the
same day.
- Not tested: a live run of the patched workflow. `pull_request_target`
always runs the default branch's copy, so this change takes effect only
after merge; the first `edited` event on an older PR after that is the
live check.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. CI workflow only; nothing in the
shipped package changes.
- Minimum rollout channel: N/A
- Stable/default behavior changed: No runtime behavior changes. In CI,
the governance and label jobs run the default branch's scripts instead
of each PR's stale base copy.
- Kill switch / disable path: N/A (CI only). Revert the PR.
- Unsafe override required: No
- Qualification impact: One contract test added; no existing test
changed or relaxed.
- Rollback path: Revert the PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Documentation: N/A, no user-facing behavior changes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-30 10:45:46 -05:00

133 lines
5.0 KiB
Python

"""Tests for the PR governance workflow contract."""
from __future__ import annotations
import json
import shutil
import subprocess
import sys
from pathlib import Path
from textwrap import dedent
import pytest
def test_incomplete_pr_template_is_reported_without_failing_job() -> None:
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
assert "Fetch current PR body" in workflow
assert "--body-file .pr-body.md" in workflow
assert "Report incomplete PR body" in workflow
assert "PR template validation found missing fields" in workflow
assert "Fail when the PR body is incomplete" not in workflow
assert 'echo "PR template validation failed' not in workflow
def test_ready_for_review_label_is_removed_when_changes_are_requested() -> None:
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
assert "reviewDecision" in workflow
assert 'review_decision="$(jq -r \'.reviewDecision // ""\'' in workflow
assert '$review_decision" == "CHANGES_REQUESTED"' in workflow
def test_merge_state_unknown_does_not_clear_conflict_or_rebase_labels() -> None:
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
assert 'elif [[ "$merge_state" != "UNKNOWN" ]]; then' in workflow
assert 'gh pr edit "$pr" --repo "$REPO" --remove-label "status: needs rebase"' in workflow
assert 'gh pr edit "$pr" --repo "$REPO" --remove-label "status: has conflicts"' in workflow
def test_rebase_label_uses_ref_comparison_instead_of_merge_state() -> None:
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
assert 'if [[ "$merge_state" == "BEHIND" ]]; then' not in workflow
assert 'gh api "repos/$REPO/compare/$base_ref...$head_oid"' in workflow
assert "--field drift" in workflow
assert 'if [[ "$drift" == "stale" ]]; then' in workflow
def test_unknown_drift_does_not_clear_the_rebase_label() -> None:
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
assert 'elif [[ "$drift" == "current" ]]; then' in workflow
def _base_files_snippet(workflow: str) -> str:
"""The shell block that asks GitHub which files the base branch moved."""
start = workflow.index(" # Files the base branch changed")
end = workflow.index(' drift="$(python3', start)
return dedent(workflow[start:end])
def test_failed_base_file_comparison_keeps_an_unknown_sentinel() -> None:
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
assert "echo '[]'" not in workflow
assert "base_files=''" in workflow
assert "|| base_files=''" in workflow
def test_failed_base_file_comparison_does_not_clear_the_rebase_label() -> None:
bash = shutil.which("bash")
if bash is None:
pytest.skip("bash is unavailable")
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
script = "\n".join(
[
"set -euo pipefail",
# Stands in for a throttled API call that writes partial output and fails.
"gh() { printf 'partia'; return 1; }",
'REPO="headroomlabs-ai/headroom"',
'base_ref="main"',
'behind_by="13"',
'merge_base="0123456789abcdef0123456789abcdef01234567"',
_base_files_snippet(workflow),
'printf "%s" "$base_files"',
]
)
result = subprocess.run([bash, "-c", script], capture_output=True, text=True)
assert result.returncode == 0, result.stderr
assert result.stdout == ""
drift = subprocess.run(
[
sys.executable,
".github/scripts/pr-health-labels.py",
"--state-json",
json.dumps(
{"mergeStateStatus": "CLEAN", "files": [{"path": "headroom/proxy/server.py"}]}
),
"--field",
"drift",
"--behind-by",
"13",
"--base-files",
result.stdout,
],
capture_output=True,
text=True,
)
assert drift.returncode == 0, drift.stderr
assert drift.stdout.strip() == "unknown"
# An unknown verdict matches neither branch, so no gh pr edit runs for the label.
start = workflow.index(' if [[ "$drift" == "stale" ]]; then')
dispatch = workflow[start : workflow.index("\n fi\n", start)]
assert 'elif [[ "$drift" == "current" ]]; then' in dispatch
assert "else" not in dispatch
def test_checkouts_use_the_commit_the_workflow_runs_from() -> None:
# pull_request_target runs this workflow file from the default branch.
# Checking out the PR's recorded base.sha instead pairs it with whatever
# helper scripts existed when the PR was last synced, so a new flag here
# fails every older PR's run with "unrecognized arguments".
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
assert "pull_request.base.sha" not in workflow
assert workflow.count("ref: ${{ github.sha }}") == workflow.count("actions/checkout@")