mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description The `PR Governance / label` job fails with `pr-health-labels.py: error: unrecognized arguments: --field drift --behind-by ... --base-files ...` on any pull request whose recorded base commit predates #3229. `pull_request_target` always runs this workflow file from the default branch, so the job invokes `.github/scripts/pr-health-labels.py` with the flags #3229 added. But both jobs check out `github.event.pull_request.base.sha`, which is the base commit recorded when the PR was last synced, not the commit the workflow came from. On an older PR that checkout carries the pre-#3229 script, which does not know those flags. Editing the description of any such PR (an `edited` event) produces a red run and a failure email, even though nothing about the PR changed. Today this hit #2626, #2627, #2942 and #3127, whose base SHAs are all older than c22cd1f1; #3617 and #3108, synced today, pass. This checks out `github.sha` instead. For `pull_request_target` that is the last commit on the base branch, the same commit the workflow file is read from, so the workflow and its helper scripts always match. It is still trusted base-branch code: the PR head is never checked out, so the `pull_request_target` safety property is unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `.github/workflows/pr-health.yml`: both `actions/checkout` steps (governance and label jobs) use `ref: ${{ github.sha }}` instead of `ref: ${{ github.event.pull_request.base.sha }}`. - `scripts/tests/test_pr_health_workflow.py`: a contract test that no checkout uses `pull_request.base.sha` and every checkout pins `github.sha`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest scripts/tests/test_pr_health_workflow.py scripts/tests/test_pr_health_labels.py -q 21 passed in 0.08s # new test against main's pr-health.yml (before this change) $ uv run --frozen --extra dev pytest scripts/tests/test_pr_health_workflow.py -q -k checkouts_use 1 failed, 7 deselected in 0.04s $ uvx ruff check scripts/tests/test_pr_health_workflow.py && uvx ruff format --check scripts/tests/test_pr_health_workflow.py All checks passed! 1 file already formatted ``` ## Real Behavior Proof - Environment: GitHub Actions on headroomlabs-ai/headroom, PR Governance runs 36719570172, 36719565247, 36719562482 and 36719559715 (2026-09-30, `pull_request_target` `edited` events). - Exact command / steps: edited the PR body of #2626, #2627, #2942 and #3127 (all with `base.sha` older thanc22cd1f1), then read the failed `label` job logs and compared each PR's `base.sha` againstc22cd1f1with `git merge-base --is-ancestor`. - Observed result: all four `label` jobs failed with `pr-health-labels.py: error: unrecognized arguments: --field drift --behind-by 150 --base-files [...]`; all four base SHAs lackc22cd1f1, while #3617 and #3108 (baseffc6edb4, which contains it) ran green the same day. - Not tested: a live run of the patched workflow. `pull_request_target` always runs the default branch's copy, so this change takes effect only after merge; the first `edited` event on an older PR after that is the live check. ## Runtime Rollout Safety - Rollout-managed feature(s): None. CI workflow only; nothing in the shipped package changes. - Minimum rollout channel: N/A - Stable/default behavior changed: No runtime behavior changes. In CI, the governance and label jobs run the default branch's scripts instead of each PR's stale base copy. - Kill switch / disable path: N/A (CI only). Revert the PR. - Unsafe override required: No - Qualification impact: One contract test added; no existing test changed or relaxed. - Rollback path: Revert the PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes Documentation: N/A, no user-facing behavior changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
133 lines
5.0 KiB
Python
133 lines
5.0 KiB
Python
"""Tests for the PR governance workflow contract."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import shutil
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
from textwrap import dedent
|
|
|
|
import pytest
|
|
|
|
|
|
def test_incomplete_pr_template_is_reported_without_failing_job() -> None:
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
|
|
assert "Fetch current PR body" in workflow
|
|
assert "--body-file .pr-body.md" in workflow
|
|
assert "Report incomplete PR body" in workflow
|
|
assert "PR template validation found missing fields" in workflow
|
|
assert "Fail when the PR body is incomplete" not in workflow
|
|
assert 'echo "PR template validation failed' not in workflow
|
|
|
|
|
|
def test_ready_for_review_label_is_removed_when_changes_are_requested() -> None:
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
|
|
assert "reviewDecision" in workflow
|
|
assert 'review_decision="$(jq -r \'.reviewDecision // ""\'' in workflow
|
|
assert '$review_decision" == "CHANGES_REQUESTED"' in workflow
|
|
|
|
|
|
def test_merge_state_unknown_does_not_clear_conflict_or_rebase_labels() -> None:
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
|
|
assert 'elif [[ "$merge_state" != "UNKNOWN" ]]; then' in workflow
|
|
assert 'gh pr edit "$pr" --repo "$REPO" --remove-label "status: needs rebase"' in workflow
|
|
assert 'gh pr edit "$pr" --repo "$REPO" --remove-label "status: has conflicts"' in workflow
|
|
|
|
|
|
def test_rebase_label_uses_ref_comparison_instead_of_merge_state() -> None:
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
|
|
assert 'if [[ "$merge_state" == "BEHIND" ]]; then' not in workflow
|
|
assert 'gh api "repos/$REPO/compare/$base_ref...$head_oid"' in workflow
|
|
assert "--field drift" in workflow
|
|
assert 'if [[ "$drift" == "stale" ]]; then' in workflow
|
|
|
|
|
|
def test_unknown_drift_does_not_clear_the_rebase_label() -> None:
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
|
|
assert 'elif [[ "$drift" == "current" ]]; then' in workflow
|
|
|
|
|
|
def _base_files_snippet(workflow: str) -> str:
|
|
"""The shell block that asks GitHub which files the base branch moved."""
|
|
start = workflow.index(" # Files the base branch changed")
|
|
end = workflow.index(' drift="$(python3', start)
|
|
return dedent(workflow[start:end])
|
|
|
|
|
|
def test_failed_base_file_comparison_keeps_an_unknown_sentinel() -> None:
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
|
|
assert "echo '[]'" not in workflow
|
|
assert "base_files=''" in workflow
|
|
assert "|| base_files=''" in workflow
|
|
|
|
|
|
def test_failed_base_file_comparison_does_not_clear_the_rebase_label() -> None:
|
|
bash = shutil.which("bash")
|
|
if bash is None:
|
|
pytest.skip("bash is unavailable")
|
|
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
script = "\n".join(
|
|
[
|
|
"set -euo pipefail",
|
|
# Stands in for a throttled API call that writes partial output and fails.
|
|
"gh() { printf 'partia'; return 1; }",
|
|
'REPO="headroomlabs-ai/headroom"',
|
|
'base_ref="main"',
|
|
'behind_by="13"',
|
|
'merge_base="0123456789abcdef0123456789abcdef01234567"',
|
|
_base_files_snippet(workflow),
|
|
'printf "%s" "$base_files"',
|
|
]
|
|
)
|
|
result = subprocess.run([bash, "-c", script], capture_output=True, text=True)
|
|
|
|
assert result.returncode == 0, result.stderr
|
|
assert result.stdout == ""
|
|
|
|
drift = subprocess.run(
|
|
[
|
|
sys.executable,
|
|
".github/scripts/pr-health-labels.py",
|
|
"--state-json",
|
|
json.dumps(
|
|
{"mergeStateStatus": "CLEAN", "files": [{"path": "headroom/proxy/server.py"}]}
|
|
),
|
|
"--field",
|
|
"drift",
|
|
"--behind-by",
|
|
"13",
|
|
"--base-files",
|
|
result.stdout,
|
|
],
|
|
capture_output=True,
|
|
text=True,
|
|
)
|
|
|
|
assert drift.returncode == 0, drift.stderr
|
|
assert drift.stdout.strip() == "unknown"
|
|
# An unknown verdict matches neither branch, so no gh pr edit runs for the label.
|
|
start = workflow.index(' if [[ "$drift" == "stale" ]]; then')
|
|
dispatch = workflow[start : workflow.index("\n fi\n", start)]
|
|
assert 'elif [[ "$drift" == "current" ]]; then' in dispatch
|
|
assert "else" not in dispatch
|
|
|
|
|
|
def test_checkouts_use_the_commit_the_workflow_runs_from() -> None:
|
|
# pull_request_target runs this workflow file from the default branch.
|
|
# Checking out the PR's recorded base.sha instead pairs it with whatever
|
|
# helper scripts existed when the PR was last synced, so a new flag here
|
|
# fails every older PR's run with "unrecognized arguments".
|
|
workflow = Path(".github/workflows/pr-health.yml").read_text(encoding="utf-8")
|
|
|
|
assert "pull_request.base.sha" not in workflow
|
|
assert workflow.count("ref: ${{ github.sha }}") == workflow.count("actions/checkout@")
|