fix(binaries): fail closed on HEADROOM_BINARIES_ALLOW_UNVERIFIED=0/false (#3218)

## Description

The sha256-verification bypass for fetched tool binaries is a
supply-chain
integrity control, but both call sites gated it on a bare presence
check:

```python
if os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
    ...skip sha256 verification...
```

Any non-empty value is truthy — including `0`, `false`, `no`, `off`. An
operator who set `HEADROOM_BINARIES_ALLOW_UNVERIFIED=0` (the natural way
to
say "keep verification on") therefore **silently disabled** sha256
verification of every fetched binary (`difft`, `scc`, and the in-memory
`rtk` / `lean-ctx` / `codebase-memory-mcp` installer path via
`verify_download_bytes`). The failure direction is backwards for a
security
control: it fails **open**.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Parse `HEADROOM_BINARIES_ALLOW_UNVERIFIED` as an affirmative flag
(`1`/`true`/`yes`/`on`) via a shared `_allow_unverified_binaries()`
helper,
  used by both `_verify_sha256` and `verify_download_bytes`.
- Every non-affirmative value — and an unset or empty variable — now
**enforces** verification (fail closed). Matches the truthy-set parsing
  already used across the codebase (`onnx_runtime`, `cold_prefix`,
  `ttl_observations`, `qdrant_env`).
- Added regression tests over the env-value matrix on both call sites.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom --ignore-missing-imports`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_binaries.py -q
35 passed, 1 warning in 1.15s

# New tests fail before the fix (0/false/no/off/whitespace + in-memory path):
$ git stash push -- headroom/binaries.py && \
  python -m pytest tests/test_binaries.py -q -k "allow_unverified or verify_download_bytes_zero"
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[0]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[false]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[no]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[off]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[  ]
FAILED ...::test_verify_download_bytes_zero_still_verifies
6 failed, 7 passed, 22 deselected

$ python -m ruff check headroom/binaries.py tests/test_binaries.py && mypy headroom --ignore-missing-imports
All checks passed!
Success: no issues found in 527 source files
```

## Real Behavior Proof

- Environment: macOS (Darwin arm64), Python 3.13. Operator sets
`HEADROOM_BINARIES_ALLOW_UNVERIFIED=0` (intending to keep verification
enabled). A tampered binary whose bytes do not match the registry-pinned
sha256 is passed to `binaries._verify_sha256`.
- Exact command / steps: write a file containing
`b"TAMPERED-BINARY-CONTENT"`, then call `binaries._verify_sha256(path,
<wrong pin>)` under `HEADROOM_BINARIES_ALLOW_UNVERIFIED=0`, once with
the fix stashed (before) and once applied (after).
- Observed result: BEFORE the fix the mismatched binary is accepted
(verification skipped); AFTER the fix it is rejected with
`Sha256Mismatch`.
  ```text
### HEADROOM_BINARIES_ALLOW_UNVERIFIED=0 (operator intent: keep
verifying)
  --- BEFORE fix ---
skipping sha256 verification for difft
(HEADROOM_BINARIES_ALLOW_UNVERIFIED=1)
  RESULT: verification SKIPPED — tampered binary ACCEPTED  (fail-open)
  --- AFTER fix ----
RESULT: Sha256Mismatch raised — tampered binary REJECTED (fail-closed)
  ```
- Not tested: a live end-to-end download of difft/scc from GitHub
releases (network-dependent); the verification function that gates every
such download is exercised directly above with a real byte payload.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — unconditional hardening of an
existing security control.
- Minimum rollout channel: N/A
- Stable/default behavior changed: only for non-affirmative env values
that previously bypassed verification; `=1`/`true`/`yes`/`on` still
bypass exactly as before, and the default (unset) still verifies.
- Kill switch / disable path: `HEADROOM_BINARIES_ALLOW_UNVERIFIED=1`
still bypasses verification for operators who genuinely need it.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [] I have made corresponding changes to the documentation (N/A —
behavior now matches the existing `=1` docstrings)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3218"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `8c981e8248ff7dca7fd36bc55805a096d3943473`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Clean mechanical current-main integration with upstream
c01cf07d94. Re-reviewed fail-closed
environment parsing and both file/in-memory verification callers
alongside current author/Devin discussion. Only explicit
affirmative1/true/yes/on values bypass verification, case-insensitive
and trimmed; unset,false,unknown retain checks, with bypass warnings
shared through main helper. All44 binary-verification tests pass,
including mismatched-pin rejection and bypass cases;2-file Ruff
lint/format and main-relative whitespace pass. Maintainer
documentation-only follow-up addresses Devin note: module
environment-variable documentation now lists affirmative values and
fail-closed fallback. No contributor verification logic rewritten. Tests
use local payloads and temporary files; no live external binary download
or fresh native build claimed. Formal approval awaits exact-head hosted
qualification; maintainers handle CI. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
This commit is contained in:
Rajat Nagda
2026-11-06 09:33:27 -06:00
committed by GitHub
co-authored by Jerrett Davis
parent d21f74e396
commit da6d9736cd
2 changed files with 64 additions and 3 deletions
+20 -3
View File
@@ -19,8 +19,10 @@ Env vars:
HEADROOM_BINARIES_CACHE override cache dir
HEADROOM_BINARIES_OFFLINE if set, never reach the network
HEADROOM_BINARIES_ALLOW_UNVERIFIED
Escape hatch: if set, skip SHA-256 verification entirely and accept
whatever the mirror served. This disables Headroom's only defence
Escape hatch: only 1, true, yes, or on (case-insensitive, with surrounding
whitespace ignored) skip SHA-256 verification and accept whatever the
mirror served. Unset, false, and unrecognized values retain verification.
This disables Headroom's only defence
against a tampered or substituted binary, so it is announced on
stderr every time it takes effect. Intended for developers testing a
locally built asset or an off-registry version; never set it in CI or
@@ -58,6 +60,19 @@ from headroom.offline import guard_egress
logger = logging.getLogger(__name__)
# Affirmative values for the sha256-verification bypass. This is a security
# control (supply-chain integrity), so it must fail CLOSED: only an explicit
# affirmative disables verification. A bare presence check treated
# `HEADROOM_BINARIES_ALLOW_UNVERIFIED=0` / `=false` — the natural way to say
# "keep verifying" — as truthy and silently skipped verification.
_TRUTHY = frozenset({"1", "true", "yes", "on"})
def _allow_unverified_binaries() -> bool:
"""Whether sha256 verification of fetched binaries is bypassed."""
return os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "").strip().lower() in _TRUTHY
__all__ = [
"BinaryError",
"BinaryFetchError",
@@ -385,7 +400,7 @@ def _allow_unverified(name: str) -> bool:
when ``lastResort`` will not fire, and when the log may be going somewhere
an operator is not watching, such as a file.
"""
if not os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
if not _allow_unverified_binaries():
return False
message = (
f"headroom: WARNING: accepting {name} WITHOUT sha256 verification "
@@ -399,6 +414,7 @@ def _allow_unverified(name: str) -> bool:
def _verify_sha256(path: Path, expected: str | None) -> None:
if _allow_unverified(path.name):
return
if not expected:
@@ -439,6 +455,7 @@ def verify_download_bytes(data: bytes, *, url: str, name: str) -> None:
Sha256Mismatch if the bytes do not match it. Setting
HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 skips both checks.
"""
if _allow_unverified(name):
return
expected = sha256_for_url(url)
+44
View File
@@ -507,6 +507,50 @@ def test_status_reports_every_registered_tool(monkeypatch):
assert r["state"] in ("on-path", "cached", "missing", "unsupported-platform")
# -------- HEADROOM_BINARIES_ALLOW_UNVERIFIED (security: fail closed) ------- #
def _write(tmp_path, data: bytes):
p = tmp_path / "asset.bin"
p.write_bytes(data)
return p
@pytest.mark.parametrize("value", ["0", "false", "no", "off", "", " "])
def test_allow_unverified_non_affirmative_still_verifies(monkeypatch, tmp_path, value):
# A security bypass must fail CLOSED: only an explicit affirmative disables
# verification. Setting the var to "0"/"false" (the natural way to say "keep
# verifying") previously passed a bare presence check and silently skipped
# sha256 verification of the downloaded binary.
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", value)
path = _write(tmp_path, b"payload")
with pytest.raises(binaries.Sha256Mismatch):
binaries._verify_sha256(path, "deadbeef" * 8) # wrong pin
def test_allow_unverified_unset_still_verifies(monkeypatch, tmp_path):
monkeypatch.delenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", raising=False)
path = _write(tmp_path, b"payload")
with pytest.raises(binaries.Sha256Mismatch):
binaries._verify_sha256(path, "deadbeef" * 8)
@pytest.mark.parametrize("value", ["1", "true", "TRUE", "yes", "on"])
def test_allow_unverified_affirmative_skips_verification(monkeypatch, tmp_path, value):
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", value)
path = _write(tmp_path, b"payload")
# Wrong pin, but an explicit affirmative bypass returns without raising.
binaries._verify_sha256(path, "deadbeef" * 8)
def test_verify_download_bytes_zero_still_verifies(monkeypatch):
# Same fail-closed contract on the in-memory installer path.
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "0")
monkeypatch.setattr(binaries, "sha256_for_url", lambda _url: "deadbeef" * 8)
with pytest.raises(binaries.Sha256Mismatch):
binaries.verify_download_bytes(b"payload", url="https://x/y", name="y")
# -------- Registry key hygiene ------------------------------------------- #