fix(binaries): fail closed when a download carries no sha256 pin (#3724)

## A-7 (S2, supply chain) — binary verification failed open on an
unpinned URL

`headroom/binaries.py` skipped SHA-256 verification entirely, with only
a log line, when the registry carried **no pin** for a download URL — in
both `_verify_sha256()` (the `resolve()` fetch path) and
`verify_download_bytes()` (the in-memory installer path). An
off-registry version override, or an installer building a filename the
registry does not describe, therefore installed whatever the mirror
served, protected only by HTTPS transport trust.

`verify_download_bytes`'s own docstring already claimed it *"refuses an
unpinned URL"*. It did not — the documentation described behaviour the
code did not have.

**Both claims in the backlog verified against `origin/main` (5ff4ea1e);
line numbers had not drifted from the 0.37.0 review.**

### What changed

- New `UnpinnedDownload(BinaryError)`. Both verification paths now
**raise** on a missing pin instead of falling through — the partial
download is deleted first, so nothing unverified is left in the cache
for a later call to pick up.
- `HEADROOM_BINARIES_ALLOW_UNVERIFIED` is kept as the escape hatch but
is now **loud**: a shared `_allow_unverified()` helper logs a warning
*and* prints to stderr. The logger is routinely unconfigured at proxy
startup, which is exactly when this fires, so the log-only warning was
invisible in practice.
- The env var is now documented in the module docstring (it was missing
from the `Env vars:` block entirely) with an explicit "never set this in
CI or production; pin the asset instead".
- `verify_download_bytes`'s docstring corrected to describe what the
code does, and its stale caller list trimmed (see below).
- `ensure_tools()` catches `UnpinnedDownload` alongside the other fetch
failures, so proxy startup still degrades to "tool missing" rather than
crashing.

### ⚠️ This changes install-time behaviour

A download with no `tools.json` pin now **fails** instead of installing.
No shipped path is affected (see the caller trace), but anyone relying
on the old silent fall-through — a local off-registry version override,
a fork with a hand-edited registry — will now get a hard error. The fix
is to pin the asset (`scripts/refresh_tool_hashes.py`), not to set the
escape hatch.

### Callers traced

`_verify_sha256` has exactly one caller and `verify_download_bytes`
exactly one; everything else reaches them transitively.

| Path | Pinned today? | Breaks? |
|---|---|---|
| `resolve()` → `_verify_sha256` (binaries.py) | Yes — pin comes from
`tools.json`, **all 21 assets across difft/scc/codebase-memory-mcp are
pinned** | No |
| `ensure_tools()` → `resolve()` (proxy startup, `cli/proxy.py:1216`) |
Yes | No — and a refusal is caught, so startup still succeeds with the
tool skipped |
| `cli/tools.py:43,230` → `resolve()` (`headroom tools install` /
`doctor`) | Yes | No |
| `proxy/interceptors/astgrep.py:122` → `resolve("ast-grep")` | n/a —
ast-grep is PyPI-only, never downloaded here | No |
| `graph/installer.py:82` `download_cbm()` → `verify_download_bytes` |
Yes for the default `CBM_VERSION = v0.8.1` on linux-amd64/arm64 and
darwin-amd64/arm64 | No |
| …same, with a `version=` override | **No** — off-registry URL | **Yes,
by design.** No production caller passes `version`; `ensure_cbm()` calls
`download_cbm()` with no args. Only tests did. |
| …same, on **Windows** | **No** — see "things the brief did not know"
below | Already broken today; now fails earlier with a clearer message |
| ONNX / embedding models (`headroom/onnx_runtime.py`) | n/a —
`hf_hub_download` at a pinned HF revision, never touches this module |
No |
| Rust / SmartCrusher | n/a — built by cargo (`ort-download-binaries` is
a cargo feature), not fetched through `binaries.py` | No |
| rtk / lean-ctx | n/a — those download paths were **removed** (see
`context_tool_cleanup.py`) | No |

The registry really does pin everything the product ships:
`.github/workflows/tools-hash-refresh.yml` runs
`scripts/refresh_tool_hashes.py --check`, which exits 1 if any pin is
*missing or stale*, on every `tools.json` PR and weekly. A new unit test
(`test_every_shipped_asset_is_pinned`) asserts the same thing so an
unpinned entry fails fast in unit tests too — that gate is what makes
failing closed safe.

### Fail-before / pass-after

Method: commit the work, `git checkout origin/main --
headroom/binaries.py` (keeping the new tests), run, restore with `git
checkout HEAD -- headroom/binaries.py`.

**Before** (new tests against `origin/main`'s `binaries.py`):

```
tests/test_binaries.py ...................FFFF.......                    [ 63%]
tests/test_graph.py ..........F......                                    [100%]

______________ test_unpinned_asset_is_refused_and_partial_deleted ______________
E   Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>
_________ test_unpinned_asset_allowed_by_escape_hatch_warns_on_stderr __________
E   AssertionError: assert 'WITHOUT sha256 verification' in ''
_______________ test_verify_download_bytes_refuses_unpinned_url ________________
E   Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>
___________ test_verify_download_bytes_escape_hatch_warns_on_stderr ____________
E   AssertionError: assert 'WITHOUT sha256 verification' in ''
______________ test_download_cbm_refuses_an_off_registry_version _______________
E   Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>

========================= 5 failed, 42 passed in 5.21s =========================
```

The three `DID NOT RAISE` failures are the defect itself: on `main`, an
unpinned download is accepted and installed.

**After**:

```
tests/test_binaries.py ..............................                    [ 63%]
tests/test_graph.py .................                                    [100%]

============================== 47 passed in 3.74s ==============================
```

Wider sweep (every test module that touches `binaries`) — `test_binaries
test_graph test_cli_tools test_release_workflows
test_context_tool_cleanup test_memory/test_traffic_learner`: **294
passed**.

Lint: `uvx ruff@0.16.7 check .` → clean; `format --check` → clean.

### Existing tests that had to be adjusted

`tests/test_binaries.py`'s autouse fixture nulls every shipped pin so
mechanics tests can serve tiny mock archives. That made *every* such
fetch an unpinned one, so four fetch/extract-mechanics tests now opt
into the escape hatch via a new `allow_unverified` fixture, and the
autouse fixture `delenv`s the variable so a developer's exported value
cannot silently mask the guard. Two `download_cbm` tests in
`tests/test_graph.py` needed the same, for the same reason — they drive
off-registry URLs on purpose.

### Things the brief got wrong / found along the way (not fixed here)

1. **`verify_download_bytes`'s caller list was stale.** It named rtk and
lean-ctx; both downloaders were removed
(`headroom/context_tool_cleanup.py` exists precisely to clean up after
them). Only codebase-memory-mcp remains. Docstring corrected.
2. **Pre-existing, unrelated bug: `download_cbm` is broken on Windows.**
`graph/installer.py` builds `codebase-memory-mcp-windows-amd64.tar.gz`,
but the registry pins `...-windows-amd64.zip`, and the extractor calls
`tarfile.open(mode="r:gz")` unconditionally. So the Windows path already
fails (404, or a tar error on a zip) before this change. This PR makes
it fail one step earlier, at verification, with a message that names the
cause. Worth a separate issue — I did not widen scope to fix it.
3. **`ensure_cbm()` catches only `RuntimeError`**, so a `Sha256Mismatch`
— a genuine tamper signal — already escapes it today, and
`UnpinnedDownload` would too. Since no production caller passes a
`version` override, the shipped path cannot reach it, so I left it alone
rather than converting a loud integrity failure into a silent `None`.
Flagging it as a judgement call.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tejas Chopra
2026-09-23 20:21:41 +02:00
committed by GitHub
co-authored by Claude Opus 5
parent 34da21e89e
commit 26a2c4939c
7 changed files with 378 additions and 36 deletions
+84 -23
View File
@@ -9,10 +9,22 @@ Supported platforms: linux (glibc + musl) x86_64/aarch64, macOS x86_64/arm64,
Windows x86_64. Unsupported platforms raise PlatformNotSupported; callers in
the compression pipeline should fall back to their non-accelerated path.
Every asset this package ships is pinned by SHA-256 in `tools.json`, and the
tools-hash-refresh CI gate fails the build if a pin is missing or stale. So
verification fails closed: a download whose URL carries no pin is refused
rather than trusted on transport (HTTPS) alone.
Env vars:
HEADROOM_BINARIES_MIRROR base URL that replaces https://github.com
HEADROOM_BINARIES_CACHE override cache dir
HEADROOM_BINARIES_OFFLINE if set, never reach the network
HEADROOM_BINARIES_ALLOW_UNVERIFIED
Escape hatch: if set, skip SHA-256 verification entirely and accept
whatever the mirror served. This disables Headroom's only defence
against a tampered or substituted binary, so it is announced on
stderr every time it takes effect. Intended for developers testing a
locally built asset or an off-registry version; never set it in CI or
in production. Pin the asset in `tools.json` instead.
"""
from __future__ import annotations
@@ -47,6 +59,7 @@ __all__ = [
"BinaryFetchError",
"PlatformNotSupported",
"Sha256Mismatch",
"UnpinnedDownload",
"OfflineError",
"PlatformKey",
"detect_platform",
@@ -77,6 +90,18 @@ class Sha256Mismatch(BinaryError):
"""Raised when a downloaded asset's SHA256 does not match the pin."""
class UnpinnedDownload(BinaryError):
"""Raised when a downloaded asset has no SHA256 pin to verify it against.
Every shipped asset is pinned (tools-hash-refresh CI gate), so a missing
pin means an off-registry fetch — a version override, a hand-edited
registry, or a URL an installer built that the registry does not describe.
Accepting it would silently downgrade integrity to transport trust, so we
refuse. Pin the asset in `tools.json`, or set
HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 to accept the risk explicitly.
"""
class OfflineError(BinaryError):
"""Raised when a network fetch is required but HEADROOM_BINARIES_OFFLINE is set."""
@@ -310,19 +335,47 @@ def _sha256_file(path: Path) -> str:
return h.hexdigest()
def _allow_unverified(name: str) -> bool:
"""True if the escape hatch is set — announced loudly each time it fires.
Skipping verification turns a supply-chain guarantee into transport trust,
so it goes to stderr as well as the logger: the logger is frequently
unconfigured at proxy startup, and a silent downgrade is the failure mode
this whole module exists to avoid.
Exactly once, though. With no handlers configured, logging's ``lastResort``
already writes WARNING to stderr, so printing unconditionally produced the
same sentence twice. Print only when a handler exists -- that is precisely
when ``lastResort`` will not fire, and when the log may be going somewhere
an operator is not watching, such as a file.
"""
if not os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
return False
message = (
f"headroom: WARNING: accepting {name} WITHOUT sha256 verification "
"(HEADROOM_BINARIES_ALLOW_UNVERIFIED is set). A tampered or "
"substituted binary will not be detected."
)
logger.warning("%s", message)
if logger.hasHandlers():
print(message, file=sys.stderr)
return True
def _verify_sha256(path: Path, expected: str | None) -> None:
if os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
logger.warning(
"skipping sha256 verification for %s (HEADROOM_BINARIES_ALLOW_UNVERIFIED=1)",
path.name,
)
if _allow_unverified(path.name):
return
if not expected:
# No pin in the registry (e.g. an off-registry version override). All
# shipped assets ARE pinned — enforced by the tools-hash-refresh CI gate —
# so a missing pin means an off-registry fetch; fall back to HTTPS trust.
logger.info("binary %s downloaded without sha256 pin (HTTPS trust only)", path.name)
return
# No pin in the registry. All shipped assets ARE pinned — enforced by
# the tools-hash-refresh CI gate — so a missing pin means an
# off-registry fetch. Fail closed: HTTPS proves only who served the
# bytes, not that they are the bytes we pinned.
path.unlink(missing_ok=True)
raise UnpinnedDownload(
f"refusing unpinned download {path.name}: no sha256 in the registry. "
"Pin it in headroom/tools.json (scripts/refresh_tool_hashes.py), or set "
"HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 to accept it unverified."
)
got = _sha256_file(path)
if got.lower() != expected.lower():
path.unlink(missing_ok=True)
@@ -342,22 +395,26 @@ def sha256_for_url(url: str) -> str | None:
def verify_download_bytes(data: bytes, *, url: str, name: str) -> None:
"""Fail-closed integrity check for an in-memory downloaded archive.
Used by installers (rtk, lean-ctx, codebase-memory-mcp) that download and
extract on their own instead of going through the fetch path above. Verifies
the bytes against the tools.json pin for ``url`` and refuses an unpinned URL
unless HEADROOM_BINARIES_ALLOW_UNVERIFIED=1.
Used by installers (codebase-memory-mcp) that download and extract on their
own instead of going through the fetch path above. Verifies the bytes
against the tools.json pin for ``url``.
Raises UnpinnedDownload if ``url`` has no pin in the registry and
Sha256Mismatch if the bytes do not match it. Setting
HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 skips both checks.
"""
if os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
logger.warning(
"skipping sha256 verification for %s (HEADROOM_BINARIES_ALLOW_UNVERIFIED=1)", name
)
if _allow_unverified(name):
return
expected = sha256_for_url(url)
if not expected:
# Off-registry URL (e.g. a version override); shipped assets are all
# pinned via the CI gate, so fall back to HTTPS trust here.
logger.info("%s downloaded without sha256 pin (HTTPS trust only)", name)
return
# Off-registry URL — a version override, or an installer building a
# filename the registry does not carry. Refuse rather than silently
# downgrading to transport trust.
raise UnpinnedDownload(
f"refusing unpinned download {name}: {url} has no sha256 in the registry. "
"Pin it in headroom/tools.json (scripts/refresh_tool_hashes.py), or set "
"HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 to accept it unverified."
)
got = hashlib.sha256(data).hexdigest()
if got.lower() != expected.lower():
raise Sha256Mismatch(f"sha256 mismatch for {name}: expected {expected}, got {got}")
@@ -490,7 +547,8 @@ def resolve(tool: str) -> Path:
Raises PlatformNotSupported if the tool is unavailable on this platform,
OfflineError if a fetch is required but HEADROOM_BINARIES_OFFLINE is set,
Sha256Mismatch if verification fails, BinaryFetchError on other IO errors.
Sha256Mismatch if verification fails, UnpinnedDownload if the asset carries
no registry pin, BinaryFetchError on other IO errors.
"""
on_path = _path_lookup(tool)
if on_path:
@@ -562,6 +620,9 @@ def ensure_tools(quiet: bool = False) -> dict[str, Path | None]:
OfflineError,
BinaryFetchError,
Sha256Mismatch,
# An unpinned asset is refused, not installed. Startup still
# degrades to "tool missing" rather than failing outright.
UnpinnedDownload,
# Catch readonly / sandboxed filesystems (e.g. containerized
# home dirs) so proxy startup never fails because the cache dir
# can't be created. The interceptor fall back to no-op.
+10 -2
View File
@@ -52,7 +52,12 @@ def _exec_tool(tool: str, argv: Sequence[str]) -> None:
err=True,
)
sys.exit(2)
except (binaries.Sha256Mismatch, binaries.BinaryFetchError) as e:
# BinaryError, not a list of subclasses: every failure here is a refusal to
# run an unverified tool, and enumerating them meant UnpinnedDownload -- added
# later -- escaped as a traceback instead of the `error: ...` + exit 2 every
# sibling produces. PlatformNotSupported and OfflineError are caught above for
# their specific hints; this is the catch-all for the rest.
except binaries.BinaryError as e:
click.secho(f"error: {e}", fg="red", err=True)
sys.exit(2)
@@ -232,7 +237,10 @@ def tools_install_cmd(tools: tuple[str, ...], force: bool) -> None:
except binaries.PlatformNotSupported as e:
click.secho(f"{name}: {e}", fg="red")
exit_code = 1
except (binaries.BinaryFetchError, binaries.Sha256Mismatch, binaries.OfflineError) as e:
# Base class for the same reason as _exec_tool: an escaping subclass here
# also aborted the loop over the remaining tools and skipped the final
# sys.exit(exit_code), so one unpinned asset silently stopped the install.
except binaries.BinaryError as e:
click.secho(f"{name}: {e}", fg="red")
exit_code = 1
sys.exit(exit_code)
+10
View File
@@ -126,8 +126,18 @@ def ensure_cbm() -> Path | None:
if existing:
return existing
# Local import to match this module's lazy-import style (see download_cbm).
from headroom.binaries import UnpinnedDownload
try:
return download_cbm()
except RuntimeError as e:
logger.warning("Failed to install codebase-memory-mcp: %s", e)
return None
except UnpinnedDownload as e:
# Documented contract is "path, or None if the download failed", and a
# refusal is a failure to install -- the feature is simply unavailable.
# Deliberately NOT widened to BinaryError: Sha256Mismatch is a tamper
# signal and must keep propagating rather than becoming a quiet None.
logger.warning("Refusing to install codebase-memory-mcp: %s", e)
return None
+22 -7
View File
@@ -164,10 +164,17 @@ def _null_binary_pins():
Installer tests fetch small mock archives, whose digests can't match the
real published pins. Nulling the pins lets those download/extract mechanics
tests run (verification then falls back to HTTPS trust); the tests that
specifically exercise verification set their own pin explicitly. Production
keeps the real pins (this fixture is test-only) and the tools-hash-refresh
CI gate guarantees they stay correct.
tests run; the tests that specifically exercise verification set their own
pin explicitly. Production keeps the real pins (this fixture is test-only)
and the tools-hash-refresh CI gate guarantees they stay correct.
Nulling a pin used to mean "fall back to HTTPS trust". It now means
"refuse", so the escape hatch has to be set alongside it or every test that
reaches a real download fails closed -- which is what happened to
test_bundled_tools_savings.py on a cold CI cache, while passing locally
against an already-populated one. Setting both together keeps this fixture
saying one thing: "verification is not what these tests are about."
Verification tests delenv it in their own fixture.
"""
try:
from headroom import binaries
@@ -185,9 +192,17 @@ def _null_binary_pins():
]
for asset, _original in saved:
asset["sha256"] = None
yield
for asset, original in saved:
asset["sha256"] = original
previous = os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED")
os.environ["HEADROOM_BINARIES_ALLOW_UNVERIFIED"] = "1"
try:
yield
finally:
if previous is None:
os.environ.pop("HEADROOM_BINARIES_ALLOW_UNVERIFIED", None)
else:
os.environ["HEADROOM_BINARIES_ALLOW_UNVERIFIED"] = previous
for asset, original in saved:
asset["sha256"] = original
@pytest.fixture(autouse=True)
+112 -4
View File
@@ -32,6 +32,10 @@ def _clear_caches(monkeypatch, tmp_path):
monkeypatch.setenv("HEADROOM_BINARIES_CACHE", str(tmp_path / "cache"))
monkeypatch.delenv("HEADROOM_BINARIES_MIRROR", raising=False)
monkeypatch.delenv("HEADROOM_BINARIES_OFFLINE", raising=False)
# Verification fails closed on a nulled (unpinned) asset, so mechanics
# tests opt into the escape hatch via the `allow_unverified` fixture; a
# developer's exported value must not leak in and mask the guard.
monkeypatch.delenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", raising=False)
yield
binaries.detect_platform.cache_clear()
binaries._registry.cache_clear()
@@ -81,6 +85,17 @@ class _FakeResponse:
return False
@pytest.fixture
def allow_unverified(monkeypatch):
"""Opt out of SHA-256 verification for tests about fetch/extract mechanics.
The autouse fixture nulls the shipped pins so these tests can serve small
mock archives; without the escape hatch every such fetch is refused as
unpinned, which is the point of `test_unpinned_*` below.
"""
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
@pytest.fixture
def fake_urlopen(monkeypatch):
"""Install a fake urllib.request.urlopen that serves registered URLs."""
@@ -203,7 +218,9 @@ def test_mirror_substitution(monkeypatch):
assert binaries._mirror_url("https://example.com/x") == "https://example.com/x"
def test_mirror_url_with_query_params_strips_them_from_download_filename(monkeypatch, fake_urlopen):
def test_mirror_url_with_query_params_strips_them_from_download_filename(
monkeypatch, fake_urlopen, allow_unverified
):
"""Mirror URLs with `?token=...` must not leak into the download filename.
Regression test for the case where `Path(url).name` gave
@@ -231,7 +248,7 @@ def test_mirror_url_with_query_params_strips_them_from_download_filename(monkeyp
asset["url"] = original_url
def test_fetch_extract_and_cache_tar_gz(monkeypatch, fake_urlopen, tmp_path):
def test_fetch_extract_and_cache_tar_gz(monkeypatch, fake_urlopen, tmp_path, allow_unverified):
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
@@ -249,7 +266,7 @@ def test_fetch_extract_and_cache_tar_gz(monkeypatch, fake_urlopen, tmp_path):
assert path2 == path
def test_fetch_extract_zip(monkeypatch, fake_urlopen):
def test_fetch_extract_zip(monkeypatch, fake_urlopen, allow_unverified):
_set_platform(monkeypatch, sys_plat="win32", machine="AMD64")
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
payload = b"MZfake"
@@ -321,6 +338,97 @@ def test_sha256_match_passes(monkeypatch, fake_urlopen):
asset["sha256"] = None
# -------- Unpinned downloads fail closed (A-7) --------------------------- #
def test_unpinned_asset_is_refused_and_partial_deleted(monkeypatch, fake_urlopen):
"""An asset with no registry pin must raise, not fall back to HTTPS trust.
The autouse fixture has already nulled every shipped pin, which is exactly
the shape of an off-registry version override.
"""
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
asset = binaries._registry()["tools"]["difft"]["assets"]["darwin-aarch64"]
assert asset["sha256"] is None
fake_urlopen[asset["url"]] = _make_tar_gz({"difft": b"untrusted"})
# Caught as BinaryError so the assertion is about behaviour, not the name
# of the subclass; the type is pinned on the next line.
with pytest.raises(binaries.BinaryError) as exc:
binaries.resolve("difft")
assert type(exc.value) is binaries.UnpinnedDownload
assert "HEADROOM_BINARIES_ALLOW_UNVERIFIED" in str(exc.value)
# Nothing unverified is left in the cache for a later call to pick up.
# Version read from the registry, not hardcoded: a hardcoded one silently
# starts asserting about a path that never existed after a version bump.
version = binaries._registry()["tools"]["difft"]["version"]
assert not binaries._cached_path("difft", version, binaries.detect_platform()).exists()
def test_unpinned_asset_allowed_by_escape_hatch_warns_on_stderr(
monkeypatch, fake_urlopen, capsys, allow_unverified
):
"""The escape hatch still installs, but says so where an operator sees it."""
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
asset = binaries._registry()["tools"]["difft"]["assets"]["darwin-aarch64"]
fake_urlopen[asset["url"]] = _make_tar_gz({"difft": b"untrusted"})
path = binaries.resolve("difft")
assert path.read_bytes() == b"untrusted"
err = capsys.readouterr().err
assert "WITHOUT sha256 verification" in err
assert "HEADROOM_BINARIES_ALLOW_UNVERIFIED" in err
def test_verify_download_bytes_refuses_unpinned_url():
with pytest.raises(binaries.BinaryError) as exc:
binaries.verify_download_bytes(
b"payload", url="https://example.test/off-registry.tar.gz", name="some-tool"
)
assert type(exc.value) is binaries.UnpinnedDownload
assert "off-registry.tar.gz" in str(exc.value)
def test_verify_download_bytes_escape_hatch_warns_on_stderr(capsys, allow_unverified):
binaries.verify_download_bytes(
b"payload", url="https://example.test/off-registry.tar.gz", name="some-tool"
)
assert "WITHOUT sha256 verification" in capsys.readouterr().err
def test_verify_download_bytes_checks_a_pinned_url():
payload = b"payload"
asset = binaries._registry()["tools"]["difft"]["assets"]["darwin-aarch64"]
asset["sha256"] = hashlib.sha256(payload).hexdigest()
try:
binaries.verify_download_bytes(payload, url=asset["url"], name="difft")
with pytest.raises(binaries.Sha256Mismatch):
binaries.verify_download_bytes(b"tampered", url=asset["url"], name="difft")
finally:
asset["sha256"] = None
def test_every_shipped_asset_is_pinned():
"""The fail-closed guard is only safe because the registry pins everything.
Enforced against the published assets by the tools-hash-refresh workflow;
checked here so a new unpinned entry fails fast in unit tests too. Reads
the registry from disk because the autouse fixture nulls the live pins.
"""
import json
registry = json.loads(binaries._REGISTRY_PATH.read_text(encoding="utf-8"))
unpinned = [
f"{tool}/{key}"
for tool, entry in registry.get("tools", {}).items()
for key, asset in entry.get("assets", {}).items()
if not asset.get("sha256")
]
assert unpinned == []
# -------- status() ------------------------------------------------------- #
@@ -349,7 +457,7 @@ def test_ensure_tools_survives_readonly_cache_dir(monkeypatch, tmp_path):
readonly_parent.chmod(0o700) # so pytest tmp cleanup succeeds
def test_ensure_tools_partial_failure_proxy_still_starts(monkeypatch):
def test_ensure_tools_partial_failure_proxy_still_starts(monkeypatch, allow_unverified):
"""If one tool fails to fetch, others still install and ensure_tools returns."""
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
@@ -0,0 +1,113 @@
"""A refusal is only useful if the caller turns it into a message.
`UnpinnedDownload` was added as a new `BinaryError` subclass, but three callers
caught a hand-written tuple of subclasses rather than the base, so the new one
escaped as a traceback. These pin the contract at the boundaries a person or a
calling feature actually sees.
"""
from __future__ import annotations
import logging
import pytest
from click.testing import CliRunner
from headroom import binaries
@pytest.fixture
def _refuse(monkeypatch: pytest.MonkeyPatch):
"""Make `resolve` refuse the way an off-registry asset does."""
def _raise(tool: str):
raise binaries.UnpinnedDownload(
f"refusing unpinned download {tool}: no sha256 in the registry. "
"Pin it, or set HEADROOM_BINARIES_ALLOW_UNVERIFIED=1."
)
monkeypatch.setattr(binaries, "resolve", _raise)
monkeypatch.delenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", raising=False)
def test_tools_run_reports_a_refusal_instead_of_a_traceback(_refuse) -> None:
"""`headroom tools <name>` must exit 2 with `error: ...`, like every sibling."""
from headroom.cli.tools import diff_cmd
result = CliRunner().invoke(diff_cmd, ["--version"])
assert result.exit_code == 2, result.output
assert not isinstance(result.exception, binaries.UnpinnedDownload), (
"the refusal escaped the CLI as a traceback"
)
assert "refusing unpinned download" in result.output
def test_tools_install_keeps_going_and_still_sets_an_exit_code(_refuse) -> None:
"""A refusal on one tool must not abort the loop over the others.
The escape skipped the final `sys.exit(exit_code)` entirely, so the command
reported success-by-omission for every tool after the first refusal.
"""
from headroom.cli.tools import tools_group
result = CliRunner().invoke(tools_group, ["install"])
assert not isinstance(result.exception, binaries.UnpinnedDownload), (
"the refusal aborted `tools install`"
)
assert result.exit_code == 1, result.output
def test_ensure_cbm_honours_its_documented_none_contract(monkeypatch) -> None:
"""`ensure_cbm` promises "path, or None if the download failed"."""
from headroom.graph import installer
monkeypatch.setattr(installer, "get_cbm_path", lambda: None)
def _raise():
raise binaries.UnpinnedDownload("refusing unpinned download codebase-memory-mcp")
monkeypatch.setattr(installer, "download_cbm", _raise)
assert installer.ensure_cbm() is None
def test_a_tamper_signal_still_propagates_from_ensure_cbm(monkeypatch) -> None:
"""The None contract must not swallow a sha256 mismatch.
An unpinned asset is a misconfiguration; a mismatch is evidence of tampering
and has to stay loud rather than becoming a quiet "feature unavailable".
"""
from headroom.graph import installer
monkeypatch.setattr(installer, "get_cbm_path", lambda: None)
def _raise():
raise binaries.Sha256Mismatch("sha256 mismatch for codebase-memory-mcp")
monkeypatch.setattr(installer, "download_cbm", _raise)
with pytest.raises(binaries.Sha256Mismatch):
installer.ensure_cbm()
def test_the_escape_hatch_warning_is_not_printed_twice(monkeypatch, capsys) -> None:
"""With no handlers configured, logging's lastResort already writes stderr.
Printing unconditionally as well produced the same sentence twice.
"""
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
logger = logging.getLogger("headroom.binaries")
saved, logger.handlers = logger.handlers, []
saved_propagate, logger.propagate = logger.propagate, False
try:
assert binaries._allow_unverified("difft") is True
finally:
logger.handlers, logger.propagate = saved, saved_propagate
stderr = capsys.readouterr().err
assert stderr.count("WITHOUT sha256 verification") <= 1, (
f"the warning was emitted more than once:\n{stderr}"
)
+27
View File
@@ -76,6 +76,9 @@ def test_get_cbm_path_prefers_path_then_install_dir(monkeypatch, tmp_path: Path)
def test_download_cbm_success_and_verification_paths(monkeypatch, tmp_path: Path) -> None:
# v1.2.3 is not in the registry and the mock archive would not match a pin
# anyway; this test is about the install mechanics, not verification.
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
monkeypatch.setattr(installer, "CBM_BIN_DIR", tmp_path)
monkeypatch.setattr(installer, "_detect_platform", lambda: "linux-amd64")
monkeypatch.setattr(
@@ -109,6 +112,9 @@ def test_download_cbm_success_and_verification_paths(monkeypatch, tmp_path: Path
def test_download_cbm_invalid_url_download_failure_and_extract_errors(
monkeypatch, tmp_path: Path
) -> None:
# These failure modes are reached with an overridden (off-registry) release
# host, so verification would refuse them before the code under test runs.
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
monkeypatch.setattr(installer, "CBM_BIN_DIR", tmp_path)
monkeypatch.setattr(installer, "_detect_platform", lambda: "linux-amd64")
@@ -138,6 +144,27 @@ def test_download_cbm_invalid_url_download_failure_and_extract_errors(
installer.download_cbm()
def test_download_cbm_refuses_an_off_registry_version(monkeypatch, tmp_path: Path) -> None:
"""A version override points at a URL with no pin — refuse it (A-7).
The default CBM_VERSION is pinned for every platform in tools.json, so only
an override reaches this path.
"""
from headroom import binaries
monkeypatch.delenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", raising=False)
monkeypatch.setattr(installer, "CBM_BIN_DIR", tmp_path)
monkeypatch.setattr(installer, "_detect_platform", lambda: "linux-amd64")
monkeypatch.setattr(
installer, "urlopen", lambda url, timeout=60: FakeResponse(_build_archive())
)
with pytest.raises(binaries.BinaryError) as exc:
installer.download_cbm(version="v9.9.9")
assert type(exc.value) is binaries.UnpinnedDownload
assert not (tmp_path / installer.CBM_BIN_NAME).exists()
def test_ensure_cbm_uses_existing_or_returns_none_on_failure(monkeypatch, tmp_path: Path) -> None:
existing = tmp_path / installer.CBM_BIN_NAME
monkeypatch.setattr(installer, "get_cbm_path", lambda: existing)