mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(binaries): fail closed when a download carries no sha256 pin (#3724)
## A-7 (S2, supply chain) — binary verification failed open on an
unpinned URL
`headroom/binaries.py` skipped SHA-256 verification entirely, with only
a log line, when the registry carried **no pin** for a download URL — in
both `_verify_sha256()` (the `resolve()` fetch path) and
`verify_download_bytes()` (the in-memory installer path). An
off-registry version override, or an installer building a filename the
registry does not describe, therefore installed whatever the mirror
served, protected only by HTTPS transport trust.
`verify_download_bytes`'s own docstring already claimed it *"refuses an
unpinned URL"*. It did not — the documentation described behaviour the
code did not have.
**Both claims in the backlog verified against `origin/main` (5ff4ea1e);
line numbers had not drifted from the 0.37.0 review.**
### What changed
- New `UnpinnedDownload(BinaryError)`. Both verification paths now
**raise** on a missing pin instead of falling through — the partial
download is deleted first, so nothing unverified is left in the cache
for a later call to pick up.
- `HEADROOM_BINARIES_ALLOW_UNVERIFIED` is kept as the escape hatch but
is now **loud**: a shared `_allow_unverified()` helper logs a warning
*and* prints to stderr. The logger is routinely unconfigured at proxy
startup, which is exactly when this fires, so the log-only warning was
invisible in practice.
- The env var is now documented in the module docstring (it was missing
from the `Env vars:` block entirely) with an explicit "never set this in
CI or production; pin the asset instead".
- `verify_download_bytes`'s docstring corrected to describe what the
code does, and its stale caller list trimmed (see below).
- `ensure_tools()` catches `UnpinnedDownload` alongside the other fetch
failures, so proxy startup still degrades to "tool missing" rather than
crashing.
### ⚠️ This changes install-time behaviour
A download with no `tools.json` pin now **fails** instead of installing.
No shipped path is affected (see the caller trace), but anyone relying
on the old silent fall-through — a local off-registry version override,
a fork with a hand-edited registry — will now get a hard error. The fix
is to pin the asset (`scripts/refresh_tool_hashes.py`), not to set the
escape hatch.
### Callers traced
`_verify_sha256` has exactly one caller and `verify_download_bytes`
exactly one; everything else reaches them transitively.
| Path | Pinned today? | Breaks? |
|---|---|---|
| `resolve()` → `_verify_sha256` (binaries.py) | Yes — pin comes from
`tools.json`, **all 21 assets across difft/scc/codebase-memory-mcp are
pinned** | No |
| `ensure_tools()` → `resolve()` (proxy startup, `cli/proxy.py:1216`) |
Yes | No — and a refusal is caught, so startup still succeeds with the
tool skipped |
| `cli/tools.py:43,230` → `resolve()` (`headroom tools install` /
`doctor`) | Yes | No |
| `proxy/interceptors/astgrep.py:122` → `resolve("ast-grep")` | n/a —
ast-grep is PyPI-only, never downloaded here | No |
| `graph/installer.py:82` `download_cbm()` → `verify_download_bytes` |
Yes for the default `CBM_VERSION = v0.8.1` on linux-amd64/arm64 and
darwin-amd64/arm64 | No |
| …same, with a `version=` override | **No** — off-registry URL | **Yes,
by design.** No production caller passes `version`; `ensure_cbm()` calls
`download_cbm()` with no args. Only tests did. |
| …same, on **Windows** | **No** — see "things the brief did not know"
below | Already broken today; now fails earlier with a clearer message |
| ONNX / embedding models (`headroom/onnx_runtime.py`) | n/a —
`hf_hub_download` at a pinned HF revision, never touches this module |
No |
| Rust / SmartCrusher | n/a — built by cargo (`ort-download-binaries` is
a cargo feature), not fetched through `binaries.py` | No |
| rtk / lean-ctx | n/a — those download paths were **removed** (see
`context_tool_cleanup.py`) | No |
The registry really does pin everything the product ships:
`.github/workflows/tools-hash-refresh.yml` runs
`scripts/refresh_tool_hashes.py --check`, which exits 1 if any pin is
*missing or stale*, on every `tools.json` PR and weekly. A new unit test
(`test_every_shipped_asset_is_pinned`) asserts the same thing so an
unpinned entry fails fast in unit tests too — that gate is what makes
failing closed safe.
### Fail-before / pass-after
Method: commit the work, `git checkout origin/main --
headroom/binaries.py` (keeping the new tests), run, restore with `git
checkout HEAD -- headroom/binaries.py`.
**Before** (new tests against `origin/main`'s `binaries.py`):
```
tests/test_binaries.py ...................FFFF....... [ 63%]
tests/test_graph.py ..........F...... [100%]
______________ test_unpinned_asset_is_refused_and_partial_deleted ______________
E Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>
_________ test_unpinned_asset_allowed_by_escape_hatch_warns_on_stderr __________
E AssertionError: assert 'WITHOUT sha256 verification' in ''
_______________ test_verify_download_bytes_refuses_unpinned_url ________________
E Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>
___________ test_verify_download_bytes_escape_hatch_warns_on_stderr ____________
E AssertionError: assert 'WITHOUT sha256 verification' in ''
______________ test_download_cbm_refuses_an_off_registry_version _______________
E Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>
========================= 5 failed, 42 passed in 5.21s =========================
```
The three `DID NOT RAISE` failures are the defect itself: on `main`, an
unpinned download is accepted and installed.
**After**:
```
tests/test_binaries.py .............................. [ 63%]
tests/test_graph.py ................. [100%]
============================== 47 passed in 3.74s ==============================
```
Wider sweep (every test module that touches `binaries`) — `test_binaries
test_graph test_cli_tools test_release_workflows
test_context_tool_cleanup test_memory/test_traffic_learner`: **294
passed**.
Lint: `uvx ruff@0.16.7 check .` → clean; `format --check` → clean.
### Existing tests that had to be adjusted
`tests/test_binaries.py`'s autouse fixture nulls every shipped pin so
mechanics tests can serve tiny mock archives. That made *every* such
fetch an unpinned one, so four fetch/extract-mechanics tests now opt
into the escape hatch via a new `allow_unverified` fixture, and the
autouse fixture `delenv`s the variable so a developer's exported value
cannot silently mask the guard. Two `download_cbm` tests in
`tests/test_graph.py` needed the same, for the same reason — they drive
off-registry URLs on purpose.
### Things the brief got wrong / found along the way (not fixed here)
1. **`verify_download_bytes`'s caller list was stale.** It named rtk and
lean-ctx; both downloaders were removed
(`headroom/context_tool_cleanup.py` exists precisely to clean up after
them). Only codebase-memory-mcp remains. Docstring corrected.
2. **Pre-existing, unrelated bug: `download_cbm` is broken on Windows.**
`graph/installer.py` builds `codebase-memory-mcp-windows-amd64.tar.gz`,
but the registry pins `...-windows-amd64.zip`, and the extractor calls
`tarfile.open(mode="r:gz")` unconditionally. So the Windows path already
fails (404, or a tar error on a zip) before this change. This PR makes
it fail one step earlier, at verification, with a message that names the
cause. Worth a separate issue — I did not widen scope to fix it.
3. **`ensure_cbm()` catches only `RuntimeError`**, so a `Sha256Mismatch`
— a genuine tamper signal — already escapes it today, and
`UnpinnedDownload` would too. Since no production caller passes a
`version` override, the shipped path cannot reach it, so I left it alone
rather than converting a loud integrity failure into a silent `None`.
Flagging it as a judgement call.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
34da21e89e
commit
26a2c4939c
+84
-23
@@ -9,10 +9,22 @@ Supported platforms: linux (glibc + musl) x86_64/aarch64, macOS x86_64/arm64,
|
||||
Windows x86_64. Unsupported platforms raise PlatformNotSupported; callers in
|
||||
the compression pipeline should fall back to their non-accelerated path.
|
||||
|
||||
Every asset this package ships is pinned by SHA-256 in `tools.json`, and the
|
||||
tools-hash-refresh CI gate fails the build if a pin is missing or stale. So
|
||||
verification fails closed: a download whose URL carries no pin is refused
|
||||
rather than trusted on transport (HTTPS) alone.
|
||||
|
||||
Env vars:
|
||||
HEADROOM_BINARIES_MIRROR base URL that replaces https://github.com
|
||||
HEADROOM_BINARIES_CACHE override cache dir
|
||||
HEADROOM_BINARIES_OFFLINE if set, never reach the network
|
||||
HEADROOM_BINARIES_ALLOW_UNVERIFIED
|
||||
Escape hatch: if set, skip SHA-256 verification entirely and accept
|
||||
whatever the mirror served. This disables Headroom's only defence
|
||||
against a tampered or substituted binary, so it is announced on
|
||||
stderr every time it takes effect. Intended for developers testing a
|
||||
locally built asset or an off-registry version; never set it in CI or
|
||||
in production. Pin the asset in `tools.json` instead.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -47,6 +59,7 @@ __all__ = [
|
||||
"BinaryFetchError",
|
||||
"PlatformNotSupported",
|
||||
"Sha256Mismatch",
|
||||
"UnpinnedDownload",
|
||||
"OfflineError",
|
||||
"PlatformKey",
|
||||
"detect_platform",
|
||||
@@ -77,6 +90,18 @@ class Sha256Mismatch(BinaryError):
|
||||
"""Raised when a downloaded asset's SHA256 does not match the pin."""
|
||||
|
||||
|
||||
class UnpinnedDownload(BinaryError):
|
||||
"""Raised when a downloaded asset has no SHA256 pin to verify it against.
|
||||
|
||||
Every shipped asset is pinned (tools-hash-refresh CI gate), so a missing
|
||||
pin means an off-registry fetch — a version override, a hand-edited
|
||||
registry, or a URL an installer built that the registry does not describe.
|
||||
Accepting it would silently downgrade integrity to transport trust, so we
|
||||
refuse. Pin the asset in `tools.json`, or set
|
||||
HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 to accept the risk explicitly.
|
||||
"""
|
||||
|
||||
|
||||
class OfflineError(BinaryError):
|
||||
"""Raised when a network fetch is required but HEADROOM_BINARIES_OFFLINE is set."""
|
||||
|
||||
@@ -310,19 +335,47 @@ def _sha256_file(path: Path) -> str:
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def _allow_unverified(name: str) -> bool:
|
||||
"""True if the escape hatch is set — announced loudly each time it fires.
|
||||
|
||||
Skipping verification turns a supply-chain guarantee into transport trust,
|
||||
so it goes to stderr as well as the logger: the logger is frequently
|
||||
unconfigured at proxy startup, and a silent downgrade is the failure mode
|
||||
this whole module exists to avoid.
|
||||
|
||||
Exactly once, though. With no handlers configured, logging's ``lastResort``
|
||||
already writes WARNING to stderr, so printing unconditionally produced the
|
||||
same sentence twice. Print only when a handler exists -- that is precisely
|
||||
when ``lastResort`` will not fire, and when the log may be going somewhere
|
||||
an operator is not watching, such as a file.
|
||||
"""
|
||||
if not os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
|
||||
return False
|
||||
message = (
|
||||
f"headroom: WARNING: accepting {name} WITHOUT sha256 verification "
|
||||
"(HEADROOM_BINARIES_ALLOW_UNVERIFIED is set). A tampered or "
|
||||
"substituted binary will not be detected."
|
||||
)
|
||||
logger.warning("%s", message)
|
||||
if logger.hasHandlers():
|
||||
print(message, file=sys.stderr)
|
||||
return True
|
||||
|
||||
|
||||
def _verify_sha256(path: Path, expected: str | None) -> None:
|
||||
if os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
|
||||
logger.warning(
|
||||
"skipping sha256 verification for %s (HEADROOM_BINARIES_ALLOW_UNVERIFIED=1)",
|
||||
path.name,
|
||||
)
|
||||
if _allow_unverified(path.name):
|
||||
return
|
||||
if not expected:
|
||||
# No pin in the registry (e.g. an off-registry version override). All
|
||||
# shipped assets ARE pinned — enforced by the tools-hash-refresh CI gate —
|
||||
# so a missing pin means an off-registry fetch; fall back to HTTPS trust.
|
||||
logger.info("binary %s downloaded without sha256 pin (HTTPS trust only)", path.name)
|
||||
return
|
||||
# No pin in the registry. All shipped assets ARE pinned — enforced by
|
||||
# the tools-hash-refresh CI gate — so a missing pin means an
|
||||
# off-registry fetch. Fail closed: HTTPS proves only who served the
|
||||
# bytes, not that they are the bytes we pinned.
|
||||
path.unlink(missing_ok=True)
|
||||
raise UnpinnedDownload(
|
||||
f"refusing unpinned download {path.name}: no sha256 in the registry. "
|
||||
"Pin it in headroom/tools.json (scripts/refresh_tool_hashes.py), or set "
|
||||
"HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 to accept it unverified."
|
||||
)
|
||||
got = _sha256_file(path)
|
||||
if got.lower() != expected.lower():
|
||||
path.unlink(missing_ok=True)
|
||||
@@ -342,22 +395,26 @@ def sha256_for_url(url: str) -> str | None:
|
||||
def verify_download_bytes(data: bytes, *, url: str, name: str) -> None:
|
||||
"""Fail-closed integrity check for an in-memory downloaded archive.
|
||||
|
||||
Used by installers (rtk, lean-ctx, codebase-memory-mcp) that download and
|
||||
extract on their own instead of going through the fetch path above. Verifies
|
||||
the bytes against the tools.json pin for ``url`` and refuses an unpinned URL
|
||||
unless HEADROOM_BINARIES_ALLOW_UNVERIFIED=1.
|
||||
Used by installers (codebase-memory-mcp) that download and extract on their
|
||||
own instead of going through the fetch path above. Verifies the bytes
|
||||
against the tools.json pin for ``url``.
|
||||
|
||||
Raises UnpinnedDownload if ``url`` has no pin in the registry and
|
||||
Sha256Mismatch if the bytes do not match it. Setting
|
||||
HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 skips both checks.
|
||||
"""
|
||||
if os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
|
||||
logger.warning(
|
||||
"skipping sha256 verification for %s (HEADROOM_BINARIES_ALLOW_UNVERIFIED=1)", name
|
||||
)
|
||||
if _allow_unverified(name):
|
||||
return
|
||||
expected = sha256_for_url(url)
|
||||
if not expected:
|
||||
# Off-registry URL (e.g. a version override); shipped assets are all
|
||||
# pinned via the CI gate, so fall back to HTTPS trust here.
|
||||
logger.info("%s downloaded without sha256 pin (HTTPS trust only)", name)
|
||||
return
|
||||
# Off-registry URL — a version override, or an installer building a
|
||||
# filename the registry does not carry. Refuse rather than silently
|
||||
# downgrading to transport trust.
|
||||
raise UnpinnedDownload(
|
||||
f"refusing unpinned download {name}: {url} has no sha256 in the registry. "
|
||||
"Pin it in headroom/tools.json (scripts/refresh_tool_hashes.py), or set "
|
||||
"HEADROOM_BINARIES_ALLOW_UNVERIFIED=1 to accept it unverified."
|
||||
)
|
||||
got = hashlib.sha256(data).hexdigest()
|
||||
if got.lower() != expected.lower():
|
||||
raise Sha256Mismatch(f"sha256 mismatch for {name}: expected {expected}, got {got}")
|
||||
@@ -490,7 +547,8 @@ def resolve(tool: str) -> Path:
|
||||
|
||||
Raises PlatformNotSupported if the tool is unavailable on this platform,
|
||||
OfflineError if a fetch is required but HEADROOM_BINARIES_OFFLINE is set,
|
||||
Sha256Mismatch if verification fails, BinaryFetchError on other IO errors.
|
||||
Sha256Mismatch if verification fails, UnpinnedDownload if the asset carries
|
||||
no registry pin, BinaryFetchError on other IO errors.
|
||||
"""
|
||||
on_path = _path_lookup(tool)
|
||||
if on_path:
|
||||
@@ -562,6 +620,9 @@ def ensure_tools(quiet: bool = False) -> dict[str, Path | None]:
|
||||
OfflineError,
|
||||
BinaryFetchError,
|
||||
Sha256Mismatch,
|
||||
# An unpinned asset is refused, not installed. Startup still
|
||||
# degrades to "tool missing" rather than failing outright.
|
||||
UnpinnedDownload,
|
||||
# Catch readonly / sandboxed filesystems (e.g. containerized
|
||||
# home dirs) so proxy startup never fails because the cache dir
|
||||
# can't be created. The interceptor fall back to no-op.
|
||||
|
||||
+10
-2
@@ -52,7 +52,12 @@ def _exec_tool(tool: str, argv: Sequence[str]) -> None:
|
||||
err=True,
|
||||
)
|
||||
sys.exit(2)
|
||||
except (binaries.Sha256Mismatch, binaries.BinaryFetchError) as e:
|
||||
# BinaryError, not a list of subclasses: every failure here is a refusal to
|
||||
# run an unverified tool, and enumerating them meant UnpinnedDownload -- added
|
||||
# later -- escaped as a traceback instead of the `error: ...` + exit 2 every
|
||||
# sibling produces. PlatformNotSupported and OfflineError are caught above for
|
||||
# their specific hints; this is the catch-all for the rest.
|
||||
except binaries.BinaryError as e:
|
||||
click.secho(f"error: {e}", fg="red", err=True)
|
||||
sys.exit(2)
|
||||
|
||||
@@ -232,7 +237,10 @@ def tools_install_cmd(tools: tuple[str, ...], force: bool) -> None:
|
||||
except binaries.PlatformNotSupported as e:
|
||||
click.secho(f"{name}: {e}", fg="red")
|
||||
exit_code = 1
|
||||
except (binaries.BinaryFetchError, binaries.Sha256Mismatch, binaries.OfflineError) as e:
|
||||
# Base class for the same reason as _exec_tool: an escaping subclass here
|
||||
# also aborted the loop over the remaining tools and skipped the final
|
||||
# sys.exit(exit_code), so one unpinned asset silently stopped the install.
|
||||
except binaries.BinaryError as e:
|
||||
click.secho(f"{name}: {e}", fg="red")
|
||||
exit_code = 1
|
||||
sys.exit(exit_code)
|
||||
|
||||
@@ -126,8 +126,18 @@ def ensure_cbm() -> Path | None:
|
||||
if existing:
|
||||
return existing
|
||||
|
||||
# Local import to match this module's lazy-import style (see download_cbm).
|
||||
from headroom.binaries import UnpinnedDownload
|
||||
|
||||
try:
|
||||
return download_cbm()
|
||||
except RuntimeError as e:
|
||||
logger.warning("Failed to install codebase-memory-mcp: %s", e)
|
||||
return None
|
||||
except UnpinnedDownload as e:
|
||||
# Documented contract is "path, or None if the download failed", and a
|
||||
# refusal is a failure to install -- the feature is simply unavailable.
|
||||
# Deliberately NOT widened to BinaryError: Sha256Mismatch is a tamper
|
||||
# signal and must keep propagating rather than becoming a quiet None.
|
||||
logger.warning("Refusing to install codebase-memory-mcp: %s", e)
|
||||
return None
|
||||
|
||||
+22
-7
@@ -164,10 +164,17 @@ def _null_binary_pins():
|
||||
|
||||
Installer tests fetch small mock archives, whose digests can't match the
|
||||
real published pins. Nulling the pins lets those download/extract mechanics
|
||||
tests run (verification then falls back to HTTPS trust); the tests that
|
||||
specifically exercise verification set their own pin explicitly. Production
|
||||
keeps the real pins (this fixture is test-only) and the tools-hash-refresh
|
||||
CI gate guarantees they stay correct.
|
||||
tests run; the tests that specifically exercise verification set their own
|
||||
pin explicitly. Production keeps the real pins (this fixture is test-only)
|
||||
and the tools-hash-refresh CI gate guarantees they stay correct.
|
||||
|
||||
Nulling a pin used to mean "fall back to HTTPS trust". It now means
|
||||
"refuse", so the escape hatch has to be set alongside it or every test that
|
||||
reaches a real download fails closed -- which is what happened to
|
||||
test_bundled_tools_savings.py on a cold CI cache, while passing locally
|
||||
against an already-populated one. Setting both together keeps this fixture
|
||||
saying one thing: "verification is not what these tests are about."
|
||||
Verification tests delenv it in their own fixture.
|
||||
"""
|
||||
try:
|
||||
from headroom import binaries
|
||||
@@ -185,9 +192,17 @@ def _null_binary_pins():
|
||||
]
|
||||
for asset, _original in saved:
|
||||
asset["sha256"] = None
|
||||
yield
|
||||
for asset, original in saved:
|
||||
asset["sha256"] = original
|
||||
previous = os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED")
|
||||
os.environ["HEADROOM_BINARIES_ALLOW_UNVERIFIED"] = "1"
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
if previous is None:
|
||||
os.environ.pop("HEADROOM_BINARIES_ALLOW_UNVERIFIED", None)
|
||||
else:
|
||||
os.environ["HEADROOM_BINARIES_ALLOW_UNVERIFIED"] = previous
|
||||
for asset, original in saved:
|
||||
asset["sha256"] = original
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
|
||||
+112
-4
@@ -32,6 +32,10 @@ def _clear_caches(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("HEADROOM_BINARIES_CACHE", str(tmp_path / "cache"))
|
||||
monkeypatch.delenv("HEADROOM_BINARIES_MIRROR", raising=False)
|
||||
monkeypatch.delenv("HEADROOM_BINARIES_OFFLINE", raising=False)
|
||||
# Verification fails closed on a nulled (unpinned) asset, so mechanics
|
||||
# tests opt into the escape hatch via the `allow_unverified` fixture; a
|
||||
# developer's exported value must not leak in and mask the guard.
|
||||
monkeypatch.delenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", raising=False)
|
||||
yield
|
||||
binaries.detect_platform.cache_clear()
|
||||
binaries._registry.cache_clear()
|
||||
@@ -81,6 +85,17 @@ class _FakeResponse:
|
||||
return False
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def allow_unverified(monkeypatch):
|
||||
"""Opt out of SHA-256 verification for tests about fetch/extract mechanics.
|
||||
|
||||
The autouse fixture nulls the shipped pins so these tests can serve small
|
||||
mock archives; without the escape hatch every such fetch is refused as
|
||||
unpinned, which is the point of `test_unpinned_*` below.
|
||||
"""
|
||||
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_urlopen(monkeypatch):
|
||||
"""Install a fake urllib.request.urlopen that serves registered URLs."""
|
||||
@@ -203,7 +218,9 @@ def test_mirror_substitution(monkeypatch):
|
||||
assert binaries._mirror_url("https://example.com/x") == "https://example.com/x"
|
||||
|
||||
|
||||
def test_mirror_url_with_query_params_strips_them_from_download_filename(monkeypatch, fake_urlopen):
|
||||
def test_mirror_url_with_query_params_strips_them_from_download_filename(
|
||||
monkeypatch, fake_urlopen, allow_unverified
|
||||
):
|
||||
"""Mirror URLs with `?token=...` must not leak into the download filename.
|
||||
|
||||
Regression test for the case where `Path(url).name` gave
|
||||
@@ -231,7 +248,7 @@ def test_mirror_url_with_query_params_strips_them_from_download_filename(monkeyp
|
||||
asset["url"] = original_url
|
||||
|
||||
|
||||
def test_fetch_extract_and_cache_tar_gz(monkeypatch, fake_urlopen, tmp_path):
|
||||
def test_fetch_extract_and_cache_tar_gz(monkeypatch, fake_urlopen, tmp_path, allow_unverified):
|
||||
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
|
||||
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
|
||||
|
||||
@@ -249,7 +266,7 @@ def test_fetch_extract_and_cache_tar_gz(monkeypatch, fake_urlopen, tmp_path):
|
||||
assert path2 == path
|
||||
|
||||
|
||||
def test_fetch_extract_zip(monkeypatch, fake_urlopen):
|
||||
def test_fetch_extract_zip(monkeypatch, fake_urlopen, allow_unverified):
|
||||
_set_platform(monkeypatch, sys_plat="win32", machine="AMD64")
|
||||
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
|
||||
payload = b"MZfake"
|
||||
@@ -321,6 +338,97 @@ def test_sha256_match_passes(monkeypatch, fake_urlopen):
|
||||
asset["sha256"] = None
|
||||
|
||||
|
||||
# -------- Unpinned downloads fail closed (A-7) --------------------------- #
|
||||
|
||||
|
||||
def test_unpinned_asset_is_refused_and_partial_deleted(monkeypatch, fake_urlopen):
|
||||
"""An asset with no registry pin must raise, not fall back to HTTPS trust.
|
||||
|
||||
The autouse fixture has already nulled every shipped pin, which is exactly
|
||||
the shape of an off-registry version override.
|
||||
"""
|
||||
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
|
||||
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
|
||||
asset = binaries._registry()["tools"]["difft"]["assets"]["darwin-aarch64"]
|
||||
assert asset["sha256"] is None
|
||||
fake_urlopen[asset["url"]] = _make_tar_gz({"difft": b"untrusted"})
|
||||
|
||||
# Caught as BinaryError so the assertion is about behaviour, not the name
|
||||
# of the subclass; the type is pinned on the next line.
|
||||
with pytest.raises(binaries.BinaryError) as exc:
|
||||
binaries.resolve("difft")
|
||||
assert type(exc.value) is binaries.UnpinnedDownload
|
||||
assert "HEADROOM_BINARIES_ALLOW_UNVERIFIED" in str(exc.value)
|
||||
# Nothing unverified is left in the cache for a later call to pick up.
|
||||
# Version read from the registry, not hardcoded: a hardcoded one silently
|
||||
# starts asserting about a path that never existed after a version bump.
|
||||
version = binaries._registry()["tools"]["difft"]["version"]
|
||||
assert not binaries._cached_path("difft", version, binaries.detect_platform()).exists()
|
||||
|
||||
|
||||
def test_unpinned_asset_allowed_by_escape_hatch_warns_on_stderr(
|
||||
monkeypatch, fake_urlopen, capsys, allow_unverified
|
||||
):
|
||||
"""The escape hatch still installs, but says so where an operator sees it."""
|
||||
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
|
||||
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
|
||||
asset = binaries._registry()["tools"]["difft"]["assets"]["darwin-aarch64"]
|
||||
fake_urlopen[asset["url"]] = _make_tar_gz({"difft": b"untrusted"})
|
||||
|
||||
path = binaries.resolve("difft")
|
||||
assert path.read_bytes() == b"untrusted"
|
||||
err = capsys.readouterr().err
|
||||
assert "WITHOUT sha256 verification" in err
|
||||
assert "HEADROOM_BINARIES_ALLOW_UNVERIFIED" in err
|
||||
|
||||
|
||||
def test_verify_download_bytes_refuses_unpinned_url():
|
||||
with pytest.raises(binaries.BinaryError) as exc:
|
||||
binaries.verify_download_bytes(
|
||||
b"payload", url="https://example.test/off-registry.tar.gz", name="some-tool"
|
||||
)
|
||||
assert type(exc.value) is binaries.UnpinnedDownload
|
||||
assert "off-registry.tar.gz" in str(exc.value)
|
||||
|
||||
|
||||
def test_verify_download_bytes_escape_hatch_warns_on_stderr(capsys, allow_unverified):
|
||||
binaries.verify_download_bytes(
|
||||
b"payload", url="https://example.test/off-registry.tar.gz", name="some-tool"
|
||||
)
|
||||
assert "WITHOUT sha256 verification" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_verify_download_bytes_checks_a_pinned_url():
|
||||
payload = b"payload"
|
||||
asset = binaries._registry()["tools"]["difft"]["assets"]["darwin-aarch64"]
|
||||
asset["sha256"] = hashlib.sha256(payload).hexdigest()
|
||||
try:
|
||||
binaries.verify_download_bytes(payload, url=asset["url"], name="difft")
|
||||
with pytest.raises(binaries.Sha256Mismatch):
|
||||
binaries.verify_download_bytes(b"tampered", url=asset["url"], name="difft")
|
||||
finally:
|
||||
asset["sha256"] = None
|
||||
|
||||
|
||||
def test_every_shipped_asset_is_pinned():
|
||||
"""The fail-closed guard is only safe because the registry pins everything.
|
||||
|
||||
Enforced against the published assets by the tools-hash-refresh workflow;
|
||||
checked here so a new unpinned entry fails fast in unit tests too. Reads
|
||||
the registry from disk because the autouse fixture nulls the live pins.
|
||||
"""
|
||||
import json
|
||||
|
||||
registry = json.loads(binaries._REGISTRY_PATH.read_text(encoding="utf-8"))
|
||||
unpinned = [
|
||||
f"{tool}/{key}"
|
||||
for tool, entry in registry.get("tools", {}).items()
|
||||
for key, asset in entry.get("assets", {}).items()
|
||||
if not asset.get("sha256")
|
||||
]
|
||||
assert unpinned == []
|
||||
|
||||
|
||||
# -------- status() ------------------------------------------------------- #
|
||||
|
||||
|
||||
@@ -349,7 +457,7 @@ def test_ensure_tools_survives_readonly_cache_dir(monkeypatch, tmp_path):
|
||||
readonly_parent.chmod(0o700) # so pytest tmp cleanup succeeds
|
||||
|
||||
|
||||
def test_ensure_tools_partial_failure_proxy_still_starts(monkeypatch):
|
||||
def test_ensure_tools_partial_failure_proxy_still_starts(monkeypatch, allow_unverified):
|
||||
"""If one tool fails to fetch, others still install and ensure_tools returns."""
|
||||
_set_platform(monkeypatch, sys_plat="darwin", machine="arm64")
|
||||
monkeypatch.setattr(binaries.shutil, "which", lambda _name: None)
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
"""A refusal is only useful if the caller turns it into a message.
|
||||
|
||||
`UnpinnedDownload` was added as a new `BinaryError` subclass, but three callers
|
||||
caught a hand-written tuple of subclasses rather than the base, so the new one
|
||||
escaped as a traceback. These pin the contract at the boundaries a person or a
|
||||
calling feature actually sees.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
from headroom import binaries
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _refuse(monkeypatch: pytest.MonkeyPatch):
|
||||
"""Make `resolve` refuse the way an off-registry asset does."""
|
||||
|
||||
def _raise(tool: str):
|
||||
raise binaries.UnpinnedDownload(
|
||||
f"refusing unpinned download {tool}: no sha256 in the registry. "
|
||||
"Pin it, or set HEADROOM_BINARIES_ALLOW_UNVERIFIED=1."
|
||||
)
|
||||
|
||||
monkeypatch.setattr(binaries, "resolve", _raise)
|
||||
monkeypatch.delenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", raising=False)
|
||||
|
||||
|
||||
def test_tools_run_reports_a_refusal_instead_of_a_traceback(_refuse) -> None:
|
||||
"""`headroom tools <name>` must exit 2 with `error: ...`, like every sibling."""
|
||||
from headroom.cli.tools import diff_cmd
|
||||
|
||||
result = CliRunner().invoke(diff_cmd, ["--version"])
|
||||
|
||||
assert result.exit_code == 2, result.output
|
||||
assert not isinstance(result.exception, binaries.UnpinnedDownload), (
|
||||
"the refusal escaped the CLI as a traceback"
|
||||
)
|
||||
assert "refusing unpinned download" in result.output
|
||||
|
||||
|
||||
def test_tools_install_keeps_going_and_still_sets_an_exit_code(_refuse) -> None:
|
||||
"""A refusal on one tool must not abort the loop over the others.
|
||||
|
||||
The escape skipped the final `sys.exit(exit_code)` entirely, so the command
|
||||
reported success-by-omission for every tool after the first refusal.
|
||||
"""
|
||||
from headroom.cli.tools import tools_group
|
||||
|
||||
result = CliRunner().invoke(tools_group, ["install"])
|
||||
|
||||
assert not isinstance(result.exception, binaries.UnpinnedDownload), (
|
||||
"the refusal aborted `tools install`"
|
||||
)
|
||||
assert result.exit_code == 1, result.output
|
||||
|
||||
|
||||
def test_ensure_cbm_honours_its_documented_none_contract(monkeypatch) -> None:
|
||||
"""`ensure_cbm` promises "path, or None if the download failed"."""
|
||||
from headroom.graph import installer
|
||||
|
||||
monkeypatch.setattr(installer, "get_cbm_path", lambda: None)
|
||||
|
||||
def _raise():
|
||||
raise binaries.UnpinnedDownload("refusing unpinned download codebase-memory-mcp")
|
||||
|
||||
monkeypatch.setattr(installer, "download_cbm", _raise)
|
||||
|
||||
assert installer.ensure_cbm() is None
|
||||
|
||||
|
||||
def test_a_tamper_signal_still_propagates_from_ensure_cbm(monkeypatch) -> None:
|
||||
"""The None contract must not swallow a sha256 mismatch.
|
||||
|
||||
An unpinned asset is a misconfiguration; a mismatch is evidence of tampering
|
||||
and has to stay loud rather than becoming a quiet "feature unavailable".
|
||||
"""
|
||||
from headroom.graph import installer
|
||||
|
||||
monkeypatch.setattr(installer, "get_cbm_path", lambda: None)
|
||||
|
||||
def _raise():
|
||||
raise binaries.Sha256Mismatch("sha256 mismatch for codebase-memory-mcp")
|
||||
|
||||
monkeypatch.setattr(installer, "download_cbm", _raise)
|
||||
|
||||
with pytest.raises(binaries.Sha256Mismatch):
|
||||
installer.ensure_cbm()
|
||||
|
||||
|
||||
def test_the_escape_hatch_warning_is_not_printed_twice(monkeypatch, capsys) -> None:
|
||||
"""With no handlers configured, logging's lastResort already writes stderr.
|
||||
|
||||
Printing unconditionally as well produced the same sentence twice.
|
||||
"""
|
||||
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
|
||||
|
||||
logger = logging.getLogger("headroom.binaries")
|
||||
saved, logger.handlers = logger.handlers, []
|
||||
saved_propagate, logger.propagate = logger.propagate, False
|
||||
try:
|
||||
assert binaries._allow_unverified("difft") is True
|
||||
finally:
|
||||
logger.handlers, logger.propagate = saved, saved_propagate
|
||||
|
||||
stderr = capsys.readouterr().err
|
||||
assert stderr.count("WITHOUT sha256 verification") <= 1, (
|
||||
f"the warning was emitted more than once:\n{stderr}"
|
||||
)
|
||||
@@ -76,6 +76,9 @@ def test_get_cbm_path_prefers_path_then_install_dir(monkeypatch, tmp_path: Path)
|
||||
|
||||
|
||||
def test_download_cbm_success_and_verification_paths(monkeypatch, tmp_path: Path) -> None:
|
||||
# v1.2.3 is not in the registry and the mock archive would not match a pin
|
||||
# anyway; this test is about the install mechanics, not verification.
|
||||
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
|
||||
monkeypatch.setattr(installer, "CBM_BIN_DIR", tmp_path)
|
||||
monkeypatch.setattr(installer, "_detect_platform", lambda: "linux-amd64")
|
||||
monkeypatch.setattr(
|
||||
@@ -109,6 +112,9 @@ def test_download_cbm_success_and_verification_paths(monkeypatch, tmp_path: Path
|
||||
def test_download_cbm_invalid_url_download_failure_and_extract_errors(
|
||||
monkeypatch, tmp_path: Path
|
||||
) -> None:
|
||||
# These failure modes are reached with an overridden (off-registry) release
|
||||
# host, so verification would refuse them before the code under test runs.
|
||||
monkeypatch.setenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", "1")
|
||||
monkeypatch.setattr(installer, "CBM_BIN_DIR", tmp_path)
|
||||
monkeypatch.setattr(installer, "_detect_platform", lambda: "linux-amd64")
|
||||
|
||||
@@ -138,6 +144,27 @@ def test_download_cbm_invalid_url_download_failure_and_extract_errors(
|
||||
installer.download_cbm()
|
||||
|
||||
|
||||
def test_download_cbm_refuses_an_off_registry_version(monkeypatch, tmp_path: Path) -> None:
|
||||
"""A version override points at a URL with no pin — refuse it (A-7).
|
||||
|
||||
The default CBM_VERSION is pinned for every platform in tools.json, so only
|
||||
an override reaches this path.
|
||||
"""
|
||||
from headroom import binaries
|
||||
|
||||
monkeypatch.delenv("HEADROOM_BINARIES_ALLOW_UNVERIFIED", raising=False)
|
||||
monkeypatch.setattr(installer, "CBM_BIN_DIR", tmp_path)
|
||||
monkeypatch.setattr(installer, "_detect_platform", lambda: "linux-amd64")
|
||||
monkeypatch.setattr(
|
||||
installer, "urlopen", lambda url, timeout=60: FakeResponse(_build_archive())
|
||||
)
|
||||
|
||||
with pytest.raises(binaries.BinaryError) as exc:
|
||||
installer.download_cbm(version="v9.9.9")
|
||||
assert type(exc.value) is binaries.UnpinnedDownload
|
||||
assert not (tmp_path / installer.CBM_BIN_NAME).exists()
|
||||
|
||||
|
||||
def test_ensure_cbm_uses_existing_or_returns_none_on_failure(monkeypatch, tmp_path: Path) -> None:
|
||||
existing = tmp_path / installer.CBM_BIN_NAME
|
||||
monkeypatch.setattr(installer, "get_cbm_path", lambda: existing)
|
||||
|
||||
Reference in New Issue
Block a user