Commit Graph
10 Commits
Author SHA1 Message Date
7a0f15f563 fix(binaries): ignore tool binaries found via the current directory (#4838)
## Description

`headroom.binaries._path_lookup()` (used by `which()`, `resolve()` and
`ensure_tools()`) and `status()` returned `shutil.which(name)` before
looking at the SHA-256-pinned cache. On Windows `shutil.which` searches
the current directory before PATH (unless
`NoDefaultCurrentDirectoryInExePath` is set), and on any OS a relative
PATH entry (`.`, an empty entry, `bin`) resolves against the cwd. The
cwd is usually the project being worked on, so a file there named
`ast-grep`, `difft`, `scc`, ... was run in place of the real tool, and
the cache's pin never applied to it.

A PATH hit is now ignored when it comes from a relative PATH entry or
lives in the current directory; lookup then falls back to the
interpreter's Scripts/bin dir and the verified cache (or a pinned fetch
in `resolve()`). `status()` (`headroom tools doctor`) uses the same rule
so it no longer reports such a file as `on-path`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/binaries.py`: new `_which_outside_cwd()`; `_path_lookup()`
and `status()` use it. A warning names the ignored file.
- `tests/test_binaries.py`: real `shutil.which` with `PATH=.` and a tool
in the cwd for `which()`, `resolve()` and `status()`, plus a test that a
normal PATH hit outside the cwd is still used.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# new tests on origin/main (fix stashed)
$ pytest -q tests/test_binaries.py
FAILED tests/test_binaries.py::test_which_ignores_a_tool_in_the_current_directory
FAILED tests/test_binaries.py::test_resolve_ignores_a_tool_in_the_current_directory
FAILED tests/test_binaries.py::test_status_does_not_report_a_cwd_tool_as_on_path
========================= 3 failed, 45 passed in 3.19s =========================

# with the fix
$ pytest -q tests/test_binaries.py tests/test_binaries_refusal_reaches_callers.py tests/test_bundled_tools_savings.py tests/test_cli_tools.py tests/test_offline_egress_chokepoint.py tests/test_ssl_context.py
================== 286 passed, 4 skipped in 156.19s (0:02:36) ==================

$ ruff check / ruff format --check (0.17.3) on changed files
All checks passed!
2 files already formatted
$ mypy headroom/binaries.py --ignore-missing-imports
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS, Python 3.12 venv; empty `HEADROOM_BINARIES_CACHE`,
`HEADROOM_BINARIES_OFFLINE=1`.
- Exact command / steps: in a directory containing an executable
`difft`, run with `PATH=.`: `python -c 'from headroom import binaries as
b; print(b.which("difft")); print([(r["state"], r["path"]) for r in
b.status() if r["tool"]=="difft"])'`.
- Observed result:
  - main: `which: difft`, `status: [('on-path', './difft')]`
- this branch: `Ignoring difft found via the current directory
(.../proj/difft); using the interpreter's Scripts dir or the Headroom
cache instead`, `which: None`, `status: [('missing', None)]`
- Not tested: on a Windows host (the Windows case is the same code path:
`shutil.which` returns a relative `.\name` hit there).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: a tool binary that is only reachable
through the current directory or a relative PATH entry is no longer
used; Headroom uses its verified cached copy instead. Running Headroom
from inside a directory that is itself listed on PATH also falls back to
the cache for tools in that directory.
- Kill switch / disable path: put the tool in a directory listed on PATH
by absolute path.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

No docs change needed. Fix adapted from a proposal by HetCreep (this
version also ignores hits from relative PATH entries other than `.`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4838"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:45:32 +05:30
Rajat NagdaandJerrett Davis da6d9736cd fix(binaries): fail closed on HEADROOM_BINARIES_ALLOW_UNVERIFIED=0/false (#3218)
## Description

The sha256-verification bypass for fetched tool binaries is a
supply-chain
integrity control, but both call sites gated it on a bare presence
check:

```python
if os.environ.get("HEADROOM_BINARIES_ALLOW_UNVERIFIED"):
    ...skip sha256 verification...
```

Any non-empty value is truthy — including `0`, `false`, `no`, `off`. An
operator who set `HEADROOM_BINARIES_ALLOW_UNVERIFIED=0` (the natural way
to
say "keep verification on") therefore **silently disabled** sha256
verification of every fetched binary (`difft`, `scc`, and the in-memory
`rtk` / `lean-ctx` / `codebase-memory-mcp` installer path via
`verify_download_bytes`). The failure direction is backwards for a
security
control: it fails **open**.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Parse `HEADROOM_BINARIES_ALLOW_UNVERIFIED` as an affirmative flag
(`1`/`true`/`yes`/`on`) via a shared `_allow_unverified_binaries()`
helper,
  used by both `_verify_sha256` and `verify_download_bytes`.
- Every non-affirmative value — and an unset or empty variable — now
**enforces** verification (fail closed). Matches the truthy-set parsing
  already used across the codebase (`onnx_runtime`, `cold_prefix`,
  `ttl_observations`, `qdrant_env`).
- Added regression tests over the env-value matrix on both call sites.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom --ignore-missing-imports`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_binaries.py -q
35 passed, 1 warning in 1.15s

# New tests fail before the fix (0/false/no/off/whitespace + in-memory path):
$ git stash push -- headroom/binaries.py && \
  python -m pytest tests/test_binaries.py -q -k "allow_unverified or verify_download_bytes_zero"
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[0]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[false]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[no]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[off]
FAILED ...::test_allow_unverified_non_affirmative_still_verifies[  ]
FAILED ...::test_verify_download_bytes_zero_still_verifies
6 failed, 7 passed, 22 deselected

$ python -m ruff check headroom/binaries.py tests/test_binaries.py && mypy headroom --ignore-missing-imports
All checks passed!
Success: no issues found in 527 source files
```

## Real Behavior Proof

- Environment: macOS (Darwin arm64), Python 3.13. Operator sets
`HEADROOM_BINARIES_ALLOW_UNVERIFIED=0` (intending to keep verification
enabled). A tampered binary whose bytes do not match the registry-pinned
sha256 is passed to `binaries._verify_sha256`.
- Exact command / steps: write a file containing
`b"TAMPERED-BINARY-CONTENT"`, then call `binaries._verify_sha256(path,
<wrong pin>)` under `HEADROOM_BINARIES_ALLOW_UNVERIFIED=0`, once with
the fix stashed (before) and once applied (after).
- Observed result: BEFORE the fix the mismatched binary is accepted
(verification skipped); AFTER the fix it is rejected with
`Sha256Mismatch`.
  ```text
### HEADROOM_BINARIES_ALLOW_UNVERIFIED=0 (operator intent: keep
verifying)
  --- BEFORE fix ---
skipping sha256 verification for difft
(HEADROOM_BINARIES_ALLOW_UNVERIFIED=1)
  RESULT: verification SKIPPED — tampered binary ACCEPTED  (fail-open)
  --- AFTER fix ----
RESULT: Sha256Mismatch raised — tampered binary REJECTED (fail-closed)
  ```
- Not tested: a live end-to-end download of difft/scc from GitHub
releases (network-dependent); the verification function that gates every
such download is exercised directly above with a real byte payload.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — unconditional hardening of an
existing security control.
- Minimum rollout channel: N/A
- Stable/default behavior changed: only for non-affirmative env values
that previously bypassed verification; `=1`/`true`/`yes`/`on` still
bypass exactly as before, and the default (unset) still verifies.
- Kill switch / disable path: `HEADROOM_BINARIES_ALLOW_UNVERIFIED=1`
still bypasses verification for operators who genuinely need it.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [] I have made corresponding changes to the documentation (N/A —
behavior now matches the existing `=1` docstrings)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3218"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `8c981e8248ff7dca7fd36bc55805a096d3943473`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Clean mechanical current-main integration with upstream
c01cf07d94. Re-reviewed fail-closed
environment parsing and both file/in-memory verification callers
alongside current author/Devin discussion. Only explicit
affirmative1/true/yes/on values bypass verification, case-insensitive
and trimmed; unset,false,unknown retain checks, with bypass warnings
shared through main helper. All44 binary-verification tests pass,
including mismatched-pin rejection and bypass cases;2-file Ruff
lint/format and main-relative whitespace pass. Maintainer
documentation-only follow-up addresses Devin note: module
environment-variable documentation now lists affirmative values and
fail-closed fallback. No contributor verification logic rewritten. Tests
use local payloads and temporary files; no live external binary download
or fresh native build claimed. Formal approval awaits exact-head hosted
qualification; maintainers handle CI. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
2026-11-06 09:33:27 -06:00
22d0905a73 fix(cli): pass configured SSL context to stdlib urlopen calls (#4011)
## Description

Fixes #3976. On `uv tool` installs and other environments where certifi
is not
wired into the stdlib OpenSSL trust path, `headroom update` and
bundled-binary
downloads can fail behind corporate TLS inspection. These urllib calls
now use
the configured Headroom trust context, matching the proxy's TLS policy.

The lazy helper import still falls back to urllib's default when the
optional
trust module cannot be imported. Errors while constructing an explicitly
configured context now propagate instead of silently replacing the
requested
trust roots with stdlib defaults. The update checker retains its outer
error-handling behavior and returns no update on failure.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring

## Changes Made

- Pass `build_urlopen_context()` to the PyPI version check and
release-binary
  download paths.
- Catch only `ImportError` around the lazy helper import; let
trust-context
  construction errors reach the caller.
- Add valid and invalid replacement-bundle tests for both helpers and
verify
the update check still handles the error through its existing outer
handler.
- Update mocked `urlopen` functions to accept the `context` keyword used
by the
  production call path.

## Testing

- [x] Targeted unit tests pass
- [ ] Full test suite run
- [x] Ruff lint and format checks pass for changed files
- [x] New tests added for the requested behavior
- [ ] Type checking run
- [ ] Manual live-network testing performed

### Test Output

```text
python -m pytest --disable-plugin-autoload -p pytest_asyncio.plugin tests/test_ssl_context.py tests/test_binaries.py tests/test_update_check.py
95 passed in 5.10s

python -m ruff check headroom/binaries.py headroom/update_check.py tests/test_ssl_context.py tests/test_binaries.py
All checks passed!

python -m ruff format --check headroom/binaries.py headroom/update_check.py tests/test_ssl_context.py tests/test_binaries.py
4 files already formatted

git diff --check
passed
```

## Real Behavior Proof

**Original reproduction:** On macOS behind a corporate TLS-inspecting
network,
`headroom update` failed with `CERTIFICATE_VERIFY_FAILED` when the
inspection
root was present in the OS trust store but absent from Python's stdlib
OpenSSL
bundle. The same machine's proxy reached its upstream through the
configured
root. The original fix routed the direct urllib calls through
`build_urlopen_context()`.

**Review regression checks:** On Windows with Python 3.13.13, tests
using a
valid `REQUESTS_CA_BUNDLE` confirm both helpers return a verifying
`SSLContext`. With an existing file containing invalid PEM, both helpers
now
raise `ssl.SSLError`; the update-check caller still returns `None`
through its
outer error handler. All 95 tests in the affected TLS, binary-download,
and
update-check suites pass.

**Not tested:** A live corporate TLS-inspection connection, Windows `uv
tool`
end-to-end behavior behind such a network, the full repository test
suite, and
the separate eval dataset download path.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Not applicable.
- Stable/default behavior changed: An invalid explicitly configured
trust
bundle now fails visibly rather than silently falling back to stdlib
roots;
  unconfigured default behavior is unchanged.
- Kill switch / disable path: No new switch; existing TLS environment
settings
  remain available.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this PR's changes.

## Review Readiness

- [x] I have performed a self-review.
- [x] This PR is ready for human review.

## Checklist

- [x] My code follows the project's style guidelines.
- [x] I have performed a self-review of my code.
- [ ] I have added comments where the code needs additional explanation.
- [ ] I have made corresponding documentation changes (none needed for
this
  internal error-handling change).
- [x] The changed files pass Ruff checks and the focused tests add no
warnings.
- [x] I have added tests that prove the requested behavior.
- [ ] The full repository unit-test suite was run.
- [x] I did not edit `CHANGELOG.md`.

---------

Co-authored-by: dajiaohuang <dajiaohuang@users.noreply.github.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-07 20:51:49 -05:00
Tejas ChopraandClaude Opus 5 26a2c4939c fix(binaries): fail closed when a download carries no sha256 pin (#3724)
## A-7 (S2, supply chain) — binary verification failed open on an
unpinned URL

`headroom/binaries.py` skipped SHA-256 verification entirely, with only
a log line, when the registry carried **no pin** for a download URL — in
both `_verify_sha256()` (the `resolve()` fetch path) and
`verify_download_bytes()` (the in-memory installer path). An
off-registry version override, or an installer building a filename the
registry does not describe, therefore installed whatever the mirror
served, protected only by HTTPS transport trust.

`verify_download_bytes`'s own docstring already claimed it *"refuses an
unpinned URL"*. It did not — the documentation described behaviour the
code did not have.

**Both claims in the backlog verified against `origin/main` (5ff4ea1e);
line numbers had not drifted from the 0.37.0 review.**

### What changed

- New `UnpinnedDownload(BinaryError)`. Both verification paths now
**raise** on a missing pin instead of falling through — the partial
download is deleted first, so nothing unverified is left in the cache
for a later call to pick up.
- `HEADROOM_BINARIES_ALLOW_UNVERIFIED` is kept as the escape hatch but
is now **loud**: a shared `_allow_unverified()` helper logs a warning
*and* prints to stderr. The logger is routinely unconfigured at proxy
startup, which is exactly when this fires, so the log-only warning was
invisible in practice.
- The env var is now documented in the module docstring (it was missing
from the `Env vars:` block entirely) with an explicit "never set this in
CI or production; pin the asset instead".
- `verify_download_bytes`'s docstring corrected to describe what the
code does, and its stale caller list trimmed (see below).
- `ensure_tools()` catches `UnpinnedDownload` alongside the other fetch
failures, so proxy startup still degrades to "tool missing" rather than
crashing.

### ⚠️ This changes install-time behaviour

A download with no `tools.json` pin now **fails** instead of installing.
No shipped path is affected (see the caller trace), but anyone relying
on the old silent fall-through — a local off-registry version override,
a fork with a hand-edited registry — will now get a hard error. The fix
is to pin the asset (`scripts/refresh_tool_hashes.py`), not to set the
escape hatch.

### Callers traced

`_verify_sha256` has exactly one caller and `verify_download_bytes`
exactly one; everything else reaches them transitively.

| Path | Pinned today? | Breaks? |
|---|---|---|
| `resolve()` → `_verify_sha256` (binaries.py) | Yes — pin comes from
`tools.json`, **all 21 assets across difft/scc/codebase-memory-mcp are
pinned** | No |
| `ensure_tools()` → `resolve()` (proxy startup, `cli/proxy.py:1216`) |
Yes | No — and a refusal is caught, so startup still succeeds with the
tool skipped |
| `cli/tools.py:43,230` → `resolve()` (`headroom tools install` /
`doctor`) | Yes | No |
| `proxy/interceptors/astgrep.py:122` → `resolve("ast-grep")` | n/a —
ast-grep is PyPI-only, never downloaded here | No |
| `graph/installer.py:82` `download_cbm()` → `verify_download_bytes` |
Yes for the default `CBM_VERSION = v0.8.1` on linux-amd64/arm64 and
darwin-amd64/arm64 | No |
| …same, with a `version=` override | **No** — off-registry URL | **Yes,
by design.** No production caller passes `version`; `ensure_cbm()` calls
`download_cbm()` with no args. Only tests did. |
| …same, on **Windows** | **No** — see "things the brief did not know"
below | Already broken today; now fails earlier with a clearer message |
| ONNX / embedding models (`headroom/onnx_runtime.py`) | n/a —
`hf_hub_download` at a pinned HF revision, never touches this module |
No |
| Rust / SmartCrusher | n/a — built by cargo (`ort-download-binaries` is
a cargo feature), not fetched through `binaries.py` | No |
| rtk / lean-ctx | n/a — those download paths were **removed** (see
`context_tool_cleanup.py`) | No |

The registry really does pin everything the product ships:
`.github/workflows/tools-hash-refresh.yml` runs
`scripts/refresh_tool_hashes.py --check`, which exits 1 if any pin is
*missing or stale*, on every `tools.json` PR and weekly. A new unit test
(`test_every_shipped_asset_is_pinned`) asserts the same thing so an
unpinned entry fails fast in unit tests too — that gate is what makes
failing closed safe.

### Fail-before / pass-after

Method: commit the work, `git checkout origin/main --
headroom/binaries.py` (keeping the new tests), run, restore with `git
checkout HEAD -- headroom/binaries.py`.

**Before** (new tests against `origin/main`'s `binaries.py`):

```
tests/test_binaries.py ...................FFFF.......                    [ 63%]
tests/test_graph.py ..........F......                                    [100%]

______________ test_unpinned_asset_is_refused_and_partial_deleted ______________
E   Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>
_________ test_unpinned_asset_allowed_by_escape_hatch_warns_on_stderr __________
E   AssertionError: assert 'WITHOUT sha256 verification' in ''
_______________ test_verify_download_bytes_refuses_unpinned_url ________________
E   Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>
___________ test_verify_download_bytes_escape_hatch_warns_on_stderr ____________
E   AssertionError: assert 'WITHOUT sha256 verification' in ''
______________ test_download_cbm_refuses_an_off_registry_version _______________
E   Failed: DID NOT RAISE <class 'headroom.binaries.BinaryError'>

========================= 5 failed, 42 passed in 5.21s =========================
```

The three `DID NOT RAISE` failures are the defect itself: on `main`, an
unpinned download is accepted and installed.

**After**:

```
tests/test_binaries.py ..............................                    [ 63%]
tests/test_graph.py .................                                    [100%]

============================== 47 passed in 3.74s ==============================
```

Wider sweep (every test module that touches `binaries`) — `test_binaries
test_graph test_cli_tools test_release_workflows
test_context_tool_cleanup test_memory/test_traffic_learner`: **294
passed**.

Lint: `uvx ruff@0.16.7 check .` → clean; `format --check` → clean.

### Existing tests that had to be adjusted

`tests/test_binaries.py`'s autouse fixture nulls every shipped pin so
mechanics tests can serve tiny mock archives. That made *every* such
fetch an unpinned one, so four fetch/extract-mechanics tests now opt
into the escape hatch via a new `allow_unverified` fixture, and the
autouse fixture `delenv`s the variable so a developer's exported value
cannot silently mask the guard. Two `download_cbm` tests in
`tests/test_graph.py` needed the same, for the same reason — they drive
off-registry URLs on purpose.

### Things the brief got wrong / found along the way (not fixed here)

1. **`verify_download_bytes`'s caller list was stale.** It named rtk and
lean-ctx; both downloaders were removed
(`headroom/context_tool_cleanup.py` exists precisely to clean up after
them). Only codebase-memory-mcp remains. Docstring corrected.
2. **Pre-existing, unrelated bug: `download_cbm` is broken on Windows.**
`graph/installer.py` builds `codebase-memory-mcp-windows-amd64.tar.gz`,
but the registry pins `...-windows-amd64.zip`, and the extractor calls
`tarfile.open(mode="r:gz")` unconditionally. So the Windows path already
fails (404, or a tar error on a zip) before this change. This PR makes
it fail one step earlier, at verification, with a message that names the
cause. Worth a separate issue — I did not widen scope to fix it.
3. **`ensure_cbm()` catches only `RuntimeError`**, so a `Sha256Mismatch`
— a genuine tamper signal — already escapes it today, and
`UnpinnedDownload` would too. Since no production caller passes a
`version` override, the shipped path cannot reach it, so I left it alone
rather than converting a loud integrity failure into a silent `None`.
Flagging it as a judgement call.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 20:21:41 +02:00
M Barry d62a3c4ad0 fix(tools): key codebase-memory-mcp assets by platform, not upstream naming (#3432)
## Description

`_asset_for_platform()` resolves a tool's download with a plain
`assets.get(plat.key())`, and `PlatformKey.key()` normalizes arch to
`aarch64`/`x86_64`. The `codebase-memory-mcp` entry in `tools.json` was
keyed in the upstream project's own naming (`darwin-arm64`,
`linux-amd64`, …), so the lookup missed on **every** platform: the tool
ships binaries for four platforms and resolved on none, reporting
`unsupported-platform` everywhere.

`difft` and `scc` already transliterate upstream names into the
registry's scheme (including duplicating one asset across `-gnu`/`-musl`
with a `_comment`), so this fixes the data to match that convention
rather than adding an alias layer in the lookup.

This went unnoticed because `headroom/graph/installer.py` fetches the
same tool over its own code path, building the URL from upstream naming
directly — so the graph feature works and only the registry view
(`headroom tools doctor`) reports the tool as unavailable.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Refile the four existing `codebase-memory-mcp` assets under the keys
`PlatformKey.key()` actually emits. URLs and sha256 pins are unchanged,
and all four match the release's published `checksums.txt`.
- Add the assets the entry was missing: `linux-{x86_64,aarch64}-musl`
pointing at upstream's `-portable` builds, and `windows-x86_64`. The
entry carried `"_comment": "No Windows build published for v0.8.1."`,
but v0.8.1 publishes `codebase-memory-mcp-windows-amd64.zip`; that stale
comment is replaced.
- Add `test_every_registry_asset_key_is_reachable` — every asset key
must be one `PlatformKey.key()` can produce. This is the guard that
would have caught the bug.
- Add `test_asset_lookup_matches_declared_platforms` — a tool resolves
iff it declares that platform. It builds the `PlatformKey` from the
platform axes rather than by parsing the registry key, so a mis-keyed
asset cannot make it pass by round-tripping, and it forbids "fixing" a
future mis-key with a silent alias fallback instead of correcting the
table.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_binaries.py -q
24 passed, 1 warning in 1.84s

$ python -m pytest tests/test_binaries.py tests/test_graph.py tests/test_cli_tools.py \
                   tests/test_bundled_tools_savings.py -q
53 passed, 2 skipped, 1 warning in 2.07s

$ ruff check .
All checks passed!

$ ruff format --check tests/test_binaries.py
1 file already formatted

$ mypy headroom
Success: no issues found in 530 source files
```

Test fails before the fix, passes after — re-introducing the original
key in-process:

```text
$ python -c "import tests.test_binaries as t; from headroom import binaries
a = binaries._registry()['tools']['codebase-memory-mcp']['assets']
a['darwin-arm64'] = a.pop('darwin-aarch64')
t.test_every_registry_asset_key_is_reachable()"
AssertionError: codebase-memory-mcp: asset key 'darwin-arm64' is unreachable;
expected one of ['darwin-aarch64', 'darwin-x86_64', 'linux-aarch64-gnu',
'linux-aarch64-musl', 'linux-x86_64-gnu', 'linux-x86_64-musl', 'windows-x86_64']
```

## Real Behavior Proof

- Environment: macOS 26 (Darwin 27.0.0), Apple silicon →
`darwin-aarch64`; Python 3.13.13; branch = `upstream/main` + this
commit. "Before" numbers come from headroom-ai 0.37.0 installed from
PyPI in a separate site-packages, unpatched.

- Exact command / steps: ran `headroom tools doctor` and
`binaries.status()` against the unpatched installed 0.37.0, then
resolved all seven declared platforms against the patched tree; full
transcript below.

  Before (unpatched 0.37.0, imported from outside the checkout):

  ```text
$ python -c "import headroom.binaries as b; print([r for r in b.status()
if r['tool']=='codebase-memory-mcp'])"
  codebase-memory-mcp -> unsupported-platform

  $ headroom tools doctor
  │ codebase-memo… │ unsupported-pl… │ v0.8.1 │ darwin-aarch64 │ - │
  codebase-memory-mcp: no prebuilt binary for darwin-aarch64;
supported: ['darwin-amd64', 'darwin-arm64', 'linux-amd64',
'linux-arm64']
  ```

After (patched tree), resolving every platform the registry now
declares:

  ```text
$ python -c "from headroom.binaries import PlatformKey,
_asset_for_platform; ..."
  darwin-aarch64       -> codebase-memory-mcp-darwin-arm64.tar.gz
  darwin-x86_64        -> codebase-memory-mcp-darwin-amd64.tar.gz
  linux-x86_64-gnu     -> codebase-memory-mcp-linux-amd64.tar.gz
linux-x86_64-musl -> codebase-memory-mcp-linux-amd64-portable.tar.gz
  linux-aarch64-gnu    -> codebase-memory-mcp-linux-arm64.tar.gz
linux-aarch64-musl -> codebase-memory-mcp-linux-arm64-portable.tar.gz
  windows-x86_64       -> codebase-memory-mcp-windows-amd64.zip
  ```

- Observed result: the tool resolves on all seven declared platforms
instead of none, and no longer reports `unsupported-platform` on this
host. On my machine `status()` now reports `cached` — note that the
cached binary was already present from an earlier run, so `cached`
reflects the existing cache, not a download performed by this patch.

- Not tested: two gaps, detailed below.
- I did not download the newly added `-portable` or `windows-amd64`
archives locally; their pins were taken from the release's published
`checksums.txt`. CI's `tools-hash-refresh / verify-pins` has since
fetched and re-hashed every pin in this PR and passed, so the URLs and
digests are confirmed. What remains unverified is the `member` name
*inside* those three archives, which CI does not extract; they follow
the existing convention (`codebase-memory-mcp`,
`codebase-memory-mcp.exe`). A wrong `member` fails loudly and safely —
the pin is verified before extraction and `_extract_member_from_tar/zip`
raises `archive did not contain expected member`. Happy to drop those
hunks and land only the key fix if you would rather not take them.
- No musl or Windows host was available. Mapping the `-portable` build
to the `-musl` keys is inferred from upstream shipping it as a separate
variant; the release notes do not state its linking. This is recorded in
a `_comment` on those assets.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — registry data plus tests.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: yes, in the sense that a tool that
previously never resolved now resolves. No proxy behavior, compression
path, or default is affected.
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this commit; the registry returns to its prior
state.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- No linked issue: `CONTRIBUTING.md` routes a bug fix with a repro and a
test straight to a PR.
- No other consumer regresses: `headroom/graph/installer.py` fetches
this tool over its own path and resolves its pin by URL via
`sha256_for_url()`, and no pre-existing URL changes here, so that lookup
still resolves (checked on this host).
- Docs checkbox is N/A — the platform key scheme is not user-facing
documentation; the reasoning is recorded in a `_comment` on the registry
entry instead.
- The two added tests are registry-wide, not specific to this tool, so
any future asset added under an upstream naming scheme fails at test
time rather than silently on every machine.
2026-09-05 10:06:36 +05:30
Tejas Chopra 1f96dabc19 fix(security): address u9up assessment findings (WEB-01–07) (#2207)
Hardens client-selected upstreams, memory identity resolution, downloaded binary integrity, telemetry import, Docker defaults, Neo4j credentials, and archive extraction. Refreshes the branch against current main and preserves newer same-origin and loopback protections.
2026-08-20 09:02:44 -05:00
93f2d7a2da chore: release main (#2792)
🤖 I have created a release *beep* *boop*
---


<details><summary>0.35.0</summary>

##
[0.35.0](https://github.com/headroomlabs-ai/headroom/compare/v0.34.0...v0.35.0)
(2026-08-12)


### Features

* **beacon:** allowlist the routing summary key
([#2818](https://github.com/headroomlabs-ai/headroom/issues/2818))
([7940c05](https://github.com/headroomlabs-ai/headroom/commit/7940c05ebf4486c6b9d00984067ae33cedf4dddb))
* **beacon:** hourly R2 compaction, per-strategy savings, and a stack
that reports
([#2853](https://github.com/headroomlabs-ai/headroom/issues/2853))
([e0870ef](https://github.com/headroomlabs-ai/headroom/commit/e0870ef931e5ea6cc6cb52551f5d80cd9e3dc715))
* **cli,pricing:** add CLI extension seam and prompt-cache TTL pricing
([#2802](https://github.com/headroomlabs-ai/headroom/issues/2802))
([6ec3e34](https://github.com/headroomlabs-ai/headroom/commit/6ec3e3478abf058fe1460f91342bcdadf54a1ba8))


### Bug Fixes

* **anthropic:** strip first-party tool search on custom upstreams
([#2539](https://github.com/headroomlabs-ai/headroom/issues/2539))
([7f6950b](https://github.com/headroomlabs-ai/headroom/commit/7f6950be34e29304deae0fa5138b852491b092fe))
* **backends/anyllm:** convert Anthropic tools and tool_choice to OpenAI
shape
([0d6866b](https://github.com/headroomlabs-ai/headroom/commit/0d6866b91a3777475abd58cd8b63a10cd0621e7f))
* **backends/anyllm:** stream tool_use blocks and map finish_reason on
the streaming path
([e4904e2](https://github.com/headroomlabs-ai/headroom/commit/e4904e23a6ba6f5cff2488332946481172446922))
* **backends/litellm:** None-guard core token counts in OpenAI usage
block ([#2324](https://github.com/headroomlabs-ai/headroom/issues/2324))
([12f9f58](https://github.com/headroomlabs-ai/headroom/commit/12f9f58cb3dcfc67af1238424d404d8dd9bad1dd))
* **beacon:** report all-layers savings, not context-compression only
([#2796](https://github.com/headroomlabs-ai/headroom/issues/2796))
([e9a24f3](https://github.com/headroomlabs-ai/headroom/commit/e9a24f3ec1ffd278b0b3ca547a90942c40c99ec8))
* **beacon:** split session failures by status code
([#2815](https://github.com/headroomlabs-ai/headroom/issues/2815))
([2954e37](https://github.com/headroomlabs-ai/headroom/commit/2954e37048f8dcffe16e1c37b8f71afb0094a0a2))
* **cache:** bound compression cache bookkeeping
([0ae948c](https://github.com/headroomlabs-ai/headroom/commit/0ae948c1510735df39317bf0861f8a8750cdbf9d))
* **cache:** enforce Anthropic's 1h-before-5m cache_control ordering
before forwarding
([#2941](https://github.com/headroomlabs-ai/headroom/issues/2941))
([3752458](https://github.com/headroomlabs-ai/headroom/commit/3752458022f736c779f7b5a6c2d6d2ef0bc89f72))
* **cache:** mirror client cache_control positions instead of
single-marker consolidation
([def3d76](https://github.com/headroomlabs-ai/headroom/commit/def3d76e5ab4665e609b51bfba54dd6d25116925))
* **cache:** stabilize Anthropic block-growing lineages
([#2917](https://github.com/headroomlabs-ai/headroom/issues/2917))
([1a04c95](https://github.com/headroomlabs-ai/headroom/commit/1a04c957f53ef25ab1209166f425a7876913c4d3))
* **ccr:** avoid injecting tool on chat streaming
([d0c1f5b](https://github.com/headroomlabs-ai/headroom/commit/d0c1f5b8ad68c7a44ed3aaa0fe40e3a656950123))
* **ccr:** preserve exact SQLite TTL boundary
([#2669](https://github.com/headroomlabs-ai/headroom/issues/2669))
([d0a86d4](https://github.com/headroomlabs-ai/headroom/commit/d0a86d409fab377f9c642d1f3680b6ece7f97b8a))
* **ccr:** report embedded hashes from compress endpoint
([#717](https://github.com/headroomlabs-ai/headroom/issues/717))
([685ebe4](https://github.com/headroomlabs-ai/headroom/commit/685ebe457d727922ba4057515556a2d2aac0f616))
* **ccr:** resolve &lt;&lt;ccr:...&gt;&gt; markers inline when no
retrieve-tool path exists
([#2512](https://github.com/headroomlabs-ai/headroom/issues/2512))
([ce8ce83](https://github.com/headroomlabs-ai/headroom/commit/ce8ce8313f8cebf060392a62f9adaab18c0df386))
* **ccr:** tolerate null/malformed OpenAI data in response handling
([#2467](https://github.com/headroomlabs-ai/headroom/issues/2467))
([e583e08](https://github.com/headroomlabs-ai/headroom/commit/e583e082d8dee942229ac6211c742f9c9448a905))
* **ci:** publish latest from the root Docker manifest
([#2252](https://github.com/headroomlabs-ai/headroom/issues/2252))
([5568d73](https://github.com/headroomlabs-ai/headroom/commit/5568d738afb5e080d8df56e64500026996cbf025))
* **claude:** stop forcing tool search on Foundry
([#2477](https://github.com/headroomlabs-ai/headroom/issues/2477))
([7981396](https://github.com/headroomlabs-ai/headroom/commit/798139608c0fb5118eb3a7a183b8b2abe92341f1))
* **cli/update:** let install ownership win over bare /.dockerenv so
venv installs self-update
([#2830](https://github.com/headroomlabs-ai/headroom/issues/2830))
([7092b53](https://github.com/headroomlabs-ai/headroom/commit/7092b53c466bf5dbda8a1cda88403d1a4b16deb1))
* **codex:** route alpha search through the Codex backend
([#2538](https://github.com/headroomlabs-ai/headroom/issues/2538))
([a540eb2](https://github.com/headroomlabs-ai/headroom/commit/a540eb2c61b1a47e5ab8b07ea4a80fee780b6514))
* **content-router:** protect custom-tag blocks before mixed-content
section split
([d7bc1e2](https://github.com/headroomlabs-ai/headroom/commit/d7bc1e275f411788abffa2d007db14aa17fd31c5))
* **deps:** bump h2 to 4.4.1 for CVE-2026-71554
([#2839](https://github.com/headroomlabs-ai/headroom/issues/2839))
([564e0a8](https://github.com/headroomlabs-ai/headroom/commit/564e0a8d0fe440dff21a6c405c88e05698b3059f))
* **deps:** enforce audited transitive dependency floors
([#2791](https://github.com/headroomlabs-ai/headroom/issues/2791))
([64e2039](https://github.com/headroomlabs-ai/headroom/commit/64e203931b9810e5a010f063d26d154419016f86))
* **doctor:** flag `ollama launch claude` proxy bypass instead of
misdirecting
([#2566](https://github.com/headroomlabs-ai/headroom/issues/2566))
([7f24d69](https://github.com/headroomlabs-ai/headroom/commit/7f24d695eea00b9bb3265fbaa6629acf0c2ff181))
* emit SSE ping before message_start on Bedrock streaming path (issue
[#902](https://github.com/headroomlabs-ai/headroom/issues/902))
([#1080](https://github.com/headroomlabs-ai/headroom/issues/1080))
([4dab254](https://github.com/headroomlabs-ai/headroom/commit/4dab254d52914c39ffe13071848604e1771b1bd1))
* **gemini:** resolve native CCR retrieval calls
([#2253](https://github.com/headroomlabs-ai/headroom/issues/2253))
([2483f57](https://github.com/headroomlabs-ai/headroom/commit/2483f570025763cd9183a93749ea8cf38f1aeb85))
* **health:** label kompress as degraded/optional when not yet loaded
([#2865](https://github.com/headroomlabs-ai/headroom/issues/2865))
([8949371](https://github.com/headroomlabs-ai/headroom/commit/89493714d2cffdc1f81a8f417ea09891453d7009))
* **image:** decouple routing types from trained_router so importing the
compressor doesn't import torch
([#2513](https://github.com/headroomlabs-ai/headroom/issues/2513))
([#2537](https://github.com/headroomlabs-ai/headroom/issues/2537))
([d7cf981](https://github.com/headroomlabs-ai/headroom/commit/d7cf981093cf505192a3736dadd0254a120830a1))
* **install/windows:** register persistent-task from S4U hidden XML
([#2453](https://github.com/headroomlabs-ai/headroom/issues/2453))
([#2459](https://github.com/headroomlabs-ai/headroom/issues/2459))
([1edaeb8](https://github.com/headroomlabs-ai/headroom/commit/1edaeb8b76f6b872a6c810d404c944caf1a594b2))
* **install:** don't crash the PowerShell installer when $PROFILE is
unset ([#2469](https://github.com/headroomlabs-ai/headroom/issues/2469))
([fc5c4e2](https://github.com/headroomlabs-ai/headroom/commit/fc5c4e239ce32f2b90a6777772a01bdf49c66cb6))
* **install:** trust Docker bridge for dashboard metadata
([e044139](https://github.com/headroomlabs-ai/headroom/commit/e044139001680fd5198147bf373df6f00db32cc7))
* **install:** use --userns=keep-id under Podman so bind-mount writes
don't fail
([#2846](https://github.com/headroomlabs-ai/headroom/issues/2846))
([3488f8d](https://github.com/headroomlabs-ai/headroom/commit/3488f8d4b5fae4eab157e0c4031ccf712bcbcc0d))
* **learn/gemini:** stop double-counting session tokens
([#2230](https://github.com/headroomlabs-ai/headroom/issues/2230))
([29d8a5e](https://github.com/headroomlabs-ai/headroom/commit/29d8a5e563cf16dbd3a53a1571f4f352e61e1b33))
* **learn/grok:** detect a Windows absolute project path
([#2283](https://github.com/headroomlabs-ai/headroom/issues/2283))
([e240df2](https://github.com/headroomlabs-ai/headroom/commit/e240df2b698e601324b85956bd93cb304f6030ab))
* **learn:** stop classifying a successful exit code 0 as an error
([#2289](https://github.com/headroomlabs-ai/headroom/issues/2289))
([a24fe7d](https://github.com/headroomlabs-ai/headroom/commit/a24fe7dcbfe5ab30d0cef631c936e2245c12d123))
* **litellm:** add async_post_call_success_hook to HeadroomCallback
([#1322](https://github.com/headroomlabs-ai/headroom/issues/1322))
([3107994](https://github.com/headroomlabs-ai/headroom/commit/3107994aed5fd42e713d3c26f3f08121a62b980e))
* **litellm:** don't forward a caller key the target cannot accept
([#2883](https://github.com/headroomlabs-ai/headroom/issues/2883))
([2f2950a](https://github.com/headroomlabs-ai/headroom/commit/2f2950a626cebf851aac29255e7188fbb1639f5a))
* **memory:** bound the TrafficLearner pending-pattern accumulator
(memory leak)
([#2579](https://github.com/headroomlabs-ai/headroom/issues/2579))
([1f5feff](https://github.com/headroomlabs-ai/headroom/commit/1f5fefffd3e82c73bddd928cfd53334031e807bc))
* **memory:** close DirectMem0 resources
([6596182](https://github.com/headroomlabs-ai/headroom/commit/65961827cf5e90d7b4e7026feb89aac000a73ea3))
* **memory:** close MCP backend on shutdown
([4bd8ecd](https://github.com/headroomlabs-ai/headroom/commit/4bd8ecd1e31475365801791d35630f66f7393553))
* **memory:** don't crash inline memory extraction on a non-object
&lt;memory&gt; block
([#2470](https://github.com/headroomlabs-ai/headroom/issues/2470))
([e00c6ff](https://github.com/headroomlabs-ai/headroom/commit/e00c6ff81ce2003e04042b8f2d1bd6aa3c6e885c))
* **memory:** keep vector metadata in sync
([#2295](https://github.com/headroomlabs-ai/headroom/issues/2295))
([c471800](https://github.com/headroomlabs-ai/headroom/commit/c471800e8ee22986c308464b02a85da5575f34cc))
* **memory:** make explicit-project and user store keys
collision-resistant
([#2231](https://github.com/headroomlabs-ai/headroom/issues/2231))
([f840d5f](https://github.com/headroomlabs-ai/headroom/commit/f840d5f2fe938432e542c3f71f2218eeecd06b05))
* **memory:** skip &lt;system-reminder&gt; blocks when building the
retrieval query
([#2195](https://github.com/headroomlabs-ai/headroom/issues/2195))
([#2541](https://github.com/headroomlabs-ai/headroom/issues/2541))
([4e5a67a](https://github.com/headroomlabs-ai/headroom/commit/4e5a67a342be4be659b62c7863a9e72422605788))
* **memory:** sync FTS5 and vector indexes on CLI
delete/edit/prune/purge
([fd4628d](https://github.com/headroomlabs-ai/headroom/commit/fd4628d82156c65d4fa22df9513315790a6cd2fb))
* **oauth2:** make repository lint checks pass
([c85abf7](https://github.com/headroomlabs-ai/headroom/commit/c85abf7a87920012e01f0a677f6fbd98c4b08de0))
* **observability:** aggregate tool savings in OTEL
([#2936](https://github.com/headroomlabs-ai/headroom/issues/2936))
([941c25d](https://github.com/headroomlabs-ai/headroom/commit/941c25d31e6c6e0b436c307cbe212771ff76b45f))
* **onnx:** stop ONNX thread pools from spinning idle cores
([#2495](https://github.com/headroomlabs-ai/headroom/issues/2495))
([#2540](https://github.com/headroomlabs-ai/headroom/issues/2540))
([5c561bd](https://github.com/headroomlabs-ai/headroom/commit/5c561bd913ea60fad2c3c53f4b65e679e7d248d0))
* **openai:** skip Responses tool-search deferral for clients that
cannot execute it
([#2696](https://github.com/headroomlabs-ai/headroom/issues/2696))
([54ea28d](https://github.com/headroomlabs-ai/headroom/commit/54ea28d9839a0dcfa4dd0cf4210a4421f03beeff))
* **opencode:** ship the transport hook-shim so wheel installs route
Node child traffic
([702dbc5](https://github.com/headroomlabs-ai/headroom/commit/702dbc5902ff184a7c20178958a811beb9c78fa3))
* **providers/anthropic:** don't crash token estimation on null
tool_calls
([#2472](https://github.com/headroomlabs-ai/headroom/issues/2472))
([08466f3](https://github.com/headroomlabs-ai/headroom/commit/08466f3cae4dbb2647dc6f249fe42c4e840600c5))
* **providers/openai:** bound tiktoken vocab loads with the guarded
loader
([#2554](https://github.com/headroomlabs-ai/headroom/issues/2554))
([0805e8e](https://github.com/headroomlabs-ai/headroom/commit/0805e8e410543d75c7ddd3b83dde5eda3bc13144))
* **proxy/anthropic:** inject headroom_retrieve whenever a CCR marker is
present, not only for new markers
([#2848](https://github.com/headroomlabs-ai/headroom/issues/2848))
([3808f60](https://github.com/headroomlabs-ai/headroom/commit/3808f60ca61e84faf3ea8f8e003a6e6c8e9af4da))
* **proxy/anthropic:** None-guard usage token counts on the direct
buffered path
([#2434](https://github.com/headroomlabs-ai/headroom/issues/2434))
([2b5ee7c](https://github.com/headroomlabs-ai/headroom/commit/2b5ee7cde809ca37f6998d9679b1eb2133ab50ca))
* **proxy/anthropic:** run tool-search history repair after turn hooks
([c6f9948](https://github.com/headroomlabs-ai/headroom/commit/c6f99482e1bea024db6014a70c8e6da419543957))
* **proxy/batch:** don't crash an OpenAI batch on a valid-JSON
non-object line
([#2316](https://github.com/headroomlabs-ai/headroom/issues/2316))
([1f2c681](https://github.com/headroomlabs-ai/headroom/commit/1f2c681c0b48150a569277d3ebd5e95709dc7c39))
* **proxy/bedrock:** report uncached input tokens from backend usage,
not the live-zone count
([#2318](https://github.com/headroomlabs-ai/headroom/issues/2318))
([c19e412](https://github.com/headroomlabs-ai/headroom/commit/c19e412b3356d80dece001887d4ff48b6fd5150b))
* **proxy/gemini:** keep streaming-parity baseline so eligible_pct can't
exceed 100
([#2824](https://github.com/headroomlabs-ai/headroom/issues/2824))
([b97c7c6](https://github.com/headroomlabs-ai/headroom/commit/b97c7c6e99eac84df49c7a7e5f21dedb298716fe))
* **proxy/metrics:** cap client-supplied model label cardinality
([#2480](https://github.com/headroomlabs-ai/headroom/issues/2480))
([e24a7e6](https://github.com/headroomlabs-ai/headroom/commit/e24a7e66b95fa908c4ea6fd079809ece7692e6b2))
* **proxy/metrics:** escape label values in the Prometheus export
([#2463](https://github.com/headroomlabs-ai/headroom/issues/2463))
([6a53861](https://github.com/headroomlabs-ai/headroom/commit/6a53861063c3839e698bbec7194517bdfd851c38))
* **proxy/openai:** don't crash the Responses memory tool loops on null
arguments
([#2273](https://github.com/headroomlabs-ai/headroom/issues/2273))
([a30db2c](https://github.com/headroomlabs-ai/headroom/commit/a30db2cae49b4ef03ebbd404ec1fc6c4f5f2404d))
* **proxy/openai:** feed Codex WS traffic into the traffic learner
([#2334](https://github.com/headroomlabs-ai/headroom/issues/2334))
([f669149](https://github.com/headroomlabs-ai/headroom/commit/f6691497692869b7067438597421ff12aace6bf4))
* **proxy/openai:** run response hooks on Responses, and bill their
re-drives
([#2872](https://github.com/headroomlabs-ai/headroom/issues/2872))
([675d13f](https://github.com/headroomlabs-ai/headroom/commit/675d13f08d42455c8fa17bda878c1a11b905cee4))
* **proxy:** allow settings routes for trusted gateway/dashboard clients
([#2491](https://github.com/headroomlabs-ai/headroom/issues/2491))
([a5b0a8f](https://github.com/headroomlabs-ai/headroom/commit/a5b0a8f4cc54d68afcf371a422b3a4a9635b7e7f))
* **proxy:** cache litellm model resolution to stop repeated Provider
List spam
([99f07e7](https://github.com/headroomlabs-ai/headroom/commit/99f07e7bbdded9dadc70e35ee6ab025279d1aa22))
* **proxy:** cancel periodic TOIN task on shutdown
([739fdef](https://github.com/headroomlabs-ai/headroom/commit/739fdef423fa8cbc82537481c875d4570b0ecad4))
* **proxy:** close the upstream stream when a streaming body is never
consumed
([0951663](https://github.com/headroomlabs-ai/headroom/commit/09516635621caccf7e3db4f537eb49ea49b8a453))
* **proxy:** compress cache-mode cold starts and tag prefix-mismatch
passthrough
([#2365](https://github.com/headroomlabs-ai/headroom/issues/2365))
([aaeba0a](https://github.com/headroomlabs-ai/headroom/commit/aaeba0a319f12b98cad3bfcf1cf991b694b946bf))
* **proxy:** emit request log timestamps in UTC
([620028f](https://github.com/headroomlabs-ai/headroom/commit/620028fa18843622d3e454bd40fb91a93e607dbf))
* **proxy:** enable tool search by default and repair poisoned
transcripts
([#2807](https://github.com/headroomlabs-ai/headroom/issues/2807))
([0237cbf](https://github.com/headroomlabs-ai/headroom/commit/0237cbffbbc456ad8a7398005602d76881862d99))
* **proxy:** gate mid-turn message coalescing to Claude Code clients
([#1643](https://github.com/headroomlabs-ai/headroom/issues/1643))
([a4bd2e6](https://github.com/headroomlabs-ai/headroom/commit/a4bd2e62a5bb73f15b3b12e979c69e2b555bee10))
* **proxy:** give each Codex /v1/responses WS turn a unique request_id
([#2164](https://github.com/headroomlabs-ai/headroom/issues/2164))
([d02df10](https://github.com/headroomlabs-ai/headroom/commit/d02df1075894b414d60626aca2bbcadd7a3577a0))
* **proxy:** graceful shutdown and reliable Ctrl+C exit
([#621](https://github.com/headroomlabs-ai/headroom/issues/621))
([17cdb18](https://github.com/headroomlabs-ai/headroom/commit/17cdb185bc79d8cfec104e781a7e555af3ef11e1))
* **proxy:** guard telemetry and TOIN endpoints
([cde1513](https://github.com/headroomlabs-ai/headroom/commit/cde1513c91b6c6c240869bc5660f4b8966197bbc))
* **proxy:** include tool_search_deferral savings in the savings ledger
([12149f7](https://github.com/headroomlabs-ai/headroom/commit/12149f74466c08b69be8d5fe751425be63c2fda4))
* **proxy:** pass through cross-region prefixed Bedrock model IDs
directly
([#2330](https://github.com/headroomlabs-ai/headroom/issues/2330))
([64cb46e](https://github.com/headroomlabs-ai/headroom/commit/64cb46e24bf7b223ea71b14b6f5e86e78fa7ac45))
* **proxy:** port session-sticky beta headers to the Rust proxy
([#2381](https://github.com/headroomlabs-ai/headroom/issues/2381))
([f6398a6](https://github.com/headroomlabs-ai/headroom/commit/f6398a64768a095b722a5fb0b2445c7953dee1c6))
* **proxy:** preserve merged session and quarantine contracts
([#2943](https://github.com/headroomlabs-ai/headroom/issues/2943))
([039cd24](https://github.com/headroomlabs-ai/headroom/commit/039cd2431aaec7d59fefaf7e97aeda1fd7ab3afa))
* **proxy:** preserve signed Anthropic thinking blocks on outbound
re-serialize
([#2254](https://github.com/headroomlabs-ai/headroom/issues/2254))
([dc163bc](https://github.com/headroomlabs-ai/headroom/commit/dc163bcd1cba4cd8898f23286eb1365fcf6e0356))
* **proxy:** stop discarding compressed Codex WS later-frame payloads
([#2823](https://github.com/headroomlabs-ai/headroom/issues/2823))
([4ec416d](https://github.com/headroomlabs-ai/headroom/commit/4ec416df8899036544e679f561f1cf921f3da0dd))
* **proxy:** time-cap the compression timeout-debt quarantine
([#2360](https://github.com/headroomlabs-ai/headroom/issues/2360))
([#2412](https://github.com/headroomlabs-ai/headroom/issues/2412))
([c5a08d2](https://github.com/headroomlabs-ai/headroom/commit/c5a08d22e05a7dd2b929f3cca76ee3fb42f122db))
* **proxy:** unwrap Hermes tool_call bridge in tool name map
([#2717](https://github.com/headroomlabs-ai/headroom/issues/2717))
([a97b824](https://github.com/headroomlabs-ai/headroom/commit/a97b82413bdc86655c064417ed4628ff4d9d7c9d))
* publish headroom-opencode in release workflow
([#2372](https://github.com/headroomlabs-ai/headroom/issues/2372))
([7859154](https://github.com/headroomlabs-ai/headroom/commit/78591545ceb8303fdf9b93cd5ff02b626df97d2b))
* **settings:** accept documented HEADROOM_* env names as settings keys
([#2833](https://github.com/headroomlabs-ai/headroom/issues/2833))
([de9e052](https://github.com/headroomlabs-ai/headroom/commit/de9e0523dad47b700062464adecd60f82547f332))
* **subscription:** dedup transcript usage by message id
([#2340](https://github.com/headroomlabs-ai/headroom/issues/2340) token
inflation)
([#2408](https://github.com/headroomlabs-ai/headroom/issues/2408))
([74275b7](https://github.com/headroomlabs-ai/headroom/commit/74275b7c3e2b39be5198f9efa35057a5e026e665))
* **toin:** bound private query and pattern retention
([8cd1380](https://github.com/headroomlabs-ai/headroom/commit/8cd138039edbfc295080ec474325d527fb3aedf3))
* **tokenizer:** coerce non-string tool_call fields before counting
([#2801](https://github.com/headroomlabs-ai/headroom/issues/2801))
([b6f9877](https://github.com/headroomlabs-ai/headroom/commit/b6f9877c78b3fa3b1d705426bd27d74be77f4fa0))
* **tokenizer:** price CJK in the Rust fixed-ratio estimator (Python
parity)
([#2260](https://github.com/headroomlabs-ai/headroom/issues/2260))
([6840153](https://github.com/headroomlabs-ai/headroom/commit/6840153473caa0d61e982215e16a8cf54b0b6cc7))
* **transforms/adaptive-sizer:** honor max_k on small-input fast path
([#2319](https://github.com/headroomlabs-ai/headroom/issues/2319))
([8a90523](https://github.com/headroomlabs-ai/headroom/commit/8a905232091d993fac9e19a59bc449f201d4cdf3))
* **transforms/smart_crusher:** don't crash on a tool call with a null
function
([#2232](https://github.com/headroomlabs-ai/headroom/issues/2232))
([3bb02f8](https://github.com/headroomlabs-ai/headroom/commit/3bb02f8f75f12cf8258a5b1c2a7fbdc190f9d074))
* Vertex model pricing shows $0.00 for versioned model names and
vertex:anthropic provider
([#2517](https://github.com/headroomlabs-ai/headroom/issues/2517))
([eb5b5e4](https://github.com/headroomlabs-ai/headroom/commit/eb5b5e41988f5c27d29ae8ae3e5fe74e56493b8c))
* **wrap/claude:** keep --1m effective when an explicit --model is
passed through
([c093bf1](https://github.com/headroomlabs-ai/headroom/commit/c093bf11eb5f356f71367ebb7b56ae3c2b434a12))
* **wrap/opencode:** verify the opencode binary before mutating config
([ae38486](https://github.com/headroomlabs-ai/headroom/commit/ae384862a4950cec057103e9daf75e74107640df))
* **wrap/serena:** install Serena from the serena-agent PyPI wheel, not
the git source
([d7b25ae](https://github.com/headroomlabs-ai/headroom/commit/d7b25ae3bb3364cde4931509ecb65e32085e5b09))
* **wrap:** honor Copilot OAuth wire-api override and model default
([#2387](https://github.com/headroomlabs-ai/headroom/issues/2387))
([1db6d88](https://github.com/headroomlabs-ai/headroom/commit/1db6d88ab4ea25654b8277358902b7df700db6b4))
* **wrap:** serialize shared proxy startup
([#2946](https://github.com/headroomlabs-ai/headroom/issues/2946))
([e540d64](https://github.com/headroomlabs-ai/headroom/commit/e540d64febf27f2e7997d3a1a1d89478cc1ef658))
* **wrap:** stop the launch cwd from shadowing the installed package in
the proxy subprocess
([#2843](https://github.com/headroomlabs-ai/headroom/issues/2843))
([c49be26](https://github.com/headroomlabs-ai/headroom/commit/c49be269a18446779cd8a048caaa7f0ba3a3b48b))


### Performance Improvements

* cut hot-path latency 27% (token-count memo, startup preloads, JSON
scan memo)
([#2838](https://github.com/headroomlabs-ai/headroom/issues/2838))
([53af90d](https://github.com/headroomlabs-ai/headroom/commit/53af90d68c723f644a5a41dd273a606117109866))
* **proxy:** bound upstream calls and hot-path costs
([#2852](https://github.com/headroomlabs-ai/headroom/issues/2852))
([f624d3a](https://github.com/headroomlabs-ai/headroom/commit/f624d3a00ac271db7947443ddeb0c8bc2e93d3eb))
* **subscription:** skip transcripts older than the window in
compute_window_tokens
([#2861](https://github.com/headroomlabs-ai/headroom/issues/2861))
([91d6bf3](https://github.com/headroomlabs-ai/headroom/commit/91d6bf33cde777b541375fb182d4479fdd78f81b))


### Dependencies

* bump brace-expansion from 5.0.7 to 5.0.9 in /docs
([#2751](https://github.com/headroomlabs-ai/headroom/issues/2751))
([56ee57b](https://github.com/headroomlabs-ai/headroom/commit/56ee57be98bf109f0a46de522724ef169a4bc51c))
* bump bytesize from 1.3.3 to 2.4.2
([#2286](https://github.com/headroomlabs-ai/headroom/issues/2286))
([6448545](https://github.com/headroomlabs-ai/headroom/commit/6448545a7f5a1dee88bce6f0830bdbfd1c99c617))
* bump hf-hub from 0.4.3 to 0.5.0
([#2285](https://github.com/headroomlabs-ai/headroom/issues/2285))
([4925bf6](https://github.com/headroomlabs-ai/headroom/commit/4925bf6a829735977bab5000b469c3edb19c75b1))
* bump next from 16.2.10 to 16.3.0 in /docs
([#2750](https://github.com/headroomlabs-ai/headroom/issues/2750))
([0fd0b99](https://github.com/headroomlabs-ai/headroom/commit/0fd0b996a4b58a166491b145f4d3885c21b27cc0))
* bump postcss from 8.5.19 to 8.5.25 in /plugins/openclaw
([#2749](https://github.com/headroomlabs-ai/headroom/issues/2749))
([cd60ee9](https://github.com/headroomlabs-ai/headroom/commit/cd60ee9ae886b32ba5da3203e35bb6b088031fd3))
* bump postcss from 8.5.19 to 8.5.25 in /plugins/opencode
([#2748](https://github.com/headroomlabs-ai/headroom/issues/2748))
([ff4e016](https://github.com/headroomlabs-ai/headroom/commit/ff4e0167bbccbd4ae51bf23ddec144e61c94cd68))
* bump postcss from 8.5.19 to 8.5.25 in /sdk/typescript
([#2747](https://github.com/headroomlabs-ai/headroom/issues/2747))
([267c2bd](https://github.com/headroomlabs-ai/headroom/commit/267c2bdcb56e132b2dd9c065dab3498dbf730ca3))
* bump postcss from 8.5.19 to 8.5.26 in /docs
([#2881](https://github.com/headroomlabs-ai/headroom/issues/2881))
([e6e5826](https://github.com/headroomlabs-ai/headroom/commit/e6e5826423a0a700a8c544ce2c8cbcdef694160e))
* bump ruff from 0.15.17 to 0.15.22 in the pip-minor-patch group
([#2501](https://github.com/headroomlabs-ai/headroom/issues/2501))
([ecf130d](https://github.com/headroomlabs-ai/headroom/commit/ecf130d3ac6fb864098cb93fafd2621ae3ac7e12))
* bump rusqlite from 0.32.1 to 0.40.1
([#2287](https://github.com/headroomlabs-ai/headroom/issues/2287))
([522faa1](https://github.com/headroomlabs-ai/headroom/commit/522faa1a59aa94e4adfd4a4afe0202d1126e187d))
* bump the cargo-minor-patch group across 1 directory with 22 updates
([#2916](https://github.com/headroomlabs-ai/headroom/issues/2916))
([148d860](https://github.com/headroomlabs-ai/headroom/commit/148d8605e2087f3c8d6a3fa4b8d248ad2da5858f))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-12 19:02:51 -05:00
chopratejas 78d935fcf9 fix: docker install fails with PermissionError in readonly cache dir
Two bugs collided to break the Docker-native install CI:

1. `ensure_tools()` ran unconditionally at every proxy startup, even when
   `--intercept-tool-results` was not passed. The feature is opt-in, so
   there's no reason to pay the binary-fetch cost (or risk a failure) when
   nothing will use them.

2. The fetch loop caught PlatformNotSupported / OfflineError /
   BinaryFetchError / Sha256Mismatch but not `PermissionError`. In
   containerized environments where the home dir / cache dir isn't
   writable, `binary_path.parent.mkdir()` raises PermissionError
   (subclass of OSError), which propagated out of ensure_tools() and
   crashed proxy startup.

Fixes:
- Move ensure_tools() inside the `if intercept_tool_results:` branch in
  cli/proxy.py so the base case never triggers a fetch.
- Catch OSError (covers PermissionError, ENOSPC, etc.) in ensure_tools()
  so sandboxed / readonly filesystems degrade to no-op instead of
  crashing. Interceptors fall back to pass-through when their tool isn't
  resolvable.

Adds regression test `test_ensure_tools_survives_readonly_cache_dir` that
points the cache at a chmod-0500 parent and asserts ensure_tools()
returns without raising.
2026-04-20 17:36:44 -07:00
chopratejas e165a579f8 fix(review): address PR #210 feedback — race, query-params, SHA logging, frozen prefix, etc
Addresses all 24 inline comments across the two review passes.

**CRITICAL fixes:**
- binaries.py: PID-scoped partial-file name prevents concurrent `headroom proxy`
  starts from clobbering each other's downloads.
- binaries.py: strip URL query params before computing the download filename
  (was breaking archive-type detection for mirror URLs with `?token=...`).
- cli/tools.py: `--force` cleanup now logs failures and bumps exit_code
  instead of silently swallowing exceptions.

**HIGH fixes:**
- binaries.py: log at INFO when SHA256 is unpinned; expose `sha_pinned` in
  doctor's status output.
- proxy/interceptors/base.py: `_FAILURES` counter + `interceptor_failure_counts()`
  getter; incremented on every `matches()`/`transform()`/`key()` exception so
  dashboards can distinguish "nothing eligible" from "everything crashing".
- cli/proxy.py: validate critical tools resolved when
  `--intercept-tool-results` is set; warn (don't fail) if a dependency is
  missing.
- proxy/interceptors/base.py: compute `tokens_before` from the original
  messages via `count_messages()` instead of back-calculating from
  `tokens_after + sum(saved)` (which double-counted message-level overhead).
- proxy/interceptors/astgrep.py: write untrusted tool_output into a private
  mode-0700 `tempfile.mkdtemp()` directory, not directly into shared `/tmp`.
- proxy/interceptors/base.py: `ToolResultInterceptorTransform.apply()` now
  honors `frozen_message_count` — leading cached-prefix messages are passed
  through untouched to preserve provider prefix caches.

**MEDIUM fixes:**
- proxy/interceptors/base.py: pre-built O(1) tool_use index replaces the
  O(n²) per-tool-result linear scan.
- proxy/interceptors/base.py: broken `progressive_disclosure_key()` now
  skips the interceptor entirely rather than firing without key protection.
- proxy/interceptors/astgrep.py: distinguish ast-grep rc=1 (no matches) from
  rc>=2 (real errors — bad syntax, missing grammar, corrupt binary).
- proxy/interceptors/astgrep.py: count JSON parse failures; warn when all
  lines fail to parse (indicates version mismatch).
- binaries.py: musl detection falls back to checking `/lib/ld-musl-*.so.1`
  when `ldd` is absent (Alpine).
- proxy/interceptors/astgrep.py: use `tempfile.mkdtemp()` + `shutil.rmtree`
  instead of `NamedTemporaryFile(delete=False)`; cleans up on Windows.
- binaries.py: chmod failures on POSIX now log a warning (only swallow on
  Windows where .exe is implicitly executable).
- tests/test_binaries.py: `test_mirror_substitution` now uses
  `monkeypatch.setenv()` instead of raw `os.environ` manipulation.
- tools.json: add `linux-x86_64-musl` and `linux-aarch64-musl` entries for
  `difft`; document the shared-asset strategy for both tools.
- proxy/interceptors/astgrep.py: log a debug line when
  `progressive_disclosure_key()` returns None for a tool whose tool_input
  shape we don't recognize.
- proxy/interceptors/base.py: moved `import json` to module top (was inside
  `_find_tool_use` hot loop).
- binaries.py: fix bare `.gz` detection — now explicitly excludes
  `.tar.gz`/`.tgz` instead of relying on a brittle "no dots" heuristic.
- proxy/interceptors/astgrep.py: provenance comment on each `_RANGE_KEYS`
  entry so future maintainers know which tool defined which key.
- cli/tools.py: comment explaining os.execv's lack of Python finalizer
  cleanup.
- proxy/interceptors/base.py: `InterceptionResult` now `frozen=True`.

**Test gaps closed:**
- Interceptor failure isolation (transform() raises → request survives,
  counter increments).
- Broken key() skips interceptor entirely.
- Refuse-to-enlarge guard (rewrite larger than original → pass through).
- Orphaned tool_result (no matching tool_use) doesn't crash.
- ToolResultInterceptorTransform.apply() happy path + frozen_message_count.
- ensure_tools() partial failure (one tool fetch fails, others succeed,
  proxy still starts).
- Mirror URL with query params doesn't leak into download filename.

44 tests total; ruff + mypy clean.
2026-04-20 17:36:44 -07:00
chopratejas 7cd67ef06d fix: bundle ast-grep/difftastic/scc + generic tool_result interceptor framework
What this does, in plain terms:

Headroom's proxy now ships with three CLI tools (ast-grep, difftastic,
scc) that it can use to shrink tool_result payloads before they reach
the model. The goal is simple: when Claude Code (or Codex, Aider, etc.)
asks the model to reason about a big file or diff, we swap the verbose
output for a compact, same-meaning version. Fewer tokens per turn, same
answers, lower bill.

Today a single interceptor is wired: ast-grep on Read. When an agent
reads a large code file, the proxy replaces the file body with an
outline of its top-level functions/classes plus docstrings. In live
tests that cut prompt tokens 74–76% on both OpenAI and Anthropic,
same answer either way.

How it works:
- `pip install headroom-ai` now installs ast-grep via a PyPI wheel
  (core dep). difftastic and scc are fetched once at proxy startup
  from pinned upstream GitHub releases and cached per-user.
- A generic registry (`headroom/proxy/interceptors/`) lets us add more
  tool-aware rewrites in one file each: declare `matches()` and
  `transform()`, call `register()`, done. No proxy or metrics plumbing
  per tool.
- Safety rails built in: pass-through when a Read specifies a line
  range; second Read of the same file in a conversation returns full
  content (progressive disclosure); any failing interceptor logs and
  skips, never crashes a request.

Opt-in for now:
- Off by default while this ships. Turn on with
  `headroom proxy --intercept-tool-results` or
  `HEADROOM_INTERCEPT_ENABLED=1`, so we can measure before flipping
  defaults.

What users see after turning it on:
- First `headroom wrap claude` boot is ~5s longer (binaries fetched).
  Every subsequent run is cache-only.
- Existing `transforms_applied` field in metrics gets entries like
  `interceptor:ast-grep`, so savings show up in current dashboards
  and HTML reports with no UI change.

Other housekeeping in this PR:
- uv.lock moved to .gitignore — regenerated locally per environment.
- 35 unit + integration tests, ruff + mypy clean.
- Dead-code audit done: removed `binaries.run()`, `needs_filesystem`
  plumbing, unused `_kind` tuple elements, unused `tool_output`
  parameter, and the never-set HEADROOM_SKIP_TOOLS_BOOTSTRAP env.
2026-04-20 17:36:44 -07:00