## Description
Two routes in the Rust front (`crates/headroom-proxy`) attach the host's
own cloud credentials to whatever request reaches them, without checking
the caller:
- **Bedrock native** (`/model/{id}/invoke`, `/converse`,
`/invoke-with-response-stream`, `/converse-stream`) is on by default and
signs every request with SigV4 using the AWS credentials found on the
host.
- **Vertex** (`…/publishers/anthropic/models/{m}:rawPredict` /
`:streamRawPredict`) is mounted unconditionally, replaces the caller's
`Authorization` with a bearer from the host's Google ADC, and forwards
to the generic `--upstream`, whatever host that is. So with a non-Google
upstream (the Python proxy, another provider) the host's Google token is
sent there. `--vertex-adc-scope` was parsed but never used.
This PR makes both routes explicit opt-ins, only sends the ADC bearer to
a Google API host, and uses the configured scope.
**Dependency:** these routes still do not authenticate callers
themselves. Callers are expected to be authenticated by the inbound
`HEADROOM_PROXY_TOKEN` gate added in #4840 (which also moves the default
listen address to loopback). This PR is based on `main` and does not
require #4840 to merge first, but operators who enable either route on a
non-loopback address need #4840 to restrict who can use them.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `config.rs`: `--enable-bedrock-native` /
`HEADROOM_PROXY_ENABLE_BEDROCK_NATIVE` now defaults to `false`. New
`--enable-vertex-native` / `HEADROOM_PROXY_ENABLE_VERTEX_NATIVE`
(default `false`) and `--vertex-allow-non-google-upstream` /
`HEADROOM_PROXY_VERTEX_ALLOW_NON_GOOGLE_UPSTREAM` (default `false`).
`Config::for_test` keeps both routes on so the existing route tests run
unchanged.
- `proxy.rs`: the Vertex route is mounted only when enabled; with it
off, Vertex paths go through the catch-all with the caller's own headers
(as the Python proxy's Vertex passthrough expects). The ADC token source
is built with `--vertex-adc-scope`. The "Bedrock route not mounted"
startup line is now `info`, since off is the default.
- `vertex/raw_predict.rs`: before fetching the ADC bearer, the handler
refuses (502 `vertex upstream is not a Google API host`,
`vertex_upstream_not_google` error log) unless `--upstream` is
`https://` on a `*.googleapis.com` host (regional, global and
`*.p.googleapis.com` endpoints), or the operator set
`--vertex-allow-non-google-upstream=true` (local emulators, gateways).
No token is fetched on that path.
- `RUST_DEV.md`: the three flags in the configuration table, with what
enabling them means.
**Behaviour change for operators:** deployments that used the Rust
front's native Bedrock or Vertex signing must now pass
`--enable-bedrock-native true` / `--enable-vertex-native true` (or the
env vars). Vertex with a non-Google `--upstream` additionally needs
`--vertex-allow-non-google-upstream true`. Without the flags, those
paths are forwarded to `--upstream` unsigned with the client's own
credentials (Claude Code in Vertex mode keeps sending its own ADC token,
which is what `docs/content/docs/claude-code-vertex.mdx` describes).
`scripts/qualify_cargo_dependencies.py` already passes
`--enable-bedrock-native true`.
## Testing
- [x] `cargo test -p headroom-proxy` (45 test binaries, 596 tests pass)
- [x] `cargo clippy -p headroom-proxy --all-targets -- -D warnings` and
`cargo fmt --all -- --check` clean
- [x] New tests added for new functionality
- [x] Manual testing performed
- [ ] `pytest` / `ruff` / `mypy`: N/A, no Python changes
New tests in `tests/integration_cloud_routes_opt_in.rs` build the config
from the real CLI defaults (`CliArgs` → `Config::from_cli`), give the
app host AWS credentials and a host ADC bearer, and record what a
`wiremock` upstream receives:
- `default_config_does_not_sign_bedrock_requests`: no `AWS4-HMAC-SHA256`
Authorization on `invoke`, `invoke-with-response-stream`, `converse`.
- `default_config_never_attaches_host_adc_bearer`: `rawPredict` and
`streamRawPredict` reach the upstream without the host bearer.
- `vertex_bearer_is_not_sent_to_a_non_google_upstream`: with the route
enabled and an `http://127.0.0.1` upstream, 502 and the upstream sees
nothing.
- `vertex_opt_in_still_signs_when_non_google_upstream_is_allowed`: the
explicit escape hatch still works.
- `configured_adc_scope_reaches_token_source`.
- Unit test `vertex::raw_predict::tests::google_api_host_check`
(`http://`, `googleapis.com.example.net`, `example-googleapis.com`,
loopback are rejected).
### Test Output
Before (the three tests that compile against `main`, run with this PR's
`src/` changes stashed):
```text
test configured_adc_scope_reaches_token_source ... FAILED
test default_config_never_attaches_host_adc_bearer ... FAILED
test default_config_does_not_sign_bedrock_requests ... FAILED
test result: FAILED. 0 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.77s
```
After:
```text
$ cargo test -p headroom-proxy --test integration_cloud_routes_opt_in
test configured_adc_scope_reaches_token_source ... ok
test vertex_opt_in_still_signs_when_non_google_upstream_is_allowed ... ok
test vertex_bearer_is_not_sent_to_a_non_google_upstream ... ok
test default_config_never_attaches_host_adc_bearer ... ok
test default_config_does_not_sign_bedrock_requests ... ok
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.69s
```
## Real Behavior Proof
- Environment: macOS (Darwin 27.2), debug builds of `headroom-proxy`
from `origin/main` (c21b0813) and this branch. AWS credentials supplied
to the proxy through `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` (fake
values), IMDS disabled. Upstream and Bedrock endpoint are a small Python
recorder on `127.0.0.1:18788` that prints the `Authorization` it
receives.
- Exact command / steps: `headroom-proxy --listen 127.0.0.1:18787
--upstream http://127.0.0.1:18788 --bedrock-endpoint
http://127.0.0.1:18788` (no Bedrock/Vertex flags), then `curl -d
'<anthropic body>'
http://127.0.0.1:18787/model/anthropic.claude-3-haiku-20240307-v1:0/invoke`
with no client credentials. Repeated on this branch with
`--enable-bedrock-native true`.
- Observed result (key id and signature redacted):
```text
== origin/main (no flags)
POST /model/anthropic.claude-3-haiku-20240307-v1:0/invoke, no client credentials -> 200
UPSTREAM SAW {"path": "/model/...:0/invoke", "authorization": "AWS4-HMAC-SHA256 Credential=<redacted>/20261113/us-east-1/bedrock/aws4_request, SignedHeaders=..., Signature=<sig>"}
== this branch (no flags)
POST /model/anthropic.claude-3-haiku-20240307-v1:0/invoke, no client credentials -> 200
UPSTREAM SAW {"path": "/model/...:0/invoke", "authorization": null}
== this branch, --enable-bedrock-native true
UPSTREAM SAW {"path": "/model/...:0/invoke", "authorization": "AWS4-HMAC-SHA256 Credential=<redacted>/20261113/us-east-1/bedrock/aws4_request, SignedHeaders=..., Signature=<sig>"}
```
- Not tested: Vertex against real Google ADC and a real
`*.googleapis.com` upstream (covered by the integration tests with a
static token source; no GCP credentials on the test machine).
## Runtime Rollout Safety
- Rollout-managed feature(s): `native_bedrock` (existing). The CLI flag
already maps `false` to a rollout disable, so with the new default the
feature reports disabled unless `--enable-bedrock-native true` is
passed. Vertex is not rollout-managed.
- Minimum rollout channel: unchanged
- Stable/default behavior changed: yes, both native routes are off by
default
- Kill switch / disable path: the routes are now off unless enabled
- Unsafe override required: no
- Qualification impact: none, the qualification script already passes
`--enable-bedrock-native true`
- Rollback path: pass `--enable-bedrock-native true` /
`--enable-vertex-native true` (plus `--vertex-allow-non-google-upstream
true` if the upstream is not a Google host), or revert this PR
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
Fix adapted from a proposal by HetCreep.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4851"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Reported by @JoaoMarcos44. Thank you for your contribution! 🩵
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
26 KiB
Headroom Rust Rewrite — Developer Guide
This document covers the Rust port of Headroom. It is the only new top-level doc created in Phase 0; longer-form design/plan writeups live elsewhere and are not versioned in this repo.
Workspace layout
Cargo.toml # workspace root
rust-toolchain.toml # pins stable rustc with rustfmt+clippy
crates/
headroom-core/ # library: shared types + transform trait surface
headroom-proxy/ # binary: axum /healthz (Phase 2 grows this)
headroom-py/ # PyO3 cdylib exposing `headroom._core`
headroom-parity/ # lib + `parity-run` CLI for Python parity tests
headroom-simulators/ # binary: deterministic local upstream stub for proxy tests
tests/parity/
fixtures/<transform>/*.json # recorded Python outputs (Phase 1 ports match)
recorder.py # Python-side fixture recorder
scripts/record_fixtures.py # entry point for running the recorder
cargo build --workspace builds every crate. default-members drops
headroom-py from cargo run/bare-cargo test flows so that cargo test --workspace does not try to execute the PyO3 cdylib standalone (it can't
find libpython without a Python interpreter hosting it).
Common commands
just is not installed on dev boxes here; a Makefile at the repo root
exposes the same targets:
| Target | What it does |
|---|---|
make test |
cargo test --workspace |
make test-parity |
Builds headroom-py via maturin, runs parity-run run |
make bench |
cargo bench --workspace |
make build-proxy |
Release-builds headroom-proxy, strips, prints size |
make build-wheel |
maturin build --release -m crates/headroom-py/Cargo.toml |
make fmt |
cargo fmt --all |
make lint |
cargo fmt --check + cargo clippy --workspace -- -D warnings |
Running the proxy
headroom-proxy is a transparent reverse proxy. Phase 1 forwards HTTP/1.1,
HTTP/2, SSE, and WebSocket traffic verbatim to a configured upstream — no
provider logic yet. The intent is that operators run the existing Python
proxy on a private port and put headroom-proxy on the public port pointed
at it; end users notice nothing.
# Build
make build-proxy
./target/release/headroom-proxy --help
# Run against a local upstream (listens on 127.0.0.1:8787 by default)
./target/release/headroom-proxy \
--upstream http://127.0.0.1:8788
# Health checks
curl -s http://127.0.0.1:8787/healthz # => {"ok":true,...}
curl -s http://127.0.0.1:8787/healthz/upstream # => 200 if upstream reachable
Operator runbook (Phase 1 cutover)
# 1. Move the Python proxy to a private port (e.g. 8788)
HEADROOM_HOST=127.0.0.1 HEADROOM_PORT=8788 python -m headroom.proxy & # or your existing launcher
# 2. Run the Rust proxy on the previously-public port (8787) pointing at it.
# The Python proxy sees every relayed request as coming from 127.0.0.1,
# so the Rust proxy enforces HEADROOM_PROXY_TOKEN itself (see below).
export HEADROOM_PROXY_TOKEN="$(openssl rand -hex 32)"
./target/release/headroom-proxy --listen 0.0.0.0:8787 --upstream http://127.0.0.1:8788 &
# 3. End users keep hitting :8787 unchanged.
# 4. Confirm passthrough:
curl -si http://127.0.0.1:8787/v1/models
# 5. Rollback = stop the Rust proxy and rebind Python back to 8787.
Configuration flags
| Flag | Env var | Default | Notes |
|---|---|---|---|
--listen |
HEADROOM_PROXY_LISTEN |
127.0.0.1:8787 |
bind address; a non-loopback address refuses to start without HEADROOM_PROXY_TOKEN unless HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1 |
--proxy-token |
HEADROOM_PROXY_TOKEN |
(unset) | inbound token required from non-loopback peers (below); prefer the env var |
--metrics-require-loopback |
HEADROOM_PROXY_METRICS_REQUIRE_LOOPBACK |
false |
when set, /metrics is served only to loopback peers (403 otherwise); recommended on non-loopback binds |
--upstream |
HEADROOM_PROXY_UPSTREAM |
(required) | base URL the proxy forwards to |
--upstream-timeout |
600s |
end-to-end request timeout (long for streams) | |
--upstream-connect-timeout |
10s |
TCP/TLS connect timeout | |
--max-body-bytes |
100MB |
for buffered cases; streams bypass | |
--log-level |
info |
RUST_LOG-style filter |
|
--rewrite-host / --no-rewrite-host |
rewrite | rewrite Host to upstream (default) | |
--graceful-shutdown-timeout |
30s |
wait for in-flight on SIGTERM/SIGINT | |
--stats |
HEADROOM_PROXY_STATS |
true |
native savings stats + /dashboard (below) |
--stats-path |
HEADROOM_PROXY_STATS_PATH |
~/.headroom/native_stats.json |
ledger persistence (honours HEADROOM_WORKSPACE_DIR) |
--enable-bedrock-native |
HEADROOM_PROXY_ENABLE_BEDROCK_NATIVE |
(unset: off) | true or --features native_bedrock mounts /model/{id}/invoke, /converse and the streaming variants; they sign every request with this host's AWS credentials and do not authenticate the caller |
--enable-vertex-native |
HEADROOM_PROXY_ENABLE_VERTEX_NATIVE |
false |
mount Vertex :rawPredict / :streamRawPredict; replaces the caller's Authorization with this host's Google ADC bearer (scope --vertex-adc-scope) and does not authenticate the caller |
--vertex-allow-non-google-upstream |
HEADROOM_PROXY_VERTEX_ALLOW_NON_GOOGLE_UPSTREAM |
false |
by default the Vertex route only sends the ADC bearer when --upstream is https://*.googleapis.com, and answers 502 otherwise |
Inbound authentication
The Rust proxy connects to the Python proxy from 127.0.0.1, and the
Python proxy trusts loopback callers (no token needed, and access to
/admin, /debug and /settings). So the Rust proxy applies the same
HEADROOM_PROXY_TOKEN rules itself, before anything is forwarded:
- With a token set, a non-loopback peer must send it as
X-Headroom-Proxy-Token: <token>orAuthorization: Bearer <token>, or gets401 {"error":"unauthorized"}. Loopback peers are exempt, as in the Python proxy. The token is removed from the request before it is forwarded (Authorizationonly when it carries the token). - A caller that is not local here (a non-loopback peer, or a loopback
peer whose
Hostis not a loopback name) is never forwarded with aHostthat may name loopback (including scoped IPv6 such as[::1%lo]), whatever--rewrite-hostis and whatever the upstream name resolves to: thatHostis replaced withheadroom-remote.invalid, so the Python proxy's local-only routes (/admin,/debug,/settings, ...) stay closed to it, with or without a token. GET /healthz,/healthz/upstream,/health,/livezand/readyzstay reachable without the token, andGET/HEAD /api/hellois answered locally with an empty200.- A non-loopback
--listenwith no token refuses to start. SetHEADROOM_ALLOW_UNAUTHENTICATED_BIND=1only when something else already restricts who can reach the port.
Savings stats & dashboard (native)
The Rust proxy records per-request savings/cost telemetry on every
lane it forwards — Anthropic /v1/messages, OpenAI Chat/Responses,
all four native Bedrock routes, and Vertex rawPredict /
streamRawPredict when those are enabled — and serves it locally (never tunnelled
upstream):
curl -s http://127.0.0.1:8787/stats | jq '.session' # since-boot totals
curl -s http://127.0.0.1:8787/stats | jq '.lifetime_by_model'
curl -s "http://127.0.0.1:8787/stats/timeseries?bucket=day" | jq '.points[-7:]'
curl -s "http://127.0.0.1:8787/stats/events?limit=5" # recent-request feed
open http://127.0.0.1:8787/dashboard # embedded UI, zero deps
Recording is deferred until the response is fully observed, so
streaming output/cache token counts come from the real SSE /
EventStream usage frames. USD figures come from the vendored LiteLLM
price table (data/model_prices_and_context_window.json); a model
missing from it records $0 and logs one WARN — refresh via
scripts/refresh_model_limits.sh. Input and output are priced
separately (input_cost_usd / output_cost_usd) because they bill
at different rates — output is typically 4-5x input, so a spend
figure that counted input alone would understate a short-prompt,
long-answer turn by most of its cost. total_cost_usd is the sum
and is what /dashboard shows as "Spend". Cache savings are NET: the
cache-read discount minus the cache-write premium (writes bill
above list price), floored at $0 per request — a warm-up turn that
only writes is not counted as negative savings, and gross-read
figures that overstate the benefit are never shown. Failed upstreams count as
failures and accrue no savings — as do proxy-side rejections
(missing AWS credentials, SigV4/ADC failures, oversized bodies), so
an operator-side outage reads as a failure spike rather than as
zero traffic. Aggregates (lifetime, per-model, 48 h hourly +
~13 mo daily buckets) persist as one atomic snapshot written off
the hot path every 10 s and on shutdown.
What reads 0 today. Spend, token counts, and prompt-cache savings are live.
tokens_saved/compression_savings_usdare wired end-to-end but will report 0 on real traffic until the live-zone per-type compressors are implemented — the dispatcher currently returnsNoCompressionfor every well-formed body (live_zone_mode_with_valid_body_returns_no_compression_pr_b2pins that invariant). Nothing needs changing here when they land; the numbers start flowing on their own.
No auth; exposure is split by tier. These endpoints bind wherever
--listenpoints (default127.0.0.1:8787) and, for loopback peers or with noHEADROOM_PROXY_TOKEN, carry no auth, so the payload is split the same way the Python proxy splits its own/stats:
- Aggregates — spend, tokens, per-model/provider rollups, history — are served to any caller that gets past the inbound gate (above), like the Prometheus
/metricsendpoint next door: withHEADROOM_PROXY_TOKENset, remote callers need the token for these too; without it, they are open to anyone who can reach the port.- Per-request rows —
recent_requests, all of/stats/events— plus the ledger's filesystem path are served only to local callers: loopback peer and a loopbackHostheader (the second is the DNS-rebinding defence — a rebound request reaches the proxy from loopback, so the peer check alone doesn't hold). Remote callers getrecent_requests: nulland a 404 from/stats/events; the dashboard says so rather than erroring.A request id + model + timestamp is a different sensitivity tier from a counter, and
/metricsnever exposes it — so "same as /metrics" isn't a licence to hand it to the open internet. Opening the local-only tier to a trusted reverse proxy is tracked in #1959, which is settling that policy for the Python dashboard first; this surface should adopt it rather than grow a second scheme.--stats=falseremoves the routes entirely.
Picking the next port: invocation telemetry
Before porting another Python compressor to Rust, check what's actually
running. The Python proxy already exposes per-transform telemetry on
/stats (headroom.proxy.prometheus_metrics):
# Top compressors by invocation count (last process lifetime)
curl -s http://127.0.0.1:8788/stats | jq '.compressions_by_strategy'
# {
# "intelligent_context": 12453,
# "smart_crusher": 487,
# "search": 312,
# "diff": 28,
# "code": 0, # ← never fires; safe to defer porting
# ...
# }
# Per-transform timing (avg/max/count by transform name)
curl -s http://127.0.0.1:8788/stats | jq '.pipeline_timing'
# Token savings attributable to each strategy
curl -s http://127.0.0.1:8788/stats | jq '.tokens_saved_by_strategy'
This is the data the audit-cleanup PR (2026-04-30) recommended for prioritizing the next Python → Rust port. Strategies with zero or near-zero invocations are deferral candidates; strategies on the hot path are porting candidates regardless of LOC count.
Reserved paths
/healthz and /healthz/upstream are intercepted by the Rust proxy and
not forwarded. Operators must not name a real upstream route either of
these. Everything else is a catch-all forward.
Maturin + Python wiring
headroom-py is a PyO3 cdylib that exposes headroom._core in Python. The
extension-module feature is opt-in so plain cargo build --workspace does
not try to link against libpython on systems that don't have it.
First-time setup (clean venv recommended)
python3.11 -m venv /tmp/hr-rust-venv
source /tmp/hr-rust-venv/bin/activate
pip install maturin
cd crates/headroom-py
maturin develop # editable dev build, installs headroom._core
cd /tmp # IMPORTANT: step out of the repo root first
python -c "from headroom._core import hello; print(hello())"
# => headroom-core
Why
cd /tmp? The repo root also contains the Pythonheadroom/package. Running the smoke import from the repo root makes Python resolveheadroomto./headroom/__init__.py(the full SDK, which pulls in heavy deps) instead of the lightweight namespace package installed by maturin. Tests should either run outside the repo root, or ensureheadroomis installed into the same venv (then the maturin-installed_core.solands alongside it and both imports resolve).
Release wheels
make build-wheel
# wheels land under target/wheels/
CI (.github/workflows/rust.yml) builds linux-x86_64, macos-arm64, and
macos-x86_64 wheels via PyO3/maturin-action and uploads them as artifacts.
Parity harness
crates/headroom-parity owns the Rust-vs-Python oracle:
- JSON fixtures under
tests/parity/fixtures/<transform>/(schema:{ transform, input, config, output, recorded_at, input_sha256 }). TransformComparatortrait — one impl per transform. Phase 0 stubs returnErr(...); the harness flags those asSkipped, not panics.parity-runCLI:cargo run -p headroom-parity -- run [--only TRANSFORM].- Unit tests in
crates/headroom-parity/src/lib.rsinclude a negative test (harness_reports_diff_for_divergent_comparator) proving the harness detects mismatched output before any real port lands.
Recording fresh fixtures
source .venv/bin/activate # the main Python SDK venv
python scripts/record_fixtures.py # uses tests/parity/recorder.py
ls tests/parity/fixtures/*/ | sort | uniq -c
The recorder monkey-patches the in-process transform classes (see
record_all() in tests/parity/recorder.py). It does not modify any
file under headroom/.
Known regressions in retired-Python components
The Stage 3b/3c.1b retirements deleted Python source for DiffCompressor
and SmartCrusher and replaced them with PyO3-delegating shims. The
2026-04-28 audit found that the retirements shipped with subsystems
silently disconnected. This section tracks each gap and its disposition
so they don't regress further or get forgotten.
SmartCrusher
| Subsystem | State | Tracked by |
|---|---|---|
| TOIN learning loop | Re-attached 2026-04-28. Shim's crush() and _smart_crush_content() now call toin.record_compression() after a real compression. Filtered on strategy != "passthrough" to ignore JSON re-canonicalization. Best-effort: TOIN failures are logged at debug level and don't break compression. |
tests/test_smart_crusher_toin_attachment.py |
| CCR marker emission knob | Honored end-to-end 2026-04-29. New enable_ccr_marker: bool field on Rust SmartCrusherConfig; crush_array checks it before emitting the <<ccr:HASH>> marker text and the CCR store write. Python shim flips it from ccr_config.enabled and ccr_config.inject_retrieval_marker — both flags collapse to the same Rust gate, since storing payloads under either off-switch makes no sense. Scope: gates only the row-drop sentinel path; Stage-3c.2 opaque-string CCR substitutions still emit always (no Python equivalent, no production caller asks for suppression). |
tests/test_smart_crusher_toin_attachment.py + crates/headroom-core/.../crusher.rs::tests::enable_ccr_marker_* |
| Custom relevance scorer | Closed (fail-loud) 2026-04-29. relevance_config and scorer constructor args remain in the signature for source compat, but the shim raises NotImplementedError when either is non-None — silently dropping a user-supplied scorer is a textbook silent-fallback bug. Full plumbing waits on Stage-3c.2's relevance-crate Python bridge. |
tests/test_smart_crusher_toin_attachment.py::test_custom_*_arg_raises_not_implemented |
| Per-tool TOIN learning hook | Re-attached partially. _smart_crush_content accepts tool_name and now threads it into the TOIN record. The hook is best-effort — it improves query_context aggregation but doesn't drive per-tool overrides yet. |
tests/test_smart_crusher_toin_attachment.py::test_smart_crush_content_records_to_toin |
DiffCompressor
| Subsystem | State |
|---|---|
| Adaptive context windows | Honored byte-for-byte (parity fixture-locked). |
| TOIN integration | Never had one — DiffCompressor records via _record_to_toin in ContentRouter, which already runs for non-SmartCrusher strategies. No regression. |
Phase 3e.1 — signals/ trait module + KeywordDetector (2026-04-29)
The Python error_detection.py regex registry was retired and reborn as a
trait + tier system in crates/headroom-core/src/signals/. See
signals/README.md for the full architecture; the highlights:
- Per-granularity traits.
LineImportanceDetectorships today; futureContentTypeDetectorandItemImportanceDetector<I>will follow as their consumers get touched. Tiered<T>combinator. Composition, not inheritance. Future ML detectors slot in as new tiers without changes toKeywordDetectoror any caller.- One concrete impl.
KeywordDetector(aho-corasick) is the only tier registered today. No NoOp/stub impls — per project no-silent-fallbacks rule, future tiers land with their real implementations. - Bug fixes baked in.
ERROR_KEYWORDSregex now includestimeout|abort|denied|rejected(previously drifted from the keyword set);tokendropped fromSECURITY_KEYWORDS(false-positived on every LLM metric reference). Both fixed in the Python regex too via the shim that recompiles patterns from the Rust-exposed keyword tables. - Companion canonical extension path.
signals/README.mddocuments the BGE classifier head — a 384-dim → 4-class softmax on top of the already-loadedbge-small-en-v1.5embedder — as the natural ML tier. Two alternatives kept open: distilled tinyBERT in ONNX, logistic regression on lexical features.
Phase 3g (queued) — Compression Pipeline Formalization (issue #315)
Strategic decision 2026-04-29: after Phase 3e (compressor ports) and
Phase 3f (Rust MCP scaffold) wrap, formalize the lossless-then-lossy-
then-CCR ordering as a cross-cutting CompressionPipeline orchestrator
LosslessTransform/LossyTransformtraits incrates/headroom-core/src/pipeline/. Existing compressors get refactored as compositions of pluggable transforms. The crucial design choice — parsers for structure, models at the prose/structure boundary — is captured in issue #315 andmemory/project_lossless_first_pipeline.md. Do NOT start coding before 3e/3f finish.
Watch list (potential regressions, not yet audited)
CCRConfig.enabled=Falseend-to-end — closed 2026-04-29. Bothenabled=Falseandinject_retrieval_marker=Falsecollapse to the same Rustenable_ccr_marker=Falsegate (no marker, no store write). See the SmartCrusher table above.SmartCrusherConfig.use_feedback_hints=False— config field is forwarded to Rust but its honoring inside the Rust crusher hasn't been verified against a parity fixture for the disabled path.
When any item above changes, update both this section and the test file. The shim's docstring also references this section — keep them aligned.
Phase 0 Blockers
These are known limitations for Phase 0. They are tracked here so Phase 1 doesn't rediscover them.
cache_alignerfixtures:CacheAligner.apply()takes(messages, tokenizer, **kwargs)— aTokenizeris provider-specific and its cheapestNoopTokenCounter/TiktokenTokenCounterconstruction still requires pullingheadroom.providers.*which imports the full observability stack (opentelemetry, etc). The recorder recordscache_aligneronly if a usable tokenizer is cheaply available; otherwise it logs a blocker and skips. Seerecorder.py::_build_cache_aligner_tokenizer.ccris not a single class: The repo hasCCRToolInjector,CCRResponseHandler,CCRToolCall,CCRToolResultetc. rather than a singleCCRclass. The recorder targets the encoder-style entry point most analogous to the Rust port (CCRToolInjector.inject_toolandCCRResponseHandler.parse_response). If Phase 1 wants a different split it should updaterecorder.py::record_allaccordingly.- Pre-commit hook noise:
scripts/sync-plugin-versions.pymutates.claude-plugin/marketplace.json,.github/plugin/marketplace.json, andplugins/headroom-agent-hooks/**/plugin.jsonon every commit. Those changes are harmless but each commit in Phase 0 picks them up. Phase 1 does not need to do anything special — just let the hook run. rust-toolchain.tomlnow pinschannel = "1.95.0"(previously tracked"stable", which let CI drift ahead of local dev boxes — see the file's own comment for the 2026-04-27 incident this fixed). Resolved; kept here for history.
Multi-worker deployment — CCR fragmentation
Status: two persistent CCR backends are available. The single---workers
recommendation no longer applies once you select a persistent backend.
Backend selection
crates/headroom-core/src/ccr/backends/ ships three implementations of
the CcrStore trait:
| Backend | When to use | Persistence | Multi-worker safe |
|---|---|---|---|
InMemoryCcrStore |
Tests, single-worker prototyping | No | No |
SqliteCcrStore (default) |
Single-instance prod / single-host fleet | Yes (file) | Yes (sticky session) |
RedisCcrStore (opt-in) |
Multi-host / horizontally-scaled prod | Yes (Redis) | Yes (no stickiness needed) |
backends::from_config picks one at startup from the operator's
CcrBackendConfig. Init failures surface to the caller
(feedback_no_silent_fallbacks.md) — a misconfigured DB path or
unreachable Redis URL aborts startup rather than silently degrading to
in-memory.
When does what work?
SqliteCcrStoreis the default for new deploys. The DB file lives on the local disk; multiple workers on the same host share it via SQLite's WAL-mode locking, so--workers Nworks as long as a sticky load balancer routes each session to the same host. Survives proxy restarts: a new worker that opens the same DB file recovers every in-flight<<ccr:HASH>>marker.RedisCcrStore(cfg-gated behind theredisfeature) is the drop-in for horizontally-scaled deployments. Every worker on every host hits the same Redis instance; no sticky session is required at any layer of the LB. Enable with--features redisin the proxy crate's Cargo build.InMemoryCcrStoreis fine for tests and single-worker development. Production deployments using it lose every<<ccr:HASH>>marker on restart and fragment across workers — keep it confined to local boxes.
What goes wrong with the in-memory backend on --workers N > 1
Each uvicorn worker is a separate Python process. The following state is fragmented across workers:
- Python
CompressionStore— defaults toSQLiteBackendatworkspace_dir()/ccr_store.db(restart-safe, shared across workers) whenHEADROOM_CCR_BACKENDis unset or"sqlite"(_create_default_ccr_backendinheadroom/cache/compression_store.py). SetHEADROOM_CCR_BACKEND=memoryto opt into the per-processInMemoryBackendinstead — that is the setting this section's fragmentation risk actually applies to; the log messages inheadroom/proxy/server.py(see below) still assume in-memory-by-default and are stale on this point. HeadroomProxy._compression_caches(headroom/proxy/server.py) — per-sessionCompressionCachedict (instance var, always per-worker).HeadroomProxy.session_tracker_store— per-session prefix-tracker state derived from Anthropic'scache_read_input_tokensresponses (instance var, always per-worker).- TOIN learner state — writes snapshots to
~/.headroom/toin.jsonbut keeps per-process in-memory state; pattern statistics on one worker are not visible to others until the next disk flush.
When uvicorn round-robins requests across workers, a session whose
turn-1 landed on worker A may have turn-2 land on worker B. Worker B has
zero knowledge of what worker A did, the <<ccr:HASH>> marker resolves
to None, and the model sees an opaque directive it can't act on.
Switching to SqliteCcrStore (default) or RedisCcrStore resolves the
CCR fragmentation; a sticky-session load balancer resolves all of them.
Detecting it in the wild
The proxy emits a WARNING-level log line on startup when --workers N > 1.
When HEADROOM_CCR_BACKEND is unset (default InMemoryBackend), the warning
includes CCR retrieval failures and suggests setting HEADROOM_CCR_BACKEND=sqlite.
When a cross-worker backend is already configured, the warning covers only the
remaining per-worker stores (compression cache, prefix tracker, TOIN, CostTracker).