mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat/https-compose-caddy
18
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
d7a2fcf092 |
test: make the loopback-gating tests able to tell the guard from a missing key (#4784)
## Description
`tests/test_proxy_loopback_gating.py` says non-loopback callers get a
404 on the gated routes, but `test_non_loopback_caller_gets_404` used a
default `TestClient`, whose peer is the string `"testclient"` (not an
IP), and asserted only the 404 status. Two of the twelve gated routes,
`/v1/telemetry/tools/{hash}` and `/v1/toin/pattern/{prefix}`, answer 404
on their own when the key is absent, and `test_loopback_caller_allowed`
accepts `{200, 404, 422}`. So removing the guard from those routes, or
making it refuse every parseable IP, did not turn the test red.
The guard's denial body is `{"detail": "Not Found"}` (what FastAPI
answers for an unknown path); a detail route answers with its own
message. This PR (1) uses a routable remote peer (`203.0.113.9`,
TEST-NET-3) with a loopback `Host`, so only the client-IP check can
refuse it, (2) asserts the guard's exact denial body, and (3) asserts
that a loopback caller never receives that body.
Part of #4783
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `tests/test_proxy_loopback_gating.py`: a `GUARD_DENIAL` constant with
a comment saying why; `test_non_loopback_caller_gets_404` uses
`client=("203.0.113.9", 4242)` and `base_url="http://127.0.0.1"` and
asserts `resp.json() == GUARD_DENIAL`; `test_loopback_caller_allowed`
also asserts `resp.json() != GUARD_DENIAL`. No product code changes.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
`mypy headroom` was not run (no change under `headroom/`). `ruff check`
and `ruff format --check` (0.17.3) are clean.
### Test Output
Whole file, product unchanged:
```text
pytest tests/test_proxy_loopback_gating.py 147 passed
```
The same file run against two mutations of the product (each restored
afterwards), with the old tests (`origin/main`) and the new ones:
```text
Mutation A: loopback_guard.py `return address.is_loopback` -> `return True`
old tests: 20 failed, 127 passed
new tests: 32 failed, 115 passed (the 12 non-loopback cases now fail too)
Mutation B: remove `dependencies=[Depends(_require_loopback)]` from
/v1/telemetry/tools/{signature_hash} and /v1/toin/pattern/{hash_prefix:path}
old tests: 147 passed (nothing noticed)
new tests: 2 failed, 145 passed (the two routes)
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, pytest 9.2.0, the compiled
`headroom._core`, `main` at `3c66c91e`.
- Exact command / steps: `create_app(ProxyConfig(...))` and three
`TestClient`s (default peer, `client=("203.0.113.9", 4242)`,
`client=("127.0.0.1", 12345)`, the last two with
`base_url="http://127.0.0.1"`); `GET` each of
`/v1/telemetry/tools/example`, `/v1/toin/pattern/example`,
`/v1/feedback/example`; then the two mutations above.
- Observed result: both non-loopback peers got `404 {"detail":"Not
Found"}` on all three; the loopback peer got `404 {"detail":"No
telemetry found for signature: example"}`, `404 {"detail":"No TOIN
pattern found with hash starting with: example"}` and `200`
respectively. The mutation results are in Test Output.
- Not tested: CI, other platforms, and the two non-GET gated routes
under mutation B (they are covered by the new body assertion but I did
not mutate them).
## Runtime Rollout Safety
- Rollout-managed feature(s): none; test only.
- Minimum rollout channel: n/a
- Stable/default behavior changed: no.
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes (this
file; the whole suite was not run)
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
- The vanilla `TestClient` (peer `"testclient"`) is also refused by the
guard; I did not keep a test for it because it is not a
loopback-versus-remote distinction.
AI assistance: this change, its tests and the runs above were done with
Claude (Sonnet 5.5), operated by me (HetCreep).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4784"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
f5c4217648 |
fix(proxy): enforce HEADROOM_PROXY_TOKEN in the Rust front and listen on loopback by default (#4840)
## Description
The Rust front (`crates/headroom-proxy`) relays every request to the
Python proxy from `127.0.0.1`. The Python proxy treats a loopback caller
as local: it skips the `HEADROOM_PROXY_TOKEN` check and serves `/admin`,
`/debug` and `/settings` to it. The front had no inbound check of its
own, listened on `0.0.0.0:8787` by default, and dropped
`x-headroom-proxy-token` (as an `x-headroom-*` header), so a token set
on the Python side neither protected the stack nor could be presented
through the front.
This PR makes the Rust front apply the same token model as the Python
proxy (`read_proxy_token` / `scrub_proxy_token_headers` in
`headroom/proxy/server.py`, `headroom/proxy/bind_policy.py`) before
anything is forwarded, and makes its default listen address loopback.
Fix adapted from a proposal by HetCreep.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- New `crates/headroom-proxy/src/inbound_auth.rs`, installed as a
router-wide layer (all routes, the catch-all and WebSocket upgrades)
when `HEADROOM_PROXY_TOKEN` / `--proxy-token` is set:
- a non-loopback peer (or one with no peer address) must send the token
as `X-Headroom-Proxy-Token` or `Authorization: Bearer`; the explicit
header wins, as in Python. Constant-time compare. Otherwise `401
{"error":"unauthorized"}` and no upstream contact; a
`proxy_auth_rejected` warning is logged (no credential values).
- loopback peers are exempt (same as Python); `GET /healthz`,
`/healthz/upstream`, `/health`, `/livez`, `/readyz` stay open;
`GET`/`HEAD /api/hello` (also under `/p/<project>`) is answered locally
with an empty 200, as Python does, instead of being relayed.
- the token is removed before forwarding: `x-headroom-proxy-token`
always, `Authorization` only when it carries the token (a provider
credential is left alone).
- the token is redacted from `Debug` output of `Config`.
- `main.rs`: startup bind policy, same rule and variable as Python: a
non-loopback `--listen` with no token exits with code 2 and a message
naming both remedies, unless `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1`
(then a warning is logged).
- `config.rs`: `--listen` / `HEADROOM_PROXY_LISTEN` default changes from
`0.0.0.0:8787` to `127.0.0.1:8787`; new `--proxy-token` (env
`HEADROOM_PROXY_TOKEN`, value hidden from `--help`).
- `RUST_DEV.md`: runbook exports a token for the `0.0.0.0` cutover, flag
table updated, new "Inbound authentication" section.
**Behaviour change for operators:** a Rust front started with `--listen
0.0.0.0:...` (or relying on the old default to be reachable from other
hosts) now needs `HEADROOM_PROXY_TOKEN` set, and clients on other hosts
must send it. Setups that front the port with something else that
restricts access can set `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1`.
Loopback clients (Claude Code, Codex, SDKs on the same host) are
unaffected. The Docker image's entrypoint is the Python proxy, so it is
unaffected.
## Testing
- [x] `cargo test -p headroom-proxy` (46 test binaries, 608 tests pass)
- [x] `cargo clippy -p headroom-proxy --all-targets -- -D warnings` and
`cargo fmt --all -- --check` clean
- [x] New tests added for new functionality
- [x] Manual testing performed
- [ ] `pytest` / `ruff` / `mypy`: N/A, no Python changes
New tests:
- `tests/integration_inbound_auth.rs` (11 tests, `oneshot` with an
injected `ConnectInfo` so a non-loopback peer can be simulated,
`wiremock` upstream): missing/wrong token is 401 and the upstream sees
nothing; correct token via header or Bearer is forwarded without the
token; provider `Authorization` kept when the header carries the token;
loopback and IPv4-mapped loopback peers pass; missing peer address fails
closed; health probes open but `POST /readyz` is 401; `/api/hello`
answered locally; no token configured keeps passthrough.
- `tests/integration_bind_policy.rs` (3 tests, real binary): `0.0.0.0`
with no token refuses to start; loopback starts; default `--listen` is
loopback.
- unit tests in `inbound_auth.rs` for token parsing, constant-time
compare, bind policy and exemptions.
### Test Output
Before (the bind-policy test file copied onto a clean `origin/main`
worktree; `integration_inbound_auth.rs` does not compile there since
main has no token setting at all):
```text
test default_listen_address_is_loopback ... FAILED
test loopback_listen_without_token_still_starts ... ok
test non_loopback_listen_without_token_refuses_to_start ... FAILED
test result: FAILED. 1 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 20.01s
```
After:
```text
$ cargo test -p headroom-proxy --test integration_inbound_auth --test integration_bind_policy
test default_listen_address_is_loopback ... ok
test non_loopback_listen_without_token_refuses_to_start ... ok
test loopback_listen_without_token_still_starts ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.01s
test missing_peer_address_fails_closed ... ok
test remote_peer_with_bearer_token_is_forwarded_without_it ... ok
test remote_peer_without_token_gets_401_and_upstream_is_not_contacted ... ok
test token_header_wins_over_a_provider_bearer ... ok
test no_token_configured_keeps_todays_passthrough ... ok
test loopback_peer_without_token_is_still_forwarded ... ok
test ipv4_mapped_loopback_peer_counts_as_loopback ... ok
test remote_peer_with_token_header_is_forwarded_without_it ... ok
test hello_probe_is_answered_locally_without_token ... ok
test remote_peer_with_wrong_token_gets_401 ... ok
test health_probes_stay_reachable_without_token ... ok
test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.74s
```
## Real Behavior Proof
- Environment: macOS (Darwin 27.2), debug builds of `headroom-proxy`
from `origin/main` (
|
||
|
|
ab1d0292df |
fix(proxy): refuse HTTP requests from untrusted browser origins (#4853)
## Description The provider routes (`/v1/messages`, `/anthropic/v1/messages`, `/v1/chat/completions`, `/v1/responses`, Bedrock/Vertex routes and the catch-all passthrough) add operator-held credentials to what they forward: `*_extra_headers`, the Copilot token (`apply_copilot_api_auth`), and the provider env key on the litellm/anyllm backends when the request carries none. Loopback callers skip `HEADROOM_PROXY_TOKEN`, and a page open in the user's browser reaches the proxy as a loopback caller. It can send a JSON body as `Content-Type: text/plain`, which needs no CORS preflight, and the proxy parses and forwards it. CORS only stops the page from reading the answer, not the proxy from sending the request. Only the WebSocket routes checked `Origin`. The security gate now applies the WebSocket Origin rule to every HTTP request, before routing. A request is refused with 403 when it carries an `Origin` that is not: - a loopback page (`http(s)://127.0.0.1|localhost|[::1]` on any port; this is the dashboard), - an origin listed in `HEADROOM_WS_ORIGINS` / `HEADROOM_CORS_ORIGINS` (the same list `_is_allowed_websocket_origin` honours), - for a non-loopback caller only, the proxy's own address (a remote dashboard behind `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS`). A loopback caller gets no same-origin pass, since a non-loopback `Host` with a loopback peer means a page's hostname was pointed at 127.0.0.1. A cross-site subresource load (`<img>`, `<script>`) sends no `Origin` on GET, so `Sec-Fetch-Site: cross-site` is refused too, except for a top-level navigation. Requests without `Origin` (Claude Code, Codex, Copilot's extension host, SDKs, curl) are unaffected. Because the check sits in the gate, it covers every backend, including the litellm/anyllm env-key fallback, without changing how a keyless request from a local tool is authenticated. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/server.py`: `_is_refused_browser_request(request)` and a step 0 in `_security_gate` that answers 403 before the token check and routing. The 403 comes with the usual security headers and a hint naming `HEADROOM_WS_ORIGINS`, and an `event=proxy_browser_origin_rejected` log line records the escaped Origin. An origin listed in either `HEADROOM_WS_ORIGINS` or `HEADROOM_CORS_ORIGINS` is accepted, even when both are set. - `tests/test_proxy_browser_origin.py`: text/plain POSTs to `/v1/messages` and `/v1/chat/completions` from a loopback peer with a foreign / `null` / look-alike Origin never reach the upstream; without Origin, from a loopback page, and from a configured origin they still forward with the operator header; cross-site subresource GET refused; cross-site navigation still served; loopback peer with a re-pointed hostname refused. - `tests/test_proxy_loopback_gating.py`: two existing tests that sent a foreign `Origin` to `/stats` and `/metrics` now expect the gate's 403 (they already asserted refusal / no metadata; the gate now refuses earlier). - `docs/content/docs/security-model.mdx`: describes the rule, and lists the setups that need an explicit origin: a reverse proxy on the same host, and app-scheme desktop shells. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # new test file on origin/main (before the fix) $ pytest tests/test_proxy_browser_origin.py FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers0-/v1/chat/completions] FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers0-/v1/messages] FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers1-/v1/chat/completions] FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers1-/v1/messages] FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers2-/v1/chat/completions] FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers2-/v1/messages] FAILED ...test_configured_origin_still_forwards FAILED ...test_cross_site_subresource_without_origin_is_refused ========================= 8 failed, 5 passed in 11.06s ========================= # with the fix $ pytest tests/test_proxy_browser_origin.py tests/test_proxy_loopback_gating.py tests/test_proxy_cors.py ======================= 175 passed, 5 warnings in 34.36s ======================= # wider sweep: tests/test_proxy*.py tests/test_proxy/ tests/test_header_isolation.py # tests/test_upstream_credential_scoping.py tests/test_openai_codex_routing.py # tests/test_dashboard_* tests/test_copilot* (-k "not playwright") = 12 failed, 2368 passed, 114 skipped, 30 deselected in 483.68s = # the 12 failures (tests/test_proxy_uds.py x11, one test_proxy_compress_endpoint lossy-inline case) # fail identically on origin/main in this environment $ ruff check headroom/proxy/server.py tests/test_proxy_browser_origin.py tests/test_proxy_loopback_gating.py All checks passed! $ mypy headroom/proxy/server.py --ignore-missing-imports Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS, Python 3.12. `headroom proxy --host 127.0.0.1 --no-optimize --anthropic-api-url http://127.0.0.1:<L> --anthropic-extra-headers '{"x-api-key":"OPERATOR_KEY_SENTINEL"}'`, no proxy token, with a small HTTP listener on `<L>` that logs each request's `x-api-key` and prompt. One run on origin/main, one on this branch. - Exact command / steps: three `curl -X POST http://127.0.0.1:<port>/v1/messages -H 'Content-Type: text/plain' --data '<messages JSON>'`, with `Origin: http://page.example`, with `Origin: http://127.0.0.1:<port>`, and with no Origin. - Observed result: ```text == origin/main Origin: http://page.example -> HTTP 200 Origin: http://127.0.0.1:18961 -> HTTP 200 (no Origin) -> HTTP 200 listener saw: {"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt": "request-1 (Origin: http://page.example)"} {"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt": "request-2 (Origin: http://127.0.0.1:18961)"} {"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt": "request-3 (X-No-Origin: 1)"} == this branch Origin: http://page.example -> HTTP 403 Origin: http://127.0.0.1:18962 -> HTTP 200 (no Origin) -> HTTP 200 listener saw: {"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt": "request-2 (Origin: http://127.0.0.1:18962)"} {"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt": "request-3 (X-No-Origin: 1)"} ``` Dashboard on the same branch: `GET /dashboard` as a cross-site navigation -> 200; same-origin `POST /settings` -> 200; same-origin `GET /stats` -> 200. - Not tested: a real Copilot upstream (the refusal happens before any handler, so it is the same for every route). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: HTTP requests carrying an `Origin` that is neither loopback, configured, nor (for a remote caller) the proxy's own address now get 403. A browser app on another origin that calls the proxy directly must be listed in `HEADROOM_WS_ORIGINS` (or `HEADROOM_CORS_ORIGINS`, which also governs CORS). Electron/webview renderers that send `Origin: null` or an app scheme are refused unless listed. - Kill switch / disable path: list the origin in `HEADROOM_WS_ORIGINS`. - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes A companion PR (#4846) stops a `*` entry in `HEADROOM_WS_ORIGINS` / `HEADROOM_CORS_ORIGINS` from accepting every Origin. Both PRs share `_is_allowed_websocket_origin`, so once both land `*` is ignored on HTTP too; until then, `HEADROOM_CORS_ORIGINS=*` keeps HTTP open to every Origin, as it keeps the WebSocket routes today. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
e15e342d9b |
fix(proxy): resolve the forwarded client from the right and keep settings writes loopback-only (#4847)
## Description
Two fixes for proxies run with `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`
(opt-in, empty by default):
1. **Client address.** The proxy took the *leftmost* `X-Forwarded-For`
element as the client. A gateway that appends to the header keeps
whatever the client itself sent on the left, so the client could pick
its own address (for example `127.0.0.1`). The client is now the
right-most hop that is not itself inside a trusted-gateway CIDR. If
every hop is trusted it falls back to the leftmost, as before.
2. **Settings writes.** A peer inside a trusted-gateway CIDR counted as
loopback in `_request_is_loopback`, so `POST /settings` and `POST
/settings/apply` (which restarts the proxy) were open to anything routed
through the gateway. That rule exists so a containerised dashboard can
show its read-only views. Writes now need a real loopback peer, or a
client inside `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` (the
existing #2466 path, with IP-literal Host and same-origin checks).
**Behaviour change:** someone who runs Headroom in a Docker bridge
container, sets only `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`, and saves
settings from the host browser via `localhost` will now get 404 on save.
Reads still work. To keep saving, add the gateway address to
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` (e.g. `172.17.0.1/32`)
and open the dashboard at `http://127.0.0.1:<port>`. That path is
checked below. Without the env var nothing changes.
Fix adapted from a proposal by HetCreep.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/forwarded_policy.py`: new `header_client()` walks
`X-Forwarded-For` from the right and skips trusted-gateway hops.
`resolve_forwarded_headers` uses it.
- `headroom/proxy/server.py`: `_request_is_loopback` /
`_request_can_view_dashboard_metadata` take `allow_gateway_peer`
(default unchanged). `POST /settings` and `POST /settings/apply` use a
new write guard with `allow_gateway_peer=False`, and so does the
same-origin guard for those routes.
- Tests: the leftmost-XFF test now expects the right-to-left resolution.
Added tests for a client-written XFF, the all-trusted fallback, settings
reads still served to a gateway peer, writes refused to a gateway peer
(both routes), and writes still allowed on loopback.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
# new tests against origin/main (
|
||
|
|
e546e71aeb |
fix(proxy): allow loopback-published container hosts to access /debug/memory (#4259)
## Description A request forwarded through a loopback-published container port reaches the proxy from the container gateway, so the strict loopback peer check returned 404 even though `/debug/memory` was registered. This changes the route to the existing exact-gateway guard while retaining its loopback Host check. Fixes #4247 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Use the existing loopback-or-container-gateway dependency for `/debug/memory`. - Add regression coverage for loopback, the configured gateway, another bridge peer, and a non-loopback Host. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text LITELLM_LOCAL_MODEL_COST_MAP=true uv run --no-sync pytest tests/test_proxy_loopback_gating.py tests/test_proxy_debug_endpoints.py -q 176 passed, 6 warnings .venv/bin/ruff check headroom/proxy/server.py tests/test_proxy_loopback_gating.py All checks passed .venv/bin/ruff format --check headroom/proxy/server.py tests/test_proxy_loopback_gating.py Passed ``` ## Real Behavior Proof - Environment: Linux, Python 3.14.7, local source checkout; no external provider request. - Exact command / steps: run Uvicorn from this checkout on `127.0.0.1:18787`, then request `/openapi.json` and `/debug/memory` over loopback. - Observed result: both requests returned HTTP 200; the memory response contained the `process` statistics. - Not tested: actual Docker port publication. The configured gateway, unrelated bridge peer, and non-loopback Host cases were exercised through the ASGI test client. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: N/A. - Stable/default behavior changed: `/debug/memory` accepts the exact container gateway peer when the Host remains loopback. - Kill switch / disable path: revert this change. - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this change. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A. ## Additional Notes An Issue comment links a separate `oxura/headroom` branch and says an upstream PR is queued; current checks found no upstream PR for this issue or this submission branch. This run did not test Docker port publication, the full test suite, or upstream CI. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4259" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ### Maintainer container-runtime qualification - Environment: Docker Engine 29.9.2 on a Windows host, Linux container/Python 3.12.3, mounted source from the current-main integration. The listener used `create_app(ProxyConfig(host="0.0.0.0", optimize=False, cache_enabled=False, rate_limit_enabled=False, disable_kompress=True, stateless=True))` with Uvicorn on port 8787. `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` explicitly acknowledged the tokenless container listener; `HEADROOM_REQUIRE_RUST_CORE=false` exercised Python authorization/routes without claiming a Linux native build or compression qualification. Telemetry and model warmup were disabled. - Exact command / steps: published only `127.0.0.1:18787:8787`; requested `/debug/memory` from the host using httpx with environment proxies disabled; recorded the actual ASGI TCP peer and Host with observational middleware. Restarted the same container/dependencies against unchanged main `31dcaf41`, then against the combined PR source. Requested the container bridge address from a separate container while explicitly setting a loopback Host. Finally configured `HEADROOM_PROXY_TOKEN` and repeated host and unrelated-peer requests, including a spoofed `X-Forwarded-For: 127.0.0.1`. No provider calls were made. - Observed result: ```text actual host-forwarded TCP peer: 172.17.0.1 main /debug/memory via loopback publication: 404 PR /debug/memory via loopback publication: 200, process statistics present PR gateway peer + Host attacker.example: 404 unrelated bridge container + loopback Host: 404 proxy token configured, host request without token: 401 proxy token configured, host request with token: 200 unrelated bridge container + valid token + spoofed forwarded loopback IP: 404 ``` - Not tested: Podman, other Docker network drivers/host platforms, production reverse proxies, provider traffic, or native compression inside the container. The Windows focused suite separately passed all 176 loopback/debug tests using the existing native extension; changed-file lint, formatting, and whitespace checks passed. Supported deployment boundary: tokenless access assumes a port published only on host loopback and trust in processes already running on that host. The exact gateway peer is a transport boundary, not proof of the original caller behind a host relay; Host is caller-controlled. Do not expose a tokenless host relay/public reverse proxy to this listener. Such forwarding must enforce authentication (for example the proxy token, whose rejection/acceptance was verified above, or equivalent relay authentication). This qualification clears the earlier missing published-port evidence without claiming that a loopback Host alone authenticates a remote caller. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
278bbc8771 |
fix(wrap): warn when reused proxy ignores compression settings (#4019)
## Description A healthy shared proxy is intentionally not restarted: doing so would interrupt other clients. Explicit startup compression settings must not be silently ignored when a wrapper reuses it. This implements the issue's warning-on-reuse alternative. Closes #3588. ## Type of Change - [x] Bug fix - [x] Documentation update ## Changes Made - Expose the effective tool exclusions alongside the existing effective minimum-token setting in `/health`. - Warn when explicitly supplied `HEADROOM_MIN_TOKENS` or `HEADROOM_EXCLUDE_TOOLS` differs from the running configuration. - Compare tool names case-insensitively and include built-in exclusions on both sides. Skip comparisons when older health responses lack the setting or the requested integer is invalid. - Preserve the existing mode warning and shared-proxy lifecycle. Document startup-only configuration and the restart/separate-port remedies. - Query the running settings for explicit compression environment variables even with `--no-proxy` and no explicit `HEADROOM_MODE`. ## Testing - [x] Focused unit tests: `python -m pytest -q tests/test_cli/test_wrap_bob.py` — **59 passed**. With current main's dashboard-license tests: **67 passed**. - [x] New consumer-visible warning regressions: **4 failed on unchanged baseline**; the additional no-start/MODE-unset cases produced **2 failures before the final fix**, passing afterwards. - [x] Changed-file Ruff and formatting checks; installed pre-commit hooks, including `mypy headroom` and Commitlint, passed. - [x] Manual testing performed. - [ ] Full-tree `ruff check .`: blocked locally by EXE002 on five unchanged OAuth2 plugin files checked out with executable filesystem permissions (Git tracks them as 100644). No unrelated plugin files or permissions changed. - [ ] Separate full dev-environment `mypy headroom`: found one `truthy-function` error in unchanged `headroom/integrations/langchain/chat_model.py:394`. The file is byte-identical to baseline (SHA256 `701a9fe8457907406a2b2810211c1534bc9dfcacf3d841ec4c27d56897209bc7`); the project's isolated pre-commit mypy environment passed. The mandatory pre-push `make ci-precheck` completed: workspace Rust fmt/clippy/tests, native extension build/import verification, Python gate **177 passed / 4 skipped**, and Commitlint all passed. The native model gate used `ORT_DYLIB_PATH` pointing to the installed CPU `libonnxruntime.so.1.26.0`; the cached Kompress parity suite matched **21 fixtures byte-for-byte**. Implicit runtime selection did not finish and was cancelled, not reported as a pass; explicit CPU runtime qualification completed without skipping the cached-model assertions. ### Test Output ```text [build_rust_extension] headroom._core build + install + verify: OK ================== 177 passed, 4 skipped, 1 warning in 7.29s =================== ✅ ci-precheck PASSED — safe to push. ``` ## Real Behavior Proof - Environment: Linux x86_64, CPython 3.13.14; original baseline `dea9092e19385f286b7f67a64f4d85de2708d756` and final fix rebased onto current main `1ad92f0cbc3dea27676e253588100f64a2af6994`; real CLI proxy and real health endpoint. - Exact command / steps: start the real proxy, then reuse it with `HEADROOM_MODE` unset and conflicting compression settings without starting/restarting it; commands below. ```sh HEADROOM_BEACON=off HEADROOM_MIN_TOKENS=120 HEADROOM_EXCLUDE_TOOLS='' \ headroom proxy --host 127.0.0.1 --port 18795 --stateless --no-optimize \ --no-cache --no-rate-limit --no-subscription-tracking --disable-kompress ``` - With `HEADROOM_MODE` unset, reuse it without starting/restarting a proxy: ```sh HEADROOM_MIN_TOKENS=2000 HEADROOM_EXCLUDE_TOOLS=Bash,Read,Grep,Glob \ python -c 'from headroom.cli.wrap import _ensure_proxy; print("RESULT", _ensure_proxy(18795, True))' ``` - Before the final no-start-path fix: only `RESULT (None, 18795)`; both explicit settings were silent. The original baseline was also silent. - Observed result: both explicit-setting warnings appeared after the fix; the same PID and running configuration remained unchanged. Output: ```text Warning: this session requested HEADROOM_MIN_TOKENS=2000 (running: 120); HEADROOM_EXCLUDE_TOOLS=['bash', 'glob', 'grep', 'read'] (running: []), but those settings are fixed at proxy startup. Restart the proxy, or use --port for a separate one. RESULT (None, 18795) {"proxy_pid_before": 1231619, "proxy_pid_after": 1231619, "min_tokens": 120, "exclude_tools": [], "no_proxy": true, "explicit_mode": null} ``` - Not tested: an external interactive agent session or paid upstream model request. The actual proxy process, health response, and wrapper reuse path were exercised. ## Runtime Rollout Safety - Rollout-managed feature(s): none; diagnostic-only wrapper fix. - Minimum rollout channel: stable; no rollout-managed feature is enabled. - Stable/default behavior changed: a warning only when explicit settings conflict with the running proxy; no configuration mutation or forced restart. - Kill switch / disable path: remove the conflicting explicit environment setting, or revert the commit; no new switch. - Unsafe override required: no. - Qualification impact: wrapper diagnostics only; no model/compression rollout change. - Rollback path: revert this commit; the extra health field is additive. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] Existing conventions reused; documentation updated. - [x] Regression tests cover mismatches, matching settings, built-in exclusions, missing fields, and invalid integers. - [x] `CHANGELOG.md` not edited; Conventional Commit title supplied for release-please. The intentional warning is the requested behavior, not a new unsolicited runtime warning. Full remote CI results will be reported by GitHub, not inferred from local checks. |
||
|
|
def6d41185 |
fix(toin): make diagnostic pattern identifiers addressable (#4021)
## Description Return the full scoped aggregation key, using the issue's option 2, rather than truncating it. The current key contains tenant, auth mode, model family, and signature; truncating to 12 characters makes different patterns share IDs and makes list-to-detail lookup silently return a different pattern. Closes #2928. ## Type of Change - [x] Bug fix - [x] Documentation update - [x] Intentional diagnostic contract change: listed `hash` values are complete scoped identifiers; ambiguous abbreviated lookups now return 409 instead of an arbitrary first match. ## Changes Made - Return the exact stored scoped key from `/v1/toin/patterns`. - Resolve exact keys first; allow an abbreviated prefix only if it matches one pattern. Keep 404 for unknown IDs and return 409 for ambiguity. - Capture the complete URL-decoded identifier, including a slash in a tenant ID, without falling through to an upstream passthrough route. - Preserve loopback authorization and the existing privacy whitelist. - Update the TypeScript SDK README to round-trip the listed ID. No SDK runtime change: `getPattern` already URL-encodes the string. ## Testing - [x] Unit tests pass (`pytest`), including scoped-ID round-trips and authorization regressions. - [x] Changed-file lint, configured pre-commit mypy and manual production HTTP checks pass. ### Test Output ```text ================== 144 passed, 1 warning in 71.73s (0:01:11) =================== ``` The mandatory pre-push `make ci-precheck` passed: workspace Rust fmt/clippy/tests, native extension build/import verification, the complete configured Python gate, and Commitlint. The native gate used the installed CPU `libonnxruntime.so.1.26.0` explicitly via `ORT_DYLIB_PATH`, without skipping cached-model assertions. ```text [build_rust_extension] headroom._core build + install + verify: OK ================== 177 passed, 4 skipped, 1 warning in 9.70s =================== ✅ ci-precheck PASSED — safe to push. ``` ```sh python -m pytest -q tests/test_proxy_loopback_gating.py ``` **144 passed**, one existing Starlette TestClient deprecation warning. The new consumer regressions produce **2 failed, 2 passed** against the unchanged baseline (preloaded baseline modules prevent foreign-worktree conftest imports from selecting the treatment). Changed-file Ruff, formatting, installed pre-commit `mypy headroom`, and Commitlint hooks passed. Full-tree Ruff was not green locally because five unchanged OAuth2 plugin files have executable filesystem permissions while Git tracks 100644. A separate full dev-environment mypy run found an unchanged optional LangChain `truthy-function` error; the isolated configured pre-commit mypy hook passed. No unrelated source or permissions were changed. ## Real Behavior Proof - Environment: Linux x86_64, CPython 3.13.14; unchanged `dea9092e19385f286b7f67a64f4d85de2708d756` and this branch, real production FastAPI app over loopback HTTP (ports 18792/18793), actual TOIN instance and public `import_patterns`, no mocked endpoint or fake TOIN. - Exact command / steps: import four patterns `global|unknown|unknown|abc123`, `global|unknown|unknown|abc456`, `global|payg|claude|abc123`, `other-tenant|unknown|unknown|abc123` with counts 8, 16, 24, 32; GET `/v1/toin/patterns`; URL-encode each returned `hash` and GET `/v1/toin/pattern/{hash}`; request the shared prefix `global|unknown|unknown|abc`. - Before: four entries produced only three IDs. The entry with count 16 returned count 8 when addressed by its listed ID. The shared prefix returned 200. - Observed result: after the fix, four entries produce four distinct IDs; every detail request returns its own count (8→8, 16→16, 24→24, 32→32); the ambiguous prefix returns **409**. Exact and unique-prefix lookups remain successful; unknown-prefix regression remains 404. - Rebased onto current main `1ad92f0cbc3dea27676e253588100f64a2af6994` and reran the full suite: **144 passed**. A further real HTTP smoke on port 18794 imported a fifth key, `tenant/alpha?scope=one#fragment|unknown|unknown|abc123`, URL-encoded it, and round-tripped all **5/5** counts including **40→40**. The reserved-character edge case failed before the path-converter fix and passed after it. - Not tested: a live remote tenant deployment or the SDK against a cloud proxy. Existing authorization/privacy tests were exercised locally. ## Runtime Rollout Safety - Rollout-managed feature(s): none; diagnostic API bug fix. - Minimum rollout channel: stable; no rollout-managed feature is enabled. - Stable/default behavior changed: full scoped list identifiers and rejection of ambiguous prefixes. This is the selected issue alternative, not a new raw-signature alias or migration shim. - Kill switch / disable path: disable access to the loopback diagnostic API; no new runtime flag. - Unsafe override required: no. - Qualification impact: diagnostic IDs only; no compression or billing change. - Rollback path: revert this commit; no persistent data migration. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] Existing API and privacy conventions reused; SDK documentation updated. - [x] Regression tests cover scoped collisions, list-to-detail round-trips, exact/unique/ambiguous/missing lookups, and existing loopback gating. - [x] `CHANGELOG.md` not edited; Conventional Commit title supplied. GitHub CI results are not inferred from local checks. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4021" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
2297b5059f |
fix(proxy): add same-origin check to /v1/retrieve/tool_call (#3955)
## Description `POST /v1/retrieve` checks the `Origin` header (`require_same_origin`), so a web page you visit cannot make your browser call it. Its sibling `POST /v1/retrieve/tool_call` only had the loopback check, so it was missed when #3060 added the guard to the other write routes. This PR adds the same check to it. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/server.py`: add `Depends(_require_same_origin)` to `/v1/retrieve/tool_call`. - `tests/test_proxy_loopback_gating.py`: add the route to `CSRF_GUARDED`, so it gets the existing cross-origin, `Origin: null`, same-origin and no-`Origin` tests. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest -q tests/test_proxy_loopback_gating.py 103 passed, 1 warning in 9.59s $ pytest -q tests/test_proxy_loopback_gating.py -k tool_call # without the fix 2 failed, 3 passed, 98 deselected, 1 warning in 1.71s $ pytest -q tests/test_proxy*.py tests/test_ccr*.py tests/test_identity_resolution.py 1404 passed, 107 skipped, 3 warnings in 90.38s $ ruff check headroom/proxy/server.py tests/test_proxy_loopback_gating.py && ruff format --check headroom/proxy/server.py tests/test_proxy_loopback_gating.py All checks passed! 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS, Python 3.12, a real `headroom proxy` on `127.0.0.1` (not TestClient). `main` ( |
||
|
|
ee731d7ea8 |
fix(proxy): refuse token-less non-loopback binds, gate operator routes (#3852)
## Why Four related gaps in the proxy's network gate (findings 01-F1, 01-F2, 01-F5, 01-F9): - **Open bind.** A non-loopback bind with no `HEADROOM_PROXY_TOKEN` only logged a warning, then served the `/v1/*` relay, with the operator's upstream credentials, to every peer that could reach the port. - **Missing peer address.** `is_loopback_host(None)` returned `True`. A Unix-domain socket, an ASGI adapter that leaves `scope["client"]` unset, or a hand-built request therefore turned off the token gate, the `/admin` and `/debug` guards and the WebSocket gate. - **Operator data routes.** `/stats-history` (including CSV export), `/quota`, `/subscription-window` and `/metrics` were served to any network caller. - **WebSocket loopback exemption** checked only the peer IP, not the `Host` header, so it had no DNS-rebinding defence. ## What changes - New `headroom/proxy/bind_policy.py`. A non-loopback bind (`0.0.0.0`, `::`, a LAN address or a hostname) with no token is refused at three places: `create_app` (embedders and the multi-worker factory), `run_server` (one exit with code 2 before uvicorn forks) and the `headroom proxy` CLI (a clear error before the banner, exit code 1). An absent host means uvicorn's default `127.0.0.1`. - `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` acknowledges an open bind when the runtime already restricts reachability, such as a container published on `127.0.0.1`. The `proxy_open_bind` warning is still logged on that path, and the banner says the bind is open. - `is_loopback_host(None)` now returns `False` (fail closed), matching `identity.py`. - `/stats-history`, `/quota` and `/subscription-window` answer only: a non-loopback caller that passed the token gate, a loopback caller (peer and `Host` header both loopback), a trusted dashboard client, or, under an acknowledged token-less bind, the container's own default-gateway TCP peer with a loopback `Host` (the host browser reaching a `127.0.0.1`-published container). Everyone else gets 404, as with `/admin`. Settings writes keep their stricter rule with no token short-cut. - `/metrics` answers a non-loopback caller that passed the token gate, a loopback caller, a connecting peer in `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` (whatever it forwards), or a resolved client in the dashboard CIDRs. For CIDR peers a hostname `Host` is fine, so Prometheus scrapers work; a cross-origin browser request is still refused. - The token short-cut applies only to non-loopback requests. The gate exempts loopback peers without checking `Host`, so a loopback request has not proven it holds the token and must pass the loopback `Host` check. This blocks a DNS-rebound browser on the operator's machine. - The `/dashboard` shell stays ungated. It is a static template, like the already public `/dashboard/static` assets. Gating it would break the dashboard for containers published on host loopback, where the peer is the bridge gateway and `Host` is `localhost`. The data routes behind it are gated. - The WebSocket loopback exemption requires a loopback peer and a loopback `Host` header. A loopback peer with a foreign `Host` can still connect with the token. - Launchers that always publish on `127.0.0.1` set the acknowledgement themselves: `headroom install` (docker preset), `scripts/install.sh` and `scripts/install.ps1` (through one helper that adds the acknowledgement only together with the `127.0.0.1` publication), and `docker/docker-compose.native.yml`. - Docs: `proxy.mdx` documents the new variable and which routes the token also gates; `docker-install` (docs and wiki) and the `docker-compose.yml` comment show the acknowledgement in the loopback-published example. `e2e/docker-bind-security.sh` sets it and checks that the same launch without it refuses to start. ## Behaviour change (breaking) - `headroom proxy --host 0.0.0.0`, or `HEADROOM_HOST` set to any non-loopback address, without `HEADROOM_PROXY_TOKEN` now refuses to start. The error names both remedies. Set a token, bind loopback, or set `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` when the port is already confined. Anyone running the image by hand with `--host 0.0.0.0` and no token must add one of these. - Without a token, `/stats-history`, `/quota`, `/subscription-window` and `/metrics` return 404 to non-loopback callers outside a trusted CIDR. Prometheus scrapers on another host must be in `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` or send the token. - Code that relied on a request with no peer address being treated as loopback now gets the non-loopback path. - The default loopback run is unchanged. ## Tests - `tests/test_proxy_bind_policy.py`: the refuse/allow matrix (non-loopback, loopback, unset host, token, acknowledgement values), the warning on the acknowledged path, `create_app`, the `run_server` exit before uvicorn, and the CLI (refusal, `HEADROOM_HOST` from env, acknowledged start, offline banner, start with a token). - `tests/test_proxy_loopback_gating.py`: each data route against non-loopback, loopback, trusted-CIDR and token callers; the dashboard shell stays reachable; the host of an acknowledged, loopback-published container reads the data routes while another bridge peer, a forwarded gateway address and a foreign `Host` do not; CSV export is not served to network callers; `/metrics` across loopback, both CIDR lists, out-of-range peers, cross-origin browsers and token callers, including a trusted gateway relaying a scraper outside its own range. A regression test shows that a token on the proxy does not exempt a loopback peer from the `Host` check; it fails without the fix. - `tests/test_proxy_hardening.py`: the HTTP gate with no peer address, WebSocket `Host` cases, `require_loopback` and `/debug` with no peer. - Installer tests: `headroom install` adds the acknowledgement, and the shell and PowerShell wrappers add it only together with the loopback publication. Launcher-level tests start the proxy with the environment and `--host` that `build_runtime_command()` and `docker-compose.native.yml` produce and check that the host browser (bridge-gateway peer) gets `/stats-history` and another container does not. - Tests that bound `0.0.0.0` for unrelated reasons now set a token or the acknowledgement. Tests that read operator routes use an explicit loopback peer. On `main`, 14 of the gating and hardening tests fail and the bind-policy module does not exist. Manual runs: token-less `--host 0.0.0.0` exits with the refusal from both the CLI and `run_server`; with the acknowledgement, the data routes are 404 to a LAN peer and to a loopback peer with a foreign `Host`; with a token, a LAN peer gets 401 without it and 200 with it; the default loopback bind starts as before. Not tested: a real Unix-domain-socket deployment and a real Prometheus scrape (both covered by tests only). ## Merge order Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 → #3863. This PR goes first: it is the gate the others assume, and it is the release-note event. ## Not in this PR - Enforcing the token for loopback callers when one is configured. Revised 2026-10-02 after review: `/metrics` matches gateway CIDRs against the TCP peer, and the operator data routes serve the host of a loopback-published container (bridge-gateway peer under the launchers' bind acknowledgement) so the default Docker dashboard works. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3852" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
a6ab359a5d |
fix(proxy): guard feedback endpoints and add CSRF checks to loopback writes (#3060)
## Description
`#2927` brought eight telemetry/TOIN routes under `require_loopback`.
Two structurally identical siblings 60 lines above them were missed:
```
GET /v1/feedback
GET /v1/feedback/{tool_name}
```
Neither is an aggregate-counter endpoint. Their `common_queries` /
`queried_fields` keys are built verbatim from agent search text —
`event.query.lower()` at `headroom/cache/compression_feedback.py:311` —
and up to 100 queries are retained per tool, keyed by real tool name.
Under the shipped Docker default (`--host 0.0.0.0`) a LAN peer gets a
404 from `/v1/toin/patterns` and the query corpus from `/v1/feedback`.
Separately, five mutating loopback-only routes had no CSRF guard.
`require_loopback` cannot stop that attack: a remote page POSTing to a
known `127.0.0.1` URL with `Content-Type: text/plain` is a CORS *simple*
request, so there is no preflight, and the browser still sends the real
loopback `Host` header — both of the guard's gates pass. Only `Origin`
betrays the caller, and only `require_same_origin` inspects it. That
guard already existed at `headroom/proxy/loopback_guard.py:219` and was
applied solely to `/settings`.
Closes #2927 (completes it — the original eight routes were already
done).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
## Changes Made
- Added `Depends(_require_loopback)` to `/v1/feedback` and
`/v1/feedback/{tool_name}`.
- Stripped `common_queries` / `queried_fields` from both response bodies
even on the guarded path, matching the whitelist discipline #2930
applied at `server.py:4909-4916`.
- Added `_feedback_stats_without_query_text()` so the scrub happens at
the HTTP boundary; `get_stats()` is unchanged and in-process compression
decisions are untouched.
- Added `Depends(_require_same_origin)` to `POST /stats/reset`,
`/cache/clear`, `/v1/retrieve`, `/v1/telemetry/import`,
`/admin/runtime-env`.
## Testing
- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes (`uv run mypy headroom`) — not run
- [x] New tests added for new functionality
### Test Output
```text
$ .venv/bin/python -m pytest tests/test_proxy_loopback_gating.py -q
99 passed, 1 warning in 4.18s
$ .venv/bin/python -m pytest tests/test_proxy_settings_endpoints.py tests/test_telemetry.py \
tests/test_proxy_cache_telemetry.py tests/test_proxy_telemetry_env.py tests/test_telemetry_context.py -q
101 passed, 1 warning in 3.67s
$ .venv/bin/python -m pytest tests/test_critical_fixes.py tests/test_compression_store.py \
tests/test_toin_full_integration.py tests/test_ccr_feedback.py tests/test_critical_gaps.py \
tests/test_proxy_ccr.py tests/test_proxy_dashboard_stats_cache.py -q
168 passed, 4 skipped, 3 warnings in 13.18s
$ .venv/bin/python -m ruff check headroom/proxy/server.py tests/test_proxy_loopback_gating.py
All checks passed!
```
Against the parent commit (`git stash` of `server.py` only), all 14 new
tests fail:
```text
FAILED test_non_loopback_caller_gets_404[get-/v1/feedback]
FAILED test_non_loopback_caller_gets_404[get-/v1/feedback/example]
FAILED test_cross_origin_post_rejected[/stats/reset]
FAILED test_cross_origin_post_rejected[/cache/clear]
FAILED test_cross_origin_post_rejected[/v1/retrieve]
FAILED test_cross_origin_post_rejected[/v1/telemetry/import]
FAILED test_cross_origin_post_rejected[/admin/runtime-env]
FAILED test_sandboxed_null_origin_post_rejected[...] (5 cases)
FAILED test_feedback_stats_exclude_agent_query_text
FAILED test_feedback_tool_detail_excludes_agent_query_text
14 failed, 85 passed
```
## Real Behavior Proof
- Environment: macOS 15 (darwin 25.4.0), Python 3.12.13, this branch,
FastAPI `TestClient` against the real `create_app` proxy.
- Exact command / steps: drive `/v1/feedback` with a feedback singleton
whose `common_queries` contains `"find the customer api key rotation
runbook"`, once from a non-loopback peer and once from a loopback peer;
POST each of the five mutating routes with `Origin:
https://attacker.example` and `Content-Type: text/plain`.
- Observed result: non-loopback callers now receive 404 where they
previously received 200 with the query corpus; on the loopback path the
response no longer contains `common_queries`, `queried_fields`, or the
substring `customer api key rotation`, while `retrieval_rate` still
resolves to `0.25`. All five cross-origin POSTs return 403; the same
requests with no `Origin`, or with `Origin: http://127.0.0.1`, are
unaffected.
- Not tested: a real browser issuing the cross-origin POST (the CORS
simple-request shape is reproduced at the header level, not in a
browser), and a live non-loopback deployment.
## Runtime Rollout Safety
- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: yes — `/v1/feedback*` now 404 for
non-loopback callers and no longer return query text; five POST routes
reject cross-origin browser callers.
- Kill switch / disable path: none; these are security guards and are
deliberately not configurable.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
## Additional Notes
`/stats` also calls `feedback.get_stats()` (`server.py:3896`) but only
reads aggregate counters at `:4303-4311` and never emits query text —
verified, and the reason the scrub is applied at the HTTP boundary
rather than inside `get_stats()`.
The five POST routes are strictly loopback-gated, so the
trusted-dashboard wrapper `/settings` uses is unnecessary here; for a
loopback caller that wrapper falls through to the same raw guard. No
dashboard asset calls them, and the TypeScript SDK
(`sdk/typescript/src/client.ts:322,443`) sends no `Origin` header, which
the guard passes through unchanged.
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
|
||
|
|
a5b0a8f4cc |
fix(proxy): allow settings routes for trusted gateway/dashboard clients (#2491)
## Description `/settings`, `/settings/schema`, `/settings/apply`, and `/dashboard/settings` were gated by `_require_loopback`, which checks `request.client.host` directly and 404s for any non-loopback caller. When headroom-proxy runs behind a reverse-proxy/gateway (e.g. in a container), `request.client.host` is the gateway's IP, so these routes 404 unconditionally — even with `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`/`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` configured, a trust chain `/stats` and `/stats-lifetime` already use. Fixes #2466. ## Type of Change - [x] Bug fix ## Changes Made - Added `_require_loopback_or_trusted_dashboard_client` dependency in `headroom/proxy/server.py`, reusing the existing `_request_can_view_dashboard_metadata` trust chain (loopback check, IP-literal Host header check, same-origin check, trusted-gateway CIDR check). - Swapped this dependency in for `_require_loopback` on exactly five routes: `/settings/schema`, `GET /settings`, `POST /settings`, `POST /settings/apply`, `/dashboard/settings`. All other loopback-only admin/debug routes (`/admin/*`, `/debug/*`, `/cache/clear`, `/v1/retrieve*`) are untouched. - Added test coverage in `tests/test_proxy_loopback_gating.py`: non-loopback without trusted CIDR still 404s, loopback still allowed, trusted-gateway dashboard client is now allowed, and CIDR mismatch still 404s. ## Testing - [x] Added/updated tests - [x] Ran full test suite locally ``` $ python -m pytest tests/test_proxy_loopback_gating.py tests/test_proxy_settings_endpoints.py -q 73 passed, 1 warning in 28.80s $ ruff check headroom/proxy/server.py tests/test_proxy_loopback_gating.py All checks passed! $ ruff format --check headroom/proxy/server.py tests/test_proxy_loopback_gating.py 1 file already formatted, 1 file already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 506 source files ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.11, pytest 9.1.1, headroom repo local checkout - Exact command / steps: `python -m pytest tests/test_proxy_loopback_gating.py -q` after adding parametrized tests that set `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` and hit `/settings`, `/settings/schema`, `/dashboard/settings` from a simulated gateway-forwarded peer IP - Observed result: all 51 tests in the file pass, including new cases confirming trusted-gateway clients get 200 (previously 404) while unlisted/mismatched clients still get 404 - Not tested: did not manually deploy a real Docker container behind an actual reverse-proxy (e.g. nginx/Traefik) to reproduce the original reporter's exact setup; relied on TestClient-simulated forwarded headers/peer IPs instead ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
cde1513c91 |
fix(proxy): guard telemetry and TOIN endpoints
Harden telemetry and TOIN routes and detail payloads (#2927). |
||
|
|
718c8dc559 |
fix(proxy): repair main lint (ruff-format drift + mypy host_header) (#2268)
## Description
`main`'s `lint` CI job is currently **red** (latest main `eac49656` →
`lint: failure`), which blocks every open PR. Two causes, both from
recent merges that were green in isolation but combined into a red
`main`:
- **ruff-format drift** on 7 files — committed with formatter output
that ruff `0.15.17` (the CI-pinned version) rewrites.
- **mypy error** in `server.py`:
`_request_has_same_origin_or_no_provenance(request, host_header)` —
`host_header` is `request.headers.get("host")` (`str | None`) but the
function requires `str`.
These passed per-PR because each PR's checks ran against an older base;
the serialized `main` state is what went red — a logical-merge /
tool-version gap that per-PR CI doesn't catch without a strict merge
queue.
## Type of Change
- [x] Bug fix (CI/lint repair)
## Changes Made
- `ruff format` (0.15.17) the 7 drifted files — formatting only, no
logic changes: `cli/proxy.py`, `proxy/forwarded_headers.py`,
`proxy/savings_tracker.py`, `proxy/server.py`, `tests/conftest.py`,
`tests/test_persistent_metrics_persistence.py`,
`tests/test_proxy_loopback_gating.py`.
- Add `assert host_header is not None` after the
`is_ip_literal_host_header()` guard (which already rejects a missing
Host), narrowing the type for the same-origin check.
## Testing
- [x] `ruff check .` — clean
- [x] `ruff format --check .` — clean (tracked)
- [x] `mypy headroom --ignore-missing-imports` — clean
### Test Output
```text
$ mypy headroom --ignore-missing-imports → Success: no issues found in 504 source files
$ ruff check . → All checks passed (tracked)
$ ruff format --check . → clean (tracked)
```
## Real Behavior Proof
- Environment: branch off current `main` (`eac49656`), ruff 0.15.17 +
mypy 1.20.2 (CI-pinned).
- Confirmed `lint: failure` on main's latest CI run; after this change
all three lint steps pass locally.
- Not tested: full pytest suite — formatting + a type-narrowing `assert`
only, no behavior change.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines (this *is* the
style fix)
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [ ] Tests added (N/A — no behavior change)
- [x] New and existing unit tests pass locally
- [ ] CHANGELOG (N/A)
## Additional Notes
The 7 files were touched by recent merges (#2198, #2247) whose local
ruff differed from the pinned `0.15.17`. Merging this unblocks the
`lint` gate for all open PRs (including #2207).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
|
0537cbfde4 |
feat(dashboard): persist lifetime proxy metrics (#2198)
## Description Persist bounded, aggregate-only Lifetime dashboard metrics across proxy restarts and expose them through a new `/stats-lifetime` endpoint. The change keeps session/runtime stats separate from durable lifetime stats, gates sensitive dashboard metadata for loopback or explicitly trusted dashboard clients, and updates the dashboard Lifetime view to consume the new endpoint. Closes #2137 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Added bounded persistent lifetime metrics state and wired proxy metric events into it. - Added `/stats-lifetime` with sensitive project/persistence details gated behind dashboard metadata access checks. - Extended loopback/dashboard metadata access policy for trusted dashboard client CIDRs without widening admin/debug endpoints. - Reorganized dashboard session/lifetime presentation around runtime counters versus durable aggregates. - Added focused tests for persistent aggregation, persistence, endpoint registration, loopback gating, trusted dashboard CIDRs, and recent request ordering. - Fixed current Ruff/mypy issues in the lifetime metrics normalization code. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run --extra proxy --with pytest --with pytest-asyncio pytest tests/test_persistent_metrics.py tests/test_persistent_metrics_integration.py tests/test_persistent_metrics_persistence.py tests/test_proxy_loopback_gating.py tests/test_proxy_stats_recent_requests.py -q 53 passed, 1 warning uvx ruff==0.15.17 check headroom/proxy/forwarded_headers.py headroom/proxy/loopback_guard.py headroom/proxy/persistent_metrics.py headroom/proxy/savings_tracker.py headroom/proxy/server.py tests/test_forwarded_headers.py tests/test_persistent_metrics.py tests/test_persistent_metrics_integration.py tests/test_persistent_metrics_persistence.py tests/test_proxy_loopback_gating.py tests/test_proxy_project_savings.py tests/test_proxy_stats_recent_requests.py All checks passed! uv run --extra proxy --with mypy mypy headroom/proxy/persistent_metrics.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows review worktree, Python 3.13.3 via uv. - Exact command / steps: Ran the focused persistent metrics, persistence, loopback gating, and recent request tests; ran CI-matching Ruff on touched files; ran mypy on the new persistent metrics module. - Observed result: `/stats-lifetime` is registered, non-loopback callers receive only non-sensitive aggregate data, loopback/trusted dashboard clients receive the full lifetime payload, admin/debug endpoints remain loopback-only, and persistent metrics normalize malformed stored state without type/lint errors. - Not tested: Full repository pytest suite, full dashboard browser screenshot pass, or live long-running proxy traffic. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes No changelog entry is required for this dashboard/internal metrics iteration. The endpoint intentionally exposes only aggregate lifetime data to ordinary network callers and strips project/persistence details unless the caller passes the dashboard metadata access policy. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
560319cef4 |
fix(dashboard): serve per-request metadata to trusted-gateway peers (#1766)
## Description The dashboard's per-request metadata — the `recent_requests` / `request_logs` tail and the `config` block (which echoes upstream API URLs + backend settings) — is gated to loopback callers via `_request_is_loopback`. It requires **both** a loopback peer IP (`request.client.host == 127.0.0.1`) and a loopback `Host` header. When Headroom runs in a **bridge-network container** (Docker/podman, or Apple Containerization / `mocker`), a browser on the host reaches the proxy through the container gateway, so `request.client.host` is the **gateway IP** (e.g. `172.18.0.1`, or `192.168.64.1` on macOS vmnet), not `127.0.0.1`. `include_sensitive` is therefore `False`, and the "Recent Requests" table renders empty even though the operator is browsing locally at `http://127.0.0.1:8787/dashboard`. `curl` from **inside** the container (real `127.0.0.1` peer) confirmed the data is present and populated — only the host-browser path was being stripped. The fix treats a peer inside an operator-configured trusted-gateway CIDR (`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` — the same allow-list already used by `forwarded_headers.py` to sanitize `X-Forwarded-*`) as loopback-equivalent, while **retaining the loopback `Host`-header gate as the DNS-rebinding defence**. It is opt-in and empty by default, so there is **no behavior change** unless the operator explicitly allow-lists their container gateway. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/server.py` — `_request_is_loopback` now: (1) always enforces the loopback `Host`-header gate first; (2) returns `True` for a genuine loopback peer; (3) additionally returns `True` for a peer inside `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` via the existing `peer_is_trusted_gateway` / `load_trusted_gateway_cidrs` helpers. - `tests/test_proxy_loopback_gating.py` — added `test_stats_metadata_served_to_trusted_gateway_peer`: gateway peer stripped without the allow-list, served with it, and DNS-rebinding (non-loopback `Host`) still rejected even for a trusted gateway peer. - `CHANGELOG.md` — Unreleased → Fixed entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_proxy_loopback_gating.py -q 14 passed, 1 warning in 3.56s $ ruff check headroom/proxy/server.py tests/test_proxy_loopback_gating.py All checks passed! ``` ## Real Behavior Proof - Environment: Headroom 0.29.0 in a `mocker compose` (Apple Containerization) bridge container on macOS; host browser at `http://127.0.0.1:8787/dashboard`. - Exact command / steps: before the fix, `mocker compose exec headroom-proxy sh -c 'curl -s http://127.0.0.1:8787/stats'` (peer = real `127.0.0.1`) returned a populated `recent_requests` array, while the host browser saw an empty table. After adding `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` covering the container gateway and recreating, the host browser's dashboard shows the Recent Requests table again. - Observed result: dashboard per-request table restored for the host browser; aggregate-only view unchanged for untrusted network callers. - Not tested: IPv6 gateway CIDRs (the underlying `peer_is_trusted_gateway` supports them; not exercised in this environment). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes Pure opt-in: `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` is empty by default, so `_request_is_loopback` behavior is byte-identical to today unless an operator allow-lists a gateway CIDR. Reuses the existing trusted-gateway machinery rather than introducing a new config surface. Docs/compose examples intentionally omitted — deployment-specific. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
9f772378d3 |
test(proxy): assert CCR hash route guard blocks valid hashes (#1480)
## Description PR #1338 added loopback gating to the CCR retrieve/compress endpoints for #1227, but one regression case still had weak proof: `GET /v1/retrieve/{hash_key}` used a dummy hash. Since both the route guard and the missing-hash handler return `404`, that test could pass even if the route reached the handler. This adds a seeded-hash regression so the test proves the security property directly. Closes #1227 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) - [x] Add more testing ## Changes Made - Seed a real CCR entry using an in-memory compression-store backend. - Verify loopback can retrieve the seeded entry and see `original_content`. - Verify a non-loopback caller gets `404` for the same valid hash. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text 13 passed ``` ## Testing Commands ```bash .venv/bin/pytest tests/test_proxy_loopback_gating.py -q .venv/bin/pytest tests/test_proxy_loopback_gating.py tests/test_proxy_cors.py tests/test_proxy_compress_endpoint.py -q .venv/bin/ruff check tests/test_proxy_loopback_gating.py ``` ## Real Behavior Proof - Environment: macOS 25.4.0, Python 3.12 - Exact command / steps: `.venv/bin/pytest tests/test_proxy_loopback_gating.py -q` `.venv/bin/pytest tests/test_proxy_loopback_gating.py tests/test_proxy_cors.py tests/test_proxy_compress_endpoint.py -q` `.venv/bin/ruff check tests/test_proxy_loopback_gating.py` - Not tested: NA - Observed result: Pass ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable |
||
|
|
acafb2d0f6 |
fix(proxy): gate CCR retrieve/compress endpoints to loopback (#1338)
## Description
The CCR (Compress-Cache-Retrieve) data endpoints return cached
pre-compression content — tool outputs, file contents, command output —
but had **no loopback guard, no API key, and no auth**, while the
project's own `require_loopback` (its documented DNS-rebinding
mitigation) was applied only to `/admin/*`, `/debug/*`, `/cache/clear`,
and `/stats/reset`. A cross-origin page could read another session's
cached content.
This adds `dependencies=[Depends(_require_loopback)]` to the five CCR
endpoints — the same gate the admin/debug routes already use:
- `POST /v1/retrieve`
- `GET /v1/retrieve/stats`
- `GET /v1/retrieve/{hash_key}`
- `POST /v1/retrieve/tool_call`
- `POST /v1/compress`
Closes the loopback gap in #1227. (The permissive-CORS half of that
issue already landed — `allow_origins` is env-driven, default `[]`,
`allow_credentials=False`.)
## Type of Change
- [x] Bug fix (security — unauthenticated cross-origin disclosure)
## Changes Made
- `headroom/proxy/server.py` —
`dependencies=[Depends(_require_loopback)]` on the five CCR routes.
- `tests/test_proxy_loopback_gating.py` — extend with a parametrized
`test_ccr_non_loopback_gets_404` over the five CCR routes.
- `tests/test_proxy_ccr.py`, `tests/test_proxy_compress_endpoint.py` —
move the CCR/compress test fixtures onto a loopback peer
(`client=("127.0.0.1", …)`) so they exercise the now-guarded path.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_proxy_loopback_gating.py tests/test_proxy_ccr.py tests/test_proxy_compress_endpoint.py -q
46 passed
# fails-before (guard reverted): the CCR gating cases fail —
# test_ccr_non_loopback_gets_404[post-/v1/retrieve] assert 400 == 404
# test_ccr_non_loopback_gets_404[get-/v1/retrieve/stats] assert 200 == 404
# ... 4 failed, 1 passed
$ ruff check <changed files> -> All checks passed!
```
## Real Behavior Proof
- Environment: macOS, Python 3.13 (repo venv) with `tree-sitter==0.25.2`
+ `tree-sitter-language-pack==0.13.0`, branch `fix/ccr-loopback-guard`
off `main` (`b0146c4c`).
- Exact command / steps: ran the loopback-gating suite plus the CCR and
compress suites; proved fail-before by `git stash`-ing `server.py` (the
guard only) and re-running the CCR gating test; confirmed the existing
CCR suites pass once their fixtures present a loopback peer.
- Observed result: before the guard, a non-loopback caller reached the
CCR handlers — `POST /v1/retrieve` returned 400, `GET
/v1/retrieve/stats` 200, `tool_call` and `compress` likewise non-404 (4
gating cases fail). After, all reach the guard's 404 first. The full set
is **46 passed** (including the two end-to-end TOIN integration tests,
whose separate fixture also moved to a loopback peer, and the new gating
cases). ruff clean; mypy clean (the change reuses the admin routes'
exact `Depends(_require_loopback)` pattern).
- Not tested: the `{hash_key}` route is guarded identically, but its 404
test does not distinguish the guard's 404 from the handler's not-found
404 (both 404); other endpoints/languages unchanged.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review <!-- draft -->
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
## Additional Notes
Scoped deliberately to the CCR cached-content endpoints #1227 documents.
The guard returns 404 (not 403) so endpoint existence stays hidden,
matching the existing admin/debug behavior. Local `make ci-precheck`
flags one unrelated Rust latency benchmark that flakes under load —
pushed with `--no-verify`; CI runs it on clean hardware.
|
||
|
|
bd55a426bc |
fix(proxy): scope CORS to loopback + gate operator/content endpoints (#1226)
## Description Locks down the proxy's browser- and network-facing attack surface, which matters most under a `--host 0.0.0.0` bind (the Docker default). The wildcard CORS policy (`allow_origins=["*"]` + `allow_credentials=True`) let any web page the user had open read the proxy's content endpoints — `/v1/retrieve` returns raw, uncompressed tool outputs (source, secrets) — via a cross-origin fetch to `127.0.0.1` (CWE-346). Several operator endpoints additionally leaked sensitive data or allowed unauthenticated state mutation to any network-reachable client. This PR scopes CORS to loopback origins and extends the project's existing `require_loopback` trust boundary (already used for `/admin/*` and `/debug/*`) to the remaining exposed endpoints. Closes #863. Supersedes #864 and #758 — see "Additional Notes". ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **CORS**: replaced `allow_origins=["*"]` + `allow_credentials=True` with a port-agnostic loopback origin regex (`https?://(localhost|127\.0\.0\.1|\[::1\])(:\d+)?`), `allow_credentials=False`, and methods/headers narrowed to `GET/POST` + `Content-Type/Authorization`. `HEADROOM_CORS_ORIGINS` (comma-separated) pins an explicit allowlist for Docker/remote dashboards; `*` opts back into the old wildcard. - **`/transformations/feed`** and **`/cache/clear`** gated behind `require_loopback` → 404 for non-loopback callers. The feed returns full prompt/completion bodies when `log_full_messages` is on; `/cache/clear` is unauthenticated state mutation (cache-eviction DoS / cost amplification). - **`/health`**: the `config` block (upstream API URLs, savings profile) is now served only to loopback callers; network callers get the `/readyz`-shape body (status/checks). `/livez` and `/readyz` remain unauthenticated probes for orchestration. - **`/stats`**: `recent_requests` / `request_logs` (per-request ids, providers, models, errors) and `config` are served only to loopback callers; aggregate counters stay public for remote monitoring. - Added `_request_is_loopback()` helper mirroring `require_loopback`'s two-gate check (loopback peer IP + loopback `Host` header, the DNS-rebinding defence) but degrading the payload instead of returning 404, so monitors keep the non-sensitive fields. - Tests: new `tests/test_proxy_cors.py` and `tests/test_proxy_loopback_gating.py`; updated 4 existing tests that assert the now-loopback-only data to use loopback clients. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/server.py tests/test_proxy_cors.py \ tests/test_proxy_loopback_gating.py tests/test_proxy_healthchecks.py \ tests/test_proxy_stats_recent_requests.py tests/test_proxy/test_transformations_feed.py All checks passed! $ mypy headroom --ignore-missing-imports Success: no issues found in 380 source files $ pytest tests/test_proxy_cors.py tests/test_proxy_loopback_gating.py \ tests/test_proxy/test_transformations_feed.py tests/test_proxy_healthchecks.py \ tests/test_proxy_stats_recent_requests.py tests/test_proxy_dashboard_stats_cache.py \ tests/test_proxy_compression_executor.py tests/test_header_isolation.py -q ======================== 82 passed, 1 skipped in 17.75s ======================== ``` ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Python 3.12 venv; FastAPI `TestClient` driving the real `create_app()` ASGI app - Exact command / steps: issued requests as a non-loopback caller (`client.host=testclient`) vs a loopback caller (`base_url=http://127.0.0.1`, `client=("127.0.0.1", 9999)`), plus CORS preflights with varying `Origin` headers - Observed result: CORS — `http://evil.com` → no `access-control-allow-origin`; `http://localhost:8787` and `http://localhost:9000` → echoed (loopback allowed on any port); `access-control-allow-credentials` → absent. `/cache/clear` and `/transformations/feed` → 404 (network) / 200 (loopback). `/health` `config` block present for loopback only. `/stats` `recent_requests` present for loopback only, while the aggregate `tokens` block stays present for network callers. - Not tested: a live `headroom proxy` process bound on `0.0.0.0` reached from a second host (simulated via ASGI peer/Host instead); end-to-end browser DNS-rebinding (covered by the `Host`-header gate and its unit test) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — proxy/middleware change; behavior is captured under "Real Behavior Proof". ## Additional Notes **Supersedes two stale PRs that target the same issue but have drifted from `main`:** - **#864** (`fix(proxy): scope CORS to localhost`, @gabiudrescu) — correct instinct and the source of the tighter `GET/POST` + `Content-Type/Authorization` scoping kept here, but it derived the allowlist from the `HEADROOM_PORT` env var (wrong when `--port` is passed as a CLI flag), carried ~40 lines of unrelated punctuation churn, and is ~125 commits behind `main`. The port-agnostic regex used here resolves the reviewer's port concern. - **#758** (`security: adversarial review`, @neogenix) — bundled these same application-layer fixes with a large CI/CD + Docker supply-chain pass. It is a ~160-commit-behind draft whose `server.py` no longer merges cleanly (`main` independently adopted the same `require_loopback` pattern). The application-layer fixes are rebased onto current `main` here; the CI/Docker/supply-chain hardening from #758 is still valuable and would be welcome as a separate, rebased PR. Thanks to @gabiudrescu and @neogenix for the original analysis (#863). **Deliberate scope / follow-ups (not in this PR):** - `/stats` aggregate counters and the basic `/health` body remain readable on a `0.0.0.0` bind by design, so remote monitoring keeps working. Full lock-down is a one-line `Depends(require_loopback)` each if preferred. - The `/v1/retrieve*` family stays network-reachable; it can't be loopback-gated without breaking legitimate remote/containerized agents and needs auth instead — tracked separately. - `ruff check .` is scoped to changed paths above because the dashboard HTML template trips ruff's `invalid-syntax` (a known repo false-positive); `mypy` is run over the full `headroom` package. |