mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): answer /api/hello probes behind the token gate, log rejection context (#4150)
## Description With `HEADROOM_PROXY_TOKEN` set, every Anthropic SDK client logs a `proxy_auth_rejected` line once per process, Claude Code included and even when it is configured correctly. Claude Code sends an unauthenticated `HEAD /api/hello` reachability probe. That probe never carries `ANTHROPIC_CUSTOM_HEADERS`, so it cannot hold the proxy token: it gets a 401, which the client ignores, and the rejection lands in the log. On a busy deployment these lines bury real rejections. Behind a gateway, the rejection line also showed only the gateway's peer address, so a rejected caller could not be identified. This PR: 1. **Answers the probe locally, behind the token.** The HTTP security gate lets GET/HEAD on exactly `/api/hello` (also under `/p/<project>`) past the token check and marks it. Extension middleware still runs, so an extension can still refuse it. The `/api/hello` passthrough route answers a marked probe with an empty 200 (the gate adds the security headers) and never relays it. This only happens when a token is configured. It is deliberately **not** added to `_AUTH_EXEMPT_PATHS`, which would hand the probe to the passthrough relay: an anonymous upstream request (to Anthropic since #4059, to the OpenAI default target before it). Without a token, the probe keeps its #4059 route unchanged. 2. **Gives the rejection line context.** `proxy_auth_rejected` (HTTP and WebSocket) adds `method=`, `ua=` and `xff=` fields. Both headers are caller-controlled, so neither is logged as received; each maps to a fixed vocabulary: - `ua`: the harness name `CLIENT_UA_MAP` gives the User-Agent, the same name the outcome and cost records use (`claude-code`, `codex`, `cursor`, `opencode`, `aider`, `copilot`, …). Failing that, the product token of a generic HTTP client (`node`, `anthropic`, `openai`, `curl`, `mozilla`, `python-httpx`, …). Otherwise `other`. `X-Client` is not consulted, since it is free text. - `xff`: the leftmost `X-Forwarded-For` hop when it parses as a plain IP address, else `invalid`. An IPv6 zone index (`fe80::1%<zone>`) is free text, so it counts as `invalid`. The hop is unverified, used only for attribution, and drives no decision. - Either is `none` when its header is absent. - No credential header is read for the log. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/server.py`: - **`_security_gate`**: when a token is configured, `GET`/`HEAD` on exactly `/api/hello`, matched on the routed path so `/p/<project>/api/hello` counts too, skips the token check and sets `request.state.headroom_local_hello_probe`. It then calls `call_next` like any request, so extension middleware (installed inside the gate) still runs and can refuse it. `POST /api/hello`, `/api/hello/` and `/api/hello/x` still need the token. - **New helper**: `_auth_rejection_context` returns `(ua, xff)` from the fixed vocabulary above: `CLIENT_UA_MAP` then an allowlist of generic clients (`_AUTH_REJECTION_CLIENTS`) for the User-Agent, and `ipaddress.ip_address` on the first hop (from `forwarded_policy.header_first`). - **Both `proxy_auth_rejected` warnings** gain the new fields: the HTTP gate gets `method=`, and the WebSocket middleware gets the same `ua=`/`xff=` fields, so the two transports stay aligned. - `headroom/providers/proxy_routes.py`: the provider passthrough route answers a marked probe with an empty 200 and never relays it upstream. - `tests/test_proxy_hardening.py`: - **Probe answer** (`test_api_hello_probe_is_answered_locally`): GET/HEAD on `/api/hello` and `/p/demo/api/hello` from a non-loopback peer gets 200 with an empty body and security headers, and the passthrough spy shows nothing relayed. `POST /p/demo/api/hello` and `GET /p/demo/stats` still get 401. `POST /api/hello`, `GET /api/hello/x`, `GET /api/hello/` and `GET /stats` still get 401, with nothing relayed. With no token configured, `/api/hello` still reaches the passthrough. - **Log line**: `claude-cli/2.1.0 (external, cli)` with `X-Forwarded-For: 198.51.100.7, 10.0.0.9` logs `ua=claude-code xff=198.51.100.7`. Absent headers log `ua=none xff=none`. `test_known_clients_keep_their_proxy_wide_name` covers `claude-code`, `codex`, `cursor`, `opencode`, `aider` behind a wrapper prefix, the Anthropic SDK and `node`. - **Caller text never reaches the line** (`test_caller_text_never_reaches_the_line`): a `PRIVATE_CREDENTIAL_CANARY` in the User-Agent (bare, as a product token, behind a fake quoted field) and in X-Forwarded-For (as the first hop, as a zone index, as a port suffix) never appears in the line, which logs `other`/`invalid` instead. CR/LF in either header maps to `other`/`invalid` (tested on the helper directly, since an HTTP client refuses to send CR/LF). - **No credentials**: neither the configured token, a wrong `x-headroom-proxy-token`, `Authorization: Bearer …` nor `x-api-key` ever appears in the line. - **WebSocket**: the rejection line matches the HTTP gate's, and the same canary in both headers never appears in it (`test_rejection_log_keeps_caller_text_out`). - `tests/test_proxy_extension_ordering.py`: - **Extension policy** (`test_api_hello_probe_still_runs_extension_policy`): an extension that answers 403 to everything gets the token-less GET/HEAD probe and refuses it. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_proxy_hardening.py tests/test_anthropic_hello_probe_routing.py \ tests/test_provider_proxy_routes.py tests/test_provider_route_specs.py \ tests/test_proxy_loopback_gating.py tests/test_proxy_extension_ordering.py \ tests/test_proxy_bind_policy.py tests/test_auth_mode.py -q 365 passed in 78.96s $ ruff check headroom/proxy/server.py tests/test_proxy_hardening.py All checks passed! $ ruff format --check headroom/proxy/server.py tests/test_proxy_hardening.py 2 files already formatted $ python -m mypy headroom/proxy/server.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: this branch on Windows 11 with Python 3.13. Ran `headroom proxy --port 18787` from the worktree with a throwaway `HEADROOM_PROXY_TOKEN`. - Exact command / steps: `curl -I http://127.0.0.1:18787/api/hello`, `curl http://127.0.0.1:18787/api/hello`, then checked the proxy log. - Observed result: HEAD and GET returned `200` with 0 bytes and `x-content-type-options: nosniff`, `x-frame-options: DENY`, `referrer-policy: no-referrer` and `strict-transport-security`. The proxy log has no `proxy_inbound_request` and no `outbound_headers` line for either probe, so nothing was forwarded. For contrast, a loopback `POST /api/hello` (loopback is token-exempt, existing behavior) logs `forwarder=openai_passthrough`. - Not tested: the new rejection-line fields behind a real gateway. They were checked against a real non-loopback peer, below. Rejection diagnostics, at `d70ec3c` (contains current `main`): the same worktree proxy, bound to a non-loopback LAN address (`--host 192.168.0.48 --port 18787 --no-optimize`, `HEADROOM_OFFLINE=1`) with a throwaway token, called from that address: ```text GET /stats UA=PRIVATE_CREDENTIAL_CANARY XFF=PRIVATE_CREDENTIAL_CANARY, 192.0.2.1 -> 401 GET /stats UA=claude-cli/2.1.0 (external, cli) XFF=198.51.100.7, 10.0.0.9 -> 401 GET /stats UA=codex-cli/0.46.0 -> 401 GET /stats UA=PRIVATE_CREDENTIAL_CANARY/9 opencode/1.2.3 -> 401 GET /stats no UA, no XFF -> 401 GET /stats UA=PRIVATE_CREDENTIAL_CANARY/1.0 XFF=fe80::1%PRIVATE_CREDENTIAL_CANARY -> 401 GET /stats valid x-headroom-proxy-token -> 200 HEAD /api/hello -> 200 WS upgrade /v1/responses UA=PRIVATE_CREDENTIAL_CANARY XFF=PRIVATE_CREDENTIAL_CANARY -> 403 event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=other xff=invalid event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=claude-code xff=198.51.100.7 event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=codex xff=none event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=opencode xff=none event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=none xff=none event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=other xff=invalid event=proxy_auth_rejected transport=websocket path=/v1/responses client=192.168.0.48 reason=missing_token ua=other xff=invalid $ grep -c PRIVATE_CREDENTIAL_CANARY proxy.out 0 ``` The HEAD `/api/hello` probe from that non-loopback peer logs `proxy_inbound_request` / `proxy_inbound_response` (`status=200`, `response_content_length=0`) and no `outbound_headers` line. Since `d125805` the probe runs through the inbound request logging like any routed request; it is still never forwarded. The loopback run above predates that commit. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: n/a. - Stable/default behavior changed: only when `HEADROOM_PROXY_TOKEN` is set. GET/HEAD `/api/hello` then gets a local 200 instead of a 401, and the rejection line gains three fields. Answered probes appear in the inbound request log (`proxy_inbound_request` / `proxy_inbound_response`) instead of as `proxy_auth_rejected` warnings. With no token, behavior is unchanged. - Kill switch / disable path: none needed. The probe answer carries no data and makes no upstream call. - Unsafe override required: no. - Qualification impact: log parsers that match the exact old `proxy_auth_rejected` line need to accept the extra trailing fields. The `path=`, `client=` and `reason=` fields are unchanged; the HTTP line now starts with `method=` before `path=`. - Rollback path: revert the commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Documentation: N/A. The docs don't list the token-exempt paths or the `proxy_auth_rejected` format. - Why only when a token is configured: without a token, the #4059 passthrough route is fine. Answering locally only matters when the alternative is a 401 for every client. - `xff` is not resolved through `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`: the gate runs before forwarded-header resolution, and resolving here would emit untrusted-peer events on every rejection. It is logged only as a validated IP literal (or `invalid`/`none`), unverified, for attribution. - `ua` reuses `CLIENT_UA_MAP`, so a harness added there is named in the rejection line too. Generic clients outside `_AUTH_REJECTION_CLIENTS` log `other`; adding one is a one-line change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4150" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ### Maintainer combined-tree qualification On the integration with main `31dcaf41`, all 375 focused hardening, hello-routing, provider-route/spec, loopback, extension-ordering, bind-policy, and auth-mode tests passed on Windows/Python 3.12 using the existing native extension. Changed-file lint, formatting, and whitespace checks passed. The provider-route insertion conflict retains the PR local-probe early return before main per-request upstream-override processing; AST comparison proves the combined route is the unchanged PR guard followed by the unchanged main route body. The probe test now also supplies a loopback or metadata-address upstream override and confirms the response stays empty/local with nothing relayed. This checks the integration boundary without rewriting contributor authorization or forwarding logic. Hosted qualification for the published sync remains pending. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JD Davis
parent
e9331b6b96
commit
673831a456
@@ -108,6 +108,11 @@ def _register_provider_passthrough_route(
|
||||
spec: ProviderPassthroughRoute,
|
||||
) -> None:
|
||||
async def provider_passthrough(request: Request):
|
||||
# A reachability probe the token gate let through without a credential
|
||||
# (/api/hello): answer it here, never relay it upstream anonymously.
|
||||
if getattr(request.state, "headroom_local_hello_probe", False):
|
||||
return Response(status_code=200)
|
||||
|
||||
# Honor the per-request upstream override like /v1/messages and the
|
||||
# catch-all do. Clients that speak one wire format but authenticate
|
||||
# against a custom OpenAI-compatible gateway must reach that gateway
|
||||
|
||||
@@ -119,7 +119,7 @@ from headroom.providers.registry import (
|
||||
)
|
||||
from headroom.proxy import runtime_env
|
||||
from headroom.proxy.audit import is_auditable_path, record_admin_action
|
||||
from headroom.proxy.auth_mode import should_stamp_codex_client
|
||||
from headroom.proxy.auth_mode import CLIENT_UA_MAP, should_stamp_codex_client
|
||||
from headroom.proxy.background_compression import BackgroundCompressor
|
||||
from headroom.proxy.budget_basis_policy import resolve_estimated_basis_policy
|
||||
from headroom.proxy.buffered_ccr_response import DEFAULT_BUFFERED_CCR_GRACE_SECONDS
|
||||
@@ -185,6 +185,7 @@ from headroom.proxy.project_context import (
|
||||
set_registered_cwd,
|
||||
strip_project_path_prefix,
|
||||
)
|
||||
from headroom.proxy.project_policy import split_project_path
|
||||
from headroom.proxy.prometheus_metrics import PrometheusMetrics
|
||||
from headroom.proxy.rate_limiter import TokenBucketRateLimiter
|
||||
from headroom.proxy.request_body_limit import RequestBodyLimitMiddleware
|
||||
@@ -3033,6 +3034,75 @@ def read_proxy_token(headers: Mapping[str, str]) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
# Generic HTTP clients a ``proxy_auth_rejected`` line may name by their
|
||||
# User-Agent product token, after the agent harnesses in CLIENT_UA_MAP. Any
|
||||
# other User-Agent logs as ``other``: the header is caller-controlled text.
|
||||
_AUTH_REJECTION_CLIENTS = frozenset(
|
||||
{
|
||||
"anthropic",
|
||||
"axios",
|
||||
"codex_cli_rs",
|
||||
"curl",
|
||||
"go-http-client",
|
||||
"mozilla",
|
||||
"node",
|
||||
"okhttp",
|
||||
"openai",
|
||||
"python-httpx",
|
||||
"python-requests",
|
||||
"python-urllib",
|
||||
"wget",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _auth_rejection_context(headers: Mapping[str, str]) -> tuple[str, str]:
|
||||
"""``(ua, xff)`` for a rejection log line, drawn from a fixed vocabulary.
|
||||
|
||||
Both headers are caller-controlled, so neither is logged as received.
|
||||
``ua`` is the harness name ``CLIENT_UA_MAP`` gives the User-Agent (the
|
||||
same names the rest of the proxy uses, e.g. ``claude-code``, ``codex``),
|
||||
else its product token when ``_AUTH_REJECTION_CLIENTS`` lists it, else
|
||||
``other``. ``X-Client`` is deliberately not consulted: unlike
|
||||
``classify_client``, nothing here may echo free text. ``xff`` is the leftmost
|
||||
``X-Forwarded-For`` hop when it is a plain IP address, else ``invalid``.
|
||||
Either is ``none`` when its header is absent. ``xff`` is unverified, so it
|
||||
only helps attribute rejected callers behind a gateway whose peer address
|
||||
is all the gate sees. It is never used for any decision. Credentials are
|
||||
never read here.
|
||||
"""
|
||||
from headroom.proxy.forwarded_policy import header_first
|
||||
|
||||
ua_lower = str(headers.get("user-agent") or "").lower()
|
||||
product = ua_lower.partition("/")[0].strip()
|
||||
harness = next((name for needle, name in CLIENT_UA_MAP if needle in ua_lower), None)
|
||||
if not ua_lower:
|
||||
ua = "none"
|
||||
elif harness is not None:
|
||||
ua = harness
|
||||
else:
|
||||
ua = product if product in _AUTH_REJECTION_CLIENTS else "other"
|
||||
|
||||
hop = header_first(str(headers.get("x-forwarded-for") or ""))
|
||||
try:
|
||||
address = ipaddress.ip_address(hop)
|
||||
except ValueError:
|
||||
return ua, "invalid" if hop else "none"
|
||||
# An IPv6 zone index (fe80::1%<zone>) is free text, so refuse it too.
|
||||
return ua, "invalid" if getattr(address, "scope_id", None) else str(address)
|
||||
|
||||
|
||||
def _auth_rejection_path(path: object) -> str:
|
||||
"""The request path for a rejection log line, with control characters escaped.
|
||||
|
||||
The ASGI server percent-decodes the path, so an unauthenticated caller can
|
||||
put raw CR/LF, NUL or terminal escapes in it (``%0A``, ``%1B``) and forge or
|
||||
garble log lines. ``unicode_escape`` renders every non-printable or
|
||||
non-ASCII character (and the backslash itself) as an escape sequence.
|
||||
"""
|
||||
return str(path).encode("unicode_escape").decode("ascii")
|
||||
|
||||
|
||||
_PROXY_TOKEN_HEADER = b"x-headroom-proxy-token"
|
||||
|
||||
|
||||
@@ -3157,11 +3227,15 @@ class WebSocketAuthMiddleware:
|
||||
await self.app(scope, receive, send)
|
||||
return
|
||||
|
||||
user_agent, forwarded_for = _auth_rejection_context(headers)
|
||||
logger.warning(
|
||||
"event=proxy_auth_rejected transport=websocket path=%s client=%s reason=%s",
|
||||
scope.get("path"),
|
||||
"event=proxy_auth_rejected transport=websocket path=%s client=%s reason=%s "
|
||||
"ua=%s xff=%s",
|
||||
_auth_rejection_path(scope.get("path")),
|
||||
client_host,
|
||||
"missing_token" if provided is None else "bad_token",
|
||||
user_agent,
|
||||
forwarded_for,
|
||||
)
|
||||
# Receive the handshake before refusing it: ASGI servers send
|
||||
# ``websocket.connect`` and wait for the application to answer, and
|
||||
@@ -4290,17 +4364,35 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
|
||||
path = request.url.path
|
||||
client = getattr(request, "client", None)
|
||||
client_host = getattr(client, "host", None) if client is not None else None
|
||||
exempt = request.method == "GET" and path in _AUTH_EXEMPT_PATHS
|
||||
# Anthropic SDK clients (Claude Code among them) probe reachability
|
||||
# with an unauthenticated GET/HEAD /api/hello that never carries
|
||||
# custom headers, so it cannot hold the proxy token. Let it past
|
||||
# the token check, marked so its passthrough route answers it
|
||||
# locally and empty instead of relaying an anonymous request
|
||||
# upstream. It still runs through extension middleware, which sits
|
||||
# inside this gate and may refuse it. The path is matched as
|
||||
# routed: a /p/<project> prefix is only stripped further in.
|
||||
hello_probe = (
|
||||
request.method in ("GET", "HEAD") and split_project_path(path)[1] == "/api/hello"
|
||||
)
|
||||
if hello_probe:
|
||||
request.state.headroom_local_hello_probe = True
|
||||
exempt = hello_probe or (request.method == "GET" and path in _AUTH_EXEMPT_PATHS)
|
||||
if not exempt and not is_loopback_host(client_host):
|
||||
provided = _extract_proxy_token(request.headers)
|
||||
if provided is None or not hmac.compare_digest(
|
||||
provided.encode("utf-8", "replace"), _proxy_token_bytes
|
||||
):
|
||||
user_agent, forwarded_for = _auth_rejection_context(request.headers)
|
||||
logger.warning(
|
||||
"event=proxy_auth_rejected path=%s client=%s reason=%s",
|
||||
path,
|
||||
"event=proxy_auth_rejected method=%s path=%s client=%s reason=%s "
|
||||
"ua=%s xff=%s",
|
||||
request.method,
|
||||
_auth_rejection_path(path),
|
||||
client_host,
|
||||
"missing_token" if provided is None else "bad_token",
|
||||
user_agent,
|
||||
forwarded_for,
|
||||
)
|
||||
rejection = JSONResponse(status_code=401, content={"error": "unauthorized"})
|
||||
_apply_security_headers(rejection)
|
||||
|
||||
@@ -104,6 +104,31 @@ class _BufferingExtension:
|
||||
await self.app(scope, replay_receive, send)
|
||||
|
||||
|
||||
class _DenyAllExtension:
|
||||
"""Raw ASGI middleware that refuses every HTTP request with 403.
|
||||
|
||||
Mirrors an extension that enforces its own access policy (an allowlist, a
|
||||
licence gate) on everything the proxy serves.
|
||||
"""
|
||||
|
||||
seen: list[str] = []
|
||||
|
||||
def __init__(self, app: Any) -> None:
|
||||
self.app = app
|
||||
|
||||
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
|
||||
if scope["type"] != "http":
|
||||
await self.app(scope, receive, send)
|
||||
return
|
||||
type(self).seen.append(scope.get("path"))
|
||||
await send({"type": "http.response.start", "status": 403, "headers": []})
|
||||
await send({"type": "http.response.body", "body": b""})
|
||||
|
||||
|
||||
def _install_deny_all(app: Any, config: Any) -> None:
|
||||
app.add_middleware(_DenyAllExtension)
|
||||
|
||||
|
||||
def _install_recording(app: Any, config: Any) -> None:
|
||||
app.add_middleware(_RecordingExtension)
|
||||
|
||||
@@ -116,6 +141,7 @@ def _install_buffering(app: Any, config: Any) -> None:
|
||||
def _reset_state() -> None:
|
||||
_RecordingExtension.seen = []
|
||||
_BufferingExtension.max_seen = 0
|
||||
_DenyAllExtension.seen = []
|
||||
|
||||
|
||||
def _make_app(monkeypatch, install, **overrides) -> Any:
|
||||
@@ -163,6 +189,16 @@ class TestExtensionMiddlewareIsInsideTheSecurityGate:
|
||||
assert resp.status_code == 200
|
||||
assert [s["path"] for s in _RecordingExtension.seen] == ["/"]
|
||||
|
||||
@pytest.mark.parametrize("method", ["GET", "HEAD"])
|
||||
def test_api_hello_probe_still_runs_extension_policy(self, monkeypatch, method):
|
||||
"""The token-less /api/hello probe passes the token gate, but an
|
||||
extension that refuses everything still refuses it."""
|
||||
app = _make_app(monkeypatch, _install_deny_all, proxy_token=TOKEN)
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
resp = c.request(method, "/api/hello")
|
||||
assert resp.status_code == 403
|
||||
assert _DenyAllExtension.seen == ["/api/hello"]
|
||||
|
||||
def test_health_probe_stays_exempt_with_extension_installed(self, monkeypatch):
|
||||
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
|
||||
@@ -159,6 +159,225 @@ class TestInboundAuthToken:
|
||||
assert resp.status_code == 401
|
||||
assert relayed == []
|
||||
|
||||
@pytest.mark.parametrize("path", ["/api/hello", "/p/demo/api/hello"])
|
||||
@pytest.mark.parametrize("method", ["GET", "HEAD"])
|
||||
@pytest.mark.parametrize("override", [None, "http://127.0.0.1:1", "http://169.254.169.254"])
|
||||
def test_api_hello_probe_is_answered_locally(self, monkeypatch, method, path, override):
|
||||
"""The SDK reachability probe never carries custom headers, so it can't
|
||||
hold the token. It is answered locally: 200, empty, never relayed, also
|
||||
under a /p/<project> base URL."""
|
||||
relayed: list[tuple[str, str]] = []
|
||||
|
||||
async def _spy_passthrough(self, request, base_url, *args, **kwargs):
|
||||
relayed.append((request.method, request.url.path))
|
||||
return JSONResponse({"relayed_to": base_url})
|
||||
|
||||
monkeypatch.setattr(OpenAIHandlerMixin, "handle_passthrough", _spy_passthrough)
|
||||
app = _make_app(proxy_token="s3cr3t-token")
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
resp = c.request(
|
||||
method,
|
||||
path,
|
||||
headers={"x-headroom-base-url": override} if override else None,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert resp.content == b""
|
||||
assert resp.headers["X-Content-Type-Options"] == "nosniff"
|
||||
assert relayed == []
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("method", "path"),
|
||||
[
|
||||
("POST", "/api/hello"),
|
||||
("GET", "/api/hello/x"),
|
||||
("GET", "/api/hello/"),
|
||||
("POST", "/p/demo/api/hello"),
|
||||
("GET", "/p/demo/stats"),
|
||||
("GET", "/stats"),
|
||||
],
|
||||
)
|
||||
def test_api_hello_answer_is_exact(self, monkeypatch, method, path):
|
||||
"""Only GET/HEAD on exactly /api/hello skips the token."""
|
||||
relayed: list[tuple[str, str]] = []
|
||||
|
||||
async def _spy_passthrough(self, request, base_url, *args, **kwargs):
|
||||
relayed.append((request.method, request.url.path))
|
||||
return JSONResponse({"relayed_to": base_url})
|
||||
|
||||
monkeypatch.setattr(OpenAIHandlerMixin, "handle_passthrough", _spy_passthrough)
|
||||
app = _make_app(proxy_token="s3cr3t-token")
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
resp = c.request(method, path)
|
||||
assert resp.status_code == 401
|
||||
assert relayed == []
|
||||
|
||||
def test_api_hello_without_token_is_left_to_the_passthrough(self, monkeypatch):
|
||||
"""No token configured: the probe keeps its normal Anthropic route."""
|
||||
relayed: list[tuple[str, str]] = []
|
||||
|
||||
async def _spy_passthrough(self, request, base_url, *args, **kwargs):
|
||||
relayed.append((request.method, request.url.path))
|
||||
return JSONResponse({"relayed_to": base_url})
|
||||
|
||||
monkeypatch.setattr(OpenAIHandlerMixin, "handle_passthrough", _spy_passthrough)
|
||||
app = _make_app()
|
||||
with TestClient(app, base_url="http://127.0.0.1", client=LOOPBACK) as c:
|
||||
c.get("/api/hello")
|
||||
assert relayed == [("GET", "/api/hello")]
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _captured_proxy_warnings():
|
||||
"""Collect ``headroom.proxy`` log messages (attached to the logger itself:
|
||||
the proxy's logging setup controls propagation, so caplog can miss them)."""
|
||||
messages: list[str] = []
|
||||
|
||||
class _Capture(logging.Handler):
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
messages.append(record.getMessage())
|
||||
|
||||
handler = _Capture(level=logging.WARNING)
|
||||
proxy_logger = logging.getLogger("headroom.proxy")
|
||||
previous_level = proxy_logger.level
|
||||
proxy_logger.setLevel(logging.WARNING)
|
||||
proxy_logger.addHandler(handler)
|
||||
try:
|
||||
yield messages
|
||||
finally:
|
||||
proxy_logger.removeHandler(handler)
|
||||
proxy_logger.setLevel(previous_level)
|
||||
|
||||
|
||||
def _rejections(messages: list[str]) -> list[str]:
|
||||
return [m for m in messages if "event=proxy_auth_rejected" in m]
|
||||
|
||||
|
||||
# Stands in for anything sensitive a caller may put in a header.
|
||||
CANARY = "PRIVATE_CREDENTIAL_CANARY"
|
||||
|
||||
|
||||
class TestAuthRejectionLog:
|
||||
"""``proxy_auth_rejected`` carries enough to attribute a caller behind a
|
||||
gateway (method, User-Agent, leftmost X-Forwarded-For hop), and never a
|
||||
credential."""
|
||||
|
||||
def test_line_carries_method_client_and_first_forwarded_hop(self):
|
||||
app = _make_app(proxy_token="s3cr3t-token")
|
||||
with _captured_proxy_warnings() as messages:
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
c.get(
|
||||
"/stats",
|
||||
headers={
|
||||
"User-Agent": "claude-cli/2.1.0 (external, cli)",
|
||||
"X-Forwarded-For": "198.51.100.7, 10.0.0.9",
|
||||
},
|
||||
)
|
||||
(line,) = _rejections(messages)
|
||||
assert "method=GET path=/stats" in line
|
||||
assert "reason=missing_token" in line
|
||||
assert line.endswith("ua=claude-code xff=198.51.100.7")
|
||||
|
||||
def test_absent_headers_log_as_none(self):
|
||||
app = _make_app(proxy_token="s3cr3t-token")
|
||||
with _captured_proxy_warnings() as messages:
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
c.get("/stats", headers={"User-Agent": ""})
|
||||
(line,) = _rejections(messages)
|
||||
assert line.endswith("ua=none xff=none")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("user_agent", "forwarded_for", "tail"),
|
||||
[
|
||||
(CANARY, f"{CANARY}, 192.0.2.1", "ua=other xff=invalid"),
|
||||
(f"{CANARY}/1.0", "192.0.2.1", "ua=other xff=192.0.2.1"),
|
||||
(f'curl" xff="{CANARY}', f"fe80::1%{CANARY}", "ua=other xff=invalid"),
|
||||
("node", f"192.0.2.1:{CANARY}", "ua=node xff=invalid"),
|
||||
(f"{CANARY}/9 opencode/1.2.3", "192.0.2.1", "ua=opencode xff=192.0.2.1"),
|
||||
],
|
||||
)
|
||||
def test_caller_text_never_reaches_the_line(self, user_agent, forwarded_for, tail):
|
||||
app = _make_app(proxy_token="s3cr3t-token")
|
||||
with _captured_proxy_warnings() as messages:
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
c.get(
|
||||
"/stats", headers={"User-Agent": user_agent, "X-Forwarded-For": forwarded_for}
|
||||
)
|
||||
(line,) = _rejections(messages)
|
||||
assert CANARY not in line
|
||||
assert line.endswith(tail)
|
||||
|
||||
def test_path_cannot_smuggle_control_characters(self):
|
||||
# The ASGI server percent-decodes the path. Starlette's URL parsing
|
||||
# drops CR/LF/TAB from request.url.path, but every other control
|
||||
# character (NUL, ESC for terminal escapes, U+2028) reaches the line.
|
||||
app = _make_app(proxy_token="s3cr3t-token")
|
||||
with _captured_proxy_warnings() as messages:
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
c.get("/stats%0D%0A%1B%5B1Aevent=forged%00%E2%80%A8")
|
||||
(line,) = _rejections(messages)
|
||||
assert line.isprintable()
|
||||
assert "path=/stats\\x1b[1Aevent=forged\\x00\\u2028 client=" in line
|
||||
|
||||
def test_context_is_a_fixed_vocabulary(self):
|
||||
# Exercised directly: an HTTP client refuses to send CR/LF in a header,
|
||||
# but an ASGI server in front of the proxy is not guaranteed to.
|
||||
from headroom.proxy.server import _auth_rejection_context
|
||||
|
||||
assert _auth_rejection_context(
|
||||
{"user-agent": "curl\r\nevent=forged", "x-forwarded-for": "192.0.2.1\r\nevent=forged"}
|
||||
) == ("other", "invalid")
|
||||
assert _auth_rejection_context(
|
||||
{"user-agent": "Mozilla/5.0 (Windows NT 10.0)", "x-forwarded-for": "2001:DB8::1"}
|
||||
) == ("mozilla", "2001:db8::1")
|
||||
assert _auth_rejection_context({}) == ("none", "none")
|
||||
# X-Client is free text, so it never names the client here.
|
||||
assert _auth_rejection_context({"user-agent": "x", "x-client": "claude-code"}) == (
|
||||
"other",
|
||||
"none",
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("user_agent", "client"),
|
||||
[
|
||||
("claude-cli/2.1.0 (external, cli)", "claude-code"),
|
||||
("claude-code/2.0.14", "claude-code"),
|
||||
("codex-cli/0.46.0", "codex"),
|
||||
("Cursor/1.7.0", "cursor"),
|
||||
("opencode/1.2.3", "opencode"),
|
||||
("CorpWrapper/3.1 aider/0.86", "aider"),
|
||||
("Anthropic/Python 0.69.0", "anthropic"),
|
||||
("node", "node"),
|
||||
],
|
||||
)
|
||||
def test_known_clients_keep_their_proxy_wide_name(self, user_agent, client):
|
||||
# Harnesses log under the same name the rest of the proxy gives them
|
||||
# (CLIENT_UA_MAP); generic HTTP clients under their product token.
|
||||
from headroom.proxy.server import _auth_rejection_context
|
||||
|
||||
assert _auth_rejection_context({"user-agent": user_agent}) == (client, "none")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"headers",
|
||||
[
|
||||
{"Authorization": "Bearer wrong-bearer-value"},
|
||||
{
|
||||
"x-headroom-proxy-token": "wrong-header-value",
|
||||
"Authorization": "Bearer sk-ant-oat01-upstream-oauth",
|
||||
},
|
||||
{"x-api-key": "sk-ant-api03-provider-key"},
|
||||
],
|
||||
)
|
||||
def test_no_credential_ever_reaches_the_line(self, headers):
|
||||
app = _make_app(proxy_token="s3cr3t-token")
|
||||
with _captured_proxy_warnings() as messages:
|
||||
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
||||
c.post("/v1/messages", headers={"User-Agent": "claude-cli/2.1", **headers})
|
||||
(line,) = _rejections(messages)
|
||||
assert "method=POST" in line
|
||||
for secret in ("s3cr3t-token", *headers.values()):
|
||||
assert secret.removeprefix("Bearer ") not in line
|
||||
assert "authorization" not in line.lower()
|
||||
|
||||
|
||||
# ──────────────────── 2.1b inbound auth token over WebSocket ──────────────
|
||||
|
||||
@@ -231,6 +450,51 @@ class TestWebSocketAuthMiddleware:
|
||||
assert downstream.called is False
|
||||
assert _closed_with_policy_violation(sent)
|
||||
|
||||
async def test_rejection_log_matches_the_http_gate(self):
|
||||
mw = WebSocketAuthMiddleware(_SpyApp(), proxy_token="s3cr3t-token")
|
||||
scope = _ws_scope(
|
||||
headers=[
|
||||
("user-agent", "node"),
|
||||
("x-forwarded-for", "198.51.100.8, 10.0.0.1"),
|
||||
("authorization", "Bearer wrong-ws-bearer"),
|
||||
]
|
||||
)
|
||||
|
||||
with _captured_proxy_warnings() as messages:
|
||||
await _drive(mw, scope)
|
||||
|
||||
(line,) = _rejections(messages)
|
||||
assert "transport=websocket" in line and "reason=bad_token" in line
|
||||
assert line.endswith("ua=node xff=198.51.100.8")
|
||||
|
||||
async def test_rejection_log_keeps_caller_text_out(self):
|
||||
mw = WebSocketAuthMiddleware(_SpyApp(), proxy_token="s3cr3t-token")
|
||||
scope = _ws_scope(
|
||||
headers=[
|
||||
("user-agent", CANARY),
|
||||
("x-forwarded-for", f"{CANARY}, 192.0.2.1"),
|
||||
]
|
||||
)
|
||||
|
||||
with _captured_proxy_warnings() as messages:
|
||||
await _drive(mw, scope)
|
||||
|
||||
(line,) = _rejections(messages)
|
||||
assert CANARY not in line
|
||||
assert line.endswith("ua=other xff=invalid")
|
||||
assert "wrong-ws-bearer" not in line and "s3cr3t-token" not in line
|
||||
|
||||
async def test_rejection_log_path_cannot_forge_a_line(self):
|
||||
mw = WebSocketAuthMiddleware(_SpyApp(), proxy_token="s3cr3t-token")
|
||||
scope = _ws_scope(path="/v1/responses\r\nevent=proxy_auth_rejected")
|
||||
|
||||
with _captured_proxy_warnings() as messages:
|
||||
await _drive(mw, scope)
|
||||
|
||||
(line,) = _rejections(messages)
|
||||
assert "\r" not in line and "\n" not in line
|
||||
assert "path=/v1/responses\\r\\nevent=proxy_auth_rejected " in line
|
||||
|
||||
async def test_accepts_correct_bearer(self):
|
||||
downstream = _SpyApp()
|
||||
mw = WebSocketAuthMiddleware(downstream, proxy_token="s3cr3t-token")
|
||||
|
||||
Reference in New Issue
Block a user