Files
headroom/tests/test_proxy_extension_ordering.py
673831a456 fix(proxy): answer /api/hello probes behind the token gate, log rejection context (#4150)
## Description

With `HEADROOM_PROXY_TOKEN` set, every Anthropic SDK client logs a
`proxy_auth_rejected` line once per process, Claude Code included and
even when it is configured correctly. Claude Code sends an
unauthenticated `HEAD /api/hello` reachability probe. That probe never
carries `ANTHROPIC_CUSTOM_HEADERS`, so it cannot hold the proxy token:
it gets a 401, which the client ignores, and the rejection lands in the
log. On a busy deployment these lines bury real rejections.

Behind a gateway, the rejection line also showed only the gateway's peer
address, so a rejected caller could not be identified.

This PR:

1. **Answers the probe locally, behind the token.** The HTTP security
gate lets GET/HEAD on exactly `/api/hello` (also under `/p/<project>`)
past the token check and marks it. Extension middleware still runs, so
an extension can still refuse it. The `/api/hello` passthrough route
answers a marked probe with an empty 200 (the gate adds the security
headers) and never relays it. This only happens when a token is
configured. It is deliberately **not** added to `_AUTH_EXEMPT_PATHS`,
which would hand the probe to the passthrough relay: an anonymous
upstream request (to Anthropic since #4059, to the OpenAI default target
before it). Without a token, the probe keeps its #4059 route unchanged.
2. **Gives the rejection line context.** `proxy_auth_rejected` (HTTP and
WebSocket) adds `method=`, `ua=` and `xff=` fields. Both headers are
caller-controlled, so neither is logged as received; each maps to a
fixed vocabulary:
- `ua`: the harness name `CLIENT_UA_MAP` gives the User-Agent, the same
name the outcome and cost records use (`claude-code`, `codex`, `cursor`,
`opencode`, `aider`, `copilot`, …). Failing that, the product token of a
generic HTTP client (`node`, `anthropic`, `openai`, `curl`, `mozilla`,
`python-httpx`, …). Otherwise `other`. `X-Client` is not consulted,
since it is free text.
- `xff`: the leftmost `X-Forwarded-For` hop when it parses as a plain IP
address, else `invalid`. An IPv6 zone index (`fe80::1%<zone>`) is free
text, so it counts as `invalid`. The hop is unverified, used only for
attribution, and drives no decision.
   - Either is `none` when its header is absent.
   - No credential header is read for the log.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/server.py`:
- **`_security_gate`**: when a token is configured, `GET`/`HEAD` on
exactly `/api/hello`, matched on the routed path so
`/p/<project>/api/hello` counts too, skips the token check and sets
`request.state.headroom_local_hello_probe`. It then calls `call_next`
like any request, so extension middleware (installed inside the gate)
still runs and can refuse it. `POST /api/hello`, `/api/hello/` and
`/api/hello/x` still need the token.
- **New helper**: `_auth_rejection_context` returns `(ua, xff)` from the
fixed vocabulary above: `CLIENT_UA_MAP` then an allowlist of generic
clients (`_AUTH_REJECTION_CLIENTS`) for the User-Agent, and
`ipaddress.ip_address` on the first hop (from
`forwarded_policy.header_first`).
- **Both `proxy_auth_rejected` warnings** gain the new fields: the HTTP
gate gets `method=`, and the WebSocket middleware gets the same
`ua=`/`xff=` fields, so the two transports stay aligned.
- `headroom/providers/proxy_routes.py`: the provider passthrough route
answers a marked probe with an empty 200 and never relays it upstream.
- `tests/test_proxy_hardening.py`:
- **Probe answer** (`test_api_hello_probe_is_answered_locally`):
GET/HEAD on `/api/hello` and `/p/demo/api/hello` from a non-loopback
peer gets 200 with an empty body and security headers, and the
passthrough spy shows nothing relayed. `POST /p/demo/api/hello` and `GET
/p/demo/stats` still get 401. `POST /api/hello`, `GET /api/hello/x`,
`GET /api/hello/` and `GET /stats` still get 401, with nothing relayed.
With no token configured, `/api/hello` still reaches the passthrough.
- **Log line**: `claude-cli/2.1.0 (external, cli)` with
`X-Forwarded-For: 198.51.100.7, 10.0.0.9` logs `ua=claude-code
xff=198.51.100.7`. Absent headers log `ua=none xff=none`.
`test_known_clients_keep_their_proxy_wide_name` covers `claude-code`,
`codex`, `cursor`, `opencode`, `aider` behind a wrapper prefix, the
Anthropic SDK and `node`.
- **Caller text never reaches the line**
(`test_caller_text_never_reaches_the_line`): a
`PRIVATE_CREDENTIAL_CANARY` in the User-Agent (bare, as a product token,
behind a fake quoted field) and in X-Forwarded-For (as the first hop, as
a zone index, as a port suffix) never appears in the line, which logs
`other`/`invalid` instead. CR/LF in either header maps to
`other`/`invalid` (tested on the helper directly, since an HTTP client
refuses to send CR/LF).
- **No credentials**: neither the configured token, a wrong
`x-headroom-proxy-token`, `Authorization: Bearer …` nor `x-api-key` ever
appears in the line.
- **WebSocket**: the rejection line matches the HTTP gate's, and the
same canary in both headers never appears in it
(`test_rejection_log_keeps_caller_text_out`).
- `tests/test_proxy_extension_ordering.py`:
- **Extension policy**
(`test_api_hello_probe_still_runs_extension_policy`): an extension that
answers 403 to everything gets the token-less GET/HEAD probe and refuses
it.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_proxy_hardening.py tests/test_anthropic_hello_probe_routing.py \
    tests/test_provider_proxy_routes.py tests/test_provider_route_specs.py \
    tests/test_proxy_loopback_gating.py tests/test_proxy_extension_ordering.py \
    tests/test_proxy_bind_policy.py tests/test_auth_mode.py -q
365 passed in 78.96s

$ ruff check headroom/proxy/server.py tests/test_proxy_hardening.py
All checks passed!
$ ruff format --check headroom/proxy/server.py tests/test_proxy_hardening.py
2 files already formatted

$ python -m mypy headroom/proxy/server.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: this branch on Windows 11 with Python 3.13. Ran `headroom
proxy --port 18787` from the worktree with a throwaway
`HEADROOM_PROXY_TOKEN`.
- Exact command / steps: `curl -I http://127.0.0.1:18787/api/hello`,
`curl http://127.0.0.1:18787/api/hello`, then checked the proxy log.
- Observed result: HEAD and GET returned `200` with 0 bytes and
`x-content-type-options: nosniff`, `x-frame-options: DENY`,
`referrer-policy: no-referrer` and `strict-transport-security`. The
proxy log has no `proxy_inbound_request` and no `outbound_headers` line
for either probe, so nothing was forwarded. For contrast, a loopback
`POST /api/hello` (loopback is token-exempt, existing behavior) logs
`forwarder=openai_passthrough`.
- Not tested: the new rejection-line fields behind a real gateway. They
were checked against a real non-loopback peer, below.

Rejection diagnostics, at `d70ec3c` (contains current `main`): the same
worktree proxy, bound to a non-loopback LAN address (`--host
192.168.0.48 --port 18787 --no-optimize`, `HEADROOM_OFFLINE=1`) with a
throwaway token, called from that address:

```text
GET /stats  UA=PRIVATE_CREDENTIAL_CANARY  XFF=PRIVATE_CREDENTIAL_CANARY, 192.0.2.1          -> 401
GET /stats  UA=claude-cli/2.1.0 (external, cli)  XFF=198.51.100.7, 10.0.0.9                -> 401
GET /stats  UA=codex-cli/0.46.0                                                             -> 401
GET /stats  UA=PRIVATE_CREDENTIAL_CANARY/9 opencode/1.2.3                                   -> 401
GET /stats  no UA, no XFF                                                                   -> 401
GET /stats  UA=PRIVATE_CREDENTIAL_CANARY/1.0  XFF=fe80::1%PRIVATE_CREDENTIAL_CANARY         -> 401
GET /stats  valid x-headroom-proxy-token                                                    -> 200
HEAD /api/hello                                                                             -> 200
WS upgrade /v1/responses  UA=PRIVATE_CREDENTIAL_CANARY  XFF=PRIVATE_CREDENTIAL_CANARY       -> 403

event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=other xff=invalid
event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=claude-code xff=198.51.100.7
event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=codex xff=none
event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=opencode xff=none
event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=none xff=none
event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=other xff=invalid
event=proxy_auth_rejected transport=websocket path=/v1/responses client=192.168.0.48 reason=missing_token ua=other xff=invalid

$ grep -c PRIVATE_CREDENTIAL_CANARY proxy.out
0
```

The HEAD `/api/hello` probe from that non-loopback peer logs
`proxy_inbound_request` / `proxy_inbound_response` (`status=200`,
`response_content_length=0`) and no `outbound_headers` line. Since
`d125805` the probe runs through the inbound request logging like any
routed request; it is still never forwarded. The loopback run above
predates that commit.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: only when `HEADROOM_PROXY_TOKEN` is
set. GET/HEAD `/api/hello` then gets a local 200 instead of a 401, and
the rejection line gains three fields. Answered probes appear in the
inbound request log (`proxy_inbound_request` / `proxy_inbound_response`)
instead of as `proxy_auth_rejected` warnings. With no token, behavior is
unchanged.
- Kill switch / disable path: none needed. The probe answer carries no
data and makes no upstream call.
- Unsafe override required: no.
- Qualification impact: log parsers that match the exact old
`proxy_auth_rejected` line need to accept the extra trailing fields. The
`path=`, `client=` and `reason=` fields are unchanged; the HTTP line now
starts with `method=` before `path=`.
- Rollback path: revert the commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- Documentation: N/A. The docs don't list the token-exempt paths or the
`proxy_auth_rejected` format.
- Why only when a token is configured: without a token, the #4059
passthrough route is fine. Answering locally only matters when the
alternative is a 401 for every client.
- `xff` is not resolved through `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`:
the gate runs before forwarded-header resolution, and resolving here
would emit untrusted-peer events on every rejection. It is logged only
as a validated IP literal (or `invalid`/`none`), unverified, for
attribution.
- `ua` reuses `CLIENT_UA_MAP`, so a harness added there is named in the
rejection line too. Generic clients outside `_AUTH_REJECTION_CLIENTS`
log `other`; adding one is a one-line change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4150"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer combined-tree qualification

On the integration with main `31dcaf41`, all 375 focused hardening,
hello-routing, provider-route/spec, loopback, extension-ordering,
bind-policy, and auth-mode tests passed on Windows/Python 3.12 using the
existing native extension. Changed-file lint, formatting, and whitespace
checks passed. The provider-route insertion conflict retains the PR
local-probe early return before main per-request upstream-override
processing; AST comparison proves the combined route is the unchanged PR
guard followed by the unchanged main route body. The probe test now also
supplies a loopback or metadata-address upstream override and confirms
the response stays empty/local with nothing relayed. This checks the
integration boundary without rewriting contributor authorization or
forwarding logic. Hosted qualification for the published sync remains
pending.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-04 22:38:51 -06:00

580 lines
24 KiB
Python

"""Extension middleware runs inside the proxy's security gate and body ceiling.
Regression tests for the ordering contract documented in
``headroom/proxy/extensions.py``. Before this contract existed, ``install_all``
ran after the gate was registered, and Starlette's prepend-on-register stack
made every extension middleware the OUTERMOST layer: an extension that answered
``POST /`` itself served unauthenticated callers, one that buffered the body
held an unbounded body before the 401 was ever issued, and one that rewrote
``Authorization`` did so before the gate compared it.
The extension used here is a stand-in for the shapes the private extensions
actually take (a middleware that claims a request and answers it, and one that
buffers the whole body), installed through the real ``install_all`` seam by
monkeypatching discovery, so the tests exercise the app exactly as
``headroom proxy --proxy-extension <name>`` would.
"""
from __future__ import annotations
import json
from typing import Any
import pytest
from fastapi.testclient import TestClient
from headroom.cache.compression_store import reset_compression_store
from headroom.proxy import extensions
from headroom.proxy import helpers as proxy_helpers
from headroom.proxy.request_body_limit import RequestBodyLimitMiddleware
from headroom.proxy.server import ProxyConfig, WebSocketAuthMiddleware, create_app
NONLOOPBACK = ("203.0.113.5", 44444) # TEST-NET-3, never loopback
LOOPBACK = ("127.0.0.1", 12345)
TOKEN = "s3cr3t-token"
class _RecordingExtension:
"""Raw ASGI middleware that records every scope it sees and answers ``POST /``.
Mirrors the shape of an extension that intercepts a vendor protocol on the
root path and relays it upstream itself, never calling the wrapped app.
"""
seen: list[dict[str, Any]] = []
def __init__(self, app: Any) -> None:
self.app = app
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
if scope["type"] in {"http", "websocket"}:
type(self).seen.append({"type": scope["type"], "path": scope.get("path")})
if scope["type"] == "http" and scope.get("path") == "/" and scope.get("method") == "POST":
body = b'{"claimed_by":"extension"}'
await send(
{
"type": "http.response.start",
"status": 200,
"headers": [
(b"content-type", b"application/json"),
(b"content-length", str(len(body)).encode()),
],
}
)
await send({"type": "http.response.body", "body": body})
return
await self.app(scope, receive, send)
class _BufferingExtension:
"""Raw ASGI middleware that buffers the full request body before passing it on.
Mirrors an extension that must read the JSON to decide how to route or
rewrite the request. ``max_seen`` records the largest body it ever held.
"""
max_seen = 0
def __init__(self, app: Any) -> None:
self.app = app
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
chunks = bytearray()
while True:
message = await receive()
if message["type"] != "http.request":
break
chunks.extend(message.get("body") or b"")
type(self).max_seen = max(type(self).max_seen, len(chunks))
if not message.get("more_body"):
break
replay = {"type": "http.request", "body": bytes(chunks), "more_body": False}
sent = False
async def replay_receive() -> dict[str, Any]:
nonlocal sent
if not sent:
sent = True
return replay
return {"type": "http.disconnect"}
await self.app(scope, replay_receive, send)
class _DenyAllExtension:
"""Raw ASGI middleware that refuses every HTTP request with 403.
Mirrors an extension that enforces its own access policy (an allowlist, a
licence gate) on everything the proxy serves.
"""
seen: list[str] = []
def __init__(self, app: Any) -> None:
self.app = app
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
type(self).seen.append(scope.get("path"))
await send({"type": "http.response.start", "status": 403, "headers": []})
await send({"type": "http.response.body", "body": b""})
def _install_deny_all(app: Any, config: Any) -> None:
app.add_middleware(_DenyAllExtension)
def _install_recording(app: Any, config: Any) -> None:
app.add_middleware(_RecordingExtension)
def _install_buffering(app: Any, config: Any) -> None:
app.add_middleware(_BufferingExtension)
@pytest.fixture(autouse=True)
def _reset_state() -> None:
_RecordingExtension.seen = []
_BufferingExtension.max_seen = 0
_DenyAllExtension.seen = []
def _make_app(monkeypatch, install, **overrides) -> Any:
reset_compression_store()
monkeypatch.setattr(extensions, "discover", lambda: iter([("test_ext", install)]))
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
proxy_extensions=["test_ext"],
**overrides,
)
return create_app(config)
class TestExtensionMiddlewareIsInsideTheSecurityGate:
def test_unauthenticated_request_never_reaches_extension_middleware(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post("/", json={}, headers={"x-amz-target": "Vendor.Generate"})
assert resp.status_code == 401
assert resp.json() == {"error": "unauthorized"}
assert _RecordingExtension.seen == [], "extension ran before the proxy token was checked"
def test_authenticated_request_reaches_extension_middleware(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post("/", json={}, headers={"Authorization": f"Bearer {TOKEN}"})
assert resp.status_code == 200
assert resp.json() == {"claimed_by": "extension"}
assert [s["path"] for s in _RecordingExtension.seen] == ["/"]
def test_wrong_token_never_reaches_extension_middleware(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post("/", json={}, headers={"Authorization": "Bearer nope"})
assert resp.status_code == 401
assert _RecordingExtension.seen == []
def test_loopback_caller_is_exempt_and_reaches_extension(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
with TestClient(app, base_url="http://127.0.0.1", client=LOOPBACK) as c:
resp = c.post("/", json={})
assert resp.status_code == 200
assert [s["path"] for s in _RecordingExtension.seen] == ["/"]
@pytest.mark.parametrize("method", ["GET", "HEAD"])
def test_api_hello_probe_still_runs_extension_policy(self, monkeypatch, method):
"""The token-less /api/hello probe passes the token gate, but an
extension that refuses everything still refuses it."""
app = _make_app(monkeypatch, _install_deny_all, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.request(method, "/api/hello")
assert resp.status_code == 403
assert _DenyAllExtension.seen == ["/api/hello"]
def test_health_probe_stays_exempt_with_extension_installed(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.get("/health")
assert resp.status_code == 200
# Exempt from auth, so the extension legitimately sees it — the gate
# passes it through rather than the extension pre-empting the gate.
assert [s["path"] for s in _RecordingExtension.seen] == ["/health"]
def test_gate_security_headers_apply_to_extension_responses(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post("/", json={}, headers={"Authorization": f"Bearer {TOKEN}"})
assert resp.headers["X-Content-Type-Options"] == "nosniff"
def test_unauthenticated_websocket_never_reaches_extension_middleware(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
with pytest.raises(Exception): # noqa: B017 — TestClient surfaces the refused upgrade
with c.websocket_connect("/v1/responses"):
pass
assert _RecordingExtension.seen == [], "extension saw a refused WebSocket handshake"
class TestBodyCeilingAppliesBeforeExtensionMiddleware:
def test_declared_oversize_body_is_refused_before_extension_buffers_it(self, monkeypatch):
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
payload = b"x" * 4096
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post(
"/v1/chat/completions",
content=payload,
headers={
"Authorization": f"Bearer {TOKEN}",
"content-type": "application/json",
},
)
assert resp.status_code == 413
assert resp.json()["error"]["code"] == "request_too_large"
assert _BufferingExtension.max_seen == 0
def test_chunked_oversize_body_is_cut_off_before_extension_holds_it_all(self, monkeypatch):
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
chunk = b"x" * 256
def body():
for _ in range(64): # 16 KiB, no Content-Length
yield chunk
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post(
"/v1/chat/completions",
content=body(),
headers={
"Authorization": f"Bearer {TOKEN}",
"content-type": "application/json",
},
)
assert resp.status_code == 413
# The ceiling plus at most one delivered message; never the whole 16 KiB.
# (TestClient delivers the generator as one message, so the extension
# sees nothing at all here; the multi-message case is covered by
# TestRequestBodyLimitMiddlewareDirectly below.)
assert _BufferingExtension.max_seen <= 1024 + len(chunk)
def test_anthropic_dialect_payload_on_messages_path(self, monkeypatch):
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post(
"/v1/messages",
content=b"x" * 4096,
headers={"Authorization": f"Bearer {TOKEN}", "content-type": "application/json"},
)
assert resp.status_code == 413
assert resp.json() == {
"type": "error",
"error": {
"type": "request_too_large",
"message": "Request body too large. Maximum size is 0MB",
},
}
@pytest.mark.parametrize(
"route",
[
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke",
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke-with-response-stream",
],
ids=["invoke", "invoke-with-response-stream"],
)
def test_bedrock_invoke_dialect_payload_on_model_paths(self, monkeypatch, route):
"""The two Bedrock InvokeModel routes keep handle_bedrock_invoke's wire
dialect on the body ceiling: exactly
``{"error": {"type": "request_too_large", "message": ...}}`` — not the
OpenAI-style payload — and the extension never buffers the body.
"""
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
app = _make_app(
monkeypatch,
_install_buffering,
proxy_token=TOKEN,
bedrock_api_url="http://127.0.0.1:4000",
)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post(
route,
content=b"x" * 4096,
headers={"Authorization": f"Bearer {TOKEN}", "content-type": "application/json"},
)
assert resp.status_code == 413
assert resp.json() == {
"error": {
"type": "request_too_large",
"message": "Request body too large. Maximum size is 0MB",
}
}
assert _BufferingExtension.max_seen == 0
def test_body_within_ceiling_reaches_extension_intact(self, monkeypatch):
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
# /stats is a local route, so the request completes without an upstream.
resp = c.post(
"/stats",
content=b"y" * 512,
headers={"Authorization": f"Bearer {TOKEN}"},
)
assert resp.status_code != 413
assert _BufferingExtension.max_seen == 512
def test_unauthenticated_oversize_body_is_refused_by_the_gate_first(self, monkeypatch):
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
resp = c.post("/v1/chat/completions", content=b"x" * 4096)
assert resp.status_code == 401
assert _BufferingExtension.max_seen == 0
class TestStackOrderIsPinned:
"""The contract is structural, so check the stack itself, not only its effects."""
@staticmethod
def _index_of(app, predicate) -> int:
for index, entry in enumerate(app.user_middleware):
if predicate(entry):
return index
raise AssertionError("middleware not found in stack")
def test_extension_middleware_is_inner_to_gate_and_body_limit(self, monkeypatch):
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
added = app.state.extension_middleware
assert len(added) == 1
assert added[0].cls is _RecordingExtension
# Lower index == outer layer (Starlette prepends on add_middleware).
ext_index = self._index_of(app, lambda e: e.cls is _RecordingExtension)
limit_index = self._index_of(app, lambda e: e.cls is RequestBodyLimitMiddleware)
ws_auth_index = self._index_of(app, lambda e: e.cls is WebSocketAuthMiddleware)
gate_index = self._index_of(
app,
lambda e: getattr(e.kwargs.get("dispatch"), "__name__", "") == "_security_gate",
)
assert ws_auth_index < gate_index < limit_index < ext_index
def test_install_all_records_nothing_for_routes_only_extensions(self, monkeypatch):
def routes_only(app: Any, config: Any) -> None:
@app.get("/ext/ping")
async def ping():
return {"ok": True}
app = _make_app(monkeypatch, routes_only, proxy_token=TOKEN)
assert app.state.extension_middleware == []
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
assert c.get("/ext/ping").status_code == 401
assert c.get("/ext/ping", headers={"Authorization": f"Bearer {TOKEN}"}).json() == {
"ok": True
}
class TestRequestBodyLimitMiddlewareDirectly:
"""Drive the layer with a hand-rolled ASGI conversation to pin the metering."""
@staticmethod
def _http_scope(path="/v1/chat/completions", headers=()):
return {"type": "http", "method": "POST", "path": path, "headers": list(headers)}
@staticmethod
def _chunked_receive(chunks):
messages = [{"type": "http.request", "body": c, "more_body": True} for c in chunks]
messages[-1]["more_body"] = False
messages.append({"type": "http.disconnect"})
it = iter(messages)
async def receive():
return next(it)
return receive
async def test_stream_over_ceiling_answers_413_once_and_stops_the_reader(self):
held = bytearray()
downstream_sent: list[dict] = []
downstream_error: list[BaseException] = []
async def downstream(scope, receive, send):
try:
while True:
m = await receive()
if m["type"] != "http.request":
break
held.extend(m["body"])
if not m.get("more_body"):
break
except proxy_helpers.RequestBodyTooLarge as exc:
downstream_error.append(exc)
# A real handler that swallowed this would now try to answer;
# that answer must be discarded because the 413 already went out.
await send({"type": "http.response.start", "status": 400, "headers": []})
await send({"type": "http.response.body", "body": b"{}"})
downstream_sent.append({"attempted": True})
return
raise AssertionError("reader should have been stopped")
sent: list[dict] = []
async def send(m):
sent.append(m)
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
await mw(self._http_scope(), self._chunked_receive([b"a" * 400] * 5), send)
assert downstream_error, "the reader was not told to stop"
# The message that trips the ceiling is never delivered, so the reader
# holds at most the ceiling itself.
assert 0 < len(held) <= 1000
statuses = [m["status"] for m in sent if m["type"] == "http.response.start"]
assert statuses == [413] # exactly one refusal; the downstream 400 was dropped
assert downstream_sent == [{"attempted": True}]
@pytest.mark.parametrize(
"path",
[
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke",
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke-with-response-stream",
],
ids=["invoke", "invoke-with-response-stream"],
)
async def test_stream_trip_answers_once_in_bedrock_dialect(self, path):
"""On a metered-stream trip over a Bedrock InvokeModel route the single
refusal carries handle_bedrock_invoke's dialect, and the handler's own
413 answer (its RequestBodyTooLarge response) is discarded by the
response-once state: exactly one response reaches the wire.
"""
downstream_sent: list[dict] = []
async def downstream(scope, receive, send):
try:
while True:
m = await receive()
if m["type"] != "http.request":
break
if not m.get("more_body"):
break
except proxy_helpers.RequestBodyTooLarge:
# handle_bedrock_invoke answers RequestBodyTooLarge with its own
# 413; response-once must drop it because the ceiling already answered.
body = b'{"error": {"type": "request_too_large", "message": "from handler"}}'
await send(
{
"type": "http.response.start",
"status": 413,
"headers": [
(b"content-type", b"application/json"),
(b"content-length", str(len(body)).encode()),
],
}
)
await send({"type": "http.response.body", "body": body})
downstream_sent.append({"attempted": True})
return
raise AssertionError("reader should have been stopped")
sent: list[dict] = []
async def send(m):
sent.append(m)
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
await mw(self._http_scope(path=path), self._chunked_receive([b"a" * 400] * 5), send)
statuses = [m["status"] for m in sent if m["type"] == "http.response.start"]
assert statuses == [413] # exactly one response; the handler's 413 was dropped
body = b"".join(m.get("body") or b"" for m in sent if m["type"] == "http.response.body")
assert json.loads(body) == {
"error": {
"type": "request_too_large",
"message": "Request body too large. Maximum size is 0MB",
}
}
assert downstream_sent == [{"attempted": True}]
async def test_body_under_ceiling_is_delivered_untouched(self):
held = bytearray()
async def downstream(scope, receive, send):
while True:
m = await receive()
held.extend(m["body"])
if not m.get("more_body"):
break
await send({"type": "http.response.start", "status": 200, "headers": []})
await send({"type": "http.response.body", "body": b"ok"})
sent: list[dict] = []
async def send(m):
sent.append(m)
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
await mw(self._http_scope(), self._chunked_receive([b"a" * 300] * 3), send)
assert bytes(held) == b"a" * 900
assert [m["status"] for m in sent if m["type"] == "http.response.start"] == [200]
async def test_response_already_started_is_not_clobbered(self):
"""A streaming handler that answered before finishing the body keeps its response."""
async def downstream(scope, receive, send):
await send({"type": "http.response.start", "status": 200, "headers": []})
await send({"type": "http.response.body", "body": b"partial", "more_body": True})
try:
while True:
m = await receive()
if not m.get("more_body"):
break
except proxy_helpers.RequestBodyTooLarge:
await send({"type": "http.response.body", "body": b"", "more_body": False})
sent: list[dict] = []
async def send(m):
sent.append(m)
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
await mw(self._http_scope(), self._chunked_receive([b"a" * 600] * 3), send)
assert [m["status"] for m in sent if m["type"] == "http.response.start"] == [200]
assert sent[-1] == {"type": "http.response.body", "body": b"", "more_body": False}
async def test_unparseable_content_length_falls_back_to_metering(self):
async def downstream(scope, receive, send):
while True:
m = await receive()
if not m.get("more_body"):
break
sent: list[dict] = []
async def send(m):
sent.append(m)
mw = RequestBodyLimitMiddleware(downstream, max_bytes=100)
scope = self._http_scope(headers=[(b"content-length", b"not-a-number")])
await mw(scope, self._chunked_receive([b"a" * 80, b"a" * 80]), send)
assert [m["status"] for m in sent if m["type"] == "http.response.start"] == [413]
async def test_non_http_scope_passes_through(self):
seen = []
async def downstream(scope, receive, send):
seen.append(scope["type"])
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1)
await mw({"type": "websocket", "path": "/v1/live"}, None, None)
await mw({"type": "lifespan"}, None, None)
assert seen == ["websocket", "lifespan"]