mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description With `HEADROOM_PROXY_TOKEN` set, every Anthropic SDK client logs a `proxy_auth_rejected` line once per process, Claude Code included and even when it is configured correctly. Claude Code sends an unauthenticated `HEAD /api/hello` reachability probe. That probe never carries `ANTHROPIC_CUSTOM_HEADERS`, so it cannot hold the proxy token: it gets a 401, which the client ignores, and the rejection lands in the log. On a busy deployment these lines bury real rejections. Behind a gateway, the rejection line also showed only the gateway's peer address, so a rejected caller could not be identified. This PR: 1. **Answers the probe locally, behind the token.** The HTTP security gate lets GET/HEAD on exactly `/api/hello` (also under `/p/<project>`) past the token check and marks it. Extension middleware still runs, so an extension can still refuse it. The `/api/hello` passthrough route answers a marked probe with an empty 200 (the gate adds the security headers) and never relays it. This only happens when a token is configured. It is deliberately **not** added to `_AUTH_EXEMPT_PATHS`, which would hand the probe to the passthrough relay: an anonymous upstream request (to Anthropic since #4059, to the OpenAI default target before it). Without a token, the probe keeps its #4059 route unchanged. 2. **Gives the rejection line context.** `proxy_auth_rejected` (HTTP and WebSocket) adds `method=`, `ua=` and `xff=` fields. Both headers are caller-controlled, so neither is logged as received; each maps to a fixed vocabulary: - `ua`: the harness name `CLIENT_UA_MAP` gives the User-Agent, the same name the outcome and cost records use (`claude-code`, `codex`, `cursor`, `opencode`, `aider`, `copilot`, …). Failing that, the product token of a generic HTTP client (`node`, `anthropic`, `openai`, `curl`, `mozilla`, `python-httpx`, …). Otherwise `other`. `X-Client` is not consulted, since it is free text. - `xff`: the leftmost `X-Forwarded-For` hop when it parses as a plain IP address, else `invalid`. An IPv6 zone index (`fe80::1%<zone>`) is free text, so it counts as `invalid`. The hop is unverified, used only for attribution, and drives no decision. - Either is `none` when its header is absent. - No credential header is read for the log. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/server.py`: - **`_security_gate`**: when a token is configured, `GET`/`HEAD` on exactly `/api/hello`, matched on the routed path so `/p/<project>/api/hello` counts too, skips the token check and sets `request.state.headroom_local_hello_probe`. It then calls `call_next` like any request, so extension middleware (installed inside the gate) still runs and can refuse it. `POST /api/hello`, `/api/hello/` and `/api/hello/x` still need the token. - **New helper**: `_auth_rejection_context` returns `(ua, xff)` from the fixed vocabulary above: `CLIENT_UA_MAP` then an allowlist of generic clients (`_AUTH_REJECTION_CLIENTS`) for the User-Agent, and `ipaddress.ip_address` on the first hop (from `forwarded_policy.header_first`). - **Both `proxy_auth_rejected` warnings** gain the new fields: the HTTP gate gets `method=`, and the WebSocket middleware gets the same `ua=`/`xff=` fields, so the two transports stay aligned. - `headroom/providers/proxy_routes.py`: the provider passthrough route answers a marked probe with an empty 200 and never relays it upstream. - `tests/test_proxy_hardening.py`: - **Probe answer** (`test_api_hello_probe_is_answered_locally`): GET/HEAD on `/api/hello` and `/p/demo/api/hello` from a non-loopback peer gets 200 with an empty body and security headers, and the passthrough spy shows nothing relayed. `POST /p/demo/api/hello` and `GET /p/demo/stats` still get 401. `POST /api/hello`, `GET /api/hello/x`, `GET /api/hello/` and `GET /stats` still get 401, with nothing relayed. With no token configured, `/api/hello` still reaches the passthrough. - **Log line**: `claude-cli/2.1.0 (external, cli)` with `X-Forwarded-For: 198.51.100.7, 10.0.0.9` logs `ua=claude-code xff=198.51.100.7`. Absent headers log `ua=none xff=none`. `test_known_clients_keep_their_proxy_wide_name` covers `claude-code`, `codex`, `cursor`, `opencode`, `aider` behind a wrapper prefix, the Anthropic SDK and `node`. - **Caller text never reaches the line** (`test_caller_text_never_reaches_the_line`): a `PRIVATE_CREDENTIAL_CANARY` in the User-Agent (bare, as a product token, behind a fake quoted field) and in X-Forwarded-For (as the first hop, as a zone index, as a port suffix) never appears in the line, which logs `other`/`invalid` instead. CR/LF in either header maps to `other`/`invalid` (tested on the helper directly, since an HTTP client refuses to send CR/LF). - **No credentials**: neither the configured token, a wrong `x-headroom-proxy-token`, `Authorization: Bearer …` nor `x-api-key` ever appears in the line. - **WebSocket**: the rejection line matches the HTTP gate's, and the same canary in both headers never appears in it (`test_rejection_log_keeps_caller_text_out`). - `tests/test_proxy_extension_ordering.py`: - **Extension policy** (`test_api_hello_probe_still_runs_extension_policy`): an extension that answers 403 to everything gets the token-less GET/HEAD probe and refuses it. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_proxy_hardening.py tests/test_anthropic_hello_probe_routing.py \ tests/test_provider_proxy_routes.py tests/test_provider_route_specs.py \ tests/test_proxy_loopback_gating.py tests/test_proxy_extension_ordering.py \ tests/test_proxy_bind_policy.py tests/test_auth_mode.py -q 365 passed in 78.96s $ ruff check headroom/proxy/server.py tests/test_proxy_hardening.py All checks passed! $ ruff format --check headroom/proxy/server.py tests/test_proxy_hardening.py 2 files already formatted $ python -m mypy headroom/proxy/server.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: this branch on Windows 11 with Python 3.13. Ran `headroom proxy --port 18787` from the worktree with a throwaway `HEADROOM_PROXY_TOKEN`. - Exact command / steps: `curl -I http://127.0.0.1:18787/api/hello`, `curl http://127.0.0.1:18787/api/hello`, then checked the proxy log. - Observed result: HEAD and GET returned `200` with 0 bytes and `x-content-type-options: nosniff`, `x-frame-options: DENY`, `referrer-policy: no-referrer` and `strict-transport-security`. The proxy log has no `proxy_inbound_request` and no `outbound_headers` line for either probe, so nothing was forwarded. For contrast, a loopback `POST /api/hello` (loopback is token-exempt, existing behavior) logs `forwarder=openai_passthrough`. - Not tested: the new rejection-line fields behind a real gateway. They were checked against a real non-loopback peer, below. Rejection diagnostics, at `d70ec3c` (contains current `main`): the same worktree proxy, bound to a non-loopback LAN address (`--host 192.168.0.48 --port 18787 --no-optimize`, `HEADROOM_OFFLINE=1`) with a throwaway token, called from that address: ```text GET /stats UA=PRIVATE_CREDENTIAL_CANARY XFF=PRIVATE_CREDENTIAL_CANARY, 192.0.2.1 -> 401 GET /stats UA=claude-cli/2.1.0 (external, cli) XFF=198.51.100.7, 10.0.0.9 -> 401 GET /stats UA=codex-cli/0.46.0 -> 401 GET /stats UA=PRIVATE_CREDENTIAL_CANARY/9 opencode/1.2.3 -> 401 GET /stats no UA, no XFF -> 401 GET /stats UA=PRIVATE_CREDENTIAL_CANARY/1.0 XFF=fe80::1%PRIVATE_CREDENTIAL_CANARY -> 401 GET /stats valid x-headroom-proxy-token -> 200 HEAD /api/hello -> 200 WS upgrade /v1/responses UA=PRIVATE_CREDENTIAL_CANARY XFF=PRIVATE_CREDENTIAL_CANARY -> 403 event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=other xff=invalid event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=claude-code xff=198.51.100.7 event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=codex xff=none event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=opencode xff=none event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=none xff=none event=proxy_auth_rejected method=GET path=/stats client=192.168.0.48 reason=missing_token ua=other xff=invalid event=proxy_auth_rejected transport=websocket path=/v1/responses client=192.168.0.48 reason=missing_token ua=other xff=invalid $ grep -c PRIVATE_CREDENTIAL_CANARY proxy.out 0 ``` The HEAD `/api/hello` probe from that non-loopback peer logs `proxy_inbound_request` / `proxy_inbound_response` (`status=200`, `response_content_length=0`) and no `outbound_headers` line. Since `d125805` the probe runs through the inbound request logging like any routed request; it is still never forwarded. The loopback run above predates that commit. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: n/a. - Stable/default behavior changed: only when `HEADROOM_PROXY_TOKEN` is set. GET/HEAD `/api/hello` then gets a local 200 instead of a 401, and the rejection line gains three fields. Answered probes appear in the inbound request log (`proxy_inbound_request` / `proxy_inbound_response`) instead of as `proxy_auth_rejected` warnings. With no token, behavior is unchanged. - Kill switch / disable path: none needed. The probe answer carries no data and makes no upstream call. - Unsafe override required: no. - Qualification impact: log parsers that match the exact old `proxy_auth_rejected` line need to accept the extra trailing fields. The `path=`, `client=` and `reason=` fields are unchanged; the HTTP line now starts with `method=` before `path=`. - Rollback path: revert the commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Documentation: N/A. The docs don't list the token-exempt paths or the `proxy_auth_rejected` format. - Why only when a token is configured: without a token, the #4059 passthrough route is fine. Answering locally only matters when the alternative is a 401 for every client. - `xff` is not resolved through `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`: the gate runs before forwarded-header resolution, and resolving here would emit untrusted-peer events on every rejection. It is logged only as a validated IP literal (or `invalid`/`none`), unverified, for attribution. - `ua` reuses `CLIENT_UA_MAP`, so a harness added there is named in the rejection line too. Generic clients outside `_AUTH_REJECTION_CLIENTS` log `other`; adding one is a one-line change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4150" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ### Maintainer combined-tree qualification On the integration with main `31dcaf41`, all 375 focused hardening, hello-routing, provider-route/spec, loopback, extension-ordering, bind-policy, and auth-mode tests passed on Windows/Python 3.12 using the existing native extension. Changed-file lint, formatting, and whitespace checks passed. The provider-route insertion conflict retains the PR local-probe early return before main per-request upstream-override processing; AST comparison proves the combined route is the unchanged PR guard followed by the unchanged main route body. The probe test now also supplies a loopback or metadata-address upstream override and confirms the response stays empty/local with nothing relayed. This checks the integration boundary without rewriting contributor authorization or forwarding logic. Hosted qualification for the published sync remains pending. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
580 lines
24 KiB
Python
580 lines
24 KiB
Python
"""Extension middleware runs inside the proxy's security gate and body ceiling.
|
|
|
|
Regression tests for the ordering contract documented in
|
|
``headroom/proxy/extensions.py``. Before this contract existed, ``install_all``
|
|
ran after the gate was registered, and Starlette's prepend-on-register stack
|
|
made every extension middleware the OUTERMOST layer: an extension that answered
|
|
``POST /`` itself served unauthenticated callers, one that buffered the body
|
|
held an unbounded body before the 401 was ever issued, and one that rewrote
|
|
``Authorization`` did so before the gate compared it.
|
|
|
|
The extension used here is a stand-in for the shapes the private extensions
|
|
actually take (a middleware that claims a request and answers it, and one that
|
|
buffers the whole body), installed through the real ``install_all`` seam by
|
|
monkeypatching discovery, so the tests exercise the app exactly as
|
|
``headroom proxy --proxy-extension <name>`` would.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from typing import Any
|
|
|
|
import pytest
|
|
from fastapi.testclient import TestClient
|
|
|
|
from headroom.cache.compression_store import reset_compression_store
|
|
from headroom.proxy import extensions
|
|
from headroom.proxy import helpers as proxy_helpers
|
|
from headroom.proxy.request_body_limit import RequestBodyLimitMiddleware
|
|
from headroom.proxy.server import ProxyConfig, WebSocketAuthMiddleware, create_app
|
|
|
|
NONLOOPBACK = ("203.0.113.5", 44444) # TEST-NET-3, never loopback
|
|
LOOPBACK = ("127.0.0.1", 12345)
|
|
TOKEN = "s3cr3t-token"
|
|
|
|
|
|
class _RecordingExtension:
|
|
"""Raw ASGI middleware that records every scope it sees and answers ``POST /``.
|
|
|
|
Mirrors the shape of an extension that intercepts a vendor protocol on the
|
|
root path and relays it upstream itself, never calling the wrapped app.
|
|
"""
|
|
|
|
seen: list[dict[str, Any]] = []
|
|
|
|
def __init__(self, app: Any) -> None:
|
|
self.app = app
|
|
|
|
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
|
|
if scope["type"] in {"http", "websocket"}:
|
|
type(self).seen.append({"type": scope["type"], "path": scope.get("path")})
|
|
if scope["type"] == "http" and scope.get("path") == "/" and scope.get("method") == "POST":
|
|
body = b'{"claimed_by":"extension"}'
|
|
await send(
|
|
{
|
|
"type": "http.response.start",
|
|
"status": 200,
|
|
"headers": [
|
|
(b"content-type", b"application/json"),
|
|
(b"content-length", str(len(body)).encode()),
|
|
],
|
|
}
|
|
)
|
|
await send({"type": "http.response.body", "body": body})
|
|
return
|
|
await self.app(scope, receive, send)
|
|
|
|
|
|
class _BufferingExtension:
|
|
"""Raw ASGI middleware that buffers the full request body before passing it on.
|
|
|
|
Mirrors an extension that must read the JSON to decide how to route or
|
|
rewrite the request. ``max_seen`` records the largest body it ever held.
|
|
"""
|
|
|
|
max_seen = 0
|
|
|
|
def __init__(self, app: Any) -> None:
|
|
self.app = app
|
|
|
|
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
|
|
if scope["type"] != "http":
|
|
await self.app(scope, receive, send)
|
|
return
|
|
chunks = bytearray()
|
|
while True:
|
|
message = await receive()
|
|
if message["type"] != "http.request":
|
|
break
|
|
chunks.extend(message.get("body") or b"")
|
|
type(self).max_seen = max(type(self).max_seen, len(chunks))
|
|
if not message.get("more_body"):
|
|
break
|
|
replay = {"type": "http.request", "body": bytes(chunks), "more_body": False}
|
|
sent = False
|
|
|
|
async def replay_receive() -> dict[str, Any]:
|
|
nonlocal sent
|
|
if not sent:
|
|
sent = True
|
|
return replay
|
|
return {"type": "http.disconnect"}
|
|
|
|
await self.app(scope, replay_receive, send)
|
|
|
|
|
|
class _DenyAllExtension:
|
|
"""Raw ASGI middleware that refuses every HTTP request with 403.
|
|
|
|
Mirrors an extension that enforces its own access policy (an allowlist, a
|
|
licence gate) on everything the proxy serves.
|
|
"""
|
|
|
|
seen: list[str] = []
|
|
|
|
def __init__(self, app: Any) -> None:
|
|
self.app = app
|
|
|
|
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
|
|
if scope["type"] != "http":
|
|
await self.app(scope, receive, send)
|
|
return
|
|
type(self).seen.append(scope.get("path"))
|
|
await send({"type": "http.response.start", "status": 403, "headers": []})
|
|
await send({"type": "http.response.body", "body": b""})
|
|
|
|
|
|
def _install_deny_all(app: Any, config: Any) -> None:
|
|
app.add_middleware(_DenyAllExtension)
|
|
|
|
|
|
def _install_recording(app: Any, config: Any) -> None:
|
|
app.add_middleware(_RecordingExtension)
|
|
|
|
|
|
def _install_buffering(app: Any, config: Any) -> None:
|
|
app.add_middleware(_BufferingExtension)
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _reset_state() -> None:
|
|
_RecordingExtension.seen = []
|
|
_BufferingExtension.max_seen = 0
|
|
_DenyAllExtension.seen = []
|
|
|
|
|
|
def _make_app(monkeypatch, install, **overrides) -> Any:
|
|
reset_compression_store()
|
|
monkeypatch.setattr(extensions, "discover", lambda: iter([("test_ext", install)]))
|
|
config = ProxyConfig(
|
|
optimize=False,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
proxy_extensions=["test_ext"],
|
|
**overrides,
|
|
)
|
|
return create_app(config)
|
|
|
|
|
|
class TestExtensionMiddlewareIsInsideTheSecurityGate:
|
|
def test_unauthenticated_request_never_reaches_extension_middleware(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post("/", json={}, headers={"x-amz-target": "Vendor.Generate"})
|
|
assert resp.status_code == 401
|
|
assert resp.json() == {"error": "unauthorized"}
|
|
assert _RecordingExtension.seen == [], "extension ran before the proxy token was checked"
|
|
|
|
def test_authenticated_request_reaches_extension_middleware(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post("/", json={}, headers={"Authorization": f"Bearer {TOKEN}"})
|
|
assert resp.status_code == 200
|
|
assert resp.json() == {"claimed_by": "extension"}
|
|
assert [s["path"] for s in _RecordingExtension.seen] == ["/"]
|
|
|
|
def test_wrong_token_never_reaches_extension_middleware(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post("/", json={}, headers={"Authorization": "Bearer nope"})
|
|
assert resp.status_code == 401
|
|
assert _RecordingExtension.seen == []
|
|
|
|
def test_loopback_caller_is_exempt_and_reaches_extension(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://127.0.0.1", client=LOOPBACK) as c:
|
|
resp = c.post("/", json={})
|
|
assert resp.status_code == 200
|
|
assert [s["path"] for s in _RecordingExtension.seen] == ["/"]
|
|
|
|
@pytest.mark.parametrize("method", ["GET", "HEAD"])
|
|
def test_api_hello_probe_still_runs_extension_policy(self, monkeypatch, method):
|
|
"""The token-less /api/hello probe passes the token gate, but an
|
|
extension that refuses everything still refuses it."""
|
|
app = _make_app(monkeypatch, _install_deny_all, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.request(method, "/api/hello")
|
|
assert resp.status_code == 403
|
|
assert _DenyAllExtension.seen == ["/api/hello"]
|
|
|
|
def test_health_probe_stays_exempt_with_extension_installed(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.get("/health")
|
|
assert resp.status_code == 200
|
|
# Exempt from auth, so the extension legitimately sees it — the gate
|
|
# passes it through rather than the extension pre-empting the gate.
|
|
assert [s["path"] for s in _RecordingExtension.seen] == ["/health"]
|
|
|
|
def test_gate_security_headers_apply_to_extension_responses(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post("/", json={}, headers={"Authorization": f"Bearer {TOKEN}"})
|
|
assert resp.headers["X-Content-Type-Options"] == "nosniff"
|
|
|
|
def test_unauthenticated_websocket_never_reaches_extension_middleware(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
with pytest.raises(Exception): # noqa: B017 — TestClient surfaces the refused upgrade
|
|
with c.websocket_connect("/v1/responses"):
|
|
pass
|
|
assert _RecordingExtension.seen == [], "extension saw a refused WebSocket handshake"
|
|
|
|
|
|
class TestBodyCeilingAppliesBeforeExtensionMiddleware:
|
|
def test_declared_oversize_body_is_refused_before_extension_buffers_it(self, monkeypatch):
|
|
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
|
|
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
|
|
payload = b"x" * 4096
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post(
|
|
"/v1/chat/completions",
|
|
content=payload,
|
|
headers={
|
|
"Authorization": f"Bearer {TOKEN}",
|
|
"content-type": "application/json",
|
|
},
|
|
)
|
|
assert resp.status_code == 413
|
|
assert resp.json()["error"]["code"] == "request_too_large"
|
|
assert _BufferingExtension.max_seen == 0
|
|
|
|
def test_chunked_oversize_body_is_cut_off_before_extension_holds_it_all(self, monkeypatch):
|
|
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
|
|
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
|
|
chunk = b"x" * 256
|
|
|
|
def body():
|
|
for _ in range(64): # 16 KiB, no Content-Length
|
|
yield chunk
|
|
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post(
|
|
"/v1/chat/completions",
|
|
content=body(),
|
|
headers={
|
|
"Authorization": f"Bearer {TOKEN}",
|
|
"content-type": "application/json",
|
|
},
|
|
)
|
|
assert resp.status_code == 413
|
|
# The ceiling plus at most one delivered message; never the whole 16 KiB.
|
|
# (TestClient delivers the generator as one message, so the extension
|
|
# sees nothing at all here; the multi-message case is covered by
|
|
# TestRequestBodyLimitMiddlewareDirectly below.)
|
|
assert _BufferingExtension.max_seen <= 1024 + len(chunk)
|
|
|
|
def test_anthropic_dialect_payload_on_messages_path(self, monkeypatch):
|
|
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
|
|
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post(
|
|
"/v1/messages",
|
|
content=b"x" * 4096,
|
|
headers={"Authorization": f"Bearer {TOKEN}", "content-type": "application/json"},
|
|
)
|
|
assert resp.status_code == 413
|
|
assert resp.json() == {
|
|
"type": "error",
|
|
"error": {
|
|
"type": "request_too_large",
|
|
"message": "Request body too large. Maximum size is 0MB",
|
|
},
|
|
}
|
|
|
|
@pytest.mark.parametrize(
|
|
"route",
|
|
[
|
|
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke",
|
|
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke-with-response-stream",
|
|
],
|
|
ids=["invoke", "invoke-with-response-stream"],
|
|
)
|
|
def test_bedrock_invoke_dialect_payload_on_model_paths(self, monkeypatch, route):
|
|
"""The two Bedrock InvokeModel routes keep handle_bedrock_invoke's wire
|
|
dialect on the body ceiling: exactly
|
|
``{"error": {"type": "request_too_large", "message": ...}}`` — not the
|
|
OpenAI-style payload — and the extension never buffers the body.
|
|
"""
|
|
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
|
|
app = _make_app(
|
|
monkeypatch,
|
|
_install_buffering,
|
|
proxy_token=TOKEN,
|
|
bedrock_api_url="http://127.0.0.1:4000",
|
|
)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post(
|
|
route,
|
|
content=b"x" * 4096,
|
|
headers={"Authorization": f"Bearer {TOKEN}", "content-type": "application/json"},
|
|
)
|
|
assert resp.status_code == 413
|
|
assert resp.json() == {
|
|
"error": {
|
|
"type": "request_too_large",
|
|
"message": "Request body too large. Maximum size is 0MB",
|
|
}
|
|
}
|
|
assert _BufferingExtension.max_seen == 0
|
|
|
|
def test_body_within_ceiling_reaches_extension_intact(self, monkeypatch):
|
|
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
|
|
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
# /stats is a local route, so the request completes without an upstream.
|
|
resp = c.post(
|
|
"/stats",
|
|
content=b"y" * 512,
|
|
headers={"Authorization": f"Bearer {TOKEN}"},
|
|
)
|
|
assert resp.status_code != 413
|
|
assert _BufferingExtension.max_seen == 512
|
|
|
|
def test_unauthenticated_oversize_body_is_refused_by_the_gate_first(self, monkeypatch):
|
|
monkeypatch.setattr(proxy_helpers, "MAX_REQUEST_BODY_SIZE", 1024)
|
|
app = _make_app(monkeypatch, _install_buffering, proxy_token=TOKEN)
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
resp = c.post("/v1/chat/completions", content=b"x" * 4096)
|
|
assert resp.status_code == 401
|
|
assert _BufferingExtension.max_seen == 0
|
|
|
|
|
|
class TestStackOrderIsPinned:
|
|
"""The contract is structural, so check the stack itself, not only its effects."""
|
|
|
|
@staticmethod
|
|
def _index_of(app, predicate) -> int:
|
|
for index, entry in enumerate(app.user_middleware):
|
|
if predicate(entry):
|
|
return index
|
|
raise AssertionError("middleware not found in stack")
|
|
|
|
def test_extension_middleware_is_inner_to_gate_and_body_limit(self, monkeypatch):
|
|
app = _make_app(monkeypatch, _install_recording, proxy_token=TOKEN)
|
|
added = app.state.extension_middleware
|
|
assert len(added) == 1
|
|
assert added[0].cls is _RecordingExtension
|
|
|
|
# Lower index == outer layer (Starlette prepends on add_middleware).
|
|
ext_index = self._index_of(app, lambda e: e.cls is _RecordingExtension)
|
|
limit_index = self._index_of(app, lambda e: e.cls is RequestBodyLimitMiddleware)
|
|
ws_auth_index = self._index_of(app, lambda e: e.cls is WebSocketAuthMiddleware)
|
|
gate_index = self._index_of(
|
|
app,
|
|
lambda e: getattr(e.kwargs.get("dispatch"), "__name__", "") == "_security_gate",
|
|
)
|
|
assert ws_auth_index < gate_index < limit_index < ext_index
|
|
|
|
def test_install_all_records_nothing_for_routes_only_extensions(self, monkeypatch):
|
|
def routes_only(app: Any, config: Any) -> None:
|
|
@app.get("/ext/ping")
|
|
async def ping():
|
|
return {"ok": True}
|
|
|
|
app = _make_app(monkeypatch, routes_only, proxy_token=TOKEN)
|
|
assert app.state.extension_middleware == []
|
|
with TestClient(app, base_url="http://testserver", client=NONLOOPBACK) as c:
|
|
assert c.get("/ext/ping").status_code == 401
|
|
assert c.get("/ext/ping", headers={"Authorization": f"Bearer {TOKEN}"}).json() == {
|
|
"ok": True
|
|
}
|
|
|
|
|
|
class TestRequestBodyLimitMiddlewareDirectly:
|
|
"""Drive the layer with a hand-rolled ASGI conversation to pin the metering."""
|
|
|
|
@staticmethod
|
|
def _http_scope(path="/v1/chat/completions", headers=()):
|
|
return {"type": "http", "method": "POST", "path": path, "headers": list(headers)}
|
|
|
|
@staticmethod
|
|
def _chunked_receive(chunks):
|
|
messages = [{"type": "http.request", "body": c, "more_body": True} for c in chunks]
|
|
messages[-1]["more_body"] = False
|
|
messages.append({"type": "http.disconnect"})
|
|
it = iter(messages)
|
|
|
|
async def receive():
|
|
return next(it)
|
|
|
|
return receive
|
|
|
|
async def test_stream_over_ceiling_answers_413_once_and_stops_the_reader(self):
|
|
held = bytearray()
|
|
downstream_sent: list[dict] = []
|
|
downstream_error: list[BaseException] = []
|
|
|
|
async def downstream(scope, receive, send):
|
|
try:
|
|
while True:
|
|
m = await receive()
|
|
if m["type"] != "http.request":
|
|
break
|
|
held.extend(m["body"])
|
|
if not m.get("more_body"):
|
|
break
|
|
except proxy_helpers.RequestBodyTooLarge as exc:
|
|
downstream_error.append(exc)
|
|
# A real handler that swallowed this would now try to answer;
|
|
# that answer must be discarded because the 413 already went out.
|
|
await send({"type": "http.response.start", "status": 400, "headers": []})
|
|
await send({"type": "http.response.body", "body": b"{}"})
|
|
downstream_sent.append({"attempted": True})
|
|
return
|
|
raise AssertionError("reader should have been stopped")
|
|
|
|
sent: list[dict] = []
|
|
|
|
async def send(m):
|
|
sent.append(m)
|
|
|
|
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
|
|
await mw(self._http_scope(), self._chunked_receive([b"a" * 400] * 5), send)
|
|
|
|
assert downstream_error, "the reader was not told to stop"
|
|
# The message that trips the ceiling is never delivered, so the reader
|
|
# holds at most the ceiling itself.
|
|
assert 0 < len(held) <= 1000
|
|
statuses = [m["status"] for m in sent if m["type"] == "http.response.start"]
|
|
assert statuses == [413] # exactly one refusal; the downstream 400 was dropped
|
|
assert downstream_sent == [{"attempted": True}]
|
|
|
|
@pytest.mark.parametrize(
|
|
"path",
|
|
[
|
|
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke",
|
|
"/model/anthropic.claude-3-5-sonnet-20241022-v2:0/invoke-with-response-stream",
|
|
],
|
|
ids=["invoke", "invoke-with-response-stream"],
|
|
)
|
|
async def test_stream_trip_answers_once_in_bedrock_dialect(self, path):
|
|
"""On a metered-stream trip over a Bedrock InvokeModel route the single
|
|
refusal carries handle_bedrock_invoke's dialect, and the handler's own
|
|
413 answer (its RequestBodyTooLarge response) is discarded by the
|
|
response-once state: exactly one response reaches the wire.
|
|
"""
|
|
downstream_sent: list[dict] = []
|
|
|
|
async def downstream(scope, receive, send):
|
|
try:
|
|
while True:
|
|
m = await receive()
|
|
if m["type"] != "http.request":
|
|
break
|
|
if not m.get("more_body"):
|
|
break
|
|
except proxy_helpers.RequestBodyTooLarge:
|
|
# handle_bedrock_invoke answers RequestBodyTooLarge with its own
|
|
# 413; response-once must drop it because the ceiling already answered.
|
|
body = b'{"error": {"type": "request_too_large", "message": "from handler"}}'
|
|
await send(
|
|
{
|
|
"type": "http.response.start",
|
|
"status": 413,
|
|
"headers": [
|
|
(b"content-type", b"application/json"),
|
|
(b"content-length", str(len(body)).encode()),
|
|
],
|
|
}
|
|
)
|
|
await send({"type": "http.response.body", "body": body})
|
|
downstream_sent.append({"attempted": True})
|
|
return
|
|
raise AssertionError("reader should have been stopped")
|
|
|
|
sent: list[dict] = []
|
|
|
|
async def send(m):
|
|
sent.append(m)
|
|
|
|
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
|
|
await mw(self._http_scope(path=path), self._chunked_receive([b"a" * 400] * 5), send)
|
|
|
|
statuses = [m["status"] for m in sent if m["type"] == "http.response.start"]
|
|
assert statuses == [413] # exactly one response; the handler's 413 was dropped
|
|
body = b"".join(m.get("body") or b"" for m in sent if m["type"] == "http.response.body")
|
|
assert json.loads(body) == {
|
|
"error": {
|
|
"type": "request_too_large",
|
|
"message": "Request body too large. Maximum size is 0MB",
|
|
}
|
|
}
|
|
assert downstream_sent == [{"attempted": True}]
|
|
|
|
async def test_body_under_ceiling_is_delivered_untouched(self):
|
|
held = bytearray()
|
|
|
|
async def downstream(scope, receive, send):
|
|
while True:
|
|
m = await receive()
|
|
held.extend(m["body"])
|
|
if not m.get("more_body"):
|
|
break
|
|
await send({"type": "http.response.start", "status": 200, "headers": []})
|
|
await send({"type": "http.response.body", "body": b"ok"})
|
|
|
|
sent: list[dict] = []
|
|
|
|
async def send(m):
|
|
sent.append(m)
|
|
|
|
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
|
|
await mw(self._http_scope(), self._chunked_receive([b"a" * 300] * 3), send)
|
|
assert bytes(held) == b"a" * 900
|
|
assert [m["status"] for m in sent if m["type"] == "http.response.start"] == [200]
|
|
|
|
async def test_response_already_started_is_not_clobbered(self):
|
|
"""A streaming handler that answered before finishing the body keeps its response."""
|
|
|
|
async def downstream(scope, receive, send):
|
|
await send({"type": "http.response.start", "status": 200, "headers": []})
|
|
await send({"type": "http.response.body", "body": b"partial", "more_body": True})
|
|
try:
|
|
while True:
|
|
m = await receive()
|
|
if not m.get("more_body"):
|
|
break
|
|
except proxy_helpers.RequestBodyTooLarge:
|
|
await send({"type": "http.response.body", "body": b"", "more_body": False})
|
|
|
|
sent: list[dict] = []
|
|
|
|
async def send(m):
|
|
sent.append(m)
|
|
|
|
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1000)
|
|
await mw(self._http_scope(), self._chunked_receive([b"a" * 600] * 3), send)
|
|
assert [m["status"] for m in sent if m["type"] == "http.response.start"] == [200]
|
|
assert sent[-1] == {"type": "http.response.body", "body": b"", "more_body": False}
|
|
|
|
async def test_unparseable_content_length_falls_back_to_metering(self):
|
|
async def downstream(scope, receive, send):
|
|
while True:
|
|
m = await receive()
|
|
if not m.get("more_body"):
|
|
break
|
|
|
|
sent: list[dict] = []
|
|
|
|
async def send(m):
|
|
sent.append(m)
|
|
|
|
mw = RequestBodyLimitMiddleware(downstream, max_bytes=100)
|
|
scope = self._http_scope(headers=[(b"content-length", b"not-a-number")])
|
|
await mw(scope, self._chunked_receive([b"a" * 80, b"a" * 80]), send)
|
|
assert [m["status"] for m in sent if m["type"] == "http.response.start"] == [413]
|
|
|
|
async def test_non_http_scope_passes_through(self):
|
|
seen = []
|
|
|
|
async def downstream(scope, receive, send):
|
|
seen.append(scope["type"])
|
|
|
|
mw = RequestBodyLimitMiddleware(downstream, max_bytes=1)
|
|
await mw({"type": "websocket", "path": "/v1/live"}, None, None)
|
|
await mw({"type": "lifespan"}, None, None)
|
|
assert seen == ["websocket", "lifespan"]
|