fix(proxy): gate the raw request body, not just its Content-Length header (#3338)

## Description

Closes #3326.

The four handlers named in the issue (`openai.py`, `batch.py`,
`gemini.py`,
`anthropic.py`) all gate on the `Content-Length` header before calling
`_read_request_json`, but that header is only present when the client
sends
one. A chunked-transfer-encoding request has no `Content-Length`, so it
walks straight past every one of those checks, and
`_read_request_body_bytes`
then reads the whole body through the unbounded `request.body()`.

`_read_request_json` and `_read_request_body_bytes` are the single
chokepoint all six call sites route through (confirmed by reading each
of
the six named lines, plus a repo-wide grep for other
`_read_request_json`
callers), so one change at that chokepoint covers all of them.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Added `_read_request_body_bounded` in `headroom/proxy/helpers.py`:
reads
the body via `Request.stream()` instead of `Request.body()`, refusing
the
  request with `RequestBodyTooLarge` once the running total crosses
  `MAX_REQUEST_BODY_SIZE`, before the rest of the body arrives.
- `_read_request_body_bytes` now reads through it instead of calling
  `request.body()` directly.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_request_body_decompression_limits.py -q
25 passed, 2 skipped in 4.25s   # 2 skipped = brotli not installed in this environment

$ pytest tests/ -k "helpers or request_body or decompress or proxy_body or size_limit or content_length" -q
246 passed, 12 skipped, 3 failed in 62.57s
# The 3 failures are in tests/test_cli/test_wrap_helpers.py::TestProxyClientRefCounting
# (process ref-counting / PID checks) and reproduce identically against
# unmodified main in this sandboxed container, unrelated to this change.

$ ruff check headroom/proxy/helpers.py tests/test_request_body_decompression_limits.py tests/test_proxy_handler_helpers.py
All checks passed!
$ ruff format --check headroom/proxy/helpers.py tests/test_request_body_decompression_limits.py tests/test_proxy_handler_helpers.py
3 files already formatted
```

`mypy headroom --ignore-missing-imports` did not run cleanly in this
environment: it fails at `numpy/__init__.pyi` with "Type statement is
only
supported in Python 3.12 and greater" before reaching this file, a
pre-existing mismatch between the installed numpy stubs and this repo's
pinned `python_version = "3.10"` mypy target, not something this diff
introduces.

One existing test needed a fixture update:
`test_proxy_handler_helpers.py`'s
`_VertexGeminiImageRequest` stand-in only implemented `.body()`, and the
new
code path calls `.stream()` (a real Starlette `Request` always has
both).
Added a `.stream()` method that yields the same body in one chunk.

## Real Behavior Proof

- Environment: Docker `python:3.12-slim`, headroom-ai 0.37.0's published
  wheel (for the compiled `_core` extension) with this branch's
  `headroom/proxy/helpers.py` overlaid on top.
- Exact command / steps: built a genuine Starlette `Request` over a
hand-rolled
chunked ASGI `receive()` channel (no `content-length` header, matching
what
a real chunked-transfer-encoding request looks like at the ASGI layer),
and
called the actual `_read_request_body_bytes` with
`MAX_REQUEST_BODY_SIZE`
turned down to 10KB so the demonstration does not need to stream tens of
  megabytes.
- Observed result: against unmodified `main`, a 50,000-byte chunked body
with
  no `Content-Length` header is read in full with no error. Against this
branch, the same request raises `RequestBodyTooLarge` before the stream
finishes, and a 5,000-byte chunked body under the cap still round-trips
  byte for byte.
- Not tested: brotli-encoded bodies (package not installed in this
sandbox,
same as the two skipped unit tests above), and the repo's own CI matrix
  (Rust extension build, HuggingFace model prefetch) end to end.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: no for any request under
`MAX_REQUEST_BODY_SIZE` (100MB by default); a chunked request that would
previously have been read unbounded is now refused past that size, which
  is the fix.
- Kill switch / disable path: none by design; raising
`MAX_REQUEST_BODY_SIZE`
  widens the ceiling if an operator needs to.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit, one function plus a two-line
call-site
  change in a single file.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did not edit `CHANGELOG.md`

## Additional Notes

Not in scope: a handful of other call sites in these same handler files
(`batch.py`'s and `bedrock.py`'s passthrough forwarders, and two
streaming
passthrough paths in `openai.py`/`anthropic.py`) also call
`request.body()` directly rather than through this chokepoint. None of
them
are among the six lines the issue names, they exist to forward a request
byte-for-byte rather than to parse it as JSON, and fixing them is a
larger,
separate change than this one. Flagging it here rather than leaving it
implicit.

---------

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Amir Fathi
2026-09-29 15:34:50 -05:00
committed by GitHub
co-authored by JD Davis
parent b1b005a9a8
commit c946b6b0b0
4 changed files with 105 additions and 7 deletions
+24
View File
@@ -76,6 +76,8 @@ class BedrockHandlerMixin:
from headroom.proxy.helpers import (
COMPRESSION_TIMEOUT_SECONDS,
MAX_MESSAGE_ARRAY_LENGTH,
MAX_REQUEST_BODY_SIZE,
RequestBodyTooLarge,
_headroom_bypass_enabled,
_strip_internal_headers,
extract_tags,
@@ -134,12 +136,34 @@ class BedrockHandlerMixin:
if isinstance(err, ClientDisconnect):
logger.debug("[%s] %s client disconnected during body read", request_id, LOG_TAG)
return Response(status_code=204)
if isinstance(err, RequestBodyTooLarge):
# The read aborted mid-stream once the size ceiling was
# crossed, so there is no raw body to fail open with, and
# forwarding one anyway would defeat the ceiling. Reject.
logger.warning("[%s] %s request body too large: %s", request_id, LOG_TAG, err)
return JSONResponse(
status_code=413,
content={
"error": {
"type": "request_too_large",
"message": (
f"Request body too large. Maximum size is "
f"{MAX_REQUEST_BODY_SIZE // (1024 * 1024)}MB"
),
}
},
)
logger.warning(
"[%s] %s could not parse body; forwarding verbatim: %s",
request_id,
LOG_TAG,
err,
)
# The raw bytes were already fully read by
# read_request_json_with_bytes before it failed decoding them,
# and it caches them onto request._body the same way Starlette's
# own Request.body() would, so this returns the cached bytes
# instead of re-draining an already-consumed stream.
raw_only = await request.body()
return await self._forward_bedrock(
url=url,
+7 -5
View File
@@ -2893,7 +2893,7 @@ def apply_session_sticky_ccr_tool(
class RequestBodyTooLarge(ValueError):
"""A decompressed request body exceeded :data:`MAX_DECOMPRESSED_BODY_SIZE`.
"""A raw or decompressed request body exceeded its size ceiling.
Subclasses ``ValueError`` so every existing ``except ValueError`` call site
keeps answering 400 unchanged, while giving a caller that would rather
@@ -3024,8 +3024,9 @@ async def _read_request_body_bytes(request: Request) -> bytes:
encoding = (request.headers.get("content-encoding") or "").lower().strip()
# Content-Length is an optimization only, not the enforcement boundary: it
# can be absent, understated, or belong to a chunked transfer. The
# streaming loop below is what actually bounds every case (#3479).
# can be absent, understated, or belong to a chunked transfer (the
# original gap this fixes, #3326). The streaming loop below is what
# actually bounds every case, chunked or not (#3479).
content_length = request.headers.get("content-length")
if content_length is not None:
try:
@@ -3047,8 +3048,9 @@ async def _read_request_body_bytes(request: Request) -> bytes:
)
raw: bytes = bytes(chunks)
# Cache like Starlette's own body() would, so any other .body() caller on
# this request (there is none today, but future callers get the same
# semantics) sees the bytes already read rather than a consumed stream.
# this request (e.g. a handler's except-branch falling open to a verbatim
# forward after a decode failure) sees the bytes already read rather than
# a consumed stream.
request._body = raw
# Every branch below decompresses incrementally against
@@ -328,6 +328,30 @@ def test_upstream_connect_error_returns_502():
assert resp.status_code == 502
def test_oversized_body_returns_413_and_never_forwards(monkeypatch):
"""A body over MAX_REQUEST_BODY_SIZE gets a 413 fail-closed response, and
_forward_bedrock is never reached: a partially-read oversized body must
not go out to the gateway.
"""
import headroom.proxy.helpers as helpers_mod
small_cap = 1024
monkeypatch.setattr(helpers_mod, "MAX_REQUEST_BODY_SIZE", small_cap)
app = create_app(_make_config())
with TestClient(app) as client:
proxy = client.app.state.proxy
http = _install_fake_client(proxy, _FakeUpstream())
proxy.anthropic_pipeline.apply = MagicMock(side_effect=AssertionError("should not run"))
body = {"messages": [{"role": "user", "content": "x" * (small_cap * 4)}], "max_tokens": 8}
resp = client.post(INVOKE, json=body)
assert resp.status_code == 413
assert resp.json()["error"]["type"] == "request_too_large"
http.build_request.assert_not_called()
http.send.assert_not_called()
# ── metrics ───────────────────────────────────────────────────────────
@@ -74,16 +74,29 @@ class _Request:
"""Minimal stand-in for the Starlette Request the reader actually takes."""
def __init__(
self, body: bytes, content_encoding: str = "", headers: dict | None = None
self,
body: bytes,
content_encoding: str = "",
*,
headers: dict | None = None,
stream_chunk_size: int = 64 * 1024,
) -> None:
self._body = body
self.headers = {"content-encoding": content_encoding, **(headers or {})}
self._stream_chunk_size = stream_chunk_size
self.chunks_yielded = 0
async def body(self) -> bytes:
return self._body
async def stream(self):
yield self._body
# Real ASGI servers hand the body over in chunks, not one blob, which
# is exactly what lets the streaming bound-check in
# `_read_request_body_bytes` refuse a body before all of it has
# arrived.
for start in range(0, len(self._body), self._stream_chunk_size):
self.chunks_yielded += 1
yield self._body[start : start + self._stream_chunk_size]
class _NeverStreamedRequest(_Request):
@@ -298,6 +311,26 @@ def test_truncated_gzip_still_errors() -> None:
)
# ───────────────── raw body ceiling (the chunked-transfer bypass) ──────────
#
# The handlers only ever see the *compressed*, wire-level size via
# `Content-Length`, and that header is absent on a chunked-transfer-encoding
# request. The streaming loop in `_read_request_body_bytes` is the backstop
# the handlers fall through to regardless of whether that header was there
# (#3326).
async def test_raw_body_refused_without_consuming_the_whole_stream(small_raw_cap: int) -> None:
"""The point is refusing early, not just refusing eventually."""
request = _Request(b"a" * (small_raw_cap * 10), stream_chunk_size=small_raw_cap // 4)
total_chunks = -(-len(request._body) // request._stream_chunk_size) # ceil div
with pytest.raises(_helpers().RequestBodyTooLarge):
await _helpers()._read_request_body_bytes(request)
assert request.chunks_yielded < total_chunks
# ──────────────────────────── through the entry point ──────────────────────
@@ -318,6 +351,21 @@ async def test_reader_leaves_uncompressed_bodies_alone(small_cap: int) -> None:
assert await _helpers()._read_request_body_bytes(_Request(PAYLOAD, "identity")) == PAYLOAD
async def test_reader_refuses_an_oversized_plain_body_with_no_content_length(
small_raw_cap: int,
) -> None:
"""The regression this PR fixes: no `Content-Length` header, no compression.
Before this fix, `_read_request_body_bytes` called `request.body()`
directly, so a handler's `Content-Length` precheck was the only gate and a
chunked request (no `Content-Length` to check) reached this function
regardless of size.
"""
oversized = b"a" * (small_raw_cap + 1)
with pytest.raises(_helpers().RequestBodyTooLarge):
await _helpers()._read_request_body_bytes(_Request(oversized, ""))
async def test_reader_still_rejects_an_unknown_encoding() -> None:
with pytest.raises(ValueError, match="Unsupported Content-Encoding"):
await _helpers()._read_request_body_bytes(_Request(PAYLOAD, "snappy"))