mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): handle ClientDisconnect in anthropic_messages (#3532)
## Description `handle_anthropic_messages()` (the main `/v1/messages` handler) has no `ClientDisconnect` guard around its call to `read_request_json_with_bytes()` — documented as the shared reader for the Anthropic, OpenAI, and Bedrock handlers. Any client disconnecting mid-request (an interrupted turn, a client-side restart/timeout) crashes the request with an unhandled 500 instead of a quiet 204. The sibling Anthropic batch-passthrough handler a few hundred lines below in the same file already has this exact guard (from #2033/#2067), but the shared reader used by the main handler was never covered. Found while running this proxy in front of Claude Code on a home server and seeing repeated `starlette.requests.ClientDisconnect` tracebacks in production. Closes: (no existing issue — opening this as a direct fix per the "🐛 Bug or small fix" row in CONTRIBUTING.md) ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/anthropic.py`: wrap the `read_request_json_with_bytes()` call in `handle_anthropic_messages()` with a `ClientDisconnect` guard, mirroring the existing local-import + try/except idiom already used at the batch-passthrough call site in the same file. - `tests/test_anthropic_messages_client_disconnect.py` (new): reproduces the crash and proves the fix. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_anthropic_messages_client_disconnect.py tests/test_proxy_handler_helpers.py tests/test_anthropic_pre_upstream_backpressure.py -q ............................................................................ [100%] 80 passed in ~2.3s $ python -m ruff check headroom/proxy/handlers/anthropic.py tests/test_anthropic_messages_client_disconnect.py All checks passed! $ python -m ruff format --check headroom/proxy/handlers/anthropic.py tests/test_anthropic_messages_client_disconnect.py 2 files already formatted $ python -m mypy headroom/proxy/handlers/anthropic.py --ignore-missing-imports Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS (aarch64-darwin), Python 3.14.7, `pip install -e ".[proxy]"` (built the Rust `headroom._core` extension via maturin/cargo locally), pytest 9.1.1. - Exact command / steps: wrote `tests/test_anthropic_messages_client_disconnect.py`, which builds a real Starlette `Request` whose ASGI `receive()` callable returns `{"type": "http.disconnect"}` — the actual protocol-level signal, not a mocked `.body()` exception — and calls `handle_anthropic_messages()` on a minimal handler. Ran it against the unpatched handler first (`git checkout <pre-fix-commit> -- headroom/proxy/handlers/anthropic.py`, ran the test, restored the fix), then against the patched handler. - Observed result: against the unpatched handler the test fails with exactly the live traceback — `starlette.requests.ClientDisconnect` raised through `handle_anthropic_messages -> read_request_json_with_bytes -> _read_request_body_bytes -> request.body()`. Against the patched handler it returns `204` and passes. Also ran the full existing `tests/test_proxy_handler_helpers.py` (which already covers the same failure mode on the three passthrough paths from #2033/#2067) and `tests/test_anthropic_pre_upstream_backpressure.py` (26 tests exercising the same handler's pre-upstream phase) — no regressions, all pass. - Not tested: end-to-end against a live upstream Anthropic API call — the test isolates the body-read phase, which is where the bug lives and where the fix applies; nothing downstream of the guard is touched. (`mypy headroom/proxy/handlers/anthropic.py --ignore-missing-imports`, pinned to the CI toolchain versions — `mypy==1.20.2`, `numpy==2.2.6` from `uv.lock` — now run and clean; see Testing → Test Output above.) ## Runtime Rollout Safety - Rollout-managed feature(s): none — this is a bug fix in an existing, always-on code path, not behind a rollout flag. - Minimum rollout channel: n/a - Stable/default behavior changed: yes, but only for the failure case (a disconnected client now gets no response instead of the server logging an unhandled exception) — no change to any successful-request behavior. - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert the commit; behavior returns to the pre-fix unhandled-exception path. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation — n/a, no user-facing docs describe this failure path - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes This is one of two small, independently-scoped fixes from the same audit pass; the other (telemetry/beacon status disclosure) is a separate PR per the "one logical change per PR" guidance in CONTRIBUTING.md. Opening as a draft while I give it a final look — will mark ready for review shortly. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3532" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JD Davis
parent
87a4394d49
commit
251d149a13
@@ -1047,6 +1047,7 @@ class AnthropicHandlerMixin:
|
||||
|
||||
from fastapi import HTTPException
|
||||
from fastapi.responses import JSONResponse, Response, StreamingResponse
|
||||
from starlette.requests import ClientDisconnect
|
||||
|
||||
from headroom.cache.compression_store import get_compression_store
|
||||
from headroom.ccr import CCRToolInjector
|
||||
@@ -1218,6 +1219,16 @@ class AnthropicHandlerMixin:
|
||||
try:
|
||||
async with stage_timer.measure("read_request_json"):
|
||||
body, original_body_bytes = await read_request_json_with_bytes(request)
|
||||
except ClientDisconnect:
|
||||
# The client (e.g. an interrupted/compacting Claude Code
|
||||
# session) went away mid-read. There is no one left to answer
|
||||
# and nothing to compress — bail out quietly instead of
|
||||
# letting Starlette's own ClientDisconnect propagate up as an
|
||||
# unhandled 500. Mirrors the existing guard on the batch
|
||||
# passthrough path a few hundred lines below.
|
||||
logger.debug("Client disconnected during body read for anthropic messages")
|
||||
await _finalize_pre_upstream()
|
||||
return Response(status_code=204)
|
||||
except (json.JSONDecodeError, ValueError) as e:
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
"""``handle_anthropic_messages`` must not crash when the client disconnects
|
||||
mid-body-read.
|
||||
|
||||
Companion to the passthrough-path regression coverage in
|
||||
``tests/test_proxy_handler_helpers.py`` (``test_handle_passthrough_client_disconnect``
|
||||
/ ``test_handle_streaming_passthrough_client_disconnect``, added in #2033/#2067).
|
||||
Those two guard the three named passthrough paths. They do not cover
|
||||
``read_request_json_with_bytes`` / ``_read_request_body_bytes`` in
|
||||
``headroom/proxy/helpers.py`` — the shared reader documented as used by the
|
||||
Anthropic, OpenAI, and Bedrock handlers — so the main ``/v1/messages`` handler
|
||||
had no guard at all: a real client disconnect (an interrupted turn, a
|
||||
client-side restart) crashed the request as an unhandled 500 instead of a
|
||||
quiet 204.
|
||||
|
||||
Reproduces the real-world trigger at the ASGI level: ``receive()`` returns
|
||||
``{"type": "http.disconnect"}``, which is exactly what Starlette's own
|
||||
``Request.body()`` checks for before raising ``ClientDisconnect`` — not a
|
||||
mocked exception, the actual protocol-level signal.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
from fastapi import Request
|
||||
|
||||
from headroom.proxy.handlers.anthropic import AnthropicHandlerMixin
|
||||
|
||||
|
||||
class _MinimalAnthropicHandler(AnthropicHandlerMixin):
|
||||
"""The smallest handler that can reach the ClientDisconnect guard.
|
||||
|
||||
The guard sits before any of the heavier machinery (compression,
|
||||
memory, CCR, upstream dispatch) executes, so none of that needs
|
||||
stubbing here — only ``_next_request_id``, which the real
|
||||
implementation backs with an ``asyncio.Lock`` + counter this
|
||||
minimal handler never initializes.
|
||||
"""
|
||||
|
||||
async def _next_request_id(self) -> str:
|
||||
return "req-test"
|
||||
|
||||
|
||||
def _build_disconnecting_request() -> Request:
|
||||
"""A real ``Request`` whose ASGI receive channel signals disconnect
|
||||
before any body bytes arrive — the actual protocol-level trigger,
|
||||
not a mocked ``.body()`` override."""
|
||||
|
||||
async def receive():
|
||||
return {"type": "http.disconnect"}
|
||||
|
||||
scope = {
|
||||
"type": "http",
|
||||
"asgi": {"version": "3.0"},
|
||||
"http_version": "1.1",
|
||||
"method": "POST",
|
||||
"scheme": "https",
|
||||
"path": "/v1/messages",
|
||||
"raw_path": b"/v1/messages",
|
||||
"query_string": b"",
|
||||
"headers": [
|
||||
(b"content-type", b"application/json"),
|
||||
(b"authorization", b"Bearer sk-ant-api-test"),
|
||||
],
|
||||
"client": ("127.0.0.1", 12345),
|
||||
"server": ("testserver", 443),
|
||||
}
|
||||
return Request(scope, receive)
|
||||
|
||||
|
||||
def test_handle_anthropic_messages_client_disconnect_returns_204():
|
||||
"""Before the fix this raised an unhandled starlette.requests.ClientDisconnect
|
||||
(observed live in production as a 500 traceback through
|
||||
handle_anthropic_messages -> read_request_json_with_bytes ->
|
||||
_read_request_body_bytes -> request.body()). After the fix it returns a
|
||||
quiet 204, matching the existing guard on the sibling batch-passthrough
|
||||
handler a few hundred lines below in the same file."""
|
||||
handler = _MinimalAnthropicHandler()
|
||||
response = asyncio.run(handler.handle_anthropic_messages(_build_disconnecting_request()))
|
||||
assert response.status_code == 204
|
||||
Reference in New Issue
Block a user