mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): import Response at runtime in the Bedrock handler (#4397)
## Description
`handle_bedrock_invoke` returns `Response(status_code=204)` when the
client disconnects during the body read, but `Response` was imported
only under `TYPE_CHECKING`, so that branch raised `NameError`. This
imports it next to `JSONResponse` inside the handler.
Closes #4393
Fixes #4393
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/bedrock.py`: `from fastapi.responses import
JSONResponse, Response` in `handle_bedrock_invoke`.
- `tests/test_proxy/test_bedrock_passthrough.py`:
`test_client_disconnect_during_body_read_returns_204`.
## Testing
- [x] Unit tests pass (`pytest`) (the five Bedrock test files only)
- [x] Linting passes (`ruff check .`) (changed files only)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
# source change reverted, new test only
E starlette.requests.ClientDisconnect
E NameError: name 'Response' is not defined
1 failed, 23 deselected
# with the fix, 5 Bedrock test files
52 passed, 1 warning in 24.57s
ruff check: All checks passed!
ruff format --check: 2 files already formatted
```
## Real Behavior Proof
- Environment: Linux (WSL2), Python 3.12.15, the real `python -m
headroom.proxy.server --bedrock-api-url http://127.0.0.1:18792`, Rust
core from the headroom-ai 0.41.0 wheel, a local stub upstream
- Exact command / steps: a raw-socket client sends `POST
/model/anthropic.claude-3-haiku/invoke` with `content-length: 1000`,
writes about 50 bytes and closes; then a normal request; run with the
import change reverted and on this branch (details below)
- Observed result: reverted, the log shows `NameError: name 'Response'
is not defined` while handling `ClientDisconnect`; this branch logs
`status=204` with no traceback, the partial request is not forwarded,
and the next request returns 200
- Not tested: real Bedrock or a gateway, the streaming route, a client
behind a TCP proxy, a Rust core built from this checkout
Details:
Setup: WSL2 Ubuntu (Linux 6.18 x86_64), Python 3.12.15, uvicorn 0.54.0,
fastapi 0.158.1, starlette 1.9.0. Worktree at PR head `c7d9d59e` (base
`e995eb9f`).
`headroom/_core.abi3.so` was copied (untracked, not committed) from the
official `headroom_ai-0.41.0-cp310-abi3-manylinux_2_28_x86_64.whl`
because the checkout has no built Rust core. The core is from 0.41.0,
the Python source is upstream/main plus the PR.
Stub upstream: a 20-line `http.server` on 127.0.0.1:18792 that logs
every POST it receives. Proxy: the real server, `python -P -m
headroom.proxy.server --host 127.0.0.1 --port 18791 --bedrock-api-url
http://127.0.0.1:18792`, started with `HEADROOM_LOG_LEVEL=DEBUG` (the
debug line goes to `~/.headroom/logs/proxy-18791.log`).
Client (raw socket, `bedrock_client.py`): connect, send `POST
/model/anthropic.claude-3-haiku/invoke` headers with `content-length:
1000`, write about 50 body bytes, wait 0.5 s, close the socket. Then, on
the same proxy, send one normal full request.
Before = `headroom/proxy/handlers/bedrock.py` with the PR's one-line
change reverted (`git apply -R`). After = PR head.
### Before (PR change reverted)
```
proxy /livez -> 200
client: sent 1000-byte Content-Length, wrote ~50 bytes, closed socket
client: follow-up full request -> 200 {"stub": "ok"}
--- proxy log
42: DEBUG - [hr_1792672318_000001] bedrock_invoke client disconnected during body read
43: ERROR - event=proxy_inbound_request_aborted id=inbound-1792672318828183190 method=POST path=/model/anthropic.claude-3-haiku/invoke reason=NameError duration_ms=500.61
45: File ".../headroom/proxy/handlers/bedrock.py", line 132, in handle_bedrock_invoke
54: raise ClientDisconnect()
55: starlette.requests.ClientDisconnect
...
110: NameError: name 'Response' is not defined (bedrock.py line 138, during handling of the ClientDisconnect)
--- stub upstream received
STUB-UPSTREAM POST /model/anthropic.claude-3-haiku/invoke body_bytes=49
```
The disconnect branch is reached (the DEBUG line), then `return
Response(status_code=204)` raises `NameError` (`Response` is imported
only under `TYPE_CHECKING`, line 38), so the request ends as
`proxy_inbound_request_aborted` with a traceback instead of a handled
disconnect.
### After (PR head c7d9d59e)
```
proxy /livez -> 200
client: sent 1000-byte Content-Length, wrote ~50 bytes, closed socket
client: follow-up full request -> 200 {"stub": "ok"}
--- proxy log
42: DEBUG - [hr_1792672327_000001] bedrock_invoke client disconnected during body read
43: INFO - event=proxy_inbound_response id=inbound-1792672327942098751 method=POST path=/model/anthropic.claude-3-haiku/invoke status=204 duration_ms=501.52
48: INFO - event=proxy_inbound_response ... method=POST path=/model/anthropic.claude-3-haiku/invoke status=200 duration_ms=55.82
--- stub upstream received
STUB-UPSTREAM POST /model/anthropic.claude-3-haiku/invoke body_bytes=49
```
The disconnect is logged and answered with status 204, no traceback,
nothing is forwarded for the partial request (the stub saw exactly one
POST: the 49-byte follow-up), and the proxy keeps serving.
### Commands
```
# stub upstream
python -P stub.py 18792 bedrock.stub.log &
# real proxy
HEADROOM_LOG_LEVEL=DEBUG PYTHONPATH=. python -P -m headroom.proxy.server --host 127.0.0.1 --port 18791 --bedrock-api-url http://127.0.0.1:18792 &
# client
python -P bedrock_client.py 18791
```
Scripts and raw outputs: `/var/tmp/cb-pub/proof/` (WSL):
`run_bedrock.sh`, `stub.py`, `bedrock_client.py`, `before_bedrock.out`,
`after_bedrock.out`.
### Not tested
- Real AWS Bedrock or a re-signing gateway (stub upstream only).
- A client behind a real TCP proxy or load balancer; the disconnect is a
local socket close.
- The Rust core is the 0.41.0 wheel's, not one built from this checkout.
- Streaming (`invoke-with-response-stream`) route; only `invoke` was
driven.
## Runtime Rollout Safety
- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no (the branch previously raised
NameError)
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A
## Additional Notes
Only the Bedrock tests were run, not the full suite. Produced with AI
assistance (Claude Code) operated by me (HetCreep); the reproduction was
run on my machine on a clone of the repository.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4397"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
41c15b311c
commit
51e4ae36a5
@@ -70,7 +70,7 @@ class BedrockHandlerMixin:
|
||||
URL path (may contain ``.``, ``:`` and ``/``).
|
||||
stream: ``True`` for ``invoke-with-response-stream``.
|
||||
"""
|
||||
from fastapi.responses import JSONResponse
|
||||
from fastapi.responses import JSONResponse, Response
|
||||
|
||||
from headroom.proxy.auth_mode import classify_client
|
||||
from headroom.proxy.helpers import (
|
||||
|
||||
@@ -352,6 +352,30 @@ def test_oversized_body_returns_413_and_never_forwards(monkeypatch):
|
||||
http.send.assert_not_called()
|
||||
|
||||
|
||||
def test_client_disconnect_during_body_read_returns_204(monkeypatch):
|
||||
"""A client that hangs up mid-body must not crash the handler: the
|
||||
ClientDisconnect branch returns an empty 204 and never forwards.
|
||||
"""
|
||||
from starlette.requests import ClientDisconnect
|
||||
|
||||
import headroom.proxy.helpers as helpers_mod
|
||||
|
||||
async def _disconnect(request):
|
||||
raise ClientDisconnect()
|
||||
|
||||
monkeypatch.setattr(helpers_mod, "read_request_json_with_bytes", _disconnect)
|
||||
|
||||
app = create_app(_make_config())
|
||||
with TestClient(app) as client:
|
||||
proxy = client.app.state.proxy
|
||||
http = _install_fake_client(proxy, _FakeUpstream())
|
||||
resp = client.post(INVOKE, json={"messages": [], "max_tokens": 8})
|
||||
|
||||
assert resp.status_code == 204
|
||||
http.build_request.assert_not_called()
|
||||
http.send.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"route", [INVOKE, INVOKE_STREAM], ids=["invoke", "invoke-with-response-stream"]
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user