mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): do not rebuild a failed turn-hook re-drive as HTTP 200 (#4398)
## Description
The Anthropic buffered turn-hook re-drive (`_turn_hook_call_model` in
`headroom/proxy/handlers/anthropic.py`) parsed the continuation body
without checking its status, so an upstream 4xx/5xx error envelope was
returned to the client as a 200 response. This raises on a non-2xx
status, as the CCR continuation path already does; `run_response_hooks`
catches the exception and keeps the original response.
Fixes #4394
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Raise `RuntimeError` in `_turn_hook_call_model` when the re-drive
status is not 2xx (5 lines).
- Add `test_anthropic_hook_redrive_error_is_not_rebuilt_as_200` to
`tests/test_turn_hook_usage.py`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
# new test, source change reverted
E KeyError: 'id'
1 failed, 14 deselected
# with the fix: tests/test_turn_hook_usage.py tests/test_turn_hooks.py tests/test_turn_hook_seams.py tests/test_openai_chat_turn_hooks.py
46 passed, 1 warning
ruff check: All checks passed!
ruff format --check: 2 files already formatted
```
## Real Behavior Proof
- Environment: Linux (WSL2), Python 3.12.15, the real `python -m
headroom.proxy.server` with `ANTHROPIC_TARGET_API_URL` at a local stub
(first POST 200, later POSTs 500), Rust core from the headroom-ai 0.41.0
wheel, a re-drive turn hook loaded through the
`headroom.proxy_extension` entry point
- Exact command / steps: one non-streaming `POST /v1/messages`, run with
the status check reverted and on this branch (details below)
- Observed result: reverted, the client gets HTTP 200 carrying the
upstream error envelope; this branch logs the hook `RuntimeError` and
the client gets HTTP 200 with the original first response
- Not tested: real Anthropic, a production hook, streaming, the OpenAI
handler paths, a Rust core built from this checkout
Details:
Setup: WSL2 Ubuntu (Linux 6.18 x86_64), Python 3.12.15, uvicorn 0.54.0,
fastapi 0.158.1, starlette 1.9.0. Worktree at PR head `e6d16cdc` (base
`e995eb9f`).
`headroom/_core.abi3.so` was copied (untracked, not committed) from the
official `headroom_ai-0.41.0-cp310-abi3-manylinux_2_28_x86_64.whl`
because the checkout has no built Rust core.
Stub upstream (`stub2.py`, 127.0.0.1:18794): the first POST to
`/v1/messages` returns 200 with a normal message (`id: msg_first`, text
"first answer"); every later POST returns 500
`{"type":"error","error":{"type":"api_error","message":"Internal server
error"}}`.
Proxy: the real server, `ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18794
HEADROOM_PROXY_EXTENSIONS=redrive_ext PYTHONPATH=.:ext python -P -m
headroom.proxy.server --host 127.0.0.1 --port 18793 --no-cache
--no-rate-limit`.
The turn hook is loaded through the real extension entry point group
`headroom.proxy_extension` (a throwaway `redrive_ext` dist-info on
PYTHONPATH; its hook does `return await call_model(ctx.messages)` once
in `on_response`, the same as the PR's test hook). No hook ships in the
repo, so a hook that re-drives is the only way to reach this path.
Client (`hook_client.py`): one non-streaming `POST /v1/messages` with
`x-api-key: sk-ant-test`.
Before = `headroom/proxy/handlers/anthropic.py` with the PR's 5-line
status check reverted (`git apply -R`). After = PR head.
### Before (PR change reverted)
```
registered turn hook: proof_redrive (priority 100)
proxy extension installed: redrive_ext
client: HTTP 200 {"type": "error", "error": {"type": "api_error", "message": "Internal server error"}}
--- stub upstream received
STUB-UPSTREAM call #1 POST /v1/messages -> 200
STUB-UPSTREAM call #2 POST /v1/messages -> 500
STUB-UPSTREAM call #3 POST /v1/messages -> 500
STUB-UPSTREAM call #4 POST /v1/messages -> 500
```
The client gets HTTP 200 whose body is the upstream error JSON, and the
good first response (`msg_first`) is lost. A client that checks the
status code sees success.
### After (PR head e6d16cdc)
```
client: HTTP 200 {"id": "msg_first", "type": "message", "role": "assistant", "model": "claude-sonnet-4-5", "content": [{"type": "text", "text": "first answer"}], "stop_reason": "end_turn", "usage": {"input_tokens": 100, "output_tokens": 10}}
--- proxy log
ERROR - turn hook 'proof_redrive' on_response failed
File ".../headroom/proxy/handlers/anthropic.py", line 4993, in _turn_hook_call_model
RuntimeError: turn hook re-drive returned non-success status 500
--- stub upstream received
STUB-UPSTREAM call #1 POST /v1/messages -> 200
STUB-UPSTREAM call #2 POST /v1/messages -> 500
STUB-UPSTREAM call #3 POST /v1/messages -> 500
STUB-UPSTREAM call #4 POST /v1/messages -> 500
```
The failed re-drive is logged by the existing hook runner
(`run_response_hooks` logs and skips a hook that raises) and the client
receives the original successful response. The upstream sees the same 4
calls in both runs (the 3 later ones are the proxy's own retries of the
500).
### Commands
```
python -P stub2.py 18794 hook.stub.log &
HEADROOM_PROXY_EXTENSIONS=redrive_ext ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18794 PYTHONPATH=.:ext \
python -P -m headroom.proxy.server --host 127.0.0.1 --port 18793 --no-cache --no-rate-limit &
python -P hook_client.py 18793
```
Scripts and raw outputs: `/var/tmp/cb-pub/proof/` (WSL): `run_hook.sh`,
`stub2.py`, `hook_client.py`, `ext/`, `before_hook.out`,
`after_hook.out`.
### Not tested
- Real Anthropic API (stub upstream only) and other failure statuses
than 500.
- A production turn hook; the hook here is a minimal stand-in that
re-drives once.
- Streaming requests (hooks that may re-drive are buffered-only).
- The OpenAI handlers' hook paths.
- The Rust core is the 0.41.0 wheel's, not one built from this checkout.
## Runtime Rollout Safety
- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: only when a turn hook re-drive fails
upstream (client now gets the original response instead of an error
envelope as 200)
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none known
- Rollback path: revert the commit
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
Only the four listed test files were run, not the whole suite. Produced
with AI assistance (Claude Code) operated by me (HetCreep); the
reproduction was run on my machine on a clone of the repository.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4398"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
043fe09d8c
commit
f59c8ffcd8
@@ -4989,6 +4989,11 @@ class AnthropicHandlerMixin:
|
||||
url=url,
|
||||
request_id=request_id,
|
||||
)
|
||||
if not 200 <= continuation_response.status_code < 300:
|
||||
raise RuntimeError(
|
||||
"turn hook re-drive returned non-success status "
|
||||
f"{continuation_response.status_code}"
|
||||
)
|
||||
continuation_json = continuation_response.json()
|
||||
_hook_usage.record(continuation_json)
|
||||
return continuation_json
|
||||
|
||||
@@ -465,3 +465,42 @@ def test_anthropic_bills_original_plus_hook_redrive(monkeypatch, _no_hooks) -> N
|
||||
assert outcome.cache_read_tokens == 120
|
||||
assert outcome.cache_write_tokens == 55
|
||||
assert outcome.uncached_input_tokens == 250
|
||||
|
||||
|
||||
@respx.mock
|
||||
def test_anthropic_hook_redrive_error_is_not_rebuilt_as_200(monkeypatch, _no_hooks) -> None:
|
||||
"""An upstream error on the re-drive must not replace the good first response."""
|
||||
register_turn_hook(_RedriveOnce())
|
||||
app, _outcomes = _app_and_outcomes(monkeypatch)
|
||||
|
||||
first = {
|
||||
"id": "a",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-sonnet-4-5",
|
||||
"content": [{"type": "text", "text": "a"}],
|
||||
"stop_reason": "end_turn",
|
||||
"usage": {"input_tokens": 100, "output_tokens": 10},
|
||||
}
|
||||
error = {"type": "error", "error": {"type": "api_error", "message": "Internal server error"}}
|
||||
# The first call succeeds; every later call (the re-drive, plus its retries) fails.
|
||||
route = respx.post("https://api.anthropic.com/v1/messages").mock(
|
||||
side_effect=[httpx.Response(200, json=first), httpx.Response(500, json=error)]
|
||||
+ [httpx.Response(500, json=error)] * 5
|
||||
)
|
||||
|
||||
with TestClient(app) as client:
|
||||
result = client.post(
|
||||
"/v1/messages",
|
||||
json={
|
||||
"model": "claude-sonnet-4-5",
|
||||
"max_tokens": 64,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
},
|
||||
headers={"x-api-key": "sk-ant-test", "anthropic-version": "2023-06-01"},
|
||||
)
|
||||
|
||||
assert route.call_count >= 2
|
||||
assert result.status_code == 200
|
||||
assert result.json()["id"] == "a"
|
||||
assert result.json().get("type") != "error"
|
||||
|
||||
Reference in New Issue
Block a user