mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): say a forwarded streaming error came from the provider (#4629)
## Description An Extra Headroom user found this line in their proxy log and could not tell whether Headroom or the provider made the 429: `Forwarding upstream streaming error status=429 url=https://chatgpt.com/backend-api/codex/responses` The proxy only passes the provider's error on. The new wording says so. The `status=` and `url=` fields and their order are unchanged, so greps on them still match. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/streaming.py`: the >=400 streaming log line now reads `[%s] Upstream provider returned an error; forwarding it to the client status=%s url=%s`. - `tests/test_proxy_streaming_ratelimit_headers.py`: the assertion on that line follows the new text. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_proxy_streaming_ratelimit_headers.py -q 22 passed in 3.60s $ uvx ruff check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_ratelimit_headers.py All checks passed! $ uvx ruff format --check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_ratelimit_headers.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS 27.2, Python 3.12 via uv, this branch at226ff6fa(on top of upstream main5c8772b7). Controlled upstream: a local stub on 127.0.0.1:6812 (source below) that answers every POST with HTTP 429, `retry-after: 3600` and an Anthropic `rate_limit_error` JSON body. No provider credentials; the API key is a dummy. - Exact command / steps: Started the stub and a real `headroom proxy` process pointed at it, then sent one streaming `/v1/messages` request through the proxy with curl (commands below). - Observed result: The running proxy logged the new warning with `status=429` and the stub's url; the client got HTTP 429 with the stub's `retry-after` header and its body byte for byte; the stub saw exactly one POST. - Not tested: A live provider error. Only the Anthropic `/v1/messages` route was run; the Codex route from the original report reaches the same log line but was not run. Only 429 was run live; the unit test covers 503. ```text $ /usr/bin/python3 -I stub429.py 6812 $ uv run --frozen --extra proxy headroom proxy --port 6811 --anthropic-api-url http://127.0.0.1:6812 --no-telemetry $ curl -sS -N -D - http://127.0.0.1:6811/v1/messages \ -H 'content-type: application/json' -H 'x-api-key: sk-ant-dummy-not-a-key' \ -H 'anthropic-version: 2023-06-01' \ -d '{"model":"claude-sonnet-4-5","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' HTTP/1.1 429 Too Many Requests server: uvicorn content-type: application/json retry-after: 3600 content-length: 98 {"type":"error","error":{"type":"rate_limit_error","message":"stub upstream: rate limited"}} # stub output stub: POST /v1/messages stream=True # proxy log 2026-11-04 16:09:09,938 - headroom.proxy - WARNING - [hr_1793804949_000001] Upstream provider returned an error; forwarding it to the client status=429 url=http://127.0.0.1:6812/v1/messages 2026-11-04 16:09:09,938 - headroom.proxy - INFO - event=proxy_inbound_response id=inbound-1793804949845998000 method=POST path=/v1/messages status=429 duration_ms=92.92 response_content_length=98 ``` <details><summary>stub429.py</summary> ```python import json import sys from http.server import BaseHTTPRequestHandler, HTTPServer BODY = json.dumps( {"type": "error", "error": {"type": "rate_limit_error", "message": "stub upstream: rate limited"}} ).encode() class H(BaseHTTPRequestHandler): def do_POST(self): n = int(self.headers.get("content-length", 0)) req = json.loads(self.rfile.read(n) or b"{}") print(f"stub: POST {self.path} stream={req.get('stream')}", flush=True) self.send_response(429) self.send_header("content-type", "application/json") self.send_header("retry-after", "3600") self.send_header("content-length", str(len(BODY))) self.end_headers() self.wfile.write(BODY) HTTPServer(("127.0.0.1", int(sys.argv[1])), H).serve_forever() ``` </details> ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Stable. - Stable/default behavior changed: No; only the text of one warning log line. - Kill switch / disable path: Not applicable. - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable. ## Additional Notes No comments or docs needed for a log string. The existing test already covers the line, so no new test. mypy was not run; the change touches no types. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4629" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
ee975d49c0
commit
956b8855a6
@@ -1479,7 +1479,7 @@ class StreamingMixin:
|
||||
|
||||
if upstream_response.status_code >= 400:
|
||||
logger.warning(
|
||||
"[%s] Forwarding upstream streaming error status=%s url=%s",
|
||||
"[%s] Upstream provider returned an error; forwarding it to the client status=%s url=%s",
|
||||
request_id,
|
||||
upstream_response.status_code,
|
||||
url,
|
||||
|
||||
@@ -493,7 +493,7 @@ class TestStreamingRatelimitHeaderForwarding:
|
||||
assert result.body == b'{"error":{"message":"capacity exhausted"}}'
|
||||
assert result.headers.get("content-encoding") is None
|
||||
fake_logger.warning.assert_any_call(
|
||||
"[%s] Forwarding upstream streaming error status=%s url=%s",
|
||||
"[%s] Upstream provider returned an error; forwarding it to the client status=%s url=%s",
|
||||
"test-http-error",
|
||||
503,
|
||||
"https://api.anthropic.com/v1/messages",
|
||||
|
||||
Reference in New Issue
Block a user