mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:07:29 +08:00
## Description An Extra Headroom user found this line in their proxy log and could not tell whether Headroom or the provider made the 429: `Forwarding upstream streaming error status=429 url=https://chatgpt.com/backend-api/codex/responses` The proxy only passes the provider's error on. The new wording says so. The `status=` and `url=` fields and their order are unchanged, so greps on them still match. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/streaming.py`: the >=400 streaming log line now reads `[%s] Upstream provider returned an error; forwarding it to the client status=%s url=%s`. - `tests/test_proxy_streaming_ratelimit_headers.py`: the assertion on that line follows the new text. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_proxy_streaming_ratelimit_headers.py -q 22 passed in 3.60s $ uvx ruff check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_ratelimit_headers.py All checks passed! $ uvx ruff format --check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_ratelimit_headers.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS 27.2, Python 3.12 via uv, this branch at226ff6fa(on top of upstream main5c8772b7). Controlled upstream: a local stub on 127.0.0.1:6812 (source below) that answers every POST with HTTP 429, `retry-after: 3600` and an Anthropic `rate_limit_error` JSON body. No provider credentials; the API key is a dummy. - Exact command / steps: Started the stub and a real `headroom proxy` process pointed at it, then sent one streaming `/v1/messages` request through the proxy with curl (commands below). - Observed result: The running proxy logged the new warning with `status=429` and the stub's url; the client got HTTP 429 with the stub's `retry-after` header and its body byte for byte; the stub saw exactly one POST. - Not tested: A live provider error. Only the Anthropic `/v1/messages` route was run; the Codex route from the original report reaches the same log line but was not run. Only 429 was run live; the unit test covers 503. ```text $ /usr/bin/python3 -I stub429.py 6812 $ uv run --frozen --extra proxy headroom proxy --port 6811 --anthropic-api-url http://127.0.0.1:6812 --no-telemetry $ curl -sS -N -D - http://127.0.0.1:6811/v1/messages \ -H 'content-type: application/json' -H 'x-api-key: sk-ant-dummy-not-a-key' \ -H 'anthropic-version: 2023-06-01' \ -d '{"model":"claude-sonnet-4-5","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' HTTP/1.1 429 Too Many Requests server: uvicorn content-type: application/json retry-after: 3600 content-length: 98 {"type":"error","error":{"type":"rate_limit_error","message":"stub upstream: rate limited"}} # stub output stub: POST /v1/messages stream=True # proxy log 2026-11-04 16:09:09,938 - headroom.proxy - WARNING - [hr_1793804949_000001] Upstream provider returned an error; forwarding it to the client status=429 url=http://127.0.0.1:6812/v1/messages 2026-11-04 16:09:09,938 - headroom.proxy - INFO - event=proxy_inbound_response id=inbound-1793804949845998000 method=POST path=/v1/messages status=429 duration_ms=92.92 response_content_length=98 ``` <details><summary>stub429.py</summary> ```python import json import sys from http.server import BaseHTTPRequestHandler, HTTPServer BODY = json.dumps( {"type": "error", "error": {"type": "rate_limit_error", "message": "stub upstream: rate limited"}} ).encode() class H(BaseHTTPRequestHandler): def do_POST(self): n = int(self.headers.get("content-length", 0)) req = json.loads(self.rfile.read(n) or b"{}") print(f"stub: POST {self.path} stream={req.get('stream')}", flush=True) self.send_response(429) self.send_header("content-type", "application/json") self.send_header("retry-after", "3600") self.send_header("content-length", str(len(BODY))) self.end_headers() self.wfile.write(BODY) HTTPServer(("127.0.0.1", int(sys.argv[1])), H).serve_forever() ``` </details> ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Stable. - Stable/default behavior changed: No; only the text of one warning log line. - Kill switch / disable path: Not applicable. - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable. ## Additional Notes No comments or docs needed for a log string. The existing test already covers the line, so no new test. mypy was not run; the change touches no types. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4629" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
1047 lines
43 KiB
Python
1047 lines
43 KiB
Python
"""Tests for ratelimit header forwarding in streaming responses.
|
|
|
|
Verifies that anthropic-ratelimit-* headers from the upstream API response
|
|
are forwarded to the client in StreamingResponse, even in SSE streaming mode.
|
|
|
|
This was a bug where non-streaming responses correctly forwarded all headers
|
|
via dict(response.headers), but streaming responses used StreamingResponse
|
|
without passing any upstream headers — silently dropping ratelimit info.
|
|
"""
|
|
|
|
import json
|
|
import logging
|
|
from unittest.mock import AsyncMock, MagicMock
|
|
|
|
import httpx
|
|
import pytest
|
|
|
|
import headroom.proxy.handlers.streaming as streaming_module
|
|
from headroom.proxy.server import HeadroomProxy
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _reset_codex_rate_limit_singleton():
|
|
"""Isolate the process-global CodexRateLimitState across tests.
|
|
|
|
The tracker is a module singleton; save/restore ``_latest`` around every
|
|
test so a captured snapshot never leaks into (or depends on) another test.
|
|
"""
|
|
from headroom.subscription.codex_rate_limits import get_codex_rate_limit_state
|
|
|
|
state = get_codex_rate_limit_state()
|
|
saved = state._latest
|
|
state._latest = None
|
|
try:
|
|
yield
|
|
finally:
|
|
state._latest = saved
|
|
|
|
|
|
class TestStreamingRatelimitHeaderForwarding:
|
|
"""Test that upstream ratelimit headers are forwarded in streaming responses."""
|
|
|
|
def _create_mock_proxy(self):
|
|
"""Create a HeadroomProxy with mocked internals for unit testing."""
|
|
proxy = object.__new__(HeadroomProxy)
|
|
proxy.http_client = MagicMock(spec=httpx.AsyncClient)
|
|
proxy.metrics = MagicMock()
|
|
proxy.metrics.record_request = AsyncMock(return_value=None)
|
|
proxy.metrics.record_failed = AsyncMock(return_value=None)
|
|
proxy.metrics.record_rate_limited = AsyncMock(return_value=None)
|
|
proxy.cost_tracker = MagicMock()
|
|
proxy.cost_tracker.estimate_cost.return_value = 0.001
|
|
proxy.cost_tracker.record_request.return_value = None
|
|
proxy.stats = {
|
|
"requests_total": 0,
|
|
"requests_optimized": 0,
|
|
"tokens": {"original": 0, "optimized": 0, "saved": 0},
|
|
"cost": {"total_usd": 0, "savings_usd": 0},
|
|
"errors": 0,
|
|
"active_requests": 0,
|
|
"requests_per_model": {},
|
|
}
|
|
proxy.memory_manager = None
|
|
proxy._config = MagicMock()
|
|
proxy._config.memory_enabled = False
|
|
proxy._config.ccr_inject_tool = False
|
|
proxy._config.retry_max_attempts = 3
|
|
proxy._config.retry_base_delay_ms = 0
|
|
proxy._config.retry_max_delay_ms = 0
|
|
proxy.config = proxy._config
|
|
proxy._parse_sse_usage_from_buffer = MagicMock(return_value=None)
|
|
proxy.memory_handler = None
|
|
return proxy
|
|
|
|
def _create_mock_upstream_response(self, extra_headers=None):
|
|
"""Create a mock httpx streaming response with ratelimit headers."""
|
|
mock_response = AsyncMock()
|
|
headers = {
|
|
"content-type": "text/event-stream",
|
|
"anthropic-ratelimit-tokens-limit": "80000",
|
|
"anthropic-ratelimit-tokens-remaining": "75000",
|
|
"anthropic-ratelimit-tokens-reset": "2026-03-25T12:00:00Z",
|
|
"anthropic-ratelimit-requests-limit": "60",
|
|
"anthropic-ratelimit-requests-remaining": "59",
|
|
"anthropic-ratelimit-requests-reset": "2026-03-25T12:00:00Z",
|
|
"anthropic-ratelimit-input-tokens-limit": "50000",
|
|
"anthropic-ratelimit-input-tokens-remaining": "48000",
|
|
"anthropic-ratelimit-input-tokens-reset": "2026-03-25T12:00:00Z",
|
|
"anthropic-ratelimit-output-tokens-limit": "30000",
|
|
"anthropic-ratelimit-output-tokens-remaining": "27000",
|
|
"anthropic-ratelimit-output-tokens-reset": "2026-03-25T12:00:00Z",
|
|
# request-id is forwarded (clients record it per transcript turn);
|
|
# cf-ray is a non-allowlisted header that must NOT be forwarded.
|
|
"x-request-id": "req-12345",
|
|
"cf-ray": "abc123",
|
|
}
|
|
if extra_headers:
|
|
headers.update(extra_headers)
|
|
mock_response.headers = httpx.Headers(headers)
|
|
mock_response.status_code = 200
|
|
|
|
# Simulate a simple SSE stream
|
|
sse_data = (
|
|
b'event: message_start\ndata: {"type":"message_start","message":{"id":"msg_01"}}\n\n'
|
|
b'event: message_stop\ndata: {"type":"message_stop"}\n\n'
|
|
)
|
|
|
|
async def aiter_bytes():
|
|
yield sse_data
|
|
|
|
mock_response.aiter_bytes = aiter_bytes
|
|
mock_response.aclose = AsyncMock()
|
|
return mock_response
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_ratelimit_headers_forwarded_in_streaming(self):
|
|
"""Ratelimit headers from upstream should appear in the StreamingResponse."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
|
|
# Mock build_request + send to return our mock response
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test", "anthropic-version": "2023-06-01"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-123",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
# Verify ratelimit headers are present in the StreamingResponse
|
|
assert result.headers.get("anthropic-ratelimit-tokens-limit") == "80000"
|
|
assert result.headers.get("anthropic-ratelimit-tokens-remaining") == "75000"
|
|
assert result.headers.get("anthropic-ratelimit-tokens-reset") == "2026-03-25T12:00:00Z"
|
|
assert result.headers.get("anthropic-ratelimit-requests-limit") == "60"
|
|
assert result.headers.get("anthropic-ratelimit-requests-remaining") == "59"
|
|
assert result.headers.get("anthropic-ratelimit-input-tokens-limit") == "50000"
|
|
assert result.headers.get("anthropic-ratelimit-output-tokens-limit") == "30000"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_non_ratelimit_headers_not_forwarded(self):
|
|
"""Arbitrary upstream headers stay dropped; the request-id family is allowed."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-456",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
# request-id is forwarded; other non-ratelimit headers are not.
|
|
assert result.headers.get("x-request-id") == "req-12345"
|
|
assert result.headers.get("cf-ray") is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_compression_metrics_headers_on_streaming(self):
|
|
"""The streaming path stamps the same x-headroom-* metrics as the buffered one."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-metrics",
|
|
original_tokens=1000,
|
|
optimized_tokens=400,
|
|
tokens_saved=600,
|
|
transforms_applied=["smart_crusher", "read_lifecycle:stale:/a,b.py"],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.headers.get("x-headroom-tokens-before") == "1000"
|
|
assert result.headers.get("x-headroom-tokens-after") == "400"
|
|
assert result.headers.get("x-headroom-tokens-saved") == "600"
|
|
assert result.headers.get("x-headroom-model") == "claude-sonnet-4-20250514"
|
|
# Comma-bearing detail is collapsed so the header still splits into tags.
|
|
transforms = result.headers.get("x-headroom-transforms")
|
|
assert transforms is not None and transforms.split(",")[0] == "smart_crusher"
|
|
assert "/a,b.py" not in transforms
|
|
# Upstream headers are still forwarded alongside.
|
|
assert result.headers.get("x-request-id") == "req-12345"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_no_transforms_header_when_nothing_was_applied(self):
|
|
"""A request nothing touched still reports its counts, but no empty transforms."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={"model": "m", "max_tokens": 1, "stream": True, "messages": []},
|
|
provider="anthropic",
|
|
model="m",
|
|
request_id="test-noop",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.headers.get("x-headroom-tokens-saved") == "0"
|
|
assert result.headers.get("x-headroom-transforms") is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_request_id_headers_forwarded_in_streaming(self):
|
|
"""The request-id family is forwarded on the streaming path (#1100)."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.headers = httpx.Headers(
|
|
{
|
|
"content-type": "text/event-stream",
|
|
"request-id": "req-aaa",
|
|
"anthropic-request-id": "req-bbb",
|
|
"x-request-id": "req-ccc",
|
|
# Non-allowlisted header: must NOT be forwarded.
|
|
"cf-ray": "ray-123",
|
|
}
|
|
)
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-1100",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.media_type == "text/event-stream"
|
|
assert result.headers.get("request-id") == "req-aaa"
|
|
assert result.headers.get("anthropic-request-id") == "req-bbb"
|
|
assert result.headers.get("x-request-id") == "req-ccc"
|
|
assert result.headers.get("cf-ray") is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_litellm_key_headers_forwarded_in_streaming(self):
|
|
"""The x-litellm-key-* family is forwarded on the streaming path.
|
|
|
|
LiteLLM virtual-key spend/budget headers (``x-litellm-key-spend``,
|
|
``x-litellm-key-budget-remaining``, etc.) are dropped on the streaming
|
|
path because they match no allowlist clause, even though the buffered
|
|
path forwards every upstream header. Downstream status widgets that
|
|
read them from the streaming response stop updating — the same class
|
|
of omission fixed for ``*ratelimit*``, ``x-codex-*`` and ``request-id``.
|
|
"""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.headers = httpx.Headers(
|
|
{
|
|
"content-type": "text/event-stream",
|
|
"x-litellm-key-spend": "12.847",
|
|
"x-litellm-key-budget-remaining": "87.153",
|
|
# Non-allowlisted header: must NOT be forwarded.
|
|
"cf-ray": "ray-456",
|
|
}
|
|
)
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-litellm-key",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.media_type == "text/event-stream"
|
|
assert result.headers.get("x-litellm-key-spend") == "12.847"
|
|
assert result.headers.get("x-litellm-key-budget-remaining") == "87.153"
|
|
assert result.headers.get("cf-ray") is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_copilot_accounting_headers_forwarded_in_streaming(self, monkeypatch):
|
|
"""Copilot's ``x-github-*`` / ``*copilot*`` headers survive the streaming path.
|
|
|
|
The Copilot CLI derives AI-credit accounting from these headers; when
|
|
they are dropped it records ``aiCreditsStatus: unavailable`` and
|
|
under-reports credits while token counts still look correct.
|
|
"""
|
|
|
|
# A Copilot URL makes the proxy mint a Copilot OAuth token, which needs a
|
|
# real login; stub it so the test runs without one.
|
|
async def fake_copilot_auth(headers: dict[str, str], *, url: str) -> dict[str, str]:
|
|
return {**headers, "authorization": "Bearer copilot-test-token"}
|
|
|
|
monkeypatch.setattr(streaming_module, "apply_copilot_api_auth", fake_copilot_auth)
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.headers = httpx.Headers(
|
|
{
|
|
"content-type": "text/event-stream",
|
|
"x-github-request-id": "ABCD:1234:5678",
|
|
"x-copilot-usage": "nano-aiu=30213500000",
|
|
"cf-ray": "ray-789",
|
|
}
|
|
)
|
|
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.githubcopilot.com/v1/messages",
|
|
headers={"authorization": "Bearer test"},
|
|
body={
|
|
"model": "claude-opus-4.8",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-opus-4.8",
|
|
request_id="test-copilot-accounting",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.media_type == "text/event-stream"
|
|
assert result.headers.get("x-github-request-id") == "ABCD:1234:5678"
|
|
assert result.headers.get("x-copilot-usage") == "nano-aiu=30213500000"
|
|
assert result.headers.get("cf-ray") is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_no_ratelimit_headers_still_works(self):
|
|
"""When upstream has no ratelimit headers, streaming should still work."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
# Remove all ratelimit headers
|
|
mock_response.headers = httpx.Headers(
|
|
{
|
|
"content-type": "text/event-stream",
|
|
"x-request-id": "req-999",
|
|
}
|
|
)
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-789",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
# Should still return a valid StreamingResponse
|
|
assert result.media_type == "text/event-stream"
|
|
# No ratelimit headers to forward
|
|
assert result.headers.get("anthropic-ratelimit-tokens-limit") is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_upstream_http_error_preserves_status_body_and_metrics(self, monkeypatch):
|
|
"""Upstream non-200 streaming responses should preserve status/body and metrics."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.status_code = 503
|
|
mock_response.headers = httpx.Headers(
|
|
{
|
|
"content-type": "application/json",
|
|
"content-encoding": "gzip",
|
|
"content-length": "42",
|
|
}
|
|
)
|
|
mock_response.aread = AsyncMock(return_value=b'{"error":{"message":"capacity exhausted"}}')
|
|
mock_response.aclose = AsyncMock()
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
fake_logger = MagicMock()
|
|
monkeypatch.setattr(streaming_module, "logger", fake_logger)
|
|
prefix_tracker = MagicMock()
|
|
prefix_tracker.classify_cache_miss.return_value.is_miss = False
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-http-error",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
prefix_tracker=prefix_tracker,
|
|
)
|
|
|
|
assert result.status_code == 503
|
|
assert result.body == b'{"error":{"message":"capacity exhausted"}}'
|
|
assert result.headers.get("content-encoding") is None
|
|
fake_logger.warning.assert_any_call(
|
|
"[%s] Upstream provider returned an error; forwarding it to the client status=%s url=%s",
|
|
"test-http-error",
|
|
503,
|
|
"https://api.anthropic.com/v1/messages",
|
|
)
|
|
# A 503 is an upstream failure, so it books via record_failed and stops
|
|
# before the savings/cost success path — otherwise a failed request
|
|
# inflates the save-rate (#1568).
|
|
proxy.metrics.record_failed.assert_awaited_once()
|
|
proxy.metrics.record_request.assert_not_awaited()
|
|
proxy.cost_tracker.record_tokens.assert_not_called()
|
|
prefix_tracker.classify_cache_miss.assert_not_called()
|
|
prefix_tracker.update_from_response.assert_not_called()
|
|
mock_response.aclose.assert_awaited_once()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_upstream_redirect_does_not_mutate_prefix_tracker(self):
|
|
"""Upstream redirects should not change provider prefix state."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.status_code = 307
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
prefix_tracker = MagicMock()
|
|
prefix_tracker.classify_cache_miss.return_value.is_miss = False
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-redirect",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
prefix_tracker=prefix_tracker,
|
|
)
|
|
|
|
async for _ in result.body_iterator:
|
|
pass
|
|
|
|
prefix_tracker.classify_cache_miss.assert_not_called()
|
|
prefix_tracker.update_from_response.assert_not_called()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_upstream_http_error_closes_response_when_body_read_fails(self, monkeypatch):
|
|
"""Reading a streaming error body should still close the upstream response."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.status_code = 502
|
|
mock_response.aread = AsyncMock(side_effect=RuntimeError("boom"))
|
|
mock_response.aclose = AsyncMock()
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
fake_logger = MagicMock()
|
|
monkeypatch.setattr(streaming_module, "logger", fake_logger)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-http-error-read-fail",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.status_code == 502
|
|
assert result.headers.get("content-type") == "application/json"
|
|
assert b"Failed to read upstream error response body" in result.body
|
|
fake_logger.warning.assert_any_call(
|
|
"[%s] Failed reading upstream error body status=%s url=%s error=%s",
|
|
"test-http-error-read-fail",
|
|
502,
|
|
"https://api.anthropic.com/v1/messages",
|
|
mock_response.aread.side_effect,
|
|
)
|
|
mock_response.aclose.assert_awaited_once()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_connect_error_returns_sse_error(self):
|
|
"""Connection errors should return an SSE error event (not crash)."""
|
|
proxy = self._create_mock_proxy()
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(
|
|
side_effect=httpx.ConnectError("Connection refused to 10.0.0.7:443")
|
|
)
|
|
|
|
# Attach directly to the proxy logger: caplog does not reliably see
|
|
# ``headroom.*`` records once proxy logging is configured.
|
|
records: list[logging.LogRecord] = []
|
|
|
|
class _Collect(logging.Handler):
|
|
def emit(self, record: logging.LogRecord) -> None:
|
|
records.append(record)
|
|
|
|
handler = _Collect(level=logging.DEBUG)
|
|
proxy_logger = logging.getLogger("headroom.proxy")
|
|
proxy_logger.addHandler(handler)
|
|
try:
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-error",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
# Should return a StreamingResponse with error SSE event
|
|
assert result.media_type == "text/event-stream"
|
|
|
|
# Consume the generator to get the error event
|
|
chunks = []
|
|
async for chunk in result.body_iterator:
|
|
chunks.append(chunk)
|
|
finally:
|
|
proxy_logger.removeHandler(handler)
|
|
|
|
assert len(chunks) == 1
|
|
raw = chunks[0].decode("utf-8")
|
|
assert "event: error" in raw
|
|
error_data = json.loads(raw.split("data: ")[1].strip())
|
|
assert error_data["error"]["type"] == "connection_error"
|
|
# Client sees the fixed vocabulary plus a correlation id, never the
|
|
# transport text (which can carry upstream hosts and addresses).
|
|
assert error_data["error"]["code"] == "upstream_unreachable"
|
|
assert error_data["request_id"] == "test-error"
|
|
assert "request_id=test-error" in error_data["error"]["message"]
|
|
assert "Connection refused" not in raw
|
|
assert "10.0.0.7" not in raw
|
|
# The operator still gets the full detail, keyed by the same id.
|
|
logged = [r.getMessage() for r in records]
|
|
assert any(
|
|
"[test-error]" in m and "Connection refused to 10.0.0.7:443" in m for m in logged
|
|
), logged
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_connect_timeout_retries_before_returning_stream(self):
|
|
"""Transient connect timeouts should retry before failing the stream."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
attempts = {"count": 0}
|
|
|
|
async def flaky_send(*args, **kwargs):
|
|
attempts["count"] += 1
|
|
if attempts["count"] == 1:
|
|
raise httpx.ConnectTimeout("timed out")
|
|
return mock_response
|
|
|
|
proxy.http_client.send = AsyncMock(side_effect=flaky_send)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-retry",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
chunks = []
|
|
async for chunk in result.body_iterator:
|
|
chunks.append(chunk)
|
|
|
|
assert attempts["count"] == 2
|
|
assert chunks
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_upstream_stream_closed_when_body_never_consumed(self):
|
|
"""A never-iterated streaming body must still release the upstream stream (#2797).
|
|
|
|
The upstream stream is opened before the body generator, and the
|
|
generator's own ``aclosing`` only runs if the body is iterated. When a
|
|
client disconnects before Starlette starts sending the body the
|
|
generator never runs, so the close must come from the response's
|
|
``background`` task instead — otherwise every such request leaks an open
|
|
HTTP/2 stream and the pooled upstream connection eventually exhausts its
|
|
100 concurrent streams ("Max outbound streams is 100, 100 open").
|
|
"""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-abandoned-stream",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
# Simulate the client disconnecting before the body is consumed: the
|
|
# generator is never iterated, so its aclosing never runs.
|
|
mock_response.aclose.assert_not_awaited()
|
|
|
|
# Starlette runs the response's background task in exactly this case.
|
|
assert result.background is not None, "streaming response must carry a cleanup task"
|
|
await result.background()
|
|
|
|
mock_response.aclose.assert_awaited()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_upstream_stream_released_over_asgi_lifecycle_on_disconnect(self):
|
|
"""Driving the real ASGI response through an early disconnect releases the stream.
|
|
|
|
Rather than calling ``result.background()`` directly, this exercises the
|
|
Starlette response lifecycle with a client that disconnects immediately,
|
|
and asserts the upstream stream is closed by the end of it -- proving the
|
|
cleanup this PR attaches is actually invoked by Starlette, not merely
|
|
present on the response object.
|
|
"""
|
|
import asyncio
|
|
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-asgi-lifecycle",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
async def receive():
|
|
# The client is already gone before the body is streamed.
|
|
return {"type": "http.disconnect"}
|
|
|
|
async def send(_message):
|
|
return None
|
|
|
|
scope = {"type": "http", "method": "POST", "headers": []}
|
|
await asyncio.wait_for(result(scope, receive, send), timeout=5.0)
|
|
|
|
# By the end of the response lifecycle the upstream stream is released.
|
|
mock_response.aclose.assert_awaited()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_codex_rate_limit_headers_captured_and_forwarded_in_streaming(self):
|
|
"""Codex x-codex-* headers must refresh /stats state AND reach the client.
|
|
|
|
Regression guard for the bug where Codex session/weekly usage never
|
|
updated on the streaming SSE transport: the proxy neither captured the
|
|
``x-codex-*`` headers into ``CodexRateLimitState`` nor forwarded them to
|
|
the client (the old ``"ratelimit" in k`` filter dropped them, so the
|
|
Codex CLI's own usage display also went stale).
|
|
"""
|
|
from headroom.subscription.codex_rate_limits import get_codex_rate_limit_state
|
|
|
|
state = get_codex_rate_limit_state()
|
|
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response(
|
|
extra_headers={
|
|
"x-codex-primary-used-percent": "42.0",
|
|
"x-codex-primary-window-minutes": "300",
|
|
"x-codex-secondary-used-percent": "8.0",
|
|
"x-codex-secondary-window-minutes": "10080",
|
|
"x-codex-limit-name": "gpt-5.4-codex",
|
|
}
|
|
)
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://chatgpt.com/backend-api/codex/responses",
|
|
headers={"authorization": "Bearer sk-test"},
|
|
body={"model": "gpt-5.4", "stream": True, "input": "hi"},
|
|
provider="openai",
|
|
model="gpt-5.4",
|
|
request_id="test-codex-sse",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
# 1. Rate-limit state refreshed from the *streaming* response.
|
|
snap = state.latest
|
|
assert snap is not None
|
|
assert snap.primary is not None
|
|
assert snap.primary.used_percent == 42.0
|
|
assert snap.primary.window_minutes == 300
|
|
assert snap.secondary is not None
|
|
assert snap.secondary.used_percent == 8.0
|
|
assert snap.secondary.window_minutes == 10080
|
|
assert snap.limit_name == "gpt-5.4-codex"
|
|
|
|
# 2. x-codex headers forwarded so the Codex CLI's native usage display
|
|
# keeps working through the proxy on the streaming path.
|
|
assert result.headers.get("x-codex-primary-used-percent") == "42.0"
|
|
assert result.headers.get("x-codex-limit-name") == "gpt-5.4-codex"
|
|
# 3. Generic ratelimit headers and the request-id family forwarded;
|
|
# other unrelated headers dropped.
|
|
assert result.headers.get("anthropic-ratelimit-tokens-limit") == "80000"
|
|
assert result.headers.get("x-request-id") == "req-12345"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_codex_rate_limit_captured_on_streaming_429(self):
|
|
"""A streaming 429 carrying x-codex-* must still refresh /stats.
|
|
|
|
The capture runs *before* the >=400 early-return, matching the
|
|
non-streaming HTTP handlers (which capture on all statuses). A 429 is
|
|
exactly when the session/weekly windows are most worth surfacing, so the
|
|
previous success-only placement left the most important update missing.
|
|
"""
|
|
from headroom.subscription.codex_rate_limits import get_codex_rate_limit_state
|
|
|
|
state = get_codex_rate_limit_state()
|
|
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.status_code = 429
|
|
mock_response.headers = httpx.Headers(
|
|
{
|
|
"content-type": "application/json",
|
|
"x-codex-primary-used-percent": "99.5",
|
|
"x-codex-primary-window-minutes": "300",
|
|
}
|
|
)
|
|
mock_response.aread = AsyncMock(return_value=b'{"error":{"message":"rate limited"}}')
|
|
mock_response.aclose = AsyncMock()
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://chatgpt.com/backend-api/codex/responses",
|
|
headers={"authorization": "Bearer sk-test"},
|
|
body={"model": "gpt-5.4", "stream": True, "input": "hi"},
|
|
provider="openai",
|
|
model="gpt-5.4",
|
|
request_id="test-codex-429",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.status_code == 429
|
|
snap = state.latest
|
|
assert snap is not None
|
|
assert snap.primary is not None
|
|
assert snap.primary.used_percent == 99.5
|
|
|
|
@pytest.mark.asyncio
|
|
@pytest.mark.parametrize(
|
|
("hint_headers", "expected_sends"),
|
|
[
|
|
({"retry-after": "3600"}, 1),
|
|
({"x-should-retry": "false"}, 1),
|
|
({}, 3),
|
|
],
|
|
)
|
|
async def test_streaming_429_skips_futile_retries(self, hint_headers, expected_sends):
|
|
"""A 429 whose reset lies beyond the backoff cap is forwarded at once."""
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response()
|
|
mock_response.status_code = 429
|
|
mock_response.headers = httpx.Headers({"content-type": "application/json", **hint_headers})
|
|
mock_response.aread = AsyncMock(
|
|
return_value=b'{"type":"error","error":{"type":"rate_limit_error"}}'
|
|
)
|
|
mock_response.aclose = AsyncMock()
|
|
|
|
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-ant-test"},
|
|
body={"model": "claude-sonnet-4-6", "stream": True, "messages": []},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-6",
|
|
request_id="test-futile-429",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.status_code == 429
|
|
assert proxy.http_client.send.await_count == expected_sends
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_streaming_429_retry_after_reachable_across_retries_recovers(self):
|
|
"""A reset beyond one capped wait but within the remaining retries is retried."""
|
|
proxy = self._create_mock_proxy()
|
|
proxy._config.retry_max_delay_ms = 20
|
|
|
|
def _rate_limited(retry_after):
|
|
response = self._create_mock_upstream_response()
|
|
response.status_code = 429
|
|
response.headers = httpx.Headers(
|
|
{"content-type": "application/json", "retry-after": retry_after}
|
|
)
|
|
response.aread = AsyncMock(return_value=b'{"type":"error"}')
|
|
response.aclose = AsyncMock()
|
|
return response
|
|
|
|
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
|
proxy.http_client.send = AsyncMock(
|
|
side_effect=[
|
|
_rate_limited("0.031"),
|
|
_rate_limited("0.001"),
|
|
self._create_mock_upstream_response(),
|
|
]
|
|
)
|
|
|
|
result = await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-ant-test"},
|
|
body={"model": "claude-sonnet-4-6", "stream": True, "messages": []},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-6",
|
|
request_id="test-reachable-429",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert result.status_code == 200
|
|
assert proxy.http_client.send.await_count == 3
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_anthropic_stream_leaves_codex_state_untouched(self):
|
|
"""The now-unconditional capture must be a no-op for non-Codex streams."""
|
|
from headroom.subscription.codex_rate_limits import get_codex_rate_limit_state
|
|
|
|
state = get_codex_rate_limit_state()
|
|
|
|
proxy = self._create_mock_proxy()
|
|
mock_response = self._create_mock_upstream_response() # anthropic-ratelimit-* only
|
|
|
|
mock_request = MagicMock()
|
|
proxy.http_client.build_request = MagicMock(return_value=mock_request)
|
|
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
|
|
|
await proxy._stream_response(
|
|
url="https://api.anthropic.com/v1/messages",
|
|
headers={"x-api-key": "sk-test"},
|
|
body={
|
|
"model": "claude-sonnet-4-20250514",
|
|
"max_tokens": 100,
|
|
"stream": True,
|
|
"messages": [{"role": "user", "content": "hi"}],
|
|
},
|
|
provider="anthropic",
|
|
model="claude-sonnet-4-20250514",
|
|
request_id="test-anthropic-noop",
|
|
original_tokens=10,
|
|
optimized_tokens=10,
|
|
tokens_saved=0,
|
|
transforms_applied=[],
|
|
tags={},
|
|
optimization_latency=0.0,
|
|
)
|
|
|
|
assert state.latest is None
|