mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): stamp x-headroom-* compression metrics on streaming responses (#3769)
## Description The buffered (non-streaming) path stamps `x-headroom-tokens-before`, `-after`, `-saved`, `-model` and `-transforms` on its response. The streaming path does not: `_stream_response_inner` forwards only the upstream rate-limit and request-id headers. Coding agents stream, so every Claude Code and Codex turn comes back without the compression metrics, and a client or a metering layer in front of the proxy cannot tell what a request saved without scraping `/stats` or the logs. Every value is already a `_stream_response` argument, known before the first byte, so this adds them to `forwarded_headers`, the same way the buffered path does. Related: #2531 / #2532 forward LiteLLM spend headers on the same streaming block. That is a different header set, and the two changes don't depend on each other. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/handlers/streaming.py`: add `x-headroom-tokens-before/-after/-saved` and `x-headroom-model` to the streaming response headers, plus `x-headroom-transforms` (via `header_safe_transforms`, as on the buffered path) when anything was applied. - `tests/test_proxy_streaming_ratelimit_headers.py`: two tests, one for the full set including comma-safe transform tags alongside the forwarded upstream headers, and one for a no-op request (counts present, no empty transforms header). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_proxy_streaming_ratelimit_headers.py -q 16 passed in 1.77s # Same two new tests against upstream/main's streaming.py (fix reverted): 2 failed, 14 passed in 2.21s $ uv run --frozen --extra dev pytest tests -q -k "stream or header or x_headroom" 887 passed, 29 skipped, 12593 deselected in 125.81s $ uvx ruff check <changed files> -> All checks passed! $ uvx ruff format --check <changed files> -> 2 files already formatted $ uv run --frozen --extra dev mypy headroom/proxy/handlers/streaming.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS 15 (arm64), headroom-ai 0.38.0 behind Headroom Desktop, real Claude Code 2.1.28x traffic (terminal and VS Code) to api.anthropic.com. The same header stamping ran there as a local patch wrapping `_stream_response` from 2026-09-23 on. - Exact command / steps: Used Claude Code normally through the proxy. A client in front of the proxy read `x-headroom-tokens-saved` from each streaming `/v1/messages` response head and summed it per `x-claude-code-session-id`. - Observed result: Streaming responses carried the headers, and the per-request savings the client recorded matched the proxy log's `tok_saved` for the same turns (e.g. `tok_saved=2865` in the log, +2865 recorded), across 64 conversations in about a day. Without the patch the 0.38.0 streaming handler sets none of these headers (read from source), so the client had nothing to record. - Not tested: This exact branch end to end against a live provider (the evidence above is the equivalent patch on 0.38.0; this branch is covered by the unit tests), and the Bedrock and Gemini streaming handlers, which build their own responses and are unchanged. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Stable. - Stable/default behavior changed: Yes, additively: streaming responses gain five `x-headroom-*` headers the buffered path already sends. The body and all existing headers are unchanged. - Kill switch / disable path: None needed; clients ignore unknown response headers. Reverting the commit removes them. - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable. ## Additional Notes Documentation: no doc lists the streaming response headers today, so nothing to update. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
df8ebdf97d
commit
2c4dc44627
@@ -1571,6 +1571,21 @@ class StreamingMixin:
|
||||
or k.lower().startswith("x-codex")
|
||||
or k.lower() in ("request-id", "anthropic-request-id", "x-request-id")
|
||||
}
|
||||
# Headroom's own compression metrics, as the buffered path stamps them.
|
||||
# Every counted value is known before the first byte, and streaming is
|
||||
# how real coding agents talk to the proxy, so without these a client
|
||||
# (or a metering layer in front of it) never learns what its request
|
||||
# saved.
|
||||
forwarded_headers["x-headroom-tokens-before"] = str(original_tokens)
|
||||
forwarded_headers["x-headroom-tokens-after"] = str(optimized_tokens)
|
||||
forwarded_headers["x-headroom-tokens-saved"] = str(tokens_saved)
|
||||
forwarded_headers["x-headroom-model"] = model
|
||||
if transforms_applied:
|
||||
from headroom.proxy.cost import header_safe_transforms
|
||||
|
||||
forwarded_headers["x-headroom-transforms"] = ",".join(
|
||||
header_safe_transforms(transforms_applied)
|
||||
)
|
||||
|
||||
async def generate():
|
||||
nonlocal body, memory_enabled # May need to modify for continuation requests
|
||||
|
||||
@@ -185,6 +185,71 @@ class TestStreamingRatelimitHeaderForwarding:
|
||||
assert result.headers.get("x-request-id") == "req-12345"
|
||||
assert result.headers.get("cf-ray") is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_compression_metrics_headers_on_streaming(self):
|
||||
"""The streaming path stamps the same x-headroom-* metrics as the buffered one."""
|
||||
proxy = self._create_mock_proxy()
|
||||
mock_response = self._create_mock_upstream_response()
|
||||
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
||||
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
||||
|
||||
result = await proxy._stream_response(
|
||||
url="https://api.anthropic.com/v1/messages",
|
||||
headers={"x-api-key": "sk-test"},
|
||||
body={
|
||||
"model": "claude-sonnet-4-20250514",
|
||||
"max_tokens": 100,
|
||||
"stream": True,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
},
|
||||
provider="anthropic",
|
||||
model="claude-sonnet-4-20250514",
|
||||
request_id="test-metrics",
|
||||
original_tokens=1000,
|
||||
optimized_tokens=400,
|
||||
tokens_saved=600,
|
||||
transforms_applied=["smart_crusher", "read_lifecycle:stale:/a,b.py"],
|
||||
tags={},
|
||||
optimization_latency=0.0,
|
||||
)
|
||||
|
||||
assert result.headers.get("x-headroom-tokens-before") == "1000"
|
||||
assert result.headers.get("x-headroom-tokens-after") == "400"
|
||||
assert result.headers.get("x-headroom-tokens-saved") == "600"
|
||||
assert result.headers.get("x-headroom-model") == "claude-sonnet-4-20250514"
|
||||
# Comma-bearing detail is collapsed so the header still splits into tags.
|
||||
transforms = result.headers.get("x-headroom-transforms")
|
||||
assert transforms is not None and transforms.split(",")[0] == "smart_crusher"
|
||||
assert "/a,b.py" not in transforms
|
||||
# Upstream headers are still forwarded alongside.
|
||||
assert result.headers.get("x-request-id") == "req-12345"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_transforms_header_when_nothing_was_applied(self):
|
||||
"""A request nothing touched still reports its counts, but no empty transforms."""
|
||||
proxy = self._create_mock_proxy()
|
||||
mock_response = self._create_mock_upstream_response()
|
||||
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
|
||||
proxy.http_client.send = AsyncMock(return_value=mock_response)
|
||||
|
||||
result = await proxy._stream_response(
|
||||
url="https://api.anthropic.com/v1/messages",
|
||||
headers={"x-api-key": "sk-test"},
|
||||
body={"model": "m", "max_tokens": 1, "stream": True, "messages": []},
|
||||
provider="anthropic",
|
||||
model="m",
|
||||
request_id="test-noop",
|
||||
original_tokens=10,
|
||||
optimized_tokens=10,
|
||||
tokens_saved=0,
|
||||
transforms_applied=[],
|
||||
tags={},
|
||||
optimization_latency=0.0,
|
||||
)
|
||||
|
||||
assert result.headers.get("x-headroom-tokens-saved") == "0"
|
||||
assert result.headers.get("x-headroom-transforms") is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_request_id_headers_forwarded_in_streaming(self):
|
||||
"""The request-id family is forwarded on the streaming path (#1100)."""
|
||||
|
||||
Reference in New Issue
Block a user