fix(proxy): stamp x-headroom-* compression metrics on streaming responses (#3769)

## Description

The buffered (non-streaming) path stamps `x-headroom-tokens-before`,
`-after`, `-saved`, `-model` and `-transforms` on its response. The
streaming path does not: `_stream_response_inner` forwards only the
upstream rate-limit and request-id headers. Coding agents stream, so
every Claude Code and Codex turn comes back without the compression
metrics, and a client or a metering layer in front of the proxy cannot
tell what a request saved without scraping `/stats` or the logs.

Every value is already a `_stream_response` argument, known before the
first byte, so this adds them to `forwarded_headers`, the same way the
buffered path does.

Related: #2531 / #2532 forward LiteLLM spend headers on the same
streaming block. That is a different header set, and the two changes
don't depend on each other.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/proxy/handlers/streaming.py`: add
`x-headroom-tokens-before/-after/-saved` and `x-headroom-model` to the
streaming response headers, plus `x-headroom-transforms` (via
`header_safe_transforms`, as on the buffered path) when anything was
applied.
- `tests/test_proxy_streaming_ratelimit_headers.py`: two tests, one for
the full set including comma-safe transform tags alongside the forwarded
upstream headers, and one for a no-op request (counts present, no empty
transforms header).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality

### Test Output

```text
$ uv run --frozen --extra dev pytest tests/test_proxy_streaming_ratelimit_headers.py -q
16 passed in 1.77s

# Same two new tests against upstream/main's streaming.py (fix reverted):
2 failed, 14 passed in 2.21s

$ uv run --frozen --extra dev pytest tests -q -k "stream or header or x_headroom"
887 passed, 29 skipped, 12593 deselected in 125.81s

$ uvx ruff check <changed files>   -> All checks passed!
$ uvx ruff format --check <changed files>   -> 2 files already formatted
$ uv run --frozen --extra dev mypy headroom/proxy/handlers/streaming.py   -> Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS 15 (arm64), headroom-ai 0.38.0 behind Headroom
Desktop, real Claude Code 2.1.28x traffic (terminal and VS Code) to
api.anthropic.com. The same header stamping ran there as a local patch
wrapping `_stream_response` from 2026-09-23 on.
- Exact command / steps: Used Claude Code normally through the proxy. A
client in front of the proxy read `x-headroom-tokens-saved` from each
streaming `/v1/messages` response head and summed it per
`x-claude-code-session-id`.
- Observed result: Streaming responses carried the headers, and the
per-request savings the client recorded matched the proxy log's
`tok_saved` for the same turns (e.g. `tok_saved=2865` in the log, +2865
recorded), across 64 conversations in about a day. Without the patch the
0.38.0 streaming handler sets none of these headers (read from source),
so the client had nothing to record.
- Not tested: This exact branch end to end against a live provider (the
evidence above is the equivalent patch on 0.38.0; this branch is covered
by the unit tests), and the Bedrock and Gemini streaming handlers, which
build their own responses and are unchanged.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes, additively: streaming responses
gain five `x-headroom-*` headers the buffered path already sends. The
body and all existing headers are unchanged.
- Kill switch / disable path: None needed; clients ignore unknown
response headers. Reverting the commit removes them.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

Documentation: no doc lists the streaming response headers today, so
nothing to update.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
gglucass
2026-09-24 16:12:20 -05:00
committed by GitHub
co-authored by Claude Opus 5.5
parent df8ebdf97d
commit 2c4dc44627
2 changed files with 80 additions and 0 deletions
+15
View File
@@ -1571,6 +1571,21 @@ class StreamingMixin:
or k.lower().startswith("x-codex")
or k.lower() in ("request-id", "anthropic-request-id", "x-request-id")
}
# Headroom's own compression metrics, as the buffered path stamps them.
# Every counted value is known before the first byte, and streaming is
# how real coding agents talk to the proxy, so without these a client
# (or a metering layer in front of it) never learns what its request
# saved.
forwarded_headers["x-headroom-tokens-before"] = str(original_tokens)
forwarded_headers["x-headroom-tokens-after"] = str(optimized_tokens)
forwarded_headers["x-headroom-tokens-saved"] = str(tokens_saved)
forwarded_headers["x-headroom-model"] = model
if transforms_applied:
from headroom.proxy.cost import header_safe_transforms
forwarded_headers["x-headroom-transforms"] = ",".join(
header_safe_transforms(transforms_applied)
)
async def generate():
nonlocal body, memory_enabled # May need to modify for continuation requests
@@ -185,6 +185,71 @@ class TestStreamingRatelimitHeaderForwarding:
assert result.headers.get("x-request-id") == "req-12345"
assert result.headers.get("cf-ray") is None
@pytest.mark.asyncio
async def test_compression_metrics_headers_on_streaming(self):
"""The streaming path stamps the same x-headroom-* metrics as the buffered one."""
proxy = self._create_mock_proxy()
mock_response = self._create_mock_upstream_response()
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
proxy.http_client.send = AsyncMock(return_value=mock_response)
result = await proxy._stream_response(
url="https://api.anthropic.com/v1/messages",
headers={"x-api-key": "sk-test"},
body={
"model": "claude-sonnet-4-20250514",
"max_tokens": 100,
"stream": True,
"messages": [{"role": "user", "content": "hi"}],
},
provider="anthropic",
model="claude-sonnet-4-20250514",
request_id="test-metrics",
original_tokens=1000,
optimized_tokens=400,
tokens_saved=600,
transforms_applied=["smart_crusher", "read_lifecycle:stale:/a,b.py"],
tags={},
optimization_latency=0.0,
)
assert result.headers.get("x-headroom-tokens-before") == "1000"
assert result.headers.get("x-headroom-tokens-after") == "400"
assert result.headers.get("x-headroom-tokens-saved") == "600"
assert result.headers.get("x-headroom-model") == "claude-sonnet-4-20250514"
# Comma-bearing detail is collapsed so the header still splits into tags.
transforms = result.headers.get("x-headroom-transforms")
assert transforms is not None and transforms.split(",")[0] == "smart_crusher"
assert "/a,b.py" not in transforms
# Upstream headers are still forwarded alongside.
assert result.headers.get("x-request-id") == "req-12345"
@pytest.mark.asyncio
async def test_no_transforms_header_when_nothing_was_applied(self):
"""A request nothing touched still reports its counts, but no empty transforms."""
proxy = self._create_mock_proxy()
mock_response = self._create_mock_upstream_response()
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
proxy.http_client.send = AsyncMock(return_value=mock_response)
result = await proxy._stream_response(
url="https://api.anthropic.com/v1/messages",
headers={"x-api-key": "sk-test"},
body={"model": "m", "max_tokens": 1, "stream": True, "messages": []},
provider="anthropic",
model="m",
request_id="test-noop",
original_tokens=10,
optimized_tokens=10,
tokens_saved=0,
transforms_applied=[],
tags={},
optimization_latency=0.0,
)
assert result.headers.get("x-headroom-tokens-saved") == "0"
assert result.headers.get("x-headroom-transforms") is None
@pytest.mark.asyncio
async def test_request_id_headers_forwarded_in_streaming(self):
"""The request-id family is forwarded on the streaming path (#1100)."""