fix(proxy): forward x-litellm-key-* headers on streaming path (#2532)

## Description

The streaming SSE path rebuilds downstream headers from an allowlist
that forwards `*ratelimit*`, `x-codex-*`, and the `request-id` family —
but drops `x-litellm-key-*`, while the buffered (non-streaming) path
forwards every upstream header. LiteLLM virtual keys report a per-key
spend/budget envelope on every response (`x-litellm-key-spend`,
`x-litellm-key-budget-remaining`, …); these are silently lost for
streaming clients.

This adds `k.lower().startswith("x-litellm-key-")` to the streaming
allowlist — the same one-line widening applied for `*ratelimit*` (#57),
`x-codex-*` (#794), and `request-id` (#1100) — keeping the two paths
symmetric.

Closes #2531

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/proxy/handlers/streaming.py`: added `or
k.lower().startswith("x-litellm-key-")` to the `forwarded_headers`
allowlist in `_stream_response`, and expanded the existing comment to
explain the new clause.
- `tests/test_proxy_streaming_ratelimit_headers.py`: added
`test_litellm_key_headers_forwarded_in_streaming`, mirroring the
existing `test_request_id_headers_forwarded_in_streaming` (#1100). It
asserts `x-litellm-key-spend` / `x-litellm-key-budget-remaining` are
forwarded and that a non-allowlisted header (`cf-ray`) is still dropped.

## Testing

- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality

### Test Output

```text
$ ruff check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_ratelimit_headers.py --config pyproject.toml
[]
$ ruff format --check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_ratelimit_headers.py --config pyproject.toml
2 files already formatted
```

## Real Behavior Proof

- Environment: Arch Linux, kernel 7.1.4, Python 3.14.6. Upstream LiteLLM
proxy as the upstream provider; Headroom built locally from the 0.32.1
source with the streaming allowlist patched for this change, running as
`headroom proxy`.
- Exact command / steps: A streaming POST through the patched Headroom
proxy (with a LiteLLM upstream) returns `x-litellm-key-spend` in the
response headers, where the unpatched build dropped it. A status widget
reading `x-litellm-key-*` from the streaming response resumes updating
after the patch.
- Observed result: `x-litellm-key-spend` (and sibling `x-litellm-key-*`)
present in the streaming response headers with the patch; absent without
it. The buffered path carried them in both cases.
- Not tested: The full `pytest` suite was not run locally. The
backend/compression streaming paths (`_stream_*_via_backend`) forward no
response headers at all and would need a separate change, as noted in
#1100 — out of scope here.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my changes
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2532"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Bartosz Oudekerk <bartosz.oudekerk@bloomreach.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Bartosz Oudekerk
2026-10-08 10:58:06 -05:00
committed by GitHub
co-authored by Bartosz Oudekerk JD Davis
parent b1fb162b62
commit 33ade5b0e1
2 changed files with 59 additions and 2 deletions
+7 -2
View File
@@ -1674,13 +1674,18 @@ class StreamingMixin:
# session/weekly display would stop updating on the streaming path.
# We also forward the ``request-id`` family: clients such as Claude Code
# record it per transcript turn, and downstream usage/cost tools dedup by
# message id + request id. The buffered (non-streaming) path already
# forwards every upstream header, so this keeps the two paths symmetric.
# message id + request id. ``x-litellm-key-*`` carries the per-key
# spend/budget envelope a LiteLLM virtual key reports per request;
# downstream status widgets read it from the streaming response the
# same way they already do from the buffered path. The buffered
# (non-streaming) path already forwards every upstream header, so
# this keeps the two paths symmetric.
forwarded_headers = {
k: v
for k, v in upstream_response.headers.items()
if "ratelimit" in k.lower()
or k.lower().startswith("x-codex")
or k.lower().startswith("x-litellm-key-")
or k.lower() in ("request-id", "anthropic-request-id", "x-request-id")
}
# Headroom's own compression metrics, as the buffered path stamps them.
@@ -297,6 +297,58 @@ class TestStreamingRatelimitHeaderForwarding:
assert result.headers.get("x-request-id") == "req-ccc"
assert result.headers.get("cf-ray") is None
@pytest.mark.asyncio
async def test_litellm_key_headers_forwarded_in_streaming(self):
"""The x-litellm-key-* family is forwarded on the streaming path.
LiteLLM virtual-key spend/budget headers (``x-litellm-key-spend``,
``x-litellm-key-budget-remaining``, etc.) are dropped on the streaming
path because they match no allowlist clause, even though the buffered
path forwards every upstream header. Downstream status widgets that
read them from the streaming response stop updating — the same class
of omission fixed for ``*ratelimit*``, ``x-codex-*`` and ``request-id``.
"""
proxy = self._create_mock_proxy()
mock_response = self._create_mock_upstream_response()
mock_response.headers = httpx.Headers(
{
"content-type": "text/event-stream",
"x-litellm-key-spend": "12.847",
"x-litellm-key-budget-remaining": "87.153",
# Non-allowlisted header: must NOT be forwarded.
"cf-ray": "ray-456",
}
)
mock_request = MagicMock()
proxy.http_client.build_request = MagicMock(return_value=mock_request)
proxy.http_client.send = AsyncMock(return_value=mock_response)
result = await proxy._stream_response(
url="https://api.anthropic.com/v1/messages",
headers={"x-api-key": "sk-test"},
body={
"model": "claude-sonnet-4-20250514",
"max_tokens": 100,
"stream": True,
"messages": [{"role": "user", "content": "hi"}],
},
provider="anthropic",
model="claude-sonnet-4-20250514",
request_id="test-litellm-key",
original_tokens=10,
optimized_tokens=10,
tokens_saved=0,
transforms_applied=[],
tags={},
optimization_latency=0.0,
)
assert result.media_type == "text/event-stream"
assert result.headers.get("x-litellm-key-spend") == "12.847"
assert result.headers.get("x-litellm-key-budget-remaining") == "87.153"
assert result.headers.get("cf-ray") is None
@pytest.mark.asyncio
async def test_no_ratelimit_headers_still_works(self):
"""When upstream has no ratelimit headers, streaming should still work."""