fix(proxy): forward Copilot accounting headers on streaming path (#4431)

## Description

The streaming path only forwards upstream headers on a short list
(`*ratelimit*`, `x-codex*`, `x-litellm-key-*`, request-id family).
GitHub Copilot uses `x-github-*` / `*copilot*` headers for AI-credit
accounting, and those are not on the list, so they get dropped. When
they're missing, the Copilot CLI stamps streamed usage records
`aiCreditsStatus: "unavailable"`, leaves out
`serviceRequestId`/`providerCallId` and the `copilotUsage` block (which
carries `totalNanoAiu`), and stops advancing
`session.usage_checkpoint.totalNanoAiu`. Token counts arrive in the SSE
body and look correct, so nothing errors. Credits are just silently
under-reported by about 4–5x.

The buffered path already forwards every upstream header. Real coding
agents stream, so the streaming path is the one that matters for
Copilot.

In a 9-task agentic A/B on `claude-opus-4.8` (one arm direct, one
through Headroom):

| | direct | via Headroom |
|---|---|---|
| `aiCreditsStatus: complete` | 990 (100%) | 247 (34.9%) |
| `aiCreditsStatus: unavailable` | 0 | 460 (65.1%) |
| reported AI credits | 8,015.9 | 1,471.8 (−82%) |
| actual cacheRead tokens | 98.5M | 68.8M (−30%) |

A −82% drop in credits against a −30% drop in tokens is a measurement
gap, not a saving.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/streaming.py`: forward headers that start
with `x-github` or contain `copilot`, and update the comment to explain
why.
- `tests/test_proxy_streaming_ratelimit_headers.py`: add
`test_copilot_accounting_headers_forwarded_in_streaming`, which checks
that `x-github-request-id` and `x-copilot-usage` are forwarded on a
streamed response and an unrelated header (`cf-ray`) is still dropped.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_proxy_streaming_ratelimit_headers.py
22 passed in 30.59s

# The new test without the fix:
$ pytest -q tests/test_proxy_streaming_ratelimit_headers.py -k copilot
FAILED ...::test_copilot_accounting_headers_forwarded_in_streaming
1 failed, 21 deselected

$ ruff check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_ratelimit_headers.py
All checks passed!
$ ruff format --check <same files>
2 files already formatted
```

## Real Behavior Proof

## Real Behavior Proof

- Environment:
  - macOS 26.6.2 (arm64)
  - Python 3.12.12
- headroom-ai 0.40.0 from PyPI in a venv (`~/.venvs/headroom-040`), with
this PR's two-line change applied to
`headroom/proxy/handlers/streaming.py` in site-packages (same diff as
this branch)
- GitHub Copilot CLI 1.0.103, model `claude-opus-4.8`, reasoning effort
`medium`
  - Upstream: `https://api.githubcopilot.com`

- Exact command / steps:
  1. Start the proxy, with telemetry off and routed to Copilot:
     ```bash
HEADROOM_COMPRESS_WORKERS=8 HEADROOM_COMPRESSION_DEADLINE_MS=60000 \
     ANTHROPIC_TARGET_API_URL=https://api.githubcopilot.com \
     OPENAI_TARGET_API_URL=https://api.githubcopilot.com \
     GITHUB_COPILOT_API_URL=https://api.githubcopilot.com \
     headroom proxy --port 8787 > /tmp/hr-proxy-hdrfix.log 2>&1
     ```
  2. Send one streamed Copilot CLI request through it:
     ```bash
     cd /tmp && COPILOT_API_URL=http://127.0.0.1:8787 \
copilot --no-experimental --model claude-opus-4.8 --allow-all-tools \
       -p "Reply with exactly: OK"
     ```
  3. Confirm the streaming path was used:
     ```bash
     grep "forwarder=streaming" /tmp/hr-proxy-hdrfix.log
     ```
4. Check the CLI session transcript
(`~/.copilot/session-state/<session-id>/events.jsonl`):
     ```bash
     grep -o '"aiCreditsStatus":"[a-z]*"' events.jsonl | sort | uniq -c
     grep -o '"serviceRequestId":"[^"]*"' events.jsonl | head -1
     grep -o '"totalNanoAiu":[0-9]*' events.jsonl | tail -1
     ```

- Observed result: Verified end-to-end. The proxy log shows
`forwarder=streaming` (`keeping the streaming path (#3071)`). In the
resulting transcript, all 9 records are `aiCreditsStatus: complete` and
`serviceRequestId` is present. The last checkpoint is `totalNanoAiu =
30,213,500,000` → 30.21 AIC, exactly matching the CLI's displayed `AI
Credits 30.21`.
- Before the patch: in the A/B eval through unpatched 0.40.0 with the
same proxy settings, streamed records were stamped `aiCreditsStatus:
"unavailable"` with no `serviceRequestId` and no `copilotUsage`. For
example, one task had 70 of 113 records unavailable, and the checkpoint
covered only 22% of cacheRead tokens.

- Not tested: a live run built from this branch (the identical patch was
verified on the 0.40.0 wheel instead); non-Copilot providers end-to-end
(unit tests only); `mypy`.

## Runtime Rollout Safety

- Rollout-managed feature(s): None
- Minimum rollout channel: N/A
- Stable/default behavior changed: Yes. The streaming path now also
forwards `x-github*` and `*copilot*` response headers, which the
buffered path already forwards.
- Kill switch / disable path: None; revert the commit.
- Unsafe override required: No
- Qualification impact: None. Both are response metadata headers, never
hop-by-hop.
- Rollback path: Revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A


<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4431"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: dmanepal <dmanepal@linkedin.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
divyamnpll
2026-11-03 22:53:38 -06:00
committed by GitHub
co-authored by dmanepal JD Davis
parent 6cc0801219
commit 66023a0187
2 changed files with 60 additions and 1 deletions
+7 -1
View File
@@ -1616,7 +1616,11 @@ class StreamingMixin:
# message id + request id. ``x-litellm-key-*`` carries the per-key
# spend/budget envelope a LiteLLM virtual key reports per request;
# downstream status widgets read it from the streaming response the
# same way they already do from the buffered path. The buffered
# same way they already do from the buffered path. GitHub Copilot keys
# its AI-credit accounting off ``x-github-*`` / ``*copilot*`` headers;
# without them the Copilot CLI marks streamed usage records
# ``aiCreditsStatus: unavailable`` and silently under-reports credits.
# The buffered
# (non-streaming) path already forwards every upstream header, so
# this keeps the two paths symmetric.
forwarded_headers = {
@@ -1625,6 +1629,8 @@ class StreamingMixin:
if "ratelimit" in k.lower()
or k.lower().startswith("x-codex")
or k.lower().startswith("x-litellm-key-")
or k.lower().startswith("x-github")
or "copilot" in k.lower()
or k.lower() in ("request-id", "anthropic-request-id", "x-request-id")
}
# Headroom's own compression metrics, as the buffered path stamps them.
@@ -349,6 +349,59 @@ class TestStreamingRatelimitHeaderForwarding:
assert result.headers.get("x-litellm-key-budget-remaining") == "87.153"
assert result.headers.get("cf-ray") is None
@pytest.mark.asyncio
async def test_copilot_accounting_headers_forwarded_in_streaming(self, monkeypatch):
"""Copilot's ``x-github-*`` / ``*copilot*`` headers survive the streaming path.
The Copilot CLI derives AI-credit accounting from these headers; when
they are dropped it records ``aiCreditsStatus: unavailable`` and
under-reports credits while token counts still look correct.
"""
# A Copilot URL makes the proxy mint a Copilot OAuth token, which needs a
# real login; stub it so the test runs without one.
async def fake_copilot_auth(headers: dict[str, str], *, url: str) -> dict[str, str]:
return {**headers, "authorization": "Bearer copilot-test-token"}
monkeypatch.setattr(streaming_module, "apply_copilot_api_auth", fake_copilot_auth)
proxy = self._create_mock_proxy()
mock_response = self._create_mock_upstream_response()
mock_response.headers = httpx.Headers(
{
"content-type": "text/event-stream",
"x-github-request-id": "ABCD:1234:5678",
"x-copilot-usage": "nano-aiu=30213500000",
"cf-ray": "ray-789",
}
)
proxy.http_client.build_request = MagicMock(return_value=MagicMock())
proxy.http_client.send = AsyncMock(return_value=mock_response)
result = await proxy._stream_response(
url="https://api.githubcopilot.com/v1/messages",
headers={"authorization": "Bearer test"},
body={
"model": "claude-opus-4.8",
"max_tokens": 100,
"stream": True,
"messages": [{"role": "user", "content": "hi"}],
},
provider="anthropic",
model="claude-opus-4.8",
request_id="test-copilot-accounting",
original_tokens=10,
optimized_tokens=10,
tokens_saved=0,
transforms_applied=[],
tags={},
optimization_latency=0.0,
)
assert result.media_type == "text/event-stream"
assert result.headers.get("x-github-request-id") == "ABCD:1234:5678"
assert result.headers.get("x-copilot-usage") == "nano-aiu=30213500000"
assert result.headers.get("cf-ray") is None
@pytest.mark.asyncio
async def test_no_ratelimit_headers_still_works(self):
"""When upstream has no ratelimit headers, streaming should still work."""