fix(proxy): honor bypass header and license gate in Anthropic batch create (#4411)

## Description

`handle_anthropic_batch_create` ignored `x-headroom-bypass: true`,
`x-headroom-mode: passthrough` and the `usage_reporter.should_compress`
license gate, and always re-serialized the body. The OpenAI and Google
batch paths enforce all of these. This PR makes the Anthropic batch path
do the same: compression is skipped for each item in either case, and on
bypass the client's original bytes are forwarded.

Fixes #4404

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/anthropic.py`: read the body with
`read_request_json_with_bytes`; compute `_bypass`
(`_headroom_bypass_enabled`) and `license_ok` (same expression as
`batch.py`, fails open without a reporter); tag `passthrough_reason`;
skip the per-item compression loop body when either gate applies; on
bypass call `_retry_request` with `original_body_bytes` and
`body_mutated=False`.
- `tests/test_proxy/test_anthropic_batch_bypass.py`: 4 tests (bypass
header, `x-headroom-mode: passthrough`, license denied, unchanged
compression path).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

With only the source change reverted (new test file kept):

```text
FAILED tests/test_proxy/test_anthropic_batch_bypass.py::test_batch_create_bypass_skips_compression_and_forwards_original_bytes[bypass_headers0]
FAILED tests/test_proxy/test_anthropic_batch_bypass.py::test_batch_create_bypass_skips_compression_and_forwards_original_bytes[bypass_headers1]
FAILED tests/test_proxy/test_anthropic_batch_bypass.py::test_batch_create_license_denied_skips_compression
==================== 3 failed, 1 passed, 1 warning in 6.77s ====================
```

With the fix, new tests plus neighbouring files
(`tests/test_proxy/test_anthropic_batch_bypass.py
tests/test_proxy/test_anthropic_buffered_timeout.py
tests/test_proxy_anthropic_cache_stability.py
tests/test_proxy_handlers_batch.py`):

```text
======================== 92 passed, 1 warning in 13.36s ========================
```

```text
$ ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy/test_anthropic_batch_bypass.py
All checks passed!
$ ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy/test_anthropic_batch_bypass.py
2 files already formatted
```

## Real Behavior Proof

- Environment: WSL2 Ubuntu on Windows 11, Python 3.12.15, branch based
on upstream/main `e995eb9f`; `headroom/_core.abi3.so` taken from the
official headroom-ai 0.41.0 wheel (untracked, not committed). Real proxy
on 127.0.0.1:18860, local stub Anthropic upstream on 127.0.0.1:18861
that logs the byte count and sha256 of each request body.
- Exact command / steps:
`ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18861 python -P -m
headroom.proxy.server --host 127.0.0.1 --port 18860 --no-cache
--no-rate-limit`, then a stdlib client POSTs the same 47340-byte
`/v1/messages/batches` body twice (non-canonical whitespace, one large
tool_result): once without headers, once with `x-headroom-bypass: true`.
Run on the code without the fix, then with it.
- Observed result: without the fix, the bypass request reaches the stub
as 47284 bytes, sha256 `d2e7efe52f09facb` (re-serialized; the sent body
was `09cbbe29568d899d`). With the fix, the bypass request reaches the
stub as 47340 bytes, sha256 `09cbbe29568d899d`, identical to what the
client sent; the request without the header is unchanged by the fix
(47284 bytes, `d2e7efe52f09facb`).

  Without the fix:
  ```text
  STUB POST /v1/messages/batches bytes=47284 sha256=d2e7efe52f09facb
  client sent bytes=47340 sha256=09cbbe29568d899d headers={}
  STUB POST /v1/messages/batches bytes=47284 sha256=d2e7efe52f09facb
client sent bytes=47340 sha256=09cbbe29568d899d
headers={'x-headroom-bypass': 'true'}
  ```
  With the fix:
  ```text
  STUB POST /v1/messages/batches bytes=47284 sha256=d2e7efe52f09facb
  client sent bytes=47340 sha256=09cbbe29568d899d headers={}
  STUB POST /v1/messages/batches bytes=47340 sha256=09cbbe29568d899d
client sent bytes=47340 sha256=09cbbe29568d899d
headers={'x-headroom-bypass': 'true'}
  ```
In this run the compression pipeline did not shrink the body (the
56-byte difference is canonical re-serialization), so the real-process
check demonstrates the byte-faithful forward, not a compression change.
- Not tested: the license-denied branch in a real process (covered only
by the unit test with a stub reporter); a real Anthropic API; batch
results processing (`handle_anthropic_batch_results`) after a bypassed
create; compressed (gzip/zstd) request bodies; `mypy headroom` was not
run.

Scripts and raw output:
`/var/tmp/cb-pub/proof/fix-anthropic-batch-bypass/` (WSL).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no for requests without the bypass
header and with a license that permits compression; with the header or a
denied license, batch items are no longer compressed
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none expected
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- On bypass the CCR batch context store may still run if
`ccr_inject_tool` is on; it is harmless because no markers are injected.
Left unchanged to keep the diff small.
- Documentation: none changed; the bypass contract is already documented
for the other handlers.

Produced with AI assistance (Claude Code) operated by me (HetCreep); the
reproduction was run on my machine on a clone of the repository.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4411"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
HetCreep
2026-10-24 08:38:24 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent 9f5cfcfd2b
commit 4fd9cddfd2
2 changed files with 130 additions and 5 deletions
+28 -5
View File
@@ -5771,7 +5771,11 @@ class AnthropicHandlerMixin:
return capability_error
from headroom.ccr import CCRToolInjector
from headroom.proxy.helpers import MAX_REQUEST_BODY_SIZE, _read_request_json
from headroom.proxy.helpers import (
MAX_REQUEST_BODY_SIZE,
_headroom_bypass_enabled,
read_request_json_with_bytes,
)
from headroom.proxy.modes import is_cache_mode
from headroom.utils import extract_user_query
@@ -5794,7 +5798,7 @@ class AnthropicHandlerMixin:
# Parse request
try:
body = await _read_request_json(request)
body, original_body_bytes = await read_request_json_with_bytes(request)
except (json.JSONDecodeError, ValueError) as e:
return JSONResponse(
status_code=400,
@@ -5848,6 +5852,20 @@ class AnthropicHandlerMixin:
request_id=request_id,
)
# Same two gates the OpenAI and Google batch paths apply: the client's
# bypass/passthrough header, and the commercial gate (fails open when no
# reporter is configured or no license info exists yet).
_bypass = _headroom_bypass_enabled(request.headers)
license_ok = (
self.usage_reporter.should_compress if self.usage_reporter is not None else True
)
if _bypass:
logger.info(f"[{request_id}] Compression skipped: reason=bypass_header")
tags["passthrough_reason"] = "bypass_header"
elif not license_ok:
logger.info(f"[{request_id}] Compression skipped: reason=license_denied")
tags["passthrough_reason"] = "license_denied"
# Track compression stats across all batch requests
total_original_tokens = 0
total_optimized_tokens = 0
@@ -5875,6 +5893,10 @@ class AnthropicHandlerMixin:
model, original_messages, params.get("system")
)
if _bypass or not license_ok:
compressed_requests.append(batch_req)
continue
if not messages or not self.config.optimize:
# No messages or optimization disabled - pass through unchanged
compressed_requests.append(
@@ -6024,14 +6046,15 @@ class AnthropicHandlerMixin:
url = f"{self.ANTHROPIC_API_URL}/v1/messages/batches"
try:
# Body is always mutated for batch (compressed requests).
# Mutated unless bypassed: bypass forwards the client's exact bytes.
response = await self._retry_request(
"POST",
url,
headers,
body,
body_mutated=True,
mutation_reasons=["batch_compression"],
original_body_bytes=original_body_bytes if _bypass else None,
body_mutated=not _bypass,
mutation_reasons=[] if _bypass else ["batch_compression"],
request_id=request_id,
forwarder_name="anthropic_batch",
path_for_log="/v1/messages/batches",
+102
View File
@@ -0,0 +1,102 @@
from __future__ import annotations
import json
from types import SimpleNamespace
import pytest
fastapi = pytest.importorskip("fastapi")
httpx = pytest.importorskip("httpx")
from fastapi.testclient import TestClient # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
# Whitespace the canonical serializer would normalise away.
RAW_BODY = (
b'{ "requests": [ { "custom_id": "req-1", "params": { "model": "claude-sonnet-4-6",'
b' "max_tokens": 64, "messages": [ { "role": "user", "content": "hello" } ] } } ] }'
)
def _post(*, headers: dict[str, str], license_ok: bool = True):
config = ProxyConfig(
optimize=True,
mode="token",
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
)
captured: dict[str, object] = {"pipeline_calls": 0}
with TestClient(create_app(config)) as client:
proxy = client.app.state.proxy
real_reporter = proxy.usage_reporter
if not license_ok:
proxy.usage_reporter = SimpleNamespace(should_compress=False)
def _apply(**kwargs): # noqa: ANN003
captured["pipeline_calls"] += 1
return SimpleNamespace(
messages=kwargs["messages"],
transforms_applied=[],
timing={},
tokens_before=100,
tokens_after=100,
waste_signals=None,
)
proxy.anthropic_pipeline.apply = _apply
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["kwargs"] = kwargs
captured["body"] = body
return httpx.Response(200, json={"id": "msgbatch_1", "type": "message_batch"})
proxy._retry_request = _fake_retry
response = client.post(
"/v1/messages/batches",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"content-type": "application/json",
**headers,
},
content=RAW_BODY,
)
proxy.usage_reporter = real_reporter # shutdown calls the real reporter's stop()
assert response.status_code == 200, response.text
return captured
@pytest.mark.parametrize(
"bypass_headers",
[{"x-headroom-bypass": "true"}, {"x-headroom-mode": "passthrough"}],
)
def test_batch_create_bypass_skips_compression_and_forwards_original_bytes(bypass_headers):
captured = _post(headers=bypass_headers)
assert captured["pipeline_calls"] == 0
assert captured["kwargs"]["body_mutated"] is False
assert captured["kwargs"]["original_body_bytes"] == RAW_BODY
assert captured["body"]["requests"][0]["params"]["messages"] == [
{"role": "user", "content": "hello"}
]
def test_batch_create_license_denied_skips_compression():
captured = _post(headers={}, license_ok=False)
assert captured["pipeline_calls"] == 0
assert captured["body"]["requests"][0]["params"]["messages"] == [
{"role": "user", "content": "hello"}
]
def test_batch_create_without_bypass_still_compresses():
captured = _post(headers={})
assert captured["pipeline_calls"] == 1
assert captured["kwargs"]["body_mutated"] is True
assert json.loads(RAW_BODY)["requests"][0]["custom_id"] == "req-1"