mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): honor bypass header and license gate in Anthropic batch create (#4411)
## Description `handle_anthropic_batch_create` ignored `x-headroom-bypass: true`, `x-headroom-mode: passthrough` and the `usage_reporter.should_compress` license gate, and always re-serialized the body. The OpenAI and Google batch paths enforce all of these. This PR makes the Anthropic batch path do the same: compression is skipped for each item in either case, and on bypass the client's original bytes are forwarded. Fixes #4404 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/anthropic.py`: read the body with `read_request_json_with_bytes`; compute `_bypass` (`_headroom_bypass_enabled`) and `license_ok` (same expression as `batch.py`, fails open without a reporter); tag `passthrough_reason`; skip the per-item compression loop body when either gate applies; on bypass call `_retry_request` with `original_body_bytes` and `body_mutated=False`. - `tests/test_proxy/test_anthropic_batch_bypass.py`: 4 tests (bypass header, `x-headroom-mode: passthrough`, license denied, unchanged compression path). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output With only the source change reverted (new test file kept): ```text FAILED tests/test_proxy/test_anthropic_batch_bypass.py::test_batch_create_bypass_skips_compression_and_forwards_original_bytes[bypass_headers0] FAILED tests/test_proxy/test_anthropic_batch_bypass.py::test_batch_create_bypass_skips_compression_and_forwards_original_bytes[bypass_headers1] FAILED tests/test_proxy/test_anthropic_batch_bypass.py::test_batch_create_license_denied_skips_compression ==================== 3 failed, 1 passed, 1 warning in 6.77s ==================== ``` With the fix, new tests plus neighbouring files (`tests/test_proxy/test_anthropic_batch_bypass.py tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_handlers_batch.py`): ```text ======================== 92 passed, 1 warning in 13.36s ======================== ``` ```text $ ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy/test_anthropic_batch_bypass.py All checks passed! $ ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy/test_anthropic_batch_bypass.py 2 files already formatted ``` ## Real Behavior Proof - Environment: WSL2 Ubuntu on Windows 11, Python 3.12.15, branch based on upstream/main `e995eb9f`; `headroom/_core.abi3.so` taken from the official headroom-ai 0.41.0 wheel (untracked, not committed). Real proxy on 127.0.0.1:18860, local stub Anthropic upstream on 127.0.0.1:18861 that logs the byte count and sha256 of each request body. - Exact command / steps: `ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18861 python -P -m headroom.proxy.server --host 127.0.0.1 --port 18860 --no-cache --no-rate-limit`, then a stdlib client POSTs the same 47340-byte `/v1/messages/batches` body twice (non-canonical whitespace, one large tool_result): once without headers, once with `x-headroom-bypass: true`. Run on the code without the fix, then with it. - Observed result: without the fix, the bypass request reaches the stub as 47284 bytes, sha256 `d2e7efe52f09facb` (re-serialized; the sent body was `09cbbe29568d899d`). With the fix, the bypass request reaches the stub as 47340 bytes, sha256 `09cbbe29568d899d`, identical to what the client sent; the request without the header is unchanged by the fix (47284 bytes, `d2e7efe52f09facb`). Without the fix: ```text STUB POST /v1/messages/batches bytes=47284 sha256=d2e7efe52f09facb client sent bytes=47340 sha256=09cbbe29568d899d headers={} STUB POST /v1/messages/batches bytes=47284 sha256=d2e7efe52f09facb client sent bytes=47340 sha256=09cbbe29568d899d headers={'x-headroom-bypass': 'true'} ``` With the fix: ```text STUB POST /v1/messages/batches bytes=47284 sha256=d2e7efe52f09facb client sent bytes=47340 sha256=09cbbe29568d899d headers={} STUB POST /v1/messages/batches bytes=47340 sha256=09cbbe29568d899d client sent bytes=47340 sha256=09cbbe29568d899d headers={'x-headroom-bypass': 'true'} ``` In this run the compression pipeline did not shrink the body (the 56-byte difference is canonical re-serialization), so the real-process check demonstrates the byte-faithful forward, not a compression change. - Not tested: the license-denied branch in a real process (covered only by the unit test with a stub reporter); a real Anthropic API; batch results processing (`handle_anthropic_batch_results`) after a bypassed create; compressed (gzip/zstd) request bodies; `mypy headroom` was not run. Scripts and raw output: `/var/tmp/cb-pub/proof/fix-anthropic-batch-bypass/` (WSL). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: no for requests without the bypass header and with a license that permits compression; with the header or a denied license, batch items are no longer compressed - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none expected - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - On bypass the CCR batch context store may still run if `ccr_inject_tool` is on; it is harmless because no markers are injected. Left unchanged to keep the diff small. - Documentation: none changed; the bypass contract is already documented for the other handlers. Produced with AI assistance (Claude Code) operated by me (HetCreep); the reproduction was run on my machine on a clone of the repository. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4411" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
9f5cfcfd2b
commit
4fd9cddfd2
@@ -5771,7 +5771,11 @@ class AnthropicHandlerMixin:
|
||||
return capability_error
|
||||
|
||||
from headroom.ccr import CCRToolInjector
|
||||
from headroom.proxy.helpers import MAX_REQUEST_BODY_SIZE, _read_request_json
|
||||
from headroom.proxy.helpers import (
|
||||
MAX_REQUEST_BODY_SIZE,
|
||||
_headroom_bypass_enabled,
|
||||
read_request_json_with_bytes,
|
||||
)
|
||||
from headroom.proxy.modes import is_cache_mode
|
||||
from headroom.utils import extract_user_query
|
||||
|
||||
@@ -5794,7 +5798,7 @@ class AnthropicHandlerMixin:
|
||||
|
||||
# Parse request
|
||||
try:
|
||||
body = await _read_request_json(request)
|
||||
body, original_body_bytes = await read_request_json_with_bytes(request)
|
||||
except (json.JSONDecodeError, ValueError) as e:
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
@@ -5848,6 +5852,20 @@ class AnthropicHandlerMixin:
|
||||
request_id=request_id,
|
||||
)
|
||||
|
||||
# Same two gates the OpenAI and Google batch paths apply: the client's
|
||||
# bypass/passthrough header, and the commercial gate (fails open when no
|
||||
# reporter is configured or no license info exists yet).
|
||||
_bypass = _headroom_bypass_enabled(request.headers)
|
||||
license_ok = (
|
||||
self.usage_reporter.should_compress if self.usage_reporter is not None else True
|
||||
)
|
||||
if _bypass:
|
||||
logger.info(f"[{request_id}] Compression skipped: reason=bypass_header")
|
||||
tags["passthrough_reason"] = "bypass_header"
|
||||
elif not license_ok:
|
||||
logger.info(f"[{request_id}] Compression skipped: reason=license_denied")
|
||||
tags["passthrough_reason"] = "license_denied"
|
||||
|
||||
# Track compression stats across all batch requests
|
||||
total_original_tokens = 0
|
||||
total_optimized_tokens = 0
|
||||
@@ -5875,6 +5893,10 @@ class AnthropicHandlerMixin:
|
||||
model, original_messages, params.get("system")
|
||||
)
|
||||
|
||||
if _bypass or not license_ok:
|
||||
compressed_requests.append(batch_req)
|
||||
continue
|
||||
|
||||
if not messages or not self.config.optimize:
|
||||
# No messages or optimization disabled - pass through unchanged
|
||||
compressed_requests.append(
|
||||
@@ -6024,14 +6046,15 @@ class AnthropicHandlerMixin:
|
||||
url = f"{self.ANTHROPIC_API_URL}/v1/messages/batches"
|
||||
|
||||
try:
|
||||
# Body is always mutated for batch (compressed requests).
|
||||
# Mutated unless bypassed: bypass forwards the client's exact bytes.
|
||||
response = await self._retry_request(
|
||||
"POST",
|
||||
url,
|
||||
headers,
|
||||
body,
|
||||
body_mutated=True,
|
||||
mutation_reasons=["batch_compression"],
|
||||
original_body_bytes=original_body_bytes if _bypass else None,
|
||||
body_mutated=not _bypass,
|
||||
mutation_reasons=[] if _bypass else ["batch_compression"],
|
||||
request_id=request_id,
|
||||
forwarder_name="anthropic_batch",
|
||||
path_for_log="/v1/messages/batches",
|
||||
|
||||
+102
@@ -0,0 +1,102 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
fastapi = pytest.importorskip("fastapi")
|
||||
httpx = pytest.importorskip("httpx")
|
||||
|
||||
from fastapi.testclient import TestClient # noqa: E402
|
||||
|
||||
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
|
||||
|
||||
# Whitespace the canonical serializer would normalise away.
|
||||
RAW_BODY = (
|
||||
b'{ "requests": [ { "custom_id": "req-1", "params": { "model": "claude-sonnet-4-6",'
|
||||
b' "max_tokens": 64, "messages": [ { "role": "user", "content": "hello" } ] } } ] }'
|
||||
)
|
||||
|
||||
|
||||
def _post(*, headers: dict[str, str], license_ok: bool = True):
|
||||
config = ProxyConfig(
|
||||
optimize=True,
|
||||
mode="token",
|
||||
cache_enabled=False,
|
||||
rate_limit_enabled=False,
|
||||
cost_tracking_enabled=False,
|
||||
log_requests=False,
|
||||
ccr_inject_tool=False,
|
||||
ccr_handle_responses=False,
|
||||
ccr_context_tracking=False,
|
||||
image_optimize=False,
|
||||
)
|
||||
captured: dict[str, object] = {"pipeline_calls": 0}
|
||||
with TestClient(create_app(config)) as client:
|
||||
proxy = client.app.state.proxy
|
||||
real_reporter = proxy.usage_reporter
|
||||
if not license_ok:
|
||||
proxy.usage_reporter = SimpleNamespace(should_compress=False)
|
||||
|
||||
def _apply(**kwargs): # noqa: ANN003
|
||||
captured["pipeline_calls"] += 1
|
||||
return SimpleNamespace(
|
||||
messages=kwargs["messages"],
|
||||
transforms_applied=[],
|
||||
timing={},
|
||||
tokens_before=100,
|
||||
tokens_after=100,
|
||||
waste_signals=None,
|
||||
)
|
||||
|
||||
proxy.anthropic_pipeline.apply = _apply
|
||||
|
||||
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
|
||||
captured["kwargs"] = kwargs
|
||||
captured["body"] = body
|
||||
return httpx.Response(200, json={"id": "msgbatch_1", "type": "message_batch"})
|
||||
|
||||
proxy._retry_request = _fake_retry
|
||||
response = client.post(
|
||||
"/v1/messages/batches",
|
||||
headers={
|
||||
"x-api-key": "test-key",
|
||||
"anthropic-version": "2023-06-01",
|
||||
"content-type": "application/json",
|
||||
**headers,
|
||||
},
|
||||
content=RAW_BODY,
|
||||
)
|
||||
proxy.usage_reporter = real_reporter # shutdown calls the real reporter's stop()
|
||||
assert response.status_code == 200, response.text
|
||||
return captured
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bypass_headers",
|
||||
[{"x-headroom-bypass": "true"}, {"x-headroom-mode": "passthrough"}],
|
||||
)
|
||||
def test_batch_create_bypass_skips_compression_and_forwards_original_bytes(bypass_headers):
|
||||
captured = _post(headers=bypass_headers)
|
||||
assert captured["pipeline_calls"] == 0
|
||||
assert captured["kwargs"]["body_mutated"] is False
|
||||
assert captured["kwargs"]["original_body_bytes"] == RAW_BODY
|
||||
assert captured["body"]["requests"][0]["params"]["messages"] == [
|
||||
{"role": "user", "content": "hello"}
|
||||
]
|
||||
|
||||
|
||||
def test_batch_create_license_denied_skips_compression():
|
||||
captured = _post(headers={}, license_ok=False)
|
||||
assert captured["pipeline_calls"] == 0
|
||||
assert captured["body"]["requests"][0]["params"]["messages"] == [
|
||||
{"role": "user", "content": "hello"}
|
||||
]
|
||||
|
||||
|
||||
def test_batch_create_without_bypass_still_compresses():
|
||||
captured = _post(headers={})
|
||||
assert captured["pipeline_calls"] == 1
|
||||
assert captured["kwargs"]["body_mutated"] is True
|
||||
assert json.loads(RAW_BODY)["requests"][0]["custom_id"] == "req-1"
|
||||
Reference in New Issue
Block a user