Files
headroom/tests/test_proxy_copilot_auth_hooks.py
Mansurshakh JaparovandJerrettDavis 50bf775e8c fix(proxy): bypass compression for large admitted histories (#4142)
## Description

Closes #4139.

A history that is too large to compress should still reach the provider.
This keeps the 100 MiB hard admission limit and bypasses request-side
transforms above 16 MiB decoded size. The hard limit still applies
before forwarding.

Current head: `62ffe3defef1684f70d96f1640818d99445a43bb`, merging
current main `a2d4fe72` into `1902d8ef`. The
42-test/native/manual/pre-push results below qualify `1902d8ef`, not
this new merge. New-head shard 4 has a cached-prefix fixture failure
under investigation; coverage uploads are still incomplete.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
- [x] Documentation update

## Changes Made

- Keep the startup override and both wire/decompressed hard bounds.
Large admitted histories skip normalization, image processing, memory
injection, tokenization and compression on Messages, Chat/Responses,
Gemini and Bedrock HTTP routes.
- Keep provider routing and backend translation, including Copilot Chat,
Codex `store:false` and Vertex route-forced streaming. Native forwarding
retains decoded bytes unless the provider's wire contract requires a
rewrite; translation retains the full message text.
- Report actual provider usage, with zero compression savings or
attempted tokens for the bypass. Keep `/v1/compress` and WebSocket
contracts unchanged.
- Add real-provider coverage for large Gemini JSON/SSE requests with
query-key auth, and OpenAI JSON/SSE through the installed LiteLLM
adapter. No admission-policy or CI changes in this follow-up.

## Testing

- [x] Unit tests pass (focused suite and normal critical pre-push suite)
- [x] Linting passes (normal hooks)
- [x] New tests added for admission, recovery and passthrough accounting
- [x] Manual testing performed against the real native build

### Test Output

```sh
python -m pytest -q tests/test_proxy_compression_admission.py tests/test_proxy/test_gemini_plugin_routed_streaming.py --cov=headroom --cov-branch
# 42 passed, 41 warnings
```

Normal pre-push `make ci-precheck` passed: 1668 Rust tests (3 ignored),
181 Python tests, lint/type checks and commitlint. No hook bypass. The
initial deep-module coverage command hit a Pydantic import-isolation
error; using the repo's `--cov=headroom` convention passed both complete
modules without disabling plugins or tests.

The earlier `96c5fe7e` qualification covered the other native protocols,
configured/per-request backends, hard-ceiling recovery and complete
local shards 2/3. Those runs are historical, not a new whole-suite
claim. New-head hosted success is not claimed.

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.13.14, freshly built real
`headroom._core`, public Headroom ASGI routes, a real localhost HTTP
provider and installed LiteLLM. Synthetic credentials, offline mode and
`DO_NOT_TRACK=1`; no live provider calls.
- Exact command / steps: Start
`create_app(ProxyConfig(backend="litellm-openai", ...))` with all
upstream URLs set to the localhost peer and
`litellm.api_base=peer+"/v1"`. Leave the production 16 MiB soft/100 MiB
hard limits unchanged. POST `"history " * (16777216 // 8 + 1024) + "LAST
TURN"` as a user message to Chat Completions, then as a Gemini content
part to generateContent/streamGenerateContent, with streaming off/on.
The peer rejects a changed history, requires Gemini query-key auth and
one `alt=sse`, and returns protocol-valid JSON/SSE with 101 input/1
output tokens. The maintained four small-ceiling cases are in
`tests/test_proxy_compression_admission.py` and run with the command
above.
- Observed result: All four >16 MiB requests returned 200 with the full
history intact. JSON/SSE and Gemini auth survived; streamGenerateContent
forced SSE even with incoming `alt=json`. Each request recorded exactly
101 provider input/1 output tokens, zero compression attempts/savings
and no failures; `/livez` stayed 200. Earlier recorded qualification
also covered all five native HTTP protocols, real translating backends
and 413 recovery.
- Not tested: Live cloud accounts, the reporter's deployed 4 GB VPS,
screenshot/image quality or a zero-JSON-parse path. One envelope/model
JSON parse remains; no proxy-only RSS guarantee is claimed.

## Runtime Rollout Safety

- Rollout-managed feature(s): None; this is the admission/compatibility
fix.
- Minimum rollout channel: Stable/default.
- Stable/default behavior changed: Keep the 100 MiB hard ceiling; bypass
request-side transforms above 16 MiB decoded size on the named HTTP
routes.
- Kill switch / disable path: Set
`HEADROOM_REQUEST_BODY_MAX_BYTES=16777216` (or lower) and restart,
accepting local 413s instead of the large-body path.
- Unsafe override required: No.
- Qualification impact: Local native protocols, backend translation,
hard bounds and recovery exercised; live-provider/VPS qualification
remains external.
- Rollback path: Revert the admission revision and restart; no storage
migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] I have made corresponding changes to the documentation
- [x] I did not edit `CHANGELOG.md`

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 22:37:21 -06:00

179 lines
6.4 KiB
Python

from __future__ import annotations
import httpx
import pytest
from starlette.requests import Request
from starlette.responses import StreamingResponse
from headroom import paths
from headroom.proxy.handlers import openai as openai_mod
from headroom.proxy.server import HeadroomProxy, ProxyConfig
def test_openai_chat_routes_copilot_requests_per_model() -> None:
copilot_base = "https://api.githubcopilot.com"
gpt54_mini_url = openai_mod.build_copilot_upstream_url(
copilot_base,
openai_mod._resolve_openai_handler_path(
{},
handler_path=openai_mod._resolve_openai_chat_handler_path(copilot_base, "gpt-5.4-mini"),
),
)
claude_url = openai_mod.build_copilot_upstream_url(
copilot_base,
openai_mod._resolve_openai_handler_path(
{},
handler_path=openai_mod._resolve_openai_chat_handler_path(
copilot_base, "claude-sonnet-5"
),
),
)
openai_url = openai_mod.build_copilot_upstream_url(
"https://api.openai.com",
openai_mod._resolve_openai_handler_path(
{},
handler_path=openai_mod._resolve_openai_chat_handler_path(
"https://api.openai.com", "gpt-5.4-mini"
),
),
)
assert gpt54_mini_url == "https://api.githubcopilot.com/responses"
assert claude_url == "https://api.githubcopilot.com/chat/completions"
assert openai_url == "https://api.openai.com/v1/chat/completions"
@pytest.mark.asyncio
@pytest.mark.parametrize("entrypoint", ["passthrough", "streaming", "streaming-disconnected"])
@pytest.mark.parametrize(
("path", "expected_auth"),
[
(
"/v1/engines/gpt-41-copilot/completions",
"Bearer ghu_editor_seat_fixture",
),
("/chat/completions", "Bearer tid_operator_seat_fixture"),
],
ids=["native-editor-seat", "chat-operator-seat"],
)
async def test_real_forwarders_enforce_credential_and_secret_boundaries(
monkeypatch: pytest.MonkeyPatch, entrypoint: str, path: str, expected_auth: str
) -> None:
monkeypatch.setenv("GITHUB_COPILOT_API_TOKEN", "tid_operator_seat_fixture")
monkeypatch.setenv("GITHUB_COPILOT_PROXY_URL", "https://gateway.example.invalid/v1")
monkeypatch.delenv("GITHUB_COPILOT_REFRESH_OAUTH_TOKEN", raising=False)
monkeypatch.setattr(paths, "_PROCESS_STATELESS", paths._PROCESS_STATELESS)
for name in (
"ANTHROPIC_API_URL",
"OPENAI_API_URL",
"GEMINI_API_URL",
"CLOUDCODE_API_URL",
"VERTEX_API_URL",
):
monkeypatch.setattr(HeadroomProxy, name, getattr(HeadroomProxy, name))
proxy = HeadroomProxy(
ProxyConfig(
stateless=True,
optimize=False,
disable_kompress=True,
discover_pipeline_extensions=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
)
)
content = b'data: {"choices":[{"text":"accepted"}]}\n\ndata: [DONE]\n\n'
class UpstreamStream(httpx.AsyncByteStream):
def __init__(self) -> None:
self.closed = False
async def __aiter__(self):
yield content
async def aclose(self) -> None:
self.closed = True
upstream_stream = UpstreamStream()
def credential_gate(request: httpx.Request) -> httpx.Response:
admitted = (
request.headers.get("authorization") == expected_auth
and request.headers.get("copilot-integration-id") == "vscode"
and "x-api-key" not in request.headers
)
if admitted:
return httpx.Response(
200,
stream=upstream_stream,
headers={"content-type": "text/event-stream"},
)
return httpx.Response(
401,
content=b'{"error":"wrong credential or leaked secret"}',
headers={"content-type": "application/json"},
)
headers = {
"authorization": "Bearer ghu_editor_seat_fixture",
"copilot-integration-id": "vscode",
"x-api-key": "unrelated-provider-secret",
}
base = (
"https://proxy.business.githubcopilot.com"
if path.startswith("/v1/engines/")
else "https://api.githubcopilot.com"
)
try:
async with httpx.AsyncClient(transport=httpx.MockTransport(credential_gate)) as client:
proxy.http_client = client
if entrypoint == "passthrough":
async def receive() -> dict[str, object]:
return {"type": "http.request", "body": b"{}", "more_body": False}
request = Request(
{
"type": "http",
"method": "POST",
"scheme": "http",
"server": ("proxy.test", 80),
"path": path,
"query_string": b"",
"headers": [(k.encode(), v.encode()) for k, v in headers.items()],
},
receive,
)
response = await proxy.handle_passthrough(request, base)
else:
response = await proxy._stream_response(
url=base + path,
headers=headers,
body={"model": "gpt-41-copilot"},
provider="openai",
model="gpt-41-copilot",
request_id="inline-auth-regression",
original_tokens=0,
optimized_tokens=0,
tokens_saved=0,
transforms_applied=[],
tags={},
optimization_latency=0.0,
)
assert response.status_code == 200
if isinstance(response, StreamingResponse):
assert not upstream_stream.closed
if entrypoint == "streaming":
received = b"".join([chunk async for chunk in response.body_iterator])
assert received == content
if response.background is not None:
await response.background()
else:
assert response.body == content
assert upstream_stream.closed
finally:
proxy._compression_executor.shutdown()
proxy._background_compression_executor.shutdown()