Files
headroom/tests/test_integrations/test_compress_backend.py
58caafb59b refactor(integrations): share the local/cloud compress backend of asgi and litellm_callback (#4124)
## Description

`CompressionMiddleware` (`headroom/integrations/asgi.py`) and
`HeadroomCallback` (`headroom/integrations/litellm_callback.py`) carried
identical copies of `_local_compress` and `_cloud_compress`; `diff` of
the two method blocks reported no difference. The cloud URL constant,
the `HEADROOM_API_KEY` / `HEADROOM_API_URL` reads and `cloud_mode` were
copied too.

This PR moves them into a `CompressBackend` mixin in
`headroom/integrations/_compress_backend.py`. That module imports
neither starlette nor litellm, so each integration still works without
the other's dependency.

Refs #4136

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring

## Changes Made

- New `headroom/integrations/_compress_backend.py` with
`CompressBackend`, which provides:
  - `_init_compress_backend(model_limit, hooks, api_key, api_url)`;
  - `cloud_mode`;
  - `_local_compress`;
  - `_cloud_compress`, which keeps the `guard_egress` offline check.
- `CompressionMiddleware(CompressBackend)` and
`HeadroomCallback(CompressBackend, CustomLogger)` call
`_init_compress_backend` and drop their copies.
- The method names stay, because
`tests/test_offline_egress_chokepoint.py` calls `_cloud_compress`
directly.
- Each class sets `_log` to its own module logger, so the "Headroom
Cloud API error" warning keeps its logger name
(`headroom.integrations.asgi` /
`headroom.integrations.litellm_callback`).
- One log change: that WARNING used to include the first 200 characters
of the cloud response body, which can echo request content. It now reads
`Headroom Cloud API error: HTTP <status> from <api_url>; request sent
uncompressed`, with the URL passed through `redact_url` from
`headroom/log_safety.py` (#4138): scheme, host and port only, the path
shown as `/<path>` and any query as `?<redacted>`. The body is not
logged at any level: a separate DEBUG line gives only the bare media
type (`type/subtype`, header parameters dropped) and the body length.
- `_DEFAULT_CLOUD_URL` and `guard_egress` stay importable from both
modules.
- Left alone on purpose:
- each class's `aclose`: they differ slightly in when `_client` is
cleared;
- `min_tokens`: it is stored and never read in either class, as on main.
The maintainer should decide separately whether to wire it up.
- No `CustomLogger` attribute clashes with the mixin's names. I checked
`dir(CustomLogger)` and the instance `vars()`.
- Tests:
- New `tests/test_integrations/test_compress_backend.py`, parametrized
over `CompressionMiddleware` and `HeadroomCallback` with an
`httpx.MockTransport` client: a 200 returns the service result (URL,
`X-Headroom-Key` header and JSON body checked), and a 503 whose body
echoes the prompt returns `None`.
`test_cloud_error_logs_no_echoed_content_at_any_level` runs both
integrations with the logger at DEBUG and at WARNING, formats every
record with `logging.Formatter().format`, and asserts that prompt,
API-key, URL-query and URL-path canaries (plain and `%`-encoded) are
absent, including a prompt canary echoed in the `Content-Type` header.
It fails on 95c608ee; the path cases fail before #4138's b566a758, and
the header case fails on 4fa6bb34.
`test_direct_load_does_not_import_the_integrations_package` now also
checks that `headroom._core` is not loaded, and the fixture comment
names what the loaded files import.
- `tests/test_integrations/test_litellm_callback.py`: the direct-file
loader registers a file-loaded `_compress_backend` before loading
`litellm_callback.py`, so the test again avoids importing
`headroom.integrations` and LangChain. New
`test_direct_load_does_not_import_the_integrations_package` checks that
in a fresh interpreter.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_compress_api.py tests/test_litellm_callback.py tests/test_integrations/test_litellm_callback.py
origin/main: 37 passed     this branch: 37 passed

$ LITELLM_LOCAL_MODEL_COST_MAP=True pytest tests/test_offline_egress_chokepoint.py -k "asgi or litellm"
origin/main: 2 passed      this branch: 2 passed

$ pytest tests/test_offline_egress_chokepoint.py
origin/main: 2 failed, 141 passed     this branch: 2 failed, 141 passed
(pre-existing and environmental: the same two asgi/litellm tests fail on main, because importing litellm
 fetches its model cost map from raw.githubusercontent.com and the test's no-socket fixture refuses it;
 with LITELLM_LOCAL_MODEL_COST_MAP=True both pass on main and on this branch)

$ pytest tests/test_integrations/test_compress_backend.py tests/test_integrations/test_litellm_callback.py tests/test_compress_api.py tests/test_litellm_callback.py tests/test_offline_egress_chokepoint.py
this branch: 185 passed
# test_compress_backend.py against the previous commit's backend (04d38103): 2 failed (warning carried the body), 2 passed

$ uvx ruff@0.16.8 check / format <touched files>  -> All checks passed! / unchanged
$ mypy headroom/integrations/_compress_backend.py asgi.py litellm_callback.py --ignore-missing-imports
headroom/integrations/langchain/chat_model.py:394: error: ... [truthy-function]   (pre-existing, followed import, untouched file)
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, repo `.venv` (litellm, starlette and
httpx installed), this branch's worktree. Temp `HOME`,
`HEADROOM_WORKSPACE_DIR` and `CODEX_HOME`.
- Exact command / steps: two runs, listed below.
1. Shared smoke harness: a real proxy from this checkout against a local
fake upstream (`smoke.sh <worktree>`).
2. The shared cloud path over real HTTP, with both module loggers at
DEBUG: a local HTTP server answers 503 (echoing the request, which
carries a prompt canary) and then 200, for `HeadroomCallback` and then
`CompressionMiddleware` (`_cloud_compress`). The API key, a `/bot…` path
segment and a `?token=` query on the URL are canaries too.
- Observed result: the output of both runs is in the two blocks below.

```text
branch: cleanup/cli-integrations-shared-compress-backend @ fad5714c
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines (<tmp>/ws/logs/proxy-58395.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: <tmp>)
```

```text
HeadroomCallback 503 -> None | 200 -> 3
CompressionMiddleware 503 -> None | 200 -> 3
WARNING headroom.integrations.litellm_callback: Headroom Cloud API error: HTTP 503 from http://127.0.0.1:PORT/<path>?<redacted>; request sent uncompressed
DEBUG headroom.integrations.litellm_callback: Headroom Cloud API error response: content-type=text/plain, 138 bytes
WARNING headroom.integrations.asgi: Headroom Cloud API error: HTTP 503 from http://127.0.0.1:PORT/<path>?<redacted>; request sent uncompressed
DEBUG headroom.integrations.asgi: Headroom Cloud API error response: content-type=text/plain, 138 bytes
canaries in log: 0
```

- Not tested: the real Headroom Cloud service.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a, library refactor.
- Stable/default behavior changed: no. The env vars, default URL,
payload, timeout, offline refusal and log text are all the same.
- Kill switch / disable path: n/a.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.

Depends on #4138; its two files show in this diff until it merges.

- `tests/test_integrations/test_litellm_callback.py` loads
`litellm_callback.py` from its file path, to avoid importing the
`headroom.integrations` package. Because the module now imports
`headroom.integrations._compress_backend`, the loader registers a
file-loaded copy of that module first; a module already in `sys.modules`
is returned without importing its parent package. Normal imports of
`headroom.integrations.litellm_callback` load the package anyway, as
before.
- A test that patched `headroom.integrations.asgi.guard_egress` would no
longer intercept the call, because the call now lives in
`_compress_backend`. No such test exists.
- Docs impact (the changed warning text) is recorded in the docs ledger;
there are no doc edits in this PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-16 14:30:57 -07:00

178 lines
7.0 KiB
Python

"""Cloud path of the compression backend shared by the ASGI and LiteLLM integrations."""
from __future__ import annotations
import asyncio
import json
import logging
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
import httpx
import pytest
from headroom.integrations.asgi import CompressionMiddleware
from headroom.integrations.litellm_callback import HeadroomCallback
_MESSAGES = [{"role": "user", "content": "secret prompt text"}]
@pytest.fixture(params=["asgi", "litellm"])
def backend(request, monkeypatch: pytest.MonkeyPatch):
monkeypatch.delenv("HEADROOM_OFFLINE", raising=False)
if request.param == "asgi":
return CompressionMiddleware(app=None, api_key="hdr_test", api_url="https://cloud.test/")
return HeadroomCallback(api_key="hdr_test", api_url="https://cloud.test/")
def _serve(backend, handler) -> list[httpx.Request]:
seen: list[httpx.Request] = []
def record(request: httpx.Request) -> httpx.Response:
seen.append(request)
return handler(request)
backend._client = httpx.AsyncClient(transport=httpx.MockTransport(record))
return seen
def test_cloud_success_returns_the_service_result(backend) -> None:
payload = {"messages": [{"role": "user", "content": "short"}], "tokens_saved": 3}
seen = _serve(backend, lambda request: httpx.Response(200, json=payload))
result = asyncio.run(backend._cloud_compress(_MESSAGES, "gpt-4o"))
assert result == payload
(request,) = seen
assert str(request.url) == "https://cloud.test/v1/saas/compress"
assert request.headers["X-Headroom-Key"] == "hdr_test"
assert json.loads(request.content) == {
"messages": _MESSAGES,
"model": "gpt-4o",
"model_limit": backend._model_limit,
}
class _Records(logging.Handler):
def __init__(self) -> None:
super().__init__(logging.DEBUG)
self.records: list[logging.LogRecord] = []
def emit(self, record: logging.LogRecord) -> None:
self.records.append(record)
_PROMPT_CANARY = "PROMPT-CANARY-7f3e"
_KEY_CANARY = "hdr_KEY-CANARY-91ab"
_URL_CANARY = "URL-CANARY-c0de"
_PATH_CANARY = "PRIVATE_PATH_CANARY"
# (api_url, how the WARNING shows it); every canary must stay out of every record.
_API_URLS = {
"query": (f"https://cloud.test/?token={_URL_CANARY}", "https://cloud.test/?<redacted>"),
"path": (f"https://cloud.test/bot{_PATH_CANARY}", "https://cloud.test/<path>"),
# %50 is "P": the encoded form must not slip through either.
"encoded-path": ("https://cloud.test/bot%50RIVATE_PATH_CANARY", "https://cloud.test/<path>"),
}
@pytest.mark.parametrize("kind", ["asgi", "litellm"])
def test_real_cloud_error_media_type_cannot_echo_credentials(kind, monkeypatch) -> None:
canary = "PRIVATE_CREDENTIAL_CANARY"
body = b"service error"
received = []
class ErrorServer(BaseHTTPRequestHandler):
def do_POST(self):
received.append(json.loads(self.rfile.read(int(self.headers["Content-Length"]))))
self.send_response(503)
self.send_header("Content-Type", f"text/{canary}")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
monkeypatch.delenv("HEADROOM_OFFLINE", raising=False)
monkeypatch.delenv("HEADROOM_DEBUG_DUMP", raising=False)
with HTTPServer(("127.0.0.1", 0), ErrorServer) as server:
worker = threading.Thread(target=server.serve_forever, daemon=True)
worker.start()
api_url = f"http://127.0.0.1:{server.server_port}"
integration = CompressionMiddleware if kind == "asgi" else HeadroomCallback
kwargs = {"app": None} if kind == "asgi" else {}
instance = integration(api_key=_KEY_CANARY, api_url=api_url, **kwargs)
handler = _Records()
monkeypatch.setattr(instance._log, "level", logging.DEBUG)
monkeypatch.setattr(instance._log, "disabled", False)
instance._log.addHandler(handler)
async def request():
try:
return await instance._cloud_compress(_MESSAGES, "gpt-4o")
finally:
if instance._client is not None:
await instance._client.aclose()
try:
assert asyncio.run(request()) is None
finally:
instance._log.removeHandler(handler)
server.shutdown()
worker.join(timeout=5)
assert received[0]["messages"] == _MESSAGES
records = [logging.Formatter().format(record) for record in handler.records]
assert any("HTTP 503" in record for record in records)
assert any(f"{len(body)} bytes" in record for record in records)
assert all(canary.lower() not in record.lower() for record in records)
@pytest.mark.parametrize("kind", ["asgi", "litellm"])
@pytest.mark.parametrize("level", [logging.DEBUG, logging.WARNING])
@pytest.mark.parametrize("url_case", sorted(_API_URLS))
def test_cloud_error_logs_no_echoed_content_at_any_level(
kind: str, level: int, url_case: str, monkeypatch: pytest.MonkeyPatch
) -> None:
"""An error response echoing the prompt must not put it, the key, or URL
credentials (query or path) in any formatted record, with DEBUG on or off."""
monkeypatch.delenv("HEADROOM_OFFLINE", raising=False)
monkeypatch.delenv("HEADROOM_DEBUG_DUMP", raising=False)
api_url, shown_url = _API_URLS[url_case]
if kind == "asgi":
backend = CompressionMiddleware(app=None, api_key=_KEY_CANARY, api_url=api_url)
else:
backend = HeadroomCallback(api_key=_KEY_CANARY, api_url=api_url)
messages = [{"role": "user", "content": f"my secret is {_PROMPT_CANARY}"}]
echo = f"bad request: {json.dumps(messages)} key={_KEY_CANARY}"
_serve(
backend,
lambda request: httpx.Response(
503,
text=echo,
# A header can echo content too: only the bare media type may be logged.
headers={"content-type": f"text/plain; note={_PROMPT_CANARY}"},
),
)
# Attach to the module logger itself: proxy startup elsewhere in the suite can
# stop headroom.* records from reaching pytest's root-level caplog handler.
handler = _Records()
monkeypatch.setattr(backend._log, "level", level)
monkeypatch.setattr(backend._log, "disabled", False)
backend._log.addHandler(handler)
try:
assert asyncio.run(backend._cloud_compress(messages, "gpt-4o")) is None
finally:
backend._log.removeHandler(handler)
formatted = [logging.Formatter().format(r) for r in handler.records]
warnings = [r.getMessage() for r in handler.records if r.levelno == logging.WARNING]
assert warnings == [
f"Headroom Cloud API error: HTTP 503 from {shown_url}; request sent uncompressed"
]
if level == logging.DEBUG:
assert any("content-type=text/plain," in line for line in formatted)
for line in formatted:
for canary in (_PROMPT_CANARY, _KEY_CANARY, _URL_CANARY, _PATH_CANARY, "%50RIVATE"):
assert canary not in line