mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description `CompressionMiddleware` (`headroom/integrations/asgi.py`) and `HeadroomCallback` (`headroom/integrations/litellm_callback.py`) carried identical copies of `_local_compress` and `_cloud_compress`; `diff` of the two method blocks reported no difference. The cloud URL constant, the `HEADROOM_API_KEY` / `HEADROOM_API_URL` reads and `cloud_mode` were copied too. This PR moves them into a `CompressBackend` mixin in `headroom/integrations/_compress_backend.py`. That module imports neither starlette nor litellm, so each integration still works without the other's dependency. Refs #4136 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring ## Changes Made - New `headroom/integrations/_compress_backend.py` with `CompressBackend`, which provides: - `_init_compress_backend(model_limit, hooks, api_key, api_url)`; - `cloud_mode`; - `_local_compress`; - `_cloud_compress`, which keeps the `guard_egress` offline check. - `CompressionMiddleware(CompressBackend)` and `HeadroomCallback(CompressBackend, CustomLogger)` call `_init_compress_backend` and drop their copies. - The method names stay, because `tests/test_offline_egress_chokepoint.py` calls `_cloud_compress` directly. - Each class sets `_log` to its own module logger, so the "Headroom Cloud API error" warning keeps its logger name (`headroom.integrations.asgi` / `headroom.integrations.litellm_callback`). - One log change: that WARNING used to include the first 200 characters of the cloud response body, which can echo request content. It now reads `Headroom Cloud API error: HTTP <status> from <api_url>; request sent uncompressed`, with the URL passed through `redact_url` from `headroom/log_safety.py` (#4138): scheme, host and port only, the path shown as `/<path>` and any query as `?<redacted>`. The body is not logged at any level: a separate DEBUG line gives only the bare media type (`type/subtype`, header parameters dropped) and the body length. - `_DEFAULT_CLOUD_URL` and `guard_egress` stay importable from both modules. - Left alone on purpose: - each class's `aclose`: they differ slightly in when `_client` is cleared; - `min_tokens`: it is stored and never read in either class, as on main. The maintainer should decide separately whether to wire it up. - No `CustomLogger` attribute clashes with the mixin's names. I checked `dir(CustomLogger)` and the instance `vars()`. - Tests: - New `tests/test_integrations/test_compress_backend.py`, parametrized over `CompressionMiddleware` and `HeadroomCallback` with an `httpx.MockTransport` client: a 200 returns the service result (URL, `X-Headroom-Key` header and JSON body checked), and a 503 whose body echoes the prompt returns `None`. `test_cloud_error_logs_no_echoed_content_at_any_level` runs both integrations with the logger at DEBUG and at WARNING, formats every record with `logging.Formatter().format`, and asserts that prompt, API-key, URL-query and URL-path canaries (plain and `%`-encoded) are absent, including a prompt canary echoed in the `Content-Type` header. It fails on 95c608ee; the path cases fail before #4138'sb566a758, and the header case fails on4fa6bb34. `test_direct_load_does_not_import_the_integrations_package` now also checks that `headroom._core` is not loaded, and the fixture comment names what the loaded files import. - `tests/test_integrations/test_litellm_callback.py`: the direct-file loader registers a file-loaded `_compress_backend` before loading `litellm_callback.py`, so the test again avoids importing `headroom.integrations` and LangChain. New `test_direct_load_does_not_import_the_integrations_package` checks that in a fresh interpreter. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_compress_api.py tests/test_litellm_callback.py tests/test_integrations/test_litellm_callback.py origin/main: 37 passed this branch: 37 passed $ LITELLM_LOCAL_MODEL_COST_MAP=True pytest tests/test_offline_egress_chokepoint.py -k "asgi or litellm" origin/main: 2 passed this branch: 2 passed $ pytest tests/test_offline_egress_chokepoint.py origin/main: 2 failed, 141 passed this branch: 2 failed, 141 passed (pre-existing and environmental: the same two asgi/litellm tests fail on main, because importing litellm fetches its model cost map from raw.githubusercontent.com and the test's no-socket fixture refuses it; with LITELLM_LOCAL_MODEL_COST_MAP=True both pass on main and on this branch) $ pytest tests/test_integrations/test_compress_backend.py tests/test_integrations/test_litellm_callback.py tests/test_compress_api.py tests/test_litellm_callback.py tests/test_offline_egress_chokepoint.py this branch: 185 passed # test_compress_backend.py against the previous commit's backend (04d38103): 2 failed (warning carried the body), 2 passed $ uvx ruff@0.16.8 check / format <touched files> -> All checks passed! / unchanged $ mypy headroom/integrations/_compress_backend.py asgi.py litellm_callback.py --ignore-missing-imports headroom/integrations/langchain/chat_model.py:394: error: ... [truthy-function] (pre-existing, followed import, untouched file) ``` ## Real Behavior Proof - Environment: macOS, Python 3.12, repo `.venv` (litellm, starlette and httpx installed), this branch's worktree. Temp `HOME`, `HEADROOM_WORKSPACE_DIR` and `CODEX_HOME`. - Exact command / steps: two runs, listed below. 1. Shared smoke harness: a real proxy from this checkout against a local fake upstream (`smoke.sh <worktree>`). 2. The shared cloud path over real HTTP, with both module loggers at DEBUG: a local HTTP server answers 503 (echoing the request, which carries a prompt canary) and then 200, for `HeadroomCallback` and then `CompressionMiddleware` (`_cloud_compress`). The API key, a `/bot…` path segment and a `?token=` query on the URL are canaries too. - Observed result: the output of both runs is in the two blocks below. ```text branch: cleanup/cli-integrations-shared-compress-backend @fad5714chealth: 200 POST /v1/messages -> HTTP200 smoke-ok POST /v1/chat/completions -> HTTP200 smoke-ok stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82 proxy.log WARNING/ERROR lines (<tmp>/ws/logs/proxy-58395.log): 2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes. 4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base 4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection. (smoke dir: <tmp>) ``` ```text HeadroomCallback 503 -> None | 200 -> 3 CompressionMiddleware 503 -> None | 200 -> 3 WARNING headroom.integrations.litellm_callback: Headroom Cloud API error: HTTP 503 from http://127.0.0.1:PORT/<path>?<redacted>; request sent uncompressed DEBUG headroom.integrations.litellm_callback: Headroom Cloud API error response: content-type=text/plain, 138 bytes WARNING headroom.integrations.asgi: Headroom Cloud API error: HTTP 503 from http://127.0.0.1:PORT/<path>?<redacted>; request sent uncompressed DEBUG headroom.integrations.asgi: Headroom Cloud API error response: content-type=text/plain, 138 bytes canaries in log: 0 ``` - Not tested: the real Headroom Cloud service. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: n/a, library refactor. - Stable/default behavior changed: no. The env vars, default URL, payload, timeout, offline refusal and log text are all the same. - Kill switch / disable path: n/a. - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes Part of maintainer-requested cleanup #4136 (dead code, duplication, debuggability), split into small PRs for review. Depends on #4138; its two files show in this diff until it merges. - `tests/test_integrations/test_litellm_callback.py` loads `litellm_callback.py` from its file path, to avoid importing the `headroom.integrations` package. Because the module now imports `headroom.integrations._compress_backend`, the loader registers a file-loaded copy of that module first; a module already in `sys.modules` is returned without importing its parent package. Normal imports of `headroom.integrations.litellm_callback` load the package anyway, as before. - A test that patched `headroom.integrations.asgi.guard_egress` would no longer intercept the call, because the call now lives in `_compress_backend`. No such test exists. - Docs impact (the changed warning text) is recorded in the docs ledger; there are no doc edits in this PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
106 lines
3.7 KiB
Python
106 lines
3.7 KiB
Python
"""Tests for headroom.integrations.litellm_callback."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import importlib
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
from typing import Any
|
|
from unittest.mock import AsyncMock, MagicMock
|
|
|
|
import pytest
|
|
|
|
_INTEGRATIONS = Path(__file__).resolve().parents[2] / "headroom" / "integrations"
|
|
_BACKEND = "headroom.integrations._compress_backend"
|
|
|
|
|
|
def _load_file(name: str, filename: str) -> Any:
|
|
spec = importlib.util.spec_from_file_location(name, _INTEGRATIONS / filename)
|
|
assert spec is not None and spec.loader is not None
|
|
mod = importlib.util.module_from_spec(spec)
|
|
spec.loader.exec_module(mod)
|
|
return mod
|
|
|
|
|
|
def _import_callback() -> type:
|
|
# Load the module files directly to avoid triggering headroom/integrations/__init__.py,
|
|
# which pulls in langchain and the native .so extension. litellm_callback imports
|
|
# _compress_backend by its dotted name, so register a file-loaded copy for that one
|
|
# import; with the name already in sys.modules, the integrations package is not
|
|
# imported. Both files still import the top-level headroom package and its
|
|
# stdlib-only helpers (headroom.offline, headroom.log_safety); the subprocess test
|
|
# below checks that neither langchain nor headroom._core is loaded.
|
|
saved = sys.modules.get(_BACKEND)
|
|
sys.modules[_BACKEND] = _load_file(_BACKEND, "_compress_backend.py")
|
|
try:
|
|
mod = _load_file("headroom.integrations.litellm_callback", "litellm_callback.py")
|
|
finally:
|
|
if saved is None:
|
|
del sys.modules[_BACKEND]
|
|
else:
|
|
sys.modules[_BACKEND] = saved
|
|
return mod.HeadroomCallback # type: ignore[no-any-return]
|
|
|
|
|
|
HeadroomCallback = _import_callback()
|
|
|
|
|
|
class TestHeadroomCallbackPostCallSuccessHook:
|
|
"""async_post_call_success_hook must exist and return response unchanged."""
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_returns_response_unchanged(self) -> None:
|
|
cb = HeadroomCallback()
|
|
sentinel = object()
|
|
result = await cb.async_post_call_success_hook(
|
|
data={},
|
|
user_api_key_dict=None,
|
|
response=sentinel,
|
|
)
|
|
assert result is sentinel
|
|
|
|
|
|
class TestHeadroomCallbackClientLifecycle:
|
|
"""Cloud client cleanup must be explicit and safe to repeat."""
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_aclose_closes_and_clears_initialized_client(self) -> None:
|
|
cb = HeadroomCallback(api_key="hdr_test")
|
|
client = MagicMock()
|
|
client.aclose = AsyncMock()
|
|
cb._client = client
|
|
|
|
await cb.aclose()
|
|
|
|
client.aclose.assert_awaited_once_with()
|
|
assert cb._client is None
|
|
|
|
await cb.aclose()
|
|
client.aclose.assert_awaited_once_with()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_aclose_without_initialized_client_is_a_noop(self) -> None:
|
|
cb = HeadroomCallback(api_key="hdr_test")
|
|
|
|
await cb.aclose()
|
|
|
|
assert cb._client is None
|
|
|
|
|
|
def test_direct_load_does_not_import_the_integrations_package() -> None:
|
|
"""The loader above must keep this file isolated from LangChain, the native
|
|
extension and the integrations package; check it in a fresh interpreter."""
|
|
code = (
|
|
"import importlib.util, sys\n"
|
|
f"spec = importlib.util.spec_from_file_location('t', {str(Path(__file__))!r})\n"
|
|
"mod = importlib.util.module_from_spec(spec); spec.loader.exec_module(mod)\n"
|
|
"bad = [m for m in sys.modules if m in ('headroom.integrations', 'headroom._core')"
|
|
" or m.startswith('langchain')]\n"
|
|
"print(bad)\n"
|
|
)
|
|
out = subprocess.run(
|
|
[sys.executable, "-c", code], capture_output=True, text=True, check=True, timeout=120
|
|
)
|
|
assert out.stdout.strip() == "[]", out.stdout + out.stderr
|