Files
headroom/tests/test_proxy_disable_kompress.py
NAZIYAandJerrettDavis a656eb4370 fix(kompress): prevent loading model when enable_kompress is False (#4667)
## Description

Fixes an issue where setting `HEADROOM_DISABLE_KOMPRESS=1` (or
configuring `ContentRouterConfig(enable_kompress=False)`) still loaded
the Kompress ONNX model when `_get_kompress()` was accessed (such as
during `/v1/compress` pipelines or reasoning compaction handlers). When
Kompress is disabled, `_get_kompress()` now returns `None` immediately
without downloading or loading ONNX model weights into memory, saving ~1
GB of resident memory.

Closes #4666

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `if not self.config.enable_kompress: return None` at the entry
of `ContentRouter._get_kompress()` in
`headroom/transforms/content_router.py`.
- Added check `and _rc_router.config.enable_kompress` before querying
`_rc_kompress` in `headroom/proxy/handlers/openai.py`.
- Explicitly passed `enable_kompress=False` in `ContentRouterConfig`
during `cold_recompact_messages()` in
`headroom/transforms/cold_prefix.py`.
- Added test cases in `tests/test_proxy_disable_kompress.py` asserting
that when `enable_kompress=False`, neither `is_kompress_available()`,
`KompressCompressor()`, nor `_get_remote_kompress()` are ever called or
instantiated.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
============================= test session starts =============================
platform win32 -- Python 3.14.5, pytest-9.0.3, pluggy-1.6.0
rootdir: C:\Users\dorni\OneDrive\Desktop\New folder\scratch\headroom
configfile: pyproject.toml
plugins: anyio-4.13.0, hypothesis-6.165.9, cov-7.1.0, schemathesis-4.24.3
collected 8 items

tests\test_proxy_disable_kompress.py ........                            [100%]

======================== 8 passed, 1 warning in 4.91s =========================
```

## Real Behavior Proof

- Environment: Windows 11 x64, Python 3.14.5, pytest 9.0.3
- Exact command / steps: python -c "from
headroom.transforms.content_router import ContentRouter,
ContentRouterConfig; r =
ContentRouter(ContentRouterConfig(enable_kompress=False));
print('Kompress instance with enable_kompress=False:',
r._get_kompress())"
- Observed result: Kompress instance with enable_kompress=False: None.
Verified via mock spied call sites that is_kompress_available(),
KompressCompressor(), and _get_remote_kompress() are never invoked when
enable_kompress=False.
- Not tested: Live Kubernetes cluster deployment, GPU ONNX execution
provider.

## Runtime Rollout Safety

- Rollout-managed feature(s): Kompress ONNX model loader bypass on
ContentRouter
- Minimum rollout channel: dev
- Stable/default behavior changed: No (when enable_kompress is True or
default, existing Kompress instantiation logic executes unchanged)
- Kill switch / disable path: HEADROOM_DISABLE_KOMPRESS=1 /
--disable-kompress
- Unsafe override required: None
- Qualification impact: Low (purely safeguards the disabled codepath
from unintended initialization)
- Rollback path: Revert commit cleanly

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Closes #4666. All tests verified locally and CI governance checks
conform to repository standards.


### Maintainer current-main requalification

Exact head `be97f120eb9926e99d4008471d09bd93972564c1`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Maintainer current-main review against
c01cf07d94. Clean mechanical
synchronization preserves all contributor production changes. Added only
a real reasoning-request regression test: actual /v1/chat/completions
routing reaches a controlled upstream, forwards reasoning unchanged when
disabled, and never queries the remote loader. Isolating the old handler
guard makes that test fail with one unwanted loader call; restoring
exact contributor bytes passes. All55
disable/cold-prefix/sticky-reasoning/remote-Kompress tests pass with the
fresh main native DLL; touched test Ruff/format and whitespace pass. No
production logic rewritten.

### Maintainer independent live HTTP proof

Environment: Windows x64, Python3.12.13, fresh native binding whose Rust
input tree matches current main; real Uvicorn HTTP listener plus two
disposable localhost HTTP fixtures (upstream and remote Kompress). No ML
model weights or cloud service are needed. This is separate from the
pytest assertions.

Observed results:
- Current main control, disable_kompress=True: HTTP200,1upstream
request,1remote inference request,remote client constructed,plaintext
reasoning changed. The disabled-state assertion fails as expected.
- Reviewed PR, disable_kompress=True: HTTP200,1upstream request,0remote
inference requests,no local or remote Kompress instance,original
reasoning preserved.
- Reviewed PR, disable_kompress=False positive control:
HTTP200,1upstream request,1remote inference request,remote client
constructed,reasoning compacted.

The untested scope is local ONNX/GPU inference, quantitative RSS savings
and deployed Kubernetes; this proves disabled request
routing/initialization and preserves enabled remote inference.

Reproduction: in an isolated checkout install the proxy extra (`python
-m pip install -e ".[proxy]"`), save the code below as
`kompress_probe.py`, then run `python kompress_probe.py disabled` and
`python kompress_probe.py enabled`. The same disabled command on main
c01cf07d reproduces the baseline failure. The script binds only
ephemeral loopback ports and shuts all servers down.

<details>
<summary>Standalone live HTTP probe</summary>

```python
import json
import os
from pathlib import Path
import socket
import sys
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

disabled = sys.argv[1] == 'disabled'
for key in ['HTTP_PROXY', 'HTTPS_PROXY', 'ALL_PROXY', 'HEADROOM_KOMPRESS_ENDPOINT_TOKEN', 'HEADROOM_KOMPRESS_ENDPOINT_HEADERS_JSON']:
    os.environ.pop(key, None)
os.environ.update(HEADROOM_OFFLINE='0', HEADROOM_NO_TELEMETRY='1', HEADROOM_SKIP_UPSTREAM_CHECK='1', HEADROOM_REQUIRE_RUST_CORE='false', HEADROOM_HTTP2='false', HEADROOM_DETECT_BACKEND='python', HEADROOM_THINKING_COMPACT='1', HEADROOM_THINKING_COMPACT_KEEP_LAST='0', HEADROOM_KOMPRESS_ENDPOINT_PATH='/compress', NO_PROXY='127.0.0.1,localhost')
sys.path.insert(0, str(Path.cwd()))
import httpx
import uvicorn
from headroom.proxy.server import ProxyConfig, create_app
from headroom.transforms.compression_units import find_content_router

seen = {'remote_calls': 0, 'upstream': []}
class Fixture(BaseHTTPRequestHandler):
    def log_message(self, *args):
        pass
    def do_POST(self):
        body=json.loads(self.rfile.read(int(self.headers['content-length'])))
        if self.server.remote:
            seen['remote_calls']+=1
            result={'compressed':'concise reasoning', 'original_tokens':300, 'compressed_tokens':2}
        else:
            seen['upstream'].append(body)
            result={'id':'fixture','object':'chat.completion','model':body['model'],'choices':[{'index':0,'message':{'role':'assistant','content':'ok'},'finish_reason':'stop'}],'usage':{'prompt_tokens':300,'completion_tokens':1,'total_tokens':301}}
        data=json.dumps(result).encode();self.send_response(200);self.send_header('content-type','application/json');self.send_header('content-length',str(len(data)));self.end_headers();self.wfile.write(data)

upstream=ThreadingHTTPServer(('127.0.0.1',0),Fixture);upstream.remote=False
remote=ThreadingHTTPServer(('127.0.0.1',0),Fixture);remote.remote=True
for fixture in [upstream,remote]:threading.Thread(target=fixture.serve_forever,daemon=True).start()
os.environ['HEADROOM_KOMPRESS_ENDPOINT']='http://127.0.0.1:'+str(remote.server_port)
app=create_app(ProxyConfig(optimize=True, disable_kompress=disabled, openai_api_url='http://127.0.0.1:'+str(upstream.server_port), cache_enabled=False,rate_limit_enabled=False,cost_tracking_enabled=False,log_requests=False,ccr_inject_tool=False,ccr_handle_responses=False,ccr_context_tracking=False,image_optimize=False))
listener=socket.socket();listener.bind(('127.0.0.1',0));listener.listen(128)
port=listener.getsockname()[1]
server=uvicorn.Server(uvicorn.Config(app,log_level='error',lifespan='on'))
thread=threading.Thread(target=server.run,kwargs={'sockets':[listener]},daemon=True);thread.start()
reasoning=' '.join('reason'+str(i) for i in range(100))
try:
    deadline=time.monotonic()+30
    while not server.started and thread.is_alive() and time.monotonic()<deadline:time.sleep(.02)
    assert server.started,'proxy failed to start'
    with httpx.Client(trust_env=False,timeout=30) as client:
        response=client.post('http://127.0.0.1:'+str(port)+'/v1/chat/completions',headers={'authorization':'Bearer local-fixture'},json={'model':'kimi-k2.7','messages':[{'role':'user','content':'question'},{'role':'assistant','content':'answer','reasoning_content':reasoning},{'role':'user','content':'continue'}]})
    assert response.status_code==200,(response.status_code,response.text)
    assert len(seen['upstream'])==1
    forwarded=seen['upstream'][0]['messages'][1].get('reasoning_content')
    router=find_content_router(app.state.proxy.openai_pipeline)
    result={'disabled':disabled,'status':response.status_code,'upstream_calls':len(seen['upstream']),'remote_calls':seen['remote_calls'],'remote_constructed':getattr(router,'_kompress_remote',None) is not None,'local_constructed':router._kompress is not None,'reasoning_preserved':forwarded==reasoning}
    print('RUNTIME_RESULT '+json.dumps(result),flush=True)
    if disabled:
        assert seen['remote_calls']==0 and not result['remote_constructed'] and not result['local_constructed']
        assert result['reasoning_preserved']
    else:
        assert seen['remote_calls']>0 and result['remote_constructed'] and not result['reasoning_preserved']
finally:
    server.should_exit=True;thread.join(10)
    upstream.shutdown();remote.shutdown();upstream.server_close();remote.server_close()

```
</details>

Exact-head hosted qualification is handled by maintainers; approval
remains pending that result. No contributor CI action or merge
requested.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-06 08:43:44 -06:00

265 lines
8.7 KiB
Python

"""Proxy configuration for disabling Kompress while keeping optimization on."""
from __future__ import annotations
import pytest
pytest.importorskip("fastapi")
from headroom.proxy.server import ProxyConfig, create_app
from headroom.transforms import CompressionStrategy, ContentRouter
@pytest.fixture(autouse=True)
def _stub_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("HEADROOM_REQUIRE_RUST_CORE", "false")
monkeypatch.setenv("HEADROOM_HTTP2", "false")
def _proxy_router(config: ProxyConfig):
app = create_app(config)
proxy = app.state.proxy
return next(
transform
for transform in proxy.anthropic_pipeline.transforms
if isinstance(transform, ContentRouter)
)
def test_disable_kompress_config_keeps_optimization_but_disables_ml_fallback() -> None:
router = _proxy_router(
ProxyConfig(
optimize=True,
disable_kompress=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
)
assert router.config.enable_kompress is False
assert router.config.fallback_strategy == CompressionStrategy.KOMPRESS
def test_disable_kompress_defaults_to_existing_kompress_behavior() -> None:
router = _proxy_router(
ProxyConfig(
optimize=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
)
assert router.config.enable_kompress is True
assert router.config.fallback_strategy == CompressionStrategy.KOMPRESS
def test_health_config_reports_disable_kompress_fallback(monkeypatch: pytest.MonkeyPatch) -> None:
from fastapi.testclient import TestClient
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
app = create_app(
ProxyConfig(
optimize=True,
disable_kompress=True,
disable_kompress_fallback=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
)
with TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 12345)) as client:
config = client.get("/health").json()["config"]
assert config["disable_kompress"] is True
assert config["disable_kompress_fallback"] is True
def test_disable_kompress_fallback_restores_passthrough() -> None:
router = _proxy_router(
ProxyConfig(
optimize=True,
disable_kompress=True,
disable_kompress_fallback=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
)
assert router.config.enable_kompress is False
assert router.config.fallback_strategy == CompressionStrategy.PASSTHROUGH
def test_disable_kompress_fallback_without_disable_kompress_is_noop() -> None:
router = _proxy_router(
ProxyConfig(
optimize=True,
disable_kompress_fallback=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
)
assert router.config.enable_kompress is True
assert router.config.fallback_strategy == CompressionStrategy.KOMPRESS
def test_disable_kompress_prevents_kompress_instantiation_in_content_router(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from unittest.mock import MagicMock
import headroom.transforms.kompress_compressor as kc
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
mock_avail = MagicMock(return_value=True)
mock_compressor = MagicMock()
monkeypatch.setattr(kc, "is_kompress_available", mock_avail)
monkeypatch.setattr(kc, "KompressCompressor", mock_compressor)
router = ContentRouter(ContentRouterConfig(enable_kompress=False))
result = router._get_kompress()
assert result is None
mock_avail.assert_not_called()
mock_compressor.assert_not_called()
def test_disable_kompress_bypasses_remote_kompress(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from unittest.mock import MagicMock
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
mock_remote = MagicMock()
router = ContentRouter(ContentRouterConfig(enable_kompress=False))
monkeypatch.setattr(router, "_get_remote_kompress", mock_remote)
result = router._get_kompress()
assert result is None
mock_remote.assert_not_called()
def test_disable_kompress_reasoning_compaction_skips_kompress_loader(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from unittest.mock import MagicMock
import headroom.transforms.kompress_compressor as kc
from headroom.proxy.server import ProxyConfig, create_app
from headroom.transforms.compression_units import find_content_router
mock_avail = MagicMock(return_value=True)
mock_compressor = MagicMock()
monkeypatch.setattr(kc, "is_kompress_available", mock_avail)
monkeypatch.setattr(kc, "KompressCompressor", mock_compressor)
config = ProxyConfig(
optimize=True,
disable_kompress=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
app = create_app(config)
proxy = app.state.proxy
router = find_content_router(proxy.openai_pipeline)
assert router is not None
assert router.config.enable_kompress is False
assert router._get_kompress() is None
mock_avail.assert_not_called()
mock_compressor.assert_not_called()
def test_disabled_kompress_reasoning_request_never_queries_remote_loader(
monkeypatch: pytest.MonkeyPatch,
) -> None:
import json
from unittest.mock import MagicMock
import httpx
from fastapi.testclient import TestClient
from headroom.transforms.compression_units import find_content_router
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
monkeypatch.setenv("HEADROOM_THINKING_COMPACT", "1")
monkeypatch.setenv("HEADROOM_THINKING_COMPACT_KEEP_LAST", "0")
monkeypatch.setenv("HEADROOM_KOMPRESS_ENDPOINT", "http://127.0.0.1:9")
app = create_app(
ProxyConfig(
optimize=True,
disable_kompress=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
)
)
forwarded = []
def upstream(request: httpx.Request) -> httpx.Response:
forwarded.append(json.loads(request.content))
return httpx.Response(
200,
json={
"id": "fixture",
"object": "chat.completion",
"model": "kimi-k2.7",
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": "ok"},
"finish_reason": "stop",
}
],
"usage": {"prompt_tokens": 100, "completion_tokens": 1, "total_tokens": 101},
},
)
reasoning = " ".join(f"reason{i}" for i in range(100))
with TestClient(app, client=("127.0.0.1", 50000)) as client:
proxy = app.state.proxy
async def install_transport() -> None:
await proxy.http_client.aclose()
if proxy.http_client_h1 is not None:
await proxy.http_client_h1.aclose()
proxy.http_client = httpx.AsyncClient(transport=httpx.MockTransport(upstream))
proxy.http_client_h1 = None
client.portal.call(install_transport)
router = find_content_router(proxy.openai_pipeline)
assert router is not None
remote_loader = MagicMock(return_value=None)
monkeypatch.setattr(router, "_get_remote_kompress", remote_loader)
response = client.post(
"/v1/chat/completions",
headers={"authorization": "Bearer local-fixture"},
json={
"model": "kimi-k2.7",
"messages": [
{"role": "user", "content": "question"},
{"role": "assistant", "content": "answer", "reasoning_content": reasoning},
{"role": "user", "content": "continue"},
],
},
)
assert response.status_code == 200
remote_loader.assert_not_called()
assert len(forwarded) == 1
assert forwarded[0]["messages"][1]["reasoning_content"] == reasoning