Files
headroom/tests/test_compress_api.py
Tejas ChopraandClaude Opus 5.5 c6d708144c fix(asgi): stop buffering when the client disconnects (GHSA-mc2f-vg4v-2fh4) (#4701)
## Description

Fixes the unauthenticated worker hang in the ASGI
`CompressionMiddleware`, tracked in draft advisory GHSA-mc2f-vg4v-2fh4
and reported by @hardness1020.

The middleware reads the request body in a `while True` loop that only
exits on an `http.request` message without `more_body`. When the client
disconnects mid-body, the server answers every further `receive()` with
`http.disconnect` immediately, so the loop spins forever without
yielding and the worker stops serving. One partial POST followed by a
close is enough.

This is the fix reviewed in the advisory's private fork (PR #1 there),
cherry-picked onto current main.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/integrations/asgi.py`: any message other than `http.request`
ends buffering and the middleware returns without calling the app
(nobody is left to answer).
- `tests/test_compress_api.py`:
`test_client_disconnect_mid_body_stops_buffering`, parametrized over
`http.disconnect` and an unknown message type. It fails if the
middleware reads again after the non-body message.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_compress_api.py -k disconnect      # with this PR
2 passed

$ pytest -q tests/test_compress_api.py -k disconnect      # main's asgi.py swapped back in
FAILED ...test_client_disconnect_mid_body_stops_buffering[http.disconnect]
FAILED ...test_client_disconnect_mid_body_stops_buffering[http.bogus]
2 failed

$ ruff check / ruff format --check on the changed files: clean
```

Locally `test_compress_api.py` also shows 3 failures
(`test_compact_json_counts_tokens_not_whitespace`, two
`TestFrozenMessageCount` tests). They fail identically on unmodified
main in a worktree without the Rust `headroom._core` extension built, so
they're environmental, not from this change. mypy reports the same 8
pre-existing errors on main and on this branch.

## Real Behavior Proof

- Environment: reporter's setup, macOS, Python 3.14.2, uvicorn 0.40.0,
starlette 1.3.1, fork PR head 7f614956 in a fresh venv.
- Exact command / steps: partial POST to an app wrapped in
`CompressionMiddleware` under live uvicorn, then close the connection;
plus a disconnect as the first message and a two-chunk complete body.
- Observed result: before, the worker hangs on the disconnect cases.
After, both disconnect cases return, and the chunked body still reaches
the app byte-for-byte. The reporter also confirmed the fix merges
cleanly after #4124.
- Not tested: the live uvicorn run was not repeated on this cherry-pick.
The diff to `asgi.py` is identical to the reviewed fork head.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: only for requests whose client
disconnected before the body finished; those no longer reach the app.
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Docs: N/A, no user-facing behavior to document. After this ships, the
advisory's patched version gets set to that release and it can be
published. The affected range is already updated to `>= 0.3.7, <=
0.43.1` (0.41.0 through 0.43.1 still carry the old loop).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4701"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 12:34:00 -06:00

453 lines
17 KiB
Python

"""Tests for the one-function compress() API and integrations."""
import asyncio
import json
from dataclasses import replace as _dc_replace
import pytest
from headroom.compress import CompressConfig, CompressResult, compress
from headroom.hooks import CompressionHooks
try:
from starlette.applications import Starlette
from starlette.requests import Request
from starlette.responses import JSONResponse
from starlette.routing import Route
from starlette.testclient import TestClient
from headroom.integrations.asgi import CompressionMiddleware
HAS_STARLETTE = True
except ImportError:
HAS_STARLETTE = False
# =============================================================================
# Tests: compress() function
# =============================================================================
class TestCompressFunction:
def test_empty_messages(self):
result = compress([], model="test")
assert result.messages == []
assert result.tokens_saved == 0
def test_small_messages_passthrough(self):
"""Small messages below compression threshold pass through unchanged."""
messages = [{"role": "user", "content": "hello"}]
result = compress(messages, model="gpt-4o")
assert result.messages[0]["content"] == "hello"
assert result.tokens_saved == 0
def test_returns_compress_result(self):
result = compress([{"role": "user", "content": "hi"}])
assert isinstance(result, CompressResult)
assert hasattr(result, "messages")
assert hasattr(result, "tokens_saved")
assert hasattr(result, "compression_ratio")
assert hasattr(result, "transforms_applied")
def test_large_tool_output_compressed(self):
"""Large JSON tool output should be compressed."""
big_data = json.dumps(
[
{"id": i, "status": "active", "name": f"item_{i}", "value": i * 17}
for i in range(200)
]
)
messages = [
{"role": "user", "content": "What are the top items?"},
{"role": "tool", "content": big_data, "tool_call_id": "call_1"},
]
result = compress(messages, model="gpt-4o")
assert result.tokens_after <= result.tokens_before
assert len(result.messages) == 2
def test_compact_json_counts_tokens_not_whitespace(self):
"""Compact JSON arrays should still compress under token thresholds."""
numbers = [42.0 + i * 0.1 for i in range(200)]
messages = [
{"role": "system", "content": "You are helpful."},
{"role": "user", "content": "Show metrics"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": "get_metrics", "arguments": "{}"},
}
],
},
{"role": "tool", "tool_call_id": "call_1", "content": json.dumps(numbers)},
]
result = compress(messages, min_tokens_to_compress=250)
assert result.tokens_saved > 0
assert any(
transform.startswith("router:smart_crusher") for transform in result.transforms_applied
)
def test_optimize_false_passthrough(self):
"""optimize=False returns messages unchanged."""
messages = [{"role": "user", "content": "hello world " * 100}]
result = compress(messages, optimize=False)
assert result.messages is messages
assert result.tokens_saved == 0
def test_kwargs_do_not_mutate_caller_config(self):
"""kwargs must not smuggle their values onto the caller's CompressConfig.
Regression: ``compress`` did ``cfg = config or CompressConfig()`` and
then ``setattr(cfg, key, value)`` for every matching kwarg — so a caller
who passed ``config=my_cfg, protect_recent=0`` came back to find their
long-lived ``my_cfg`` silently rewritten. A shared, per-agent config
was corrupted by every request that overrode a single option.
"""
big_data = json.dumps([{"id": i, "status": "active"} for i in range(200)])
messages = [
{"role": "user", "content": "analyze"},
{"role": "tool", "content": big_data, "tool_call_id": "c1"},
]
cfg = CompressConfig(protect_recent=4, target_ratio=0.8)
snapshot = _dc_replace(cfg)
compress(
messages,
model="claude-sonnet-4-5-20250929",
config=cfg,
protect_recent=0,
target_ratio=0.2,
)
assert cfg.protect_recent == snapshot.protect_recent, (
"compress() mutated caller's config.protect_recent via kwargs"
)
assert cfg.target_ratio == snapshot.target_ratio, (
"compress() mutated caller's config.target_ratio via kwargs"
)
def test_with_custom_hooks(self):
"""Hooks are called when provided."""
calls = []
class TrackingHooks(CompressionHooks):
def pre_compress(self, messages, ctx):
calls.append(("pre", len(messages)))
return messages
def compute_biases(self, messages, ctx):
calls.append(("biases", len(messages)))
return {}
def post_compress(self, event):
calls.append(("post", event.tokens_saved))
big_data = json.dumps([{"id": i, "status": "active"} for i in range(100)])
messages = [
{"role": "user", "content": "analyze"},
{"role": "tool", "content": big_data, "tool_call_id": "c1"},
]
compress(messages, hooks=TrackingHooks())
assert any(c[0] == "pre" for c in calls)
assert any(c[0] == "biases" for c in calls)
class TestCompressResultFields:
def test_fields_populated(self):
big_data = json.dumps([{"id": i, "type": "log"} for i in range(100)])
messages = [
{"role": "user", "content": "summarize"},
{"role": "tool", "content": big_data, "tool_call_id": "c1"},
]
result = compress(messages, model="claude-sonnet-4-5-20250929")
assert result.tokens_before > 0
assert result.tokens_after >= 0
assert result.tokens_saved >= 0
assert 0.0 <= result.compression_ratio <= 1.0
# =============================================================================
# Tests: ASGI CompressionMiddleware (requires starlette)
# =============================================================================
def _make_asgi_app(middleware_kwargs=None):
"""Create a test ASGI app with CompressionMiddleware."""
async def chat_endpoint(request: Request) -> JSONResponse:
body = await request.json()
return JSONResponse(
{
"model": "gpt-4o",
"choices": [{"message": {"content": "response"}}],
"usage": {"prompt_tokens": 10, "completion_tokens": 5},
"_message_count": len(body.get("messages", [])),
}
)
async def health(request: Request) -> JSONResponse:
return JSONResponse({"status": "ok"})
app = Starlette(
routes=[
Route("/health", health),
Route("/v1/chat/completions", chat_endpoint, methods=["POST"]),
Route("/v1/messages", chat_endpoint, methods=["POST"]),
]
)
app.add_middleware(CompressionMiddleware, **(middleware_kwargs or {}))
return app
@pytest.mark.skipif(not HAS_STARLETTE, reason="starlette not installed")
class TestASGIMiddleware:
def test_non_llm_paths_passthrough(self):
app = _make_asgi_app()
client = TestClient(app)
resp = client.get("/health")
assert resp.status_code == 200
assert resp.json()["status"] == "ok"
def test_small_messages_passthrough(self):
app = _make_asgi_app()
client = TestClient(app)
resp = client.post(
"/v1/chat/completions",
json={"model": "gpt-4o", "messages": [{"role": "user", "content": "hi"}]},
)
assert resp.status_code == 200
def test_large_messages_compressed(self):
"""Large tool output should be compressed by middleware."""
app = _make_asgi_app()
client = TestClient(app)
big_data = json.dumps([{"id": i, "status": "active"} for i in range(200)])
resp = client.post(
"/v1/chat/completions",
json={
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "analyze"},
{"role": "tool", "content": big_data, "tool_call_id": "c1"},
],
},
)
assert resp.status_code == 200
def test_anthropic_path(self):
"""Works with Anthropic /v1/messages path."""
app = _make_asgi_app()
client = TestClient(app)
resp = client.post(
"/v1/messages",
json={
"model": "claude-sonnet-4-5-20250929",
"messages": [{"role": "user", "content": "hello"}],
},
)
assert resp.status_code == 200
def test_get_requests_passthrough(self):
"""GET requests to LLM paths pass through."""
app = _make_asgi_app()
client = TestClient(app)
resp = client.get("/v1/chat/completions")
assert resp.status_code in (200, 405)
@pytest.mark.parametrize("after_partial", ["http.disconnect", "http.bogus"])
def test_client_disconnect_mid_body_stops_buffering(self, after_partial):
"""A client that sends part of a body and disconnects must not hang the worker.
After a disconnect, uvicorn answers every receive() with http.disconnect
without suspending, so the buffering loop used to spin forever at full
CPU, starving the event loop (GHSA-mc2f-vg4v-2fh4). Any message other
than more body ends buffering the same way.
"""
app_called = False
async def app(scope, receive, send):
nonlocal app_called
app_called = True
calls = 0
async def receive():
nonlocal calls
calls += 1
if calls > 100:
raise AssertionError("middleware kept reading after http.disconnect")
if calls == 1:
return {"type": "http.request", "body": b'{"partial"', "more_body": True}
return {"type": after_partial}
async def send(message):
raise AssertionError(f"nothing should be sent to a gone client: {message}")
scope = {"type": "http", "method": "POST", "path": "/x/v1/chat/completions"}
asyncio.run(CompressionMiddleware(app)(scope, receive, send))
assert calls == 2
assert not app_called
# =============================================================================
# Tests: LiteLLM Callback
# =============================================================================
class TestLiteLLMCallback:
def test_callback_imports(self):
"""Verify the callback can be imported."""
from headroom.integrations.litellm_callback import HeadroomCallback
callback = HeadroomCallback()
assert callback.total_tokens_saved == 0
def test_callback_compresses_messages(self):
"""Callback compresses messages in pre_call_hook."""
import asyncio
from headroom.integrations.litellm_callback import HeadroomCallback
callback = HeadroomCallback()
big_data = json.dumps([{"id": i, "status": "active"} for i in range(200)])
data = {
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "analyze"},
{"role": "tool", "content": big_data, "tool_call_id": "c1"},
],
}
result = asyncio.run(callback.async_pre_call_hook("key", data, "completion"))
assert result is data
def test_callback_ignores_non_completion(self):
"""Non-completion calls are passed through."""
import asyncio
from headroom.integrations.litellm_callback import HeadroomCallback
callback = HeadroomCallback()
data = {"messages": [{"role": "user", "content": "hi"}]}
result = asyncio.run(callback.async_pre_call_hook("key", data, "embedding"))
assert result is data
# =============================================================================
# Tests: frozen_message_count through library-mode compress()
# =============================================================================
class TestFrozenMessageCount:
"""The frozen prefix must be reachable from library mode.
Proxy handlers pass frozen_message_count so transforms never rewrite
messages already anchored in the provider's prompt cache. Library-mode
callers manage their own conversation loop and need the same control —
without it, read_lifecycle rewrites sent history and converts cached
prefix reads into full-price rewrites.
"""
@staticmethod
def _stale_read_conversation() -> list[dict]:
"""Anthropic-format conversation with a stale Read: file read early,
edited later. read_lifecycle should classify the Read as STALE."""
big_content = "\n".join(f"line {i}: some file content here" for i in range(80))
return [
{"role": "user", "content": "read then edit the config"},
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "t_read",
"name": "Read",
"input": {"file_path": "/app/config.py"},
}
],
},
{
"role": "user",
"content": [
{"type": "tool_result", "tool_use_id": "t_read", "content": big_content}
],
},
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "t_edit",
"name": "Edit",
"input": {
"file_path": "/app/config.py",
"old_string": "old",
"new_string": "new",
},
}
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t_edit", "content": "ok"}],
},
{"role": "assistant", "content": "edited."},
{"role": "user", "content": "now summarize the change"},
]
@staticmethod
def _read_result_content(messages: list[dict]) -> str:
for msg in messages:
content = msg.get("content")
if not isinstance(content, list):
continue
for block in content:
if isinstance(block, dict) and block.get("tool_use_id") == "t_read":
return str(block.get("content"))
raise AssertionError("t_read tool_result not found")
def test_stale_read_rewritten_without_frozen_prefix(self):
"""Baseline: with no frozen prefix, the stale Read is rewritten."""
messages = self._stale_read_conversation()
original = self._read_result_content(messages)
result = compress(messages, model="claude-sonnet-4-5-20250929")
assert self._read_result_content(result.messages) != original
def test_frozen_prefix_blocks_stale_read_rewrite(self):
"""frozen_message_count as kwarg: messages inside the frozen prefix
must come back byte-identical, even though the Read is stale."""
messages = self._stale_read_conversation()
original = self._read_result_content(messages)
result = compress(
messages,
model="claude-sonnet-4-5-20250929",
frozen_message_count=5,
)
assert self._read_result_content(result.messages) == original
def test_frozen_prefix_via_config_object(self):
"""frozen_message_count set on CompressConfig behaves identically."""
messages = self._stale_read_conversation()
original = self._read_result_content(messages)
cfg = CompressConfig(frozen_message_count=5)
result = compress(messages, model="claude-sonnet-4-5-20250929", config=cfg)
assert self._read_result_content(result.messages) == original
def test_frozen_zero_is_legacy_behavior(self):
"""Explicit 0 matches the default: stale Read gets rewritten."""
messages = self._stale_read_conversation()
original = self._read_result_content(messages)
result = compress(messages, model="claude-sonnet-4-5-20250929", frozen_message_count=0)
assert self._read_result_content(result.messages) != original