mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(asgi): stop buffering when the client disconnects (GHSA-mc2f-vg4v-2fh4) (#4701)
## Description Fixes the unauthenticated worker hang in the ASGI `CompressionMiddleware`, tracked in draft advisory GHSA-mc2f-vg4v-2fh4 and reported by @hardness1020. The middleware reads the request body in a `while True` loop that only exits on an `http.request` message without `more_body`. When the client disconnects mid-body, the server answers every further `receive()` with `http.disconnect` immediately, so the loop spins forever without yielding and the worker stops serving. One partial POST followed by a close is enough. This is the fix reviewed in the advisory's private fork (PR #1 there), cherry-picked onto current main. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/integrations/asgi.py`: any message other than `http.request` ends buffering and the middleware returns without calling the app (nobody is left to answer). - `tests/test_compress_api.py`: `test_client_disconnect_mid_body_stops_buffering`, parametrized over `http.disconnect` and an unknown message type. It fails if the middleware reads again after the non-body message. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest -q tests/test_compress_api.py -k disconnect # with this PR 2 passed $ pytest -q tests/test_compress_api.py -k disconnect # main's asgi.py swapped back in FAILED ...test_client_disconnect_mid_body_stops_buffering[http.disconnect] FAILED ...test_client_disconnect_mid_body_stops_buffering[http.bogus] 2 failed $ ruff check / ruff format --check on the changed files: clean ``` Locally `test_compress_api.py` also shows 3 failures (`test_compact_json_counts_tokens_not_whitespace`, two `TestFrozenMessageCount` tests). They fail identically on unmodified main in a worktree without the Rust `headroom._core` extension built, so they're environmental, not from this change. mypy reports the same 8 pre-existing errors on main and on this branch. ## Real Behavior Proof - Environment: reporter's setup, macOS, Python 3.14.2, uvicorn 0.40.0, starlette 1.3.1, fork PR head 7f614956 in a fresh venv. - Exact command / steps: partial POST to an app wrapped in `CompressionMiddleware` under live uvicorn, then close the connection; plus a disconnect as the first message and a two-chunk complete body. - Observed result: before, the worker hangs on the disconnect cases. After, both disconnect cases return, and the chunked body still reaches the app byte-for-byte. The reporter also confirmed the fix merges cleanly after #4124. - Not tested: the live uvicorn run was not repeated on this cherry-pick. The diff to `asgi.py` is identical to the reviewed fork head. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: only for requests whose client disconnected before the body finished; those no longer reach the app. - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert this PR ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Docs: N/A, no user-facing behavior to document. After this ships, the advisory's patched version gets set to that release and it can be published. The affected range is already updated to `>= 0.3.7, <= 0.43.1` (0.41.0 through 0.43.1 still carry the old loop). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4701" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
13347e339c
commit
c6d708144c
@@ -114,9 +114,14 @@ class CompressionMiddleware(CompressBackend):
|
||||
# Read the full body
|
||||
while True:
|
||||
msg = await buffering_receive()
|
||||
if msg.get("type") == "http.request":
|
||||
if not msg.get("more_body", False):
|
||||
break
|
||||
# After a disconnect the server answers every receive() with
|
||||
# http.disconnect at once, so looping on it would spin forever
|
||||
# without yielding. Anything but more body means nobody is left
|
||||
# to answer; stop here.
|
||||
if msg["type"] != "http.request":
|
||||
return
|
||||
if not msg.get("more_body", False):
|
||||
break
|
||||
|
||||
full_body = b"".join(body_chunks)
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Tests for the one-function compress() API and integrations."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from dataclasses import replace as _dc_replace
|
||||
|
||||
@@ -261,6 +262,41 @@ class TestASGIMiddleware:
|
||||
resp = client.get("/v1/chat/completions")
|
||||
assert resp.status_code in (200, 405)
|
||||
|
||||
@pytest.mark.parametrize("after_partial", ["http.disconnect", "http.bogus"])
|
||||
def test_client_disconnect_mid_body_stops_buffering(self, after_partial):
|
||||
"""A client that sends part of a body and disconnects must not hang the worker.
|
||||
|
||||
After a disconnect, uvicorn answers every receive() with http.disconnect
|
||||
without suspending, so the buffering loop used to spin forever at full
|
||||
CPU, starving the event loop (GHSA-mc2f-vg4v-2fh4). Any message other
|
||||
than more body ends buffering the same way.
|
||||
"""
|
||||
app_called = False
|
||||
|
||||
async def app(scope, receive, send):
|
||||
nonlocal app_called
|
||||
app_called = True
|
||||
|
||||
calls = 0
|
||||
|
||||
async def receive():
|
||||
nonlocal calls
|
||||
calls += 1
|
||||
if calls > 100:
|
||||
raise AssertionError("middleware kept reading after http.disconnect")
|
||||
if calls == 1:
|
||||
return {"type": "http.request", "body": b'{"partial"', "more_body": True}
|
||||
return {"type": after_partial}
|
||||
|
||||
async def send(message):
|
||||
raise AssertionError(f"nothing should be sent to a gone client: {message}")
|
||||
|
||||
scope = {"type": "http", "method": "POST", "path": "/x/v1/chat/completions"}
|
||||
asyncio.run(CompressionMiddleware(app)(scope, receive, send))
|
||||
|
||||
assert calls == 2
|
||||
assert not app_called
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Tests: LiteLLM Callback
|
||||
|
||||
Reference in New Issue
Block a user