mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): use HTTP/1.1 client for ChatGPT streaming and model metadata (#4006)
## Description Ensures that ChatGPT endpoints (`chatgpt.com`, including `/v1/responses` forwarded to `https://chatgpt.com/backend-api/codex/responses`, `/v1/models`, Codex subpaths, and retried requests) consistently utilize `http_client_h1` instead of multiplexing over the default HTTP/2 client. `server.py` already provisions `self.http_client_h1` specifically because ChatGPT's Cloudflare edge challenges HTTP/2 fingerprints on sensitive endpoints. When large context requests (e.g. 500KB - 800KB payload turns) are forwarded over HTTP/2 streaming, HTTP/2 connection/stream flow-control window stalls block the single multiplexed TCP connection in `httpcore`. When this occurs, all subsequent requests to ChatGPT hosts (including `/v1/models` and subsequent turn completions) become blocked waiting behind the stalled connection, eventually leading to proxy unresponsiveness and accumulated asyncio task queues. While `handle_passthrough` in `openai.py` and `openai_images.py` already selected `http_client_h1`, `StreamingMixin._stream_response_generator`, `handle_model_metadata_endpoint`, `_retry_request` in `server.py`, and Codex subpath routes in `proxy_routes.py` previously hardcoded `self.http_client`. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Expose `prefers_http1_passthrough` in `headroom/proxy/handlers/openai.py` (aliasing `_prefers_http1_passthrough`). - Add `_select_streaming_client` to `StreamingMixin` in `headroom/proxy/handlers/streaming.py` that selects `http_client_h1` when the target URL matches ChatGPT endpoints. - Update `_stream_response_generator` and `_compact_sse_stream` to use `_select_streaming_client`. - Update `_retry_request` in `headroom/proxy/server.py` to route ChatGPT target URLs through `http_client_h1`. - Update `_register_openai_responses_subpath_route` in `headroom/providers/proxy_routes.py` to forward through `http_client_h1` for `handle_chatgpt_codex_responses_subpath`. - In `headroom/providers/model_metadata.py`, pass `getattr(proxy, "http_client_h1", None) or proxy.http_client` to `handle_chatgpt_model_metadata`. - Add unit tests in `tests/test_proxy_handler_helpers.py` and `tests/test_provider_model_metadata.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py ======================== 76 passed, 1 warning in 8.24s ======================== uv run ruff check headroom/ tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS 27 (Darwin arm64), Python 3.13, Headroom Proxy v0.39.1 / v0.40.0, Codex CLI and OpenClaw Desktop client. - Exact command / steps: Sent 800 KB streaming turns to /v1/responses targeting https://chatgpt.com/backend-api/codex/responses while monitoring TCP socket states via lsof/netstat and triggering concurrent /v1/models calls. - Observed result: Streaming responses and model metadata resolve over HTTP/1.1 without multiplexed stream stall; Send-Q drains cleanly, turns finish in ~2.1s, and all unit tests pass. - Not tested: Third-party proprietary non-ChatGPT OAuth endpoints (e.g., enterprise SSO providers). ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: stable - Stable/default behavior changed: No (aligns streaming, retries, and subpath routes with existing `http_client_h1` passthrough policy for ChatGPT hosts) - Kill switch / disable path: Fallback to `self.http_client` when `http_client_h1` is unset - Unsafe override required: No - Qualification impact: None - Rollback path: Revert commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ### Maintainer current-main requalification Exact head `d6cee507ff0aae7f09705b2a5b9a8bd8264b137f`, upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. Updated against upstream/maine4a60ed7. Resolved the adjacent-helper conflict by retaining both the contributor's HTTP/1.1 compatibility alias and main's telemetry helper. HTTP/1.1 selection stays confined to ChatGPT hosts across streaming/continuations, retries, metadata and authenticated Responses subpaths; other upstreams retain their existing client. No contributor production logic was rewritten. All 136 focused helper/route/metadata tests pass. Three newer route fixtures initially mocked only the default client; mechanically assigned their same controlled response client to http_client_h1 too, preserving every forwarding assertion. Changed-file Ruff lint/format and whitespace checks pass. Existing macOS/Codex/OpenClaw live-stream evidence remains scoped; no new live ChatGPT stream or fresh native-build claim is made. Applied shared test-only Gemini streamed-request fixture repair for independently established main failures; all ten cases pass. Formal approval awaits exact-head hosted qualification. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. ### Maintainer current-main requalification Exact head `96dfae54717c13249f3bd9305cc6930030f2c85d`, upstream main `ecbaf86d78cf52d185fbffa306157b9a4cc1b906`. Maintainer test-only repair for hosted run42031595103: the newer Codex alias tests mocked only http_client, while this PR correctly routes authenticated ChatGPT requests through http_client_h1. Assigned the existing controlled fake to both client attributes, preserving every auth/forwarding assertion; no production logic changed. Main advanced during qualification; cleanly merged upstream/mainecbaf86d78and re-reviewed its dependency/workflow/docs delta. All 154 helper/route/metadata/Codex-alias/Gemini tests pass on the final tree; lint/format and whitespace checks pass. Existing runtime evidence remains explicitly scoped, with no new live ChatGPT or fresh native-build claim. Exact-head hosted checks are handled by maintainers. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JD Davis
parent
398d7cdcf4
commit
e1becdf732
@@ -51,8 +51,9 @@ async def handle_model_metadata_endpoint(
|
||||
) -> Response:
|
||||
"""Handle OpenAI-compatible model metadata with Codex ChatGPT-auth support."""
|
||||
assert proxy.http_client is not None
|
||||
chatgpt_client = getattr(proxy, "http_client_h1", None) or proxy.http_client
|
||||
chatgpt_response = await handle_chatgpt_model_metadata(
|
||||
proxy.http_client,
|
||||
chatgpt_client,
|
||||
request,
|
||||
endpoint.upstream_path,
|
||||
)
|
||||
|
||||
@@ -212,9 +212,9 @@ def _register_openai_responses_subpath_route(
|
||||
spec: OpenAIResponsesSubpathRoute,
|
||||
) -> None:
|
||||
async def openai_responses_subpath(request: Request, sub_path: str):
|
||||
assert proxy.http_client is not None
|
||||
codex_subpath_client = getattr(proxy, "http_client_h1", None) or proxy.http_client
|
||||
chatgpt_response = await handle_chatgpt_codex_responses_subpath(
|
||||
proxy.http_client,
|
||||
codex_subpath_client,
|
||||
request,
|
||||
sub_path,
|
||||
)
|
||||
|
||||
@@ -2074,16 +2074,21 @@ def _extract_responses_usage(event: dict[str, Any]) -> tuple[int, int, int, int,
|
||||
return input_tokens, output_tokens, cached, write, uncached, inferred
|
||||
|
||||
|
||||
def _prefers_http1_passthrough(base_url: str) -> bool:
|
||||
"""Whether passthrough to this host must use HTTP/1.1.
|
||||
def prefers_http1_passthrough(base_url: str) -> bool:
|
||||
"""Whether passthrough/streaming to this host must use HTTP/1.1.
|
||||
|
||||
ChatGPT's Cloudflare edge issues a managed challenge to our HTTP/2
|
||||
fingerprint on sensitive account endpoints; HTTP/1.1 is accepted.
|
||||
fingerprint on sensitive account endpoints, and multiplexing large
|
||||
streaming request bodies over HTTP/2 can stall on flow-control windows.
|
||||
HTTP/1.1 avoids both issues.
|
||||
"""
|
||||
host = (urlparse(base_url).hostname or "").lower()
|
||||
return host == "chatgpt.com" or host.endswith(".chatgpt.com")
|
||||
|
||||
|
||||
_prefers_http1_passthrough = prefers_http1_passthrough
|
||||
|
||||
|
||||
def _openai_outcome_provider(
|
||||
method: str, path: str, custom_base_url: str | None, upstream_base_url: str
|
||||
) -> str:
|
||||
|
||||
@@ -219,6 +219,18 @@ class StreamingMixin:
|
||||
_mid_turn_queues: dict[str, asyncio.Queue] = {}
|
||||
_active_streams: set[str] = set()
|
||||
|
||||
def _select_streaming_client(self, url: str) -> httpx.AsyncClient:
|
||||
"""Select HTTP client for upstream streaming.
|
||||
|
||||
ChatGPT endpoints prefer HTTP/1.1 to avoid Cloudflare bot-management
|
||||
challenges and HTTP/2 flow-control window stalls on large turns.
|
||||
"""
|
||||
from headroom.proxy.handlers.openai import prefers_http1_passthrough
|
||||
|
||||
if prefers_http1_passthrough(url):
|
||||
return getattr(self, "http_client_h1", None) or self.http_client
|
||||
return self.http_client
|
||||
|
||||
@staticmethod
|
||||
def _get_session_key(body: dict, session_header: str | None = None) -> str:
|
||||
"""Return session identity from an explicit header or a body-derived hash.
|
||||
@@ -697,10 +709,11 @@ class StreamingMixin:
|
||||
]
|
||||
continuation_bytes = json.dumps({**base_body, "messages": messages}).encode("utf-8")
|
||||
assert self.http_client is not None
|
||||
continuation_request = self.http_client.build_request(
|
||||
continuation_client = self._select_streaming_client(url)
|
||||
continuation_request = continuation_client.build_request(
|
||||
"POST", url, content=continuation_bytes, headers=headers
|
||||
)
|
||||
continuation = await self.http_client.send(continuation_request, stream=True)
|
||||
continuation = await continuation_client.send(continuation_request, stream=True)
|
||||
rounds += 1
|
||||
async with contextlib.aclosing(continuation) as upstream:
|
||||
if upstream.status_code >= 400:
|
||||
@@ -1415,6 +1428,7 @@ class StreamingMixin:
|
||||
# Open connection before generator to capture upstream response headers
|
||||
# (needed to forward ratelimit headers to the client via StreamingResponse)
|
||||
assert self.http_client is not None, "http_client must be initialized before streaming"
|
||||
streaming_client = self._select_streaming_client(url)
|
||||
try:
|
||||
retry_attempts = max(1, getattr(self.config, "retry_max_attempts", 3))
|
||||
upstream_response = None
|
||||
@@ -1422,10 +1436,10 @@ class StreamingMixin:
|
||||
|
||||
for attempt in range(retry_attempts):
|
||||
try:
|
||||
_upstream_req = self.http_client.build_request(
|
||||
_upstream_req = streaming_client.build_request(
|
||||
"POST", url, content=outbound_bytes, headers=outbound_headers
|
||||
)
|
||||
upstream_response = await self.http_client.send(_upstream_req, stream=True)
|
||||
upstream_response = await streaming_client.send(_upstream_req, stream=True)
|
||||
if _codex_wire_debug:
|
||||
capture_codex_wire_debug(
|
||||
"http_stream_upstream_response_headers",
|
||||
|
||||
@@ -2693,15 +2693,23 @@ class HeadroomProxy(
|
||||
if timeout is not None:
|
||||
post_kwargs["timeout"] = timeout
|
||||
|
||||
from headroom.proxy.handlers.openai import prefers_http1_passthrough
|
||||
|
||||
req_client = (
|
||||
getattr(self, "http_client_h1", None) or self.http_client
|
||||
if prefers_http1_passthrough(url)
|
||||
else self.http_client
|
||||
)
|
||||
|
||||
for attempt in range(self.config.retry_max_attempts):
|
||||
try:
|
||||
if stream:
|
||||
# For streaming, we return early - retry happens at higher level
|
||||
return await self.http_client.post( # type: ignore[union-attr]
|
||||
return await req_client.post( # type: ignore[union-attr]
|
||||
url, **post_kwargs
|
||||
)
|
||||
else:
|
||||
response = await self.http_client.post( # type: ignore[union-attr]
|
||||
response = await req_client.post( # type: ignore[union-attr]
|
||||
url, **post_kwargs
|
||||
)
|
||||
|
||||
|
||||
@@ -54,6 +54,37 @@ def test_handle_model_metadata_endpoint_returns_chatgpt_response_when_present(mo
|
||||
assert response.json() == {"client": "h2", "upstream_path": "/backend-api/models"}
|
||||
|
||||
|
||||
def test_handle_model_metadata_endpoint_prefers_http1_client(monkeypatch) -> None:
|
||||
async def fake_chatgpt_metadata(
|
||||
http_client,
|
||||
request: Request,
|
||||
upstream_path: str,
|
||||
) -> Response:
|
||||
return JSONResponse({"client": http_client, "upstream_path": upstream_path})
|
||||
|
||||
monkeypatch.setattr(
|
||||
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
||||
fake_chatgpt_metadata,
|
||||
)
|
||||
proxy = type("Proxy", (), {"http_client": "h2", "http_client_h1": "h1"})()
|
||||
app = FastAPI()
|
||||
|
||||
@app.get("/probe")
|
||||
async def probe(request: Request):
|
||||
return await handle_model_metadata_endpoint(
|
||||
proxy,
|
||||
request,
|
||||
endpoint=MODEL_METADATA_LIST_ENDPOINT,
|
||||
provider_api_base_url="https://api.openai.test",
|
||||
provider_name="openai",
|
||||
)
|
||||
|
||||
with TestClient(app) as client:
|
||||
response = client.get("/probe")
|
||||
|
||||
assert response.json() == {"client": "h1", "upstream_path": "/backend-api/models"}
|
||||
|
||||
|
||||
def test_handle_model_metadata_endpoint_falls_back_to_selected_provider(monkeypatch) -> None:
|
||||
async def fake_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
|
||||
return None
|
||||
|
||||
@@ -757,6 +757,7 @@ def test_openai_response_subpath_aliases_and_chatgpt_auth_use_expected_targets(m
|
||||
with TestClient(_app()) as client:
|
||||
fake = FakeAsyncClient()
|
||||
client.app.state.proxy.http_client = fake
|
||||
client.app.state.proxy.http_client_h1 = fake
|
||||
assert client.get("/v1/codex/responses/items/resp_1").status_code == 200
|
||||
assert client.post("/backend-api/responses/items/resp_2").status_code == 200
|
||||
assert client.delete("/backend-api/codex/responses/items/resp_3").status_code == 200
|
||||
@@ -1071,6 +1072,7 @@ def test_v1_models_fetches_codex_registry_under_chatgpt_auth(monkeypatch) -> Non
|
||||
with TestClient(_app()) as client:
|
||||
fake_http_client = FakeAsyncClient()
|
||||
client.app.state.proxy.http_client = fake_http_client
|
||||
client.app.state.proxy.http_client_h1 = fake_http_client
|
||||
response = client.get(
|
||||
"/v1/models?client_version=0.135.0",
|
||||
headers={
|
||||
@@ -1194,6 +1196,7 @@ def test_v1_models_get_single_dynamic_under_chatgpt_auth() -> None:
|
||||
with TestClient(_app()) as client:
|
||||
fake_http_client = FakeAsyncClient()
|
||||
client.app.state.proxy.http_client = fake_http_client
|
||||
client.app.state.proxy.http_client_h1 = fake_http_client
|
||||
ok = client.get(
|
||||
"/v1/models/gpt-5.3-codex-spark",
|
||||
headers={
|
||||
|
||||
@@ -79,6 +79,7 @@ def test_codex_responses_subpath_aliases_delegate_to_passthrough():
|
||||
with TestClient(create_app(ProxyConfig())) as client:
|
||||
fake_http_client = FakeAsyncClient()
|
||||
client.app.state.proxy.http_client = fake_http_client
|
||||
client.app.state.proxy.http_client_h1 = fake_http_client
|
||||
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
|
||||
|
||||
pi_response = client.post(
|
||||
@@ -141,6 +142,7 @@ def test_codex_responses_subpath_passthrough_derives_chatgpt_routing_from_jwt(pa
|
||||
with TestClient(create_app(ProxyConfig())) as client:
|
||||
fake_http_client = FakeAsyncClient()
|
||||
client.app.state.proxy.http_client = fake_http_client
|
||||
client.app.state.proxy.http_client_h1 = fake_http_client
|
||||
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
|
||||
|
||||
response = client.post(
|
||||
@@ -192,6 +194,7 @@ def test_codex_model_metadata_fetches_codex_registry_for_chatgpt_auth(monkeypatc
|
||||
with TestClient(create_app(ProxyConfig())) as client:
|
||||
fake_http_client = FakeAsyncClient()
|
||||
client.app.state.proxy.http_client = fake_http_client
|
||||
client.app.state.proxy.http_client_h1 = fake_http_client
|
||||
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
|
||||
|
||||
list_response = client.get(
|
||||
|
||||
@@ -17,7 +17,9 @@ from headroom.proxy.handlers.openai import (
|
||||
_decode_openai_bearer_payload,
|
||||
_passthrough_usage_from_json,
|
||||
_prefers_http1_passthrough,
|
||||
prefers_http1_passthrough,
|
||||
)
|
||||
from headroom.proxy.handlers.streaming import StreamingMixin
|
||||
from headroom.proxy.helpers import (
|
||||
_headroom_bypass_enabled,
|
||||
is_third_party_model,
|
||||
@@ -778,6 +780,78 @@ def test_chatgpt_passthrough_falls_back_when_h1_client_missing() -> None:
|
||||
assert handler.http_client.calls == 1
|
||||
|
||||
|
||||
def test_streaming_client_selection() -> None:
|
||||
assert prefers_http1_passthrough("https://chatgpt.com/backend-api/codex/responses") is True
|
||||
assert prefers_http1_passthrough("https://api.openai.com/v1/chat/completions") is False
|
||||
|
||||
mixin = object.__new__(StreamingMixin)
|
||||
mixin.http_client = "h2"
|
||||
mixin.http_client_h1 = "h1"
|
||||
|
||||
assert mixin._select_streaming_client("https://chatgpt.com/backend-api/codex/responses") == "h1"
|
||||
assert mixin._select_streaming_client("https://api.openai.com/v1/chat/completions") == "h2"
|
||||
|
||||
mixin.http_client_h1 = None
|
||||
assert mixin._select_streaming_client("https://chatgpt.com/backend-api/codex/responses") == "h2"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retry_request_selects_http1_for_chatgpt(monkeypatch) -> None:
|
||||
calls = []
|
||||
|
||||
class DummyClient:
|
||||
def __init__(self, name: str):
|
||||
self.name = name
|
||||
|
||||
async def post(self, url, **kwargs):
|
||||
calls.append((self.name, url))
|
||||
return httpx.Response(200, json={"ok": True})
|
||||
|
||||
proxy = object.__new__(HeadroomProxy)
|
||||
proxy.config = type("Config", (), {"retry_max_attempts": 1, "retry_enabled": False})()
|
||||
proxy.http_client = DummyClient("h2")
|
||||
proxy.http_client_h1 = DummyClient("h1")
|
||||
|
||||
await proxy._retry_request("POST", "https://chatgpt.com/backend-api/codex/responses", {}, {})
|
||||
assert calls == [("h1", "https://chatgpt.com/backend-api/codex/responses")]
|
||||
|
||||
calls.clear()
|
||||
await proxy._retry_request("POST", "https://api.openai.com/v1/chat/completions", {}, {})
|
||||
assert calls == [("h2", "https://api.openai.com/v1/chat/completions")]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_proxy_routes_openai_responses_subpath_selects_http1(monkeypatch) -> None:
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from headroom.providers.openai_responses import OpenAIResponsesSubpathRoute
|
||||
from headroom.providers.proxy_routes import _register_openai_responses_subpath_route
|
||||
|
||||
received_clients = []
|
||||
|
||||
async def fake_subpath(http_client, request, sub_path):
|
||||
received_clients.append(http_client)
|
||||
return httpx.Response(200, json={"ok": True})
|
||||
|
||||
monkeypatch.setattr(
|
||||
"headroom.providers.proxy_routes.handle_chatgpt_codex_responses_subpath",
|
||||
fake_subpath,
|
||||
)
|
||||
|
||||
app = FastAPI()
|
||||
proxy = type("Proxy", (), {"http_client": "h2", "http_client_h1": "h1"})()
|
||||
spec = OpenAIResponsesSubpathRoute(
|
||||
path="/v1/responses/{sub_path:path}",
|
||||
methods=("GET", "POST"),
|
||||
)
|
||||
_register_openai_responses_subpath_route(app, proxy, spec)
|
||||
|
||||
client = TestClient(app)
|
||||
client.get("/v1/responses/items/resp_1")
|
||||
assert received_clients == ["h1"]
|
||||
|
||||
|
||||
def test_passthrough_usage_normalizes_vertex_usage_metadata() -> None:
|
||||
usage = _passthrough_usage_from_json(
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user