fix(proxy): use HTTP/1.1 client for ChatGPT streaming and model metadata (#4006)

## Description

Ensures that ChatGPT endpoints (`chatgpt.com`, including `/v1/responses`
forwarded to `https://chatgpt.com/backend-api/codex/responses`,
`/v1/models`, Codex subpaths, and retried requests) consistently utilize
`http_client_h1` instead of multiplexing over the default HTTP/2 client.

`server.py` already provisions `self.http_client_h1` specifically
because ChatGPT's Cloudflare edge challenges HTTP/2 fingerprints on
sensitive endpoints. When large context requests (e.g. 500KB - 800KB
payload turns) are forwarded over HTTP/2 streaming, HTTP/2
connection/stream flow-control window stalls block the single
multiplexed TCP connection in `httpcore`. When this occurs, all
subsequent requests to ChatGPT hosts (including `/v1/models` and
subsequent turn completions) become blocked waiting behind the stalled
connection, eventually leading to proxy unresponsiveness and accumulated
asyncio task queues.

While `handle_passthrough` in `openai.py` and `openai_images.py` already
selected `http_client_h1`, `StreamingMixin._stream_response_generator`,
`handle_model_metadata_endpoint`, `_retry_request` in `server.py`, and
Codex subpath routes in `proxy_routes.py` previously hardcoded
`self.http_client`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Expose `prefers_http1_passthrough` in
`headroom/proxy/handlers/openai.py` (aliasing
`_prefers_http1_passthrough`).
- Add `_select_streaming_client` to `StreamingMixin` in
`headroom/proxy/handlers/streaming.py` that selects `http_client_h1`
when the target URL matches ChatGPT endpoints.
- Update `_stream_response_generator` and `_compact_sse_stream` to use
`_select_streaming_client`.
- Update `_retry_request` in `headroom/proxy/server.py` to route ChatGPT
target URLs through `http_client_h1`.
- Update `_register_openai_responses_subpath_route` in
`headroom/providers/proxy_routes.py` to forward through `http_client_h1`
for `handle_chatgpt_codex_responses_subpath`.
- In `headroom/providers/model_metadata.py`, pass `getattr(proxy,
"http_client_h1", None) or proxy.http_client` to
`handle_chatgpt_model_metadata`.
- Add unit tests in `tests/test_proxy_handler_helpers.py` and
`tests/test_provider_model_metadata.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run pytest tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py
======================== 76 passed, 1 warning in 8.24s ========================

uv run ruff check headroom/ tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py
All checks passed!
```

## Real Behavior Proof

- Environment: macOS 27 (Darwin arm64), Python 3.13, Headroom Proxy
v0.39.1 / v0.40.0, Codex CLI and OpenClaw Desktop client.
- Exact command / steps: Sent 800 KB streaming turns to /v1/responses
targeting https://chatgpt.com/backend-api/codex/responses while
monitoring TCP socket states via lsof/netstat and triggering concurrent
/v1/models calls.
- Observed result: Streaming responses and model metadata resolve over
HTTP/1.1 without multiplexed stream stall; Send-Q drains cleanly, turns
finish in ~2.1s, and all unit tests pass.
- Not tested: Third-party proprietary non-ChatGPT OAuth endpoints (e.g.,
enterprise SSO providers).

## Runtime Rollout Safety

- Rollout-managed feature(s): None
- Minimum rollout channel: stable
- Stable/default behavior changed: No (aligns streaming, retries, and
subpath routes with existing `http_client_h1` passthrough policy for
ChatGPT hosts)
- Kill switch / disable path: Fallback to `self.http_client` when
`http_client_h1` is unset
- Unsafe override required: No
- Qualification impact: None
- Rollback path: Revert commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)


### Maintainer current-main requalification

Exact head `d6cee507ff0aae7f09705b2a5b9a8bd8264b137f`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Updated against upstream/main e4a60ed7. Resolved the adjacent-helper
conflict by retaining both the contributor's HTTP/1.1 compatibility
alias and main's telemetry helper. HTTP/1.1 selection stays confined to
ChatGPT hosts across streaming/continuations, retries, metadata and
authenticated Responses subpaths; other upstreams retain their existing
client. No contributor production logic was rewritten.

All 136 focused helper/route/metadata tests pass. Three newer route
fixtures initially mocked only the default client; mechanically assigned
their same controlled response client to http_client_h1 too, preserving
every forwarding assertion. Changed-file Ruff lint/format and whitespace
checks pass. Existing macOS/Codex/OpenClaw live-stream evidence remains
scoped; no new live ChatGPT stream or fresh native-build claim is made.
Applied shared test-only Gemini streamed-request fixture repair for
independently established main failures; all ten cases pass. Formal
approval awaits exact-head hosted qualification.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `96dfae54717c13249f3bd9305cc6930030f2c85d`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Maintainer test-only repair for hosted run42031595103: the newer Codex
alias tests mocked only http_client, while this PR correctly routes
authenticated ChatGPT requests through http_client_h1. Assigned the
existing controlled fake to both client attributes, preserving every
auth/forwarding assertion; no production logic changed.

Main advanced during qualification; cleanly merged upstream/main
ecbaf86d78 and re-reviewed its
dependency/workflow/docs delta. All 154
helper/route/metadata/Codex-alias/Gemini tests pass on the final tree;
lint/format and whitespace checks pass. Existing runtime evidence
remains explicitly scoped, with no new live ChatGPT or fresh
native-build claim. Exact-head hosted checks are handled by maintainers.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
pokemonred06-creator
2026-11-05 10:51:47 -06:00
committed by GitHub
co-authored by JD Davis
parent 398d7cdcf4
commit e1becdf732
9 changed files with 151 additions and 12 deletions
+2 -1
View File
@@ -51,8 +51,9 @@ async def handle_model_metadata_endpoint(
) -> Response:
"""Handle OpenAI-compatible model metadata with Codex ChatGPT-auth support."""
assert proxy.http_client is not None
chatgpt_client = getattr(proxy, "http_client_h1", None) or proxy.http_client
chatgpt_response = await handle_chatgpt_model_metadata(
proxy.http_client,
chatgpt_client,
request,
endpoint.upstream_path,
)
+2 -2
View File
@@ -212,9 +212,9 @@ def _register_openai_responses_subpath_route(
spec: OpenAIResponsesSubpathRoute,
) -> None:
async def openai_responses_subpath(request: Request, sub_path: str):
assert proxy.http_client is not None
codex_subpath_client = getattr(proxy, "http_client_h1", None) or proxy.http_client
chatgpt_response = await handle_chatgpt_codex_responses_subpath(
proxy.http_client,
codex_subpath_client,
request,
sub_path,
)
+8 -3
View File
@@ -2074,16 +2074,21 @@ def _extract_responses_usage(event: dict[str, Any]) -> tuple[int, int, int, int,
return input_tokens, output_tokens, cached, write, uncached, inferred
def _prefers_http1_passthrough(base_url: str) -> bool:
"""Whether passthrough to this host must use HTTP/1.1.
def prefers_http1_passthrough(base_url: str) -> bool:
"""Whether passthrough/streaming to this host must use HTTP/1.1.
ChatGPT's Cloudflare edge issues a managed challenge to our HTTP/2
fingerprint on sensitive account endpoints; HTTP/1.1 is accepted.
fingerprint on sensitive account endpoints, and multiplexing large
streaming request bodies over HTTP/2 can stall on flow-control windows.
HTTP/1.1 avoids both issues.
"""
host = (urlparse(base_url).hostname or "").lower()
return host == "chatgpt.com" or host.endswith(".chatgpt.com")
_prefers_http1_passthrough = prefers_http1_passthrough
def _openai_outcome_provider(
method: str, path: str, custom_base_url: str | None, upstream_base_url: str
) -> str:
+18 -4
View File
@@ -219,6 +219,18 @@ class StreamingMixin:
_mid_turn_queues: dict[str, asyncio.Queue] = {}
_active_streams: set[str] = set()
def _select_streaming_client(self, url: str) -> httpx.AsyncClient:
"""Select HTTP client for upstream streaming.
ChatGPT endpoints prefer HTTP/1.1 to avoid Cloudflare bot-management
challenges and HTTP/2 flow-control window stalls on large turns.
"""
from headroom.proxy.handlers.openai import prefers_http1_passthrough
if prefers_http1_passthrough(url):
return getattr(self, "http_client_h1", None) or self.http_client
return self.http_client
@staticmethod
def _get_session_key(body: dict, session_header: str | None = None) -> str:
"""Return session identity from an explicit header or a body-derived hash.
@@ -697,10 +709,11 @@ class StreamingMixin:
]
continuation_bytes = json.dumps({**base_body, "messages": messages}).encode("utf-8")
assert self.http_client is not None
continuation_request = self.http_client.build_request(
continuation_client = self._select_streaming_client(url)
continuation_request = continuation_client.build_request(
"POST", url, content=continuation_bytes, headers=headers
)
continuation = await self.http_client.send(continuation_request, stream=True)
continuation = await continuation_client.send(continuation_request, stream=True)
rounds += 1
async with contextlib.aclosing(continuation) as upstream:
if upstream.status_code >= 400:
@@ -1415,6 +1428,7 @@ class StreamingMixin:
# Open connection before generator to capture upstream response headers
# (needed to forward ratelimit headers to the client via StreamingResponse)
assert self.http_client is not None, "http_client must be initialized before streaming"
streaming_client = self._select_streaming_client(url)
try:
retry_attempts = max(1, getattr(self.config, "retry_max_attempts", 3))
upstream_response = None
@@ -1422,10 +1436,10 @@ class StreamingMixin:
for attempt in range(retry_attempts):
try:
_upstream_req = self.http_client.build_request(
_upstream_req = streaming_client.build_request(
"POST", url, content=outbound_bytes, headers=outbound_headers
)
upstream_response = await self.http_client.send(_upstream_req, stream=True)
upstream_response = await streaming_client.send(_upstream_req, stream=True)
if _codex_wire_debug:
capture_codex_wire_debug(
"http_stream_upstream_response_headers",
+10 -2
View File
@@ -2693,15 +2693,23 @@ class HeadroomProxy(
if timeout is not None:
post_kwargs["timeout"] = timeout
from headroom.proxy.handlers.openai import prefers_http1_passthrough
req_client = (
getattr(self, "http_client_h1", None) or self.http_client
if prefers_http1_passthrough(url)
else self.http_client
)
for attempt in range(self.config.retry_max_attempts):
try:
if stream:
# For streaming, we return early - retry happens at higher level
return await self.http_client.post( # type: ignore[union-attr]
return await req_client.post( # type: ignore[union-attr]
url, **post_kwargs
)
else:
response = await self.http_client.post( # type: ignore[union-attr]
response = await req_client.post( # type: ignore[union-attr]
url, **post_kwargs
)
+31
View File
@@ -54,6 +54,37 @@ def test_handle_model_metadata_endpoint_returns_chatgpt_response_when_present(mo
assert response.json() == {"client": "h2", "upstream_path": "/backend-api/models"}
def test_handle_model_metadata_endpoint_prefers_http1_client(monkeypatch) -> None:
async def fake_chatgpt_metadata(
http_client,
request: Request,
upstream_path: str,
) -> Response:
return JSONResponse({"client": http_client, "upstream_path": upstream_path})
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
fake_chatgpt_metadata,
)
proxy = type("Proxy", (), {"http_client": "h2", "http_client_h1": "h1"})()
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
proxy,
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.openai.test",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.json() == {"client": "h1", "upstream_path": "/backend-api/models"}
def test_handle_model_metadata_endpoint_falls_back_to_selected_provider(monkeypatch) -> None:
async def fake_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
+3
View File
@@ -757,6 +757,7 @@ def test_openai_response_subpath_aliases_and_chatgpt_auth_use_expected_targets(m
with TestClient(_app()) as client:
fake = FakeAsyncClient()
client.app.state.proxy.http_client = fake
client.app.state.proxy.http_client_h1 = fake
assert client.get("/v1/codex/responses/items/resp_1").status_code == 200
assert client.post("/backend-api/responses/items/resp_2").status_code == 200
assert client.delete("/backend-api/codex/responses/items/resp_3").status_code == 200
@@ -1071,6 +1072,7 @@ def test_v1_models_fetches_codex_registry_under_chatgpt_auth(monkeypatch) -> Non
with TestClient(_app()) as client:
fake_http_client = FakeAsyncClient()
client.app.state.proxy.http_client = fake_http_client
client.app.state.proxy.http_client_h1 = fake_http_client
response = client.get(
"/v1/models?client_version=0.135.0",
headers={
@@ -1194,6 +1196,7 @@ def test_v1_models_get_single_dynamic_under_chatgpt_auth() -> None:
with TestClient(_app()) as client:
fake_http_client = FakeAsyncClient()
client.app.state.proxy.http_client = fake_http_client
client.app.state.proxy.http_client_h1 = fake_http_client
ok = client.get(
"/v1/models/gpt-5.3-codex-spark",
headers={
+3
View File
@@ -79,6 +79,7 @@ def test_codex_responses_subpath_aliases_delegate_to_passthrough():
with TestClient(create_app(ProxyConfig())) as client:
fake_http_client = FakeAsyncClient()
client.app.state.proxy.http_client = fake_http_client
client.app.state.proxy.http_client_h1 = fake_http_client
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
pi_response = client.post(
@@ -141,6 +142,7 @@ def test_codex_responses_subpath_passthrough_derives_chatgpt_routing_from_jwt(pa
with TestClient(create_app(ProxyConfig())) as client:
fake_http_client = FakeAsyncClient()
client.app.state.proxy.http_client = fake_http_client
client.app.state.proxy.http_client_h1 = fake_http_client
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
response = client.post(
@@ -192,6 +194,7 @@ def test_codex_model_metadata_fetches_codex_registry_for_chatgpt_auth(monkeypatc
with TestClient(create_app(ProxyConfig())) as client:
fake_http_client = FakeAsyncClient()
client.app.state.proxy.http_client = fake_http_client
client.app.state.proxy.http_client_h1 = fake_http_client
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
list_response = client.get(
+74
View File
@@ -17,7 +17,9 @@ from headroom.proxy.handlers.openai import (
_decode_openai_bearer_payload,
_passthrough_usage_from_json,
_prefers_http1_passthrough,
prefers_http1_passthrough,
)
from headroom.proxy.handlers.streaming import StreamingMixin
from headroom.proxy.helpers import (
_headroom_bypass_enabled,
is_third_party_model,
@@ -778,6 +780,78 @@ def test_chatgpt_passthrough_falls_back_when_h1_client_missing() -> None:
assert handler.http_client.calls == 1
def test_streaming_client_selection() -> None:
assert prefers_http1_passthrough("https://chatgpt.com/backend-api/codex/responses") is True
assert prefers_http1_passthrough("https://api.openai.com/v1/chat/completions") is False
mixin = object.__new__(StreamingMixin)
mixin.http_client = "h2"
mixin.http_client_h1 = "h1"
assert mixin._select_streaming_client("https://chatgpt.com/backend-api/codex/responses") == "h1"
assert mixin._select_streaming_client("https://api.openai.com/v1/chat/completions") == "h2"
mixin.http_client_h1 = None
assert mixin._select_streaming_client("https://chatgpt.com/backend-api/codex/responses") == "h2"
@pytest.mark.asyncio
async def test_retry_request_selects_http1_for_chatgpt(monkeypatch) -> None:
calls = []
class DummyClient:
def __init__(self, name: str):
self.name = name
async def post(self, url, **kwargs):
calls.append((self.name, url))
return httpx.Response(200, json={"ok": True})
proxy = object.__new__(HeadroomProxy)
proxy.config = type("Config", (), {"retry_max_attempts": 1, "retry_enabled": False})()
proxy.http_client = DummyClient("h2")
proxy.http_client_h1 = DummyClient("h1")
await proxy._retry_request("POST", "https://chatgpt.com/backend-api/codex/responses", {}, {})
assert calls == [("h1", "https://chatgpt.com/backend-api/codex/responses")]
calls.clear()
await proxy._retry_request("POST", "https://api.openai.com/v1/chat/completions", {}, {})
assert calls == [("h2", "https://api.openai.com/v1/chat/completions")]
@pytest.mark.asyncio
async def test_proxy_routes_openai_responses_subpath_selects_http1(monkeypatch) -> None:
from fastapi import FastAPI
from fastapi.testclient import TestClient
from headroom.providers.openai_responses import OpenAIResponsesSubpathRoute
from headroom.providers.proxy_routes import _register_openai_responses_subpath_route
received_clients = []
async def fake_subpath(http_client, request, sub_path):
received_clients.append(http_client)
return httpx.Response(200, json={"ok": True})
monkeypatch.setattr(
"headroom.providers.proxy_routes.handle_chatgpt_codex_responses_subpath",
fake_subpath,
)
app = FastAPI()
proxy = type("Proxy", (), {"http_client": "h2", "http_client_h1": "h1"})()
spec = OpenAIResponsesSubpathRoute(
path="/v1/responses/{sub_path:path}",
methods=("GET", "POST"),
)
_register_openai_responses_subpath_route(app, proxy, spec)
client = TestClient(app)
client.get("/v1/responses/items/resp_1")
assert received_clients == ["h1"]
def test_passthrough_usage_normalizes_vertex_usage_metadata() -> None:
usage = _passthrough_usage_from_json(
{