mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description Ensures that ChatGPT endpoints (`chatgpt.com`, including `/v1/responses` forwarded to `https://chatgpt.com/backend-api/codex/responses`, `/v1/models`, Codex subpaths, and retried requests) consistently utilize `http_client_h1` instead of multiplexing over the default HTTP/2 client. `server.py` already provisions `self.http_client_h1` specifically because ChatGPT's Cloudflare edge challenges HTTP/2 fingerprints on sensitive endpoints. When large context requests (e.g. 500KB - 800KB payload turns) are forwarded over HTTP/2 streaming, HTTP/2 connection/stream flow-control window stalls block the single multiplexed TCP connection in `httpcore`. When this occurs, all subsequent requests to ChatGPT hosts (including `/v1/models` and subsequent turn completions) become blocked waiting behind the stalled connection, eventually leading to proxy unresponsiveness and accumulated asyncio task queues. While `handle_passthrough` in `openai.py` and `openai_images.py` already selected `http_client_h1`, `StreamingMixin._stream_response_generator`, `handle_model_metadata_endpoint`, `_retry_request` in `server.py`, and Codex subpath routes in `proxy_routes.py` previously hardcoded `self.http_client`. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Expose `prefers_http1_passthrough` in `headroom/proxy/handlers/openai.py` (aliasing `_prefers_http1_passthrough`). - Add `_select_streaming_client` to `StreamingMixin` in `headroom/proxy/handlers/streaming.py` that selects `http_client_h1` when the target URL matches ChatGPT endpoints. - Update `_stream_response_generator` and `_compact_sse_stream` to use `_select_streaming_client`. - Update `_retry_request` in `headroom/proxy/server.py` to route ChatGPT target URLs through `http_client_h1`. - Update `_register_openai_responses_subpath_route` in `headroom/providers/proxy_routes.py` to forward through `http_client_h1` for `handle_chatgpt_codex_responses_subpath`. - In `headroom/providers/model_metadata.py`, pass `getattr(proxy, "http_client_h1", None) or proxy.http_client` to `handle_chatgpt_model_metadata`. - Add unit tests in `tests/test_proxy_handler_helpers.py` and `tests/test_provider_model_metadata.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py ======================== 76 passed, 1 warning in 8.24s ======================== uv run ruff check headroom/ tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS 27 (Darwin arm64), Python 3.13, Headroom Proxy v0.39.1 / v0.40.0, Codex CLI and OpenClaw Desktop client. - Exact command / steps: Sent 800 KB streaming turns to /v1/responses targeting https://chatgpt.com/backend-api/codex/responses while monitoring TCP socket states via lsof/netstat and triggering concurrent /v1/models calls. - Observed result: Streaming responses and model metadata resolve over HTTP/1.1 without multiplexed stream stall; Send-Q drains cleanly, turns finish in ~2.1s, and all unit tests pass. - Not tested: Third-party proprietary non-ChatGPT OAuth endpoints (e.g., enterprise SSO providers). ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: stable - Stable/default behavior changed: No (aligns streaming, retries, and subpath routes with existing `http_client_h1` passthrough policy for ChatGPT hosts) - Kill switch / disable path: Fallback to `self.http_client` when `http_client_h1` is unset - Unsafe override required: No - Qualification impact: None - Rollback path: Revert commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ### Maintainer current-main requalification Exact head `d6cee507ff0aae7f09705b2a5b9a8bd8264b137f`, upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. Updated against upstream/maine4a60ed7. Resolved the adjacent-helper conflict by retaining both the contributor's HTTP/1.1 compatibility alias and main's telemetry helper. HTTP/1.1 selection stays confined to ChatGPT hosts across streaming/continuations, retries, metadata and authenticated Responses subpaths; other upstreams retain their existing client. No contributor production logic was rewritten. All 136 focused helper/route/metadata tests pass. Three newer route fixtures initially mocked only the default client; mechanically assigned their same controlled response client to http_client_h1 too, preserving every forwarding assertion. Changed-file Ruff lint/format and whitespace checks pass. Existing macOS/Codex/OpenClaw live-stream evidence remains scoped; no new live ChatGPT stream or fresh native-build claim is made. Applied shared test-only Gemini streamed-request fixture repair for independently established main failures; all ten cases pass. Formal approval awaits exact-head hosted qualification. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. ### Maintainer current-main requalification Exact head `96dfae54717c13249f3bd9305cc6930030f2c85d`, upstream main `ecbaf86d78cf52d185fbffa306157b9a4cc1b906`. Maintainer test-only repair for hosted run42031595103: the newer Codex alias tests mocked only http_client, while this PR correctly routes authenticated ChatGPT requests through http_client_h1. Assigned the existing controlled fake to both client attributes, preserving every auth/forwarding assertion; no production logic changed. Main advanced during qualification; cleanly merged upstream/mainecbaf86d78and re-reviewed its dependency/workflow/docs delta. All 154 helper/route/metadata/Codex-alias/Gemini tests pass on the final tree; lint/format and whitespace checks pass. Existing runtime evidence remains explicitly scoped, with no new live ChatGPT or fresh native-build claim. Exact-head hosted checks are handled by maintainers. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com>
304 lines
11 KiB
Python
304 lines
11 KiB
Python
import base64
|
|
import json
|
|
|
|
import httpx
|
|
import pytest
|
|
from fastapi import WebSocket
|
|
from fastapi.responses import JSONResponse
|
|
from fastapi.testclient import TestClient
|
|
|
|
from headroom.proxy.server import HeadroomProxy, ProxyConfig, create_app
|
|
|
|
|
|
def _jwt(payload: dict) -> str:
|
|
header = {"alg": "none", "typ": "JWT"}
|
|
|
|
def encode(part: dict) -> str:
|
|
raw = json.dumps(part, separators=(",", ":")).encode("utf-8")
|
|
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
|
|
|
return f"{encode(header)}.{encode(payload)}."
|
|
|
|
|
|
def test_codex_responses_aliases_delegate_to_openai_handler(monkeypatch):
|
|
async def fake_handle(self, request): # type: ignore[no-untyped-def]
|
|
return JSONResponse({"ok": True, "path": request.url.path})
|
|
|
|
monkeypatch.setattr(HeadroomProxy, "handle_openai_responses", fake_handle)
|
|
|
|
with TestClient(create_app(ProxyConfig())) as client:
|
|
for path in (
|
|
"/v1/codex/responses",
|
|
"/backend-api/responses",
|
|
"/backend-api/codex/responses",
|
|
):
|
|
response = client.post(path, json={"model": "gpt-5.3-codex"})
|
|
assert response.status_code == 200
|
|
assert response.json() == {"ok": True, "path": path}
|
|
|
|
|
|
def test_codex_responses_websocket_aliases_delegate_to_openai_handler(monkeypatch):
|
|
seen_paths: list[str] = []
|
|
|
|
async def fake_handle_ws(self, websocket: WebSocket): # type: ignore[no-untyped-def]
|
|
seen_paths.append(websocket.url.path)
|
|
await websocket.accept()
|
|
await websocket.send_json({"ok": True, "path": websocket.url.path})
|
|
await websocket.close()
|
|
|
|
monkeypatch.setattr(HeadroomProxy, "handle_openai_responses_ws", fake_handle_ws)
|
|
|
|
with TestClient(create_app(ProxyConfig())) as client:
|
|
for path in (
|
|
"/v1/codex/responses",
|
|
"/backend-api/responses",
|
|
"/backend-api/codex/responses",
|
|
):
|
|
with client.websocket_connect(path) as websocket:
|
|
assert websocket.receive_json() == {"ok": True, "path": path}
|
|
|
|
assert seen_paths == [
|
|
"/v1/codex/responses",
|
|
"/backend-api/responses",
|
|
"/backend-api/codex/responses",
|
|
]
|
|
|
|
|
|
def test_codex_responses_subpath_aliases_delegate_to_passthrough():
|
|
class FakeAsyncClient:
|
|
def __init__(self) -> None:
|
|
self.calls: list[tuple[str, str]] = []
|
|
|
|
async def request(self, method, url, **_kwargs): # type: ignore[no-untyped-def]
|
|
self.calls.append((method, url))
|
|
return httpx.Response(200, json={"method": method, "url": url})
|
|
|
|
async def aclose(self) -> None:
|
|
return None
|
|
|
|
with TestClient(create_app(ProxyConfig())) as client:
|
|
fake_http_client = FakeAsyncClient()
|
|
client.app.state.proxy.http_client = fake_http_client
|
|
client.app.state.proxy.http_client_h1 = fake_http_client
|
|
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
|
|
|
|
pi_response = client.post(
|
|
"/v1/codex/responses/compact?trace=0",
|
|
json={"model": "gpt-5.3-codex"},
|
|
)
|
|
api_key_response = client.post(
|
|
"/backend-api/responses/compact?trace=1",
|
|
json={"model": "gpt-5.3-codex"},
|
|
)
|
|
chatgpt_response = client.post(
|
|
"/backend-api/codex/responses/compact?trace=2",
|
|
headers={"chatgpt-account-id": "acct_123"},
|
|
json={"model": "gpt-5.3-codex"},
|
|
)
|
|
|
|
assert pi_response.status_code == 200
|
|
assert api_key_response.status_code == 200
|
|
assert chatgpt_response.status_code == 200
|
|
assert fake_http_client.calls == [
|
|
("POST", "https://api.openai.test/v1/responses/compact?trace=0"),
|
|
("POST", "https://api.openai.test/v1/responses/compact?trace=1"),
|
|
("POST", "https://chatgpt.com/backend-api/codex/responses/compact?trace=2"),
|
|
]
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("path", "expected_url"),
|
|
[
|
|
(
|
|
"/v1/codex/responses/compact?trace=jwt",
|
|
"https://chatgpt.com/backend-api/codex/responses/compact?trace=jwt",
|
|
),
|
|
(
|
|
"/v1/responses/compact?trace=jwt-old",
|
|
"https://chatgpt.com/backend-api/codex/responses/compact?trace=jwt-old",
|
|
),
|
|
],
|
|
)
|
|
def test_codex_responses_subpath_passthrough_derives_chatgpt_routing_from_jwt(path, expected_url):
|
|
class FakeAsyncClient:
|
|
def __init__(self) -> None:
|
|
self.calls: list[tuple[str, str, dict[str, str]]] = []
|
|
|
|
async def request(self, method, url, **kwargs): # type: ignore[no-untyped-def]
|
|
self.calls.append((method, url, dict(kwargs.get("headers", {}))))
|
|
return httpx.Response(200, json={"method": method, "url": url})
|
|
|
|
async def aclose(self) -> None:
|
|
return None
|
|
|
|
token = _jwt(
|
|
{
|
|
"https://api.openai.com/auth": {
|
|
"chatgpt_account_id": "acct-from-jwt",
|
|
}
|
|
}
|
|
)
|
|
|
|
with TestClient(create_app(ProxyConfig())) as client:
|
|
fake_http_client = FakeAsyncClient()
|
|
client.app.state.proxy.http_client = fake_http_client
|
|
client.app.state.proxy.http_client_h1 = fake_http_client
|
|
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
|
|
|
|
response = client.post(
|
|
path,
|
|
headers={"Authorization": f"Bearer {token}"},
|
|
json={"model": "gpt-5.4"},
|
|
)
|
|
|
|
assert response.status_code == 200
|
|
assert len(fake_http_client.calls) == 1
|
|
|
|
method, url, headers = fake_http_client.calls[0]
|
|
assert method == "POST"
|
|
assert url == expected_url
|
|
assert headers["authorization"] == f"Bearer {token}"
|
|
assert headers["ChatGPT-Account-ID"] == "acct-from-jwt"
|
|
|
|
|
|
def test_codex_model_metadata_fetches_codex_registry_for_chatgpt_auth(monkeypatch):
|
|
"""Issue #478: under Codex ChatGPT-subscription OAuth, the proxy
|
|
must NOT forward `/v1/models[/{id}]` to chatgpt.com/backend-api —
|
|
that endpoint returns 403 to OAuth tokens. Instead, Headroom should
|
|
fetch the Codex-specific model registry and synthesize an
|
|
OpenAI-compatible payload from its slugs.
|
|
"""
|
|
|
|
class FakeAsyncClient:
|
|
def __init__(self):
|
|
self.calls: list[tuple[str, str, dict[str, str]]] = []
|
|
|
|
async def get(self, url, **kwargs): # type: ignore[no-untyped-def]
|
|
self.calls.append(("GET", url, dict(kwargs.get("headers", {}))))
|
|
return httpx.Response(
|
|
200,
|
|
json={"models": [{"slug": "gpt-5.5"}, {"slug": "gpt-5.3-codex-spark"}]},
|
|
)
|
|
|
|
async def aclose(self):
|
|
return None
|
|
|
|
token = _jwt(
|
|
{
|
|
"https://api.openai.com/auth": {
|
|
"chatgpt_account_id": "acct-from-jwt",
|
|
}
|
|
}
|
|
)
|
|
|
|
with TestClient(create_app(ProxyConfig())) as client:
|
|
fake_http_client = FakeAsyncClient()
|
|
client.app.state.proxy.http_client = fake_http_client
|
|
client.app.state.proxy.http_client_h1 = fake_http_client
|
|
client.app.state.proxy.OPENAI_API_URL = "https://api.openai.test"
|
|
|
|
list_response = client.get(
|
|
"/v1/models?client_version=0.130.0",
|
|
headers={"Authorization": f"Bearer {token}"},
|
|
)
|
|
known_response = client.get(
|
|
"/v1/models/gpt-5.5",
|
|
headers={"Authorization": f"Bearer {token}"},
|
|
)
|
|
unknown_response = client.get(
|
|
"/v1/models/gpt-99-future?client_version=0.130.0",
|
|
headers={"Authorization": f"Bearer {token}"},
|
|
)
|
|
|
|
assert len(fake_http_client.calls) == 3
|
|
for method, url, headers in fake_http_client.calls:
|
|
assert method == "GET"
|
|
assert url == "https://chatgpt.com/backend-api/codex/models?client_version=0.130.0"
|
|
assert headers["authorization"] == f"Bearer {token}"
|
|
assert headers["chatgpt-account-id"] == "acct-from-jwt"
|
|
assert headers["accept"] == "application/json"
|
|
assert "ChatGPT-Account-ID" not in headers
|
|
assert "Accept" not in headers
|
|
|
|
# List endpoint returns a non-empty OpenAI-compatible list.
|
|
assert list_response.status_code == 200
|
|
list_payload = list_response.json()
|
|
assert list_payload["object"] == "list"
|
|
assert isinstance(list_payload["data"], list)
|
|
assert len(list_payload["data"]) > 0
|
|
assert {entry["id"] for entry in list_payload["data"]} == {
|
|
"gpt-5.5",
|
|
"gpt-5.3-codex-spark",
|
|
}
|
|
assert {entry["slug"] for entry in list_payload["models"]} == {
|
|
"gpt-5.5",
|
|
"gpt-5.3-codex-spark",
|
|
}
|
|
for entry in list_payload["models"]:
|
|
assert entry["display_name"]
|
|
assert entry["default_reasoning_level"] == "medium"
|
|
assert entry["supports_parallel_tool_calls"] is True
|
|
|
|
# Single-model GET returns a model object when known.
|
|
assert known_response.status_code == 200
|
|
known_payload = known_response.json()
|
|
assert known_payload == {
|
|
"id": "gpt-5.5",
|
|
"object": "model",
|
|
"created": 0,
|
|
"owned_by": "openai",
|
|
}
|
|
|
|
# Unknown model variants 404 against the dynamic registry.
|
|
assert unknown_response.status_code == 404
|
|
|
|
|
|
_CODEX_DESKTOP_UA = (
|
|
"Codex Desktop/0.140.0-alpha.2 (Mac OS 15.7.7; arm64) unknown (Codex Desktop; 26.609.71450)"
|
|
)
|
|
|
|
|
|
def test_responses_middleware_stamps_x_client_codex_for_unidentified_caller(monkeypatch):
|
|
# Codex Desktop's User-Agent isn't a known codex UA, so the HTTP middleware
|
|
# must stamp X-Client: codex on /v1/responses before the handler classifies
|
|
# the caller — otherwise a compression timeout is refused with a 413 that
|
|
# Codex treats as a hard connection failure.
|
|
seen: dict[str, str | None] = {}
|
|
|
|
async def fake_handle(self, request): # type: ignore[no-untyped-def]
|
|
seen["x-client"] = request.headers.get("x-client")
|
|
return JSONResponse({"ok": True})
|
|
|
|
monkeypatch.setattr(HeadroomProxy, "handle_openai_responses", fake_handle)
|
|
|
|
with TestClient(create_app(ProxyConfig())) as client:
|
|
response = client.post(
|
|
"/v1/responses",
|
|
headers={"user-agent": _CODEX_DESKTOP_UA},
|
|
json={"model": "gpt-5.3-codex"},
|
|
)
|
|
|
|
assert response.status_code == 200
|
|
assert seen["x-client"] == "codex"
|
|
|
|
|
|
def test_responses_middleware_preserves_explicit_x_client(monkeypatch):
|
|
# A caller that already self-identifies is left untouched by the stamp.
|
|
seen: dict[str, str | None] = {}
|
|
|
|
async def fake_handle(self, request): # type: ignore[no-untyped-def]
|
|
seen["x-client"] = request.headers.get("x-client")
|
|
return JSONResponse({"ok": True})
|
|
|
|
monkeypatch.setattr(HeadroomProxy, "handle_openai_responses", fake_handle)
|
|
|
|
with TestClient(create_app(ProxyConfig())) as client:
|
|
response = client.post(
|
|
"/v1/responses",
|
|
headers={"x-client": "aider", "user-agent": _CODEX_DESKTOP_UA},
|
|
json={"model": "gpt-5.3-codex"},
|
|
)
|
|
|
|
assert response.status_code == 200
|
|
assert seen["x-client"] == "aider"
|