mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description Ensures that ChatGPT endpoints (`chatgpt.com`, including `/v1/responses` forwarded to `https://chatgpt.com/backend-api/codex/responses`, `/v1/models`, Codex subpaths, and retried requests) consistently utilize `http_client_h1` instead of multiplexing over the default HTTP/2 client. `server.py` already provisions `self.http_client_h1` specifically because ChatGPT's Cloudflare edge challenges HTTP/2 fingerprints on sensitive endpoints. When large context requests (e.g. 500KB - 800KB payload turns) are forwarded over HTTP/2 streaming, HTTP/2 connection/stream flow-control window stalls block the single multiplexed TCP connection in `httpcore`. When this occurs, all subsequent requests to ChatGPT hosts (including `/v1/models` and subsequent turn completions) become blocked waiting behind the stalled connection, eventually leading to proxy unresponsiveness and accumulated asyncio task queues. While `handle_passthrough` in `openai.py` and `openai_images.py` already selected `http_client_h1`, `StreamingMixin._stream_response_generator`, `handle_model_metadata_endpoint`, `_retry_request` in `server.py`, and Codex subpath routes in `proxy_routes.py` previously hardcoded `self.http_client`. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Expose `prefers_http1_passthrough` in `headroom/proxy/handlers/openai.py` (aliasing `_prefers_http1_passthrough`). - Add `_select_streaming_client` to `StreamingMixin` in `headroom/proxy/handlers/streaming.py` that selects `http_client_h1` when the target URL matches ChatGPT endpoints. - Update `_stream_response_generator` and `_compact_sse_stream` to use `_select_streaming_client`. - Update `_retry_request` in `headroom/proxy/server.py` to route ChatGPT target URLs through `http_client_h1`. - Update `_register_openai_responses_subpath_route` in `headroom/providers/proxy_routes.py` to forward through `http_client_h1` for `handle_chatgpt_codex_responses_subpath`. - In `headroom/providers/model_metadata.py`, pass `getattr(proxy, "http_client_h1", None) or proxy.http_client` to `handle_chatgpt_model_metadata`. - Add unit tests in `tests/test_proxy_handler_helpers.py` and `tests/test_provider_model_metadata.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py ======================== 76 passed, 1 warning in 8.24s ======================== uv run ruff check headroom/ tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS 27 (Darwin arm64), Python 3.13, Headroom Proxy v0.39.1 / v0.40.0, Codex CLI and OpenClaw Desktop client. - Exact command / steps: Sent 800 KB streaming turns to /v1/responses targeting https://chatgpt.com/backend-api/codex/responses while monitoring TCP socket states via lsof/netstat and triggering concurrent /v1/models calls. - Observed result: Streaming responses and model metadata resolve over HTTP/1.1 without multiplexed stream stall; Send-Q drains cleanly, turns finish in ~2.1s, and all unit tests pass. - Not tested: Third-party proprietary non-ChatGPT OAuth endpoints (e.g., enterprise SSO providers). ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: stable - Stable/default behavior changed: No (aligns streaming, retries, and subpath routes with existing `http_client_h1` passthrough policy for ChatGPT hosts) - Kill switch / disable path: Fallback to `self.http_client` when `http_client_h1` is unset - Unsafe override required: No - Qualification impact: None - Rollback path: Revert commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ### Maintainer current-main requalification Exact head `d6cee507ff0aae7f09705b2a5b9a8bd8264b137f`, upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. Updated against upstream/maine4a60ed7. Resolved the adjacent-helper conflict by retaining both the contributor's HTTP/1.1 compatibility alias and main's telemetry helper. HTTP/1.1 selection stays confined to ChatGPT hosts across streaming/continuations, retries, metadata and authenticated Responses subpaths; other upstreams retain their existing client. No contributor production logic was rewritten. All 136 focused helper/route/metadata tests pass. Three newer route fixtures initially mocked only the default client; mechanically assigned their same controlled response client to http_client_h1 too, preserving every forwarding assertion. Changed-file Ruff lint/format and whitespace checks pass. Existing macOS/Codex/OpenClaw live-stream evidence remains scoped; no new live ChatGPT stream or fresh native-build claim is made. Applied shared test-only Gemini streamed-request fixture repair for independently established main failures; all ten cases pass. Formal approval awaits exact-head hosted qualification. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. ### Maintainer current-main requalification Exact head `96dfae54717c13249f3bd9305cc6930030f2c85d`, upstream main `ecbaf86d78cf52d185fbffa306157b9a4cc1b906`. Maintainer test-only repair for hosted run42031595103: the newer Codex alias tests mocked only http_client, while this PR correctly routes authenticated ChatGPT requests through http_client_h1. Assigned the existing controlled fake to both client attributes, preserving every auth/forwarding assertion; no production logic changed. Main advanced during qualification; cleanly merged upstream/mainecbaf86d78and re-reviewed its dependency/workflow/docs delta. All 154 helper/route/metadata/Codex-alias/Gemini tests pass on the final tree; lint/format and whitespace checks pass. Existing runtime evidence remains explicitly scoped, with no new live ChatGPT or fresh native-build claim. Exact-head hosted checks are handled by maintainers. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com>
378 lines
13 KiB
Python
378 lines
13 KiB
Python
from __future__ import annotations
|
|
|
|
from fastapi import FastAPI, Request
|
|
from fastapi.responses import JSONResponse, Response
|
|
from fastapi.testclient import TestClient
|
|
|
|
from headroom.providers.model_metadata import (
|
|
MODEL_METADATA_LIST_ENDPOINT,
|
|
ModelMetadataEndpoint,
|
|
handle_model_metadata_endpoint,
|
|
model_metadata_get_endpoint,
|
|
)
|
|
|
|
|
|
def test_model_metadata_endpoints_are_explicit() -> None:
|
|
assert MODEL_METADATA_LIST_ENDPOINT == ModelMetadataEndpoint(
|
|
"/v1/models",
|
|
"/backend-api/models",
|
|
)
|
|
assert model_metadata_get_endpoint("gpt-5") == ModelMetadataEndpoint(
|
|
"/v1/models/{model_id}",
|
|
"/backend-api/models/gpt-5",
|
|
)
|
|
|
|
|
|
def test_handle_model_metadata_endpoint_returns_chatgpt_response_when_present(monkeypatch) -> None:
|
|
async def fake_chatgpt_metadata(
|
|
http_client,
|
|
request: Request,
|
|
upstream_path: str,
|
|
) -> Response:
|
|
return JSONResponse({"client": http_client, "upstream_path": upstream_path})
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
fake_chatgpt_metadata,
|
|
)
|
|
proxy = type("Proxy", (), {"http_client": "h2"})()
|
|
app = FastAPI()
|
|
|
|
@app.get("/probe")
|
|
async def probe(request: Request):
|
|
return await handle_model_metadata_endpoint(
|
|
proxy,
|
|
request,
|
|
endpoint=MODEL_METADATA_LIST_ENDPOINT,
|
|
provider_api_base_url="https://api.openai.test",
|
|
provider_name="openai",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
response = client.get("/probe")
|
|
|
|
assert response.json() == {"client": "h2", "upstream_path": "/backend-api/models"}
|
|
|
|
|
|
def test_handle_model_metadata_endpoint_prefers_http1_client(monkeypatch) -> None:
|
|
async def fake_chatgpt_metadata(
|
|
http_client,
|
|
request: Request,
|
|
upstream_path: str,
|
|
) -> Response:
|
|
return JSONResponse({"client": http_client, "upstream_path": upstream_path})
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
fake_chatgpt_metadata,
|
|
)
|
|
proxy = type("Proxy", (), {"http_client": "h2", "http_client_h1": "h1"})()
|
|
app = FastAPI()
|
|
|
|
@app.get("/probe")
|
|
async def probe(request: Request):
|
|
return await handle_model_metadata_endpoint(
|
|
proxy,
|
|
request,
|
|
endpoint=MODEL_METADATA_LIST_ENDPOINT,
|
|
provider_api_base_url="https://api.openai.test",
|
|
provider_name="openai",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
response = client.get("/probe")
|
|
|
|
assert response.json() == {"client": "h1", "upstream_path": "/backend-api/models"}
|
|
|
|
|
|
def test_handle_model_metadata_endpoint_falls_back_to_selected_provider(monkeypatch) -> None:
|
|
async def fake_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
|
|
return None
|
|
|
|
calls: list[tuple[str, str, str]] = []
|
|
|
|
class Proxy:
|
|
http_client = "h2"
|
|
|
|
async def handle_passthrough(
|
|
self,
|
|
request: Request,
|
|
base_url: str,
|
|
sub_path: str = "",
|
|
provider_name: str = "",
|
|
) -> Response:
|
|
calls.append((base_url, sub_path, provider_name))
|
|
return JSONResponse({"provider": provider_name, "sub_path": sub_path})
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
fake_chatgpt_metadata,
|
|
)
|
|
app = FastAPI()
|
|
|
|
@app.get("/probe")
|
|
async def probe(request: Request):
|
|
return await handle_model_metadata_endpoint(
|
|
Proxy(),
|
|
request,
|
|
endpoint=model_metadata_get_endpoint("claude-opus"),
|
|
provider_api_base_url="https://api.anthropic.test",
|
|
provider_name="anthropic",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
response = client.get("/probe")
|
|
|
|
assert response.json() == {"provider": "anthropic", "sub_path": "models"}
|
|
assert calls == [("https://api.anthropic.test", "models", "anthropic")]
|
|
|
|
|
|
def test_grok_dispatch_adapts_xai_model_list(monkeypatch) -> None:
|
|
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
|
|
return None
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
no_chatgpt_metadata,
|
|
)
|
|
|
|
class Proxy:
|
|
http_client = "h2"
|
|
|
|
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
|
|
return Response(
|
|
content=b'{"object":"list","data":[{"id":"grok-4.6","context_length":500000}]}',
|
|
status_code=200,
|
|
headers={"content-type": "application/json"},
|
|
)
|
|
|
|
app = FastAPI()
|
|
|
|
@app.get("/probe")
|
|
async def probe(request: Request):
|
|
return await handle_model_metadata_endpoint(
|
|
Proxy(),
|
|
request,
|
|
endpoint=MODEL_METADATA_LIST_ENDPOINT,
|
|
provider_api_base_url="https://api.x.ai/v1",
|
|
provider_name="openai",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
response = client.get("/probe")
|
|
|
|
assert response.json()["data"] == [
|
|
{"id": "grok-4.6", "context_length": 500000, "context_window": 500000}
|
|
]
|
|
|
|
|
|
def test_grok_response_preserves_status_and_safe_headers_without_stale_framing(monkeypatch) -> None:
|
|
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
|
|
return None
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
no_chatgpt_metadata,
|
|
)
|
|
|
|
class Proxy:
|
|
http_client = "h2"
|
|
|
|
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
|
|
return Response(
|
|
content=b'{"data":[{"id":"grok","context_length":1}]}',
|
|
status_code=206,
|
|
headers={
|
|
"content-type": "application/json",
|
|
"x-upstream": "kept",
|
|
"etag": '"stale"',
|
|
"last-modified": "Thu, 01 Jan 1970 00:00:00 GMT",
|
|
"cache-control": "max-age=60",
|
|
"content-length": "44",
|
|
"content-encoding": "gzip",
|
|
"transfer-encoding": "chunked",
|
|
},
|
|
)
|
|
|
|
app = FastAPI()
|
|
|
|
@app.get("/probe")
|
|
async def probe(request: Request):
|
|
return await handle_model_metadata_endpoint(
|
|
Proxy(),
|
|
request,
|
|
endpoint=MODEL_METADATA_LIST_ENDPOINT,
|
|
provider_api_base_url="https://api.x.ai",
|
|
provider_name="openai",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
response = client.get("/probe")
|
|
|
|
assert response.status_code == 206
|
|
assert response.headers["x-upstream"] == "kept"
|
|
assert response.headers.get("etag") is None
|
|
assert response.headers.get("last-modified") is None
|
|
assert response.headers.get("cache-control") is None
|
|
assert response.headers.get("content-encoding") is None
|
|
assert response.headers.get("transfer-encoding") is None
|
|
assert response.json()["data"][0]["context_window"] == 1
|
|
|
|
|
|
def test_grok_non_success_and_non_json_responses_are_unchanged(monkeypatch) -> None:
|
|
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
|
|
return None
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
no_chatgpt_metadata,
|
|
)
|
|
responses = {
|
|
"/error": Response(
|
|
content=b'{"data":[{"context_length":500000}]}',
|
|
status_code=500,
|
|
headers={"x-upstream": "error"},
|
|
),
|
|
"/non-json": Response(
|
|
content=b"upstream text",
|
|
status_code=200,
|
|
headers={"x-upstream": "text"},
|
|
),
|
|
"/surrogate": Response(
|
|
content=b'{"data":[{"id":"grok","context_length":1,"label":"\\ud800"}]}',
|
|
status_code=200,
|
|
headers={"x-upstream": "surrogate"},
|
|
),
|
|
"/non-finite": Response(
|
|
content=b'{"data":[{"id":"grok","context_length":1}],"extra":NaN}',
|
|
status_code=200,
|
|
headers={"x-upstream": "non-finite"},
|
|
),
|
|
}
|
|
|
|
class Proxy:
|
|
http_client = "h2"
|
|
|
|
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
|
|
return responses[request.url.path]
|
|
|
|
app = FastAPI()
|
|
|
|
@app.get("/{kind}")
|
|
async def probe(request: Request, kind: str):
|
|
return await handle_model_metadata_endpoint(
|
|
Proxy(),
|
|
request,
|
|
endpoint=MODEL_METADATA_LIST_ENDPOINT,
|
|
provider_api_base_url="https://api.x.ai",
|
|
provider_name="openai",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
error = client.get("/error")
|
|
non_json = client.get("/non-json")
|
|
surrogate = client.get("/surrogate")
|
|
non_finite = client.get("/non-finite")
|
|
|
|
assert error.status_code == 500
|
|
assert error.content == b'{"data":[{"context_length":500000}]}'
|
|
assert error.headers["x-upstream"] == "error"
|
|
assert non_json.status_code == 200
|
|
assert non_json.content == b"upstream text"
|
|
assert non_json.headers["x-upstream"] == "text"
|
|
assert surrogate.content == b'{"data":[{"id":"grok","context_length":1,"label":"\\ud800"}]}'
|
|
assert surrogate.headers["x-upstream"] == "surrogate"
|
|
assert non_finite.content == b'{"data":[{"id":"grok","context_length":1}],"extra":NaN}'
|
|
assert non_finite.headers["x-upstream"] == "non-finite"
|
|
|
|
|
|
def test_grok_response_unchanged_preserves_entity_validators(monkeypatch) -> None:
|
|
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
|
|
return None
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
no_chatgpt_metadata,
|
|
)
|
|
|
|
class Proxy:
|
|
http_client = "h2"
|
|
|
|
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
|
|
return Response(
|
|
content=b'{"data":[{"id":"grok","context_length":1,"context_window":1}]}',
|
|
status_code=200,
|
|
headers={
|
|
"content-type": "application/json",
|
|
"etag": '"current"',
|
|
"last-modified": "Thu, 01 Jan 1970 00:00:00 GMT",
|
|
"cache-control": "max-age=60",
|
|
},
|
|
)
|
|
|
|
app = FastAPI()
|
|
|
|
@app.get("/probe")
|
|
async def probe(request: Request):
|
|
return await handle_model_metadata_endpoint(
|
|
Proxy(),
|
|
request,
|
|
endpoint=MODEL_METADATA_LIST_ENDPOINT,
|
|
provider_api_base_url="https://api.x.ai",
|
|
provider_name="openai",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
response = client.get("/probe")
|
|
|
|
assert response.headers["etag"] == '"current"'
|
|
assert response.headers["last-modified"] == "Thu, 01 Jan 1970 00:00:00 GMT"
|
|
assert response.headers["cache-control"] == "max-age=60"
|
|
|
|
|
|
def test_grok_negative_space_bypasses_non_xai_detail_and_aliases(monkeypatch) -> None:
|
|
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
|
|
return None
|
|
|
|
monkeypatch.setattr(
|
|
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
|
|
no_chatgpt_metadata,
|
|
)
|
|
|
|
class Proxy:
|
|
http_client = "h2"
|
|
|
|
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
|
|
if request.url.path == "/non-xai":
|
|
content = b'{"data":[{"context_length":500000}]}'
|
|
elif request.url.path == "/detail":
|
|
content = b'{"id":"grok-4.6","context_length":500000}'
|
|
else:
|
|
content = b'{"data":[{"context_length":500000,"contextWindow":123}]}'
|
|
return Response(content=content, status_code=200, media_type="application/json")
|
|
|
|
app = FastAPI()
|
|
|
|
@app.get("/{kind}")
|
|
async def probe(request: Request, kind: str):
|
|
return await handle_model_metadata_endpoint(
|
|
Proxy(),
|
|
request,
|
|
endpoint=MODEL_METADATA_LIST_ENDPOINT
|
|
if kind == "list"
|
|
else model_metadata_get_endpoint("grok"),
|
|
provider_api_base_url="https://api.x.ai"
|
|
if kind != "non-xai"
|
|
else "https://api.openai.com",
|
|
provider_name="openai",
|
|
)
|
|
|
|
with TestClient(app) as client:
|
|
non_xai = client.get("/non-xai")
|
|
detail = client.get("/detail")
|
|
alias = client.get("/list")
|
|
|
|
assert non_xai.content == b'{"data":[{"context_length":500000}]}'
|
|
assert detail.content == b'{"id":"grok-4.6","context_length":500000}'
|
|
assert alias.json()["data"][0]["contextWindow"] == 123
|