Files
headroom/tests/test_provider_model_metadata.py
pokemonred06-creatorandJD Davis e1becdf732 fix(proxy): use HTTP/1.1 client for ChatGPT streaming and model metadata (#4006)
## Description

Ensures that ChatGPT endpoints (`chatgpt.com`, including `/v1/responses`
forwarded to `https://chatgpt.com/backend-api/codex/responses`,
`/v1/models`, Codex subpaths, and retried requests) consistently utilize
`http_client_h1` instead of multiplexing over the default HTTP/2 client.

`server.py` already provisions `self.http_client_h1` specifically
because ChatGPT's Cloudflare edge challenges HTTP/2 fingerprints on
sensitive endpoints. When large context requests (e.g. 500KB - 800KB
payload turns) are forwarded over HTTP/2 streaming, HTTP/2
connection/stream flow-control window stalls block the single
multiplexed TCP connection in `httpcore`. When this occurs, all
subsequent requests to ChatGPT hosts (including `/v1/models` and
subsequent turn completions) become blocked waiting behind the stalled
connection, eventually leading to proxy unresponsiveness and accumulated
asyncio task queues.

While `handle_passthrough` in `openai.py` and `openai_images.py` already
selected `http_client_h1`, `StreamingMixin._stream_response_generator`,
`handle_model_metadata_endpoint`, `_retry_request` in `server.py`, and
Codex subpath routes in `proxy_routes.py` previously hardcoded
`self.http_client`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Expose `prefers_http1_passthrough` in
`headroom/proxy/handlers/openai.py` (aliasing
`_prefers_http1_passthrough`).
- Add `_select_streaming_client` to `StreamingMixin` in
`headroom/proxy/handlers/streaming.py` that selects `http_client_h1`
when the target URL matches ChatGPT endpoints.
- Update `_stream_response_generator` and `_compact_sse_stream` to use
`_select_streaming_client`.
- Update `_retry_request` in `headroom/proxy/server.py` to route ChatGPT
target URLs through `http_client_h1`.
- Update `_register_openai_responses_subpath_route` in
`headroom/providers/proxy_routes.py` to forward through `http_client_h1`
for `handle_chatgpt_codex_responses_subpath`.
- In `headroom/providers/model_metadata.py`, pass `getattr(proxy,
"http_client_h1", None) or proxy.http_client` to
`handle_chatgpt_model_metadata`.
- Add unit tests in `tests/test_proxy_handler_helpers.py` and
`tests/test_provider_model_metadata.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run pytest tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py
======================== 76 passed, 1 warning in 8.24s ========================

uv run ruff check headroom/ tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py
All checks passed!
```

## Real Behavior Proof

- Environment: macOS 27 (Darwin arm64), Python 3.13, Headroom Proxy
v0.39.1 / v0.40.0, Codex CLI and OpenClaw Desktop client.
- Exact command / steps: Sent 800 KB streaming turns to /v1/responses
targeting https://chatgpt.com/backend-api/codex/responses while
monitoring TCP socket states via lsof/netstat and triggering concurrent
/v1/models calls.
- Observed result: Streaming responses and model metadata resolve over
HTTP/1.1 without multiplexed stream stall; Send-Q drains cleanly, turns
finish in ~2.1s, and all unit tests pass.
- Not tested: Third-party proprietary non-ChatGPT OAuth endpoints (e.g.,
enterprise SSO providers).

## Runtime Rollout Safety

- Rollout-managed feature(s): None
- Minimum rollout channel: stable
- Stable/default behavior changed: No (aligns streaming, retries, and
subpath routes with existing `http_client_h1` passthrough policy for
ChatGPT hosts)
- Kill switch / disable path: Fallback to `self.http_client` when
`http_client_h1` is unset
- Unsafe override required: No
- Qualification impact: None
- Rollback path: Revert commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)


### Maintainer current-main requalification

Exact head `d6cee507ff0aae7f09705b2a5b9a8bd8264b137f`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Updated against upstream/main e4a60ed7. Resolved the adjacent-helper
conflict by retaining both the contributor's HTTP/1.1 compatibility
alias and main's telemetry helper. HTTP/1.1 selection stays confined to
ChatGPT hosts across streaming/continuations, retries, metadata and
authenticated Responses subpaths; other upstreams retain their existing
client. No contributor production logic was rewritten.

All 136 focused helper/route/metadata tests pass. Three newer route
fixtures initially mocked only the default client; mechanically assigned
their same controlled response client to http_client_h1 too, preserving
every forwarding assertion. Changed-file Ruff lint/format and whitespace
checks pass. Existing macOS/Codex/OpenClaw live-stream evidence remains
scoped; no new live ChatGPT stream or fresh native-build claim is made.
Applied shared test-only Gemini streamed-request fixture repair for
independently established main failures; all ten cases pass. Formal
approval awaits exact-head hosted qualification.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `96dfae54717c13249f3bd9305cc6930030f2c85d`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Maintainer test-only repair for hosted run42031595103: the newer Codex
alias tests mocked only http_client, while this PR correctly routes
authenticated ChatGPT requests through http_client_h1. Assigned the
existing controlled fake to both client attributes, preserving every
auth/forwarding assertion; no production logic changed.

Main advanced during qualification; cleanly merged upstream/main
ecbaf86d78 and re-reviewed its
dependency/workflow/docs delta. All 154
helper/route/metadata/Codex-alias/Gemini tests pass on the final tree;
lint/format and whitespace checks pass. Existing runtime evidence
remains explicitly scoped, with no new live ChatGPT or fresh
native-build claim. Exact-head hosted checks are handled by maintainers.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-05 10:51:47 -06:00

378 lines
13 KiB
Python

from __future__ import annotations
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse, Response
from fastapi.testclient import TestClient
from headroom.providers.model_metadata import (
MODEL_METADATA_LIST_ENDPOINT,
ModelMetadataEndpoint,
handle_model_metadata_endpoint,
model_metadata_get_endpoint,
)
def test_model_metadata_endpoints_are_explicit() -> None:
assert MODEL_METADATA_LIST_ENDPOINT == ModelMetadataEndpoint(
"/v1/models",
"/backend-api/models",
)
assert model_metadata_get_endpoint("gpt-5") == ModelMetadataEndpoint(
"/v1/models/{model_id}",
"/backend-api/models/gpt-5",
)
def test_handle_model_metadata_endpoint_returns_chatgpt_response_when_present(monkeypatch) -> None:
async def fake_chatgpt_metadata(
http_client,
request: Request,
upstream_path: str,
) -> Response:
return JSONResponse({"client": http_client, "upstream_path": upstream_path})
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
fake_chatgpt_metadata,
)
proxy = type("Proxy", (), {"http_client": "h2"})()
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
proxy,
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.openai.test",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.json() == {"client": "h2", "upstream_path": "/backend-api/models"}
def test_handle_model_metadata_endpoint_prefers_http1_client(monkeypatch) -> None:
async def fake_chatgpt_metadata(
http_client,
request: Request,
upstream_path: str,
) -> Response:
return JSONResponse({"client": http_client, "upstream_path": upstream_path})
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
fake_chatgpt_metadata,
)
proxy = type("Proxy", (), {"http_client": "h2", "http_client_h1": "h1"})()
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
proxy,
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.openai.test",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.json() == {"client": "h1", "upstream_path": "/backend-api/models"}
def test_handle_model_metadata_endpoint_falls_back_to_selected_provider(monkeypatch) -> None:
async def fake_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
calls: list[tuple[str, str, str]] = []
class Proxy:
http_client = "h2"
async def handle_passthrough(
self,
request: Request,
base_url: str,
sub_path: str = "",
provider_name: str = "",
) -> Response:
calls.append((base_url, sub_path, provider_name))
return JSONResponse({"provider": provider_name, "sub_path": sub_path})
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
fake_chatgpt_metadata,
)
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=model_metadata_get_endpoint("claude-opus"),
provider_api_base_url="https://api.anthropic.test",
provider_name="anthropic",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.json() == {"provider": "anthropic", "sub_path": "models"}
assert calls == [("https://api.anthropic.test", "models", "anthropic")]
def test_grok_dispatch_adapts_xai_model_list(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return Response(
content=b'{"object":"list","data":[{"id":"grok-4.6","context_length":500000}]}',
status_code=200,
headers={"content-type": "application/json"},
)
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai/v1",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.json()["data"] == [
{"id": "grok-4.6", "context_length": 500000, "context_window": 500000}
]
def test_grok_response_preserves_status_and_safe_headers_without_stale_framing(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return Response(
content=b'{"data":[{"id":"grok","context_length":1}]}',
status_code=206,
headers={
"content-type": "application/json",
"x-upstream": "kept",
"etag": '"stale"',
"last-modified": "Thu, 01 Jan 1970 00:00:00 GMT",
"cache-control": "max-age=60",
"content-length": "44",
"content-encoding": "gzip",
"transfer-encoding": "chunked",
},
)
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.status_code == 206
assert response.headers["x-upstream"] == "kept"
assert response.headers.get("etag") is None
assert response.headers.get("last-modified") is None
assert response.headers.get("cache-control") is None
assert response.headers.get("content-encoding") is None
assert response.headers.get("transfer-encoding") is None
assert response.json()["data"][0]["context_window"] == 1
def test_grok_non_success_and_non_json_responses_are_unchanged(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
responses = {
"/error": Response(
content=b'{"data":[{"context_length":500000}]}',
status_code=500,
headers={"x-upstream": "error"},
),
"/non-json": Response(
content=b"upstream text",
status_code=200,
headers={"x-upstream": "text"},
),
"/surrogate": Response(
content=b'{"data":[{"id":"grok","context_length":1,"label":"\\ud800"}]}',
status_code=200,
headers={"x-upstream": "surrogate"},
),
"/non-finite": Response(
content=b'{"data":[{"id":"grok","context_length":1}],"extra":NaN}',
status_code=200,
headers={"x-upstream": "non-finite"},
),
}
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return responses[request.url.path]
app = FastAPI()
@app.get("/{kind}")
async def probe(request: Request, kind: str):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai",
provider_name="openai",
)
with TestClient(app) as client:
error = client.get("/error")
non_json = client.get("/non-json")
surrogate = client.get("/surrogate")
non_finite = client.get("/non-finite")
assert error.status_code == 500
assert error.content == b'{"data":[{"context_length":500000}]}'
assert error.headers["x-upstream"] == "error"
assert non_json.status_code == 200
assert non_json.content == b"upstream text"
assert non_json.headers["x-upstream"] == "text"
assert surrogate.content == b'{"data":[{"id":"grok","context_length":1,"label":"\\ud800"}]}'
assert surrogate.headers["x-upstream"] == "surrogate"
assert non_finite.content == b'{"data":[{"id":"grok","context_length":1}],"extra":NaN}'
assert non_finite.headers["x-upstream"] == "non-finite"
def test_grok_response_unchanged_preserves_entity_validators(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return Response(
content=b'{"data":[{"id":"grok","context_length":1,"context_window":1}]}',
status_code=200,
headers={
"content-type": "application/json",
"etag": '"current"',
"last-modified": "Thu, 01 Jan 1970 00:00:00 GMT",
"cache-control": "max-age=60",
},
)
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.headers["etag"] == '"current"'
assert response.headers["last-modified"] == "Thu, 01 Jan 1970 00:00:00 GMT"
assert response.headers["cache-control"] == "max-age=60"
def test_grok_negative_space_bypasses_non_xai_detail_and_aliases(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
if request.url.path == "/non-xai":
content = b'{"data":[{"context_length":500000}]}'
elif request.url.path == "/detail":
content = b'{"id":"grok-4.6","context_length":500000}'
else:
content = b'{"data":[{"context_length":500000,"contextWindow":123}]}'
return Response(content=content, status_code=200, media_type="application/json")
app = FastAPI()
@app.get("/{kind}")
async def probe(request: Request, kind: str):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT
if kind == "list"
else model_metadata_get_endpoint("grok"),
provider_api_base_url="https://api.x.ai"
if kind != "non-xai"
else "https://api.openai.com",
provider_name="openai",
)
with TestClient(app) as client:
non_xai = client.get("/non-xai")
detail = client.get("/detail")
alias = client.get("/list")
assert non_xai.content == b'{"data":[{"context_length":500000}]}'
assert detail.content == b'{"id":"grok-4.6","context_length":500000}'
assert alias.json()["data"][0]["contextWindow"] == 123