feat(proxy): let turn hooks rewrite OpenAI Responses requests on HTTP and WebSocket (#4232)

## Description

Add an optional on_responses_create turn hook so extensions can replace
an OpenAI Responses request before model-dependent proxy processing. The
hook applies to HTTP requests and WebSocket response.create frames,
including subsequent frames, and respects the Headroom bypass header.

## Type of Change
- [x] New feature (optional turn-hook method)

## Changes Made
- Expose ResponsesCreate with request body, headers, transport, and
connection identity.
- Run ordered synchronous/asynchronous hooks and isolate hook
exceptions.
- Apply replacement bodies before compression, model-dependent
accounting, and HTTP dispatch.
- Apply the hook to initial and subsequent WebSocket create frames.
- Add handler-level regression coverage for replacement, no-op, bypass,
and streaming behavior.

## Testing
- [x] Maintainer focused hook and WebSocket lifecycle suites passed.
- [x] Touched-file Ruff and whitespace checks passed.

### Test Output
```text
python -m pytest tests/test_responses_create_hooks.py tests/test_turn_hooks.py tests/test_openai_codex_ws_lifecycle.py tests/test_original_snapshot_in_place_hook.py -q
76 passed, 1 warning in 13.93s

ruff check headroom/proxy/turn_hooks.py headroom/proxy/handlers/openai.py tests/test_responses_create_hooks.py
All checks passed!
```

The latest main sync incorporates the merged snapshot-preservation fix
from #4241. All 76 focused hook, WebSocket, and snapshot tests passed
after that clean sync. Hosted CI is qualifying the newly pushed head.
Maintainer tests used the available development native extension; no
newly built native binary or independent live-provider validation is
claimed.

## Real Behavior Proof
- Environment: isolated Windows/Python 3.12.13 maintainer worktree, real
HTTP/WebSocket handlers exercised through repository test fixtures.
- Exact command / steps: run the four pytest suites listed above;
inspect replacement, bypass, no-op, and WebSocket lifecycle cases.
- Observed result: all 76 focused tests passed, including routed
model/effort reaching the captured upstream request and preserving
no-op/bypass behavior.
- Not tested: independent live inference through an installed routing
extension. The original author-supplied live/replay evidence is
preserved below and remains attributed to the author.

## Runtime Rollout Safety
- Rollout-managed feature(s): no new global rollout switch; the optional
hook runs only when registered.
- Minimum rollout channel: existing extension deployment policy applies.
- Stable/default behavior changed: registered hooks may replace
Responses bodies before proxy processing; deployments without this hook
retain the existing path.
- Kill switch / disable path: disable or unregister the consuming
extension; the Headroom bypass header skips this hook.
- Unsafe override required: no.
- Qualification impact: routing/model/stream changes require
extension-specific runtime qualification in addition to the core tests.
- Rollback path: disable the extension or retain the prior qualified
release.

## Review Readiness
- [ ] I have performed a self-review
- [ ] This PR is ready for human review

Maintainer qualification is recorded above; author attestations have not
been filled on the author's behalf.

## Additional Notes

The original author description and supplied evidence follow, with
headings demoted for the required template structure. The read-only body
view is shallow: hooks must construct replacements rather than mutate
nested request values.
#### Summary

Proxy extensions can now route Codex's `/v1/responses` traffic on the
WebSocket transport as well as over HTTP. This adds one optional
turn-hook method, `on_responses_create(ResponsesCreate) -> dict | None`.
It runs on every HTTP `/v1/responses` body and on every WebSocket
`response.create` frame, at the same point in the pipeline on both. A
hook reads `request.body`, which is a read-only view. To change the
request it returns a replacement body (for example with a new `model` or
`reasoning`); returning `None` leaves the request alone. This mirrors
how `on_response` already works. Chained hooks each see the previous
hook's replacement. `ResponsesCreate` carries `body`, `headers`,
`transport` (`"http"` or `"websocket"`) and `connection_id`.
`connection_id` is the WS session id and is `None` over HTTP. A
`previous_response_id` chain lives within one connection, so a hook that
holds state across a chain's frames should key it on `connection_id`.

#### The test we ran

- **Setup:** a Daytona sandbox running Headroom Enterprise 0.4.3 (core
0.39.1), Codex CLI 0.162.1 with the real GitHub MCP server, and
RouteMeGood in apply mode.
- **HTTP Codex:** 0 RouteMeGood decisions before routemegood#33 and 3
after it, on a probe.
- **Wrapped Codex:** still 0 decisions. `headroom wrap codex` writes
`supports_websockets = true`, so wrapped Codex sends `/v1/responses`
over a WebSocket. That traffic never passes through
`@app.middleware("http")`, so routing silently did nothing.

#### Why a new hook rather than a writable `TurnContext.model`

Turn hooks already run on the WS path. They run inside
`_compress_openai_responses_payload`, and that call site is the wrong
place for a router to change the model:

- It runs only when `config.optimize` is on, and on the executor under
the compression timeout. A timeout or a compression failure would drop
the routing decision.
- Built-in tool-search deferral
(`inject_tool_search_deferral_openai(model)`) has already run against
the *old* model. Routing to a model without tool search would forward a
`tool_search` tool the new model rejects.
- The handler's `model` (tokenizer, pricing, metrics, outcome) is
captured before compression, so a changed `ctx.model` would be priced as
the old model.

`on_responses_create` runs before all of that:

- **HTTP:** right after the inbound debug capture and before `model =
body["model"]`.
- **First WS frame:** before the ChatGPT `store=false` force, memory and
compression.
- **Later WS frames:** before memory and compression.

So everything downstream sees the routed model: compression, deferral,
`store=false`, token counting, pricing, and the WS memory-continuation
template, which is rebuilt from the outbound frame. It is skipped under
`x-headroom-bypass`, and it is inert when no hook defines it.

- **`previous_response_id`:** passed through untouched. OpenAI allows a
model switch mid-chain, and the continuation template already drops
`previous_response_id`.
- **Byte-faithful forwarding (#1542):** the HTTP body is marked mutated
only when a hook returns a replacement. An unchanged request still
forwards the client's original bytes, and nothing has to serialize the
body to find out.
- **A failing hook leaves no trace:** a hook never edits the body in
place, so a hook that raises cannot leave a half-made edit behind.
Without that, pricing would see the edit while the forwarded bytes did
not.
- **`stream` is read after the hook:** a replacement that turns
streaming on takes the streaming dispatch path.
- **Headers:** WS handshake headers are handed over read-only
(`MappingProxyType`).

#### Before / after

| | before | after |
|---|---|---|
| Unit: WS `response.create` frames, hook sets `gpt-5.4-mini` +
`reasoning.effort=low` | upstream frames carry the client's `gpt-5.5` |
both frames carry `gpt-5.4-mini` / `low` (compression off: routing
doesn't depend on it) |
| Unit: HTTP `/v1/responses`, same hook | upstream body +
`RequestOutcome.model` = `gpt-5.5` | upstream body and outcome =
`gpt-5.4-mini` |
| Live, local proxy (steps below) | 0 decisions (2 WS create frames, 0
HTTP POSTs); RouteMeGood logs `this Headroom has no Responses create
hook: Codex over WebSocket passes through unrouted` | 2 decisions, one
per WS create frame |

##### Live check: steps

- **Machine:** macOS (Darwin 25.6), Python 3.12.13.
- **Codex:** codex-cli 0.153.2 with ChatGPT auth.
- **Extension:** RouteMeGood from the stacked PR branch, in shadow mode.
- **Proxy:** the extension's `.venv`, started from this branch:

```sh
HEADROOM_BEACON=off ROUTEMEGOOD_BEACON=off HEADROOM_ROUTEMEGOOD_LICENSE_DEV=1 ROUTEMEGOOD_MODE=shadow ROUTEMEGOOD_HARNESS=codex ROUTEMEGOOD_PERSIST=0 ROUTEMEGOOD_REQUIRE_CREDS=0 PYTHONPATH=<this branch>:<routemegood feat/responses-ws> python -m headroom.cli proxy --port 8799 --proxy-extension routemegood

codex exec --skip-git-repo-check -c model_provider=headroom   -c 'model_providers.headroom={name="OpenAI via Headroom proxy",base_url="http://127.0.0.1:8799/v1",supports_websockets=true,requires_openai_auth=true,wire_api="responses"}'   "Reply with the single word: pong"
curl -s localhost:8799/routemegood/stats
```

**Observed:**

- The proxy log showed `WS /v1/responses accepted
(route=chatgpt_subscription ...)`, then `completed (... client_frames=2
...)`.
- There were no HTTP `POST /v1/responses`.
- Codex printed `pong`.
- **After:** `/routemegood/stats` returned
`{"decisions":2,"harness":"codex","picked":{"gpt-5.6-terra":2},...}`.
- **Before:** I ran the same steps with `PYTHONPATH` set to a `main`
checkout (976aa714). That gave `{"decisions":0,"note":"no inference
traffic seen yet"}` and the one-line degrade warning.

**Not tested live:** apply mode, i.e. whether the rewritten model
reaches OpenAI's servers. Without a learned predictor RouteMeGood keeps
the caller's model. The unit tests above cover the rewrite. The live run
was in shadow mode, and with no learned predictor RouteMeGood keeps the
caller's model, so it shows decisions being *made* on WS. The rewrite
reaching the upstream is shown by the unit tests. The Daytona sandbox
was not used for this check: another audit is running through its proxy.

#### Tests

`tests/test_responses_create_hooks.py`, all through the handlers:

- WS: the hook's model and effort reach the upstream on the first and
later frames. `previous_response_id` is kept and non-create frames are
untouched.
- WS: the bypass header skips the hook.
- WS: a hook that raises does not break the session.
- HTTP: the upstream body and the recorded outcome both carry the routed
model.
- HTTP: a no-op hook keeps `body_mutated=False`, so the original bytes
are forwarded.
- HTTP: the bypass header skips the hook.

The WS and HTTP routing tests fail on `main`. Touched-area suites pass:
532 tests across turn hooks, Codex WS lifecycle, memory, savings,
compression, routing, byte-faithful forwarding and the extension
ordering suites. ruff check, ruff format and mypy are clean.

The RouteMeGood side: headroomlabs-ai/routemegood PR stacked on #33.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Tejas Chopra
2026-10-17 11:29:16 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JD Davis
parent 1d112cd36c
commit fbf7bc3278
3 changed files with 435 additions and 19 deletions
+48 -12
View File
@@ -22,6 +22,7 @@ from collections import OrderedDict
from concurrent.futures import ThreadPoolExecutor, as_completed
from dataclasses import replace
from datetime import datetime
from types import MappingProxyType
from typing import TYPE_CHECKING, Any
from urllib.parse import quote, unquote, urlparse
@@ -55,6 +56,7 @@ from headroom.proxy.tool_schema_savings_policy import (
net_hook_tool_saving,
without_deferral_flags,
)
from headroom.proxy.turn_hooks import run_responses_create_hooks
from headroom.proxy.upstream_guard import is_safe_upstream_url
from headroom.proxy.ws_headers import WS_HOP_BY_HOP_HEADERS
from headroom.proxy.ws_session_registry import (
@@ -1962,6 +1964,24 @@ class OpenAIHandlerMixin:
except Exception as exc:
logger.debug("[%s] Traffic learner (ws): %s", request_id, exc)
@staticmethod
async def _run_ws_create_hooks(
frame: Any, ws_headers: dict[str, str], connection_id: str
) -> bool:
"""Run ``on_responses_create`` on a WS create frame's inner payload.
True when a hook changed it, so the caller re-serializes the frame.
"""
inner = frame.get("response", frame) if isinstance(frame, dict) else None
if not isinstance(inner, dict):
return False
return await run_responses_create_hooks(
inner,
headers=MappingProxyType(ws_headers),
transport="websocket",
connection_id=connection_id,
)
@staticmethod
def _headroom_bypass_enabled(headers: Any) -> bool:
"""Return True when inbound headers request full passthrough."""
@@ -5672,8 +5692,6 @@ class OpenAIHandlerMixin:
},
)
model = body.get("model", "unknown")
stream = body.get("stream", False)
body_mutation_tracker = BodyMutationTracker()
_bypass = self._headroom_bypass_enabled(request.headers)
if _bypass:
@@ -5693,8 +5711,17 @@ class OpenAIHandlerMixin:
url=str(request.url),
headers=dict(request.headers.items()),
body=body,
metadata={"path": request.url.path, "stream": stream},
metadata={"path": request.url.path, "stream": body.get("stream", False)},
)
# Before every stage that reads the body, so a hook's replacement is
# what compression, deferral, token counting, pricing and the
# stream/buffered dispatch all see.
if not _bypass and await run_responses_create_hooks(
body, headers=request.headers, transport="http"
):
body_mutation_tracker.mark_mutated("responses_create_hook")
model = body.get("model", "unknown")
stream = body.get("stream", False)
# /v1/responses uses provider-specific CompressionUnit extraction
# below, then routes mutable text through ContentRouter. The
@@ -7662,6 +7689,19 @@ class OpenAIHandlerMixin:
except json.JSONDecodeError:
# Not JSON — pass through as-is
pass
first_frame_is_create = isinstance(body, dict) and (
body.get("type") == "response.create" or ("type" not in body and "input" in body)
)
_ws_bypass = self._headroom_bypass_enabled(ws_headers)
if _ws_bypass:
logger.info(
"[%s] WS /v1/responses passthrough reason=bypass_header mutation=disabled",
request_id,
)
elif first_frame_is_create and await self._run_ws_create_hooks(
body, ws_headers, session_id
):
first_msg_raw = json.dumps(body)
if isinstance(body, dict) and body:
ws_response_body_for_store = (
body["response"] if isinstance(body.get("response"), dict) else body
@@ -7707,12 +7747,6 @@ class OpenAIHandlerMixin:
ws_recorded_compression_timing_totals: dict[str, float] = {}
ws_ttfb_ms: float | None = None
ws_recorded_ttfb_ms = False
_ws_bypass = self._headroom_bypass_enabled(ws_headers)
if _ws_bypass:
logger.info(
"[%s] WS /v1/responses passthrough reason=bypass_header mutation=disabled",
request_id,
)
capture_codex_wire_debug(
"ws_inbound_first_frame",
@@ -8144,9 +8178,7 @@ class OpenAIHandlerMixin:
logger.warning(f"[{request_id}] WS Memory injection failed: {e}")
return frame_raw
if isinstance(body, dict) and (
body.get("type") == "response.create" or ("type" not in body and "input" in body)
):
if first_frame_is_create:
first_msg_raw = await _prepare_memory_frame(body, first_msg_raw)
first_response_body = body.get("response", body)
current_response_input = _responses_input_to_items(
@@ -8872,6 +8904,10 @@ class OpenAIHandlerMixin:
inbound_response = _inbound_frame_body.get(
"response", _inbound_frame_body
)
if not _ws_bypass and await self._run_ws_create_hooks(
_inbound_frame_body, ws_headers, session_id
):
msg = json.dumps(_inbound_frame_body)
current_response_input = _responses_input_to_items(
inbound_response.get("input")
if isinstance(inbound_response, dict)
+91 -7
View File
@@ -21,10 +21,12 @@ the same deprecation policy.
from __future__ import annotations
import inspect
import logging
from collections.abc import Awaitable, Callable
from collections.abc import Awaitable, Callable, Mapping
from dataclasses import dataclass, field
from typing import Any, Protocol, runtime_checkable
from types import MappingProxyType
from typing import Any, Literal, Protocol, runtime_checkable
from headroom.proxy.tool_schema_savings_policy import (
deferred_booking,
@@ -90,10 +92,31 @@ class TurnContext:
)
@dataclass(frozen=True)
class ResponsesCreate:
"""One OpenAI Responses create request, before the proxy rewrites it.
``body`` is a read-only view of the Responses payload (for a WebSocket
``response.create`` frame, the inner ``response`` object). ``headers``
are the client's request headers, or the WebSocket handshake headers,
looked up by lowercase name. ``connection_id`` names the WebSocket
connection the frame arrived on (``None`` over HTTP): a
``previous_response_id`` chain lives within one connection, so it is the
key for anything a hook must hold across a chain's frames. A reconnect
gets a new id; Codex's first frame on it replays the full transcript
rather than continuing the old chain, so it arrives as a fresh request.
"""
body: Mapping[str, Any]
headers: Mapping[str, str]
transport: Literal["http", "websocket"]
connection_id: str | None = None
@runtime_checkable
class TurnHook(Protocol):
"""A registered turn observer. Both methods are optional (a hook may define
either); missing methods are simply skipped."""
"""A registered turn observer. Every method is optional (a hook may define
any of them); missing methods are simply skipped."""
name: str
@@ -104,9 +127,10 @@ class TurnHook(Protocol):
stream_safe: bool
# Optional: run order among registered hooks, lowest first (stable for
# ties; absent ⇒ DEFAULT_HOOK_PRIORITY). Convention: routers that change
# ``ctx.model`` 10-49, message folds 50-99, tool-catalog shaping that is
# gated on the final model (deferral) 200+. Registration order is
# ties; absent ⇒ DEFAULT_HOOK_PRIORITY). Convention: routers 10-49 (an
# OpenAI Responses router rewrites the model in ``on_responses_create``,
# which runs before every ``on_request``), message folds 50-99,
# tool-catalog shaping that is gated on the final model (deferral) 200+. Registration order is
# entry-point discovery order, which an operator does not control, so a
# hook whose correctness depends on running after another must say so.
priority: int
@@ -122,6 +146,26 @@ class TurnHook(Protocol):
May ``await call_model(messages)`` to re-drive the model (e.g. to resolve
an injected tool call), looping as needed before returning the final."""
async def on_responses_create(self, request: ResponsesCreate) -> dict[str, Any] | None:
"""Return a replacement body for an OpenAI Responses create request,
or ``None`` to leave it unchanged.
Called once per HTTP ``/v1/responses`` body and once per WebSocket
``response.create`` frame, on the event loop, before compression and
every model-gated stage (not when the client sent the Headroom bypass
header). This is where a router changes the model or reasoning effort:
unlike ``on_request`` it runs whether or not compression is enabled.
Everything after it -- memory, compression, tool-search deferral,
token counting, pricing, metrics, the WS continuation template -- sees
the replacement; the next hook gets it as its ``request.body``. The
view is read-only at the top level and nested values are shared with
the live request, so build replacements (``{**body, "reasoning":
{...}}``) rather than mutating nested dicts or lists. A hook that
raises then leaves nothing half-applied, and a request no hook
replaced keeps the client's original bytes. Keep it fast: it is awaited inline on every frame, so
no network I/O."""
_hooks: list[TurnHook] = []
@@ -301,3 +345,43 @@ async def run_response_hooks(
if replacement is not None:
current = replacement
return current
async def run_responses_create_hooks(
body: dict[str, Any],
*,
headers: Mapping[str, str],
transport: Literal["http", "websocket"],
connection_id: str | None = None,
) -> bool:
"""Run every hook's ``on_responses_create`` and apply the final body to
``body`` in place. True if any hook replaced it.
Inert (returns False) when no hook defines the method. A failing hook is
logged and skipped. A sync method is accepted as well as a coroutine.
"""
current: dict[str, Any] = body
for hook in registered_turn_hooks():
fn = getattr(hook, "on_responses_create", None)
if fn is None:
continue
request = ResponsesCreate(
body=MappingProxyType(current),
headers=headers,
transport=transport,
connection_id=connection_id,
)
try:
result = fn(request)
if inspect.isawaitable(result):
result = await result
except Exception: # a hook must never break the proxy
log.exception("turn hook %r on_responses_create failed", getattr(hook, "name", hook))
continue
if isinstance(result, dict):
current = result
if current is body:
return False
body.clear()
body.update(current)
return True
+296
View File
@@ -0,0 +1,296 @@
"""``on_responses_create``: one routing seam for OpenAI Responses on both transports.
Codex sends ``/v1/responses`` over a WebSocket when wrapped, and an HTTP
middleware never sees those frames, so a router installed as middleware did
nothing for wrapped Codex. A turn hook's ``on_responses_create`` runs on the
HTTP body and on every WebSocket ``response.create`` frame, before the proxy's
own stages, so a model/effort rewrite reaches the upstream and the accounting.
"""
from __future__ import annotations
import json
import sys
from typing import Any
from unittest.mock import patch
import pytest
from headroom.proxy.turn_hooks import (
ResponsesCreate,
clear_turn_hooks,
register_turn_hook,
)
from tests.test_openai_codex_ws_lifecycle import (
_DummyOpenAIHandler,
_FakeUpstream,
_FakeWebSocket,
_make_fake_websockets_module,
)
@pytest.fixture(autouse=True)
def _clean_registry():
clear_turn_hooks()
yield
clear_turn_hooks()
class _Router:
name = "test-router"
def __init__(self) -> None:
self.seen: list[tuple[str, str, str | None]] = []
self.connections: list[str | None] = []
async def on_responses_create(self, request: ResponsesCreate) -> dict:
self.seen.append(
(request.transport, str(request.body.get("model")), request.headers.get("user-agent"))
)
self.connections.append(request.connection_id)
return {**request.body, "model": "gpt-5.4-mini", "reasoning": {"effort": "low"}}
def _create(text: str, **extra: Any) -> str:
return json.dumps(
{"type": "response.create", "response": {"model": "gpt-5.5", "input": text, **extra}}
)
def _upstream() -> _FakeUpstream:
return _FakeUpstream(
[
json.dumps({"type": "response.created", "response": {"id": "r_1"}}),
json.dumps({"type": "response.completed", "response": {"id": "r_1"}}),
],
hold_after_events=True,
)
async def _run_ws(frames: list[str], headers: dict[str, str] | None = None) -> list[dict]:
upstream = _upstream()
client_ws = _FakeWebSocket(frames=frames, headers=headers)
with patch.dict(sys.modules, {"websockets": _make_fake_websockets_module(upstream)}):
await _DummyOpenAIHandler().handle_openai_responses_ws(client_ws)
return [json.loads(frame) for frame in upstream.sent]
@pytest.mark.asyncio
async def test_ws_create_frames_carry_the_hooks_model_and_effort_upstream() -> None:
router = _Router()
register_turn_hook(router)
# previous_response_id chains are left to the client and OpenAI: a model
# switch mid-chain is allowed, so the hook's rewrite rides alongside it.
sent = await _run_ws(
[
_create("first"),
json.dumps({"type": "response.cancel"}),
_create("next", previous_response_id="r_1"),
]
)
creates = [frame["response"] for frame in sent if frame.get("type") == "response.create"]
assert [c["model"] for c in creates] == ["gpt-5.4-mini", "gpt-5.4-mini"]
assert [c["reasoning"] for c in creates] == [{"effort": "low"}] * 2
assert creates[1]["previous_response_id"] == "r_1"
assert {"type": "response.cancel"} in sent
# Compression is off in the dummy handler: routing must not depend on it.
assert router.seen == [
("websocket", "gpt-5.5", "codex-cli/0.5"),
("websocket", "gpt-5.5", "codex-cli/0.5"),
]
# One connection, one id: what a hook keys a previous_response_id chain on.
assert router.connections[0] and len(set(router.connections)) == 1
@pytest.mark.asyncio
async def test_ws_bypass_header_skips_the_hook() -> None:
router = _Router()
register_turn_hook(router)
frames = [_create("first"), _create("next")]
sent = await _run_ws(frames, headers={"authorization": "Bearer t", "x-headroom-bypass": "true"})
assert router.seen == []
assert [frame["response"]["model"] for frame in sent] == ["gpt-5.5", "gpt-5.5"]
@pytest.mark.asyncio
async def test_a_hook_that_raises_does_not_break_the_session() -> None:
class _Broken:
name = "broken"
async def on_responses_create(self, request: ResponsesCreate) -> None:
# The body is read-only: a hook cannot half-apply an edit and then fail.
request.body["model"] = "gpt-5.4-mini" # type: ignore[index]
register_turn_hook(_Broken())
frames = [_create("first")]
sent = await _run_ws(frames)
assert sent == [json.loads(frames[0])]
class _NoOp:
name = "no-op"
async def on_responses_create(self, request: ResponsesCreate) -> None:
return None
def _post_http(raw: bytes, headers: dict[str, str] | None = None) -> tuple[Any, list, list]:
"""POST raw bytes to /v1/responses.
Returns the response, what reached the upstream call as
``(body, body_mutated)`` pairs, and the recorded outcomes.
"""
pytest.importorskip("fastapi")
httpx = pytest.importorskip("httpx")
from fastapi.testclient import TestClient
from headroom.proxy.loopback_guard import require_loopback
from headroom.proxy.server import ProxyConfig, create_app
app = create_app(
ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
)
)
app.dependency_overrides[require_loopback] = lambda: None
sent: list[Any] = []
outcomes: list[Any] = []
with TestClient(app) as client:
server = app.state.proxy
async def fake_retry(method, url, headers, body, stream=False, **kwargs):
sent.append((_as_json(body), kwargs.get("body_mutated")))
return httpx.Response(
200,
json={
"id": "resp_1",
"object": "response",
"status": "completed",
"model": "gpt-5.4-mini",
"output": [],
"usage": {"input_tokens": 10, "output_tokens": 2},
},
request=httpx.Request("POST", url),
)
record = server._record_request_outcome
async def capture(outcome):
outcomes.append(outcome)
await record(outcome)
server._retry_request = fake_retry
server._record_request_outcome = capture
resp = client.post(
"/v1/responses",
content=raw,
headers={
"Authorization": "Bearer sk-test",
"Content-Type": "application/json",
"User-Agent": "codex_exec/0.162.1",
**(headers or {}),
},
)
return resp, sent, outcomes
def _as_json(body: Any) -> dict:
return json.loads(body) if isinstance(body, (bytes, str)) else body
_RAW = json.dumps({"model": "gpt-5.5", "input": "hi", "stream": False}).encode()
def test_http_body_carries_the_hooks_model_and_the_outcome_prices_it() -> None:
router = _Router()
register_turn_hook(router)
resp, sent, outcomes = _post_http(_RAW)
assert resp.status_code == 200, resp.text
assert router.seen == [("http", "gpt-5.5", "codex_exec/0.162.1")]
assert router.connections == [None]
body, mutated = sent[0]
assert body["model"] == "gpt-5.4-mini"
assert body["reasoning"] == {"effort": "low"}
assert mutated is True
assert outcomes and all(o.model == "gpt-5.4-mini" for o in outcomes)
def test_http_hook_that_changes_nothing_keeps_the_clients_bytes() -> None:
register_turn_hook(_NoOp())
resp, sent, _ = _post_http(_RAW)
assert resp.status_code == 200, resp.text
# Unmutated is what sends the client's original bytes (#1542).
assert sent == [(json.loads(_RAW), False)]
def test_http_bypass_header_skips_the_hook() -> None:
router = _Router()
register_turn_hook(router)
resp, sent, _ = _post_http(_RAW, headers={"x-headroom-bypass": "true"})
assert resp.status_code == 200, resp.text
assert router.seen == []
assert sent == [(json.loads(_RAW), False)]
def test_http_hook_that_turns_on_streaming_takes_the_streaming_path() -> None:
class _Streamer:
name = "streamer"
async def on_responses_create(self, request: ResponsesCreate) -> dict:
return {**request.body, "stream": True}
register_turn_hook(_Streamer())
pytest.importorskip("fastapi")
from fastapi.responses import JSONResponse
from fastapi.testclient import TestClient
from headroom.proxy.loopback_guard import require_loopback
from headroom.proxy.server import ProxyConfig, create_app
app = create_app(
ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
)
)
app.dependency_overrides[require_loopback] = lambda: None
streamed: list[dict] = []
with TestClient(app) as client:
server = app.state.proxy
async def fake_stream(url, headers, body, *args, **kwargs):
streamed.append(body)
return JSONResponse({"ok": True})
async def no_buffered(*args, **kwargs):
raise AssertionError("a streaming upstream request took the buffered path")
server._stream_response = fake_stream
server._retry_request = no_buffered
resp = client.post(
"/v1/responses",
content=_RAW,
headers={"Authorization": "Bearer sk-test", "Content-Type": "application/json"},
)
assert resp.status_code == 200, resp.text
assert [b["stream"] for b in streamed] == [True]