mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description Plugins that pin wording per conversation (headroom-condense, and through it skill-search and tool-search) cannot see a real conversation id. Today: - **Claude:** they hash the first typed prompt, so two sessions that open with the same words share a key. - **Codex:** they get no key at all, so every new skill or MCP description waits out the one-hour hold. This PR adds an optional `TurnContext.conversation` that core fills from the client's own conversation id, so hooks can pin per real conversation. It is item #4 of the compression-gaps plan; the plugin work for #3, #6, #8 and #9 needs it to stay cache-safe. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **New field:** `TurnContext.conversation: str | None = None`. It is an opaque key that stays the same for every turn and retry of one client conversation, or `None` when the client names none. Hooks get a digest, never the raw id. | Path | Source | | --- | --- | | `/v1/messages` | sha256 of `X-Claude-Code-Session-Id` (Claude Code sends it on every request), falling back to `X-Headroom-Session-Id`; each source has its own namespace | | `/v1/responses` (HTTP and WebSocket) | the new `turn_conversation_key`, computed from the body before compression and passed through `_compress_openai_responses_payload_in_executor` | | `/v1/chat/completions` | `X-Headroom-Session-Id` only | | `/v1/compress` (gateway) | the gateway session id; `None` without one | - **Responses key.** `turn_conversation_key` uses the same identity and digest as `savings_conversation_key` (`thread_id` before `session_id`, then `client_metadata.*`, then the session header). Unlike the savings key, it also applies on `previous_response_id` turns and on bodies that name a server-side `conversation` (a string, or an object with an `id`), so the key does not drop out after the first WebSocket frame. `savings_conversation_key` keeps its existing exclusions. - **Gateway seam.** `CompressTurn.prepare()` is a third-party seam, so its signature is unchanged. Instead, the handler sets an optional writable `conversation` attribute on the turn before calling `prepare`. This is documented in `compress_turn.py`. - **No new ids.** Nothing is generated. The per-connection WebSocket uuid is not used, because a reconnect would change it. - **One path left out.** The catch-all passthrough Responses route still passes `None`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`): 3 errors, all in files this PR does not touch (embedders, model_metadata, langchain) - [x] New tests added for new functionality - [x] Manual testing performed - **New tests:** - `tests/test_turn_context_conversation.py`: - Anthropic: the same session gives one key, a different session with the same prompt gives a different key, the fallback header gets its own key, and no header gives `None`. - Responses HTTP: the same thread gives one key, including on a `previous_response_id` turn, and a different thread gives a different key. - Responses server-side `conversation`: string and object forms give one key across turns; a different conversation gives a different key; the savings key stays `None`. - Responses WebSocket: first and later frames share one key; a different thread gets a different key. - Chat Completions: covered the same way. - `tests/gateway/test_contract_request_half.py`: per-session gateway keys, and `None` without a session. - Every new test fails on `main`. - **Updated test:** `tests/test_proxy_openai.py` now pins `conversation=None` for a body with no ids. ### Test Output ```text pytest (turn hooks, gateway, Codex WS lifecycle, proxy openai, conversation savings, passthrough, hook usage): 411 passed, 5 skipped after the server-side conversation fix, on the synced head: 101 passed (conversation, gateway, Responses suites) ruff@0.17.0 check / format --check: clean mypy headroom --ignore-missing-imports: 3 errors, none in changed files ``` `tests/test_turn_hook_usage.py` needs `LITELLM_LOCAL_MODEL_COST_MAP=True` in a fresh venv; that is an existing litellm price-map fetch, and it fails the same way on `main`. ## Real Behavior Proof - Environment: - macOS 27.0.1 (Darwin 27.0.0, arm64), Python 3.12.13. - The proxy ran from this branch in a fresh `uv` venv (`uv venv -p 3.12 && uv pip install -e ".[dev]"`) on 127.0.0.1:9100, with default `ProxyConfig` (optimize on). - Upstream was the real Anthropic API, with my own API key passed by environment variable. - Client was Claude Code 2.1.311 (`claude --version`) with `--model claude-haiku-5-5`. - Exact command / steps: 1. Start the proxy with a scratch-only launcher (not committed). It calls `register_turn_hook()` with a hook whose `on_request` appends `ctx.provider`, `ctx.model`, `ctx.conversation` and `len(ctx.messages)` to a JSONL file, then calls `run_server(ProxyConfig(host="127.0.0.1", port=9100))`. Run it with `PYTHONSAFEPATH=1 HEADROOM_TELEMETRY=off LITELLM_LOCAL_MODEL_COST_MAP=True python run_proxy.py hooks.jsonl`, from a directory outside the repo. 2. Point Claude Code at the proxy: `export ANTHROPIC_BASE_URL=http://127.0.0.1:9100 CLAUDE_CONFIG_DIR=<fresh empty dir>`. 3. Start two sessions with the same prompt: `claude -p "say hi" --model claude-haiku-5-5 --output-format json`, run twice. Note each `session_id`. 4. Add two more turns to session 1: `claude -p "now say bye" --resume <session 1 id> --model claude-haiku-5-5`, then `claude -p "one more word please" --resume <session 1 id> --model claude-haiku-5-5`. 5. Read `hooks.jsonl`, and compare each key with sha256(`claude-code\0` + session_id). - Observed result (from `hooks.jsonl`, keys truncated): | Session | Turn | Messages | `ctx.conversation` | | --- | --- | --- | --- | | `305d64a9…` | first | 2 | `306306eeb9bc164b…` | | `4e50609f…` (same prompt) | first | 2 | `4a9e51bc7f8005f2…` | | `305d64a9…` | first `--resume` | 4 | `306306eeb9bc164b…` | | `305d64a9…` | second `--resume` | 6 | `306306eeb9bc164b…` | Each key equals sha256(`claude-code\0<session id>`) of the session id that Claude Code reported. All three runs answered normally, and the proxy log had no errors. - Not tested: - Codex over Responses HTTP or WebSocket (unit and handler tests cover these). - Chat Completions with `X-Headroom-Session-Id`. - The `/v1/compress` gateway path. - The server-side `conversation` body form, live (covered by the handler test). - The `X-Headroom-Session-Id` fallback on `/v1/messages`. - Claude Code `/compact` and subagents. - Bedrock and Vertex upstreams. - Linux and Windows. - A real plugin consuming the key. That is wired in headroomlabs-ai/headroom-condense#7; skill-search and tool-search adoption comes next. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: stable. - Stable/default behavior changed: no request or response bytes change. Registered turn hooks receive one new optional field; hooks that do not read it behave as before. - Kill switch / disable path: none needed; a hook that should not use the key ignores it (headroom-condense falls back to its previous key when the field is `None` or absent). - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this PR. Plugins read the field with `getattr`, so they keep working on cores without it. Cache note: compare keys only within one endpoint, since the same header gives different keys on `/v1/messages` and `/v1/responses`. Each key comes from a client id that stays the same for the whole conversation (Claude Code session ids survive retries, `--resume` and `/compact`; Codex `thread_id` wins over `session_id`, so guardian and subagent requests do not collide), and the Responses key is taken before compression rewrites the body, so a pinning hook keeps the same pin on every turn. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A (proxy-side change). ## Additional Notes - "No new warnings" is unchecked because the run was not compared warning by warning against `main`. - Follow-ups: skill-search and tool-search switch to `conversation_for(ctx)` once headroom-condense#7 is released. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Maintainer post-merge qualification Published `819f82117a4737de621528e9c49aee6aeeb4a793` includes upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 28 conversation/Responses/per-frame-memory/disconnect tests pass; Ruff/format/diff pass. Clean main merge; hook conversation policy unchanged, no new code finding. Author explicitly reserves unchecked self-review/readiness attestations; existing hold remains. Contributor correctness logic was not rewritten. Existing Changes Requested decision retained without duplicate findings. Scoped Windows validation uses the existing development native extension, not a fresh build. Hosted qualification is handled by maintainers; nothing merged. ### Maintainer current-main requalification Exact head `965ff21ffbea9b446e5e39269812bb0d8c43c108`, upstream main `ecbaf86d78cf52d185fbffa306157b9a4cc1b906`. Mechanically updated from upstream/mainecbaf86d78with no conflicts and no contributor conversation/ownership policy changes. Re-reviewed current discussion and the integration delta. All 38 focused conversation-context, Responses, per-frame WebSocket memory, disconnect and Gemini tests pass; Ruff0.17.3 lint/format and whitespace checks pass. Existing runtime evidence remains scoped; local tests use the existing native extension, not a fresh build. No new source blocker was found. The existing readiness action remains: the author has explicitly reserved and still not checked the self-review and ready-for-human-review attestations. They are not asserted on the author's behalf. Preserve the existing Changes Requested readiness hold until the author completes that action. Maintainers handle exact-head hosted checks; no contributor CI action requested. Nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `db69aa725d085d75b0881f0b2e1f2237480ee0c6`, upstream main `8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`. Clean mechanical sync with upstream main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a. Re-reviewed conversation-context integration and fresh author/Devin discussion without changing contributor identity/ownership policy.38conversation/Responses/per-frame-WebSocket/disconnect/Gemini tests pass;10changed Python files pass Ruff0.17.3check/format and whitespace checks. Existing runtime evidence remains scoped; copied extension supports Python startup, not a fresh native build. No new source blocker found. The current body still leaves author self-review and ready-for-human-review attestations unchecked and reserves those actions for the author. Preserve that existing readiness hold; no boxes were ticked or attested on their behalf. This is not an inherited-main CI defect. Maintainers handle exact-head hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `7411de5607beabfcbbd63b134795b0053da0e681`, upstream main `7592e681459f14a2041eb5f524a658cb6ad1dd92`. Clean current-main sync7592e681459f14a2041eb5f524a658cb6ad1dd92,including latestmemoryfixes. Conversationidentity/ownership policyunchanged.46conversation/Responses/perframeWS/disconnect/memory-regression/Gemini tests pass;whitespace passes. PriorRuff0.17.3check/format applies to unchangedPRfiles. Freshbodystillreservesuncheckedself-review/ready-for-humanreviewboxesfortheauthor. No newsourceblockerfound;retainexistingauthor-readinessholdwithouttickingboxesonbehalf. Existingruntimeevidence scoped,no freshnativebuildclaim. Maintainershandlechecks;nothingmerged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `cdf48ae9561dd2075625a4085594ac429a421231`, upstream main `2d3ee51846435ef5b320eea4806af14e602b0d8c`. Clean integration with upstream/main2d3ee51846after the external 0.43.1 release merge. Exact headroom/crates/tests comparison against previously reviewed7411de560shows no changes. Re-reviewed generated metadata; 78 release/version/MCP tests pass on this tree and whitespace passes. Earlier 46 conversation/Responses/WebSocket/disconnect/memory/Gemini passes remain scoped to unchanged inputs. No new source blocker found in reviewed scope. Existing author-owned readiness declaration remains the hold; fresh body must be checked before publication without checking author attestations on their behalf. No contributor identity/ownership policy rewritten, no fresh native-build/provider claim. Maintainers handle checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `d343c187f73b6a9005f0268e280ec657f435f1df`, upstream main `ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`. Clean integration with upstream/mainea59ac3eefafter the Click/macOS setup merge. Re-reviewed the OpenAI response replay transport-header overlap; conversation and existing tool-name map parameters remain additive.14 conversation/context-compaction tests and3 targeted route/gateway cases pass (71 unrelated cases deselected); Ruff and whitespace pass. No new source blocker found. Fresh body still leaves author-owned self-review/readiness attestations unchecked; preserve the existing hold without attesting on their behalf. Existing wider runtime evidence remains scoped; no fresh native-build or live-provider claim. Maintainers handle hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `741582654b9d0817ca8318af6aaf2dd000d1ec81`, upstream main `c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`. Final post-Dependabot requalification against upstream/mainc01cf07d94. All six Dependabot updates are now merged. Relative to this branch's freshly tested prepared tree, the final main sync changes only the three intended Cargo.lock records; Python/source/test inputs are unchanged. Re-reviewed the integration and confirmed complete native source/lock/toolchain objects match the tested combined build and exact fresh native DLL used for this branch's local tests. Focused fresh-native qualification: 18 passed, 70 deselected, 1 warning in 20.19s. Selected real dependency overlays and native DLL paths are recorded in final4496-fresh-native-tests.log; the DLL provenance/input hashes are in final-native-artifact-proof.json. No redundant rerun is implied after this Cargo-only merge. Whitespace passes. Prior source review and scoped real-behavior evidence remain applicable to unchanged contributor inputs. No new source blocker found. Fresh body still leaves author self-review and ready-for-human-review attestations unchecked. Retain the author-owned readiness hold without checking those boxes on their behalf. Maintainers handle hosted checks; no external-provider claim, contributor semantic rewrite or non-Dependabot merge. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `cb5c34b6221880a811dd94d21c5e3f3e874ee565`, upstream main `da6d9736cd82074988c7d0a2360b49d9d8b6c0a0`. Current-main requalification againstda6d9736cd. All69 integration patches exactly equal c01cf07-to-current-main delta after diff-location normalization; conversation identity/ownership policy unchanged.88 conversation,Responses context-compaction,OpenAI route,gateway request-contract tests pass;1 deprecation warning. Post-suite PyTorch cache-stat logging reports closed-stream logging errors during process exit; pytest exit0 and test results are retained without claiming full runtime qualification.31 changed Python files pass Ruff lint/format; offline lock and whitespace pass. Existing pr3982 native DLL verified full input hashes/artifact checksum, no fresh build claimed. Author/Devin discussion considered; previous substantive Devin findings resolved. No new source blocker found. Author explicitly reserves self-review/ready-for-human-review attestations and both remain unchecked; retain existing readiness hold without attesting on their behalf. Maintainers handle hosted checks; no contributor logic rewrite or merge. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `951bdf5853aaf8757ba46253763c91b6601b5c6d`, upstream main `0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`. Clean integration with upstream/main0ddaa56c21. All 19 integration patches exactly match the prior-reviewed-mainda6d9736to current-main delta after diff-location normalization; conversation identity/ownership policy is unchanged. Fresh conversation/turn-hook/Responses/gateway request-contract, doctor and incoming Vertex/backend/CLI suite: 367 passed, 11 skipped, 1 warning. Fourteen changed Python files pass Ruff lint/format; offline lock and whitespace checks pass. Author discussion and Devin findings read: both historical Devin items are marked resolved, including server-side conversation IDs. No new source blocker found. Existing author-reserved self-review/ready-for-human-review attestations remain a hold; this maintainer does not attest on the author's behalf. Static source/test scope and security inspection found no new workflow/dependency/setup path; the inherited flagged workflow-validation job is excluded by this exact PR's unchanged workflow paths. Conversation digests are not an authentication boundary. Verified recorded native input-set/hashes and artifact checksum before DLL reuse; no fresh build or new live-client qualification claimed. Maintainers handle exact-head hosted checks; no PR merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
457 lines
16 KiB
Python
457 lines
16 KiB
Python
"""Responses tool-search deferral is skipped for harnesses that cannot run it (GH #2660)."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
from types import SimpleNamespace
|
|
|
|
import httpx
|
|
import pytest
|
|
from starlette.datastructures import Headers
|
|
|
|
from headroom.proxy.auth_mode import classify_client
|
|
from headroom.proxy.handlers.openai import OpenAIHandlerMixin
|
|
from headroom.proxy.helpers import (
|
|
inject_tool_search_deferral_openai,
|
|
openai_tool_search_client_supported,
|
|
)
|
|
|
|
TOOL_SEARCH_MODEL = "gpt-5.5"
|
|
|
|
|
|
def _tool_payload() -> list[dict[str, object]]:
|
|
"""Six core coding tools plus ten non-core ones, over the injection minimum."""
|
|
names = ["bash", "read", "write", "edit", "grep", "glob"]
|
|
names += [f"slack_{index}" for index in range(10)]
|
|
return [
|
|
{"type": "function", "name": name, "parameters": {"type": "object", "properties": {}}}
|
|
for name in names
|
|
]
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("headers", "expected_client", "supported"),
|
|
[
|
|
({"user-agent": "opencode/0.4.2"}, "opencode", False),
|
|
({"x-client": "opencode"}, "opencode", False),
|
|
({"user-agent": "codex-cli/1.2.3"}, "codex", False),
|
|
({"user-agent": "claude-code/2.0"}, "claude-code", True),
|
|
({"user-agent": "cursor/1.0"}, "cursor", True),
|
|
({}, None, True),
|
|
({"user-agent": "some-unknown-sdk/1.0"}, None, True),
|
|
# CLIENT_UA_MAP matches by substring, so a wrapper that embeds the
|
|
# opencode UA is classified as opencode and excluded with it.
|
|
({"user-agent": "acme-wrapper opencode/1.0"}, "opencode", False),
|
|
],
|
|
)
|
|
def test_only_the_reported_harness_is_excluded(
|
|
headers: dict[str, str], expected_client: str | None, supported: bool
|
|
) -> None:
|
|
"""The exclusion keys on the client name the proxy already resolves."""
|
|
assert classify_client(headers) == expected_client
|
|
assert openai_tool_search_client_supported(classify_client(headers)) is supported
|
|
|
|
|
|
def test_a_similar_client_name_does_not_match() -> None:
|
|
"""The exclusion set is exact membership on the resolved client name.
|
|
|
|
Substring matching happens upstream in ``CLIENT_UA_MAP``; this pins that the
|
|
set itself does not widen a name that already classified.
|
|
"""
|
|
assert openai_tool_search_client_supported("opencode-fork") is True
|
|
assert openai_tool_search_client_supported("open") is True
|
|
assert openai_tool_search_client_supported("opencode") is False
|
|
|
|
|
|
def test_request_headers_decide_the_outbound_tools_payload() -> None:
|
|
"""End of the route: real request headers in, final Responses tools out.
|
|
|
|
This is the symptom the issue reports. An opencode request must not find an
|
|
injected ``{"type": "tool_search"}`` tool it cannot execute, and every other
|
|
client must still get the deferral it got before.
|
|
"""
|
|
tools = _tool_payload()
|
|
|
|
opencode = Headers({"user-agent": "opencode/0.4.2", "content-type": "application/json"})
|
|
forwarded = inject_tool_search_deferral_openai(
|
|
tools,
|
|
TOOL_SEARCH_MODEL,
|
|
client=classify_client(opencode),
|
|
)
|
|
assert forwarded is tools
|
|
assert not any(tool.get("type") == "tool_search" for tool in forwarded)
|
|
assert not any(tool.get("defer_loading") for tool in forwarded)
|
|
|
|
codex = Headers({"user-agent": "codex-cli/1.2.3", "content-type": "application/json"})
|
|
forwarded = inject_tool_search_deferral_openai(
|
|
tools,
|
|
TOOL_SEARCH_MODEL,
|
|
client=classify_client(codex),
|
|
)
|
|
assert forwarded is tools
|
|
assert not any(tool.get("type") == "tool_search" for tool in forwarded)
|
|
assert not any(tool.get("defer_loading") for tool in tools)
|
|
|
|
|
|
def test_websocket_and_http_header_shapes_classify_alike() -> None:
|
|
"""The WebSocket path builds a plain dict from the same multidict."""
|
|
multidict = Headers({"user-agent": "opencode/0.4.2"})
|
|
|
|
assert classify_client(multidict) == "opencode"
|
|
assert classify_client(dict(multidict)) == "opencode"
|
|
|
|
|
|
def test_native_responses_compressor_scopes_the_exclusion_per_call() -> None:
|
|
"""The flag rides one request; a later request is unaffected by an earlier one."""
|
|
seen: list[dict[str, object]] = []
|
|
handler = object.__new__(OpenAIHandlerMixin)
|
|
|
|
async def _run_compression(fn, *, timeout): # noqa: ANN001, ANN202
|
|
return fn()
|
|
|
|
def _compress(payload, *, model, request_id, **kwargs): # noqa: ANN001, ANN202
|
|
seen.append(kwargs)
|
|
return (payload, False, 0, [], "no-op", 0, 0, 0, {})
|
|
|
|
handler._run_compression_in_executor = _run_compression
|
|
handler._compress_openai_responses_payload = _compress
|
|
|
|
async def _run() -> None:
|
|
await handler._compress_openai_responses_payload_in_executor(
|
|
{"input": "hello"},
|
|
model=TOOL_SEARCH_MODEL,
|
|
request_id="req-opencode",
|
|
client="opencode",
|
|
)
|
|
await handler._compress_openai_responses_payload_in_executor(
|
|
{"input": "hello"},
|
|
model=TOOL_SEARCH_MODEL,
|
|
request_id="req-codex",
|
|
)
|
|
|
|
asyncio.run(_run())
|
|
|
|
assert [
|
|
{key: value for key, value in call.items() if key not in {"timing", "deadline_started_at"}}
|
|
for call in seen
|
|
] == [
|
|
{"client": "opencode"},
|
|
{"client": None},
|
|
]
|
|
|
|
|
|
def test_supported_clients_send_no_extra_compressor_argument() -> None:
|
|
"""A compressor override written before this change keeps its exact signature."""
|
|
calls: list[str] = []
|
|
handler = object.__new__(OpenAIHandlerMixin)
|
|
|
|
async def _run_compression(fn, *, timeout): # noqa: ANN001, ANN202
|
|
return fn()
|
|
|
|
def _narrow_compress(payload, *, model, request_id, timing=None): # noqa: ANN001, ANN202
|
|
calls.append(request_id)
|
|
return (payload, False, 0, [], "no-op", 0, 0, 0, {})
|
|
|
|
handler._run_compression_in_executor = _run_compression
|
|
handler._compress_openai_responses_payload = _narrow_compress
|
|
|
|
asyncio.run(
|
|
handler._compress_openai_responses_payload_in_executor(
|
|
{"input": "hello"},
|
|
model=TOOL_SEARCH_MODEL,
|
|
request_id="req-codex",
|
|
)
|
|
)
|
|
|
|
assert calls == ["req-codex"]
|
|
|
|
|
|
def test_a_narrow_compressor_override_still_works_for_an_excluded_client() -> None:
|
|
"""The retry drops the optional keywords rather than failing the request."""
|
|
calls: list[str] = []
|
|
handler = object.__new__(OpenAIHandlerMixin)
|
|
|
|
async def _run_compression(fn, *, timeout): # noqa: ANN001, ANN202
|
|
return fn()
|
|
|
|
def _narrow_compress(payload, *, model, request_id): # noqa: ANN001, ANN202
|
|
calls.append(request_id)
|
|
return (payload, False, 0, [], "no-op", 0, 0, 0, {})
|
|
|
|
handler._run_compression_in_executor = _run_compression
|
|
handler._compress_openai_responses_payload = _narrow_compress
|
|
|
|
asyncio.run(
|
|
handler._compress_openai_responses_payload_in_executor(
|
|
{"input": "hello"},
|
|
model=TOOL_SEARCH_MODEL,
|
|
request_id="req-opencode",
|
|
client="opencode",
|
|
)
|
|
)
|
|
|
|
assert calls == ["req-opencode"]
|
|
|
|
|
|
def test_native_responses_compressor_reraises_internal_type_error() -> None:
|
|
"""An internal compressor TypeError is propagated without a signature retry."""
|
|
calls = 0
|
|
handler = object.__new__(OpenAIHandlerMixin)
|
|
sentinel = TypeError("internal compressor failure")
|
|
|
|
async def _run_compression(fn, *, timeout): # noqa: ANN001, ANN202
|
|
return fn()
|
|
|
|
def _compress(payload, *, model, request_id, client, timing=None): # noqa: ANN001, ANN202
|
|
nonlocal calls
|
|
calls += 1
|
|
raise sentinel
|
|
|
|
handler._run_compression_in_executor = _run_compression
|
|
handler._compress_openai_responses_payload = _compress
|
|
|
|
with pytest.raises(TypeError) as exc_info:
|
|
asyncio.run(
|
|
handler._compress_openai_responses_payload_in_executor(
|
|
{"input": "hello"},
|
|
model=TOOL_SEARCH_MODEL,
|
|
request_id="req-opencode",
|
|
client="opencode",
|
|
)
|
|
)
|
|
|
|
assert exc_info.value is sentinel
|
|
assert calls == 1
|
|
|
|
|
|
class _ResponsesRequest:
|
|
method = "POST"
|
|
url = SimpleNamespace(path="/custom/v1/responses", query="")
|
|
|
|
def __init__(self, headers: dict[str, str]) -> None:
|
|
self.headers = headers
|
|
|
|
async def body(self) -> bytes:
|
|
return json.dumps({"model": TOOL_SEARCH_MODEL, "input": "hello"}).encode()
|
|
|
|
|
|
class _UpstreamClient:
|
|
async def request(self, **kwargs): # noqa: ANN001, ANN201
|
|
request = httpx.Request(kwargs["method"], kwargs["url"])
|
|
return httpx.Response(200, request=request, json={"ok": True})
|
|
|
|
|
|
def _passthrough_handler(seen: list[dict[str, object]]) -> OpenAIHandlerMixin:
|
|
handler = object.__new__(OpenAIHandlerMixin)
|
|
handler.config = SimpleNamespace(
|
|
optimize=True,
|
|
compress_passthrough=True,
|
|
openai_extra_headers=None,
|
|
)
|
|
handler.http_client = _UpstreamClient()
|
|
handler.http_client_h1 = None
|
|
|
|
async def _next_request_id() -> str:
|
|
return "req-test"
|
|
|
|
async def _compress(payload, *, model, request_id, **kwargs): # noqa: ANN001, ANN202
|
|
seen.append(kwargs)
|
|
return (payload, False, 0, [], "no-op", 0, len(json.dumps(payload)), 0, {})
|
|
|
|
handler._next_request_id = _next_request_id
|
|
handler._compress_openai_responses_payload_in_executor = _compress
|
|
return handler
|
|
|
|
|
|
def test_custom_base_path_excludes_the_reported_harness() -> None:
|
|
seen: list[dict[str, object]] = []
|
|
|
|
asyncio.run(
|
|
_passthrough_handler(seen).handle_passthrough(
|
|
_ResponsesRequest({"user-agent": "opencode/0.4.2", "content-type": "application/json"}),
|
|
"https://api.example.com",
|
|
)
|
|
)
|
|
|
|
assert seen == [{"client": "opencode"}]
|
|
|
|
|
|
def test_custom_base_path_leaves_other_clients_alone() -> None:
|
|
seen: list[dict[str, object]] = []
|
|
|
|
asyncio.run(
|
|
_passthrough_handler(seen).handle_passthrough(
|
|
_ResponsesRequest(
|
|
{"user-agent": "codex-cli/1.2.3", "content-type": "application/json"}
|
|
),
|
|
"https://api.example.com",
|
|
)
|
|
)
|
|
|
|
assert seen == [{"client": "codex"}]
|
|
|
|
|
|
# --- production route: the native /v1/responses handler -----------------------
|
|
|
|
_OPENAI_OK_RESPONSE = {
|
|
"id": "resp_test",
|
|
"object": "response",
|
|
"status": "completed",
|
|
"model": TOOL_SEARCH_MODEL,
|
|
"output": [],
|
|
"usage": {"input_tokens": 10, "output_tokens": 2},
|
|
}
|
|
|
|
|
|
class _NativeCapturingTransport(httpx.AsyncBaseTransport):
|
|
def __init__(self) -> None:
|
|
self.call_count = 0
|
|
|
|
async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
|
|
self.call_count += 1
|
|
async for _ in request.stream:
|
|
pass
|
|
return httpx.Response(200, json=_OPENAI_OK_RESPONSE)
|
|
|
|
|
|
def _native_responses_client(): # noqa: ANN202
|
|
"""Boot the real app and observe what the Responses compressor is handed.
|
|
|
|
The transport and the spy are installed after the lifespan runs, because
|
|
startup builds the proxy's HTTP clients.
|
|
"""
|
|
pytest.importorskip("fastapi")
|
|
from fastapi.testclient import TestClient
|
|
|
|
from headroom.proxy.server import ProxyConfig, create_app
|
|
|
|
config = ProxyConfig(
|
|
optimize=True,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
ccr_inject_tool=False,
|
|
ccr_handle_responses=False,
|
|
ccr_context_tracking=False,
|
|
image_optimize=False,
|
|
)
|
|
app = create_app(config)
|
|
seen: list[dict[str, object]] = []
|
|
|
|
# Loopback client so the proxy-token middleware treats this as a local call.
|
|
with TestClient(app, client=("127.0.0.1", 50000)) as client:
|
|
proxy = app.state.proxy
|
|
transport = _NativeCapturingTransport()
|
|
proxy.http_client = httpx.AsyncClient(transport=transport)
|
|
proxy.http_client_h1 = httpx.AsyncClient(transport=transport)
|
|
|
|
original = proxy._compress_openai_responses_payload_in_executor
|
|
|
|
async def _spy(payload, **kwargs): # noqa: ANN001, ANN202
|
|
seen.append({k: v for k, v in kwargs.items() if k not in {"model", "request_id"}})
|
|
return await original(payload, **kwargs)
|
|
|
|
proxy._compress_openai_responses_payload_in_executor = _spy
|
|
yield client, seen, transport
|
|
|
|
|
|
def _responses_body() -> dict[str, object]:
|
|
return {"model": TOOL_SEARCH_MODEL, "input": "hello", "tools": _tool_payload()}
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("user_agent", "expected"),
|
|
[
|
|
# No conversation id in the body or headers: hooks get no conversation.
|
|
("opencode/0.4.2", {"client": "opencode", "conversation": None}),
|
|
("codex-cli/1.2.3", {"client": "codex", "conversation": None}),
|
|
],
|
|
)
|
|
def test_native_responses_route_carries_the_client_decision(
|
|
user_agent: str, expected: dict[str, object]
|
|
) -> None:
|
|
"""Drives POST /v1/responses on the real app, not a handler method in isolation.
|
|
|
|
Deleting the kwargs splat at the native call site leaves every other test in
|
|
this file green; this one fails.
|
|
"""
|
|
for client, seen, transport in _native_responses_client():
|
|
response = client.post(
|
|
"/v1/responses",
|
|
headers={
|
|
"content-type": "application/json",
|
|
"authorization": "Bearer sk-test-0000000000000000000000000000000000000000000",
|
|
"user-agent": user_agent,
|
|
},
|
|
json=_responses_body(),
|
|
)
|
|
|
|
assert transport.call_count == 1, response.text
|
|
assert response.status_code == 200, response.text
|
|
assert seen, "the Responses compressor was never reached"
|
|
assert {k: v for k, v in seen[0].items() if k not in {"timing", "savings_tags"}} == expected
|
|
|
|
|
|
# --- production route: the Codex WebSocket handler ---------------------------
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("user_agent", "expected"),
|
|
[
|
|
("opencode/0.4.2", {"client": "opencode"}),
|
|
("codex-cli/1.2.3", {"client": "codex"}),
|
|
],
|
|
)
|
|
def test_websocket_route_carries_the_client_decision(
|
|
user_agent: str, expected: dict[str, object]
|
|
) -> None:
|
|
"""The WS frame path resolves the client the same way the HTTP path does.
|
|
|
|
Reuses the repo's existing Codex WS harness so the real
|
|
``handle_openai_responses_ws`` ingress runs, rather than asserting the
|
|
wiring structurally.
|
|
"""
|
|
import sys
|
|
from unittest.mock import patch
|
|
|
|
from tests.test_openai_codex_ws_lifecycle import (
|
|
_DummyOpenAIHandler,
|
|
_FakeUpstream,
|
|
_FakeWebSocket,
|
|
_first_frame,
|
|
_make_fake_websockets_module,
|
|
)
|
|
|
|
upstream = _FakeUpstream(
|
|
[
|
|
json.dumps({"type": "response.created", "response": {"id": "r_1"}}),
|
|
json.dumps({"type": "response.completed", "response": {"id": "r_1"}}),
|
|
]
|
|
)
|
|
client_ws = _FakeWebSocket(
|
|
frames=[_first_frame()],
|
|
headers={"authorization": "Bearer test", "user-agent": user_agent},
|
|
)
|
|
handler = _DummyOpenAIHandler()
|
|
handler.config.optimize = True
|
|
|
|
seen: list[dict[str, object]] = []
|
|
|
|
def _compress(payload, *, model, request_id, **kwargs): # noqa: ANN001, ANN202
|
|
seen.append(kwargs)
|
|
return (payload, False, 0, [], "router_no_compression", 10, 10)
|
|
|
|
handler._compress_openai_responses_payload = _compress
|
|
|
|
with patch.dict(sys.modules, {"websockets": _make_fake_websockets_module(upstream)}):
|
|
asyncio.run(handler.handle_openai_responses_ws(client_ws))
|
|
|
|
assert seen, "the Responses compressor was never reached on the WS path"
|
|
assert {
|
|
k: v
|
|
for k, v in seen[0].items()
|
|
if k not in {"timing", "deadline_started_at", "savings_tags"}
|
|
} == expected
|