mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description Plugins that pin wording per conversation (headroom-condense, and through it skill-search and tool-search) cannot see a real conversation id. Today: - **Claude:** they hash the first typed prompt, so two sessions that open with the same words share a key. - **Codex:** they get no key at all, so every new skill or MCP description waits out the one-hour hold. This PR adds an optional `TurnContext.conversation` that core fills from the client's own conversation id, so hooks can pin per real conversation. It is item #4 of the compression-gaps plan; the plugin work for #3, #6, #8 and #9 needs it to stay cache-safe. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **New field:** `TurnContext.conversation: str | None = None`. It is an opaque key that stays the same for every turn and retry of one client conversation, or `None` when the client names none. Hooks get a digest, never the raw id. | Path | Source | | --- | --- | | `/v1/messages` | sha256 of `X-Claude-Code-Session-Id` (Claude Code sends it on every request), falling back to `X-Headroom-Session-Id`; each source has its own namespace | | `/v1/responses` (HTTP and WebSocket) | the new `turn_conversation_key`, computed from the body before compression and passed through `_compress_openai_responses_payload_in_executor` | | `/v1/chat/completions` | `X-Headroom-Session-Id` only | | `/v1/compress` (gateway) | the gateway session id; `None` without one | - **Responses key.** `turn_conversation_key` uses the same identity and digest as `savings_conversation_key` (`thread_id` before `session_id`, then `client_metadata.*`, then the session header). Unlike the savings key, it also applies on `previous_response_id` turns and on bodies that name a server-side `conversation` (a string, or an object with an `id`), so the key does not drop out after the first WebSocket frame. `savings_conversation_key` keeps its existing exclusions. - **Gateway seam.** `CompressTurn.prepare()` is a third-party seam, so its signature is unchanged. Instead, the handler sets an optional writable `conversation` attribute on the turn before calling `prepare`. This is documented in `compress_turn.py`. - **No new ids.** Nothing is generated. The per-connection WebSocket uuid is not used, because a reconnect would change it. - **One path left out.** The catch-all passthrough Responses route still passes `None`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`): 3 errors, all in files this PR does not touch (embedders, model_metadata, langchain) - [x] New tests added for new functionality - [x] Manual testing performed - **New tests:** - `tests/test_turn_context_conversation.py`: - Anthropic: the same session gives one key, a different session with the same prompt gives a different key, the fallback header gets its own key, and no header gives `None`. - Responses HTTP: the same thread gives one key, including on a `previous_response_id` turn, and a different thread gives a different key. - Responses server-side `conversation`: string and object forms give one key across turns; a different conversation gives a different key; the savings key stays `None`. - Responses WebSocket: first and later frames share one key; a different thread gets a different key. - Chat Completions: covered the same way. - `tests/gateway/test_contract_request_half.py`: per-session gateway keys, and `None` without a session. - Every new test fails on `main`. - **Updated test:** `tests/test_proxy_openai.py` now pins `conversation=None` for a body with no ids. ### Test Output ```text pytest (turn hooks, gateway, Codex WS lifecycle, proxy openai, conversation savings, passthrough, hook usage): 411 passed, 5 skipped after the server-side conversation fix, on the synced head: 101 passed (conversation, gateway, Responses suites) ruff@0.17.0 check / format --check: clean mypy headroom --ignore-missing-imports: 3 errors, none in changed files ``` `tests/test_turn_hook_usage.py` needs `LITELLM_LOCAL_MODEL_COST_MAP=True` in a fresh venv; that is an existing litellm price-map fetch, and it fails the same way on `main`. ## Real Behavior Proof - Environment: - macOS 27.0.1 (Darwin 27.0.0, arm64), Python 3.12.13. - The proxy ran from this branch in a fresh `uv` venv (`uv venv -p 3.12 && uv pip install -e ".[dev]"`) on 127.0.0.1:9100, with default `ProxyConfig` (optimize on). - Upstream was the real Anthropic API, with my own API key passed by environment variable. - Client was Claude Code 2.1.311 (`claude --version`) with `--model claude-haiku-5-5`. - Exact command / steps: 1. Start the proxy with a scratch-only launcher (not committed). It calls `register_turn_hook()` with a hook whose `on_request` appends `ctx.provider`, `ctx.model`, `ctx.conversation` and `len(ctx.messages)` to a JSONL file, then calls `run_server(ProxyConfig(host="127.0.0.1", port=9100))`. Run it with `PYTHONSAFEPATH=1 HEADROOM_TELEMETRY=off LITELLM_LOCAL_MODEL_COST_MAP=True python run_proxy.py hooks.jsonl`, from a directory outside the repo. 2. Point Claude Code at the proxy: `export ANTHROPIC_BASE_URL=http://127.0.0.1:9100 CLAUDE_CONFIG_DIR=<fresh empty dir>`. 3. Start two sessions with the same prompt: `claude -p "say hi" --model claude-haiku-5-5 --output-format json`, run twice. Note each `session_id`. 4. Add two more turns to session 1: `claude -p "now say bye" --resume <session 1 id> --model claude-haiku-5-5`, then `claude -p "one more word please" --resume <session 1 id> --model claude-haiku-5-5`. 5. Read `hooks.jsonl`, and compare each key with sha256(`claude-code\0` + session_id). - Observed result (from `hooks.jsonl`, keys truncated): | Session | Turn | Messages | `ctx.conversation` | | --- | --- | --- | --- | | `305d64a9…` | first | 2 | `306306eeb9bc164b…` | | `4e50609f…` (same prompt) | first | 2 | `4a9e51bc7f8005f2…` | | `305d64a9…` | first `--resume` | 4 | `306306eeb9bc164b…` | | `305d64a9…` | second `--resume` | 6 | `306306eeb9bc164b…` | Each key equals sha256(`claude-code\0<session id>`) of the session id that Claude Code reported. All three runs answered normally, and the proxy log had no errors. - Not tested: - Codex over Responses HTTP or WebSocket (unit and handler tests cover these). - Chat Completions with `X-Headroom-Session-Id`. - The `/v1/compress` gateway path. - The server-side `conversation` body form, live (covered by the handler test). - The `X-Headroom-Session-Id` fallback on `/v1/messages`. - Claude Code `/compact` and subagents. - Bedrock and Vertex upstreams. - Linux and Windows. - A real plugin consuming the key. That is wired in headroomlabs-ai/headroom-condense#7; skill-search and tool-search adoption comes next. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: stable. - Stable/default behavior changed: no request or response bytes change. Registered turn hooks receive one new optional field; hooks that do not read it behave as before. - Kill switch / disable path: none needed; a hook that should not use the key ignores it (headroom-condense falls back to its previous key when the field is `None` or absent). - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this PR. Plugins read the field with `getattr`, so they keep working on cores without it. Cache note: compare keys only within one endpoint, since the same header gives different keys on `/v1/messages` and `/v1/responses`. Each key comes from a client id that stays the same for the whole conversation (Claude Code session ids survive retries, `--resume` and `/compact`; Codex `thread_id` wins over `session_id`, so guardian and subagent requests do not collide), and the Responses key is taken before compression rewrites the body, so a pinning hook keeps the same pin on every turn. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A (proxy-side change). ## Additional Notes - "No new warnings" is unchecked because the run was not compared warning by warning against `main`. - Follow-ups: skill-search and tool-search switch to `conversation_for(ctx)` once headroom-condense#7 is released. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Maintainer post-merge qualification Published `819f82117a4737de621528e9c49aee6aeeb4a793` includes upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 28 conversation/Responses/per-frame-memory/disconnect tests pass; Ruff/format/diff pass. Clean main merge; hook conversation policy unchanged, no new code finding. Author explicitly reserves unchecked self-review/readiness attestations; existing hold remains. Contributor correctness logic was not rewritten. Existing Changes Requested decision retained without duplicate findings. Scoped Windows validation uses the existing development native extension, not a fresh build. Hosted qualification is handled by maintainers; nothing merged. ### Maintainer current-main requalification Exact head `965ff21ffbea9b446e5e39269812bb0d8c43c108`, upstream main `ecbaf86d78cf52d185fbffa306157b9a4cc1b906`. Mechanically updated from upstream/mainecbaf86d78with no conflicts and no contributor conversation/ownership policy changes. Re-reviewed current discussion and the integration delta. All 38 focused conversation-context, Responses, per-frame WebSocket memory, disconnect and Gemini tests pass; Ruff0.17.3 lint/format and whitespace checks pass. Existing runtime evidence remains scoped; local tests use the existing native extension, not a fresh build. No new source blocker was found. The existing readiness action remains: the author has explicitly reserved and still not checked the self-review and ready-for-human-review attestations. They are not asserted on the author's behalf. Preserve the existing Changes Requested readiness hold until the author completes that action. Maintainers handle exact-head hosted checks; no contributor CI action requested. Nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `db69aa725d085d75b0881f0b2e1f2237480ee0c6`, upstream main `8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`. Clean mechanical sync with upstream main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a. Re-reviewed conversation-context integration and fresh author/Devin discussion without changing contributor identity/ownership policy.38conversation/Responses/per-frame-WebSocket/disconnect/Gemini tests pass;10changed Python files pass Ruff0.17.3check/format and whitespace checks. Existing runtime evidence remains scoped; copied extension supports Python startup, not a fresh native build. No new source blocker found. The current body still leaves author self-review and ready-for-human-review attestations unchecked and reserves those actions for the author. Preserve that existing readiness hold; no boxes were ticked or attested on their behalf. This is not an inherited-main CI defect. Maintainers handle exact-head hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `7411de5607beabfcbbd63b134795b0053da0e681`, upstream main `7592e681459f14a2041eb5f524a658cb6ad1dd92`. Clean current-main sync7592e681459f14a2041eb5f524a658cb6ad1dd92,including latestmemoryfixes. Conversationidentity/ownership policyunchanged.46conversation/Responses/perframeWS/disconnect/memory-regression/Gemini tests pass;whitespace passes. PriorRuff0.17.3check/format applies to unchangedPRfiles. Freshbodystillreservesuncheckedself-review/ready-for-humanreviewboxesfortheauthor. No newsourceblockerfound;retainexistingauthor-readinessholdwithouttickingboxesonbehalf. Existingruntimeevidence scoped,no freshnativebuildclaim. Maintainershandlechecks;nothingmerged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `cdf48ae9561dd2075625a4085594ac429a421231`, upstream main `2d3ee51846435ef5b320eea4806af14e602b0d8c`. Clean integration with upstream/main2d3ee51846after the external 0.43.1 release merge. Exact headroom/crates/tests comparison against previously reviewed7411de560shows no changes. Re-reviewed generated metadata; 78 release/version/MCP tests pass on this tree and whitespace passes. Earlier 46 conversation/Responses/WebSocket/disconnect/memory/Gemini passes remain scoped to unchanged inputs. No new source blocker found in reviewed scope. Existing author-owned readiness declaration remains the hold; fresh body must be checked before publication without checking author attestations on their behalf. No contributor identity/ownership policy rewritten, no fresh native-build/provider claim. Maintainers handle checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `d343c187f73b6a9005f0268e280ec657f435f1df`, upstream main `ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`. Clean integration with upstream/mainea59ac3eefafter the Click/macOS setup merge. Re-reviewed the OpenAI response replay transport-header overlap; conversation and existing tool-name map parameters remain additive.14 conversation/context-compaction tests and3 targeted route/gateway cases pass (71 unrelated cases deselected); Ruff and whitespace pass. No new source blocker found. Fresh body still leaves author-owned self-review/readiness attestations unchecked; preserve the existing hold without attesting on their behalf. Existing wider runtime evidence remains scoped; no fresh native-build or live-provider claim. Maintainers handle hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `741582654b9d0817ca8318af6aaf2dd000d1ec81`, upstream main `c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`. Final post-Dependabot requalification against upstream/mainc01cf07d94. All six Dependabot updates are now merged. Relative to this branch's freshly tested prepared tree, the final main sync changes only the three intended Cargo.lock records; Python/source/test inputs are unchanged. Re-reviewed the integration and confirmed complete native source/lock/toolchain objects match the tested combined build and exact fresh native DLL used for this branch's local tests. Focused fresh-native qualification: 18 passed, 70 deselected, 1 warning in 20.19s. Selected real dependency overlays and native DLL paths are recorded in final4496-fresh-native-tests.log; the DLL provenance/input hashes are in final-native-artifact-proof.json. No redundant rerun is implied after this Cargo-only merge. Whitespace passes. Prior source review and scoped real-behavior evidence remain applicable to unchanged contributor inputs. No new source blocker found. Fresh body still leaves author self-review and ready-for-human-review attestations unchecked. Retain the author-owned readiness hold without checking those boxes on their behalf. Maintainers handle hosted checks; no external-provider claim, contributor semantic rewrite or non-Dependabot merge. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `cb5c34b6221880a811dd94d21c5e3f3e874ee565`, upstream main `da6d9736cd82074988c7d0a2360b49d9d8b6c0a0`. Current-main requalification againstda6d9736cd. All69 integration patches exactly equal c01cf07-to-current-main delta after diff-location normalization; conversation identity/ownership policy unchanged.88 conversation,Responses context-compaction,OpenAI route,gateway request-contract tests pass;1 deprecation warning. Post-suite PyTorch cache-stat logging reports closed-stream logging errors during process exit; pytest exit0 and test results are retained without claiming full runtime qualification.31 changed Python files pass Ruff lint/format; offline lock and whitespace pass. Existing pr3982 native DLL verified full input hashes/artifact checksum, no fresh build claimed. Author/Devin discussion considered; previous substantive Devin findings resolved. No new source blocker found. Author explicitly reserves self-review/ready-for-human-review attestations and both remain unchecked; retain existing readiness hold without attesting on their behalf. Maintainers handle hosted checks; no contributor logic rewrite or merge. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `951bdf5853aaf8757ba46253763c91b6601b5c6d`, upstream main `0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`. Clean integration with upstream/main0ddaa56c21. All 19 integration patches exactly match the prior-reviewed-mainda6d9736to current-main delta after diff-location normalization; conversation identity/ownership policy is unchanged. Fresh conversation/turn-hook/Responses/gateway request-contract, doctor and incoming Vertex/backend/CLI suite: 367 passed, 11 skipped, 1 warning. Fourteen changed Python files pass Ruff lint/format; offline lock and whitespace checks pass. Author discussion and Devin findings read: both historical Devin items are marked resolved, including server-side conversation IDs. No new source blocker found. Existing author-reserved self-review/ready-for-human-review attestations remain a hold; this maintainer does not attest on the author's behalf. Static source/test scope and security inspection found no new workflow/dependency/setup path; the inherited flagged workflow-validation job is excluded by this exact PR's unchanged workflow paths. Conversation digests are not an authentication boundary. Verified recorded native input-set/hashes and artifact checksum before DLL reuse; no fresh build or new live-client qualification claimed. Maintainers handle exact-head hosted checks; no PR merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
257 lines
8.7 KiB
Python
257 lines
8.7 KiB
Python
"""``TurnContext.conversation``: one opaque key per client conversation.
|
|
|
|
Hooks that pin wording per conversation (headroom-condense) key on it, so two
|
|
sessions that open with the same prompt must get different keys, and every
|
|
turn of one conversation, retries and incremental Responses bodies included,
|
|
the same key.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
import sys
|
|
from typing import Any
|
|
from unittest.mock import patch
|
|
|
|
import httpx
|
|
import pytest
|
|
import respx
|
|
|
|
pytest.importorskip("fastapi")
|
|
|
|
from fastapi.testclient import TestClient # noqa: E402
|
|
|
|
import headroom.proxy.handlers.openai as openai_module # noqa: E402
|
|
from headroom.proxy.conversation_savings import savings_conversation_key # noqa: E402
|
|
from headroom.proxy.loopback_guard import require_loopback # noqa: E402
|
|
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
|
|
from headroom.proxy.turn_hooks import clear_turn_hooks, register_turn_hook # noqa: E402
|
|
from tests.test_openai_codex_ws_lifecycle import ( # noqa: E402
|
|
_DummyOpenAIHandler,
|
|
_FakeUpstream,
|
|
_FakeWebSocket,
|
|
_make_fake_websockets_module,
|
|
)
|
|
|
|
|
|
class _Recorder:
|
|
name = "conversation_recorder"
|
|
stream_safe = True
|
|
|
|
def __init__(self) -> None:
|
|
self.seen: list[str | None] = []
|
|
|
|
def on_request(self, ctx: Any) -> None:
|
|
self.seen.append(ctx.conversation)
|
|
|
|
|
|
@pytest.fixture
|
|
def recorder(monkeypatch):
|
|
# litellm fetches its price map on import unless told not to; respx would
|
|
# reject that request.
|
|
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
|
|
clear_turn_hooks()
|
|
hook = _Recorder()
|
|
register_turn_hook(hook)
|
|
yield hook
|
|
clear_turn_hooks()
|
|
|
|
|
|
def _app(*, optimize: bool):
|
|
app = create_app(
|
|
ProxyConfig(
|
|
optimize=optimize,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
)
|
|
)
|
|
app.dependency_overrides[require_loopback] = lambda: None
|
|
return app
|
|
|
|
|
|
PROMPT = [{"role": "user", "content": "fix the bug"}]
|
|
|
|
|
|
@respx.mock
|
|
def test_claude_code_sessions_with_the_same_prompt_get_different_conversations(recorder):
|
|
respx.post("https://api.anthropic.com/v1/messages").mock(
|
|
return_value=httpx.Response(
|
|
200,
|
|
json={
|
|
"id": "msg_1",
|
|
"type": "message",
|
|
"role": "assistant",
|
|
"model": "claude-haiku-4-5",
|
|
"content": [{"type": "text", "text": "ok"}],
|
|
"stop_reason": "end_turn",
|
|
"usage": {"input_tokens": 10, "output_tokens": 1},
|
|
},
|
|
)
|
|
)
|
|
body = {"model": "claude-haiku-4-5", "max_tokens": 16, "messages": PROMPT}
|
|
sessions = [
|
|
{"x-claude-code-session-id": "s-a"},
|
|
{"x-claude-code-session-id": "s-a"}, # a retry, or the next turn
|
|
{"x-claude-code-session-id": "s-b"},
|
|
{"x-headroom-session-id": "s-a"},
|
|
{},
|
|
]
|
|
with TestClient(_app(optimize=False)) as client:
|
|
for headers in sessions:
|
|
r = client.post(
|
|
"/v1/messages", json=body, headers={"x-api-key": "sk-ant-test", **headers}
|
|
)
|
|
assert r.status_code == 200, r.text
|
|
a, a_again, b, headroom_a, none = recorder.seen
|
|
assert a == a_again and a is not None
|
|
assert b not in (None, a)
|
|
# Namespaced by source, and never the raw client id.
|
|
assert headroom_a not in (None, a)
|
|
assert "s-a" not in (a, headroom_a)
|
|
assert none is None
|
|
|
|
|
|
@respx.mock
|
|
def test_responses_http_turns_of_one_thread_share_a_conversation(recorder):
|
|
respx.post("https://api.openai.com/v1/responses").mock(
|
|
return_value=httpx.Response(
|
|
200,
|
|
json={
|
|
"id": "resp_1",
|
|
"output": [{"type": "message", "role": "assistant", "content": []}],
|
|
"usage": {"input_tokens": 10, "output_tokens": 1},
|
|
},
|
|
)
|
|
)
|
|
|
|
def body(thread: str, **extra: Any) -> dict[str, Any]:
|
|
return {
|
|
"model": "gpt-4o",
|
|
"input": [{"type": "message", "role": "user", "content": "fix the bug"}],
|
|
"stream": False,
|
|
"client_metadata": {"thread_id": thread},
|
|
**extra,
|
|
}
|
|
|
|
with TestClient(_app(optimize=True)) as client:
|
|
for sent in (body("t-1"), body("t-1", previous_response_id="resp_0"), body("t-2")):
|
|
r = client.post("/v1/responses", json=sent, headers={"authorization": "Bearer sk-t"})
|
|
assert r.status_code == 200, r.text
|
|
first, incremental, other = recorder.seen
|
|
assert first == incremental and first is not None
|
|
assert other not in (None, first)
|
|
|
|
|
|
@respx.mock
|
|
def test_responses_server_side_conversation_names_the_conversation(recorder):
|
|
respx.post("https://api.openai.com/v1/responses").mock(
|
|
return_value=httpx.Response(
|
|
200,
|
|
json={
|
|
"id": "resp_1",
|
|
"output": [{"type": "message", "role": "assistant", "content": []}],
|
|
"usage": {"input_tokens": 10, "output_tokens": 1},
|
|
},
|
|
)
|
|
)
|
|
|
|
def body(conversation: Any) -> dict[str, Any]:
|
|
return {
|
|
"model": "gpt-4o",
|
|
"input": [{"type": "message", "role": "user", "content": "fix the bug"}],
|
|
"stream": False,
|
|
"conversation": conversation,
|
|
}
|
|
|
|
sent = [body("conv_1"), body("conv_1"), body({"id": "conv_1"}), body("conv_2")]
|
|
with TestClient(_app(optimize=True)) as client:
|
|
for one in sent:
|
|
r = client.post("/v1/responses", json=one, headers={"authorization": "Bearer sk-t"})
|
|
assert r.status_code == 200, r.text
|
|
first, again, as_object, other = recorder.seen
|
|
assert first == again == as_object and first is not None
|
|
assert other not in (None, first)
|
|
# The savings funnel still books these per request: the provider holds the history.
|
|
assert all(savings_conversation_key(one) is None for one in sent)
|
|
|
|
|
|
def _ws_frame(thread: str, **extra: Any) -> str:
|
|
response = {
|
|
"model": "gpt-5.4",
|
|
"input": "fix the bug",
|
|
"client_metadata": {"thread_id": thread},
|
|
**extra,
|
|
}
|
|
return json.dumps({"type": "response.create", "response": response})
|
|
|
|
|
|
async def _ws_conversations(frames: list[str]) -> list[str | None]:
|
|
"""The ``conversation`` each WS frame's compression pass was handed."""
|
|
seen: list[str | None] = []
|
|
|
|
def _compress(payload, *, model, request_id, conversation=None, **_kw):
|
|
seen.append(conversation)
|
|
return payload, False, 0, [], "router_no_compression", 10, 10, 0
|
|
|
|
client_ws = _FakeWebSocket(frames=frames, hold_after_initial=True)
|
|
handler = _DummyOpenAIHandler()
|
|
handler.config.optimize = True
|
|
handler._compress_openai_responses_payload = _compress # type: ignore[method-assign]
|
|
upstream = _FakeUpstream([], hold_after_events=True)
|
|
|
|
async def _disconnect() -> None:
|
|
await asyncio.sleep(0.05)
|
|
client_ws.trigger_disconnect()
|
|
|
|
with (
|
|
patch.object(openai_module, "COMPRESSION_TIMEOUT_SECONDS", 30.0),
|
|
patch.dict(sys.modules, {"websockets": _make_fake_websockets_module(upstream)}),
|
|
):
|
|
trigger = asyncio.create_task(_disconnect())
|
|
try:
|
|
await asyncio.wait_for(handler.handle_openai_responses_ws(client_ws), timeout=2.0)
|
|
finally:
|
|
trigger.cancel()
|
|
return seen
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_responses_ws_frames_of_one_thread_share_a_conversation():
|
|
first, later = await _ws_conversations(
|
|
[_ws_frame("t-1"), _ws_frame("t-1", previous_response_id="resp_1")]
|
|
)
|
|
(other,) = await _ws_conversations([_ws_frame("t-2")])
|
|
assert first == later and first is not None
|
|
assert other not in (None, first)
|
|
|
|
|
|
@respx.mock
|
|
def test_chat_completions_take_the_headroom_session_header(recorder):
|
|
respx.post("https://api.openai.com/v1/chat/completions").mock(
|
|
return_value=httpx.Response(
|
|
200,
|
|
json={
|
|
"id": "c",
|
|
"choices": [
|
|
{"message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}
|
|
],
|
|
"usage": {"prompt_tokens": 10, "completion_tokens": 1},
|
|
},
|
|
)
|
|
)
|
|
body = {"model": "gpt-4o", "messages": PROMPT}
|
|
with TestClient(_app(optimize=False)) as client:
|
|
for headers in ({"x-headroom-session-id": "h-1"}, {"x-headroom-session-id": "h-2"}, {}):
|
|
r = client.post(
|
|
"/v1/chat/completions",
|
|
json=body,
|
|
headers={"authorization": "Bearer sk-t", **headers},
|
|
)
|
|
assert r.status_code == 200, r.text
|
|
one, two, none = recorder.seen
|
|
assert one is not None and two not in (None, one)
|
|
assert none is None
|