mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description Plugins that pin wording per conversation (headroom-condense, and through it skill-search and tool-search) cannot see a real conversation id. Today: - **Claude:** they hash the first typed prompt, so two sessions that open with the same words share a key. - **Codex:** they get no key at all, so every new skill or MCP description waits out the one-hour hold. This PR adds an optional `TurnContext.conversation` that core fills from the client's own conversation id, so hooks can pin per real conversation. It is item #4 of the compression-gaps plan; the plugin work for #3, #6, #8 and #9 needs it to stay cache-safe. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **New field:** `TurnContext.conversation: str | None = None`. It is an opaque key that stays the same for every turn and retry of one client conversation, or `None` when the client names none. Hooks get a digest, never the raw id. | Path | Source | | --- | --- | | `/v1/messages` | sha256 of `X-Claude-Code-Session-Id` (Claude Code sends it on every request), falling back to `X-Headroom-Session-Id`; each source has its own namespace | | `/v1/responses` (HTTP and WebSocket) | the new `turn_conversation_key`, computed from the body before compression and passed through `_compress_openai_responses_payload_in_executor` | | `/v1/chat/completions` | `X-Headroom-Session-Id` only | | `/v1/compress` (gateway) | the gateway session id; `None` without one | - **Responses key.** `turn_conversation_key` uses the same identity and digest as `savings_conversation_key` (`thread_id` before `session_id`, then `client_metadata.*`, then the session header). Unlike the savings key, it also applies on `previous_response_id` turns and on bodies that name a server-side `conversation` (a string, or an object with an `id`), so the key does not drop out after the first WebSocket frame. `savings_conversation_key` keeps its existing exclusions. - **Gateway seam.** `CompressTurn.prepare()` is a third-party seam, so its signature is unchanged. Instead, the handler sets an optional writable `conversation` attribute on the turn before calling `prepare`. This is documented in `compress_turn.py`. - **No new ids.** Nothing is generated. The per-connection WebSocket uuid is not used, because a reconnect would change it. - **One path left out.** The catch-all passthrough Responses route still passes `None`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`): 3 errors, all in files this PR does not touch (embedders, model_metadata, langchain) - [x] New tests added for new functionality - [x] Manual testing performed - **New tests:** - `tests/test_turn_context_conversation.py`: - Anthropic: the same session gives one key, a different session with the same prompt gives a different key, the fallback header gets its own key, and no header gives `None`. - Responses HTTP: the same thread gives one key, including on a `previous_response_id` turn, and a different thread gives a different key. - Responses server-side `conversation`: string and object forms give one key across turns; a different conversation gives a different key; the savings key stays `None`. - Responses WebSocket: first and later frames share one key; a different thread gets a different key. - Chat Completions: covered the same way. - `tests/gateway/test_contract_request_half.py`: per-session gateway keys, and `None` without a session. - Every new test fails on `main`. - **Updated test:** `tests/test_proxy_openai.py` now pins `conversation=None` for a body with no ids. ### Test Output ```text pytest (turn hooks, gateway, Codex WS lifecycle, proxy openai, conversation savings, passthrough, hook usage): 411 passed, 5 skipped after the server-side conversation fix, on the synced head: 101 passed (conversation, gateway, Responses suites) ruff@0.17.0 check / format --check: clean mypy headroom --ignore-missing-imports: 3 errors, none in changed files ``` `tests/test_turn_hook_usage.py` needs `LITELLM_LOCAL_MODEL_COST_MAP=True` in a fresh venv; that is an existing litellm price-map fetch, and it fails the same way on `main`. ## Real Behavior Proof - Environment: - macOS 27.0.1 (Darwin 27.0.0, arm64), Python 3.12.13. - The proxy ran from this branch in a fresh `uv` venv (`uv venv -p 3.12 && uv pip install -e ".[dev]"`) on 127.0.0.1:9100, with default `ProxyConfig` (optimize on). - Upstream was the real Anthropic API, with my own API key passed by environment variable. - Client was Claude Code 2.1.311 (`claude --version`) with `--model claude-haiku-5-5`. - Exact command / steps: 1. Start the proxy with a scratch-only launcher (not committed). It calls `register_turn_hook()` with a hook whose `on_request` appends `ctx.provider`, `ctx.model`, `ctx.conversation` and `len(ctx.messages)` to a JSONL file, then calls `run_server(ProxyConfig(host="127.0.0.1", port=9100))`. Run it with `PYTHONSAFEPATH=1 HEADROOM_TELEMETRY=off LITELLM_LOCAL_MODEL_COST_MAP=True python run_proxy.py hooks.jsonl`, from a directory outside the repo. 2. Point Claude Code at the proxy: `export ANTHROPIC_BASE_URL=http://127.0.0.1:9100 CLAUDE_CONFIG_DIR=<fresh empty dir>`. 3. Start two sessions with the same prompt: `claude -p "say hi" --model claude-haiku-5-5 --output-format json`, run twice. Note each `session_id`. 4. Add two more turns to session 1: `claude -p "now say bye" --resume <session 1 id> --model claude-haiku-5-5`, then `claude -p "one more word please" --resume <session 1 id> --model claude-haiku-5-5`. 5. Read `hooks.jsonl`, and compare each key with sha256(`claude-code\0` + session_id). - Observed result (from `hooks.jsonl`, keys truncated): | Session | Turn | Messages | `ctx.conversation` | | --- | --- | --- | --- | | `305d64a9…` | first | 2 | `306306eeb9bc164b…` | | `4e50609f…` (same prompt) | first | 2 | `4a9e51bc7f8005f2…` | | `305d64a9…` | first `--resume` | 4 | `306306eeb9bc164b…` | | `305d64a9…` | second `--resume` | 6 | `306306eeb9bc164b…` | Each key equals sha256(`claude-code\0<session id>`) of the session id that Claude Code reported. All three runs answered normally, and the proxy log had no errors. - Not tested: - Codex over Responses HTTP or WebSocket (unit and handler tests cover these). - Chat Completions with `X-Headroom-Session-Id`. - The `/v1/compress` gateway path. - The server-side `conversation` body form, live (covered by the handler test). - The `X-Headroom-Session-Id` fallback on `/v1/messages`. - Claude Code `/compact` and subagents. - Bedrock and Vertex upstreams. - Linux and Windows. - A real plugin consuming the key. That is wired in headroomlabs-ai/headroom-condense#7; skill-search and tool-search adoption comes next. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: stable. - Stable/default behavior changed: no request or response bytes change. Registered turn hooks receive one new optional field; hooks that do not read it behave as before. - Kill switch / disable path: none needed; a hook that should not use the key ignores it (headroom-condense falls back to its previous key when the field is `None` or absent). - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this PR. Plugins read the field with `getattr`, so they keep working on cores without it. Cache note: compare keys only within one endpoint, since the same header gives different keys on `/v1/messages` and `/v1/responses`. Each key comes from a client id that stays the same for the whole conversation (Claude Code session ids survive retries, `--resume` and `/compact`; Codex `thread_id` wins over `session_id`, so guardian and subagent requests do not collide), and the Responses key is taken before compression rewrites the body, so a pinning hook keeps the same pin on every turn. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A (proxy-side change). ## Additional Notes - "No new warnings" is unchecked because the run was not compared warning by warning against `main`. - Follow-ups: skill-search and tool-search switch to `conversation_for(ctx)` once headroom-condense#7 is released. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Maintainer post-merge qualification Published `819f82117a4737de621528e9c49aee6aeeb4a793` includes upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 28 conversation/Responses/per-frame-memory/disconnect tests pass; Ruff/format/diff pass. Clean main merge; hook conversation policy unchanged, no new code finding. Author explicitly reserves unchecked self-review/readiness attestations; existing hold remains. Contributor correctness logic was not rewritten. Existing Changes Requested decision retained without duplicate findings. Scoped Windows validation uses the existing development native extension, not a fresh build. Hosted qualification is handled by maintainers; nothing merged. ### Maintainer current-main requalification Exact head `965ff21ffbea9b446e5e39269812bb0d8c43c108`, upstream main `ecbaf86d78cf52d185fbffa306157b9a4cc1b906`. Mechanically updated from upstream/mainecbaf86d78with no conflicts and no contributor conversation/ownership policy changes. Re-reviewed current discussion and the integration delta. All 38 focused conversation-context, Responses, per-frame WebSocket memory, disconnect and Gemini tests pass; Ruff0.17.3 lint/format and whitespace checks pass. Existing runtime evidence remains scoped; local tests use the existing native extension, not a fresh build. No new source blocker was found. The existing readiness action remains: the author has explicitly reserved and still not checked the self-review and ready-for-human-review attestations. They are not asserted on the author's behalf. Preserve the existing Changes Requested readiness hold until the author completes that action. Maintainers handle exact-head hosted checks; no contributor CI action requested. Nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `db69aa725d085d75b0881f0b2e1f2237480ee0c6`, upstream main `8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`. Clean mechanical sync with upstream main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a. Re-reviewed conversation-context integration and fresh author/Devin discussion without changing contributor identity/ownership policy.38conversation/Responses/per-frame-WebSocket/disconnect/Gemini tests pass;10changed Python files pass Ruff0.17.3check/format and whitespace checks. Existing runtime evidence remains scoped; copied extension supports Python startup, not a fresh native build. No new source blocker found. The current body still leaves author self-review and ready-for-human-review attestations unchecked and reserves those actions for the author. Preserve that existing readiness hold; no boxes were ticked or attested on their behalf. This is not an inherited-main CI defect. Maintainers handle exact-head hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `7411de5607beabfcbbd63b134795b0053da0e681`, upstream main `7592e681459f14a2041eb5f524a658cb6ad1dd92`. Clean current-main sync7592e681459f14a2041eb5f524a658cb6ad1dd92,including latestmemoryfixes. Conversationidentity/ownership policyunchanged.46conversation/Responses/perframeWS/disconnect/memory-regression/Gemini tests pass;whitespace passes. PriorRuff0.17.3check/format applies to unchangedPRfiles. Freshbodystillreservesuncheckedself-review/ready-for-humanreviewboxesfortheauthor. No newsourceblockerfound;retainexistingauthor-readinessholdwithouttickingboxesonbehalf. Existingruntimeevidence scoped,no freshnativebuildclaim. Maintainershandlechecks;nothingmerged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `cdf48ae9561dd2075625a4085594ac429a421231`, upstream main `2d3ee51846435ef5b320eea4806af14e602b0d8c`. Clean integration with upstream/main2d3ee51846after the external 0.43.1 release merge. Exact headroom/crates/tests comparison against previously reviewed7411de560shows no changes. Re-reviewed generated metadata; 78 release/version/MCP tests pass on this tree and whitespace passes. Earlier 46 conversation/Responses/WebSocket/disconnect/memory/Gemini passes remain scoped to unchanged inputs. No new source blocker found in reviewed scope. Existing author-owned readiness declaration remains the hold; fresh body must be checked before publication without checking author attestations on their behalf. No contributor identity/ownership policy rewritten, no fresh native-build/provider claim. Maintainers handle checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `d343c187f73b6a9005f0268e280ec657f435f1df`, upstream main `ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`. Clean integration with upstream/mainea59ac3eefafter the Click/macOS setup merge. Re-reviewed the OpenAI response replay transport-header overlap; conversation and existing tool-name map parameters remain additive.14 conversation/context-compaction tests and3 targeted route/gateway cases pass (71 unrelated cases deselected); Ruff and whitespace pass. No new source blocker found. Fresh body still leaves author-owned self-review/readiness attestations unchecked; preserve the existing hold without attesting on their behalf. Existing wider runtime evidence remains scoped; no fresh native-build or live-provider claim. Maintainers handle hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `741582654b9d0817ca8318af6aaf2dd000d1ec81`, upstream main `c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`. Final post-Dependabot requalification against upstream/mainc01cf07d94. All six Dependabot updates are now merged. Relative to this branch's freshly tested prepared tree, the final main sync changes only the three intended Cargo.lock records; Python/source/test inputs are unchanged. Re-reviewed the integration and confirmed complete native source/lock/toolchain objects match the tested combined build and exact fresh native DLL used for this branch's local tests. Focused fresh-native qualification: 18 passed, 70 deselected, 1 warning in 20.19s. Selected real dependency overlays and native DLL paths are recorded in final4496-fresh-native-tests.log; the DLL provenance/input hashes are in final-native-artifact-proof.json. No redundant rerun is implied after this Cargo-only merge. Whitespace passes. Prior source review and scoped real-behavior evidence remain applicable to unchanged contributor inputs. No new source blocker found. Fresh body still leaves author self-review and ready-for-human-review attestations unchecked. Retain the author-owned readiness hold without checking those boxes on their behalf. Maintainers handle hosted checks; no external-provider claim, contributor semantic rewrite or non-Dependabot merge. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `cb5c34b6221880a811dd94d21c5e3f3e874ee565`, upstream main `da6d9736cd82074988c7d0a2360b49d9d8b6c0a0`. Current-main requalification againstda6d9736cd. All69 integration patches exactly equal c01cf07-to-current-main delta after diff-location normalization; conversation identity/ownership policy unchanged.88 conversation,Responses context-compaction,OpenAI route,gateway request-contract tests pass;1 deprecation warning. Post-suite PyTorch cache-stat logging reports closed-stream logging errors during process exit; pytest exit0 and test results are retained without claiming full runtime qualification.31 changed Python files pass Ruff lint/format; offline lock and whitespace pass. Existing pr3982 native DLL verified full input hashes/artifact checksum, no fresh build claimed. Author/Devin discussion considered; previous substantive Devin findings resolved. No new source blocker found. Author explicitly reserves self-review/ready-for-human-review attestations and both remain unchecked; retain existing readiness hold without attesting on their behalf. Maintainers handle hosted checks; no contributor logic rewrite or merge. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `951bdf5853aaf8757ba46253763c91b6601b5c6d`, upstream main `0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`. Clean integration with upstream/main0ddaa56c21. All 19 integration patches exactly match the prior-reviewed-mainda6d9736to current-main delta after diff-location normalization; conversation identity/ownership policy is unchanged. Fresh conversation/turn-hook/Responses/gateway request-contract, doctor and incoming Vertex/backend/CLI suite: 367 passed, 11 skipped, 1 warning. Fourteen changed Python files pass Ruff lint/format; offline lock and whitespace checks pass. Author discussion and Devin findings read: both historical Devin items are marked resolved, including server-side conversation IDs. No new source blocker found. Existing author-reserved self-review/ready-for-human-review attestations remain a hold; this maintainer does not attest on the author's behalf. Static source/test scope and security inspection found no new workflow/dependency/setup path; the inherited flagged workflow-validation job is excluded by this exact PR's unchanged workflow paths. Conversation digests are not an authentication boundary. Verified recorded native input-set/hashes and artifact checksum before DLL reuse; no fresh build or new live-client qualification claimed. Maintainers handle exact-head hosted checks; no PR merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
663 lines
25 KiB
Python
663 lines
25 KiB
Python
"""Request half of the gateway turn contract: ``POST /v1/compress`` (spec section 1).
|
|
|
|
Invariants protected here:
|
|
|
|
* I-LEGACY - a body WITHOUT a top-level ``gateway`` object behaves exactly as
|
|
today: same response keys, no hooks run, no pending turn.
|
|
* I-KEYS - gateway mode adds ``body``/``turn_id``/``route``/``obligations``/
|
|
``gateway`` on EVERY answer, including fail-open ones.
|
|
* I-BODY - ``body`` is the complete provider request: pass-through fields
|
|
echoed untouched, ``config``/``gateway``/``token_budget`` stripped,
|
|
``body.messages`` is the top-level ``messages``.
|
|
* I-VALID - malformed ``gateway`` blocks are 400 ``invalid_request``.
|
|
* I-OBLIG - obligations follow the capability matrix: ``redrive`` only when a
|
|
re-driving hook ran under ``can_redrive`` + ``session_affinity``;
|
|
``relay_usage`` iff ``can_relay_response``.
|
|
* I-NOSHRINK - a re-driving hook never shrinks when nothing can re-drive.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import re
|
|
from typing import Any
|
|
|
|
import pytest
|
|
|
|
from headroom.proxy.turn_hooks import register_turn_hook
|
|
from tests.gateway import fold_only_hook, redrive_hook_ext
|
|
from tests.gateway.conftest import compress, registry_of
|
|
from tests.gateway.samples import (
|
|
SYSTEM_PROMPT,
|
|
anthropic_tool_history,
|
|
anthropic_tools,
|
|
big_tool_history,
|
|
canonical,
|
|
openai_tools,
|
|
tool_names,
|
|
)
|
|
|
|
LEGACY_STATELESS_KEYS = {
|
|
"messages",
|
|
"tokens_before",
|
|
"tokens_after",
|
|
"tokens_saved",
|
|
"compression_ratio",
|
|
"transforms_applied",
|
|
"transforms_summary",
|
|
"ccr_hashes",
|
|
}
|
|
GATEWAY_EXTRA_KEYS = {"body", "turn_id", "route", "obligations", "gateway"}
|
|
ROUTE_KEYS = {"model", "provider", "service_tier", "reason"}
|
|
GATEWAY_ECHO_KEYS = {"can_redrive", "can_relay_response", "session_affinity"}
|
|
HEX32 = re.compile(r"^[0-9a-f]{32}$")
|
|
|
|
|
|
class _RecordingHook:
|
|
"""A hook that records whether it ran; never changes anything."""
|
|
|
|
name = "recording"
|
|
stream_safe = True
|
|
|
|
def __init__(self) -> None:
|
|
self.requests = 0
|
|
|
|
def on_request(self, ctx: Any) -> None:
|
|
self.requests += 1
|
|
|
|
|
|
def _body(messages: list[dict[str, Any]] | None = None, **extra: Any) -> dict[str, Any]:
|
|
out: dict[str, Any] = {"model": "gpt-4o", "messages": messages or big_tool_history()}
|
|
out.update(extra)
|
|
return out
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# I-LEGACY (these pass on the current tree and must keep passing) #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_legacy_mode_keys_unchanged_stateless(headroom_client) -> None:
|
|
"""I-LEGACY: no ``gateway`` key -> exactly today's stateless key set."""
|
|
resp = compress(headroom_client, _body())
|
|
assert resp.status_code == 200, resp.text
|
|
assert set(resp.json()) == LEGACY_STATELESS_KEYS
|
|
|
|
|
|
def test_legacy_mode_keys_unchanged_session(headroom_client) -> None:
|
|
"""I-LEGACY: session mode adds only ``session``."""
|
|
resp = compress(headroom_client, _body(config={"session_id": "legacy-s"}))
|
|
assert resp.status_code == 200, resp.text
|
|
assert set(resp.json()) == LEGACY_STATELESS_KEYS | {"session"}
|
|
|
|
|
|
def test_legacy_mode_ignores_pass_through_fields(headroom_client) -> None:
|
|
"""I-LEGACY: extra provider fields are neither echoed nor rejected."""
|
|
resp = compress(
|
|
headroom_client, _body(tools=openai_tools(), system=SYSTEM_PROMPT, temperature=0.2)
|
|
)
|
|
assert resp.status_code == 200, resp.text
|
|
assert set(resp.json()) == LEGACY_STATELESS_KEYS
|
|
|
|
|
|
def test_legacy_mode_runs_no_hooks(headroom_client) -> None:
|
|
"""I-LEGACY: turn hooks are a gateway-mode feature; legacy callers never see them."""
|
|
hook = _RecordingHook()
|
|
register_turn_hook(hook)
|
|
fold = fold_only_hook.register(max_chars=100)
|
|
resp = compress(headroom_client, _body())
|
|
assert resp.status_code == 200
|
|
assert hook.requests == 0
|
|
assert fold.request_calls == 0
|
|
# And the payload is untouched by the fold hook.
|
|
assert len(resp.json()["messages"][2]["content"]) > 100
|
|
|
|
|
|
def test_legacy_mode_registers_no_pending_turn(headroom_client) -> None:
|
|
"""I-LEGACY: nothing to wait for, nothing registered."""
|
|
compress(headroom_client, _body(config={"session_id": "legacy-p"}))
|
|
registry = registry_of(headroom_client)
|
|
assert registry is None or len(registry) == 0
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# I-KEYS / I-BODY #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def _assert_contract_shape(data: dict[str, Any], sent: dict[str, Any]) -> None:
|
|
assert GATEWAY_EXTRA_KEYS <= set(data), sorted(data)
|
|
assert HEX32.match(data["turn_id"]), data["turn_id"]
|
|
assert set(data["route"]) == ROUTE_KEYS
|
|
assert isinstance(data["obligations"], list)
|
|
assert set(data["gateway"]) == GATEWAY_ECHO_KEYS
|
|
body = data["body"]
|
|
assert isinstance(body, dict)
|
|
for forbidden in ("config", "gateway", "token_budget"):
|
|
assert forbidden not in body, forbidden
|
|
assert body["model"] == sent["model"]
|
|
assert body["messages"] == data["messages"]
|
|
|
|
|
|
def test_gateway_mode_adds_contract_keys(headroom_client) -> None:
|
|
"""I-KEYS: ``gateway: {}`` is gateway mode; the legacy keys are all still there."""
|
|
sent = _body(gateway={})
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert LEGACY_STATELESS_KEYS <= set(data)
|
|
_assert_contract_shape(data, sent)
|
|
assert data["obligations"] == []
|
|
assert data["gateway"] == {
|
|
"can_redrive": False,
|
|
"can_relay_response": False,
|
|
"session_affinity": True,
|
|
}
|
|
assert data["route"]["model"] == "gpt-4o"
|
|
assert data["route"]["provider"] == "openai"
|
|
assert data["route"]["service_tier"] is None
|
|
assert data["route"]["reason"] == ""
|
|
# Compression still happened in gateway mode.
|
|
assert data["tokens_saved"] > 0
|
|
assert data["body"]["messages"][2]["content"] != sent["messages"][2]["content"]
|
|
|
|
|
|
def test_gateway_mode_with_session(headroom_client) -> None:
|
|
"""I-KEYS: session mode + gateway mode -> both key sets."""
|
|
sent = _body(config={"session_id": "gw-s1"}, gateway={"can_relay_response": True})
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
_assert_contract_shape(data, sent)
|
|
assert data["session"]["id"] == "gw-s1"
|
|
assert data["gateway"]["can_relay_response"] is True
|
|
|
|
|
|
def test_pass_through_fields_echoed_untouched(headroom_client) -> None:
|
|
"""I-BODY: unknown provider fields come back byte-identical inside ``body``."""
|
|
metadata = {"user_id": "u-1", "nested": {"k": [1, 2, {"z": None}]}}
|
|
sent = _body(
|
|
gateway={},
|
|
temperature=0.25,
|
|
top_p=0.9,
|
|
service_tier="flex",
|
|
metadata=metadata,
|
|
stream=False,
|
|
response_format={"type": "json_object"},
|
|
token_budget=50_000,
|
|
)
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
body = resp.json()["body"]
|
|
for key in ("temperature", "top_p", "service_tier", "metadata", "stream", "response_format"):
|
|
assert canonical(body[key]) == canonical(sent[key]), key
|
|
assert "token_budget" not in body
|
|
assert resp.json()["route"]["service_tier"] == "flex"
|
|
|
|
|
|
def test_tools_and_system_echoed(headroom_client) -> None:
|
|
"""I-BODY: ``system`` passes through as sent; ``tools`` come back (possibly
|
|
compacted) with every tool name preserved and in order."""
|
|
sent = _body(gateway={}, tools=openai_tools(), system=SYSTEM_PROMPT)
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
body = resp.json()["body"]
|
|
assert body["system"] == SYSTEM_PROMPT
|
|
assert tool_names(body["tools"]) == tool_names(sent["tools"])
|
|
|
|
|
|
def test_no_tools_means_no_tools_key(headroom_client) -> None:
|
|
"""I-BODY: a request without ``tools`` never grows one (an empty ``tools``
|
|
list is a provider 400 on Anthropic)."""
|
|
resp = compress(headroom_client, _body(gateway={}))
|
|
assert resp.status_code == 200
|
|
assert "tools" not in resp.json()["body"]
|
|
|
|
|
|
def test_tool_schema_compaction_is_deterministic(headroom_client) -> None:
|
|
"""I-BODY / prefix stability: identical tools in -> identical tool bytes out,
|
|
on every call, and the transform is named when it changed anything."""
|
|
sent = _body(gateway={}, tools=openai_tools())
|
|
first = compress(headroom_client, sent).json()
|
|
second = compress(headroom_client, sent).json()
|
|
assert canonical(first["body"]["tools"]) == canonical(second["body"]["tools"])
|
|
changed = canonical(first["body"]["tools"]) != canonical(sent["tools"])
|
|
assert ("tool_schema_compaction" in first["transforms_applied"]) == changed
|
|
|
|
|
|
def test_route_provider_inferred_from_model(headroom_client) -> None:
|
|
"""I-BODY: ``route.provider`` follows the tracker-provider rule (claude/anthropic
|
|
in the model name -> anthropic, else openai) and ``route.model`` is the body model."""
|
|
sent = {
|
|
"model": "claude-sonnet-4-5",
|
|
"messages": anthropic_tool_history(),
|
|
"system": SYSTEM_PROMPT,
|
|
"tools": anthropic_tools(),
|
|
"max_tokens": 64,
|
|
"gateway": {},
|
|
}
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["route"]["provider"] == "anthropic"
|
|
assert data["route"]["model"] == "claude-sonnet-4-5"
|
|
assert data["body"]["max_tokens"] == 64
|
|
assert data["body"]["system"] == SYSTEM_PROMPT
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# I-VALID #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"gateway",
|
|
[
|
|
[],
|
|
"yes",
|
|
1,
|
|
True,
|
|
{"can_redrive": "true"},
|
|
{"can_relay_response": 1},
|
|
{"session_affinity": "no"},
|
|
{"can_redrive": None},
|
|
],
|
|
ids=[
|
|
"list",
|
|
"string",
|
|
"int",
|
|
"bool",
|
|
"redrive-str",
|
|
"relay-int",
|
|
"affinity-str",
|
|
"redrive-null",
|
|
],
|
|
)
|
|
def test_invalid_gateway_block_is_400(headroom_client, gateway: Any) -> None:
|
|
"""I-VALID: ``gateway`` must be an object whose flags are real booleans."""
|
|
resp = compress(headroom_client, _body(gateway=gateway))
|
|
assert resp.status_code == 400, resp.text
|
|
assert resp.json()["error"]["type"] == "invalid_request"
|
|
|
|
|
|
def test_unknown_gateway_keys_are_ignored(headroom_client) -> None:
|
|
"""I-VALID: forward-compatible - unknown keys inside ``gateway`` are dropped."""
|
|
resp = compress(
|
|
headroom_client,
|
|
_body(gateway={"plugin_version": "kong-headroom/0.1.0", "future_flag": {"x": 1}}),
|
|
)
|
|
assert resp.status_code == 200, resp.text
|
|
assert set(resp.json()["gateway"]) == GATEWAY_ECHO_KEYS
|
|
|
|
|
|
def test_legacy_validation_still_applies_in_gateway_mode(headroom_client) -> None:
|
|
"""I-VALID: the pre-existing 400s (missing model/messages) are unchanged."""
|
|
resp = compress(
|
|
headroom_client, {"messages": [{"role": "user", "content": "x"}], "gateway": {}}
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "model" in resp.json()["error"]["message"]
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# I-KEYS on fail-open answers #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_bypass_header_in_gateway_mode_still_returns_contract_keys(headroom_client) -> None:
|
|
"""I-KEYS: the bypass short-circuit must still hand the gateway a ``body`` it can
|
|
forward, a ``turn_id`` and an empty obligation list."""
|
|
sent = _body(gateway={"can_relay_response": True}, tools=openai_tools(), temperature=0.1)
|
|
resp = compress(headroom_client, sent, headers={"x-headroom-bypass": "true"})
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["messages"] == sent["messages"]
|
|
assert data["tokens_before"] == 0 and data["tokens_saved"] == 0
|
|
assert GATEWAY_EXTRA_KEYS <= set(data)
|
|
assert data["obligations"] == []
|
|
assert HEX32.match(data["turn_id"])
|
|
body = data["body"]
|
|
assert body["messages"] == sent["messages"]
|
|
assert canonical(body["tools"]) == canonical(sent["tools"])
|
|
assert body["temperature"] == 0.1
|
|
assert "gateway" not in body and "config" not in body
|
|
|
|
|
|
def test_empty_messages_in_gateway_mode_returns_contract_keys(headroom_client) -> None:
|
|
"""I-KEYS: the empty-messages short-circuit is also a gateway answer."""
|
|
sent = _body(messages=[], gateway={})
|
|
sent["messages"] = []
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["messages"] == []
|
|
assert GATEWAY_EXTRA_KEYS <= set(data)
|
|
assert data["obligations"] == []
|
|
assert data["body"]["messages"] == []
|
|
assert data["route"]["model"] == "gpt-4o"
|
|
|
|
|
|
def test_compression_timeout_fails_open_with_contract_keys(headroom_client, monkeypatch) -> None:
|
|
"""I-KEYS: stateless timeout -> 200 + originals + ``compression_skipped`` AND the
|
|
contract keys, so the gateway can forward without a special case."""
|
|
from unittest.mock import AsyncMock
|
|
|
|
proxy = headroom_client.app.state.proxy
|
|
monkeypatch.setattr(proxy, "_run_compression_in_executor", AsyncMock(side_effect=TimeoutError))
|
|
sent = _body(gateway={}, tools=openai_tools(), temperature=0.3)
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["compression_skipped"] is True
|
|
assert data["skip_reason"] == "compression_timeout"
|
|
assert data["messages"] == sent["messages"]
|
|
assert GATEWAY_EXTRA_KEYS <= set(data)
|
|
assert data["obligations"] == []
|
|
assert data["body"]["messages"] == sent["messages"]
|
|
assert data["body"]["temperature"] == 0.3
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# I-OBLIG / I-NOSHRINK: the capability matrix #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def _expected_obligations(
|
|
hook: str, can_redrive: bool, can_relay: bool, affinity: bool
|
|
) -> list[str]:
|
|
out: list[str] = []
|
|
if hook == "redrive" and can_redrive and affinity:
|
|
out.append("redrive")
|
|
if can_relay:
|
|
out.append("relay_usage")
|
|
return out
|
|
|
|
|
|
@pytest.mark.parametrize("hook", ["none", "fold", "redrive"])
|
|
@pytest.mark.parametrize("can_redrive", [False, True])
|
|
@pytest.mark.parametrize("can_relay", [False, True])
|
|
@pytest.mark.parametrize("affinity", [False, True])
|
|
def test_obligations_matrix(
|
|
headroom_client, hook: str, can_redrive: bool, can_relay: bool, affinity: bool
|
|
) -> None:
|
|
"""I-OBLIG + I-NOSHRINK over every capability combination and hook kind."""
|
|
fold = redrive = None
|
|
if hook == "fold":
|
|
fold = fold_only_hook.register(max_chars=100)
|
|
elif hook == "redrive":
|
|
redrive = redrive_hook_ext.register()
|
|
sent = _body(
|
|
gateway={
|
|
"can_redrive": can_redrive,
|
|
"can_relay_response": can_relay,
|
|
"session_affinity": affinity,
|
|
},
|
|
tools=openai_tools(),
|
|
)
|
|
resp = compress(headroom_client, sent)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["obligations"] == _expected_obligations(hook, can_redrive, can_relay, affinity)
|
|
assert data["gateway"] == {
|
|
"can_redrive": can_redrive,
|
|
"can_relay_response": can_relay,
|
|
"session_affinity": affinity,
|
|
}
|
|
names = tool_names(data["body"]["tools"])
|
|
if "redrive" in data["obligations"]:
|
|
assert redrive is not None and redrive.shrink_calls == 1
|
|
assert "search_tools" in names
|
|
assert not any(n.startswith("deferred_") for n in names)
|
|
else:
|
|
# I-NOSHRINK: tools are a subset of what came in, in order.
|
|
assert "search_tools" not in names
|
|
assert names == tool_names(sent["tools"])
|
|
if redrive is not None:
|
|
assert redrive.shrink_calls == 0
|
|
if fold is not None:
|
|
# A stream-safe hook runs regardless of the redrive capability.
|
|
assert fold.request_calls == 1
|
|
assert data["body"]["messages"][2]["content"].endswith(fold_only_hook.FOLD_MARKER)
|
|
assert "turn_hook" in data["transforms_applied"]
|
|
# A pending turn exists iff there is something to wait for.
|
|
registry = registry_of(headroom_client)
|
|
assert registry is not None
|
|
if data["obligations"]:
|
|
assert registry.get(data["turn_id"]) is not None
|
|
else:
|
|
assert registry.get(data["turn_id"]) is None
|
|
|
|
|
|
def test_redrive_hook_tools_delta_is_reported(headroom_client) -> None:
|
|
"""I-OBLIG: a hook that shrinks tools is reported as ``turn_hook:tools:<n>tok``
|
|
(mirrors the chat path) so the savings are attributable."""
|
|
redrive_hook_ext.register()
|
|
sent = _body(gateway={"can_redrive": True}, tools=openai_tools())
|
|
data = compress(headroom_client, sent).json()
|
|
assert data["obligations"] == ["redrive"]
|
|
tags = [t for t in data["transforms_applied"] if t.startswith("turn_hook:tools:")]
|
|
assert len(tags) == 1, data["transforms_applied"]
|
|
assert re.match(r"^turn_hook:tools:\d+tok$", tags[0])
|
|
|
|
|
|
def test_session_mode_replays_hook_output(headroom_client) -> None:
|
|
"""I-BODY in session mode: the hook-rewritten messages are what the gateway
|
|
forwards, so they must be what turn 2 replays byte-for-byte."""
|
|
fold_only_hook.register(max_chars=120)
|
|
history = big_tool_history()
|
|
turn1 = compress(
|
|
headroom_client, _body(history, config={"session_id": "gw-fold"}, gateway={})
|
|
).json()
|
|
folded = turn1["body"]["messages"][2]["content"]
|
|
assert folded.endswith(fold_only_hook.FOLD_MARKER)
|
|
turn2 = compress(
|
|
headroom_client,
|
|
_body(
|
|
history + [{"role": "assistant", "content": "ok"}, {"role": "user", "content": "more"}],
|
|
config={"session_id": "gw-fold"},
|
|
gateway={},
|
|
),
|
|
).json()
|
|
assert canonical(turn2["body"]["messages"][: len(turn1["body"]["messages"])]) == canonical(
|
|
turn1["body"]["messages"]
|
|
)
|
|
assert turn2["body"]["messages"][-1]["content"] == "more"
|
|
|
|
|
|
def test_turn_ids_are_unique_per_request(headroom_client) -> None:
|
|
"""I-KEYS: every request half mints a fresh id, registered or not."""
|
|
ids = {compress(headroom_client, _body(gateway={})).json()["turn_id"] for _ in range(5)}
|
|
assert len(ids) == 5
|
|
|
|
|
|
class _ConversationHook:
|
|
name = "conversation"
|
|
stream_safe = True
|
|
|
|
def __init__(self) -> None:
|
|
self.seen: list[str | None] = []
|
|
|
|
def on_request(self, ctx: Any) -> None:
|
|
self.seen.append(ctx.conversation)
|
|
|
|
|
|
def test_hooks_see_the_gateway_session_as_the_conversation(headroom_client) -> None:
|
|
"""Each gateway session is one conversation; no session, no conversation."""
|
|
hook = _ConversationHook()
|
|
register_turn_hook(hook)
|
|
for session in ("gw-c1", "gw-c1", "gw-c2"):
|
|
resp = compress(headroom_client, _body(config={"session_id": session}, gateway={}))
|
|
assert resp.status_code == 200, resp.text
|
|
resp = compress(headroom_client, _body(gateway={}))
|
|
assert resp.status_code == 200, resp.text
|
|
first, again, other, none = hook.seen
|
|
assert first == again and first is not None and first != "gw-c1"
|
|
assert other not in (None, first)
|
|
assert none is None
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("label", "shape", "model", "provider", "route_provider"),
|
|
[
|
|
(
|
|
"anthropic_system",
|
|
{"messages": [{"role": "user", "content": "x"}], "system": SYSTEM_PROMPT},
|
|
"gemini/gemini-2.5-pro",
|
|
"anthropic",
|
|
"openai",
|
|
),
|
|
(
|
|
"anthropic_tools",
|
|
{"messages": [{"role": "user", "content": "x"}], "tools": anthropic_tools()},
|
|
"gemini/gemini-2.5-pro",
|
|
"anthropic",
|
|
"openai",
|
|
),
|
|
(
|
|
"anthropic_history_with_server_tool",
|
|
{
|
|
"messages": anthropic_tool_history(),
|
|
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
|
|
},
|
|
"gpt-5.5",
|
|
"anthropic",
|
|
"openai",
|
|
),
|
|
(
|
|
"anthropic_first_turn_server_tool_only",
|
|
{
|
|
"messages": [{"role": "user", "content": "x"}],
|
|
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
|
|
},
|
|
"gemini/gemini-2.5-pro",
|
|
"anthropic",
|
|
"openai",
|
|
),
|
|
(
|
|
"openai_tools_to_claude",
|
|
{"messages": [{"role": "user", "content": "x"}], "tools": openai_tools()},
|
|
"claude-opus-5-5",
|
|
"openai",
|
|
"anthropic",
|
|
),
|
|
(
|
|
"openai_tool_row_to_claude",
|
|
{"messages": [{"role": "tool", "tool_call_id": "c", "content": "x"}]},
|
|
"claude-opus-5-5",
|
|
"openai",
|
|
"anthropic",
|
|
),
|
|
(
|
|
"responses_input_to_claude",
|
|
{"input": [{"role": "user", "content": "x"}], "instructions": "be brief"},
|
|
"claude-opus-5-5",
|
|
"openai",
|
|
"anthropic",
|
|
),
|
|
(
|
|
"chat_system_role_to_claude",
|
|
{
|
|
"messages": [
|
|
{"role": "system", "content": "be brief"},
|
|
{"role": "user", "content": "x"},
|
|
]
|
|
},
|
|
"claude-opus-5-5",
|
|
"openai",
|
|
"anthropic",
|
|
),
|
|
(
|
|
"chat_image_to_claude",
|
|
{
|
|
"messages": [
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{"type": "image_url", "image_url": {"url": "https://x.test/a.png"}}
|
|
],
|
|
}
|
|
]
|
|
},
|
|
"claude-opus-5-5",
|
|
"openai",
|
|
"anthropic",
|
|
),
|
|
(
|
|
"anthropic_thinking_to_gpt",
|
|
{
|
|
"messages": [
|
|
{"role": "user", "content": "x"},
|
|
{
|
|
"role": "assistant",
|
|
"content": [{"type": "thinking", "thinking": "t", "signature": "s"}],
|
|
},
|
|
{"role": "user", "content": "y"},
|
|
]
|
|
},
|
|
"gpt-5.5",
|
|
"anthropic",
|
|
"openai",
|
|
),
|
|
(
|
|
"anthropic_server_tool_result_to_gpt",
|
|
{
|
|
"messages": [
|
|
{"role": "user", "content": "x"},
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{"type": "web_search_tool_result", "tool_use_id": "s", "content": []}
|
|
],
|
|
},
|
|
{"role": "user", "content": "y"},
|
|
],
|
|
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
|
|
},
|
|
"gpt-5.5",
|
|
"anthropic",
|
|
"openai",
|
|
),
|
|
(
|
|
"plain_text_follows_model",
|
|
{"messages": [{"role": "user", "content": "x"}], "system": None},
|
|
"claude-opus-5-5",
|
|
"anthropic",
|
|
"anthropic",
|
|
),
|
|
],
|
|
)
|
|
def test_hooks_see_the_wire_shape_and_route_follows_the_model(
|
|
headroom_client, label, shape, model, provider, route_provider
|
|
) -> None:
|
|
"""A gateway can send any client's shape to any model."""
|
|
seen: list[str] = []
|
|
|
|
class Recorder:
|
|
name = f"provider_recorder_{label}"
|
|
stream_safe = True
|
|
|
|
def on_request(self, ctx: Any) -> None:
|
|
seen.append(ctx.provider)
|
|
|
|
register_turn_hook(Recorder())
|
|
data = compress(headroom_client, {**shape, "model": model, "gateway": {}}).json()
|
|
assert seen == [provider]
|
|
assert data["route"]["provider"] == route_provider
|
|
|
|
|
|
def test_chat_body_to_claude_stays_within_the_cache_breakpoint_budget(headroom_client) -> None:
|
|
"""The budget belongs to the upstream model, not to the body's wire shape."""
|
|
marked = [
|
|
{"role": "user", "content": [{"type": "text", "text": f"m{i}", "cache_control": {}}]}
|
|
for i in range(7)
|
|
]
|
|
body = _body(messages=marked, tools=openai_tools(), model="claude-opus-5-5", gateway={})
|
|
data = compress(headroom_client, body).json()
|
|
sent = [
|
|
block
|
|
for message in data["body"]["messages"]
|
|
for block in message["content"]
|
|
if isinstance(block, dict) and "cache_control" in block
|
|
]
|
|
assert len(sent) <= 4
|
|
assert "cache_breakpoint_budget" in data["transforms_applied"]
|