Files
headroom/tests/gateway/test_contract_request_half.py
94a99ed20f feat(proxy): give turn hooks one conversation id per client conversation (#4496)
## Description

Plugins that pin wording per conversation (headroom-condense, and
through it skill-search and tool-search) cannot see a real conversation
id. Today:

- **Claude:** they hash the first typed prompt, so two sessions that
open with the same words share a key.
- **Codex:** they get no key at all, so every new skill or MCP
description waits out the one-hour hold.

This PR adds an optional `TurnContext.conversation` that core fills from
the client's own conversation id, so hooks can pin per real
conversation. It is item #4 of the compression-gaps plan; the plugin
work for #3, #6, #8 and #9 needs it to stay cache-safe.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **New field:** `TurnContext.conversation: str | None = None`. It is an
opaque key that stays the same for every turn and retry of one client
conversation, or `None` when the client names none. Hooks get a digest,
never the raw id.

  | Path | Source |
  | --- | --- |
| `/v1/messages` | sha256 of `X-Claude-Code-Session-Id` (Claude Code
sends it on every request), falling back to `X-Headroom-Session-Id`;
each source has its own namespace |
| `/v1/responses` (HTTP and WebSocket) | the new
`turn_conversation_key`, computed from the body before compression and
passed through `_compress_openai_responses_payload_in_executor` |
  | `/v1/chat/completions` | `X-Headroom-Session-Id` only |
| `/v1/compress` (gateway) | the gateway session id; `None` without one
|

- **Responses key.** `turn_conversation_key` uses the same identity and
digest as `savings_conversation_key` (`thread_id` before `session_id`,
then `client_metadata.*`, then the session header). Unlike the savings
key, it also applies on `previous_response_id` turns and on bodies that
name a server-side `conversation` (a string, or an object with an `id`),
so the key does not drop out after the first WebSocket frame.
`savings_conversation_key` keeps its existing exclusions.
- **Gateway seam.** `CompressTurn.prepare()` is a third-party seam, so
its signature is unchanged. Instead, the handler sets an optional
writable `conversation` attribute on the turn before calling `prepare`.
This is documented in `compress_turn.py`.
- **No new ids.** Nothing is generated. The per-connection WebSocket
uuid is not used, because a reconnect would change it.
- **One path left out.** The catch-all passthrough Responses route still
passes `None`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`): 3 errors, all in files
this PR does not touch (embedders, model_metadata, langchain)
- [x] New tests added for new functionality
- [x] Manual testing performed

- **New tests:**
  - `tests/test_turn_context_conversation.py`:
- Anthropic: the same session gives one key, a different session with
the same prompt gives a different key, the fallback header gets its own
key, and no header gives `None`.
- Responses HTTP: the same thread gives one key, including on a
`previous_response_id` turn, and a different thread gives a different
key.
- Responses server-side `conversation`: string and object forms give one
key across turns; a different conversation gives a different key; the
savings key stays `None`.
- Responses WebSocket: first and later frames share one key; a different
thread gets a different key.
    - Chat Completions: covered the same way.
- `tests/gateway/test_contract_request_half.py`: per-session gateway
keys, and `None` without a session.
  - Every new test fails on `main`.
- **Updated test:** `tests/test_proxy_openai.py` now pins
`conversation=None` for a body with no ids.

### Test Output

```text
pytest (turn hooks, gateway, Codex WS lifecycle, proxy openai, conversation savings, passthrough, hook usage): 411 passed, 5 skipped
after the server-side conversation fix, on the synced head: 101 passed (conversation, gateway, Responses suites)
ruff@0.17.0 check / format --check: clean
mypy headroom --ignore-missing-imports: 3 errors, none in changed files
```

`tests/test_turn_hook_usage.py` needs
`LITELLM_LOCAL_MODEL_COST_MAP=True` in a fresh venv; that is an existing
litellm price-map fetch, and it fails the same way on `main`.

## Real Behavior Proof

- Environment:
  - macOS 27.0.1 (Darwin 27.0.0, arm64), Python 3.12.13.
- The proxy ran from this branch in a fresh `uv` venv (`uv venv -p 3.12
&& uv pip install -e ".[dev]"`) on 127.0.0.1:9100, with default
`ProxyConfig` (optimize on).
- Upstream was the real Anthropic API, with my own API key passed by
environment variable.
- Client was Claude Code 2.1.311 (`claude --version`) with `--model
claude-haiku-5-5`.
- Exact command / steps:
1. Start the proxy with a scratch-only launcher (not committed). It
calls `register_turn_hook()` with a hook whose `on_request` appends
`ctx.provider`, `ctx.model`, `ctx.conversation` and `len(ctx.messages)`
to a JSONL file, then calls `run_server(ProxyConfig(host="127.0.0.1",
port=9100))`. Run it with `PYTHONSAFEPATH=1 HEADROOM_TELEMETRY=off
LITELLM_LOCAL_MODEL_COST_MAP=True python run_proxy.py hooks.jsonl`, from
a directory outside the repo.
2. Point Claude Code at the proxy: `export
ANTHROPIC_BASE_URL=http://127.0.0.1:9100 CLAUDE_CONFIG_DIR=<fresh empty
dir>`.
3. Start two sessions with the same prompt: `claude -p "say hi" --model
claude-haiku-5-5 --output-format json`, run twice. Note each
`session_id`.
4. Add two more turns to session 1: `claude -p "now say bye" --resume
<session 1 id> --model claude-haiku-5-5`, then `claude -p "one more word
please" --resume <session 1 id> --model claude-haiku-5-5`.
5. Read `hooks.jsonl`, and compare each key with sha256(`claude-code\0`
+ session_id).
- Observed result (from `hooks.jsonl`, keys truncated):

  | Session | Turn | Messages | `ctx.conversation` |
  | --- | --- | --- | --- |
  | `305d64a9…` | first | 2 | `306306eeb9bc164b…` |
  | `4e50609f…` (same prompt) | first | 2 | `4a9e51bc7f8005f2…` |
  | `305d64a9…` | first `--resume` | 4 | `306306eeb9bc164b…` |
  | `305d64a9…` | second `--resume` | 6 | `306306eeb9bc164b…` |

Each key equals sha256(`claude-code\0<session id>`) of the session id
that Claude Code reported. All three runs answered normally, and the
proxy log had no errors.
- Not tested:
- Codex over Responses HTTP or WebSocket (unit and handler tests cover
these).
  - Chat Completions with `X-Headroom-Session-Id`.
  - The `/v1/compress` gateway path.
- The server-side `conversation` body form, live (covered by the handler
test).
  - The `X-Headroom-Session-Id` fallback on `/v1/messages`.
  - Claude Code `/compact` and subagents.
  - Bedrock and Vertex upstreams.
  - Linux and Windows.
- A real plugin consuming the key. That is wired in
headroomlabs-ai/headroom-condense#7; skill-search and tool-search
adoption comes next.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: stable.
- Stable/default behavior changed: no request or response bytes change.
Registered turn hooks receive one new optional field; hooks that do not
read it behave as before.
- Kill switch / disable path: none needed; a hook that should not use
the key ignores it (headroom-condense falls back to its previous key
when the field is `None` or absent).
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this PR. Plugins read the field with `getattr`,
so they keep working on cores without it.

Cache note: compare keys only within one endpoint, since the same header
gives different keys on `/v1/messages` and `/v1/responses`. Each key
comes from a client id that stays the same for the whole conversation
(Claude Code session ids survive retries, `--resume` and `/compact`;
Codex `thread_id` wins over `session_id`, so guardian and subagent
requests do not collide), and the Responses key is taken before
compression rewrites the body, so a pinning hook keeps the same pin on
every turn.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A (proxy-side change).

## Additional Notes

- "No new warnings" is unchecked because the run was not compared
warning by warning against `main`.
- Follow-ups: skill-search and tool-search switch to
`conversation_for(ctx)` once headroom-condense#7 is released.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


## Maintainer post-merge qualification
Published `819f82117a4737de621528e9c49aee6aeeb4a793` includes upstream
main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 28
conversation/Responses/per-frame-memory/disconnect tests pass;
Ruff/format/diff pass. Clean main merge; hook conversation policy
unchanged, no new code finding. Author explicitly reserves unchecked
self-review/readiness attestations; existing hold remains. Contributor
correctness logic was not rewritten. Existing Changes Requested decision
retained without duplicate findings. Scoped Windows validation uses the
existing development native extension, not a fresh build. Hosted
qualification is handled by maintainers; nothing merged.


### Maintainer current-main requalification

Exact head `965ff21ffbea9b446e5e39269812bb0d8c43c108`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Mechanically updated from upstream/main
ecbaf86d78 with no conflicts and no
contributor conversation/ownership policy changes. Re-reviewed current
discussion and the integration delta. All 38 focused
conversation-context, Responses, per-frame WebSocket memory, disconnect
and Gemini tests pass; Ruff0.17.3 lint/format and whitespace checks
pass. Existing runtime evidence remains scoped; local tests use the
existing native extension, not a fresh build.

No new source blocker was found. The existing readiness action remains:
the author has explicitly reserved and still not checked the self-review
and ready-for-human-review attestations. They are not asserted on the
author's behalf. Preserve the existing Changes Requested readiness hold
until the author completes that action. Maintainers handle exact-head
hosted checks; no contributor CI action requested. Nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `db69aa725d085d75b0881f0b2e1f2237480ee0c6`, upstream main
`8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`.

Clean mechanical sync with upstream
main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a. Re-reviewed
conversation-context integration and fresh author/Devin discussion
without changing contributor identity/ownership
policy.38conversation/Responses/per-frame-WebSocket/disconnect/Gemini
tests pass;10changed Python files pass Ruff0.17.3check/format and
whitespace checks. Existing runtime evidence remains scoped; copied
extension supports Python startup, not a fresh native build.

No new source blocker found. The current body still leaves author
self-review and ready-for-human-review attestations unchecked and
reserves those actions for the author. Preserve that existing readiness
hold; no boxes were ticked or attested on their behalf. This is not an
inherited-main CI defect. Maintainers handle exact-head hosted checks;
nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `7411de5607beabfcbbd63b134795b0053da0e681`, upstream main
`7592e681459f14a2041eb5f524a658cb6ad1dd92`.

Clean current-main
sync7592e681459f14a2041eb5f524a658cb6ad1dd92,including
latestmemoryfixes. Conversationidentity/ownership
policyunchanged.46conversation/Responses/perframeWS/disconnect/memory-regression/Gemini
tests pass;whitespace passes. PriorRuff0.17.3check/format applies to
unchangedPRfiles.
Freshbodystillreservesuncheckedself-review/ready-for-humanreviewboxesfortheauthor.

No
newsourceblockerfound;retainexistingauthor-readinessholdwithouttickingboxesonbehalf.
Existingruntimeevidence scoped,no freshnativebuildclaim.
Maintainershandlechecks;nothingmerged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `cdf48ae9561dd2075625a4085594ac429a421231`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Clean integration with upstream/main
2d3ee51846 after the external 0.43.1
release merge. Exact headroom/crates/tests comparison against previously
reviewed 7411de560 shows no changes. Re-reviewed generated metadata; 78
release/version/MCP tests pass on this tree and whitespace passes.
Earlier 46 conversation/Responses/WebSocket/disconnect/memory/Gemini
passes remain scoped to unchanged inputs.

No new source blocker found in reviewed scope. Existing author-owned
readiness declaration remains the hold; fresh body must be checked
before publication without checking author attestations on their behalf.
No contributor identity/ownership policy rewritten, no fresh
native-build/provider claim. Maintainers handle checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `d343c187f73b6a9005f0268e280ec657f435f1df`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.

Clean integration with upstream/main
ea59ac3eef after the Click/macOS setup
merge. Re-reviewed the OpenAI response replay transport-header overlap;
conversation and existing tool-name map parameters remain additive.14
conversation/context-compaction tests and3 targeted route/gateway cases
pass (71 unrelated cases deselected); Ruff and whitespace pass. No new
source blocker found. Fresh body still leaves author-owned
self-review/readiness attestations unchecked; preserve the existing hold
without attesting on their behalf. Existing wider runtime evidence
remains scoped; no fresh native-build or live-provider claim.
Maintainers handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `741582654b9d0817ca8318af6aaf2dd000d1ec81`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 18 passed, 70 deselected, 1 warning
in 20.19s. Selected real dependency overlays and native DLL paths are
recorded in final4496-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

No new source blocker found. Fresh body still leaves author self-review
and ready-for-human-review attestations unchecked. Retain the
author-owned readiness hold without checking those boxes on their
behalf.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `cb5c34b6221880a811dd94d21c5e3f3e874ee565`, upstream main
`da6d9736cd82074988c7d0a2360b49d9d8b6c0a0`.

Current-main requalification against
da6d9736cd. All69 integration patches
exactly equal c01cf07-to-current-main delta after diff-location
normalization; conversation identity/ownership policy unchanged.88
conversation,Responses context-compaction,OpenAI route,gateway
request-contract tests pass;1 deprecation warning. Post-suite PyTorch
cache-stat logging reports closed-stream logging errors during process
exit; pytest exit0 and test results are retained without claiming full
runtime qualification.31 changed Python files pass Ruff lint/format;
offline lock and whitespace pass. Existing pr3982 native DLL verified
full input hashes/artifact checksum, no fresh build claimed.
Author/Devin discussion considered; previous substantive Devin findings
resolved. No new source blocker found. Author explicitly reserves
self-review/ready-for-human-review attestations and both remain
unchecked; retain existing readiness hold without attesting on their
behalf. Maintainers handle hosted checks; no contributor logic rewrite
or merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `951bdf5853aaf8757ba46253763c91b6601b5c6d`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 19 integration patches
exactly match the prior-reviewed-main da6d9736 to current-main delta
after diff-location normalization; conversation identity/ownership
policy is unchanged. Fresh conversation/turn-hook/Responses/gateway
request-contract, doctor and incoming Vertex/backend/CLI suite: 367
passed, 11 skipped, 1 warning. Fourteen changed Python files pass Ruff
lint/format; offline lock and whitespace checks pass.

Author discussion and Devin findings read: both historical Devin items
are marked resolved, including server-side conversation IDs. No new
source blocker found. Existing author-reserved
self-review/ready-for-human-review attestations remain a hold; this
maintainer does not attest on the author's behalf. Static source/test
scope and security inspection found no new workflow/dependency/setup
path; the inherited flagged workflow-validation job is excluded by this
exact PR's unchanged workflow paths. Conversation digests are not an
authentication boundary. Verified recorded native input-set/hashes and
artifact checksum before DLL reuse; no fresh build or new live-client
qualification claimed. Maintainers handle exact-head hosted checks; no
PR merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-09 14:56:34 -06:00

663 lines
25 KiB
Python

"""Request half of the gateway turn contract: ``POST /v1/compress`` (spec section 1).
Invariants protected here:
* I-LEGACY - a body WITHOUT a top-level ``gateway`` object behaves exactly as
today: same response keys, no hooks run, no pending turn.
* I-KEYS - gateway mode adds ``body``/``turn_id``/``route``/``obligations``/
``gateway`` on EVERY answer, including fail-open ones.
* I-BODY - ``body`` is the complete provider request: pass-through fields
echoed untouched, ``config``/``gateway``/``token_budget`` stripped,
``body.messages`` is the top-level ``messages``.
* I-VALID - malformed ``gateway`` blocks are 400 ``invalid_request``.
* I-OBLIG - obligations follow the capability matrix: ``redrive`` only when a
re-driving hook ran under ``can_redrive`` + ``session_affinity``;
``relay_usage`` iff ``can_relay_response``.
* I-NOSHRINK - a re-driving hook never shrinks when nothing can re-drive.
"""
from __future__ import annotations
import re
from typing import Any
import pytest
from headroom.proxy.turn_hooks import register_turn_hook
from tests.gateway import fold_only_hook, redrive_hook_ext
from tests.gateway.conftest import compress, registry_of
from tests.gateway.samples import (
SYSTEM_PROMPT,
anthropic_tool_history,
anthropic_tools,
big_tool_history,
canonical,
openai_tools,
tool_names,
)
LEGACY_STATELESS_KEYS = {
"messages",
"tokens_before",
"tokens_after",
"tokens_saved",
"compression_ratio",
"transforms_applied",
"transforms_summary",
"ccr_hashes",
}
GATEWAY_EXTRA_KEYS = {"body", "turn_id", "route", "obligations", "gateway"}
ROUTE_KEYS = {"model", "provider", "service_tier", "reason"}
GATEWAY_ECHO_KEYS = {"can_redrive", "can_relay_response", "session_affinity"}
HEX32 = re.compile(r"^[0-9a-f]{32}$")
class _RecordingHook:
"""A hook that records whether it ran; never changes anything."""
name = "recording"
stream_safe = True
def __init__(self) -> None:
self.requests = 0
def on_request(self, ctx: Any) -> None:
self.requests += 1
def _body(messages: list[dict[str, Any]] | None = None, **extra: Any) -> dict[str, Any]:
out: dict[str, Any] = {"model": "gpt-4o", "messages": messages or big_tool_history()}
out.update(extra)
return out
# --------------------------------------------------------------------------- #
# I-LEGACY (these pass on the current tree and must keep passing) #
# --------------------------------------------------------------------------- #
def test_legacy_mode_keys_unchanged_stateless(headroom_client) -> None:
"""I-LEGACY: no ``gateway`` key -> exactly today's stateless key set."""
resp = compress(headroom_client, _body())
assert resp.status_code == 200, resp.text
assert set(resp.json()) == LEGACY_STATELESS_KEYS
def test_legacy_mode_keys_unchanged_session(headroom_client) -> None:
"""I-LEGACY: session mode adds only ``session``."""
resp = compress(headroom_client, _body(config={"session_id": "legacy-s"}))
assert resp.status_code == 200, resp.text
assert set(resp.json()) == LEGACY_STATELESS_KEYS | {"session"}
def test_legacy_mode_ignores_pass_through_fields(headroom_client) -> None:
"""I-LEGACY: extra provider fields are neither echoed nor rejected."""
resp = compress(
headroom_client, _body(tools=openai_tools(), system=SYSTEM_PROMPT, temperature=0.2)
)
assert resp.status_code == 200, resp.text
assert set(resp.json()) == LEGACY_STATELESS_KEYS
def test_legacy_mode_runs_no_hooks(headroom_client) -> None:
"""I-LEGACY: turn hooks are a gateway-mode feature; legacy callers never see them."""
hook = _RecordingHook()
register_turn_hook(hook)
fold = fold_only_hook.register(max_chars=100)
resp = compress(headroom_client, _body())
assert resp.status_code == 200
assert hook.requests == 0
assert fold.request_calls == 0
# And the payload is untouched by the fold hook.
assert len(resp.json()["messages"][2]["content"]) > 100
def test_legacy_mode_registers_no_pending_turn(headroom_client) -> None:
"""I-LEGACY: nothing to wait for, nothing registered."""
compress(headroom_client, _body(config={"session_id": "legacy-p"}))
registry = registry_of(headroom_client)
assert registry is None or len(registry) == 0
# --------------------------------------------------------------------------- #
# I-KEYS / I-BODY #
# --------------------------------------------------------------------------- #
def _assert_contract_shape(data: dict[str, Any], sent: dict[str, Any]) -> None:
assert GATEWAY_EXTRA_KEYS <= set(data), sorted(data)
assert HEX32.match(data["turn_id"]), data["turn_id"]
assert set(data["route"]) == ROUTE_KEYS
assert isinstance(data["obligations"], list)
assert set(data["gateway"]) == GATEWAY_ECHO_KEYS
body = data["body"]
assert isinstance(body, dict)
for forbidden in ("config", "gateway", "token_budget"):
assert forbidden not in body, forbidden
assert body["model"] == sent["model"]
assert body["messages"] == data["messages"]
def test_gateway_mode_adds_contract_keys(headroom_client) -> None:
"""I-KEYS: ``gateway: {}`` is gateway mode; the legacy keys are all still there."""
sent = _body(gateway={})
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
data = resp.json()
assert LEGACY_STATELESS_KEYS <= set(data)
_assert_contract_shape(data, sent)
assert data["obligations"] == []
assert data["gateway"] == {
"can_redrive": False,
"can_relay_response": False,
"session_affinity": True,
}
assert data["route"]["model"] == "gpt-4o"
assert data["route"]["provider"] == "openai"
assert data["route"]["service_tier"] is None
assert data["route"]["reason"] == ""
# Compression still happened in gateway mode.
assert data["tokens_saved"] > 0
assert data["body"]["messages"][2]["content"] != sent["messages"][2]["content"]
def test_gateway_mode_with_session(headroom_client) -> None:
"""I-KEYS: session mode + gateway mode -> both key sets."""
sent = _body(config={"session_id": "gw-s1"}, gateway={"can_relay_response": True})
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
data = resp.json()
_assert_contract_shape(data, sent)
assert data["session"]["id"] == "gw-s1"
assert data["gateway"]["can_relay_response"] is True
def test_pass_through_fields_echoed_untouched(headroom_client) -> None:
"""I-BODY: unknown provider fields come back byte-identical inside ``body``."""
metadata = {"user_id": "u-1", "nested": {"k": [1, 2, {"z": None}]}}
sent = _body(
gateway={},
temperature=0.25,
top_p=0.9,
service_tier="flex",
metadata=metadata,
stream=False,
response_format={"type": "json_object"},
token_budget=50_000,
)
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
body = resp.json()["body"]
for key in ("temperature", "top_p", "service_tier", "metadata", "stream", "response_format"):
assert canonical(body[key]) == canonical(sent[key]), key
assert "token_budget" not in body
assert resp.json()["route"]["service_tier"] == "flex"
def test_tools_and_system_echoed(headroom_client) -> None:
"""I-BODY: ``system`` passes through as sent; ``tools`` come back (possibly
compacted) with every tool name preserved and in order."""
sent = _body(gateway={}, tools=openai_tools(), system=SYSTEM_PROMPT)
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
body = resp.json()["body"]
assert body["system"] == SYSTEM_PROMPT
assert tool_names(body["tools"]) == tool_names(sent["tools"])
def test_no_tools_means_no_tools_key(headroom_client) -> None:
"""I-BODY: a request without ``tools`` never grows one (an empty ``tools``
list is a provider 400 on Anthropic)."""
resp = compress(headroom_client, _body(gateway={}))
assert resp.status_code == 200
assert "tools" not in resp.json()["body"]
def test_tool_schema_compaction_is_deterministic(headroom_client) -> None:
"""I-BODY / prefix stability: identical tools in -> identical tool bytes out,
on every call, and the transform is named when it changed anything."""
sent = _body(gateway={}, tools=openai_tools())
first = compress(headroom_client, sent).json()
second = compress(headroom_client, sent).json()
assert canonical(first["body"]["tools"]) == canonical(second["body"]["tools"])
changed = canonical(first["body"]["tools"]) != canonical(sent["tools"])
assert ("tool_schema_compaction" in first["transforms_applied"]) == changed
def test_route_provider_inferred_from_model(headroom_client) -> None:
"""I-BODY: ``route.provider`` follows the tracker-provider rule (claude/anthropic
in the model name -> anthropic, else openai) and ``route.model`` is the body model."""
sent = {
"model": "claude-sonnet-4-5",
"messages": anthropic_tool_history(),
"system": SYSTEM_PROMPT,
"tools": anthropic_tools(),
"max_tokens": 64,
"gateway": {},
}
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["route"]["provider"] == "anthropic"
assert data["route"]["model"] == "claude-sonnet-4-5"
assert data["body"]["max_tokens"] == 64
assert data["body"]["system"] == SYSTEM_PROMPT
# --------------------------------------------------------------------------- #
# I-VALID #
# --------------------------------------------------------------------------- #
@pytest.mark.parametrize(
"gateway",
[
[],
"yes",
1,
True,
{"can_redrive": "true"},
{"can_relay_response": 1},
{"session_affinity": "no"},
{"can_redrive": None},
],
ids=[
"list",
"string",
"int",
"bool",
"redrive-str",
"relay-int",
"affinity-str",
"redrive-null",
],
)
def test_invalid_gateway_block_is_400(headroom_client, gateway: Any) -> None:
"""I-VALID: ``gateway`` must be an object whose flags are real booleans."""
resp = compress(headroom_client, _body(gateway=gateway))
assert resp.status_code == 400, resp.text
assert resp.json()["error"]["type"] == "invalid_request"
def test_unknown_gateway_keys_are_ignored(headroom_client) -> None:
"""I-VALID: forward-compatible - unknown keys inside ``gateway`` are dropped."""
resp = compress(
headroom_client,
_body(gateway={"plugin_version": "kong-headroom/0.1.0", "future_flag": {"x": 1}}),
)
assert resp.status_code == 200, resp.text
assert set(resp.json()["gateway"]) == GATEWAY_ECHO_KEYS
def test_legacy_validation_still_applies_in_gateway_mode(headroom_client) -> None:
"""I-VALID: the pre-existing 400s (missing model/messages) are unchanged."""
resp = compress(
headroom_client, {"messages": [{"role": "user", "content": "x"}], "gateway": {}}
)
assert resp.status_code == 400
assert "model" in resp.json()["error"]["message"]
# --------------------------------------------------------------------------- #
# I-KEYS on fail-open answers #
# --------------------------------------------------------------------------- #
def test_bypass_header_in_gateway_mode_still_returns_contract_keys(headroom_client) -> None:
"""I-KEYS: the bypass short-circuit must still hand the gateway a ``body`` it can
forward, a ``turn_id`` and an empty obligation list."""
sent = _body(gateway={"can_relay_response": True}, tools=openai_tools(), temperature=0.1)
resp = compress(headroom_client, sent, headers={"x-headroom-bypass": "true"})
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["messages"] == sent["messages"]
assert data["tokens_before"] == 0 and data["tokens_saved"] == 0
assert GATEWAY_EXTRA_KEYS <= set(data)
assert data["obligations"] == []
assert HEX32.match(data["turn_id"])
body = data["body"]
assert body["messages"] == sent["messages"]
assert canonical(body["tools"]) == canonical(sent["tools"])
assert body["temperature"] == 0.1
assert "gateway" not in body and "config" not in body
def test_empty_messages_in_gateway_mode_returns_contract_keys(headroom_client) -> None:
"""I-KEYS: the empty-messages short-circuit is also a gateway answer."""
sent = _body(messages=[], gateway={})
sent["messages"] = []
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["messages"] == []
assert GATEWAY_EXTRA_KEYS <= set(data)
assert data["obligations"] == []
assert data["body"]["messages"] == []
assert data["route"]["model"] == "gpt-4o"
def test_compression_timeout_fails_open_with_contract_keys(headroom_client, monkeypatch) -> None:
"""I-KEYS: stateless timeout -> 200 + originals + ``compression_skipped`` AND the
contract keys, so the gateway can forward without a special case."""
from unittest.mock import AsyncMock
proxy = headroom_client.app.state.proxy
monkeypatch.setattr(proxy, "_run_compression_in_executor", AsyncMock(side_effect=TimeoutError))
sent = _body(gateway={}, tools=openai_tools(), temperature=0.3)
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["compression_skipped"] is True
assert data["skip_reason"] == "compression_timeout"
assert data["messages"] == sent["messages"]
assert GATEWAY_EXTRA_KEYS <= set(data)
assert data["obligations"] == []
assert data["body"]["messages"] == sent["messages"]
assert data["body"]["temperature"] == 0.3
# --------------------------------------------------------------------------- #
# I-OBLIG / I-NOSHRINK: the capability matrix #
# --------------------------------------------------------------------------- #
def _expected_obligations(
hook: str, can_redrive: bool, can_relay: bool, affinity: bool
) -> list[str]:
out: list[str] = []
if hook == "redrive" and can_redrive and affinity:
out.append("redrive")
if can_relay:
out.append("relay_usage")
return out
@pytest.mark.parametrize("hook", ["none", "fold", "redrive"])
@pytest.mark.parametrize("can_redrive", [False, True])
@pytest.mark.parametrize("can_relay", [False, True])
@pytest.mark.parametrize("affinity", [False, True])
def test_obligations_matrix(
headroom_client, hook: str, can_redrive: bool, can_relay: bool, affinity: bool
) -> None:
"""I-OBLIG + I-NOSHRINK over every capability combination and hook kind."""
fold = redrive = None
if hook == "fold":
fold = fold_only_hook.register(max_chars=100)
elif hook == "redrive":
redrive = redrive_hook_ext.register()
sent = _body(
gateway={
"can_redrive": can_redrive,
"can_relay_response": can_relay,
"session_affinity": affinity,
},
tools=openai_tools(),
)
resp = compress(headroom_client, sent)
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["obligations"] == _expected_obligations(hook, can_redrive, can_relay, affinity)
assert data["gateway"] == {
"can_redrive": can_redrive,
"can_relay_response": can_relay,
"session_affinity": affinity,
}
names = tool_names(data["body"]["tools"])
if "redrive" in data["obligations"]:
assert redrive is not None and redrive.shrink_calls == 1
assert "search_tools" in names
assert not any(n.startswith("deferred_") for n in names)
else:
# I-NOSHRINK: tools are a subset of what came in, in order.
assert "search_tools" not in names
assert names == tool_names(sent["tools"])
if redrive is not None:
assert redrive.shrink_calls == 0
if fold is not None:
# A stream-safe hook runs regardless of the redrive capability.
assert fold.request_calls == 1
assert data["body"]["messages"][2]["content"].endswith(fold_only_hook.FOLD_MARKER)
assert "turn_hook" in data["transforms_applied"]
# A pending turn exists iff there is something to wait for.
registry = registry_of(headroom_client)
assert registry is not None
if data["obligations"]:
assert registry.get(data["turn_id"]) is not None
else:
assert registry.get(data["turn_id"]) is None
def test_redrive_hook_tools_delta_is_reported(headroom_client) -> None:
"""I-OBLIG: a hook that shrinks tools is reported as ``turn_hook:tools:<n>tok``
(mirrors the chat path) so the savings are attributable."""
redrive_hook_ext.register()
sent = _body(gateway={"can_redrive": True}, tools=openai_tools())
data = compress(headroom_client, sent).json()
assert data["obligations"] == ["redrive"]
tags = [t for t in data["transforms_applied"] if t.startswith("turn_hook:tools:")]
assert len(tags) == 1, data["transforms_applied"]
assert re.match(r"^turn_hook:tools:\d+tok$", tags[0])
def test_session_mode_replays_hook_output(headroom_client) -> None:
"""I-BODY in session mode: the hook-rewritten messages are what the gateway
forwards, so they must be what turn 2 replays byte-for-byte."""
fold_only_hook.register(max_chars=120)
history = big_tool_history()
turn1 = compress(
headroom_client, _body(history, config={"session_id": "gw-fold"}, gateway={})
).json()
folded = turn1["body"]["messages"][2]["content"]
assert folded.endswith(fold_only_hook.FOLD_MARKER)
turn2 = compress(
headroom_client,
_body(
history + [{"role": "assistant", "content": "ok"}, {"role": "user", "content": "more"}],
config={"session_id": "gw-fold"},
gateway={},
),
).json()
assert canonical(turn2["body"]["messages"][: len(turn1["body"]["messages"])]) == canonical(
turn1["body"]["messages"]
)
assert turn2["body"]["messages"][-1]["content"] == "more"
def test_turn_ids_are_unique_per_request(headroom_client) -> None:
"""I-KEYS: every request half mints a fresh id, registered or not."""
ids = {compress(headroom_client, _body(gateway={})).json()["turn_id"] for _ in range(5)}
assert len(ids) == 5
class _ConversationHook:
name = "conversation"
stream_safe = True
def __init__(self) -> None:
self.seen: list[str | None] = []
def on_request(self, ctx: Any) -> None:
self.seen.append(ctx.conversation)
def test_hooks_see_the_gateway_session_as_the_conversation(headroom_client) -> None:
"""Each gateway session is one conversation; no session, no conversation."""
hook = _ConversationHook()
register_turn_hook(hook)
for session in ("gw-c1", "gw-c1", "gw-c2"):
resp = compress(headroom_client, _body(config={"session_id": session}, gateway={}))
assert resp.status_code == 200, resp.text
resp = compress(headroom_client, _body(gateway={}))
assert resp.status_code == 200, resp.text
first, again, other, none = hook.seen
assert first == again and first is not None and first != "gw-c1"
assert other not in (None, first)
assert none is None
@pytest.mark.parametrize(
("label", "shape", "model", "provider", "route_provider"),
[
(
"anthropic_system",
{"messages": [{"role": "user", "content": "x"}], "system": SYSTEM_PROMPT},
"gemini/gemini-2.5-pro",
"anthropic",
"openai",
),
(
"anthropic_tools",
{"messages": [{"role": "user", "content": "x"}], "tools": anthropic_tools()},
"gemini/gemini-2.5-pro",
"anthropic",
"openai",
),
(
"anthropic_history_with_server_tool",
{
"messages": anthropic_tool_history(),
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
},
"gpt-5.5",
"anthropic",
"openai",
),
(
"anthropic_first_turn_server_tool_only",
{
"messages": [{"role": "user", "content": "x"}],
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
},
"gemini/gemini-2.5-pro",
"anthropic",
"openai",
),
(
"openai_tools_to_claude",
{"messages": [{"role": "user", "content": "x"}], "tools": openai_tools()},
"claude-opus-5-5",
"openai",
"anthropic",
),
(
"openai_tool_row_to_claude",
{"messages": [{"role": "tool", "tool_call_id": "c", "content": "x"}]},
"claude-opus-5-5",
"openai",
"anthropic",
),
(
"responses_input_to_claude",
{"input": [{"role": "user", "content": "x"}], "instructions": "be brief"},
"claude-opus-5-5",
"openai",
"anthropic",
),
(
"chat_system_role_to_claude",
{
"messages": [
{"role": "system", "content": "be brief"},
{"role": "user", "content": "x"},
]
},
"claude-opus-5-5",
"openai",
"anthropic",
),
(
"chat_image_to_claude",
{
"messages": [
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": "https://x.test/a.png"}}
],
}
]
},
"claude-opus-5-5",
"openai",
"anthropic",
),
(
"anthropic_thinking_to_gpt",
{
"messages": [
{"role": "user", "content": "x"},
{
"role": "assistant",
"content": [{"type": "thinking", "thinking": "t", "signature": "s"}],
},
{"role": "user", "content": "y"},
]
},
"gpt-5.5",
"anthropic",
"openai",
),
(
"anthropic_server_tool_result_to_gpt",
{
"messages": [
{"role": "user", "content": "x"},
{
"role": "assistant",
"content": [
{"type": "web_search_tool_result", "tool_use_id": "s", "content": []}
],
},
{"role": "user", "content": "y"},
],
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
},
"gpt-5.5",
"anthropic",
"openai",
),
(
"plain_text_follows_model",
{"messages": [{"role": "user", "content": "x"}], "system": None},
"claude-opus-5-5",
"anthropic",
"anthropic",
),
],
)
def test_hooks_see_the_wire_shape_and_route_follows_the_model(
headroom_client, label, shape, model, provider, route_provider
) -> None:
"""A gateway can send any client's shape to any model."""
seen: list[str] = []
class Recorder:
name = f"provider_recorder_{label}"
stream_safe = True
def on_request(self, ctx: Any) -> None:
seen.append(ctx.provider)
register_turn_hook(Recorder())
data = compress(headroom_client, {**shape, "model": model, "gateway": {}}).json()
assert seen == [provider]
assert data["route"]["provider"] == route_provider
def test_chat_body_to_claude_stays_within_the_cache_breakpoint_budget(headroom_client) -> None:
"""The budget belongs to the upstream model, not to the body's wire shape."""
marked = [
{"role": "user", "content": [{"type": "text", "text": f"m{i}", "cache_control": {}}]}
for i in range(7)
]
body = _body(messages=marked, tools=openai_tools(), model="claude-opus-5-5", gateway={})
data = compress(headroom_client, body).json()
sent = [
block
for message in data["body"]["messages"]
for block in message["content"]
if isinstance(block, dict) and "cache_control" in block
]
assert len(sent) <= 4
assert "cache_breakpoint_budget" in data["transforms_applied"]