fix(proxy): reconcile Chat content-block cache markers after prefix replay (#4333)

## Description

OpenCode V2 talking to Claude through an OpenAI-compatible gateway
(LiteLLM → Bedrock) marks
Chat **content blocks**: two system text blocks, the newest user or tool
text block, plus one tool
definition. After the Chat prefix replay (`_restore_frozen_prefix` /
`finalize_turn`), last turn's
block marker is forwarded again next to the client's moved one. The
client sends four markers, the
proxy forwards five, and the provider rejects the turn:

```text
A maximum of 4 blocks with cache_control may be provided. Found 5.
```

#3895 fixed the same accumulation for message-dict and `tool_calls`
markers, and its helper
explicitly left block markers alone. This PR extends
`mirror_client_message_cache_control` to
content blocks, so all three Chat marker holders follow the client's
current request.

Related: #3894 (same symptom, message-level variant), #4332 (separate
token-mode lost breakpoint).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `mirror_client_message_cache_control` now also reconciles
`content[].cache_control`:
- The client marks a block → copy the client's own value (`ttl` kept per
position).
- The client marks no block in that message → drop replayed block
markers.
This also covers OpenCode resending an older turn as a **string** while
the replay restores
    the block form it forwarded when that turn was newest.
- The client marks blocks but the block count no longer aligns → leave
the message unchanged.
    No index guessing.
- Only `cache_control` keys change; never creates a marker and never
moves one to another block.
- Why not reuse `normalize_message_cache_control`: in Chat the system
prompt lives in `messages`.
Its legacy fallback strips every block marker and re-places one, which
would also drop the
  client's system breakpoints.
- Why not rely on `enforce_cache_breakpoint_budget`: it only runs for
`provider == "anthropic"`.
  This traffic is OpenAI Chat to a gateway, so it never runs here.
- The insertion point is unchanged. The existing mirror call already
runs after both Chat replays
and before forwarding and the tracker snapshot, so cache and token mode
and streaming and
  buffered forwarding are all covered.
- Docstring and comment no longer claim OpenAI-compatible OpenCode never
marks blocks.

## Testing

- [x] Unit tests pass (`pytest`): see below
- [x] Linting passes (`ruff check .`, `ruff format --check`)
- [x] Type checking passes (`mypy headroom/cache/prefix_tracker.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed

New: `tests/test_openai_block_cache_control_replay.py` (20 tests).
Without the fix, 17 fail.

- Helper cases:
  - moved user → tool breakpoint
  - byte-identical replayed content
  - mid-message positions with `ttl: 1h` and `5m`
  - client with no block markers (none created)
  - client with no markers at all
  - mixed block, message and `tool_calls` markers
  - block-count mismatch (left unchanged)
  - string client content vs replayed blocks
- literal `cache_control` inside tool text and arguments stays user data
  - idempotence
- Handler cases capture the real forwarded body through
`/v1/chat/completions`:
  - cache and token mode (token with frozen floor 2)
  - streaming and buffered
  - block-form and string-form older user turn
- the tracker snapshot records the reconciled layout, and turn 3 stays
bounded

### Test Output

```text
$ pytest tests/test_openai_block_cache_control_replay.py tests/test_openai_message_level_cache_control.py
29 passed in 5.19s

# same new file with the fix reverted
17 failed, 3 passed

# related cache / prefix / OpenAI / proxy / LiteLLM / compression-cache suites
1217 passed, 33 skipped

# full suite, before vs after: identical failure set except three tests that
# fail intermittently under -n 8 on both builds and pass in isolation on both.
# All remaining failures are local-environment only (no Rust toolchain,
# sandboxed network, missing optional extras).
41 failed, 16305 passed, 860 skipped, 14 errors

$ ruff check . && ruff format --check headroom tests     # clean
$ mypy headroom/cache/prefix_tracker.py                  # Success: no issues found
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.13, OpenCode 2.1.2 with the bundled
Headroom plugin → `headroom proxy --mode cache|token` → LiteLLM gateway
→ Bedrock Claude Haiku; synthetic data only.
- Exact command / steps: Ran the same OpenCode agent task (one tool
returning 120 synthetic JSON records, called 3 times in sequence)
through the proxy, once on installed 0.41.0 and once importing this
branch's Python source via `PYTHONPATH` (Rust crates unchanged since
0.41.0); an `httpx` hook in the proxy recorded only the `cache_control`
paths of each outbound body.
- Observed result: 0.41.0 forwards 5 markers on the second request and
the gateway returns 400 "Found 5"; with this branch, cache and token
mode forward at most 4 markers on every request and the agent returns
the exact expected answer after 3 tool calls.
- Not tested: Windows; Anthropic `/v1/messages` (unchanged path);
Responses API; clients other than OpenCode V2; live sessions longer than
4 requests; compression of cache-protected Chat text blocks.

### Details

| Build | Mode | Inbound markers per request | Outbound markers per
request | Result |
| --- | --- | --- | --- | --- |
| 0.41.0 | cache | 4, 4 | 4, **5** | 400 "Found 5", turn aborted |
| this PR | cache | 4, 4, 4, 4 | 4, 4, 4, 4 | exact answer after 3 tool
calls |
| this PR | token | 4, 4, 4, 4 | 4, 4, 4, 3 | exact answer after 3 tool
calls (see note) |

Loopback HTTP replay of the same shapes (4 steps, cache and token, with
and without a
provider-confirmed frozen floor):

- 0.41.0 forwards 4/5/6/7 markers.
- This branch forwards 4/4/4/4 in all four cases.

Paths and values match the client, and the non-marker message bytes are
unchanged.

The token-mode `3` also reproduces on unpatched 0.41.0 and is a separate
issue.
`CompressionCache._swap_tool_result_content` writes a cached compression
back as a string for
`role: "tool"`, which drops the client's block marker on the newest tool
result. That is a lost
breakpoint, not a 400. Not changed here; tracked in #4332.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: Chat Completions forwarding no longer
re-sends replayed
content-block `cache_control` markers the client did not send this turn.
- Kill switch / disable path: none (bug fix; output is the client's own
marker layout)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines

Co-authored-by: Thomas Bartik <22518863+tbartik@users.noreply.github.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
tbartik
2026-11-03 22:57:17 -06:00
committed by GitHub
co-authored by Thomas Bartik JD Davis
parent 98018333f9
commit 35ca9b8356
4 changed files with 459 additions and 29 deletions
+57 -23
View File
@@ -1105,33 +1105,37 @@ def mirror_client_message_cache_control(
messages: list[dict[str, Any]],
client_messages: list[dict[str, Any]] | None,
) -> list[dict[str, Any]]:
"""Keep ``cache_control`` outside content blocks exactly where the client put it.
"""Keep Chat Completions ``cache_control`` exactly where the client put it.
OpenAI-compatible clients that talk to Anthropic models through a gateway
(OpenCode via ``@ai-sdk/openai-compatible``, then LiteLLM) do not mark
content blocks. They mark the MESSAGE dict — ``{"role": "tool", "content":
"...", "cache_control": {...}}`` — and, on an assistant turn, the entries of
its ``tool_calls`` list, which the gateway turns into ``tool_use`` blocks.
``normalize_message_cache_control`` only sees block markers
(``_client_marker_positions`` reads list content), so after
``overlay_cached_prefix`` / ``finalize_turn`` replay an earlier turn's
forwarded messages, these markers ride along untouched. A client that marks
three places per request (system, the newest tool call, the newest tool
result) ends up forwarding four, then five and more, and the gateway's
translation to Anthropic blocks crosses the limit of four (``A maximum of 4
blocks with cache_control may be provided. Found 6``).
(OpenCode via ``@ai-sdk/openai-compatible``, then LiteLLM) mark three kinds
of holder: the MESSAGE dict — ``{"role": "tool", "content": "...",
"cache_control": {...}}`` —, the entries of an assistant's ``tool_calls``
list, which the gateway turns into ``tool_use`` blocks, and the content
blocks of list-form content (OpenCode V2 sends system, user and tool
content as text blocks). After ``overlay_cached_prefix`` /
``finalize_turn`` replay an earlier turn's forwarded messages, last turn's
markers ride along with the replayed bytes. A client that moves its
breakpoint forward each step then forwards one marker more per step, and
the gateway's translation to Anthropic blocks crosses the limit of four
(``A maximum of 4 blocks with cache_control may be provided. Found 5``).
The client's current request is the authority for these markers, the same
rule the block-level normalizer applies: on each message and on each of its
``tool_calls``, drop the marker unless the client put one there in this
request, in which case copy the client's own value. Content — and any
block-level marker in it — is left alone, so the replayed bytes the provider
cached stay byte-identical.
The client's current request is the authority for these markers: on each
message, each of its ``tool_calls`` and each of its content blocks, drop the
marker unless the client put one there in this request, in which case copy
the client's own value (an explicit ``ttl`` survives per position). Only the
``cache_control`` keys move; every other byte of the replayed content stays
what the provider cached.
Index alignment is the invariant (the pipeline preserves message count and
each message's tool calls); where it does not hold, that message or list is
returned unchanged rather than guessing, and the breakpoint budget guard
still bounds the total.
Unlike ``normalize_message_cache_control`` this never creates a marker and
never moves one to a different block: in Chat the system prompt is part of
``messages``, so a consolidating fallback would also strip the client's
system breakpoints. Index alignment is the invariant (the pipeline preserves
message count and each message's tool calls); where it does not hold, that
message or list is left unchanged rather than guessing. A replay may also
bring back an earlier turn's block-form content where the client now sends
a string (OpenCode does this for older user turns): the client marks no
block there, so the replayed block markers are dropped.
"""
if not isinstance(client_messages, list) or len(client_messages) != len(messages):
return messages
@@ -1153,11 +1157,41 @@ def mirror_client_message_cache_control(
if any(c for _, c in mirrored):
new_msg = {**new_msg, "tool_calls": [c for c, _ in mirrored]}
msg_changed = True
blocks = new_msg.get("content")
client_blocks = client.get("content") if isinstance(client, dict) else None
if isinstance(blocks, list):
mirrored = _mirror_block_markers(blocks, client_blocks)
if mirrored is not blocks:
new_msg = {**new_msg, "content": mirrored}
msg_changed = True
out.append(new_msg)
changed = changed or msg_changed
return out if changed else messages
def _mirror_block_markers(blocks: list[Any], client_content: Any) -> list[Any]:
"""Give each forwarded content block the client's block marker (or none).
A client message that marks no block (string content, which a replay may
bring back in an earlier turn's block form, or unmarked blocks) leaves no
block marker to keep, so every one is dropped. When the client does mark
blocks, positions map 1:1 only if the block counts match; otherwise the
blocks are returned unchanged rather than guessing.
"""
client_blocks = client_content if isinstance(client_content, list) else []
client_marks = any(
isinstance(b, dict) and isinstance(b.get("cache_control"), dict) for b in client_blocks
)
if not client_marks:
client_blocks = [None] * len(blocks)
elif len(client_blocks) != len(blocks):
return blocks
mirrored = [_mirror_marker(b, cb) for b, cb in zip(blocks, client_blocks)]
if not any(c for _, c in mirrored):
return blocks
return [b for b, _ in mirrored]
def _mirror_marker(holder: Any, client_holder: Any) -> tuple[Any, bool]:
"""Give ``holder`` the client's own ``cache_control`` (or none); report change."""
if not isinstance(holder, dict):
+4 -5
View File
@@ -4323,11 +4323,10 @@ class OpenAIHandlerMixin:
# Both replays above forward earlier turns' messages byte-identical,
# including the cache_control each carried back then. Chat Completions
# clients mark the message dict and an assistant's tool_calls, which the
# block-level normalizer does not see, so without this the markers
# accumulate turn over turn and an Anthropic-backed gateway rejects the
# request (more than four breakpoints). Content is untouched: only the
# markers move.
# clients mark the message dict, an assistant's tool_calls and content
# blocks, so without this the markers accumulate turn over turn and an
# Anthropic-backed gateway rejects the request (more than four
# breakpoints). Only the markers move; every other byte stays replayed.
from headroom.cache.prefix_tracker import mirror_client_message_cache_control
optimized_messages = mirror_client_message_cache_control(
@@ -0,0 +1,397 @@
"""Chat Completions content-block ``cache_control`` must not accumulate across replays.
OpenCode V2 talking to a Claude model through an OpenAI-compatible gateway marks
content BLOCKS in Chat messages (``{"role": "user", "content": [{"type": "text",
..., "cache_control": {...}}]}``), not only the message dict and ``tool_calls``.
The prefix replay forwards earlier turns' messages byte-identical, so last turn's
block marker came back next to the client's moved one: the client sent four, the
proxy forwarded five, and the gateway's Anthropic translation failed with
``A maximum of 4 blocks with cache_control may be provided. Found 5``.
"""
from __future__ import annotations
import copy
import json
from types import SimpleNamespace
import httpx
import pytest
pytest.importorskip("fastapi")
from fastapi.testclient import TestClient
from headroom.cache.prefix_tracker import mirror_client_message_cache_control
from headroom.proxy.server import ProxyConfig, create_app
CC = {"type": "ephemeral"}
CC_1H = {"type": "ephemeral", "ttl": "1h"}
def _text(text: str, marker: dict | None = None) -> dict:
block = {"type": "text", "text": text}
return {**block, "cache_control": dict(marker)} if marker else block
def _marker_paths(value, path: str = "") -> list[str]: # noqa: ANN001
found: list[str] = []
if isinstance(value, dict):
for key, item in value.items():
if key == "cache_control":
found.append(path or ".")
else:
found.extend(_marker_paths(item, f"{path}.{key}" if path else key))
elif isinstance(value, list):
for index, item in enumerate(value):
found.extend(_marker_paths(item, f"{path}.{index}" if path else str(index)))
return found
def _strip_markers(value): # noqa: ANN001, ANN201
if isinstance(value, dict):
return {k: _strip_markers(v) for k, v in value.items() if k != "cache_control"}
if isinstance(value, list):
return [_strip_markers(v) for v in value]
return value
def _system(first: dict | None = CC, last: dict | None = CC) -> dict:
return {
"role": "system",
"content": [
_text("You are an agent.", first),
_text("Preserve records."),
_text("Use the tool."),
_text("Answer briefly.", last),
],
}
def _call(cid: str = "c1") -> dict:
return {
"role": "assistant",
"content": "",
"tool_calls": [
{"id": cid, "type": "function", "function": {"name": "read", "arguments": "{}"}}
],
}
# ── the helper ──────────────────────────────────────────────────────────────
def test_moved_user_breakpoint_drops_the_replayed_block_marker() -> None:
client = [
_system(),
{"role": "user", "content": [_text("q")]},
_call(),
{"role": "tool", "tool_call_id": "c1", "content": [_text("result", CC)]},
]
replayed = copy.deepcopy(client)
replayed[1]["content"][0]["cache_control"] = dict(CC) # last turn's breakpoint
before = copy.deepcopy(replayed)
out = mirror_client_message_cache_control(replayed, client)
assert (
_marker_paths(out) == _marker_paths(client) == ["0.content.0", "0.content.3", "3.content.0"]
)
assert replayed == before # input not mutated
assert _strip_markers(out) == _strip_markers(replayed)
def test_replayed_block_content_stays_byte_identical() -> None:
client = [
{"role": "tool", "tool_call_id": "c1", "content": [_text("raw " * 50)]},
{"role": "user", "content": [_text("next", CC)]},
]
replayed = [
{"role": "tool", "tool_call_id": "c1", "content": [_text("<compressed>", CC)]},
copy.deepcopy(client[1]),
]
out = mirror_client_message_cache_control(replayed, client)
assert out[0]["content"] == [_text("<compressed>")]
assert out[1] is replayed[1]
def test_explicit_mid_message_positions_and_ttls_are_copied_verbatim() -> None:
client = [
_system(first=CC_1H, last=CC),
{"role": "user", "content": [_text("a"), _text("b", CC_1H), _text("c")]},
]
replayed = [
_system(first=CC, last=CC),
{"role": "user", "content": [_text("a"), _text("b"), _text("c", CC)]},
]
out = mirror_client_message_cache_control(replayed, client)
assert out[0]["content"][0]["cache_control"] == CC_1H
assert out[0]["content"][3]["cache_control"] == CC
assert [("cache_control" in b) for b in out[1]["content"]] == [False, True, False]
assert out[1]["content"][1]["cache_control"] == CC_1H
def test_client_without_block_markers_gets_none_back_and_none_created() -> None:
client = [_system(first=None, last=None), {"role": "user", "content": [_text("q")]}]
replayed = [_system(), {"role": "user", "content": [_text("q", CC)]}]
out = mirror_client_message_cache_control(replayed, client)
assert _marker_paths(out) == []
def test_client_without_any_markers_changes_nothing() -> None:
client = [
{"role": "system", "content": [_text("s")]},
{"role": "user", "content": [_text("q")]},
]
messages = copy.deepcopy(client)
assert mirror_client_message_cache_control(messages, client) is messages
def test_mixed_block_message_and_tool_call_markers() -> None:
call = _call()
marked_call = {**call, "tool_calls": [{**call["tool_calls"][0], "cache_control": dict(CC)}]}
client = [
_system(),
{"role": "user", "content": [_text("q")]},
marked_call,
{"role": "tool", "tool_call_id": "c1", "content": "out", "cache_control": dict(CC)},
]
replayed = [
_system(),
{"role": "user", "content": [_text("q", CC)], "cache_control": dict(CC)},
call,
{"role": "tool", "tool_call_id": "c1", "content": "out"},
]
out = mirror_client_message_cache_control(replayed, client)
assert (
_marker_paths(out)
== _marker_paths(client)
== ["0.content.0", "0.content.3", "2.tool_calls.0", "3"]
)
def test_block_count_mismatch_leaves_that_message_unchanged() -> None:
client = [
{"role": "user", "content": [_text("a"), _text("b", CC)]},
{"role": "user", "content": [_text("c", CC)]},
]
# The pipeline merged two blocks; there is no safe 1:1 mapping, so no guessing.
replayed = [
{"role": "user", "content": [_text("ab", CC)]},
{"role": "user", "content": [_text("c")]},
]
out = mirror_client_message_cache_control(replayed, client)
assert out[0] is replayed[0]
assert out[1]["content"][0]["cache_control"] == CC
def test_replayed_blocks_for_a_string_client_message_lose_their_markers() -> None:
# OpenCode V2 sends an older user turn as a string; the replay brings back
# the block form it forwarded when that turn was the newest, marker included.
client = [{"role": "user", "content": "q"}]
replayed = [{"role": "user", "content": [_text("q", CC)]}]
out = mirror_client_message_cache_control(replayed, client)
assert out[0]["content"] == [_text("q")]
def test_block_count_mismatch_without_client_block_markers_only_drops_markers() -> None:
client = [{"role": "user", "content": [_text("a"), _text("b")]}]
replayed = [{"role": "user", "content": [_text("ab", CC)]}]
out = mirror_client_message_cache_control(replayed, client)
assert out[0]["content"] == [_text("ab")]
def test_literal_cache_control_in_tool_text_and_arguments_is_user_data() -> None:
payload = json.dumps({"cache_control": {"type": "ephemeral"}})
call = {
"role": "assistant",
"content": "",
"tool_calls": [
{"id": "c1", "type": "function", "function": {"name": "write", "arguments": payload}}
],
}
client = [call, {"role": "tool", "tool_call_id": "c1", "content": [_text(payload)]}]
replayed = copy.deepcopy(client)
out = mirror_client_message_cache_control(replayed, client)
assert out is replayed
assert out[1]["content"][0]["text"] == payload
def test_reconciliation_is_idempotent() -> None:
client = [
_system(),
{"role": "user", "content": [_text("q")]},
{"role": "user", "content": [_text("r", CC)]},
]
replayed = [
_system(),
{"role": "user", "content": [_text("q", CC)]},
{"role": "user", "content": [_text("r", CC)]},
]
once = mirror_client_message_cache_control(replayed, client)
twice = mirror_client_message_cache_control(once, client)
assert twice is once
# ── through the Chat Completions handler ────────────────────────────────────
BIG = "synthetic result " * 400
class _Tracker:
"""Session tracker with last turn recorded; records what this turn stores."""
def __init__(self, previous_original, previous_forwarded, frozen: int): # noqa: ANN001
self._original = previous_original
self._forwarded = previous_forwarded
self._frozen = frozen
self.recorded: list[dict] | None = None
def get_frozen_message_count(self) -> int:
return self._frozen
def get_last_original_messages(self): # noqa: ANN201
return copy.deepcopy(self._original)
def get_last_forwarded_messages(self): # noqa: ANN201
return copy.deepcopy(self._forwarded)
def update_from_response(self, **kwargs): # noqa: ANN003
self.recorded = copy.deepcopy(kwargs.get("messages"))
def _opencode_turns(string_user: bool = False) -> tuple[list[dict], list[dict], list[dict]]:
"""The block-marked shape OpenCode V2 sends to Claude over Chat, two steps in.
Last step the client marked two system blocks and the user block, and the
proxy forwarded that. This step the client moved the moving breakpoint to the
tool result and left the user block unmarked. Live OpenCode V2 also resends
that older user turn as a plain string (``string_user``), so the replay
brings back a block form the client no longer sends.
"""
user = {
"role": "user",
"content": "Read fixture.json" if string_user else [_text("Read fixture.json")],
}
tool = {"role": "tool", "tool_call_id": "c1", "content": [_text(BIG)]}
previous = [_system(), {"role": "user", "content": [_text("Read fixture.json", CC)]}]
current = [_system(), user, _call(), {**tool, "content": [_text(BIG, CC)]}]
return copy.deepcopy(previous), copy.deepcopy(previous), current
def _app_config() -> ProxyConfig:
return ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
)
def _send(mode: str, frozen: int, stream: bool, turns=None): # noqa: ANN001, ANN202
previous_original, previous_forwarded, current = turns or _opencode_turns()
captured: dict = {}
tracker = _Tracker(previous_original, previous_forwarded, frozen)
with TestClient(create_app(_app_config())) as client:
proxy = client.app.state.proxy
proxy.config.optimize = True
proxy.config.mode = mode
proxy.session_tracker_store.compute_session_id = lambda request, model, messages: "s"
proxy.session_tracker_store.get_or_create = (
lambda session_id, provider, cache_ttl_seconds=None: tracker
)
proxy.openai_pipeline.apply = lambda **kwargs: SimpleNamespace(
messages=kwargs["messages"],
transforms_applied=[],
timing={},
tokens_before=100,
tokens_after=100,
waste_signals=None,
)
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["body"] = json.loads(body) if isinstance(body, (bytes, str)) else body
return httpx.Response(
200,
json={
"id": "c",
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": "ok"},
"finish_reason": "stop",
}
],
"usage": {"prompt_tokens": 100, "completion_tokens": 1, "total_tokens": 101},
},
)
async def _fake_stream(url, headers, body, *args, **kwargs): # noqa: ANN001, ANN002
captured["body"] = body
return httpx.Response(200, text="data: [DONE]\n\n")
proxy._retry_request = _fake_retry
proxy._stream_response = _fake_stream
response = client.post(
"/v1/chat/completions",
headers={"authorization": "Bearer test-key"},
json={"model": "claude-haiku-4-5", "stream": stream, "messages": current},
)
assert response.status_code == 200, response.text
return captured["body"], current, tracker
@pytest.mark.parametrize("string_user", [False, True], ids=["block-user", "string-user"])
@pytest.mark.parametrize("stream", [False, True], ids=["buffered", "streaming"])
@pytest.mark.parametrize("mode,frozen", [("cache", 0), ("token", 2)])
def test_forwarded_block_markers_follow_the_client(
mode: str, frozen: int, stream: bool, string_user: bool
) -> None:
forwarded, current, _ = _send(mode, frozen, stream, _opencode_turns(string_user))
paths = _marker_paths(forwarded["messages"])
assert paths == _marker_paths(current) == ["0.content.0", "0.content.3", "3.content.0"]
if not string_user:
assert _strip_markers(forwarded["messages"]) == _strip_markers(current)
else:
# The replayed block form of the old user turn is what the provider cached.
assert forwarded["messages"][1]["content"] == [_text("Read fixture.json")]
assert _strip_markers(forwarded["messages"][2:]) == _strip_markers(current[2:])
def test_tracker_records_the_reconciled_layout_so_turn_three_stays_bounded() -> None:
forwarded, current, tracker = _send("cache", 0, False)
assert tracker.recorded is not None
assert _marker_paths(tracker.recorded) == _marker_paths(forwarded["messages"])
# Turn 3 replays what turn 2 recorded; the moving breakpoint moves again.
tool2 = {"role": "tool", "tool_call_id": "c2", "content": [_text("second", CC)]}
turn3 = copy.deepcopy(current)
turn3[3]["content"][0].pop("cache_control")
turn3 += [_call("c2"), tool2]
forwarded3, _, _ = _send("cache", 0, False, (current, tracker.recorded, turn3))
assert _marker_paths(forwarded3["messages"]) == ["0.content.0", "0.content.3", "5.content.0"]
@@ -117,7 +117,7 @@ def test_nothing_to_do_returns_the_same_list() -> None:
assert mirror_client_message_cache_control(messages, client) is messages
def test_block_level_markers_are_left_to_the_block_normalizer() -> None:
def test_block_markers_already_where_the_client_put_them_are_left_alone() -> None:
block = {"type": "text", "text": "x", "cache_control": CC}
client = [{"role": "user", "content": [dict(block)]}]
messages = [{"role": "user", "content": [dict(block)]}]