Files
headroom/tests/test_read_maturation_handler_nobust.py
Alicia f1035ac737 fix(cache): preserve OpenAI tool block metadata on cached replay (#4334)
Current status (2026-10-24 20:50 UTC): exact head
`2333a5ccf7700f86f8767d1c5d02fbfb903c966c` is conflict-free and
[approved by
JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639834650).
All seven held workflows are now successful:
[CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40128152028)
has 17 successful jobs and five skips;
[Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151955)
has 12 successful jobs. Both [Wrap
E2E](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151965/job/120974772335)
and CI's Docker-native job actually execute the new npm hook: two tests
pass, zero fail or skip. Logs check out synthetic merge `91669e2` (this
head into `3fef15d3`). The maintainer independently reports 146 tests
passed with ten optional skips, including the nine new regressions, plus
Ruff lint/formatting. These hosted and maintainer results supplement,
rather than replace or relabel, the author's limited local checks.
Security passes upstream policy, including three excepted bundled npm
advisories until 2026-11-15; this is not a zero-findings claim.
Live-provider billing/cache behavior remains unqualified. The prior
approval-pending statement and local Docker limitation in the [author
response](https://github.com/headroomlabs-ai/headroom/pull/4334#issuecomment-6388924792)
describe the earlier observation, now superseded by these hosted
results.

### Previous-head qualification (historical)

Current status (2026-10-24 19:54 UTC): exact head
`a618561542a8bb48fdb29b07c66c3c6647b7a690` is conflict-free and
[approved by
JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639593849);
the maintainer explicitly cleared the earlier dependency qualification
hold. All seven previously held workflows completed successfully,
including
[CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172110)
(17 successful jobs, five skipped),
[Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172235)
(12 successful jobs), Rust and Wrap E2E. Logs check out synthetic merge
`2ff995d67ef4bb52590c1c0ca95514c85e9693ab` (this head into `3fef15d3`).
Maintainer qualification reports 137 tests passed with ten optional
skips plus Ruff/whitespace; these are their results, not another
author-run suite. Security passes upstream policy, not a zero-findings
claim: E2E npm retains three excepted bundled advisories under
upstream's existing policy until 2026-11-15. No additional exception was
introduced here. Live-provider behavior remains unqualified. Earlier
dated results below are historical.

## Description

Fixes #4332. Replaying a cached compression for an OpenAI `role: tool`
message currently converts list-form content to a string, discarding its
client-supplied `cache_control` and other block metadata. A single text
block now keeps its shape and metadata; only its text is replaced in the
existing deep copy.

For multiple blocks or non-text/malformed block shapes, replay leaves
the message unchanged, before a cache lookup is counted. The cached
joined string does not contain enough information to safely reconstruct
block boundaries or preserve media. String-form OpenAI and Anthropic
behavior are unchanged. This does not duplicate #4333, which explicitly
excludes the token-mode lost-breakpoint issue.

## Type of Change

- [x] Bug fix

## Changes Made

- Preserve the deep-copied single-text OpenAI content block and replace
text only.
- Leave ambiguous lists unchanged before cache hit accounting.
- Share the eligibility guard across cache replay, the rewrite helper
and the real interceptor path before callbacks, disclosure state and
savings accounting.
- Add direct-helper/cache/interceptor regressions for unsupported
shapes, metadata isolation and progressive-disclosure isolation.
- Earlier compatible dependency updates are now incorporated in upstream
main; the final PR delta keeps the cache fix and regressions, with
dependency constraints and locks unchanged from main; the tools manifest
now adds only a test script.

## Testing

- [x] Focused regression tests pass.
- [x] Lint / formatting and changed-module type checks passed at
d74aa5c; runtime/test code is unchanged by a00f387.
- [x] Independent review performed.
- [x] Current 2333a5cc hosted CI/native and Docker regression-hook
checks passed; live-provider qualification is not claimed.
- [x] Current 2333a5cc Security gates pass under upstream policy, with
existing exceptions disclosed above.

### Test Output

```text
# Final lockfile bytes published as a00f387; Linux / Python 3.12.14
PYTHONPATH=. python -m pytest -q tests/test_compression_cache.py tests/test_tool_result_interceptors.py --disable-warnings
115 passed in 31.60s
uv lock --check: passed
cargo metadata --locked: passed
cargo audit (0.22.2): zero reported vulnerabilities
pip-audit of frozen production [all] export: 24 vulnerabilities in Pillow 12.3.0 remain
```

The Rust audit retains the repository's existing RUSTSEC-2024-0436
ignore, unchanged; no new ignores were added. certifi's updated CA
bundle successfully loads 120 CA certificates. The exact remote
Cargo.lock / uv.lock Git blob hashes match the tested files.

At d74aa5c, the five adjacent
cache/registry/interceptor/session/token-mode suites passed 169 tests
with zero skips. Whole-repository Ruff and formatting (1805 files), mypy
for both modified source modules and whitespace checks passed. The
maintainer independently reported 144 Windows/Python 3.12 tests passing
with ten optional skips; that is maintainer evidence, not my local run.

Reproduce focused coverage at d74aa5c (compare added executable lines
against base 4c7acba):

```sh
python -m pytest tests/test_compression_cache.py tests/test_tool_result_interceptors.py --cov=headroom.cache.compression_cache --cov=headroom.proxy.interceptors.base --cov-branch --cov-report=json:coverage.json --cov-report=term-missing -q
```

Independent local coverage at d74aa5c: 115 cache/interceptor tests
passed; all 19 changed executable lines were covered and changed-line
branches had no misses. Whole-module coverage was 92% / 86%, not a
whole-project coverage claim. Codecov's 00:01:55 UTC comment preceded
the relevant shard uploads at 00:02:59 and 00:05:00; final service
aggregation remains unverified.

Historical hosted CI run39657491771 and rust run39657491768 passed for
d74aa5c; Security run39657491814 failed. These are not final a00f387
checks. Maintainers verify hosted checks; no contributor workflow rerun
was requested or initiated.

### Historical dependency blocker (cleared on a618561)

a00f387 updates rustls-webpki to 0.103.16, webpki-roots to 1.0.10 and
certifi to 2026.10.19, only in the lockfiles. Pillow 12.3.0 remains
vulnerable. Pillow 13 requires Python >=3.11; Headroom supports >=3.10.
fastembed 0.8.0 also requires Pillow <13. An actual uv resolver attempt
using conditional Pillow >=13 failed; the attempted manifest edit was
reverted. No overrides, audit suppressions or Python-support removal
were introduced. A compatible fastembed/Pillow release or a separately
agreed dependency/support-policy change is still required.

The initial environment's full-suite collection failed on missing
optional/native dependencies; full native/runtime/live-provider
qualification has not been established locally. Focused checks do not
imply complete qualification.

## Real Behavior Proof

- Environment: Linux, Python 3.12.14, pytest 9.1.1, Ruff 0.17.1.
Synthetic data only, HEADROOM_OFFLINE=1, HEADROOM_BEACON=off,
DO_NOT_TRACK=1; no LLM/provider calls or mocks.

- Exact command / steps: Run the following Python snippet in the project
environment. I called the real public
`CompressionCache.update_from_result` API to seed a cached string
compression, then the real shared `prepare_turn` session engine with a
new list-form tool result. Minimal reproduction:

```python
from headroom.cache.compression_cache import CompressionCache
from headroom.proxy.session_engine import prepare_turn, FREEZE_POLICY_REPLAYABLE

cache = CompressionCache()
raw = "synthetic records " * 50
older = {"role": "tool", "tool_call_id": "c1", "content": raw}
cache.update_from_result([older], [{**older, "content": "<compressed>"}])
newest = {"role": "tool", "tool_call_id": "c2", "content": [
    {"type": "text", "text": raw,
     "cache_control": {"type": "ephemeral", "ttl": "1h"}}
]}
prepared = prepare_turn(cache, [
    {"role": "user", "content": "synthetic request"}, newest
], policy=FREEZE_POLICY_REPLAYABLE)
print(prepared.pipeline_input[-1])
```

- Observed result: Before: content is the string `<compressed>`; the
breakpoint is absent.
After:

```text
{'role': 'tool', 'tool_call_id': 'c2', 'content': [{'type': 'text', 'text': '<compressed>', 'cache_control': {'type': 'ephemeral', 'ttl': '1h'}}]}
original_unchanged: True
frozen_message_count: 1
```

- Not tested: live OpenCode/gateway/Bedrock requests, provider cache
billing or hit rates, author-run Windows/macOS, final-head
native/runtime behavior, full suite. Maintainer Windows results are
separately attributed above. This proof establishes session-preparation
output, not a live provider outcome.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; no new feature/configuration.
Compatible certificate dependency lock versions are refreshed; no
manifest constraints changed.
- Minimum rollout channel: not applicable; this is a cache correctness
fix, not a rollout-managed feature.
- Kill switch / disable path: no new switch; revert this patch to
restore previous behavior.
- Qualification impact: no qualification settings changed; hosted
Rust/native checks passed on the current merge tree; live-provider
qualification remains unverified.
- Stable/default behavior changed: cached replay and direct interception
preserve single-text-block shape/metadata; unsupported layouts pass
through before transformations or savings accounting.
- Unsafe override required: no.
- Rollback path: revert the change.

## Review Readiness

- [x] I have performed a self-review

Independent review has also been performed.
- [x] This PR is ready for human review

Current-head hosted checks and the regression hook passed; maintainer
approval and real behavior proof are recorded above.
- [x] No CHANGELOG edits.
2026-10-29 20:27:26 -05:00

629 lines
25 KiB
Python

"""Integration: Mechanism B (read maturation) cache behaviour, through the
REAL Anthropic handler, across a multi-turn session.
The held Read is cached with its turn: the client's breakpoints are forwarded
where the client put them, and the Read matures once its file quiesces even
though the provider-confirmed prefix has moved past it by then. The unit
tests in ``test_read_maturation.py`` call the manager in isolation with
``frozen_message_count=0``; the live test in ``test_live/`` does a
2-request hold->mature with no intermediate turns. Neither exercises the
realistic path where a held Read sits across several turns while the prefix
tracker advances ``frozen_message_count`` from the provider's reported cache
usage.
This is a regression test for that path. It drives the real handler with a
mocked upstream that echoes the cache usage Anthropic would report (caching
everything up to the breakpoint the handler chose, system blocks included),
so the prefix tracker advances exactly as in production. It then asserts,
directly on the FORWARDED bytes:
1. the client's breakpoint is forwarded where the client put it on every
turn, including the turns that hold the Read;
2. the mechanism actually engages: the Read is held verbatim while the file
is active, then matures into a CCR marker once it quiesces, in that order.
Note on cache-state isolation: the CCR store is persistent (SQLite at
~/.headroom/ccr_store.db by default) and shared across processes, so stale
entries from prior runs can perturb maturation timing. Run against a clean
store for deterministic results.
"""
from __future__ import annotations
import copy
import json
import pytest
pytest.importorskip("fastapi")
import httpx
from fastapi.testclient import TestClient
from headroom.proxy.server import ProxyConfig, create_app
MODEL = "claude-haiku-4-5-20251001"
SYSTEM = [
{
"type": "text",
"text": "You are a coding assistant. Be terse. " * 200,
"cache_control": {"type": "ephemeral"},
}
]
READ_TOOL = {
"name": "Read",
"description": "Read a file",
"input_schema": {
"type": "object",
"properties": {"file_path": {"type": "string"}},
"required": ["file_path"],
},
}
READ_ID = "toolu_r1"
# Big enough to dominate message tokens and clear the maturation min-size gate.
BIG = "".join(f" {i}\tdef f_{i}(): return {i} # line {i}\n" for i in range(700))
def _read_pair(tail: bool) -> list[dict]:
tr = {"type": "tool_result", "tool_use_id": READ_ID, "content": BIG}
if tail:
tr["cache_control"] = {"type": "ephemeral"}
return [
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": READ_ID,
"name": "Read",
"input": {"file_path": "/x/foo.py"},
}
],
},
{"role": "user", "content": [tr]},
]
def _quiet_pair(i: int, tail: bool) -> list[dict]:
u = {"type": "text", "text": f"Unrelated question {i}: what is {i}+{i}?"}
if tail:
u["cache_control"] = {"type": "ephemeral"}
return [
{"role": "assistant", "content": [{"type": "text", "text": str(2 * i)}]},
{"role": "user", "content": [u]},
]
def _convo(nquiet: int) -> list[dict]:
"""Read of /x/foo.py followed by ``nquiet`` turns that never touch it.
The Claude-Code-style tail breakpoint rides the newest user block."""
msgs: list[dict] = [{"role": "user", "content": [{"type": "text", "text": "Read /x/foo.py"}]}]
msgs += _read_pair(tail=(nquiet == 0))
for i in range(1, nquiet + 1):
msgs += _quiet_pair(i, tail=(i == nquiet))
return msgs
def _busy_convo(nquiet: int) -> list[dict]:
"""``_convo`` with a large JSON Bash output before the Read and on every
quiet turn, as in a real coding session. The router compresses those, and
a compression run is what marks the held Read as locally replayable, so
the frozen count follows the provider's cache past it."""
def bash(tid: str) -> list[dict]:
out = json.dumps([{"path": f"src/{tid}_{j}.py", "size": j} for j in range(300)])
return [
{
"role": "assistant",
"content": [
{"type": "tool_use", "id": tid, "name": "Bash", "input": {"command": "ls"}}
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": tid, "content": out}],
},
]
msgs: list[dict] = [{"role": "user", "content": [{"type": "text", "text": "Read /x/foo.py"}]}]
msgs += bash("toolu_b0") + _read_pair(tail=False)
for i in range(1, nquiet + 1):
msgs += bash(f"toolu_b{i}")
msgs[-1]["content"][-1]["cache_control"] = {"type": "ephemeral"}
return msgs
def _breakpoint_index(messages: list[dict]) -> int:
"""Index of the last message carrying a cache_control block (-1 if none).
Anthropic caches everything up to AND INCLUDING this message."""
bp = -1
for i, m in enumerate(messages):
c = m.get("content")
if isinstance(c, list) and any(isinstance(b, dict) and "cache_control" in b for b in c):
bp = i
return bp
def _read_result_content(message: dict) -> str | None:
c = message.get("content")
if isinstance(c, list):
for b in c:
if (
isinstance(b, dict)
and b.get("type") == "tool_result"
and b.get("tool_use_id") == READ_ID
):
return b.get("content")
return None
def _est_tokens(message: dict) -> int:
return max(1, len(str(message.get("content", ""))) // 4)
def test_held_read_keeps_client_breakpoint_then_matures(monkeypatch):
# Isolate the CCR store: it is persistent (SQLite) and shared across
# processes by default, so stale entries from other runs would perturb
# maturation timing and make this test non-deterministic. The in-memory
# backend gives a pristine store per test.
from headroom.cache.compression_store import reset_compression_store
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
reset_compression_store()
# Match the real proxy: cache machinery ON (the prefix tracker + compression
# cache are what maturation's hold/frozen-count logic depends on). Disabling
# them masks the behavior under test.
config = ProxyConfig(
optimize=True,
read_maturation=True,
mode="token",
cache_enabled=True,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
# The provider confirms no cache (as after a lapse), so nothing is frozen
# and a quiet Read may mature. Once the provider has cached it, it stays
# verbatim instead: test_read_inside_the_cached_prefix_stays_verbatim.
forwarded = _drive_session(config, n_turns=7, session_id="nobust-1", cache_hits=False)
assert forwarded, "no requests were forwarded"
# Per-turn classification of the Read's forwarded form.
held_verbatim = [] # turns where the Read is forwarded verbatim (held)
matured = [] # turns where the Read has become a CCR marker
for turn, msgs in enumerate(forwarded):
for m in msgs:
content = _read_result_content(m)
if content is None:
continue
if content == BIG:
held_verbatim.append(turn)
elif "Retrieve original: hash=" in content:
matured.append(turn)
# INVARIANT 1: the client's breakpoint is forwarded where the client put
# it. This used to be the opposite ("the verbatim Read is never
# cache-written"): the handler moved the breakpoint in front of the held
# Read, which dropped Claude Code's tail breakpoint for the whole hold and
# re-sent the Read and everything after it uncached on every held turn, to
# save one cache write of the Read. The held Read is now cached with its
# turn.
moved = [
turn
for turn, msgs in enumerate(forwarded)
if _breakpoint_index(msgs) != _breakpoint_index(_convo(turn))
]
assert not moved, f"client cache breakpoint moved or dropped on turn(s) {moved}"
# INVARIANT 2 (mechanism actually engages): the Read is held verbatim while
# the file is active, then matures once it quiesces. Guards against a
# vacuous pass where maturation silently no-ops.
assert held_verbatim, "expected the fresh Read to be held verbatim on early turns"
assert matured, "expected the Read to mature into a CCR marker after quiescing"
# The matured marker only appears AFTER the verbatim hold (ordering).
assert min(matured) > max(held_verbatim), (
f"maturation must follow the hold: held={held_verbatim} matured={matured}"
)
def _drive_session(
config,
n_turns: int,
session_id: str,
saved_out: list[int] | None = None,
convo=_convo,
cache_hits: bool = True,
tools: list[dict] | None = None,
) -> list[list[dict]]:
"""Drive ``n_turns`` cumulative turns of ``convo`` through the real handler
with a mocked upstream; return the forwarded message arrays per turn. When
``saved_out`` is given, each turn's emitted ``x-headroom-tokens-saved`` is
appended to it. The upstream caches up to the handler's breakpoint, or
reports no cache at all when ``cache_hits`` is False."""
app = create_app(config)
forwarded: list[list[dict]] = []
with TestClient(app) as client:
proxy = client.app.state.proxy
original_retry = proxy._retry_request
async def _mock_upstream(method, url, headers, body, stream=False, **kwargs):
msgs = body.get("messages", []) or []
forwarded.append(copy.deepcopy(msgs))
bp = _breakpoint_index(msgs)
sys_tokens = sum(
max(1, len(str(b.get("text", ""))) // 4)
for b in (body.get("system") or [])
if isinstance(b, dict)
)
cached = sys_tokens + (sum(_est_tokens(m) for m in msgs[: bp + 1]) if bp >= 0 else 0)
if not cache_hits:
cached = 0
return httpx.Response(
200,
json={
"id": "msg_x",
"type": "message",
"role": "assistant",
"content": [{"type": "text", "text": "ok"}],
"usage": {
"input_tokens": 20,
"output_tokens": 2,
"cache_read_input_tokens": cached,
"cache_creation_input_tokens": 0,
},
},
)
proxy._retry_request = _mock_upstream
try:
for n in range(n_turns):
r = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"x-headroom-session-id": session_id,
"content-type": "application/json",
},
json={
"model": MODEL,
"max_tokens": 20,
"system": SYSTEM,
"tools": tools or [READ_TOOL],
"messages": convo(n),
},
)
assert r.status_code == 200, f"turn {n}: {r.text[:300]}"
if saved_out is not None:
saved_out.append(int(r.headers.get("x-headroom-tokens-saved", 0)))
finally:
proxy._retry_request = original_retry
return forwarded
def _first_matured_turn(forwarded: list[list[dict]]) -> int | None:
"""The first turn index whose forwarded Read is a CCR marker."""
for turn, msgs in enumerate(forwarded):
for m in msgs:
content = _read_result_content(m)
if content and "Retrieve original: hash=" in content:
return turn
return None
def test_read_cached_with_its_turn_stays_verbatim(monkeypatch):
"""In a session whose tool output compresses, the frozen count follows the
provider's cache past the held Read from the next request on. The Read is
then left verbatim for good: maturing it would re-write everything after it
at the cache-write rate, which costs more than the Read saves."""
from headroom.cache.compression_store import reset_compression_store
from headroom.transforms.read_maturation import ReadMaturationManager
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
reset_compression_store()
frozen_seen: list[int] = []
apply = ReadMaturationManager.apply
def _recording_apply(self, messages, frozen_message_count=0):
frozen_seen.append(frozen_message_count)
return apply(self, messages, frozen_message_count=frozen_message_count)
monkeypatch.setattr(ReadMaturationManager, "apply", _recording_apply)
config = ProxyConfig(
optimize=True,
read_maturation=True,
mode="token",
cache_enabled=True,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
forwarded = _drive_session(config, n_turns=7, session_id="cached-read-1", convo=_busy_convo)
moved = [
turn
for turn, msgs in enumerate(forwarded)
if _breakpoint_index(msgs) != _breakpoint_index(_busy_convo(turn))
]
assert not moved, f"client cache breakpoint moved or dropped on turn(s) {moved}"
read_at = 4 # the Read's tool_result message in _busy_convo
assert all(f > read_at for f in frozen_seen[1:]), (
f"fixture must cache the held Read: frozen counts {frozen_seen}"
)
assert _first_matured_turn(forwarded) is None, "a cached Read was matured"
DEFERRED_TOOL = {
"name": "mcp__notes__search",
"description": "Search notes",
"defer_loading": True,
"input_schema": {"type": "object", "properties": {"q": {"type": "string"}}},
}
def _tool_search_convo(nquiet: int) -> list[dict]:
"""``_busy_convo`` after a ToolSearch whose result holds only tool_reference
blocks, as Claude Code loads deferred tools. Such textless results are
stable; tests must not rely on them stopping the local replay window."""
msgs = _busy_convo(nquiet)
search = [
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "toolu_ts",
"name": "ToolSearch",
"input": {"query": "notes"},
}
],
},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_ts",
"content": [{"type": "tool_reference", "tool_name": DEFERRED_TOOL["name"]}],
}
],
},
]
return msgs[:1] + search + msgs[1:]
def _without_cache_control(messages: list[dict]) -> list[dict]:
return json.loads(
json.dumps(messages),
object_hook=lambda d: {k: v for k, v in d.items() if k != "cache_control"},
)
def _prefix_rewrites(forwarded: list[list[dict]]) -> list[int]:
"""Turns whose forwarded messages do not start with the previous turn's.
The mock caches each turn through its last message, so any such turn
re-writes the provider cache. Breakpoints move every turn and are ignored."""
rewrites = []
for turn in range(1, len(forwarded)):
prev = _without_cache_control(forwarded[turn - 1])
if _without_cache_control(forwarded[turn])[: len(prev)] != prev:
rewrites.append(turn)
return rewrites
def _record_pipeline_freeze(monkeypatch) -> list[int]:
from headroom.proxy import session_engine
seen: list[int] = []
prepare = session_engine.prepare_turn
def _recording_prepare(*args, **kwargs):
prep = prepare(*args, **kwargs)
seen.append(prep.frozen_message_count)
return prep
monkeypatch.setattr(session_engine, "prepare_turn", _recording_prepare)
return seen
@pytest.mark.parametrize(
("convo", "tools", "pipeline_stop"),
[
# Both input shapes exercise the same explicitly bounded local cache.
(_convo, [READ_TOOL], 2),
(_tool_search_convo, [READ_TOOL, DEFERRED_TOOL], 2),
],
ids=["quiet", "tool-reference"],
)
@pytest.mark.parametrize("cache_hits", [True, False], ids=["provider-cached", "provider-uncached"])
def test_read_inside_the_cached_prefix_stays_verbatim(
monkeypatch, convo, tools, pipeline_stop, cache_hits
):
"""By the time the Read's file quiesces the provider has cached it, so
maturing it would re-write the cache from the Read onward. The pipeline's
own freeze stops below the Read here; maturation must follow the
provider-confirmed floor that finalize_turn replays instead. The uncached
control must still mature, so retaining the Read cannot pass vacuously."""
from headroom.cache.compression_cache import CompressionCache
from headroom.cache.compression_store import reset_compression_store
from headroom.transforms.read_maturation import ReadMaturationManager
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
reset_compression_store()
# Explicitly model a local replay window shorter than the provider's
# confirmed prefix (e.g. after local cache eviction). Keep real preparation
# and tracking; do not depend on any particular tool-result shape being a
# cache miss. Textless ToolSearch results are legitimately stable now.
compute_frozen = CompressionCache.compute_frozen_count
def _bounded_local_replay(self, messages):
return min(compute_frozen(self, messages), pipeline_stop)
monkeypatch.setattr(CompressionCache, "compute_frozen_count", _bounded_local_replay)
pipeline_freeze = _record_pipeline_freeze(monkeypatch)
maturation_freeze: list[int] = []
apply = ReadMaturationManager.apply
def _recording_apply(self, messages, frozen_message_count=0):
maturation_freeze.append(frozen_message_count)
return apply(self, messages, frozen_message_count=frozen_message_count)
monkeypatch.setattr(ReadMaturationManager, "apply", _recording_apply)
config = ProxyConfig(
optimize=True,
read_maturation=True,
mode="token",
cache_enabled=True,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
forwarded = _drive_session(
config,
n_turns=8,
session_id="cached-read-2",
convo=convo,
tools=tools,
cache_hits=cache_hits,
)
assert pipeline_freeze and max(pipeline_freeze) <= pipeline_stop, (
f"fixture must keep the pipeline freeze below the Read: {pipeline_freeze}"
)
assert len(forwarded) == 8, "every turn must reach the upstream"
read_at = next(i for i, msg in enumerate(convo(0)) if _read_result_content(msg) is not None)
assert pipeline_stop <= read_at, "the local replay window must exclude the held Read"
if not cache_hits:
assert maturation_freeze and not any(maturation_freeze)
assert _first_matured_turn(forwarded) is not None, "uncached control never matured"
return
assert len(maturation_freeze) == 8 and all(f > read_at for f in maturation_freeze[1:]), (
f"provider-confirmed maturation floor must protect the Read: {maturation_freeze}"
)
assert all(_read_result_content(msgs[read_at]) == BIG for msgs in forwarded), (
"the held Read must remain present and verbatim on every turn"
)
assert _first_matured_turn(forwarded) is None, "a cached Read was matured"
assert not _prefix_rewrites(forwarded), "the forwarded cached prefix changed"
def test_quiesce_turns_config_is_honored(monkeypatch):
"""`quiesce_turns` must be runtime-configurable end-to-end: a fresh Read of
/x/foo.py matures `quiesce_turns` quiet turns after it appears (the convo
builds one quiet assistant turn per step, and the Read sits at assistant
turn 1). With quiesce_turns=2 it must mature at turn 2 — not the built-in
default of 5. Currently the handler hardcodes ReadMaturationConfig(enabled=
True), ignoring the configured value, so this fails (matures at 5)."""
from headroom.cache.compression_store import reset_compression_store
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
reset_compression_store()
config = ProxyConfig(
optimize=True,
read_maturation=True,
read_maturation_quiesce_turns=2,
mode="token",
cache_enabled=True,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
forwarded = _drive_session(config, n_turns=4, session_id="quiesce-cfg-1", cache_hits=False)
first = _first_matured_turn(forwarded)
assert first == 2, (
f"expected the Read to mature at turn 2 with quiesce_turns=2, "
f"but first matured at turn {first} (handler ignored the configured value)"
)
def test_read_maturation_knobs_from_env(monkeypatch):
"""Operators must be able to tune maturation via env vars (the pilot
playbook says 'pick quiesce_turns')."""
from headroom.proxy.server import _MULTI_WORKER_CONFIG_ENV, _proxy_config_from_env
# _proxy_config_from_env short-circuits on a prebuilt multi-worker JSON
# config and ignores the HEADROOM_* vars entirely. Clear it so this test
# actually exercises the env-var parsing path it claims to (and isn't
# poisoned by a leaked HEADROOM_PROXY_CONFIG_JSON from another test).
monkeypatch.delenv(_MULTI_WORKER_CONFIG_ENV, raising=False)
monkeypatch.setenv("HEADROOM_READ_MATURATION", "1")
monkeypatch.setenv("HEADROOM_ROLLOUT_CHANNEL", "beta")
monkeypatch.setenv("HEADROOM_READ_MATURATION_QUIESCE_TURNS", "3")
monkeypatch.setenv("HEADROOM_READ_MATURATION_MAX_HOLD_TURNS", "10")
monkeypatch.setenv("HEADROOM_READ_MATURATION_MIN_SIZE_BYTES", "4096")
cfg = _proxy_config_from_env()
assert cfg.read_maturation is True
assert cfg.read_maturation_quiesce_turns == 3
assert cfg.read_maturation_max_hold_turns == 10
assert cfg.read_maturation_min_size_bytes == 4096
def test_read_maturation_env_cannot_bypass_stable_rollout(monkeypatch):
"""Every env-driven server composition root must enforce the beta gate."""
from headroom.proxy.server import _MULTI_WORKER_CONFIG_ENV, _proxy_config_from_env
monkeypatch.delenv(_MULTI_WORKER_CONFIG_ENV, raising=False)
monkeypatch.setenv("HEADROOM_READ_MATURATION", "1")
monkeypatch.setenv("HEADROOM_ROLLOUT_CHANNEL", "stable")
cfg = _proxy_config_from_env()
assert cfg.read_maturation is False
assert cfg.rollout is not None
assert cfg.rollout.decision("read_maturation").reason.value == "blocked_by_channel"
def test_replayed_marker_is_not_rebooked_in_the_emitted_savings(monkeypatch):
"""The savings a request EMITS must be first-appearance.
Subtracting the replay debt inside the maturation block is not enough on
its own: the handler recomputes ``tokens_saved`` from the plain
original-vs-optimized diff twice more before the outcome is recorded (the
pre-send hook recount and the consistency recount), and either one throws
the adjustment away — so every replay turn re-books a removal that was
already booked when the Read matured. Assert on the emitted header, which
is the same figure the outcome and PERF line carry, not on the manager.
"""
from headroom.cache.compression_store import reset_compression_store
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
reset_compression_store()
config = ProxyConfig(
optimize=True,
read_maturation=True,
read_maturation_quiesce_turns=2,
mode="token",
cache_enabled=True,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
)
saved: list[int] = []
forwarded = _drive_session(
config, n_turns=5, session_id="first-appearance-1", saved_out=saved, cache_hits=False
)
first = _first_matured_turn(forwarded)
assert first is not None, "the Read never matured, so there is nothing to replay"
replays = saved[first + 1 :]
assert replays, "expected at least one replay turn after maturation"
# The turn that matures the Read books its removal, once.
assert saved[first] > 0, "the maturing turn booked nothing"
# Later turns forward the same marker instead of the verbatim Read, so the
# wire diff is just as large — but it is the SAME removal, already booked.
assert max(replays) < saved[first] * 0.2, (
f"replayed marker re-booked through the final accounting path: "
f"matured turn saved {saved[first]}, replay turns saved {replays}"
)