mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
Current status (2026-10-24 20:50 UTC): exact head `2333a5ccf7700f86f8767d1c5d02fbfb903c966c` is conflict-free and [approved by JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639834650). All seven held workflows are now successful: [CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40128152028) has 17 successful jobs and five skips; [Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151955) has 12 successful jobs. Both [Wrap E2E](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151965/job/120974772335) and CI's Docker-native job actually execute the new npm hook: two tests pass, zero fail or skip. Logs check out synthetic merge `91669e2` (this head into `3fef15d3`). The maintainer independently reports 146 tests passed with ten optional skips, including the nine new regressions, plus Ruff lint/formatting. These hosted and maintainer results supplement, rather than replace or relabel, the author's limited local checks. Security passes upstream policy, including three excepted bundled npm advisories until 2026-11-15; this is not a zero-findings claim. Live-provider billing/cache behavior remains unqualified. The prior approval-pending statement and local Docker limitation in the [author response](https://github.com/headroomlabs-ai/headroom/pull/4334#issuecomment-6388924792) describe the earlier observation, now superseded by these hosted results. ### Previous-head qualification (historical) Current status (2026-10-24 19:54 UTC): exact head `a618561542a8bb48fdb29b07c66c3c6647b7a690` is conflict-free and [approved by JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639593849); the maintainer explicitly cleared the earlier dependency qualification hold. All seven previously held workflows completed successfully, including [CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172110) (17 successful jobs, five skipped), [Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172235) (12 successful jobs), Rust and Wrap E2E. Logs check out synthetic merge `2ff995d67ef4bb52590c1c0ca95514c85e9693ab` (this head into `3fef15d3`). Maintainer qualification reports 137 tests passed with ten optional skips plus Ruff/whitespace; these are their results, not another author-run suite. Security passes upstream policy, not a zero-findings claim: E2E npm retains three excepted bundled advisories under upstream's existing policy until 2026-11-15. No additional exception was introduced here. Live-provider behavior remains unqualified. Earlier dated results below are historical. ## Description Fixes #4332. Replaying a cached compression for an OpenAI `role: tool` message currently converts list-form content to a string, discarding its client-supplied `cache_control` and other block metadata. A single text block now keeps its shape and metadata; only its text is replaced in the existing deep copy. For multiple blocks or non-text/malformed block shapes, replay leaves the message unchanged, before a cache lookup is counted. The cached joined string does not contain enough information to safely reconstruct block boundaries or preserve media. String-form OpenAI and Anthropic behavior are unchanged. This does not duplicate #4333, which explicitly excludes the token-mode lost-breakpoint issue. ## Type of Change - [x] Bug fix ## Changes Made - Preserve the deep-copied single-text OpenAI content block and replace text only. - Leave ambiguous lists unchanged before cache hit accounting. - Share the eligibility guard across cache replay, the rewrite helper and the real interceptor path before callbacks, disclosure state and savings accounting. - Add direct-helper/cache/interceptor regressions for unsupported shapes, metadata isolation and progressive-disclosure isolation. - Earlier compatible dependency updates are now incorporated in upstream main; the final PR delta keeps the cache fix and regressions, with dependency constraints and locks unchanged from main; the tools manifest now adds only a test script. ## Testing - [x] Focused regression tests pass. - [x] Lint / formatting and changed-module type checks passed at d74aa5c; runtime/test code is unchanged bya00f387. - [x] Independent review performed. - [x] Current2333a5cchosted CI/native and Docker regression-hook checks passed; live-provider qualification is not claimed. - [x] Current2333a5ccSecurity gates pass under upstream policy, with existing exceptions disclosed above. ### Test Output ```text # Final lockfile bytes published as a00f387; Linux / Python 3.12.14 PYTHONPATH=. python -m pytest -q tests/test_compression_cache.py tests/test_tool_result_interceptors.py --disable-warnings 115 passed in 31.60s uv lock --check: passed cargo metadata --locked: passed cargo audit (0.22.2): zero reported vulnerabilities pip-audit of frozen production [all] export: 24 vulnerabilities in Pillow 12.3.0 remain ``` The Rust audit retains the repository's existing RUSTSEC-2024-0436 ignore, unchanged; no new ignores were added. certifi's updated CA bundle successfully loads 120 CA certificates. The exact remote Cargo.lock / uv.lock Git blob hashes match the tested files. Atd74aa5c, the five adjacent cache/registry/interceptor/session/token-mode suites passed 169 tests with zero skips. Whole-repository Ruff and formatting (1805 files), mypy for both modified source modules and whitespace checks passed. The maintainer independently reported 144 Windows/Python 3.12 tests passing with ten optional skips; that is maintainer evidence, not my local run. Reproduce focused coverage atd74aa5c(compare added executable lines against base4c7acba): ```sh python -m pytest tests/test_compression_cache.py tests/test_tool_result_interceptors.py --cov=headroom.cache.compression_cache --cov=headroom.proxy.interceptors.base --cov-branch --cov-report=json:coverage.json --cov-report=term-missing -q ``` Independent local coverage atd74aa5c: 115 cache/interceptor tests passed; all 19 changed executable lines were covered and changed-line branches had no misses. Whole-module coverage was 92% / 86%, not a whole-project coverage claim. Codecov's 00:01:55 UTC comment preceded the relevant shard uploads at 00:02:59 and 00:05:00; final service aggregation remains unverified. Historical hosted CI run39657491771 and rust run39657491768 passed for d74aa5c; Security run39657491814 failed. These are not finala00f387checks. Maintainers verify hosted checks; no contributor workflow rerun was requested or initiated. ### Historical dependency blocker (cleared ona618561)a00f387updates rustls-webpki to 0.103.16, webpki-roots to 1.0.10 and certifi to 2026.10.19, only in the lockfiles. Pillow 12.3.0 remains vulnerable. Pillow 13 requires Python >=3.11; Headroom supports >=3.10. fastembed 0.8.0 also requires Pillow <13. An actual uv resolver attempt using conditional Pillow >=13 failed; the attempted manifest edit was reverted. No overrides, audit suppressions or Python-support removal were introduced. A compatible fastembed/Pillow release or a separately agreed dependency/support-policy change is still required. The initial environment's full-suite collection failed on missing optional/native dependencies; full native/runtime/live-provider qualification has not been established locally. Focused checks do not imply complete qualification. ## Real Behavior Proof - Environment: Linux, Python 3.12.14, pytest 9.1.1, Ruff 0.17.1. Synthetic data only, HEADROOM_OFFLINE=1, HEADROOM_BEACON=off, DO_NOT_TRACK=1; no LLM/provider calls or mocks. - Exact command / steps: Run the following Python snippet in the project environment. I called the real public `CompressionCache.update_from_result` API to seed a cached string compression, then the real shared `prepare_turn` session engine with a new list-form tool result. Minimal reproduction: ```python from headroom.cache.compression_cache import CompressionCache from headroom.proxy.session_engine import prepare_turn, FREEZE_POLICY_REPLAYABLE cache = CompressionCache() raw = "synthetic records " * 50 older = {"role": "tool", "tool_call_id": "c1", "content": raw} cache.update_from_result([older], [{**older, "content": "<compressed>"}]) newest = {"role": "tool", "tool_call_id": "c2", "content": [ {"type": "text", "text": raw, "cache_control": {"type": "ephemeral", "ttl": "1h"}} ]} prepared = prepare_turn(cache, [ {"role": "user", "content": "synthetic request"}, newest ], policy=FREEZE_POLICY_REPLAYABLE) print(prepared.pipeline_input[-1]) ``` - Observed result: Before: content is the string `<compressed>`; the breakpoint is absent. After: ```text {'role': 'tool', 'tool_call_id': 'c2', 'content': [{'type': 'text', 'text': '<compressed>', 'cache_control': {'type': 'ephemeral', 'ttl': '1h'}}]} original_unchanged: True frozen_message_count: 1 ``` - Not tested: live OpenCode/gateway/Bedrock requests, provider cache billing or hit rates, author-run Windows/macOS, final-head native/runtime behavior, full suite. Maintainer Windows results are separately attributed above. This proof establishes session-preparation output, not a live provider outcome. ## Runtime Rollout Safety - Rollout-managed feature(s): none; no new feature/configuration. Compatible certificate dependency lock versions are refreshed; no manifest constraints changed. - Minimum rollout channel: not applicable; this is a cache correctness fix, not a rollout-managed feature. - Kill switch / disable path: no new switch; revert this patch to restore previous behavior. - Qualification impact: no qualification settings changed; hosted Rust/native checks passed on the current merge tree; live-provider qualification remains unverified. - Stable/default behavior changed: cached replay and direct interception preserve single-text-block shape/metadata; unsupported layouts pass through before transformations or savings accounting. - Unsafe override required: no. - Rollback path: revert the change. ## Review Readiness - [x] I have performed a self-review Independent review has also been performed. - [x] This PR is ready for human review Current-head hosted checks and the regression hook passed; maintainer approval and real behavior proof are recorded above. - [x] No CHANGELOG edits.
629 lines
25 KiB
Python
629 lines
25 KiB
Python
"""Integration: Mechanism B (read maturation) cache behaviour, through the
|
|
REAL Anthropic handler, across a multi-turn session.
|
|
|
|
The held Read is cached with its turn: the client's breakpoints are forwarded
|
|
where the client put them, and the Read matures once its file quiesces even
|
|
though the provider-confirmed prefix has moved past it by then. The unit
|
|
tests in ``test_read_maturation.py`` call the manager in isolation with
|
|
``frozen_message_count=0``; the live test in ``test_live/`` does a
|
|
2-request hold->mature with no intermediate turns. Neither exercises the
|
|
realistic path where a held Read sits across several turns while the prefix
|
|
tracker advances ``frozen_message_count`` from the provider's reported cache
|
|
usage.
|
|
|
|
This is a regression test for that path. It drives the real handler with a
|
|
mocked upstream that echoes the cache usage Anthropic would report (caching
|
|
everything up to the breakpoint the handler chose, system blocks included),
|
|
so the prefix tracker advances exactly as in production. It then asserts,
|
|
directly on the FORWARDED bytes:
|
|
|
|
1. the client's breakpoint is forwarded where the client put it on every
|
|
turn, including the turns that hold the Read;
|
|
2. the mechanism actually engages: the Read is held verbatim while the file
|
|
is active, then matures into a CCR marker once it quiesces, in that order.
|
|
|
|
Note on cache-state isolation: the CCR store is persistent (SQLite at
|
|
~/.headroom/ccr_store.db by default) and shared across processes, so stale
|
|
entries from prior runs can perturb maturation timing. Run against a clean
|
|
store for deterministic results.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import copy
|
|
import json
|
|
|
|
import pytest
|
|
|
|
pytest.importorskip("fastapi")
|
|
|
|
import httpx
|
|
from fastapi.testclient import TestClient
|
|
|
|
from headroom.proxy.server import ProxyConfig, create_app
|
|
|
|
MODEL = "claude-haiku-4-5-20251001"
|
|
SYSTEM = [
|
|
{
|
|
"type": "text",
|
|
"text": "You are a coding assistant. Be terse. " * 200,
|
|
"cache_control": {"type": "ephemeral"},
|
|
}
|
|
]
|
|
READ_TOOL = {
|
|
"name": "Read",
|
|
"description": "Read a file",
|
|
"input_schema": {
|
|
"type": "object",
|
|
"properties": {"file_path": {"type": "string"}},
|
|
"required": ["file_path"],
|
|
},
|
|
}
|
|
READ_ID = "toolu_r1"
|
|
# Big enough to dominate message tokens and clear the maturation min-size gate.
|
|
BIG = "".join(f" {i}\tdef f_{i}(): return {i} # line {i}\n" for i in range(700))
|
|
|
|
|
|
def _read_pair(tail: bool) -> list[dict]:
|
|
tr = {"type": "tool_result", "tool_use_id": READ_ID, "content": BIG}
|
|
if tail:
|
|
tr["cache_control"] = {"type": "ephemeral"}
|
|
return [
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": READ_ID,
|
|
"name": "Read",
|
|
"input": {"file_path": "/x/foo.py"},
|
|
}
|
|
],
|
|
},
|
|
{"role": "user", "content": [tr]},
|
|
]
|
|
|
|
|
|
def _quiet_pair(i: int, tail: bool) -> list[dict]:
|
|
u = {"type": "text", "text": f"Unrelated question {i}: what is {i}+{i}?"}
|
|
if tail:
|
|
u["cache_control"] = {"type": "ephemeral"}
|
|
return [
|
|
{"role": "assistant", "content": [{"type": "text", "text": str(2 * i)}]},
|
|
{"role": "user", "content": [u]},
|
|
]
|
|
|
|
|
|
def _convo(nquiet: int) -> list[dict]:
|
|
"""Read of /x/foo.py followed by ``nquiet`` turns that never touch it.
|
|
The Claude-Code-style tail breakpoint rides the newest user block."""
|
|
msgs: list[dict] = [{"role": "user", "content": [{"type": "text", "text": "Read /x/foo.py"}]}]
|
|
msgs += _read_pair(tail=(nquiet == 0))
|
|
for i in range(1, nquiet + 1):
|
|
msgs += _quiet_pair(i, tail=(i == nquiet))
|
|
return msgs
|
|
|
|
|
|
def _busy_convo(nquiet: int) -> list[dict]:
|
|
"""``_convo`` with a large JSON Bash output before the Read and on every
|
|
quiet turn, as in a real coding session. The router compresses those, and
|
|
a compression run is what marks the held Read as locally replayable, so
|
|
the frozen count follows the provider's cache past it."""
|
|
|
|
def bash(tid: str) -> list[dict]:
|
|
out = json.dumps([{"path": f"src/{tid}_{j}.py", "size": j} for j in range(300)])
|
|
return [
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{"type": "tool_use", "id": tid, "name": "Bash", "input": {"command": "ls"}}
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [{"type": "tool_result", "tool_use_id": tid, "content": out}],
|
|
},
|
|
]
|
|
|
|
msgs: list[dict] = [{"role": "user", "content": [{"type": "text", "text": "Read /x/foo.py"}]}]
|
|
msgs += bash("toolu_b0") + _read_pair(tail=False)
|
|
for i in range(1, nquiet + 1):
|
|
msgs += bash(f"toolu_b{i}")
|
|
msgs[-1]["content"][-1]["cache_control"] = {"type": "ephemeral"}
|
|
return msgs
|
|
|
|
|
|
def _breakpoint_index(messages: list[dict]) -> int:
|
|
"""Index of the last message carrying a cache_control block (-1 if none).
|
|
Anthropic caches everything up to AND INCLUDING this message."""
|
|
bp = -1
|
|
for i, m in enumerate(messages):
|
|
c = m.get("content")
|
|
if isinstance(c, list) and any(isinstance(b, dict) and "cache_control" in b for b in c):
|
|
bp = i
|
|
return bp
|
|
|
|
|
|
def _read_result_content(message: dict) -> str | None:
|
|
c = message.get("content")
|
|
if isinstance(c, list):
|
|
for b in c:
|
|
if (
|
|
isinstance(b, dict)
|
|
and b.get("type") == "tool_result"
|
|
and b.get("tool_use_id") == READ_ID
|
|
):
|
|
return b.get("content")
|
|
return None
|
|
|
|
|
|
def _est_tokens(message: dict) -> int:
|
|
return max(1, len(str(message.get("content", ""))) // 4)
|
|
|
|
|
|
def test_held_read_keeps_client_breakpoint_then_matures(monkeypatch):
|
|
# Isolate the CCR store: it is persistent (SQLite) and shared across
|
|
# processes by default, so stale entries from other runs would perturb
|
|
# maturation timing and make this test non-deterministic. The in-memory
|
|
# backend gives a pristine store per test.
|
|
from headroom.cache.compression_store import reset_compression_store
|
|
|
|
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
|
|
reset_compression_store()
|
|
|
|
# Match the real proxy: cache machinery ON (the prefix tracker + compression
|
|
# cache are what maturation's hold/frozen-count logic depends on). Disabling
|
|
# them masks the behavior under test.
|
|
config = ProxyConfig(
|
|
optimize=True,
|
|
read_maturation=True,
|
|
mode="token",
|
|
cache_enabled=True,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
)
|
|
# The provider confirms no cache (as after a lapse), so nothing is frozen
|
|
# and a quiet Read may mature. Once the provider has cached it, it stays
|
|
# verbatim instead: test_read_inside_the_cached_prefix_stays_verbatim.
|
|
forwarded = _drive_session(config, n_turns=7, session_id="nobust-1", cache_hits=False)
|
|
|
|
assert forwarded, "no requests were forwarded"
|
|
|
|
# Per-turn classification of the Read's forwarded form.
|
|
held_verbatim = [] # turns where the Read is forwarded verbatim (held)
|
|
matured = [] # turns where the Read has become a CCR marker
|
|
for turn, msgs in enumerate(forwarded):
|
|
for m in msgs:
|
|
content = _read_result_content(m)
|
|
if content is None:
|
|
continue
|
|
if content == BIG:
|
|
held_verbatim.append(turn)
|
|
elif "Retrieve original: hash=" in content:
|
|
matured.append(turn)
|
|
|
|
# INVARIANT 1: the client's breakpoint is forwarded where the client put
|
|
# it. This used to be the opposite ("the verbatim Read is never
|
|
# cache-written"): the handler moved the breakpoint in front of the held
|
|
# Read, which dropped Claude Code's tail breakpoint for the whole hold and
|
|
# re-sent the Read and everything after it uncached on every held turn, to
|
|
# save one cache write of the Read. The held Read is now cached with its
|
|
# turn.
|
|
moved = [
|
|
turn
|
|
for turn, msgs in enumerate(forwarded)
|
|
if _breakpoint_index(msgs) != _breakpoint_index(_convo(turn))
|
|
]
|
|
assert not moved, f"client cache breakpoint moved or dropped on turn(s) {moved}"
|
|
|
|
# INVARIANT 2 (mechanism actually engages): the Read is held verbatim while
|
|
# the file is active, then matures once it quiesces. Guards against a
|
|
# vacuous pass where maturation silently no-ops.
|
|
assert held_verbatim, "expected the fresh Read to be held verbatim on early turns"
|
|
assert matured, "expected the Read to mature into a CCR marker after quiescing"
|
|
# The matured marker only appears AFTER the verbatim hold (ordering).
|
|
assert min(matured) > max(held_verbatim), (
|
|
f"maturation must follow the hold: held={held_verbatim} matured={matured}"
|
|
)
|
|
|
|
|
|
def _drive_session(
|
|
config,
|
|
n_turns: int,
|
|
session_id: str,
|
|
saved_out: list[int] | None = None,
|
|
convo=_convo,
|
|
cache_hits: bool = True,
|
|
tools: list[dict] | None = None,
|
|
) -> list[list[dict]]:
|
|
"""Drive ``n_turns`` cumulative turns of ``convo`` through the real handler
|
|
with a mocked upstream; return the forwarded message arrays per turn. When
|
|
``saved_out`` is given, each turn's emitted ``x-headroom-tokens-saved`` is
|
|
appended to it. The upstream caches up to the handler's breakpoint, or
|
|
reports no cache at all when ``cache_hits`` is False."""
|
|
app = create_app(config)
|
|
forwarded: list[list[dict]] = []
|
|
with TestClient(app) as client:
|
|
proxy = client.app.state.proxy
|
|
original_retry = proxy._retry_request
|
|
|
|
async def _mock_upstream(method, url, headers, body, stream=False, **kwargs):
|
|
msgs = body.get("messages", []) or []
|
|
forwarded.append(copy.deepcopy(msgs))
|
|
bp = _breakpoint_index(msgs)
|
|
sys_tokens = sum(
|
|
max(1, len(str(b.get("text", ""))) // 4)
|
|
for b in (body.get("system") or [])
|
|
if isinstance(b, dict)
|
|
)
|
|
cached = sys_tokens + (sum(_est_tokens(m) for m in msgs[: bp + 1]) if bp >= 0 else 0)
|
|
if not cache_hits:
|
|
cached = 0
|
|
return httpx.Response(
|
|
200,
|
|
json={
|
|
"id": "msg_x",
|
|
"type": "message",
|
|
"role": "assistant",
|
|
"content": [{"type": "text", "text": "ok"}],
|
|
"usage": {
|
|
"input_tokens": 20,
|
|
"output_tokens": 2,
|
|
"cache_read_input_tokens": cached,
|
|
"cache_creation_input_tokens": 0,
|
|
},
|
|
},
|
|
)
|
|
|
|
proxy._retry_request = _mock_upstream
|
|
try:
|
|
for n in range(n_turns):
|
|
r = client.post(
|
|
"/v1/messages",
|
|
headers={
|
|
"x-api-key": "test-key",
|
|
"anthropic-version": "2023-06-01",
|
|
"x-headroom-session-id": session_id,
|
|
"content-type": "application/json",
|
|
},
|
|
json={
|
|
"model": MODEL,
|
|
"max_tokens": 20,
|
|
"system": SYSTEM,
|
|
"tools": tools or [READ_TOOL],
|
|
"messages": convo(n),
|
|
},
|
|
)
|
|
assert r.status_code == 200, f"turn {n}: {r.text[:300]}"
|
|
if saved_out is not None:
|
|
saved_out.append(int(r.headers.get("x-headroom-tokens-saved", 0)))
|
|
finally:
|
|
proxy._retry_request = original_retry
|
|
return forwarded
|
|
|
|
|
|
def _first_matured_turn(forwarded: list[list[dict]]) -> int | None:
|
|
"""The first turn index whose forwarded Read is a CCR marker."""
|
|
for turn, msgs in enumerate(forwarded):
|
|
for m in msgs:
|
|
content = _read_result_content(m)
|
|
if content and "Retrieve original: hash=" in content:
|
|
return turn
|
|
return None
|
|
|
|
|
|
def test_read_cached_with_its_turn_stays_verbatim(monkeypatch):
|
|
"""In a session whose tool output compresses, the frozen count follows the
|
|
provider's cache past the held Read from the next request on. The Read is
|
|
then left verbatim for good: maturing it would re-write everything after it
|
|
at the cache-write rate, which costs more than the Read saves."""
|
|
from headroom.cache.compression_store import reset_compression_store
|
|
from headroom.transforms.read_maturation import ReadMaturationManager
|
|
|
|
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
|
|
reset_compression_store()
|
|
|
|
frozen_seen: list[int] = []
|
|
apply = ReadMaturationManager.apply
|
|
|
|
def _recording_apply(self, messages, frozen_message_count=0):
|
|
frozen_seen.append(frozen_message_count)
|
|
return apply(self, messages, frozen_message_count=frozen_message_count)
|
|
|
|
monkeypatch.setattr(ReadMaturationManager, "apply", _recording_apply)
|
|
|
|
config = ProxyConfig(
|
|
optimize=True,
|
|
read_maturation=True,
|
|
mode="token",
|
|
cache_enabled=True,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
)
|
|
forwarded = _drive_session(config, n_turns=7, session_id="cached-read-1", convo=_busy_convo)
|
|
|
|
moved = [
|
|
turn
|
|
for turn, msgs in enumerate(forwarded)
|
|
if _breakpoint_index(msgs) != _breakpoint_index(_busy_convo(turn))
|
|
]
|
|
assert not moved, f"client cache breakpoint moved or dropped on turn(s) {moved}"
|
|
read_at = 4 # the Read's tool_result message in _busy_convo
|
|
assert all(f > read_at for f in frozen_seen[1:]), (
|
|
f"fixture must cache the held Read: frozen counts {frozen_seen}"
|
|
)
|
|
assert _first_matured_turn(forwarded) is None, "a cached Read was matured"
|
|
|
|
|
|
DEFERRED_TOOL = {
|
|
"name": "mcp__notes__search",
|
|
"description": "Search notes",
|
|
"defer_loading": True,
|
|
"input_schema": {"type": "object", "properties": {"q": {"type": "string"}}},
|
|
}
|
|
|
|
|
|
def _tool_search_convo(nquiet: int) -> list[dict]:
|
|
"""``_busy_convo`` after a ToolSearch whose result holds only tool_reference
|
|
blocks, as Claude Code loads deferred tools. Such textless results are
|
|
stable; tests must not rely on them stopping the local replay window."""
|
|
msgs = _busy_convo(nquiet)
|
|
search = [
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_ts",
|
|
"name": "ToolSearch",
|
|
"input": {"query": "notes"},
|
|
}
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_ts",
|
|
"content": [{"type": "tool_reference", "tool_name": DEFERRED_TOOL["name"]}],
|
|
}
|
|
],
|
|
},
|
|
]
|
|
return msgs[:1] + search + msgs[1:]
|
|
|
|
|
|
def _without_cache_control(messages: list[dict]) -> list[dict]:
|
|
return json.loads(
|
|
json.dumps(messages),
|
|
object_hook=lambda d: {k: v for k, v in d.items() if k != "cache_control"},
|
|
)
|
|
|
|
|
|
def _prefix_rewrites(forwarded: list[list[dict]]) -> list[int]:
|
|
"""Turns whose forwarded messages do not start with the previous turn's.
|
|
The mock caches each turn through its last message, so any such turn
|
|
re-writes the provider cache. Breakpoints move every turn and are ignored."""
|
|
rewrites = []
|
|
for turn in range(1, len(forwarded)):
|
|
prev = _without_cache_control(forwarded[turn - 1])
|
|
if _without_cache_control(forwarded[turn])[: len(prev)] != prev:
|
|
rewrites.append(turn)
|
|
return rewrites
|
|
|
|
|
|
def _record_pipeline_freeze(monkeypatch) -> list[int]:
|
|
from headroom.proxy import session_engine
|
|
|
|
seen: list[int] = []
|
|
prepare = session_engine.prepare_turn
|
|
|
|
def _recording_prepare(*args, **kwargs):
|
|
prep = prepare(*args, **kwargs)
|
|
seen.append(prep.frozen_message_count)
|
|
return prep
|
|
|
|
monkeypatch.setattr(session_engine, "prepare_turn", _recording_prepare)
|
|
return seen
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("convo", "tools", "pipeline_stop"),
|
|
[
|
|
# Both input shapes exercise the same explicitly bounded local cache.
|
|
(_convo, [READ_TOOL], 2),
|
|
(_tool_search_convo, [READ_TOOL, DEFERRED_TOOL], 2),
|
|
],
|
|
ids=["quiet", "tool-reference"],
|
|
)
|
|
@pytest.mark.parametrize("cache_hits", [True, False], ids=["provider-cached", "provider-uncached"])
|
|
def test_read_inside_the_cached_prefix_stays_verbatim(
|
|
monkeypatch, convo, tools, pipeline_stop, cache_hits
|
|
):
|
|
"""By the time the Read's file quiesces the provider has cached it, so
|
|
maturing it would re-write the cache from the Read onward. The pipeline's
|
|
own freeze stops below the Read here; maturation must follow the
|
|
provider-confirmed floor that finalize_turn replays instead. The uncached
|
|
control must still mature, so retaining the Read cannot pass vacuously."""
|
|
from headroom.cache.compression_cache import CompressionCache
|
|
from headroom.cache.compression_store import reset_compression_store
|
|
from headroom.transforms.read_maturation import ReadMaturationManager
|
|
|
|
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
|
|
reset_compression_store()
|
|
# Explicitly model a local replay window shorter than the provider's
|
|
# confirmed prefix (e.g. after local cache eviction). Keep real preparation
|
|
# and tracking; do not depend on any particular tool-result shape being a
|
|
# cache miss. Textless ToolSearch results are legitimately stable now.
|
|
compute_frozen = CompressionCache.compute_frozen_count
|
|
|
|
def _bounded_local_replay(self, messages):
|
|
return min(compute_frozen(self, messages), pipeline_stop)
|
|
|
|
monkeypatch.setattr(CompressionCache, "compute_frozen_count", _bounded_local_replay)
|
|
pipeline_freeze = _record_pipeline_freeze(monkeypatch)
|
|
maturation_freeze: list[int] = []
|
|
apply = ReadMaturationManager.apply
|
|
|
|
def _recording_apply(self, messages, frozen_message_count=0):
|
|
maturation_freeze.append(frozen_message_count)
|
|
return apply(self, messages, frozen_message_count=frozen_message_count)
|
|
|
|
monkeypatch.setattr(ReadMaturationManager, "apply", _recording_apply)
|
|
config = ProxyConfig(
|
|
optimize=True,
|
|
read_maturation=True,
|
|
mode="token",
|
|
cache_enabled=True,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
)
|
|
|
|
forwarded = _drive_session(
|
|
config,
|
|
n_turns=8,
|
|
session_id="cached-read-2",
|
|
convo=convo,
|
|
tools=tools,
|
|
cache_hits=cache_hits,
|
|
)
|
|
|
|
assert pipeline_freeze and max(pipeline_freeze) <= pipeline_stop, (
|
|
f"fixture must keep the pipeline freeze below the Read: {pipeline_freeze}"
|
|
)
|
|
assert len(forwarded) == 8, "every turn must reach the upstream"
|
|
read_at = next(i for i, msg in enumerate(convo(0)) if _read_result_content(msg) is not None)
|
|
assert pipeline_stop <= read_at, "the local replay window must exclude the held Read"
|
|
if not cache_hits:
|
|
assert maturation_freeze and not any(maturation_freeze)
|
|
assert _first_matured_turn(forwarded) is not None, "uncached control never matured"
|
|
return
|
|
assert len(maturation_freeze) == 8 and all(f > read_at for f in maturation_freeze[1:]), (
|
|
f"provider-confirmed maturation floor must protect the Read: {maturation_freeze}"
|
|
)
|
|
assert all(_read_result_content(msgs[read_at]) == BIG for msgs in forwarded), (
|
|
"the held Read must remain present and verbatim on every turn"
|
|
)
|
|
assert _first_matured_turn(forwarded) is None, "a cached Read was matured"
|
|
assert not _prefix_rewrites(forwarded), "the forwarded cached prefix changed"
|
|
|
|
|
|
def test_quiesce_turns_config_is_honored(monkeypatch):
|
|
"""`quiesce_turns` must be runtime-configurable end-to-end: a fresh Read of
|
|
/x/foo.py matures `quiesce_turns` quiet turns after it appears (the convo
|
|
builds one quiet assistant turn per step, and the Read sits at assistant
|
|
turn 1). With quiesce_turns=2 it must mature at turn 2 — not the built-in
|
|
default of 5. Currently the handler hardcodes ReadMaturationConfig(enabled=
|
|
True), ignoring the configured value, so this fails (matures at 5)."""
|
|
from headroom.cache.compression_store import reset_compression_store
|
|
|
|
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
|
|
reset_compression_store()
|
|
|
|
config = ProxyConfig(
|
|
optimize=True,
|
|
read_maturation=True,
|
|
read_maturation_quiesce_turns=2,
|
|
mode="token",
|
|
cache_enabled=True,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
)
|
|
forwarded = _drive_session(config, n_turns=4, session_id="quiesce-cfg-1", cache_hits=False)
|
|
first = _first_matured_turn(forwarded)
|
|
assert first == 2, (
|
|
f"expected the Read to mature at turn 2 with quiesce_turns=2, "
|
|
f"but first matured at turn {first} (handler ignored the configured value)"
|
|
)
|
|
|
|
|
|
def test_read_maturation_knobs_from_env(monkeypatch):
|
|
"""Operators must be able to tune maturation via env vars (the pilot
|
|
playbook says 'pick quiesce_turns')."""
|
|
from headroom.proxy.server import _MULTI_WORKER_CONFIG_ENV, _proxy_config_from_env
|
|
|
|
# _proxy_config_from_env short-circuits on a prebuilt multi-worker JSON
|
|
# config and ignores the HEADROOM_* vars entirely. Clear it so this test
|
|
# actually exercises the env-var parsing path it claims to (and isn't
|
|
# poisoned by a leaked HEADROOM_PROXY_CONFIG_JSON from another test).
|
|
monkeypatch.delenv(_MULTI_WORKER_CONFIG_ENV, raising=False)
|
|
|
|
monkeypatch.setenv("HEADROOM_READ_MATURATION", "1")
|
|
monkeypatch.setenv("HEADROOM_ROLLOUT_CHANNEL", "beta")
|
|
monkeypatch.setenv("HEADROOM_READ_MATURATION_QUIESCE_TURNS", "3")
|
|
monkeypatch.setenv("HEADROOM_READ_MATURATION_MAX_HOLD_TURNS", "10")
|
|
monkeypatch.setenv("HEADROOM_READ_MATURATION_MIN_SIZE_BYTES", "4096")
|
|
|
|
cfg = _proxy_config_from_env()
|
|
|
|
assert cfg.read_maturation is True
|
|
assert cfg.read_maturation_quiesce_turns == 3
|
|
assert cfg.read_maturation_max_hold_turns == 10
|
|
assert cfg.read_maturation_min_size_bytes == 4096
|
|
|
|
|
|
def test_read_maturation_env_cannot_bypass_stable_rollout(monkeypatch):
|
|
"""Every env-driven server composition root must enforce the beta gate."""
|
|
from headroom.proxy.server import _MULTI_WORKER_CONFIG_ENV, _proxy_config_from_env
|
|
|
|
monkeypatch.delenv(_MULTI_WORKER_CONFIG_ENV, raising=False)
|
|
monkeypatch.setenv("HEADROOM_READ_MATURATION", "1")
|
|
monkeypatch.setenv("HEADROOM_ROLLOUT_CHANNEL", "stable")
|
|
|
|
cfg = _proxy_config_from_env()
|
|
|
|
assert cfg.read_maturation is False
|
|
assert cfg.rollout is not None
|
|
assert cfg.rollout.decision("read_maturation").reason.value == "blocked_by_channel"
|
|
|
|
|
|
def test_replayed_marker_is_not_rebooked_in_the_emitted_savings(monkeypatch):
|
|
"""The savings a request EMITS must be first-appearance.
|
|
|
|
Subtracting the replay debt inside the maturation block is not enough on
|
|
its own: the handler recomputes ``tokens_saved`` from the plain
|
|
original-vs-optimized diff twice more before the outcome is recorded (the
|
|
pre-send hook recount and the consistency recount), and either one throws
|
|
the adjustment away — so every replay turn re-books a removal that was
|
|
already booked when the Read matured. Assert on the emitted header, which
|
|
is the same figure the outcome and PERF line carry, not on the manager.
|
|
"""
|
|
from headroom.cache.compression_store import reset_compression_store
|
|
|
|
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
|
|
reset_compression_store()
|
|
|
|
config = ProxyConfig(
|
|
optimize=True,
|
|
read_maturation=True,
|
|
read_maturation_quiesce_turns=2,
|
|
mode="token",
|
|
cache_enabled=True,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
)
|
|
saved: list[int] = []
|
|
forwarded = _drive_session(
|
|
config, n_turns=5, session_id="first-appearance-1", saved_out=saved, cache_hits=False
|
|
)
|
|
|
|
first = _first_matured_turn(forwarded)
|
|
assert first is not None, "the Read never matured, so there is nothing to replay"
|
|
replays = saved[first + 1 :]
|
|
assert replays, "expected at least one replay turn after maturation"
|
|
|
|
# The turn that matures the Read books its removal, once.
|
|
assert saved[first] > 0, "the maturing turn booked nothing"
|
|
# Later turns forward the same marker instead of the verbatim Read, so the
|
|
# wire diff is just as large — but it is the SAME removal, already booked.
|
|
assert max(replays) < saved[first] * 0.2, (
|
|
f"replayed marker re-booked through the final accounting path: "
|
|
f"matured turn saved {saved[first]}, replay turns saved {replays}"
|
|
)
|