fix(cache): preserve OpenAI tool block metadata on cached replay (#4334)

Current status (2026-10-24 20:50 UTC): exact head
`2333a5ccf7700f86f8767d1c5d02fbfb903c966c` is conflict-free and
[approved by
JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639834650).
All seven held workflows are now successful:
[CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40128152028)
has 17 successful jobs and five skips;
[Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151955)
has 12 successful jobs. Both [Wrap
E2E](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151965/job/120974772335)
and CI's Docker-native job actually execute the new npm hook: two tests
pass, zero fail or skip. Logs check out synthetic merge `91669e2` (this
head into `3fef15d3`). The maintainer independently reports 146 tests
passed with ten optional skips, including the nine new regressions, plus
Ruff lint/formatting. These hosted and maintainer results supplement,
rather than replace or relabel, the author's limited local checks.
Security passes upstream policy, including three excepted bundled npm
advisories until 2026-11-15; this is not a zero-findings claim.
Live-provider billing/cache behavior remains unqualified. The prior
approval-pending statement and local Docker limitation in the [author
response](https://github.com/headroomlabs-ai/headroom/pull/4334#issuecomment-6388924792)
describe the earlier observation, now superseded by these hosted
results.

### Previous-head qualification (historical)

Current status (2026-10-24 19:54 UTC): exact head
`a618561542a8bb48fdb29b07c66c3c6647b7a690` is conflict-free and
[approved by
JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639593849);
the maintainer explicitly cleared the earlier dependency qualification
hold. All seven previously held workflows completed successfully,
including
[CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172110)
(17 successful jobs, five skipped),
[Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172235)
(12 successful jobs), Rust and Wrap E2E. Logs check out synthetic merge
`2ff995d67ef4bb52590c1c0ca95514c85e9693ab` (this head into `3fef15d3`).
Maintainer qualification reports 137 tests passed with ten optional
skips plus Ruff/whitespace; these are their results, not another
author-run suite. Security passes upstream policy, not a zero-findings
claim: E2E npm retains three excepted bundled advisories under
upstream's existing policy until 2026-11-15. No additional exception was
introduced here. Live-provider behavior remains unqualified. Earlier
dated results below are historical.

## Description

Fixes #4332. Replaying a cached compression for an OpenAI `role: tool`
message currently converts list-form content to a string, discarding its
client-supplied `cache_control` and other block metadata. A single text
block now keeps its shape and metadata; only its text is replaced in the
existing deep copy.

For multiple blocks or non-text/malformed block shapes, replay leaves
the message unchanged, before a cache lookup is counted. The cached
joined string does not contain enough information to safely reconstruct
block boundaries or preserve media. String-form OpenAI and Anthropic
behavior are unchanged. This does not duplicate #4333, which explicitly
excludes the token-mode lost-breakpoint issue.

## Type of Change

- [x] Bug fix

## Changes Made

- Preserve the deep-copied single-text OpenAI content block and replace
text only.
- Leave ambiguous lists unchanged before cache hit accounting.
- Share the eligibility guard across cache replay, the rewrite helper
and the real interceptor path before callbacks, disclosure state and
savings accounting.
- Add direct-helper/cache/interceptor regressions for unsupported
shapes, metadata isolation and progressive-disclosure isolation.
- Earlier compatible dependency updates are now incorporated in upstream
main; the final PR delta keeps the cache fix and regressions, with
dependency constraints and locks unchanged from main; the tools manifest
now adds only a test script.

## Testing

- [x] Focused regression tests pass.
- [x] Lint / formatting and changed-module type checks passed at
d74aa5c; runtime/test code is unchanged by a00f387.
- [x] Independent review performed.
- [x] Current 2333a5cc hosted CI/native and Docker regression-hook
checks passed; live-provider qualification is not claimed.
- [x] Current 2333a5cc Security gates pass under upstream policy, with
existing exceptions disclosed above.

### Test Output

```text
# Final lockfile bytes published as a00f387; Linux / Python 3.12.14
PYTHONPATH=. python -m pytest -q tests/test_compression_cache.py tests/test_tool_result_interceptors.py --disable-warnings
115 passed in 31.60s
uv lock --check: passed
cargo metadata --locked: passed
cargo audit (0.22.2): zero reported vulnerabilities
pip-audit of frozen production [all] export: 24 vulnerabilities in Pillow 12.3.0 remain
```

The Rust audit retains the repository's existing RUSTSEC-2024-0436
ignore, unchanged; no new ignores were added. certifi's updated CA
bundle successfully loads 120 CA certificates. The exact remote
Cargo.lock / uv.lock Git blob hashes match the tested files.

At d74aa5c, the five adjacent
cache/registry/interceptor/session/token-mode suites passed 169 tests
with zero skips. Whole-repository Ruff and formatting (1805 files), mypy
for both modified source modules and whitespace checks passed. The
maintainer independently reported 144 Windows/Python 3.12 tests passing
with ten optional skips; that is maintainer evidence, not my local run.

Reproduce focused coverage at d74aa5c (compare added executable lines
against base 4c7acba):

```sh
python -m pytest tests/test_compression_cache.py tests/test_tool_result_interceptors.py --cov=headroom.cache.compression_cache --cov=headroom.proxy.interceptors.base --cov-branch --cov-report=json:coverage.json --cov-report=term-missing -q
```

Independent local coverage at d74aa5c: 115 cache/interceptor tests
passed; all 19 changed executable lines were covered and changed-line
branches had no misses. Whole-module coverage was 92% / 86%, not a
whole-project coverage claim. Codecov's 00:01:55 UTC comment preceded
the relevant shard uploads at 00:02:59 and 00:05:00; final service
aggregation remains unverified.

Historical hosted CI run39657491771 and rust run39657491768 passed for
d74aa5c; Security run39657491814 failed. These are not final a00f387
checks. Maintainers verify hosted checks; no contributor workflow rerun
was requested or initiated.

### Historical dependency blocker (cleared on a618561)

a00f387 updates rustls-webpki to 0.103.16, webpki-roots to 1.0.10 and
certifi to 2026.10.19, only in the lockfiles. Pillow 12.3.0 remains
vulnerable. Pillow 13 requires Python >=3.11; Headroom supports >=3.10.
fastembed 0.8.0 also requires Pillow <13. An actual uv resolver attempt
using conditional Pillow >=13 failed; the attempted manifest edit was
reverted. No overrides, audit suppressions or Python-support removal
were introduced. A compatible fastembed/Pillow release or a separately
agreed dependency/support-policy change is still required.

The initial environment's full-suite collection failed on missing
optional/native dependencies; full native/runtime/live-provider
qualification has not been established locally. Focused checks do not
imply complete qualification.

## Real Behavior Proof

- Environment: Linux, Python 3.12.14, pytest 9.1.1, Ruff 0.17.1.
Synthetic data only, HEADROOM_OFFLINE=1, HEADROOM_BEACON=off,
DO_NOT_TRACK=1; no LLM/provider calls or mocks.

- Exact command / steps: Run the following Python snippet in the project
environment. I called the real public
`CompressionCache.update_from_result` API to seed a cached string
compression, then the real shared `prepare_turn` session engine with a
new list-form tool result. Minimal reproduction:

```python
from headroom.cache.compression_cache import CompressionCache
from headroom.proxy.session_engine import prepare_turn, FREEZE_POLICY_REPLAYABLE

cache = CompressionCache()
raw = "synthetic records " * 50
older = {"role": "tool", "tool_call_id": "c1", "content": raw}
cache.update_from_result([older], [{**older, "content": "<compressed>"}])
newest = {"role": "tool", "tool_call_id": "c2", "content": [
    {"type": "text", "text": raw,
     "cache_control": {"type": "ephemeral", "ttl": "1h"}}
]}
prepared = prepare_turn(cache, [
    {"role": "user", "content": "synthetic request"}, newest
], policy=FREEZE_POLICY_REPLAYABLE)
print(prepared.pipeline_input[-1])
```

- Observed result: Before: content is the string `<compressed>`; the
breakpoint is absent.
After:

```text
{'role': 'tool', 'tool_call_id': 'c2', 'content': [{'type': 'text', 'text': '<compressed>', 'cache_control': {'type': 'ephemeral', 'ttl': '1h'}}]}
original_unchanged: True
frozen_message_count: 1
```

- Not tested: live OpenCode/gateway/Bedrock requests, provider cache
billing or hit rates, author-run Windows/macOS, final-head
native/runtime behavior, full suite. Maintainer Windows results are
separately attributed above. This proof establishes session-preparation
output, not a live provider outcome.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; no new feature/configuration.
Compatible certificate dependency lock versions are refreshed; no
manifest constraints changed.
- Minimum rollout channel: not applicable; this is a cache correctness
fix, not a rollout-managed feature.
- Kill switch / disable path: no new switch; revert this patch to
restore previous behavior.
- Qualification impact: no qualification settings changed; hosted
Rust/native checks passed on the current merge tree; live-provider
qualification remains unverified.
- Stable/default behavior changed: cached replay and direct interception
preserve single-text-block shape/metadata; unsupported layouts pass
through before transformations or savings accounting.
- Unsafe override required: no.
- Rollback path: revert the change.

## Review Readiness

- [x] I have performed a self-review

Independent review has also been performed.
- [x] This PR is ready for human review

Current-head hosted checks and the regression hook passed; maintainer
approval and real behavior proof are recorded above.
- [x] No CHANGELOG edits.
This commit is contained in:
Alicia
2026-10-29 20:27:26 -05:00
committed by GitHub
parent 0ae116d46c
commit f1035ac737
11 changed files with 540 additions and 15 deletions
+4
View File
@@ -93,6 +93,10 @@ RUN npm install -g --no-fund --no-audit npm@11.16.0
COPY e2e/wrap/tools/package.json e2e/wrap/tools/package-lock.json /opt/wrap-tools/
RUN npm ci --prefix /opt/wrap-tools --no-fund --no-audit
# Exercise the installed dependency fixes in both existing wrap-E2E CI callers.
COPY e2e/wrap/tools/dependency-regression.test.cjs /opt/wrap-tools/
RUN npm test --prefix /opt/wrap-tools
WORKDIR /workspace
# Bring in the prebuilt manylinux wheel from stage 1.
+7
View File
@@ -17,11 +17,18 @@ OpenClaw version no longer supports Node 22):
cd e2e/wrap/tools
npm install --package-lock-only --no-fund --no-audit
npm ci --no-fund --no-audit
npm test
cd ../../..
docker build -f e2e/wrap/Dockerfile -t headroom-wrap-e2e .
docker run --rm headroom-wrap-e2e
```
`npm test` runs the Ajv URI and Express body-parser regression checks against
the installed locked dependencies. The Dockerfile runs the same command from
`/opt/wrap-tools` after `npm ci`, so failures stop the image build in both the
existing Wrap E2E workflow and CI's Docker-native wrap E2E job. Their path
filters include `e2e/**`, covering dependency and regression-test updates.
To intentionally refresh transitive versions without changing the pinned CLI
versions, use `npm update --package-lock-only --no-fund --no-audit` in this
directory instead of `npm install --package-lock-only`, then run the same
@@ -0,0 +1,85 @@
'use strict';
// From e2e/wrap/tools after installing its locked dependencies:
// node --test --test-concurrency=1 --test-reporter=tap dependency-regression.test.cjs
const path = require('node:path');
const { createRequire } = require('node:module');
const fromTools = createRequire(path.join(process.cwd(), 'package.json'));
if (process.argv[2] === '--serve') {
const express = fromTools('express');
const app = express();
app.get('/health', (_req, res) => res.json({ ok: true }));
app.post('/text', express.text({ verify() {} }), (req, res) => res.json(req.body));
app.use((error, _req, res, _next) => res.status(error.status || 500).json({ type: error.type }));
const server = app.listen(0, '127.0.0.1', () => process.send({ port: server.address().port }));
process.on('message', (message) => {
if (message === 'stop') { server.close(() => process.exit(0)); server.closeAllConnections(); }
});
} else {
const test = require('node:test');
const assert = require('node:assert/strict');
const http = require('node:http');
const { fork } = require('node:child_process');
test('Ajv rejects a percent-encoded scheme instead of aliasing an HTTPS schema', () => {
const Ajv = fromTools('ajv');
const ajv = new Ajv();
ajv.addSchema({ $id: 'https://example.invalid/defs.json', type: 'integer' });
const valid = ajv.compile({ $ref: 'https://example.invalid/defs.json' });
assert.equal(valid(42), true);
assert.equal(valid('42'), false);
assert.throws(
() => ajv.compile({ $ref: '%68ttps://example.invalid/defs.json' }),
{ message: 'URI scheme is malformed.' },
);
});
test('unsupported charset with invalid gzip leaves the Express process healthy', async (t) => {
const child = fork(__filename, ['--serve'], {
cwd: process.cwd(), env: process.env, execArgv: [], silent: true,
});
let stderr = '';
child.stdout.resume();
child.stderr.on('data', (chunk) => { stderr += chunk; });
t.after(async () => {
t.diagnostic(JSON.stringify({ exitCode: child.exitCode, signal: child.signalCode, stderr }));
if (child.exitCode !== null || child.signalCode !== null) return;
await new Promise((resolve) => {
const timer = setTimeout(() => { child.kill(); resolve(); }, 1000);
child.once('exit', () => { clearTimeout(timer); resolve(); });
if (child.connected) child.send('stop'); else child.kill();
});
});
const port = await new Promise((resolve, reject) => {
const timer = setTimeout(() => reject(new Error('server startup timeout')), 3000);
child.once('message', (msg) => { clearTimeout(timer); resolve(msg.port); });
child.once('error', (err) => { clearTimeout(timer); reject(err); });
child.once('exit', (code) => { clearTimeout(timer); reject(new Error('server exited: ' + code + stderr)); });
});
function request(route, body, headers = {}, method = 'POST') {
return new Promise((resolve, reject) => {
const req = http.request({ hostname: '127.0.0.1', port, path: route, method, agent: false,
headers: { ...headers, 'Content-Length': Buffer.byteLength(body) } }, (res) => {
const chunks = [];
res.on('data', (chunk) => chunks.push(chunk));
res.on('error', reject);
res.on('end', () => resolve({ status: res.statusCode, body: Buffer.concat(chunks).toString() }));
});
req.setTimeout(2000, () => req.destroy(new Error('localhost request timeout')));
req.on('error', reject);
req.end(body);
});
}
const response = await request('/text', 'not-gzip', {
'Content-Type': 'text/plain; charset=not-a-real-charset', 'Content-Encoding': 'gzip',
});
assert.equal(response.status, 415);
assert.equal(JSON.parse(response.body).type, 'charset.unsupported');
// Let a pending zlib error surface before checking the child's health.
await new Promise((resolve) => setTimeout(resolve, 80));
assert.equal(child.exitCode, null, stderr);
const health = await request('/health', '', {}, 'GET');
assert.equal(health.status, 200);
assert.deepEqual(JSON.parse(health.body), { ok: true });
});
}
+3
View File
@@ -3,6 +3,9 @@
"version": "1.0.0",
"private": true,
"description": "Locked CLI dependencies for the Docker wrap E2E harness",
"scripts": {
"test": "node --test --test-concurrency=1 --test-reporter=tap dependency-regression.test.cjs"
},
"dependencies": {
"@openai/codex": "0.163.0",
"openclaw": "2026.10.6"
+44 -6
View File
@@ -76,6 +76,28 @@ def _extract_tool_result_content(msg: dict) -> str | None:
return None
def _can_swap_tool_result_content(msg: dict) -> bool:
"""Whether replacing extracted text preserves the OpenAI content shape.
A joined replacement cannot be mapped back to multiple blocks without
losing boundaries, metadata or media. All rewrite callers must check this
before extraction, transformation or savings accounting. Anthropic's
existing tool-result rewrite contract is unchanged.
"""
if msg.get("role") != "tool":
return True
content = msg.get("content")
if isinstance(content, str):
return True
return (
isinstance(content, list)
and len(content) == 1
and isinstance(content[0], dict)
and content[0].get("type") == "text"
and isinstance(content[0].get("text"), str)
)
def _carries_tool_result_text(msg: dict) -> bool:
"""Whether any tool result in ``msg`` holds text (both formats)."""
if msg.get("role") == "tool":
@@ -93,11 +115,19 @@ def _carries_tool_result_text(msg: dict) -> bool:
def _swap_tool_result_content(msg: dict, new_content: str) -> dict:
"""Deep copy msg and replace tool result content with new_content."""
"""Deep copy and rewrite supported content; preserve unsupported layouts."""
new_msg = copy.deepcopy(msg)
if not _can_swap_tool_result_content(msg):
return new_msg
# OpenAI format
if new_msg.get("role") == "tool":
new_msg["content"] = new_content
content = new_msg.get("content")
if isinstance(content, list):
# The shared guard permits only an unambiguous single text block.
# Keep its shape and metadata, including gateway cache breakpoints.
content[0]["text"] = new_content
else:
new_msg["content"] = new_content
return new_msg
# Anthropic format
content = new_msg.get("content")
@@ -310,10 +340,10 @@ class CompressionCache:
"""Count consecutive stable messages from the start.
A message is stable if it is a plain user/assistant/system message,
an assistant message with tool_use blocks, a tool_result whose
content hash is already in the cache, or a tool_result with no text
(nothing to replay). The first unstable tool_result (cache miss)
stops the count.
an assistant message with tool_use blocks, an Anthropic tool_result with no text
(nothing to replay), or a tool_result whose cached text and shape can
be replayed. A cache miss or unsupported text shape stops the count.
A joined text hash alone does not prove shape-safe replay.
The trailing message is *always* excluded from the frozen prefix
(cap of ``len(messages) - 1``). The trailing message represents
@@ -331,6 +361,11 @@ class CompressionCache:
count = 0
for msg in messages:
if _is_tool_result_message(msg):
# A joined-text cache/stable hash cannot prove replay for
# shapes that apply_cached deliberately leaves untouched.
# Anthropic textless results retain their no-rewrite path.
if not _can_swap_tool_result_content(msg):
break
content = _extract_tool_result_content(msg)
if content is not None:
h = self.content_hash(content)
@@ -364,6 +399,9 @@ class CompressionCache:
with self._lock:
result: list[dict] = []
for msg in messages:
if not _can_swap_tool_result_content(msg):
result.append(msg)
continue
if _is_tool_result_message(msg):
content = _extract_tool_result_content(msg)
if content is not None:
+5
View File
@@ -9,6 +9,7 @@ from dataclasses import dataclass
from typing import Any, Protocol, runtime_checkable
from headroom.cache.compression_cache import (
_can_swap_tool_result_content,
_extract_tool_result_content,
_is_tool_result_message,
_swap_tool_result_content,
@@ -253,6 +254,10 @@ def apply_to_messages(
new_messages.append(msg)
continue
if not _can_swap_tool_result_content(msg):
new_messages.append(msg)
continue
original = _extract_tool_result_content(msg)
if not isinstance(original, str) or not original:
new_messages.append(msg)
+165
View File
@@ -0,0 +1,165 @@
"""Freeze eligibility must match the bytes the cache can actually replay."""
from __future__ import annotations
import copy
import unittest
from headroom.cache.compression_cache import CompressionCache
from headroom.proxy.session_engine import (
FREEZE_POLICY_CONFIRMED_CLAMP,
FREEZE_POLICY_REPLAYABLE,
prepare_turn,
)
class TestCacheReplayShapes(unittest.TestCase):
def setUp(self) -> None:
self.cache = CompressionCache()
self.cache.store_compressed(self.cache.content_hash("A\nB"), "short", 1)
self.blocks = [{"type": "text", "text": "A"}, {"type": "text", "text": "B"}]
self.messages = [
{"role": "tool", "tool_call_id": "c1", "content": self.blocks},
{"role": "user", "content": "next"},
]
def test_multiblock_cache_hit_remains_mutable(self) -> None:
before = copy.deepcopy(self.messages)
prep = prepare_turn(self.cache, self.messages, policy=FREEZE_POLICY_REPLAYABLE)
self.assertEqual(prep.frozen_message_count, 0)
self.assertEqual(prep.pipeline_input, before)
self.assertEqual(self.messages, before)
self.assertEqual(self.cache.get_stats()["hits"], 0)
self.assertNotIn(self.cache.content_hash("A\nB"), self.cache._stable_hashes)
def test_stable_joined_hash_does_not_prove_shape_replay(self) -> None:
self.cache.mark_stable(self.cache.content_hash("A\nB"))
self.assertEqual(self.cache.compute_frozen_count(self.messages), 0)
def test_prefix_stops_before_unsupported_shape(self) -> None:
messages = [{"role": "user", "content": "earlier"}, *self.messages]
self.assertEqual(self.cache.compute_frozen_count(messages), 1)
def test_confirmed_policy_is_still_clamped_by_replayability(self) -> None:
prep = prepare_turn(
self.cache,
self.messages,
policy=FREEZE_POLICY_CONFIRMED_CLAMP,
tracker_frozen=1,
)
self.assertEqual(prep.frozen_message_count, 0)
def test_explicit_caller_pin_still_wins(self) -> None:
prep = prepare_turn(
self.cache, self.messages, policy=FREEZE_POLICY_REPLAYABLE, explicit_frozen=1
)
self.assertEqual(prep.frozen_message_count, 1)
self.assertEqual(prep.pipeline_input, self.messages)
def test_supported_string_and_single_block_replay(self) -> None:
for content in ["A\nB", [{"type": "text", "text": "A\nB", "custom": [1]}]]:
with self.subTest(content=content):
messages = [{"role": "tool", "content": content}, self.messages[-1]]
prep = prepare_turn(self.cache, messages, policy=FREEZE_POLICY_REPLAYABLE)
self.assertEqual(prep.frozen_message_count, 1)
expected = (
"short"
if isinstance(content, str)
else [{"type": "text", "text": "short", "custom": [1]}]
)
self.assertEqual(prep.pipeline_input[0]["content"], expected)
def test_anthropic_replay_contract_is_unchanged(self) -> None:
content = [{"type": "tool_result", "tool_use_id": "c1", "content": "A\nB"}]
prep = prepare_turn(
self.cache,
[{"role": "user", "content": content}, self.messages[-1]],
policy=FREEZE_POLICY_REPLAYABLE,
)
self.assertEqual(prep.frozen_message_count, 1)
self.assertEqual(prep.pipeline_input[0]["content"][0]["content"], "short")
def test_malformed_and_media_shapes_remain_mutable(self) -> None:
for content in [
None,
{},
[],
[{"type": "text", "text": 7}],
[
{"type": "image_url", "image_url": {"url": "synthetic"}},
{"type": "text", "text": "A\nB"},
],
]:
with self.subTest(content=content):
messages = [{"role": "tool", "content": content}, self.messages[-1]]
self.assertEqual(self.cache.compute_frozen_count(messages), 0)
def test_textless_anthropic_prefix_stops_at_unsupported_openai_shape(self) -> None:
textless = {
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t0", "content": []}],
}
for blocks in [
self.blocks,
[
{"type": "image_url", "image_url": {"url": "synthetic"}},
{"type": "text", "text": "A\nB"},
],
]:
with self.subTest(blocks=blocks):
messages = [textless, {"role": "tool", "content": blocks}, self.messages[-1]]
before = copy.deepcopy(messages)
stats_before = self.cache.get_stats()
prep = prepare_turn(self.cache, messages, policy=FREEZE_POLICY_REPLAYABLE)
self.assertEqual(prep.frozen_message_count, 1)
self.assertEqual(prep.pipeline_input, before)
self.assertEqual(messages, before)
self.assertEqual(self.cache.get_stats()["hits"], 0)
self.assertEqual(
self.cache.get_stats()["tokens_saved"], stats_before["tokens_saved"]
)
def test_textless_anthropic_prefix_still_obeys_confirmed_clamp(self) -> None:
textless = {
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t0", "content": []}],
}
messages = [textless, *self.messages]
for confirmed in [0, 1, 2]:
with self.subTest(confirmed=confirmed):
prep = prepare_turn(
self.cache,
messages,
policy=FREEZE_POLICY_CONFIRMED_CLAMP,
tracker_frozen=confirmed,
)
self.assertEqual(prep.frozen_message_count, min(confirmed, 1))
def test_cached_text_after_textless_anthropic_result_remains_mutable(self) -> None:
messages = [
{
"role": "user",
"content": [
{"type": "tool_result", "tool_use_id": "t0", "content": []},
{"type": "tool_result", "tool_use_id": "t1", "content": "A\nB"},
],
},
self.messages[-1],
]
before = copy.deepcopy(messages)
prep = prepare_turn(self.cache, messages, policy=FREEZE_POLICY_REPLAYABLE)
self.assertEqual(prep.frozen_message_count, 0)
self.assertEqual(prep.pipeline_input, before)
self.assertEqual(messages, before)
self.assertEqual(self.cache.get_stats()["hits"], 0)
def test_supported_stable_hash_and_trailing_live_zone_are_unchanged(self) -> None:
cache = CompressionCache()
cache.mark_stable(cache.content_hash("stable"))
messages = [{"role": "tool", "content": "stable"}, self.messages[-1]]
self.assertEqual(cache.compute_frozen_count(messages), 1)
self.assertEqual(cache.compute_frozen_count(messages[:1]), 0)
if __name__ == "__main__":
unittest.main()
+77
View File
@@ -455,6 +455,53 @@ class TestCompressionCacheFrozenCount:
class TestCompressionCacheApplyAndUpdate:
@pytest.mark.parametrize(
"metadata",
[{}, {"cache_control": {"type": "ephemeral", "ttl": "1h"}}, {"custom": {"nested": [1, 2]}}],
)
def test_openai_cached_text_block_preserves_shape_and_metadata(self, metadata: dict) -> None:
import copy
cache = CompressionCache()
original = "synthetic records " * 50
older = {"role": "tool", "tool_call_id": "old", "content": original}
cache.update_from_result([older], [{**older, "content": "short output"}])
message = {
"role": "tool",
"tool_call_id": "new",
"content": [{"type": "text", "text": original, **metadata}],
}
snapshot = copy.deepcopy(message)
result = cache.apply_cached([message])[0]
assert result == {**message, "content": [{**message["content"][0], "text": "short output"}]}
assert message == snapshot
assert result is not message
assert result["content"][0] is not message["content"][0]
@pytest.mark.parametrize(
"extra",
[
{"type": "text", "text": "second", "cache_control": {"type": "ephemeral"}},
{"type": "image_url", "image_url": {"url": "https://example.com/synthetic.png"}},
{"type": "unknown", "value": "keep me"},
],
)
def test_openai_cached_multiblock_content_is_not_collapsed(self, extra: dict) -> None:
import copy
cache = CompressionCache()
blocks = [{"type": "text", "text": "first"}, extra]
text = "first\nsecond" if extra["type"] == "text" else "first"
cache.store_compressed(cache.content_hash(text), "short", tokens_saved=1)
message = {"role": "tool", "tool_call_id": "new", "content": blocks}
snapshot = copy.deepcopy(message)
assert cache.apply_cached([message]) == [snapshot]
assert message == snapshot
assert cache.get_stats()["hits"] == 0
def test_apply_cached_swaps_tool_results(self, cache: CompressionCache) -> None:
original_content = "big tool output"
h = CompressionCache.content_hash(original_content)
@@ -851,3 +898,33 @@ def test_get_compression_cache_returns_same_instance_under_contention() -> None:
first = results[0]
for c in results[1:]:
assert c is first, "Concurrent _get_compression_cache returned different instances"
@pytest.mark.parametrize(
"content",
[
[{"type": "text", "text": "first"}, {"type": "text", "text": "second"}],
[
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AA=="}},
{"type": "text", "text": "second"},
],
[],
["not a block"],
[{"type": "text"}],
[{"type": "text", "text": 42}],
],
)
def test_shared_swap_and_cache_preserve_unsupported_openai_blocks(content):
from copy import deepcopy
from headroom.cache.compression_cache import _swap_tool_result_content
message = {"role": "tool", "content": content}
original = deepcopy(message)
assert _swap_tool_result_content(message, "short") == original
cache = CompressionCache()
cache.store_compressed(cache.content_hash("first\nsecond"), "short", tokens_saved=10)
before = cache.get_stats().copy()
assert cache.apply_cached([message]) == [original]
assert cache.get_stats() == before
assert message == original
+48 -8
View File
@@ -367,9 +367,8 @@ DEFERRED_TOOL = {
def _tool_search_convo(nquiet: int) -> list[dict]:
"""``_busy_convo`` after a ToolSearch whose result holds only tool_reference
blocks, as Claude Code loads deferred tools. The compression cache reads no
text from that result, so its count never passes it, however much of the
later tool output compresses."""
blocks, as Claude Code loads deferred tools. Such textless results are
stable; tests must not rely on them stopping the local replay window."""
msgs = _busy_convo(nquiet)
search = [
{
@@ -434,23 +433,46 @@ def _record_pipeline_freeze(monkeypatch) -> list[int]:
@pytest.mark.parametrize(
("convo", "tools", "pipeline_stop"),
[
# Nothing compresses, so the compression cache never stores the Read.
# Both input shapes exercise the same explicitly bounded local cache.
(_convo, [READ_TOOL], 2),
# Tool output compresses, but the count stops at the ToolSearch result.
(_tool_search_convo, [READ_TOOL, DEFERRED_TOOL], 2),
],
ids=["quiet", "tool-reference"],
)
def test_read_inside_the_cached_prefix_stays_verbatim(monkeypatch, convo, tools, pipeline_stop):
@pytest.mark.parametrize("cache_hits", [True, False], ids=["provider-cached", "provider-uncached"])
def test_read_inside_the_cached_prefix_stays_verbatim(
monkeypatch, convo, tools, pipeline_stop, cache_hits
):
"""By the time the Read's file quiesces the provider has cached it, so
maturing it would re-write the cache from the Read onward. The pipeline's
own freeze stops below the Read here; maturation must follow the
provider-confirmed floor that finalize_turn replays instead."""
provider-confirmed floor that finalize_turn replays instead. The uncached
control must still mature, so retaining the Read cannot pass vacuously."""
from headroom.cache.compression_cache import CompressionCache
from headroom.cache.compression_store import reset_compression_store
from headroom.transforms.read_maturation import ReadMaturationManager
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
reset_compression_store()
# Explicitly model a local replay window shorter than the provider's
# confirmed prefix (e.g. after local cache eviction). Keep real preparation
# and tracking; do not depend on any particular tool-result shape being a
# cache miss. Textless ToolSearch results are legitimately stable now.
compute_frozen = CompressionCache.compute_frozen_count
def _bounded_local_replay(self, messages):
return min(compute_frozen(self, messages), pipeline_stop)
monkeypatch.setattr(CompressionCache, "compute_frozen_count", _bounded_local_replay)
pipeline_freeze = _record_pipeline_freeze(monkeypatch)
maturation_freeze: list[int] = []
apply = ReadMaturationManager.apply
def _recording_apply(self, messages, frozen_message_count=0):
maturation_freeze.append(frozen_message_count)
return apply(self, messages, frozen_message_count=frozen_message_count)
monkeypatch.setattr(ReadMaturationManager, "apply", _recording_apply)
config = ProxyConfig(
optimize=True,
read_maturation=True,
@@ -462,12 +484,30 @@ def test_read_inside_the_cached_prefix_stays_verbatim(monkeypatch, convo, tools,
)
forwarded = _drive_session(
config, n_turns=8, session_id="cached-read-2", convo=convo, tools=tools
config,
n_turns=8,
session_id="cached-read-2",
convo=convo,
tools=tools,
cache_hits=cache_hits,
)
assert pipeline_freeze and max(pipeline_freeze) <= pipeline_stop, (
f"fixture must keep the pipeline freeze below the Read: {pipeline_freeze}"
)
assert len(forwarded) == 8, "every turn must reach the upstream"
read_at = next(i for i, msg in enumerate(convo(0)) if _read_result_content(msg) is not None)
assert pipeline_stop <= read_at, "the local replay window must exclude the held Read"
if not cache_hits:
assert maturation_freeze and not any(maturation_freeze)
assert _first_matured_turn(forwarded) is not None, "uncached control never matured"
return
assert len(maturation_freeze) == 8 and all(f > read_at for f in maturation_freeze[1:]), (
f"provider-confirmed maturation floor must protect the Read: {maturation_freeze}"
)
assert all(_read_result_content(msgs[read_at]) == BIG for msgs in forwarded), (
"the held Read must remain present and verbatim on every turn"
)
assert _first_matured_turn(forwarded) is None, "a cached Read was matured"
assert not _prefix_rewrites(forwarded), "the forwarded cached prefix changed"
+5 -1
View File
@@ -528,8 +528,12 @@ class TestSwapToolResultListContent:
from headroom.cache.compression_cache import _swap_tool_result_content
msg = _make_openai_tool_list_content_msg("tc1", ["original"])
msg["content"][0]["cache_control"] = {"type": "ephemeral"}
swapped = _swap_tool_result_content(msg, "compressed")
assert swapped["content"] == "compressed"
assert swapped["content"] == [
{"type": "text", "text": "compressed", "cache_control": {"type": "ephemeral"}}
]
assert msg["content"][0]["text"] == "original"
def test_swap_does_not_mutate_original(self):
"""_swap_tool_result_content performs a deep copy."""
+97
View File
@@ -1307,3 +1307,100 @@ def test_proxy_pipeline_excludes_interceptor_when_env_not_set(monkeypatch):
for pipeline in (proxy.anthropic_pipeline, proxy.openai_pipeline):
transforms = pipeline.transforms
assert not any(isinstance(t, ToolResultInterceptorTransform) for t in transforms)
@pytest.mark.parametrize(
"content",
[
[
{"type": "text", "text": "first long block " * 20},
{"type": "text", "text": "second long block " * 20},
],
[
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AA=="}},
{"type": "text", "text": "long text " * 40},
],
[],
["not a block"],
[{"type": "text"}],
[{"type": "text", "text": 42}],
],
)
def test_unsupported_openai_blocks_skip_interceptors_and_savings(content, tokenizer, monkeypatch):
from copy import deepcopy
class Shortener:
name = "shape-regression"
calls = 0
def progressive_disclosure_key(self, *_):
self.calls += 1
return "same-key"
def matches(self, *_):
self.calls += 1
return True
def transform(self, *_):
self.calls += 1
return "short"
interceptor = Shortener()
monkeypatch.setattr("headroom.proxy.interceptors.base.INTERCEPTORS", [interceptor])
unsupported = {"role": "tool", "tool_call_id": "1", "content": content}
snapshot = deepcopy(unsupported)
result = apply_to_messages([unsupported], tokenizer)
assert result.messages == [snapshot]
assert result.messages[0] is unsupported
assert result.spans == []
assert interceptor.calls == 0
assert unsupported == snapshot
pipeline_result = ToolResultInterceptorTransform().apply([unsupported], tokenizer)
assert pipeline_result.messages == [snapshot]
assert pipeline_result.transforms_applied == []
assert pipeline_result.tokens_before == pipeline_result.tokens_after
assert interceptor.calls == 0
# A skipped layout must not consume progressive-disclosure state.
valid = {"role": "tool", "tool_call_id": "2", "content": "long output " * 40}
result = apply_to_messages([unsupported, valid], tokenizer)
assert result.messages[0] == snapshot
assert result.messages[1]["content"] == "short"
assert len(result.spans) == 1
assert interceptor.calls == 3
def test_single_openai_text_block_interceptor_preserves_metadata(tokenizer, monkeypatch):
from copy import deepcopy
class Shortener:
name = "metadata-regression"
def matches(self, *_):
return True
def transform(self, *_):
return "short"
monkeypatch.setattr("headroom.proxy.interceptors.base.INTERCEPTORS", [Shortener()])
message = {
"role": "tool",
"tool_call_id": "1",
"content": [
{
"type": "text",
"text": "long output " * 40,
"cache_control": {"type": "ephemeral", "ttl": "1h"},
"metadata": {"nested": [1, 2]},
}
],
}
original = deepcopy(message)
result = apply_to_messages([message], tokenizer)
expected = deepcopy(message)
expected["content"][0]["text"] = "short"
assert result.messages == [expected]
assert len(result.spans) == 1
assert result.spans[0].tokens_saved > 0
result.messages[0]["content"][0]["metadata"]["nested"].append(3)
assert message == original