mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(cache): preserve OpenAI tool block metadata on cached replay (#4334)
Current status (2026-10-24 20:50 UTC): exact head `2333a5ccf7700f86f8767d1c5d02fbfb903c966c` is conflict-free and [approved by JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639834650). All seven held workflows are now successful: [CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40128152028) has 17 successful jobs and five skips; [Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151955) has 12 successful jobs. Both [Wrap E2E](https://github.com/headroomlabs-ai/headroom/actions/runs/40128151965/job/120974772335) and CI's Docker-native job actually execute the new npm hook: two tests pass, zero fail or skip. Logs check out synthetic merge `91669e2` (this head into `3fef15d3`). The maintainer independently reports 146 tests passed with ten optional skips, including the nine new regressions, plus Ruff lint/formatting. These hosted and maintainer results supplement, rather than replace or relabel, the author's limited local checks. Security passes upstream policy, including three excepted bundled npm advisories until 2026-11-15; this is not a zero-findings claim. Live-provider billing/cache behavior remains unqualified. The prior approval-pending statement and local Docker limitation in the [author response](https://github.com/headroomlabs-ai/headroom/pull/4334#issuecomment-6388924792) describe the earlier observation, now superseded by these hosted results. ### Previous-head qualification (historical) Current status (2026-10-24 19:54 UTC): exact head `a618561542a8bb48fdb29b07c66c3c6647b7a690` is conflict-free and [approved by JerrettDavis](https://github.com/headroomlabs-ai/headroom/pull/4334#pullrequestreview-5639593849); the maintainer explicitly cleared the earlier dependency qualification hold. All seven previously held workflows completed successfully, including [CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172110) (17 successful jobs, five skipped), [Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40116172235) (12 successful jobs), Rust and Wrap E2E. Logs check out synthetic merge `2ff995d67ef4bb52590c1c0ca95514c85e9693ab` (this head into `3fef15d3`). Maintainer qualification reports 137 tests passed with ten optional skips plus Ruff/whitespace; these are their results, not another author-run suite. Security passes upstream policy, not a zero-findings claim: E2E npm retains three excepted bundled advisories under upstream's existing policy until 2026-11-15. No additional exception was introduced here. Live-provider behavior remains unqualified. Earlier dated results below are historical. ## Description Fixes #4332. Replaying a cached compression for an OpenAI `role: tool` message currently converts list-form content to a string, discarding its client-supplied `cache_control` and other block metadata. A single text block now keeps its shape and metadata; only its text is replaced in the existing deep copy. For multiple blocks or non-text/malformed block shapes, replay leaves the message unchanged, before a cache lookup is counted. The cached joined string does not contain enough information to safely reconstruct block boundaries or preserve media. String-form OpenAI and Anthropic behavior are unchanged. This does not duplicate #4333, which explicitly excludes the token-mode lost-breakpoint issue. ## Type of Change - [x] Bug fix ## Changes Made - Preserve the deep-copied single-text OpenAI content block and replace text only. - Leave ambiguous lists unchanged before cache hit accounting. - Share the eligibility guard across cache replay, the rewrite helper and the real interceptor path before callbacks, disclosure state and savings accounting. - Add direct-helper/cache/interceptor regressions for unsupported shapes, metadata isolation and progressive-disclosure isolation. - Earlier compatible dependency updates are now incorporated in upstream main; the final PR delta keeps the cache fix and regressions, with dependency constraints and locks unchanged from main; the tools manifest now adds only a test script. ## Testing - [x] Focused regression tests pass. - [x] Lint / formatting and changed-module type checks passed at d74aa5c; runtime/test code is unchanged bya00f387. - [x] Independent review performed. - [x] Current2333a5cchosted CI/native and Docker regression-hook checks passed; live-provider qualification is not claimed. - [x] Current2333a5ccSecurity gates pass under upstream policy, with existing exceptions disclosed above. ### Test Output ```text # Final lockfile bytes published as a00f387; Linux / Python 3.12.14 PYTHONPATH=. python -m pytest -q tests/test_compression_cache.py tests/test_tool_result_interceptors.py --disable-warnings 115 passed in 31.60s uv lock --check: passed cargo metadata --locked: passed cargo audit (0.22.2): zero reported vulnerabilities pip-audit of frozen production [all] export: 24 vulnerabilities in Pillow 12.3.0 remain ``` The Rust audit retains the repository's existing RUSTSEC-2024-0436 ignore, unchanged; no new ignores were added. certifi's updated CA bundle successfully loads 120 CA certificates. The exact remote Cargo.lock / uv.lock Git blob hashes match the tested files. Atd74aa5c, the five adjacent cache/registry/interceptor/session/token-mode suites passed 169 tests with zero skips. Whole-repository Ruff and formatting (1805 files), mypy for both modified source modules and whitespace checks passed. The maintainer independently reported 144 Windows/Python 3.12 tests passing with ten optional skips; that is maintainer evidence, not my local run. Reproduce focused coverage atd74aa5c(compare added executable lines against base4c7acba): ```sh python -m pytest tests/test_compression_cache.py tests/test_tool_result_interceptors.py --cov=headroom.cache.compression_cache --cov=headroom.proxy.interceptors.base --cov-branch --cov-report=json:coverage.json --cov-report=term-missing -q ``` Independent local coverage atd74aa5c: 115 cache/interceptor tests passed; all 19 changed executable lines were covered and changed-line branches had no misses. Whole-module coverage was 92% / 86%, not a whole-project coverage claim. Codecov's 00:01:55 UTC comment preceded the relevant shard uploads at 00:02:59 and 00:05:00; final service aggregation remains unverified. Historical hosted CI run39657491771 and rust run39657491768 passed for d74aa5c; Security run39657491814 failed. These are not finala00f387checks. Maintainers verify hosted checks; no contributor workflow rerun was requested or initiated. ### Historical dependency blocker (cleared ona618561)a00f387updates rustls-webpki to 0.103.16, webpki-roots to 1.0.10 and certifi to 2026.10.19, only in the lockfiles. Pillow 12.3.0 remains vulnerable. Pillow 13 requires Python >=3.11; Headroom supports >=3.10. fastembed 0.8.0 also requires Pillow <13. An actual uv resolver attempt using conditional Pillow >=13 failed; the attempted manifest edit was reverted. No overrides, audit suppressions or Python-support removal were introduced. A compatible fastembed/Pillow release or a separately agreed dependency/support-policy change is still required. The initial environment's full-suite collection failed on missing optional/native dependencies; full native/runtime/live-provider qualification has not been established locally. Focused checks do not imply complete qualification. ## Real Behavior Proof - Environment: Linux, Python 3.12.14, pytest 9.1.1, Ruff 0.17.1. Synthetic data only, HEADROOM_OFFLINE=1, HEADROOM_BEACON=off, DO_NOT_TRACK=1; no LLM/provider calls or mocks. - Exact command / steps: Run the following Python snippet in the project environment. I called the real public `CompressionCache.update_from_result` API to seed a cached string compression, then the real shared `prepare_turn` session engine with a new list-form tool result. Minimal reproduction: ```python from headroom.cache.compression_cache import CompressionCache from headroom.proxy.session_engine import prepare_turn, FREEZE_POLICY_REPLAYABLE cache = CompressionCache() raw = "synthetic records " * 50 older = {"role": "tool", "tool_call_id": "c1", "content": raw} cache.update_from_result([older], [{**older, "content": "<compressed>"}]) newest = {"role": "tool", "tool_call_id": "c2", "content": [ {"type": "text", "text": raw, "cache_control": {"type": "ephemeral", "ttl": "1h"}} ]} prepared = prepare_turn(cache, [ {"role": "user", "content": "synthetic request"}, newest ], policy=FREEZE_POLICY_REPLAYABLE) print(prepared.pipeline_input[-1]) ``` - Observed result: Before: content is the string `<compressed>`; the breakpoint is absent. After: ```text {'role': 'tool', 'tool_call_id': 'c2', 'content': [{'type': 'text', 'text': '<compressed>', 'cache_control': {'type': 'ephemeral', 'ttl': '1h'}}]} original_unchanged: True frozen_message_count: 1 ``` - Not tested: live OpenCode/gateway/Bedrock requests, provider cache billing or hit rates, author-run Windows/macOS, final-head native/runtime behavior, full suite. Maintainer Windows results are separately attributed above. This proof establishes session-preparation output, not a live provider outcome. ## Runtime Rollout Safety - Rollout-managed feature(s): none; no new feature/configuration. Compatible certificate dependency lock versions are refreshed; no manifest constraints changed. - Minimum rollout channel: not applicable; this is a cache correctness fix, not a rollout-managed feature. - Kill switch / disable path: no new switch; revert this patch to restore previous behavior. - Qualification impact: no qualification settings changed; hosted Rust/native checks passed on the current merge tree; live-provider qualification remains unverified. - Stable/default behavior changed: cached replay and direct interception preserve single-text-block shape/metadata; unsupported layouts pass through before transformations or savings accounting. - Unsafe override required: no. - Rollback path: revert the change. ## Review Readiness - [x] I have performed a self-review Independent review has also been performed. - [x] This PR is ready for human review Current-head hosted checks and the regression hook passed; maintainer approval and real behavior proof are recorded above. - [x] No CHANGELOG edits.
This commit is contained in:
@@ -93,6 +93,10 @@ RUN npm install -g --no-fund --no-audit npm@11.16.0
|
||||
COPY e2e/wrap/tools/package.json e2e/wrap/tools/package-lock.json /opt/wrap-tools/
|
||||
RUN npm ci --prefix /opt/wrap-tools --no-fund --no-audit
|
||||
|
||||
# Exercise the installed dependency fixes in both existing wrap-E2E CI callers.
|
||||
COPY e2e/wrap/tools/dependency-regression.test.cjs /opt/wrap-tools/
|
||||
RUN npm test --prefix /opt/wrap-tools
|
||||
|
||||
WORKDIR /workspace
|
||||
|
||||
# Bring in the prebuilt manylinux wheel from stage 1.
|
||||
|
||||
@@ -17,11 +17,18 @@ OpenClaw version no longer supports Node 22):
|
||||
cd e2e/wrap/tools
|
||||
npm install --package-lock-only --no-fund --no-audit
|
||||
npm ci --no-fund --no-audit
|
||||
npm test
|
||||
cd ../../..
|
||||
docker build -f e2e/wrap/Dockerfile -t headroom-wrap-e2e .
|
||||
docker run --rm headroom-wrap-e2e
|
||||
```
|
||||
|
||||
`npm test` runs the Ajv URI and Express body-parser regression checks against
|
||||
the installed locked dependencies. The Dockerfile runs the same command from
|
||||
`/opt/wrap-tools` after `npm ci`, so failures stop the image build in both the
|
||||
existing Wrap E2E workflow and CI's Docker-native wrap E2E job. Their path
|
||||
filters include `e2e/**`, covering dependency and regression-test updates.
|
||||
|
||||
To intentionally refresh transitive versions without changing the pinned CLI
|
||||
versions, use `npm update --package-lock-only --no-fund --no-audit` in this
|
||||
directory instead of `npm install --package-lock-only`, then run the same
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
'use strict';
|
||||
// From e2e/wrap/tools after installing its locked dependencies:
|
||||
// node --test --test-concurrency=1 --test-reporter=tap dependency-regression.test.cjs
|
||||
const path = require('node:path');
|
||||
const { createRequire } = require('node:module');
|
||||
const fromTools = createRequire(path.join(process.cwd(), 'package.json'));
|
||||
|
||||
if (process.argv[2] === '--serve') {
|
||||
const express = fromTools('express');
|
||||
const app = express();
|
||||
app.get('/health', (_req, res) => res.json({ ok: true }));
|
||||
app.post('/text', express.text({ verify() {} }), (req, res) => res.json(req.body));
|
||||
app.use((error, _req, res, _next) => res.status(error.status || 500).json({ type: error.type }));
|
||||
const server = app.listen(0, '127.0.0.1', () => process.send({ port: server.address().port }));
|
||||
process.on('message', (message) => {
|
||||
if (message === 'stop') { server.close(() => process.exit(0)); server.closeAllConnections(); }
|
||||
});
|
||||
} else {
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const http = require('node:http');
|
||||
const { fork } = require('node:child_process');
|
||||
|
||||
test('Ajv rejects a percent-encoded scheme instead of aliasing an HTTPS schema', () => {
|
||||
const Ajv = fromTools('ajv');
|
||||
const ajv = new Ajv();
|
||||
ajv.addSchema({ $id: 'https://example.invalid/defs.json', type: 'integer' });
|
||||
const valid = ajv.compile({ $ref: 'https://example.invalid/defs.json' });
|
||||
assert.equal(valid(42), true);
|
||||
assert.equal(valid('42'), false);
|
||||
assert.throws(
|
||||
() => ajv.compile({ $ref: '%68ttps://example.invalid/defs.json' }),
|
||||
{ message: 'URI scheme is malformed.' },
|
||||
);
|
||||
});
|
||||
|
||||
test('unsupported charset with invalid gzip leaves the Express process healthy', async (t) => {
|
||||
const child = fork(__filename, ['--serve'], {
|
||||
cwd: process.cwd(), env: process.env, execArgv: [], silent: true,
|
||||
});
|
||||
let stderr = '';
|
||||
child.stdout.resume();
|
||||
child.stderr.on('data', (chunk) => { stderr += chunk; });
|
||||
t.after(async () => {
|
||||
t.diagnostic(JSON.stringify({ exitCode: child.exitCode, signal: child.signalCode, stderr }));
|
||||
if (child.exitCode !== null || child.signalCode !== null) return;
|
||||
await new Promise((resolve) => {
|
||||
const timer = setTimeout(() => { child.kill(); resolve(); }, 1000);
|
||||
child.once('exit', () => { clearTimeout(timer); resolve(); });
|
||||
if (child.connected) child.send('stop'); else child.kill();
|
||||
});
|
||||
});
|
||||
const port = await new Promise((resolve, reject) => {
|
||||
const timer = setTimeout(() => reject(new Error('server startup timeout')), 3000);
|
||||
child.once('message', (msg) => { clearTimeout(timer); resolve(msg.port); });
|
||||
child.once('error', (err) => { clearTimeout(timer); reject(err); });
|
||||
child.once('exit', (code) => { clearTimeout(timer); reject(new Error('server exited: ' + code + stderr)); });
|
||||
});
|
||||
function request(route, body, headers = {}, method = 'POST') {
|
||||
return new Promise((resolve, reject) => {
|
||||
const req = http.request({ hostname: '127.0.0.1', port, path: route, method, agent: false,
|
||||
headers: { ...headers, 'Content-Length': Buffer.byteLength(body) } }, (res) => {
|
||||
const chunks = [];
|
||||
res.on('data', (chunk) => chunks.push(chunk));
|
||||
res.on('error', reject);
|
||||
res.on('end', () => resolve({ status: res.statusCode, body: Buffer.concat(chunks).toString() }));
|
||||
});
|
||||
req.setTimeout(2000, () => req.destroy(new Error('localhost request timeout')));
|
||||
req.on('error', reject);
|
||||
req.end(body);
|
||||
});
|
||||
}
|
||||
const response = await request('/text', 'not-gzip', {
|
||||
'Content-Type': 'text/plain; charset=not-a-real-charset', 'Content-Encoding': 'gzip',
|
||||
});
|
||||
assert.equal(response.status, 415);
|
||||
assert.equal(JSON.parse(response.body).type, 'charset.unsupported');
|
||||
// Let a pending zlib error surface before checking the child's health.
|
||||
await new Promise((resolve) => setTimeout(resolve, 80));
|
||||
assert.equal(child.exitCode, null, stderr);
|
||||
const health = await request('/health', '', {}, 'GET');
|
||||
assert.equal(health.status, 200);
|
||||
assert.deepEqual(JSON.parse(health.body), { ok: true });
|
||||
});
|
||||
}
|
||||
@@ -3,6 +3,9 @@
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"description": "Locked CLI dependencies for the Docker wrap E2E harness",
|
||||
"scripts": {
|
||||
"test": "node --test --test-concurrency=1 --test-reporter=tap dependency-regression.test.cjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@openai/codex": "0.163.0",
|
||||
"openclaw": "2026.10.6"
|
||||
|
||||
Vendored
+44
-6
@@ -76,6 +76,28 @@ def _extract_tool_result_content(msg: dict) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _can_swap_tool_result_content(msg: dict) -> bool:
|
||||
"""Whether replacing extracted text preserves the OpenAI content shape.
|
||||
|
||||
A joined replacement cannot be mapped back to multiple blocks without
|
||||
losing boundaries, metadata or media. All rewrite callers must check this
|
||||
before extraction, transformation or savings accounting. Anthropic's
|
||||
existing tool-result rewrite contract is unchanged.
|
||||
"""
|
||||
if msg.get("role") != "tool":
|
||||
return True
|
||||
content = msg.get("content")
|
||||
if isinstance(content, str):
|
||||
return True
|
||||
return (
|
||||
isinstance(content, list)
|
||||
and len(content) == 1
|
||||
and isinstance(content[0], dict)
|
||||
and content[0].get("type") == "text"
|
||||
and isinstance(content[0].get("text"), str)
|
||||
)
|
||||
|
||||
|
||||
def _carries_tool_result_text(msg: dict) -> bool:
|
||||
"""Whether any tool result in ``msg`` holds text (both formats)."""
|
||||
if msg.get("role") == "tool":
|
||||
@@ -93,11 +115,19 @@ def _carries_tool_result_text(msg: dict) -> bool:
|
||||
|
||||
|
||||
def _swap_tool_result_content(msg: dict, new_content: str) -> dict:
|
||||
"""Deep copy msg and replace tool result content with new_content."""
|
||||
"""Deep copy and rewrite supported content; preserve unsupported layouts."""
|
||||
new_msg = copy.deepcopy(msg)
|
||||
if not _can_swap_tool_result_content(msg):
|
||||
return new_msg
|
||||
# OpenAI format
|
||||
if new_msg.get("role") == "tool":
|
||||
new_msg["content"] = new_content
|
||||
content = new_msg.get("content")
|
||||
if isinstance(content, list):
|
||||
# The shared guard permits only an unambiguous single text block.
|
||||
# Keep its shape and metadata, including gateway cache breakpoints.
|
||||
content[0]["text"] = new_content
|
||||
else:
|
||||
new_msg["content"] = new_content
|
||||
return new_msg
|
||||
# Anthropic format
|
||||
content = new_msg.get("content")
|
||||
@@ -310,10 +340,10 @@ class CompressionCache:
|
||||
"""Count consecutive stable messages from the start.
|
||||
|
||||
A message is stable if it is a plain user/assistant/system message,
|
||||
an assistant message with tool_use blocks, a tool_result whose
|
||||
content hash is already in the cache, or a tool_result with no text
|
||||
(nothing to replay). The first unstable tool_result (cache miss)
|
||||
stops the count.
|
||||
an assistant message with tool_use blocks, an Anthropic tool_result with no text
|
||||
(nothing to replay), or a tool_result whose cached text and shape can
|
||||
be replayed. A cache miss or unsupported text shape stops the count.
|
||||
A joined text hash alone does not prove shape-safe replay.
|
||||
|
||||
The trailing message is *always* excluded from the frozen prefix
|
||||
(cap of ``len(messages) - 1``). The trailing message represents
|
||||
@@ -331,6 +361,11 @@ class CompressionCache:
|
||||
count = 0
|
||||
for msg in messages:
|
||||
if _is_tool_result_message(msg):
|
||||
# A joined-text cache/stable hash cannot prove replay for
|
||||
# shapes that apply_cached deliberately leaves untouched.
|
||||
# Anthropic textless results retain their no-rewrite path.
|
||||
if not _can_swap_tool_result_content(msg):
|
||||
break
|
||||
content = _extract_tool_result_content(msg)
|
||||
if content is not None:
|
||||
h = self.content_hash(content)
|
||||
@@ -364,6 +399,9 @@ class CompressionCache:
|
||||
with self._lock:
|
||||
result: list[dict] = []
|
||||
for msg in messages:
|
||||
if not _can_swap_tool_result_content(msg):
|
||||
result.append(msg)
|
||||
continue
|
||||
if _is_tool_result_message(msg):
|
||||
content = _extract_tool_result_content(msg)
|
||||
if content is not None:
|
||||
|
||||
@@ -9,6 +9,7 @@ from dataclasses import dataclass
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
|
||||
from headroom.cache.compression_cache import (
|
||||
_can_swap_tool_result_content,
|
||||
_extract_tool_result_content,
|
||||
_is_tool_result_message,
|
||||
_swap_tool_result_content,
|
||||
@@ -253,6 +254,10 @@ def apply_to_messages(
|
||||
new_messages.append(msg)
|
||||
continue
|
||||
|
||||
if not _can_swap_tool_result_content(msg):
|
||||
new_messages.append(msg)
|
||||
continue
|
||||
|
||||
original = _extract_tool_result_content(msg)
|
||||
if not isinstance(original, str) or not original:
|
||||
new_messages.append(msg)
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Freeze eligibility must match the bytes the cache can actually replay."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import unittest
|
||||
|
||||
from headroom.cache.compression_cache import CompressionCache
|
||||
from headroom.proxy.session_engine import (
|
||||
FREEZE_POLICY_CONFIRMED_CLAMP,
|
||||
FREEZE_POLICY_REPLAYABLE,
|
||||
prepare_turn,
|
||||
)
|
||||
|
||||
|
||||
class TestCacheReplayShapes(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.cache = CompressionCache()
|
||||
self.cache.store_compressed(self.cache.content_hash("A\nB"), "short", 1)
|
||||
self.blocks = [{"type": "text", "text": "A"}, {"type": "text", "text": "B"}]
|
||||
self.messages = [
|
||||
{"role": "tool", "tool_call_id": "c1", "content": self.blocks},
|
||||
{"role": "user", "content": "next"},
|
||||
]
|
||||
|
||||
def test_multiblock_cache_hit_remains_mutable(self) -> None:
|
||||
before = copy.deepcopy(self.messages)
|
||||
prep = prepare_turn(self.cache, self.messages, policy=FREEZE_POLICY_REPLAYABLE)
|
||||
self.assertEqual(prep.frozen_message_count, 0)
|
||||
self.assertEqual(prep.pipeline_input, before)
|
||||
self.assertEqual(self.messages, before)
|
||||
self.assertEqual(self.cache.get_stats()["hits"], 0)
|
||||
self.assertNotIn(self.cache.content_hash("A\nB"), self.cache._stable_hashes)
|
||||
|
||||
def test_stable_joined_hash_does_not_prove_shape_replay(self) -> None:
|
||||
self.cache.mark_stable(self.cache.content_hash("A\nB"))
|
||||
self.assertEqual(self.cache.compute_frozen_count(self.messages), 0)
|
||||
|
||||
def test_prefix_stops_before_unsupported_shape(self) -> None:
|
||||
messages = [{"role": "user", "content": "earlier"}, *self.messages]
|
||||
self.assertEqual(self.cache.compute_frozen_count(messages), 1)
|
||||
|
||||
def test_confirmed_policy_is_still_clamped_by_replayability(self) -> None:
|
||||
prep = prepare_turn(
|
||||
self.cache,
|
||||
self.messages,
|
||||
policy=FREEZE_POLICY_CONFIRMED_CLAMP,
|
||||
tracker_frozen=1,
|
||||
)
|
||||
self.assertEqual(prep.frozen_message_count, 0)
|
||||
|
||||
def test_explicit_caller_pin_still_wins(self) -> None:
|
||||
prep = prepare_turn(
|
||||
self.cache, self.messages, policy=FREEZE_POLICY_REPLAYABLE, explicit_frozen=1
|
||||
)
|
||||
self.assertEqual(prep.frozen_message_count, 1)
|
||||
self.assertEqual(prep.pipeline_input, self.messages)
|
||||
|
||||
def test_supported_string_and_single_block_replay(self) -> None:
|
||||
for content in ["A\nB", [{"type": "text", "text": "A\nB", "custom": [1]}]]:
|
||||
with self.subTest(content=content):
|
||||
messages = [{"role": "tool", "content": content}, self.messages[-1]]
|
||||
prep = prepare_turn(self.cache, messages, policy=FREEZE_POLICY_REPLAYABLE)
|
||||
self.assertEqual(prep.frozen_message_count, 1)
|
||||
expected = (
|
||||
"short"
|
||||
if isinstance(content, str)
|
||||
else [{"type": "text", "text": "short", "custom": [1]}]
|
||||
)
|
||||
self.assertEqual(prep.pipeline_input[0]["content"], expected)
|
||||
|
||||
def test_anthropic_replay_contract_is_unchanged(self) -> None:
|
||||
content = [{"type": "tool_result", "tool_use_id": "c1", "content": "A\nB"}]
|
||||
prep = prepare_turn(
|
||||
self.cache,
|
||||
[{"role": "user", "content": content}, self.messages[-1]],
|
||||
policy=FREEZE_POLICY_REPLAYABLE,
|
||||
)
|
||||
self.assertEqual(prep.frozen_message_count, 1)
|
||||
self.assertEqual(prep.pipeline_input[0]["content"][0]["content"], "short")
|
||||
|
||||
def test_malformed_and_media_shapes_remain_mutable(self) -> None:
|
||||
for content in [
|
||||
None,
|
||||
{},
|
||||
[],
|
||||
[{"type": "text", "text": 7}],
|
||||
[
|
||||
{"type": "image_url", "image_url": {"url": "synthetic"}},
|
||||
{"type": "text", "text": "A\nB"},
|
||||
],
|
||||
]:
|
||||
with self.subTest(content=content):
|
||||
messages = [{"role": "tool", "content": content}, self.messages[-1]]
|
||||
self.assertEqual(self.cache.compute_frozen_count(messages), 0)
|
||||
|
||||
def test_textless_anthropic_prefix_stops_at_unsupported_openai_shape(self) -> None:
|
||||
textless = {
|
||||
"role": "user",
|
||||
"content": [{"type": "tool_result", "tool_use_id": "t0", "content": []}],
|
||||
}
|
||||
for blocks in [
|
||||
self.blocks,
|
||||
[
|
||||
{"type": "image_url", "image_url": {"url": "synthetic"}},
|
||||
{"type": "text", "text": "A\nB"},
|
||||
],
|
||||
]:
|
||||
with self.subTest(blocks=blocks):
|
||||
messages = [textless, {"role": "tool", "content": blocks}, self.messages[-1]]
|
||||
before = copy.deepcopy(messages)
|
||||
stats_before = self.cache.get_stats()
|
||||
prep = prepare_turn(self.cache, messages, policy=FREEZE_POLICY_REPLAYABLE)
|
||||
self.assertEqual(prep.frozen_message_count, 1)
|
||||
self.assertEqual(prep.pipeline_input, before)
|
||||
self.assertEqual(messages, before)
|
||||
self.assertEqual(self.cache.get_stats()["hits"], 0)
|
||||
self.assertEqual(
|
||||
self.cache.get_stats()["tokens_saved"], stats_before["tokens_saved"]
|
||||
)
|
||||
|
||||
def test_textless_anthropic_prefix_still_obeys_confirmed_clamp(self) -> None:
|
||||
textless = {
|
||||
"role": "user",
|
||||
"content": [{"type": "tool_result", "tool_use_id": "t0", "content": []}],
|
||||
}
|
||||
messages = [textless, *self.messages]
|
||||
for confirmed in [0, 1, 2]:
|
||||
with self.subTest(confirmed=confirmed):
|
||||
prep = prepare_turn(
|
||||
self.cache,
|
||||
messages,
|
||||
policy=FREEZE_POLICY_CONFIRMED_CLAMP,
|
||||
tracker_frozen=confirmed,
|
||||
)
|
||||
self.assertEqual(prep.frozen_message_count, min(confirmed, 1))
|
||||
|
||||
def test_cached_text_after_textless_anthropic_result_remains_mutable(self) -> None:
|
||||
messages = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "tool_result", "tool_use_id": "t0", "content": []},
|
||||
{"type": "tool_result", "tool_use_id": "t1", "content": "A\nB"},
|
||||
],
|
||||
},
|
||||
self.messages[-1],
|
||||
]
|
||||
before = copy.deepcopy(messages)
|
||||
prep = prepare_turn(self.cache, messages, policy=FREEZE_POLICY_REPLAYABLE)
|
||||
self.assertEqual(prep.frozen_message_count, 0)
|
||||
self.assertEqual(prep.pipeline_input, before)
|
||||
self.assertEqual(messages, before)
|
||||
self.assertEqual(self.cache.get_stats()["hits"], 0)
|
||||
|
||||
def test_supported_stable_hash_and_trailing_live_zone_are_unchanged(self) -> None:
|
||||
cache = CompressionCache()
|
||||
cache.mark_stable(cache.content_hash("stable"))
|
||||
messages = [{"role": "tool", "content": "stable"}, self.messages[-1]]
|
||||
self.assertEqual(cache.compute_frozen_count(messages), 1)
|
||||
self.assertEqual(cache.compute_frozen_count(messages[:1]), 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -455,6 +455,53 @@ class TestCompressionCacheFrozenCount:
|
||||
|
||||
|
||||
class TestCompressionCacheApplyAndUpdate:
|
||||
@pytest.mark.parametrize(
|
||||
"metadata",
|
||||
[{}, {"cache_control": {"type": "ephemeral", "ttl": "1h"}}, {"custom": {"nested": [1, 2]}}],
|
||||
)
|
||||
def test_openai_cached_text_block_preserves_shape_and_metadata(self, metadata: dict) -> None:
|
||||
import copy
|
||||
|
||||
cache = CompressionCache()
|
||||
original = "synthetic records " * 50
|
||||
older = {"role": "tool", "tool_call_id": "old", "content": original}
|
||||
cache.update_from_result([older], [{**older, "content": "short output"}])
|
||||
message = {
|
||||
"role": "tool",
|
||||
"tool_call_id": "new",
|
||||
"content": [{"type": "text", "text": original, **metadata}],
|
||||
}
|
||||
snapshot = copy.deepcopy(message)
|
||||
|
||||
result = cache.apply_cached([message])[0]
|
||||
|
||||
assert result == {**message, "content": [{**message["content"][0], "text": "short output"}]}
|
||||
assert message == snapshot
|
||||
assert result is not message
|
||||
assert result["content"][0] is not message["content"][0]
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"extra",
|
||||
[
|
||||
{"type": "text", "text": "second", "cache_control": {"type": "ephemeral"}},
|
||||
{"type": "image_url", "image_url": {"url": "https://example.com/synthetic.png"}},
|
||||
{"type": "unknown", "value": "keep me"},
|
||||
],
|
||||
)
|
||||
def test_openai_cached_multiblock_content_is_not_collapsed(self, extra: dict) -> None:
|
||||
import copy
|
||||
|
||||
cache = CompressionCache()
|
||||
blocks = [{"type": "text", "text": "first"}, extra]
|
||||
text = "first\nsecond" if extra["type"] == "text" else "first"
|
||||
cache.store_compressed(cache.content_hash(text), "short", tokens_saved=1)
|
||||
message = {"role": "tool", "tool_call_id": "new", "content": blocks}
|
||||
snapshot = copy.deepcopy(message)
|
||||
|
||||
assert cache.apply_cached([message]) == [snapshot]
|
||||
assert message == snapshot
|
||||
assert cache.get_stats()["hits"] == 0
|
||||
|
||||
def test_apply_cached_swaps_tool_results(self, cache: CompressionCache) -> None:
|
||||
original_content = "big tool output"
|
||||
h = CompressionCache.content_hash(original_content)
|
||||
@@ -851,3 +898,33 @@ def test_get_compression_cache_returns_same_instance_under_contention() -> None:
|
||||
first = results[0]
|
||||
for c in results[1:]:
|
||||
assert c is first, "Concurrent _get_compression_cache returned different instances"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"content",
|
||||
[
|
||||
[{"type": "text", "text": "first"}, {"type": "text", "text": "second"}],
|
||||
[
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AA=="}},
|
||||
{"type": "text", "text": "second"},
|
||||
],
|
||||
[],
|
||||
["not a block"],
|
||||
[{"type": "text"}],
|
||||
[{"type": "text", "text": 42}],
|
||||
],
|
||||
)
|
||||
def test_shared_swap_and_cache_preserve_unsupported_openai_blocks(content):
|
||||
from copy import deepcopy
|
||||
|
||||
from headroom.cache.compression_cache import _swap_tool_result_content
|
||||
|
||||
message = {"role": "tool", "content": content}
|
||||
original = deepcopy(message)
|
||||
assert _swap_tool_result_content(message, "short") == original
|
||||
cache = CompressionCache()
|
||||
cache.store_compressed(cache.content_hash("first\nsecond"), "short", tokens_saved=10)
|
||||
before = cache.get_stats().copy()
|
||||
assert cache.apply_cached([message]) == [original]
|
||||
assert cache.get_stats() == before
|
||||
assert message == original
|
||||
|
||||
@@ -367,9 +367,8 @@ DEFERRED_TOOL = {
|
||||
|
||||
def _tool_search_convo(nquiet: int) -> list[dict]:
|
||||
"""``_busy_convo`` after a ToolSearch whose result holds only tool_reference
|
||||
blocks, as Claude Code loads deferred tools. The compression cache reads no
|
||||
text from that result, so its count never passes it, however much of the
|
||||
later tool output compresses."""
|
||||
blocks, as Claude Code loads deferred tools. Such textless results are
|
||||
stable; tests must not rely on them stopping the local replay window."""
|
||||
msgs = _busy_convo(nquiet)
|
||||
search = [
|
||||
{
|
||||
@@ -434,23 +433,46 @@ def _record_pipeline_freeze(monkeypatch) -> list[int]:
|
||||
@pytest.mark.parametrize(
|
||||
("convo", "tools", "pipeline_stop"),
|
||||
[
|
||||
# Nothing compresses, so the compression cache never stores the Read.
|
||||
# Both input shapes exercise the same explicitly bounded local cache.
|
||||
(_convo, [READ_TOOL], 2),
|
||||
# Tool output compresses, but the count stops at the ToolSearch result.
|
||||
(_tool_search_convo, [READ_TOOL, DEFERRED_TOOL], 2),
|
||||
],
|
||||
ids=["quiet", "tool-reference"],
|
||||
)
|
||||
def test_read_inside_the_cached_prefix_stays_verbatim(monkeypatch, convo, tools, pipeline_stop):
|
||||
@pytest.mark.parametrize("cache_hits", [True, False], ids=["provider-cached", "provider-uncached"])
|
||||
def test_read_inside_the_cached_prefix_stays_verbatim(
|
||||
monkeypatch, convo, tools, pipeline_stop, cache_hits
|
||||
):
|
||||
"""By the time the Read's file quiesces the provider has cached it, so
|
||||
maturing it would re-write the cache from the Read onward. The pipeline's
|
||||
own freeze stops below the Read here; maturation must follow the
|
||||
provider-confirmed floor that finalize_turn replays instead."""
|
||||
provider-confirmed floor that finalize_turn replays instead. The uncached
|
||||
control must still mature, so retaining the Read cannot pass vacuously."""
|
||||
from headroom.cache.compression_cache import CompressionCache
|
||||
from headroom.cache.compression_store import reset_compression_store
|
||||
from headroom.transforms.read_maturation import ReadMaturationManager
|
||||
|
||||
monkeypatch.setenv("HEADROOM_CCR_BACKEND", "memory")
|
||||
reset_compression_store()
|
||||
# Explicitly model a local replay window shorter than the provider's
|
||||
# confirmed prefix (e.g. after local cache eviction). Keep real preparation
|
||||
# and tracking; do not depend on any particular tool-result shape being a
|
||||
# cache miss. Textless ToolSearch results are legitimately stable now.
|
||||
compute_frozen = CompressionCache.compute_frozen_count
|
||||
|
||||
def _bounded_local_replay(self, messages):
|
||||
return min(compute_frozen(self, messages), pipeline_stop)
|
||||
|
||||
monkeypatch.setattr(CompressionCache, "compute_frozen_count", _bounded_local_replay)
|
||||
pipeline_freeze = _record_pipeline_freeze(monkeypatch)
|
||||
maturation_freeze: list[int] = []
|
||||
apply = ReadMaturationManager.apply
|
||||
|
||||
def _recording_apply(self, messages, frozen_message_count=0):
|
||||
maturation_freeze.append(frozen_message_count)
|
||||
return apply(self, messages, frozen_message_count=frozen_message_count)
|
||||
|
||||
monkeypatch.setattr(ReadMaturationManager, "apply", _recording_apply)
|
||||
config = ProxyConfig(
|
||||
optimize=True,
|
||||
read_maturation=True,
|
||||
@@ -462,12 +484,30 @@ def test_read_inside_the_cached_prefix_stays_verbatim(monkeypatch, convo, tools,
|
||||
)
|
||||
|
||||
forwarded = _drive_session(
|
||||
config, n_turns=8, session_id="cached-read-2", convo=convo, tools=tools
|
||||
config,
|
||||
n_turns=8,
|
||||
session_id="cached-read-2",
|
||||
convo=convo,
|
||||
tools=tools,
|
||||
cache_hits=cache_hits,
|
||||
)
|
||||
|
||||
assert pipeline_freeze and max(pipeline_freeze) <= pipeline_stop, (
|
||||
f"fixture must keep the pipeline freeze below the Read: {pipeline_freeze}"
|
||||
)
|
||||
assert len(forwarded) == 8, "every turn must reach the upstream"
|
||||
read_at = next(i for i, msg in enumerate(convo(0)) if _read_result_content(msg) is not None)
|
||||
assert pipeline_stop <= read_at, "the local replay window must exclude the held Read"
|
||||
if not cache_hits:
|
||||
assert maturation_freeze and not any(maturation_freeze)
|
||||
assert _first_matured_turn(forwarded) is not None, "uncached control never matured"
|
||||
return
|
||||
assert len(maturation_freeze) == 8 and all(f > read_at for f in maturation_freeze[1:]), (
|
||||
f"provider-confirmed maturation floor must protect the Read: {maturation_freeze}"
|
||||
)
|
||||
assert all(_read_result_content(msgs[read_at]) == BIG for msgs in forwarded), (
|
||||
"the held Read must remain present and verbatim on every turn"
|
||||
)
|
||||
assert _first_matured_turn(forwarded) is None, "a cached Read was matured"
|
||||
assert not _prefix_rewrites(forwarded), "the forwarded cached prefix changed"
|
||||
|
||||
|
||||
@@ -528,8 +528,12 @@ class TestSwapToolResultListContent:
|
||||
from headroom.cache.compression_cache import _swap_tool_result_content
|
||||
|
||||
msg = _make_openai_tool_list_content_msg("tc1", ["original"])
|
||||
msg["content"][0]["cache_control"] = {"type": "ephemeral"}
|
||||
swapped = _swap_tool_result_content(msg, "compressed")
|
||||
assert swapped["content"] == "compressed"
|
||||
assert swapped["content"] == [
|
||||
{"type": "text", "text": "compressed", "cache_control": {"type": "ephemeral"}}
|
||||
]
|
||||
assert msg["content"][0]["text"] == "original"
|
||||
|
||||
def test_swap_does_not_mutate_original(self):
|
||||
"""_swap_tool_result_content performs a deep copy."""
|
||||
|
||||
@@ -1307,3 +1307,100 @@ def test_proxy_pipeline_excludes_interceptor_when_env_not_set(monkeypatch):
|
||||
for pipeline in (proxy.anthropic_pipeline, proxy.openai_pipeline):
|
||||
transforms = pipeline.transforms
|
||||
assert not any(isinstance(t, ToolResultInterceptorTransform) for t in transforms)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"content",
|
||||
[
|
||||
[
|
||||
{"type": "text", "text": "first long block " * 20},
|
||||
{"type": "text", "text": "second long block " * 20},
|
||||
],
|
||||
[
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AA=="}},
|
||||
{"type": "text", "text": "long text " * 40},
|
||||
],
|
||||
[],
|
||||
["not a block"],
|
||||
[{"type": "text"}],
|
||||
[{"type": "text", "text": 42}],
|
||||
],
|
||||
)
|
||||
def test_unsupported_openai_blocks_skip_interceptors_and_savings(content, tokenizer, monkeypatch):
|
||||
from copy import deepcopy
|
||||
|
||||
class Shortener:
|
||||
name = "shape-regression"
|
||||
calls = 0
|
||||
|
||||
def progressive_disclosure_key(self, *_):
|
||||
self.calls += 1
|
||||
return "same-key"
|
||||
|
||||
def matches(self, *_):
|
||||
self.calls += 1
|
||||
return True
|
||||
|
||||
def transform(self, *_):
|
||||
self.calls += 1
|
||||
return "short"
|
||||
|
||||
interceptor = Shortener()
|
||||
monkeypatch.setattr("headroom.proxy.interceptors.base.INTERCEPTORS", [interceptor])
|
||||
unsupported = {"role": "tool", "tool_call_id": "1", "content": content}
|
||||
snapshot = deepcopy(unsupported)
|
||||
result = apply_to_messages([unsupported], tokenizer)
|
||||
assert result.messages == [snapshot]
|
||||
assert result.messages[0] is unsupported
|
||||
assert result.spans == []
|
||||
assert interceptor.calls == 0
|
||||
assert unsupported == snapshot
|
||||
pipeline_result = ToolResultInterceptorTransform().apply([unsupported], tokenizer)
|
||||
assert pipeline_result.messages == [snapshot]
|
||||
assert pipeline_result.transforms_applied == []
|
||||
assert pipeline_result.tokens_before == pipeline_result.tokens_after
|
||||
assert interceptor.calls == 0
|
||||
|
||||
# A skipped layout must not consume progressive-disclosure state.
|
||||
valid = {"role": "tool", "tool_call_id": "2", "content": "long output " * 40}
|
||||
result = apply_to_messages([unsupported, valid], tokenizer)
|
||||
assert result.messages[0] == snapshot
|
||||
assert result.messages[1]["content"] == "short"
|
||||
assert len(result.spans) == 1
|
||||
assert interceptor.calls == 3
|
||||
|
||||
|
||||
def test_single_openai_text_block_interceptor_preserves_metadata(tokenizer, monkeypatch):
|
||||
from copy import deepcopy
|
||||
|
||||
class Shortener:
|
||||
name = "metadata-regression"
|
||||
|
||||
def matches(self, *_):
|
||||
return True
|
||||
|
||||
def transform(self, *_):
|
||||
return "short"
|
||||
|
||||
monkeypatch.setattr("headroom.proxy.interceptors.base.INTERCEPTORS", [Shortener()])
|
||||
message = {
|
||||
"role": "tool",
|
||||
"tool_call_id": "1",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "long output " * 40,
|
||||
"cache_control": {"type": "ephemeral", "ttl": "1h"},
|
||||
"metadata": {"nested": [1, 2]},
|
||||
}
|
||||
],
|
||||
}
|
||||
original = deepcopy(message)
|
||||
result = apply_to_messages([message], tokenizer)
|
||||
expected = deepcopy(message)
|
||||
expected["content"][0]["text"] = "short"
|
||||
assert result.messages == [expected]
|
||||
assert len(result.spans) == 1
|
||||
assert result.spans[0].tokens_saved > 0
|
||||
result.messages[0]["content"][0]["metadata"]["nested"].append(3)
|
||||
assert message == original
|
||||
|
||||
Reference in New Issue
Block a user