mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-23 02:49:17 +08:00
`/v1/compress` returns a body that someone else's client forwards
unchanged, so anything Headroom adds there reaches the provider. Two
things got added that the caller never asked for.
## Stale-Read retrieval hashes in marker-free modes
Read lifecycle replaces a Read that a later edit made stale with a short
notice. That notice always ended in `Retrieve original: hash=…` and
wrote the original to the CCR store, even in the default and
`lossy_inline` modes. The docs call those modes marker-free, and their
callers have no retrieval tool, so the hash pointed nowhere.
- Marker-free modes keep the notice ("re-read the file for current
content") but drop the hash and skip the store write.
- Lossless routers (proxy `--lossless`,
`ContentRouterConfig(lossless=True)`) skip read lifecycle entirely,
because the notice drops the file's bytes.
- Read maturation now recognises the hashless notice. Otherwise it would
replace the notice with a marker whose hash points at the notice itself.
## Cache breakpoints the client didn't send
On the session path, `normalize_message_cache_control` fell back to
placing `{"type": "ephemeral"}` on the last block message when the
client marked nothing. A caller that chose not to cache got billed cache
writes. `/v1/compress` now passes `place_unmarked=False`: if the client
marked no message, markers are stripped and none are placed. Proxy
handlers keep today's behaviour, and client markers in a misaligned list
still go through the old path.
## Tests
Each new test fails on main.
- `test_marker_free_modes_mark_a_stale_read_without_a_retrieval_hash`,
for the default mode and `lossy_inline`
- `test_lossless_router_keeps_a_stale_read`
- `test_session_turns_add_no_marker_the_caller_did_not_send`
- `test_respects_lifecycle_markers`, extended to the hashless notice
The related suites (gateway, transforms, read maturation, prefix
tracker, cache_control, compress endpoint and session) pass: 1371 passed
and 9 skipped before the review fixes, then 222 of 222 on the touched
files after them. Lint is clean on CI's ruff 0.17.0, and mypy is clean
on the touched modules.
## Not in this PR
- Read maturation (`handlers/anthropic.py`) is not gated on `--lossless`
or `--no-ccr` either. It's the same kind of bug on the proxy path, so it
belongs in a separate fix.
- The "lossless densify only" pipeline in `compress.py` sets
`ccr_enabled=False` without `lossless=True`, so read lifecycle still
runs there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4678"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
## Real Behavior Proof
Maintainer qualification on Windows, Python 3.12.13, using an actual
Uvicorn server bound to an owned loopback socket and HTTPX over real
HTTP. The normal `/v1/compress` endpoint and compression pipeline ran
with offline mode and Kompress disabled; no handler, compression stage,
HTTP response or store was mocked. Workspace, configuration, savings and
TOIN paths were isolated. All 243 native inputs and the reused artifact
SHA-256
`5be63a4f31d3e38e7427d28380ecb977616e3f6a3dd6a1c59af5ff6361a05978` were
verified; this was not a fresh native build.
- Exact command / steps: the committed probe is `python -m pytest -q -s
tests/test_compress_runtime_http.py`. The qualifying run used the
maintainer loopback-only launcher with an explicit verified native path
and SHA-256, followed by the four focused
session/API/maturation/lifecycle suites. All 138 tests passed. The probe
starts the actual server, GETs `/health`, and sends nine POSTs to
`/v1/compress`: a Read followed by Edit in default, lossy-inline and
lossless modes, then three session turns without caller cache markers
and three turns with a caller marker. It stops and joins the server in
`finally`.
- Observed result: health and all nine compression requests returned
HTTP 200. Default and lossy-inline emitted stale notices with no
retrieval hash or CCR hashes, saving 3,576 tokens per request. Lossless
preserved the original Read bytes. All three unmarked session turns
returned zero cache markers; all three marked controls retained exactly
one. Session compression saved 7,350 / 15,901 / 25,958 tokens. Server
shutdown joined successfully.
- Before-fix control: the same real-server requests with all six
affected production modules replaced by current main
`200c838c7393d20ae56a0c338f832ed27133c398` failed five assertions:
default and lossy-inline included retrieval hashes, and each unmarked
session turn added a cache marker. Lossless and the three marked
controls passed on main and are not claimed as reproduced regressions.
Candidate source was restored byte-for-byte.
- Not tested: provider forwarding or billing, a live model, deployment
behavior, or the separate proxy read-maturation scope already listed
above. External startup connection attempts were blocked by the
qualifying launcher; this is not a zero-attempt network claim. No author
readiness or self-review declaration is added by this maintainer
evidence.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
399 lines
15 KiB
Python
399 lines
15 KiB
Python
"""Tests for Mechanism B: activity-based hold-back Read maturation.
|
|
|
|
The invariants under test, beyond decision behavior:
|
|
1. The provider-cached (frozen) prefix is never mutated, and a client
|
|
cache_control breakpoint is never moved or dropped.
|
|
2. Replay is deterministic — once matured, the same marker is applied on
|
|
every subsequent request, byte-identical.
|
|
3. Holding is derived from the conversation (file activity), so the
|
|
decision survives state loss; only matured markers are stateful.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from headroom.config import ReadMaturationConfig
|
|
from headroom.transforms.read_maturation import ReadMaturationManager
|
|
|
|
CONTENT = " 1\tdef foo():\n 2\t return 42\n" * 60 # > 2048B
|
|
SMALL = " 1\tok\n"
|
|
|
|
|
|
def anthropic_read(tc_id: str, file_path: str, content: str) -> list[dict]:
|
|
return [
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{"type": "tool_use", "id": tc_id, "name": "Read", "input": {"file_path": file_path}}
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [{"type": "tool_result", "tool_use_id": tc_id, "content": content}],
|
|
},
|
|
]
|
|
|
|
|
|
def anthropic_edit(tc_id: str, file_path: str) -> list[dict]:
|
|
return [
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": tc_id,
|
|
"name": "Edit",
|
|
"input": {"file_path": file_path, "old_string": "a", "new_string": "b"},
|
|
}
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [{"type": "tool_result", "tool_use_id": tc_id, "content": "ok"}],
|
|
},
|
|
]
|
|
|
|
|
|
def openai_read(tc_id: str, file_path: str, content: str) -> list[dict]:
|
|
return [
|
|
{
|
|
"role": "assistant",
|
|
"tool_calls": [
|
|
{
|
|
"id": tc_id,
|
|
"function": {"name": "Read", "arguments": f'{{"file_path": "{file_path}"}}'},
|
|
}
|
|
],
|
|
},
|
|
{"role": "tool", "tool_call_id": tc_id, "content": content},
|
|
]
|
|
|
|
|
|
def quiet(n: int) -> list[dict]:
|
|
"""n assistant turns with no file activity (advances the quiet clock)."""
|
|
return [
|
|
{"role": "assistant", "content": [{"type": "text", "text": f"thinking {i}"}]}
|
|
for i in range(n)
|
|
]
|
|
|
|
|
|
def base_conv() -> list[dict]:
|
|
return [{"role": "user", "content": "look"}, *anthropic_read("r1", "/x/foo.py", CONTENT)]
|
|
|
|
|
|
def manager(**overrides) -> ReadMaturationManager:
|
|
cfg = ReadMaturationConfig(enabled=True, **overrides)
|
|
return ReadMaturationManager(cfg)
|
|
|
|
|
|
def read_content(res, idx=2):
|
|
return res.messages[idx]["content"][0]["content"]
|
|
|
|
|
|
class TestActivityDecision:
|
|
def test_disabled_is_noop(self):
|
|
m = ReadMaturationManager(ReadMaturationConfig(enabled=False))
|
|
res = m.apply(base_conv())
|
|
assert res.messages == base_conv()
|
|
assert res.holding_msg_indices == []
|
|
|
|
def test_fresh_read_holds_verbatim(self):
|
|
res = manager().apply(base_conv())
|
|
assert read_content(res) == CONTENT
|
|
assert res.holding_msg_indices == [2]
|
|
assert res.holding_reads == 1
|
|
assert res.newly_matured == 0
|
|
|
|
def test_holds_while_file_quiet_below_quiesce(self):
|
|
msgs = [*base_conv(), *quiet(4)] # quiet = 4 < 5
|
|
res = manager(quiesce_turns=5).apply(msgs)
|
|
assert res.holding_msg_indices == [2]
|
|
assert read_content(res) == CONTENT
|
|
|
|
def test_matures_after_quiesce(self):
|
|
msgs = [*base_conv(), *quiet(5)] # quiet = 5 >= 5
|
|
res = manager(quiesce_turns=5).apply(msgs)
|
|
assert res.newly_matured == 1
|
|
assert res.holding_msg_indices == []
|
|
marker = read_content(res)
|
|
assert "compressed after use" in marker
|
|
assert "/x/foo.py" in marker
|
|
assert "Retrieve original: hash=" in marker
|
|
assert res.bytes_saved > 0
|
|
|
|
def test_file_activity_resets_quiet_clock(self):
|
|
# read, 4 quiet, edit same file, 4 quiet: quiet=4 < 5 → still held
|
|
msgs = [*base_conv(), *quiet(4), *anthropic_edit("e1", "/x/foo.py"), *quiet(4)]
|
|
res = manager(quiesce_turns=5).apply(msgs)
|
|
assert res.holding_msg_indices == [2]
|
|
# one more quiet turn → file quiesced → matures
|
|
res = manager(quiesce_turns=5).apply([*msgs, *quiet(1)])
|
|
assert res.newly_matured == 1
|
|
|
|
def test_activity_on_other_file_does_not_reset(self):
|
|
msgs = [*base_conv(), *quiet(3), *anthropic_edit("e1", "/x/OTHER.py"), *quiet(1)]
|
|
# foo.py quiet for 5 assistant turns (3 quiet + edit-turn + 1 quiet)
|
|
res = manager(quiesce_turns=5).apply(msgs)
|
|
assert res.newly_matured == 1
|
|
|
|
def test_max_hold_caps_busy_files(self):
|
|
# File touched every turn — never quiesces — but the hold cap fires.
|
|
msgs = base_conv()
|
|
for i in range(6):
|
|
msgs += anthropic_edit(f"e{i}", "/x/foo.py")
|
|
res = manager(quiesce_turns=100, max_hold_turns=6).apply(msgs)
|
|
assert res.newly_matured == 1
|
|
assert res.holding_msg_indices == []
|
|
|
|
def test_replay_is_deterministic_and_stateful(self):
|
|
m = manager(quiesce_turns=5)
|
|
matured_msgs = [*base_conv(), *quiet(5)]
|
|
a = read_content(m.apply(matured_msgs))
|
|
# Replay applies even when the conversation grows and the file is
|
|
# touched again later (matured is final).
|
|
later = [*matured_msgs, *anthropic_edit("e1", "/x/foo.py")]
|
|
b = read_content(m.apply(later))
|
|
c = read_content(m.apply([*later, *quiet(3)]))
|
|
assert a == b == c
|
|
|
|
def test_small_reads_ignored(self):
|
|
msgs = [
|
|
{"role": "user", "content": "look"},
|
|
*anthropic_read("r1", "/x/a.py", SMALL),
|
|
*quiet(10),
|
|
]
|
|
res = manager().apply(msgs)
|
|
assert res.holding_msg_indices == []
|
|
assert read_content(res) == SMALL
|
|
|
|
def test_frozen_prefix_untouched(self):
|
|
msgs = [*base_conv(), *quiet(10)]
|
|
m = manager(quiesce_turns=5)
|
|
res = m.apply(msgs, frozen_message_count=len(msgs))
|
|
assert res.holding_msg_indices == []
|
|
assert read_content(res) == CONTENT
|
|
|
|
def test_held_read_matures_only_outside_frozen_prefix(self):
|
|
"""A held Read is cached with its turn. At quiesce it matures if it
|
|
is still outside the frozen prefix (e.g. a cold prefix); once the
|
|
frozen prefix covers it, it stays verbatim — maturing it would
|
|
re-write everything after it at the cache-write rate."""
|
|
held = base_conv()
|
|
msgs = [*held, *quiet(5)]
|
|
|
|
m = manager(quiesce_turns=5)
|
|
assert m.apply(held).holding_msg_indices == [2]
|
|
res = m.apply(msgs, frozen_message_count=2) # Read in the live zone
|
|
assert res.newly_matured == 1
|
|
assert "compressed after use" in read_content(res)
|
|
|
|
m = manager(quiesce_turns=5)
|
|
assert m.apply(held).holding_msg_indices == [2]
|
|
res = m.apply(msgs, frozen_message_count=3) # frozen prefix covers it
|
|
assert res.newly_matured == 0
|
|
assert res.messages is msgs
|
|
|
|
@pytest.mark.parametrize(
|
|
"notice",
|
|
[
|
|
"[Read content stale: /x/foo.py ... Retrieve original: hash=abc123]",
|
|
"[Read content stale: /x/foo.py was modified after this read]",
|
|
],
|
|
)
|
|
def test_respects_lifecycle_markers(self, notice):
|
|
marker = notice + " " * 2048
|
|
msgs = [
|
|
{"role": "user", "content": "look"},
|
|
*anthropic_read("r1", "/x/foo.py", marker),
|
|
*quiet(10),
|
|
]
|
|
res = manager().apply(msgs)
|
|
assert res.holding_msg_indices == []
|
|
assert res.newly_matured == 0
|
|
|
|
def test_client_breakpoint_on_fresh_read_is_held_and_kept(self):
|
|
"""Claude Code parks its tail breakpoint on the newest block —
|
|
right after a Read, that's the Read result itself. The read must
|
|
still be held (verbatim) and the breakpoint must stay on it, so
|
|
the turn is cached; it also survives the block's maturation."""
|
|
msgs = base_conv()
|
|
msgs[2]["content"][0]["cache_control"] = {"type": "ephemeral"}
|
|
m = manager(quiesce_turns=5)
|
|
res = m.apply(msgs)
|
|
assert res.holding_msg_indices == [2]
|
|
assert res.messages[2]["content"][0] == msgs[2]["content"][0]
|
|
|
|
res = m.apply([*msgs, *quiet(5)])
|
|
assert res.newly_matured == 1
|
|
assert res.messages[2]["content"][0]["cache_control"] == {"type": "ephemeral"}
|
|
|
|
def test_openai_format(self):
|
|
msgs = [{"role": "user", "content": "look"}, *openai_read("r1", "/x/foo.py", CONTENT)]
|
|
m = manager(quiesce_turns=5)
|
|
res = m.apply(msgs)
|
|
assert res.holding_msg_indices == [2]
|
|
res = m.apply([*msgs, *quiet(5)])
|
|
assert "compressed after use" in res.messages[2]["content"]
|
|
|
|
def test_files_mature_independently(self):
|
|
# foo.py quiet for ages; bar.py just read → foo matures, bar holds.
|
|
msgs = [
|
|
*base_conv(),
|
|
*quiet(6),
|
|
*anthropic_read("r2", "/x/bar.py", CONTENT),
|
|
]
|
|
res = manager(quiesce_turns=5).apply(msgs)
|
|
assert res.newly_matured == 1 # foo.py
|
|
assert "compressed after use" in read_content(res, 2)
|
|
assert res.holding_msg_indices == [10] # bar.py result message
|
|
assert read_content(res, 10) == CONTENT
|
|
|
|
def test_decision_survives_state_loss(self):
|
|
# A fresh manager (proxy restart) makes the same holding/matured
|
|
# decision because holding is derived from the conversation.
|
|
msgs = [*base_conv(), *quiet(5)]
|
|
first = read_content(manager(quiesce_turns=5).apply(msgs))
|
|
second = read_content(manager(quiesce_turns=5).apply(msgs))
|
|
assert "compressed after use" in first and "compressed after use" in second
|
|
# Markers differ only if CCR hashing differed — it must not.
|
|
assert first == second
|
|
|
|
|
|
class TestCcrIntegration:
|
|
def test_original_stored_and_retrievable(self):
|
|
from headroom.cache.backends.memory import InMemoryBackend
|
|
from headroom.cache.compression_store import CompressionStore
|
|
|
|
store = CompressionStore(backend=InMemoryBackend())
|
|
m = ReadMaturationManager(
|
|
ReadMaturationConfig(enabled=True, quiesce_turns=5), compression_store=store
|
|
)
|
|
res = m.apply([*base_conv(), *quiet(5)])
|
|
|
|
marker = read_content(res)
|
|
ccr_hash = marker.split("hash=")[1].rstrip("]")
|
|
entry = store.retrieve(ccr_hash)
|
|
assert entry is not None
|
|
assert entry.original_content == CONTENT
|
|
assert entry.compression_strategy == "read_maturation"
|
|
|
|
|
|
class TestProxyWiring:
|
|
def test_proxy_config_flag_default_off(self):
|
|
from headroom.proxy.models import ProxyConfig
|
|
|
|
assert ProxyConfig().read_maturation is False
|
|
assert ProxyConfig(read_maturation=True).read_maturation is True
|
|
|
|
def test_session_state_rides_on_prefix_tracker(self):
|
|
"""The handler's session-state flow: a manager attached to the
|
|
tracker carries matured markers across requests."""
|
|
from headroom.cache.prefix_tracker import PrefixCacheTracker
|
|
from headroom.config import ReadMaturationConfig
|
|
from headroom.transforms.read_maturation import ReadMaturationManager
|
|
|
|
tracker = PrefixCacheTracker("anthropic")
|
|
assert tracker.read_maturation_manager is None
|
|
|
|
# Request 1: fresh read — held.
|
|
tracker.read_maturation_manager = ReadMaturationManager(
|
|
ReadMaturationConfig(enabled=True, quiesce_turns=5)
|
|
)
|
|
msgs = base_conv()
|
|
msgs[2]["content"][0] = {
|
|
**msgs[2]["content"][0],
|
|
}
|
|
res = tracker.read_maturation_manager.apply(msgs)
|
|
assert res.holding_msg_indices == [2]
|
|
|
|
# Request N (file quiet): same manager matures and replays.
|
|
later = [*base_conv(), *quiet(5)]
|
|
res = tracker.read_maturation_manager.apply(later)
|
|
assert res.newly_matured == 1
|
|
replay = tracker.read_maturation_manager.apply(later)
|
|
assert replay.replacements_applied == 1
|
|
assert replay.newly_matured == 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
pytest.main([__file__, "-v"])
|
|
|
|
|
|
class TestFirstAppearanceAccounting:
|
|
"""A matured Read's savings must book once, on the turn it matures.
|
|
|
|
The client re-sends the raw conversation every turn, so a plain
|
|
original-vs-optimized token diff re-counts the same removal on every
|
|
later request. ``replayed_token_debt`` measures that from the
|
|
request's own endpoints — raw client snapshot vs forwarded messages —
|
|
rather than from what ``apply`` replaced, because after maturation
|
|
the marker normally reaches the wire through the cached-prefix replay
|
|
and ``apply`` replaces nothing at all.
|
|
"""
|
|
|
|
def _matured(self):
|
|
"""Returns (manager, raw client messages, forwarded messages)."""
|
|
m = manager(quiesce_turns=5)
|
|
raw = [*base_conv(), *quiet(5)]
|
|
res = m.apply(raw)
|
|
assert res.newly_matured == 1
|
|
return m, raw, res.messages
|
|
|
|
@staticmethod
|
|
def _marker(sent):
|
|
return sent[2]["content"][0]["content"]
|
|
|
|
def test_first_appearance_books_in_full(self):
|
|
m, raw, sent = self._matured()
|
|
assert m.replayed_token_debt(raw, sent, len) == 0
|
|
|
|
def test_later_request_is_charged_even_without_a_replacement(self):
|
|
# The second call passes the same forwarded form WITHOUT running
|
|
# apply again: that is the cached-prefix replay, the path that
|
|
# actually re-books the removal in production.
|
|
m, raw, sent = self._matured()
|
|
m.replayed_token_debt(raw, sent, len)
|
|
expected = len(CONTENT) - len(self._marker(sent))
|
|
assert m.replayed_token_debt(raw, sent, len) == expected
|
|
|
|
def test_marker_echo_is_not_a_replay(self):
|
|
# The client echoes the marker form back: nothing was removed this
|
|
# request, so there is nothing to un-book.
|
|
m, _, sent = self._matured()
|
|
assert m.replayed_token_debt(sent, sent, len) == 0
|
|
|
|
def test_unchanged_wire_form_is_not_a_replay(self):
|
|
# Forwarded verbatim (maturation state lost, hold re-established):
|
|
# no removal on the wire, no debt.
|
|
m, raw, _ = self._matured()
|
|
assert m.replayed_token_debt(raw, raw, len) == 0
|
|
|
|
def test_delta_is_tokenized_once_per_tool_call(self):
|
|
m, raw, sent = self._matured()
|
|
m.replayed_token_debt(raw, sent, len)
|
|
calls = []
|
|
|
|
def count(text: str) -> int:
|
|
calls.append(text)
|
|
return len(text)
|
|
|
|
expected = len(CONTENT) - len(self._marker(sent))
|
|
assert m.replayed_token_debt(raw, sent, count) == expected
|
|
assert len(calls) == 2 # content + marker, once
|
|
assert m.replayed_token_debt(raw, sent, count) == expected
|
|
assert len(calls) == 2 # cached for the session
|
|
|
|
def test_replay_request_nets_zero_savings(self):
|
|
# With a compositional counter the diff on a replay request equals
|
|
# exactly the debt, so first-appearance accounting books nothing new.
|
|
m, raw, sent = self._matured()
|
|
m.replayed_token_debt(raw, sent, len)
|
|
diff = len(CONTENT) - len(self._marker(sent))
|
|
assert max(0, diff - m.replayed_token_debt(raw, sent, len)) == 0
|
|
|
|
def test_nothing_matured_has_zero_debt(self):
|
|
m = manager()
|
|
res = m.apply(base_conv())
|
|
assert m.replayed_token_debt(base_conv(), res.messages, len) == 0
|