Files
headroom/tests/test_hook_protect_messages.py
Tejas ChopraandClaude Opus 5 d17ac6a174 feat(hooks): add protect_messages, a hard per-message compression veto (#3772)
## Why

`compute_biases` is a **soft** lever. The router first decides *this
message gets compressed, with this strategy*, and only then hands the
bias down to that strategy as a multiplier. By the time bias is
consulted the decision to rewrite the message has already been made, and
strategies are free to clamp it against their own floors.

So there is no bias value — 8, 1000, anything — that reliably means
*"leave this one alone."* Bias tunes **how much** survives; it is never
asked **whether** to touch the message at all.

A hook that has concluded a specific message must survive verbatim has
no way to say so today. This adds one.

## What

A fourth hook alongside the existing three:

```python
def protect_messages(self, messages, ctx) -> set[int]:
    """Return message indices that must not be compressed at all."""
    return set()
```

`ContentRouter` checks it **before any routing decision**, so a
protected message passes through untouched whatever its content type or
role — covering both the content-block path and the string path:

```python
if i in hook_protect:
    result_slots[i] = message
    transforms_applied.append("router:protected:hook")
    continue
```

Threaded through `compress()`, the Anthropic handler (all six
`compress_messages` call sites) and the OpenAI handler.

## Inert by construction, not by a flag

There is no env var and nothing to switch off, because there is no state
to switch:

- the base implementation returns an empty set;
- `config.hooks` is `None` unless an operator installs one;
- with no hook the router's veto set is empty, so the `if i in
hook_protect` branch never fires.

An operator who installs nothing sees byte-identical output. A flag here
would gate a code path that is already unreachable.

## Tests

`tests/test_hook_protect_messages.py`, 12 tests, covering **both**
directions.

Six fail without the router change — verified by stashing
`content_router.py` and re-running:

```
FAILED test_a_protected_message_survives_verbatim[log-anthropic]
FAILED test_a_protected_message_survives_verbatim[json-openai]
FAILED test_a_protected_message_survives_verbatim[json-anthropic]
FAILED test_protection_beats_every_bias
FAILED test_the_seam_is_reachable_from_the_sdk
6 failed, 6 passed
```

`test_protection_beats_every_bias` is the one that pins the argument
above — it fails precisely because a bias cannot express a veto.

The other six assert the seam stays inert: the default hook protects
nothing, a hooks object that doesn't implement the method still works,
and an empty veto set changes nothing.

## Verification

- 12/12 new tests pass; **518 passed, 45 skipped, 0 failed** across the
hook/router/compress suites.
- `ruff check` clean, `ruff format --check` clean (1633 files), `mypy
headroom --ignore-missing-imports` — *Success: no issues found in 538
source files*. All at CI's pinned ruff 0.16.7.
- Every `biases=` call site in both handlers now has a matching
`protect=` — checked mechanically, not by eye.

## Known boundary

The Anthropic **image-only pre-pass** on the newest user turn
(`anthropic.py:603`) compresses image bytes without consulting the veto.
Left alone deliberately: it only rewrites images in the live zone, and
threading the hook context into it would widen this change with no
caller that needs it.

Gemini's native handler and Anthropic cache mode don't run the
compression-hook pipeline at all, so they're unaffected — as they
already are for `compute_biases`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 13:10:56 +02:00

181 lines
6.4 KiB
Python

"""``CompressionHooks.protect_messages`` — the hard per-message veto.
``compute_biases`` is a soft multiplier, and several strategies clamp it
against their own floors or ignore it outright: measured against the log and
JSON routes, bias 0.1 and bias 1000 produce byte-identical output. A hook that
has concluded a specific message must survive verbatim therefore cannot say so
with a bias, and this seam is how it says so instead.
"""
from __future__ import annotations
import json
import pytest
from headroom import compress
from headroom.hooks import CompressionHooks
from headroom.tokenizers import get_tokenizer
from headroom.transforms.content_router import ContentRouter
MODEL = "gpt-4o"
LOG = "\n".join(
f"2026-09-2{i % 9} 10:{i % 60:02d}:00 INFO worker={i} processed batch {i} in {i * 3}ms"
for i in range(300)
)
BLOB = json.dumps([{"id": i, "name": f"item-{i}", "size": i * 7} for i in range(200)])
def openai_messages(body: str) -> list[dict]:
return [
{"role": "user", "content": "look"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "t1",
"type": "function",
"function": {"name": "run_terminal_cmd", "arguments": '{"command": "cat x"}'},
}
],
},
{"role": "tool", "tool_call_id": "t1", "content": body},
{"role": "user", "content": "now summarise"},
]
def anthropic_messages(body: str) -> list[dict]:
return [
{"role": "user", "content": "look"},
{
"role": "assistant",
"content": [
{"type": "tool_use", "id": "t1", "name": "Bash", "input": {"command": "cat x"}}
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t1", "content": body}],
},
{"role": "user", "content": "now summarise"},
]
def _text_at(messages: list[dict], index: int) -> str:
content = messages[index]["content"]
if isinstance(content, str):
return content
return "".join(
block.get("content", "") if isinstance(block.get("content"), str) else ""
for block in content
)
@pytest.mark.parametrize(
"build", [openai_messages, anthropic_messages], ids=["openai", "anthropic"]
)
@pytest.mark.parametrize("body", [LOG, BLOB], ids=["log", "json"])
def test_a_protected_message_survives_verbatim(build, body) -> None:
"""Both content shapes, and routes that ignore bias entirely."""
tokenizer = get_tokenizer(MODEL)
compressed = ContentRouter().apply(build(body), tokenizer)
assert _text_at(compressed.messages, 2) != body, "fixture must actually be compressible"
protected = ContentRouter().apply(build(body), tokenizer, protect={2})
assert _text_at(protected.messages, 2) == body
assert "router:protected:hook" in protected.transforms_applied
@pytest.mark.parametrize(
"protect", [None, set(), {99}, frozenset()], ids=["none", "empty", "miss", "frozen"]
)
def test_the_seam_is_inert_unless_used(protect) -> None:
"""The default path must be byte-identical to not passing the kwarg."""
tokenizer = get_tokenizer(MODEL)
baseline = ContentRouter().apply(openai_messages(LOG), tokenizer)
result = ContentRouter().apply(openai_messages(LOG), tokenizer, protect=protect)
assert _text_at(result.messages, 2) == _text_at(baseline.messages, 2)
assert "router:protected:hook" not in result.transforms_applied
def test_protection_beats_every_bias() -> None:
"""The point of the seam: no bias can do this."""
tokenizer = get_tokenizer(MODEL)
for bias in (0.1, 1.0, 8.0, 1000.0):
biased = ContentRouter().apply(openai_messages(LOG), tokenizer, biases={2: bias})
assert _text_at(biased.messages, 2) != LOG
protected = ContentRouter().apply(openai_messages(LOG), tokenizer, protect={2})
assert _text_at(protected.messages, 2) == LOG
def test_the_default_hook_protects_nothing() -> None:
assert CompressionHooks().protect_messages(openai_messages(LOG), None) == set()
def test_the_seam_is_reachable_from_the_sdk() -> None:
"""Wiring check: a hook installed through ``compress(hooks=...)`` reaches
the router. Without this the seam exists and nobody can use it."""
class Protecting(CompressionHooks):
def protect_messages(self, messages, ctx):
return {2}
plain = compress(openai_messages(LOG), model=MODEL, hooks=CompressionHooks())
protecting = compress(openai_messages(LOG), model=MODEL, hooks=Protecting())
assert _text_at(plain.messages, 2) != LOG
assert _text_at(protecting.messages, 2) == LOG
assert protecting.tokens_after > plain.tokens_after
def test_hooks_that_do_not_implement_it_still_work() -> None:
"""A subclass predating this method inherits the empty default."""
class Legacy(CompressionHooks):
def compute_biases(self, messages, ctx):
return {2: 1.5}
result = compress(openai_messages(LOG), model=MODEL, hooks=Legacy())
assert result.tokens_after < result.tokens_before
def test_a_hooks_object_that_is_not_a_subclass_still_compresses() -> None:
"""The real compatibility case: ``hooks`` is duck-typed, not type-checked.
Nothing requires the object passed as ``hooks`` to subclass
CompressionHooks, so one written before ``protect_messages`` existed has no
such attribute. Calling it blind raises AttributeError inside the
compression block, which the proxy catches as a *compression failure* — so
the request would silently stop being compressed. That would make this
additive seam a regression for those callers.
"""
class DuckTyped: # deliberately not a CompressionHooks subclass
def pre_compress(self, messages, ctx):
return messages
def compute_biases(self, messages, ctx):
return {}
def post_compress(self, event):
pass
def on_pipeline_event(self, event):
return None
result = compress(openai_messages(LOG), model=MODEL, hooks=DuckTyped())
assert result.tokens_after < result.tokens_before
def test_collect_protected_reports_a_missing_method_as_no_vetoes() -> None:
from headroom.hooks import CompressContext, collect_protected
class Without:
pass
ctx = CompressContext(model=MODEL)
assert collect_protected(Without(), [], ctx) is None
assert collect_protected(CompressionHooks(), [], ctx) == set()