mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat(hooks): add protect_messages, a hard per-message compression veto (#3772)
## Why
`compute_biases` is a **soft** lever. The router first decides *this
message gets compressed, with this strategy*, and only then hands the
bias down to that strategy as a multiplier. By the time bias is
consulted the decision to rewrite the message has already been made, and
strategies are free to clamp it against their own floors.
So there is no bias value — 8, 1000, anything — that reliably means
*"leave this one alone."* Bias tunes **how much** survives; it is never
asked **whether** to touch the message at all.
A hook that has concluded a specific message must survive verbatim has
no way to say so today. This adds one.
## What
A fourth hook alongside the existing three:
```python
def protect_messages(self, messages, ctx) -> set[int]:
"""Return message indices that must not be compressed at all."""
return set()
```
`ContentRouter` checks it **before any routing decision**, so a
protected message passes through untouched whatever its content type or
role — covering both the content-block path and the string path:
```python
if i in hook_protect:
result_slots[i] = message
transforms_applied.append("router:protected:hook")
continue
```
Threaded through `compress()`, the Anthropic handler (all six
`compress_messages` call sites) and the OpenAI handler.
## Inert by construction, not by a flag
There is no env var and nothing to switch off, because there is no state
to switch:
- the base implementation returns an empty set;
- `config.hooks` is `None` unless an operator installs one;
- with no hook the router's veto set is empty, so the `if i in
hook_protect` branch never fires.
An operator who installs nothing sees byte-identical output. A flag here
would gate a code path that is already unreachable.
## Tests
`tests/test_hook_protect_messages.py`, 12 tests, covering **both**
directions.
Six fail without the router change — verified by stashing
`content_router.py` and re-running:
```
FAILED test_a_protected_message_survives_verbatim[log-anthropic]
FAILED test_a_protected_message_survives_verbatim[json-openai]
FAILED test_a_protected_message_survives_verbatim[json-anthropic]
FAILED test_protection_beats_every_bias
FAILED test_the_seam_is_reachable_from_the_sdk
6 failed, 6 passed
```
`test_protection_beats_every_bias` is the one that pins the argument
above — it fails precisely because a bias cannot express a veto.
The other six assert the seam stays inert: the default hook protects
nothing, a hooks object that doesn't implement the method still works,
and an empty veto set changes nothing.
## Verification
- 12/12 new tests pass; **518 passed, 45 skipped, 0 failed** across the
hook/router/compress suites.
- `ruff check` clean, `ruff format --check` clean (1633 files), `mypy
headroom --ignore-missing-imports` — *Success: no issues found in 538
source files*. All at CI's pinned ruff 0.16.7.
- Every `biases=` call site in both handlers now has a matching
`protect=` — checked mechanically, not by eye.
## Known boundary
The Anthropic **image-only pre-pass** on the newest user turn
(`anthropic.py:603`) compresses image bytes without consulting the veto.
Left alone deliberately: it only rewrites images in the live zone, and
threading the hook context into it would widen this change with no
caller that needs it.
Gemini's native handler and Anthropic cache mode don't run the
compression-hook pipeline at all, so they're unaffected — as they
already are for `compute_biases`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
b36e59a893
commit
d17ac6a174
@@ -230,12 +230,17 @@ def compress(
|
||||
try:
|
||||
# Compute biases from hooks if provided
|
||||
biases = None
|
||||
# Hard per-message veto. Separate from ``biases`` because a bias is a
|
||||
# soft multiplier that several strategies clamp or ignore, so it cannot
|
||||
# express "leave this one alone".
|
||||
protect = None
|
||||
if hooks:
|
||||
from headroom.hooks import CompressContext
|
||||
from headroom.hooks import CompressContext, collect_protected
|
||||
|
||||
ctx = CompressContext(model=model)
|
||||
messages = hooks.pre_compress(messages, ctx)
|
||||
biases = hooks.compute_biases(messages, ctx)
|
||||
protect = collect_protected(hooks, messages, ctx)
|
||||
|
||||
received_event = pipeline_extensions.emit(
|
||||
PipelineStage.INPUT_RECEIVED,
|
||||
@@ -257,6 +262,7 @@ def compress(
|
||||
model_limit=model_limit,
|
||||
context=context,
|
||||
biases=biases,
|
||||
protect=protect,
|
||||
# Pass CompressConfig options through to transforms
|
||||
compress_user_messages=cfg.compress_user_messages,
|
||||
compress_system_messages=cfg.compress_system_messages,
|
||||
|
||||
+60
-3
@@ -1,10 +1,11 @@
|
||||
"""Compression hooks and pipeline lifecycle events.
|
||||
|
||||
Three hooks at well-defined pipeline stages:
|
||||
Four hooks at well-defined pipeline stages:
|
||||
|
||||
1. pre_compress: modify messages before compression (dedup, filter, inject)
|
||||
2. compute_biases: set per-message compression aggressiveness (position-aware, phase-aware)
|
||||
3. post_compress: observe results after compression (learning, analytics, logging)
|
||||
3. protect_messages: veto compression of specific messages outright
|
||||
4. post_compress: observe results after compression (learning, analytics, logging)
|
||||
|
||||
The canonical pipeline also emits lifecycle events through ``on_pipeline_event``.
|
||||
That gives extensions one stable contract across SDK, ``compress()``, and proxy
|
||||
@@ -39,7 +40,7 @@ from .pipeline import PipelineEvent
|
||||
|
||||
@dataclass
|
||||
class CompressContext:
|
||||
"""Context passed to pre_compress and compute_biases hooks.
|
||||
"""Context passed to the pre_compress, compute_biases and protect_messages hooks.
|
||||
|
||||
Provides enough information for hooks to make decisions without
|
||||
needing to understand the proxy's internals.
|
||||
@@ -128,6 +129,32 @@ class CompressionHooks:
|
||||
"""
|
||||
return {}
|
||||
|
||||
def protect_messages(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
ctx: CompressContext,
|
||||
) -> set[int]:
|
||||
"""Return message indices that must not be compressed at all.
|
||||
|
||||
``compute_biases`` is a *soft* lever: a multiplier on how aggressively
|
||||
a compressor prunes. Several strategies clamp it against their own
|
||||
floors or ignore it entirely, so no bias — however large — reliably
|
||||
means "leave this one alone". A hook that has concluded a specific
|
||||
message must survive verbatim needs to say so directly, and this is how.
|
||||
|
||||
The router honours these before any routing decision, so the message is
|
||||
passed through untouched whatever its content type or role. Use
|
||||
sparingly: a protected message is context the compressor cannot reclaim.
|
||||
|
||||
Args:
|
||||
messages: The full message list.
|
||||
ctx: Compression context.
|
||||
|
||||
Returns:
|
||||
Set of message indices to pass through verbatim. Empty = no vetoes.
|
||||
"""
|
||||
return set()
|
||||
|
||||
def post_compress(self, event: CompressEvent) -> None:
|
||||
"""Called after compression completes. Observational only.
|
||||
|
||||
@@ -149,3 +176,33 @@ class CompressionHooks:
|
||||
the three legacy compression-specific hooks.
|
||||
"""
|
||||
return None
|
||||
|
||||
|
||||
def collect_protected(
|
||||
hooks: Any,
|
||||
messages: list[dict[str, Any]],
|
||||
ctx: CompressContext,
|
||||
) -> set[int] | None:
|
||||
"""Ask a hooks object for its per-message vetoes, tolerating older ones.
|
||||
|
||||
``hooks`` is duck-typed at every call site: nothing requires the object to
|
||||
be a :class:`CompressionHooks` subclass, and objects written before
|
||||
``protect_messages`` existed will not have the attribute at all. Calling it
|
||||
blind would raise ``AttributeError`` inside the compression block, which the
|
||||
proxy catches as a compression failure — so adding this hook would silently
|
||||
stop compressing for those callers, turning an additive seam into a
|
||||
regression. A missing method means "no vetoes".
|
||||
|
||||
Returns None when the hook cannot be asked, which the router treats the same
|
||||
as an empty set.
|
||||
"""
|
||||
fn = getattr(hooks, "protect_messages", None)
|
||||
if fn is None:
|
||||
return None
|
||||
protected = fn(messages, ctx)
|
||||
if protected is None:
|
||||
return None
|
||||
# Normalise here rather than trusting the hook: ``hooks`` is duck-typed, so
|
||||
# a set is a convention, not a guarantee, and a list of indices is the
|
||||
# obvious thing for an implementer to return.
|
||||
return {int(i) for i in protected}
|
||||
|
||||
@@ -1739,6 +1739,16 @@ class AnthropicHandlerMixin:
|
||||
if self.config.hooks and _hook_ctx is not None
|
||||
else None
|
||||
)
|
||||
# Hard per-message veto. Separate from ``biases`` because a
|
||||
# bias is a soft multiplier that several strategies clamp or
|
||||
# ignore, so it cannot express "leave this one alone".
|
||||
from headroom.hooks import collect_protected
|
||||
|
||||
protect = (
|
||||
collect_protected(self.config.hooks, messages, _hook_ctx)
|
||||
if self.config.hooks and _hook_ctx is not None
|
||||
else None
|
||||
)
|
||||
|
||||
# F2.1 c5/5: derive the per-request CompressionPolicy
|
||||
# from the auth_mode classified at request entry. The
|
||||
@@ -1807,6 +1817,7 @@ class AnthropicHandlerMixin:
|
||||
prefix_replay_guaranteed=True,
|
||||
idle_seconds=idle_seconds,
|
||||
biases=biases,
|
||||
protect=protect,
|
||||
request_id=request_id,
|
||||
compression_policy=compression_policy,
|
||||
cache_ttl_seconds=_cc_ttl,
|
||||
@@ -1853,6 +1864,7 @@ class AnthropicHandlerMixin:
|
||||
prefix_replay_guaranteed=True,
|
||||
idle_seconds=idle_seconds,
|
||||
biases=biases,
|
||||
protect=protect,
|
||||
request_id=request_id,
|
||||
compression_policy=compression_policy,
|
||||
cache_ttl_seconds=_cc_ttl,
|
||||
@@ -1905,6 +1917,7 @@ class AnthropicHandlerMixin:
|
||||
prefix_replay_guaranteed=True,
|
||||
idle_seconds=idle_seconds,
|
||||
biases=biases,
|
||||
protect=protect,
|
||||
request_id=request_id,
|
||||
compression_policy=compression_policy,
|
||||
cache_ttl_seconds=_cc_ttl,
|
||||
@@ -1948,6 +1961,7 @@ class AnthropicHandlerMixin:
|
||||
frozen_message_count=frozen_message_count,
|
||||
prefix_replay_guaranteed=True,
|
||||
biases=biases,
|
||||
protect=protect,
|
||||
request_id=request_id,
|
||||
compression_policy=compression_policy,
|
||||
cache_ttl_seconds=_cc_ttl,
|
||||
@@ -2010,6 +2024,7 @@ class AnthropicHandlerMixin:
|
||||
frozen_message_count=frozen_message_count,
|
||||
prefix_replay_guaranteed=True,
|
||||
biases=biases,
|
||||
protect=protect,
|
||||
request_id=request_id,
|
||||
compression_policy=compression_policy,
|
||||
**proxy_pipeline_kwargs(self.config),
|
||||
@@ -2082,6 +2097,7 @@ class AnthropicHandlerMixin:
|
||||
frozen_message_count=prefix_n,
|
||||
idle_seconds=idle_seconds,
|
||||
biases=biases,
|
||||
protect=protect,
|
||||
request_id=request_id,
|
||||
compression_policy=compression_policy,
|
||||
cache_ttl_seconds=_cc_ttl,
|
||||
|
||||
@@ -3687,13 +3687,18 @@ class OpenAIHandlerMixin:
|
||||
|
||||
# Hook: pre_compress
|
||||
_hook_biases = None
|
||||
# Hard per-message veto. Separate from ``_hook_biases`` because a bias is
|
||||
# a soft multiplier that several strategies clamp or ignore, so it cannot
|
||||
# express "leave this one alone".
|
||||
_hook_protect = None
|
||||
if self.config.hooks:
|
||||
from headroom.hooks import CompressContext
|
||||
from headroom.hooks import CompressContext, collect_protected
|
||||
|
||||
_hook_ctx = CompressContext(model=model, provider="openai")
|
||||
try:
|
||||
messages = self.config.hooks.pre_compress(messages, _hook_ctx)
|
||||
_hook_biases = self.config.hooks.compute_biases(messages, _hook_ctx)
|
||||
_hook_protect = collect_protected(self.config.hooks, messages, _hook_ctx)
|
||||
except Exception as e:
|
||||
logger.debug(f"[{request_id}] Hook error: {e}")
|
||||
|
||||
@@ -3898,6 +3903,7 @@ class OpenAIHandlerMixin:
|
||||
),
|
||||
prefix_replay_guaranteed=True,
|
||||
biases=_hook_biases,
|
||||
protect=_hook_protect,
|
||||
compression_policy=compression_policy,
|
||||
cross_turn_dedup_recoverable=_dedup_pointers_recoverable,
|
||||
# Thread the savings-profile knobs (e.g.
|
||||
@@ -3940,6 +3946,7 @@ class OpenAIHandlerMixin:
|
||||
frozen_message_count=apply_frozen_count,
|
||||
prefix_replay_guaranteed=True,
|
||||
biases=_hook_biases,
|
||||
protect=_hook_protect,
|
||||
compression_policy=compression_policy,
|
||||
cross_turn_dedup_recoverable=_dedup_pointers_recoverable,
|
||||
# Same savings-profile threading as the token-mode
|
||||
|
||||
@@ -5258,6 +5258,15 @@ class ContentRouter(Transform):
|
||||
tokens_before = sum(tokenizer.count_text(str(m.get("content", ""))) for m in messages)
|
||||
context = kwargs.get("context", "")
|
||||
hook_biases: dict[int, float] = kwargs.get("biases") or {}
|
||||
# Per-message veto from a compression hook (``CompressionHooks.
|
||||
# protect_messages``). Distinct from ``biases``, which is a soft
|
||||
# multiplier on how aggressively a compressor prunes: several
|
||||
# strategies clamp or ignore it, so a bias — however large — cannot
|
||||
# express "leave this one alone". This can. Empty unless a hook is
|
||||
# installed, so the default path is unchanged.
|
||||
hook_protect: set[int] = {
|
||||
int(i) for i in (kwargs.get("protect") or ()) if isinstance(i, (int, bool))
|
||||
}
|
||||
|
||||
# Build tool name map for exclusion checking
|
||||
tool_name_map = self._build_tool_name_map(messages)
|
||||
@@ -5557,6 +5566,14 @@ class ContentRouter(Transform):
|
||||
content = message.get("content", "")
|
||||
bias = 1.0 # Default bias, may be overridden for tool messages
|
||||
|
||||
# Hook veto, checked before any routing decision so it covers both
|
||||
# the content-block path and the string path below.
|
||||
if i in hook_protect:
|
||||
result_slots[i] = message
|
||||
transforms_applied.append("router:protected:hook")
|
||||
route_counts["hook_protected"] = route_counts.get("hook_protected", 0) + 1
|
||||
continue
|
||||
|
||||
messages_from_end = num_messages - i
|
||||
# The caller's own words stay verbatim on a replaying path even
|
||||
# when user messages are compressible for their tool observations:
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
"""``CompressionHooks.protect_messages`` — the hard per-message veto.
|
||||
|
||||
``compute_biases`` is a soft multiplier, and several strategies clamp it
|
||||
against their own floors or ignore it outright: measured against the log and
|
||||
JSON routes, bias 0.1 and bias 1000 produce byte-identical output. A hook that
|
||||
has concluded a specific message must survive verbatim therefore cannot say so
|
||||
with a bias, and this seam is how it says so instead.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom import compress
|
||||
from headroom.hooks import CompressionHooks
|
||||
from headroom.tokenizers import get_tokenizer
|
||||
from headroom.transforms.content_router import ContentRouter
|
||||
|
||||
MODEL = "gpt-4o"
|
||||
LOG = "\n".join(
|
||||
f"2026-09-2{i % 9} 10:{i % 60:02d}:00 INFO worker={i} processed batch {i} in {i * 3}ms"
|
||||
for i in range(300)
|
||||
)
|
||||
BLOB = json.dumps([{"id": i, "name": f"item-{i}", "size": i * 7} for i in range(200)])
|
||||
|
||||
|
||||
def openai_messages(body: str) -> list[dict]:
|
||||
return [
|
||||
{"role": "user", "content": "look"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "t1",
|
||||
"type": "function",
|
||||
"function": {"name": "run_terminal_cmd", "arguments": '{"command": "cat x"}'},
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "t1", "content": body},
|
||||
{"role": "user", "content": "now summarise"},
|
||||
]
|
||||
|
||||
|
||||
def anthropic_messages(body: str) -> list[dict]:
|
||||
return [
|
||||
{"role": "user", "content": "look"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "tool_use", "id": "t1", "name": "Bash", "input": {"command": "cat x"}}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [{"type": "tool_result", "tool_use_id": "t1", "content": body}],
|
||||
},
|
||||
{"role": "user", "content": "now summarise"},
|
||||
]
|
||||
|
||||
|
||||
def _text_at(messages: list[dict], index: int) -> str:
|
||||
content = messages[index]["content"]
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
return "".join(
|
||||
block.get("content", "") if isinstance(block.get("content"), str) else ""
|
||||
for block in content
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"build", [openai_messages, anthropic_messages], ids=["openai", "anthropic"]
|
||||
)
|
||||
@pytest.mark.parametrize("body", [LOG, BLOB], ids=["log", "json"])
|
||||
def test_a_protected_message_survives_verbatim(build, body) -> None:
|
||||
"""Both content shapes, and routes that ignore bias entirely."""
|
||||
tokenizer = get_tokenizer(MODEL)
|
||||
|
||||
compressed = ContentRouter().apply(build(body), tokenizer)
|
||||
assert _text_at(compressed.messages, 2) != body, "fixture must actually be compressible"
|
||||
|
||||
protected = ContentRouter().apply(build(body), tokenizer, protect={2})
|
||||
assert _text_at(protected.messages, 2) == body
|
||||
assert "router:protected:hook" in protected.transforms_applied
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"protect", [None, set(), {99}, frozenset()], ids=["none", "empty", "miss", "frozen"]
|
||||
)
|
||||
def test_the_seam_is_inert_unless_used(protect) -> None:
|
||||
"""The default path must be byte-identical to not passing the kwarg."""
|
||||
tokenizer = get_tokenizer(MODEL)
|
||||
baseline = ContentRouter().apply(openai_messages(LOG), tokenizer)
|
||||
result = ContentRouter().apply(openai_messages(LOG), tokenizer, protect=protect)
|
||||
assert _text_at(result.messages, 2) == _text_at(baseline.messages, 2)
|
||||
assert "router:protected:hook" not in result.transforms_applied
|
||||
|
||||
|
||||
def test_protection_beats_every_bias() -> None:
|
||||
"""The point of the seam: no bias can do this."""
|
||||
tokenizer = get_tokenizer(MODEL)
|
||||
for bias in (0.1, 1.0, 8.0, 1000.0):
|
||||
biased = ContentRouter().apply(openai_messages(LOG), tokenizer, biases={2: bias})
|
||||
assert _text_at(biased.messages, 2) != LOG
|
||||
protected = ContentRouter().apply(openai_messages(LOG), tokenizer, protect={2})
|
||||
assert _text_at(protected.messages, 2) == LOG
|
||||
|
||||
|
||||
def test_the_default_hook_protects_nothing() -> None:
|
||||
assert CompressionHooks().protect_messages(openai_messages(LOG), None) == set()
|
||||
|
||||
|
||||
def test_the_seam_is_reachable_from_the_sdk() -> None:
|
||||
"""Wiring check: a hook installed through ``compress(hooks=...)`` reaches
|
||||
the router. Without this the seam exists and nobody can use it."""
|
||||
|
||||
class Protecting(CompressionHooks):
|
||||
def protect_messages(self, messages, ctx):
|
||||
return {2}
|
||||
|
||||
plain = compress(openai_messages(LOG), model=MODEL, hooks=CompressionHooks())
|
||||
protecting = compress(openai_messages(LOG), model=MODEL, hooks=Protecting())
|
||||
|
||||
assert _text_at(plain.messages, 2) != LOG
|
||||
assert _text_at(protecting.messages, 2) == LOG
|
||||
assert protecting.tokens_after > plain.tokens_after
|
||||
|
||||
|
||||
def test_hooks_that_do_not_implement_it_still_work() -> None:
|
||||
"""A subclass predating this method inherits the empty default."""
|
||||
|
||||
class Legacy(CompressionHooks):
|
||||
def compute_biases(self, messages, ctx):
|
||||
return {2: 1.5}
|
||||
|
||||
result = compress(openai_messages(LOG), model=MODEL, hooks=Legacy())
|
||||
assert result.tokens_after < result.tokens_before
|
||||
|
||||
|
||||
def test_a_hooks_object_that_is_not_a_subclass_still_compresses() -> None:
|
||||
"""The real compatibility case: ``hooks`` is duck-typed, not type-checked.
|
||||
|
||||
Nothing requires the object passed as ``hooks`` to subclass
|
||||
CompressionHooks, so one written before ``protect_messages`` existed has no
|
||||
such attribute. Calling it blind raises AttributeError inside the
|
||||
compression block, which the proxy catches as a *compression failure* — so
|
||||
the request would silently stop being compressed. That would make this
|
||||
additive seam a regression for those callers.
|
||||
"""
|
||||
|
||||
class DuckTyped: # deliberately not a CompressionHooks subclass
|
||||
def pre_compress(self, messages, ctx):
|
||||
return messages
|
||||
|
||||
def compute_biases(self, messages, ctx):
|
||||
return {}
|
||||
|
||||
def post_compress(self, event):
|
||||
pass
|
||||
|
||||
def on_pipeline_event(self, event):
|
||||
return None
|
||||
|
||||
result = compress(openai_messages(LOG), model=MODEL, hooks=DuckTyped())
|
||||
assert result.tokens_after < result.tokens_before
|
||||
|
||||
|
||||
def test_collect_protected_reports_a_missing_method_as_no_vetoes() -> None:
|
||||
from headroom.hooks import CompressContext, collect_protected
|
||||
|
||||
class Without:
|
||||
pass
|
||||
|
||||
ctx = CompressContext(model=MODEL)
|
||||
assert collect_protected(Without(), [], ctx) is None
|
||||
assert collect_protected(CompressionHooks(), [], ctx) == set()
|
||||
Reference in New Issue
Block a user