Files
headroom/tests/test_content_router.py
Mohit SharmaandMartinMayday 0511bcd331 fix(router): compress tool_result blocks in mutable window with trailing messages (#4717)
## Description

In Anthropic conversation structures with client tools (such as Claude
Code or custom tool harnesses), user tool_result content blocks are
frequently accompanied by trailing follow-up prompt text or subsequent
turn messages within the current mutable turn. When
prefix_replay_guaranteed=True was active,
ContentRouter._process_content_blocks() evaluated fresh turn status
strictly via messages_from_end == 1. Because the tool execution message
was not the final element in the array (messages_from_end > 1), the
router treated it as frozen prompt cache history, leaving large
tool_result blocks uncompressed.

This change determines freshness based on membership in the mutable turn
suffix (i >= mutable_turn_start, where mutable_turn_start =
max(frozen_message_count, last_assistant_index + 1)). Tool results in
the current un-forwarded turn are now compressed into dense
representations while preserving prompt cache integrity for all frozen
prefix messages (i < frozen_message_count) and leaving preceding
completed turns untouched.

Closes #4708

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Calculated last_assistant_index and mutable_turn_start =
max(frozen_message_count, last_assistant_index + 1) in
ContentRouter.apply().
- Passed in_mutable_window=(i >= mutable_turn_start) to
ContentRouter._process_content_blocks().
- Updated fresh_turn calculation to evaluate in_mutable_window when
present, releasing tool_result blocks in the mutable turn suffix even
when trailing prompt messages exist.
- Added regression tests in tests/test_content_router.py covering tool
result compression with trailing messages and verifying frozen prefix
immutability under frozen_message_count.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_content_router.py tests/test_transforms_content_router.py
======================== 107 passed, 1 skipped in 111.42s ======================

$ uv run ruff check headroom/transforms/content_router.py tests/test_content_router.py
All checks passed!
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.9, native PyO3 Rust extension
built via maturin, model claude-3-7-sonnet-20250219.
- Exact command / steps: Ran ContentRouter with 500 log lines followed
by trailing message under prefix_replay_guaranteed.
- Observed result: Tokens before: 3070 -> Tokens after: 127 (saved 2943
tokens, 95.8% reduction on repetitive logs). FATAL disk full preserved.
Frozen prefix unchanged under frozen_message_count=3.
- Not tested: Live proxy network transmission to production Anthropic
API endpoints.

## Runtime Rollout Safety

- Rollout-managed feature(s): Router transform logic for Anthropic
content blocks.
- Minimum rollout channel: N/A
- Stable/default behavior changed: No (affects only Anthropic tool
blocks in the mutable turn suffix under prefix_replay_guaranteed=True;
preserves existing cache contract).
- Kill switch / disable path: Omitting prefix_replay_guaranteed or
specifying frozen_message_count.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Clean git revert of commit fc595554.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did not edit CHANGELOG.md — it is generated by release-please
from my Conventional Commit PR title (a CI guard enforces this)

## Additional Notes

Co-authored-by: MartinMayday <MartinMayday@users.noreply.github.com>

Co-authored-by: MartinMayday <MartinMayday@users.noreply.github.com>
2026-11-09 14:52:37 -06:00

120 lines
3.7 KiB
Python

from __future__ import annotations
from types import SimpleNamespace
import pytest
from headroom.transforms.content_router import (
CompressionStrategy,
ContentRouter,
ContentRouterConfig,
)
class _TestTokenizer:
def count_tokens(self, text: str) -> int:
return max(1, len(text) // 4)
def count_text(self, text: str) -> int:
return max(1, len(text) // 4)
def count_message_tokens(self, msg: dict) -> int:
return 10
def test_tool_result_compressed_with_trailing_messages_in_mutable_window(
monkeypatch: pytest.MonkeyPatch,
) -> None:
router = ContentRouter(ContentRouterConfig())
def fake_compress(content, context="", bias=1.0, precomputed_detection=None):
return SimpleNamespace(
compressed=content[: len(content) // 2] + "[compressed]",
compression_ratio=0.5,
strategy_used=CompressionStrategy.LOG,
)
monkeypatch.setattr(router, "compress", fake_compress)
tool_output = "tool output line " * 200
# Issue #4708: Anthropic tool_result block followed by trailing message in the mutable turn.
messages = [
{"role": "user", "content": "read the module"},
{
"role": "assistant",
"content": [{"type": "tool_use", "id": "t1", "name": "bash", "input": {}}],
},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "t1",
"content": tool_output,
"cache_control": {"type": "ephemeral"},
}
],
},
{"role": "user", "content": "what failed in the test above?"},
]
result = router.apply(
messages,
_TestTokenizer(),
model_limit=100_000,
prefix_replay_guaranteed=True,
)
compressed_block = result.messages[2]["content"][0]
assert compressed_block["content"].endswith("[compressed]"), (
"tool_result followed by trailing message must be compressed in mutable turn"
)
assert compressed_block["cache_control"] == {"type": "ephemeral"}
def test_tool_result_in_frozen_prefix_not_mutated(
monkeypatch: pytest.MonkeyPatch,
) -> None:
router = ContentRouter(ContentRouterConfig())
def fake_compress(content, context="", bias=1.0, precomputed_detection=None):
return SimpleNamespace(
compressed=content[: len(content) // 2] + "[compressed]",
compression_ratio=0.5,
strategy_used=CompressionStrategy.LOG,
)
monkeypatch.setattr(router, "compress", fake_compress)
tool_output = "tool output line " * 200
# Turn 1 tool_result is part of frozen prefix (frozen_message_count=3).
messages = [
{"role": "user", "content": "read the module"},
{
"role": "assistant",
"content": [{"type": "tool_use", "id": "t1", "name": "bash", "input": {}}],
},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "t1",
"content": tool_output,
"cache_control": {"type": "ephemeral"},
}
],
},
{"role": "assistant", "content": "tests failed with error"},
{"role": "user", "content": "now fix it"},
]
result = router.apply(
messages,
_TestTokenizer(),
model_limit=100_000,
frozen_message_count=3,
prefix_replay_guaranteed=True,
)
# Message 2 is frozen (index 2 < frozen_message_count=3), must not be mutated
assert result.messages[2]["content"][0]["content"] == tool_output