fix(proxy): snapshot the original messages when a turn hook can edit them (#4241)

## Description

`snapshot_original_messages` (`headroom/proxy/helpers.py`) skips a
deepcopy by aliasing the live `messages` list. It only takes a real copy
when `config.hooks` or a pipeline extension is set, and it ignores
registered turn hooks. A turn hook that rewrites message text in place
(headroom-skill-search folds the skill listing) therefore rewrote the
recorded original as well. On Anthropic `/v1/messages`, `tok_before`
became the post-hook count and the headline booked `tokens_saved=0`,
even though the request had actually shrunk.

I found this while collecting the live skill-search proof that
@JerrettDavis asked for on #4057. That PR does not touch this helper, so
the fix is here, on its own.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `snapshot_original_messages` now also returns a deep copy when
`registered_turn_hooks()` is non-empty. With no turn hooks registered,
the cost is an empty `sorted([])`.
- Comments at the three call sites, and the docstring of the test
module, now mention turn hooks.
- Tests in `tests/test_original_snapshot_in_place_hook.py`:
  - `test_the_snapshot_is_a_copy_while_a_turn_hook_is_registered` (unit)
- `test_anthropic_headline_counts_a_turn_hook_fold_made_in_place`, a
real `/v1/messages` TestClient turn with only the upstream mocked
- an autouse fixture that clears the turn-hook registry before and after
each test

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, `ruff format --check .`, ruff
0.16.8)
- [x] Type checking passes (`mypy headroom`, mypy 1.20.2)
- [x] New tests added

### Test Output

```text
# helpers.py from main, new tests:
FAILED tests/test_original_snapshot_in_place_hook.py::test_the_snapshot_is_a_copy_while_a_turn_hook_is_registered
FAILED tests/test_original_snapshot_in_place_hook.py::test_anthropic_headline_counts_a_turn_hook_fold_made_in_place   (assert 8 > 8)
2 failed, 3 passed
# with the fix:
tests/test_original_snapshot_in_place_hook.py tests/test_turn_hooks.py tests/gateway + hook/skill suites: 401 passed, 4 skipped
```

## Real Behavior Proof

- Environment: macOS, Python 3.12.13, a real proxy run from #4057's
branch with headroom-skill-search 0.2.0 (`--proxy-extension
skill_search`, `HEADROOM_SKILL_SEARCH_HINT_MODE=first_sentence`,
`HEADROOM_BEACON=off`), real Anthropic upstream, `claude-haiku-4-5`.
- Exact command / steps: a non-streaming curl to `/v1/messages` that
lists 22 skills the way Claude Code does. Then a second turn in which
the model calls `search_skills`.
- Observed result:
  - Without this fix:
    - upstream returned 200, and the body shrank from 9116 B to 2744 B
    - the skill_search ledger booked 1327 − 108 = 1219 tokens
    - the headline read `tok_before=424 tok_after=532 tok_saved=0`
  - With this fix applied:
- turn 1: 200, `tok_before=1751 tok_after=532 tok_saved=1219`, the same
as the ledger's signed net
- turn 2: the model called `search_skills`, the hook served
`image-resize`, and the final answer came back 200 `end_turn`
    - `/stats` `by_source.skill_search`: 1 event, 1219 tokens
- Not tested:
- The headline on the OpenAI chat path. It already counted correctly
before this fix, but `original_client_messages` there, used for the
session id and debug dumps, was aliased the same way and is now a copy.
  - Streaming requests: skill search only runs on non-streaming ones.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: only when turn hooks are registered.
Each request then pays one deepcopy of `messages`, as it already does
with `config.hooks` set.
- Kill switch / disable path: unregister the turn-hook extension
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review (plus the code-reviewer agent; its
SHOULDs on test isolation and unit coverage are addressed)
- [x] This PR is ready for human review

Referenced from #4057.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4241"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Tejas Chopra
2026-10-15 21:23:01 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JD Davis
parent 4ff608975f
commit df11f27498
4 changed files with 91 additions and 13 deletions
+5 -4
View File
@@ -1248,9 +1248,10 @@ class AnthropicHandlerMixin:
pipeline_path = request.url.path if upstream_base_url else "/v1/messages"
pipeline_stream = bool(body.get("stream", False) or force_stream)
# O1 (2026-09-27 perf audit): the snapshot of the original
# conversation aliases the live list unless hooks or pipeline
# extensions are configured - those can mutate `messages` in place
# (pre_compress receives the live list), so then the snapshot is an
# conversation aliases the live list unless hooks, pipeline
# extensions or turn hooks are configured - those can mutate
# `messages` in place (pre_compress receives the live list, skill
# search folds text in place), so then the snapshot is an
# independently owned copy (snapshot_original_messages). The
# deep_copy stage is always measured so the timing summary keeps
# its key.
@@ -5779,7 +5780,7 @@ class AnthropicHandlerMixin:
pipeline_timing: dict[str, float] = {}
# O1 (2026-09-27 perf audit): the per-request snapshot aliases unless
# hooks or extensions are configured (see the main handler).
# hooks, extensions or turn hooks are configured (see the main handler).
from headroom.proxy.helpers import snapshot_original_messages
from headroom.transforms.cold_prefix import anthropic_cache_ttl_seconds
+2 -2
View File
@@ -3135,8 +3135,8 @@ class OpenAIHandlerMixin:
model = body.get("model", "unknown")
messages = body.get("messages", [])
# O1 (2026-09-27 perf audit): the snapshot of the original
# conversation aliases the live list unless hooks or pipeline
# extensions are configured; those can mutate `messages` in place
# conversation aliases the live list unless hooks, pipeline extensions
# or turn hooks are configured; those can mutate `messages` in place
# (pre_compress receives the live list), so then it is an
# independently owned copy (see snapshot_original_messages).
from headroom.proxy.helpers import snapshot_original_messages
+7 -5
View File
@@ -112,6 +112,7 @@ from headroom.proxy.tool_injection_logging import (
)
from headroom.proxy.tool_injection_tracker import SessionToolTracker as _SessionToolTracker
from headroom.proxy.tool_name_policy import extract_tool_name
from headroom.proxy.turn_hooks import registered_turn_hooks
if TYPE_CHECKING:
import httpx
@@ -4793,11 +4794,12 @@ def snapshot_original_messages(
Aliasing the live list is safe only while nothing between ingress and the
pipeline's own deepcopy can mutate it in place. A configured
``config.hooks`` (``pre_compress`` and friends receive the live list) or an
enabled pipeline extension can, so in that case the snapshot is an
independently owned deep copy; with neither configured the alias is kept
and costs nothing.
``config.hooks`` (``pre_compress`` and friends receive the live list), an
enabled pipeline extension or a registered turn hook (skill search folds
message text in place) can, so in that case the snapshot is an
independently owned deep copy; with none of them the alias is kept and
costs nothing.
"""
if hooks is not None or bool(getattr(extensions, "enabled", False)):
if hooks is not None or bool(getattr(extensions, "enabled", False)) or registered_turn_hooks():
return copy.deepcopy(messages)
return messages
+77 -2
View File
@@ -4,8 +4,10 @@ O1 aliases ``original_client_messages`` to the live ``messages`` list to skip a
deepcopy per request. That is only safe while nothing mutates the list before
the pipeline's own copy; ``config.hooks.pre_compress`` receives the live list,
so a hook that edits it in place (and returns the same list) used to rewrite the
recorded original too. With hooks or extensions configured the snapshot is an
independently owned copy; with neither, the alias stays (no env switch).
recorded original too. Registered turn hooks (skill search) fold message text
in place the same way. With hooks, extensions or turn hooks configured the
snapshot is an independently owned copy; with none, the alias stays (no env
switch).
Observed at the handlers' own consumers: ``compute_session_id`` derives the
session from the ORIGINAL client messages after the hook has run, and the
@@ -27,8 +29,18 @@ from fastapi.testclient import TestClient # noqa: E402
from headroom.hooks import CompressionHooks # noqa: E402
from headroom.proxy.helpers import snapshot_original_messages # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
from headroom.proxy.turn_hooks import clear_turn_hooks, register_turn_hook # noqa: E402
ORIGINAL = "the original user text"
@pytest.fixture(autouse=True)
def _clean_turn_hooks():
clear_turn_hooks()
yield
clear_turn_hooks()
HOOKED = "HOOKED IN PLACE"
@@ -175,3 +187,66 @@ def test_the_snapshot_aliases_only_when_nothing_can_mutate() -> None:
for kw in ({"hooks": _InPlaceHooks()}, {"extensions": _Ext()}):
snap = snapshot_original_messages(msgs, **kw)
assert snap == msgs and snap is not msgs and snap[0] is not msgs[0]
def test_the_snapshot_is_a_copy_while_a_turn_hook_is_registered() -> None:
msgs = [{"role": "user", "content": ORIGINAL}]
register_turn_hook(_InPlaceTurnHook())
snap = snapshot_original_messages(msgs)
assert snap == msgs and snap is not msgs and snap[0] is not msgs[0]
class _InPlaceTurnHook:
"""Shaped like skill search: rewrites a message's text in place, adds a tool."""
name = "in_place_fold"
stream_safe = False
def on_request(self, ctx) -> None:
for m in ctx.messages:
if m.get("role") == "user":
m["content"] = "short"
ctx.tools = [
*(ctx.tools or []),
{"name": "search", "description": "find", "input_schema": {"type": "object"}},
]
def test_anthropic_headline_counts_a_turn_hook_fold_made_in_place() -> None:
register_turn_hook(_InPlaceTurnHook())
with TestClient(create_app(_config(hooks=None))) as client:
proxy = client.app.state.proxy
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
return httpx.Response(
200,
content=json.dumps(
{
"id": "msg_1",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-5",
"content": [{"type": "text", "text": "ok"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 5, "output_tokens": 1},
}
).encode(),
headers={"content-type": "application/json"},
)
proxy._retry_request = _fake_retry
r = client.post(
"/v1/messages",
headers={"x-api-key": "sk-ant-test", "anthropic-version": "2023-06-01"},
json={
"model": "claude-sonnet-4-5",
"max_tokens": 16,
"messages": [{"role": "user", "content": ORIGINAL + " word" * 400}],
},
)
assert r.status_code == 200, r.text
before = int(r.headers["x-headroom-tokens-before"])
after = int(r.headers["x-headroom-tokens-after"])
assert before > after
assert int(r.headers["x-headroom-tokens-saved"]) == before - after > 0