mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(ccr): inject headroom_retrieve before the prefix is warm, not after (#3810)
## Description
Reported by a user on 0.38.0: a full cache miss (`reason=prefix_change`)
immediately after `inject_first_time`, in almost every session.
```
req 001611 tool_injection_decision=skip cache_read=53,420
req 001613 tool_injection_decision=inject_first_time (headroom_retrieve added)
CACHE-BUST expected_cached=84,416 actual_read=0
cache_read=0 cache_write=85,611
req 001629 tool_injection_decision=inject_sticky_replay cache_read=85,611
```
They were right, and it is not a config mistake.
`tools` is the **head** of Anthropic's cache key — the prefix is ordered
tools → system → messages — so any change to the array invalidates every
cached token behind it. `apply_session_sticky_ccr_tool` gated the first
injection on `has_compressed_content_this_turn`, so the tool enters the
array **at the first compression**: typically many turns in, against a
fully warm prefix. The idle time and the subagent hand-off in their logs
are coincidental; they just correlate with when the first compression
happens.
### The trade is not close
Their session: 113,888 tokens of cache write to save 2,205 tokens of
content. Pricing the classes against each other (read 0.1×, 5m write
1.25×):
| | |
|---|---|
| one-time bust | **130,971** base-token-equivalents |
| benefit per later turn | 220 |
| **payback** | **~594 turns of the same session** |
At Sonnet list that is **$0.393 spent to save $0.00066/turn**, and more
on Opus. PR-B7's sticky tracker already turned this from a per-turn flap
into a once-per-session event; this fixes the one that remained.
### The fix is timing, not mechanism
The definition is **~119 tokens**. Injecting it on the session's first
request folds that cost into the cache write the session was always
going to pay, and the tools array never changes again. Eager:
**$0.00045, once**. Lazy: **$0.393**.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
## Changes Made
- `headroom/proxy/helpers.py`: `apply_session_sticky_ccr_tool` gains
`allow_eager`; both branches now arm on `allow_eager and bool(tools_out)
and get_ccr_tool_injection_mode() == "eager"` instead of waiting for
`has_compressed_content_this_turn`.
- New `HEADROOM_CCR_TOOL_INJECTION` (`eager` | `lazy`, default `eager`)
with `resolve_ccr_tool_injection_mode` / `get_ccr_tool_injection_mode`,
following the `HEADROOM_TOOL_INJECTION_STICKY` precedent. An unknown
value raises rather than silently defaulting.
- New `inject_eager` member of the `ToolInjectionDecision` literal, so
the existing `event=tool_injection_decision` line distinguishes "entered
the array cold" from "entered against a warm prefix".
- Call sites pass `allow_eager` from what the request can actually do:
`not preserve_tool_order` (anthropic), `bool(self.config.optimize) and
not _bypass` (openai) — a request that will never compress is not
pre-armed.
- `tests/test_ccr_tool_injection_cache_stability.py`: new, 13 tests (10
functions, three parametrized over both providers / both empty-tools
spellings).
### Scope: only when the client already sends tools
My first cut injected eagerly always, and it broke 8 existing tests.
Rather than flip them I looked at why — one was asserting that a request
arriving with **no** tools must not gain a `tools` key. That is a real
invariant: it would let the model emit `tool_use` blocks the client
never expected (#728), and such a client has no tool results to
compress, so no warm tools segment to protect either. Narrowing eager
injection to "the client already sent tools" made **all 8 failures
disappear on their own**. The existing tests were right; the first
design was wrong.
### Safety
- A spurious call is **not an error path**: an unknown hash resolves to
a structured `{"status": "missing"}` tool result via
`CCRResponseHandler._execute_retrieval`, not an exception or a 400.
- `has_done_ccr` / `record_ccr_done` have no consumers outside the
injection path, so pinning the golden bytes early cannot mislead
anything else.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_ccr_tool_injection_cache_stability.py -q
13 passed in 0.46s
# same file with the default flipped back to the pre-fix timing
# (CCR_TOOL_INJECTION_DEFAULT = "lazy"), i.e. the gate this PR replaces:
6 failed, 7 passed
FAILED test_tools_array_is_byte_stable_across_the_first_compression[anthropic]
FAILED test_tools_array_is_byte_stable_across_the_first_compression[openai]
FAILED test_the_tool_is_present_from_the_very_first_request[anthropic]
FAILED test_the_tool_is_present_from_the_very_first_request[openai]
FAILED test_sticky_replay_is_byte_identical_to_the_eager_injection
FAILED test_the_sessionless_path_is_also_stable
# and the diff is the bug itself - one turn's array carries
# {"name": "headroom_retrieve", ...} and the other's does not
$ pytest -k "ccr or injection or sticky or cache"
2,079 passed (clean origin/main: 2,067 passed)
# same single failure on both: test_verbatim_read_never_cache_written_before_maturation
# pre-existing, passes in isolation; its own docstring names the cause
# (~/.headroom/ccr_store.db is shared across processes)
$ ruff check . && ruff format --check .
All checks passed!
1659 files already formatted
$ mypy headroom --ignore-missing-imports
Success: no issues found in 543 source files
```
CI on `f5b24f1d`: green.
## Real Behavior Proof
- Environment: live Anthropic API, real `create_ccr_tool_definition()`
output, 6 client tools, nothing else changed between turns.
- Exact command / steps: three turns against a warm prefix — turn 1
cold, turn 2 identical tools array, turn 3 with `headroom_retrieve`
appended and nothing else altered.
- Observed result: `turn 1 read=0 write=4,335` (cold) → `turn 2
read=4,335 write=0` (warm) → `turn 3 read=0 write=4,458`. One 478-byte
tool, entire prefix gone. This is the customer's `reason=prefix_change`
reproduced from first principles.
- Not tested: no live-API run of the *fixed* path end to end — the
post-fix evidence is the byte-identity assertion over the tools array
plus the suite, not a second live capture. My first verification sweep
also used a `-k "ccr or injection or sticky or cache"` filter that
matched nothing in `test_anthropic_stage_timings.py`, which is why CI
caught the `--no-optimize` flaw and I did not; the filter was the hole,
not the coverage.
## Runtime Rollout Safety
- Rollout-managed feature(s): CCR retrieval-tool injection timing, via
the new `HEADROOM_CCR_TOOL_INJECTION` env var.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes — the default changes from `lazy`
(inject at first compression) to `eager` (inject on the session's first
request, only when the client already sent tools). Deliberate: the old
default is the bug. The tool's *contents* and the sticky golden-bytes
replay are unchanged, so only the timing of the first injection moves.
- Kill switch / disable path: `HEADROOM_CCR_TOOL_INJECTION=lazy`
restores the historical gate exactly. `--no-ccr` (env `HEADROOM_NO_CCR`)
still disables CCR injection entirely.
- Unsafe override required: No.
- Qualification impact: Requests that already send tools now carry
`headroom_retrieve` from turn 1, so any fixture or assertion pinning the
turn-1 tools array for a tools-bearing request will see one extra entry.
No-tools requests, `--no-optimize`, and bypassed requests are unaffected
by construction and covered by tests.
- Rollback path: Set `HEADROOM_CCR_TOOL_INJECTION=lazy` for an immediate
revert with no deploy, or revert the commit. Nothing is persisted and
the session tracker is in-memory, so a rollback takes effect on the next
session with no migration.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
Documentation: the new env var is self-documenting in the resolver and
carries its rationale in comments; say the word if you want it in the
proxy env-var reference and I will add it here.
## Additional Notes
Known, not fixed here: on the OpenAI **chat-streaming** path
`can_inject_ccr_tool` is False, which also skips sticky replay — so a
session mixing streaming and non-streaming turns toggles the tool
repeatedly, busting the tools segment each way. Worse than the bug fixed
here because it recurs, but resolving it properly needs buffered-CCR
work rather than a timing change. Tracked as #3815; #3816 is its first
step and is independent of this PR (zero file overlap, either merge
order is fine).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f074711726
commit
bb8c285714
@@ -2532,6 +2532,12 @@ class AnthropicHandlerMixin:
|
||||
existing_tools=tools,
|
||||
has_compressed_content_this_turn=injector.has_compressed_content,
|
||||
history_has_ccr_reference=history_references_ccr_tool(optimized_messages),
|
||||
# `preserve_tool_order` is this handler's existing name
|
||||
# for "leave the client's tools alone" (bypass, or
|
||||
# optimization off). Those requests never compress, so
|
||||
# injecting ahead of a compression that will not happen
|
||||
# would leave an unredeemable tool in the array.
|
||||
allow_eager=not preserve_tool_order,
|
||||
)
|
||||
if ccr_tool_injected:
|
||||
logger.debug(
|
||||
|
||||
@@ -4202,6 +4202,9 @@ class OpenAIHandlerMixin:
|
||||
existing_tools=tools,
|
||||
has_compressed_content_this_turn=has_new_compressed_content,
|
||||
history_has_ccr_reference=history_references_ccr_tool(optimized_messages),
|
||||
# Same rule as the Anthropic handler: only pre-arm the tool
|
||||
# for a request that can actually compress.
|
||||
allow_eager=bool(self.config.optimize) and not _bypass,
|
||||
)
|
||||
if ccr_tool_injected:
|
||||
logger.debug(
|
||||
|
||||
@@ -93,8 +93,12 @@ from headroom.proxy.tool_definition_serialization import (
|
||||
serialize_tool_definition_canonical as _serialize_tool_definition_canonical,
|
||||
)
|
||||
from headroom.proxy.tool_injection_config import (
|
||||
CcrToolInjectionMode,
|
||||
ToolInjectionStickyMode,
|
||||
)
|
||||
from headroom.proxy.tool_injection_config import (
|
||||
get_ccr_tool_injection_mode as _get_ccr_tool_injection_mode,
|
||||
)
|
||||
from headroom.proxy.tool_injection_config import (
|
||||
get_tool_injection_sticky_mode as _get_tool_injection_sticky_mode,
|
||||
)
|
||||
@@ -2287,6 +2291,15 @@ def get_tool_injection_sticky_mode() -> ToolInjectionStickyMode:
|
||||
return _get_tool_injection_sticky_mode()
|
||||
|
||||
|
||||
def get_ccr_tool_injection_mode() -> CcrToolInjectionMode:
|
||||
"""Return when the CCR retrieval tool enters the tools array.
|
||||
|
||||
Read at request time so operators can flip behaviour without a restart.
|
||||
Unknown values raise loudly per the no-silent-fallback build constraint.
|
||||
"""
|
||||
return _get_ccr_tool_injection_mode()
|
||||
|
||||
|
||||
def get_tool_tracker_max_sessions() -> int:
|
||||
"""Return the LRU bound for `SessionToolTracker` (sessions cap)."""
|
||||
return _get_tool_tracker_max_sessions()
|
||||
@@ -2686,6 +2699,7 @@ def apply_session_sticky_ccr_tool(
|
||||
existing_tools: list[dict[str, Any]] | None,
|
||||
has_compressed_content_this_turn: bool,
|
||||
history_has_ccr_reference: bool = False,
|
||||
allow_eager: bool = False,
|
||||
) -> tuple[list[dict[str, Any]], bool]:
|
||||
"""Apply sticky-on CCR retrieval-tool injection per :class:`SessionCcrTracker`.
|
||||
|
||||
@@ -2741,7 +2755,9 @@ def apply_session_sticky_ccr_tool(
|
||||
# definition and the provider rejects the request because history still
|
||||
# references it (#2440).
|
||||
if not session_id:
|
||||
if not (has_compressed_content_this_turn or history_has_ccr_reference):
|
||||
# See the fresh-session branch below for what gates eager injection.
|
||||
eager = allow_eager and bool(tools_out) and get_ccr_tool_injection_mode() == "eager"
|
||||
if not (eager or has_compressed_content_this_turn or history_has_ccr_reference):
|
||||
log_tool_injection_decision(
|
||||
provider=provider,
|
||||
session_id=None,
|
||||
@@ -2757,7 +2773,7 @@ def apply_session_sticky_ccr_tool(
|
||||
session_id=None,
|
||||
decision="inject_first_time"
|
||||
if has_compressed_content_this_turn
|
||||
else "inject_history_reference",
|
||||
else ("inject_history_reference" if history_has_ccr_reference else "inject_eager"),
|
||||
tool_definition_bytes_count=len(replay.canonical_bytes),
|
||||
request_id=request_id,
|
||||
)
|
||||
@@ -2807,8 +2823,37 @@ def apply_session_sticky_ccr_tool(
|
||||
)
|
||||
return tools_out, True
|
||||
|
||||
# Fresh session — only inject when this turn produced compressed content.
|
||||
if not has_compressed_content_this_turn:
|
||||
# Fresh session. `tools` is the head of Anthropic's cache key, so the
|
||||
# moment this tool enters the array decides what it costs. Waiting for the
|
||||
# first compression means entering against a fully warm prefix and
|
||||
# invalidating all of it — measured on a customer session at 113,888 tokens
|
||||
# of cache write to save 2,205 tokens of content, a ~594-turn payback that
|
||||
# no session reaches. The definition is ~119 tokens; injecting it on the
|
||||
# first request instead folds that cost into the cache write the session
|
||||
# was always going to pay, and the array never changes again.
|
||||
#
|
||||
# The historical gate is still reachable via HEADROOM_CCR_TOOL_INJECTION=lazy
|
||||
# for operators who would rather keep the tool out of conversations that
|
||||
# never compress. A spurious call costs a round trip, not an error: an
|
||||
# unknown hash resolves to a structured {"status": "missing"} tool result
|
||||
# (see CCRResponseHandler._execute_retrieval), not an exception or a 400.
|
||||
#
|
||||
# Eager injection needs three things to be true, because it is the one path
|
||||
# that touches the tools array before anything has been compressed:
|
||||
#
|
||||
# allow_eager -- the caller confirms it may rewrite this request's tools
|
||||
# at all. Under `--no-optimize` or a bypass header nothing will ever be
|
||||
# compressed, so the tool would be permanently unredeemable; the old
|
||||
# gate got this for free because no compression meant no injection.
|
||||
# a non-empty client tools array -- adding the first entry would turn a
|
||||
# no-tools request into a tools request and let the model emit tool_use
|
||||
# blocks the client never expected (#728). Such a client also has no
|
||||
# tool results to compress, so there is no warm tools segment to
|
||||
# protect. The cache problem is a harness problem, and harnesses always
|
||||
# send tools.
|
||||
# the eager mode -- operators can restore the historical gate.
|
||||
eager = allow_eager and bool(tools_out) and get_ccr_tool_injection_mode() == "eager"
|
||||
if not (eager or has_compressed_content_this_turn):
|
||||
log_tool_injection_decision(
|
||||
provider=provider,
|
||||
session_id=session_id,
|
||||
@@ -2824,7 +2869,7 @@ def apply_session_sticky_ccr_tool(
|
||||
log_tool_injection_decision(
|
||||
provider=provider,
|
||||
session_id=session_id,
|
||||
decision="inject_first_time",
|
||||
decision="inject_first_time" if has_compressed_content_this_turn else "inject_eager",
|
||||
tool_definition_bytes_count=len(replay.canonical_bytes),
|
||||
request_id=request_id,
|
||||
)
|
||||
|
||||
@@ -5,26 +5,40 @@ from __future__ import annotations
|
||||
import os
|
||||
|
||||
from headroom.proxy.tool_injection_policy import (
|
||||
CCR_TOOL_INJECTION_DEFAULT,
|
||||
CCR_TOOL_INJECTION_ENV,
|
||||
TOOL_INJECTION_STICKY_DEFAULT,
|
||||
TOOL_INJECTION_STICKY_ENV,
|
||||
TOOL_TRACKER_MAX_SESSIONS_DEFAULT,
|
||||
TOOL_TRACKER_MAX_SESSIONS_ENV,
|
||||
CcrToolInjectionMode,
|
||||
ToolInjectionStickyMode,
|
||||
resolve_ccr_tool_injection_mode,
|
||||
resolve_tool_injection_sticky_mode,
|
||||
resolve_tool_tracker_max_sessions,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CCR_TOOL_INJECTION_DEFAULT",
|
||||
"CCR_TOOL_INJECTION_ENV",
|
||||
"CcrToolInjectionMode",
|
||||
"TOOL_INJECTION_STICKY_DEFAULT",
|
||||
"TOOL_INJECTION_STICKY_ENV",
|
||||
"TOOL_TRACKER_MAX_SESSIONS_DEFAULT",
|
||||
"TOOL_TRACKER_MAX_SESSIONS_ENV",
|
||||
"ToolInjectionStickyMode",
|
||||
"get_ccr_tool_injection_mode",
|
||||
"get_tool_injection_sticky_mode",
|
||||
"get_tool_tracker_max_sessions",
|
||||
]
|
||||
|
||||
|
||||
def get_ccr_tool_injection_mode() -> CcrToolInjectionMode:
|
||||
"""Return when the CCR retrieval tool enters the tools array."""
|
||||
|
||||
return resolve_ccr_tool_injection_mode(os.environ.get(CCR_TOOL_INJECTION_ENV))
|
||||
|
||||
|
||||
def get_tool_injection_sticky_mode() -> ToolInjectionStickyMode:
|
||||
"""Return the active memory-tool stickiness mode."""
|
||||
|
||||
|
||||
@@ -8,6 +8,12 @@ from typing import Literal
|
||||
ToolInjectionDecision = Literal[
|
||||
"inject_first_time",
|
||||
"inject_sticky_replay",
|
||||
# CCR, eager mode (the default): the retrieval tool is injected on the
|
||||
# session's first request, before anything has been compressed, so the
|
||||
# tools array — the head of the provider cache key — never changes
|
||||
# mid-session. Distinguished from inject_first_time so operators can tell
|
||||
# "entered the array cold" from "entered it against a warm prefix".
|
||||
"inject_eager",
|
||||
# Sessionless path: history already references headroom_retrieve, so the
|
||||
# tool definition is re-injected even without fresh compression (#2440).
|
||||
"inject_history_reference",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Policy helpers for memory tool injection stickiness configuration."""
|
||||
"""Policy helpers for proxy tool injection configuration."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -8,6 +8,15 @@ TOOL_INJECTION_STICKY_ENV = "HEADROOM_TOOL_INJECTION_STICKY"
|
||||
ToolInjectionStickyMode = Literal["enabled", "disabled"]
|
||||
TOOL_INJECTION_STICKY_DEFAULT: ToolInjectionStickyMode = "enabled"
|
||||
|
||||
CCR_TOOL_INJECTION_ENV = "HEADROOM_CCR_TOOL_INJECTION"
|
||||
CcrToolInjectionMode = Literal["eager", "lazy"]
|
||||
#: Eager by default. ``tools`` is the head of Anthropic's cache key, so adding
|
||||
#: ``headroom_retrieve`` at the first compression — typically many turns in,
|
||||
#: against a fully warm prefix — invalidates the entire prefix behind it. The
|
||||
#: definition is ~119 tokens; paying for it in the session's first (unavoidable)
|
||||
#: cache write is orders of magnitude cheaper than a mid-session rewrite.
|
||||
CCR_TOOL_INJECTION_DEFAULT: CcrToolInjectionMode = "eager"
|
||||
|
||||
TOOL_TRACKER_MAX_SESSIONS_ENV = "HEADROOM_TOOL_TRACKER_MAX_SESSIONS"
|
||||
TOOL_TRACKER_MAX_SESSIONS_DEFAULT = 1000
|
||||
|
||||
@@ -25,6 +34,24 @@ def resolve_tool_injection_sticky_mode(raw: str | None) -> ToolInjectionStickyMo
|
||||
)
|
||||
|
||||
|
||||
def resolve_ccr_tool_injection_mode(raw: str | None) -> CcrToolInjectionMode:
|
||||
"""Resolve when the CCR retrieval tool enters the tools array.
|
||||
|
||||
``eager`` injects from the session's first request, so the tools array is
|
||||
byte-stable for the life of the session. ``lazy`` restores the historical
|
||||
behaviour of waiting for the first compression, which keeps the tool out of
|
||||
conversations that never compress at the cost of one full prefix
|
||||
invalidation in those that do.
|
||||
"""
|
||||
|
||||
normalized = (raw or "").strip().lower()
|
||||
if not normalized:
|
||||
return CCR_TOOL_INJECTION_DEFAULT
|
||||
if normalized in ("eager", "lazy"):
|
||||
return cast(CcrToolInjectionMode, normalized)
|
||||
raise ValueError(f"Invalid {CCR_TOOL_INJECTION_ENV}={normalized!r}; expected 'eager' or 'lazy'")
|
||||
|
||||
|
||||
def resolve_tool_tracker_max_sessions(raw: str | None) -> int:
|
||||
"""Resolve the positive LRU session bound for tool injection tracking."""
|
||||
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
"""The CCR retrieval tool must not enter the tools array mid-session.
|
||||
|
||||
``tools`` is the head of Anthropic's cache key — the prefix is ordered
|
||||
tools → system → messages — so any change to the array invalidates every
|
||||
cached token behind it. Injecting ``headroom_retrieve`` at the first
|
||||
compression means entering the array against a fully warm prefix.
|
||||
|
||||
Measured against the live API with the real definition: a warm prefix reading
|
||||
4,335 tokens dropped to 0 read and 4,458 written the moment the 478-byte tool
|
||||
was appended. On a customer session the same effect cost 113,888 tokens of
|
||||
cache write to save 2,205 tokens of content — a payback of ~594 turns *within
|
||||
one session*, which no session reaches.
|
||||
|
||||
The fix is timing, not mechanism: inject on the first request, while the
|
||||
prefix is still cold, and the array never changes again.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.ccr.tool_injection import CCR_TOOL_NAME, create_ccr_tool_definition
|
||||
from headroom.proxy.helpers import (
|
||||
_reset_session_ccr_tracker_for_test,
|
||||
apply_session_sticky_ccr_tool,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_tracker():
|
||||
_reset_session_ccr_tracker_for_test()
|
||||
yield
|
||||
_reset_session_ccr_tracker_for_test()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _default_eager(monkeypatch):
|
||||
"""Pin the default explicitly so a stray env cannot mask a regression."""
|
||||
monkeypatch.delenv("HEADROOM_CCR_TOOL_INJECTION", raising=False)
|
||||
|
||||
|
||||
CLIENT_TOOLS = [
|
||||
{
|
||||
"name": "Read",
|
||||
"description": "Read a file.",
|
||||
"input_schema": {"type": "object", "properties": {"path": {"type": "string"}}},
|
||||
},
|
||||
{
|
||||
"name": "Bash",
|
||||
"description": "Run a command.",
|
||||
"input_schema": {"type": "object", "properties": {"cmd": {"type": "string"}}},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def _apply(session_id, *, compressed, provider="anthropic", allow_eager=True):
|
||||
"""``allow_eager=True`` mirrors what both handlers pass for a request that
|
||||
can actually compress (optimization on, no bypass header)."""
|
||||
return apply_session_sticky_ccr_tool(
|
||||
provider=provider,
|
||||
session_id=session_id,
|
||||
request_id="req-1",
|
||||
existing_tools=CLIENT_TOOLS,
|
||||
has_compressed_content_this_turn=compressed,
|
||||
allow_eager=allow_eager,
|
||||
)
|
||||
|
||||
|
||||
def _names(tools):
|
||||
return [t.get("name") or t.get("function", {}).get("name") for t in tools]
|
||||
|
||||
|
||||
# ── the regression this file exists for ───────────────────────────────
|
||||
|
||||
|
||||
@pytest.mark.parametrize("provider", ["anthropic", "openai"])
|
||||
def test_tools_array_is_byte_stable_across_the_first_compression(provider):
|
||||
"""The turn that first compresses must not change the tools array.
|
||||
|
||||
This is the whole bug. Before the fix, turn 1 returned two tools and turn 2
|
||||
returned three — and because tools sit first in the cache prefix, that one
|
||||
appended entry invalidated the entire warm prefix behind it.
|
||||
"""
|
||||
session = f"sess-{provider}-stability"
|
||||
|
||||
before, _ = _apply(session, compressed=False, provider=provider)
|
||||
during, _ = _apply(session, compressed=True, provider=provider)
|
||||
after, _ = _apply(session, compressed=False, provider=provider)
|
||||
|
||||
assert json.dumps(before) == json.dumps(during) == json.dumps(after), (
|
||||
"tools array changed across the first compression — this invalidates "
|
||||
"the whole provider cache prefix"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("provider", ["anthropic", "openai"])
|
||||
def test_the_tool_is_present_from_the_very_first_request(provider):
|
||||
"""Cold prefix is the only cheap moment to add to the tools array."""
|
||||
tools, injected = _apply(f"sess-{provider}-first", compressed=False, provider=provider)
|
||||
assert injected is True
|
||||
assert CCR_TOOL_NAME in _names(tools)
|
||||
|
||||
|
||||
def test_the_tool_definition_is_small_enough_that_eager_is_obviously_right():
|
||||
"""Guards the premise: if this ever grows large, revisit the trade."""
|
||||
size = len(json.dumps(create_ccr_tool_definition()))
|
||||
assert size < 2000, (
|
||||
f"headroom_retrieve is now {size} bytes; eager injection was justified "
|
||||
"on it being ~478 bytes (~119 tokens) of the session's first cache write"
|
||||
)
|
||||
|
||||
|
||||
# ── the escape hatch still works ──────────────────────────────────────
|
||||
|
||||
|
||||
def test_lazy_mode_restores_the_historical_gate(monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_CCR_TOOL_INJECTION", "lazy")
|
||||
session = "sess-lazy"
|
||||
|
||||
tools, injected = _apply(session, compressed=False)
|
||||
assert injected is False
|
||||
assert CCR_TOOL_NAME not in _names(tools)
|
||||
|
||||
tools, injected = _apply(session, compressed=True)
|
||||
assert injected is True
|
||||
assert CCR_TOOL_NAME in _names(tools)
|
||||
|
||||
|
||||
def test_an_invalid_mode_raises_rather_than_silently_defaulting(monkeypatch):
|
||||
from headroom.proxy.helpers import get_ccr_tool_injection_mode
|
||||
|
||||
monkeypatch.setenv("HEADROOM_CCR_TOOL_INJECTION", "sometimes")
|
||||
with pytest.raises(ValueError, match="HEADROOM_CCR_TOOL_INJECTION"):
|
||||
get_ccr_tool_injection_mode()
|
||||
|
||||
|
||||
# ── properties the fix must not break ─────────────────────────────────
|
||||
|
||||
|
||||
@pytest.mark.parametrize("empty", [None, []])
|
||||
def test_a_request_with_no_tools_is_not_armed_eagerly(empty):
|
||||
"""Eager injection must not turn a no-tools request into a tools request.
|
||||
|
||||
Adding the first entry to an absent tools array lets the model emit
|
||||
tool_use blocks the client never expected (#728). Such a client is also not
|
||||
a harness: it has no tool results to compress, so there is no warm tools
|
||||
segment for eager injection to protect. It keeps the historical gate.
|
||||
"""
|
||||
tools, injected = apply_session_sticky_ccr_tool(
|
||||
provider="anthropic",
|
||||
session_id="sess-no-tools",
|
||||
request_id="req-1",
|
||||
existing_tools=empty,
|
||||
has_compressed_content_this_turn=False,
|
||||
allow_eager=True,
|
||||
)
|
||||
assert injected is False
|
||||
assert tools == []
|
||||
|
||||
|
||||
def test_a_client_provided_tool_still_wins(monkeypatch):
|
||||
"""An MCP-registered headroom_retrieve must not be doubled up."""
|
||||
client_owned = [*CLIENT_TOOLS, {"name": CCR_TOOL_NAME, "description": "client's own"}]
|
||||
tools, injected = apply_session_sticky_ccr_tool(
|
||||
provider="anthropic",
|
||||
session_id="sess-client-owned",
|
||||
request_id="req-1",
|
||||
existing_tools=client_owned,
|
||||
has_compressed_content_this_turn=False,
|
||||
allow_eager=True,
|
||||
)
|
||||
assert injected is False
|
||||
assert _names(tools).count(CCR_TOOL_NAME) == 1
|
||||
|
||||
|
||||
def test_sticky_replay_is_byte_identical_to_the_eager_injection():
|
||||
"""Later turns replay the golden bytes recorded by the eager injection."""
|
||||
session = "sess-golden"
|
||||
first, _ = _apply(session, compressed=False)
|
||||
later, _ = _apply(session, compressed=True)
|
||||
assert json.dumps(first) == json.dumps(later)
|
||||
|
||||
|
||||
def test_the_sessionless_path_is_also_stable():
|
||||
"""No session id means no tracker, so the flag alone drove the toggle."""
|
||||
a, _ = _apply(None, compressed=False)
|
||||
b, _ = _apply(None, compressed=True)
|
||||
assert json.dumps(a) == json.dumps(b)
|
||||
assert CCR_TOOL_NAME in _names(a)
|
||||
|
||||
|
||||
def test_a_request_that_cannot_compress_is_not_armed():
|
||||
"""`--no-optimize` or a bypass header means nothing will ever be compressed.
|
||||
|
||||
Pre-arming there would leave a permanently unredeemable tool in the client's
|
||||
array. The old gate got this for free: no compression meant no injection.
|
||||
"""
|
||||
tools, injected = _apply("sess-no-optimize", compressed=False, allow_eager=False)
|
||||
assert injected is False
|
||||
assert CCR_TOOL_NAME not in _names(tools)
|
||||
Reference in New Issue
Block a user