fix(router): keep ccr_retrieve exemption through orchestrator wrappers (#3915)

## Description

A `headroom_retrieve` result delivered through a generic orchestrator
wrapper is re-offloaded: OpenCode V2 Code Mode runs every MCP call
inside its built-in `execute` tool and Codex code mode uses `exec` /
`functions.exec`, so the wire tool name is the wrapper, the inner name
never reaches the proxy, and the existing `ccr_retrieve` exemption
(which keys on the tool name) misses. SmartCrusher then rewrites the
retrieved bytes into a fresh `<<ccr:hash>>` marker the model can never
redeem (same unresolvable-retrieval-loop class as #1077 / #2698). Refs
#3563.

This change resolves a recognized orchestrator call to the retrieval
tool's own name when -- and only when -- the call's own payload invokes
`headroom_retrieve`. Per #3563's constraints: the wrapper is never
exempted as a whole, and the result content is never consulted (an
`original_content` property is user-controllable data, not recovery
proof).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/config.py`: `unwrap_tool_call()` now resolves an
orchestrator wrapper (`execute`, `exec`, `functions.exec`) whose payload
contains a JavaScript call to `headroom_retrieve` -- bare, dotted chain,
`mcp__`-alias or bracket access, JSON-escaped or not -- to
`headroom_retrieve`. The two identity consumers
(`ContentRouter._build_tool_name_map` and the Responses handler in
`proxy/handlers/openai.py`) pick this up through the shared helper, so
both OpenAI-chat and Anthropic shapes are covered.
- `tests/test_orchestrator_wrapper_ccr_retrieve.py`: new regression
tests (helper-level and ContentRouter-level, plus a non-retrieve control
proving the exemption stays narrow).

## Testing

- [x] Unit tests pass (`pytest`; full tree ran on the patched clone: 14
047 passed, 600 skipped, 12 failed - all 12 triaged, none related to
this change: 7 pass in isolation, 2 hit a local proxy already holding
port 8787, 1 needs the `image` extra (`PIL`), 2 fail identically on
unpatched `main` in this environment)
- [x] Linting passes (`ruff check .`)
- [x] Formatting clean (`ruff format --check .` - 1692 files)
- [x] Type checking: `mypy headroom` -> 1 error, pre-existing and
unchanged: `headroom/integrations/langchain/chat_model.py:394
truthy-function`. It reproduces when that file alone is checked, and
neither file in this PR reports any error.
- [x] New tests added for new functionality
- [x] Manual testing performed (local dev proxy, `/v1/compress` only)

### Test Output

```text
$ uv run pytest tests/test_orchestrator_wrapper_ccr_retrieve.py -q
# before the fix: 5 failed, 4 passed
# after  the fix: 9 passed

$ uv run pytest tests/test_hermes_tool_call_unwrap.py \
    tests/test_transforms/test_content_router_ccr_retrieve_exemption.py \
    tests/test_transforms/test_smart_crusher_ccr_retrieve_exemption.py \
    tests/test_transforms/test_content_router.py \
    tests/test_openai_responses_compression_units.py -q
142 passed

$ uv run ruff check .
All checks passed!

$ uv run ruff format --check .
1692 files already formatted

$ uv run mypy headroom
headroom/integrations/langchain/chat_model.py:394: error: Function "RunnableBinding"
could always be true in boolean context  [truthy-function]
Found 1 error in 1 file (checked 550 source files)   # pre-existing, unrelated
```

## Real Behavior Proof

- Environment: Linux (x86_64), Python 3.13.12, headroom built from this
branch (`uv sync --extra dev --extra proxy`), dev proxy on
`127.0.0.1:8789` with an isolated `HEADROOM_WORKSPACE_DIR`, cache mode,
SmartCrusher + CCR enabled, ML compression as shipped. No provider
endpoint is contacted.
- Exact command / steps: start `HEADROOM_WORKSPACE_DIR=... uv run
headroom proxy --port 8789`, then `uv run python probe.py
http://127.0.0.1:8789`. The probe is adapted from the #3563 repro script
(`/v1/compress` + `/v1/retrieve` only); the OpenCode case is an
assistant tool_call named `execute` whose arguments contain
`tools.headroom.headroom_retrieve({hash: "..."})`, followed by the tool
result carrying the retrieve JSON (object and MCP-envelope shapes).
- Observed result: before -> after, same probe run against the unpatched
then patched proxy:
- `execute` calling retrieve, JSON object result:
`router:smart_crusher:0.01` + `<<ccr:2cdc115141a5,string,8.0KB>>` ->
`router:excluded:ccr_retrieve`, byte-exact.
- `execute` calling retrieve, MCP JSON envelope:
`router:smart_crusher:0.01` -> `router:excluded:ccr_retrieve`,
byte-exact.
- `functions.exec` calling retrieve, JSON object:
`router:smart_crusher:0.01` -> `router:excluded:ccr_retrieve`,
byte-exact.
- named `mcp__headroom__headroom_retrieve` baseline:
`router:excluded:ccr_retrieve` both before and after (unchanged).
- control, `execute` NOT calling the retrieve tool:
`router:smart_crusher:0.01` both before and after (stays compressible).
Raw outputs: before/after probe JSON in `/tmp/opencode/headroom-pr/`.
- Live session proof (OpenCode V2, patched proxy live on 127.0.0.1:8787,
build 0.40.0-dev + this fix): `headroom_compress` of a 2359-char payload
(hash 252e0dc46447386438bcf077) then `headroom_retrieve` through the
Code Mode `execute` wrapper delivered the full 2359 chars to the model
verbatim; the same flow returned a fresh `<<ccr:...>>` marker before the
fix.
- Not tested: no live Codex client session; the Responses
`custom_tool_call` path is covered at unit level only (through
`unwrap_tool_call_name`). The full tree was run separately: 14 047
passed, 600 skipped, 12 failed - all 12 triaged as pre-existing or
environment-specific (see Testing above).

## Runtime Rollout Safety

- Rollout-managed feature(s): none (no flag added)
- Minimum rollout channel: n/a
- Stable/default behavior changed: yes, deliberately and narrowly -- an
`execute` / `exec` / `functions.exec` call whose script invokes
`headroom_retrieve` now keeps its output verbatim. Wrapper output that
does not invoke the retrieval tool is unchanged (control case above).
- Kill switch / disable path: revert this change; no new config surface
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes (subsets
above)
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- Known parity gap, not introduced here: `SmartCrusher.apply()`'s own
guard (`_build_tool_name_index`) builds its name index without the
shared unwrap, so a direct `SmartCrusher.apply()` caller still misses
orchestrator wrappers -- exactly as it already misses the Hermes
`tool_call` bridge (#3837). The proxy path is unaffected because
ContentRouter's exemption fires first.
- Accepted tradeoff (documented in the code comment): a wrapper payload
that merely mentions `headroom_retrieve(` inside a string (e.g. a shell
command being exec'd) over-protects that one output. Losing compression
on one output is cheap; losing retrieved bytes is not.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3915"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Benoit MELO
2026-10-05 15:18:20 -05:00
committed by GitHub
co-authored by JD Davis
parent b9a0d298cf
commit 6151ed1f52
2 changed files with 486 additions and 1 deletions
+125 -1
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import fnmatch
import json
import re
from collections.abc import Iterable
from dataclasses import InitVar, dataclass, field
from datetime import datetime
@@ -405,8 +406,124 @@ def _load_json_value(raw: Any) -> Any:
return raw
# headroom's own retrieval tool. Kept as a literal (rather than imported from
# headroom.ccr.tool_injection) so this leaf module stays free of the ccr
# package's import graph; mirrors CCR_TOOL_NAME there.
_CCR_RETRIEVE_TOOL_NAME = "headroom_retrieve"
# Orchestrator wrappers. OpenCode V2 Code Mode runs every MCP call inside its
# built-in `execute` tool, and Codex code mode sends calls as `exec` /
# `functions.exec` custom tool calls whose payload is JavaScript. On the wire
# the name is the wrapper -- the inner tool name never reaches the proxy -- so
# when the script invokes the retrieval tool the ccr_retrieve exemption misses
# and the retrieved bytes are re-offloaded into a <<ccr:hash>> marker the agent
# can never redeem (unresolvable retrieval loop, #3563).
#
# Only a call whose own payload invokes the retrieval tool resolves to it: the
# wrapper is never exempted as a whole (an `execute` call running anything else
# stays compressible), and the match keys on the model-authored call arguments
# -- never on the result content (an `original_content` property is data the
# model may echo, not recovery proof). Tradeoff, same family as the
# mcp__<server>__ alias matching above: a wrapper payload that merely mentions
# `headroom_retrieve(` inside a string (e.g. a shell command being exec'd)
# over-protects that one output; losing compression is cheap, losing retrieved
# bytes is not.
#
# Matching runs on decoded script text: the OpenAI wire hands the payload over
# JSON-encoded, where a real newline between the callee and `(` arrives as the
# two characters `\n` and a raw-text scan misses the call.
_ORCHESTRATOR_WRAPPER_NAMES = frozenset({"execute", "exec", "functions.exec"})
# JavaScript call expressions inside a wrapper payload: `headroom_retrieve(`,
# `tools.headroom.headroom_retrieve(`, plus bracket access
# `tools["headroom_retrieve"](` / `tools['mcp__headroom__headroom_retrieve'](`.
# Optional chaining counts as an invocation too -- `headroom_retrieve?.()`,
# `tools?.headroom.headroom_retrieve(...)`, `tools["headroom_retrieve"]?.()`
# -- since the tool still runs when present and its bytes need the exemption.
# Decoded script text carries unescaped quotes; the escaped form of a raw
# payload is tolerated too.
_JS_CALL_CHAIN_RE = re.compile(
r"(?<![\w$.])([A-Za-z_$][\w$]*(?:\??\.[A-Za-z_$][\w$]*)*)\s*(?:\?\.)?\s*\("
)
_JS_BRACKET_KEY_RE = re.compile(
r"""\[\s*\\?(?:"([^"\n\\]{1,200})\\?"|'([^'\n\\]{1,200})\\?')\s*\]"""
r"""\s*(?:\?\.)?\s*\("""
)
def _decoded_strings(value: Any) -> list[str]:
"""Every string nested in a decoded payload (dicts and lists walk
through); scalars contribute nothing."""
if isinstance(value, str):
return [value]
if isinstance(value, dict):
items: Iterable[Any] = value.values()
elif isinstance(value, (list, tuple)):
items = value
else:
return []
strings: list[str] = []
for item in items:
strings.extend(_decoded_strings(item))
return strings
def _orchestrator_invokes_retrieve(name: str, arguments: Any) -> bool:
"""True when an orchestrator wrapper's payload invokes the retrieval tool.
See the comment above ``_ORCHESTRATOR_WRAPPER_NAMES``. ``arguments`` is the
model-authored call payload: JSON text on the OpenAI wire, a decoded dict on
the Anthropic wire, raw JavaScript for a custom_tool_call. JSON text is
decoded before matching so a script newline is seen as a real newline. Any
other value fails closed (the call keeps its wrapper name).
"""
if name not in _ORCHESTRATOR_WRAPPER_NAMES:
return False
texts: list[str]
if isinstance(arguments, str):
decoded = _load_json_value(arguments)
if isinstance(decoded, dict):
texts = _decoded_strings(decoded)
elif decoded is None:
# Not JSON: a raw JavaScript custom_tool_call payload.
texts = [arguments]
else:
# A JSON scalar, string or array: no orchestrator payload shape.
return False
elif isinstance(arguments, dict):
texts = _decoded_strings(arguments)
else:
return False
def _names_retrieve(candidate: str) -> bool:
# A dotted chain resolves on its last segment
# (`tools.headroom.headroom_retrieve`); is_tool_excluded() then applies
# the usual mcp__<server>__ aliases to it.
return is_tool_excluded(candidate.rsplit(".", 1)[-1], (_CCR_RETRIEVE_TOOL_NAME,))
for text in texts:
if _CCR_RETRIEVE_TOOL_NAME not in text:
continue
for match in _JS_CALL_CHAIN_RE.finditer(text):
if _names_retrieve(match.group(1)):
return True
for match in _JS_BRACKET_KEY_RE.finditer(text):
key = next(group for group in match.groups() if group)
if _names_retrieve(key):
return True
return False
def unwrap_tool_call(name: str, arguments: Any) -> tuple[str, Any]:
"""Resolve a Hermes deferred ``tool_call`` wrapper to ``(real_name, real_arguments)``.
"""Resolve a wrapper tool call to ``(effective_name, effective_arguments)``.
Two wrapper families resolve here beyond the pass-through: the Hermes
deferred ``tool_call`` bridge (see comment above), and an orchestrator
wrapper (``execute`` / ``exec`` / ``functions.exec``) whose payload invokes
the retrieval tool -- the latter reports the retrieval tool's own name so
its result keeps the ccr_retrieve exemption (#3563). A Hermes bridge over
an orchestrator call (``tool_call`` -> ``execute`` -> script) resolves
through that same orchestrator rule.
Non-wrapper names pass through with their arguments unchanged. A batch
resolves when every entry names the same tool (Hermes rejects multi-local
@@ -416,6 +533,8 @@ def unwrap_tool_call(name: str, arguments: Any) -> tuple[str, Any]:
means).
"""
if name != _HERMES_TOOL_CALL_WRAPPER:
if _orchestrator_invokes_retrieve(name, arguments):
return _CCR_RETRIEVE_TOOL_NAME, arguments
return name, arguments
payload = _load_json_value(arguments)
if not isinstance(payload, dict):
@@ -444,6 +563,11 @@ def unwrap_tool_call(name: str, arguments: Any) -> tuple[str, Any]:
# Per-call arguments are only meaningful for a single call; a batch keeps
# the wrapper payload so nothing downstream reads one entry as the whole.
real_arguments = entries[0].get("arguments") if len(entries) == 1 else arguments
if _orchestrator_invokes_retrieve(real_name, real_arguments):
# Hermes bridge over an orchestrator wrapper (`tool_call` -> `execute`
# -> script): the effective call is the orchestrator's, so its
# retrieval check decides the name; the bridge alone is never exempted.
return _CCR_RETRIEVE_TOOL_NAME, real_arguments
return real_name, real_arguments
@@ -0,0 +1,361 @@
"""Regression tests: a headroom_retrieve result delivered through an orchestrator
wrapper must keep the ccr_retrieve exemption (#3563).
OpenCode V2 Code Mode runs every MCP call inside its built-in ``execute`` tool,
and Codex code mode sends calls as ``exec`` / ``functions.exec`` custom tool
calls whose payload is JavaScript. On the wire the tool name is the wrapper --
the inner name never reaches the proxy -- so the exemption that keys on the tool
name missed, SmartCrusher re-offloaded the retrieved bytes into a fresh
``<<ccr:hash>>`` marker, and the model could never redeem it (the same
unresolvable retrieval loop class as #1077 / #2698).
The fix resolves such a call to the retrieval tool's own name when -- and only
when -- the call's own payload invokes ``headroom_retrieve``. The wrapper is
never exempted as a whole, and the result content is never consulted (an
``original_content`` property is user-controllable data, not recovery proof).
"""
from __future__ import annotations
import json
from headroom.config import unwrap_tool_call
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
def _get_tokenizer():
from headroom.providers import OpenAIProvider
from headroom.tokenizer import Tokenizer
provider = OpenAIProvider()
token_counter = provider.get_token_counter("gpt-4o")
return Tokenizer(token_counter, "gpt-4o")
def _big_retrieve_json() -> str:
"""The JSON object the retrieval tool returns: big enough to clear the
compression threshold, shaped like the #3563 repro (``original_content``)."""
rows = "\n".join(
f"Row {i:03d}: synthetic recovery check; status=healthy; payload=harmless sample."
for i in range(90)
)
return json.dumps({"original_content": rows})
def _orchestrator_messages(wrapper_name: str, code: str, content: str) -> list[dict]:
"""OpenAI-shape conversation: an outer wrapper call, then its tool result."""
return [
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_ccr_exec",
"type": "function",
"function": {"name": wrapper_name, "arguments": json.dumps({"code": code})},
}
],
},
{"role": "tool", "tool_call_id": "call_ccr_exec", "content": content},
]
RETRIEVE_CODE = (
'const r = await tools.headroom.headroom_retrieve({hash: "abc123def456"});\nreturn r;'
)
NON_RETRIEVE_CODE = 'const r = await tools.bash.bash({command: "ls"});\nreturn r;'
# A real newline between the callee and its `(`. On the OpenAI wire the payload
# is JSON, so this arrives as the two escaped characters `\n` and a scan of the
# raw JSON text misses it; the call must be recognized on the decoded script.
RETRIEVE_CODE_MULTILINE = (
'const r = await tools.headroom.headroom_retrieve\n({hash: "abc123def456"});\nreturn r;'
)
class TestOrchestratorUnwrap:
"""Unit coverage for the identity derivation itself."""
def test_opencode_execute_wrapper_resolves_to_retrieve(self):
name, _args = unwrap_tool_call("execute", json.dumps({"code": RETRIEVE_CODE}))
assert name == "headroom_retrieve"
def test_qualified_mcp_name_inside_script_resolves(self):
code = 'const r = await tools.mcp__headroom__headroom_retrieve({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_bracket_access_inside_script_resolves(self):
code = 'const r = await tools["headroom_retrieve"]({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_execute_wrapper_without_retrieve_call_keeps_name(self):
assert unwrap_tool_call("execute", json.dumps({"code": NON_RETRIEVE_CODE}))[0] == "execute"
def test_non_wrapper_mentioning_retrieve_keeps_name(self):
# A non-orchestrator tool whose arguments merely name the tool is not
# recovery output; only recognized wrappers resolve.
args = '{"command": "grep -n headroom_retrieve( README.md"}'
assert unwrap_tool_call("bash", args)[0] == "bash"
def test_wrapper_payload_mentioning_retrieve_without_a_call_keeps_name(self):
assert unwrap_tool_call("execute", json.dumps({"code": "// see headroom_retrieve"}))[0] == (
"execute"
)
def test_json_encoded_newline_before_call_resolves(self):
"""The OpenAI wire JSON-encodes the payload, so the script's real
newline arrives as the two characters ``\\n``; resolving must happen on
the decoded script text, not on the raw JSON text (#3915 review)."""
code = 'const r = await tools.headroom.headroom_retrieve\n({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_json_encoded_tab_before_call_resolves(self):
code = 'const r = await tools.headroom.headroom_retrieve\t({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_json_encoded_newline_before_bracket_call_resolves(self):
code = 'const r = await tools["headroom_retrieve"]\n({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_anthropic_dict_payload_with_newline_resolves(self):
"""The Anthropic wire hands over a decoded dict; it must be scanned
directly, without a json.dumps round-trip re-encoding the newline."""
args = {"code": 'const r = await tools.headroom.headroom_retrieve\n({hash: "abc"});'}
assert unwrap_tool_call("execute", args)[0] == "headroom_retrieve"
def test_nested_containers_in_dict_payload_resolve(self):
args = {"meta": {"attempt": 1}, "calls": [{"code": RETRIEVE_CODE_MULTILINE}]}
assert unwrap_tool_call("execute", args)[0] == "headroom_retrieve"
def test_raw_javascript_payload_with_newline_resolves(self):
"""A custom_tool_call payload that is not JSON is scanned as-is."""
code = 'const r = await tools.headroom.headroom_retrieve\n({hash: "abc"});'
assert unwrap_tool_call("execute", code)[0] == "headroom_retrieve"
def test_string_values_without_the_name_are_skipped(self):
args = {"note": "no tool call here", "code": RETRIEVE_CODE_MULTILINE}
assert unwrap_tool_call("execute", args)[0] == "headroom_retrieve"
def test_dict_payload_without_string_scripts_keeps_name(self):
args = {"code": 42, "meta": {"attempt": 1}, "flags": [True, None]}
assert unwrap_tool_call("execute", args)[0] == "execute"
def test_non_string_non_dict_arguments_fail_closed(self):
for args in (None, 42, ["headroom_retrieve("]):
assert unwrap_tool_call("execute", args)[0] == "execute"
def test_json_scalar_arguments_fail_closed(self):
assert unwrap_tool_call("execute", "42")[0] == "execute"
def test_unrelated_call_chain_keeps_name(self):
# The mention satisfies the substring prefilter but no chain resolves.
code = "// see headroom_retrieve docs\nrun(1);"
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "execute"
def test_unrelated_bracket_access_keeps_name(self):
code = '// see headroom_retrieve docs\ntools["other"]();'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "execute"
def test_optional_invocation_resolves(self):
"""`retrieve?.(...)` is a valid JavaScript call; its bytes need the
exemption like a plain invocation (#3915 review)."""
code = 'const r = await tools.headroom.headroom_retrieve?.({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_optional_property_access_resolves(self):
code = 'const r = await tools?.headroom.headroom_retrieve({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_optional_bracket_invocation_resolves(self):
code = 'const r = await tools["headroom_retrieve"]?.({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_optional_access_around_bracket_call_resolves(self):
code = 'const r = await tools?.["headroom_retrieve"]?.({hash: "abc"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "headroom_retrieve"
def test_optional_chain_without_retrieve_keeps_name(self):
code = 'const r = await tools?.bash.bash?.({command: "ls"});'
assert unwrap_tool_call("execute", json.dumps({"code": code}))[0] == "execute"
def test_hermes_bridge_over_orchestrator_retrieve_resolves(self):
"""`tool_call` wrapping an `execute` that runs the retrieval tool: the
name must resolve through the nested orchestrator, not stop at
`execute` (#3915 review)."""
args = json.dumps({"name": "execute", "arguments": {"code": RETRIEVE_CODE}})
name, inner = unwrap_tool_call("tool_call", args)
assert name == "headroom_retrieve"
assert inner == {"code": RETRIEVE_CODE}
def test_hermes_bridge_over_orchestrator_without_retrieve_keeps_name(self):
args = {"name": "execute", "arguments": {"code": NON_RETRIEVE_CODE}}
assert unwrap_tool_call("tool_call", args)[0] == "execute"
class TestOrchestratorWrapperCcrRetrieveExemption:
def test_execute_wrapper_retrieve_result_not_recompressed(self):
"""OpenCode V2 Code Mode: the execute wrapper's retrieval result must
pass through verbatim instead of being re-offloaded."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
messages = _orchestrator_messages("execute", RETRIEVE_CODE, content)
result = router.apply(messages, tokenizer)
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
assert tool_msg["content"] == content, (
"headroom_retrieve result behind the execute wrapper was recompressed "
"(unresolvable retrieval loop, #3563)"
)
assert "<<ccr:" not in tool_msg["content"]
assert "router:excluded:ccr_retrieve" in result.transforms_applied
def test_functions_exec_wrapper_retrieve_result_not_recompressed(self):
"""Codex code-mode shape: same guarantee under `functions.exec`."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
code = 'const r = await tools.mcp__headroom__headroom_retrieve({hash: "abc"});\nreturn r;'
messages = _orchestrator_messages("functions.exec", code, content)
result = router.apply(messages, tokenizer)
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
assert tool_msg["content"] == content
assert "router:excluded:ccr_retrieve" in result.transforms_applied
def test_execute_wrapper_multiline_retrieve_result_not_recompressed(self):
"""Codex code-mode shapes can break the call line before the `(`; the
decoded script must still resolve and keep the exemption."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
messages = _orchestrator_messages("execute", RETRIEVE_CODE_MULTILINE, content)
result = router.apply(messages, tokenizer)
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
assert tool_msg["content"] == content, (
"multiline headroom_retrieve call behind the execute wrapper was recompressed "
"(unresolvable retrieval loop, #3563)"
)
assert "<<ccr:" not in tool_msg["content"]
assert "router:excluded:ccr_retrieve" in result.transforms_applied
def test_anthropic_execute_wrapper_multiline_retrieve_not_recompressed(self):
"""Anthropic-shape orchestrator call: the tool_use input is a decoded
dict carrying a real newline, with no JSON text to scan."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
messages = [
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "toolu_exec_multiline",
"name": "execute",
"input": {"code": RETRIEVE_CODE_MULTILINE},
}
],
},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_exec_multiline",
"content": content,
}
],
},
]
result = router.apply(messages, tokenizer)
tool_result_block = result.messages[1]["content"][0]
assert tool_result_block["content"] == content
assert "<<ccr:" not in tool_result_block["content"]
assert "router:excluded:ccr_retrieve" in result.transforms_applied
def test_execute_wrapper_multiline_without_retrieve_still_compressed(self):
"""Negative control: a multiline script that does not call the
retrieval tool keeps the wrapper name and stays compressible."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
code = "const r = await tools.bash.bash\n({command: 'ls'});\nreturn r;"
messages = _orchestrator_messages("execute", code, content)
result = router.apply(messages, tokenizer)
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
assert "router:excluded:ccr_retrieve" not in result.transforms_applied
assert tool_msg["content"] != content or result.tokens_after < result.tokens_before
def test_execute_wrapper_without_retrieve_call_still_compressed(self):
"""The exemption stays narrow: an execute call that does not invoke the
retrieval tool has ordinary, compressible output."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
messages = _orchestrator_messages("execute", NON_RETRIEVE_CODE, content)
result = router.apply(messages, tokenizer)
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
assert "router:excluded:ccr_retrieve" not in result.transforms_applied
assert tool_msg["content"] != content or result.tokens_after < result.tokens_before
def test_hermes_bridge_over_execute_retrieve_result_not_recompressed(self):
"""A Hermes `tool_call` over an `execute` running the retrieval tool:
the paired output must keep the exemption (nested wrapper, #3915
review)."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
messages = [
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_ccr_hermes",
"type": "function",
"function": {
"name": "tool_call",
"arguments": json.dumps(
{"name": "execute", "arguments": {"code": RETRIEVE_CODE}}
),
},
}
],
},
{"role": "tool", "tool_call_id": "call_ccr_hermes", "content": content},
]
result = router.apply(messages, tokenizer)
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
assert tool_msg["content"] == content, (
"nested Hermes/execute retrieval result was recompressed "
"(unresolvable retrieval loop, #3563)"
)
assert "<<ccr:" not in tool_msg["content"]
assert "router:excluded:ccr_retrieve" in result.transforms_applied
def test_execute_wrapper_optional_call_retrieve_not_recompressed(self):
"""Optional-chaining invocation (`retrieve?.(...)`) keeps the exemption
through the router (#3915 review)."""
content = _big_retrieve_json()
router = ContentRouter(ContentRouterConfig(min_section_tokens=10))
tokenizer = _get_tokenizer()
code = 'const r = await tools.headroom.headroom_retrieve?.({hash: "abc"});\nreturn r;'
messages = _orchestrator_messages("execute", code, content)
result = router.apply(messages, tokenizer)
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
assert tool_msg["content"] == content
assert "router:excluded:ccr_retrieve" in result.transforms_applied