mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(ccr): unwrap Hermes batched tool_call so headroom_retrieve stays exempt (#3839)
## Description Fixes #3837. `headroom_retrieve` results issued through Hermes' deferred-tool bridge were re-compressed into `<<ccr:…>>` markers, making retrieved content unreachable. **Root cause.** Hermes' `tool_call` bridge advertises a **batch** shape, `{"calls": [{"name", "arguments"}, …]}`, and only tolerates the legacy `{"name", "arguments"}` form (Hermes `tools/tool_search_validation.py::normalize_tool_call_entries`). `unwrap_tool_call_name` read only the legacy form, so every batched call resolved to the wrapper name `tool_call`: - The `ccr_retrieve` guard never fired. SmartCrusher re-compressed the retrieved rows into a marker whose hash matches the original. - `--protect-tool-results` and exclude lists silently no-op'd for **every** deferred tool. Since late August, Hermes also defers its core tools (terminal, file reads) behind the same bridge. Reproduced end to end through a real proxy (token mode, `--target-ratio 0.7`) against a fake Anthropic upstream. A Hermes batch-shaped retrieve of a compressed JSON output: | | retrieve result forwarded upstream | PERF transforms | |---|---|---| | main | `{"hash":…,"original_content":"<<ccr:3c73cd7e951b,string,14.4KB>>",…}` | `router:tool_result:smart_crusher` | | this PR | verbatim, on every later turn too | `router:excluded:ccr_retrieve` | The reporter's "0.38.0 works, 0.39.1 doesn't" is most likely a Hermes-side change to the advertised batch shape, not a Headroom regression. v0.38.0's unwrap is identical and fails the same way on the batch shape. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/config.py`: new `unwrap_tool_call(name, arguments) -> (real_name, real_arguments)`, mirroring Hermes' normalizer: - `calls` may be a list, a bare object or a JSON string; the legacy shape is still handled. - A batch resolves only when every entry names the same tool. Malformed or mixed-name batches fail open to the wrapper name, as before. - `unwrap_tool_call_name` now delegates to it. - `content_router._build_tool_name_map`: uses the unwrapped **arguments** too, so read protection (`_tool_call_commands`) and the relevance query see the real call, not the wrapper envelope. - `handlers/openai.py` (Responses units path): `headroom_retrieve` detection uses the shared `is_tool_excluded` matcher instead of `== / endswith("__headroom_retrieve")`. That also covers Hermes' `mcp_srv_x` and OpenCode's `headroom_headroom_retrieve`. The import is hoisted to module level. ## Testing - [x] Unit tests pass. 270 tests across `test_hermes_tool_call_unwrap`, `test_openai_responses_*`, `test_content_router*`, `test_tool_name_aliases` and the exclude-tools suites. - [x] Linting passes: CI's `ruff 0.16.8` check plus `format --check`. - [x] Type checking passes: `mypy` on the three changed modules. - [x] New tests added. 8 of them fail on the unfixed code, including an end-to-end `ContentRouter.apply()` case with a non-retrieve control. The existing Responses retrieve-protection test is parametrized over the alias and bridge forms. - [x] Manual testing performed (proxy + fake upstream, table above). ## Follow-up (not in this PR) A separate hole turned up while reproducing: in token mode the session compression cache (`CompressionCache.apply_cached`, via `session_engine.prepare_turn`) swaps a tool result's content by **content hash** before the router runs, without looking at the tool name. So a retrieve result whose bytes equal an earlier compressed output gets the cached compressed form back, even though the router logs `excluded:ccr_retrieve`. The MCP retrieve returns a JSON envelope, so this doesn't trigger for it in practice today. It will for any client that returns the raw original. It is present in 0.38.0 too. It will be filed separately. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
25274310ca
commit
2f076687b0
+65
-19
@@ -381,32 +381,78 @@ def _tool_name_aliases(name: str) -> tuple[str, ...]:
|
||||
# Hermes Agent's deferred-tool bridge. Hermes loads on-demand tools via a
|
||||
# `tool_search`/`tool_describe`/`tool_call` indirection; on the wire the
|
||||
# emitted tool call is named `tool_call` and the REAL tool name lives in the
|
||||
# arguments payload (`{"name": "...", "arguments": {...}}`). Tool exclusion /
|
||||
# protect lists match on the real name, so we must unwrap this bridge before
|
||||
# building the tool_call_id -> name map, or whitelists silently no-op for all
|
||||
# deferred tools.
|
||||
# arguments payload. Tool exclusion / protect lists match on the real name, so
|
||||
# we must unwrap this bridge before building the tool_call_id -> name map, or
|
||||
# whitelists silently no-op for all deferred tools.
|
||||
#
|
||||
# Two payload shapes, mirroring Hermes' own `normalize_tool_call_entries`
|
||||
# (tools/tool_search_validation.py):
|
||||
# - batch (advertised): {"calls": [{"name": ..., "arguments": {...}}, ...]}
|
||||
# where `calls` may also arrive as a JSON string or a bare object;
|
||||
# - legacy single: {"name": ..., "arguments": {...}}.
|
||||
# Only the legacy shape was handled before, so once Hermes advertised the
|
||||
# batch shape every deferred tool (headroom_retrieve included, #3837) resolved
|
||||
# to the wrapper name and its exclusion stopped firing.
|
||||
_HERMES_TOOL_CALL_WRAPPER = "tool_call"
|
||||
|
||||
|
||||
def _load_json_value(raw: Any) -> Any:
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
return json.loads(raw)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
return raw
|
||||
|
||||
|
||||
def unwrap_tool_call(name: str, arguments: Any) -> tuple[str, Any]:
|
||||
"""Resolve a Hermes deferred ``tool_call`` wrapper to ``(real_name, real_arguments)``.
|
||||
|
||||
Non-wrapper names pass through with their arguments unchanged. A batch
|
||||
resolves when every entry names the same tool (Hermes rejects multi-local
|
||||
batches, so in practice a batch of more than one is connector-only); a
|
||||
mixed batch has no single real name. Malformed or ambiguous wrappers fail
|
||||
open and return ``(name, arguments)`` unchanged (caller decides what that
|
||||
means).
|
||||
"""
|
||||
if name != _HERMES_TOOL_CALL_WRAPPER:
|
||||
return name, arguments
|
||||
payload = _load_json_value(arguments)
|
||||
if not isinstance(payload, dict):
|
||||
return name, arguments
|
||||
|
||||
calls = payload.get("calls")
|
||||
if calls is None:
|
||||
entries: list[Any] = [payload]
|
||||
else:
|
||||
calls = _load_json_value(calls)
|
||||
if isinstance(calls, dict):
|
||||
calls = [calls]
|
||||
if not isinstance(calls, list) or not calls:
|
||||
return name, arguments
|
||||
entries = calls
|
||||
|
||||
names: set[str] = set()
|
||||
for entry in entries:
|
||||
inner = entry.get("name") if isinstance(entry, dict) else None
|
||||
if not isinstance(inner, str) or not inner.strip():
|
||||
return name, arguments
|
||||
names.add(inner.strip())
|
||||
if len(names) != 1:
|
||||
return name, arguments
|
||||
real_name = names.pop()
|
||||
# Per-call arguments are only meaningful for a single call; a batch keeps
|
||||
# the wrapper payload so nothing downstream reads one entry as the whole.
|
||||
real_arguments = entries[0].get("arguments") if len(entries) == 1 else arguments
|
||||
return real_name, real_arguments
|
||||
|
||||
|
||||
def unwrap_tool_call_name(name: str, arguments: Any) -> str:
|
||||
"""Extract the real tool name from a Hermes deferred ``tool_call`` wrapper.
|
||||
|
||||
Non-wrapper names pass through unchanged. Malformed/unparseable wrappers
|
||||
fail open and return the wrapper name (caller decides what that means).
|
||||
See :func:`unwrap_tool_call`; this returns only the name.
|
||||
"""
|
||||
if name != _HERMES_TOOL_CALL_WRAPPER:
|
||||
return name
|
||||
raw = arguments
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
raw = json.loads(raw)
|
||||
except (ValueError, TypeError):
|
||||
return name
|
||||
if isinstance(raw, dict):
|
||||
inner = raw.get("name")
|
||||
if isinstance(inner, str) and inner.strip():
|
||||
return inner.strip()
|
||||
return name
|
||||
return unwrap_tool_call(name, arguments)[0]
|
||||
|
||||
|
||||
def is_tool_excluded(name: str, exclude_tools: Iterable[str]) -> bool:
|
||||
|
||||
@@ -56,7 +56,7 @@ import httpx
|
||||
|
||||
from headroom.agent_savings import proxy_pipeline_kwargs
|
||||
from headroom.ccr.marker_resolution import resolve_markers_in_response
|
||||
from headroom.config import unwrap_tool_call_name
|
||||
from headroom.config import is_tool_excluded, unwrap_tool_call_name
|
||||
from headroom.copilot_auth import (
|
||||
apply_copilot_api_auth,
|
||||
build_copilot_upstream_url,
|
||||
@@ -2177,9 +2177,10 @@ class OpenAIHandlerMixin:
|
||||
name = unwrap_tool_call_name(name, item.get("arguments") or item.get("input"))
|
||||
if isinstance(name, str) and isinstance(call_id, str) and call_id:
|
||||
function_name_by_call_id[call_id] = name
|
||||
if isinstance(name, str) and (
|
||||
name == "headroom_retrieve" or name.endswith("__headroom_retrieve")
|
||||
):
|
||||
# Same matcher ContentRouter uses for its ccr_retrieve guard, so the
|
||||
# MCP alias forms (mcp__srv__x, Hermes' mcp_srv_x, OpenCode's
|
||||
# headroom_headroom_retrieve) are recognized here too.
|
||||
if isinstance(name, str) and is_tool_excluded(name, ("headroom_retrieve",)):
|
||||
if isinstance(call_id, str) and call_id:
|
||||
headroom_retrieve_call_ids.add(call_id)
|
||||
|
||||
@@ -2191,7 +2192,6 @@ class OpenAIHandlerMixin:
|
||||
DEFAULT_BYTE_EXACT_EXCLUDE_TOOLS,
|
||||
DEFAULT_EXCLUDE_TOOLS,
|
||||
DEFAULT_VERBATIM_EXCLUDE_TOOLS,
|
||||
is_tool_excluded,
|
||||
)
|
||||
|
||||
router_exclude_tools = getattr(router.config, "exclude_tools", None)
|
||||
|
||||
@@ -60,7 +60,7 @@ from ..config import (
|
||||
RelevanceScorerConfig,
|
||||
TransformResult,
|
||||
is_tool_excluded,
|
||||
unwrap_tool_call_name,
|
||||
unwrap_tool_call,
|
||||
)
|
||||
from ..parser import CCR_RETRIEVAL_MARKER_RE
|
||||
from ..tokenizer import Tokenizer
|
||||
@@ -5069,16 +5069,17 @@ class ContentRouter(Transform):
|
||||
tc_id = tc.get("id", "")
|
||||
fn = tc.get("function", {})
|
||||
name = fn.get("name", "")
|
||||
call_args = fn.get("arguments")
|
||||
if name:
|
||||
# Hermes deferred tools arrive wrapped as `tool_call`
|
||||
# with the real name inside the arguments payload.
|
||||
name = unwrap_tool_call_name(name, fn.get("arguments"))
|
||||
# with the real name and arguments inside the payload.
|
||||
name, call_args = unwrap_tool_call(name, call_args)
|
||||
if tc_id and name:
|
||||
mapping[tc_id] = name
|
||||
args = _tool_call_args_text(fn.get("arguments"))
|
||||
args = _tool_call_args_text(call_args)
|
||||
if args:
|
||||
args_map[tc_id] = args
|
||||
command = _tool_call_command_text(fn.get("arguments"))
|
||||
command = _tool_call_command_text(call_args)
|
||||
if command:
|
||||
commands_map[tc_id] = command
|
||||
|
||||
@@ -5089,16 +5090,17 @@ class ContentRouter(Transform):
|
||||
if isinstance(block, dict) and block.get("type") == "tool_use":
|
||||
tc_id = block.get("id", "")
|
||||
name = block.get("name", "")
|
||||
call_input = block.get("input")
|
||||
if name:
|
||||
# Hermes deferred tools arrive wrapped as `tool_call`
|
||||
# with the real name inside the input payload.
|
||||
name = unwrap_tool_call_name(name, block.get("input"))
|
||||
# with the real name and arguments inside the input.
|
||||
name, call_input = unwrap_tool_call(name, call_input)
|
||||
if tc_id and name:
|
||||
mapping[tc_id] = name
|
||||
args = _tool_call_args_text(block.get("input"))
|
||||
args = _tool_call_args_text(call_input)
|
||||
if args:
|
||||
args_map[tc_id] = args
|
||||
command = _tool_call_command_text(block.get("input"))
|
||||
command = _tool_call_command_text(call_input)
|
||||
if command:
|
||||
commands_map[tc_id] = command
|
||||
|
||||
|
||||
@@ -13,9 +13,12 @@ into `ContentRouter._build_tool_name_map` (OpenAI + Anthropic paths).
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from headroom.config import (
|
||||
DEFAULT_EXCLUDE_TOOLS,
|
||||
is_tool_excluded,
|
||||
unwrap_tool_call,
|
||||
unwrap_tool_call_name,
|
||||
)
|
||||
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
|
||||
@@ -195,3 +198,160 @@ def test_build_tool_name_map_exclusion_after_unwrap() -> None:
|
||||
mapping = router._build_tool_name_map(messages)
|
||||
assert mapping["call_rf_1"] == "read_file"
|
||||
assert is_tool_excluded(mapping["call_rf_1"], router.config.exclude_tools or set()) is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Batch shape: {"calls": [{"name", "arguments"}, ...]} (#3837)
|
||||
#
|
||||
# Hermes advertises `tool_call(calls)`; the legacy top-level {"name", ...} form
|
||||
# is only tolerated. Mirrors Hermes' normalize_tool_call_entries.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_unwrap_batch_of_one_dict() -> None:
|
||||
args = {"calls": [{"name": "headroom_retrieve", "arguments": {"hash": "abc"}}]}
|
||||
assert unwrap_tool_call("tool_call", args) == ("headroom_retrieve", {"hash": "abc"})
|
||||
|
||||
|
||||
def test_unwrap_batch_of_one_json_string() -> None:
|
||||
args = '{"calls": [{"name": "read_file", "arguments": {"path": "/x"}}]}'
|
||||
assert unwrap_tool_call("tool_call", args) == ("read_file", {"path": "/x"})
|
||||
|
||||
|
||||
def test_unwrap_batch_calls_as_json_string() -> None:
|
||||
args = {"calls": '[{"name": "web_search", "arguments": {"query": "q"}}]'}
|
||||
assert unwrap_tool_call_name("tool_call", args) == "web_search"
|
||||
|
||||
|
||||
def test_unwrap_batch_calls_as_bare_object() -> None:
|
||||
args = {"calls": {"name": "web_search", "arguments": {}}}
|
||||
assert unwrap_tool_call_name("tool_call", args) == "web_search"
|
||||
|
||||
|
||||
def test_unwrap_batch_same_name_keeps_wrapper_arguments() -> None:
|
||||
args = {
|
||||
"calls": [
|
||||
{"name": "connectors__gh__search", "arguments": {"q": "a"}},
|
||||
{"name": "connectors__gh__search", "arguments": {"q": "b"}},
|
||||
]
|
||||
}
|
||||
assert unwrap_tool_call("tool_call", args) == ("connectors__gh__search", args)
|
||||
|
||||
|
||||
def test_unwrap_batch_mixed_names_fails_open() -> None:
|
||||
args = {
|
||||
"calls": [
|
||||
{"name": "connectors__gh__search", "arguments": {}},
|
||||
{"name": "connectors__jira__search", "arguments": {}},
|
||||
]
|
||||
}
|
||||
assert unwrap_tool_call("tool_call", args) == ("tool_call", args)
|
||||
|
||||
|
||||
def test_unwrap_batch_malformed_fails_open() -> None:
|
||||
for args in (
|
||||
{"calls": []},
|
||||
{"calls": "not json"},
|
||||
{"calls": [{"arguments": {}}]},
|
||||
{"calls": ["headroom_retrieve"]},
|
||||
):
|
||||
assert unwrap_tool_call("tool_call", args) == ("tool_call", args)
|
||||
|
||||
|
||||
def test_unwrap_legacy_shape_returns_inner_arguments() -> None:
|
||||
args = {"name": "read_file", "arguments": {"path": "/x"}}
|
||||
assert unwrap_tool_call("tool_call", args) == ("read_file", {"path": "/x"})
|
||||
|
||||
|
||||
def test_unwrap_plain_name_keeps_arguments() -> None:
|
||||
assert unwrap_tool_call("Read", {"file_path": "/x"}) == ("Read", {"file_path": "/x"})
|
||||
|
||||
|
||||
def test_build_tool_name_map_batch_shape_both_formats() -> None:
|
||||
batch = {"calls": [{"name": "terminal", "arguments": {"command": "cat a.py"}}]}
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_b1",
|
||||
"type": "function",
|
||||
"function": {"name": "tool_call", "arguments": json.dumps(batch)},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "tool_use", "id": "toolu_b2", "name": "tool_call", "input": batch}
|
||||
],
|
||||
},
|
||||
]
|
||||
router = _router()
|
||||
mapping = router._build_tool_name_map(messages)
|
||||
assert mapping == {"call_b1": "terminal", "toolu_b2": "terminal"}
|
||||
# The per-call arguments, not the wrapper envelope, feed read protection
|
||||
# and the relevance query.
|
||||
assert router._tool_call_commands == {"call_b1": "cat a.py", "toolu_b2": "cat a.py"}
|
||||
assert router._tool_call_args["toolu_b2"] == "cat a.py"
|
||||
|
||||
|
||||
def _retrieve_rows_payload() -> str:
|
||||
rows = [
|
||||
{"id": i, "name": f"item-{i}", "status": "ok" if i % 3 else "fail", "value": i * 7}
|
||||
for i in range(300)
|
||||
]
|
||||
return json.dumps(rows)
|
||||
|
||||
|
||||
def _anthropic_retrieve_conversation(tool_use: dict, payload: str) -> list[dict]:
|
||||
return [
|
||||
{"role": "user", "content": "get the data"},
|
||||
{"role": "assistant", "content": [tool_use]},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [{"type": "tool_result", "tool_use_id": tool_use["id"], "content": payload}],
|
||||
},
|
||||
{"role": "assistant", "content": "ok"},
|
||||
{"role": "user", "content": "continue"},
|
||||
]
|
||||
|
||||
|
||||
def test_batch_wrapped_headroom_retrieve_result_is_not_recompressed() -> None:
|
||||
"""#3837: a retrieve issued through Hermes' batch bridge must pass through.
|
||||
|
||||
Before the fix the name map resolved to `tool_call`, the ccr_retrieve guard
|
||||
never fired, and SmartCrusher re-compressed the retrieved rows back into a
|
||||
marker, leaving the original unreachable.
|
||||
"""
|
||||
from headroom.providers import AnthropicProvider
|
||||
from headroom.tokenizer import Tokenizer
|
||||
|
||||
tokenizer = Tokenizer(AnthropicProvider().get_token_counter("claude-sonnet-4-5"))
|
||||
payload = _retrieve_rows_payload()
|
||||
for retrieve_name in ("headroom_retrieve", "mcp_headroom_headroom_retrieve"):
|
||||
tool_use = {
|
||||
"type": "tool_use",
|
||||
"id": "toolu_retrieve",
|
||||
"name": "tool_call",
|
||||
"input": {"calls": [{"name": retrieve_name, "arguments": {"hash": "abc123"}}]},
|
||||
}
|
||||
result = ContentRouter(ContentRouterConfig()).apply(
|
||||
_anthropic_retrieve_conversation(tool_use, payload), tokenizer
|
||||
)
|
||||
assert result.messages[2]["content"][0]["content"] == payload, retrieve_name
|
||||
assert "router:excluded:ccr_retrieve" in result.transforms_applied, retrieve_name
|
||||
|
||||
# Control: the same payload behind a non-retrieve deferred tool does
|
||||
# compress, so the assertion above is not vacuous.
|
||||
control = {
|
||||
"type": "tool_use",
|
||||
"id": "toolu_retrieve",
|
||||
"name": "tool_call",
|
||||
"input": {"calls": [{"name": "mcp_db_query", "arguments": {"q": "all"}}]},
|
||||
}
|
||||
result = ContentRouter(ContentRouterConfig()).apply(
|
||||
_anthropic_retrieve_conversation(control, payload), tokenizer
|
||||
)
|
||||
assert result.messages[2]["content"][0]["content"] != payload
|
||||
|
||||
@@ -587,7 +587,22 @@ def test_openai_responses_adapter_accepts_empty_input_list():
|
||||
assert strategy_chain == []
|
||||
|
||||
|
||||
def test_openai_responses_adapter_preserves_headroom_retrieve_outputs():
|
||||
@pytest.mark.parametrize(
|
||||
("name", "arguments"),
|
||||
[
|
||||
("mcp__headroom__headroom_retrieve", "{}"),
|
||||
# Hermes' single-underscore MCP alias and OpenCode's doubled prefix,
|
||||
# which the old ``== / endswith("__headroom_retrieve")`` check missed.
|
||||
("mcp_headroom_headroom_retrieve", "{}"),
|
||||
("headroom_headroom_retrieve", "{}"),
|
||||
# Hermes deferred-tool bridge, batch shape (#3837).
|
||||
(
|
||||
"tool_call",
|
||||
'{"calls": [{"name": "headroom_retrieve", "arguments": {"hash": "abc"}}]}',
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_openai_responses_adapter_preserves_headroom_retrieve_outputs(name, arguments):
|
||||
router = ContentRouter()
|
||||
|
||||
def compress(self, content: str, **_kwargs):
|
||||
@@ -606,8 +621,8 @@ def test_openai_responses_adapter_preserves_headroom_retrieve_outputs():
|
||||
{
|
||||
"type": "function_call",
|
||||
"call_id": "call_retrieve",
|
||||
"name": "mcp__headroom__headroom_retrieve",
|
||||
"arguments": "{}",
|
||||
"name": name,
|
||||
"arguments": arguments,
|
||||
},
|
||||
{
|
||||
"type": "function_call_output",
|
||||
|
||||
Reference in New Issue
Block a user