mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(smart-crusher): preserve wrapped retrieval results in apply (#4149)
## Current-head security-gate integration Head `17ac399ac38930a265d8e5108c703110cd3e2ebb` incorporates upstream main `49751ec4` (#4178). The upstream change is Cargo.lock only; contributor logic and behavioral regressions are unchanged. This section supersedes older current-head/check-status references below; earlier runtime evidence is retained as historical implementation qualification. Normal commit/pre-push hooks passed without bypass: Rust formatting, warnings-as-errors Clippy, complete workspace tests, native editable rebuild/import verification, **181 critical Python tests**, and branch-range commitlint. Fresh local `cargo audit --file Cargo.lock` also passed (546 dependencies, no findings). Hosted checks for this new head must complete independently; local results are not a claim that all hosted checks are green. ## Description Closes #4147. Related: #3563, #3837 and the already merged router fix #3915. Close the remaining direct Python `SmartCrusher.apply()` parity gap: resolve originating wrapper calls with the existing shared `unwrap_tool_call_name` parser before applying the CCR retrieval exemption. This is not a duplicate proxy/router fix, a blanket wrapper exclusion, or a new content-based recovery heuristic. ## Type of Change - [x] Bug fix (non-breaking correction) - [x] Documentation update ## Changes Made - Resolve OpenAI originating function arguments and Anthropic tool-use input to their effective tool names in the existing call-ID index. - Preserve null-function handling, missing-ID/name behavior and the shared parser's conservative ambiguity/malformed-input rules. - Add four real-native regression cases: OpenAI/Anthropic × Hermes `tool_call`/`functions.exec`, each checking byte-exact retrieval output and a genuinely compressed ordinary wrapper-output control. - Remove old tautological compression assertions and retain named-retrieval consumer coverage. - Document direct `apply()` recovery identity, the need for originating call history, and the context-free `crush()` limit. Native compression, thresholds, CCR storage and the parser itself are unchanged. ## Testing - [x] Real native regression and shared wrapper safety suites pass - [x] Ruff passes on the two changed Python files - [x] Scoped mypy passes on the changed source file - [x] Actual rebuilt native API and real tokenizer exercised before and after ### Test Output ```text Before: 4 wrapped-recovery regressions failed at byte preservation. After: 65 passed, 1 warning in 126.20s Ruff: All checks passed! mypy: Success: no issues found in 1 source file ``` The warning is the existing Anthropic tokenizer approximation in an unchanged Hermes test. The scoped suite comprises the direct retrieval exemption tests plus Hermes and orchestrator-wrapper safety coverage; this is not a full-Python-suite claim. Normal commit/push hooks completed without bypass: Rust fmt, workspace Clippy with `-D warnings`, workspace Rust tests, native extension build/install/import verification, critical Python suite (**177 passed, 4 skipped, 1 warning in 25.48s**) and commitlint. The ordinary push succeeded with the genuine installed ORT library explicitly resolved. No native backend or gate threshold changed. ### Review follow-up verification The non-string-name caller correction is in `40fa055854b1c4b60da84f78d534976bc3fd40fb`; the reviewed thread is resolved. Verification against that exact published Python source: **83 passed, 1 existing Anthropic approximation warning in 16.44s**, across direct retrieval, Hermes and orchestrator safety suites. Changed-file Ruff and scoped mypy passed. A separate real native API smoke reused the genuine unchanged Rust extension and real OpenAI tokenizer: all four wire/wrapper cases returned exactly 27,490 recovery bytes, and all ordinary controls still compressed. An OpenAI `name=["Bash"]` returned its original `"ok"` result. Anthropic malformed names are tested at the identity-index boundary: the bundled tokenizer rejects such malformed tool-use names earlier, and this PR does not change or claim that tokenizer behavior. No mocked native backend, live provider request, or live retrieval-store claim. ## Real Behavior Proof - Environment: Linux x86_64, Python 3.13.14, latest-main base `976aa714ef3e19174dc0cb586cee7b20d064e4ef`, freshly built `headroom/_core.abi3.so`, real OpenAI `gpt-4o` tokenizer. Resolve the installed genuine `libonnxruntime.so.1.26.0` via `ORT_DYLIB_PATH`; no native fallback, mocked compressor or provider request. - Exact command / steps: run `python -m pytest -q tests/test_transforms/test_smart_crusher_ccr_retrieve_exemption.py tests/test_hermes_tool_call_unwrap.py tests/test_orchestrator_wrapper_ccr_retrieve.py`, Ruff on the two changed files and mypy on `headroom/transforms/smart_crusher.py`. Separately call the production native-backed `SmartCrusher.apply()` with a 27,490-byte synthetic recovery result and an equally sized ordinary result; originating wrapper arguments invoke retrieval only for the recovery call. Exercise both wire formats and both wrappers. The isolated reproduction uses the existing test configuration `min_tokens_to_crush=0` unchanged before/after; production/native thresholds are not changed. - Observed result: before, all four recovery payloads became CCR-marked 505/509-byte outputs. After, all four returned exactly 27,490 bytes, matching the original byte for byte, with no new recovery CCR marker. Every ordinary wrapper result still compressed. Copied runtime results are below. - Not tested: a live provider session, proxy HTTP route, real client integration, or a complete retrieval loop. This is a real direct-library reproduction using synthetic returned recovery bytes, not a claim that a synthetic hash was retrieved from a live store. | Wire | Wrapper | Recovered bytes before | Recovered bytes after | Exact after | Ordinary output compressed after | |---|---|---:|---:|---|---| | OpenAI | `tool_call` | 505 | 27490 | Yes | Yes | | OpenAI | `functions.exec` | 505 | 27490 | Yes | Yes | | Anthropic | `tool_call` | 509 | 27490 | Yes | Yes | | Anthropic | `functions.exec` | 509 | 27490 | Yes | Yes | ## Runtime Rollout Safety - Rollout-managed feature(s): None; existing originating-call identity and CCR exemption only. - Minimum rollout channel: Stable; existing rollout controls unchanged. - Stable/default behavior changed: Direct `apply()` no longer re-compresses recognized wrapped recovery results. Unrelated wrapper output still compresses; malformed/ambiguous wrappers retain normal shared-parser behavior. - Kill switch / disable path: No new switch; revert this identity-map correction to restore prior direct-transform behavior. - Unsafe override required: None; runtime qualification used only synthetic local messages. - Qualification impact: Both wire formats, both wrapper families, real native output/control behavior and shared parser/router safety coverage qualified. Live-client/provider/proxy limits remain explicit above. - Rollback path: Revert the code/tests/docs to base `976aa714ef3e19174dc0cb586cee7b20d064e4ef`; no data/schema migration. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] Shared wrapper parsing reused, no second identity convention - [x] Documentation updated - [x] Native and existing wrapper safety regressions pass - [x] No new source warnings, dependencies or native policy changes - [x] `CHANGELOG.md` untouched; Conventional Commit PR title used --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JerrettDavis
parent
2721f06044
commit
9762a14a40
@@ -73,6 +73,22 @@ print(result.strategy) # e.g. "lossless:table(...)" or "smart_sample(1000->15)"
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
### Recovered tool output in the Python message transform
|
||||
|
||||
`SmartCrusher.apply(messages, tokenizer)` preserves recognized
|
||||
`headroom_retrieve` results byte for byte when their call ID matches the
|
||||
originating assistant call. The originating identity is resolved with the
|
||||
shared wrapper parser for both OpenAI `tool_calls`/`arguments` and Anthropic
|
||||
`tool_use`/`input`, including Hermes `tool_call` bridges and retrieval calls
|
||||
inside `execute`, `exec`, or `functions.exec`.
|
||||
|
||||
Other wrapper output remains compressible; an unrecognized or ambiguous
|
||||
wrapper does not gain a blanket recovery exemption. Keep the originating
|
||||
assistant call in the message history so the result can be identified.
|
||||
The context-free `crush(json_string)` API has no originating-call history and
|
||||
does not provide this message-level exemption.
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
<Tabs groupId="lang" items={['TypeScript', 'Python']}>
|
||||
|
||||
@@ -52,7 +52,7 @@ from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from ..ccr.tool_injection import CCR_TOOL_NAME
|
||||
from ..config import CCRConfig, TransformResult, is_tool_excluded
|
||||
from ..config import CCRConfig, TransformResult, is_tool_excluded, unwrap_tool_call_name
|
||||
from ..tokenizer import Tokenizer
|
||||
from ..utils import compute_short_hash, create_tool_digest_marker, deep_copy_messages
|
||||
from .base import Transform
|
||||
@@ -129,7 +129,7 @@ def strip_ccr_sentinels(items: Any) -> Any:
|
||||
|
||||
|
||||
def _build_tool_name_index(messages: list[dict[str, Any]]) -> dict[str, str]:
|
||||
"""Map tool_call_id/tool_use_id → tool name across OpenAI + Anthropic formats.
|
||||
"""Map tool_call_id/tool_use_id → effective tool name in both wire formats.
|
||||
|
||||
Skips entries where id or name is missing; those calls still crush, but
|
||||
won't contribute a tool-name to the ``smart_crush`` tag.
|
||||
@@ -140,9 +140,10 @@ def _build_tool_name_index(messages: list[dict[str, Any]]) -> dict[str, str]:
|
||||
continue
|
||||
for tc in msg.get("tool_calls") or []:
|
||||
tc_id = tc.get("id")
|
||||
name = (tc.get("function") or {}).get("name")
|
||||
if tc_id and name:
|
||||
index[tc_id] = name
|
||||
function = tc.get("function") or {}
|
||||
name = function.get("name")
|
||||
if tc_id and isinstance(name, str) and name:
|
||||
index[tc_id] = unwrap_tool_call_name(name, function.get("arguments"))
|
||||
content = msg.get("content")
|
||||
if isinstance(content, list):
|
||||
for block in content:
|
||||
@@ -150,8 +151,8 @@ def _build_tool_name_index(messages: list[dict[str, Any]]) -> dict[str, str]:
|
||||
continue
|
||||
bid = block.get("id")
|
||||
name = block.get("name")
|
||||
if bid and name:
|
||||
index[bid] = name
|
||||
if bid and isinstance(name, str) and name:
|
||||
index[bid] = unwrap_tool_call_name(name, block.get("input"))
|
||||
return index
|
||||
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ that content again → new <<ccr:hash>> marker → infinite retrieval loop.
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from copy import deepcopy
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -80,29 +81,6 @@ class TestHeadroomRetrieveExemptionOpenAI:
|
||||
# No CCR transforms should have fired
|
||||
assert not any("smart_crush" in t for t in result.transforms_applied)
|
||||
|
||||
def test_non_retrieve_tool_still_compressed(self):
|
||||
"""Normal tool results are still compressed — exemption is narrow."""
|
||||
content = _big_content()
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_xyz",
|
||||
"function": {"name": "Bash", "arguments": '{"cmd":"ls"}'},
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "call_xyz", "content": content},
|
||||
]
|
||||
crusher = _make_crusher(min_tokens=0)
|
||||
tokenizer = _get_tokenizer()
|
||||
result = crusher.apply(messages, tokenizer)
|
||||
|
||||
tool_msg = result.messages[1]
|
||||
# Content should have been modified (compressed)
|
||||
assert tool_msg["content"] != content or result.tokens_after <= result.tokens_before
|
||||
|
||||
|
||||
class TestHeadroomRetrieveExemptionAnthropic:
|
||||
"""Anthropic-style tool_result content blocks from headroom_retrieve must not be crushed."""
|
||||
@@ -145,42 +123,6 @@ class TestHeadroomRetrieveExemptionAnthropic:
|
||||
)
|
||||
assert not any("smart_crush" in t for t in result.transforms_applied)
|
||||
|
||||
def test_non_retrieve_anthropic_tool_still_compressed(self):
|
||||
"""Normal Anthropic tool_result blocks are still compressed."""
|
||||
content = _big_content()
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": "toolu_bash_1",
|
||||
"name": "Bash",
|
||||
"input": {"cmd": "ls"},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": "toolu_bash_1",
|
||||
"content": content,
|
||||
}
|
||||
],
|
||||
},
|
||||
]
|
||||
crusher = _make_crusher(min_tokens=0)
|
||||
tokenizer = _get_tokenizer()
|
||||
result = crusher.apply(messages, tokenizer)
|
||||
|
||||
# Either the content changed (compressed) or tokens went down
|
||||
tool_result_block = result.messages[1]["content"][0]
|
||||
assert (
|
||||
tool_result_block["content"] != content or result.tokens_after <= result.tokens_before
|
||||
)
|
||||
|
||||
def test_mixed_retrieve_and_normal_only_normal_compressed(self):
|
||||
"""With two tool_results in one user turn, only the non-CCR one is compressed."""
|
||||
content = _big_content()
|
||||
@@ -224,11 +166,139 @@ class TestHeadroomRetrieveExemptionAnthropic:
|
||||
|
||||
blocks = result.messages[1]["content"]
|
||||
ccr_block = blocks[0]
|
||||
bash_block = blocks[1]
|
||||
|
||||
# headroom_retrieve result must be untouched
|
||||
assert ccr_block["content"] == content, (
|
||||
"headroom_retrieve result was compressed — infinite loop bug #1077"
|
||||
)
|
||||
# The bash result should differ (or at least the crush count > 0)
|
||||
assert bash_block["content"] != content or result.tokens_after < result.tokens_before
|
||||
|
||||
|
||||
@pytest.mark.parametrize("wire", ["openai", "anthropic"])
|
||||
@pytest.mark.parametrize("wrapper", ["tool_call", "functions.exec"])
|
||||
def test_wrapped_retrieval_is_exact_and_unrelated_wrapper_remains_compressible(wire, wrapper):
|
||||
content = json.dumps([{"id": i, "value": "sample record " * 8} for i in range(200)])
|
||||
retrieval_args = (
|
||||
{"name": CCR_TOOL_NAME, "arguments": {"hash": "abc123def456"}}
|
||||
if wrapper == "tool_call"
|
||||
else {"code": 'text(await tools.mcp__headroom__headroom_retrieve({hash: "abc123def456"}))'}
|
||||
)
|
||||
ordinary_args = (
|
||||
{"name": "Bash", "arguments": {"command": "echo sample"}}
|
||||
if wrapper == "tool_call"
|
||||
else {"code": 'text(await tools.exec_command({cmd: "echo sample"}))'}
|
||||
)
|
||||
if wire == "openai":
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "retrieved",
|
||||
"function": {"name": wrapper, "arguments": json.dumps(retrieval_args)},
|
||||
},
|
||||
{
|
||||
"id": "ordinary",
|
||||
"function": {"name": wrapper, "arguments": json.dumps(ordinary_args)},
|
||||
},
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "retrieved", "content": content},
|
||||
{"role": "tool", "tool_call_id": "ordinary", "content": content},
|
||||
]
|
||||
else:
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": "retrieved",
|
||||
"name": wrapper,
|
||||
"input": retrieval_args,
|
||||
},
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": "ordinary",
|
||||
"name": wrapper,
|
||||
"input": ordinary_args,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "tool_result", "tool_use_id": "retrieved", "content": content},
|
||||
{"type": "tool_result", "tool_use_id": "ordinary", "content": content},
|
||||
],
|
||||
},
|
||||
]
|
||||
result = _make_crusher(min_tokens=0).apply(messages, _get_tokenizer())
|
||||
if wire == "openai":
|
||||
recovered = result.messages[1]["content"]
|
||||
ordinary = result.messages[2]["content"]
|
||||
else:
|
||||
recovered = result.messages[1]["content"][0]["content"]
|
||||
ordinary = result.messages[1]["content"][1]["content"]
|
||||
assert recovered == content
|
||||
assert ordinary != content
|
||||
|
||||
|
||||
def _messages_with_tool_name(wire, name):
|
||||
if wire == "openai":
|
||||
return [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{"id": "call_1", "function": {"name": name, "arguments": "{}"}},
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "call_1", "content": "ok"},
|
||||
]
|
||||
return [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "tool_use", "id": "call_1", "name": name, "input": {}},
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "tool_result", "tool_use_id": "call_1", "content": "ok"},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("wire", ["openai", "anthropic"])
|
||||
@pytest.mark.parametrize(
|
||||
"name",
|
||||
[["Bash"], {"name": "Bash"}, 7, True, None, ""],
|
||||
ids=["list", "dict", "integer", "boolean", "null", "empty"],
|
||||
)
|
||||
def test_invalid_tool_names_are_unresolved(wire, name):
|
||||
"""Both wire formats reject non-string names before shared wrapper parsing."""
|
||||
from headroom.transforms.smart_crusher import _build_tool_name_index
|
||||
|
||||
messages = _messages_with_tool_name(wire, name)
|
||||
original = deepcopy(messages)
|
||||
assert _build_tool_name_index(messages) == {}
|
||||
assert messages == original
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name",
|
||||
[["Bash"], {"name": "Bash"}, 7, True, None, ""],
|
||||
ids=["list", "dict", "integer", "boolean", "null", "empty"],
|
||||
)
|
||||
def test_invalid_openai_tool_names_leave_small_results_unchanged(name):
|
||||
"""The real OpenAI counter reaches indexing even for malformed function names."""
|
||||
messages = _messages_with_tool_name("openai", name)
|
||||
original = deepcopy(messages)
|
||||
result = _make_crusher(min_tokens=200).apply(messages, _get_tokenizer())
|
||||
|
||||
assert result.messages == original
|
||||
assert messages == original
|
||||
assert result.transforms_applied == []
|
||||
assert result.markers_inserted == []
|
||||
assert result.tokens_before == result.tokens_after
|
||||
|
||||
Reference in New Issue
Block a user