mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(transforms): tolerate malformed assistant tool calls in ContentRouter.apply (#4459)
## Description A tool call with `"function": null` (or a non-object), `arguments` that decode to a list/string/number, or a `file_path` that is not a string raised out of `ReadLifecycleManager`, `ReadMaturationManager` and `ContentRouter._build_tool_name_map`. `TransformPipeline.apply` re-raised and counted a circuit-breaker failure, so three such requests in a row switched compression off for the cooldown (60 s default) for every request on that pipeline. The three scanners now skip the malformed call, as the surrounding `isinstance(tc, dict)` checks already do for a non-dict `tool_calls` entry. Fixes #4458 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/read_lifecycle.py` (`_build_tool_metadata`): skip a call whose `function` is not a dict; only read `file_path` / `offset` / `limit` when the decoded `arguments` is a dict; treat a non-string `file_path` as absent (it is later used as a dict key). Same non-string `file_path` guard in the Anthropic `tool_use` branch. - `headroom/transforms/read_maturation.py` (`_scan_activity`): same `function` and decoded-`arguments` guards; non-dict `input` and non-string `file_path` treated as absent in both branches. - `headroom/transforms/content_router.py` (`_build_tool_name_map`): skip a call whose `function` is not a dict. - `tests/test_transforms/test_malformed_tool_calls.py`: 9 malformed shapes (`function` null / string, `arguments` decoding to list / string / number, `file_path` list in OpenAI arguments and in `tool_use`, `path` dict, `input` list) run through each of the three entry points (27 cases). ## Testing Rebased onto upstream/main `3fef15d3` (2026-10-24): the new tests pass (27 passed); with only the source change reverted, 25 fail. The real-behavior run below was done on `e995eb9f`. Review round 1 (42ddb602, Devin findings): results of calls whose `function` is not an object are now passed through verbatim (before: an 18,690-char log result was compressed to 103 chars), and read maturation skips calls whose file cannot be read from the arguments. The tests now assert the tool result is unchanged and the maturation test runs eight quiet turns: 27 passed; on the previous head 9 of them fail. Neighbouring suites (134 files touching the router, read lifecycle or maturation): the only failures are environment ones (memory_bridge fixtures, offline-egress, kompress), and they fail on main too. - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (`ruff check` and `ruff format --check` were run on the four changed files only; `pytest` on the new file and the neighbouring read-lifecycle, read-maturation and content-router test files, not the whole suite.) ### Test Output ```text # new test file with only the source change absent (tests added on top of upstream/maine995eb9f) 25 failed, 2 passed e.g. AttributeError: 'NoneType' object has no attribute 'get' TypeError: unhashable type: 'dict' (file_ops[file_path].append in _build_file_operation_index) # with the fix tests/test_transforms/test_malformed_tool_calls.py ..................... [ 77%] ...... [100%] 27 passed in 1.52s # neighbouring tests with the fix tests/test_transforms/test_read_lifecycle.py tests/test_read_maturation.py tests/test_read_maturation_handler_nobust.py tests/test_transforms/test_content_router.py 120 passed, 1 warning in 8.93s ruff check: All checks passed! ruff format --check: 4 files already formatted ``` ## Real Behavior Proof - Environment: Ubuntu on WSL2 (Linux 6.18, x86_64), Python 3.12.15, upstream/maine995eb9fwith and without this commit, Rust core `headroom/_core.abi3.so` copied (untracked) from the official headroom-ai 0.41.0 wheel. The real proxy `python -m headroom.proxy.server --host 127.0.0.1 --port 18970 --no-cache --no-rate-limit` with `OPENAI_TARGET_API_URL=http://127.0.0.1:18971`, a local stub upstream that logs every request it receives (so the stub log shows the proxy pointed at it). - Exact command / steps: `client36.py` sends six `POST /v1/chat/completions` in order: a well-formed control (assistant call `list_users`, tool result of 400 JSON rows, 31,260 bytes), three requests whose assistant `tool_calls[0].function` is `null` (same tool result), then two well-formed requests with new tool-result content (the proxy's compression cache would otherwise answer for repeated content). Run once on the base commit and once with this commit. The stub records the body size it receives. - Observed result: before, the three malformed requests are each forwarded unoptimized with `Optimization failed: AttributeError: 'NoneType' object has no attribute 'get'`, the third opens `Pipeline circuit breaker OPEN after 3 consecutive failures; passing messages through for 60s`, and the two well-formed requests after it reach the upstream uncompressed (31,234 bytes). After, no request fails, the breaker never opens, and every request is compressed (the malformed ones too, since the bad call is simply ignored). Excerpts below. - Not tested: a real OpenAI or Anthropic API (stub upstream only); the Anthropic `/v1/messages` and the OpenAI Responses handlers over HTTP (the `tool_use` and string-`file_path` shapes are covered by the unit tests only); streaming requests; malformed `offset` / `limit` values (a non-numeric `offset` is still not guarded); Windows or macOS; a Rust core built from this checkout (the 0.41.0 wheel's was used); how many callers share one pipeline in a real deployment. Before (basee995eb9f), proxy log and stub: ```text [hr_1792691400_000002] Optimization failed: AttributeError: 'NoneType' object has no attribute 'get' (request forwarded unoptimized) [hr_1792691400_000003] Optimization failed: AttributeError: 'NoneType' object has no attribute 'get' (request forwarded unoptimized) Pipeline circuit breaker OPEN after 3 consecutive failures; passing messages through for 60s [hr_1792691400_000004] Optimization failed: AttributeError: 'NoneType' object has no attribute 'get' (request forwarded unoptimized) STUB-UPSTREAM call #1 body_bytes=10090 tool_content_chars=9433 <- control, compressed STUB-UPSTREAM call #2 body_bytes=31200 tool_content_chars=26180 <- malformed STUB-UPSTREAM call #3 body_bytes=31200 tool_content_chars=26180 STUB-UPSTREAM call #4 body_bytes=31200 tool_content_chars=26180 STUB-UPSTREAM call #5 body_bytes=31234 tool_content_chars=26180 <- well-formed, NOT compressed STUB-UPSTREAM call #6 body_bytes=31234 tool_content_chars=26180 ``` After (this commit): no `Optimization failed`, no `circuit breaker` line, six `Pipeline complete: ... saved 6786 ... 62.6% reduction` lines, and: ```text STUB-UPSTREAM call #1 body_bytes=10090 tool_content_chars=9433 STUB-UPSTREAM call #2 body_bytes=10056 tool_content_chars=9433 STUB-UPSTREAM call #3 body_bytes=10056 tool_content_chars=9433 STUB-UPSTREAM call #4 body_bytes=10056 tool_content_chars=9433 STUB-UPSTREAM call #5 body_bytes=10090 tool_content_chars=9433 STUB-UPSTREAM call #6 body_bytes=10090 tool_content_chars=9433 ``` Scripts and raw outputs (WSL): `/var/tmp/cb-pub/proof/fix-malformed-tool-calls-apply/` (`stub36.py`, `client36.py`, `run36proof.sh`, `before.*`, `after.*`, `src.patch`). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: yes, narrowly: requests with a malformed tool call are now compressed instead of failing the router and counting toward the circuit breaker; well-formed requests are unaffected - Kill switch / disable path: none; no config involved - Unsafe override required: no - Qualification impact: none expected - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes Design choice to review: a malformed call is skipped (it contributes no tool name, no file path), not rejected, so the request still goes through compression. The alternative is validating tool calls at handler ingress; I did not find such validation today and kept the fix local to the three scanners that crashed. Documentation: no doc describes this behavior. Produced with AI assistance (Claude Code) operated by me (HetCreep); the reproduction was run on my machine on a clone of the repository. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
03590cd61e
commit
cddd2fd60e
@@ -596,6 +596,24 @@ def _external_compressor_matches(descriptor: CompressorDescriptor, content_mime:
|
||||
return any(d in ("*", "*/*") or d == type_wildcard for d in declared)
|
||||
|
||||
|
||||
def _malformed_tool_call_ids(messages: list[dict[str, Any]]) -> set[str]:
|
||||
"""Ids of OpenAI tool calls whose ``function`` is not an object.
|
||||
|
||||
Such a call has no readable tool name, so its result cannot be classified;
|
||||
it is passed through like a verbatim-excluded tool rather than compressed.
|
||||
"""
|
||||
ids: set[str] = set()
|
||||
for msg in messages:
|
||||
if msg.get("role") != "assistant":
|
||||
continue
|
||||
for tc in msg.get("tool_calls") or []:
|
||||
if isinstance(tc, dict) and not isinstance(tc.get("function"), dict):
|
||||
tc_id = tc.get("id")
|
||||
if isinstance(tc_id, str) and tc_id:
|
||||
ids.add(tc_id)
|
||||
return ids
|
||||
|
||||
|
||||
def _tool_call_args_text(raw: Any) -> str:
|
||||
"""Compact, query-usable text from a tool call's args.
|
||||
|
||||
@@ -5832,7 +5850,9 @@ class ContentRouter(Transform):
|
||||
for tc in msg.get("tool_calls") or []:
|
||||
if isinstance(tc, dict):
|
||||
tc_id = tc.get("id", "")
|
||||
fn = tc.get("function", {})
|
||||
fn = tc.get("function")
|
||||
if not isinstance(fn, dict): # "function": null / non-object
|
||||
continue
|
||||
name = fn.get("name", "")
|
||||
call_args = fn.get("arguments")
|
||||
if name:
|
||||
@@ -6184,6 +6204,9 @@ class ContentRouter(Transform):
|
||||
for tool_id, name in tool_name_map.items()
|
||||
if is_tool_excluded(name, exclude_tools)
|
||||
}
|
||||
# A call with no readable name keeps its result untouched, as it was
|
||||
# before such calls stopped raising out of apply().
|
||||
excluded_tool_ids |= _malformed_tool_call_ids(messages)
|
||||
|
||||
# CCR-retrieve tool IDs, precomputed once (mirrors excluded_tool_ids
|
||||
# above, rather than calling is_tool_excluded() per message/block). A
|
||||
@@ -6579,7 +6602,7 @@ class ContentRouter(Transform):
|
||||
if role == "tool":
|
||||
if tool_call_id in excluded_tool_ids:
|
||||
tool_name = tool_name_map.get(tool_call_id, "")
|
||||
if tool_name and is_tool_excluded(tool_name, DEFAULT_VERBATIM_EXCLUDE_TOOLS):
|
||||
if not tool_name or is_tool_excluded(tool_name, DEFAULT_VERBATIM_EXCLUDE_TOOLS):
|
||||
result_slots[i] = message
|
||||
transforms_applied.append("router:excluded:tool")
|
||||
route_counts["excluded_tool"] += 1
|
||||
@@ -7753,7 +7776,7 @@ class ContentRouter(Transform):
|
||||
continue
|
||||
if tool_use_id in excluded_tool_ids:
|
||||
tool_name = tool_name_map.get(tool_use_id, "") if tool_name_map else ""
|
||||
if tool_name and is_tool_excluded(tool_name, DEFAULT_VERBATIM_EXCLUDE_TOOLS):
|
||||
if not tool_name or is_tool_excluded(tool_name, DEFAULT_VERBATIM_EXCLUDE_TOOLS):
|
||||
new_blocks.append(block)
|
||||
transforms_applied.append("router:excluded:tool")
|
||||
if route_counts is not None:
|
||||
|
||||
@@ -181,7 +181,9 @@ class ReadLifecycleManager:
|
||||
if not isinstance(tc, dict):
|
||||
continue
|
||||
tc_id = tc.get("id", "")
|
||||
func = tc.get("function", {})
|
||||
func = tc.get("function")
|
||||
if not isinstance(func, dict): # "function": null / non-object
|
||||
continue
|
||||
name = func.get("name", "")
|
||||
if not tc_id or not name:
|
||||
continue
|
||||
@@ -191,11 +193,15 @@ class ReadLifecycleManager:
|
||||
limit = None
|
||||
try:
|
||||
args = json.loads(func.get("arguments", "{}"))
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
args = None
|
||||
if isinstance(args, dict): # arguments may decode to a list/str/number
|
||||
file_path = args.get("file_path") or args.get("path")
|
||||
offset = args.get("offset")
|
||||
limit = args.get("limit")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
# file_path is a dict key downstream; an unhashable value must not reach it.
|
||||
if not isinstance(file_path, str):
|
||||
file_path = None
|
||||
metadata[tc_id] = (name, file_path, offset, limit)
|
||||
|
||||
# Anthropic format: content blocks with type=tool_use
|
||||
@@ -218,6 +224,8 @@ class ReadLifecycleManager:
|
||||
file_path = inp.get("file_path") or inp.get("path")
|
||||
offset = inp.get("offset")
|
||||
limit = inp.get("limit")
|
||||
if not isinstance(file_path, str):
|
||||
file_path = None
|
||||
metadata[tc_id] = (name, file_path, offset, limit)
|
||||
|
||||
return metadata
|
||||
|
||||
@@ -284,15 +284,21 @@ class ReadMaturationManager:
|
||||
for tc in msg.get("tool_calls", []) or []:
|
||||
if not isinstance(tc, dict):
|
||||
continue
|
||||
func = tc.get("function", {})
|
||||
func = tc.get("function")
|
||||
if not isinstance(func, dict): # "function": null / non-object
|
||||
continue
|
||||
name = func.get("name", "")
|
||||
if name not in _TOUCH_TOOLS:
|
||||
continue
|
||||
try:
|
||||
args = json.loads(func.get("arguments", "{}"))
|
||||
except (ValueError, TypeError):
|
||||
args = {}
|
||||
except (ValueError, TypeError): # not JSON: unknown file
|
||||
continue
|
||||
if not isinstance(args, dict): # list/str/number: unknown file
|
||||
continue
|
||||
fp = args.get("file_path") or args.get("path") or ""
|
||||
if not isinstance(fp, str): # unknown file: never mature this call
|
||||
continue
|
||||
if fp:
|
||||
act.file_last_touch[fp] = turn
|
||||
if name in _READ_TOOLS:
|
||||
@@ -306,8 +312,12 @@ class ReadMaturationManager:
|
||||
name = b.get("name", "")
|
||||
if name not in _TOUCH_TOOLS:
|
||||
continue
|
||||
inp = b.get("input") or {}
|
||||
inp = b.get("input")
|
||||
if not isinstance(inp, dict): # unknown file: never mature this call
|
||||
continue
|
||||
fp = inp.get("file_path") or inp.get("path") or ""
|
||||
if not isinstance(fp, str): # unknown file: never mature this call
|
||||
continue
|
||||
if fp:
|
||||
act.file_last_touch[fp] = turn
|
||||
if name in _READ_TOOLS:
|
||||
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
"""Malformed assistant tool calls must not raise out of ContentRouter.apply.
|
||||
|
||||
A request body can carry ``"function": null``, ``arguments`` that decode to a
|
||||
non-object, or a ``tool_use`` whose ``file_path`` is not a string. Those used to
|
||||
raise out of ``ReadLifecycleManager`` / ``ReadMaturationManager`` /
|
||||
``ContentRouter._build_tool_name_map``; ``TransformPipeline.apply`` re-raised
|
||||
and counted a circuit-breaker failure, so three such requests in a row switched
|
||||
compression off for the breaker cooldown.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.config import ReadLifecycleConfig, ReadMaturationConfig
|
||||
from headroom.tokenizers import EstimatingTokenCounter
|
||||
from headroom.transforms.content_router import ContentRouter
|
||||
from headroom.transforms.read_lifecycle import ReadLifecycleManager
|
||||
from headroom.transforms.read_maturation import ReadMaturationManager
|
||||
|
||||
|
||||
def _openai(function: object) -> dict:
|
||||
return {
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [{"id": "c1", "type": "function", "function": function}],
|
||||
}
|
||||
|
||||
|
||||
def _anthropic(tool_input: object) -> dict:
|
||||
return {
|
||||
"role": "assistant",
|
||||
"content": [{"type": "tool_use", "id": "c1", "name": "Read", "input": tool_input}],
|
||||
}
|
||||
|
||||
|
||||
MALFORMED = {
|
||||
"function_null": _openai(None),
|
||||
"function_string": _openai("Read"),
|
||||
"arguments_list": _openai({"name": "Read", "arguments": "[1]"}),
|
||||
"arguments_string": _openai({"name": "Read", "arguments": '"s"'}),
|
||||
"arguments_number": _openai({"name": "Read", "arguments": "5"}),
|
||||
"arguments_invalid_json": _openai({"name": "Read", "arguments": "{"}),
|
||||
"arguments_file_path_list": _openai({"name": "Read", "arguments": '{"file_path": ["a"]}'}),
|
||||
"tool_use_file_path_list": _anthropic({"file_path": ["a"]}),
|
||||
"tool_use_path_dict": _anthropic({"path": {"a": 1}}),
|
||||
"tool_use_input_list": _anthropic([1]),
|
||||
}
|
||||
|
||||
|
||||
# Compressible log text: a well-formed call to an unknown tool would be compressed,
|
||||
# so an unchanged result shows the malformed call's output was left alone.
|
||||
RESULT = "".join(f"2026-10-24 INFO worker {i % 7} processed item {i} ok\n" for i in range(400))
|
||||
|
||||
|
||||
def _conversation(assistant: dict, quiet_turns: int = 0) -> list[dict]:
|
||||
result = RESULT
|
||||
if "tool_calls" in assistant:
|
||||
tool_msg = {"role": "tool", "tool_call_id": "c1", "content": result}
|
||||
else:
|
||||
tool_msg = {
|
||||
"role": "user",
|
||||
"content": [{"type": "tool_result", "tool_use_id": "c1", "content": result}],
|
||||
}
|
||||
messages = [{"role": "user", "content": "go"}, assistant, tool_msg]
|
||||
for _ in range(quiet_turns):
|
||||
messages += [{"role": "assistant", "content": "ok"}, {"role": "user", "content": "next"}]
|
||||
return messages
|
||||
|
||||
|
||||
def _tool_result_text(messages: list[dict]) -> str:
|
||||
content = messages[2]["content"]
|
||||
return content if isinstance(content, str) else content[0]["content"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("assistant", MALFORMED.values(), ids=MALFORMED.keys())
|
||||
def test_read_lifecycle_tolerates_malformed_tool_calls(assistant: dict) -> None:
|
||||
messages = _conversation(assistant)
|
||||
result = ReadLifecycleManager(ReadLifecycleConfig(enabled=True)).apply(messages)
|
||||
assert result.messages == messages
|
||||
|
||||
|
||||
@pytest.mark.parametrize("assistant", MALFORMED.values(), ids=MALFORMED.keys())
|
||||
def test_read_maturation_tolerates_malformed_tool_calls(assistant: dict) -> None:
|
||||
# Enough quiet turns that a well-formed Read would mature; a call whose file
|
||||
# cannot be read from its arguments must never be matured.
|
||||
messages = _conversation(assistant, quiet_turns=8)
|
||||
result = ReadMaturationManager(ReadMaturationConfig(enabled=True)).apply(messages)
|
||||
assert _tool_result_text(result.messages) == RESULT
|
||||
|
||||
|
||||
@pytest.mark.parametrize("assistant", MALFORMED.values(), ids=MALFORMED.keys())
|
||||
def test_content_router_apply_tolerates_malformed_tool_calls(assistant: dict) -> None:
|
||||
messages = _conversation(assistant)
|
||||
result = ContentRouter().apply(messages, EstimatingTokenCounter())
|
||||
assert len(result.messages) == len(messages)
|
||||
assert _tool_result_text(result.messages) == RESULT
|
||||
Reference in New Issue
Block a user