fix(proxy): don't send a memory continuation with unanswered client tool calls (#4009) (#4013)

## Description

When the model calls one of Headroom's injected memory tools on a
buffered Anthropic turn (a `stream:false` request, or a streaming
request that CCR switched to buffered), Headroom runs the tool itself
and then asks the model to continue. That follow-up request repeats the
model's whole turn, but it only includes answers for the memory calls.

So if the model called a memory tool **and** one of the client's own
tools in the same turn (parallel tool use makes this common), the client
tool was left unanswered. Anthropic rejects that with a 400 (`tool_use
ids were found without tool_result blocks`), the 400 replaced the
perfectly good first response, and the client's turn failed. The tool id
in the error was never sent to the client, so it had no way to repair
its history. The log still said `continuation complete`, which hid the
failure.

The streaming path already handles this correctly since #3947: it only
continues when the turn called nothing but memory tools, and otherwise
gives the turn back to the client. This PR applies the same rule to the
buffered path.

Closes #4009

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/anthropic.py`, buffered memory block:
- Continue only when every `tool_use` in the model's turn has a
`tool_result`.
- Otherwise, return the turn to the client with the proxy's own memory
calls removed (the client never declared those tools and they already
ran). The client's own tool calls stay, so it can run them as usual. A
memory tool the client declared itself is left alone, same as on the
streaming path.
- Build the continuation from `body["messages"]` (what was actually sent
upstream) instead of `optimized_messages`. They normally match, but
later steps such as the system-role relocation (#765) only update
`body["messages"]`.
- Log a warning with the upstream status when the continuation fails,
instead of `continuation complete`.
- `tests/test_anthropic_buffered_memory_continuation.py` (new): drives
the real app with a faked upstream.
- A turn with a memory call plus a `Bash` call: no continuation is sent,
the memory tool still runs, and the client receives only the `Bash`
call. Fails on `main` (a continuation is sent).
- A memory-only turn: continues as before, with a result for every call,
on top of the messages that were sent.
- A continuation that returns 400 is logged as a failure. Fails on
`main`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

Before the fix (new tests on unmodified `main`):

```text
FAILED tests/test_anthropic_buffered_memory_continuation.py::test_mixed_turn_returns_client_tool_without_continuation
    assert len(sent) == 1
E   AssertionError: assert 2 == 1
FAILED tests/test_anthropic_buffered_memory_continuation.py::test_failed_continuation_is_not_logged_as_complete
======================== 2 failed, 1 passed in 15.54s =========================
```

After the fix:

```text
$ pytest tests/test_anthropic_buffered_memory_continuation.py tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_memory_tool_stream.py -q
============================= 70 passed in 18.42s =============================

$ ruff check headroom/proxy/handlers/anthropic.py tests/test_anthropic_buffered_memory_continuation.py
All checks passed!
$ ruff format --check headroom/proxy/handlers/anthropic.py tests/test_anthropic_buffered_memory_continuation.py
2 files already formatted

$ mypy headroom --ignore-missing-imports
headroom\fileperms.py:92: error: Module has no attribute "fchmod"; maybe "chmod"?  [attr-defined]
Found 1 error in 1 file (checked 562 source files)
```

The one mypy error is Windows-only (`os.fchmod` does not exist there)
and is unrelated to this change; it does not appear on Linux CI.

I also ran every `tests/*memory*`, `*anthropic*`, `*buffered*`, `*ccr*`
file plus `tests/test_proxy/` (1705 passed). The failures in that run
(`test_wrap_code_memory.py`, `test_orphan_watchdog.py`, an import error
in `test_anthropic_buffered_ccr_usage.py`) fail the same way on
unmodified `main` on my Windows machine. `test_cc_switch_reconciler.py`
failed once under xdist and passed on three serial reruns; it is
timing-sensitive and does not touch this code.

## Real Behavior Proof

- Environment: Windows 11, Python 3.13, headroom at this branch, full
FastAPI app via `TestClient` with the upstream call faked (memory
handler faked to behave like the real one: results only for memory
tools).
- Exact command / steps: `pytest
tests/test_anthropic_buffered_memory_continuation.py -q`, which sends a
`stream:false` `/v1/messages` request with memory tools injected, where
the upstream answers with `[memory_search tool_use, Bash tool_use]`.
- Observed result: on `main` Headroom sends a second upstream request
whose user turn answers only `toolu_mem`, leaving `toolu_bash`
unanswered (the exact shape Anthropic rejects in #4009). With this
change no second request is sent, `memory_search` still runs, and the
client gets back a response containing only the `Bash` tool call.
- Not tested: against a live Anthropic or Bedrock-backed gateway, and
the reporter's exact 214-message session. The issue's error points at
`messages.210` while the request had 214 messages; Headroom appends the
continuation after the full list, so I could not explain that index from
Headroom's code (possibly the gateway re-indexing).

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This changes the existing
memory-tool handling on buffered Anthropic turns.
- Minimum rollout channel: stable
- Stable/default behavior changed: Yes, only for turns that previously
failed: when the model calls an injected memory tool together with a
client tool, the turn now goes back to the client instead of producing
an upstream 400. Memory-only turns behave as before.
- Kill switch / disable path: Disable memory tool injection
(`--no-memory-tools` / memory `inject_tools` off), which skips this
block entirely.
- Unsafe override required: No
- Qualification impact: None
- Rollback path: Revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` (it is generated by
release-please from my Conventional Commit PR title)

## Additional Notes

- No docs change: this is internal proxy behavior with no user-facing
setting.
- Follow-up, not in this PR: the OpenAI Chat Completions memory path
(`openai.py`, around the `Memory: Handled ... tool call(s) with
continuation` log) builds its continuation the same way. It does not
break the client, because it checks the continuation's status and falls
back to the original response, but the continuation itself is still
malformed on a mixed turn and wastes a request.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4013"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Parideboy
2026-10-07 14:29:17 -05:00
committed by GitHub
co-authored by Claude Opus 5.5 JD Davis
parent 278bbc8771
commit 87e05b68b8
2 changed files with 218 additions and 6 deletions
+57 -6
View File
@@ -4731,18 +4731,62 @@ class AnthropicHandlerMixin:
request_context=memory_request_ctx,
)
if tool_results:
turn_content = resp_json.get("content") or []
answered = {result.get("tool_use_id") for result in tool_results}
unanswered = [
block
for block in turn_content
if isinstance(block, dict)
and block.get("type") == "tool_use"
and block.get("id") not in answered
]
if tool_results and unanswered:
# The continuation replays this whole turn, and
# Anthropic rejects any tool_use without a result
# (#4009). Only the client can answer its own
# tools, so the turn goes back to it, minus the
# memory calls it never declared and that already
# ran here (same rule as the streaming path, #3947).
logger.info(
f"[{request_id}] Memory: Turn also called a client "
"tool; returning the turn to the client"
)
resp_json = {
**resp_json,
"content": [
block
for block in turn_content
if not (
isinstance(block, dict)
and block.get("type") == "tool_use"
and block.get("name") in server_memory_tool_names
)
],
}
response = httpx.Response(
status_code=200,
content=json.dumps(resp_json).encode(),
headers={
key: value
for key, value in response.headers.items()
if key.lower()
not in ("content-encoding", "content-length")
},
)
elif tool_results:
# Create continuation messages
assistant_msg = {
"role": "assistant",
"content": resp_json.get("content", []),
"content": turn_content,
}
user_msg = {
"role": "user",
"content": tool_results,
}
continuation_messages = optimized_messages + [
# body["messages"], not optimized_messages: it is
# what this turn actually sent upstream.
continuation_messages = body["messages"] + [
assistant_msg,
user_msg,
]
@@ -4763,9 +4807,16 @@ class AnthropicHandlerMixin:
# Update response with continuation
resp_json = cont_response.json()
response = cont_response
logger.info(
f"[{request_id}] Memory: Tool calls handled, continuation complete"
)
if cont_response.status_code >= 400:
logger.warning(
f"[{request_id}] Memory: Continuation failed with "
f"upstream status {cont_response.status_code}"
)
else:
logger.info(
f"[{request_id}] Memory: Tool calls handled, "
"continuation complete"
)
except Exception as e:
logger.warning(
@@ -0,0 +1,161 @@
"""Buffered memory continuations answer every tool_use they replay (GH #4009).
On a buffered Anthropic turn the proxy runs its injected memory tools and sends
the model a continuation. That continuation replays the whole assistant turn,
so a client tool called alongside a memory tool used to go upstream with no
``tool_result`` and the provider rejected it with a 400.
"""
from __future__ import annotations
import json
import logging
from typing import Any
import httpx
import pytest
from fastapi.testclient import TestClient
from headroom.proxy.server import ProxyConfig, create_app
MEMORY_TOOLS = ("memory_save", "memory_search")
class _MemoryHandler:
def __init__(self) -> None:
self.config = type(
"MemoryConfig",
(),
{"inject_context": False, "inject_tools": True, "project_root_override": ""},
)()
self.initialized = False
self.backend = None
self.executed: list[str] = []
def compute_memory_tool_definitions(self, provider: str) -> list[dict[str, Any]]:
return [
{"name": name, "description": name, "input_schema": {"type": "object"}}
for name in MEMORY_TOOLS
]
def get_beta_headers(self) -> dict[str, str]:
return {}
def has_memory_tool_calls(self, response: dict[str, Any], provider: str) -> bool:
return bool(self._memory_calls(response))
async def handle_memory_tool_calls(
self, response: dict[str, Any], user_id: str, provider: str, **kwargs: Any
) -> list[dict[str, Any]]:
# Like the real handler: results only for memory tools.
results = []
for block in self._memory_calls(response):
self.executed.append(block["name"])
results.append({"type": "tool_result", "tool_use_id": block["id"], "content": "ok"})
return results
@staticmethod
def _memory_calls(response: dict[str, Any]) -> list[dict[str, Any]]:
return [
block
for block in response.get("content") or []
if block.get("type") == "tool_use" and block.get("name") in MEMORY_TOOLS
]
def _message(content: list[dict[str, Any]], stop_reason: str) -> dict[str, Any]:
return {
"id": "msg_1",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-6",
"content": content,
"stop_reason": stop_reason,
"usage": {"input_tokens": 10, "output_tokens": 5},
}
MEMORY_CALL = {
"type": "tool_use",
"id": "toolu_mem",
"name": "memory_search",
"input": {"query": "deploy region"},
}
CLIENT_CALL = {"type": "tool_use", "id": "toolu_bash", "name": "Bash", "input": {"command": "ls"}}
def _run(upstream: list[httpx.Response]) -> tuple[httpx.Response, list[dict], _MemoryHandler]:
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
)
sent: list[dict] = []
handler = _MemoryHandler()
with TestClient(create_app(config)) as client:
proxy = client.app.state.proxy
proxy.memory_handler = handler
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
sent.append(json.loads(json.dumps(body)))
return upstream[len(sent) - 1]
proxy._retry_request = _fake_retry # type: ignore[assignment]
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"x-headroom-user-id": "u1",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"tools": [{"name": "Bash", "input_schema": {"type": "object"}}],
"messages": [{"role": "user", "content": "check the deploy region"}],
},
)
return response, sent, handler
def test_mixed_turn_returns_client_tool_without_continuation() -> None:
first = _message([MEMORY_CALL, CLIENT_CALL], "tool_use")
response, sent, handler = _run([httpx.Response(200, json=first)])
assert response.status_code == 200, response.text
# No continuation: it could not answer the Bash call.
assert len(sent) == 1
assert handler.executed == ["memory_search"]
# The client gets its own tool call back, not the proxy's.
assert [block["name"] for block in response.json()["content"]] == ["Bash"]
def test_memory_only_turn_continues_with_a_result_for_every_call() -> None:
first = _message([MEMORY_CALL], "tool_use")
final = _message([{"type": "text", "text": "us-east-1"}], "end_turn")
response, sent, _ = _run([httpx.Response(200, json=first), httpx.Response(200, json=final)])
assert len(sent) == 2
assistant, user = sent[1]["messages"][-2:]
assert assistant == {"role": "assistant", "content": [MEMORY_CALL]}
assert [r["tool_use_id"] for r in user["content"]] == ["toolu_mem"]
assert sent[1]["messages"][:-2] == sent[0]["messages"]
assert response.json()["content"] == [{"type": "text", "text": "us-east-1"}]
def test_failed_continuation_is_not_logged_as_complete(caplog: pytest.LogCaptureFixture) -> None:
first = _message([MEMORY_CALL], "tool_use")
rejected = httpx.Response(
400, json={"type": "error", "error": {"type": "invalid_request_error", "message": "no"}}
)
with caplog.at_level(logging.INFO, logger="headroom.proxy"):
_run([httpx.Response(200, json=first), rejected])
assert "Memory: Continuation failed with upstream status 400" in caplog.text
assert "continuation complete" not in caplog.text