fix(memory): run only the memory tools the proxy injected on the request (#4870)

## Description

With memory enabled (`--memory`, `--learn`, `wrap/init --memory`), the
proxy ran any upstream tool call named
`memory_list`/`memory_save`/`memory_search`/`memory_update`/`memory_delete`,
and hid it from the client. That held even when the proxy had not
offered those tools on the request (`--no-memory-tools`), or when the
client had declared a tool with that name itself.

The proxy now runs only the memory tools it injected on the current
request. Every other call reaches the client unchanged. The proxy does
not run it, does not withhold it, and sends no continuation upstream.
Anthropic streaming already followed this rule
(`_server_memory_tool_names`). That helper is now shared, and every
executor uses it.

**Behaviour change**
- With `--no-memory-tools`, memory tool calls from the model now reach
the client instead of running on the proxy.
- A memory-named tool that the client declares is now the client's. The
proxy no longer runs it on any path, including the streaming path that
used to run it as a side effect.
- Injected memory tools behave as before: they run on the proxy, are
hidden from the client, and the turn continues upstream.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/memory_handler.py`: `server_memory_tool_names(tools,
client_tools)` moved here from `AnthropicHandlerMixin`. It now reads
Anthropic, Chat and Responses tool shapes through `extract_tool_name`.
`handle_memory_tool_calls` takes a required `tool_names` and skips every
other call.
- `handlers/openai.py`, Chat Completions (non-streaming): records the
injected names and passes them to the executor.
- `handlers/openai.py`, Responses (HTTP): runs a `function_call` only if
its name was injected. Before, any name in `MEMORY_TOOL_NAMES` ran.
- `handlers/openai.py`, Responses over WebSocket: the injected names are
kept per `response.create`, like the user id. The relay holds back and
runs only those calls.
- `handlers/anthropic.py`, non-streaming: passes the injected names it
already computed for streaming.
- `handlers/streaming.py`: removes the branch that ran memory calls
after the stream had already reached the client. It ran only for tools
the proxy had not injected. Memory handling in the stream (SSE
buffering, the Claude Code credential warning) now turns on only when
the proxy injected memory tools.
- Tests: new `tests/test_memory_tools_only_when_injected.py`. Existing
fixtures that relied on the old behaviour now inject the tool they call:
`test_codex_project_context.py`, `test_memory_handler_native_ops.py`,
`test_memory_handler_project_isolation.py`,
`test_memory_tool_stream.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

The new tests use `create_app` with a fake memory backend and a
`MockTransport` upstream. The WebSocket test uses a scripted upstream
socket. For each path (chat, Responses, Responses over WebSocket,
Anthropic, Anthropic streaming) they check two cases. First, injection
is off and the model calls `memory_delete`. Second, injection is on and
the client declares its own `memory_delete`. Each case expects zero
backend calls, exactly one upstream request, and the call in the
client's response. The control cases inject `memory_list` and expect it
to run as before.

On `origin/main` (0368a872):
```text
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-anthropic_stream]
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_injected_still_runs[chat]
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_injected_still_runs[responses]
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_injected_still_runs[anthropic]
PASSED tests/test_memory_tools_only_when_injected.py::test_ws_memory_call_the_proxy_injected_still_runs
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-chat]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-responses]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-anthropic]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-anthropic_stream]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-chat]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-responses]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-anthropic]
FAILED tests/test_memory_tools_only_when_injected.py::test_ws_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected]
FAILED tests/test_memory_tools_only_when_injected.py::test_ws_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared]
9 failed, 5 passed in 3.05s
```
On this branch:
```text
14 passed in 3.23s
```
The existing tests for memory, anthropic, openai and streaming (165
files under `tests/`, not `test_live`) pass on both branches:
```text
origin/main: 3183 passed, 44 skipped in 230.96s
this branch: 3197 passed, 44 skipped in 229.83s
```
`uvx ruff@0.17.3 check` and `format --check` pass on the changed files.
`mypy` on the four changed modules reports no issues.

## Real Behavior Proof

- Environment: macOS, Python 3.12,
`create_app(ProxyConfig(memory_enabled=True,
memory_storage_mode="global", ...))` in-process via
`httpx.ASGITransport`. The upstream is `httpx.MockTransport`, the memory
backend is a recording fake, and `LITELLM_LOCAL_MODEL_COST_MAP=True`.
- Exact command / steps: a harness sends one request per path. The
mocked upstream answers with a tool call to `memory_delete`. The harness
records backend calls, upstream request count, and whether the client
response contains the call.
- Observed result (`origin/main` → this branch):
```text
chat       inject=False memory_delete  upstream_calls=2 backend_calls=[delete] client_saw_tool_call=False  →  upstream_calls=1 backend_calls=[] client_saw_tool_call=True
responses  inject=False memory_delete  upstream_calls=2 backend_calls=[delete] client_saw_tool_call=False  →  upstream_calls=1 backend_calls=[] client_saw_tool_call=True
anthropic  inject=False memory_delete  upstream_calls=2 backend_calls=[delete] client_saw_tool_call=False  →  upstream_calls=1 backend_calls=[] client_saw_tool_call=True
anthropic stream inject=False memory_delete  backend_calls=[delete] client_saw_tool_call=True  →  backend_calls=[] client_saw_tool_call=True
chat       inject=True, client-declared memory_delete  upstream_calls=2 backend_calls=[delete]  →  upstream_calls=1 backend_calls=[]
anthropic  inject=True, client-declared memory_delete  upstream_calls=2 backend_calls=[delete]  →  upstream_calls=1 backend_calls=[]
```
- Not tested: a live provider and a live Codex WebSocket session. The
WebSocket path is covered by the in-process test above.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: only for memory-named tool calls that
the proxy did not inject. See "Behaviour change" above. The default
`--memory` setup, where the proxy injects the tools, is unchanged.
- Kill switch / disable path: none needed.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Follow-up, not in this PR: on the Chat, Responses and WebSocket paths, a
single turn can call an injected memory tool and a client tool together.
The proxy then continues upstream without an output for the client's
call. That was already true for a client tool such as `shell`. It now
also applies to a memory-named tool that the client declared, which the
proxy used to run. The Anthropic non-streaming path already handles this
by returning the turn to the client. Bringing the OpenAI paths in line
is a separate change.

The docs need no change: `--no-memory-tools` is already documented as
"Disable automatic memory tool injection", which is now what it does.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4870"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
Tejas Chopra
2026-11-14 10:47:05 +05:30
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent 48df6df497
commit fda75de505
9 changed files with 519 additions and 73 deletions
+5 -21
View File
@@ -727,25 +727,6 @@ class AnthropicHandlerMixin:
canonical = str(tool)
return (name, canonical)
@staticmethod
def _server_memory_tool_names(tools: Any, client_tools: Any) -> frozenset[str]:
"""Memory tools in ``tools`` that the proxy injected and must run itself.
A memory tool the client declared stays the client's: its calls are
forwarded, not withheld.
"""
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES, NATIVE_MEMORY_TOOL_NAME
client_tool_names = {t.get("name") for t in client_tools or [] if isinstance(t, dict)}
return frozenset(
name
for t in tools or []
if isinstance(t, dict)
and isinstance(name := t.get("name"), str)
and (name in MEMORY_TOOL_NAMES or name == NATIVE_MEMORY_TOOL_NAME)
and name not in client_tool_names
)
def _can_salvage_buffered_upstream(self, resp_json: Any) -> bool:
"""May this upstream response be relayed when post-processing failed?
@@ -3508,9 +3489,11 @@ class AnthropicHandlerMixin:
)
if mem_tools_injected:
memory_tools_injected = True
server_memory_tool_names = self._server_memory_tool_names(
tools, _original_tools
from headroom.proxy.memory_handler import (
server_memory_tool_names as _server_memory_tool_names,
)
server_memory_tool_names = _server_memory_tool_names(tools, _original_tools)
tool_names = [
t.get("name") or t.get("type", "")
for t in tools
@@ -5343,6 +5326,7 @@ class AnthropicHandlerMixin:
memory_user_id,
"anthropic",
request_context=memory_request_ctx,
tool_names=server_memory_tool_names,
)
turn_content = resp_json.get("content") or []
+35 -14
View File
@@ -4684,6 +4684,9 @@ class OpenAIHandlerMixin:
# invariant I2. See REALIGNMENT/03-phase-A-lockdown.md PR-A3.
memory_context_injected = False
memory_tools_injected = False
# Memory tools this proxy injected and runs itself; every other call,
# memory-named or not, goes back to the client.
server_memory_tool_names: frozenset[str] = frozenset()
if memory_decision.inject:
# Memory-handler is guaranteed present when inject=True.
# Timeout-wrap (matches Anthropic /v1/messages and
@@ -4802,7 +4805,12 @@ class OpenAIHandlerMixin:
client_declared_tools=bool(_original_tools),
)
if mem_tools_injected:
from headroom.proxy.memory_handler import (
server_memory_tool_names as _server_memory_tool_names,
)
memory_tools_injected = True
server_memory_tool_names = _server_memory_tool_names(tools, _original_tools)
logger.info(f"[{request_id}] Memory: Injected memory tools (openai)")
except Exception as e:
logger.warning(f"[{request_id}] Memory injection failed: {e!r}")
@@ -6020,6 +6028,7 @@ class OpenAIHandlerMixin:
and memory_user_id
and resp_json
and response.status_code == 200
and server_memory_tool_names
and self.memory_handler.has_memory_tool_calls(resp_json, "openai")
):
try:
@@ -6028,6 +6037,7 @@ class OpenAIHandlerMixin:
memory_user_id,
"openai",
request_context=memory_request_ctx,
tool_names=server_memory_tool_names,
)
if tool_results:
# Build continuation: original messages + assistant tool_calls + tool results
@@ -6582,6 +6592,8 @@ class OpenAIHandlerMixin:
responses_memory_tools_allowed = (
_allow_responses_memory_tools(is_chatgpt_auth) and not stream
)
# Memory tools this proxy injected and runs itself (see chat above).
responses_server_memory_tool_names: frozenset[str] = frozenset()
# PR-A6 (P5-50, preps P0-6): session-sticky `OpenAI-Beta` merge
# for /v1/responses. Compute a session_id off the same store the
@@ -6873,17 +6885,22 @@ class OpenAIHandlerMixin:
memory_tool_defs_responses.append(t)
if responses_memory_tools_allowed:
resp_tools = body.get("tools") or []
client_resp_tools = body.get("tools") or []
resp_tools, mem_tools_injected = _apply_sticky_mem_tools_resp(
provider="openai",
session_id=_responses_session_id,
request_id=request_id,
existing_tools=resp_tools,
existing_tools=client_resp_tools,
memory_tools_to_inject=memory_tool_defs_responses,
inject_this_turn=bool(self.memory_handler.config.inject_tools),
client_declared_tools=client_declared_response_tools,
)
if mem_tools_injected:
from headroom.proxy.memory_handler import server_memory_tool_names
responses_server_memory_tool_names = server_memory_tool_names(
resp_tools, client_resp_tools
)
body["tools"] = resp_tools
include = body.get("include")
if isinstance(include, list):
@@ -7535,22 +7552,20 @@ class OpenAIHandlerMixin:
if (
self.memory_handler
and memory_user_id
and responses_memory_tools_allowed
and responses_server_memory_tool_names
and resp_json
and response.status_code == 200
and self.memory_handler.has_memory_tool_calls(resp_json, "openai")
):
try:
# Extract function_call items from output
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES
# Only the calls to tools this proxy injected.
output_items = resp_json.get("output", [])
memory_fc_items = [
item
for item in output_items
if isinstance(item, dict)
and item.get("type") == "function_call"
and item.get("name") in MEMORY_TOOL_NAMES
and item.get("name") in responses_server_memory_tool_names
]
# Execute memory tool calls
@@ -8624,6 +8639,8 @@ class OpenAIHandlerMixin:
memory_user_id: str | None = None
memory_request_ctx = None
# Memory tools this proxy injected on the current response.create.
memory_tool_names: frozenset[str] = frozenset()
ws_project_resolver = CodexProjectContextResolver()
ws_pinned_project_cwd = None
ws_turn_project_key = classify_project(ws_headers)
@@ -8651,7 +8668,7 @@ class OpenAIHandlerMixin:
from headroom.proxy.memory_query import MemoryQuery
async def _prepare_memory_frame(frame_body: dict[str, Any], frame_raw: str) -> str:
nonlocal memory_user_id, memory_request_ctx
nonlocal memory_user_id, memory_request_ctx, memory_tool_names
nonlocal ws_pinned_project_cwd, ws_turn_project_key
nonlocal ws_turn_project_features_allowed
nonlocal ws_turn_learning_allowed, ws_first_project_resolution
@@ -8666,6 +8683,7 @@ class OpenAIHandlerMixin:
ws_memory_generation += 1
memory_user_id = None
memory_request_ctx = None
memory_tool_names = frozenset()
first_project_resolution = ws_first_project_resolution
ws_first_project_resolution = False
@@ -8923,12 +8941,12 @@ class OpenAIHandlerMixin:
else:
ws_mem_defs_responses.append(t)
ws_tools = ws_response_body.get("tools") or []
client_ws_tools = ws_response_body.get("tools") or []
ws_tools, mem_injected = _apply_sticky_mem_tools_ws(
provider="openai",
session_id=session_id if ws_memory_tools_allowed else None,
request_id=request_id,
existing_tools=ws_tools,
existing_tools=client_ws_tools,
memory_tools_to_inject=ws_mem_defs_responses,
inject_this_turn=bool(
self.memory_handler.config.inject_tools and ws_memory_tools_allowed
@@ -8936,6 +8954,9 @@ class OpenAIHandlerMixin:
client_declared_tools=client_declared_ws_tools,
)
if mem_injected:
from headroom.proxy.memory_handler import server_memory_tool_names
memory_tool_names = server_memory_tool_names(ws_tools, client_ws_tools)
ws_response_body["tools"] = ws_tools
include = ws_response_body.get("include")
if isinstance(include, list):
@@ -9858,8 +9879,6 @@ class OpenAIHandlerMixin:
This prevents orphaned response.created events from confusing Codex.
"""
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES
# Unit 3: surface response.completed observation
# to the outer scope so the termination-cause
# classifier can prefer ``response_completed``
@@ -9890,6 +9909,7 @@ class OpenAIHandlerMixin:
response_memory_generation = ws_memory_generation
response_memory_user_id = memory_user_id
response_memory_request_ctx = memory_request_ctx
response_memory_tool_names = memory_tool_names
response_input = list(current_response_input)
response_template = dict(current_response_template)
@@ -10141,6 +10161,7 @@ class OpenAIHandlerMixin:
response_memory_generation = ws_memory_generation
response_memory_user_id = memory_user_id
response_memory_request_ctx = memory_request_ctx
response_memory_tool_names = memory_tool_names
response_input = list(current_response_input)
response_template = dict(current_response_template)
if event_type in {
@@ -10197,7 +10218,7 @@ class OpenAIHandlerMixin:
response_output_items.append(item)
if (
item.get("type") == "function_call"
and item.get("name") in MEMORY_TOOL_NAMES
and item.get("name") in response_memory_tool_names
):
pending_fcs.append(item)
@@ -10209,7 +10230,7 @@ class OpenAIHandlerMixin:
if (
isinstance(item, dict)
and item.get("type") == "function_call"
and item.get("name") in MEMORY_TOOL_NAMES
and item.get("name") in response_memory_tool_names
):
suppress_response = True
decided = True
+3 -25
View File
@@ -633,6 +633,7 @@ class StreamingMixin:
memory_user_id,
"anthropic",
request_context=memory_request_ctx,
tool_names=server_memory_tool_names,
)
logger.info(
f"[{request_id}] Memory: Executed {len(tool_results)}/"
@@ -1432,11 +1433,12 @@ class StreamingMixin:
),
}
# Track if we need to handle memory tools
# Track if we need to handle memory tools: only those the proxy injected.
memory_enabled = (
memory_user_id is not None
and self.memory_handler is not None
and provider == "anthropic"
and bool(server_memory_tool_names)
)
# Withholding headroom_retrieve needs the Anthropic SSE shape the
# filter and the continuation round are written for.
@@ -1974,30 +1976,6 @@ class StreamingMixin:
parsed_response = (
stream_state.pop("memory_client_message", None) or parsed_response
)
elif (
memory_enabled
and parsed_response
and self.memory_handler.has_memory_tool_calls(parsed_response, provider)
):
logger.info(
f"[{request_id}] Memory: Detected tool calls in streaming response"
)
# The client declared these memory tools itself, so the
# tool_use already reached it and the client continues
# the turn; only the server-side save happens here.
tool_results = await self.memory_handler.handle_memory_tool_calls(
parsed_response,
memory_user_id,
provider,
request_context=memory_request_ctx,
)
if tool_results:
logger.info(
f"[{request_id}] Memory: Tool calls executed "
f"({len(tool_results)} results saved, SSE streaming — "
"continuation handled by client)"
)
if memory_filter is not None:
# Memory detection was skipped (empty stream or buffer cap):
+32 -2
View File
@@ -17,9 +17,12 @@ Usage:
# Search and inject context
context = await handler.search_and_format_context(user_id, messages)
# Handle tool calls in response
# Handle tool calls in response, only for the tools the proxy injected
names = server_memory_tool_names(request_tools, client_tools)
if handler.has_memory_tool_calls(response, "anthropic"):
results = await handler.handle_memory_tool_calls(response, user_id, "anthropic")
results = await handler.handle_memory_tool_calls(
response, user_id, "anthropic", tool_names=names
)
"""
from __future__ import annotations
@@ -44,6 +47,7 @@ from headroom.memory.storage_router import (
ResolvedScope,
)
from headroom.proxy.public_errors import tool_result_error
from headroom.proxy.tool_name_policy import extract_tool_name
if TYPE_CHECKING:
from headroom.memory.backends.local import LocalBackend
@@ -89,6 +93,25 @@ NATIVE_MEMORY_BETA_HEADER = "context-management-2025-06-27"
# Native memory tool type
NATIVE_MEMORY_TOOL_TYPE = "memory_20250818"
def server_memory_tool_names(tools: Any, client_tools: Any) -> frozenset[str]:
"""Memory tools in ``tools`` that the proxy injected and must run itself.
``tools`` is the tool list sent upstream, ``client_tools`` the list the
client sent. A memory-named tool the client declared stays the client's:
its calls are forwarded, never withheld or executed by the proxy.
"""
client_tool_names = {extract_tool_name(t) for t in client_tools or [] if isinstance(t, dict)}
return frozenset(
name
for t in tools or []
if isinstance(t, dict)
and isinstance(name := extract_tool_name(t), str)
and (name in MEMORY_TOOL_NAMES or name == NATIVE_MEMORY_TOOL_NAME)
and name not in client_tool_names
)
# Maximum time to wait for a single backend initialization (one-shot).
# Applies to MemoryHandler._ensure_initialized. On timeout, _initialized
# stays False so that subsequent requests retry instead of deadlocking.
@@ -1125,6 +1148,8 @@ your responses, not to drive new actions."""
user_id: str,
provider: str = "anthropic",
request_context: RequestContext | None = None,
*,
tool_names: frozenset[str],
) -> list[dict[str, Any]]:
"""Execute memory tool calls and return results.
@@ -1136,6 +1161,9 @@ your responses, not to drive new actions."""
save/search/update/delete operations route to the per-
workspace DB so projects cannot read or overwrite each
other's memories (GH #462).
tool_names: The memory tools the proxy injected on this request
(see ``server_memory_tool_names``). Calls to any other tool,
memory-named or not, are the client's and are skipped.
Returns:
List of tool results in provider format.
@@ -1151,6 +1179,8 @@ your responses, not to drive new actions."""
# the following `.get` would raise AttributeError on a malformed /
# partial upstream tool call. Coalesce to {}.
tool_name = tc.get("name") or (tc.get("function") or {}).get("name")
if tool_name not in tool_names:
continue
tool_id = tc.get("id") or tc.get("call_id", "")
# Parse input data
+14 -1
View File
@@ -659,7 +659,11 @@ async def test_ws_memory_dispatch_uses_current_turn_project(
monkeypatch.setenv("CODEX_HOME", str(codex_home))
def frame(project: str | None) -> str:
response = {"model": "gpt-5.4", "input": "hello"}
response = {
"model": "gpt-5.4",
"input": "hello",
"tools": [{"type": "function", "name": "shell", "parameters": {}}],
}
if project:
response["client_metadata"] = {
"thread_id": f"thread-{project}",
@@ -750,6 +754,15 @@ async def test_ws_memory_dispatch_uses_current_turn_project(
self.config.project_root_override = ""
self.executed = []
def compute_memory_tool_definitions(self, provider):
# Offer the tool the model calls: the proxy only runs tools it injected.
return [
{
"type": "function",
"function": {"name": tool_name, "parameters": {"type": "object"}},
}
]
async def _execute_memory_tool(
self, name, args, user_id, provider, *, request_context=None
):
+12 -1
View File
@@ -9,7 +9,14 @@ from types import SimpleNamespace
import pytest
from headroom.proxy import memory_handler as memory_handler_module
from headroom.proxy.memory_handler import MemoryConfig, MemoryHandler
from headroom.proxy.memory_handler import (
MEMORY_TOOL_NAMES,
NATIVE_MEMORY_TOOL_NAME,
MemoryConfig,
MemoryHandler,
)
ALL_MEMORY_TOOLS = frozenset(MEMORY_TOOL_NAMES | {NATIVE_MEMORY_TOOL_NAME})
@pytest.fixture
@@ -1268,6 +1275,7 @@ async def test_search_and_format_context_and_handle_memory_tool_calls(
},
"u1",
"anthropic",
tool_names=ALL_MEMORY_TOOLS,
)
assert anthropic_results == [
{
@@ -1299,6 +1307,7 @@ async def test_search_and_format_context_and_handle_memory_tool_calls(
},
"u1",
"openai",
tool_names=ALL_MEMORY_TOOLS,
)
assert openai_results == [
{
@@ -1326,6 +1335,7 @@ async def test_search_and_format_context_and_handle_memory_tool_calls(
},
"u1",
"openai",
tool_names=ALL_MEMORY_TOOLS,
)
assert skipped == []
@@ -1705,6 +1715,7 @@ async def test_extract_tool_calls_and_handle_tool_calls_parse_edges(
},
"u1",
"openai",
tool_names=ALL_MEMORY_TOOLS,
)
assert results == [{"role": "tool", "tool_call_id": "fc1", "content": "ok:memory_search:{}"}]
@@ -400,6 +400,7 @@ def test_unresolved_project_native_memory_tool_fails_closed(tmp_path: Path) -> N
"alice",
"anthropic",
request_context=ctx,
tool_names=frozenset({"memory", "memory_save"}),
)
)
+7 -9
View File
@@ -733,14 +733,14 @@ class TestContinuationFallbacks:
class TestStreamingMemoryFallbacks:
@pytest.mark.asyncio
async def test_client_handled_memory_calls_still_run_without_server_tools(self) -> None:
async def test_memory_calls_the_proxy_did_not_inject_are_left_to_the_client(self) -> None:
raw = _sse([TEXT, SAVE], "tool_use")
proxy = _proxy([raw], SAVE_RESULT)
proxy.memory_handler.has_memory_tool_calls = MagicMock(return_value=True)
events = await _client_view(proxy)
assert events == _events(raw)
proxy.memory_handler.handle_memory_tool_calls.assert_awaited_once()
proxy.memory_handler.handle_memory_tool_calls.assert_not_awaited()
@pytest.mark.asyncio
@pytest.mark.parametrize("server_tools", [MEMORY_TOOLS, frozenset()])
@@ -777,7 +777,7 @@ class TestStreamingMemoryFallbacks:
class TestServerMemoryToolNames:
def test_injected_memory_tools_are_server_side(self) -> None:
from headroom.proxy.handlers.anthropic import AnthropicHandlerMixin
from headroom.proxy.memory_handler import server_memory_tool_names
tools = [
{"name": "Bash"},
@@ -787,18 +787,16 @@ class TestServerMemoryToolNames:
{"type": "web_search"},
"not-a-tool",
]
names = AnthropicHandlerMixin._server_memory_tool_names(tools, [{"name": "Bash"}])
names = server_memory_tool_names(tools, [{"name": "Bash"}])
assert names == frozenset({"memory_save", "memory_search", "memory"})
def test_client_declared_memory_tool_stays_with_the_client(self) -> None:
from headroom.proxy.handlers.anthropic import AnthropicHandlerMixin
from headroom.proxy.memory_handler import server_memory_tool_names
tools = [{"name": "memory_save"}, {"name": "memory_search"}]
names = AnthropicHandlerMixin._server_memory_tool_names(
tools, [{"name": "memory_save"}, "junk"]
)
names = server_memory_tool_names(tools, [{"name": "memory_save"}, "junk"])
assert names == frozenset({"memory_search"})
assert AnthropicHandlerMixin._server_memory_tool_names(None, None) == frozenset()
assert server_memory_tool_names(None, None) == frozenset()
class TestHandlerPassesServerMemoryTools:
@@ -0,0 +1,410 @@
"""The proxy runs only the memory tools it injected on the request.
A memory-named call the proxy did not offer (``--no-memory-tools``) or that
the client declared itself belongs to the client: it reaches the client
unchanged, the memory backend is not touched, and no continuation is sent
upstream. Calls to tools the proxy did inject still run on the proxy.
"""
from __future__ import annotations
import asyncio
import json
import sys
from types import SimpleNamespace
from typing import Any
from unittest.mock import MagicMock, patch
import httpx
import pytest
from fastapi.testclient import TestClient
from headroom.proxy.server import ProxyConfig, create_app
class _FakeBackend:
def __init__(self) -> None:
self.calls: list[str] = []
async def list_memories(self, *, user_id: str, limit: int) -> list[Any]:
self.calls.append("list")
return [SimpleNamespace(id="mem-1", content="stored", created_at=None)]
async def delete_memory(self, memory_id: str, *args: Any, **kwargs: Any) -> bool:
self.calls.append("delete")
return True
def _make_app(tmp_path, *, inject_tools: bool) -> tuple[Any, _FakeBackend]:
config = ProxyConfig(
openai_api_url="https://openai.invalid",
anthropic_api_url="https://anthropic.invalid",
optimize=False,
image_optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
discover_pipeline_extensions=False,
compression_max_workers=1,
memory_enabled=True,
memory_storage_mode="global",
memory_inject_tools=inject_tools,
memory_inject_context=False,
memory_db_path=str(tmp_path / "memory.db"),
)
app = create_app(config)
handler = app.state.proxy.memory_handler
backend = _FakeBackend()
handler._backend = backend
handler._initialized = True
router = handler._router
if router is not None:
router.backend_for = lambda ctx: (backend, router.scope_for(ctx))
return app, backend
# --- upstream turns ---------------------------------------------------------
def _chat_turn(name: str | None) -> dict[str, Any]:
if name is None:
message: dict[str, Any] = {"role": "assistant", "content": "done"}
else:
message = {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": name, "arguments": '{"memory_id": "mem-1"}'},
}
],
}
return {
"id": "chatcmpl-1",
"object": "chat.completion",
"model": "gpt-4o-mini",
"choices": [{"index": 0, "message": message, "finish_reason": "stop"}],
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
}
def _responses_item(name: str) -> dict[str, Any]:
return {
"type": "function_call",
"id": "fc_1",
"call_id": "call_1",
"name": name,
"arguments": '{"memory_id": "mem-1"}',
"status": "completed",
}
def _responses_turn(name: str | None) -> dict[str, Any]:
item = (
_responses_item(name)
if name
else {
"type": "message",
"id": "msg_1",
"role": "assistant",
"status": "completed",
"content": [{"type": "output_text", "text": "done", "annotations": []}],
}
)
return {
"id": "resp_1",
"object": "response",
"status": "completed",
"model": "gpt-4o-mini",
"output": [item],
"usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2},
}
def _anthropic_turn(name: str | None) -> dict[str, Any]:
if name is None:
content: list[dict[str, Any]] = [{"type": "text", "text": "done"}]
else:
content = [
{"type": "tool_use", "id": "toolu_1", "name": name, "input": {"memory_id": "mem-1"}}
]
return {
"id": "msg_1",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-5",
"content": content,
"stop_reason": "tool_use" if name else "end_turn",
"stop_sequence": None,
"usage": {"input_tokens": 1, "output_tokens": 1},
}
def _anthropic_sse(name: str) -> bytes:
turn = _anthropic_turn(name)
events = [
("message_start", {"type": "message_start", "message": {**turn, "content": []}}),
(
"content_block_start",
{
"type": "content_block_start",
"index": 0,
"content_block": {"type": "tool_use", "id": "toolu_1", "name": name, "input": {}},
},
),
(
"content_block_delta",
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "input_json_delta", "partial_json": '{"memory_id": "mem-1"}'},
},
),
("content_block_stop", {"type": "content_block_stop", "index": 0}),
(
"message_delta",
{
"type": "message_delta",
"delta": {"stop_reason": "tool_use", "stop_sequence": None},
"usage": {"output_tokens": 1},
},
),
("message_stop", {"type": "message_stop"}),
]
return "".join(f"event: {e}\ndata: {json.dumps(d)}\n\n" for e, d in events).encode()
# --- HTTP paths -------------------------------------------------------------
_OPENAI_TOOL = {
"type": "function",
"function": {"name": "shell", "parameters": {"type": "object", "properties": {}}},
}
_OPENAI_CLIENT_MEMORY_DELETE = {
"type": "function",
"function": {"name": "memory_delete", "parameters": {"type": "object", "properties": {}}},
}
_RESPONSES_TOOL = {"type": "function", "name": "shell", "parameters": {"type": "object"}}
_RESPONSES_CLIENT_MEMORY_DELETE = {
"type": "function",
"name": "memory_delete",
"parameters": {"type": "object"},
}
_ANTHROPIC_TOOL = {"name": "Bash", "input_schema": {"type": "object"}}
_ANTHROPIC_CLIENT_MEMORY_DELETE = {"name": "memory_delete", "input_schema": {"type": "object"}}
_PATHS: dict[str, dict[str, Any]] = {
"chat": {
"url": "/v1/chat/completions",
"headers": {"authorization": "Bearer FAKE-KEY"},
"body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "hi"}]},
"tool": _OPENAI_TOOL,
"client_memory_delete": _OPENAI_CLIENT_MEMORY_DELETE,
"turn": _chat_turn,
},
"responses": {
"url": "/v1/responses",
"headers": {"authorization": "Bearer FAKE-KEY"},
"body": {"model": "gpt-4o-mini", "input": "hi"},
"tool": _RESPONSES_TOOL,
"client_memory_delete": _RESPONSES_CLIENT_MEMORY_DELETE,
"turn": _responses_turn,
},
"anthropic": {
"url": "/v1/messages",
"headers": {"x-api-key": "FAKE-KEY", "anthropic-version": "2023-06-01"},
"body": {
"model": "claude-sonnet-4-5",
"max_tokens": 64,
"messages": [{"role": "user", "content": "hi"}],
},
"tool": _ANTHROPIC_TOOL,
"client_memory_delete": _ANTHROPIC_CLIENT_MEMORY_DELETE,
"turn": _anthropic_turn,
},
"anthropic_stream": {
"url": "/v1/messages",
"headers": {"x-api-key": "FAKE-KEY", "anthropic-version": "2023-06-01"},
"body": {
"model": "claude-sonnet-4-5",
"max_tokens": 64,
"stream": True,
"messages": [{"role": "user", "content": "hi"}],
},
"tool": _ANTHROPIC_TOOL,
"client_memory_delete": _ANTHROPIC_CLIENT_MEMORY_DELETE,
"turn": _anthropic_turn,
},
}
async def _post(app: Any, kind: str, tools: list[dict[str, Any]], call: str) -> tuple:
spec = _PATHS[kind]
upstream_bodies: list[dict[str, Any]] = []
def upstream(request: httpx.Request) -> httpx.Response:
upstream_bodies.append(json.loads(request.content))
first = len(upstream_bodies) == 1
if kind == "anthropic_stream" and first:
return httpx.Response(
200, content=_anthropic_sse(call), headers={"content-type": "text/event-stream"}
)
return httpx.Response(200, json=spec["turn"](call if first else None))
mock = httpx.AsyncClient(transport=httpx.MockTransport(upstream))
app.state.proxy.http_client = mock
app.state.proxy.http_client_h1 = mock
transport = httpx.ASGITransport(app=app, client=("127.0.0.1", 12345))
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:
response = await client.post(
spec["url"],
json={**spec["body"], "tools": tools},
headers={**spec["headers"], "x-headroom-user-id": "user-1"},
timeout=30,
)
await mock.aclose()
return response, upstream_bodies
@pytest.mark.parametrize("kind", list(_PATHS))
@pytest.mark.parametrize("case", ["not_injected", "client_declared"])
async def test_memory_call_the_proxy_did_not_inject_reaches_the_client(
tmp_path, monkeypatch, kind: str, case: str
) -> None:
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
spec = _PATHS[kind]
app, backend = _make_app(tmp_path, inject_tools=case == "client_declared")
tools = [spec["tool"]]
if case == "client_declared":
tools.append(spec["client_memory_delete"])
response, upstream_bodies = await _post(app, kind, tools, "memory_delete")
assert response.status_code == 200
assert backend.calls == []
assert len(upstream_bodies) == 1
assert "memory_delete" in response.text
assert ("toolu_1" if kind.startswith("anthropic") else "call_1") in response.text
@pytest.mark.parametrize("kind", ["chat", "responses", "anthropic"])
async def test_memory_call_the_proxy_injected_still_runs(tmp_path, monkeypatch, kind: str) -> None:
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
app, backend = _make_app(tmp_path, inject_tools=True)
response, upstream_bodies = await _post(app, kind, [_PATHS[kind]["tool"]], "memory_list")
assert response.status_code == 200
assert backend.calls == ["list"]
assert len(upstream_bodies) == 2
assert "memory_list" not in response.text
# --- Responses over WebSocket ---------------------------------------------
class _Upstream:
"""Upstream WS that answers the first response.create with one tool call."""
def __init__(self, call: str) -> None:
self.call = call
self.sent: list[dict[str, Any]] = []
self._events: asyncio.Queue[str] = asyncio.Queue()
async def __aenter__(self) -> _Upstream:
return self
async def __aexit__(self, *exc: object) -> None:
return None
async def send(self, payload: str) -> None:
self.sent.append(json.loads(payload))
rid = f"resp_{len(self.sent)}"
if len(self.sent) == 1:
item = _responses_item(self.call)
else:
item = _responses_turn(None)["output"][0]
for event in (
{"type": "response.created", "response": {"id": rid}},
{"type": "response.output_item.added", "item": item},
{"type": "response.output_item.done", "item": item},
{"type": "response.completed", "response": {"id": rid}},
):
await self._events.put(json.dumps(event))
async def close(self) -> None:
return None
def __aiter__(self) -> _Upstream:
return self
async def __anext__(self) -> str:
return await self._events.get()
def _ws_exchange(
app: Any, tools: list[dict[str, Any]], call: str
) -> tuple[list[dict[str, Any]], _Upstream]:
upstream = _Upstream(call)
async def connect(*_args: Any, **_kwargs: Any) -> _Upstream:
return upstream
websockets_module = MagicMock()
websockets_module.connect = connect
websockets_module.Subprotocol = str
frame = {
"type": "response.create",
"response": {"model": "gpt-4o-mini", "input": "hi", "tools": tools},
}
received: list[dict[str, Any]] = []
with patch.dict(sys.modules, {"websockets": websockets_module}):
client = TestClient(app, client=("127.0.0.1", 12345))
with client.websocket_connect(
"/v1/responses",
headers={"authorization": "Bearer FAKE-KEY", "x-headroom-user-id": "user-1"},
) as websocket:
websocket.send_text(json.dumps(frame))
while True:
event = json.loads(websocket.receive_text())
received.append(event)
if event.get("type") == "response.completed":
break
return received, upstream
@pytest.mark.parametrize("case", ["not_injected", "client_declared"])
def test_ws_memory_call_the_proxy_did_not_inject_reaches_the_client(
tmp_path, monkeypatch, case: str
) -> None:
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
app, backend = _make_app(tmp_path, inject_tools=case == "client_declared")
tools = [_RESPONSES_TOOL]
if case == "client_declared":
tools.append(_RESPONSES_CLIENT_MEMORY_DELETE)
received, upstream = _ws_exchange(app, tools, "memory_delete")
assert backend.calls == []
assert len(upstream.sent) == 1
assert any(
event.get("type") == "response.output_item.done"
and event["item"].get("name") == "memory_delete"
for event in received
)
def test_ws_memory_call_the_proxy_injected_still_runs(tmp_path, monkeypatch) -> None:
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
app, backend = _make_app(tmp_path, inject_tools=True)
received, upstream = _ws_exchange(app, [_RESPONSES_TOOL], "memory_list")
assert backend.calls == ["list"]
assert len(upstream.sent) == 2
assert not any(event.get("item", {}).get("name") == "memory_list" for event in received)