mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): accept HEADROOM_MEMORY_INJECTION_MODE=tool (#4395)
## Description `memory_decision.py` handles mode `tool`, but `resolve_memory_injection_mode` rejected it with `ValueError`, so the value could not be used. The resolver now accepts `tool`; the configuration docs list it. Second commit, after review (Devin, comment 4348232164): in `tool` mode the old `MemoryDecision` returned `inject=False`, and the handlers gate the memory tool definitions on that same flag, so the agent would have had no `memory_search` to call. The intent of `tool` (commit71d5a7b5: no auto-injected context, the model calls the memory tools itself) is now implemented: the decision stays open for `tool`, and `MemoryHandler.search_and_format_context` skips the context lookup, as it already did for `MemoryMode.TOOL`. Fixes #4391 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/memory_injection_mode_policy.py`: add `"tool"` to the literal type and the accepted values; update the error text. - `headroom/proxy/helpers.py`: comment documents `tool`. - `docs/content/docs/configuration.mdx`: list `tool` for `HEADROOM_MEMORY_INJECTION_MODE`. - `tests/test_memory_injection_mode_policy.py`: resolver accepts `tool` (case/space insensitive); `get_memory_injection_mode()` returns `tool` from the env var. - `headroom/proxy/memory_decision_policy.py`, `memory_decision.py`: `tool` no longer produces a skip (`mode_tool` removed); `inject` stays True so the handlers register the memory tools. - `headroom/proxy/memory_handler.py`: `search_and_format_context` returns None when `HEADROOM_MEMORY_INJECTION_MODE=tool` (before any backend lookup). - `tests/test_memory_decision.py`, `tests/test_memory_decision_policy.py`: `tool` expectations changed to `inject=True`, no skip reason; new test in `test_memory_injection_mode_policy.py` covers the context skip and the open decision. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # with the policy change reverted (tests/test_memory_injection_mode_policy.py) E ValueError: Invalid HEADROOM_MEMORY_INJECTION_MODE='tool'; expected 'live_zone_tail' or 'disabled' FAILED tests/test_memory_injection_mode_policy.py::test_resolve_memory_injection_mode_accepts_known_values_case_insensitively FAILED tests/test_memory_injection_mode_policy.py::test_get_memory_injection_mode_accepts_tool_from_env 2 failed, 2 passed # second commit: new test with the src changes of that commit reverted E assert decision.inject is True / MemoryDecision(inject=False, skip_reason='mode_tool', ...) 1 failed, 4 passed # second commit applied $ pytest tests/test_memory_injection_mode_policy.py tests/test_memory_decision.py tests/test_memory_decision_policy.py tests/test_memory_handler_concurrent_init.py 37 passed, 1 skipped # first commit, the four files listed under Real Behavior Proof 6 failed, 25 passed (all 6 failures in test_proxy_system_prompt_immutable.py, same 6 on unmodified upstream/main) $ ruff check <changed py files> All checks passed! $ ruff format --check <changed py files> 3 files already formatted ``` ## Real Behavior Proof - Environment: Linux (WSL2), Python 3.12.15, the real proxy server (`run_server`) with persistent memory on (local backend, ONNX embedder), Rust core from the headroom-ai 0.41.0 wheel, a local stub upstream on 127.0.0.1 - Exact command / steps: seed one memory for a test user, start the proxy with `HEADROOM_MEMORY_INJECTION_MODE=tool`, send one non-streaming `POST /v1/messages` that declares one client tool; repeat on upstream/main and with `live_zone_tail` as controls (details below) - Observed result: upstream/main returns HTTP 500 (`ValueError` from the mode resolver); this branch returns HTTP 200, the stub receives the five memory tools and the user message without a `## Relevant Memories` block - Not tested: real Anthropic, streaming, the OpenAI/Gemini handlers and the WebSocket path, `use_native_tool`, a memory tool-call round trip, a Rust core built from this checkout Details: Setup: WSL2 Ubuntu (Linux 6.18 x86_64), Python 3.12.15, uvicorn 0.54.0, fastapi 0.158.1, starlette 1.9.0, numpy 2.5.4, sqlite-vec 0.1.9, onnxruntime (memory deps installed into a side directory, not the repo). Worktree at PR head `9744d2a1` (base `e995eb9f`). `headroom/_core.abi3.so` was copied (untracked, not committed) from the official `headroom_ai-0.41.0-cp310-abi3-manylinux_2_28_x86_64.whl` because the checkout has no built Rust core. Stub upstream (`stub_mem.py`, 127.0.0.1:18796): answers every `POST /v1/messages` with a normal 200 message and logs the tool names and the last user message it received. Proxy: the real server (`run_server`) with `memory_enabled=True`, `memory_storage_mode="global"`, local backend with the ONNX embedder, `ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18796`, `HEADROOM_MEMORY_INJECTION_MODE=tool`, port 18795, `--no-cache --no-rate-limit` equivalents. (`headroom proxy --memory` was not used: the CLI needs the optional `mcp` package, which is not installed in this environment. `mem_proxy.py` builds the same `ProxyConfig` and calls `run_server`.) Memory content: before each run, `mem_seed.py` saves one memory for user `proofuser` ("The user prefers PostgreSQL 16 with pgvector for every database.") through the real `MemoryHandler`/`LocalBackend`; a search for it returns 1 hit. Client (`mem_client.py`): one non-streaming `POST /v1/messages` with `x-headroom-user-id: proofuser`, the message "Which database do I prefer?" and one client tool `get_weather` (the memory tools are injected only when the client declares tools). Before = `upstream/main` (`e995eb9f`). After = PR head (`9744d2a1`). ### Before (upstream/main) ``` git head:e995eb9fHEADROOM_MEMORY_INJECTION_MODE=tool client: HTTP 500 Internal Server Error ERROR - event=proxy_inbound_request_aborted ... path=/v1/messages reason=ValueError ValueError: Invalid HEADROOM_MEMORY_INJECTION_MODE='tool'; expected 'live_zone_tail' or 'disabled' --- stub upstream received (nothing received) ``` Every request fails at `get_memory_injection_mode()` (`headroom/proxy/helpers.py:267` -> `memory_injection_mode_policy.py:19`) and never reaches the upstream. ### After (PR head9744d2a1) ``` git head:9744d2a1HEADROOM_MEMORY_INJECTION_MODE=tool client: HTTP 200 {"id": "msg_ok", "type": "message", "role": "assistant", "model": "claude-sonnet-4-5", "content": [{"type": "text", "text": "ok"}], "stop_reason": "end_turn", "usage": {"input_tokens": 10, "output_tokens": 2}} INFO - event=memory_mode_skip mode=tool user_id=proofuser reason=tool_mode_no_auto_injection INFO - [hr_1792672967_000001] Memory: Injected tools: ['memory_save', 'memory_search', 'memory_update', 'memory_delete', 'memory_list'] --- stub upstream received STUB-UPSTREAM POST /v1/messages tools=['get_weather', 'headroom_retrieve', 'memory_delete', 'memory_list', 'memory_save', 'memory_search', 'memory_update'] STUB-UPSTREAM last_user_content="Which database do I prefer?" ``` The request succeeds, the upstream receives the five memory tool definitions next to the client's own tool, and the user message is forwarded unchanged (no `## Relevant Memories` block; `grep -c "Relevant Memories"` on the stub log = 0). ### Control 1: default mode on the PR head (shows the seeded memory is retrievable) Same setup, `HEADROOM_MEMORY_INJECTION_MODE=live_zone_tail`: ``` INFO - event=memory_inject user=proofuser scope=global count=1 chars=793 budget_tokens=1024 INFO - event=memory_injection ... decision=injected_live_zone_tail bytes_injected=793 STUB-UPSTREAM ... tools=['get_weather', 'headroom_retrieve', 'memory_delete', 'memory_list', 'memory_save', 'memory_search', 'memory_update'] STUB-UPSTREAM last_user_content="Which database do I prefer?\n\n## Relevant Memories (scope: global)\n\nThese are READ-ONLY entries recalled from prior sessions ..." ``` So with the same memory and the same request, `live_zone_tail` injects the memory context into the user message, and `tool` does not. ### Control 2: only the first commit (62ec86b5, "accept tool") Same setup, `HEADROOM_MEMORY_INJECTION_MODE=tool`: ``` git head:62ec86b5HEADROOM_MEMORY_INJECTION_MODE=tool client: HTTP 200 {...} STUB-UPSTREAM POST /v1/messages tools=['get_weather', 'headroom_retrieve'] STUB-UPSTREAM last_user_content="Which database do I prefer?" ``` The request no longer fails, but the memory tool definitions are missing. This is the case the second commit (`9744d2a1`) fixes. ### Commands ``` python -P mem_seed.py <db> # PYTHONPATH=.:deps-mem, seeds one memory for proofuser python -P stub_mem.py 18796 stub.log & HEADROOM_MEMORY_INJECTION_MODE=tool ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18796 PYTHONPATH=.:deps-mem \ python -P mem_proxy.py 18795 <db> & python -P mem_client.py 18795 ``` Scripts and raw outputs: `/var/tmp/cb-pub/proof/` (WSL): `run_mem.sh`, `mem_proxy.py`, `mem_seed.py`, `stub_mem.py`, `mem_client.py`, `mem_{before,after,control,commit1}.proxy.log`, `mem_{before,after,control,commit1}.stub.log`, `before_mem.out`, `after_mem.out`, `control_mem.out`, `commit1_mem.out`. ### Not tested - Real Anthropic API (stub upstream only). - Streaming requests, the OpenAI Chat/Responses handlers, Gemini, and the WebSocket path; only Anthropic `POST /v1/messages` non-streaming was run. - `use_native_tool` (native `memory_20250818` tool) and `ProxyConfig.memory_mode="tool"`; only the env var was exercised. - Memory tool execution (a `memory_save` call round trip) in tool mode. - The Rust core is the 0.41.0 wheel's, not one built from this checkout. - `headroom proxy --memory` via the CLI (the `mcp` package is not installed here). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: no (default stays `live_zone_tail`; only a previously rejected value is accepted) - Kill switch / disable path: unset the env var - Unsafe override required: no - Qualification impact: none expected - Rollback path: revert the commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (6 existing failures in test_proxy_system_prompt_immutable.py are identical on unmodified upstream/main in my environment) - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes If maintainers would rather drop `tool` than support it, the alternative is removing it from the resolver and the docs. Produced with AI assistance (Claude Code) operated by me (HetCreep); the reproduction was run on my machine on a clone of the repository. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
9202948c1e
commit
f89c9282a8
@@ -383,7 +383,7 @@ headroom proxy --learn --min-evidence 3
|
||||
| `HEADROOM_TOIN_PATH` | Override TOIN telemetry file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |
|
||||
| `HEADROOM_SUBSCRIPTION_STATE_PATH` | Override subscription tracker state file. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |
|
||||
| `HEADROOM_PERIODIC_TOIN_STATS` | Controls periodic TOIN stats logging in long-lived proxy workers. Set to `0`, `false`, `off`, or `no` to disable the 5-minute stats loop without disabling TOIN learning or request-time feedback. | `true` |
|
||||
| `HEADROOM_MEMORY_INJECTION_MODE` | Memory-context routing mode: `live_zone_tail` (default) or `disabled`. The legacy `system_prompt` mode was retired by PR-A2; supplying it raises. | `live_zone_tail` |
|
||||
| `HEADROOM_MEMORY_INJECTION_MODE` | Memory-context routing mode: `live_zone_tail` (default), `disabled`, or `tool` (auto-injection off; the agent calls memory tools explicitly). The legacy `system_prompt` mode was retired by PR-A2; supplying it raises. | `live_zone_tail` |
|
||||
| `HEADROOM_PROXY_PYTHON_FORWARDER_MODE` | Python forwarder serialization mode. `byte_faithful` (default) forwards original request bytes verbatim when no transform mutated the body and re-serializes canonically only when needed — keeps Anthropic prompt-cache hit-rate intact. `legacy_json_kwarg` is an explicit operator opt-in for emergency rollback to the historical `httpx ... json=body` behavior. NOT a fallback — only flip on explicit operator decision. | `byte_faithful` |
|
||||
| `HEADROOM_STRIP_INTERNAL_HEADERS` | Python proxy: whether to strip internal `x-headroom-*` request headers (e.g. `x-headroom-bypass`, `x-headroom-mode`, `x-headroom-user-id`, `x-headroom-stack`, `x-headroom-base-url`) before every upstream forwarder call (PR-A5, fixes P5-49). `enabled` (default) stops fingerprinting / leakage. `disabled` is an explicit operator opt-in for diagnostic shadow tracing — NOT a fallback. Inbound reads of these headers (bypass gating, memory user-id resolution) are unaffected because they read `request.headers` directly. | `enabled` |
|
||||
| `HEADROOM_PROXY_STRIP_INTERNAL_HEADERS` | Rust proxy: same policy as `HEADROOM_STRIP_INTERNAL_HEADERS` but for the Rust transparent proxy. Stripping happens inside `build_forward_request_headers` so both HTTP and WebSocket upstream calls are gated by one flag. `enabled` default; `disabled` operator opt-in for diagnostic shadow tracing. Response-side `X-Headroom-*` injection (e.g. `x-headroom-tokens-saved`) is unrelated and stays. | `enabled` |
|
||||
|
||||
@@ -250,6 +250,9 @@ def capture_codex_wire_debug(
|
||||
# prefix) is never mutated.
|
||||
# - "disabled": Memory context lookup is skipped entirely; the request
|
||||
# forwards untouched.
|
||||
# - "tool": Auto-injection of context is off, but the memory tools are still
|
||||
# registered; the agent calls them explicitly (``search_and_format_context``
|
||||
# returns None in this mode).
|
||||
#
|
||||
# Configurable via HEADROOM_MEMORY_INJECTION_MODE env var. There is no
|
||||
# "system_prompt" option — that path is permanently retired by I2 (cache hot
|
||||
|
||||
@@ -20,9 +20,7 @@ Precedence (highest first):
|
||||
2. ``no_handler`` — no memory backend configured
|
||||
3. ``no_user_id`` — per-request user_id missing
|
||||
4. ``mode_disabled`` — operator HEADROOM_MEMORY_INJECTION_MODE=disabled
|
||||
5. ``mode_tool`` — operator HEADROOM_MEMORY_INJECTION_MODE=tool
|
||||
(auto-inject off; the agent calls memory tools explicitly)
|
||||
6. otherwise → ``inject=True``
|
||||
5. otherwise → ``inject=True``
|
||||
|
||||
This module exposes one value type + one factory + one helper. Pure
|
||||
function; same Rust-port shape as :class:`CompressionDecision`.
|
||||
@@ -59,7 +57,6 @@ class MemoryDecision:
|
||||
# * "no_handler" — no memory backend configured on the proxy
|
||||
# * "no_user_id" — per-request user_id missing
|
||||
# * "mode_disabled" — operator HEADROOM_MEMORY_INJECTION_MODE=disabled
|
||||
# * "mode_tool" — operator HEADROOM_MEMORY_INJECTION_MODE=tool
|
||||
# When ``inject`` is True, this is None.
|
||||
skip_reason: str | None
|
||||
|
||||
|
||||
@@ -42,9 +42,6 @@ def decide_memory_injection(
|
||||
elif mode_name == "disabled":
|
||||
reason = "mode_disabled"
|
||||
inject = False
|
||||
elif mode_name == "tool":
|
||||
reason = "mode_tool"
|
||||
inject = False
|
||||
else:
|
||||
reason = None
|
||||
inject = True
|
||||
|
||||
@@ -759,9 +759,12 @@ class MemoryHandler:
|
||||
if not self.config.inject_context:
|
||||
return None
|
||||
|
||||
from headroom.proxy.helpers import get_memory_injection_mode
|
||||
|
||||
# PR-B6: Tool mode disables auto-injection. The model calls
|
||||
# ``memory_search`` to retrieve when it wants to.
|
||||
if self.config.mode == MemoryMode.TOOL:
|
||||
# ``memory_search`` to retrieve when it wants to. Selected by the
|
||||
# handler config or by HEADROOM_MEMORY_INJECTION_MODE=tool.
|
||||
if self.config.mode == MemoryMode.TOOL or get_memory_injection_mode() == "tool":
|
||||
logger.info(
|
||||
"event=memory_mode_skip mode=tool user_id=%s reason=tool_mode_no_auto_injection",
|
||||
user_id,
|
||||
|
||||
@@ -5,8 +5,8 @@ from __future__ import annotations
|
||||
from typing import Literal, cast
|
||||
|
||||
MEMORY_INJECTION_MODE_ENV = "HEADROOM_MEMORY_INJECTION_MODE"
|
||||
MEMORY_INJECTION_MODE_DEFAULT: Literal["live_zone_tail", "disabled"] = "live_zone_tail"
|
||||
MemoryInjectionMode = Literal["live_zone_tail", "disabled"]
|
||||
MEMORY_INJECTION_MODE_DEFAULT: Literal["live_zone_tail", "disabled", "tool"] = "live_zone_tail"
|
||||
MemoryInjectionMode = Literal["live_zone_tail", "disabled", "tool"]
|
||||
|
||||
|
||||
def resolve_memory_injection_mode(raw: str | None) -> MemoryInjectionMode:
|
||||
@@ -14,9 +14,9 @@ def resolve_memory_injection_mode(raw: str | None) -> MemoryInjectionMode:
|
||||
normalized = (raw or "").strip().lower()
|
||||
if not normalized:
|
||||
return MEMORY_INJECTION_MODE_DEFAULT
|
||||
if normalized in ("live_zone_tail", "disabled"):
|
||||
if normalized in ("live_zone_tail", "disabled", "tool"):
|
||||
return cast(MemoryInjectionMode, normalized)
|
||||
raise ValueError(
|
||||
f"Invalid {MEMORY_INJECTION_MODE_ENV}={normalized!r}; "
|
||||
"expected 'live_zone_tail' or 'disabled'"
|
||||
"expected 'live_zone_tail', 'disabled' or 'tool'"
|
||||
)
|
||||
|
||||
@@ -143,14 +143,15 @@ def test_mode_disabled_is_skip() -> None:
|
||||
assert d.skip_reason == "mode_disabled"
|
||||
|
||||
|
||||
def test_mode_tool_is_skip() -> None:
|
||||
"""TOOL mode: auto-injection disabled (the agent calls memory
|
||||
tools explicitly instead). Distinct from disabled."""
|
||||
def test_mode_tool_keeps_tools_registered() -> None:
|
||||
"""TOOL mode: context auto-injection is off (see
|
||||
MemoryHandler.search_and_format_context) but the decision stays open
|
||||
so the memory tools are still registered for the agent."""
|
||||
d = MemoryDecision.decide(
|
||||
headers={}, memory_handler=_memory_handler(), memory_user_id="u1", mode_name="tool"
|
||||
)
|
||||
assert d.inject is False
|
||||
assert d.skip_reason == "mode_tool"
|
||||
assert d.inject is True
|
||||
assert d.skip_reason is None
|
||||
|
||||
|
||||
# ── Precedence ordering when multiple gates close ────────────────────
|
||||
@@ -197,7 +198,7 @@ def test_mode_disabled_beats_mode_tool() -> None:
|
||||
headers={}, memory_handler=_memory_handler(), memory_user_id="u1", mode_name="tool"
|
||||
)
|
||||
assert d_dis.skip_reason == "mode_disabled"
|
||||
assert d_tool.skip_reason == "mode_tool"
|
||||
assert d_tool.skip_reason is None
|
||||
|
||||
|
||||
# ── Observability fields ─────────────────────────────────────────────
|
||||
@@ -300,12 +301,6 @@ def test_apply_to_tags_for_every_skip_reason() -> None:
|
||||
"memory_user_id": "u1",
|
||||
"mode_name": "disabled",
|
||||
},
|
||||
"mode_tool": {
|
||||
"headers": {},
|
||||
"memory_handler": _memory_handler(),
|
||||
"memory_user_id": "u1",
|
||||
"mode_name": "tool",
|
||||
},
|
||||
}
|
||||
for expected, kwargs in cases.items():
|
||||
d = MemoryDecision.decide(**kwargs)
|
||||
|
||||
@@ -50,7 +50,8 @@ def test_decide_memory_injection_reports_mode_reasons() -> None:
|
||||
)
|
||||
|
||||
assert disabled.skip_reason == "mode_disabled"
|
||||
assert tool.skip_reason == "mode_tool"
|
||||
assert tool.inject is True
|
||||
assert tool.skip_reason is None
|
||||
|
||||
|
||||
def test_apply_memory_skip_reason_stamps_only_when_skipping() -> None:
|
||||
|
||||
@@ -17,8 +17,37 @@ def test_resolve_memory_injection_mode_defaults_for_missing_value() -> None:
|
||||
def test_resolve_memory_injection_mode_accepts_known_values_case_insensitively() -> None:
|
||||
assert resolve_memory_injection_mode("live_zone_tail") == "live_zone_tail"
|
||||
assert resolve_memory_injection_mode(" DISABLED ") == "disabled"
|
||||
assert resolve_memory_injection_mode(" Tool ") == "tool"
|
||||
|
||||
|
||||
def test_get_memory_injection_mode_accepts_tool_from_env(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# Documented in memory_decision.py (mode_tool); must not raise per request.
|
||||
from headroom.proxy.helpers import get_memory_injection_mode
|
||||
|
||||
monkeypatch.setenv("HEADROOM_MEMORY_INJECTION_MODE", "tool")
|
||||
assert get_memory_injection_mode() == "tool"
|
||||
|
||||
|
||||
def test_resolve_memory_injection_mode_rejects_unknown_values() -> None:
|
||||
with pytest.raises(ValueError, match="HEADROOM_MEMORY_INJECTION_MODE"):
|
||||
resolve_memory_injection_mode("system_prompt")
|
||||
|
||||
|
||||
async def test_tool_mode_skips_context_search_but_keeps_decision_open(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""tool mode: no auto-injected context, memory tools stay registrable."""
|
||||
from headroom.proxy.memory_decision import MemoryDecision
|
||||
from headroom.proxy.memory_handler import MemoryConfig, MemoryHandler
|
||||
|
||||
monkeypatch.setenv("HEADROOM_MEMORY_INJECTION_MODE", "tool")
|
||||
handler = MemoryHandler(MemoryConfig(enabled=True, backend="local"))
|
||||
messages = [{"role": "user", "content": "what did I say about databases?"}]
|
||||
|
||||
assert await handler.search_and_format_context("u1", messages) is None
|
||||
assert handler._initialized is False # returned before any backend lookup
|
||||
|
||||
decision = MemoryDecision.decide(
|
||||
headers={}, memory_handler=handler, memory_user_id="u1", mode_name="tool"
|
||||
)
|
||||
assert decision.inject is True
|
||||
|
||||
Reference in New Issue
Block a user