Commit Graph
5 Commits
Author SHA1 Message Date
fda75de505 fix(memory): run only the memory tools the proxy injected on the request (#4870)
## Description

With memory enabled (`--memory`, `--learn`, `wrap/init --memory`), the
proxy ran any upstream tool call named
`memory_list`/`memory_save`/`memory_search`/`memory_update`/`memory_delete`,
and hid it from the client. That held even when the proxy had not
offered those tools on the request (`--no-memory-tools`), or when the
client had declared a tool with that name itself.

The proxy now runs only the memory tools it injected on the current
request. Every other call reaches the client unchanged. The proxy does
not run it, does not withhold it, and sends no continuation upstream.
Anthropic streaming already followed this rule
(`_server_memory_tool_names`). That helper is now shared, and every
executor uses it.

**Behaviour change**
- With `--no-memory-tools`, memory tool calls from the model now reach
the client instead of running on the proxy.
- A memory-named tool that the client declares is now the client's. The
proxy no longer runs it on any path, including the streaming path that
used to run it as a side effect.
- Injected memory tools behave as before: they run on the proxy, are
hidden from the client, and the turn continues upstream.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/memory_handler.py`: `server_memory_tool_names(tools,
client_tools)` moved here from `AnthropicHandlerMixin`. It now reads
Anthropic, Chat and Responses tool shapes through `extract_tool_name`.
`handle_memory_tool_calls` takes a required `tool_names` and skips every
other call.
- `handlers/openai.py`, Chat Completions (non-streaming): records the
injected names and passes them to the executor.
- `handlers/openai.py`, Responses (HTTP): runs a `function_call` only if
its name was injected. Before, any name in `MEMORY_TOOL_NAMES` ran.
- `handlers/openai.py`, Responses over WebSocket: the injected names are
kept per `response.create`, like the user id. The relay holds back and
runs only those calls.
- `handlers/anthropic.py`, non-streaming: passes the injected names it
already computed for streaming.
- `handlers/streaming.py`: removes the branch that ran memory calls
after the stream had already reached the client. It ran only for tools
the proxy had not injected. Memory handling in the stream (SSE
buffering, the Claude Code credential warning) now turns on only when
the proxy injected memory tools.
- Tests: new `tests/test_memory_tools_only_when_injected.py`. Existing
fixtures that relied on the old behaviour now inject the tool they call:
`test_codex_project_context.py`, `test_memory_handler_native_ops.py`,
`test_memory_handler_project_isolation.py`,
`test_memory_tool_stream.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

The new tests use `create_app` with a fake memory backend and a
`MockTransport` upstream. The WebSocket test uses a scripted upstream
socket. For each path (chat, Responses, Responses over WebSocket,
Anthropic, Anthropic streaming) they check two cases. First, injection
is off and the model calls `memory_delete`. Second, injection is on and
the client declares its own `memory_delete`. Each case expects zero
backend calls, exactly one upstream request, and the call in the
client's response. The control cases inject `memory_list` and expect it
to run as before.

On `origin/main` (0368a872):
```text
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-anthropic_stream]
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_injected_still_runs[chat]
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_injected_still_runs[responses]
PASSED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_injected_still_runs[anthropic]
PASSED tests/test_memory_tools_only_when_injected.py::test_ws_memory_call_the_proxy_injected_still_runs
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-chat]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-responses]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-anthropic]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected-anthropic_stream]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-chat]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-responses]
FAILED tests/test_memory_tools_only_when_injected.py::test_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared-anthropic]
FAILED tests/test_memory_tools_only_when_injected.py::test_ws_memory_call_the_proxy_did_not_inject_reaches_the_client[not_injected]
FAILED tests/test_memory_tools_only_when_injected.py::test_ws_memory_call_the_proxy_did_not_inject_reaches_the_client[client_declared]
9 failed, 5 passed in 3.05s
```
On this branch:
```text
14 passed in 3.23s
```
The existing tests for memory, anthropic, openai and streaming (165
files under `tests/`, not `test_live`) pass on both branches:
```text
origin/main: 3183 passed, 44 skipped in 230.96s
this branch: 3197 passed, 44 skipped in 229.83s
```
`uvx ruff@0.17.3 check` and `format --check` pass on the changed files.
`mypy` on the four changed modules reports no issues.

## Real Behavior Proof

- Environment: macOS, Python 3.12,
`create_app(ProxyConfig(memory_enabled=True,
memory_storage_mode="global", ...))` in-process via
`httpx.ASGITransport`. The upstream is `httpx.MockTransport`, the memory
backend is a recording fake, and `LITELLM_LOCAL_MODEL_COST_MAP=True`.
- Exact command / steps: a harness sends one request per path. The
mocked upstream answers with a tool call to `memory_delete`. The harness
records backend calls, upstream request count, and whether the client
response contains the call.
- Observed result (`origin/main` → this branch):
```text
chat       inject=False memory_delete  upstream_calls=2 backend_calls=[delete] client_saw_tool_call=False  →  upstream_calls=1 backend_calls=[] client_saw_tool_call=True
responses  inject=False memory_delete  upstream_calls=2 backend_calls=[delete] client_saw_tool_call=False  →  upstream_calls=1 backend_calls=[] client_saw_tool_call=True
anthropic  inject=False memory_delete  upstream_calls=2 backend_calls=[delete] client_saw_tool_call=False  →  upstream_calls=1 backend_calls=[] client_saw_tool_call=True
anthropic stream inject=False memory_delete  backend_calls=[delete] client_saw_tool_call=True  →  backend_calls=[] client_saw_tool_call=True
chat       inject=True, client-declared memory_delete  upstream_calls=2 backend_calls=[delete]  →  upstream_calls=1 backend_calls=[]
anthropic  inject=True, client-declared memory_delete  upstream_calls=2 backend_calls=[delete]  →  upstream_calls=1 backend_calls=[]
```
- Not tested: a live provider and a live Codex WebSocket session. The
WebSocket path is covered by the in-process test above.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: only for memory-named tool calls that
the proxy did not inject. See "Behaviour change" above. The default
`--memory` setup, where the proxy injects the tools, is unchanged.
- Kill switch / disable path: none needed.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Follow-up, not in this PR: on the Chat, Responses and WebSocket paths, a
single turn can call an injected memory tool and a client tool together.
The proxy then continues upstream without an output for the client's
call. That was already true for a client tool such as `shell`. It now
also applies to a memory-named tool that the client declared, which the
proxy used to run. The Anthropic non-streaming path already handles this
by returning the turn to the client. Bringing the OpenAI paths in line
is a separate change.

The docs need no change: `--no-memory-tools` is already documented as
"Disable automatic memory tool injection", which is now what it does.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4870"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:47:05 +05:30
Mansurshakh JaparovandJD Davis 0f3b0d8a21 fix(memory): resolve pi cwd before injecting project tools (#4618)
## Description

Closes #4442.

pi's system `<cwd>` block now resolves the memory project. If project
scope is still unresolved with the `empty` fallback, the Anthropic
handler no longer advertises unusable memory tools, including tools
recorded earlier in the session.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Parse pi cwd only in system text. Keep main's line parser, header/CLI
precedence and limited user-message fallback.
- Reuse `is_project_unresolved` for context and sticky-tool replay.
Client tools and global/user storage behavior stay intact.
- Add initialized-handler scope regressions, replace
source-inspection/forwarding-only checks, and document project scope in
the published memory page.

## Testing

- [x] Unit tests pass (`pytest`) — related suite and the normal critical
gate, not the entire Python suite locally.
- [x] Linting passes (`ruff check .`) — hosted lint passed on 5ace63e2;
local filesystem limitation below.
- [x] Type checking passes (`mypy headroom`) — normal full hook with its
existing `--ignore-missing-imports` setting.
- [x] New tests added for the scope regression
- [x] Manual testing performed

### Test Output

```text
# 16 related test files
329 passed, 12 skipped, 1 warning in 43.07s

# ordinary pre-push: make ci-precheck
178 passed, 5 skipped, 1 warning in 10.87s
✅ ci-precheck PASSED — safe to push.

uv run ruff format --check .
1860 files already formatted
```

Normal commit hooks (Ruff, formatter, full mypy, commitlint) and the
complete pre-push path passed: Rust fmt/clippy/workspace tests, native
build/import verification, critical Python tests and commitlint. The
existing Starlette test-client warning was not filtered.

Local whole-tree Ruff reports EXE002 on five **unchanged** OAuth plugin
files: my `fuseblk` checkout exposes them as executable despite Git
modes `100644`; `chmod` returns zero without changing the mode. The
uncached check agrees. I did not change those files, add shebangs or
ignore the rule. Hosted Ruff, formatting and full mypy passed on
5ace63e2; the local filesystem limitation remains.

The first full CI caught a missing scope predicate on the existing
lifecycle test double. Fixed in
[5ace63e2](https://github.com/headroomlabs-ai/headroom/commit/5ace63e2cbd2295f9594d1e0442a679aad99979c);
its complete file passes (`5 passed, 1 warning in 13.01s`). This
follow-up changes no production code or assertions and adds no runtime
fallback. The behavior proof below ran on 7aeb83cf with the same
production source; that qualification is historical; current-head checks
are recorded below.

### Author follow-up: 80453a03

Moved the project-scope guidance out of the unpublished wiki and into
`docs/content/docs/memory.mdx`; this commit changes no production code
or tests. Viewed the actual Next.js `/docs/memory#proxy-project-scope`
page and its table-of-contents entry; no browser errors.

The existing router/scope tests pass: 65 passed, with four SQLite
ResourceWarnings. All seven pi-parser lines and all six branch edges are
covered, so I did not add duplicate tests for the old Codecov
percentage.

Normal commit and pre-push hooks passed, including Rust checks, native
build/import, 180 critical Python tests (5 skipped, 1
tokenizer-approximation warning), and commitlint. My first push timed
out with the wrong local ONNX path; the next found missing pip in the
fresh uv environment. Both setup issues were fixed without bypassing
checks. Current-head hosted CI is not yet qualified.

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, freshly built native core,
real TCP/Uvicorn proxy and SQLite/ONNX memory backend. Only the upstream
is a local Anthropic-shaped HTTP server; no provider charges.
- Exact command / steps: after `uv sync --locked --extra dev --extra
proxy --extra memory` and `make verify-rust-core`, run this from the
checkout. It prints the actual loaded native module/hash and leaves a
disposable `.headroom-4442-*` directory for inspecting SQLite. Both HTTP
listeners shut down before exit.

<details>
<summary>Standalone HTTP/SQLite reproduction</summary>

```bash
./.venv/bin/python - <<'PY'
import hashlib,http.client,json,os,socket,tempfile,threading,time
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from pathlib import Path
import uvicorn
import headroom._core as core
assert Path(core.__file__).resolve().parent == Path.cwd().resolve()/"headroom"
native_sha=hashlib.sha256(Path(core.__file__).read_bytes()).hexdigest()
from headroom.memory.storage_router import ProjectResolver,RequestContext,extract_system_prompt
from headroom.proxy.server import ProxyConfig,create_app
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES
from headroom.proxy.helpers import get_session_tool_tracker
root=Path(tempfile.mkdtemp(prefix=".headroom-4442-",dir=Path.cwd()))
project_a=root/"pi%2Fproject-a";project_b=root/"project-b"
project_a.mkdir(exist_ok=True);project_b.mkdir(exist_ok=True)
os.environ["HEADROOM_WORKSPACE_DIR"]=str(root/"workspace")
os.environ["HEADROOM_BEACON"]="off"
os.environ["HEADROOM_TELEMETRY"]="off"
os.environ["NO_PROXY"]="127.0.0.1,localhost"
forwarded=[]
class Gateway(BaseHTTPRequestHandler):
    def do_POST(self):
        body=json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        forwarded.append({"path":self.path,"body":body})
        payload=json.dumps({"id":"msg_local_scope","type":"message","role":"assistant","model":"claude-haiku-4-5","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","stop_sequence":None,"usage":{"input_tokens":10,"output_tokens":1}}).encode()
        self.send_response(200);self.send_header("Content-Type","application/json");self.send_header("Content-Length",str(len(payload)));self.end_headers();self.wfile.write(payload)
    def log_message(self,*args):pass
upstream=ThreadingHTTPServer(("127.0.0.1",0),Gateway)
ut=threading.Thread(target=upstream.serve_forever,daemon=True);ut.start()
config=ProxyConfig(mode="cache",anthropic_api_url=f"http://127.0.0.1:{upstream.server_port}",memory_enabled=True,memory_db_path=str(root/"legacy.db"),traffic_learning_enabled=True,subscription_tracking_enabled=False,cost_tracking_enabled=False,cache_enabled=False,rate_limit_enabled=False,log_requests=False,image_optimize=False,ccr_inject_tool=False,ccr_handle_responses=False,ccr_context_tracking=False,anthropic_pre_upstream_memory_context_timeout_seconds=30.0)
app=create_app(config)
sock=socket.socket();sock.bind(("127.0.0.1",0));sock.listen(128)
port=sock.getsockname()[1]
server=uvicorn.Server(uvicorn.Config(app,log_level="warning",lifespan="on",timeout_graceful_shutdown=3))
st=threading.Thread(target=server.run,kwargs={"sockets":[sock]},daemon=True);st.start()
try:
    deadline=time.monotonic()+180
    while not server.started:
        if not st.is_alive() or time.monotonic()>deadline:raise RuntimeError("real proxy startup failed")
        time.sleep(.02)
    session="headroom4442-native-project-scope"
    common={"Content-Type":"application/json","x-api-key":"local-unbilled-key","anthropic-version":"2023-06-01","x-headroom-user-id":"native-scope-user","x-headroom-session-id":session}
    client_tools=[{"name":"get_weather","description":"Weather","input_schema":{"type":"object","properties":{}}}]
    rows=[]
    cases=[("fresh_unresolved","You are helpful.",{},False),("pi_cwd",f"<cwd>\n{project_a}\n</cwd>",{},True),("warm_unresolved","You are helpful.",{},False),("pi_cwd_restored",f"<cwd>\n{project_a}\n</cwd>",{},True),("explicit_header_wins",f"<cwd>\n{project_b}\n</cwd>",{"x-headroom-project-id":"explicit-project"},True),("ordinary_user_cwd_ignored","You are helpful.",{},False)]
    for label,system,extra,expected in cases:
        body={"model":"claude-haiku-4-5","max_tokens":16,"system":system,"messages":[{"role":"user","content":f"cwd:\n<cwd>\n{project_b}\n</cwd>" if label=="ordinary_user_cwd_ignored" else "hello scope"}],"tools":client_tools}
        before=len(forwarded)
        conn=http.client.HTTPConnection("127.0.0.1",port,timeout=90)
        conn.request("POST","/v1/messages",body=json.dumps(body).encode(),headers={**common,**extra})
        response=conn.getresponse();payload=response.read();conn.close()
        assert response.status==200,(label,response.status,payload.decode())
        assert len(forwarded)==before+1,(label,"unexpected upstream request count")
        actual=forwarded[-1]["body"]
        names={tool.get("name") for tool in actual.get("tools",[])}
        assert names==({"get_weather"}|MEMORY_TOOL_NAMES if expected else {"get_weather"}),(label,names)
        scope=ProjectResolver().resolve(RequestContext(headers={**common,**extra},system_prompt=extract_system_prompt(body),base_user_id="native-scope-user"))
        rows.append({"case":label,"status":response.status,"memoryTools":sorted(names & MEMORY_TOOL_NAMES),"project":scope,"gatewayPath":forwarded[-1]["path"]})
    assert forwarded[1]["body"]["tools"]==forwarded[3]["body"]["tools"],"restored definitions changed"
    tracker=get_session_tool_tracker()
    assert tracker.should_inject("anthropic",session),"probe did not exercise an actual recorded sticky session"
    from urllib.parse import quote
    key_a=ProjectResolver().resolve(RequestContext(headers={"x-headroom-cwd":quote(str(project_a),safe="/")},system_prompt="",base_user_id="native-scope-user"))[0]
    assert rows[1]["project"][0]==key_a
    assert rows[4]["project"][1]=="explicit-project"
    router=app.state.proxy.memory_handler._router
    assert router is not None,"real memory backend never initialized"
    opened=router.open_backends()
    project_db=root/"memories"/"projects"/key_a/"memory.db"
    assert project_db in opened and project_db.is_file(),("real pi project database not opened",[str(x) for x in opened])
    print(json.dumps({"mode":config.mode,"realHttpListener":f"http://127.0.0.1:{port}","localGateway":f"http://127.0.0.1:{upstream.server_port}","rows":rows,"realStickyGoldenSession":True,"realPiProjectDatabase":str(project_db),"openedDatabases":[str(x) for x in opened],"nativeModule":core.__file__,"nativeSha256":native_sha,"noLivePiOrPaidProvider":True},indent=2))
finally:
    server.should_exit=True;st.join(20)
    upstream.shutdown();upstream.server_close();ut.join(5);sock.close()
    assert not st.is_alive() and not ut.is_alive(),"native proof leaked a server"


PY
```

</details>

- Observed result: unchanged `main` fc1ba891 fails at the first
unresolved request: five proxy memory tools reach upstream. With the
fix, all six requests return HTTP 200:

| Same-session request | Proxy memory tools |
| --- | ---: |
| Fresh, no cwd | 0 |
| pi system cwd | 5 |
| Cwd absent again | 0 |
| Same pi cwd restored | 5 |
| Explicit project header, different system cwd | 5 |
| Cwd block only in ordinary user text | 0 |

The client weather tool stays present. Restored definitions are
structurally equal to the first resolved turn; the explicit header wins.
The literal-percent cwd agrees with its encoded-header form, and the
actual project SQLite database opens at the expected key.

- Not tested: live pi/Caveman, paid-provider cache accounting/billing,
Windows, or other provider protocols. This does not claim to fix Caveman
header forwarding.

## Runtime Rollout Safety

- Rollout-managed feature(s): none added; existing Anthropic memory
injection.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: pi cwd resolves; unresolved
project/empty scope no longer adds unusable proxy memory tools.
- Kill switch / disable path: existing `--no-memory`; explicit
project/cwd headers still work.
- Unsafe override required: no.
- Qualification impact: no storage migration or new dependency; scoped
HTTP/SQLite behavior checked.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes — related
suite and critical gate, not a whole-suite claim
- [x] I did **not** edit `CHANGELOG.md` — release-please owns it

## Screenshots (if applicable)

N/A; the observable surface is the HTTP payload and project database.

## Additional Notes

AI-assisted; no subagents. OpenAI injection policy is unchanged. Keeping
this draft until the current hosted checks are qualified.

One earlier post-build probe completed the requests and closed both
listeners but hit its whole-process shutdown deadline. The same program
and native artifact later exited normally with faulthandler enabled; its
diagnostic abort timer did not fire. I have not identified the
intermittent cause and am not claiming a shutdown fix. No production
timeout or feature flag was changed to get that result.


The two npm audits now pass on 5ace63e2, along with the other Security
jobs. The earlier [unchanged-base
failure](https://github.com/headroomlabs-ai/headroom/actions/runs/41399323584)
is historical evidence, not a current blocker. This PR changes no
dependency manifests, lockfiles, workflow or audit exceptions.


### Maintainer current-main requalification

Exact head `60597623e00967153d947a46bf197950250f310e`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Clean mechanical update from current upstream main, preserving the pi
system-cwd resolver, request-scoped project guard, main's cache-mode
search skip/logging, and client-tool preservation. Ordinary user cwd
blocks still do not enable the pi fallback; unresolved project scope
still blocks both context injection and replayed memory tools. No
contributor production logic was rewritten and no new production finding
remains in this reviewed scope.

The initial combined tree dropped main's memory-invariant file and two
stream/timeout regression cases. Maintainer test-only repairs restore
those tests rather than reducing coverage: the invariant file is
byte-identical to main, every main assertion in the four
restored/updated test files is preserved by AST comparison, and three
fixtures now supply the handler protocol methods needed for resolved
scope and beta headers. The lifecycle main control passes without that
fixture extension; no claim is made that this particular fixture failure
independently fails main. All 222 affected
scope/invariant/stream/timeout/lifecycle/backpressure/cache-stability/system-prompt
tests pass with one skip. Ten shared Gemini request-fixture cases also
pass after the separate test-only repair for independently verified main
failures. All 12 main-relative changed Python files pass Ruff/format;
whitespace checks pass.

The author's existing real TCP/SQLite/ONNX reproduction and explicit
limits were read and remain in the body. These fresh Windows checks
qualify the combined request-scope regressions; they are not a claim of
rerunning that Linux proof, live pi/client/provider behavior, or a fresh
native build. An existing binary was supplied solely for test startup.
Draft status is preserved; readiness remains the author's decision and
nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `60e01cf7c25e90abb9f5cb66fa221da83e7c5177`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.

Clean integration with upstream/main
ea59ac3eef. Re-reviewed project-scoping
delta against intervening memory/backend/stream changes: pi cwd remains
limited to system context; ordinary user cwd blocks do not enable
fallback; unresolved projects skip context/tool injection and sticky
tool replay while retaining client tools and main cache-mode no-search
behavior.232
scope/invariant/stream/timeout/lifecycle/backpressure/cache/system-prompt/Gemini
tests pass,1optional skip; touched-source Ruff and whitespace pass. No
contributor production policy rewritten. Earlier author TCP/SQLite/ONNX
evidence remains scoped, with no fresh native build or live-pi/provider
claim. Existing draft preserved; final approval awaits exact-head hosted
qualification. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `e93ceb35df7f78bfa2dcc0611609131cf2b08b0a`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 232 passed, 1 skipped, 1 warning in
47.62s. Selected real dependency overlays and native DLL paths are
recorded in final4618-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

No new source finding in the reviewed scope. Final formal approval
awaits exact-head hosted qualification. Project resolution and
unresolved-scope injection/replay gates remain unchanged. Existing draft
status is preserved.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `d693dd2b6e33fd47ff853f7e28d938fc1dbc30ac`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378. Clean source
integration preserves pi systemcwd resolution and unresolved-scope
context/tool/sticky-replay gates.70integration patches exactly matchmain
delta;Anthropic exception patch differs only byclosing-parenthesis diff
alignment and its exception AST equalsmain. Broad
memory/router/projectscope/continuation/backpressure/cache/lifecycle/doctor/nativeops
run:405passed,1skipped,one newmain fake-handler fixture failed
missingis_project_unresolved. Maintainer adds only that fake method
returningFalse,respecting resolvedfixture scope;all39backpressure tests
nowpass,including unchanged
injectedSQLiteexception,HTTP200andsemaphorerelease assertions. No
production logic changed.33changedPythonlint/format,offline
lock/whitespacepass;fixedtestlint/formatrechecked.
Existingpr3982nativeDLLverified fullinputhashes/checksum;no
freshbuildclaimed. Currentauthor/Devin discussion checked,inline
endpoint empty;prior scoped realbehavior evidence remains applicable to
unchanged contributor inputs. No new sourcefinding;no
realprovider/agentlaunch claim. Draftpreserved.
Exactnewheadhostedqualification pending;maintainers
handlechecks,nothingmerged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `0da0b5a01958eaaac1d625227cb4bdb9477c5d3b`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main delta after diff-location
normalization. Contributor pi system-cwd resolution and
unresolved-project memory context/tool/sticky-replay gates are
unchanged. Fresh
memory-router/project-scope/streaming/continuation/backpressure/cache/lifecycle
and incoming Vertex/backend/CLI tests: 381 passed, 12 skipped, 1
warning. Twelve changed Python files pass Ruff lint/format; offline lock
and whitespace pass.

Static production/test scope and security review considered user versus
system cwd inputs, malformed/empty cwd, missing scope and stale tool
replay: tests keep pi cwd tags in ordinary user content from resolving
project scope, and unresolved scope blocks replay. No unrelated
executable, workflow, dependency or setup changes. Flagged inherited
workflow-validation job is excluded by unchanged workflow paths. Author
discussion checked; collected inline endpoint has no findings. Prior
scoped behavior evidence applies to unchanged contributor inputs; no new
real agent/provider launch claimed. Native DLL reused only after
complete tracked input-set/hash and artifact checksum verification, no
fresh build claimed. Draft preserved. Formal approval awaits exact
published-head hosted qualification. Maintainers handle checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-10 22:28:01 -06:00
lorenzozaneandJD Davis 3afaa97ffc fix(proxy): preserve Unicode separators in SSE JSON (#4337)
## Description

Complete UTF-8 SSE frames can contain U+0085, U+2028, or U+2029 inside
JSON strings. Splitting decoded text with `str.splitlines()` treats
those payload characters as framing, corrupting JSON and bypassing
memory-tool continuation index remapping. This change splits complete
frames at byte line boundaries before strict UTF-8 decoding and adds
regressions for both parser paths.

Fixes #4251

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Split SSE frame bytes into lines before decoding in the shared parser
and memory-tool filter.
- Cover all three Unicode separators, CRLF event framing, payload
preservation, and continuation index remapping.

## Testing

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .` on changed files)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed by this PR author

### Test Output

```text
uv run --no-sync pytest tests/test_sse_byte_buffer_policy.py tests/test_sse_utf8_split.py tests/test_streaming_sse_malformed_events.py tests/test_memory_tool_stream.py -q -k 'not test_injected_memory_tools_reach_the_stream'
424 passed, 1 deselected (environment-limited server setup test)

uv run --no-sync ruff check headroom/proxy/sse_byte_buffer_policy.py headroom/proxy/memory_tool_stream.py tests/test_sse_byte_buffer_policy.py tests/test_memory_tool_stream.py
passed

uv run --no-sync ruff format --check headroom/proxy/sse_byte_buffer_policy.py headroom/proxy/memory_tool_stream.py tests/test_sse_byte_buffer_policy.py tests/test_memory_tool_stream.py
passed

git diff --check
passed
```

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14.
- Exact command / steps: This PR author ran the focused regression
command above. Separately, the Issue reporter reports replaying the
frames through a localhost HTTP server and httpx with one-byte-at-a-time
reads; the Issue does not provide an exact executable command.
- Observed result: The regression suite verifies all three Unicode
separators survive parsing and the continuation block start and text
delta both use index 2. The reporter reports that its replay preserved
all three characters and produced continuation indices `[2, 2, 2]`; that
replay was not independently rerun for this PR.
- Not tested: Live model/provider behavior and a deployed proxy.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Not applicable.
- Stable/default behavior changed: SSE payload text containing Unicode
line separators remains intact.
- Kill switch / disable path: Not applicable.
- Unsafe override required: No.
- Qualification impact: None expected; no dependency or native-code
changes.
- Rollback path: Revert this change.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
(no new comments were needed)
- [ ] I have made corresponding changes to the documentation (not
applicable to this parser fix)
- [ ] My changes generate no new warnings (not independently checked)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes (focused
related suite)
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

The Issue reporter has a similar implementation on a fork branch, but no
upstream PR exists for it. The issue remains open; this PR submits the
reviewed job candidate for the maintainer's consideration.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4337"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer runtime follow-up (2026-11-04)

AI-assisted qualification on Windows x64 / Python 3.12.13, after syncing
main 31dcaf41f. All 427 focused SSE/parser/memory-stream tests passed;
lint, formatting and whitespace checks passed. This uses the existing
native binary rather than a fresh native build.

I independently ran a real loopback HTTP SSE server and consumed its
response with httpx at one byte per read through both parser paths. All
283 byte chunks were consumed; U+0085, U+2028 and U+2029 survived
exactly, and the continuation indices were [2, 2]. No live provider or
deployed proxy is claimed.

Reproduce from the checkout by saving this as qualification.py and
running `python qualification.py`:

```python
import sys,json,threading
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
import httpx
from headroom.proxy.sse_byte_buffer_policy import parse_sse_events_from_byte_buffer
from headroom.proxy.memory_tool_stream import MemoryToolStreamFilter
from headroom.memory.tools import get_tool_names
text="before\^Emiddle\u2028next\u2029after"
payloads=[{"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}},{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":text}}]
wire=b"".join(("event: "+p["type"]+"\r\ndata: "+json.dumps(p,ensure_ascii=False)+"\r\n\r\n").encode("utf-8") for p in payloads)
class Handler(BaseHTTPRequestHandler):
 def do_GET(self):
  self.send_response(200);self.send_header("Content-Type","text/event-stream");self.send_header("Content-Length",str(len(wire)));self.end_headers()
  for byte in wire:self.wfile.write(bytes([byte]));self.wfile.flush()
 def log_message(self,*args):pass
server=ThreadingHTTPServer(("127.0.0.1",0),Handler);thread=threading.Thread(target=server.serve_forever,daemon=True);thread.start()
try:
 shared=[]; filtered=[]; buf=bytearray(); outbuf=bytearray(); chunks=0
 flt=MemoryToolStreamFilter(get_tool_names(),index_offset=2,forward_message_start=False)
 with httpx.Client(trust_env=False,timeout=5) as client:
  with client.stream("GET",f"http://127.0.0.1:{server.server_port}/events") as response:
   response.raise_for_status()
   for chunk in response.iter_bytes(chunk_size=1):
    chunks+=1;buf.extend(chunk);shared.extend(parse_sse_events_from_byte_buffer(buf))
    for output in flt.feed(chunk):outbuf.extend(output)
    filtered.extend(parse_sse_events_from_byte_buffer(outbuf))
 assert chunks==len(wire) and not buf and not outbuf
 objects=[json.loads(data) for _,data in shared];remapped=[json.loads(data) for _,data in filtered]
 assert objects==payloads
 assert [p["index"] for p in remapped]==[2,2]
 assert remapped[1]["delta"]["text"]==text
 print(json.dumps({"transport":"loopback HTTP SSE","read_chunk_bytes":1,"chunks":chunks,"unicode_codepoints_preserved":["U+0085","U+2028","U+2029"],"shared_events":len(shared),"continuation_indices":[p["index"] for p in remapped],"status":"PASS"}))
finally:server.shutdown();server.server_close();thread.join(5)
```

Current-head hosted CI is separate from this local runtime evidence.


### Maintainer current-main requalification

Exact head `8253904f1d25b6c7d42d2b2bde277aee03b57202`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Updated against upstream/main e4a60ed7 and re-reviewed both SSE parser
paths. Resolved the test-only overlap by retaining both the PR's Unicode
JSON regression and main's new one-leading-space framing regression;
production parser code merged automatically. All 428
parser/UTF-8/malformed-event/memory-tool-stream tests pass. Ruff
lint/format and whitespace checks pass.

Independently reran the real localhost HTTP SSE replay through httpx in
283 one-byte chunks on this combined checkout: U+0085/U+2028/U+2029
payload text is preserved by both parser paths and continuation indices
remain [2, 2]. The existing executable reproduction in this body remains
applicable. This uses the existing development native extension and does
not claim a fresh native build, live provider or deployed proxy
qualification. No contributor production logic was rewritten.

Applied the shared test-only Gemini streamed-request fixture repair for
independently established main failures; all ten cases pass. Formal
approval awaits exact-head hosted qualification.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-05 10:41:45 -06:00
danieldutuandJerrettDavis 5f308e1cc8 fix(proxy): skip memory tool injection on streamed OpenAI requests (#4451)
## Description

On streamed OpenAI chat and Responses requests, the proxy added 5
`memory_*` tool definitions that it cannot execute. In a local run with
a stub upstream, the chat request reached the upstream with 6 tools
(`Bash` plus 5 `memory_*`). The client had declared 1. After this change
the upstream receives 1.

Root cause: `headroom/proxy/handlers/openai.py` (chat, near line 4257;
Responses, near line 5881) injects memory tools with no `stream`
condition. Execution of memory tool calls exists only on the buffered
paths and on the Anthropic stream path
(`headroom/proxy/handlers/streaming.py`, `memory_enabled` is set only
for `provider == "anthropic"`). A `memory_*` call from the model
therefore went to the client unhandled.

Fix: both sites skip memory tool injection when `stream` is true. The
chat site skips the whole `apply_session_sticky_memory_tools` call, not
only its inputs. That helper's sticky branch ignores `inject_this_turn`
and re-adds the golden memory definitions for a session that already had
a buffered turn. The first revision of this PR emptied the inputs only,
so a buffered turn followed by `stream=true` in one session still sent 5
`memory_*` tools (reported by @JerrettDavis and @tonydzi). The Responses
site already gates the whole sticky block on
`responses_memory_tools_allowed`, which now includes `not stream`. The
helper is shared with the Anthropic handler, which keeps its tools on
streams, so the condition sits at the OpenAI call sites. Memory context
injection and buffered OpenAI requests are unchanged.

Trade-off: a streamed turn's `tools` list now differs from the buffered
turns in the same session, which costs one prefix-cache miss on that
turn. Tools the stream path cannot execute are the worse outcome, so the
PR accepts the miss.

@HetCreep reported this in #4420 and listed the options. This PR
implements option 1.

Closes #4420

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/openai.py`: chat path wraps the definitions
lookup and the `apply_session_sticky_memory_tools` call in `if not
stream:`. Responses path ANDs `not stream` into
`responses_memory_tools_allowed`.
- `tests/test_memory_tool_stream.py`: streamed OpenAI chat sends only
the client's tool, for a fresh session and for a session whose earlier
buffered turn recorded the memory tools (parametrized); the existing
streamed Anthropic test now also asserts the memory tools are in the
outgoing body.
- `tests/test_openai_codex_routing.py`: streamed Responses sends no
memory tool and does not compute definitions.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — ran on handlers/openai.py
only; full mypy headroom not run
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_memory_tool_stream.py tests/test_openai_codex_routing.py tests/test_memory_tool_session_sticky.py
======================= 132 passed, 1 warning in 3.73s =========================

On the first revision (d440495a), the new buffered-then-streamed case fails:
FAILED tests/test_memory_tool_stream.py::TestHandlerPassesServerMemoryTools::test_streamed_openai_chat_gets_no_memory_tools[True]
1 failed, 1 passed (deselected 58)

$ ruff check .
All checks passed!
$ ruff format --check .
1812 files already formatted
$ mypy headroom/proxy/handlers/openai.py
Success: no issues found in 1 source file
```

Only `mypy` on `openai.py` was run, not the full `mypy headroom`.

## Real Behavior Proof

- Environment: macOS, Python 3.13,
`create_app(ProxyConfig(memory_enabled=True, openai_api_url=<stub>,
anthropic_api_url=<stub>))` under `TestClient`; the stub is a local
`http.server` that records each request body and replies with a minimal
SSE stream.
- Exact command / steps: `python proof2.py <db>` (scratch script, not
committed): a buffered then a streamed `POST /v1/chat/completions` (one
client tool `Bash`, `compute_session_id` pinned to one id) and one
streamed `POST /v1/messages`, all with `x-headroom-user-id: u1`, run on
d440495a and on this commit.
- Observed result: two-turn sequence on one pinned session id (turn 1
`stream=false`, turn 2 `stream=true`). Before (d440495a): turn 1
`/v1/chat/completions stream= False tools= ['Bash', 'memory_save',
'memory_search', 'memory_update', 'memory_delete', 'memory_list']`, turn
2 `/v1/chat/completions stream= True tools= ['Bash', 'memory_save',
'memory_search', 'memory_update', 'memory_delete', 'memory_list']`.
After: turn 1 unchanged, turn 2 `/v1/chat/completions stream= True
tools= ['Bash']`. Anthropic stream after: `/v1/messages stream= True
tools= ['Bash', 'memory_save', 'memory_search', 'memory_update',
'memory_delete', 'memory_list']`. The single-turn run on the base commit
(before any fix) also sent all 5 memory tools on a streamed chat
request.
- Not tested: streamed Responses against the stub (covered by the unit
test only), a real OpenAI upstream, WebSocket Responses (`openai.py`
near line 8106 also injects memory tools; left unchanged), Windows.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This changes request-time memory
tool injection for streamed OpenAI requests only.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes. Streamed OpenAI chat and
Responses sessions with memory enabled no longer receive `memory_*`
tools; they still receive memory context.
- Kill switch / disable path: None needed; the change removes an
injection. `--no-memory-tools` already disables injection for all
requests.
- Unsafe override required: No.
- Qualification impact: None. No qualification rule references streamed
OpenAI memory tools.
- Rollback path: Revert the commit.

## Review Readiness

- [x] I have performed a self-review
- [ ] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — 3 test
files ran (131 passed); full suite not run
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Follow-up option 2 from #4420, an OpenAI SSE filter and executor
mirroring the Anthropic path, is a larger change that I can send if the
maintainers prefer it. Documentation checklist item is unchecked because
no doc describes streamed OpenAI memory tools.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-24 08:45:05 -07:00
Kenneth WongandJD Davis 1cf496612e fix(proxy): run injected memory tools server-side on streaming turns (#3947)
## Description

With `--memory`, the proxy injects `memory_save` / `memory_search` /
`memory_update` / `memory_delete` / `memory_list` into Anthropic
requests. On the **streaming** path it then forwarded the model's call
to those tools straight to the client. A client that never declared them
can't run them. Claude Code streams every request, so it always answers:

```
<tool_use_error>Error: No such tool available: memory_save</tool_use_error>
```

The proxy did persist the save in the background, but the model is told
it failed. `memory_search` has no return path at all, so a later session
never gets its results back. In the default `--mode cache`,
auto-injection of memory context is skipped on purpose, which leaves
`memory_search` as the only recall path. Proxy memory therefore didn't
work end to end for Claude Code. This is Finding 3 of #2195. The
non-streaming path already runs the tools and continues the turn
server-side (`handlers/anthropic.py`), and the `_stream_response`
docstring already promised the same for streaming.

**Fix.** When the proxy injected memory tools, `MemoryToolStreamFilter`
sits between the upstream SSE stream and the client:

- Every event still streams live, except the injected tools'
`content_block_*` events (withheld) and `message_delta` / `message_stop`
(held until the turn's outcome is known).
- If a round stops on memory calls alone, the proxy runs them, sends the
results upstream as the next turn (same outbound body, `messages`
extended), and streams that round into the **same** client message: no
second `message_start`, content block indices offset so they stay
contiguous. It allows up to 4 rounds.
- If a round also calls a client tool, the memory calls are still
executed, but the turn goes back to the client with `stop_reason:
tool_use`, as before. The client can't carry the memory results.
- Memory-named tools that the client declared itself are never withheld
or executed by the proxy.
- The filter records each withheld call with its streamed input, so the
calls run even when a round can't be rebuilt (SSE buffer cap,
unparseable stream). Continuation rounds honour the same buffer cap, and
the client's final `message_delta` usage includes the earlier rounds.
- If a stream has no injected-tool calls, the client receives the exact
upstream bytes (asserted in a test).
- The finalizer, and with it the prefix tracker, now gets the merged
message the client actually received rather than round one. That message
is what the client sends back next turn.

Refs #2195

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/memory_tool_stream.py` (new):
`MemoryToolStreamFilter`, a pure per-round SSE event filter. It splits
frames with the canonical `find_sse_event_terminator`, so it is safe on
byte-split chunks.
- `headroom/proxy/handlers/streaming.py`:
- `_stream_response` gains `server_memory_tool_names`; the generator
routes chunks through the filter when it is set.
- New `_continue_memory_tool_stream` executes the withheld calls and
runs the continuation rounds. A continuation upstream error is surfaced
as an SSE `error` event; Anthropic error envelopes pass through.
  - Continuation-round usage is added to the turn's totals.
- The held message tail is always released, including the empty-stream
and SSE-buffer-cap paths.
- `headroom/proxy/handlers/anthropic.py`: computes the memory tool names
this proxy injected (injected tools minus client-declared names) and
passes them to `_stream_response`.
- `tests/test_memory_tool_stream.py` (new, 17 tests):
- Filter tests: byte-identical passthrough, 1-byte trickle feed,
withholding plus the `end_turn` rewrite, re-indexing, continuation
offset, mixed client tool, client-declared tool not withheld.
- `_stream_response` integration tests: server-side continuation
streamed into one message, mixed round returned to the client,
passthrough without injected tools, finalizer receives the
client-visible message, client output usage summed across rounds.
- Edge cases: hidden-call input rebuilt from the stream, a truncated
call skipped, an unrebuilt round still running its calls, the
continuation buffer cap.

## Testing

- [x] Unit tests pass (`pytest`): see scope below
- [x] Linting passes (`ruff check .`): run on the changed files with the
pinned `ruff==0.16.8`, plus `ruff format --check`
- [x] Type checking passes (`mypy headroom`): run on the changed modules
only, not the whole package
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_memory_tool_stream.py -q
...........                                                              [100%]
17 passed in 2.19s

# All 82 test modules matching stream|memory|anthropic|sse|tool_inject|sticky,
# run on this branch and on origin/main (540f4da1) in the same venv:
origin/main: 6 failed, 1577 passed, 200 skipped
this PR:     6 failed, 1594 passed, 200 skipped
# Same 6 failures on both sides, all environmental: test_memory_bridge (3, no
# sentence-transformers in this venv) and test_upstream_guard (3, socks/TLS
# pinning). No new failures.

$ ruff check headroom/proxy/memory_tool_stream.py headroom/proxy/handlers/streaming.py headroom/proxy/handlers/anthropic.py tests/test_memory_tool_stream.py
All checks passed!
$ mypy headroom/proxy/memory_tool_stream.py headroom/proxy/handlers/streaming.py headroom/proxy/handlers/anthropic.py
Success: no issues found
```

## Real Behavior Proof

- Environment: macOS, real Claude Code 2.1.287 (`claude -p
--output-format stream-json`), with `ANTHROPIC_BASE_URL` pointed at a
Headroom proxy (`--memory`, default `--mode cache`). The proxy's
`--anthropic-api-url` points at a local fake Anthropic upstream that
streams real SSE. The fake model calls `memory_save` for a "SAVE" prompt
and `memory_search` for a "RECALL" prompt, and echoes any tool_result it
gets back. Before = released 0.39.1. After = this branch, same
interpreter.
- Exact command / steps: Session 1: `claude -p "SAVE: the deploy region
for project zephyr is ap-southeast-1"`. Session 2, same
`x-headroom-project-id`: `claude -p "RECALL: what is the deploy region
for project zephyr?"`.
- Observed result: before this PR Claude Code rejects both memory tool
calls (`No such tool available`); after it, Claude Code sees no memory
tool traffic, the save persists and session 2's `memory_search` returns
it (score 0.85). Transcript excerpts:

Before (0.39.1). Claude Code receives the memory tool calls and rejects
them:
  ```text
s1: client-side tool traffic = ['memory_save',
'RESULT:<tool_use_error>Error: No such tool available: memory_save…']
final = 'ack: "<tool_use_error>Error: No such tool available:
memory_save</tool_use_error>…'
s2: client-side tool traffic = ['memory_search',
'RESULT:<tool_use_error>Error: No such tool available: memory_search…']
final = 'ack: "<tool_use_error>Error: No such tool available:
memory_search</tool_use_error>…'
  ```
After (this PR). Claude Code sees no memory tool traffic; the save
persists and session 2 recalls it:
  ```text
  s1: client-side tool traffic = []
final = 'ack: "{\"status\": \"saved\", \"memory_id\": \"93cb8225-…\",
\"content\": \"The deploy region for project zephyr is
ap-southeast-1\"…'
  s2: client-side tool traffic = []
final = 'ack: "{\"status\": \"found\", \"count\": 2, \"memories\":
[{\"id\": \"93cb8225-…\", \"content\": \"The deploy region for project
zephyr is ap-southeast-1\", \"score\": 0.85…'
  proxy log:
Memory: Executed 1/1 proxy-handled tool call(s) withheld from the SSE
stream
    Memory: Continuation round 1 streamed
  ```
- Not tested: the real Anthropic API (the upstream was a local SSE
fake); extended-thinking turns, where round one's signed `thinking`
blocks are replayed in the continuation request and the client receives
thinking blocks from two rounds in one message; the Bedrock streaming
and WebSocket paths, which this PR doesn't touch.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This changes the existing `--memory`
streaming path for Anthropic.
- Minimum rollout channel: stable.
- Stable/default behavior changed: only for streaming Anthropic requests
into which the proxy injected memory tools (`--memory` with tool
injection on). Everything else takes the unchanged code path.
- Kill switch / disable path: `--no-memory-tools` /
`HEADROOM_NO_MEMORY_TOOLS=1` stops tool injection, and the filter never
engages.
- Unsafe override required: no.
- Qualification impact: none expected.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A: no
doc describes the old behavior)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Related, but out of scope here:

- **`--mode token` auto-injection with Claude Code** works on the first
request of a session. The memory lands in the first `<system-reminder>`
block of the opening user turn. On later tool-loop requests it doesn't:
the latest user turn is a `tool_result` carrying the cache breakpoint,
so the decision is `no_eligible_user_turn`. The text injected into the
first turn's earlier block also differs from the bytes the client
re-sends next turn. The `cache_control` guard in
`_append_context_to_latest_non_frozen_user_turn` avoids exactly that,
but only for the block carrying the breakpoint itself. I'm happy to
follow up separately.
- In a round that calls both a memory tool and a client tool, the memory
results can't reach the model, because the client owns the turn. Saves
still persist.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-03 16:14:17 -05:00