Commit Graph
1 Commits
Author SHA1 Message Date
Yuchuan HeandJerrettDavis 0d9155019f fix(memory): dispatch the advertised memory_list tool (#4830)
## Description

`MemorySystem.get_tools()` advertises `memory_list`, but
`process_tool_call()` has no handler for it. An agent can see the tool
and still receive `Unknown tool: memory_list` when trying to browse
stored memories or obtain an ID for an update.

Dispatch the existing tool to the backend's scoped listing API. A
session-bound MemorySystem lists only that user's current session; an
unscoped one retains user-wide browsing. Return recent memories with
their IDs and readable fields, without embeddings or a semantic query.
Backends without the requested listing scope return an explicit
unsupported-operation error rather than falling back to a wider scope.

Refs #4362, item 1 only.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Add the `memory_list` dispatcher and public handler.
- Keep the schema's default limit of 10 and bounds of 1–100. Filter by
user and, when bound, session before applying the limit, using the
existing HierarchicalMemory.get_session_memories API through
LocalBackend.
- Cover normal and optimized tool inventories, empty results, bounds,
unsupported backends and backend failures.

## Testing

- [x] Relevant unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Formatting passes (`ruff format --check .`)
- [x] Type checking passes (original full-package mypy hook)
- [x] Regression tests added
- [x] Standalone runtime validation completed

### Test Output

```text
Initial regressions against original production code: 8 failed
Session regressions against initial PR head: 4 failed, 9 passed
Final list regressions on the corrected head: 13 passed
pytest tests/test_memory_list_dispatch.py tests/test_memory_system.py tests/test_memory_tool_project_scope.py -q:
  113 passed, 13 skipped
Original pre-commit and commit-msg hooks: passed
Whole-tree Ruff check and format --check: passed
Original pre-push make ci-precheck: passed
  Rust workspace: 1749 passed, 3 ignored
  Native extension: rebuilt, installed and import-verified
  Critical Python: 177 passed, 8 skipped
  commitlint: passed
```

The owning-suite skips require Hugging Face model files unavailable
offline. Existing datetime/optional-model deprecation warnings remain
visible in the logs.

## Real Behavior Proof

- Environment: Windows 11, CPython 3.13.9, existing dev venv; clean
production baseline `9f2cc017863f540c61f8747b3c744a97776ebaf2`, patch
`42aca035dde6734d1a9d7a0b87f1b99cf8b27e69`. Explicit checkout on
PYTHONPATH; no paid/provider endpoint. The loopback service returns
controlled vectors, not trained-model inference.
- Exact command / steps: Save the script below as `proof.py`. From each
checkout run `python proof.py list <fresh-output-directory> before` on
main, or `python proof.py list <fresh-output-directory> after` on the
patch. Identical scripts ran outside pytest. The script initializes a
real LocalBackend, saves three Alice memories and one Bob memory through
MemorySystem, then calls the advertised listing tool with limit=2. The
Ollama HTTP adapter sends actual requests to a bound loopback receiver
returning controlled 384-dimensional vectors; SQLite, FTS5 and the
configured local vector index are real.

<details>
<summary>Standalone reproduction script</summary>

```python
"""Standalone real SQLite/vector/FTS/HTTP and agent-file boundary reproductions.

Usage: python memory_http_proof.py {list,vectors,sync} FRESH_OUTPUT {before,after}
The loopback embedding service returns controlled vectors, not model inference.
"""

import asyncio
from datetime import datetime, timedelta, timezone
import hashlib
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import platform
import sqlite3
import sys
import threading

import headroom
import headroom._core
from headroom.memory.backends.local import LocalBackend, LocalBackendConfig
from headroom.memory.config import EmbedderBackend, MemoryConfig, VectorBackend
from headroom.memory.core import HierarchicalMemory
from headroom.memory.models import Memory
from headroom.memory.sync import sync, sync_import
from headroom.memory.sync_adapters.codex_agent import CodexAdapter
from headroom.memory.system import MemorySystem

sys.stdout.reconfigure(encoding="utf-8")
case, directory, expected = sys.argv[1:]
assert case == "list"
assert expected in ("before", "after")
out = Path(directory).resolve()
out.mkdir(parents=True, exist_ok=False)
os.environ["HEADROOM_STATELESS"] = "0"
os.environ["HEADROOM_WORKSPACE_DIR"] = str(out / "workspace")
os.environ["HEADROOM_CCR_BACKEND"] = "memory"
os.environ["HEADROOM_OFFLINE"] = "0"  # Only the ephemeral loopback service is called.


class Endpoint(BaseHTTPRequestHandler):
    dimension = 384
    requests = []

    def log_message(self, *_):
        pass

    def do_POST(self):
        payload = json.loads(self.rfile.read(int(self.headers["content-length"])))
        Endpoint.requests.append({"path": self.path, "body": payload, "dimension": self.dimension})
        assert self.path == "/api/embeddings", self.path
        body = json.dumps({"embedding": [1.0] + [0.0] * (self.dimension - 1)}).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)


server = ThreadingHTTPServer(("127.0.0.1", 0), Endpoint)
worker = threading.Thread(target=server.serve_forever, daemon=True)
worker.start()
url = f"http://127.0.0.1:{server.server_port}"


def backend_config(name):
    return LocalBackendConfig(
        db_path=str(out / (name + ".db")), embedder_backend="ollama",
        embedder_model="probe-fixed", ollama_base_url=url, vector_dimension=384,
    )


async def list_case():
    backend = LocalBackend(backend_config("list"))
    try:
        system = MemorySystem(backend, user_id="alice")
        ids = []
        for content in ("First scoped fact", "Second scoped fact", "Latest scoped fact"):
            response = await system.process_tool_call("memory_save", {"content": content, "importance": 0.4})
            assert response["success"], response
            ids.append(response["memory_id"])
        response = await MemorySystem(backend, "bob").process_tool_call("memory_save", {"content": "Bob's private fact", "importance": 0.4})
        assert response["success"], response
        before_requests = len(Endpoint.requests)
        result = await system.process_tool_call("memory_list", {"limit": 2})
        result["embedding_requests_for_listing"] = len(Endpoint.requests) - before_requests
        result["stored_alice_count"] = len(await backend.get_user_memories("alice"))
        if expected == "after":
            assert result["success"] and result["count"] == 2, result
            assert [m["memory_id"] for m in result["memories"]] == [ids[2], ids[1]]
            assert all("Bob" not in m["content"] for m in result["memories"])
        else:
            assert result["success"] is False and result["error"] == "Unknown tool: memory_list", result
        assert result["stored_alice_count"] == 3 and result["embedding_requests_for_listing"] == 0
        return result
    finally:
        await backend.close()


try:
    result = asyncio.run(list_case())
    result.update({
        "case": case, "expected": expected, "python": platform.python_version(),
        "source": str(Path(headroom.__file__).resolve()),
        "native": str(Path(headroom._core.__file__).resolve()),
        "http_requests": Endpoint.requests,
        "not_tested": "Trained model inference, paid APIs, provider answer quality, non-Windows runtime, non-local memory backends.",
    })
    (out / "result.json").write_text(json.dumps(result, indent=2), encoding="utf-8", newline="\n")
    print(json.dumps(result, indent=2))
finally:
    server.shutdown()
    server.server_close()
    worker.join(timeout=5)
```

</details>

- Observed result: On main the tool returns Unknown tool: memory_list
even though Alice has three stored rows. With the patch it returns
Alice's two newest memories, in order, with the saved IDs. Bob's memory
is absent, and listing makes zero additional embedding requests.


Session-scope follow-up: run the script below as `python
session_proof.py <fresh-output-directory> before` on the initial PR head
`908762a8705dea9a878f28dca4cb25b3fc5136ee`, or with `after` on the
corrected head. It saves real memories for Alice/session-a,
Alice/session-b and Bob/session-a through LocalBackend and its actual
loopback Ollama HTTP adapter. With limit=1, the initial head returns the
newer Alice/session-b fact to session-a; the correction returns
session-a's fact. A missing session returns zero results, unscoped
browsing still returns both Alice facts, Bob remains excluded, and
listing sends no embedding request.

<details>
<summary>Session isolation reproduction</summary>

```python
"""Real LocalBackend listing must filter user and session before LIMIT."""
import asyncio
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import sys
import threading

from headroom.memory.backends.local import LocalBackend, LocalBackendConfig
from headroom.memory.system import MemorySystem

sys.stdout.reconfigure(encoding="utf-8")
directory, expected = sys.argv[1:]
assert expected in ("before", "after")
out = Path(directory).resolve()
out.mkdir(parents=True, exist_ok=False)
os.environ.update(HEADROOM_OFFLINE="0", HEADROOM_STATELESS="0", HEADROOM_CCR_BACKEND="memory")


class Endpoint(BaseHTTPRequestHandler):
    calls = []

    def log_message(self, *_):
        pass

    def do_POST(self):
        request = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        assert self.path == "/api/embeddings"
        Endpoint.calls.append(request["prompt"])
        response = json.dumps({"embedding": [1.0] + [0.0] * 383}).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(response)))
        self.end_headers()
        self.wfile.write(response)


server = ThreadingHTTPServer(("127.0.0.1", 0), Endpoint)
worker = threading.Thread(target=server.serve_forever, daemon=True)
worker.start()


async def main():
    backend = LocalBackend(LocalBackendConfig(
        db_path=str(out / "memory.db"), embedder_backend="ollama", embedder_model="probe-fixed",
        ollama_base_url=f"http://127.0.0.1:{server.server_port}", vector_dimension=384,
    ))
    try:
        current = MemorySystem(backend, "alice", session_id="session-a")
        other_session = MemorySystem(backend, "alice", session_id="session-b")
        other_user = MemorySystem(backend, "bob", session_id="session-a")
        saved = []
        for system, content in ((current,"Current-session fact"),(other_session,"Newer other-session fact"),(other_user,"Newest other-user fact")):
            result = await system.process_tool_call("memory_save", {"content":content,"importance":0.4})
            assert result["success"],result
            saved.append(result["memory_id"])
        count = len(Endpoint.calls)
        listed = await current.process_tool_call("memory_list", {"limit":1})
        assert listed["success"] and listed["count"] == 1,listed
        scope_preserved = listed["memories"][0]["memory_id"] == saved[0]
        assert scope_preserved == (expected == "after"),listed
        assert listed["memories"][0]["memory_id"] != saved[2]
        missing = await MemorySystem(backend,"alice",session_id="missing").process_tool_call("memory_list", {"limit":1})
        assert missing["count"] == (0 if expected == "after" else 1),missing
        unscoped = await MemorySystem(backend,"alice").process_tool_call("memory_list", {"limit":2})
        assert [m["memory_id"] for m in unscoped["memories"]] == [saved[1],saved[0]]
        assert len(Endpoint.calls) == count
        result = {"session_isolated":scope_preserved,"scoped_content":listed["memories"][0]["content"],"missing_session_count":missing["count"],"unscoped_count":unscoped["count"],"other_user_excluded":True,"embedding_calls_for_listing":len(Endpoint.calls)-count,"save_http_calls":count,"expected":expected,"source":str(Path(__import__('headroom').__file__).resolve())}
        (out/"result.json").write_text(json.dumps(result,indent=2),encoding="utf-8",newline="\n")
        print(json.dumps(result,indent=2))
    finally:
        await backend.close()


try:
    asyncio.run(main())
finally:
    server.shutdown()
    server.server_close()
    worker.join(timeout=5)
```

</details>

- Not tested: Remote memory backends, trained embedding inference, paid
APIs, model answer quality, non-Windows runtime, or the complete Python
suite.

## Runtime Rollout Safety

- Rollout-managed feature(s): Existing memory behavior; no new feature
flag or channel expansion.
- Minimum rollout channel: Normal release after maintainer review and CI
qualification.
- Stable/default behavior changed: The advertised tool can enumerate
recent memories on backends that expose get_user_memories. Other memory
operations retain their dispatch paths.
- Kill switch / disable path: Use the existing memory-disable path, or
revert this commit.
- Unsafe override required: No.
- Qualification impact: Related tests and original local gates passed;
hosted CI is tracked separately.
- Rollback path: Revert this commit; no dependency or storage-schema
change.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have added tests that prove the fix
- [x] Relevant new and existing tests pass locally
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Listing is supported through LocalBackend's user and session APIs; the
session adapter delegates to an existing core query that applies both
filters before LIMIT. This patch does not add an enumeration API to
DirectMem0. The author's review covered the initial patch and evidence;
the session-isolation review correction was additionally reviewed and
validated with Codex.

Developed and validated with Codex assistance.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4830"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-16 23:09:26 -06:00