fix: [BUG] DirectMem0Adapter.search_memories calls removed mem0 search API (#4518)

<!-- oss-harness:v2 -->
Fixes #4246.

Focused Python fix for: [BUG] DirectMem0Adapter.search_memories calls
removed mem0 search API

Changed implementation: headroom/memory/backends/direct_mem0.py.
Regression tests: tests/test_memory/test_direct_mem0.py.

Verification (working directory `.`): `.venv/bin/python -m pytest -q -x
-p no:cacheprovider -- /workspace/tests/test_memory/test_direct_mem0.py`
- Patched code: 19 tests, zero failures/errors.
- Original code with the same regression tests: 1 failures, zero
collection/setup errors.
- Base commit: `f10725e6b9940d8bc2fcde950c86f8ac97543cb6`.

AI-assisted contribution: OpenCode using Autoloops wrote this patch. The
automated harness independently ran the tests listed above. No human
review is claimed.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4518"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: nandanadileep <116407139+nandanadileep@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Nandana Dileep <nandanadileep@users.noreply.github.com>
This commit is contained in:
Nandana Dileep
2026-11-04 08:33:51 -06:00
committed by GitHub
co-authored by nandanadileep JerrettDavis Cursor Agent Nandana Dileep
parent 3b8dfa5e3a
commit d9cee45268
4 changed files with 377 additions and 21 deletions
+39
View File
@@ -248,6 +248,45 @@ config = MemoryConfig(
)
```
### Direct Mem0 / Qdrant payload (mem0 2.x)
The `qdrant-neo4j` backend (`DirectMem0Adapter`) writes facts straight into Qdrant so Mem0 can skip its extra LLM extraction. mem0ai 2.x search (`Memory.search` → `_search_vector_store`) **skips any point whose payload has no `data` field**. New writes store the fact under both:
- `data` — required for mem0 2.x search
- `memory` — legacy alias for older payload readers
Existing collections written before this contract only have `memory`. Those points still sit in Qdrant, but `search_memories` returns nothing for them until you copy `memory` onto `data`:
```python
from qdrant_client import QdrantClient
client = QdrantClient(...) # same URL/host as Headroom
collection = "headroom_memories" # Mem0Config.collection_name
offset = None
while True:
points, offset = client.scroll(
collection_name=collection,
with_payload=True,
limit=100,
offset=offset,
)
updates = []
for point in points:
payload = point.payload or {}
if payload.get("data") or not payload.get("memory"):
continue
updates.append(point.id)
client.set_payload(
collection_name=collection,
payload={"data": payload["memory"]},
points=[point.id],
)
if offset is None:
break
```
Run this once per collection after upgrading. New saves already write `data` and do not need a second pass.
## Provider Compatibility
Memory works with any OpenAI-compatible client:
+79 -20
View File
@@ -72,6 +72,60 @@ def _utcnow() -> datetime:
return datetime.now(timezone.utc)
def _mem0_search_metadata(result: dict[str, Any]) -> dict[str, Any]:
"""Return extra payload fields from a Mem0 2.x search hit.
``MemoryItem.metadata`` defaults to ``None``. When a hit has no extra
payload keys, ``MemoryItem.model_dump()`` therefore emits
``metadata: None``. ``dict.get("metadata", {})`` still returns ``None``
in that case (the key is present), so callers must not treat the default
as a guaranteed mapping.
"""
metadata = result.get("metadata")
return metadata if isinstance(metadata, dict) else {}
def _mem0_search_content(result: dict[str, Any], metadata: dict[str, Any]) -> str:
"""Resolve memory text from a Mem0 2.x search hit.
Mem0 maps the Qdrant payload's ``data`` field onto the result's
``memory`` key. Direct-adapter writes also keep a ``memory`` alias for
older payload readers, which can surface under ``metadata`` when Mem0
treats it as extra payload.
"""
content = result.get("memory") or result.get("data")
if not content:
content = metadata.get("memory") or metadata.get("data") or result.get("content")
return content or ""
def _direct_qdrant_payload(
fact: str,
user_id: str,
importance: float,
metadata: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Build a Qdrant payload that Mem0 2.x search can round-trip.
Mem0's ``_search_vector_store`` skips hits whose payload has no ``data``
field. Headroom historically stored the fact under ``memory`` only, so
saved records were invisible to ``Memory.search``. New writes store the
text under both ``data`` (required) and ``memory`` (legacy alias).
"""
return {
"user_id": user_id,
"importance": importance,
"created_at": _utcnow().isoformat(),
"hash": hashlib.md5(fact.encode()).hexdigest(), # nosec B324
**(metadata or {}),
# ``data`` must win over caller metadata: Mem0 2.x search skips
# payloads that lack it. ``memory`` is a compatibility alias for
# records and readers that predate the mem0 2.x payload contract.
"data": fact,
"memory": fact,
}
@dataclass
class Mem0Config:
"""Configuration for Direct Mem0 adapter.
@@ -294,6 +348,11 @@ class DirectMem0Adapter:
) -> list[str]:
"""Write pre-extracted facts directly to Qdrant.
Each point's payload stores the fact under ``data`` (required by
mem0ai 2.x ``Memory.search``) and ``memory`` (legacy alias). Records
written before this contract only have ``memory`` and will not be
returned by search until they are migrated; see wiki/memory.md.
Args:
facts: List of fact strings to store.
user_id: User identifier.
@@ -315,16 +374,12 @@ class DirectMem0Adapter:
# Generate embedding
embedding = await asyncio.to_thread(self._embed, fact)
# Build metadata payload
payload = {
"data": fact, # the key Mem0 search reads; points without it are skipped
"memory": fact,
"user_id": user_id,
"importance": importance,
"created_at": _utcnow().isoformat(),
"hash": hashlib.md5(fact.encode()).hexdigest(), # nosec B324
**(metadata or {}),
}
payload = _direct_qdrant_payload(
fact=fact,
user_id=user_id,
importance=importance,
metadata=metadata,
)
points.append(
PointStruct(
@@ -810,7 +865,11 @@ class DirectMem0Adapter:
"""
await self._ensure_initialized()
# Use Mem0's search (no LLM calls, just embeddings)
# Use Mem0's search (no LLM calls, just embeddings). mem0ai 2.x is
# keyword-only and rejects top-level entity params via
# ``_reject_top_level_entity_params`` (raising for ``user_id``) while
# silently swallowing ``limit`` from ``**kwargs``. Scope with
# ``filters`` and bound the result count with ``top_k``.
results = await asyncio.to_thread(
self._mem0_client.search,
query=query,
@@ -822,17 +881,17 @@ class DirectMem0Adapter:
result_list = results if isinstance(results, list) else results.get("results", [])
for result in result_list:
# Mem0 returns memory in metadata.memory when using direct Qdrant writes
result_metadata = result.get("metadata", {})
memory_content = result.get("memory") or result_metadata.get(
"memory", result.get("content", "")
)
# Mem0 2.x may omit extra payload keys and then dump
# ``metadata: None`` (not a missing key). Entity/session filters
# and user_id construction all call ``.get`` on this mapping.
result_metadata = _mem0_search_metadata(result)
memory_content = _mem0_search_content(result, result_metadata)
memory_id = result.get("id", str(uuid.uuid4()))
similarity = result.get("score", result.get("similarity", 0.0))
memory_entities = result_metadata.get("entity_refs") or []
# Filter by entities if provided
if entities:
memory_entities = result_metadata.get("entity_refs", [])
content_lower = memory_content.lower()
if not any(e in memory_entities for e in entities):
if not any(e.lower() in content_lower for e in entities):
@@ -845,10 +904,10 @@ class DirectMem0Adapter:
memory = Memory(
id=memory_id,
content=memory_content,
user_id=result_metadata.get("user_id", user_id),
user_id=result.get("user_id") or result_metadata.get("user_id") or user_id,
session_id=result_metadata.get("session_id"),
importance=result_metadata.get("importance", 0.5),
entity_refs=result_metadata.get("entity_refs", []),
entity_refs=memory_entities,
metadata=result_metadata,
created_at=_utcnow(),
valid_from=_utcnow(),
@@ -858,7 +917,7 @@ class DirectMem0Adapter:
MemorySearchResult(
memory=memory,
score=float(similarity),
related_entities=result_metadata.get("entity_refs", []),
related_entities=memory_entities,
related_memories=[],
)
)
+225 -1
View File
@@ -3,9 +3,11 @@
from __future__ import annotations
import asyncio
import hashlib
import logging
import threading
from unittest.mock import AsyncMock, MagicMock
from typing import Any
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
@@ -301,6 +303,103 @@ async def test_huge_model_supplied_id_is_capped_in_the_warning() -> None:
assert "'short-id'" in short and "…" not in short
class _Mem0V2SearchClient:
"""Minimal stand-in for the mem0ai 2.x ``Memory.search`` contract.
Keyword-only signature with ``top_k``/``filters`` and no top-level entity
parameters; any ``user_id``/``agent_id``/``run_id`` in ``**kwargs`` raises,
and an unknown ``limit`` would be silently swallowed by ``**kwargs``.
"""
def __init__(self) -> None:
self.calls: list[dict[str, object]] = []
self._hits: object = [{"id": "m1", "memory": "Works at Netflix", "score": 0.9}]
def search(self, query, *, top_k=20, filters=None, **kwargs):
for name in ("user_id", "agent_id", "run_id"):
if name in kwargs:
raise ValueError(
f"Top-level entity parameters {{{name!r}}} are not supported in "
f"search(). Use filters={{{name!r}: '...'}} instead."
)
self.calls.append({"query": query, "top_k": top_k, "filters": filters, **kwargs})
return self._hits
class _Mem0NullableMetadataClient(_Mem0V2SearchClient):
"""Real mem0ai 2.x hit shape: MemoryItem.model_dump() emits metadata=None."""
def __init__(self) -> None:
super().__init__()
self._hits = {
"results": [
{
"id": "m1",
"memory": "Works at Netflix",
"hash": None,
"metadata": None,
"score": 0.9,
"created_at": None,
"updated_at": None,
"user_id": "u1",
}
]
}
@pytest.mark.asyncio
async def test_search_memories_uses_mem0_v2_filters_and_top_k() -> None:
"""search_memories must call mem0 2.x search with filters + top_k, not the
removed top-level user_id/limit kwargs (issue #4246)."""
adapter = _adapter()
adapter._ensure_initialized = AsyncMock() # type: ignore[method-assign]
client = _Mem0V2SearchClient()
adapter._mem0_client = client
results = await adapter.search_memories("anything", user_id="u1", top_k=3)
assert client.calls == [{"query": "anything", "top_k": 3, "filters": {"user_id": "u1"}}]
assert [r.memory.content for r in results] == ["Works at Netflix"]
@pytest.mark.asyncio
async def test_search_memories_accepts_nullable_mem0_metadata() -> None:
"""Mem0 2.x dumps metadata=None when a hit has no extra payload keys.
``result.get("metadata", {})`` still returns None, which used to crash
entity/session filtering and user_id construction (issue #4246 follow-up).
"""
adapter = _adapter()
adapter._ensure_initialized = AsyncMock() # type: ignore[method-assign]
adapter._mem0_client = _Mem0NullableMetadataClient()
results = await adapter.search_memories("anything", user_id="u1", entities=["Netflix"], top_k=3)
assert [r.memory.content for r in results] == ["Works at Netflix"]
assert results[0].memory.user_id == "u1"
assert results[0].memory.metadata == {}
missed = await adapter.search_memories("anything", user_id="u1", session_id="s1")
assert missed == []
def test_direct_qdrant_payload_includes_data_for_mem0_search() -> None:
"""Direct Qdrant writes must store text under ``data`` so mem0 2.x search
does not skip the point (``if not payload.get("data"): continue``)."""
from headroom.memory.backends.direct_mem0 import _direct_qdrant_payload
payload = _direct_qdrant_payload("direct adapter fact", "tenant-a", 0.5)
assert payload["data"] == "direct adapter fact"
assert payload["memory"] == "direct adapter fact"
assert payload["user_id"] == "tenant-a"
# Caller metadata cannot strip the field mem0 search requires.
overwritten = _direct_qdrant_payload(
"fact", "u1", 0.1, metadata={"data": "nope", "memory": "nope"}
)
assert overwritten["data"] == "fact"
assert overwritten["memory"] == "fact"
@pytest.mark.asyncio
@pytest.mark.parametrize("level", [logging.INFO, logging.DEBUG])
async def test_get_memory_survives_an_out_of_range_errno(level: int) -> None:
@@ -350,3 +449,128 @@ async def test_search_scopes_by_user_through_filters() -> None:
query="language", filters={"user_id": "u1"}, top_k=2
)
assert [r.memory.content for r in results] == ["Uses Rust"]
def _vec3(text: str) -> list[float]:
"""Deterministic 3-d unit vector so identical strings cosine-match at 1.0."""
digest = hashlib.sha256(text.encode()).digest()
raw = [((digest[i] / 255.0) * 2 - 1) for i in range(3)]
norm = sum(x * x for x in raw) ** 0.5 or 1.0
return [x / norm for x in raw]
class _Deterministic3DEmbedder:
"""Stand-in for a Mem0 embedder: ``embed`` / ``embed_batch``, 3 dimensions."""
def embed(self, text: str, memory_action: str | None = None) -> list[float]:
return _vec3(text)
def embed_batch(self, texts: list[str], memory_action: str | None = None) -> list[list[float]]:
return [_vec3(text) for text in texts]
def _inmemory_mem0(tmp_path: Any, collection_name: str = "headroom_memories"):
"""Real mem0ai Memory + in-memory Qdrant, no OpenAI/graph/network.
Mirrors the maintainer reproduction: Memory.search against the real Qdrant
store, deterministic 3-d embeddings, collection created by Mem0 itself.
"""
pytest.importorskip("mem0")
qdrant_client_mod = pytest.importorskip("qdrant_client")
from mem0.memory import main as mem0_main
client = qdrant_client_mod.QdrantClient(":memory:")
embedder = _Deterministic3DEmbedder()
def _embedder_factory(*_args: object, **_kwargs: object) -> _Deterministic3DEmbedder:
return embedder
def _llm_factory(*_args: object, **_kwargs: object) -> MagicMock:
return MagicMock()
with (
patch.object(mem0_main, "MEM0_TELEMETRY", False),
patch.object(mem0_main.EmbedderFactory, "create", side_effect=_embedder_factory),
patch.object(mem0_main.LlmFactory, "create", side_effect=_llm_factory),
patch.object(mem0_main, "capture_event", lambda *a, **k: None),
):
memory = mem0_main.Memory.from_config(
{
"vector_store": {
"provider": "qdrant",
"config": {
"collection_name": collection_name,
"embedding_model_dims": 3,
"client": client,
},
},
"history_db_path": str(tmp_path / "mem0-history.db"),
}
)
memory.embedding_model = embedder
# Keep this test on dense cosine search. The BM25 sparse slot would load
# fastembed's Qdrant/bm25 model, which is out of scope here.
if hasattr(memory.vector_store, "_has_bm25_slot"):
memory.vector_store._has_bm25_slot = False
return memory, client
@pytest.mark.asyncio
async def test_direct_write_then_mem0_search_round_trip(tmp_path) -> None:
"""Write through the adapter, then retrieve through real mem0ai 2.x search.
Covers the payload contract (``data`` vs ``memory``), nullable metadata on
a Mem0-native hit, and the adapter's search decoding of both.
"""
mem0_memory, qdrant = _inmemory_mem0(tmp_path)
from qdrant_client.models import PointStruct
adapter = DirectMem0Adapter(Mem0Config(enable_graph=False, collection_name="headroom_memories"))
adapter._ensure_initialized = AsyncMock() # type: ignore[method-assign]
adapter._initialized = True
adapter._qdrant_client = qdrant
adapter._mem0_client = mem0_memory
adapter._embed = lambda text: _vec3(text) # type: ignore[method-assign]
fact = "direct adapter fact"
user_id = "tenant-a"
await adapter._write_facts_to_qdrant(facts=[fact], user_id=user_id, importance=0.5)
found = await adapter.search_memories(fact, user_id=user_id, top_k=5)
assert [r.memory.content for r in found] == [fact]
assert found[0].memory.user_id == user_id
# Legacy payload shape: text under ``memory`` only. Mem0 2.x search skips it.
qdrant.upsert(
collection_name="headroom_memories",
points=[
PointStruct(
id="aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
vector=_vec3("legacy memory only"),
payload={
"memory": "legacy memory only",
"user_id": user_id,
"importance": 0.5,
},
)
],
)
legacy = await adapter.search_memories("legacy memory only", user_id=user_id, top_k=5)
assert "legacy memory only" not in [r.memory.content for r in legacy]
# Mem0-native payload (data + user_id, no extra keys) dumps metadata=None.
qdrant.upsert(
collection_name="headroom_memories",
points=[
PointStruct(
id="11111111-2222-3333-4444-555555555555",
vector=_vec3("native mem0 fact"),
payload={"data": "native mem0 fact", "user_id": user_id},
)
],
)
native = await adapter.search_memories("native mem0 fact", user_id=user_id, top_k=5)
native_hit = next(r for r in native if r.memory.content == "native mem0 fact")
assert native_hit.memory.metadata == {}
assert native_hit.memory.user_id == user_id
+34
View File
@@ -728,6 +728,40 @@ for m in client.memory.get_all():
3. Try a more specific search query
4. Check similarity threshold
### DirectMem0 / qdrant-neo4j search returns nothing after a mem0 2.x upgrade
mem0ai 2.x `Memory.search` only returns Qdrant points whose payload includes a `data` field. `DirectMem0Adapter` historically wrote the fact under `memory` only, so those records stay in Qdrant but are invisible to search.
New writes store both `data` (required) and `memory` (legacy alias). To migrate existing points, copy `memory` onto `data` once:
```python
from qdrant_client import QdrantClient
client = QdrantClient(...) # same URL/host as Headroom
collection = "headroom_memories" # Mem0Config.collection_name
offset = None
while True:
points, offset = client.scroll(
collection_name=collection,
with_payload=True,
limit=100,
offset=offset,
)
for point in points:
payload = point.payload or {}
if payload.get("data") or not payload.get("memory"):
continue
client.set_payload(
collection_name=collection,
payload={"data": payload["memory"]},
points=[point.id],
)
if offset is None:
break
```
Also confirm search is called with mem0 2.x `filters={"user_id": ...}` and `top_k`, not the removed top-level `user_id` / `limit` kwargs.
### High latency
1. Use local embeddings: `embedder_backend=EmbedderBackend.LOCAL`