fix(memory): preserve parent scopes when clearing memories (#4305)

## Description

Closes #4303.

Clearing an agent within one session could also delete that agent's
memories in other sessions, while leaving their search entries behind.
Apply every supplied scope to both the ID query and the delete, so they
target the same records.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Share scope predicates between queries and clears, including agent +
turn without a session.
- Keep superseded records eligible for clearing.
- Add eight scope combinations using real SQLite, FTS5 and sqlite-vec
stores; check sibling records, cache reads and search results before and
after reopening.

## Testing

- [ ] Unit tests pass (`pytest`) — focused suites verified; no
full-suite claim.
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (repository mypy hook: `mypy headroom
--ignore-missing-imports`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
Unmodified main, new regression: 4 failed, 4 passed
Local fix, focused suites: 102 passed, 8 skipped
make ci-precheck-python: 177 passed, 4 skipped
Rust workspace fmt, clippy and tests: passed
Repository pre-commit hooks, including mypy: passed
ruff check .: All checks passed!
ruff format --check .: 1803 files already formatted
```

Focused command:
```bash
HEADROOM_OFFLINE=1 HEADROOM_BEACON=off python -m pytest \
  tests/test_memory/test_clear_scope_consistency.py \
  tests/test_memory/test_query_conditions.py \
  tests/test_memory/test_hierarchical.py \
  tests/test_memory/test_core_metadata_index_sync.py \
  tests/test_sqlite_vector_index.py \
  tests/test_memory/test_supersession_repair.py \
  tests/test_cli_memory_index_sync.py -q --no-cov
```

## Real Behavior Proof

- Environment: macOS 15.7.4 arm64, Python 3.13.5, temporary on-disk
SQLite/FTS5/sqlite-vec stores. Fixed 3D embeddings for index checks; no
model service or API credentials.
- Exact command / steps: save the standalone code in #4303 as
`/tmp/headroom-clear-scope.py`, then run `PYTHONPATH=. .venv/bin/python
/tmp/headroom-clear-scope.py` from the checkout. It stores two memories
with the same user/agent in different sessions and clears only `s1`.
- Observed result: main prints `deleted: 2` and `s2 preserved: False`;
this patch prints `deleted: 1` and `s2 preserved: True`. A separate
public-API run with real vector/text stores also preserves the sibling
and keeps search results consistent after reopening. User-only and
session-only controls remain correct.
- Not tested: live embedding quality, cloud backends, production proxy
traffic, Windows/Linux, or the entire Python test suite. Optional
HNSW/model-dependent tests were skipped. No claim that default user-wide
CLI clearing is affected.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; existing SQLite memory API.
- Minimum rollout channel: existing memory users; no new channel.
- Stable/default behavior changed: scoped clearing preserves memories
outside the supplied scope.
- Kill switch / disable path: avoid granular clears; no new toggle.
- Unsafe override required: no.
- Qualification impact: scoped storage/index consistency; no model or
routing changes.
- Rollback path: revert this fix; no schema or data migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have added tests that prove my fix is effective
- [x] I did not edit CHANGELOG.md

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4305"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
takiAA
2026-11-04 22:48:52 -06:00
committed by GitHub
co-authored by JD Davis
parent fe01961b29
commit a8375a1d26
2 changed files with 106 additions and 52 deletions
+16 -52
View File
@@ -452,49 +452,18 @@ class SQLiteMemoryStore:
conditions: list[str] = []
params: list[Any] = []
# Hierarchical scope filtering
# Every supplied scope narrows the match. IDs can be reused under
# different parents, so no child scope can replace a parent predicate.
if filter.user_id is not None:
conditions.append("user_id = ?")
params.append(filter.user_id)
# Hierarchical filtering: when filtering by user_id only,
# return USER-level and below (all that user's memories)
# This is implicit - we just filter by user_id
if filter.session_id is not None:
conditions.append("session_id = ?")
params.append(filter.session_id)
# agent_id and turn_id are independent narrowing constraints:
# turn_id must be applied even when agent_id is absent. Nesting
# the turn_id check inside the agent_id block dropped the turn
# filter for a (session_id + turn_id, no agent_id) query, so it
# returned the whole session instead of the one turn.
if filter.agent_id is not None:
conditions.append("agent_id = ?")
params.append(filter.agent_id)
if filter.turn_id is not None:
conditions.append("turn_id = ?")
params.append(filter.turn_id)
elif filter.agent_id is not None:
# Agent without session - unusual but supported
conditions.append("agent_id = ?")
params.append(filter.agent_id)
elif filter.turn_id is not None:
# Turn without session/agent - unusual but supported
conditions.append("turn_id = ?")
params.append(filter.turn_id)
elif filter.session_id is not None:
# Session without user - filter by session only
if filter.session_id is not None:
conditions.append("session_id = ?")
params.append(filter.session_id)
elif filter.agent_id is not None:
# Agent without user/session
if filter.agent_id is not None:
conditions.append("agent_id = ?")
params.append(filter.agent_id)
elif filter.turn_id is not None:
# Turn only
if filter.turn_id is not None:
conditions.append("turn_id = ?")
params.append(filter.turn_id)
@@ -878,22 +847,17 @@ class SQLiteMemoryStore:
Returns:
Number of memories deleted.
"""
conditions = ["user_id = ?"]
params: list[Any] = [user_id]
if turn_id is not None:
# Clear only specific turn
conditions.append("turn_id = ?")
params.append(turn_id)
elif agent_id is not None:
# Clear agent and its turns
conditions.append("agent_id = ?")
params.append(agent_id)
elif session_id is not None:
# Clear session and its agents/turns
conditions.append("session_id = ?")
params.append(session_id)
# If only user_id, clear all user's memories
# Use the same scope as the query that selects IDs for index cleanup.
# Clearing includes historical versions as well as active memories.
conditions, params = self._build_query_conditions(
MemoryFilter(
user_id=user_id,
session_id=session_id,
agent_id=agent_id,
turn_id=turn_id,
include_superseded=True,
)
)
where_clause = " AND ".join(conditions)
@@ -0,0 +1,90 @@
"""Scoped clears must preserve sibling memories and their search indexes."""
from contextlib import asynccontextmanager
import numpy as np
import pytest
from headroom.memory.adapters.cache import LRUMemoryCache
from headroom.memory.adapters.fts5 import FTS5TextIndex
from headroom.memory.adapters.sqlite import SQLiteMemoryStore
from headroom.memory.adapters.sqlite_vector import SQLiteVectorIndex, is_sqlite_vec_available
from headroom.memory.config import MemoryConfig
from headroom.memory.core import HierarchicalMemory
from headroom.memory.ports import MemoryFilter, TextFilter
class FixedEmbedder:
async def embed(self, text: str) -> np.ndarray:
return np.array([1.0, 0.0, 0.0], dtype=np.float32)
@asynccontextmanager
async def memory_system(path):
memory = HierarchicalMemory(
SQLiteMemoryStore(path / "memory.db"),
SQLiteVectorIndex(dimension=3, db_path=path / "vectors.db"),
FTS5TextIndex(path / "memory.db"),
FixedEmbedder(),
LRUMemoryCache(),
MemoryConfig(auto_bubble=False, vector_dimension=3),
)
try:
yield memory
finally:
await memory.close()
async def assert_remaining(memory, ids, remaining):
expected = {ids[label] for label in remaining}
rows = await memory.query(MemoryFilter(include_superseded=True))
assert {row.id for row in rows} == expected
vector_hits = await memory.search("orchard", top_k=10)
assert {hit.memory.id for hit in vector_hits} == expected
text_hits = await memory.text_index.search_memories(TextFilter(query="orchard"))
assert {hit.memory.id for hit in text_hits} == expected
for label, memory_id in ids.items():
assert (await memory.get(memory_id) is not None) == (label in remaining)
@pytest.mark.skipif(not is_sqlite_vec_available(), reason="sqlite-vec unavailable")
@pytest.mark.parametrize(
("scope", "removed"),
[
({}, {"target", "other-turn", "other-agent", "other-session"}),
({"session_id": "s1"}, {"target", "other-turn", "other-agent"}),
({"agent_id": "planner"}, {"target", "other-turn", "other-session"}),
({"turn_id": "t1"}, {"target", "other-agent", "other-session"}),
({"session_id": "s1", "agent_id": "planner"}, {"target", "other-turn"}),
({"session_id": "s1", "turn_id": "t1"}, {"target", "other-agent"}),
({"agent_id": "planner", "turn_id": "t1"}, {"target", "other-session"}),
({"session_id": "s1", "agent_id": "planner", "turn_id": "t1"}, {"target"}),
],
ids=["user", "session", "agent", "turn", "session-agent", "session-turn", "agent-turn", "full"],
)
async def test_clear_scope_preserves_siblings_and_indexes_after_reopen(tmp_path, scope, removed):
rows = [
("target", "alice", "s1", "planner", "t1"),
("other-turn", "alice", "s1", "planner", "t2"),
("other-agent", "alice", "s1", "reviewer", "t1"),
("other-session", "alice", "s2", "planner", "t1"),
("other-user", "bob", "s1", "planner", "t1"),
]
ids = {}
async with memory_system(tmp_path) as memory:
for label, user, session, agent, turn in rows:
entry = await memory.add(
"orchard " + label,
user_id=user,
session_id=session,
agent_id=agent,
turn_id=turn,
)
ids[label] = entry.id
await assert_remaining(memory, ids, set(ids))
assert await memory.clear_scope(user_id="alice", **scope) == len(removed)
remaining = set(ids) - removed
await assert_remaining(memory, ids, remaining)
async with memory_system(tmp_path) as memory:
await assert_remaining(memory, ids, remaining)