Files
headroom/tests/cli/test_memory.py
Ehsan-hashandJerrettDavis c4ea31b156 fix(memory): resolve partial IDs beyond the first 1000 records (#4389)
## Description

`headroom memory show` tries a full ID first, then searches by prefix.
The fallback only examined the newest 1,000 memories. That could report
an older memory as missing or silently select a newer memory when
another matching ID existed outside that first page.

Use the existing unlimited `MemoryFilter()` query for this fallback.
Exact-ID lookup, literal prefix matching, superseded-memory filtering,
and the existing missing/ambiguous-ID errors stay the same.

## Type of Change

- [x] Bug fix

## Changes Made

- Remove the 1,000-row cap from the partial-ID fallback.
- Add real SQLite CLI regressions for an older unique match and
ambiguity across that boundary.

## Testing

- [x] Focused and affected integration tests pass.
- [x] Lint, format, type, and repository pre-push checks pass.
- [x] Actual CLI behavior verified with a synthetic SQLite database.

- Before the correction, `TestMemoryShow` had 4 passing tests and both
new regressions failed.
- Afterward, all 6 show tests passed.
- All 77 affected CLI, index-sync, query-filter, and SQLite-store tests
passed.
- Whole-project Ruff check and format check passed (1,812 files already
formatted).
- Repository pre-commit checks, including mypy and commitlint, passed.
- The full repository pre-push gate passed under pinned Rust 1.95.0 and
Node 22.20.0: Rust formatting, workspace Clippy/tests, extension
rebuild/import verification, 177 Python tests (4 skipped), and
commitlint. No hooks were bypassed.
- Independent review found no blocking findings.

```text
77 passed, 274 warnings in 2.21s
```

The warnings are existing `datetime.utcnow()` deprecations.

## Real Behavior Proof

- Environment: macOS arm64, Python 3.12.15, Rust 1.95.0, editable
checkout with its native extension built locally. The 77 affected tests
and CLI proof were repeated after the pinned extension rebuild. No
provider/model was used for this proof. All proof data was synthetic,
with offline mode enabled and telemetry disabled.

- Exact command / steps: Create a temporary `SQLiteMemoryStore`, use
`save_batch` to add 1,000 filler memories dated 2022-01-01 and
`target-memory-old` dated 2021-01-01. Run the actual installed CLI
against that database:

```sh
headroom memory show --db-path "$repro_db" target-memory-old --json
headroom memory show --db-path "$repro_db" target- --json
headroom memory show --db-path "$repro_db" absent- --json
```

- Observed result: The unique older prefix resolves, the missing ID is
rejected, and adding a newer match makes the shared prefix ambiguous:

```text
target-memory-old: exit 0, id=target-memory-old
target-:          exit 0, id=target-memory-old
absent-:          exit 1, Error: Memory not found: absent-
```

Then add `target-memory-new` dated 2023-01-01 and repeat the `target-`
command:

```text
exit 1, Error: Ambiguous ID 'target-'. Matches: ['target-m', 'target-m']
```

On the unchanged baseline, the unique older prefix instead returned
`Memory not found`; after adding the newer match, the same prefix
incorrectly exited 0 and selected `target-memory-new`. Full-ID and
missing-ID controls behaved correctly before and after. The temporary
database and all CLI child processes were cleaned up.

- Not tested: The complete Python suite, external providers or embedding
models, Windows, or Linux. The prefix fallback still materializes the
matching store's full query result; full IDs retain the direct lookup
path.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This is a local SQLite CLI
correction.
- Minimum rollout channel: Not applicable; no rollout-managed feature or
channel changes.
- Stable/default behavior changed: Yes. Partial-ID lookup searches all
eligible memories and rejects ambiguous prefixes across the former
1,000-row boundary.
- Kill switch / disable path: No new switch. Full IDs retain the direct
lookup path.
- Unsafe override required: No.
- Qualification impact: No schema, configuration, dependency, provider,
or rollout-qualification changes.
- Rollback path: Revert commit 017c9a5 to restore the previous behavior.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

The contributing agent completed self-review and verification; a
separate reviewer checked the complete patch.

## Checklist

- [x] Regression tests fail before the fix and pass afterward.
- [x] Focused and affected integration tests pass.
- [x] No manual changes to `CHANGELOG.md`.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4389"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-24 08:44:26 -07:00

546 lines
19 KiB
Python

"""Tests for memory CLI commands.
These tests use real SQLite databases (temp files) - no mocks.
"""
import asyncio
import json
from datetime import datetime, timedelta
from pathlib import Path
import pytest
from click.testing import CliRunner
from headroom.cli.main import main
from headroom.memory.adapters.sqlite import SQLiteMemoryStore
from headroom.memory.models import Memory
@pytest.fixture
def runner() -> CliRunner:
"""Create a CLI test runner."""
return CliRunner()
@pytest.fixture
def temp_db(tmp_path: Path) -> str:
"""Create a temporary database path."""
return str(tmp_path / "test_memory.db")
@pytest.fixture
def populated_db(temp_db: str) -> str:
"""Create a database with sample memories."""
store = SQLiteMemoryStore(temp_db)
# Create memories at different scopes and ages
memories = [
# USER scope (no session/agent/turn)
Memory(
id="user-mem-001",
content="User prefers TypeScript over JavaScript",
user_id="test-user",
session_id=None,
agent_id=None,
turn_id=None,
importance=0.9,
created_at=datetime.now() - timedelta(days=5),
valid_from=datetime.now() - timedelta(days=5),
),
# SESSION scope
Memory(
id="session-mem-001",
content="Working on authentication feature",
user_id="test-user",
session_id="session-123",
agent_id=None,
turn_id=None,
importance=0.7,
created_at=datetime.now() - timedelta(hours=2),
valid_from=datetime.now() - timedelta(hours=2),
),
Memory(
id="session-mem-002",
content="Database uses PostgreSQL",
user_id="test-user",
session_id="session-123",
agent_id=None,
turn_id=None,
importance=0.6,
created_at=datetime.now() - timedelta(days=10),
valid_from=datetime.now() - timedelta(days=10),
),
# AGENT scope
Memory(
id="agent-mem-001",
content="Agent is exploring code structure",
user_id="test-user",
session_id="session-123",
agent_id="agent-456",
turn_id=None,
importance=0.4,
created_at=datetime.now() - timedelta(hours=1),
valid_from=datetime.now() - timedelta(hours=1),
),
# TURN scope (ephemeral)
Memory(
id="turn-mem-001",
content="Tool output from grep search",
user_id="test-user",
session_id="session-123",
agent_id="agent-456",
turn_id="turn-789",
importance=0.2,
created_at=datetime.now() - timedelta(minutes=5),
valid_from=datetime.now() - timedelta(minutes=5),
),
# Low importance memory for pruning tests
Memory(
id="low-importance-001",
content="Temporary note",
user_id="test-user",
session_id="session-123",
agent_id=None,
turn_id=None,
importance=0.1,
created_at=datetime.now() - timedelta(days=45),
valid_from=datetime.now() - timedelta(days=45),
),
]
for mem in memories:
asyncio.run(store.save(mem))
return temp_db
class TestMemoryList:
"""Tests for 'headroom memory list' command."""
def test_list_all(self, runner: CliRunner, populated_db: str) -> None:
"""List all memories."""
result = runner.invoke(main, ["memory", "list", "--db-path", populated_db])
assert result.exit_code == 0
# IDs are truncated to 8 chars in display, check for partial matches
assert "user-mem" in result.output
assert "session-" in result.output # "session-mem" truncated to "session-"
def test_list_with_limit(self, runner: CliRunner, populated_db: str) -> None:
"""List with limit."""
result = runner.invoke(main, ["memory", "list", "--db-path", populated_db, "--limit", "2"])
assert result.exit_code == 0
# Should show limited results
assert "2 shown" in result.output or "Memories" in result.output
def test_list_by_scope(self, runner: CliRunner, populated_db: str) -> None:
"""Filter by scope level."""
result = runner.invoke(
main, ["memory", "list", "--db-path", populated_db, "--scope", "USER"]
)
assert result.exit_code == 0
assert "TypeScript" in result.output # USER scope memory content
def test_list_empty_db(self, runner: CliRunner, temp_db: str) -> None:
"""List from empty database."""
# Initialize empty db
SQLiteMemoryStore(temp_db)
result = runner.invoke(main, ["memory", "list", "--db-path", temp_db])
assert result.exit_code == 0
assert "No memories found" in result.output
class TestMemoryShow:
"""Tests for 'headroom memory show' command."""
def test_show_by_id(self, runner: CliRunner, populated_db: str) -> None:
"""Show memory by full ID."""
result = runner.invoke(main, ["memory", "show", "--db-path", populated_db, "user-mem-001"])
assert result.exit_code == 0
assert "TypeScript" in result.output
assert "0.9" in result.output or "0.90" in result.output # importance
def test_show_by_partial_id(self, runner: CliRunner, populated_db: str) -> None:
"""Show memory by partial ID."""
result = runner.invoke(main, ["memory", "show", "--db-path", populated_db, "user-mem"])
assert result.exit_code == 0
assert "TypeScript" in result.output
def test_show_by_partial_id_beyond_default_query_limit(
self, runner: CliRunner, temp_db: str
) -> None:
"""Show a unique partial ID when the match is older than 1,000 rows."""
store = SQLiteMemoryStore(temp_db)
filler_start = datetime(2024, 1, 1)
memories = [
Memory(
id=f"filler-{index:04d}",
content=f"Filler memory {index}",
user_id="test-user",
created_at=filler_start + timedelta(days=index),
valid_from=filler_start + timedelta(days=index),
)
for index in range(1000)
]
memories.append(
Memory(
id="legacy-target-memory",
content="Legacy target outside the first page",
user_id="test-user",
created_at=datetime(2020, 1, 1),
valid_from=datetime(2020, 1, 1),
)
)
asyncio.run(store.save_batch(memories))
result = runner.invoke(main, ["memory", "show", "--db-path", temp_db, "legacy-target-"])
assert result.exit_code == 0
assert "Legacy target outside the first page" in result.output
def test_show_by_partial_id_reports_ambiguity_beyond_default_query_limit(
self, runner: CliRunner, temp_db: str
) -> None:
"""Report ambiguity when matching IDs span beyond the first 1,000 rows."""
store = SQLiteMemoryStore(temp_db)
filler_start = datetime(2024, 1, 1)
memories = [
Memory(
id=f"filler-{index:04d}",
content=f"Filler memory {index}",
user_id="test-user",
created_at=filler_start + timedelta(days=index),
valid_from=filler_start + timedelta(days=index),
)
for index in range(1000)
]
memories.extend(
[
Memory(
id="shared-prefix-old",
content="Older matching memory",
user_id="test-user",
created_at=datetime(2020, 1, 1),
valid_from=datetime(2020, 1, 1),
),
Memory(
id="shared-prefix-new",
content="Newer matching memory",
user_id="test-user",
created_at=datetime(2026, 1, 1),
valid_from=datetime(2026, 1, 1),
),
]
)
asyncio.run(store.save_batch(memories))
result = runner.invoke(main, ["memory", "show", "--db-path", temp_db, "shared-prefix-"])
assert result.exit_code != 0
assert "Ambiguous ID 'shared-prefix-'" in result.output
def test_show_json_output(self, runner: CliRunner, populated_db: str) -> None:
"""Show memory as JSON."""
result = runner.invoke(
main, ["memory", "show", "--db-path", populated_db, "user-mem-001", "--json"]
)
assert result.exit_code == 0
# Should be valid JSON
data = json.loads(result.output)
assert data["id"] == "user-mem-001"
assert "TypeScript" in data["content"]
def test_show_not_found(self, runner: CliRunner, populated_db: str) -> None:
"""Show non-existent memory."""
result = runner.invoke(
main, ["memory", "show", "--db-path", populated_db, "nonexistent-id"]
)
assert result.exit_code != 0 or "not found" in result.output.lower()
class TestMemoryStats:
"""Tests for 'headroom memory stats' command."""
def test_stats(self, runner: CliRunner, populated_db: str) -> None:
"""Show stats for populated database."""
result = runner.invoke(main, ["memory", "stats", "--db-path", populated_db])
assert result.exit_code == 0
assert "Total" in result.output or "Memories" in result.output
assert "6" in result.output # 6 memories
def test_stats_empty_db(self, runner: CliRunner, temp_db: str) -> None:
"""Stats for empty database."""
SQLiteMemoryStore(temp_db)
result = runner.invoke(main, ["memory", "stats", "--db-path", temp_db])
assert result.exit_code == 0
assert "0" in result.output
class TestMemoryEdit:
"""Tests for 'headroom memory edit' command."""
def test_edit_content(self, runner: CliRunner, populated_db: str) -> None:
"""Edit memory content."""
result = runner.invoke(
main,
[
"memory",
"edit",
"--db-path",
populated_db,
"user-mem-001",
"--content",
"Updated content",
],
)
assert result.exit_code == 0
# Verify change
show_result = runner.invoke(
main, ["memory", "show", "--db-path", populated_db, "user-mem-001"]
)
assert "Updated content" in show_result.output
def test_edit_importance(self, runner: CliRunner, populated_db: str) -> None:
"""Edit memory importance."""
result = runner.invoke(
main,
["memory", "edit", "--db-path", populated_db, "user-mem-001", "--importance", "0.5"],
)
assert result.exit_code == 0
# Verify change
show_result = runner.invoke(
main, ["memory", "show", "--db-path", populated_db, "user-mem-001"]
)
assert "0.5" in show_result.output
def test_edit_not_found(self, runner: CliRunner, populated_db: str) -> None:
"""Edit non-existent memory."""
result = runner.invoke(
main,
["memory", "edit", "--db-path", populated_db, "nonexistent", "--content", "test"],
)
assert result.exit_code != 0 or "not found" in result.output.lower()
class TestMemoryDelete:
"""Tests for 'headroom memory delete' command."""
def test_delete_single(self, runner: CliRunner, populated_db: str) -> None:
"""Delete single memory with force."""
result = runner.invoke(
main,
["memory", "delete", "--db-path", populated_db, "turn-mem-001", "--force"],
)
assert result.exit_code == 0
# Verify deleted
show_result = runner.invoke(
main, ["memory", "show", "--db-path", populated_db, "turn-mem-001"]
)
assert "not found" in show_result.output.lower() or show_result.exit_code != 0
def test_delete_multiple(self, runner: CliRunner, populated_db: str) -> None:
"""Delete multiple memories."""
result = runner.invoke(
main,
[
"memory",
"delete",
"--db-path",
populated_db,
"turn-mem-001",
"agent-mem-001",
"--force",
],
)
assert result.exit_code == 0
def test_delete_requires_confirmation(self, runner: CliRunner, populated_db: str) -> None:
"""Delete prompts for confirmation without --force."""
# Invoke delete and say no to confirmation
runner.invoke(
main,
["memory", "delete", "--db-path", populated_db, "turn-mem-001"],
input="n\n", # Say no
)
# Verify memory still exists since we said no
show_result = runner.invoke(
main, ["memory", "show", "--db-path", populated_db, "turn-mem-001"]
)
# Memory should still exist since we said no
assert "Tool output" in show_result.output or show_result.exit_code == 0
class TestMemoryPrune:
"""Tests for 'headroom memory prune' command."""
def test_prune_dry_run(self, runner: CliRunner, populated_db: str) -> None:
"""Prune with dry-run shows what would be deleted."""
result = runner.invoke(
main,
["memory", "prune", "--db-path", populated_db, "--older-than", "30d", "--dry-run"],
)
assert result.exit_code == 0
assert "would" in result.output.lower() or "dry" in result.output.lower()
def test_prune_by_age(self, runner: CliRunner, populated_db: str) -> None:
"""Prune old memories."""
result = runner.invoke(
main,
["memory", "prune", "--db-path", populated_db, "--older-than", "30d", "--force"],
)
assert result.exit_code == 0
# Should have deleted the 45-day old memory
def test_prune_by_scope(self, runner: CliRunner, populated_db: str) -> None:
"""Prune by scope level."""
result = runner.invoke(
main,
["memory", "prune", "--db-path", populated_db, "--scope", "TURN", "--force"],
)
assert result.exit_code == 0
# Verify TURN memories are gone
list_result = runner.invoke(
main, ["memory", "list", "--db-path", populated_db, "--scope", "TURN"]
)
assert "No memories found" in list_result.output or "turn-mem" not in list_result.output
def test_prune_low_importance(self, runner: CliRunner, populated_db: str) -> None:
"""Prune low importance memories."""
result = runner.invoke(
main,
["memory", "prune", "--db-path", populated_db, "--low-importance", "0.3", "--force"],
)
assert result.exit_code == 0
class TestMemoryPurge:
"""Tests for 'headroom memory purge' command."""
def test_purge_requires_confirm_flag(self, runner: CliRunner, populated_db: str) -> None:
"""Purge requires --confirm flag."""
result = runner.invoke(main, ["memory", "purge", "--db-path", populated_db])
assert result.exit_code != 0 or "confirm" in result.output.lower()
def test_purge_with_confirm(self, runner: CliRunner, populated_db: str) -> None:
"""Purge deletes all memories."""
result = runner.invoke(
main,
["memory", "purge", "--db-path", populated_db, "--confirm"],
input="y\n", # Confirm
)
assert result.exit_code == 0
# Verify empty
stats_result = runner.invoke(main, ["memory", "stats", "--db-path", populated_db])
assert "0" in stats_result.output
class TestMemoryExportImport:
"""Tests for export/import commands."""
def test_export_to_stdout(self, runner: CliRunner, populated_db: str) -> None:
"""Export memories to stdout."""
result = runner.invoke(main, ["memory", "export", "--db-path", populated_db])
assert result.exit_code == 0
# Should be valid JSON array
data = json.loads(result.output)
assert isinstance(data, list)
assert len(data) == 6
def test_export_to_file(self, runner: CliRunner, populated_db: str, tmp_path: Path) -> None:
"""Export memories to file."""
output_file = tmp_path / "export.json"
result = runner.invoke(
main,
["memory", "export", "--db-path", populated_db, "--output", str(output_file)],
)
assert result.exit_code == 0
# Verify file
with open(output_file) as f:
data = json.load(f)
assert len(data) == 6
def test_import_from_file(self, runner: CliRunner, temp_db: str, tmp_path: Path) -> None:
"""Import memories from file."""
# Create import file
import_data = [
{
"id": "imported-001",
"content": "Imported memory",
"user_id": "test-user",
"importance": 0.8,
"created_at": datetime.now().isoformat(),
"valid_from": datetime.now().isoformat(),
}
]
import_file = tmp_path / "import.json"
with open(import_file, "w") as f:
json.dump(import_data, f)
# Initialize empty db
SQLiteMemoryStore(temp_db)
result = runner.invoke(
main,
["memory", "import", "--db-path", temp_db, str(import_file), "--force"],
)
assert result.exit_code == 0
# Verify imported
show_result = runner.invoke(main, ["memory", "show", "--db-path", temp_db, "imported-001"])
assert "Imported memory" in show_result.output
def test_export_import_roundtrip(
self, runner: CliRunner, populated_db: str, tmp_path: Path
) -> None:
"""Export and import should be lossless."""
export_file = tmp_path / "roundtrip.json"
new_db = str(tmp_path / "new.db")
# Export
runner.invoke(
main, ["memory", "export", "--db-path", populated_db, "--output", str(export_file)]
)
# Import to new db
SQLiteMemoryStore(new_db)
runner.invoke(main, ["memory", "import", "--db-path", new_db, str(export_file), "--force"])
# Compare stats
orig_stats = runner.invoke(main, ["memory", "stats", "--db-path", populated_db])
new_stats = runner.invoke(main, ["memory", "stats", "--db-path", new_db])
# Should have same count
assert "6" in orig_stats.output
assert "6" in new_stats.output
class TestMemoryHelp:
"""Tests for help output."""
def test_memory_help(self, runner: CliRunner) -> None:
"""Memory group shows help."""
result = runner.invoke(main, ["memory", "--help"])
assert result.exit_code == 0
assert "list" in result.output
assert "show" in result.output
assert "stats" in result.output
assert "edit" in result.output
assert "delete" in result.output
assert "prune" in result.output
assert "purge" in result.output
assert "export" in result.output
assert "import" in result.output
def test_list_help(self, runner: CliRunner) -> None:
"""List command shows help."""
result = runner.invoke(main, ["memory", "list", "--help"])
assert result.exit_code == 0
assert "--limit" in result.output
assert "--scope" in result.output
assert "--since" in result.output