Files
headroom/tests/test_memory/test_writers.py
Tejas ChopraandClaude Opus 5.5 2e497cbec3 refactor(memory): deprecate the never-wired agent memory writers (#4101)
## Description

`headroom/memory/writers/` exports memories to agent files (CLAUDE.md,
`.cursor/rules`, AGENTS.md, generic markdown). It arrived in 0fd6dfca,
and no product code has ever imported it. `headroom learn` writes agent
context files through `headroom.learn.writer`, a separate writer. The
package lists its classes in `__all__`, so it is public API. This PR
deprecates it instead of deleting it. Importing
`headroom.memory.writers` now emits a `DeprecationWarning`.

**What to use instead:** there is no replacement. For writing agent
context files, `headroom learn` (`headroom.learn.writer`) is the
supported path.

Evidence:
- `grep -rnE 'memory\.writers|MemoryWriter|AgentWriter|ExportResult'`
finds only the package itself, `memory/budget.py` (removed in a separate
PR) and tests. The search covered `headroom crates plugins sdk docs wiki
scripts benchmarks examples e2e .github pyproject.toml` and the consumer
repos.
- The package is not re-exported from `headroom.memory` and not
documented.

Refs #4136

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `headroom/memory/writers/__init__.py`:
- Emit `DeprecationWarning("headroom.memory.writers is deprecated and
will be removed in a future release; it has no replacement …")` when the
package is imported. Importing any submodule, such as
`headroom.memory.writers.base`, runs the package first, so that warns
too.
  - Add `.. deprecated::` to the docstring.
- `tests/test_memory/test_writers.py`: add one test that asserts the
warning fires on import. The existing writer tests stay.
- Unchanged: `import headroom.memory`, `headroom.memory.sync` and the
sync adapters stay quiet. Checked with `-W error::DeprecationWarning`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q -p no:cacheprovider tests/test_memory/test_writers.py tests/test_managed_block_injection.py tests/test_memory/test_budget.py tests/test_memory_sync.py
======================== 92 passed, 1 warning in 1.21s =========================
$ python -W error::DeprecationWarning -c "import headroom.memory, headroom.memory.sync, headroom.memory.sync_adapters.claude_code; print('quiet')"
quiet
$ uvx ruff@0.16.8 check headroom/memory/writers/__init__.py tests/test_memory/test_writers.py
All checks passed!
```

## Real Behavior Proof

- Environment: macOS, Python 3.12.13, repo .venv, branch checkout at
58e5702d (base a9815bbb), temp HOME / HEADROOM_WORKSPACE_DIR /
CODEX_HOME, offline.
- Exact command / steps:
- Ran the shared smoke harness, which starts a real proxy from this
checkout against a local fake upstream and sends one Anthropic and one
OpenAI request with a compressible tool result.
- Ran `python -W always`. It imports `headroom.memory` (the live path),
then imports a writer from the deprecated package directly.
  - Ran the full `tests/test_memory` suite.
- Observed result:
- Both proxy requests return HTTP 200 with savings, and the proxy log
has no deprecation warning.
- `headroom.memory` imports quietly. The direct writers import emits the
new warning, and the writer still loads.
  - 615 passed.
- Not tested: No runtime path imports the package.
`headroom/memory/budget.py` imports it, and #4099 deletes that file.

```text
$ bash smoke.sh <worktree>
branch: cleanup/memory-cache-memory-writers @ 58e5702d
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines:
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you 
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.

$ HOME=$TMP/home HEADROOM_WORKSPACE_DIR=$TMP/ws CODEX_HOME=$TMP/codex python -u -W always -c "import headroom.memory; print(...); from headroom.memory.writers import GenericMemoryWriter; print(...)"
headroom.memory imported
<string>:4: DeprecationWarning: headroom.memory.writers is deprecated and will be removed in a future release; it has no replacement (headroom learn writes agent context files through headroom.learn.writer).
writer: GenericMemoryWriter
```

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A; deprecation notice only.
- Stable/default behavior changed: No. The writers behave as before.
Importing them additionally emits a DeprecationWarning, which Python
hides by default outside `__main__` and tests.
- Kill switch / disable path: Standard `warnings` filters.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A.

## Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.

- The audit first proposed deleting the package. Under the
deprecate-first rule for public API, it is deprecated here. Deletion can
follow in a later release.
- The unused budget manager (`memory/budget.py`), which depends on this
package, is removed in a separate PR.
- "My changes generate no new warnings" refers to code that is used. The
only new warning is the intended DeprecationWarning.
- mypy was not run, because no types changed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:03:14 -07:00

428 lines
16 KiB
Python

"""Tests for agent-native memory writers."""
from __future__ import annotations
import asyncio
import importlib
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
from types import SimpleNamespace
import pytest
from headroom.memory.writers.base import (
MARKER_END,
MARKER_START,
MemoryEntry,
_estimate_tokens,
_merge_section,
)
from headroom.memory.writers.claude_writer import ClaudeCodeMemoryWriter
from headroom.memory.writers.codex_writer import CodexMemoryWriter
from headroom.memory.writers.cursor_writer import CursorMemoryWriter
from headroom.memory.writers.generic_writer import GenericMemoryWriter
def test_importing_the_writers_package_warns_deprecated(monkeypatch: pytest.MonkeyPatch) -> None:
# Restore both the sys.modules entry and the parent attribute afterwards, so
# later tests see the original package object whatever the import form.
import headroom.memory
monkeypatch.delitem(sys.modules, "headroom.memory.writers", raising=False)
monkeypatch.delattr(headroom.memory, "writers", raising=False)
with pytest.warns(DeprecationWarning, match="headroom.memory.writers is deprecated"):
importlib.import_module("headroom.memory.writers")
# =============================================================================
# Test Data
# =============================================================================
def _make_entries(count: int = 5) -> list[MemoryEntry]:
"""Create test memory entries."""
entries = []
categories = ["error_recovery", "environment", "preference", "architecture"]
for i in range(count):
entries.append(
MemoryEntry(
content=f"Test memory entry {i}: use pytest not unittest",
importance=0.5 + (i % 3) * 0.15,
category=categories[i % len(categories)],
entity_refs=[f"/path/to/file{i}.py"],
created_at=time.time() - i * 3600, # Each an hour older
access_count=max(0, 3 - i),
)
)
return entries
# =============================================================================
# Base / Shared Tests
# =============================================================================
class TestMemoryEntry:
def test_score_calculation(self):
entry = MemoryEntry(
content="test",
importance=0.9,
created_at=time.time(), # Just now
access_count=5,
)
assert entry.score > 0.5 # High importance + recent + accessed
def test_score_decays_with_age(self):
recent = MemoryEntry(content="new", importance=0.9, created_at=time.time())
old = MemoryEntry(content="old", importance=0.9, created_at=time.time() - 30 * 86400)
assert recent.score > old.score
def test_score_handles_future_timestamp(self):
# A future created_at (clock skew, a timestamp written on another
# machine) drove recency = 1/(1 + age_days*0.1) to a divide-by-zero at
# exactly +10 days and negative past that, inverting the ranking. Age is
# now clamped to >= 0, so a future memory scores like a brand-new one.
now = time.time()
boom = MemoryEntry(content="future", importance=0.9, created_at=now + 10 * 86400)
far_future = MemoryEntry(content="far", importance=0.9, created_at=now + 30 * 86400)
fresh = MemoryEntry(content="fresh", importance=0.9, created_at=now)
assert boom.score > 0 # no ZeroDivisionError at exactly +10 days
assert far_future.score > 0 # not negative past +10 days
# Any future timestamp clamps to age 0, so all future memories score
# identically to a brand-new one (never better than a real fresh one).
assert far_future.score == boom.score
assert far_future.score >= fresh.score
def test_content_hash(self):
e1 = MemoryEntry(content="same content")
e2 = MemoryEntry(content="same content")
assert e1.content_hash == e2.content_hash
def test_different_hash(self):
e1 = MemoryEntry(content="content A")
e2 = MemoryEntry(content="content B")
assert e1.content_hash != e2.content_hash
class TestTokenEstimate:
def test_rough_estimate(self):
assert _estimate_tokens("hello world") > 0
assert _estimate_tokens("a" * 400) == 100 # ~4 chars per token
class TestMergeSection:
def test_new_file(self, tmp_path: Path):
nonexistent = tmp_path / "new.md"
result = _merge_section(nonexistent, "new section")
assert result == "new section\n"
def test_append_to_existing(self, tmp_path: Path):
existing = tmp_path / "existing.md"
existing.write_text("# Existing Content\n\nSome stuff here.")
result = _merge_section(existing, "new section")
assert "# Existing Content" in result
assert "new section" in result
def test_replace_existing_markers(self, tmp_path: Path):
existing = tmp_path / "marked.md"
existing.write_text(f"# Header\n\n{MARKER_START}\nold content\n{MARKER_END}\n\n# Footer")
result = _merge_section(existing, f"{MARKER_START}\nnew content\n{MARKER_END}")
assert "new content" in result
assert "old content" not in result
assert "# Header" in result
assert "# Footer" in result
def test_replace_existing_markers_handles_literal_backslashes(self, tmp_path: Path):
existing = tmp_path / "marked.md"
existing.write_text(f"# Header\n\n{MARKER_START}\nold content\n{MARKER_END}\n")
section = f"{MARKER_START}\n- Keep C:\\Users\\john.doe\\repo and literal \\u\n{MARKER_END}"
result = _merge_section(existing, section)
assert r"C:\Users\john.doe\repo" in result
assert r"literal \u" in result
assert "old content" not in result
# =============================================================================
# Claude Code Writer Tests
# =============================================================================
class TestClaudeCodeWriter:
def test_format_memories(self):
writer = ClaudeCodeMemoryWriter()
entries = _make_entries(3)
formatted = writer.format_memories(entries)
assert "## Headroom Learned Context" in formatted
assert "Auto-maintained by Headroom" in formatted
assert "Test memory entry" in formatted
def test_export_dry_run(self, tmp_path: Path):
writer = ClaudeCodeMemoryWriter(
project_path=tmp_path,
memory_dir=tmp_path / "memory",
)
entries = _make_entries(3)
result = writer.export(entries, dry_run=True)
assert result.dry_run is True
assert result.memories_exported == 3
assert len(result.files_written) == 1
assert not (tmp_path / "memory" / "MEMORY.md").exists()
def test_export_writes_file(self, tmp_path: Path):
writer = ClaudeCodeMemoryWriter(
project_path=tmp_path,
memory_dir=tmp_path / "memory",
)
entries = _make_entries(3)
result = writer.export(entries, dry_run=False)
assert result.memories_exported == 3
written = (tmp_path / "memory" / "MEMORY.md").read_text()
assert MARKER_START in written
assert MARKER_END in written
assert "Test memory entry" in written
def test_budget_limits_output(self, tmp_path: Path):
writer = ClaudeCodeMemoryWriter(
project_path=tmp_path,
memory_dir=tmp_path / "memory",
token_budget=50, # Very small budget
)
entries = _make_entries(10)
result = writer.export(entries, dry_run=True)
assert result.memories_skipped_budget > 0
assert result.memories_exported < 10
def test_dedup(self, tmp_path: Path):
writer = ClaudeCodeMemoryWriter(
project_path=tmp_path,
memory_dir=tmp_path / "memory",
)
entries = [
MemoryEntry(content="duplicate content", importance=0.5),
MemoryEntry(content="duplicate content", importance=0.8), # Same content
MemoryEntry(content="unique content", importance=0.6),
]
result = writer.export(entries, dry_run=True)
assert result.memories_skipped_dedup == 1
assert result.memories_exported == 2
def test_export_topics(self, tmp_path: Path):
writer = ClaudeCodeMemoryWriter(
project_path=tmp_path,
memory_dir=tmp_path / "memory",
)
entries = _make_entries(6)
topics = writer.export_topics(entries, dry_run=True)
# Should produce topic files for categories with 2+ entries
assert len(topics) > 0
for filename, content in topics.items():
assert filename.startswith("headroom_")
assert "---" in content # YAML frontmatter
def test_default_path_encodes_windows_user_with_dot(self):
writer = ClaudeCodeMemoryWriter(project_path=Path(r"C:\Users\john.doe\work"))
rendered = str(writer.default_path())
assert "-C-Users-john.doe-work" in rendered
assert "john-doe" not in rendered
assert rendered.endswith("MEMORY.md")
# =============================================================================
# Cursor Writer Tests
# =============================================================================
class TestCursorWriter:
def test_format_memories(self):
writer = CursorMemoryWriter()
entries = _make_entries(3)
formatted = writer.format_memories(entries)
assert "##" in formatted
assert "Test memory entry" in formatted
def test_creates_mdc_with_frontmatter(self, tmp_path: Path):
writer = CursorMemoryWriter(project_path=tmp_path)
entries = _make_entries(3)
writer.export(entries, dry_run=False)
mdc_path = tmp_path / ".cursor" / "rules" / "headroom-memory.mdc"
assert mdc_path.exists()
content = mdc_path.read_text()
assert "---" in content
assert "alwaysApply: true" in content
assert "description:" in content
assert MARKER_START in content
def test_updates_existing_mdc(self, tmp_path: Path):
mdc_dir = tmp_path / ".cursor" / "rules"
mdc_dir.mkdir(parents=True)
mdc_file = mdc_dir / "headroom-memory.mdc"
mdc_file.write_text(
"---\ndescription: test\nalwaysApply: true\n---\n\n"
f"# Header\n\n{MARKER_START}\nold stuff\n{MARKER_END}\n"
)
writer = CursorMemoryWriter(project_path=tmp_path)
entries = _make_entries(2)
writer.export(entries, dry_run=False)
content = mdc_file.read_text()
assert "old stuff" not in content
assert "Test memory entry" in content
assert "alwaysApply: true" in content # Preserved
# =============================================================================
# Codex Writer Tests
# =============================================================================
class TestCodexWriter:
def test_format_memories(self):
writer = CodexMemoryWriter()
entries = _make_entries(3)
formatted = writer.format_memories(entries)
assert "## Headroom Learned Context" in formatted
def test_default_path(self, tmp_path: Path):
writer = CodexMemoryWriter(project_path=tmp_path)
assert writer.default_path() == tmp_path / "AGENTS.md"
def test_export(self, tmp_path: Path):
writer = CodexMemoryWriter(project_path=tmp_path)
entries = _make_entries(3)
writer.export(entries, dry_run=False)
agents_md = tmp_path / "AGENTS.md"
assert agents_md.exists()
content = agents_md.read_text()
assert MARKER_START in content
# =============================================================================
# Generic Writer Tests
# =============================================================================
class TestGenericWriter:
def test_custom_filename(self, tmp_path: Path):
writer = GenericMemoryWriter(project_path=tmp_path, filename="GEMINI.md")
assert writer.default_path() == tmp_path / "GEMINI.md"
def test_export(self, tmp_path: Path):
writer = GenericMemoryWriter(project_path=tmp_path)
entries = _make_entries(3)
result = writer.export(entries, dry_run=False)
assert (tmp_path / "HEADROOM_MEMORY.md").exists()
assert result.memories_exported == 3
# =============================================================================
# LF newline contract (#3698)
# =============================================================================
@pytest.mark.windows_newline
class TestNewlineContract:
"""Every memory write pins ``newline="\n"`` — see #3698.
These files overlap with the learn writers (``CLAUDE.md``, ``AGENTS.md``,
``MEMORY.md``), so an unpinned write here flips a file the learn writers
just wrote as LF back to CRLF on Windows, producing spurious whole-file
diffs in any repo that commits them.
Asserted at the call, not the artifact: ``TextIOWrapper`` picks its
translation target at C-compile time (``#ifdef MS_WINDOWS``), so on POSIX
an unpinned write is indistinguishable from a pinned one on disk.
"""
@staticmethod
def _spy(monkeypatch: pytest.MonkeyPatch) -> list[tuple[Path, str | None]]:
calls: list[tuple[Path, str | None]] = []
original = Path.write_text
def spy(self, data, encoding=None, errors=None, newline=None):
calls.append((self, newline))
return original(self, data, encoding=encoding, errors=errors, newline=newline)
monkeypatch.setattr(Path, "write_text", spy)
return calls
@staticmethod
def _assert_all_lf(calls: list[tuple[Path, str | None]]) -> None:
assert calls, "no writes captured — this test no longer drives the writers"
unpinned = sorted(str(path) for path, newline in calls if newline != "\n")
assert not unpinned, f"memory writers wrote without newline='\\n': {unpinned}"
def test_memory_writers_pin_lf(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
"""base.py, cursor_writer.py and claude_writer.py write sites."""
entries = _make_entries(6)
calls = self._spy(monkeypatch)
# base.MemoryWriter.export — shared by the codex/generic writers.
CodexMemoryWriter(project_path=tmp_path).export(entries, dry_run=False)
GenericMemoryWriter(project_path=tmp_path).export(entries, dry_run=False)
# cursor_writer.export overrides base.export with its own write.
CursorMemoryWriter(project_path=tmp_path).export(entries, dry_run=False)
# claude_writer.export_topics writes per-category topic files.
claude = ClaudeCodeMemoryWriter(project_path=tmp_path, memory_dir=tmp_path / "memory")
claude.export(entries, dry_run=False)
claude.export_topics(entries, dry_run=False)
self._assert_all_lf(calls)
names = {path.name for path, _ in calls}
assert {"AGENTS.md", "HEADROOM_MEMORY.md", "headroom-memory.mdc", "MEMORY.md"} <= names
assert any(name.startswith("headroom_") for name in names), names
def test_bridge_writes_pin_lf(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
"""bridge.py write sites: markdown export, markdown append, sync state."""
from headroom.memory.bridge import MemoryBridge
from headroom.memory.bridge_config import BridgeConfig
memories = [
SimpleNamespace(
content="use uv, not pip",
metadata={"topic": "Environment"},
entity_refs=[],
importance=0.8,
created_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
]
async def get_user_memories(user_id, limit=200):
return list(memories)
config = BridgeConfig(sync_state_path=tmp_path / "state" / "bridge_state.json")
bridge = MemoryBridge(
config=config, backend=SimpleNamespace(get_user_memories=get_user_memories)
)
target = tmp_path / "CLAUDE.md"
target.write_text("# Notes\n\n## Environment\n- existing\n", encoding="utf-8")
calls = self._spy(monkeypatch)
# export_to_markdown covers the export write *and* the sync-state write.
asyncio.run(bridge.export_to_markdown(path=tmp_path / "export" / "MEMORY.md"))
asyncio.run(bridge._append_to_markdown(target, memories))
self._assert_all_lf(calls)
assert {path.name for path, _ in calls} == {
"MEMORY.md", # export_to_markdown
"bridge_state.json", # _save_sync_state
"CLAUDE.md", # _append_to_markdown
}