perf(memory/budget): precompute word sets once in _merge_similar (#3275)

## Description

`MemoryBudgetManager._merge_similar` collapses near-duplicate memories
with an O(n^2) pairwise Jaccard scan. But `_text_similarity` rebuilt the
word set for **both** sides on every comparison:

```python
for i, m1 in enumerate(memories):
    for j, m2 in enumerate(memories[i + 1:], start=i + 1):
        if self._text_similarity(m1.content, m2.content) > threshold:  # re-splits both sides
            ...

@staticmethod
def _text_similarity(a, b):
    words_a = set(a.lower().split())   # m1.content re-tokenized on every inner j
    words_b = set(b.lower().split())
    ...
```

So each memory's content was `lower().split()` into a set O(n) times per
optimization pass. The pairwise structure is inherent to the greedy
grouping, but the re-tokenization is pure waste.

This tokenizes each memory's word set **once** up front and compares the
cached sets. `_text_similarity` now delegates to a module-level
`_jaccard(set_a, set_b)` helper, and the Jaccard skips materializing the
union set (`|A| + |B| - |A ∩ B|`). Results are unchanged — the merged
output is identical to the original per-pair scan.

Benchmark (`_merge_similar`, 250 candidate memories of ~80 words each,
mean of 10 passes):

```
before : 662.8 ms/pass
after  :  57.4 ms/pass   (~11.5x faster)
```

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/memory/budget.py`: added a module-level `_jaccard(words_a,
words_b)` helper. `_merge_similar` precomputes `word_sets =
[set(m.content.lower().split()) for m in memories]` once and compares
cached sets via `_jaccard`. `_text_similarity` now delegates to
`_jaccard`, so its behavior (including the empty-input -> 0.0 guard) is
unchanged.
- `tests/test_memory/test_budget.py`: added
`test_merge_groups_transitively_like_pairwise_scan` (three
identical-content entries collapse to the highest-importance
representative; an unrelated entry survives) and
`test_text_similarity_matches_explicit_jaccard` (value equals an
explicit Jaccard; empty side yields 0.0, not a ZeroDivisionError).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality

### Test Output

```text
tests/test_memory/test_budget.py  ->  13 passed
uvx ruff@0.16.2 check headroom/memory/budget.py tests/test_memory/test_budget.py  ->  All checks passed!
uvx mypy@1.20.2 headroom/memory/budget.py  ->  Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.16.2 and mypy 1.20.2 via uvx.
- Exact command / steps: (1) checked `_text_similarity` equals the
original two-set formula over 1000 random string pairs; (2) ran
`_merge_similar` against a reference implementation using the original
per-pair `_text_similarity` on 120 memories with real content overlap
and confirmed byte-identical merge output (same surviving-entry
identities); (3) benchmarked `_merge_similar` on 250 memories at 662.8ms
before vs 57.4ms after; (4) ran the full
`tests/test_memory/test_budget.py` suite.
- Observed result: identical merge results (same entries merged, same
highest-importance representative kept, same entity-ref/access-count
aggregation) with each memory tokenized once instead of O(n) times,
cutting the merge step ~11x on a 250-memory batch.
- Not tested: end-to-end optimize() against a live memory backend (this
exercises `_merge_similar` directly and through `optimize`, which the
existing suite already covers).

## Runtime Rollout Safety

- Rollout-managed feature(s): none — no feature flag or rollout channel
involved.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: no. Merge output is identical; only
redundant re-tokenization is removed.
- Kill switch / disable path: N/A (no config surface added).
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this commit; `_merge_similar` goes back to
re-tokenizing per comparison.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal behavior, merge output unchanged)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

The `_jaccard` helper is deliberately module-level so the same
tokenize-once pattern is reusable, and `_text_similarity` stays as a
thin public wrapper for callers/tests that pass raw strings.
This commit is contained in:
Abhay Singh
2026-09-25 00:01:16 -05:00
committed by GitHub
parent c81378c878
commit 3aa5012857
2 changed files with 55 additions and 13 deletions
+26 -13
View File
@@ -25,6 +25,19 @@ from headroom.memory.writers.base import MemoryEntry, _estimate_tokens
logger = logging.getLogger(__name__)
def _jaccard(words_a: set[str], words_b: set[str]) -> float:
"""Jaccard similarity between two word sets.
Operates on already-tokenized sets so callers scanning many pairs can
tokenize each document once instead of per comparison.
"""
if not words_a or not words_b:
return 0.0
intersection = len(words_a & words_b)
union = len(words_a) + len(words_b) - intersection
return intersection / union
@dataclass
class BudgetConfig:
"""Configuration for memory budget management."""
@@ -244,6 +257,13 @@ class MemoryBudgetManager:
if len(memories) <= 1:
return memories
# Precompute each memory's word set once. The pairwise scan below is
# O(n^2); recomputing set(content.lower().split()) inside every
# _text_similarity call re-tokenized the same content O(n) times. With
# the sets cached, each pair is a plain set intersection/union.
word_sets = [set(m.content.lower().split()) for m in memories]
threshold = self._config.similarity_merge_threshold
merged: list[MemoryEntry] = []
used: set[int] = set()
@@ -251,16 +271,15 @@ class MemoryBudgetManager:
if i in used:
continue
words_i = word_sets[i]
# Find similar memories
group = [m1]
for j, m2 in enumerate(memories[i + 1 :], start=i + 1):
for j in range(i + 1, len(memories)):
if j in used:
continue
if (
self._text_similarity(m1.content, m2.content)
> self._config.similarity_merge_threshold
):
group.append(m2)
if _jaccard(words_i, word_sets[j]) > threshold:
group.append(memories[j])
used.add(j)
if len(group) == 1:
@@ -280,10 +299,4 @@ class MemoryBudgetManager:
@staticmethod
def _text_similarity(a: str, b: str) -> float:
"""Simple Jaccard similarity on word sets."""
words_a = set(a.lower().split())
words_b = set(b.lower().split())
if not words_a or not words_b:
return 0.0
intersection = words_a & words_b
union = words_a | words_b
return len(intersection) / len(union)
return _jaccard(set(a.lower().split()), set(b.lower().split()))
+29
View File
@@ -105,6 +105,35 @@ class TestBudgetManager:
assert report.merged >= 1
assert report.kept <= 2
def test_merge_groups_transitively_like_pairwise_scan(self, manager: MemoryBudgetManager):
"""The precomputed word-set scan must merge exactly the pairs the
original per-pair Jaccard scan did: high-overlap entries collapse to
the highest-importance representative, distinct entries survive."""
shared = "alpha beta gamma delta epsilon zeta eta theta iota kappa"
entries = [
_make_entry(shared, importance=0.3),
_make_entry(shared, importance=0.9), # highest -> kept representative
_make_entry(shared, importance=0.5),
_make_entry("wholly unrelated content about something else", importance=0.6),
]
merged = manager._merge_similar(list(entries))
# Three identical-content entries collapse to one; the unrelated one stays.
assert len(merged) == 2
kept_shared = [m for m in merged if m.content == shared]
assert len(kept_shared) == 1
# The surviving representative is the highest-importance of the group.
assert kept_shared[0].importance == 0.9
def test_text_similarity_matches_explicit_jaccard(self, manager: MemoryBudgetManager):
a = "the quick brown fox jumps"
b = "the quick brown dog runs"
wa, wb = set(a.split()), set(b.split())
expected = len(wa & wb) / len(wa | wb)
assert manager._text_similarity(a, b) == pytest.approx(expected)
# Empty side yields 0.0, not a ZeroDivisionError.
assert manager._text_similarity("", "anything") == 0.0
def test_very_old_pruned(self, manager: MemoryBudgetManager):
"""Test that very old, low-importance memories get pruned by decay."""
entries = [