fix(memory): replace the HNSW vector when a memory is re-indexed (#4649)

## Description

Item 5 of #4362. `HierarchicalMemory.update`, `refresh` and `detach`
re-index a memory under the same id after its content, and so its
embedding, changed (`headroom/memory/core.py:493-504`, `:520-522`,
`:626`). On the HNSW backend that re-index never reached the graph:
`HNSWVectorIndex.index()` on an existing id went to
`_update_embedding_internal`, which only stored the new vector in the
`_embeddings` cache, and the update branch of `index_batch()` and
`update_embedding()` did the same (the docstrings said so). Search kept
matching the memory by its old content.

hnswlib's `add_items` with a label that already exists replaces that
element's vector in place: `addPointWithLevel` looks the label up and
calls `updatePoint` instead of adding a new element (nmslib/hnswlib
`hnswlib/hnswalg.h:1282-1305`, v0.9.0). All three paths now call it with
the memory's existing label. Ids, the id mappings and `size` are
unchanged.

Refs #4362

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/memory/adapters/hnsw.py`:
- `_update_embedding_internal` calls `self._index.add_items(embedding,
[existing_label])` before updating the cache (covers `index()` on an
existing id and `update_embedding()`); docstring rewritten.
- `index_batch()`: the update branch writes all updated vectors with one
`add_items` call on their existing labels. Updates are first collapsed
by id, keeping the last entry, so a batch that names an id twice ends
with the graph, the cached embedding and the metadata on the same vector
(review round 2).
- `update_embedding()` docstring no longer says the graph keeps the
original vector.
- `tests/test_memory/test_hnsw_update_replaces_vector.py` (new, skipped
when hnswlib is absent, like the existing HNSW tests): for each of the
three paths, a memory moved from vector A to vector B is the top hit for
B with similarity 1.0, and `size` stays 2. Round 2 adds
`test_index_batch_repeated_id_ends_on_the_last_vector`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) (run on
`headroom/memory/adapters/hnsw.py` only, see below)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# fail before (source at e4a60ed7, new test kept, hnswlib installed)
FAILED ...::test_index_on_existing_id_replaces_the_vector        (assert 'm2' == 'm1')
FAILED ...::test_update_embedding_replaces_the_vector
FAILED ...::test_index_batch_on_existing_id_replaces_the_vector
3 failed

# pass after (head 9668df0f): new test + tests/test_memory/test_hnsw_batch_capacity.py + tests/test_hnsw_only.py
12 passed

# round 2: the repeated-id test against the round-1 head 454c1c10, 20 runs
20 passed   (the out-of-order update did not reproduce here; the dedupe is a guard,
             not a fail-before fix)

# tests/test_memory + test_hnsw_only.py + test_memory_system.py + test_private_state_files.py
#   with hnswlib:    e4a60ed7 87 failed / 797 passed;  this branch 84 failed / 800 passed
#                    the 84 are the same on both sides: "sentence-transformers is required
#                    for LocalEmbedder" (not installed here); the 3 extra on e4a60ed7 are the new tests
#   without hnswlib: this branch 716 passed, 177 skipped

# ruff check / ruff format --check: All checks passed! / 2 files already formatted
# mypy headroom/memory/adapters/hnsw.py: Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Ubuntu (WSL2) on Windows 11, Python 3.12.15, `headroom`
Python source at `e4a60ed7` and at this branch, offline. hnswlib: the
`chroma-hnswlib` 0.7.6 binary wheel (it installs the `hnswlib` module;
this machine has no C++ toolchain to build `hnswlib>=0.8.0` from
source). The re-add-replaces behaviour it shows is the same code path as
upstream v0.9.0 cited above.
- Exact command / steps: a real `HierarchicalMemory(SQLiteMemoryStore,
HNSWVectorIndex, FTS5TextIndex, embedder)` on a temporary database, with
a deterministic offline embedder that maps each phrase to its own unit
vector. Add `"user prefers tabs"`, `"deploys on fridays"`, `"uses
postgres"`; `update(<first id>, content="user prefers spaces")`; then
`search(query, top_k=1)` for the new and the old phrase.
- Observed result: on `e4a60ed7` the updated memory matches its old
content at similarity 1.000 and its new content at 0.000; on this branch
it is the other way round (table below).

  | query | `e4a60ed7` | this branch |
  |---|---|---|
| `"user prefers spaces"` (new content) | similarity 0.000 | similarity
1.000 |
| `"user prefers tabs"` (old content) | similarity 1.000 | similarity
0.000 |

On `e4a60ed7` the updated memory still matches its old content exactly
and does not match its new content.
- Not tested: `hnswlib` built from source at 0.8.x/0.9.x (only the
chroma wheel ran here), a real embedding model, and `mypy headroom` as a
whole. Persistence is unchanged: `save_index` writes the graph as
before, now with the replaced vector.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A
- Stable/default behavior changed: Yes, on the opt-in HNSW backend only
(`VectorBackend.HNSW`, the `[vector]` extra): an updated memory is
searched by its new embedding. The default sqlite-vec backend is not
touched.
- Kill switch / disable path: None added; revert.
- Unsafe override required: No
- Qualification impact: None expected
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

One logical change per PR, as CONTRIBUTING asks: this covers item 5 of
#4362 only (item 7 is #4647, item 10 is #4648). It does not touch item 4
(loading a persisted HNSW index at startup, #3308).

Produced with AI assistance (Claude Code) operated by me (HetCreep); the
reproduction was run on my machine against a clone of the repository.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


### Maintainer qualification against current main

Qualified source head `0e798693966d65f3afc445b8f943e36db86e4471` against
upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 22 focused
tests passed with native hnswlib 0.8.0. An independent
HierarchicalMemory/SQLite/HNSW/FTS runtime with deterministic embeddings
changed an existing memory from tabs to spaces: new-vector similarity
1.0, old-vector similarity 0.0; saving and loading into a fresh index
preserved 1.0. Actual main retained the old vector (new 0.0, old 1.0,
reload new 0.0). Three labels and index size remained unchanged.
Duplicate batch IDs retain the last vector; this is a regression guard,
not a claimed reproduced race.

The maintainer cherry-picked only the shared Gemini test-fixture repair
from #4644: real Starlette Request with two ASGI receive chunks,
preserving the existing test assertions. This addresses the same four
independently verified failures on main; contributor production logic is
unchanged. Ruff, formatting, and whitespace checks passed. Exact-head
hosted CI is being checked before formal approval.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
HetCreep
2026-11-05 13:05:00 -06:00
committed by GitHub
co-authored by JerrettDavis Claude Opus 5.5
parent 917edc6223
commit ee638f9532
2 changed files with 124 additions and 11 deletions
+17 -11
View File
@@ -494,11 +494,19 @@ class HNSWVectorIndex:
self._metadata[memory.id] = IndexedMemoryMetadata.from_memory(memory)
self._embeddings[memory.id] = embedding.copy()
# Handle updates (hnswlib doesn't support true updates, so we track separately)
# Updates: add_items with an existing label replaces its vector. A batch
# can name the same id more than once; keep only the last entry so the
# graph, the cached embedding and the metadata end on the same vector
# (hnswlib may apply rows for one label out of order across threads).
update_memories = list({m.id: (m, emb) for m, emb in update_memories}.values())
if update_memories:
self._index.add_items(
np.vstack([emb for _, emb in update_memories]).astype(np.float32),
np.array([self._memory_to_hnsw[m.id] for m, _ in update_memories]),
)
for memory, embedding in update_memories:
self._metadata[memory.id] = IndexedMemoryMetadata.from_memory(memory)
self._embeddings[memory.id] = embedding.copy()
# Note: HNSW embedding stays unchanged unless we remove and re-add
if self._auto_save and self._save_path:
self.save_index(self._save_path)
@@ -748,9 +756,7 @@ class HNSWVectorIndex:
async def update_embedding(self, memory_id: str, embedding: np.ndarray) -> bool:
"""Update the embedding for an indexed memory.
Note: hnswlib doesn't support in-place updates. This stores the new
embedding but the HNSW graph uses the original. For full update,
remove and re-index the memory.
The new vector replaces the old one in the HNSW graph, so search uses it.
Args:
memory_id: The unique identifier of the memory.
@@ -777,15 +783,15 @@ class HNSWVectorIndex:
async def _update_embedding_internal(self, memory_id: str, embedding: np.ndarray) -> bool:
"""Internal embedding update without lock (caller must hold lock).
hnswlib doesn't support true in-place updates, so we:
1. Store the new embedding in our local cache
2. The HNSW index continues to use the old embedding for search
For a true update, the caller should remove and re-index.
hnswlib's ``add_items`` with an existing label replaces that element's
vector in place (``addPoint`` -> ``updatePoint``), so the label and the
id mappings stay as they are and search sees the new vector.
"""
if memory_id not in self._memory_to_hnsw:
hnsw_id = self._memory_to_hnsw.get(memory_id)
if hnsw_id is None:
return False
self._index.add_items(embedding.reshape(1, -1), np.array([hnsw_id]))
self._embeddings[memory_id] = embedding.copy()
return True
@@ -0,0 +1,107 @@
"""Re-indexing a memory on the HNSW backend must move it to its new vector.
`HierarchicalMemory.update`, `refresh` and `detach` re-index a memory under the
same id after its content (and so its embedding) changed. The HNSW adapter only
cached the new embedding; the graph kept the old vector, so search still found
the memory by its OLD content and not by the new one.
See https://github.com/headroomlabs-ai/headroom/issues/4362 (item 5).
"""
from __future__ import annotations
import numpy as np
import pytest
from headroom.memory.models import Memory
from headroom.memory.ports import VectorFilter
try:
from headroom.memory.adapters.hnsw import _check_hnswlib_available
HNSW_AVAILABLE = _check_hnswlib_available()
except ImportError:
HNSW_AVAILABLE = False
pytestmark = [
pytest.mark.skipif(not HNSW_AVAILABLE, reason="hnswlib not installed"),
pytest.mark.asyncio,
]
DIM = 8
OLD = np.eye(DIM, dtype=np.float32)[0]
NEW = np.eye(DIM, dtype=np.float32)[1]
def _index():
from headroom.memory.adapters.hnsw import HNSWVectorIndex
return HNSWVectorIndex(dimension=DIM, max_elements=16)
async def _top(index, vector: np.ndarray) -> tuple[str, float]:
results = await index.search(VectorFilter(query_vector=vector, top_k=1))
return results[0].memory.id, results[0].similarity
async def _seed(index) -> Memory:
memory = Memory(id="m1", content="old", user_id="u", embedding=OLD.copy())
await index.index(memory)
# A second memory so a stale vector cannot win by being the only entry.
await index.index(
Memory(id="m2", content="other", user_id="u", embedding=np.ones(DIM, np.float32))
)
return memory
async def test_index_on_existing_id_replaces_the_vector() -> None:
index = _index()
memory = await _seed(index)
memory.embedding = NEW.copy()
await index.index(memory)
assert index.size == 2
found, similarity = await _top(index, NEW)
assert found == "m1"
assert similarity == pytest.approx(1.0, abs=1e-5)
async def test_update_embedding_replaces_the_vector() -> None:
index = _index()
await _seed(index)
assert await index.update_embedding("m1", NEW.copy()) is True
found, similarity = await _top(index, NEW)
assert found == "m1"
assert similarity == pytest.approx(1.0, abs=1e-5)
async def test_index_batch_on_existing_id_replaces_the_vector() -> None:
index = _index()
memory = await _seed(index)
memory.embedding = NEW.copy()
assert await index.index_batch([memory]) == 1
assert index.size == 2
found, similarity = await _top(index, NEW)
assert found == "m1"
assert similarity == pytest.approx(1.0, abs=1e-5)
async def test_index_batch_repeated_id_ends_on_the_last_vector() -> None:
index = _index()
await _seed(index)
first = Memory(
id="m1", content="first", user_id="u", embedding=np.eye(DIM, dtype=np.float32)[2]
)
last = Memory(id="m1", content="last", user_id="u", embedding=NEW.copy())
await index.index_batch([first, last])
assert index.size == 2
found, similarity = await _top(index, NEW)
assert found == "m1"
assert similarity == pytest.approx(1.0, abs=1e-5)
assert np.array_equal(index._embeddings["m1"], NEW)