mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
fix/plugin-launcher-isolated-python
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
de1a1e0f47 |
deps: update sqlite-vec requirement from >=0.1.6 to >=0.1.9 (#4654)
Updates the requirements on [sqlite-vec](https://TODO.com) to permit the latest version. Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4654" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ### Maintainer current-main requalification Exact head `45ff018fcda1368f0e1b3d8cf84f97203dd993d8`, upstream main `f50db67006e2fe3f43b295542b759d90bb73e5a1`. Maintainer remediation under the user's explicit directive to maintain Dependabot updates and qualify them for approval. This records an authorized routine-maintenance exception rather than inventing a newly required vector feature or claiming the old range excluded0.1.9. sqlite-vec remains in the existing optional memory extras; the base install gains no new dependency, service or network behavior. Native extension/wheel compatibility is the relevant install surface. Synchronized current main and repaired the selected sqlite-vec lock entry plus all three declared floor records to0.1.9, retaining unrelated lock metadata. Offline lock consistency and whitespace checks pass. Installed the actual0.1.9 Python3.12 native wheel in an isolated overlay and verified its module path/version. All45 vector-index/local-search/core-search/relation-key tests pass, exercising real extension loading, indexing/querying, filters, updates/deletes and persistence. The initial Windows suite exposed pre-existing fixture teardown errors: indexes cached connections but fixtures unlinked the open databases. A clean-main control reproduced that error. Fixed the test-only lifecycle to await close before unlink, and strengthened the persistence test to close the first index before reopening the database. Production vector/runtime logic and assertions are unchanged. Re-execution passes all45 with no cleanup errors. Existing Headroom native core supports startup; this does not claim a rebuilt sqlite-vec wheel or fresh Headroom Rust extension build. Hosted checks cover remaining platforms; maintainers handle final approval. No requests for changes to the bot and nothing merged. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. ### Maintainer current-main requalification Exact head `6d6cd26dff5f6f954fc737a5ac89ba9a960d2448`, upstream main `def4ef10099514d09188326852138a8b20c26237`. Maintainer remediation under the user's explicit directive to maintain Dependabot updates and qualify them for approval. This records an authorized routine-maintenance exception rather than inventing a newly required vector feature or claiming the old range excluded0.1.9. sqlite-vec remains in the existing optional memory extras; the base install gains no new dependency, service or network behavior. Native extension/wheel compatibility is the relevant install surface. Synchronized current main and repaired the selected sqlite-vec lock entry plus all three declared floor records to0.1.9, retaining unrelated lock metadata. Offline lock consistency and whitespace checks pass. Installed the actual0.1.9 Python3.12 native wheel in an isolated overlay and verified its module path/version. All45 vector-index/local-search/core-search/relation-key tests pass, exercising real extension loading, indexing/querying, filters, updates/deletes and persistence. The initial Windows suite exposed pre-existing fixture teardown errors: indexes cached connections but fixtures unlinked the open databases. A clean-main control reproduced that error. Fixed the test-only lifecycle to await close before unlink, and strengthened the persistence test to close the first index before reopening the database. Production vector/runtime logic and assertions are unchanged. Re-execution passes all45 with no cleanup errors. Existing Headroom native core supports startup; this does not claim a rebuilt sqlite-vec wheel or fresh Headroom Rust extension build. Hosted checks cover remaining platforms; maintainers handle final approval. No requests for changes to the bot and nothing merged. Post-merge requalification: main |
||
|
|
a8c2e4d903 |
fix(memory/vector): skip a corrupt vec_metadata row instead of aborting the search (#3165)
## Description
`SQLiteVectorIndex.search()` rebuilds a `VectorMetadata` for each
candidate row directly from the `vec_metadata` columns, with unguarded
deserialization:
```python
meta = VectorMetadata(
...
created_at=datetime.fromisoformat(row["created_at"]),
valid_until=(datetime.fromisoformat(row["valid_until"]) if row["valid_until"] else None),
entity_refs=json.loads(row["entity_refs"]),
content=row["content"],
metadata=json.loads(row["metadata_json"]),
)
```
A single row with an unparseable `created_at` / `valid_until` (bad
datetime) or malformed `entity_refs` / `metadata_json` (bad JSON) —
schema drift across an upgrade, or a partially written row — raises
`ValueError`/`JSONDecodeError` mid-loop and aborts the **entire**
search, losing every other (good) result.
This is the exact failure #2947 named: dict-shaped `entity_refs` "took
down whole memory searches rather than the one bad row." That fix added
`normalize_entity_refs` to `VectorMetadata.from_json` and
`Memory.from_dict`, but this search loop rebuilds metadata from
**columns** (not `from_json`), so it never got the healing — and
`created_at` / `metadata_json` in the same loop were left completely
unguarded.
The fix routes the per-row construction through a new `_row_to_metadata`
helper that skips an unreadable row (returns `None`, logs a warning) and
normalizes `entity_refs`, so one corrupt/legacy row no longer takes down
the whole search.
Reproduction:
```python
# index two memories, then corrupt one row on disk
conn.execute("UPDATE vec_metadata SET created_at=? WHERE memory_id=?", ("not-a-date", bad_id))
await index.search(VectorFilter(query_vector=good.embedding, top_k=5, user_id="alice"))
# before: raises ValueError -> zero results
# after: returns the good result; the bad row is skipped
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/memory/adapters/sqlite_vector.py`: added
`_row_to_metadata(row)` which builds `VectorMetadata` with guarded
`datetime.fromisoformat` / `json.loads` (returns `None` and logs on
failure) and runs `entity_refs` through `normalize_entity_refs`
(matching `from_json`). `search()` now uses it and skips rows that
return `None`.
- `tests/test_sqlite_vector_index.py`: added
`test_search_skips_corrupt_row_instead_of_aborting` — index two
memories, corrupt one row's `created_at` on disk, and assert `search()`
returns the good result and skips the bad row instead of raising.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
tests/test_sqlite_vector_index.py::...::test_search_skips_corrupt_row_instead_of_aborting -> passed
uvx ruff@0.16.2 check headroom/memory/adapters/sqlite_vector.py tests/test_sqlite_vector_index.py -> All checks passed!
uvx mypy@1.20.2 headroom/memory/adapters/sqlite_vector.py -> Success: no issues found in 1 source file
```
(A pre-existing Windows-only teardown `PermissionError` — the fixture
unlinks the db file while sqlite still holds it — and one pre-existing
`test_data_persists_across_instances` failure are unrelated to this
change; both are identical on clean main and pass on the Linux CI.)
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
`sqlite-vec` available, ruff 0.16.2 and mypy 1.20.2 via uvx.
- Exact command / steps: indexed two memories, `UPDATE vec_metadata SET
created_at='not-a-date'` (and a malformed `metadata_json`) on one row,
then `search()`. Before the fix it raised `ValueError` /
`JSONDecodeError` and returned nothing; after, it logs `Skipping
unreadable vec_metadata row (memory_id=...)`, returns the good result,
and omits the bad row. Reverting the loop to the inline construction
makes the new test fail (search raises); restoring it passes.
- Observed result: one corrupt row degrades to a skipped row; the rest
of the search is unaffected.
- Not tested: an actual on-disk schema-drift upgrade; the corrupt row is
injected directly, which is the exact shape the search loop consumes.
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is defensive row
deserialization in the sqlite-vec search path, not a
rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: a corrupt/legacy `vec_metadata` row
is now skipped (with a warning) instead of aborting the whole search.
Well-formed rows are built exactly as before, and `entity_refs` are
normalized (already the behavior downstream in the local backend).
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: memory vector search survives a single corrupt
row instead of returning zero results.
- Rollback path: revert this PR; `search()` returns to the inline
unguarded construction.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal adapter behavior)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
This completes #2947's "one bad row must not take down the whole search"
intent for the fields that fix did not reach on the column-based search
path (`created_at`, `valid_until`, `metadata_json`), and keeps
`entity_refs` consistent by routing it through the same
`normalize_entity_refs` helper. It is independent of, and complementary
to, the CCR-store row-tolerance fix in the sibling PR.
|
||
|
|
f5cea7c51e |
fix(memory): batch onnx embeddings and sqlite-vec ops
Make the ONNX + sqlite-vec memory path truly batched. Batch ONNX embed_batch calls, batch sqlite-vec index/remove work under a single cached connection, and update MCP warm-up to use batch embed/save/index flows. Add focused regression tests for ONNX batching, sqlite-vec single-connection batch behavior, and MCP warm-up batching. Skip the MCP-specific test when optional MCP dependencies are not installed. Refs #240 |
||
|
|
84ad47eba9 |
Add SQLiteVectorIndex using sqlite-vec for bounded vector search
A SQLite-based alternative to HNSWVectorIndex offering: - True CRUD operations (real deletes, not marks) - Bounded memory via SQLite page cache (default 8MB) - Persistent storage by default - Cosine similarity search - Native integration potential with FTS5 for hybrid search New files: - headroom/memory/adapters/sqlite_vector.py: Implementation - tests/test_sqlite_vector_index.py: 16 comprehensive tests Dependencies: - sqlite-vec added as optional dependency (pip install sqlite-vec) - Requires Python built with loadable extension support Key advantages over HNSWVectorIndex: - No SIGILL crash risk (pure C, no AVX requirement) - True deletes (not just marks, space actually reclaimed) - Simpler persistence (SQLite handles it automatically) - Consistent with SQLiteGraphStore (same technology stack) |