mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description
`SQLiteMemoryStore._build_query_conditions` binds a metadata filter
value as JSON text for anything that isn't a `str`:
```python
conditions.append(f"json_extract(metadata, '$.{key}') = ?")
params.append(json.dumps(value) if not isinstance(value, str) else value)
```
But `json_extract` returns a **native** SQLite value (INTEGER / REAL /
TEXT). Comparing that against `json.dumps(value)` — `"5"`, `"true"`,
`"1.5"` — never matches, because SQLite does not equate `5 = '5'`. So a
metadata filter on any numeric or boolean value silently returns
**nothing**; only string filters work.
Reproduction (memory stored with `metadata={"priority": 5, "archived":
True, "score": 1.5, "tag": "x"}`):
```python
await store.query(MemoryFilter(user_id="u", metadata_filters={"priority": 5})) # -> [] (want 1)
await store.query(MemoryFilter(user_id="u", metadata_filters={"archived": True})) # -> [] (want 1)
await store.query(MemoryFilter(user_id="u", metadata_filters={"score": 1.5})) # -> [] (want 1)
await store.query(MemoryFilter(user_id="u", metadata_filters={"tag": "x"})) # -> [1] (only strings work)
```
This affects both `query()` and `count()` (they share the builder), on
the public `MemoryFilter(metadata_filters=...)` API.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/memory/adapters/sqlite.py`: bind the **native** Python value
for scalar metadata filters (`str` / `int` / `float`; `bool` is an `int`
subclass, and a JSON `true` extracts to `1`, which `True` binds to). A
non-scalar value (`dict`/`list`) — not a bindable SQLite type — keeps
the JSON-text comparison so the query does not raise. The JSON-path key
validation (injection guard) is unchanged.
- `tests/test_memory/test_query_conditions.py`: added
`test_metadata_scalar_filters_bind_native_values` (int/bool/float/str
bind natively, not as `"5"`/`"true"`) and
`test_metadata_non_scalar_filter_keeps_json_text` (a list value keeps
`json.dumps`).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
tests/test_memory/test_query_conditions.py -> 5 passed in 1.27s
(the scalar test FAILS on pre-fix code, verified via git stash)
uvx ruff@0.16.2 check headroom/memory/adapters/sqlite.py tests/test_memory/test_query_conditions.py -> All checks passed!
uvx mypy@1.20.2 headroom/memory/adapters/sqlite.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.16.2 and mypy 1.20.2 via uvx.
- Exact command / steps: saved a memory with
`metadata={"priority":5,"archived":True,"score":1.5,"tag":"x"}` and
queried each field. Before the fix, `priority=5`, `archived=True`, and
`score=1.5` each returned 0 rows while `tag="x"` returned 1; after the
fix all four return 1, a non-matching `priority=6` returns 0, and a
dict-valued filter runs without raising.
- Observed result: numeric/boolean/string metadata filters all match
correctly; non-matching values still exclude; non-scalar values do not
crash.
- Not tested: no change to the JSON-path key-injection validation
(covered by existing security tests) — only the bound value is
corrected.
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is a SQLite memory-store query
builder, not a rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: string filters are unchanged;
numeric/boolean filters change from silently returning nothing to
correctly matching. No schema or API change.
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this PR.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal behavior)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
75 lines
2.7 KiB
Python
75 lines
2.7 KiB
Python
"""SQLiteMemoryStore._build_query_conditions scope filtering.
|
|
|
|
`_build_query_conditions` only reads the filter, so it is exercised directly via
|
|
``object.__new__`` (no DB, no embedder).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from headroom.memory.adapters.sqlite import SQLiteMemoryStore
|
|
from headroom.memory.ports import MemoryFilter
|
|
|
|
|
|
def _conditions(**kwargs) -> tuple[list[str], list]:
|
|
store = object.__new__(SQLiteMemoryStore)
|
|
return store._build_query_conditions(MemoryFilter(**kwargs))
|
|
|
|
|
|
def test_turn_id_is_applied_without_agent_id():
|
|
"""A (user, session, turn) filter without agent_id must still narrow to the
|
|
turn — previously the turn_id condition was nested inside the agent_id block
|
|
and silently dropped, returning the whole session."""
|
|
conditions, params = _conditions(user_id="u", session_id="s", turn_id="t")
|
|
|
|
assert "turn_id = ?" in conditions
|
|
assert "t" in params
|
|
|
|
|
|
def test_agent_id_and_turn_id_both_applied():
|
|
conditions, params = _conditions(user_id="u", session_id="s", agent_id="a", turn_id="t")
|
|
|
|
assert "agent_id = ?" in conditions
|
|
assert "turn_id = ?" in conditions
|
|
assert "a" in params and "t" in params
|
|
|
|
|
|
def test_agent_id_only_still_applied():
|
|
conditions, _ = _conditions(user_id="u", session_id="s", agent_id="a")
|
|
|
|
assert "agent_id = ?" in conditions
|
|
assert "turn_id = ?" not in conditions
|
|
|
|
|
|
def test_metadata_scalar_filters_bind_native_values():
|
|
"""A metadata filter on a numeric/boolean value must bind the NATIVE value.
|
|
|
|
``json_extract`` returns a native SQLite value, so binding ``json.dumps(value)``
|
|
("5", "true") compared the column against text and matched nothing (SQLite
|
|
never equates ``5 = '5'``). Scalars must be bound as-is; only non-scalar
|
|
(dict/list) values keep the JSON-text form.
|
|
"""
|
|
conditions, params = _conditions(
|
|
user_id="u",
|
|
metadata_filters={"priority": 5, "archived": True, "score": 1.5, "tag": "x"},
|
|
)
|
|
|
|
assert any("json_extract(metadata, '$.priority')" in c for c in conditions)
|
|
# Native scalars, not their json.dumps text forms.
|
|
assert 5 in params and "5" not in params
|
|
assert 1.5 in params
|
|
assert "x" in params
|
|
# bool binds as its native value (a JSON ``true`` extracts to 1).
|
|
assert True in params
|
|
assert "true" not in params
|
|
|
|
|
|
def test_metadata_non_scalar_filter_keeps_json_text():
|
|
"""A dict/list metadata value is not a bindable SQLite type, so it keeps the
|
|
JSON-text comparison rather than raising when the query runs."""
|
|
import json
|
|
|
|
conditions, params = _conditions(user_id="u", metadata_filters={"tags": ["a", "b"]})
|
|
|
|
assert any("json_extract(metadata, '$.tags')" in c for c in conditions)
|
|
assert json.dumps(["a", "b"]) in params
|