Commit Graph
4 Commits
Author SHA1 Message Date
d232d097cf fix(memory): bound verb-pattern word runs to stop quadratic backtracking in bridge import (#4370) (#4376)
Fixes #4370

**Root cause confirmed**: the four verb patterns in
`extract_relationships_from_section`
(`headroom/memory/bridge_parsers.py:188-193`) use `\w+` word runs. On a
long token without the verb, the engine starts a match attempt at every
position and `\w+` backtracks character by character before failing —
O(n²) in the token length, four patterns over. These run synchronously
inside the bridge auto-import coroutines, so a single base64 blob /
minified line / hash in a memory file stalls the proxy event loop.

**Fix**: bound the word runs — `\w+` → `\w{1,64}` in all four patterns
(the entity extractor already caps names at <80–100 chars, so 64-char
words cover every realistic entity). Backtracking per scan position is
now capped at 64 steps, making the scan linear in section size. Matching
semantics are unchanged for words up to 64 chars (patterns with longer
words simply don't bind, same as the entity caps elsewhere).

**Measured** (8,000-char single-token section, one pattern, same
machine):
- before: **1.09 s**
- after: **0.02 s**
- full `extract_relationships_from_section` (4 patterns + bold-colon
pass) at 20,000-char token: **0.18 s**

**Tests** (`tests/test_memory_bridge.py`, 2 new):
- `test_verb_patterns_still_match_with_long_words`: normal matching
unaffected within the bound.
- `test_long_token_is_linear_not_quadratic`: a 20,000-char token must
extract in <2 s and still find the "Headroom uses SQLite" relation after
it (fails on the old patterns, where the same call takes minutes).

Verified: `pytest tests/test_memory_bridge.py
tests/test_memory_system.py` → **116 passed, 18 skipped** (skips + 14
backend-fixture errors are pre-existing env-dependency gaps in my local
venv — `numpy` not installed there — not related to this change).
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4376"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: viwe-monai <viwe-monai@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-03 22:54:36 -06:00
chopratejas 0adc39ab7a Fix CI: guard starlette imports, asyncio.run(), deprecate datetime.utcnow()
- Guard starlette imports in test_compress_api.py (skip ASGI tests without proxy deps)
- Replace asyncio.get_event_loop().run_until_complete() with asyncio.run() (Python 3.13)
- Replace datetime.utcnow() with datetime.now(timezone.utc).replace(tzinfo=None) everywhere
2026-02-19 11:03:24 -08:00
Claude Code BotandClaude Opus 4.5 1c5a0e09fa fix(tests): add missing skip decorator and tests for exclude_tools
## What this PR fixes

1. **CI Python 3.12 failure**: Added skip decorator to `TestLocalBackend`
   in `test_memory_system.py` - these tests require hnswlib which is not
   available on all CI runners.

2. **Missing test coverage**: Added 6 tests for the `exclude_tools` feature
   in `test_content_router.py`. Tests use existing helper functions
   `generate_python_code()`, `generate_json_data()`, and
   `generate_search_results()` defined at lines 57-95 of the same file.

3. **Anthropic/OpenAI inconsistency**: Fixed `_process_content_blocks()`
   to add `router:excluded:tool` marker for Anthropic format, matching
   the OpenAI format behavior at line 1157.

4. **Dead code removal**: Removed unused `exclude_tools` field from
   `SmartCrusherConfig` - the actual implementation uses
   `ContentRouterConfig.exclude_tools` in content_router.py.

AI review: code-reviewer (2 iterations), adversarial-reviewer (2 iterations)
Issues fixed: missing test coverage, format inconsistency, dead code

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 17:44:03 -08:00
chopratejas da74341858 Add hierarchical memory system with graph + vector storage
Implement comprehensive memory system supporting:
- Local backend (SQLite + FTS5 + HNSW) for zero-dependency operation
- Mem0 backends (Neo4j + Qdrant) for production graph memory
- DirectMem0Adapter for optimized pre-extracted data (bypasses LLM)
- Memory extraction with facts, entities, and relationships
- Proxy integration with --memory flag for automatic memory injection

Key components:
- headroom/memory/backends/: LocalBackend, Mem0Backend, DirectMem0Adapter
- headroom/memory/system.py: MemorySystem with tool-based interface
- headroom/memory/extraction.py: Entity and relationship extraction
- headroom/proxy/memory_handler.py: Proxy integration layer
- headroom/prediction/feature_extractor.py: Content analysis features

Testing:
- 217 new memory system tests covering all backends
- LoCoMo evaluation framework for memory quality assessment
- Integration tests for proxy memory functionality

Also removes deprecated example files in favor of focused test coverage.
2026-01-26 21:58:47 -08:00