ai-memory/docs/research-basic-memory.md

basic-memory - Research Report

Follow-up: the September 2026 landscape refresh — how this project evolved, what the field looks like now, and what we concluded — is in research-2026-landscape.md.

Source project: basicmachines-co/basic-memory (Python, MCP-native, markdown-on-disk). Studied as inspiration and as the manual-write-note model we explicitly diverge from.

1. Purpose & Scope

Basic Memory is a local-first, MCP-native, Markdown-based personal knowledge graph. Its tagline is "Your AI never forgets again": notes live as plain Markdown files on disk, both humans (in Obsidian, VS Code) and LLMs (over MCP) read and write them, and a SQLite/Postgres index keeps a knowledge graph in sync.

The model is explicitly manual. The user (or, more often, the agent in response to the user) calls write_note, edit_note, move_note, etc. There is no implicit capture of conversation content. The "How it works" example in README.md:317-356 is literally "Ask the LLM to capture it: 'Make a note on coffee brewing methods.'" The README even prescribes user prompts ("Create a note about our project architecture decisions") as the activation step. Nothing in the codebase auto-saves conversation turns. The closest thing to automation is the continue_conversation prompt (src/basic_memory/mcp/prompts/continue_conversation.py:18-90), which only retrieves - it asks the model to search recent activity and load context; it never writes.

2. Storage Model

Both Markdown and SQL, with the files being source-of-truth. Each note is one Markdown file with YAML frontmatter:

---
title: Coffee Brewing Methods
type: note
permalink: coffee-brewing-methods
tags: [coffee, brewing]
---

The grammar is documented in docs/NOTE-FORMAT.md and parsed at src/basic_memory/markdown/entity_parser.py:1-27 using markdown-it plus custom observation_plugin and relation_plugin. Three semantic primitives:

• Entity (one per file): src/basic_memory/models/knowledge.py:28-149. Holds title, note_type, permalink, file_path, checksum, mtime, size, entity_metadata (JSON for custom frontmatter), external_id (stable UUID).
• Observation: - [category] text #tag (context) lines, indexed in the observation table (knowledge.py:220-263).
• Relation: - relation_type [[Other Entity]] lines, indexed in relation (knowledge.py:265-311). Bare [[X]] becomes links_to. Relations can be unresolved (to_id NULL until the target exists), then auto-resolved on sync.

Search is dual-stack and selected by database_backend config (src/basic_memory/config.py:222-226):

• SQLite: FTS5 virtual table search_index with custom tokenizer 'unicode61 tokenchars 0x2F' so / is searchable (src/basic_memory/models/search.py:62-94). Semantic vectors via sqlite-vec virtual table search_vector_embeddings (search.py:146-153).
• Postgres: real search_index table with tsvector GIN + pgvector (search.py:17-58) and pg_trgm for fuzzy link resolution (migration f8a9b2c3d4e5).

Hybrid search defaults: vector candidates semantic_vector_k=100, similarity threshold 0.55, model bge-small-en-v1.5 via FastEmbed (config.py:233-313). Search type is hybrid when semantic is on, else text (config.py:313-318).

There is also a NoteContent table (knowledge.py:152-217) that materializes the markdown body in the DB with a file_write_status state machine (pending|writing|synced|failed|external_change_detected) and db_version/file_version for conflict resolution between AI writes and human file edits.

3. MCP Tools Exposed

All tools are registered via the @mcp.tool decorator and exported from src/basic_memory/mcp/tools/__init__.py:9-65. Every tool is annotated with MCP readOnlyHint/destructiveHint/idempotentHint/openWorldHint so agents can pick safely. Every single tool requires explicit invocation - none are triggered by the server itself.

Tool	Hint	File
write_note	destructive, not idempotent	tools/write_note.py:21
edit_note (append/prepend/find_replace/replace_section/insert_*)	not destructive	tools/edit_note.py:225
read_note, view_note, read_content	read-only	tools/read_note.py:64, view_note.py:13, read_content.py:157
delete_note	destructive	tools/delete_note.py:184
move_note	not destructive (updates links)	tools/move_note.py:346
search_notes (advanced: tags, status, metadata_filters, after_date, search_type=text/vector/hybrid)	read-only	tools/search.py:564
recent_activity	read-only	tools/recent_activity.py:28
build_context (resolves memory:// URIs, walks relation graph N hops, depth=1..3)	read-only	tools/build_context.py:114
list_directory, canvas (Obsidian canvas), list_workspaces	mixed	various
list_memory_projects, create_memory_project, delete_project	mixed	tools/project_management.py
schema_infer, schema_validate, schema_diff (Picoschema over frontmatter)	read-only	tools/schema.py:206-440
ChatGPT-compat search / fetch	read-only	tools/chatgpt_tools.py:107-171
cloud_info, release_notes	read-only

Two MCP prompts ship: continue_conversation and recent_activity (both retrieve-only). There is also a view_note UI artifact path.

4. Memory Lifecycle

There is no lifecycle. Grep'd the entire src/ tree for decay|aging|consolidat|summariz|forget|expire|ttl|prune|archive_old - zero matches outside of unrelated OAuth-token expiry and SQLAlchemy expire_on_commit. Notes are append-only forever unless a human or agent explicitly calls delete_note, move_note, or edit_note. There is no auto-summarization, no auto-merge of duplicates, no recency weighting in search ranking (only an after_date filter), no "cold storage" tier. recent_activity (tools/recent_activity.py:28) just queries by created_at/updated_at; it doesn't shape memory.

The only background process is the WatchService (src/basic_memory/sync/watch_service.py:81-145) + SyncService (src/basic_memory/sync/sync_service.py:153-188), which reconcile file changes with the DB after a sync_delay of 1000 ms (config.py:338). That's housekeeping, not lifecycle.

5. Cross-Project / Cross-Agent

Projects are a first-class concept. Config holds a Dict[str, ProjectEntry] (config.py:184-195), each with a path, a mode (LOCAL or CLOUD - per-project routing), optional workspace_id, and bisync state. A default_project is auto-set to the first project (config.py:705-711).

Project resolution is a unified three-tier chain (docs/ARCHITECTURE.md:298-324): explicit project argument → default → single-project fallback. Every MCP tool accepts project and project_id (UUID) parameters. get_project_client(project, ...) (mcp/project_context.py) routes to local ASGI or cloud HTTP per-project, so you can mix.

For agent handoffs, there is no special handshake - basic-memory treats all MCP clients identically. The "handoff" story is: agent A writes notes to project foo, agent B (any other MCP client pointed at the same project) reads them. The Markdown files on disk are the lingua franca. There is no session/agent identifier, no per-agent scratchpad. created_by/last_updated_by columns exist (knowledge.py:99-102) but are populated only in cloud (user_profile_id), null for local/CLI.

6. Backup & Portability

• Default location: ~/basic-memory for notes (per BASIC_MEMORY_HOME env, README:197), ~/.basic-memory/ for SQLite DB + config (config.py:24, 67-80). Config in ~/.basic-memory/config.json, chmod 0600.
• Portability: Excellent for files (they're just Markdown - git clone, rsync, Syncthing, rclone all work). The DB is a derived index - bm sync rebuilds it from the files.
• Schema migrations: 22 Alembic migrations in src/basic_memory/alembic/versions/, auto-run on startup via get_or_create_db (services/initialization.py:23-38). Migrations cover both SQLite and Postgres.
• Importers for Claude conversations, ChatGPT exports, and memory.json (the original MCP "memory" server format) live in src/basic_memory/importers/.
• Cloud sync uses rclone bisync (config.py:136-138, 164-167), not a custom protocol - another portable choice.

7. Strengths Worth Borrowing

1. Files are source of truth, DB is derived index. Survives any DB corruption, plays nice with git, version control, and grep. Bidirectional human/AI editing via file-watcher + checksum.
2. MCP behavior annotations on every tool (readOnlyHint, destructiveHint, etc., tools/write_note.py:23). Agents can plan multi-step actions without trial-and-error.
3. Aggressive AliasChoices aliasing of parameter names (write_note.py:31 accepts directory|folder|dir|path; search.py:574 accepts query|q|search|text). LLMs use whatever name their training reaches for; the tool absorbs the variance.
4. memory:// URI scheme + build_context graph walker (tools/build_context.py:114-247) - turns wiki-links into navigable context. Cleaner than dumping the whole graph.
5. Unresolved relations as first-class state (knowledge.py:282) - to_id nullable, resolves later when target appears. Forward references just work.
6. Per-project routing with mixed local/cloud modes per project, not per-server.
7. Composition root + typed-client pattern (docs/ARCHITECTURE.md:14-256) keeps MCP/CLI/API entrypoints clean and testable.
8. The NoteContent table with file_write_status state machine (knowledge.py:155-217) handles the race between agent writes and on-disk human edits - worth replicating if you keep a DB cache.

8. Weaknesses / Friction (Avoid)

1. The manual write_note ceremony is the headline friction. Users must explicitly tell the model "make a note about this," and the model must decide title, directory, tags, note_type, and the semantic observation/relation grammar - all on every call. The write_note signature has 11 parameters (write_note.py:25-45). Skip this entirely: capture should be ambient (post-turn summarization, automatic salience scoring, etc.) - not a tool the model must remember to call.
2. Append-only forever. No decay, no consolidation, no automatic deduplication. Long-running graphs accumulate cruft. Recent v0.20 added a guard so write_note errors on conflict instead of silently upserting (write_note.py:240-262), which protects data but pushes the burden back onto the agent to manage identity. For agent long-term memory, decay/merge/consolidation is essential and absent here.
3. The semantic grammar is human-authored convention. - [category] text #tag (context) and - relation_type [[Target]] are intuitive for humans in Obsidian but require the LLM to generate this format correctly every time. Drift is normal. A Rust server can store edges natively and let the LLM emit prose.
4. Note identity is brittle. permalink is derived from title/path; renames create work (update_permalinks_on_move defaults to False, config.py:349-352). The 11-column entity table + separate note_content is a lot of machinery to keep in sync with files.
5. No agent/session model. No notion of "who wrote this," "which session," or "what was the user's intent." created_by exists only for cloud auth (knowledge.py:99-102). For multi-agent handoffs, you'd want provenance baked in.
6. Search ranking is keyword-or-vector with a fixed min_similarity=0.55 (config.py:307-312). No recency/importance reranking, no usage feedback, no per-query learning.
7. The Markdown source-of-truth tradeoff. Filesystem latency, checksum recomputation, FTS5 rebuilds, and circuit-breaker retry tracking (sync_service.py:179-281) are a lot of infrastructure just to keep a DB consistent with files. For agent memory where the files exist only because LLMs wrote them, this is overhead with no payoff - keep the DB as primary and only export Markdown on demand.
8. Tool surface is wide (~25 tools). Agents have to pick among write_note/edit_note/move_note/delete_note/read_note/view_note/read_content/search/search_notes/build_context/recent_activity/list_directory/list_memory_projects/canvas/schema_*. Even with hints, that's a lot of context burned describing tools. Aim for a smaller, more orthogonal set.

Bottom line: borrow the graph + observations + wiki-link primitives, the memory:// URI navigation, the typed-client + composition-root layering, the MCP annotations, and the file-as-portable-export idea - but invert the capture model (ambient, not invoked), add a lifecycle (decay/consolidation/summarization), keep the DB primary, add agent/session provenance, and ship a much narrower MCP tool surface.