Developer Guide
Architecture Overview
ai-memory is an AI-agnostic memory management system built as a single Rust binary that serves three roles:
- MCP tool server – stdio JSON-RPC server exposing 101 advertised entries at
--profile full(100 callable memory tools + the always-onmemory_capabilitiesbootstrap) + 2 MCP prompts for any MCP-compatible AI client (Claude AI, OpenAI ChatGPT, xAI Grok, META Llama, and others) - CLI tool – direct SQLite operations for store, recall, search, list, etc. (completely AI-agnostic)
- HTTP daemon – an Axum web server exposing the same operations as a REST API with 92 route registrations / 78 unique URL paths (completely AI-agnostic)
Key architectural features: Zero token cost (no context loaded until recall), TOON compact default response format (79% smaller than JSON), MCP prompts capability (recall-first behavioral rules + memory-workflow reference card), 4 feature tiers with optional local LLMs via Ollama, true dedup on title+namespace, 6-factor recall scoring with score field in responses.
All three interfaces share the same storage layer (src/storage/, exposed as the db alias) and validation layer (validate.rs). The daemon adds automatic garbage collection (every 30 minutes) and graceful shutdown with WAL checkpointing.
main.rs -- Thin CLI shim (W6 refactor); top-level Command enum now lives in daemon_runtime.rs (89 subcommands with --features sal, 87 in the default build)
daemon_runtime.rs -- HTTP daemon `serve` bootstrap, MCP `mcp` dispatch, top-level clap Command enum
models/ -- Data structures: Memory (28 fields at v0.9.0), MemoryLink (9 relations at v0.8.0), MemoryKind (Batman Form-6 + Goal/Plan/Step), LifecycleState (v0.8.0 Pillar-2 state machine), Citation/SourceSpan (Form-4), query types, constants
handlers/ -- HTTP request handlers split per domain (http.rs, federation_receive.rs, hook_subscribers.rs, transport.rs, plus per-surface modules: recall.rs, memories.rs, admin.rs, kg.rs, …); Axum extractors + JSON responses; error sanitization. Route-path SSOT in handlers/routes.rs (#1558 batch 4 — one const per production route path; lib.rs registers them, the postgres gate / federation receiver / doctor match on them)
storage/ -- sqlite SQL primitives; CRUD, FTS5, recall scoring, GC, migration (CURRENT_SCHEMA_VERSION = 81)
store/ -- SAL `MemoryStore` trait + adapter implementations (sqlite + postgres + AGE feature gates); new DB operations land here FIRST (post-#961)
mcp/ -- MCP server over stdio JSON-RPC; tool registry (registry.rs incl. the tool_names const module), per-tool handlers under tools/, JSON-RPC wire-constant SSOT (mcp/jsonrpc.rs, #1558 batch 3 — version tag, reserved error codes, method names), tool-call param-name SSOT (mcp/param_names.rs), notification handling
identity/ -- NHI identity: keypair storage (keypair.rs — DAEMON_KEYPAIR_LABEL), reserved-principal sentinel SSOT (sentinels.rs, #1558 batch 2 — DAEMON_PRINCIPAL, ANONYMOUS_INVALID, …; validate::RESERVED_AGENT_IDS is built from these), attestation (attest.rs), signing/verification (sign.rs/verify.rs), replay protection (replay.rs)
models/field_names.rs -- wire-field-name SSOT (one const per JSON response key shared across handlers/tools)
validate.rs -- Input validation for all write paths (RequestValidator + single-field free fns)
errors.rs -- Structured error types (ApiError, MemoryError), error_codes consts, the errors::msg wire-message const module, error sanitization for HTTP responses
color.rs -- ANSI color output for CLI (zero dependencies, auto-detects terminal)
config.rs -- Tier configuration system (keyword, semantic, smart, autonomous), feature gating, TtlConfig, archive_on_gc
embeddings.rs -- Embedding pipeline: HuggingFace model loading, vector generation, cosine similarity
llm.rs -- LLM integration via Ollama for query expansion, auto-tagging, contradiction detection
mine.rs -- Retroactive conversation import from Claude, ChatGPT, and Slack exports
reranker.rs -- Hybrid recall algorithm: blends semantic (embedding) and keyword (FTS5) scores
hnsw.rs -- In-memory HNSW vector index for approximate nearest-neighbor search
governance/ -- Rule engine, agent-action evaluator, signed rule storage (L1-6 substrate rules)
atomisation/ -- WT-1 atomiser engine + LlmCurator
multistep_ingest/ -- Form 3 multi-step ingest orchestrator (two-phase deterministic + LLM)
synthesis/ -- Form 1 online dedup-and-synthesis
confidence/ -- Form 5 auto-confidence + shadow + decay
persona/ -- QW-2 persona-as-artifact generator
offload/ -- QW-3 context-offload primitive + TTL sweep
forensic/ -- L2-5 forensic bundle export/verify
federation/ -- Quorum sync, peer attestation, mTLS allowlist
kg/ -- Knowledge-graph traversal (recursive-CTE + AGE Cypher)
subscriptions.rs -- HMAC-signed webhook dispatch (mandatory at v0.7.0 post R3-S1.HMAC; unsigned dispatch DISABLED), DLQ, replay
signed_events.rs -- Append-only audit chain with V-4 cross-row hash chain
Embedding Pipeline (semantic tier and above)
When running at the semantic tier or higher, ai-memory loads a HuggingFace embedding model at startup and generates dense vector embeddings for each memory. The pipeline:
- Model loading (
embeddings.rs) – downloads and caches a sentence-transformer model from HuggingFace on first run - Embedding generation – new memories are embedded at insert time; existing memories are backfilled on first startup with embeddings enabled
- Storage – embeddings are stored as BLOB columns in the
memoriestable (schema migration v3) - Hybrid recall (
reranker.rs) – at recall time, the query is embedded and compared against stored embeddings via cosine similarity, then blended with FTS5 keyword scores to produce a final ranking
Embedding models:
all-MiniLM-L6-v2(384 dimensions, ~90 MB) – used at thesemantictiernomic-embed-text-v1.5(768 dimensions, ~270 MB) – used at thesmartandautonomoustiers
Code Structure
src/main.rs
Clistruct withclapderive – defines all CLI commands and global flags (--db,--json). Lives insrc/daemon_runtime.rs(W6 refactor moved it offsrc/main.rs).Commandenum (insrc/daemon_runtime.rs) – at v0.9.0 the enum carries 90 unique variants under--features sal(88 in the default build — the gap is the two sal-gated variantsMigrate+SchemaInit; SSOT:EXPECTED_CLI_SUBCOMMANDS_DEFAULT=88/EXPECTED_CLI_SUBCOMMANDS_SAL=90insrc/lib.rs, pinned bytests/cli_subcommand_count_invariant.rs): the v0.6.x core (Serve,Mcp,Store,Update,Recall,Search,Get,List,Delete,Promote,Forget,Link,Consolidate,Resolve,Shell,Sync,SyncDaemon,AutoConsolidate,Gc,Stats,Namespaces,Namespace,Config,Export,Import,Completions,Man,Mine,Archive,Agents,Pending,Backup,Restore,Curator,Bench,Migrate(gated--features sal),SchemaInit(gated--features sal),Doctor,Boot,Install,Wrap,Logs,Audit), the v0.7 additions (Identity,Offload,Deref,Rules,Governance,VerifyReflectionChain,VerifySignedEventsChain,ExportForensicBundle,VerifyForensicBundle,ExportReflections,Atomise,Persona,Calibrate,Skill,Share,Expand,Reembed(#1598)), the FX-12/FX-C3 MCP↔CLI parity batch (KgQuery,FindPaths,RecallObservations,CheckDuplicate,Replay,Reflect,Subscribe,Unsubscribe,ListSubscriptions,SubscriptionReplay,SubscriptionDlqList,Notify,Inbox,IngestMultistep,KgInvalidate,KgTimeline,EntityRegister,EntityGetByAlias,DependentsOfInvalidated,ReflectionOrigin,QuotaStatus), the #1389 L2RecoverPreviousSession, the v0.8.0 #1720 B2Reown+ PE-8VerifyAuditTrail, the v0.8.0 #1727UndoEdit(non-destructive in-place-edit undo), the v0.9.0 #1859Lineage, and the v0.9.0 #1827Capability(macaroon capability-token keygen/mint/attenuate/inspect/verify). Runai-memory --helpfor the live list.StoreArgsincludes--expires-atand--ttl-secsflags for custom expirationUpdateArgsincludes--expires-atflag for setting expiration on existing memoriesListArgsincludes--offsetflag for paginationauto_namespace()– detects namespace from git remote URL or directory namehuman_age()– formats ISO timestamps as “2h ago”, “3d ago” for CLI outputserve()– starts the Axum server with all routes (92 production.route(...)registrations / 78 unique URL paths — includesPOST /memories/{id}/promote, the 4 archive endpoints, namespace-standard endpoints, webhook subscription endpoints, KG endpoints, approval-SSE, quota status, link-verify, capture_turn, share, skills, the 14 #1111 MCP-parity paths, federation sync, the v0.9.0 #1859 lineage read endpoint), spawns GC task, handles graceful shutdown via SIGINT with WAL checkpointcmd_*()functions – one per CLI command, each opens the DB directly
src/models/
Tierenum (Short,Mid,Long) with TTL defaults: 6h, 7d, noneMemorystruct – the core data type with 28 fields at v0.9.0 (27 at v0.8.0, 26 at v0.7.0, was 15 at v0.6.x): addsreflection_depth(Task 1/8),memory_kind(Batman Form-6 vocabulary),entity_id+persona_version(QW-2),citations+source_uri+source_span(Form-4 fact provenance),confidence_source+confidence_signals+confidence_decayed_at(Form-5 calibration),version(schema v45 — Gap-1 optimistic concurrency formemory_update),lifecycle_state(schema v64 — v0.8.0 Pillar-2 #1709 typed-cognition state machine:open→active→blocked/done/abandoned, transitions enforced onmemory_update), andcid(schema v74 — v0.9.0 G8 #1825 additive BLAKE3 content-id,Option<String>). ExtensiblemetadataJSON column still present. Canonical truth insrc/models/memory.rs.MemoryLinkstruct – typed directional links between memories. Nine relation variants at v0.8.0 (six at v0.7.0, four at v0.6.x):related_to,supersedes,contradicts,derived_from,reflects_on,derives_from,decomposes_into,depends_on,advances. Carries v0.7 temporal-validity (valid_from,valid_until,observed_by) and attestation (signature,attest_level,signed_at) columns.- Request types:
CreateMemory,UpdateMemory,SearchQuery,ListQuery,RecallQuery,RecallBody,LinkBody,ForgetQuery,ConsolidateBody,ImportBody - Response types:
Stats,TierCount,NamespaceCount TtlConfigstruct – per-tier TTL overrides loaded fromconfig.toml(short_ttl_secs,mid_ttl_secs,long_ttl_secs,short_extend_secs,mid_extend_secs)ResolvedTtlstruct – resolved TTL values after merging config defaults with per-tier overrides- Constants:
MAX_CONTENT_SIZE(65536),PROMOTION_THRESHOLD(5),SHORT_TTL_EXTEND_SECS(3600),MID_TTL_EXTEND_SECS(86400)
src/mcp/
The MCP (Model Context Protocol) server implementation. MCP is an open standard – this server works with any MCP-compatible AI client. Runs over stdio, processing one JSON-RPC message per line. The registry exposes 101 advertised entries at --profile full (100 callable “memory tools” + the always-on memory_capabilities bootstrap; both numbers are intentional, see issue #862). Default --profile core ships 7 tools (the original 5 + memory_load_family + memory_smart_load) plus the always-on bootstrap.
The pre-#1066 monolithic src/mcp.rs is GONE — the module is split: src/mcp/registry.rs owns the canonical registered_tools() iterator + tool_definitions() view + the tool_names const module (100 canonical tool-name consts at v0.8.0 — extracted per #1187 / Wave-1 PR1, then grown through the v0.7.x/v0.8.0 tool additions incl. memory_capture_turn and the #1709 coordination tools; tool_names::ALL.len() is pinned against Profile::full().expected_tool_count()); src/mcp/tools/*.rs host per-tool handlers AND each tool’s <ToolName>Request schemars struct + McpTool impl; src/mcp/mod.rs wires the JSON-RPC dispatch loop; src/mcp/jsonrpc.rs is the JSON-RPC wire-constant SSOT (#1558 batch 3) and src/mcp/param_names.rs the tool-call param-name SSOT.
Post-v0.7.0 #987 (D1.6) the source-of-truth lives in registered_tools() — a single Vec<RegisteredTool> with one entry per McpTool impl. tool_definitions() is now a thin four-line view that iterates the vec and projects each row to the wire shape (name/description/docs/inputSchema). The hand-coded json!({...}) macro that previously held every tool’s schema verbatim is GONE.
RpcRequest/RpcResponse/RpcError– JSON-RPC 2.0 typesregistered_tools()(src/mcp/registry.rs) – canonical iterator over every per-toolMcpToolimpl. Each entry is aRegisteredToolderived fromT::name()/T::description()/T::docs()/T::family()/T::input_schema()whereTis a zero-sized type defined in the per-tool module (e.g.crate::mcp::store::StoreTool). Adding a new MCP tool = ONE line here + the impl.tool_definitions()(src/mcp/registry.rs) – thin view overregistered_tools()that returns the full-surface tool schemas fortools/list(every Family; includes v0.7 additionsmemory_reflect,memory_atomise,memory_ingest_multistep,memory_persona,memory_persona_generate,memory_offload,memory_deref,memory_calibrate_confidence, the 7 L1-5 Agent Skills tools,memory_check_agent_action,memory_rule_list,memory_export_reflection,memory_dependents_of_invalidated,memory_find_paths,memory_verify,memory_quota_status, the originalmemory_capabilities,memory_expand_query,memory_auto_tag,memory_detect_contradiction, the 4 archive tools, etc.). Filtered to the active--profilebytool_definitions_for_profile(). Per issue #864: “Family” in this codebase always refers to the MCP tool-family enum — eight variants at v0.7.0 (Family::Core/Lifecycle/Graph/Governance/Power/Meta/Archive/Otherinsrc/profile.rs) — NEVER to theMemoryKindBatman vocabulary; those are unrelated taxonomies.memory_recallschema includesuntilparameter andformatparameter (enum:"json","toon","toon_compact", default:"toon_compact")memory_searchschema includesformatparameter (enum:"json","toon","toon_compact", default:"toon_compact") and enforcesmaximum: 200on limitmemory_listschema includesformatparameter (enum:"json","toon","toon_compact", default:"toon_compact") and enforcesmaximum: 200on limitmemory_consolidateschema enforcesminItems: 2, maxItems: 100on IDsmemory_updateschema includesexpires_atparameter
handle_store(),handle_recall(),handle_search(),handle_list(),handle_delete(),handle_promote(),handle_forget(),handle_stats(),handle_update(),handle_get(),handle_link(),handle_get_links(),handle_consolidate(),handle_archive_list(),handle_archive_restore(),handle_archive_purge(),handle_archive_stats()– one handler per toolhandle_request()– routes JSON-RPC methods:initialize,notifications/initialized,tools/list,tools/call,ping- Notification handling: all JSON-RPC notifications (requests without an
idfield) are correctly skipped without sending a response, per the JSON-RPC 2.0 specification run_mcp_server()– main loop: reads lines from stdin, parses JSON-RPC, dispatches, writes responses to stdout
Protocol version: 2024-11-05. All tool responses are wrapped in MCP content blocks ({"content": [{"type": "text", "text": "..."}]}). The protocol is AI-agnostic – any MCP client can connect.
MCP Prompts: The server exposes 2 prompts via prompts/list:
- recall-first – System prompt with 8 behavioral rules for proactive memory use. Supports an optional
namespaceargument for scoped recall. - memory-workflow – Quick reference card for the full tool surface.
MCP Error Codes: The server uses standard JSON-RPC 2.0 error codes:
-32700– Parse error (malformed JSON)-32600– Invalid request (missing required fields)-32601– Method not found (unknown JSON-RPC method)-32602– Invalid params (bad tool arguments)- Application-level errors are returned as text in the MCP content block with
"isError": true, not as JSON-RPC error codes.
src/validate.rs
Input validation for every write path. Called by CLI, HTTP handlers, and MCP handlers.
| Function | Validates |
|---|---|
validate_title() |
Non-empty, max 512 chars (MAX_TITLE_LEN), no control chars |
validate_content() |
Non-empty, max 64KB, no null bytes |
validate_namespace() |
Non-empty, max 512 chars (MAX_NAMESPACE_LEN); / allowed as hierarchy delimiter (no leading/trailing/empty segments); no backslashes/spaces/nulls |
validate_source() |
Must be one of VALID_SOURCES: user, nhi, claude (deprecated), hook, api, cli, import, consolidation, system, chaos, notify |
validate_tags() |
Max 50 tags, each max 128 bytes, no empty strings |
validate_id() |
Non-empty, max 128 bytes, no null bytes |
validate_expires_at() |
Valid RFC3339, not in the past |
validate_ttl_secs() |
Positive, max 1 year |
validate_relation() |
Must be one of VALID_RELATIONS (nine at v0.8.0): related_to, supersedes, contradicts, derived_from, reflects_on, derives_from, decomposes_into, depends_on, advances |
validate_confidence() |
Finite number, 0.0 to 1.0 |
validate_priority() |
Integer, 1 to 10 |
validate_create() |
Full validation for CreateMemory |
validate_memory() |
Full validation for Memory (import) |
validate_update() |
Validates only present fields |
validate_link() |
Validates both IDs, relation, and rejects self-links |
validate_consolidate() |
2-100 IDs, validates title, summary, namespace |
src/color.rs
ANSI color output for CLI – zero external dependencies. Auto-detects terminal via std::io::IsTerminal.
init()– sets global color flag based on terminal detectionshort(),mid(),long()– tier-specific colors (red, yellow, green)dim(),bold(),cyan()– semantic colorstier_color()– dispatches to tier color by string namepriority_bar()– renders a 10-character bar (█████░░░░░) colored by priority level (green for 8+, yellow for 5-7, red for 1-4)
Colors are suppressed when stdout is not a terminal (e.g., piping to file). The --json flag bypasses color output entirely.
src/errors.rs
Structured error types for the HTTP API:
ApiError– serializable error withcodeandmessagefieldsMemoryErrorenum –NotFound,ValidationFailed,DatabaseError,Conflict- Implements
IntoResponsefor Axum, mapping to appropriate HTTP status codes - Implements
From<anyhow::Error>andFrom<rusqlite::Error> - Error sanitization:
DatabaseErrorresponses return a generic"Internal server error"message to clients, never leaking internal database error details. Detailed errors are logged server-side.
src/handlers/
All HTTP handlers for the 92 production .route(...) registrations / 78 unique URL paths (canonical count from CLAUDE.md §Architecture; counted via codegraph_search kind=route limit=100). The pre-Wave-1 monolithic src/handlers.rs (~17.8k LOC) is GONE — split into src/handlers/{mod,http,transport,federation_receive,hook_subscribers}.rs. State is the Db = Arc<Mutex<(Connection, PathBuf, ResolvedTtl, bool)>> extractor defined in src/handlers/transport.rs. Each handler acquires the lock, validates input via crate::validate::RequestValidator (#966 Wave-2 Tier-C1), performs DB operations through the SAL MemoryStore trait (src/store/), and returns JSON.
Key handlers:
create_memory/bulk_create– memory creation with deduplication (bulk limited to 1,000 items)get_memory/list_memories/update_memory/delete_memory– standard CRUDpromote_memory–POST /memories/{id}/promoteendpoint for promoting to long-termsearch/recall– FTS-powered search with sanitized queriesforget/consolidate– bulk operationsimport_memories– import with 1,000 item limitarchive_list/archive_restore/archive_purge/archive_stats– archive management endpoints- All ID path parameters are validated before database access
Note: HTTP handlers are tested via integration tests (
tests/integration.rs), not unit tests.
src/storage/ (alias db via pub use storage as db in src/lib.rs:52)
The sqlite storage layer. The pre-#961 monolithic src/db.rs is GONE — split into the per-domain modules under src/storage/ (CRUD, FTS5, recall scoring, GC, schema migrations at src/storage/migrations.rs, reflect at src/storage/reflect.rs). The legacy free-function names are preserved as the db::* alias for call-site backward compat. Post-#961 SAL boundary cleanup: new DB operations land on the MemoryStore trait in src/store/mod.rs FIRST; storage/* hosts primitives the sqlite adapter delegates to. Key functions:
| Function | Description |
|---|---|
open() |
Opens DB, sets WAL mode, creates schema, runs migrations |
insert() |
Upsert on (title, namespace) – never downgrades tier, keeps max priority |
get() |
Fetch by ID |
touch() |
Bump access count, extend TTL, auto-promote mid->long at 5 accesses, reinforce priority every 10 accesses. Uses BEGIN IMMEDIATE/COMMIT transaction for atomicity. |
update() |
Partial update of any fields |
delete() |
Delete by ID (links cascade) |
forget() |
Bulk delete by namespace + FTS pattern + tier |
list() |
List with filters: namespace, tier, priority, date range, tags, offset |
search() |
FTS5 AND search with 6-factor composite scoring |
recall() |
FTS5 OR search + touch + auto-promote + TTL extension |
find_contradictions() |
Find memories in same namespace with similar titles |
consolidate() |
Merge multiple memories, delete originals, aggregate tags and max priority. Uses BEGIN IMMEDIATE/COMMIT transaction for atomicity. |
sanitize_fts_query() |
Strips special characters and quotes tokens to prevent FTS injection |
create_link() / get_links() / delete_link() |
Memory linking (ON DELETE CASCADE) |
gc() |
Delete expired memories |
stats() |
Aggregate statistics (totals, by tier, by namespace, expiring soon, links, DB size) |
list_namespaces() |
List namespaces with memory counts |
export_all() / export_links() |
Full data export |
checkpoint() |
WAL checkpoint (TRUNCATE) for clean shutdown |
archive_memory() |
Move a memory to the archive table |
list_archived() |
List all archived memories |
restore_archived() |
Restore an archived memory to the active table |
purge_archive() |
Permanently delete all archived memories |
archive_stats() |
Archive statistics (count, size, date range) |
health_check() |
Verifies DB accessibility and FTS5 integrity |
Transaction safety: touch() and consolidate() use BEGIN IMMEDIATE to acquire a write lock upfront, preventing deadlocks and ensuring the entire read-modify-write cycle is atomic. This is critical for touch() because it reads the current access count, computes promotion/reinforcement logic, and writes back – all of which must be atomic under concurrent access.
FTS query sanitization: The sanitize_fts_query() function strips all FTS5 special characters (*, ", (, ), :, +, -, ~, ^, {, }, [, ], |, \) from user input and wraps each remaining token in double quotes. This prevents injection of FTS query syntax that could cause unexpected results or errors.
Migration error handling: The migration logic only ignores “duplicate column” errors (indicating the migration already ran). All other errors are propagated, ensuring real failures are caught early.
src/hnsw.rs
In-memory HNSW (Hierarchical Navigable Small World) vector index for approximate nearest-neighbor search. The VectorIndex struct provides insert, search, and remove operations on dense embeddings. When the index is small (below the HNSW threshold), it falls back to linear scan. The index has no persistence – it is rebuilt from the database on startup. This keeps the on-disk format simple (embeddings stored as BLOBs in SQLite) while providing fast in-memory ANN search during runtime.
src/toon.rs
TOON (Token-Oriented Object Notation) serializer. Converts JSON recall/search/list responses into the compact TOON wire format. The format spec is documented in the TOON Format Specification section below. Public API: memories_to_toon(), search_to_toon(). Compact mode emits a 6-field projection (id/tier/title/namespace/score/created_at); full mode emits the complete record.
src/config.rs
Tier configuration system + global runtime config. Parses ~/.config/ai-memory/config.toml, applies environment-variable overrides (AI_MEMORY_*), validates tier capabilities (keyword, semantic, smart, autonomous), and emits the immutable Config consumed by every other module. Includes TtlConfig (per-tier TTL + extension windows), archive_on_gc, embedding-model selection, Ollama URL, and feature gating that disables higher-tier code paths when the configured tier doesn’t permit them.
src/embeddings.rs
Embedding pipeline for semantic+ tiers. Loads HuggingFace sentence-transformer models (all-MiniLM-L6-v2 384-dim or nomic-embed-text-v1.5 768-dim) on first run via hf-hub, runs inference via Candle, generates dense vectors at insert time, and backfills missing embeddings on first startup. Vectors are stored as BLOBs in the memories.embedding column. Consumed by reranker.rs for hybrid recall and hnsw.rs for approximate nearest-neighbour indexing.
src/llm.rs
Provider-agnostic LLM client (#1067) for query expansion, auto-tagging, and contradiction detection. Two wire shapes — Ollama-native (/api/chat + /api/embed, no auth) and OpenAI-compatible (/v1/chat/completions + /v1/embeddings, Bearer auth). Backend selected by AI_MEMORY_LLM_BACKEND env var with 15 vendor aliases (xai, openai, anthropic, gemini, deepseek, kimi, qwen, mistral, groq, together, cerebras, openrouter, fireworks, lmstudio, plus the generic openai-compatible escape hatch). The struct name OllamaClient is preserved post-#1066 for call-site backward compat (rename to LlmClient is non-breaking and tracked separately). Vendor identifiers in this module are legitimate per the substrate-canonical-discipline carve-out enforced by scripts/check-vendor-literals.sh (#1200). Supplies the production implementation of the AutonomyLlm trait (see src/autonomy.rs). Prompts are kept short and structured to minimize token cost; failures are non-fatal — the curator and autonomy passes log and continue.
src/mine.rs
Retroactive conversation import — bulk-imports historical Claude / ChatGPT / Slack export files into ai-memory as backfilled memories. Each conversation becomes a single memory; metadata captures source, agent_id, and timestamps from the export. Used to seed memory before live capture is available.
src/reranker.rs
Hybrid recall algorithm. Blends the FTS5 keyword score and the embedding cosine similarity into a single ranking, applying configurable weighting and a 6-factor scoring formula (recency, priority, access count, tier weight, content match, namespace match). Returns a score field in every recall response so callers can audit ranking decisions.
src/identity/
Non-Human Identity (NHI) resolution for agent_id (split from the former src/identity.rs into per-domain modules: mod.rs, attest.rs, sign.rs, verify.rs, replay.rs, plus the #1558 additions sentinels.rs — reserved caller identities / RESERVED_AGENT_IDS — and keypair.rs — DAEMON_KEYPAIR_LABEL + daemon signing-keypair load). Centralises the precedence chain across CLI, MCP, and HTTP entry points so metadata.agent_id is uniformly populated. Public API: resolve_agent_id() (CLI/MCP), resolve_http_agent_id() (HTTP body + X-Agent-Id header), preserve_agent_id() (round-trip), process_discriminator() (stable per-process identifier). Default-id formats: ai:<client>@<hostname> (MCP), host:<hostname> (CLI) — both durable, pid-free since #1720 — and anonymous:req-<uuid8> (HTTP per-request fallback). Store-path agent attestation is required by default on the HTTP direct-write surface (#1751, surface-scoped by #1985): an unsigned HTTP POST /api/v1/memories (+/bulk) is rejected (403 ATTESTATION_FAILED), not stored as claimed. The MCP memory_store and CLI store operator-as-actor surfaces are permissive by default (an unsigned write lands claimed) — the surface is resolved by identity::attest::require_agent_attestation_for(WriteSurface). A write that presents a valid Ed25519 signature is stamped agent_attested (#626 Layer-3 — see identity::attest::stamp_attestation); a forged signature is rejected on every surface. =1 forces strict everywhere, =0 permissive everywhere.
src/curator/
Autonomous curator daemon (v0.6.1; split from the former src/curator.rs into mod.rs + pipeline.rs, candidates.rs, cluster.rs, compaction.rs, persist.rs, reflection_pass.rs). Runs a periodic sweep over stored memories, invoking auto_tag and detect_contradiction via the configured LLM and persisting results into each memory’s metadata. Complements the synchronous post-store hooks (#265). Hard cap on operations per cycle (default 100); skips internal _-prefixed namespaces; honours include/exclude lists; dry-run mode emits a report without touching rows; LLM errors are logged but never abort a cycle. v0.8.0 Pillar-2.5 (#1709) adds an LLM-free size-GC pass: when CompactionConfig.max_corpus_bytes (Option<i64>, default None = disabled) is set, each scanned namespace whose live corpus (length(title)+length(content)+length(metadata) summed) exceeds the cap has its lowest-value rows evicted (archived-before-deleted, restorable; least-durable tier then lowest priority/access_count/last_accessed_at first) until back under cap — pure deterministic SQL ranking, counted in CuratorReport.memories_evicted_size_gc, gated on !dry_run. Public API: CuratorConfig, CuratorReport, run_once(), run_daemon().
src/autonomy.rs
Full-autonomy loop — stacks on the curator daemon. Four passes beyond auto-tag:
- Consolidation — find near-duplicate memories in the same namespace (Jaccard ≥ 0.55, max cluster size 8), LLM-summarise into a single canonical memory, archive originals.
- Forgetting of superseded memories — when
metadata.confirmed_contradictionsis set, demote/forget the contradicted entry. - Priority feedback — nudge
priorityup for hot memories, down for cold ones (purely arithmetic, no LLM call). - Rollback log + self-report — every autonomous action lands in
_curator/rollback/<ts>(reversible) and every cycle in_curator/reports/<ts>.
Defines the AutonomyLlm trait so the curator can be unit-tested without a live Ollama instance. Public API: run_autonomy_passes(), persist_self_report(), reverse_rollback_entry(), RollbackEntry, AutonomyPassReport.
src/replication.rs
W-of-N quorum-write layer for the peer-mesh sync (v0.7 track C). Scaffolds the contract described in ADR-0001-quorum-replication.md. The QuorumWriter sits ABOVE the existing sync-daemon — deployments without --quorum-writes keep the v0.6.0 one-way push behaviour byte-for-byte. Public API: QuorumPolicy, QuorumWriter::commit, AckTracker. Emits metrics: replication_quorum_ack_total{result}, replication_quorum_failures_total{reason}, replication_clock_skew_seconds.
src/federation/
Federation autonomy (split from the former src/federation.rs into mod.rs + quorum.rs, peer.rs, peer_attestation.rs, receive.rs, sync.rs, signing.rs, push_dlq.rs, reflection_bookkeeping.rs, identity/) — wires the quorum primitives from replication into the HTTP write path (v0.7 track C, PR 2 of N). When ai-memory serve is started with --quorum-writes N --quorum-peers <urls>, every successful HTTP write fans out a 1-memory /api/v1/sync/push POST to each peer; the write returns OK only once W-1 peer acks land within --quorum-timeout-ms. Fewer acks → 503 quorum_not_met. Public API: FederationConfig, broadcast_store_quorum().
src/subscriptions.rs
v0.6.0.0 webhook subscriptions. Subscribers register a URL + shared secret + event/namespace/agent filters; matching events POST an HMAC-SHA256-signed JSON payload (header X-Ai-Memory-Signature: sha256=<hex>) over a fire-and-forget thread. SSRF hardening: http:// only to 127.0.0.0/8 or localhost; everywhere else requires https://; RFC1918 / RFC4193 / link-local hosts rejected unless allow_private_networks=true. Stored secret is SHA-256 of the plaintext (plaintext returned once at registration). Public API: Subscription, NewSubscription, insert(), delete(), list(), dispatch_event(), validate_url().
src/migrate.rs
Cross-backend migration tool — streams memories from one SAL backend to another (v0.7 track B, PR 2 of N). Gated behind --features sal; extended transparently by --features sal-postgres. Supported URLs: sqlite:///abs/path.db, sqlite://./relative.db, postgres://user:pass@host:port/db. CLI: ai-memory migrate --from <url> --to <url> [--batch 1000] [--dry-run] [--namespace foo]. Reads via MemoryStore::list, writes via MemoryStore::store with the source memory’s id verbatim — adapter upsert-on-id semantics make repeated migration idempotent.
src/metrics.rs
v0.6.0.0 Prometheus metrics, exposed at GET /metrics by the daemon. Minimal, non-invasive instrumentation — single global Registry, a handful of IntCounter / IntCounterVec / IntGauge / HistogramVec handles. Callers increment via typed helpers (record_store(tier, ok), record_recall(mode, latency_seconds), record_autonomy_hook(kind, ok), curator_cycle_completed(...)) rather than poking the registry directly so a future metrics-backend swap stays internal. Public API: Metrics (struct), registry(), render().
Database Schema
memories table
CREATE TABLE memories (
id TEXT PRIMARY KEY,
tier TEXT NOT NULL, -- 'short', 'mid', 'long'
namespace TEXT NOT NULL DEFAULT 'global',
title TEXT NOT NULL,
content TEXT NOT NULL,
tags TEXT NOT NULL DEFAULT '[]', -- JSON array
priority INTEGER NOT NULL DEFAULT 5, -- 1-10
confidence REAL NOT NULL DEFAULT 1.0, -- 0.0-1.0
source TEXT NOT NULL DEFAULT 'api', -- 'user', 'claude', 'hook', 'api', 'cli', etc.
access_count INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL, -- ISO 8601 / RFC3339
updated_at TEXT NOT NULL,
last_accessed_at TEXT,
expires_at TEXT, -- NULL for long-term
embedding BLOB -- dense vector (v3 migration, NULL if keyword tier)
);
-- Indexes
CREATE INDEX idx_memories_tier ON memories(tier);
CREATE INDEX idx_memories_namespace ON memories(namespace);
CREATE INDEX idx_memories_priority ON memories(priority DESC);
CREATE INDEX idx_memories_expires ON memories(expires_at);
-- Unique constraint enables upsert/deduplication behavior
CREATE UNIQUE INDEX idx_memories_title_ns ON memories(title, namespace);
memories_fts virtual table
CREATE VIRTUAL TABLE memories_fts USING fts5(
title, content, tags,
content=memories, content_rowid=rowid
);
Kept in sync via AFTER INSERT, AFTER DELETE, and AFTER UPDATE triggers on memories.
memory_links table
CREATE TABLE memory_links (
source_id TEXT NOT NULL REFERENCES memories(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES memories(id) ON DELETE CASCADE,
relation TEXT NOT NULL DEFAULT 'related_to',
created_at TEXT NOT NULL,
PRIMARY KEY (source_id, target_id, relation)
);
Relation types (nine at v0.8.0): related_to, supersedes,
contradicts, derived_from, reflects_on, derives_from,
decomposes_into, depends_on, advances. The
table shown above is the original core shape — at v0.7.0 each link row
also carries the temporal-validity columns (valid_from,
valid_until, observed_by) and attestation columns (signature,
attest_level, signed_at).
archived_memories table
CREATE TABLE archived_memories (
id TEXT PRIMARY KEY,
tier TEXT NOT NULL,
namespace TEXT NOT NULL DEFAULT 'global',
title TEXT NOT NULL,
content TEXT NOT NULL,
tags TEXT NOT NULL DEFAULT '[]',
priority INTEGER NOT NULL DEFAULT 5,
confidence REAL NOT NULL DEFAULT 1.0,
source TEXT NOT NULL DEFAULT 'api',
access_count INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
last_accessed_at TEXT,
expires_at TEXT,
archived_at TEXT NOT NULL,
archive_reason TEXT NOT NULL DEFAULT 'gc'
);
-- Indexes
CREATE INDEX idx_archived_memories_namespace ON archived_memories(namespace);
CREATE INDEX idx_archived_memories_archived_at ON archived_memories(archived_at);
Added in schema migration v3 -> v4 (shown in its original 16-column shape). Stores memories archived by GC before deletion; the columns mirror the memories table with additions including archived_at (timestamp of archival) and archive_reason (e.g., 'gc'). Schema v49 (#1025) added 14 more nullable columns (reflection_depth, memory_kind, citations, version, …) so archive → restore is lossless for the full v0.7.0 26-field Memory shape; v49+ also carries original_tier / original_expires_at, re-applied on restore.
The
CREATE TABLEblocks in this section show the original core columns for orientation — the canonical current DDL is theSCHEMAconst + migration ladder insrc/storage/mod.rs/src/storage/migrations.rs.
schema_version table
Tracks migration state. Current version: 78 (CURRENT_SCHEMA_VERSION in src/storage/migrations.rs).
Recall Scoring Formula
The recall function uses a 6-factor composite score to rank results:
score = (fts_rank * -1) -- FTS5 relevance (negated: lower = better in SQLite)
+ (priority * 0.5) -- Priority weight (1-10 -> 0.5-5.0)
+ (MIN(access_count, 50) * 0.1) -- Frequency bonus
+ (confidence * 2.0) -- Certainty weight (0.0-1.0 -> 0.0-2.0)
+ tier_boost -- long=3.0, mid=1.0, short=0.0
+ (1.0 / (1.0 + (julianday('now') - julianday(updated_at)) * 0.1)) -- Recency decay
The search function uses the same formula minus the tier boost.
Hybrid Recall Algorithm (semantic tier and above)
At the semantic tier and above, the reranker.rs module blends two scoring signals:
- Semantic score – cosine similarity between the query embedding and each memory’s stored embedding (0.0 to 1.0)
- Keyword score – the existing 6-factor FTS5 composite score, normalized to 0.0-1.0
The final score is a weighted blend: final = (semantic_weight * semantic_score) + ((1 - semantic_weight) * keyword_score). The semantic weight is adaptive by content length — 0.50 for short content (≤ 500 chars) sliding to 0.15 for long content (≥ 5000 chars) — because embeddings lose information on long text. Results from both pipelines are merged, deduplicated by memory ID, and sorted by the blended score.
Tier Configuration System
The config.rs module defines 4 feature tiers that gate functionality:
| Tier | Embeddings | LLM | Capability gating |
|---|---|---|---|
keyword |
No | No | FTS5-only recall; LLM-backed tools return a tier-requirement notice |
semantic |
Yes | No | Hybrid (semantic + keyword) recall; embedding-backed tools (e.g. memory_check_duplicate) active |
smart |
Yes | Yes | Adds LLM-backed expansion / auto-tag / contradiction detection |
autonomous |
Yes | Yes | Adds cross-encoder reranking + autonomous behaviors |
The tier gates capabilities (embedder / LLM / reranker), not the advertised tool count — the tool surface is selected separately by --profile (7 entries at core, 101 at full). Tier is set at startup via ai-memory mcp --tier <tier> and cannot be changed at runtime. Post-#1067 the LLM is provider-agnostic (AI_MEMORY_LLM_BACKEND), not Ollama-only. The memory_capabilities tool reports the active tier and which features are available, allowing AI clients to adapt their behavior.
Note: Configuration is loaded once at process startup. Changes to
config.tomlrequire restarting the ai-memory process (MCP server, HTTP daemon, or CLI) to take effect.
The recency decay factor ensures that recent memories rank higher when other factors are similar. A memory updated today gets a boost of ~1.0, a memory from 10 days ago gets ~0.5, and a memory from 100 days ago gets ~0.09.
TOON Format Specification
TOON (Token-Oriented Object Notation) is a token-efficient serialization format designed for LLM communication. It replaces JSON for recall, search, and list responses, reducing output size by 40-60% by declaring field names once as a header and listing values row by row with pipe delimiters.
The implementation is in src/toon.rs.
Structure Overview
A TOON response consists of three parts in order:
- Metadata line (optional) – key:value pairs for scalar fields
- Header line – declares field names once
- Data rows – one per object, values matching header column order
Metadata Line Syntax
Scalar (non-array) response fields are serialized as pipe-delimited key:value pairs on the first line:
count:3|mode:hybrid
If there are no metadata fields, this line is omitted entirely.
Header Line Syntax
The header declares the array name followed by field names in square brackets, pipe-delimited, ending with a colon:
memories[id|title|tier|namespace|priority|confidence|score|access_count|tags|source|created_at|updated_at]:
Field names appear exactly once in the entire output regardless of how many data rows follow. This is the primary source of token savings over JSON.
Data Row Syntax
Each data row contains values pipe-delimited in the same order as the header fields:
abc-123|PostgreSQL 16 config|long|infra|9|1.0|0.763|2|postgres,database|claude|2026-04-03T15:00:00+00:00|2026-04-03T15:00:00+00:00
- Strings are output as-is (unless they require escaping)
- Numbers (integers and floats) are output as their string representation
- Booleans are output as
1(true) or0(false) - Arrays (e.g., tags) are joined with commas:
postgres,database - Objects are output as the literal
[object] - Null/missing values are represented as an empty string (zero characters between the delimiters), e.g.,
abc||midmeans the second field is null
Escaping Rules
Two characters require escaping in TOON values:
| Character | Escaped As | Reason |
|---|---|---|
\| (pipe) |
\\| |
Pipe is the field delimiter |
\n (newline) |
\\n |
Newline is the row delimiter |
Escaping is only applied when the value actually contains a pipe or newline character. Values without these characters are output verbatim with no additional escaping.
Example: a title containing a pipe like A|B is serialized as A\|B in the data row.
Compact vs Full Mode
TOON supports two modes that differ only in which fields are included:
Full mode (12 fields):
memories[id|title|tier|namespace|priority|confidence|score|access_count|tags|source|created_at|updated_at]:
Compact mode (7 fields) – omits timestamps, confidence, access_count, and source for tighter output:
memories[id|title|tier|namespace|priority|score|tags]:
The MCP server defaults to compact mode (toon_compact). Clients can request "toon" for full mode or "json" for standard JSON via the format parameter on recall, search, and list tools.
Search Response Normalization
Search responses use a "results" key instead of "memories". The TOON serializer normalizes this internally – the output always uses the memories[...] header regardless of the source key.
Complete Parsing Example
Given this JSON response:
{
"memories": [
{"id": "abc-123", "title": "PostgreSQL config", "tier": "long", "namespace": "infra", "priority": 9, "score": 0.763, "tags": ["postgres", "db"]},
{"id": "def-456", "title": "Redis cache", "tier": "long", "namespace": "infra", "priority": 8, "score": 0.541, "tags": ["redis"]},
{"id": "ghi-789", "title": "Deploy notes", "tier": "mid", "namespace": "infra", "priority": 5, "score": 0.320, "tags": []}
],
"count": 3,
"mode": "hybrid"
}
TOON compact output:
count:3|mode:hybrid
memories[id|title|tier|namespace|priority|score|tags]:
abc-123|PostgreSQL config|long|infra|9|0.763|postgres,db
def-456|Redis cache|long|infra|8|0.541|redis
ghi-789|Deploy notes|mid|infra|5|0.32|
To parse TOON:
- Read the first line. If it does not start with a bracket-containing identifier (e.g.,
memories[), parse it as metadata: split on|, then split each segment on:to get key-value pairs. - Read the header line. Extract the array name and field list: strip the trailing
:, extract the portion inside[...], and split on|to get the ordered field names. - Read each subsequent non-empty line as a data row. Split on
|(respecting\|escapes), mapping each positional value to the corresponding header field name. - Unescape
\|to|and\nto newline in each value. Empty values represent null/missing fields.
API Reference
Base URL: http://127.0.0.1:9077/api/v1
All responses are JSON. Error responses include {"error": "message"}. Database errors are sanitized – clients receive "Internal server error" instead of raw SQLite error details.
The HTTP API exposes 92 production .route(...) registrations / 78 unique URL paths (canonical count via codegraph codegraph_search kind=route limit=100 filtered to src/lib.rs excluding the #[cfg(test)]-gated test-only routes; multi-line-aware path extraction via awk '/\.route\(/{in=1}in&&/"\/[^"]*"/{match($0,/"\/[^"]*"/);print substr($0,RSTART,RLENGTH);in=0}' src/lib.rs | sort -u; v0.6.3.1 baseline of 50 and v0.6.3 baseline of 42 are frozen on the evidence page).
Health Check
GET /health
Deep health check: verifies DB is readable and FTS5 integrity-check passes.
Response (200): {"status": "ok", "service": "ai-memory"}
Response (503): {"status": "error", "service": "ai-memory"}
Create Memory
POST /memories
Content-Type: application/json
{
"title": "Project uses Axum",
"content": "The HTTP server is built with Axum 0.8.",
"tier": "mid",
"namespace": "ai-memory",
"tags": ["rust", "web"],
"priority": 6,
"confidence": 1.0,
"source": "api",
"expires_at": "2026-04-06T00:00:00Z",
"ttl_secs": 86400
}
Response (201):
{
"id": "a1b2c3d4-...",
"tier": "mid",
"namespace": "ai-memory",
"title": "Project uses Axum",
"potential_contradictions": ["id1", "id2"]
}
Defaults: tier=mid, namespace=global, priority=5, confidence=1.0, source=api.
Optional: expires_at (RFC3339), ttl_secs (overrides tier default). Deduplicates on title+namespace (upsert).
Bulk Create
POST /memories/bulk
Content-Type: application/json
[
{"title": "Memory 1", "content": "..."},
{"title": "Memory 2", "content": "..."}
]
Response: {"created": 2, "errors": []}
Limited to 1,000 items per request.
Get Memory
GET /memories/{id}
Response:
{
"memory": { ... },
"links": [ ... ]
}
Update Memory
PUT /memories/{id}
Content-Type: application/json
{
"content": "Updated content",
"priority": 8,
"expires_at": "2026-06-01T00:00:00Z"
}
All fields are optional. Only provided fields are updated. Validated before write.
Delete Memory
DELETE /memories/{id}
Response: {"deleted": true}. Links are cascade-deleted.
Promote Memory
POST /memories/{id}/promote
Promotes a memory to long-term tier and clears its expiry.
Response: {"promoted": true}
List Memories
GET /memories?namespace=my-app&tier=long&limit=20&offset=0&min_priority=5&since=2026-01-01T00:00:00Z&until=2026-12-31T23:59:59Z&tags=rust
All query parameters are optional. The limit is capped at
max_page_size (compiled default 1000; [limits].max_page_size /
AI_MEMORY_MAX_PAGE_SIZE).
Response: {"memories": [...], "count": 5}
Search (AND semantics)
GET /search?q=database+migration&namespace=my-app&tier=mid&limit=10&since=...&until=...&tags=...
Response: {"results": [...], "count": 3, "query": "database migration"}
Uses 6-factor scoring (without tier boost). Queries are sanitized to prevent FTS injection.
Recall (OR semantics + touch)
GET /recall?context=auth+flow+jwt&namespace=my-app&limit=10&tags=auth&since=2026-01-01T00:00:00Z&until=2026-12-31T23:59:59Z
Or via POST:
POST /recall
Content-Type: application/json
{"context": "auth flow jwt", "namespace": "my-app", "limit": 10}
Response: {"memories": [...], "count": 5}
Recall automatically: bumps access_count, extends TTL, and auto-promotes mid-tier memories with 5+ accesses to long-term. The touch operation is transactional.
Forget (Bulk Delete)
POST /forget
Content-Type: application/json
{"namespace": "my-app", "pattern": "deprecated API", "tier": "short"}
At least one field is required. Pattern uses FTS matching (sanitized). Response: {"deleted": 3}
Consolidate
POST /consolidate
Content-Type: application/json
{
"ids": ["id1", "id2", "id3"],
"title": "Auth system summary",
"summary": "JWT with refresh tokens, RBAC middleware, Redis sessions.",
"namespace": "my-app",
"tier": "long"
}
Requires 2-100 IDs. Deletes source memories, creates new with aggregated tags and max priority. The entire operation is transactional. Response (201): {"id": "new-id", "consolidated": 3}
Create Link
POST /links
Content-Type: application/json
{"source_id": "id1", "target_id": "id2", "relation": "related_to"}
Relations: related_to, supersedes, contradicts, derived_from. Self-links rejected. Response (201): {"linked": true}
Get Links
GET /links/{id}
Response: {"links": [{"source_id": "...", "target_id": "...", "relation": "...", "created_at": "..."}]}
Namespaces
GET /namespaces
Response: {"namespaces": [{"namespace": "my-app", "count": 42}]}
Admin-gated at v0.7.0 (#945) — as are GET /stats, POST /gc,
GET /export, POST /import, POST /forget, GET /agents,
GET /taxonomy, GET /archive, GET /archive/stats, and the
/skill/* routes. See docs/API_REFERENCE.md §”Admin-gated
endpoints”.
Stats
GET /stats
Response:
{
"total": 150,
"by_tier": [{"tier": "long", "count": 80}, ...],
"by_namespace": [{"namespace": "my-app", "count": 42}, ...],
"expiring_soon": 5,
"links_count": 12,
"db_size_bytes": 524288
}
Garbage Collection
POST /gc
Response: {"expired_deleted": 3}
Export
GET /export
Response: full JSON dump of all memories and links with exported_at timestamp.
Import
POST /import
Content-Type: application/json
{"memories": [...], "links": [...]}
Validates each memory before import. Limited to 1,000 memories per request. Response: {"imported": 50, "errors": []}
Error Code Reference
Structured error codes returned by the HTTP API and MCP server:
| Code | HTTP Status | Description |
|---|---|---|
NOT_FOUND |
404 | Memory or resource not found |
VALIDATION_FAILED |
400 | Invalid input parameters |
DATABASE_ERROR |
500 | SQLite or internal error |
CONFLICT |
409 | Duplicate or conflicting operation |
The baseline HTTP error envelope is {"error": "<message>"} (message
strings centralised in src/errors.rs::msg); typed classes
additionally carry a code field (e.g. {"code": "ATTESTATION_FAILED",
"error": …} — the full code vocabulary lives in
src/errors.rs::error_codes, which also includes v0.7.0 additions like
REFLECTION_DEPTH_EXCEEDED, GOVERNANCE_REFUSED, QUOTA_EXCEEDED).
DATABASE_ERROR-class responses are sanitized – clients receive a
generic internal-server-error message; detailed errors are logged
server-side only.
CLI Reference
Global flags:
--db <path>– database path (default:ai-memory.db, env:AI_MEMORY_DB)--json– output as machine-parseable JSON
serve
Start the HTTP daemon (92 route registrations / 78 unique URL paths).
ai-memory serve --host 127.0.0.1 --port 9077
mcp
Run as an MCP tool server over stdio. This is the primary integration path for any MCP-compatible AI client. The --profile full surface advertises 101 entries (100 callable memory tools + the always-on memory_capabilities bootstrap); the default --profile core ships 7 + the bootstrap.
ai-memory mcp
ai-memory mcp --tier semantic # default
ai-memory mcp --tier smart # enables LLM-powered tools (any backend via AI_MEMORY_LLM_BACKEND, #1067)
Reads JSON-RPC from stdin, writes responses to stdout. Logs to stderr. Correctly handles notifications (no response sent). Works with any MCP-compatible client (Claude AI, OpenAI ChatGPT, xAI Grok, META Llama, etc.).
store
ai-memory store \
-T "Title" \
-c "Content" \
--tier mid \
--namespace my-app \
--tags "tag1,tag2" \
--priority 7 \
--confidence 0.9 \
--source claude \
--expires-at "2026-04-15T00:00:00Z" \
--ttl-secs 86400
Use -c - to read content from stdin. Validates all fields before writing. --expires-at sets an explicit expiration timestamp (RFC3339). --ttl-secs sets a TTL in seconds (overrides tier default).
update
ai-memory update <id> -T "New title" -c "New content" --priority 8 --expires-at "2026-06-01T00:00:00Z"
The --expires-at flag sets or changes the expiration on an existing memory.
recall
ai-memory recall "search context" --namespace my-app --limit 10 --tags auth --since 2026-01-01T00:00:00Z
search
ai-memory search "exact terms" --namespace my-app --tier long --limit 20 --since 2026-01-01 --until 2026-12-31 --tags rust
get
ai-memory get <id>
Shows the memory plus all its links.
list
ai-memory list --namespace my-app --tier mid --limit 50 --offset 0 --since 2026-01-01 --until 2026-12-31 --tags devops
The --offset flag enables pagination. Use with --limit to page through results.
delete
ai-memory delete <id>
promote
ai-memory promote <id>
Promotes to long-term and clears expiry.
forget
ai-memory forget --namespace my-app --pattern "old stuff" --tier short
At least one filter is required.
link
ai-memory link <source-id> <target-id> --relation supersedes
Relation types: related_to (default), supersedes, contradicts, derived_from. Self-links rejected.
consolidate
ai-memory consolidate "id1,id2,id3" -T "Summary title" -s "Consolidated content" --namespace my-app
gc
ai-memory gc
stats
ai-memory stats
namespaces
ai-memory namespaces
export / import
ai-memory export > backup.json
ai-memory import < backup.json
Export includes memories and links. Import validates each memory and skips invalid ones.
resolve
Resolve a contradiction by marking one memory as superseding another.
ai-memory resolve <winner_id> <loser_id>
Creates a “supersedes” link from winner to loser. Demotes the loser (priority=1, confidence=0.1). Touches the winner (bumps access count).
shell
Interactive REPL for browsing and managing memories.
ai-memory shell
REPL commands: recall <ctx>, search <q>, list [ns], get <id>, stats, namespaces, delete <id>, help, quit. Color output with tier labels and priority bars.
sync
Sync memories between two database files.
ai-memory sync <remote.db> --direction pull|push|merge
pull– import all memories from remote into localpush– export all local memories to remotemerge– bidirectional sync (both databases get all memories)
Uses dedup-safe upsert (title+namespace). Links are synced alongside memories.
auto-consolidate
Automatically group and consolidate memories.
ai-memory auto-consolidate [--namespace <ns>] [--short-only] [--min-count 3] [--dry-run]
Groups memories by namespace+primary tag. Groups with >= min_count members are consolidated into one long-term memory. Use --dry-run to preview.
mine
Import memories from historical conversations (Claude, ChatGPT, Slack exports).
ai-memory mine --format claude <path-to-export>
ai-memory mine --format chatgpt <path-to-export>
ai-memory mine --format slack <path-to-export>
Takes --format to specify the input file format (claude, chatgpt, slack) and a path to the export file or directory.
man
Generate roff man page to stdout.
ai-memory man # print roff to stdout
ai-memory man | man -l - # view immediately
completions
ai-memory completions bash
ai-memory completions zsh
ai-memory completions fish
Adding New Features
- Add the model in
src/models/– new struct or new fields on existing structs (wire keys viasrc/models/field_names.rsconsts) - Add validation in
validate.rs– new validation function (or aRequestValidatormethod) - Add the DB operation on the SAL
MemoryStoretrait insrc/store/mod.rsFIRST (post-#961), implemented onSqliteStore(usually delegating to acrate::storage::*primitive) ANDPostgresStore— a sqlite-only free function will 501 on the postgres route gate - Add the HTTP handler in the matching per-domain module under
src/handlers/ - Add the route: one path const in
src/handlers/routes.rs(the route-path SSOT) + one.route(handlers::routes::<CONST>, …)registration insrc/lib.rs::build_router_with_timeout, then bumpEXPECTED_PRODUCTION_ROUTES_COUNT/EXPECTED_PRODUCTION_UNIQUE_PATHS_COUNTand the postgres-gate allowlist (src/handlers/postgres_gate.rs) - Add the CLI command – new variant in the
Commandenum insrc/daemon_runtime.rs, anArgsstruct (typically undersrc/cli/commands/), a dispatch arm, and bumpEXPECTED_CLI_SUBCOMMANDS_* - Add the MCP tool (post-v0.7.0 #987 D1.x): define
<ToolName>Request(schemarsJsonSchemaderive; NOdeny_unknown_fieldsper the #1052 wire-truthfulness pin) +<ToolName>Tool(zero-sized) withimpl McpToolinsrc/mcp/tools/<name>.rs; register ONERegisteredTool::of::<…>()line inregistered_tools()insrc/mcp/registry.rs; add atool_names::*const +ALLslice entry (census/pin tests track the count automatically againstProfile::full().expected_tool_count()); add the handler + dispatch arm insrc/mcp/mod.rs::handle_request(); add ad1_6_987_testsparity-test mod. The pre-D1.6 step “add JSON definition intool_definitions()” is gone —tool_definitions()is now a four-line iteration. - Add tests under
tests/(integration) and in the module’s unit-test suite
Testing
The project has 1,886 lib tests + 49+ integration tests at 93.84% line coverage as of v0.6.3.1 (was 1,600 lib / 93.08% on v0.6.3). v0.6.3 baseline numbers are frozen on the evidence page; v0.6.3.1 deltas are documented in the release notes. Modules each carry their own unit-test suite; integration tests live under tests/.
# Run all tests
cargo test
# Run with output
cargo test -- --nocapture
# Run a specific test
cargo test test_name
# Check formatting
cargo fmt --check
# Run clippy
cargo clippy -- -D warnings
Integration tests run through the CLI binary, creating temporary databases for isolation.
Benchmarks
Criterion (microbenchmarks)
Criterion benchmarks are in benches/recall.rs. They test insert, recall, and search performance at 1,000 memories scale.
cargo bench
# recall/short_query, recall/medium_query, recall/long_query
# search/simple_search, search/filtered_search
# insert/store_memory
LongMemEval (end-to-end accuracy)
The benchmarks/longmemeval/ directory evaluates recall accuracy against the LongMemEval dataset (ICLR 2025). Four harnesses are available:
| Harness | Strategy | R@5 | Speed |
|---|---|---|---|
harness_99.py --no-expand |
Parallel FTS5, 10 cores | 97.0% | 232 q/s (2.2s) |
harness_99.py |
LLM expansion + parallel FTS5 | 97.2% (Gemma 4, API venue; historical gemma3:4b 97.8%) |
142 q/s (3.5s) |
harness_fast.py |
Single-process native SQLite | 96.2% | 57 q/s (8.8s) |
harness.py |
CLI subprocess per operation | 96.2% | 1.2 q/s (414s) |
Published expansion anchor (per the #1975 ruling, 2026-07-10): 97.2% R@5, 99.6% R@10, 99.8% R@20 — measured 2026-05-31 with OpenRouter google/gemma-4-26b-a4b-it (500 questions, 0 expansion failures). The historical gemma3:4b best (97.8% R@5, 489/500) is retained in benchmarks/longmemeval/results.md as the compiled-default-model row, no longer the headline. The keyword-tier 97.0% is LLM-independent.
# Quick run (keyword, ~2s)
python3 benchmarks/longmemeval/harness_99.py \
--dataset-path /tmp/LongMemEval --variant S --no-expand --workers 10
# Full run with LLM expansion (requires Ollama + gemma3:4b)
python3 benchmarks/longmemeval/harness_99.py \
--dataset-path /tmp/LongMemEval --variant S --workers 10
See benchmarks/longmemeval/README.md for full replication instructions.
CI/CD Pipeline
GitHub Actions CI runs on every push and pull request. The four cargo
gates every PR must pass (see CLAUDE.md §Build & Test Commands):
- Check formatting –
cargo fmt --check - Clippy –
cargo clippy -- -D warnings -D clippy::all -D clippy::pedantic - Run tests –
AI_MEMORY_NO_CONFIG=1 cargo test - Dependency audit –
cargo audit
Plus the script-based HARD-BLOCK gates wired into
.github/workflows/c8-precheck.yml (#1174 PR10):
- C8 caller-context allowlist –
scripts/qc-codegraph-precheck.sh(no newCallerContext::for_agent("<literal>")/for_adminsites outsidescripts/qc-codegraph-allowlists/). - Vendor-monoculture + SECS_PER_* –
scripts/check-vendor-literals.sh(vendor identifiers only in the documented substrate carve-outs; no raw 3600/86400/604800Duration::from_secsliterals — useSECS_PER_HOUR/SECS_PER_DAY/SECS_PER_WEEK). - Hardcoded-literal duplication ratchet –
scripts/check-hardcoded-literals.shagainst the frozen baseline. - Docs-vs-SSOT drift gate –
scripts/check-docs-vs-ssot.sh(narrative counts in the docs must match the canonical Rust consts: schema version, tool counts, route/path counts, CLI subcommand counts, Memory field count, link-relation count, …). - L3-boundary perma-ban –
scripts/check-l3-boundary.sh(§25.3 S5 / RQ-10, #1853 — no reintroduction of the banned Layer-3 boundary crossings). - Cloud-init ASCII gate –
scripts/check-cloud-init-ascii.sh(#1880 — cloud-init templates must remain ASCII-only).
Each script gate also runs a --self-test step proving it is
load-bearing. Coverage floors are enforced per-module from
coverage/thresholds.toml (.github/workflows/coverage.yml) —
thresholds rise across releases, never fall. Additional workflows:
bench.yml (p95 budgets), token-budget.yml (tools/list token
ceiling), tool-count-drift.yml, fuzz.yml, mobile-runtime.yml.
Release Pipeline
On tag push (e.g., v0.2.0):
- Builds release binaries for
x86_64-unknown-linux-gnuandaarch64-apple-darwin - Packages as
.tar.gz - Creates a GitHub Release with the artifacts
Building from Source
git clone https://github.com/alphaonedev/ai-memory-mcp.git
cd ai-memory
# Debug build
cargo build
# Release build (optimized, stripped)
cargo build --release
# The binary is at target/release/ai-memory
New Dependencies (v0.4.0)
candle-core,candle-nn,candle-transformers– HuggingFace Candle for local embedding model inferencehf-hub– HuggingFace Hub client for downloading embedding modelstokenizers– HuggingFace tokenizers for text preprocessingreqwest– HTTP client for Ollama API communication (LLM inference)
All dependencies are always compiled; tier selection controls which features are activated at runtime.
Release profile settings (from Cargo.toml):
opt-level = 3strip = true(removes debug symbols)lto = "thin"(link-time optimization)
Working Under an Autonomous Campaign
When this repository is being driven by the campaign Python harness
(at alphaonedev/agentic-mem-labs/tools/campaign/, Apache 2.0 ©
AlphaOne LLC), the development workflow is the same workflow described
above plus the constraints in
ENGINEERING_STANDARDS.md §7.
Concurrent operation
- A live campaign holds a designated
release/vX.Y.Zbranch as its exclusive merge target. Human contributors can still open PRs againstdevelop(or pre-existing release branches) without conflict. -
The campaign records every decision and PR to its ai-memory namespace (named after the campaign, e.g.
campaign-v063). To see what the agent has done in the current iteration window:ai-memory --db ~/.claude/ai-memory.db list --namespace campaign-v063 -
The append-only audit trail lives on a
campaign-log/vX.Y.Zbranch ofagentic-mem-labs. One markdown report per iteration:git -C ~/agentic-mem-labs-log log --oneline campaign-log/v0.6.3
Memory namespace as the campaign’s operating substrate
Every campaign uses an ai-memory namespace named after the campaign. The namespace contains: the campaign’s overview/scope/hard rules, approvals, code-quality standards, Engineering Standards alignment, a snapshot of open issues + PRs at campaign start, one summary memory per iteration, decisions, blockers, and “future”/deferred items.
Treat the namespace as both the agent’s working memory and the
historical record. After a campaign ends, the namespace is preserved
indefinitely (tier = long).
What human reviewers should focus on under a campaign
PRs from campaign/<slug> branches into release/vX.Y.Z get
gh pr merge --squash --delete-branch once CI is green. The agent
self-reviews quality (clippy pedantic, fmt, tests). For human
spot-checks: charter alignment, hard-rule compliance, test coverage,
audit consistency on the campaign-log/vX.Y.Z branch.
The campaign is a complement to human development, not a replacement. For everything outside the active charter — bug triage, design ADRs, release cuts, dependency upgrades, security response — humans still own the work.