ai-memory Troubleshooting

Common errors, causes, and fixes. If your scenario isn’t here, check journalctl -u ai-memory --since "1 hour ago" first, then open an issue at https://github.com/alphaonedev/ai-memory-mcp/issues.

Startup

“database is locked”

Symptom: ai-memory <cmd> reports Error: database is locked.

Cause: Another ai-memory process (CLI, daemon, curator, or sync) holds the SQLite write lock. SQLite uses a process-global lock; two writers can’t coexist.

Fix:

List any running ai-memory processes: ps -ef | grep ai-memory.
If a daemon is running, route your operation through it (HTTP API or MCP) instead of the CLI.
If you suspect a stale lock, stop every process and check the WAL companion files next to the database (<db>.db-wal / <db>.db-shm); they are recovered automatically on the next open.
The busy_timeout is a compiled 5 s PRAGMA (src/storage/connection.rs) — it is not operator-tunable. For long-running imports that keep hitting the lock, stop the competing writer (or route the import through the daemon) instead.

“could not find embedding model”

Symptom: First recall or search hangs then fails. Log shows hf-hub download errors or candle model-load failure.

Cause: ai-memory downloads the embedding model lazily on first semantic recall. First run needs ~90 MB for all-MiniLM-L6-v2 (or ~270 MB for nomic-embed-text-v1.5 on smart/autonomous tiers). Network or disk issues interrupt the download.

Fix:

Confirm outbound access to huggingface.co.
Check ~/.cache/huggingface/hub/ for a partial download. Delete the model directory and retry.
For air-gapped environments, pre-stage the model via huggingface-cli download sentence-transformers/all-MiniLM-L6-v2.
If you don’t need semantic recall, run with --tier keyword — FTS5-only, zero model load.
Post-#1598, CPU-only / egress-restricted hosts can skip the local model entirely: point [embeddings] at an API backend (any #1067 alias, or openai-compatible for a self-hosted TEI/vLLM/llama.cpp /v1/embeddings endpoint). See the enterprise reference architectures.

Semantic recall degraded to keyword — “embedder init failed” / “EMBEDDER LOAD FAILED” (#1593 / #1598)

Symptom: stderr shows embedder init failed (backend=…, model=…, url=…, source=…): … — semantic recall DEGRADED to keyword (#1143, #1593, #1598) (MCP stdio) or the ERROR-level EMBEDDER LOAD FAILED marker (daemon). memory_capabilities reports embedder_loaded: false and recall_mode_active: "degraded".

Cause: embedder construction failed — for API backends usually a wrong base_url, a missing/rejected API key, or no network egress; for local backends a HuggingFace download or memory issue. This is the fail-closed posture (#1593): the substrate keeps serving keyword/FTS recall and NEVER silently routes embeddings through the chat LLM client. #1594 makes the degradation truthful at request time too — a remote embedder whose endpoint starts failing flips embedder_loaded to false in live memory_capabilities output.

Fix:

Run ai-memory doctor and inspect the Embeddings Reachability (#1598) section. It probes the resolved endpoint (ollama GET /api/tags; API backends POST /embeddings with the resolved Bearer key) and reports backend/model/base_url/config_source/key_source plus HTTP status — auth (401/403), rate-limit (429), vendor outage (5xx), wrong base_url (4xx-other), or network/DNS.
If config_source = compiled-default, no operator embeddings config exists anywhere — set AI_MEMORY_EMBED_BACKEND or write an [embeddings] section (see docs/CONFIG_SCHEMA.md).
If key_source = error(...), fix the referenced env var or key file perms (mode 0400 required for api_key_file).
If the section carries a gpu_policy WARN, you resolved backend = ollama on a host with no compatible GPU — operator policy is API embeddings on CPU-only nodes; switch the backend or move the workload to a GPU node.
To silence the degradation deliberately, set tier = "keyword".

“port 9077 already in use”

Symptom: ai-memory serve fails immediately with Address already in use.

Cause: Another ai-memory serve, a development tool, or an old process from a previous shutdown.

Fix:

# Find the offender
lsof -i :9077
# or
ss -tlpn | grep 9077

# Bind to a different port
ai-memory serve --port 19077

MCP integration

Claude Code / Desktop / Cursor don’t see ai-memory tools

Symptom: Restarted the IDE after adding the MCP config; no memory_* tools appear in the tool list.

Causes + fixes:

Wrong config path. Verify:
- Claude Code: mcpServers in ~/.claude.json (user scope) or .mcp.json in the project root (NOT settings.json).
- Claude Desktop: ~/Library/Application Support/Claude/claude_desktop_config.json (macOS).
- Cursor: Settings → Features → MCP.
JSON syntax error. Paste the config into jq '.' file.json to validate.
ai-memory not on PATH. MCP servers inherit the IDE’s PATH. Absolute path the command: "command": "/usr/local/bin/ai-memory".
Old IDE version. MCP support landed in Claude Desktop 0.7+, Cursor 0.45+, Claude Code 1.0+.
Server crashed on stdio. Run ai-memory mcp manually in a terminal; you should see it waiting on stdin. If it exits immediately, check stderr for errors.

“tools/list returned 31 tools, expected 34”

Symptom: Integration test fails on MCP tool count.

Cause: A new tool landed in src/mcp/tools/<name>.rs + registered_tools() in src/mcp/registry.rs (#987 D1.6 recipe; pre-#1066 the source was the monolithic src/mcp.rs) without updating the tool-count assertion. Harmless — it’s a test that locks the tool count to prevent accidental removal. The canonical post-#1187 source is crate::mcp::tool_names::* consts + Profile::full().expected_tool_count() in src/profile.rs. Update the assertion to match the new count and add the tool’s tool_names::* const reference where the test enumerates expected tools.

MCP tool returns “no memories found” but `ai-memory list` shows them

Cause: The MCP server and the CLI point at different databases.

Fix: Every entry point reads AI_MEMORY_DB. Set it consistently:

// Claude Code ~/.claude.json (user scope) or .mcp.json (project scope)
{
  "mcpServers": {
    "ai-memory": {
      "command": "ai-memory",
      "args": ["mcp"],
      "env": { "AI_MEMORY_DB": "/Users/you/ai-memory.db" }
    }
  }
}

Autonomy / curator

“no LLM client configured” in curator report

Symptom: ai-memory curator --once --json report shows "errors": ["no LLM client configured"] and zero operations.

Cause: The feature tier doesn’t wire an LLM, or the configured backend is unreachable.

Fix (v0.7.x — preferred):

Run ai-memory doctor and inspect the LLM Reachability (#1146) section. It probes the configured backend (Ollama, xAI, OpenAI, Anthropic, etc.) and reports the resolved backend/model/base_url/config_source/key_source plus HTTP status. The severity tag (INFO / WARN / CRIT) tells you whether the issue is auth (401), rate-limit (429), vendor outage (5xx), wrong base_url (4xx-other), or network/DNS/TLS.
If config_source = compiled-default, no operator LLM config is present anywhere. Either set AI_MEMORY_LLM_BACKEND (env) or write a [llm] section in ~/.config/ai-memory/config.toml (see docs/CONFIG_SCHEMA.md).
If key_source = error(...), the resolved API key (api_key_env / api_key_file) couldn’t be read — fix the referenced env var or file perms (0400 required for api_key_file by default).
Check the feature tier: curator has no --tier flag — it reads the tier field from config.toml. Set tier = "smart" (or "autonomous") there and re-run.

Fix (legacy v0.6.x flat-field config):

If you’re still on v0.6.x flat fields (llm_model, ollama_url), the deprecation WARN at config-load tells you it’s time to migrate:

ai-memory config migrate --dry-run    # preview the v2 shape
ai-memory config migrate              # apply with timestamped .bak
ai-memory doctor                      # verify LLM Reachability

The legacy fields continue to work in v0.7.x but will be removed in v0.8.0.

Curator cycle times are long (> 10 min)

Cause: Each eligible memory triggers an Ollama round-trip (~1–5 s). With a large corpus and --max-ops 100, a cycle can take 5–10 min.

Fix:

Lower --max-ops to fit your cycle budget.
Enable Ollama KV compression (OLLAMA_KV_CACHE_TYPE=q4_0) to speed up each call. See docs/RUNBOOK-ollama-kv-tuning.md.
Run --daemon --interval-secs 3600 and let it catch up slowly.

Curator made a bad call — how to undo it

# See the last 20 actions
ai-memory list --namespace _curator/rollback --limit 20

# Reverse a specific one
ai-memory curator --rollback <id>

# Reverse the last 5
ai-memory curator --rollback-last 5

Reversed entries are tagged _reversed, not deleted — the audit trail is preserved.

HTTP API

“401 missing or invalid API key”

Cause: The daemon has an api_key configured (the api_key field in config.toml — there is no --api-key serve flag). Pass the key:

curl -H "X-API-Key: YOUR_KEY" http://127.0.0.1:9077/api/v1/stats
# or (DEPRECATED #1574 — URL keys leak into access/proxy logs;
# accepted with a WARN at v0.7.0, slated for v0.8 rejection)
curl 'http://127.0.0.1:9077/api/v1/stats?api_key=YOUR_KEY'

/api/v1/health is always exempt — use it as a reachability probe.

“500 Internal Server Error” with no body

Cause: Error-sanitisation strips stack traces from production responses to avoid leaking internals.

Fix: Check the daemon log (journalctl -u ai-memory) for the full error. If running in foreground, look at stderr. Raise verbosity with RUST_LOG=ai_memory=debug.

“503 quorum_not_met” on every write

Cause: Federation is configured (--quorum-writes N --quorum-peers …) but peers are unreachable or slow.

Diagnosis:

Body carries {"got":X,"needed":Y,"reason":"…"}. reason:
- unreachable — no peers responded at all (network / DNS).
- timeout — some peers acked but not enough before --quorum-timeout-ms.
- id_drift — peers returned different memory ids (replication divergence).
Curl each peer directly: curl https://peer-a:9077/api/v1/health.
Check peer mTLS allowlist — your fingerprint may not be listed.

Fix: lower --quorum-writes temporarily, restore peer connectivity, restart with the original setting. For timeout on a cross-region mesh, raise --quorum-timeout-ms — the 2000 ms default is same-DC-tuned; WAN meshes need 5000-10000 ms (the do-1461 3-region reference deploy uses FED_QUORUM_TIMEOUT_MS=8000; see #1565). The write commits locally first, so the longer wait affects only the synchronous-durability gate.

Sync / federation

Memories stop syncing between peers

Cause: Multiple possibilities.

Diagnosis:

On each peer: ai-memory sync-daemon must be running. systemctl status ai-memory-sync or check the log.
Divergence check: run ai-memory stats on each peer and compare the total counts; the per-peer vector clock lives in the sync_state table (sqlite3 <db> "SELECT * FROM sync_state").
mTLS fingerprint drift: if you rotated certs, the allowlist must be regenerated on every receiver.
--batch-size 500 default may be too small for a backlog. Bump to 5000 temporarily.

Split-brain: two peers diverged

Cause: Network partition. Both halves accepted writes. Now they disagree on (title, namespace) content.

Fix: Decide which side is authoritative. On that side, run ai-memory export > snapshot.json. On the other side, ai-memory import --trust-source < snapshot.json. The upsert on (title, namespace) will overwrite the divergent copies with the authoritative ones.

Per-namespace conflict resolution is an open work item (sync-phase Layer 2b).

Federation push-DLQ backlog / quarantined rows

Symptom: the daemon logs replay: row N quarantined after 100 attempts (ceiling 100) and/or the federation_push_dlq_depth gauge stays high.

Failed quorum pushes land in the federation_push_dlq table and a background worker replays them (oldest first, batches per tick). The per-tick batch is adaptive (#1579 B5): it scales with the live backlog up to a cap (min(backlog, cap), floor 64; cap default 2048, operator-tunable via AI_MEMORY_FED_DLQ_REPLAY_MAX_BATCH), so a bulk backlog drains at thousands of rows/min instead of the historical fixed-64 ceiling of 128 rows/min/peer. Replays reuse the daemon’s pooled federation connections (no per-row TLS handshake), and the captured payload ships the source embedding vector when one was available at enqueue time (#1566), so a healthy receiver applies a replayed row in milliseconds — receivers no longer re-embed synchronously on receive (the pre-#1566 ~1 s/row embed-on-receive that inflated replay latency and quorum deadlines is gone; rows without a usable shipped vector are embedded by a background task after the ack). Rows that fail MAX_REPLAY_ATTEMPTS (100) times are quarantined: the take query excludes them (#1578) and they wait for operator review. No CLI drain surface ships at v0.7.0 — inspection and drain are direct SQL against the daemon’s store:

-- Inspect (postgres-backed daemons: table lives in the daemon's
-- schema, e.g. ic_peer_1.federation_push_dlq on shared fleets):
SELECT attempt_count, count(*), max(left(last_error, 60))
FROM federation_push_dlq WHERE replayed_at IS NULL GROUP BY 1;

-- Drain quarantined rows after confirming the target memories
-- already converged (compare distinct memory counts across peers, or
-- GET each memory_id on the destination peer). Marking replayed
-- retains the rows for audit; deleting is equivalent operationally:
UPDATE federation_push_dlq SET replayed_at = now()
WHERE replayed_at IS NULL AND attempt_count >= 100;

A large backlog of replayable (below-ceiling) rows whose memories already converged via async catch-up (e.g. a historical quota-429 burst) can be drained the same way — drop the attempt_count predicate after verifying convergence. The replay worker handles everything else on its own.

Performance

`recall` is slow (> 2 s)

Common causes:

First semantic recall after startup — model load is ~500 ms cold. Warm up with a throwaway recall call.
Async-boot HNSW warm window (#1579 B3) — serve and mcp become ready immediately and build the HNSW index in the background; until the swap lands, semantic recall serves the keyword/FTS blend (correct, but ranked without the vector phase) and can look “worse” or slower on big corpora. Watch for the readiness line: serve logs INFO HNSW index warm (#1579 B3); mcp prints ai-memory: HNSW index ready (N entries, warmed in X.Xs) on stderr. One-shot ai-memory recall CLI invocations skip the graph build entirely below 20k embedded rows (hnsw::CLI_HNSW_BUILD_MIN_ENTRIES) and linear-scan instead — that path is expected to answer in tens of ms, not to build an index.
Disk I/O bottleneck — iostat 1 to confirm. Move DB to SSD.
SQLite contention under concurrent writes — use stats output to see WAL size. If the daemon is doing a lot of writes, recall waits.

Memory usage grows unbounded

Cause: HNSW index size grows with the number of memories. At ~100k memories × 384-dim vectors × 4 bytes = ~150 MB just for the index.

Fix:

Aggressive gc + reduce retention on short tier.
Move to Postgres + pgvector for out-of-process index (--features sal-postgres, v0.7) — the canonical answer at 100k+ memory scale.

Governance

My action returned “202 Accepted” but nothing happened

Cause: Governance requires an approval. Your action is in the pending queue.

Fix:

# List pending
ai-memory pending list --status pending

# Approve (requires registered approver)
ai-memory pending approve <pending-id>

# Or reject
ai-memory pending reject <pending-id>

Consensus rules require multiple distinct registered agents — see docs/ADMIN_GUIDE.md § “Governance”.

Still stuck?

Run ai-memory stats --json and attach to the issue.
Attach the last 50 lines of journalctl -u ai-memory.
State your tier (ai-memory curator --once --dry-run --json shows effective tier + errors).
Open https://github.com/alphaonedev/ai-memory-mcp/issues.

ai-memory Troubleshooting

Startup

“database is locked”

“could not find embedding model”

Semantic recall degraded to keyword — “embedder init failed” / “EMBEDDER LOAD FAILED” (#1593 / #1598)

“port 9077 already in use”

MCP integration

Claude Code / Desktop / Cursor don’t see ai-memory tools

“tools/list returned 31 tools, expected 34”

MCP tool returns “no memories found” but ai-memory list shows them

Autonomy / curator

“no LLM client configured” in curator report

Curator cycle times are long (> 10 min)

Curator made a bad call — how to undo it

HTTP API

“401 missing or invalid API key”

“500 Internal Server Error” with no body

“503 quorum_not_met” on every write

Sync / federation

Memories stop syncing between peers

Split-brain: two peers diverged

Federation push-DLQ backlog / quarantined rows

Performance

recall is slow (> 2 s)

Memory usage grows unbounded

Governance

My action returned “202 Accepted” but nothing happened

Still stuck?

MCP tool returns “no memories found” but `ai-memory list` shows them

`recall` is slow (> 2 s)