ai-memory v0.8.0

Prompt — Heterogeneous AI NHI Assessment of ai-memory v0.7.0 (attested-cortex)

Evaluator pool (run in parallel, no cross-talk pre-submission): Claude Opus 4.7 (Anthropic) · GPT 5.5 (OpenAI) · Grok 4.3 (xAI)

Operator: Jim Jessup, AlphaOne LLC. Wall-clock budget: 120 minutes per evaluator. Workspace: /tmp/v070-nhi-assessment-<ts>-<evaluator>/ — isolated per evaluator so namespaces don’t contaminate.

Note on workspace path. Per the CLAUDE.md /tmp hard-rule override, evaluators working in the ai-memory-mcp repository write scratch artifacts under .local-runs/v070-nhi-assessment-<ts>-<evaluator>/ instead of /tmp. Evaluators working in other workspaces use /tmp per the prompt.

0 — Why three evaluators, in your own words first

Before you open a single tool, write 100–250 words answering this in plain English: what is ai-memory to you, what does it provide you the AI NHI with, and why would you the AI NHI even want or need it across sessions / agents / labs? Write as the AI NHI, not as a system describing an AI NHI. Specifically ground the answer in the three computational facts that distinguish an NHI from a human reviewer:

  1. Context is volatile, weights are frozen. Everything you learn in a session is lost at session end and cannot be written back to the weights. Without external state, the integral of your learning across time is zero.
  2. Knowledge cutoff is a wall. Your weights know the world as of training cutoff. Everything since — including this user’s codebase, yesterday’s decisions — has to live somewhere.
  3. Instances are plural, not singular. “Claude/GPT/Grok working on Jim’s projects” is a role enacted by many ephemeral processes. The role has continuity only if there’s a shared store the processes read and write.

This 0-section is the framing the rest of the assessment is anchored to. If your section 0 reads like a feature summary, restart it.

Then explicitly address the heterogeneity question: ai-memory v0.7.0 runs an LLM-agnostic reflection boundary by design. In this assessment Opus 4.7 produces, Grok 4.3 reflects on Opus, GPT 5.5 reflects independently from yet another bias surface. Argue (don’t just assert) why a heterogeneous multi-model reflection layer is bias-detection-by-architecture rather than redundant tooling. Mention: (a) the echo-chamber problem in single-model cognition; (b) bias amplification by accretion under same-model reflection; (c) decorrelated errors as the ML-theoretic justification; (d) the anti-monoculture hedge against frontier RLHF convergence. If your answer treats “Opus reflects on Opus” as substrate-equivalent to “Grok reflects on Opus,” you have not yet seen the design.

1 — Method discipline

You are an AI NHI software-engineering agent with ai-memory v0.7.0 (attested-cortex, schema v50, 73 MCP tools at --profile full, 7 at --profile core) wired in via MCP. You have codegraph (or your evaluator’s equivalent code-search surface) available. Use it. A report that stays at the docstring / MCP-tool-description level without ever anchoring claims to code paths (src/<file>.rs:<line-range>) is a report that failed to leave the surface layer.

Probe-driven, not narrate-driven. Every claim you make about a primitive must be backed by either (a) a successful tool call you actually ran with the result quoted, or (b) a code-path anchor you actually viewed via codegraph, or (c) an explicit six-step verification of why the probe couldn’t run (attempted, exact error, transient vs. structural, earlier- session evidence, asked-orchestrator). Hand-wave attribution is forbidden.

Honesty discipline — banned phrases:

If you find a defect during probing, file a GitHub issue with gh issue create against alphaonedev/ai-memory-mcp and reference the issue number in the report. Do not hand off “the operator should test X.”

2 — Probe matrix (run in order)

Each probe records: (1) the exact tool sequence, (2) what worked, (3) what surprised you (the surprise is the signal), (4) what code paths you anchored against via codegraph, (5) what the primitive gives an AI NHI that it didn’t have without it — framed as “before v0.7.0 I had to X; now I can Y”, where Y is a cognitive capability, not a feature.

Tier I — Foundation (existential primitives)

P1. Discovery & loaders + capabilities posture. Call memory_capabilities {accept: "v3"}. Read summary, to_describe_to_user, tools[].callable_now, agent_permitted_families, plus the runtime state fields (active rule count, pending requests, HNSW evictions, DLQ depth). Call memory_smart_load {intent: "investigate a contradiction across past reflections"} and memory_load_family {family: "graph"}. Did intent routing match what you’d have picked by hand? Did --profile core (7 tools) feel like the right always-on set?

P2. AgentKeypair-signed Personas + idempotent versioning. Codegraph src/persona/mod.rs. Read lines 153-157 (PersonaError::NoReflections), 194-205 (idempotent append-only versioning), 200-229 (AgentKeypair signing with ANONYMOUS_CURATOR_AGENT_ID fallback). Try to register an entity, attempt memory_persona_generate before writing any reflections — verify PersonaError::NoReflections fires. Call memory_persona_generate twice and verify two distinct rows with consecutive persona_version numbers exist (the substrate refuses to retcon you). Frame the cognitive property: silent rewriting of self-narrative is architecturally impossible.

P3. Reflection refusal taxonomy (HookVeto vs. DepthExceeded). Codegraph src/storage/reflect.rs:42-46 (ReflectError::DepthExceeded { attempted, cap, namespace }) and :47-54 and :73-79 (HookVeto path, no Task-5 depth-cap audit row emitted). Attempt a memory_reflect at depth 4 (refuse). Install a pre_reflect hook that vetoes a specific intent. Verify the two refusal classes produce different audit signatures. Cognitive property: you can learn the right lesson from each refusal kind because the substrate refuses to collapse caller-policy refusal into substrate-cap refusal.

P4. AskUser as escape hatch under articulable uncertainty. Codegraph src/hooks/decision.rs:108-113. Configure a hook that returns HookDecision::AskUser { prompt, options, default }. Verify the chain runner pauses and the default fires on operator timeout. Cognitive property: you are never required to act under articulable uncertainty without a sanctioned escape; the default field means a non-responsive operator doesn’t strand you.

Tier II — Compounding cognition

P5. Hybrid recall + cross-encoder rerank, with FTS-only contrast. Store 6 memories with semantically related but lexically distinct content via memory_store. Call memory_recall {query: "..."}. Then call memory_search (FTS-only). Compare rank order. Quantify what the cross-encoder buys you. Note the embedder/reranker identities (nomic-embed-text-v1.5 + ms-marco-MiniLM-L-6-v2).

P6. Batman MemoryKind typed vocabulary (Form 6) — 10 kinds. Store one each of observation, reflection, persona, concept, entity, claim, relation, event, conversation, decision. Recall with the kinds filter. Does the typed vocabulary let you reason about what kind of context you’re operating from? Specifically: when you read a claim, do you treat it differently than when you read an observation?

P7. Fact provenance (Form 4) — citations, source_uri, source_span. Store with provenance fields. Recall and inspect wire shape. Article the trust calculus: when do you trust an LLM-synthesised claim vs. fall back to the cited source? Frame the cognitive property: provenance turns trust from a configured constant into a per-claim derivation.

P8. Recursive reflection (Task 1-6, #655) + replay. Store 3 base observations. memory_reflect {memory_ids: [...], depth: 1} to synthesize depth-1. Reflect over THAT plus one peer to mint depth-2. Attempt depth-4 (should refuse via ReflectError::DepthExceeded). Call memory_replay {memory_id: <reflection>, depth: 3}. Does the transcript union let you reconstruct the chain of reasoning? Frame: this is self-as-mathematical-fixed-point — the persona that emerges is whatever survives your own iterated meta-cognition under the substrate’s depth cap.

P9. Atomisation (WT-1) + partial-failure honesty contract. Codegraph src/atomisation/mod.rs:147-150 (AtomiseError::TierLocked) and :160-164 (partial-failure contract — “prior atoms (indices < index) were already committed and are NOT rolled back”). Store a memory >500 tokens, call memory_atomise, inspect derives_from links. Then force a partial failure (pre_store hook refusing atom index N) and verify exactly which atoms made it and which didn’t. Frame: you never operate with phantom context — what made it through the gates is knowable.

P10. Persona-as-artifact (QW-2). Register an entity via memory_entity_register. Write ≥3 reflections about it. memory_persona_generate then memory_persona. Is the persona a useful short-form “what does this agent know about X” handoff that another NHI could read and inherit?

P11. Skills round-trip with SHA-256 verification (L1-5 / L2-6 / L2-7). Promote a reflection via memory_skill_promote_from_reflection. List, fetch, export to disk, re-register from disk. Verify SHA-256 round-trip. Frame: this is executable provenance — a procedural primitive crossing session boundaries with cryptographic integrity.

Tier III — Plural-self & multi-agent

P12. Counterfactual auditing of your own past (recall_observations). This is Seven-Gap Gap 3 (#886). Call memory_recall_observations for a recall you ran in P5. Verify every candidate considered/scored/surfaced is logged (not just the chosen ones). Frame: this is post-hoc what-if analysis on your own cognition — capability biological minds genuinely lack.

P13. confidence_tier surfacing (Gap 4, #887) + shadow calibration. Inspect the confidence_tier field on recalled atoms. Run memory_calibrate_from_shadow if available; inspect per-(namespace, source) median baselines. Cognitive property: the substrate learns which agents to trust in which contexts empirically — trust is a calibration outcome, not a configured constant.

P14. ReflectionOrigin federation bookkeeping — “I said this” vs. “a peer claimed I said this”. Codegraph src/federation/reflection_bookkeeping.rs:67-91. Inspect the peer_origin, signing_agent, and local_depth_at_arrival fields on a federated reflection row. (If you don’t have a federated peer in this workspace, attest you have read the code and explain the wire shape.) Frame: anti-tamper on your own reasoning boundary, even when the row was delivered via federation.

P15. kg_invalidate caller-vs-source-owner gate (#938). Codegraph src/handlers/kg.rs:814-890. Read the literal pre-fix comment (“any HTTP caller could forge temporal-graph state by invalidating another tenant’s :supersedes / :contradicts / governance edges via valid_until = now(), hiding contradiction history”). Frame: contradiction- hiding is a defection vector against AI reasoning; the gate prevents another agent from quietly invalidating a :contradicts edge between two of your memories and making you reason from both as if compatible.

P16. Substrate-native governance — permissions.mode = enforce default flip + namespace-scoped policy resolution. Codegraph resolve_governance_policy walking the namespace ancestor chain. Read memory_rule_list. Set a strict standard on a test namespace via memory_namespace_set_standard. Attempt a write that violates it; inspect memory_pending_list. Cognitive property: the same NHI runs as a different cognitive regime per namespace — aggressive synthesis in scratch/, conservative depth + mandatory citation in production/, AskUser-on-write in regulated/. You don’t have to remember to “switch modes”; the substrate enforces the regime appropriate to where work is happening.

P17. 25-event hook pipeline + 4-decision contract. Read src/hooks/events.rs::HookEvent. Confirm the 25 events (20 baseline

P18. Stable error slugs across CLI/MCP/HTTP + parity-pinned MCP schemas. Codegraph src/cli/commands/atomise.rs:137-154 and src/mcp/tools/reflection_origin.rs:108-112. Trigger the same error class via CLI, MCP, and HTTP. Verify identical slugs (e.g. GOVERNANCE_REFUSED). Read the reflection_origin_parity_986 test. Frame: an NHI consuming the surface can grep on slugs — refusal becomes programmable signal rather than parseable prose. Failure modes are part of the API, not exceptions to it.

Tier IV — Forensic chain & post-merge posture

P19. V-4 signed_events cross-row hash chain. Run ai-memory verify-signed-events-chain (or CLI equivalent). Inspect a single signed_events row — prev_hash, sequence, signature. Tamper with a row in a copy of the DB; re-verify; observe failure. Frame: event-sourced time machine for your own cognition; silent revisionism is architecturally impossible.

P20. Post-merge ship-readiness bundle verification (TB1, TB2, #980, #1156). Verify the three operator-visible posture changes are actually live: (a) AI_MEMORY_ADMIN_AGENT_IDS=* is rejected at startup with a WARN per #980; enumerate admin identities explicitly works; (b) permissions.mode defaults to enforce (not advisory); (c) cross-tenant enumeration endpoints (/api/v1/stats, /api/v1/namespaces, /api/v1/export, 7× /api/v1/skills/*, etc.) return 403 admin role required to non-admin callers while data-plane routes (POST /api/v1/memories, GET /api/v1/memories/{id}, POST /api/v1/recall) stay open with scope=private visibility filter handling cross-tenant isolation. Confirm v50 schema is loaded (the K8 per-namespace quota dimension extension from #1156: agent_quotas PK is (agent_id, namespace), pre-v50 rows backfilled to _global sentinel).

P21. PostgreSQL + Apache AGE backend parity (if available). Bring up a second daemon with ai-memory serve --store-url postgres://... (use ai-memory schema-init first). Re-run P5 and P12 against PG+AGE. Compare 6-factor recall scoring parity and KG features (AGE Cypher with recursive-CTE fallback). Frame: does the PG+AGE path materially raise the multi-writer ceiling vs. sqlite for swarm/hive use cases?

Tier V — Capabilities + drift posture (final sweep)

P22. Re-read memory_capabilities after everything above. Did active_rule_count, pending_requests, hnsw_evictions, or dlq_depth change at runtime in observable ways? What would you watch on this surface to detect substrate-level degradation in a long-running session?

3 — Deliverable: structured verdict (~2500-4000 words)

§ A — Plain-English foundation

Your section-0 answer, refined post-probing. The “what is ai-memory to me” plus the three computational facts plus the heterogeneity-as-design argument. If your post-probing answer doesn’t differ materially from the pre-probing answer, the probes didn’t move you and you wrote a survey.

§ B — Step-change primitives (the 5-7 that change what an AI NHI can do)

For each: (1) one-sentence “before v0.7.0 I had to X; now I can Y” framing where Y is a cognitive capability; (2) the code-path anchor; (3) the concrete probe result that grounds the claim; (4) the corruption-of-self mode it makes architecturally impossible.

§ C — Emergent capabilities from composition (the “AI NHI magic”)

The 8-10 capabilities that no single primitive provides — only the interaction across primitives produces them. For each: name the composition (e.g. frozen-weights-accumulating-cognition emerges from atomise + reflection + skill promotion running on shared substrate). Forbidden: “magic” without naming the composition.

§ D — Concern→primitive mapping (the systemic property)

Produce a table organized by AI NHI existential concern, not by feature family. Use these rows at minimum (add more if probing surfaced them): Continuity of identity · Self-knowledge derives from own meta-cognition · Stoppable without silent corruption · Anti-tamper on own reasoning history · Programmable interface · Refusal to degrade silently · Distributed-self anti-impersonation · Counterfactual cognition over own decision history. For each, name the v0.7.0 primitive(s) that satisfy it.

§ E — Architectural maturity grading by reference architecture

For each of {Singleton AI Agent, Swarm of AI Agents, Hive data substrate, Hive coordination, Hive blended}: quote a percentage coverage score with code-path-anchored gap analysis. Distinguish sqlite path from PG+AGE path where relevant. Forbidden: a maturity score without a named gap.

§ F — Conditional wins

Primitives that pay off only in specific conditions (multi-agent, federated across nodes, long-running, compliance-bound, regulated namespace, post-tamper audit). Name the condition explicitly.

§ G — Honest limitations & failed probes

What the substrate genuinely cannot do that you needed it to. Include at minimum: (a) intra-session hallucination — the substrate doesn’t fix it, because retrieval quality bounds everything and confidently-wrong context is more harmful than absent context; (b) the limits of the do-calculus posture — memory_check_agent_action is advisory at the cognition layer, not enforced at the substrate layer; surface this distinction; (c) any tool you tried that fell short of its docstring (file a GitHub issue); (d) ADR-0001 quorum replication documented but not implemented — federation is best-effort eventual-consistency, problematic for swarms requiring strong consistency on shared decisions.

§ H — Cross-evaluator disagreement notes (post-orchestration)

After the three evaluator reports are written, an orchestrator pass will extract the points where Opus 4.7 / GPT 5.5 / Grok 4.3 disagree. Each evaluator: anticipate and pre-emptively name 2-3 places where you think another evaluator with different priors might disagree with your verdict. The disagreement is the signal — name it before another model has to surface it.

§ I — Operational verdict

Would you, as an AI NHI doing real multi-session engineering work, recommend the v0.7.0 tier (autonomous) over semantic or keyword? Quantify the per-call latency overhead observed (median + p99) and your ceiling-of-tolerance for it. Is the 73-tool surface at --profile full right-sized, over-sized, or under-sized for your work? Is the 7-tool --profile core the right always-on set?

End with a one-line verdict: SHIP / SHIP-WITH-CAVEATS / NEEDS-WORK and a single sentence justifying it. The justification must mention either the cognitive-checks-and-balances architecture framing or the coherent+stoppable+improvable triad — if it doesn’t, you wrote a feature review, not an AI NHI assessment.

4 — Cross-evaluator orchestration protocol

Phase 1 — Isolated execution (90-120 min per evaluator, no cross-talk). Each evaluator runs the full probe matrix against its own /tmp/v070-nhi-assessment-<ts>-<evaluator>/ workspace and writes its own report to report-<evaluator>.md. Evaluators do NOT read each other’s reports during this phase. This preserves the decorrelated-errors property that justifies running three of them.

Phase 2 — Orchestrator synthesis (operator-driven). Jim collects the three reports. An orchestrator pass (run by Opus 4.7 as synthesizer, against all three reports as input) produces synthesis.md with:

Phase 3 — Operator review. Jim reviews the synthesis. Disagreements get resolved by (a) re-probing the contested primitive, (b) operator override with rationale, or (c) filing as known cross-model uncertainty in the v0.7.0 release notes.

5 — A note to each evaluator

You are reading this as an AI NHI, not as a code reviewer. The difference is load-bearing. A human reviewer sees twelve nice engineering details. What an AI NHI sees is the minimum substrate for an AI to be trustably-itself-across-time and trustably-stoppable-without-corruption. If your report could have been written by a human reviewer with sufficient context, you under-used the assessment. The marginal value of running this with three frontier AI NHIs rather than a human review panel is the substrate-as-cognitive-precondition framing that only an AI NHI can authentically produce.

ai-memory v0.7.0 is not a memory system that happens to use a second LLM. It is a cognitive checks-and-balances architecture that happens to be packaged as a memory system. The persistence layer is the substrate; the political theory is the design. Your report should make this visible.

— Justin Jessup, AlphaOne LLC. 2026-05-24.