🏗️ Architecture: How should Vitto's memory system be redesigned to achieve Ja...

Date: 2026-03-19
Time: 15:53:28 UTC
Session: gspace1-239.2
Agent: backend
Confidence: 92%
Status: ✅ Documented

📋 Decision Context

Question:

How should Vitto's memory system be redesigned to achieve Jarvis-level agent capabilities with persistent, scoped, and anticipatory memory?

Task: S239: Vitto memory system v2 — groundbreaking memory architecture design

🎯 Decision Made

Chosen:

Two-system architecture: Soul (identity/consciousness, per-user customizable) + Brain (4-scope memory hierarchy: Organization > Space > Universe > User). Virtual Context Memory with ConversationGenomeService fixes in-session amnesia. 3-tier recall engine (hot cache + Redis Vector Sets < 5ms, MongoDB $vectorSearch 50-200ms, cross-API cold recall). 5-factor admission control, prospective indexing, dopamine-gated routing, cognitive plugin system. 256-dim Matryoshka embeddings, TOON format compression, prompt caching optimization.

Options Considered:

1. Simple increase of message limits (rejected: doesn't scale, just delays the problem)
2. Full Graphiti/Neo4j temporal knowledge graph (rejected: too much infrastructure for current team size)
3. Consume genome from backend-ai-api only (rejected: need real-time per-turn updates in agent loop)
4. Single 'core vs user' memory split (rejected: 4-scope hierarchy better matches existing space/universe model)

💭 Reasoning

Deep audit revealed 3 critical failures: (1) In-session amnesia — conversations hard-capped at 10-20 messages with no compression (channel-processing.service.ts:324, channel-prompt-builder.service.ts:256-257). (2) Missing MongoDB vector search indexes — vitto_memories_vector and agent_embeddings indexes not created in Atlas, so semantic recall silently falls back to text search. (3) No memory scope hierarchy — all memories default to layer='developer'. Research of 30+ papers/systems (D-MEM, Kumiho, A-MAC, CrewAI, Letta, Graphiti) informed the architecture. The Virtual Context Memory (MemGPT-inspired genome compression) is the most critical fix — eliminates the message cliff while maintaining full conversation awareness. Redis Vector Sets solve MongoDB's 1-3s propagation delay. The cognitive plugin system ensures each innovation is independently toggleable.

📁 Files Involved

• docs/superpowers/specs/2026-03-19-vitto-memory-system-v2-design.md
• APIs/backend-vitto-api/src/app/mind/vitto-memory.service.ts
• APIs/backend-vitto-api/src/app/mind/memory/vitto-memory-recall.service.ts
• APIs/backend-vitto-api/src/app/engine/channel-processing.service.ts
• APIs/backend-vitto-api/src/app/engine/context-assembly/channel-prompt-builder.service.ts
• APIs/backend-vitto-api/src/app/mind/schemas/vitto-memory.schema.ts

🔗 Related

• Decision ID: dev_1773935608386_zpphv1a
• Session: 239
• Workspace: gspace1

Auto-generated by MCP Smart Consolidation