Vitto Memory & Intelligence Architecture

Overview

Vitto's intelligence comes from three layers:

1. User Profile — who the user IS (from MongoDB users collection)
2. User Memories — what Vitto has LEARNED about the user (from user_agent_memories)
3. System Intelligence — how Vitto operates (from vitto-mind database)

┌──────────────────────────────────────────────────────────────────────┐
│                        VITTO CHAT REQUEST                           │
├──────────────────────────────────────────────────────────────────────┤
│                                                                      │
│   Frontend sends:                                                    │
│   ├── prompt (enriched)     → goes to Anthropic as user message     │
│   ├── rawUserMessage        → used for memory extraction            │
│   ├── systemPrompt          → persona + context                     │
│   ├── userId (from JWT)     → identity key for everything           │
│   └── history[]             → conversation continuity               │
│                                                                      │
│   Backend enriches with:                                             │
│   ├── User Profile          ← MongoDB `users` collection           │
│   ├── User Memories         ← MongoDB `user_agent_memories`        │
│   └── Knowledge Distribution ← Multi-tier knowledge system         │
│                                                                      │
│   After response, backend:                                           │
│   ├── 1. detectAndSaveExplicitMemory() ← Regex, immediate save     │
│   └── 2. extractAndSaveMemories()      ← LLM extraction, async     │
│                                                                      │
└──────────────────────────────────────────────────────────────────────┘

Memory Creation Pipeline

Step 1: Explicit Intent Detection (immediate, no LLM)

When the user says "remember this" or states a fact explicitly, the backend uses regex patterns to detect and save immediately.

Patterns detected:

• "remember this/that, I'm ..." → saves as explicit_identity fact
• "my name is ..." → saves as full_name fact
• "I'm the/a [role]" → saves as job_title fact
• "remember that ..." → saves as user_stated_fact

Location: ChatService.detectAndSaveExplicitMemory() Confidence: 1.0 (explicit statement) Latency: ~1ms (no LLM call)

Step 2: LLM Extraction (async, after response)

After every chat response, a cheap LLM call (Claude Haiku, 300 tokens max) analyzes the user's raw message and extracts structured facts.

What it extracts:

CRITICAL: The extraction receives rawUserMessage (the clean user input), NOT the enriched prompt that contains system instructions. Without this separation, the extraction LLM sees 500+ tokens of noise and returns [].

Location: ChatService.extractAndSaveMemories() Confidence: 0.7-1.0 (based on explicitness) Latency: ~500ms (async, doesn't block chat response)

Step 3: Storage

Both steps save to user_agent_memories in gograb-api via UserAgentService.saveMemory().

Upsert behavior: If a memory with the same userId + agentId + key exists, it gets updated (not duplicated) and its reinforcementCount increments.

Memory Recall Pipeline

On every chat request, before the Anthropic API call:

1. Load User Profile from users collection → name, email, roles, subscription, spaces, account age
2. Load Agent Memories from user_agent_memories → facts, preferences, skills, personality
3. Inject into system prompt → Anthropic sees the full context

Location: ChatService.loadUserContext()

Storage Architecture

What goes WHERE (and why)

What does NOT go in user_agent_memories

• Conversation history (stored locally via flash-persistence)
• Vitto's strategies/skills (stored in vitto-mind)
• Agent performance metrics (stored in ai-agents)

Frontend Architecture

Three Chat Entry Points

All three MUST:

1. Send rawUserMessage alongside the enriched prompt
2. Include userId (from JWT/TenantContext/auth_backup)
3. NOT create memories client-side (memory creation is backend-only)

Local Persistence (flash-persistence)

Chat messages are persisted locally via VittoChatPersistenceService using autoFlashState (IndexedDB). This provides:

• Conversation history survives page refresh
• Session management (create, switch, delete conversations)
• Scoped per userId + spaceId

This is LOCAL persistence only. It does NOT replace the backend memory system.

Debugging

Check what memories exist

# Via API (requires JWT)
curl -H "Authorization: Bearer $TOKEN" http://localhost:3001/ai-api/chat/memories

# Via MongoDB directly
db.useragentmemories.find({ userId: ObjectId("YOUR_USER_ID"), agentId: "vitto" })

Backend logs to watch

[MEMORY] Explicit save: job_title="CEO and founder" for user 679abc...
[MEMORY] Extracting from: "remember this, I'm the founder and CEO" for user 679abc...
[MEMORY] Extraction result: [{"type":"fact","key":"role","value":"founder and CEO",...}]
[MEMORY] Extracted and saved 1/1 memories for user 679abc...: role

If memories are NOT being saved, check:

1. Is userId a valid ObjectId? Look for: Invalid userId format (not a valid ObjectId)
2. Is UserAgentService injected? Look for: Memory pipeline skipped: userId=..., userAgentService=false
3. Is the user blocked? Check user_preferences.agentPrivacy.blockedAgents
4. Did extraction return empty? Look for: No memories extracted from message
5. Is the Anthropic API key set? Look for: ANTHROPIC_API_KEY not set

Bugs Fixed (Session 105)

Last Updated: February 11, 2026 Session: 105