Session 75.15: Two-Layer Agent Intelligence Implementation Summary

Date: 2026-01-31 Branch: editor2-session-75.5 Status: Core Implementation Complete

What Was Built

1. Cross-Space User Agent Memories

Location: gograb-api.user_agent_memories

A system that lets AI agents truly KNOW users across all their Spaces:

// Schema: libs/libs-api/backend-core/src/lib/schemas/user-agent-memory.schema.ts
interface UserAgentMemory {
  userId: ObjectId;
  agentId: string;           // 'vitto', 'claude-code', etc.
  memoryType: 'preference' | 'fact' | 'skill' | 'personality' | 'interaction';
  key: string;               // 'response_style', 'coding_language', etc.
  value: Record<string, any>;
  confidence: number;        // 0-1, grows with reinforcement
  source: 'observed' | 'stated' | 'inferred';
  learnedFromSpaces: string[]; // Audit trail
}

Key Features:

• Upsert on userId + agentId + key (no duplicates)
• Confidence reinforcement/weakening with atomic MongoDB operations
• Privacy checks before saving (respects crossSpaceLearning: false)
• TTL support for temporary memories
• GDPR-compliant forget operations

2. Global User Preferences

Location: gograb-api.user_preferences

User's default preferences that apply across all Spaces:

// Schema: libs/libs-api/backend-core/src/lib/schemas/user-preferences.schema.ts
interface UserPreferences {
  userId: ObjectId;
  theme: 'light' | 'dark' | 'auto';
  language: string;
  timezone: string;
  agentPrivacy: {
    crossSpaceLearning: boolean;
    memoryRetention: '30d' | '90d' | '1y' | 'forever';
    blockedAgents: string[];
  };
  notifications: NotificationPreferences;
  interface: InterfacePreferences;
  aiAssistant: AIAssistantPreferences;
}

3. VITTO Memory Integration

Location: apis/backend-ai-api/src/app/vitto/vitto-orchestrator.service.ts

VITTO now:

1. Loads user memories before AI execution
2. Includes memories in system prompt for personalization
3. Saves interaction learnings after task completion

// Memory loading for AI context
const userContext = await this.loadUserContext(userId);
// Returns formatted context like:
// ## What I know about this user:
// **Preferences:**
// - response_style: {"style": "concise"} (confidence: 0.80)
// **Skills/Expertise:**
// - coding_language: {"primary": "TypeScript"}

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                          User Request                                │
└─────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────┐
│                    backend-ai-api (VITTO)                            │
│  ┌──────────────────────────────────────────────────────────────┐   │
│  │ VittoOrchestratorService                                      │   │
│  │  └─ loadUserContext(userId) ─────┐                            │   │
│  │  └─ saveUserLearning(...)        │                            │   │
│  │  └─ reinforceUserMemory(...)     │                            │   │
│  └──────────────────────────────────│────────────────────────────┘   │
│                                      │                               │
│                                      ▼                               │
│  ┌──────────────────────────────────────────────────────────────┐   │
│  │ UserAgentMemoryModule                                         │   │
│  │  └─ UserAgentService (from @libs-api/backend-core)            │   │
│  │  └─ UserPreferencesService                                    │   │
│  └──────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘
                                │
                    AI_CONNECTION_NAMES.PLATFORM
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     gograb-api (MongoDB)                             │
│  ├── user_agent_memories (WHO the user is)                          │
│  └── user_preferences (Global defaults + privacy)                   │
└─────────────────────────────────────────────────────────────────────┘

Files Changed/Created

New Files

Modified Files

API Endpoints

User Preferences (backend-api)

GET    /api/users/me/preferences           # Get global preferences
PUT    /api/users/me/preferences           # Update preferences
GET    /api/users/me/privacy-settings      # Get privacy controls
PUT    /api/users/me/privacy-settings      # Update privacy
PUT    /api/users/me/privacy-settings/block/:agentId  # Block agent
DELETE /api/users/me/privacy-settings/block/:agentId  # Unblock agent

Agent Memories (backend-api)

GET    /api/users/me/agent-memories        # List all memories
GET    /api/users/me/agent-memories/:agentId  # Memories from agent
GET    /api/users/me/agent-memories-stats  # Memory statistics
DELETE /api/users/me/agent-memories/:memoryId  # Forget specific
DELETE /api/users/me/agent-memories/agent/:agentId  # Forget all from agent
DELETE /api/users/me/agent-memories        # Forget ALL (GDPR)

Bug Fixes Applied

1. ObjectId Validation - All forget methods now validate IDs before creating ObjectId
2. Confidence Race Condition - Uses atomic MongoDB aggregation pipeline with $min/$max
3. Preference Overrides Fallback - SpaceDatabaseService creates defaults if missing

Usage Example

Saving a Memory (from VITTO or any agent)

await userAgentService.saveMemory({
  userId: 'user-123',
  agentId: 'vitto',
  memoryType: 'preference',
  key: 'response_style',
  value: { style: 'concise', reason: 'User requested shorter answers' },
  confidence: 0.8,
  source: 'stated',
  spaceId: 'space-abc',
});

Querying Memories

const memories = await userAgentService.getMemories({
  userId: 'user-123',
  agentId: 'vitto',
  memoryType: 'preference',
  minConfidence: 0.5,
  limit: 20,
});

Reinforcing (user confirmed assumption)

await userAgentService.reinforceMemory('user-123', 'vitto', 'response_style', 0.1);
// Confidence increases by 0.1, capped at 1.0

Privacy Check

const canLearn = await userAgentService.canLearnFromUser('user-123', 'vitto');
// Returns false if user has blocked agent or disabled crossSpaceLearning

What's Still Outstanding

Key Design Decisions

1. Direct MongoDB over HTTP - VITTO needs sub-10ms memory access during conversation
2. @Optional() injection - VITTO works even if memory service unavailable
3. Memories by agentId - Each agent maintains its own knowledge about users
4. Confidence-based learning - Low initial confidence grows with reinforcement
5. Privacy-first - All saves check user's privacy settings first