2026 03 19 Vitto Memory V2 Implementation
Documentation for Plans
Vitto Memory System v2 — Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Transform Vitto's memory from a 10-message amnesiac chatbot into a Jarvis-level cognitive system with scoped memory, virtual context, and intelligent recall.
Architecture: Soul (identity) and Brain (knowledge) as parallel systems. Brain uses 4-scope hierarchy (Org > Space > Universe > User). Virtual Context Memory with conversation genome eliminates in-session amnesia. 3-tier recall engine (hot cache → Redis Vector Sets → MongoDB $vectorSearch). Cognitive plugin system makes each innovation independently toggleable.
Tech Stack: NestJS, MongoDB Atlas (Mongoose), Redis 8.0+ (Vector Sets via ioredis), OpenAI text-embedding-3-small (256-dim Matryoshka), TOON format, Angular 20+ (standalone, OnPush, signals), Jest.
Spec: docs/superpowers/specs/2026-03-19-vitto-memory-system-v2-design.md
Git: All commits follow [S239.X] type: description. NO git add . — always specific files. Branch: session-238-239 (shared, coordinate with concurrent session).
File Map
New Files (Backend — APIs/backend-vitto-api/src/app/)
mind/
├── virtual-context/
│ ├── conversation-genome.service.ts # Session compression (heuristic v1)
│ ├── conversation-genome.types.ts # Genome interfaces
│ ├── window-manager.service.ts # Dynamic message window sizing
│ ├── on-demand-recall.service.ts # Reference-triggered recall from full history
│ └── __tests__/
│ ├── conversation-genome.service.spec.ts
│ ├── window-manager.service.spec.ts
│ └── on-demand-recall.service.spec.ts
├── recall/
│ ├── dopamine-router.service.ts # Fast/standard/deep query routing
│ ├── redis-vector.service.ts # Redis Vector Sets integration
│ └── __tests__/
│ ├── dopamine-router.service.spec.ts
│ └── redis-vector.service.spec.ts
├── admission/
│ ├── memory-admission.service.ts # 5-factor admission gate
│ ├── memory-admission.types.ts # Admission interfaces
│ └── __tests__/
│ └── memory-admission.service.spec.ts
├── indexing/
│ ├── prospective-indexer.service.ts # Future-question generation
│ └── __tests__/
│ └── prospective-indexer.service.spec.ts
├── plugins/
│ ├── cognitive-plugin.interface.ts # Plugin interface
│ ├── cognitive-plugin-registry.service.ts # Plugin registry
│ └── built-in/
│ ├── admission-control.plugin.ts
│ ├── prospective-index.plugin.ts
│ ├── dopamine-router.plugin.ts
│ ├── user-preference-learning.plugin.ts
│ ├── emotional-context.plugin.ts
│ └── memory-versioning.plugin.ts
├── conflict/
│ ├── memory-conflict-resolver.service.ts # Scope precedence
│ └── __tests__/
│ └── memory-conflict-resolver.service.spec.ts
├── export/
│ └── memory-docs-export.service.ts # Export to docs-angular
└── soul-compression.service.ts # Soul → TOON brief
engine/
└── context-assembly/
└── toon-serializer.service.ts # TOON format serialization
New Files (Frontend — libs/libs-ai/)
vitto-hub/src/lib/components/
├── memory-manager/
│ ├── memory-scope-filter/
│ │ ├── memory-scope-filter.component.ts
│ │ ├── memory-scope-filter.component.html
│ │ └── memory-scope-filter.component.scss
│ ├── memory-version-history/
│ │ ├── memory-version-history.component.ts
│ │ ├── memory-version-history.component.html
│ │ └── memory-version-history.component.scss
│ └── genome-viewer/
│ ├── genome-viewer.component.ts
│ ├── genome-viewer.component.html
│ └── genome-viewer.component.scss
vitto-core/src/lib/models/
└── memory-v2.models.ts # Updated MemoryScope, RecallResult types
Modified Files
# Schema & Types
mind/schemas/vitto-memory.schema.ts # Add scope, scopeId, overridable, status, history fields
mind/memory/vitto-memory.types.ts # Add scope to RecallOptions/RecallResult, 256-dim constant
# Memory Services
mind/memory/vitto-memory-recall.service.ts # Add Redis tier, dopamine routing, scope filtering
mind/memory/vitto-memory-store.service.ts # Add admission control hook, prospective indexing trigger
mind/memory/vitto-memory-embedding.service.ts # 256-dim Matryoshka support, dual-index transition
mind/memory/vitto-memory-hot-cache.service.ts # Scope-aware cache keys
# Context Assembly
engine/context-assembly/context-assembly.service.ts # Genome injection, dynamic budget
engine/context-assembly/channel-prompt-builder.service.ts # Remove hardcoded 10/20 limits
engine/channel-processing.service.ts # Remove hardcoded 10 limit, use window manager
strategy/context-window.service.ts # Integrate with window manager
# Module Registration
mind/mind.module.ts # Register new services
engine/engine.module.ts # Register toon-serializer
# Frontend
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-manager.component.ts
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-manager.component.html
libs/libs-ai/vitto-core/src/lib/services/vitto-api.service.ts # New scope endpoints
Phase 1: Critical Fixes — In-Session Amnesia & Schema
Priority: HIGHEST. This phase fixes the primary user-facing bug: Vitto forgets mid-conversation.
Task 1: Memory Schema — Add Scope Fields
Files:
Modify:
APIs/backend-vitto-api/src/app/mind/schemas/vitto-memory.schema.tsModify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory.types.tsCreate:
APIs/backend-vitto-api/src/app/mind/memory/__tests__/vitto-memory-types.spec.tsStep 1: Read the current schema and types files
Read mind/schemas/vitto-memory.schema.ts and mind/memory/vitto-memory.types.ts fully.
- Step 2: Write failing test for new scope types
// mind/memory/__tests__/vitto-memory-types.spec.ts
import { MemoryScope, MEMORY_SCOPES } from '../vitto-memory.types';
describe('MemoryScope types', () => {
it('should define all four scopes', () => {
expect(MEMORY_SCOPES).toEqual(['organization', 'space', 'universe', 'user']);
});
it('should define scope precedence order', () => {
expect(MEMORY_SCOPES.indexOf('organization')).toBeLessThan(MEMORY_SCOPES.indexOf('space'));
expect(MEMORY_SCOPES.indexOf('space')).toBeLessThan(MEMORY_SCOPES.indexOf('universe'));
expect(MEMORY_SCOPES.indexOf('universe')).toBeLessThan(MEMORY_SCOPES.indexOf('user'));
});
});
- Step 3: Run test to verify it fails
Run: npx jest APIs/backend-vitto-api/src/app/mind/memory/__tests__/vitto-memory-types.spec.ts --no-coverage
Expected: FAIL — MemoryScope not exported
- Step 4: Add scope types to vitto-memory.types.ts
Add to existing types file:
export type MemoryScope = 'organization' | 'space' | 'universe' | 'user';
export const MEMORY_SCOPES: MemoryScope[] = ['organization', 'space', 'universe', 'user'];
export type MemoryStatus = 'active' | 'quarantined' | 'archived';
// Update RecallOptions — add new fields alongside existing ones
// In the existing RecallOptions interface, ADD:
// scope?: MemoryScope | MemoryScope[];
// scopeId?: string;
// universeId?: string;
// status?: MemoryStatus;
// Update RecallResult — add new fields alongside existing ones
// In the existing RecallResult interface, ADD:
// scope: MemoryScope;
// scopeId: string;
// source: 'hot_cache' | 'redis' | 'vector_search' | 'text_search' | 'cold_recall';
// Update RememberParams — add new fields alongside existing ones
// In the existing RememberParams interface, ADD:
// scope?: MemoryScope;
// scopeId?: string;
// overridable?: boolean;
// Update EMBEDDING_DIMENSIONS constant:
// export const EMBEDDING_DIMENSIONS_V2 = 256;
// Keep EMBEDDING_DIMENSIONS = 1536 for backward compat during transition
- Step 5: Run test to verify it passes
Run: npx jest APIs/backend-vitto-api/src/app/mind/memory/__tests__/vitto-memory-types.spec.ts --no-coverage
Expected: PASS
- Step 6: Add scope fields to Mongoose schema
In vitto-memory.schema.ts, add alongside existing layer field:
@Prop({ type: String, enum: ['organization', 'space', 'universe', 'user'], default: 'user', index: true })
scope: string;
@Prop({ type: String, index: true })
scopeId: string;
@Prop({ type: Boolean, default: true })
overridable: boolean;
@Prop({ type: String, enum: ['active', 'quarantined', 'archived'], default: 'active', index: true })
status: string;
@Prop({ type: [{ summary: String, tags: [String], editedBy: String, editedAt: Date }], default: [] })
history: Array<{ summary: string; tags: string[]; editedBy: string; editedAt: Date }>;
Also update the vector search index comment to include scope and scopeId filter fields.
- Step 7: Commit
git add APIs/backend-vitto-api/src/app/mind/schemas/vitto-memory.schema.ts APIs/backend-vitto-api/src/app/mind/memory/vitto-memory.types.ts APIs/backend-vitto-api/src/app/mind/memory/__tests__/vitto-memory-types.spec.ts
git commit -m "[S239.1] feat: add memory scope hierarchy and versioning to schema"
Task 2: Conversation Genome Service (Heuristic v1)
Files:
Create:
APIs/backend-vitto-api/src/app/mind/virtual-context/conversation-genome.types.tsCreate:
APIs/backend-vitto-api/src/app/mind/virtual-context/conversation-genome.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/conversation-genome.service.spec.tsStep 1: Write failing test for genome types and incremental update
// __tests__/conversation-genome.service.spec.ts
import { ConversationGenomeService } from '../conversation-genome.service';
import { ConversationGenome } from '../conversation-genome.types';
describe('ConversationGenomeService', () => {
let service: ConversationGenomeService;
beforeEach(() => {
service = new ConversationGenomeService();
});
describe('incrementalUpdate', () => {
it('should extract topics from evicted messages', () => {
const genome: ConversationGenome = {
sessionId: 'test-session',
userId: 'user-1',
topics: [],
decisions: [],
facts: [],
actionItems: [],
summary: '',
messageRange: { from: 0, to: 0 },
tokenCount: 0,
lastUpdated: new Date(),
version: 0,
};
const evictedMessages = [
{ role: 'user', content: 'Can you help me set up MongoDB vector search for the project?' },
{ role: 'assistant', content: 'Sure, I will create the vector index on the vitto-memories collection with cosine similarity.' },
{ role: 'user', content: 'Great, and make sure it uses 256 dimensions.' },
];
const updated = service.incrementalUpdate(genome, evictedMessages);
expect(updated.topics.length).toBeGreaterThan(0);
expect(updated.version).toBe(1);
expect(updated.messageRange.to).toBe(3);
});
it('should not exceed 800 tokens', () => {
const genome: ConversationGenome = {
sessionId: 'test-session',
userId: 'user-1',
topics: [],
decisions: [],
facts: [],
actionItems: [],
summary: '',
messageRange: { from: 0, to: 0 },
tokenCount: 0,
lastUpdated: new Date(),
version: 0,
};
// Simulate 50 evicted messages
const evictedMessages = Array.from({ length: 50 }, (_, i) => ({
role: i % 2 === 0 ? 'user' : 'assistant',
content: `Message ${i} about topic ${i % 5} with some substantial content that takes up tokens.`,
}));
const updated = service.incrementalUpdate(genome, evictedMessages);
expect(updated.tokenCount).toBeLessThanOrEqual(800);
});
});
describe('hydrate', () => {
it('should produce prompt-ready text under 500 tokens', () => {
const genome: ConversationGenome = {
sessionId: 'test',
userId: 'user-1',
topics: [
{ name: 'MongoDB setup', firstMentioned: 0, lastMentioned: 5, status: 'active', keyPoints: ['Vector index needed', '256 dimensions'] },
{ name: 'Redis caching', firstMentioned: 3, lastMentioned: 8, status: 'resolved', keyPoints: ['Using Vector Sets'] },
],
decisions: [{ description: 'Use 256-dim Matryoshka embeddings', madeAt: 4 }],
facts: [{ content: 'Project uses MongoDB Atlas', establishedAt: 1 }],
actionItems: ['Create vitto_memories_vector index'],
summary: 'Discussed memory system improvements focusing on vector search and Redis caching.',
messageRange: { from: 0, to: 10 },
tokenCount: 300,
lastUpdated: new Date(),
version: 3,
};
const text = service.hydrate(genome);
expect(typeof text).toBe('string');
expect(text.length).toBeGreaterThan(0);
// Rough token estimate: ~4 chars per token
expect(text.length).toBeLessThan(2000); // ~500 tokens max
});
});
});
- Step 2: Run test to verify it fails
Run: npx jest APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/conversation-genome.service.spec.ts --no-coverage
Expected: FAIL — module not found
- Step 3: Create genome types
// mind/virtual-context/conversation-genome.types.ts
export interface ConversationGenome {
sessionId: string;
userId: string;
topics: GenomeTopic[];
decisions: GenomeDecision[];
facts: GenomeFact[];
actionItems: string[];
summary: string;
messageRange: { from: number; to: number };
tokenCount: number;
lastUpdated: Date;
version: number;
}
export interface GenomeTopic {
name: string;
firstMentioned: number;
lastMentioned: number;
status: 'active' | 'resolved' | 'abandoned';
keyPoints: string[];
}
export interface GenomeDecision {
description: string;
madeAt: number;
}
export interface GenomeFact {
content: string;
establishedAt: number;
}
export interface GenomeMessage {
role: string;
content: string;
}
- Step 4: Implement ConversationGenomeService (heuristic v1, no LLM)
// mind/virtual-context/conversation-genome.service.ts
import { Injectable } from '@nestjs/common';
import { ConversationGenome, GenomeMessage, GenomeTopic } from './conversation-genome.types';
@Injectable()
export class ConversationGenomeService {
private readonly MAX_GENOME_TOKENS = 800;
private readonly MAX_TOPICS = 10;
private readonly MAX_KEY_POINTS_PER_TOPIC = 3;
private readonly MAX_DECISIONS = 5;
private readonly MAX_FACTS = 5;
private readonly MAX_ACTION_ITEMS = 5;
incrementalUpdate(genome: ConversationGenome, evictedMessages: GenomeMessage[]): ConversationGenome {
const updated = { ...genome };
const startIndex = genome.messageRange.to;
// Extract keywords from evicted messages
for (let i = 0; i < evictedMessages.length; i++) {
const msg = evictedMessages[i];
const msgIndex = startIndex + i;
const keywords = this.extractKeywords(msg.content);
// Topic detection: group by keyword clusters
const topicName = this.detectTopic(keywords, updated.topics);
const existingTopic = updated.topics.find((t) => t.name === topicName);
if (existingTopic) {
existingTopic.lastMentioned = msgIndex;
const point = this.extractKeyPoint(msg.content);
if (point && existingTopic.keyPoints.length < this.MAX_KEY_POINTS_PER_TOPIC) {
if (!existingTopic.keyPoints.includes(point)) {
existingTopic.keyPoints.push(point);
}
}
} else if (updated.topics.length < this.MAX_TOPICS) {
updated.topics.push({
name: topicName,
firstMentioned: msgIndex,
lastMentioned: msgIndex,
status: 'active',
keyPoints: [this.extractKeyPoint(msg.content)].filter(Boolean) as string[],
});
}
// Decision detection
if (this.isDecision(msg.content) && updated.decisions.length < this.MAX_DECISIONS) {
updated.decisions.push({ description: this.summarizeSentence(msg.content, 100), madeAt: msgIndex });
}
// Fact detection
if (this.isFact(msg.content, msg.role) && updated.facts.length < this.MAX_FACTS) {
updated.facts.push({ content: this.summarizeSentence(msg.content, 80), establishedAt: msgIndex });
}
// Action item detection
if (this.isActionItem(msg.content) && updated.actionItems.length < this.MAX_ACTION_ITEMS) {
updated.actionItems.push(this.summarizeSentence(msg.content, 60));
}
}
// Update summary (simple concatenation of topic names)
updated.summary = this.buildSummary(updated);
updated.messageRange = { from: genome.messageRange.from, to: startIndex + evictedMessages.length };
updated.version = genome.version + 1;
updated.lastUpdated = new Date();
updated.tokenCount = this.estimateTokens(this.hydrate(updated));
// Prune if over token budget
if (updated.tokenCount > this.MAX_GENOME_TOKENS) {
this.prune(updated);
}
return updated;
}
hydrate(genome: ConversationGenome): string {
const parts: string[] = [];
if (genome.summary) {
parts.push(`[Session Summary] ${genome.summary}`);
}
if (genome.topics.length > 0) {
const activeTopics = genome.topics.filter((t) => t.status === 'active');
const resolvedTopics = genome.topics.filter((t) => t.status === 'resolved');
if (activeTopics.length > 0) {
parts.push(`[Active Topics] ${activeTopics.map((t) => `${t.name}: ${t.keyPoints.join('; ')}`).join(' | ')}`);
}
if (resolvedTopics.length > 0) {
parts.push(`[Resolved] ${resolvedTopics.map((t) => t.name).join(', ')}`);
}
}
if (genome.decisions.length > 0) {
parts.push(`[Decisions Made] ${genome.decisions.map((d) => d.description).join('; ')}`);
}
if (genome.facts.length > 0) {
parts.push(`[Established Facts] ${genome.facts.map((f) => f.content).join('; ')}`);
}
if (genome.actionItems.length > 0) {
parts.push(`[Pending Actions] ${genome.actionItems.join('; ')}`);
}
return parts.join('\n');
}
topicRelevance(query: string, genome: ConversationGenome): number {
const queryKeywords = this.extractKeywords(query);
let maxRelevance = 0;
for (const topic of genome.topics) {
const topicKeywords = this.extractKeywords(topic.name + ' ' + topic.keyPoints.join(' '));
const overlap = queryKeywords.filter((k) => topicKeywords.includes(k)).length;
const relevance = overlap / Math.max(queryKeywords.length, 1);
maxRelevance = Math.max(maxRelevance, relevance);
}
return maxRelevance;
}
private extractKeywords(text: string): string[] {
const stopWords = new Set([
'the', 'a', 'an', 'is', 'are', 'was', 'were', 'be', 'been', 'being',
'have', 'has', 'had', 'do', 'does', 'did', 'will', 'would', 'could',
'should', 'may', 'might', 'can', 'shall', 'to', 'of', 'in', 'for',
'on', 'with', 'at', 'by', 'from', 'it', 'this', 'that', 'i', 'you',
'we', 'they', 'he', 'she', 'and', 'or', 'but', 'not', 'so', 'if',
'then', 'than', 'just', 'also', 'very', 'really', 'about', 'into',
]);
return text
.toLowerCase()
.replace(/[^a-z0-9\s-]/g, '')
.split(/\s+/)
.filter((w) => w.length > 2 && !stopWords.has(w));
}
private detectTopic(keywords: string[], existingTopics: GenomeTopic[]): string {
// Try to match existing topic by keyword overlap
for (const topic of existingTopics) {
const topicKeywords = this.extractKeywords(topic.name);
const overlap = keywords.filter((k) => topicKeywords.some((tk) => k.includes(tk) || tk.includes(k))).length;
if (overlap >= 2 || (overlap >= 1 && topicKeywords.length <= 2)) {
return topic.name;
}
}
// Create new topic from top keywords
return keywords.slice(0, 3).join(' ') || 'general';
}
private extractKeyPoint(content: string): string | null {
const sentences = content.split(/[.!?]+/).filter((s) => s.trim().length > 10);
if (sentences.length === 0) return null;
return this.summarizeSentence(sentences[0], 80);
}
private isDecision(content: string): boolean {
const decisionPatterns = /\b(decided|will use|going with|chose|let's go|approved|selected|agreed)\b/i;
return decisionPatterns.test(content);
}
private isFact(content: string, role: string): boolean {
if (role !== 'assistant') return false;
const factPatterns = /\b(uses|requires|supports|runs on|built with|depends on|configured|set to)\b/i;
return factPatterns.test(content);
}
private isActionItem(content: string): boolean {
const actionPatterns = /\b(need to|should|must|todo|action item|next step|will create|will implement)\b/i;
return actionPatterns.test(content);
}
private summarizeSentence(text: string, maxChars: number): string {
const trimmed = text.trim();
if (trimmed.length <= maxChars) return trimmed;
return trimmed.substring(0, maxChars - 3) + '...';
}
private buildSummary(genome: ConversationGenome): string {
const topicNames = genome.topics.map((t) => t.name).slice(0, 5);
if (topicNames.length === 0) return '';
return `Discussed: ${topicNames.join(', ')}.`;
}
private estimateTokens(text: string): number {
return Math.ceil(text.length / 4); // ~4 chars per token heuristic
}
private prune(genome: ConversationGenome): void {
// Remove oldest resolved topics first
genome.topics = genome.topics
.sort((a, b) => {
if (a.status === 'resolved' && b.status !== 'resolved') return 1;
return b.lastMentioned - a.lastMentioned;
})
.slice(0, this.MAX_TOPICS);
// Trim key points
for (const topic of genome.topics) {
topic.keyPoints = topic.keyPoints.slice(-this.MAX_KEY_POINTS_PER_TOPIC);
}
// Trim oldest decisions and facts
genome.decisions = genome.decisions.slice(-this.MAX_DECISIONS);
genome.facts = genome.facts.slice(-this.MAX_FACTS);
genome.actionItems = genome.actionItems.slice(-this.MAX_ACTION_ITEMS);
genome.tokenCount = this.estimateTokens(this.hydrate(genome));
}
}
- Step 5: Run tests to verify they pass
Run: npx jest APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/conversation-genome.service.spec.ts --no-coverage
Expected: PASS (all 3 tests)
- Step 6: Commit
git add APIs/backend-vitto-api/src/app/mind/virtual-context/
git commit -m "[S239.2] feat: implement ConversationGenomeService with heuristic compression"
Task 3: Window Manager Service
Files:
Create:
APIs/backend-vitto-api/src/app/mind/virtual-context/window-manager.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/window-manager.service.spec.tsStep 1: Write failing test
// __tests__/window-manager.service.spec.ts
import { WindowManagerService } from '../window-manager.service';
import { ConversationGenomeService } from '../conversation-genome.service';
describe('WindowManagerService', () => {
let service: WindowManagerService;
beforeEach(() => {
service = new WindowManagerService(new ConversationGenomeService());
});
describe('calculateWindowSize', () => {
it('should allocate remaining budget to messages', () => {
const size = service.calculateWindowSize({
totalBudget: 80000,
pinnedTokens: 550, // soul(150) + brief(200) + system(200)
genomeTokens: 400,
recalledTokens: 500,
skillTokens: 0,
responseBuffer: 4000,
});
// Available = 80000 - 550 - 400 - 500 - 0 - 4000 = 74550
// At ~300 tokens/message avg = ~248 messages
// But capped at reasonable max (60)
expect(size).toBeGreaterThanOrEqual(30);
expect(size).toBeLessThanOrEqual(60);
});
it('should return minimum window even when budget is tight', () => {
const size = service.calculateWindowSize({
totalBudget: 8000,
pinnedTokens: 3000,
genomeTokens: 2000,
recalledTokens: 1000,
skillTokens: 1000,
responseBuffer: 4000,
});
// Budget is negative but should still return minimum (4)
expect(size).toBeGreaterThanOrEqual(4);
});
});
describe('evictAndCompress', () => {
it('should evict oldest messages when window is exceeded', () => {
const messages = Array.from({ length: 20 }, (_, i) => ({
role: i % 2 === 0 ? 'user' : 'assistant',
content: `Message ${i}`,
}));
const emptyGenome = {
sessionId: 'test', userId: 'u1', topics: [], decisions: [],
facts: [], actionItems: [], summary: '', messageRange: { from: 0, to: 0 },
tokenCount: 0, lastUpdated: new Date(), version: 0,
};
const result = service.evictAndCompress(messages, 10, emptyGenome);
expect(result.activeMessages.length).toBe(10);
expect(result.activeMessages[0].content).toBe('Message 10'); // Oldest kept
expect(result.updatedGenome.version).toBe(1); // Genome was updated
expect(result.evictedCount).toBe(10);
});
it('should not evict when within window', () => {
const messages = Array.from({ length: 5 }, (_, i) => ({
role: 'user', content: `Message ${i}`,
}));
const emptyGenome = {
sessionId: 'test', userId: 'u1', topics: [], decisions: [],
facts: [], actionItems: [], summary: '', messageRange: { from: 0, to: 0 },
tokenCount: 0, lastUpdated: new Date(), version: 0,
};
const result = service.evictAndCompress(messages, 10, emptyGenome);
expect(result.activeMessages.length).toBe(5);
expect(result.evictedCount).toBe(0);
expect(result.updatedGenome.version).toBe(0); // Unchanged
});
});
});
- Step 2: Run test to verify it fails
Run: npx jest APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/window-manager.service.spec.ts --no-coverage
Expected: FAIL
- Step 3: Implement WindowManagerService
// mind/virtual-context/window-manager.service.ts
import { Injectable } from '@nestjs/common';
import { ConversationGenomeService } from './conversation-genome.service';
import { ConversationGenome, GenomeMessage } from './conversation-genome.types';
interface TokenBudgetParams {
totalBudget: number;
pinnedTokens: number;
genomeTokens: number;
recalledTokens: number;
skillTokens: number;
responseBuffer: number;
}
interface EvictionResult {
activeMessages: GenomeMessage[];
updatedGenome: ConversationGenome;
evictedCount: number;
}
@Injectable()
export class WindowManagerService {
private readonly MIN_WINDOW = 4;
private readonly MAX_WINDOW = 60;
private readonly AVG_TOKENS_PER_MESSAGE = 300;
constructor(private readonly genomeService: ConversationGenomeService) {}
calculateWindowSize(params: TokenBudgetParams): number {
const available = params.totalBudget - params.pinnedTokens - params.genomeTokens
- params.recalledTokens - params.skillTokens - params.responseBuffer;
if (available <= 0) return this.MIN_WINDOW;
const calculated = Math.floor(available / this.AVG_TOKENS_PER_MESSAGE);
return Math.max(this.MIN_WINDOW, Math.min(this.MAX_WINDOW, calculated));
}
evictAndCompress(
messages: GenomeMessage[],
windowSize: number,
genome: ConversationGenome,
): EvictionResult {
if (messages.length <= windowSize) {
return { activeMessages: messages, updatedGenome: genome, evictedCount: 0 };
}
const evictCount = messages.length - windowSize;
const evictedMessages = messages.slice(0, evictCount);
const activeMessages = messages.slice(evictCount);
const updatedGenome = this.genomeService.incrementalUpdate(genome, evictedMessages);
return { activeMessages, updatedGenome, evictedCount: evictCount };
}
}
- Step 4: Run tests to verify they pass
Run: npx jest APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/window-manager.service.spec.ts --no-coverage
Expected: PASS
- Step 5: Commit
git add APIs/backend-vitto-api/src/app/mind/virtual-context/window-manager.service.ts APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/window-manager.service.spec.ts
git commit -m "[S239.3] feat: implement WindowManagerService with dynamic sizing and eviction"
Task 4: Wire Virtual Context into Context Assembly
Files:
Modify:
APIs/backend-vitto-api/src/app/engine/context-assembly/channel-prompt-builder.service.tsModify:
APIs/backend-vitto-api/src/app/engine/channel-processing.service.tsModify:
APIs/backend-vitto-api/src/app/mind/mind.module.tsModify:
APIs/backend-vitto-api/src/app/engine/engine.module.tsStep 1: Read current files to understand exact integration points
Read these files fully:
engine/context-assembly/channel-prompt-builder.service.ts(find the hardcoded 10/20 limits)engine/channel-processing.service.ts(find the hardcoded 10 limit)mind/mind.module.ts(understand current provider list)Step 2: Register new services in mind.module.ts
Add to the providers array:
import { ConversationGenomeService } from './virtual-context/conversation-genome.service';
import { WindowManagerService } from './virtual-context/window-manager.service';
// Add to providers:
ConversationGenomeService,
WindowManagerService,
Also export them so engine module can inject them.
- Step 3: Update channel-prompt-builder.service.ts
Find the loadChannelSessionHistory() method:
Replace
this.agentSessions.getConversationHistory(userId, 20)withthis.agentSessions.getConversationHistory(userId, 100)(load full stored history)Replace
filterMessagesByChannel(allHistory, channel, 10)with dynamic window fromWindowManagerServiceRemove the aggressive topic detection block (lines ~261-284 in the current code) — the genome handles topic awareness now
Inject
WindowManagerServiceandConversationGenomeServiceinto the constructorAfter filtering, call
windowManager.evictAndCompress()and injectgenomeService.hydrate()into the system prompt as[CONVERSATION CONTEXT]sectionStep 4: Update channel-processing.service.ts
Find getConversationHistory(userId, 10):
Replace
10with100(load full stored history)Inject
WindowManagerServiceUse
windowManager.evictAndCompress()to get the active window + genomePass both to the agent loop
Step 5: Run existing tests to verify no regressions
Run: npx jest APIs/backend-vitto-api/src/app/engine/__tests__/ --no-coverage
Expected: All existing tests should still pass (or update any that assert on the old limit values)
- Step 6: Commit
git add APIs/backend-vitto-api/src/app/mind/mind.module.ts APIs/backend-vitto-api/src/app/engine/engine.module.ts APIs/backend-vitto-api/src/app/engine/context-assembly/channel-prompt-builder.service.ts APIs/backend-vitto-api/src/app/engine/channel-processing.service.ts
git commit -m "[S239.4] feat: wire virtual context memory into context assembly pipeline"
Phase 2: Intelligence Layer — Admission, Routing, Compression
Task 5: Dopamine Router Service
Files:
Create:
APIs/backend-vitto-api/src/app/mind/recall/dopamine-router.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/recall/__tests__/dopamine-router.service.spec.tsStep 1: Write failing tests for route classification
Test cases: greeting → fast, simple question → standard, "compare X and Y" → deep, short message → fast, multiple entities → deep.
Step 2: Run test to verify it fails
Step 3: Implement DopamineRouterService
Heuristic-only (NO LLM call). Classify using: message length, question word complexity, entity count, conversation position. Return { route, scopes, maxResults, includeGraphTraversal, includeProspectiveMatching }.
Step 4: Run tests, verify pass
Step 5: Commit
git commit -m "[S239.5] feat: implement dopamine-gated query router"
Task 6: Memory Admission Service
Files:
Create:
APIs/backend-vitto-api/src/app/mind/admission/memory-admission.types.tsCreate:
APIs/backend-vitto-api/src/app/mind/admission/memory-admission.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/admission/__tests__/memory-admission.service.spec.tsStep 1: Write failing tests for 5-factor scoring
Test cases: correction type → high score, casual conversation → low score, duplicate content → low novelty score, quarantine threshold behavior.
Step 2: Run test to verify it fails
Step 3: Implement MemoryAdmissionService
5 factors: futureUtility, factualConfidence, semanticNovelty, temporalRecency, contentTypePrior. Weights: 0.25/0.20/0.25/0.10/0.20. Thresholds: admit ≥ 0.50, quarantine 0.25-0.50, reject < 0.25. Novelty check uses existing VittoMemoryStoreService.textSimilarity() pattern.
- Step 4: Wire into VittoMemoryStoreService.remember()
Add admission check before the Mem0 consolidation logic. If quarantined, set status: 'quarantined' on the memory document.
Step 5: Run all mind tests, verify no regressions
Step 6: Commit
git commit -m "[S239.6] feat: implement 5-factor memory admission control with quarantine"
Task 7: Memory Conflict Resolver
Files:
Create:
APIs/backend-vitto-api/src/app/mind/conflict/memory-conflict-resolver.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/conflict/__tests__/memory-conflict-resolver.service.spec.tsStep 1: Write failing tests
Test: user preference overrides org preference, org policy (overridable: false) overrides user preference, most recent fact wins for factual conflicts, same scope + same topic → highest importance wins.
Step 2: Implement MemoryConflictResolverService
Step 3: Run tests, verify pass
Step 4: Commit
git commit -m "[S239.7] feat: implement scope-aware memory conflict resolver"
Task 8: Scope-Aware Recall
Files:
Modify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory-recall.service.tsModify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory-hot-cache.service.tsModify:
APIs/backend-vitto-api/src/app/mind/vitto-memory.service.tsStep 1: Read current recall service fully
Step 2: Update vectorSearch() to accept scope filters
In the $vectorSearch aggregation pipeline, add scope and scopeId to the filter expression alongside existing userId, spaceId, type, layer filters. Also add status: 'active' to exclude quarantined memories from normal recall.
Step 3: Update textSearch() with same scope filters
Step 4: Update hot cache to include scope in cache keys
The cache key should incorporate scope to avoid cross-scope cache hits.
- Step 5: Integrate DopamineRouterService into recall pipeline
The VittoMemoryService.recall() method should call dopamineRouter.classifyRoute() and pass the result to VittoMemoryRecallService. Fast route skips cold recall. Deep route includes graph traversal.
Step 6: Run existing recall tests + add new scope tests
Step 7: Commit
git commit -m "[S239.8] feat: scope-aware recall with dopamine-gated routing"
Phase 3: Speed Layer — Redis, Embeddings, Prompt Caching
Task 9: Redis Vector Service
Files:
Create:
APIs/backend-vitto-api/src/app/mind/recall/redis-vector.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/recall/__tests__/redis-vector.service.spec.tsStep 1: Verify Redis is available in app.module.ts
Read app.module.ts to confirm RedisModule import and understand the existing Redis connection pattern. The service should inject the existing Redis client.
- Step 2: Write failing test with mocked Redis client
Test: addVector stores correctly, search returns sorted results, isAvailable returns false when disconnected, removeVector works.
- Step 3: Implement RedisVectorService
Use ioredis commands: VSET.ADD for adding vectors, VSIM for similarity search. Graceful degradation: if Redis is unavailable or VSET commands fail (Redis < 8.0), isAvailable() returns false and all methods are no-ops.
- Step 4: Wire into recall pipeline
In VittoMemoryRecallService, add Redis as Tier 1.5 (after hot cache, before MongoDB). Only query Redis if redisVector.isAvailable().
- Step 5: Wire write-through on memory creation
In VittoMemoryStoreService.remember(), after MongoDB insert succeeds, fire-and-forget redisVector.addVector().
- Step 6: Run tests, commit
git commit -m "[S239.9] feat: Redis Vector Sets integration as hot recall tier"
Task 10: Matryoshka Embedding Migration
Files:
Modify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory-embedding.service.tsModify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory.types.tsStep 1: Read current embedding service
Step 2: Add dual-index support
Add EMBEDDING_DIMENSIONS_V2 = 256 constant. Add VITTO_EMBEDDING_USE_DUAL_INDEX env var check. When dual-index is active, generate both 1536-dim (into embedding field) and 256-dim (into embedding_256 field). When dual-index is off + dimensions=256, only generate 256-dim into embedding field.
- Step 3: Update OpenAI API call to pass
dimensionsparameter
The text-embedding-3-small API supports dimensions: 256. Add this parameter when generating 256-dim embeddings.
- Step 4: Create backfill endpoint
Add a controller endpoint POST /engine/memories/backfill-embeddings (admin-only) that iterates all memories without embedding_256 and generates 256-dim embeddings in batches of 50.
- Step 5: Test, commit
git commit -m "[S239.10] feat: 256-dim Matryoshka embedding support with dual-index transition"
Task 11: TOON Serializer & Soul Compression
Files:
Create:
APIs/backend-vitto-api/src/app/engine/context-assembly/toon-serializer.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/soul-compression.service.tsStep 1: Add toon-format dependency
Run: yarn add toon-format (or implement a lightweight TOON serializer if the package is unavailable — TOON is essentially YAML-style for flat arrays + CSV-style for uniform objects)
- Step 2: Implement ToonSerializerService
Wrap the toon-format serialize function. If package unavailable, implement a minimal TOON serializer that handles arrays of objects (the main use case for memory/tool results).
- Step 3: Implement SoulCompressionService
Compile soul config (character + values + mood + persona) into ~150-token TOON brief. Cache with hash-based invalidation. needsFullSoul(query) heuristic: return true if query contains identity/personality-related keywords.
- Step 4: Wire into context assembly
Inject ToonSerializerService into ChannelPromptBuilderService and WebPromptBuilderService. Serialize recalled memories and user brief data in TOON format instead of JSON.
- Step 5: Test, commit
git commit -m "[S239.11] feat: TOON serialization and soul compression for context efficiency"
Phase 4: Innovation Layer — Prospective Indexing, Plugins
Task 12: Cognitive Plugin System
Files:
Create:
APIs/backend-vitto-api/src/app/mind/plugins/cognitive-plugin.interface.tsCreate:
APIs/backend-vitto-api/src/app/mind/plugins/cognitive-plugin-registry.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/plugins/__tests__/cognitive-plugin-registry.service.spec.tsStep 1: Define plugin interface
// cognitive-plugin.interface.ts
import { RememberParams, RecallResult } from '../memory/vitto-memory.types';
export interface AssemblyContext {
userId: string;
spaceId?: string;
universeId?: string;
query: string;
tokenBudget: number;
recalledMemories: RecallResult[];
}
export interface ResponseContext {
userId: string;
query: string;
assembledPrompt: string;
recalledMemories: RecallResult[];
}
export interface ResponseResult {
userId: string;
query: string;
response: string;
toolCalls: number;
}
export interface AdmissionOverride {
forceAdmit?: boolean;
forceReject?: boolean;
modifiedCandidate?: RememberParams;
}
export interface ConsolidationResult {
merged?: string[];
promoted?: string[];
demoted?: string[];
}
export interface CognitivePlugin {
name: string;
version: string;
enabled: boolean;
onBeforeContextAssembly?(context: AssemblyContext): Promise<AssemblyContext>;
onBeforeResponse?(context: ResponseContext): Promise<ResponseContext>;
onAfterResponse?(response: ResponseResult): Promise<void>;
onMemoryAdmission?(candidate: RememberParams): Promise<AdmissionOverride | null>;
onMemoryConsolidation?(memoryIds: string[]): Promise<ConsolidationResult>;
onSessionStart?(session: { userId: string; sessionId: string }): Promise<void>;
onSessionEnd?(session: { userId: string; sessionId: string }): Promise<void>;
}
- Step 2: Write failing test for registry
Test: register/unregister plugins, executeHook runs in order, timeout kills slow plugins (500ms for post-response, 100ms for pre-response), disabled plugins are skipped.
Step 3: Implement CognitivePluginRegistryService
Step 4: Register built-in plugins
Create the 6 built-in plugin files in plugins/built-in/. Each wraps an existing service (AdmissionService, DopamineRouter, etc.) as a plugin. Register them in mind.module.ts via OnModuleInit.
- Step 5: Test, commit
git commit -m "[S239.12] feat: cognitive plugin system with registry and 6 built-in plugins"
Task 13: Prospective Indexer Service
Files:
Create:
APIs/backend-vitto-api/src/app/mind/indexing/prospective-indexer.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/indexing/__tests__/prospective-indexer.service.spec.tsStep 1: Write failing test
Test: generates 3-5 questions for a user_preference memory, questions are stored as supplementary embeddings, works with mocked LLM provider.
- Step 2: Implement ProspectiveIndexerService
Use the existing LLM provider (cheapest model via smart routing). Generate 3-5 hypothetical future questions. Store each question's embedding in Redis (linked to memoryId). Fire-and-forget — don't block the write path.
- Step 3: Wire as a cognitive plugin
The ProspectiveIndexPlugin calls prospectiveIndexer.generateProspectiveQuestions() in the onMemoryConsolidation hook.
- Step 4: Test, commit
git commit -m "[S239.13] feat: prospective indexing generates future-question embeddings at write time"
Task 14: On-Demand Recall Service
Files:
Create:
APIs/backend-vitto-api/src/app/mind/virtual-context/on-demand-recall.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/on-demand-recall.service.spec.tsStep 1: Write failing test
Test: detects explicit references ("earlier you said"), detects implicit references (topic in genome but not in active window), returns relevant messages from full session history.
- Step 2: Implement OnDemandRecallService
Pattern matching for explicit triggers. Genome topic relevance check for implicit triggers. Uses Redis Vector Sets (or MongoDB fallback) to search full session message history. Returns top 3-5 relevant messages.
- Step 3: Wire into context assembly
When recall is triggered, inject [RECALLED FROM EARLIER IN CONVERSATION] block before the active window in the prompt.
- Step 4: Test, commit
git commit -m "[S239.14] feat: on-demand recall for mid-conversation references"
Phase 5: Memory Management UI
Task 15: Memory Scope Filter Component
Files:
Create:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-scope-filter/memory-scope-filter.component.tsCreate:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-scope-filter/memory-scope-filter.component.htmlCreate:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-scope-filter/memory-scope-filter.component.scssModify:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-manager.component.tsModify:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-manager.component.htmlStep 1: Read current memory-manager component fully
Understand the existing filter, search, and display patterns.
- Step 2: Create MemoryScopeFilterComponent
Standalone Angular component with OnPush. Inputs: selectedScope signal, availableScopes. Output: scopeChanged event. Renders a dropdown/chip selector with scope icons (🏢 Org, 👥 Space, 🌐 Universe, 👤 User).
- Step 3: Integrate into memory-manager.component
Add the scope filter alongside the existing type dropdown. Update the browseMemories() call to include scope parameter. Display scope badge on each memory card.
- Step 4: Update VittoApiService with scope parameter
Add scope to browseMemories() and searchMemories() query params.
- Step 5: Test visually in browser
Navigate to localhost:6400/vitto/memory. Verify scope filter appears, filtering works, badges display.
- Step 6: Commit
git commit -m "[S239.15] feat: memory scope filter UI with scope badges on cards"
Task 16: Memory Version History & Quarantine View
Files:
Create:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-version-history/memory-version-history.component.tsCreate:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-version-history/memory-version-history.component.htmlCreate:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-version-history/memory-version-history.component.scssModify:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-manager.component.tsStep 1: Create MemoryVersionHistoryComponent
Standalone, OnPush. Input: history array from memory document. Renders expandable timeline of previous versions with diff highlighting.
- Step 2: Add quarantine toggle to memory manager
Add a toggle button "Show Quarantined" that switches to status: 'quarantined' filter. Show promote/reject action buttons on quarantined memory cards. Wire to new backend endpoints: POST /engine/memories/:id/promote and POST /engine/memories/:id/reject.
- Step 3: Create backend endpoints for promote/reject
In the existing memory controller, add promote (sets status to 'active') and reject (deletes permanently) endpoints.
- Step 4: Test visually, commit
git commit -m "[S239.16] feat: memory version history and quarantine management UI"
Task 17: Genome Viewer Component
Files:
Create:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/genome-viewer/genome-viewer.component.tsCreate:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/genome-viewer/genome-viewer.component.htmlCreate:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/genome-viewer/genome-viewer.component.scssStep 1: Create GenomeViewerComponent
Standalone, OnPush. Real-time display of the current session's conversation genome. Shows: topics (active/resolved with color coding), decisions, facts, action items, token count, compression ratio. Fetches via new endpoint GET /engine/session/:sessionId/genome.
- Step 2: Create backend genome endpoint
In the existing mind controller, add GET /mind/session/:sessionId/genome that returns the current genome for the session.
- Step 3: Add to Memory Health route or as tab in memory manager
Wire into the existing /vitto/memory-health route as a section, or as a tab in the memory layout.
- Step 4: Test visually, commit
git commit -m "[S239.17] feat: real-time genome viewer showing conversation compression"
Task 18: Export to Docs-Angular
Files:
Create:
APIs/backend-vitto-api/src/app/mind/export/memory-docs-export.service.tsStep 1: Implement MemoryDocsExportService
Generates organized markdown files from Vitto's memory, grouped by scope (organization/, spaces/{name}/, universes/{name}/, users/{name}/). POSTs to backend-docs-api POST /sync/bulk using the existing docs sync protocol. Creates a vitto-memory project in doc-projects if it doesn't exist.
- Step 2: Create controller endpoint
POST /engine/memories/export-to-docs (admin-only). Optionally accepts scope filter. Returns export report (files created, token counts).
- Step 3: Add "Export to Docs" button in memory manager
Wire to the new endpoint. Show success toast with link to docs app.
- Step 4: Test, commit
git commit -m "[S239.18] feat: memory export to docs-angular as organized markdown"
Phase 6: E2E Testing
Task 19: E2E Test — In-Session Memory Continuity
Files:
Create:
APIs/backend-vitto-api/src/app/mind/__tests__/e2e-memory-v2.spec.tsStep 1: Write E2E test simulating a 30-message conversation
describe('E2E: Session Continuity', () => {
it('should maintain context across 30+ messages via genome compression', async () => {
// 1. Create a test session
// 2. Send 30 messages with distinct topics (messages 1-10: topic A, 11-20: topic B, 21-30: topic A again)
// 3. At message 30, verify:
// - Active window contains recent messages
// - Genome contains compressed topic A from messages 1-10
// - User can reference topic A from message 5 and get relevant context
});
it('should handle topic switches without losing prior context', async () => {
// 1. Discuss topic A for 5 messages
// 2. Switch to topic B for 5 messages
// 3. Ask about topic A again
// 4. Verify genome preserves topic A context
});
});
- Step 2: Write E2E test for scope-based recall
describe('E2E: Scope-Based Recall', () => {
it('should recall organization memories for all users', async () => {
// 1. Create org-scoped memory
// 2. Query as different user
// 3. Verify org memory is included in recall
});
it('should NOT recall universe memories from a different universe', async () => {
// 1. Create universe-scoped memory for Universe A
// 2. Query in Universe B context
// 3. Verify memory is NOT returned
});
it('should resolve conflicts: user preference overrides org preference', async () => {
// 1. Create org memory: "respond formally"
// 2. Create user memory: "respond casually"
// 3. Recall both, run conflict resolver
// 4. Verify user preference wins
});
});
- Step 3: Write E2E test for admission control
describe('E2E: Admission Control', () => {
it('should admit corrections with high score', async () => {
// Store a correction → verify status='active'
});
it('should quarantine low-signal conversations', async () => {
// Store a generic "hi" conversation → verify status='quarantined'
});
it('should promote quarantined memory when topic resurfaces', async () => {
// 1. Store memory that gets quarantined
// 2. Store a new high-signal memory on same topic
// 3. Run quarantine review
// 4. Verify quarantined memory promoted to active
});
});
- Step 4: Write E2E test for vector search recall
describe('E2E: Vector Search Recall', () => {
it('should return semantically similar memories via vector search', async () => {
// 1. Store memory about "MongoDB setup"
// 2. Query "database configuration"
// 3. Verify the MongoDB memory is returned with similarity score
});
it('should fall back to text search when vector search unavailable', async () => {
// Mock vector search failure
// Verify text search returns results
});
});
- Step 5: Run all E2E tests
Run: npx jest APIs/backend-vitto-api/src/app/mind/__tests__/e2e-memory-v2.spec.ts --no-coverage --runInBand
Expected: All pass
- Step 6: Commit
git commit -m "[S239.19] test: comprehensive E2E tests for session continuity, scopes, admission, and recall"
Task 20: E2E Test — Full Pipeline Integration
Files:
Create:
APIs/backend-vitto-api/src/app/engine/__tests__/e2e-memory-pipeline.spec.tsStep 1: Write full pipeline E2E test
describe('E2E: Full Memory Pipeline', () => {
it('should complete write→recall cycle with admission + genome + scoped recall', async () => {
// 1. Set up user context (space, universe)
// 2. Simulate a conversation (10 messages)
// 3. Verify genome is generated after eviction
// 4. Store a decision memory
// 5. Verify it passes admission control
// 6. Recall it via semantic search
// 7. Verify it appears in correct scope
// 8. Verify conflict resolution with an org memory
});
it('should maintain performance under load', async () => {
// 1. Store 100 memories across 4 scopes
// 2. Time recall operations
// 3. Verify hot cache hits < 5ms
// 4. Verify vector search < 500ms
});
});
- Step 2: Run all tests
Run: npx jest APIs/backend-vitto-api/ --no-coverage --runInBand
Expected: All pass (existing + new)
- Step 3: Commit
git commit -m "[S239.20] test: full pipeline E2E integration tests"
Phase 7: Library Consolidation & Deprecation Cleanup
Task 21: Update Frontend Types & Models
Files:
Create:
libs/libs-ai/vitto-core/src/lib/models/memory-v2.models.tsModify:
libs/libs-ai/vitto-core/src/lib/services/vitto-api.service.tsModify:
libs/libs-ai/vitto-hub/src/lib/components/memory-manager/memory-manager.component.tsStep 1: Create shared memory v2 models
// libs/libs-ai/vitto-core/src/lib/models/memory-v2.models.ts
export type MemoryScope = 'organization' | 'space' | 'universe' | 'user';
export type MemoryStatus = 'active' | 'quarantined' | 'archived';
export interface MemoryEntry {
_id: string;
type: string;
content: string;
tags: string[];
importance: number;
scope: MemoryScope; // NEW
scopeId: string; // NEW
status: MemoryStatus; // NEW
overridable: boolean; // NEW
history: MemoryVersion[]; // NEW
source?: string;
userId?: string;
spaceId?: string;
createdAt?: string;
updatedAt?: string;
keyEntities?: string[];
outcome?: string;
attachments?: MemoryAttachment[];
}
export interface MemoryVersion {
summary: string;
tags: string[];
editedBy: string;
editedAt: string;
}
export interface MemoryAttachment {
type: string;
url: string;
description?: string;
}
- Step 2: Update VittoApiService methods
Add scope and status parameters to browseMemories(), searchMemories(). Add new methods: promoteMemory(memoryId), rejectMemory(memoryId), getSessionGenome(sessionId), exportMemoriesToDocs(options), backfillEmbeddings().
- Step 3: Update memory-manager.component.ts
Replace the local MemoryEntry interface with the imported shared one from memory-v2.models.ts. Remove any duplicate type definitions.
- Step 4: Ensure public-api.ts exports new models
Check the barrel export file (libs/libs-ai/vitto-core/src/index.ts or public-api.ts) exports the new models.
- Step 5: Commit
git commit -m "[S239.21] refactor: shared memory v2 models and updated VittoApiService"
Task 22: Deprecate Old Layer System
Files:
Modify:
APIs/backend-vitto-api/src/app/mind/schemas/vitto-memory.schema.tsModify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory-recall.service.tsModify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory.types.tsStep 1: Add JSDoc deprecation notices
Mark the layer field in the schema with @deprecated Use scope instead. Mark layer in RecallOptions with @deprecated Use scope instead.
- Step 2: Add backward compatibility mapping
In VittoMemoryRecallService, if options.layer is provided but options.scope is not, map: layer='developer' → scope=['organization','space','universe','user'], layer='space_builder' → scope=['space','universe','user']. This ensures old callers still work.
- Step 3: Update existing code that writes
layerfield
Search for all layer: assignments in the codebase. Update them to also set scope and scopeId. Keep layer for backward compat during transition.
- Step 4: Run all tests
Run: npx jest APIs/backend-vitto-api/ --no-coverage
Expected: All pass
- Step 5: Commit
git commit -m "[S239.22] refactor: deprecate layer field in favor of scope system"
Task 23: Update NX Library Exports & Index Files
Files:
Modify:
libs/libs-ai/vitto-core/src/index.tsModify:
libs/libs-ai/vitto-hub/src/index.tsVerify: all new components are exported correctly
Step 1: Audit vitto-core exports
Read libs/libs-ai/vitto-core/src/index.ts. Ensure memory-v2.models.ts types are exported. Remove any exports of deprecated types that have been replaced.
- Step 2: Audit vitto-hub exports
Read libs/libs-ai/vitto-hub/src/index.ts. Ensure new components (MemoryScopeFilter, MemoryVersionHistory, GenomeViewer) are exported if they're used outside the library.
- Step 3: Verify no circular dependencies
Run: npx nx graph --focus=libs-ai-vitto-core
Check for circular imports between vitto-core and vitto-hub.
- Step 4: Run full build to verify
Run: npx nx build libs-ai-vitto-core and npx nx build libs-ai-vitto-hub
Expected: Build succeeds with no errors
- Step 5: Commit
git commit -m "[S239.23] chore: update NX library exports and verify no circular dependencies"
Task 24: Remove Dead Code & Final Cleanup
Files:
Audit all modified files for unused imports, dead code, orphaned types
Step 1: Search for unused imports in modified files
Check every file modified in this plan for: unused imports, unreferenced variables, commented-out code that was replaced.
- Step 2: Remove the hardcoded message limits
Verify that NO file in backend-vitto-api still has hardcoded getConversationHistory(userId, 10) or getConversationHistory(userId, 20). All should use the window manager.
- Step 3: Verify all new services are registered in modules
Cross-reference: every new service file created in this plan should be registered in either mind.module.ts or engine.module.ts. Check that none are missing.
- Step 4: Run full test suite
Run: npx jest APIs/backend-vitto-api/ --no-coverage
Expected: All tests pass
- Step 5: Run linting
Run: npx nx lint backend-vitto-api
Expected: No errors
- Step 6: Final commit
git commit -m "[S239.24] chore: remove dead code, verify module registrations, final cleanup"
Phase 8: Deferred Spec Items
These tasks complete the remaining spec items that build on top of Phases 1-7.
Task 25: Memory Versioning Write-Side Logic
Files:
Modify:
APIs/backend-vitto-api/src/app/mind/memory/vitto-memory-store.service.tsStep 1: Read current editMemory/updateMemory logic in store service
Step 2: Add version history population on edit
Before overwriting summary or tags, push the previous values into the history array:
// In the update method, before applying changes:
const existing = await this.vittoMemoryModel.findOne({ memoryId });
if (existing && (existing.summary !== newSummary || JSON.stringify(existing.tags) !== JSON.stringify(newTags))) {
await this.vittoMemoryModel.updateOne(
{ memoryId },
{ $push: { history: { summary: existing.summary, tags: existing.tags, editedBy: userId, editedAt: new Date() } } }
);
}
Step 3: Add test for version history creation
Step 4: Commit
git add APIs/backend-vitto-api/src/app/mind/memory/vitto-memory-store.service.ts
git commit -m "[S239.25] feat: populate memory version history on edits"
Task 26: Quarantine Cron Service
Files:
Create:
APIs/backend-vitto-api/src/app/mind/admission/quarantine.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/admission/__tests__/quarantine.service.spec.tsStep 1: Write failing test
Test: auto-promote if topic resurfaced (similarity > 0.8 with recently admitted memory), permanent delete after 90 days, skip memories less than 30 days old.
- Step 2: Implement QuarantineService
Monthly cron (@Cron('0 3 1 * *')). Queries vitto-memories with status: 'quarantined'. For each, check similarity against recently admitted memories (last 30 days). Auto-promote if match found. Delete if older than 90 days and no match.
Step 3: Register in mind.module.ts
Step 4: Run tests, commit
git add APIs/backend-vitto-api/src/app/mind/admission/quarantine.service.ts APIs/backend-vitto-api/src/app/mind/admission/__tests__/quarantine.service.spec.ts APIs/backend-vitto-api/src/app/mind/mind.module.ts
git commit -m "[S239.26] feat: quarantine cron service with auto-promotion and cleanup"
Task 27: ConversationGenomeService v2 (LLM-Powered Recompile)
Files:
Modify:
APIs/backend-vitto-api/src/app/mind/virtual-context/conversation-genome.service.tsStep 1: Add fullRecompile() method using cheap LLM
Inject the LLM provider service. Call Haiku (or cheapest available model) with a structured prompt that extracts topics, decisions, facts, and action items from the full message array. Called every 10 evictions (tracked by version % 10 === 0 check in incrementalUpdate).
Step 2: Add test for fullRecompile with mocked LLM
Step 3: Wire the recompile trigger into WindowManagerService.evictAndCompress()
After incrementalUpdate, check if genome.version % 10 === 0 and trigger fullRecompile() asynchronously.
- Step 4: Run tests, commit
git add APIs/backend-vitto-api/src/app/mind/virtual-context/conversation-genome.service.ts APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/conversation-genome.service.spec.ts
git commit -m "[S239.27] feat: LLM-powered genome recompile every 10 evictions"
Task 28: User Preference Learning Plugin
Files:
Modify:
APIs/backend-vitto-api/src/app/mind/plugins/built-in/user-preference-learning.plugin.tsCreate:
APIs/backend-vitto-api/src/app/mind/plugins/built-in/__tests__/user-preference-learning.plugin.spec.tsStep 1: Write failing test
Test: detects explicit corrections ("no, shorter answers"), detects implicit preferences (user consistently ignores long responses), stores as user_preference memory with high importance.
- Step 2: Implement detection logic in onAfterResponse hook
Heuristic detection patterns:
Explicit: "no", "don't", "stop doing", "I prefer", "please use", "shorter", "longer", "in Spanish"
Implicit: user repeatedly edits/corrects same type of output Store detected preferences via
VittoMemoryService.remember({ type: 'user_preference', scope: 'user' }).Step 3: Run tests, commit
git add APIs/backend-vitto-api/src/app/mind/plugins/built-in/user-preference-learning.plugin.ts APIs/backend-vitto-api/src/app/mind/plugins/built-in/__tests__/user-preference-learning.plugin.spec.ts
git commit -m "[S239.28] feat: user preference learning plugin detects corrections and preferences"
Task 29: Data Migration Script
Files:
Create:
APIs/backend-vitto-api/src/app/mind/scripts/migrate-scope-fields.tsStep 1: Write migration script
// Adds scope='user' and scopeId=userId to all existing memories that lack scope field
// Run: npx ts-node APIs/backend-vitto-api/src/app/mind/scripts/migrate-scope-fields.ts
async function migrateScopes() {
// Connect to vitto-mind database
// Find all docs where scope doesn't exist or is null
// Update: scope='user', scopeId=doc.userId, status='active'
// Report: X documents migrated
}
Step 2: Test on a small batch first (limit 10, dryRun=true)
Step 3: Run full migration
Step 4: Commit
git add APIs/backend-vitto-api/src/app/mind/scripts/migrate-scope-fields.ts
git commit -m "[S239.29] feat: data migration script for scope fields on existing memories"
Task 30: Prompt Caching Optimization
Files:
Modify:
APIs/backend-vitto-api/src/app/engine/context-assembly/context-assembly.service.tsModify:
APIs/backend-vitto-api/src/app/engine/context-assembly/channel-prompt-builder.service.tsStep 1: Read current context assembly to understand prompt structure
Step 2: Reorder system prompt for cache-friendly structure
Ensure the prompt is assembled in this exact order (stable prefix first):
- Soul brief (TOON, ~150 tokens) — STABLE across turns
- System instructions (~500 tokens) — STABLE across turns
- User brief (TOON, ~200 tokens) — STABLE within session
- Conversation genome (~300-500 tokens) — changes every ~10 turns
- Recalled memories — PER TURN
- Active conversation window — PER TURN
Add a cache_control: { type: 'ephemeral' } breakpoint after the stable prefix for Anthropic prompt caching.
- Step 3: Ensure deterministic serialization
Add a sort to all TOON serialized objects to guarantee identical output for identical input. Verify the soul brief produces the same string for the same configuration.
- Step 4: Run tests, commit
git add APIs/backend-vitto-api/src/app/engine/context-assembly/context-assembly.service.ts APIs/backend-vitto-api/src/app/engine/context-assembly/channel-prompt-builder.service.ts
git commit -m "[S239.30] feat: cache-friendly prompt structure for Anthropic prompt caching"
Task 31: Cross-Channel UserGenomeService
Files:
Create:
APIs/backend-vitto-api/src/app/mind/virtual-context/user-genome.service.tsCreate:
APIs/backend-vitto-api/src/app/mind/schemas/user-genome.schema.tsCreate:
APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/user-genome.service.spec.tsStep 1: Create Mongoose schema for user-genomes collection
Single document per user. Aggregates topics, decisions, facts across all sessions. 7-day topic eviction. Max 1K tokens.
- Step 2: Write failing test
Test: aggregates genomes from 2 different sessions, evicts topics older than 7 days, injects into context assembly after user brief.
- Step 3: Implement UserGenomeService
After each session ends (or every 5 interactions), merge the session genome into the user genome. Lightweight heuristic merge (no LLM). Every 5 merges, recompile via Haiku.
- Step 4: Wire into context assembly
Inject user genome AFTER user brief, BEFORE channel-specific history. This provides cross-channel awareness when user switches from WhatsApp to web.
Step 5: Register schema and service in mind.module.ts
Step 6: Run tests, commit
git add APIs/backend-vitto-api/src/app/mind/virtual-context/user-genome.service.ts APIs/backend-vitto-api/src/app/mind/schemas/user-genome.schema.ts APIs/backend-vitto-api/src/app/mind/virtual-context/__tests__/user-genome.service.spec.ts APIs/backend-vitto-api/src/app/mind/mind.module.ts
git commit -m "[S239.31] feat: cross-channel UserGenomeService for multi-channel continuity"
Task 32: Wire agent-loop.service.ts with Genome
Files:
Modify:
APIs/backend-vitto-api/src/app/engine/vitto-engine.service.ts(oragent-loop.service.ts)Step 1: Read agent-loop.service.ts to find message injection point
Step 2: Inject genome + active window instead of raw message array
Replace the raw conversationHistory injection with: genome hydration text prepended as a system-like message, followed by the active window messages from WindowManagerService.
Step 3: Run existing agent-loop tests to verify no regression
Step 4: Commit
git add APIs/backend-vitto-api/src/app/engine/vitto-engine.service.ts
git commit -m "[S239.32] feat: inject genome + active window into agent loop"
Summary
| Phase | Tasks | Focus |
|---|---|---|
| Phase 1: Critical Fixes | 1-4 | Schema scopes, ConversationGenome, WindowManager, context assembly wiring |
| Phase 2: Intelligence | 5-8 | Dopamine router, admission control, conflict resolver, scope-aware recall |
| Phase 3: Speed | 9-11 | Redis Vector Sets, 256-dim embeddings, TOON + soul compression |
| Phase 4: Innovation | 12-14 | Plugin system, prospective indexing, on-demand recall |
| Phase 5: UI | 15-18 | Scope filter, version history, genome viewer, docs export |
| Phase 6: E2E Testing | 19-20 | Session continuity, scope recall, admission, full pipeline |
| Phase 7: Library Cleanup | 21-24 | Shared models, deprecate layer, NX exports, dead code removal |
| Phase 8: Deferred Spec Items | 25-32 | Version history write-side, quarantine cron, genome v2 LLM, user preference plugin, data migration, prompt caching, cross-channel genome, agent-loop wiring |
Total: 32 tasks across 8 phases.