MCP Intelligent Memory System - Proposal
Revolutionary upgrade to MCP using hierarchical memory and vector search
MCP Intelligent Memory System
Goal: Solve the context window limitation with a three-tier memory architecture inspired by Anthropic's approach.
📋 Problem Analysis
Current Issues
| Issue | Impact | Severity |
|---|---|---|
| Linear text search | O(n) scan through all decisions | High |
| No semantic understanding | Keyword matching misses related concepts | High |
| Flat memory structure | All decisions equal weight | Medium |
| No auto-compression | Context fills up quickly | Critical |
| No session continuity | Each chat starts fresh | High |
What Anthropic Does
From their Context Management research:
- Context Editing - Auto-remove stale tool calls when approaching limits
- Memory Tool - Store/retrieve outside context window via files
- Initializer Agent - Sets up environment with prior context
- Clear Artifacts - Leave breadcrumbs for future sessions
🎯 Proposed Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ THREE-TIER INTELLIGENT MEMORY SYSTEM │
│ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ TIER 1: WORKING MEMORY (Hot) ~50 tokens │ │
│ │ ──────────────────────────────────────────────────────────────────── │ │
│ │ • Current session decisions (last 5) │ │
│ │ • In-memory Map, instant access (<1ms) │ │
│ │ • Auto-prunes when session changes │ │
│ │ • Format: { sessionID, summary, keyChoices } │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ TIER 2: SEMANTIC INDEX (Warm) ~200 tokens │ │
│ │ ──────────────────────────────────────────────────────────────────── │ │
│ │ • Vector embeddings via OpenAI/Voyage │ │
│ │ • MongoDB Atlas Vector Search │ │
│ │ • Cosine similarity (semantic, not keyword!) │ │
│ │ • Filters: session, success rate, decision type │ │
│ │ • Returns: Top 3 most relevant (compressed) │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ TIER 3: SESSION SUMMARIES (Cold) ~100 tokens │ │
│ │ ──────────────────────────────────────────────────────────────────── │ │
│ │ • Auto-generated session summaries │ │
│ │ • Pattern clusters (e.g., "Angular patterns", "Backend patterns") │ │
│ │ • Historical success rates per decision type │ │
│ │ • Only loaded on explicit request │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
│ TOTAL BUDGET: ~350 tokens (vs current ~2000) │
└─────────────────────────────────────────────────────────────────────────────┘
🔧 Implementation Plan
Phase 1: Enhance Decision Schema
// Current schema (flat)
interface DevelopmentDecision {
id: string;
timestamp: Date;
type: string;
question: string;
options: string[];
chosen: string;
reasoning: string;
confidence: number;
session?: SessionMetadata;
// ...
}
// Enhanced schema (hierarchical)
interface EnhancedDecision {
id: string;
timestamp: Date;
// 🆕 TIER INDICATORS
tier: 'hot' | 'warm' | 'cold';
lastAccessedAt: Date;
accessCount: number;
// 🆕 COMPRESSED FIELDS
summary: string; // Auto-generated 1-line summary
keyChoice: string; // Single most important choice
// 🆕 VECTOR EMBEDDING
embedding?: number[]; // 1536-dim vector
embeddingModel?: string; // 'text-embedding-3-small'
embeddedAt?: Date;
// 🆕 SESSION CONTEXT (always included!)
session: {
number: number; // 9
id: string; // "9.2"
title: string; // "AI Panel Integration"
isActive: boolean;
};
// Original fields (compressed on cold tier)
type: string;
question: string;
chosen: string;
reasoning: string;
confidence: number;
// 🆕 OUTCOME TRACKING
outcome?: {
wasSuccessful: boolean;
impactScore: number; // 1-10
learnings: string[]; // Key takeaways
};
}
Phase 2: Vector Search Integration
// MCP calls backend API for vector search
async function semanticSearch(query: string, options: {
sessionFilter?: number;
maxResults?: number;
minSimilarity?: number;
}): Promise<CompressedDecision[]> {
// 1. Generate embedding for query
const queryEmbedding = await generateEmbedding(query);
// 2. Vector search in MongoDB
const results = await embeddingsCollection.aggregate([
{
$vectorSearch: {
index: "decision_embeddings",
queryVector: queryEmbedding,
path: "embedding",
numCandidates: 50,
limit: options.maxResults || 5,
filter: options.sessionFilter
? { "session.number": options.sessionFilter }
: {}
}
},
{
$project: {
id: 1,
summary: 1,
keyChoice: 1,
"session.id": 1,
confidence: 1,
"outcome.wasSuccessful": 1,
score: { $meta: "vectorSearchScore" }
}
}
]).toArray();
return results.filter(r => r.score > (options.minSimilarity || 0.7));
}
Phase 3: Intelligent Preflight (New Version)
async function intelligentPreflight(params: {
task: string;
currentSessionId: string;
}): Promise<IntelligentContext> {
const budget = { used: 0, max: 350 };
// TIER 1: Working Memory (~50 tokens)
const workingMemory = getWorkingMemory(params.currentSessionId);
budget.used += estimateTokens(workingMemory);
// TIER 2: Semantic Search (~200 tokens)
let semanticResults: CompressedDecision[] = [];
if (budget.used < budget.max - 100) {
semanticResults = await semanticSearch(params.task, {
sessionFilter: getCurrentSession().number,
maxResults: 3,
minSimilarity: 0.75
});
budget.used += estimateTokens(semanticResults);
}
// TIER 3: Session Summary (~100 tokens, only if budget allows)
let sessionSummary: string | undefined;
if (budget.used < budget.max - 50) {
sessionSummary = await getSessionSummary(params.currentSessionId);
budget.used += estimateTokens(sessionSummary);
}
return {
workingMemory, // Last 5 decisions (hot)
semanticMatches: semanticResults, // Top 3 relevant (warm)
sessionSummary, // Optional overview (cold)
tokenBudget: budget,
guidance: generateGuidance(semanticResults)
};
}
Phase 4: Auto-Summarization
// Generate summaries when decisions are logged
function autoSummarize(decision: DevelopmentDecision): string {
// Pattern: "[TYPE] [VERB] [KEY_NOUN] using [APPROACH]"
const verb = {
'architecture': 'Designed',
'implementation': 'Implemented',
'refactor': 'Refactored',
'bugfix': 'Fixed',
'pattern': 'Applied',
'convention': 'Established'
}[decision.type] || 'Decided';
// Extract key noun from question
const keyNoun = extractKeyNoun(decision.question);
// Use first 3 words of chosen
const approach = decision.chosen.split(' ').slice(0, 5).join(' ');
return `${verb} ${keyNoun} using ${approach}`;
// Example: "Implemented context retrieval using semantic vector search"
}
// Generate session summaries periodically
async function generateSessionSummary(sessionNumber: number): Promise<string> {
const decisions = getDecisionsBySession(sessionNumber);
if (decisions.length === 0) return "No decisions logged";
if (decisions.length <= 3) {
return decisions.map(d => d.summary).join('. ');
}
// Group by type
const byType = groupBy(decisions, 'type');
const parts: string[] = [];
for (const [type, typeDecisions] of Object.entries(byType)) {
const successRate = calculateSuccessRate(typeDecisions);
parts.push(`${typeDecisions.length} ${type} decisions (${successRate}% success)`);
}
return parts.join(', ');
// Example: "5 implementation decisions (80% success), 2 architecture decisions (100% success)"
}
📊 Token Budget Comparison
| Component | Current | Proposed | Savings |
|---|---|---|---|
| Decisions | ~1200 tokens (5 full) | ~150 tokens (5 compressed) | 88% |
| Conventions | ~400 tokens (3 full) | ~50 tokens (3 rules only) | 88% |
| Patterns | ~200 tokens (2 full) | ~50 tokens (2 names only) | 75% |
| Session Context | ~200 tokens | ~100 tokens (summary) | 50% |
| TOTAL | ~2000 tokens | ~350 tokens | 82% |
🔄 Data Flow
User Query: "How to integrate AI panel with canvas?"
│
▼
┌──────────────────────────────────────────────────────────────┐
│ STEP 1: Check Working Memory (Tier 1) │
│ ─────────────────────────────────────────────────────────── │
│ Current session (9) has 2 recent decisions: │
│ • "Implemented CDK drag-drop for AI modules" │
│ • "Added assignContentToSection() to canvas" │
│ → MATCH FOUND (session-local) │
└──────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────┐
│ STEP 2: Semantic Search (Tier 2) │
│ ─────────────────────────────────────────────────────────── │
│ Query embedding → Vector search in MongoDB │
│ Results: │
│ • Session 6: "Implemented thought streaming" (0.82 sim) │
│ • Session 8: "Created backend-ai-api for AI work" (0.78) │
│ → 2 semantically related decisions found │
└──────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────┐
│ STEP 3: Return Compressed Context │
│ ─────────────────────────────────────────────────────────── │
│ { │
│ workingMemory: ["CDK drag-drop", "assignContentToSection"]│
│ semanticMatches: ["thought streaming", "backend-ai-api"] │
│ guidance: "Similar integrations used CDK + canvas state" │
│ tokens: { used: 180, max: 350 } │
│ } │
└──────────────────────────────────────────────────────────────┘
🚀 Implementation Phases
Phase 1: Schema Enhancement (Day 1)
- Add
summary,keyChoiceto decision schema - Add
tier,lastAccessedAt,accessCountfields - Update
log_decisionto auto-generate summaries - Migrate existing decisions (add summaries)
Phase 2: Vector Integration (Day 2)
- Add embedding generation to MCP (call backend API)
- Create MongoDB vector index on
ai-memory.embeddings - Implement
semanticSearch()function - Add embedding on decision log (async)
Phase 3: Hierarchical Memory (Day 3)
- Implement working memory (in-memory Map)
- Implement session summaries (auto-generated)
- Create
intelligentPreflight()with tiers - Add tier promotion/demotion logic
Phase 4: Testing & Tuning (Day 4)
- Test with various query types
- Tune similarity thresholds
- Measure token usage
- Document best practices