Adaptive Learning Triggers - Learning When to Learn
Intelligent triggers based on patterns, not arbitrary time intervals
🎯 Adaptive Learning Triggers
The Problem with Time-Based Triggers
❌ "Run pattern extraction every week"
Problems:
├─ Week with 500 decisions: Extraction too late, patterns missed
├─ Week with 5 decisions: Wasted compute, nothing to learn
├─ Urgent pattern (same error 10x): Waits until Sunday
└─ Arbitrary: Why 7 days? Why not 3? Why not 14?
The Solution: Event-Driven Learning
✅ "Learn when there's something to learn"
Triggers:
├─ Volume: 100 new decisions since last extraction
├─ Repetition: Same question asked 3+ times
├─ Performance: Success rate drops below threshold
├─ Confidence: Average confidence declining
└─ Novelty: New category/domain detected
Trigger System Architecture
┌─────────────────────────────────────────────────────────────────┐
│ LEARNING TRIGGER SYSTEM │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ VOLUME │ │ REPETITION │ │ PERFORMANCE │ │
│ │ Counter │ │ Detector │ │ Monitor │ │
│ │ │ │ │ │ │ │
│ │ 100 new │ │ Same Q 3x │ │ <80% rate │ │
│ │ decisions │ │ in 24hrs │ │ for type │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ └──────────────────┼──────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ TRIGGER QUEUE │ │
│ │ │ │
│ │ Priority-based │ │
│ │ De-duplicated │ │
│ └────────┬────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ LEARNING JOB │ │
│ │ EXECUTOR │ │
│ │ │ │
│ │ Pattern extract│ │
│ │ Rule promotion │ │
│ │ Archive check │ │
│ └─────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
Trigger Types
1. Volume Trigger
interface VolumeTrigger {
type: 'volume';
threshold: number; // Default: 100 decisions
scope: 'global' | 'category' | 'agent';
currentCount: number;
lastTriggered: Date;
}
// Example: Different thresholds per category
const volumeThresholds = {
'build': 50, // Build issues: learn faster
'angular': 100, // Angular: standard
'architecture': 20, // Architecture: rare but important
'default': 100
};
2. Repetition Trigger
interface RepetitionTrigger {
type: 'repetition';
question: string;
occurrences: number;
threshold: number; // Default: 3
timeWindow: number; // Default: 24 hours
firstSeen: Date;
lastSeen: Date;
}
// When triggered: Extract pattern IMMEDIATELY
// "If the same question is asked 3+ times, we should have a pattern for it"
3. Performance Trigger
interface PerformanceTrigger {
type: 'performance';
metric: 'success_rate' | 'confidence' | 'reference_rate';
category: string;
currentValue: number;
threshold: number;
trend: 'declining' | 'stable' | 'improving';
}
// Example thresholds
const performanceThresholds = {
success_rate: 0.80, // Below 80% success: re-analyze
confidence: 0.75, // Below 75% avg confidence: need patterns
reference_rate: 0.30, // Below 30% patterns used: wrong patterns?
};
4. Novelty Trigger
interface NoveltyTrigger {
type: 'novelty';
detected: 'new_category' | 'new_technology' | 'new_pattern';
identifier: string;
occurrences: number;
firstSeen: Date;
}
// When new category detected (e.g., "AI integration" decisions appearing):
// - Lower threshold for learning (learn fast from new domain)
// - Higher urgency for pattern extraction
5. Confidence Drop Trigger
interface ConfidenceDropTrigger {
type: 'confidence_drop';
category: string;
previousAvg: number;
currentAvg: number;
dropPercentage: number;
threshold: number; // Default: 15% drop
}
// If confidence drops 15%+ in a category:
// - Something changed (new patterns, deprecated patterns)
// - Trigger re-analysis
Meta-Learning: Learning When to Learn
┌─────────────────────────────────────────────────────────────────┐
│ META-LEARNING SYSTEM │
├─────────────────────────────────────────────────────────────────┤
│ │
│ After 1,000 decisions, analyze: │
│ ├─ Which triggers produced useful patterns? │
│ ├─ Which thresholds are too high/low? │
│ ├─ Which categories need faster learning? │
│ └─ What's the optimal batch size for extraction? │
│ │
│ Learn trigger effectiveness: │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ Trigger │ Patterns │ Useful │ Effectiveness │ │ │
│ │──────────────│──────────│────────│───────────────│ │ │
│ │ Volume(100) │ 45 │ 32 │ 71% │ │ │
│ │ Repetition │ 12 │ 11 │ 92% │ ⭐ │ │
│ │ Performance │ 8 │ 6 │ 75% │ │ │
│ │ Novelty │ 5 │ 4 │ 80% │ │ │
│ └────────────────────────────────────────────────────────┘ │
│ │
│ Insight: Repetition trigger most effective │
│ Action: Lower repetition threshold from 3 to 2 │
│ │
│ After 5,000 decisions: │
│ ├─ Build patterns: Stable, slow down extraction │
│ ├─ Angular patterns: Growing, maintain rate │
│ ├─ AI integration: New, accelerate extraction │
│ └─ Auto-adjust thresholds per category │
│ │
└─────────────────────────────────────────────────────────────────┘
Trigger Priority & De-duplication
interface TriggerPriority {
// Highest priority: Immediate action needed
CRITICAL: ['performance_drop', 'repeated_failure'];
// High priority: Should process soon
HIGH: ['repetition', 'confidence_drop'];
// Normal priority: Process in batch
NORMAL: ['volume', 'novelty'];
// Low priority: Background processing
LOW: ['cleanup', 'archive_check'];
}
// De-duplication: Don't run same extraction twice
interface TriggerDeduplication {
// If volume trigger fires for 'build' category:
// - Check if repetition already triggered for build
// - If yes, combine into single extraction job
// - Track which triggers contributed
}
Learning Job Types
1. Pattern Extraction
Triggered by: Volume, Repetition, Performance
Input: Batch of decisions matching criteria
Output: Extracted patterns, rule candidates
2. Rule Promotion
Triggered by: Pattern reaching threshold
Input: Pattern with 10+ successful uses
Output: Promoted to core rule
3. Anti-Pattern Detection
Triggered by: Repeated failures
Input: Failed decisions
Output: Anti-patterns (what NOT to do)
4. Archive Eligibility Check
Triggered by: Volume (after extraction)
Input: Decisions with extracted patterns
Output: Decisions ready for archive
Implementation
// Trigger Service
@Injectable()
export class AdaptiveTriggerService {
private triggers = new Map<string, TriggerState>();
private triggerQueue: PriorityQueue<Trigger>;
// Called on every new decision
async onNewDecision(decision: Decision): Promise<void> {
// Check all trigger conditions
await this.checkVolumeTrigger(decision);
await this.checkRepetitionTrigger(decision);
await this.checkPerformanceTrigger(decision);
await this.checkNoveltyTrigger(decision);
// Process any fired triggers
await this.processTriggerQueue();
}
// Called on outcome report
async onOutcomeReported(decisionId: string, success: boolean): Promise<void> {
await this.updatePerformanceMetrics(decisionId, success);
await this.checkConfidenceDropTrigger();
}
// Process triggers by priority
private async processTriggerQueue(): Promise<void> {
while (!this.triggerQueue.isEmpty()) {
const trigger = this.triggerQueue.pop();
// De-duplicate with pending jobs
if (this.isDuplicate(trigger)) continue;
// Execute appropriate learning job
await this.executeLearningJob(trigger);
}
}
}
Comparison: Time-Based vs Adaptive
| Aspect | Time-Based | Adaptive |
|---|---|---|
| Trigger | Every 7 days | When needed |
| Responsiveness | Days late | Minutes |
| Compute waste | High (slow weeks) | Minimal |
| Pattern quality | Averaged | Targeted |
| Learning speed | Fixed | Accelerates with activity |
| Optimization | Manual tuning | Self-optimizing |
Benefits
- Responsive: Learn immediately when patterns emerge
- Efficient: No wasted compute on quiet periods
- Adaptive: Thresholds optimize over time
- Targeted: Different triggers for different situations
- Self-improving: Meta-learning adjusts the system itself
Next Steps
- Implement trigger tracking in decision logging
- Build trigger evaluation service
- Create learning job queue
- Add meta-learning analysis job
- Build dashboard for trigger visibility