Architecture: How should the 6-phase module generation pipeline route to d...
Documentation for Sessions
🏗️ Architecture: How should the 6-phase module generation pipeline route to d...
Date: 2026-03-13
Time: 02:02:09 UTC
Session: gspace1-212.3
Agent: backend
Confidence: 95%
Status: ✅ Documented
📋 Decision Context
Question:
How should the 6-phase module generation pipeline route to different LLM models per phase for cost optimization?
Task: S212: Per-phase model routing in generation pipeline
🎯 Decision Made
Chosen:
Static per-phase model assignment via AI_MODELS_CONFIG.moduleGeneration config, resolved through ProviderFactoryService.getForPhase(). PLAN→Opus (architecture decisions cascade), ENHANCE/INTEGRATE→Haiku (classification/validation), ANALYZE/DECOMPOSE/EXECUTE→Sonnet (judgment + code gen). Each phase service calls getForPhase() which returns {provider, model} with ResilientLLMProvider failover.
Options Considered:
- Static per-phase config in AI_MODELS_CONFIG
- RouteLLM classifier-based dynamic routing
- CascadeFlow start-cheap-escalate-on-failure
- User-configurable per-phase model selection
💭 Reasoning
Root cause: All 6 phases called providerFactory.getDefault() → Sonnet for everything → $1-2 per generation with ~50% waste. The infrastructure (getForPhase, TaskPhase, ResilientLLMProvider) already existed but was never wired. Alternatives considered: (1) RouteLLM classifier-based routing (ICLR 2025) — adds latency, Python-only, overkill for deterministic pipeline phases; (2) CascadeFlow start-cheap-escalate — our phases have structural validation, static is better; (3) User-configurable per-phase — over-engineering, most users don't understand LLM capabilities. The static config approach is the consensus production pattern ('most routing is conditional logic'). Expected savings: ~40-50% per generation ($0.40-0.80 saved). PLAN uses Opus because architecture decisions cascade — a cheap model here causes expensive rework downstream.
📁 Files Involved
libs/libs-api/backend-core/src/lib/config/ai-models.config.tsAPIs/backend-vitto-api/src/app/llm/provider-factory.service.tsAPIs/backend-vitto-api/src/app/module-generation/phases/enhance.phase.tsAPIs/backend-vitto-api/src/app/module-generation/phases/plan.phase.tsAPIs/backend-vitto-api/src/app/module-generation/phases/execute.phase.ts
🔗 Related
- Decision ID:
dev_1773367329425_72i617p - Session: 212
- Workspace: gspace1
Auto-generated by MCP Smart Consolidation