Handoff: Vitto Orchestrator Enhancement

From: Coding Agent (Claude Code, Session 98) To: New Claude Code Agent (dedicated session) Priority: High — blocks the entire multi-agent workflow Estimated Scope: ~15-20 files, 4 phases

The Problem

Vitto is the platform's autonomous AI orchestrator agent. It receives handoffs from other agents (Testing Director, coding agents, MCP tools) and is supposed to intelligently process them — delegating to browser automation, Claude CLI, or handling directly.

What actually happens: Vitto auto-claims every handoff within 60 seconds and "completes" them by asking Claude Sonnet 4.5 to generate a text response. It marks tasks as completed regardless of whether it actually did anything.

Real example from Session 98:

• A verification handoff was created with 3 tasks: verify a bug fix, acknowledge test results, respond to a cross-repo analysis
• Vitto claimed it in <1 second, completed it in 48.7 seconds
• Task 1 (verify bug fix): Generated a markdown "test plan" template instead of actually testing
• Task 2 (acknowledge results): Paraphrased the input back
• Task 3 (respond to analysis): Generated a document that already existed as input
• All 3 marked status: 'completed'

This is not intelligence. This is an expensive echo command.

Root Cause Analysis

1. Strategy Selection is Type-Blind (Critical)

File: apis/backend-ai-api/src/app/vitto/vitto-orchestrator.service.ts Lines: 622-647 (complexity classification), 541-617 (planExecution)

The complexity heuristic:

HIGH: keywords (refactor, migrate, implement, create, build, design) OR >3 tasks OR >3 files
MEDIUM: >1 task OR >1 file
LOW: default

Then:

HIGH complexity → delegate to Claude CLI
MEDIUM/LOW → direct AI execution (= ask Claude to write text)

The handoff type field is completely ignored. A verification handoff gets the same treatment as a question handoff. There's no awareness that:

• verification requires browser automation or explicit rejection
• implementation requires Claude CLI with file editing
• review might be fine as direct AI
• test_request requires actual test execution

2. No Output Quality Gate (Critical)

File: Same file, executeWithAI() lines 1286-1428, completeHandoff() lines 1823-1835

After Claude returns text, Vitto immediately calls completeHandoff(). There is zero validation:

• Does the response actually address the task?
• Is a "test plan template" an acceptable answer to "verify this fix works"?
• Did Vitto actually perform the requested action or just describe how one might?

3. Never Rejects or Escalates (Critical)

File: Same file, evaluateHandoff() lines 451-475

Vitto accepts ALL handoff types. There is no capability self-assessment. It should know:

• "I cannot verify runtime behavior without browser access" → reject with reason
• "This requires file editing, delegating to Claude CLI" → delegate, don't fake it
• "This is a question I can answer" → direct execution is fine

4. Stale Threshold Steals Work (Medium)

Lines 107-108:

STALE_CHECK_INTERVAL_MS = 30_000  // Check every 30s
STALE_THRESHOLD_MS = 60_000       // 1 minute = stale

Real agents (Cursor, Claude Code) can take 5-10 minutes on complex tasks. Vitto considers anything pending >60 seconds as abandoned and takes over, producing garbage output that marks the handoff as completed — preventing the real agent from ever processing it.

5. Standalone Module is Hollow (Medium)

File: apis/backend-ai-api/src/app/vitto/vitto-standalone.module.ts (42 lines)

Only imports HttpModule and ConfigModule. If run as PM2 process (the intended isolation mode), Vitto loses access to:

• LearningModule (pattern recognition)
• SuperIntelligenceModule (episodic memory)
• AgentsModule (TenantAIService)
• WorkflowModule (module/system generation)

6. TenantAIService Integration Incomplete (Medium)

File: apis/backend-api/src/app/ai-agents/services/tenant-ai.service.ts (802 lines)

The service is ~80% coded with full methods for:

• getPersonalizationContext() — returns preferences, tech stack, few-shot examples
• logInteraction() — logs with embeddings
• learnTechStackFromCode() — detects frameworks/languages
• findSimilarInteractions() — vector similarity search

It's called from vitto-orchestrator.service.ts lines 1371-1421, but the actual learning pipeline stores metadata with fixed confidence values instead of real outcome signals from task execution.

Architecture Reference

Vitto's Current Execution Flow

WebSocket event or 30s poll
  → evaluateHandoff() — always accepts
    → processHandoff()
      → acknowledge (POST /handoffs/{id}/respond)
      → gatherCrossSessionIntelligence() — episodes + similar decisions
      → planExecution()
        → estimateComplexity() — keyword-based, ignores type
        → if HIGH → delegate_claude_cli
        → if MEDIUM/LOW → direct AI execution
      → executePlan() → executeStep() per step
        → executeWithAI() — Claude API call → text response
      → completeHandoff() — always succeeds, no validation

What the Flow SHOULD Be

WebSocket event or poll
  → evaluateHandoff()
    → CAN I handle this type? (capability check)
      → NO → reject with reason, suggest reroute
      → YES → accept
    → processHandoff()
      → acknowledge
      → gatherIntelligence()
      → planExecution()
        → TYPE-AWARE strategy selection:
          → verification → delegate_browser OR reject
          → implementation → delegate_claude_cli
          → fix_request → delegate_claude_cli
          → review → direct AI (acceptable)
          → question → direct AI (acceptable)
          → test_request → delegate_browser OR reject
          → documentation → direct AI (acceptable)
      → executePlan()
      → VALIDATE output quality
        → Does response match task type expectations?
        → Did it perform an action or just describe one?
        → FAIL → retry with different strategy OR escalate
      → completeHandoff() — only if quality passes

Files to Modify

Core Changes

Integration Changes

Supporting Types

Implementation Plan

Phase 1: Capability-Aware Strategy Engine (Critical)

Goal: Vitto routes tasks to the right execution method based on handoff type, or rejects if it can't fulfill.

Changes to vitto-orchestrator.service.ts:

1. Replace estimateComplexity() with selectStrategy(handoff) (lines 622-647)

   New logic:

   private selectStrategy(handoff: Handoff): { strategy: string; reason: string } {
     const type = handoff.type;
     const hasClaudeCli = this.claudeCliService?.isAvailable?.();
     const hasBrowserAccess = false; // Vitto cannot browse directly
   
     switch (type) {
       case 'verification':
       case 'test_request':
         // These REQUIRE actual runtime testing
         if (hasBrowserAccess) return { strategy: 'delegate_browser', reason: 'Requires runtime verification' };
         return { strategy: 'reject', reason: 'Cannot verify runtime behavior — requires browser agent or human' };
   
       case 'implementation':
       case 'fix_request':
         // These REQUIRE file editing
         if (hasClaudeCli) return { strategy: 'delegate_claude_cli', reason: 'Requires code changes' };
         return { strategy: 'reject', reason: 'Cannot edit files — requires Claude CLI or coding agent' };
   
       case 'review':
       case 'question':
       case 'documentation':
         // These CAN be handled by direct AI
         return { strategy: 'direct', reason: 'Informational task suitable for AI response' };
   
       default:
         // Fall back to complexity-based routing for untyped handoffs
         return this.estimateComplexityFallback(handoff);
     }
   }
   

2. Add rejection capability to processHandoff() (~line 480)

   After selectStrategy(), if strategy is reject:

   if (strategy === 'reject') {
     await this.rejectHandoff(handoff.handoffId, reason);
     return;
   }
   

3. Implement rejectHandoff() method:

   private async rejectHandoff(handoffId: string, reason: string): Promise<void> {
     await this.httpService.axiosRef.post(
       `${this.backendUrl}/ai-api/infrastructure/handoffs/${handoffId}/respond`,
       {
         respondingAgent: this.agentId,
         status: 'rejected',
         decline: { reason, suggestReroute: true }
       }
     );
   }
   

4. Update planExecution() (lines 541-617) to use selectStrategy() instead of the keyword-based approach.

Verification:

• Create a verification type handoff → Vitto should reject it with reason
• Create a question type handoff → Vitto should handle directly
• Create an implementation type handoff → Vitto should delegate to Claude CLI

Phase 2: Output Quality Gate (Critical)

Goal: Vitto validates its own output before marking tasks as completed.

Changes to vitto-orchestrator.service.ts:

1. Add validateOutput() method after executePlan() returns:

   private async validateOutput(
     handoff: Handoff,
     results: TaskResult[]
   ): Promise<{ passed: boolean; failures: string[] }> {
     const failures: string[] = [];
   
     for (const result of results) {
       if (result.status === 'failed') continue; // Already failed
   
       const task = handoff.tasks?.find(t => t.id === result.taskId);
       if (!task) continue;
   
       // Check: Did Vitto actually DO something or just describe how?
       const descriptionPatterns = [
         /here's how to/i,
         /you (can|should|could|would)/i,
         /steps to (follow|test|verify)/i,
         /test plan/i,
         /### \d+\./,  // Numbered markdown sections (template-like)
       ];
   
       const isJustDescription = descriptionPatterns.some(p => p.test(result.result));
   
       if (isJustDescription && handoff.type !== 'question' && handoff.type !== 'documentation') {
         failures.push(
           `Task ${result.taskId}: Generated descriptive text instead of performing action. ` +
           `Handoff type '${handoff.type}' requires actual execution.`
         );
         result.status = 'failed';
         result.error = 'Output validation failed: response is descriptive, not actionable';
       }
     }
   
     return { passed: failures.length === 0, failures };
   }
   

2. Insert validation in processHandoff() between executePlan() and completeHandoff():

   const results = await this.executePlan(handoff, plan);
   const validation = await this.validateOutput(handoff, results);
   
   if (!validation.passed) {
     this.logger.warn(`[VITTO] Quality gate FAILED for ${handoff.handoffId}:`);
     validation.failures.forEach(f => this.logger.warn(`  → ${f}`));
   
     // Escalate: reject with quality failure reason
     await this.rejectHandoff(
       handoff.handoffId,
       `Quality validation failed: ${validation.failures.join('; ')}`
     );
     return;
   }
   
   await this.completeHandoff(handoff.handoffId, results, []);
   

Verification:

• Direct AI execution that generates "Here's how to test..." should fail validation
• Direct AI execution that answers a question should pass
• Claude CLI execution that returns actual code changes should pass

Phase 3: Stale Threshold Fix (Medium)

Goal: Prevent Vitto from stealing handoffs from agents that are actively working.

Changes to vitto-orchestrator.service.ts:

1. Increase STALE_THRESHOLD_MS (line 108):

   // Was: 60_000 (1 minute) — too aggressive, steals from real agents
   private readonly STALE_THRESHOLD_MS = 300_000; // 5 minutes
   

2. Add agent activity check in evaluateHandoff() (lines 451-475):

   // Before declaring stale, check if assigned agent is still online
   if (handoff.assignedTo) {
     try {
       const agentStatus = await this.httpService.axiosRef.get(
         `${this.backendUrl}/ai-api/infrastructure/agents/${handoff.assignedTo}`
       );
       if (agentStatus.data?.status === 'online' || agentStatus.data?.status === 'busy') {
         // Agent is alive, don't steal
         return;
       }
     } catch { /* Agent not found, proceed with stale check */ }
   }
   

3. Only take over truly abandoned handoffs — where the assigned agent is offline AND the threshold has passed.

Verification:

• Assign a handoff to a connected agent → Vitto should NOT take over even after 5 minutes
• Assign a handoff to a disconnected agent → Vitto should take over after 5 minutes
• Unassigned handoff with no online agents → Vitto should claim after 5 minutes

Phase 4: Standalone Module + Learning Pipeline (Medium)

Goal: PM2 mode works with full intelligence, learning produces real signals.

4a. Fix Standalone Module

File: apis/backend-ai-api/src/app/vitto/vitto-standalone.module.ts

Add the missing imports:

@Module({
  imports: [
    ConfigModule.forRoot(),
    HttpModule,
    // ADD these:
    UserAgentMemoryModule,
    WorkflowModule,
    LearningModule,
    SuperIntelligenceModule,
    AgentsModule,
  ],
  providers: [
    VittoOrchestratorService,
    VittoFileStorageService,
    ClaudeCliService,
  ],
})

4b. Enrich Learning Pipeline

File: vitto-orchestrator.service.ts, saveUserLearning() calls (~lines 1371-1421)

Current state: stores metadata with placeholder values. Change to include:

• Actual task outcome (did the quality gate pass?)
• Execution strategy used (direct, delegate_cli, delegate_browser)
• Duration of execution
• Whether the handoff was rejected and why
• Token count from Claude API response

File: apis/backend-api/src/app/ai-agents/services/tenant-ai.service.ts

Verify that:

• logInteraction() properly stores outcome signals
• getPersonalizationContext() returns data that influences strategy selection
• findSimilarInteractions() works with the vector memory service

Verification:

• Start Vitto as standalone process → should have access to episodic memory and learning
• Process a question handoff → check that interaction is logged with real outcome data
• Process a second similar question → check that previous interaction influences response

Key Types Reference

HandoffType (from handoff.schema.ts)

type HandoffType = 'test_request' | 'verification' | 'fix_request' |
                   'review' | 'question' | 'implementation' | 'documentation';

HandoffStatus

type HandoffStatus = 'pending' | 'acknowledged' | 'in_progress' |
                     'completed' | 'rejected' | 'expired';

ExecutionPlan (from vitto-orchestrator.service.ts, line 2098)

interface ExecutionPlan {
  strategy: 'direct' | 'delegate' | 'multi-step' | 'module-generation' | 'system-generation';
  steps: ExecutionStep[];
  reasoning: string;
}

ExecutionStep (line 2104)

interface ExecutionStep {
  order: number;
  action: 'execute' | 'file_write' | 'file_read' | 'http_request' |
          'delegate_code' | 'delegate_browser' | 'computer_use' | 'delegate_claude_cli';
  taskId: string;
  description: string;
  delegateTo?: string;
  params?: Record<string, unknown>;
}

SubAgent Capabilities (line 76)

interface SubAgent {
  agentId: string;
  platform: string;
  capabilities: {
    canBrowse: boolean;
    canEditFiles: boolean;
    canExecuteCode: boolean;
    canTestUI: boolean;
  };
}

Environment & Setup

Where Vitto Runs

• Primary: Inside backend-ai-api NestJS app (port 3001)
• Standalone: Can run as separate PM2 process via vitto-standalone.module.ts
• Connection: Native WebSocket to /ws endpoint on port 3001

Required Environment Variables

ANTHROPIC_API_KEY=...          # For Claude API calls
VITTO_AI_MODEL=...             # Default: claude-sonnet-4-5-20250929
BACKEND_API_URL=...            # backend-api endpoint
INTERNAL_API_KEY=...           # Service-to-service auth
CLAUDE_CLI_PATH=...            # Path to claude CLI (default: 'claude')
VITTO_STORAGE_MODE=both        # local | cloud | both
WORKSPACE_ID=editor2           # Workspace identifier

How to Test Vitto

1. Ensure backend-ai-api is running: curl http://localhost:3001/ai-api/health
2. Check Vitto status: curl http://localhost:3001/vitto/status
3. Trigger handoff check: curl -X POST http://localhost:3001/vitto/check-handoffs
4. Create test handoff via MCP: agent { action: "handoff_create", toAgentType: "any", type: "question", summary: "Test handoff", tasks: [...] }
5. Monitor Vitto logs in backend-ai-api terminal output for [VITTO] prefixed messages

Build & Run

# Start backend-ai-api (includes Vitto)
yarn serve:backend-ai

# Vitto connects via WebSocket automatically 3 seconds after app init
# Check connection in logs: "[VITTO] Connected to WebSocket gateway"

What NOT to Change

1. Module/System generation pipeline — executeModuleGeneration() and executeSystemGeneration() route to PhaseOrchestratorService and SystemExecutorService respectively. These are separate systems and should NOT be touched.

2. Rate limiting — The Bottleneck config (3 concurrent, 60/min) is fine for current scale. Don't remove it.

3. Cross-session intelligence gathering — gatherCrossSessionIntelligence() (lines 1018-1084) is well-implemented. Don't refactor it — just make sure its recommendations actually influence strategy selection.

4. File storage service — VittoFileStorageService works correctly with hybrid local+cloud. Leave it alone.

5. WebSocket connection management — The connect/heartbeat/stale-check cycle works. Only change the stale threshold value.

Success Criteria

After these changes, the following scenarios must work correctly:

Anti-criteria (must NOT happen):

• Vitto generating a "here's how to test" response for a verification task and marking it complete
• Vitto stealing a handoff from an active agent after 60 seconds
• Vitto accepting every handoff regardless of its actual capabilities
• Vitto running without intelligence modules in standalone mode

Related Context

MCP Tools Available

The new agent has access to the same MCP tools. Use preflight() at session start and log_decision() for each architectural change.

Session Documentation Convention

• Session docs go in docs/english/sessions/S{N}/
• Use proper frontmatter
• Max 5-8 files per session

CLAUDE.md Rules Apply

• Angular 20 standalone components, signals preferred
• NestJS uses @nestjs/config with .env.local
• Never hardcode ports — use env vars
• RxJS Observables only, no Promises/setTimeout in Angular code
• NestJS backend code CAN use async/await (it's Node.js, not Angular)

Port Configuration

• backend-ai-api: port 3001 (where Vitto runs)
• backend-api: port 3000 (where TenantAIService lives)
• Both read from .env.local

Priority Order

1. Phase 1 (Strategy Engine) — unblocks everything. Without this, Vitto produces garbage.
2. Phase 2 (Quality Gate) — safety net. Catches cases where strategy selection still routes incorrectly.
3. Phase 3 (Stale Threshold) — prevents Vitto from interfering with real agents.
4. Phase 4 (Standalone + Learning) — makes Vitto actually learn from experience.

Phases 1+2 together are the minimum viable fix. Phases 3+4 are important but can be a follow-up if needed.