Testing Director Report: Systemic Prompt-vs-Type Mismatches in Pipeline B

Documentation for Handoffs

Systemic Prompt-vs-Type Mismatches in Pipeline B

Session 138 | February 23, 2026 | Testing Director

Executive Summary

After the coding agent fixed the enhancedPrompt.substring crash (ENHANCE phase), generation now gets past PLAN but crashes at EXECUTE with: executionPlan is not iterable.

Root cause analysis reveals this is NOT an isolated bug — it's a systemic issue across 3 of 6 phases. The LLM prompt templates tell the AI to return JSON in one shape, but the TypeScript type definitions expect a different shape. This causes runtime crashes when the code tries to access fields that don't match.

Test Results

Attempt enhancedPrompt Fix Result Error
1 (before fix) No Crashed at PLAN enhance.enhancedPrompt.substring is not a function
2 (before fix) No Crashed at PLAN Same
3 (after fix) Yes Got past PLAN → Crashed at EXECUTE EXECUTE failed: executionPlan is not iterable

The Current Blocker: PLAN Phase Output

What the Prompt Template Says (system-plan.prompt.md)

{
  "navigation": {
    "menuItems": [
      { "id": "dashboard", "label": "Dashboard", "icon": "layout-dashboard", "route": "/dashboard", "moduleId": "dashboard", "order": 1 }
    ],
    "defaultRoute": "/dashboard"
  },
  "executionPlan": {
    "phases": [
      { "order": 1, "moduleId": "user-management", "action": "generate", "templateType": "crud", "estimatedDuration": "2-5 minutes" }
    ],
    "totalModules": 3,
    "estimatedTotalDuration": "5-15 minutes"
  }
}

What the TypeScript Type Expects (ModuleGenPlanResult)

interface ModuleGenPlanResult {
  systemName: string;
  totalModules: number;
  executionPlan: ModuleExecutionPlan[];  // FLAT ARRAY, not nested object
  navigationStructure: NavigationNode[];  // different field name + flat array
  sharedEntities: string[];
}

What the Code Does (execute.phase.ts line 68-70)

const executionPlan = planResult.executionPlan || [];
for (const plan of executionPlan) {  // CRASHES: object is not iterable

The LLM returns executionPlan as { phases: [...], totalModules: 3 } (an object). The code tries to iterate it with for...of. Objects aren't iterable → crash.

Fix for This Specific Crash

In execute.phase.ts line 68:

// Handle both: array (expected by type) or object with .phases (returned by LLM)
const rawPlan = planResult.executionPlan;
const executionPlan = Array.isArray(rawPlan)
  ? rawPlan
  : (rawPlan as any)?.phases || [];

Full Mismatch Audit: All 6 Phases

ENHANCE — Fixed ✅

ANALYZE — 5 Mismatches ⚠️

Field Prompt Returns Type Expects Impact
actors[].roles roles: string[] (array) role: string (single) Data loss
coreEntities[].attributes attributes: [{name, type, required}] fields: FieldSpec[] Field name mismatch — entities would have empty fields
mainWorkflows[].steps [{order, action, entity}] (objects) string[] Type mismatch — structured data truncated to strings
nonFunctionalRequirements [{category, requirement}] (objects) string[] Data loss
enrichedPrompt Present in LLM output Not in type definition Field dropped silently

DECOMPOSE — 4 Mismatches ⚠️

Field Prompt Returns Type Expects Impact
modules[].priority 1, 2, 3 (numbers) 'core'|'secondary'|'optional' (enum) Type coercion issues
modules[].dependencies Array inside each module Separate DependencyMap object Structure mismatch — dependencies misplaced
dependencyGraph {nodes: [], edges: []} Not in type Field dropped
integrationPoints[].from/to from, to fromModule, toModule Field name mismatch

PLAN — 4 Mismatches ⚠️ (Current Crash Point)

Field Prompt Returns Type Expects Impact
executionPlan {phases: [], totalModules, estimatedTotalDuration} ModuleExecutionPlan[] (flat array) CRASH: not iterable
navigation {menuItems: [], defaultRoute} navigationStructure: NavigationNode[] Field name mismatch + structure
executionPlan.phases[].moduleId moduleId moduleName Field name mismatch
executionPlan.phases[].estimatedDuration estimatedDuration: string estimatedComplexity: enum Completely different field

EXECUTE — OK ✅

INTEGRATE — OK ✅

Recommended Approach

Option A: Fix Types to Match Prompts (Fastest)

Update the TypeScript interfaces in module-generation.types.ts to match what the LLM actually returns. This is the path of least resistance since the prompts are already working (ENHANCE and ANALYZE both complete successfully).

Option B: Fix Prompts to Match Types

Rewrite the prompt templates to match the existing TypeScript types. Riskier because:

Option C: Add Adapter Layer (Most Robust)

Add a normalizePhaseOutput() function after each parseJsonResponse() call that maps LLM field names to TypeScript field names. This handles both formats gracefully.

Recommendation: Option A for immediate unblock, then Option C for long-term robustness.

Files to Modify

File Changes Needed
module-generation.types.ts Update ModuleGenAnalyzeResult, ModuleGenDecomposeResult, ModuleGenPlanResult to match prompt output shapes
execute.phase.ts line 68 Handle executionPlan as either array or {phases: []} object
module-gen-phase.executor.ts Add safe field access for all cross-phase data references

Priority

# Action Blocks
1 Fix executionPlan iteration in execute.phase.ts Unblocks generation immediately
2 Fix navigationStructure → navigation.menuItems mapping Navigation after generation
3 Fix ANALYZE field names (attributes → fields, roles → role) Entity quality in generated modules
4 Fix DECOMPOSE field names (from → fromModule, priority number→enum) Module dependency tracking