S234: Vitto Jarvis Evolution — Testing Guide

Audience: Internal QA team. No coding knowledge required. Updated: 2026-03-18

Quick Reference: What Changed

1. Sub-Vitto (Parallel Agent Dispatch)

What to test

Vitto can now break complex tasks into sub-tasks and run them simultaneously using deploy_agents.

How to test

1. Open Vitto chat (click the Vitto icon or navigate to any chat panel)
2. Give a complex, multi-part request. Examples:
   • "Analyze the authentication system, the payment flow, and the notification system — give me a report on each"
   • "Review the user module, the settings module, and the dashboard module for code quality issues"
3. Expected behavior:
   • Vitto should call deploy_agents (you'll see a tool call in the chat)
   • A Sub-Vitto Orchestration card should appear showing:
     • Multiple sub-tasks listed
     • Progress indicators for each
     • Status (running → completed/failed)
     • A synthesis at the end summarizing all results
4. Check: Each sub-task should complete independently. If one fails, others should still succeed.

Edge cases

• Ask for just 1 sub-task → should still work (overkill but valid)
• Ask for 10+ sub-tasks → should batch them (max 5 concurrent)
• Ask Vitto to "deploy agents to deploy agents" → should be blocked (recursion guard)

2. Agent Creation via Chat

What to test

Vitto can create new specialized agent definitions when asked.

How to test

1. In Vitto chat, say: "Create an agent that reviews code for security vulnerabilities"
2. Expected behavior:
   • Vitto calls create_agent tool
   • Returns a JSON card with the new agent's:
     • Name (e.g., "Security Review Agent")
     • Role
     • Available tools
     • Trigger conditions
3. Check: Navigate to /vitto → Sub-Agents tab → Registry. The new agent should appear in the list.

Edge cases

• Vague description: "Create an agent" → should ask for more details or create a generic one
• Duplicate name: Creating two agents with same description → should create separate agents

3. Agent Marketplace

What to test

Browse, publish, and clone agent definitions.

How to test

1. Navigate to /vitto → click "Sub-Agents" in the sidebar
2. You should see 4 tabs: Registry, Editor, Training, Marketplace
3. Marketplace tab:
   • Should show a search bar and published agent count
   • If no agents are published yet, shows empty state: "No published agents yet"
   • Search should filter by name/description
4. Publishing:
   • Go to Registry tab
   • Find an agent in the list
   • Click Publish button (only visible if agent isn't already published)
   • Switch to Marketplace tab → agent should now appear
5. Cloning:
   • In Marketplace, find an agent
   • Click "Clone to My Space"
   • Should switch to Registry tab and show the cloned agent

4. OAuth — Connected Services (Google)

What to test

Users can connect their Google account so Vitto can act on their behalf.

How to test

1. Navigate to /vitto → Channel Manager → Connected Services tab
2. You should see a "Google" card with a "Connect" button
3. Click Connect:
   • Should redirect to Google OAuth consent screen
   • After granting permission, should redirect back to the app
   • Google card should now show "Connected" status with the connected email
4. In Vitto chat:
   • Ask: "What's on my calendar today?"
   • Vitto should call oauth_status first (checks if Google is connected)
   • Then call oauth_api_call with Google Calendar API
   • Expected: Returns your actual calendar events
5. Disconnecting:
   • Back in Connected Services, click "Disconnect" on Google
   • Ask Vitto about calendar again → should tell you Google isn't connected

Prerequisites

• Google OAuth must be configured in .env.local with GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET
• If not configured, the Connect button should show an error or be disabled

Edge cases

• Connect → close browser → reopen → connection should persist
• Token expiry → Vitto should auto-refresh (transparent to user)
• Ask about calendar without connecting first → Vitto should say "you need to connect Google first"

5. Module Generation (Fixed Preview)

What to test

When Vitto generates platform modules, they should now appear in the preview immediately.

How to test

1. Navigate to a Space → Universe → Studio
2. In the Vitto chat panel, say: "Generate a task management platform with projects, tasks, and team members"
3. Watch for:
   • Phase progress updates (ANALYZE → PLAN → DECOMPOSE → ENHANCE → EXECUTE → INTEGRATE)
   • Per-module progress: "Module 1/3 generated", "Module 2/3 generated", etc.
   • Files appearing in the code editor (left panel) as each module completes
   • Review panel opening automatically with generated files
   • Preview panel showing the compiled application
4. The critical check: After generation completes, the preview should NOT be blank. You should see actual rendered UI.

What was broken before

• Vitto would say "Done. 5 modules, 131 files" but the preview showed nothing
• Files were generated on the backend but never delivered to the browser

Edge cases

• Generate while preview is already open → should update
• Generate with a universe that has existing modules → new modules should coexist
• Cancel/navigate away during generation → should not crash

6. Teams Channel

What to test

Vitto can receive messages from Microsoft Teams and respond.

Prerequisites

• Microsoft Bot Framework must be configured
• Teams webhook URL must be registered

How to test

1. Send a message to the Vitto bot in Microsoft Teams
2. Expected: Vitto processes the message through the engine and responds
3. Responses should appear as Adaptive Cards in Teams (formatted with sections, headers)
4. Check that the conversation is tracked in Vitto's task history

Note

This requires Teams Bot Framework infrastructure. If not configured, skip this test.

7. Vitto Evolution & Brain Visualization

What to test

Vitto accumulates experience and "matures" over time.

How to test

1. Navigate to /vitto → Brain panel (or wherever the brain visualization is rendered)
2. Expected: You should see maturity data:
   • Maturity score (0-100)
   • Experience points
   • Traits (adaptive, analytical, creative, etc.)
   • Milestones reached
3. After having several conversations with Vitto, the maturity score should increase
4. Traits should shift based on interaction patterns

Edge cases

• New user with no history → should show base maturity (low score, default traits)
• Admin viewing another user's evolution → should show that user's data

8. Personality Drift & Conscience

What to test

Vitto's personality adapts naturally over time.

How to test

1. Navigate to /vitto → Conscience view (sidebar: Mind → Conscience)
2. Expected: SSE stream of conscience events showing Vitto's internal state
3. After interactions, you may see drift events like:
   • "Tone shifted toward more technical language"
   • "Formality increased based on user preference"
4. Drift confidence thresholds: Only significant personality shifts (confidence > 0.8) should auto-apply

9. Admin: OAuth Health Dashboard

What to test

Admins can monitor OAuth connection health across all users.

How to test

1. Navigate to /command/integrations (admin-only route)
2. Expected:
   • Overview of all OAuth connections
   • Health status per provider (Google, GitHub)
   • Token expiry warnings
   • Connection statistics

10. Operations: Live Sub-Vitto Monitor

What to test

Real-time monitoring of active sub-Vitto tasks.

How to test

1. Navigate to /operations/live
2. Trigger a deploy_agents call in Vitto chat (complex multi-part request)
3. Expected: Live view should show active sub-tasks with real-time progress

Smoke Test Checklist

Run these quickly to verify nothing is broken:

• Open Vitto chat → send a simple message → get a response
• Navigate to /vitto → all sidebar sections load without errors
• Sub-Agents tab → 4 tabs visible (Registry, Editor, Training, Marketplace)
• Channel Manager → Connected Services tab visible
• Vitto Studio → generate a simple module → preview shows UI
• Ask Vitto "what tools do you have?" → should list deploy_agents, create_agent, oauth_status, oauth_api_call among others

Known Limitations

1. GitHub OAuth: Provider config exists but full flow not yet tested end-to-end
2. Teams responses: Requires Bot Framework infrastructure to be configured
3. Slack/Notion OAuth: Not yet implemented (Google and GitHub only)
4. Sub-Vitto VM tier: Currently falls back to in-process execution (E2B/Firecracker VM routing is Phase 2.5, deferred)
5. Agent Marketplace: No rating/review system yet — just browse/clone/publish

Reporting Issues

When reporting a bug, please include:

1. Where: Which page/panel/chat
2. What you did: Step-by-step reproduction
3. What you expected: What should have happened
4. What happened: What actually happened
5. Console errors: Open browser DevTools (F12) → Console tab → screenshot any red errors
6. Network tab: Check for failed API calls (red entries in Network tab)