Vitto Jarvis — Master Specification

Single source of truth for evolving Vitto into a Jarvis-level super agent. Consolidates all prior Jarvis specs, includes codebase-verified audit, 7 capability pillars, dependency-ordered build phases, and programmatic verification gates. Terminal spec — not to be superseded.

Vitto Jarvis — Master Specification

This is the terminal Jarvis spec. It supersedes all 4 prior Jarvis specs, consolidates what's been built (20 features verified), defines the 11 remaining gaps, and includes programmatic verification gates so future sessions can check completion without reading code.

Guiding principles:

  1. Analyze first — never repeat work. Every section starts with what exists.
  2. Respect API boundaries — backend-api (user data), backend-ai-api (intelligence), backend-vitto-api (execution).
  3. Everything wired — no orphaned backend services. If it exists, it must be visible in the chat via plugins.
  4. Everything testable — verification gates with specific commands/endpoints.
  5. Document for future agents — handoff docs for deferred work.

Part I: The Jarvis Vision — 7 Capability Pillars

Pillar 1: Cognitive Core (Reasoning Layer)

Gap: Vitto's agent loop is while not done: LLM(history + tools) → execute. Reasoning is implicit in the LLM. There's no persistent reasoning trace, no explicit hypothesis tracking, no post-task analysis of WHY Vitto chose one tool over another.

Solution: A ReasoningService that decorates the existing AgentLoopService — NOT a replacement. Before each LLM call, it captures the current hypothesis. After each tool result, it links the observation back. These flow as metadata ON existing SSE events (not separate events — avoids stream flooding).

Chat visibility: New activity card renderer in the streaming service (see Design Decision 1) shows collapsible reasoning traces inline:

┌─ Reasoning ──────────────────────────────┐
│ Hypothesis: User wants module X updated  │
│ ▸ Step 1: read_file(module.ts) → ✓      │
│   Observation: File exists, 240 lines    │
│ ▸ Step 2: write_file → ✓               │
│   Confidence: 0.92                       │
│ Conclusion: Applied pattern from S234    │
└──────────────────────────────────────────┘

Collapsed by default. Expandable for debugging. Reasoning traces persisted alongside execution traces in MongoDB for eval scoring and future recall.

API ownership: backend-vitto-api.


Pillar 2: Perfect Memory

Delegated to S239. See docs/english/Plans/S239-memory-system-handoff.md.

Covers:

S238 owns the wiring: Once S239 builds these services, S238 ensures context assembly injects them and chat plugins render memory state ("I remember from WhatsApp yesterday...").


Pillar 3: Autonomous Execution (Background Tasks)

Gap: CostGuard caps all tasks at 10-15 minutes. Checkpoints exist (Redis) but only for crash recovery, not for chaining multi-step work across time windows.

Solution: A BackgroundTaskService that decomposes complex tasks into a chain of bounded subtasks. Each subtask runs through the normal AgentLoopService.run() with normal CostGuard limits. The orchestrator chains them via BullMQ job dependencies.

Key design decisions:

Chat visibility: Activity card in chat (via ActivityCardHostComponent):

┌─ Background Task: Competitor Analysis ───┐
│ ✓ Step 1/4: Web research (done, $0.12)  │
│ ⏳ Step 2/4: Analyzing findings...       │
│ ○ Step 3/4: Draft report (pending)       │
│ ○ Step 4/4: Format & deliver (pending)   │
│ [Cancel] [Pause after current step]      │
└──────────────────────────────────────────┘

API ownership: backend-vitto-api. Uses existing TaskQueueService (BullMQ) and AgentLoopService.


Pillar 4: Persistent Presence (Desktop Daemon)

What exists (verified): Tauri v2 app at apps/vitto-desktop/ — v0.3.0, signed for macOS, starts hidden, autostart via LaunchAgent, 13 native commands, global hotkey Cmd+Ctrl+V, auto-updater. See docs/english/Apps/vitto-desktop-app/changelog.md.

4 surgical changes needed (no rewrite):

4a. Smart heartbeat — Replace blind polling in VittoHeartbeatService with idle-aware logic:

4b. Infinite WebSocket reconnection:

// vitto-screen-capture.service.ts
reconnectionAttempts: Infinity  // was 10

4c. Proactive notification → chat bridge — Desktop becomes a first-class channel in ProactiveDispatcherService. Tapping a notification deep-links into chat with context pre-loaded.

4d. Version tracking — Tray "About Vitto" shows current version. /vitto/debug route shows desktop app version when connected via LocalBridge. Every release updates docs/english/Apps/vitto-desktop-app/changelog.md.

Chat visibility: Proactive findings render as summary cards:

┌─ Vitto noticed while you were away ──────┐
│ 3 pending tasks need attention           │
│ Meeting with Design team in 45 min       │
│ Module auth-guard has 2 failing tests    │
│ [Review Tasks] [Dismiss]                 │
└──────────────────────────────────────────┘

Renders via existing ProactiveToastComponent + new card renderer in the activity card system.

API ownership: Tauri app is frontend. Calls existing POST /engine/message. No new API endpoints.


Pillar 5: External Reach (Sub-Agent Power + OAuth)

What's done: OAuth full cycle, expert sub-agents with generateDefinition() + marketplace(), Teams adapter, parallel sub-Vitto orchestration, context-aware tool filtering.

What's NOT done:

5a. Sub-agent autonomy gating — Sub-agents with elevated tools (run_command, write_file) execute without approval checkpoints. Wire AutonomyService.shouldProceed() before sub-agent execution when elevated tools are present.

Chat visibility: Approval card in the activity card system:

┌─ Sub-Agent Request ──────────────────────┐
│ Agent: "Code Reviewer" (coder role)      │
│ Wants: run_command("yarn test")          │
│ Autonomy: SUGGEST (needs approval)       │
│ [Approve] [Approve All for Session] [Deny]│
└──────────────────────────────────────────┘

5b. OAuth chat wiring — OAuth works on the backend but there's no chat plugin to show connection status or trigger flows inline. When Vitto needs Google access and it's not connected, render a connect button:

┌─ Connect Google ─────────────────────────┐
│ I need access to your Google Calendar.   │
│ [Connect Google Account]                 │
│ (Opens OAuth consent in popup)           │
└──────────────────────────────────────────┘

Uses the existing oauth_connect tool which returns an auth URL — the plugin turns it into a clickable action.

VM tier: Deferred. See docs/english/Plans/S-future-vm-agent-tier-handoff.md.

API ownership: backend-vitto-api. Frontend plugins in @libs-ai/vitto-chat.


Pillar 6: Self-Improvement (Closed-Loop Learning)

What's done: Outcome loop closed, SelfLearningPlugin, eval framework (30 golden, nightly cron), learning sync (6AM), anonymous pattern extractor.

What's NOT done:

6a. Closed-loop skill learning:

Chat visibility: When Vitto uses a skill, reasoning plugin shows:

┌─ Skill Used ─────────────────────────────┐
│ "CRUD module pattern" (trusted, 92%)     │
│ 18/20 past uses succeeded                │
└──────────────────────────────────────────┘

6b. Eval system hardening:

6c. Persona drift auto-apply:

Chat visibility:

┌─ Vitto Adapted ──────────────────────────┐
│ I noticed you prefer shorter answers.    │
│ Adjusted: explanationDepth 7 → 5         │
│ [Undo] [That's right]                    │
└──────────────────────────────────────────┘

API ownership: backend-vitto-api. Existing services modified, no new services.


Pillar 7: Soul & Identity (Personality + Visual Evolution)

What's done: VittoEvolutionService (maturity scoring, 5 stages), avatar engine (14 moods, gaze tracking), VittoBrainVisualizationComponent (exists but static).

What's NOT done:

7a. Brain visualization wired to live data — Pure frontend. Connect existing SSE conscience stream + maturity score API to the brain page. Show:

7b. Avatar reflects evolution stage — MoodDefinitionService needs a stageOverlay that modifies base mood properties (aura complexity, feature richness) based on current VittoEvolutionService stage.

7c. Evolution milestones in chat — When Vitto crosses a stage boundary, render in the activity card system:

┌─ Vitto Evolved ──────────────────────────┐
│ Stage: Aware → Connected                 │
│ Trigger: 200th memory stored             │
│ Maturity: ████████░░ 62/100              │
└──────────────────────────────────────────┘

Priority: Lowest. Build last. Fun and differentiating but doesn't make Vitto smarter.

API ownership: Frontend (avatar, brain visualization). backend-vitto-api VittoEvolutionService already exists.


Part II: Verified Audit — What's Done vs What's Not

Completed (20 items — codebase-verified 2026-03-19)

# Feature Source Spec Key File(s) Verified By
1 Sub-Vitto engine (parallel orchestration) S232/S234 sub-vitto-orchestrator.service.ts (312 lines) Promise.allSettled() at line 184, calls agentLoop.run() at line 227
2 SubAgent→AgentLoop refactor S234 sub-agent.service.ts (1,049 lines) agentLoop.run() at line 719
3 Context-aware tool filtering S237 tool-registry.service.ts TOOL_CATEGORIES (lines 40-71), INTENT_TOOL_MAP (lines 74-87), getToolsForContext() (lines 199-260)
4 Memory priority fix (dossier elevated) S237 context-assembly Dossier elevated to HIGH priority in token budget
5 Conversation history injection S237 agent-loop / vitto-engine AgentSessionService.getConversationHistory() injected into loop
6 SSE ticket auth S232.1-A sse-ticket.service.ts, sse-ticket.guard.ts, sse-ticket.controller.ts JWT 5-min expiry, cross-user protection
7 AgentProcess redirect to Vitto S232.1-A agent-process.controller.ts (ai-api) Proxies to vitto-api, logs "[S232.2] PROXYING"
8 OAuth credential delegation S232 oauth.controller.ts, oauth.service.ts (backend-api), oauth.plugin.ts (vitto-api), user-oauth-connection.schema.ts Full cycle: consent, AES-256-GCM, proxy, Google+GitHub
9 Expert sub-agent creation S232 sub-agent.service.ts generateDefinition() lines 169-240, marketplace() lines 249-282
10 Vitto evolution (maturity score) S232 vitto-evolution.service.ts (131 lines) Normalized scoring, 5 stages, conscience events
11 Persona adaptation (confidence thresholds) S232 dossier-persona-adapter.service.ts (148 lines) Confidence 0-1, rate-limited 1/user/24h, auto-apply disabled
12 Teams channel adapter S232 teams-channel.adapter.ts (151 lines) RichMessage → Adaptive Cards
13 MongoDB cleanup S232 N/A channel-user-links and vitto-task-history — zero references in codebase
14 Embedding search wired S231.2 learning-embedding-adapter.service.ts Wired in LearningService.onModuleInit()
15 Cost tracker (5 bug fixes) S231.2 cost-tracker.service.ts, vitto.controller.ts MongoDB persistence in onModuleInit(), controller returns cost field
16 Eval framework S231.2 eval/eval-runner.service.ts 30 golden entries, @Cron('0 2 * * *'), 7-day rolling avg
17 In-app observability S231.2 vitto-observability.service.ts (386 lines) Metrics aggregation, anomaly detection, slow task tracking
18 Outcome loop closed S216/S232.1 outcome-feedback.service.ts sendOutcome() → postExecutionOutcome() → ai-api /learning/execution-outcomes
19 S216 wiring (9 gaps) S216 Multiple files 8/9 fixed (episodic gate, skill composition, extraction notify, duplicate outcome, channel filter, space brief, rich digest, alert ack, voice note delivery)
20 Checkpoint/resume S232.1 checkpoint.service.ts Redis-backed, persists agent loop state before each LLM call

Remaining (11 items — this spec)

# Feature Pillar Priority Sessions Est. Owner
1 Activity Card System (streaming + renderers) ALL HIGH 2 S238
2 Reasoning layer (ReasoningService) 1: Cognitive HIGH 2 S238
3 Cross-channel genome 2: Memory HIGH ~2 S239
4 User preference learning 2: Memory HIGH ~2 S239
5 Background task chains 3: Autonomous HIGH 3 S238
6 Desktop daemon (4 changes) 4: Presence MEDIUM 2 S238
7 Sub-agent autonomy gating 5: External MEDIUM 1 S238
8 OAuth chat wiring 5: External MEDIUM 1 S238
9 Closed-loop skill learning 6: Self-Improve HIGH 2 S238
10 Eval hardening + persona auto-apply 6: Self-Improve MEDIUM 1 S238
11 Brain visualization wired 7: Soul LOW 1 S238
S238 scope ~15 sessions
S239 scope (parallel) ~4 sessions

Deferred (with handoff docs)

Feature Handoff Doc Reason
VM agent tier docs/english/Plans/S-future-vm-agent-tier-handoff.md Low ROI — in-process handles 90% of cases
Voice pipeline Deferred by owner Do it right in a later session
Browser extension relay N/A VM snapshots solve 80% of same need

Part III: Build Phases (Dependency-Ordered)

Phase 0: Activity Card System (BLOCKING — All pillars depend on this)

Why first: Every pillar requires chat visibility via inline activity cards. The existing ChatPlugin interface does NOT support SSE-event-to-card dispatch (it supports tabs, slash commands, quick actions, and output components — none map to dynamic SSE card rendering). The existing GenerationPlugin also does NOT do SSE-to-card dispatch. This means Phase 0 must build a new rendering mechanism.

Architecture decision: The correct approach is NOT to extend the ChatPlugin interface. Instead, extend the streaming service and message rendering layer — which already handles tool result rendering inline. The pattern is:

  1. VittoChatStreamingService parses SSE events via handleStreamEvent() (currently a fixed StreamEvent type union: content_delta, phase_start, phase_progress, tool_start, generation_complete, error)
  2. Extend StreamEvent type union with new event types for activity cards
  3. New events are mapped to activity entries in the message stream (same as how tool results appear inline)
  4. Activity entries are rendered by a new ActivityCardHostComponent that dynamically resolves the correct card component based on event type

What to build:

File 1 — MODIFY: libs/libs-ai/vitto-chat/src/lib/services/vitto-chat-streaming.service.ts

Extend StreamEvent type union:

Existing: 'content_delta' | 'phase_start' | 'phase_progress' | 'tool_start' | 'generation_complete' | 'error'
Add: 'activity_card'

In handleStreamEvent(), add an activity_card case that creates an activity entry in the message stream. The activity entry contains: { cardType: string, data: any, interactive: boolean }.

Backend SSE events that should trigger activity cards:

Backend SSE Event cardType Interactive Pillar
tool.starting (with reasoning metadata) reasoning-trace No (collapsed) 1: Cognitive
background.step_completed, background.chain_status background-task Yes (cancel/pause) 3: Autonomous
conscience.proactive_summary proactive-summary Yes (review/dismiss) 4: Presence
subvitto.approval_required sub-agent-approval Yes (approve/deny) 5: External
tool.called where tool=oauth_connect oauth-connect Yes (connect button) 5: External
conscience.skill_used skill-usage No 6: Self-Improve
conscience.persona_adapted persona-adapted Yes (undo/confirm) 6: Self-Improve
conscience.evolution.visual evolution-milestone No 7: Soul

Note: Memory cards (Pillar 2) deferred to S239. Build 8 card types now, add memory card when S239 delivers.

File 2 — NEW: libs/libs-ai/vitto-chat/src/lib/components/activity-cards/activity-card-host.component.ts

Dynamic component host that resolves card type → component. Uses Angular's NgComponentOutlet or ViewContainerRef.createComponent(). Maps cardType string to component class via a registry map.

File 3 — NEW: libs/libs-ai/vitto-chat/src/lib/components/activity-cards/cards/ (directory with 8 card components)

Each card is a standalone Angular component with OnPush, signal inputs. All follow the same pattern:

File 4 — MODIFY: libs/libs-ai/vitto-chat/src/lib/components/chat-shell/ (message rendering area)

Register ActivityCardHostComponent as an inline renderer within the message stream, alongside the existing tool result renderers.

Dependencies: None. This is pure frontend. Estimated effort: 2 sessions.


Phase 1: Reasoning Layer (Pillar 1)

Depends on: Phase 0 (VittoActivityPlugin for chat rendering)

What to build:

File 1 — NEW: APIs/backend-vitto-api/src/app/engine/reasoning.service.ts

Purpose: Wraps AgentLoopService with reasoning metadata.

Interface:
  - enrichToolCall(toolCall, context) → adds hypothesis field to SSE event
  - enrichToolResult(toolResult, hypothesis) → adds observation + confidence
  - synthesize(observations[]) → generates conclusion with confidence score

Implementation:
  - NOT an LLM call. Uses heuristics to extract hypothesis from the LLM's
    response text (regex for "I'll", "Let me", "I think", etc.)
  - Confidence = tool success rate for this pattern (from execution traces)
  - Persists reasoning trace in execution-traces collection alongside existing data

File 2 — MODIFY: APIs/backend-vitto-api/src/app/engine/agent-loop.service.ts

Inject ReasoningService. Before each tool call emission, call enrichToolCall(). After each tool result, call enrichToolResult(). The existing SSE events (tool.starting, tool.called) gain metadata fields — no new event types needed:

// Existing event:
{ type: 'tool.starting', data: { toolName, args } }
// Becomes:
{ type: 'tool.starting', data: { toolName, args, reasoning: { hypothesis, confidence } } }

File 3 — MODIFY: APIs/backend-vitto-api/src/app/mind/schemas/execution-trace.schema.ts

Add reasoningTrace field (array of { hypothesis: string, observation: string, confidence: number, toolName: string }) to persist reasoning alongside execution data.

File 4 — MODIFY: APIs/backend-vitto-api/src/app/engine/engine.module.ts

Register ReasoningService in providers array.

File 5 — Frontend: ReasoningTraceCardComponent (from Phase 0) renders the enriched events.

Dependencies: Phase 0 for rendering. Estimated effort: 2 sessions.


Phase 2: Background Task Chains (Pillar 3)

Depends on: Phase 1 (reasoning traces help debug chain failures)

What to build:

File 1 — NEW: APIs/backend-vitto-api/src/app/engine/background-task.service.ts

Purpose: Decomposes complex tasks into bounded subtask chains.

Interface:
  - decompose(task, template?) → SubtaskChain
  - executeChain(chain, userId) → starts BullMQ job chain
  - getChainStatus(chainId) → progress across all subtasks
  - cancelChain(chainId) → cancels remaining subtasks
  - pauseChain(chainId) → stops after current subtask completes

Templates (initial set):
  - RESEARCH: search → analyze → synthesize → deliver
  - CODE_REVIEW: read → analyze → report → suggest
  - GENERATE: plan → scaffold → implement → verify

Implementation:
  - Each subtask is a BullMQ job with { chainId, stepIndex, prevOutput }
  - Job dependency: step N depends on step N-1
  - prevOutput stored in MongoDB (chain-checkpoints collection), NOT BullMQ job data
    (BullMQ has payload size limits)
  - SmartRouter picks model per step via complexity hint in template
  - CostGuard applies per-subtask (existing limits, no changes)

File 2 — MODIFY: APIs/backend-vitto-api/src/app/engine/vitto-engine.service.ts

In processTaskWorkflow(), before delegating to this.phaseExecutor.execute(task, sseSubject), check if RequestClassifierService classifies complexity as 'complex' AND the task description matches a background template. If so, route to BackgroundTaskService.decompose() instead. Note: the classifier uses 'simple' | 'moderate' | 'complex' (not HIGH/CRITICAL).

File 3 — MODIFY: Frontend BackgroundTaskCardComponent (from Phase 0) renders chain progress with cancel/pause actions.

File 4 — NEW: APIs/backend-vitto-api/src/app/engine/schemas/chain-checkpoint.schema.ts

MongoDB schema for chain state: { chainId, stepIndex, output, status, cost, startedAt, completedAt }.

File 5 — MODIFY: APIs/backend-vitto-api/src/app/engine/vitto.controller.ts (or NEW controller)

Add REST endpoints for frontend card actions:

GET    /engine/background/:chainId/status  → getChainStatus()
POST   /engine/background/:chainId/cancel  → cancelChain()
POST   /engine/background/:chainId/pause   → pauseChain()
POST   /engine/background/:chainId/resume  → resumeChain()

The BackgroundTaskCardComponent calls these when user clicks [Cancel] or [Pause].

File 6 — MODIFY: APIs/backend-vitto-api/src/app/engine/engine.module.ts

Register BackgroundTaskService in providers, register ChainCheckpoint schema via MongooseModule.forFeature().

Dependencies: Phase 0 for rendering. Can start after Phase 0 (does NOT depend on Phase 1 — reasoning traces are nice-to-have for debugging but not functionally required). Estimated effort: 3 sessions.


Phase 3: Self-Improvement (Pillar 6)

Depends on: None (can parallelize with Phase 2)

What to build:

3a. Skill confidence tiers:

File 1 — MODIFY: APIs/backend-vitto-api/src/app/engine/skill-manager.service.ts

Add to schema: confidenceTier (experimental|validated|trusted), outcomeHistory[], successCount, failureCount, rollingSuccessRate, warmStartExpiresAt.

Add/modify methods:

File 2 — MODIFY: APIs/backend-vitto-api/src/app/engine/phase-executor.service.ts

After VERIFY phase, if task.injectedSkills?.length > 0, call skillManager.recordOutcome() for each.

File 3 — MODIFY: Skill retrieval (in context assembly or skill-retrieval service)

Weight by tier: trusted=1.0, validated=0.7, experimental=0.4 (or 0.7 during warm start). Inject top 3 by weight.

3b. Eval hardening:

File 4 — MODIFY: APIs/backend-vitto-api/src/app/eval/eval-runner.service.ts

Add evalVersion (git commit hash via env var) to each eval result. Score by category (general, task, code, memory, edge). Alert on 5% per-category drop.

3c. Persona auto-apply:

File 5 — MODIFY: APIs/backend-vitto-api/src/app/engine/dossier-persona-adapter.service.ts

Enable auto-apply for explanationDepth and formality only. Keep approval gate for humorLevel, proactivity. Emit persona.adapted SSE event (consumed by PersonaAdaptedCardComponent from Phase 0).

Dependencies: Phase 0 for chat rendering of skill usage and persona adaptation. Estimated effort: 3 sessions (2 for skills, 0.5 for eval, 0.5 for persona).


Phase 4: External Reach (Pillar 5)

Depends on: Phase 0 (approval cards, OAuth cards)

What to build:

4a. Sub-agent autonomy gating:

File 1 — MODIFY: APIs/backend-vitto-api/src/app/engine/sub-agent.service.ts

Before executeTask() runs the agent loop, check if the tool list contains elevated tools (write_file, run_command, etc.). If so, call autonomyService.shouldProceed(). If autonomy level is SUGGEST, emit subvitto.approval_required SSE event and await approval via checkpoint.

File 2 — MODIFY: Frontend SubAgentApprovalCardComponent (from Phase 0) renders approval UI with [Approve], [Approve All for Session], [Deny].

4b. OAuth chat wiring:

File 3 — NEW: libs/libs-ai/vitto-chat/src/lib/plugins/vitto-activity/cards/oauth-connect-card.component.ts

When oauth_connect tool returns an auth URL, the card renders a connect button. On click, opens OAuth consent popup. On completion, refreshes connection status.

File 4 — MODIFY: APIs/backend-vitto-api/src/app/tools/plugins/oauth.plugin.ts

When oauth_api_call fails with "not connected", emit oauth.connect_required SSE event with service name and required scopes. This triggers the frontend card.

Dependencies: Phase 0 for rendering. Estimated effort: 2 sessions (1 for autonomy, 1 for OAuth wiring).


Phase 5: Desktop Daemon (Pillar 4)

Depends on: None (independent track, can parallelize)

What to build:

5a. Smart heartbeat:

NOTE: VittoHeartbeatService (S231.2) already has idle-aware logic with POLL_INTERVAL_MS=10min and IDLE_THRESHOLD_MS=5min using JS mousemove/keydown listeners. What's missing is native macOS idle detection (current JS listeners only work when the app window is focused) and native notification dispatch.

File 1 — MODIFY: libs/libs-ai/vitto-core/src/lib/services/vitto-heartbeat.service.ts

Replace JS-based idle detection with Tauri native command system_info (which already returns system metrics including idle time via CGEventSourceSecondsSinceLastEventType). This makes idle detection work system-wide, not just when the Vitto window is focused.

Add notification dispatch: on actionable result, invoke Tauri's tauri-plugin-notification (already installed). On notification tap, deep-link to chat via vitto://chat?context={taskId}.

5b. Infinite reconnection:

NOTE: vitto-screen-capture.service.ts already has reconnectionAttempts: Infinity (S231.2 fix). Verify this is also true for the LocalBridge WebSocket connection — if not, fix there.

5c. Version tracking:

File 3 — MODIFY: Tray menu in lib.rs — add "About Vitto v{VERSION}" item File 4 — MODIFY: /vitto/debug route — show desktop version when LocalBridge is connected (version sent in WebSocket handshake)

5d. Changelog rule: Every Tauri release:

  1. Bump version in Cargo.toml
  2. Update docs/english/Apps/vitto-desktop-app/changelog.md
  3. Push to vitto-updater.gograb.space

Dependencies: None. Estimated effort: 2 sessions.


Phase 6: Soul & Identity (Pillar 7) — Build Last

Depends on: Phase 0 (card rendering)

What to build:

6a. Brain visualization:

File 1 — MODIFY: libs/libs-ai/vitto-hub/src/lib/components/brain/vitto-brain-visualization.component.ts

Wire to live data sources:

6b. Avatar stage overlay:

File 2 — MODIFY: libs/libs-anim/vitto-avatar/src/lib/services/vitto-mood.service.ts

Add applyStageOverlay(stage) that modifies base mood properties:

6c. Evolution milestones in chat:

Already handled by EvolutionMilestoneCardComponent in Phase 0. Backend already emits conscience.evolution.visual events via VittoEvolutionService (line 99). The streaming service maps this to cardType evolution-milestone.

Dependencies: Phase 0. Estimated effort: 1 session.


Part IV: Verification Matrix

How to check if a feature is done

Every phase has programmatic verification gates. Run these commands to check completion without reading code.

Phase 0: Activity Card System

Gate How to Verify Expected
Card host exists ls libs/libs-ai/vitto-chat/src/lib/components/activity-cards/activity-card-host.component.ts File exists
Card components exist ls libs/libs-ai/vitto-chat/src/lib/components/activity-cards/cards/ 8 component directories
StreamEvent extended grep "activity_card" libs/libs-ai/vitto-chat/src/lib/services/vitto-chat-streaming.service.ts At least 1 hit
Compiles yarn build gograb-space 0 errors
Renders in chat Open web chat → trigger any registered SSE event → card appears inline Visual verification

Phase 1: Reasoning Layer

Gate How to Verify Expected
Service exists ls APIs/backend-vitto-api/src/app/engine/reasoning.service.ts File exists
Injected in agent loop grep -n "reasoningService|ReasoningService" APIs/backend-vitto-api/src/app/engine/agent-loop.service.ts At least 2 hits
SSE events enriched Send a chat message → inspect SSE stream → tool.starting event has reasoning field reasoning.hypothesis present
Trace persisted After task completion → db.getCollection('execution-traces').findOne({}, {sort: {createdAt: -1}}) → check for reasoningTrace field Array of hypothesis/observation pairs
Chat card renders Send a message that triggers tool use → reasoning card appears in chat Collapsible card with hypothesis/observations

Phase 2: Background Tasks

Gate How to Verify Expected
Service exists ls APIs/backend-vitto-api/src/app/engine/background-task.service.ts File exists
Schema exists ls APIs/backend-vitto-api/src/app/engine/schemas/chain-checkpoint.schema.ts File exists
Chain executes Send complex task via chat → observe chain decomposition → all subtasks complete Multi-step card shows progress
Checkpoint persists db.getCollection('chain-checkpoints').find({ chainId: '<id>' }) Documents per subtask
Cancel works Start chain → click [Cancel] → remaining subtasks don't execute Chain status = cancelled
Cost tracking Each subtask has independent cost in chat card Per-step cost displayed

Phase 3: Self-Improvement

Gate How to Verify Expected
Skill tiers in schema grep "confidenceTier" APIs/backend-vitto-api/src/app/engine/skill-manager.service.ts Schema field exists
Outcome recorded After task with injected skill → db.getCollection('vitto-skills').findOne({ _id: '<skillId>' }) outcomeHistory has entry, counters incremented
Graduation Skill with 10+ successes and ≥80% rate confidenceTier: 'validated'
Eval version db.getCollection('eval-results').findOne({}, {sort: {createdAt: -1}}) evalVersion field present
Persona auto-apply After 10+ interactions → check persona sliders explanationDepth or formality changed, conscience event emitted
Chat cards Skill usage card and persona adapted card render in chat Visual verification

Phase 4: External Reach

Gate How to Verify Expected
Autonomy gate Create sub-agent with coder role → execute → approval card appears Card with [Approve] [Deny] buttons
Approval flow Click [Approve] → sub-agent executes → result returned End-to-end success
OAuth card Ask Vitto "check my calendar" without Google connected → connect card appears Card with [Connect Google Account] button
OAuth flow Click connect → popup → authorize → Vitto retries calendar check End-to-end success

Phase 5: Desktop Daemon

Gate How to Verify Expected
Starts hidden Install .dmg → app starts → no window visible → tray icon present Tray icon visible, no window
Hotkey Press Cmd+Ctrl+V Window appears
Close-to-tray Close window → tray icon still present → process running pgrep Vitto returns PID
Heartbeat fires Leave machine idle >5 min → wait → notification appears (if actionable) Native macOS notification
Infinite reconnect Stop backend → wait 30s → restart backend → app reconnects WebSocket reconnected (check tray icon state)
Version in tray Right-click tray icon "About Vitto v0.4.0" visible
Survives logout Logout → login → tray icon present LaunchAgent auto-restart

Phase 6: Soul & Identity

Gate How to Verify Expected
Brain page live data Navigate to /vitto/brain Maturity score, stage, memory stats, skill counts displayed
Avatar stage overlay Check avatar at different maturity stages Visual difference between nascent and connected
Evolution card in chat Trigger stage boundary (create enough memories) Milestone card appears

Part V: Supersession Record

What Each Prior Spec Covered and Current Status

Spec Date Session Status Details
S216 Wiring Design 2026-03-13 S216 FULLY SUPERSEDED 9 wiring gaps. 8 fixed, 1 partially fixed (space brief bootstrap returns minimal text — acceptable). All fixes verified in codebase.
S225.4 Evolution Spec 2026-03-17 S225.4 FULLY SUPERSEDED Phase 1 complete (embedding, cost, health, Tauri prep, proxy). Phases 2-4 absorbed into S232 comprehensive spec. Voice deferred.
S232.1 Unified Plan 2026-03-18 S232.1 FULLY SUPERSEDED Track A (security): complete. Track B (sub-agents): complete via S234. Track C (Teams, OAuth): complete. Track D (skill learning, genome): absorbed into this spec Phases 3 & S239. Track E (browser, voice): deferred with handoff docs.
S232 Comprehensive Design 2026-03-18 S232 FULLY SUPERSEDED 8/9 sections complete. VM tier (Section 2) deferred with handoff doc. Remaining work (skill learning, persona auto-apply, brain visualization) absorbed into this spec Phases 3 and 6.

Delegated Work

Delegation Handoff Doc Status
Memory system improvements S239-memory-system-handoff.md Active — S239 concurrent session
VM agent tier S-future-vm-agent-tier-handoff.md Deferred — future session
Voice pipeline Not yet written Deferred by owner — do it right later
Desktop changelog changelog.md Active — maintained per release

Design Decisions (This Spec)

# Decision Chosen Rationale
1 Activity card architecture Extend streaming service + ActivityCardHostComponent with dynamic card registry — NOT the ChatPlugin interface (which lacks SSE event routing) ChatPlugin supports tabs/slashCommands/quickActions/outputComponents, NOT SSE-to-card dispatch. Activity cards are inline message elements rendered by the streaming layer, same pattern as tool result renderers.
2 Reasoning events Metadata ON existing SSE events, not separate events Avoids SSE stream flooding. tool.starting gains a reasoning field. StreamEvent type union extended with 'activity_card' for new card types.
3 Background task decomposition Template-guided, not free-form LLM LLM decomposition is unreliable. Templates (research, code_review, generate) provide structure; LLM fills specifics.
4 VM tier Deferred with handoff doc Low ROI — in-process handles 90% of cases. Computer-use already handles sandboxed execution.
5 Skill cold-start 2-week warm start period Prevents experimental skills from being permanently deprioritized due to insufficient usage data.
6 Persona auto-apply Low-risk sliders only (explanationDepth, formality) High-risk sliders (humor, proactivity) require user consent. Matches existing confidence threshold architecture.
7 Brain visualization Frontend-only changes All data sources exist. No new backend work for this pillar.
8 Desktop daemon scope 4 surgical changes, no rewrite Minimize risk for an app with 0 current users. Scale effort with adoption.
9 Voice pipeline Deferred entirely Do it right in a dedicated session. Owner's explicit decision.

Build Order Summary

Phase 0: Activity Card System ────────────── 2 sessions (BLOCKING)
   │
   ├── Phase 1: Reasoning Layer ──────────── 2 sessions
   │
   ├── Phase 2: Background Tasks ─────────── 3 sessions (PARALLEL with Phase 1)
   │
   ├── Phase 3: Self-Improvement ─────────── 3 sessions (PARALLEL with Phases 1+2)
   │
   ├── Phase 4: External Reach ──────────── 2 sessions (after Phase 0)
   │
   ├── Phase 5: Desktop Daemon ──────────── 2 sessions (INDEPENDENT, any time)
   │
   └── Phase 6: Soul & Identity ─────────── 1 session (LAST)

Total: ~15 sessions (9 S238 scope + 2 delegated S239 + 4 deferred VM)
Parallelizable: Phases 1+2+3 (all after Phase 0), Phase 5 independent
Critical path: Phase 0 → Phases 1+2+3 in parallel (5 sessions) = 7 sessions

Testing Guide

How to Test Each Pillar End-to-End

Pillar 1: Cognitive Core

  1. Open web chat at localhost:6200
  2. Send: "What modules do we have in this space?"
  3. Expand the reasoning card that appears below Vitto's response
  4. Verify: hypothesis, tool calls with observations, conclusion with confidence
  5. Navigate to /vitto/introspection → verify reasoning traces appear in the admin stream

Pillar 2: Memory (S239)

  1. Send a message on WhatsApp about "project deadline is Friday"
  2. Open web chat
  3. Send: "What did we discuss about deadlines?"
  4. Verify: Vitto recalls the WhatsApp conversation with cross-channel attribution

Pillar 3: Background Tasks

  1. Send: "Research the top 5 competitors in our market and write a summary report"
  2. Verify: Background task card appears with 4 steps
  3. Watch steps complete sequentially with per-step cost
  4. Click [Pause after current step] → verify chain pauses
  5. Resume → verify remaining steps complete
  6. Check MongoDB: db.getCollection('chain-checkpoints').find({ chainId: '<id>' })

Pillar 4: Desktop Daemon

  1. Install Vitto.dmg
  2. Verify: no window on startup, tray icon visible
  3. Press Cmd+Ctrl+V → window appears
  4. Close window → verify tray icon still present
  5. Leave machine idle 5+ min → verify notification appears if actionable
  6. Right-click tray → verify "About Vitto v0.4.0" visible
  7. Kill backend → wait 30s → restart → verify reconnection (tray icon changes state)

Pillar 5: External Reach

  1. Create a sub-agent with coder role at /vitto/agents
  2. In chat: "Ask the code reviewer agent to check the auth module"
  3. Verify: approval card appears (if autonomy = SUGGEST)
  4. Click [Approve] → sub-agent executes → result returned
  5. Ask: "Check my Google Calendar" (without Google connected)
  6. Verify: OAuth connect card appears
  7. Click [Connect Google Account] → complete OAuth → Vitto retries

Pillar 6: Self-Improvement

  1. Chat with Vitto for 10+ messages using short, direct language
  2. Verify: persona adapted card appears ("Adjusted explanationDepth")
  3. Click [Undo] → verify slider reverts
  4. Navigate to /vitto/skills → check that skills have confidence tiers
  5. Run eval manually: POST /eval/run → check results have evalVersion field

Pillar 7: Soul & Identity

  1. Navigate to /vitto/brain
  2. Verify: maturity score, current stage, memory stats, skill counts displayed live
  3. Check avatar: visual properties should reflect current evolution stage
  4. Trigger stage boundary (e.g., store 200th memory) → evolution card in chat

Owner: fguijarro Session: S238.5 Date: 2026-03-19 Estimated effort: ~15 sessions (S238 scope) + S239 (memory) + future (VM, voice) Prior specs superseded: 4 (S216, S225.4, S232, S232.1) This is the terminal Jarvis spec.