Vitto Infrastructure, Competitive Analysis & Strategic Gaps

Comprehensive analysis of Vitto's architecture vs. the 2026 AI agent landscape. Covers internal systems (backend-vitto-api, backend-ai-api), competitor deep dives (OpenClaw, OpenAI, Google, Microsoft, Anthropic, Simular, Perplexity, and 10+ others), desktop control comparison, strategic gaps, and a phased roadmap to Jarvis-parity.

Vitto Infrastructure, Competitive Analysis & Strategic Gaps

Purpose: Single source of truth for Vitto's technical position in the 2026 AI agent market. This document informs all future architectural decisions, sprint planning, and investment prioritization. Updated March 17, 2026.


Table of Contents

  1. Executive Summary
  2. Internal Architecture — What We've Built
  3. Competitive Landscape — 2026 AI Agent Market
  4. Desktop Control — Detailed Comparison
  5. Where Vitto Has No Competition
  6. Critical Gaps & Technical Debt
  7. Architecture Patterns — Industry State of the Art
  8. Strategic Differentiators & Business Moats
  9. Phased Roadmap to Jarvis-Parity
  10. Key Decisions & Open Questions
  11. Reference: Essential Files

1. Executive Summary

Vitto is architecturally ahead of every competitor except Microsoft (Work IQ + Windows 365 + M365 distribution) and Perplexity (hardware-first 24/7 agent). Our unique position is: cloud intelligence + local presence + continuous learning — nobody combines all three.

What We Have That Nobody Else Does

What We're Missing

Strategic Position

The market is splitting into: (1) Big Tech integrated agents with massive distribution, (2) Specialized startups targeting niches, and (3) Open-source democratizing access. Nobody has nailed the full "Jarvis" vision. We have the deepest technical foundation — we need to ship the local presence and close the observability gap.


2. Internal Architecture

2.1 backend-vitto-api

Role: Supreme orchestrator. Every AI interaction routes through it. Port: 3003 | Database: vitto-mind + vitto-users | Deployment: Railway

Module Inventory (17 modules)

Module Role
EngineModule Core agent loop, 4-phase workflow, 35+ services
MindModule Persistent memory, embeddings, knowledge graph, anomaly detection
LLMModule 5-provider abstraction, smart routing, resilient failover
ConnectivityModule HTTP/WS clients to all backends, channels
ProactiveModule 7 cron monitors, digests, alerts, quality triggers
ModuleGenerationModule 6-phase code generation pipeline
SecurityModule Audit, secrets vault (AES-256-GCM), sandboxing, circuit breakers
A2AModule Google Agent-to-Agent protocol
McpServerModule Vitto as MCP server
AwarenessModule Multi-channel real-time awareness hub
UsersModule User dossiers, job mastery, cross-DB sync
TrainingModule Self-learning, teaching, evaluation plugins
StudioModule GitLab/Git integration for generated code
SpaceBuilderModule NL-to-universe generation
CustomerModule Customer session intelligence
ConnectorModule External connector management
LocalBridgeModule WebSocket bridge to Vitto Desktop (global)

LLM Providers & Routing

Provider Timeout Tool Use Status
Claude (Anthropic) 600s Yes Primary
OpenAI (GPT) 60s Yes Fallback
Gemini (Google) 60s Yes Fallback
DeepSeek 120s Yes Fallback
Ollama (local) — Yes Self-hosted option

Smart Router (SmartRouterService): Routes by complexity — simple (Haiku/Flash/GPT-4o-mini), medium (Sonnet/GPT-4o), complex (Opus/o1). Saves 60-90% on simple tasks.

Resilient Provider (ResilientLLMProvider): Wraps primary with ordered fallback chain. Circuit breaker per provider (5 failures → trip, 60s reset). Failover chain configurable via VITTO_FAILOVER_CHAIN env var.

Per-phase override: VITTO_{PHASE}_MODEL=provider:model lets you use different models for ANALYZE vs EXECUTE.

4-Phase Agentic Workflow

ANALYZE → Classify request (CONVERSATION / COMMAND / COMPLEX TASK)
        → Recall past decisions from MindService
        → Assess autonomy level
PLAN    → Generate execution plan with ordered steps + tool requirements
EXECUTE → AgentLoopService: LLM ↔ tools loop
        → 17 tool plugin bundles, layer-based access control
        → Stuck detection (breaks repeated identical tool calls)
VERIFY  → QualityGateService validates output
        → Records lessons learned

Cost Guards: Max 40 iterations, 300K tokens, 10 min (configurable). Per-complexity scaling: simple = 25%, moderate = 50%, complex = 100%.

Tool Plugin System (17 bundles)

fileCodePlugin, docsPlugin, webPagePlugin, handoffPlugin, memoryPlugin, systemPlugin, httpPlugin, browserPlugin, screenLocalPlugin, computerUsePlugin, platformPlugin, mcpPlugin, moduleGenPlugin, editPlugin, messagingPlugin, visualEditingPlugin, whistlePlugin

Each plugin = (ctx: ToolPluginContext) => ToolHandler[]. Tools filtered by VittoLayer (layer-based access control). Real-time dependencies (bridge, sandbox) dynamically filtered.

Memory Architecture (Tiered)

Tier Mechanism Latency
Hot cache Per-user in-memory LRU Instant
Vector search OpenAI text-embedding-3-small (1536-dim) ~100ms
Text search MongoDB text index ~50ms
Cold recall MongoDB aged query ~200ms

Memory types: conversation, task_execution, code_review, decision, learning, user_preference, correction, space_context, documentation, visual_capture.

Time-decay model with decayProtected flag. Consolidation job synthesizes old memories into learning type summaries via Haiku.

Security Layers (6 deep)

  1. Prompt injection guard — 50+ regex patterns (warn-only since S159)
  2. Credential redactor — strips keys/tokens/passwords from tool output
  3. Secret vault — AES-256-GCM, key rotation, scope-based (global/space/skill/user)
  4. Hash chain verifier — tamper-evident audit logs
  5. Retention policy — GDPR compliance
  6. Sandbox layers — Process → Container → Firecracker microVM

2.2 backend-ai-api

Role: AI intelligence data layer. Decision tracking, learning pipeline, chat persistence, computer use. Port: 3001 | Databases: 5 separate MongoDB connections | Deployment: Railway

Module Inventory (18 modules)

Module Session Role
AgentsModule early Dev agent registry, embeddings, patterns, calibration
PlatformAIModule early Platform training data pipeline
ModuleIntelligenceModule early Module generation outcomes API
InfrastructureModule early Sessions, decisions, handoffs, context, CU loop
ArchiveModule early Cold-storage archival
LearningModule early Intelligence Pyramid (core rules, patterns, triggers)
OrchestrationModule early Multi-agent task coordination
CommandCenterModule early Real-time WebSocket command dashboard
WorkflowModule S11.2 AI-driven platform code generation (SSE)
ChatSessionsModule S11.4 Chat session persistence
AgentProcessModule S12 Unified agent process layer
SuperIntelligenceModule S12 Episodic memory, causal reasoning, curiosity, simulation
SpaceAgentsModule S12 User-facing space builder agents
PlatformApiClientModule S49.5 HTTP client to backend-api
McpGatewayModule S51.2 Cloud MCP gateway (HTTP + WS + billing)
AgentCoreModule S74 Shared agent registry
ChatModule S82 Single-turn chat endpoint
DevModule S104 Developer dashboard statistics

5 MongoDB Databases

Connection Database Key Collections
AGENTS ai-agents decisions, sessions, handoffs, agent_registry, golden_decisions, llm_usage
MEMORY ai-memory agent_embeddings (vector, 1536-dim)
MCP_DEV ai-mcp-dev core_rules, learned_patterns, trigger_states, episodes, causal_graphs, reflections
ARCHIVE ai-archive Cold-storage documents
Default gograb-api users, chat_sessions, user_agent_memories

Intelligence Pyramid (Learning Pipeline)

The full end-to-end pipeline:

1. CAPTURE   → MCP log_decision → DecisionService.create() → MongoDB decisions
2. EMBED     → generateAndStoreEmbedding() (async) → OpenAI embedding → ai-memory
3. EPISODE   → EpisodicMemoryService.storeEpisode() (async) → ai-mcp-dev.episodes
4. TRIGGER   → AdaptiveTriggerService.onNewDecision() → 5 trigger types
5. OUTCOME   → OutcomeDetectorService (cron) → processes pending outcomes
6. MINE      → PatternMiningJob every 6h → groups decisions → upserts learned_patterns
7. PROMOTE   → Patterns ≥3 uses + ≥80% success → CoreRule (daily at 1am)
8. DISTRIBUTE → getPyramidContext(task) → injected into system prompts
9. GOLDEN    → GoldenDecisionService → manually/auto-promoted exemplars
10. CALIBRATE → CalibrationService → agent confidence calibration

Additional intelligence services:

Conversation Genome

ConversationGenomeService.compile() uses Haiku to extract structured JSON: topics, decisions (what/why/status), facts (fact/confidence/source), userIntent, openQuestions, artifacts. Generates OpenAI embedding. Stored on session document. 10-30x token reduction vs raw history.

Tiered compression: Tier 1 (0-24h) full, Tier 2 (1-30d) summarized, Tier 3 (30d+) memory-extracted. Auto-archival at >100 messages AND >7 days inactive.

Computer Use Loop

Full Anthropic computer_20251124 observe-think-act loop:

LLM Resilience


2.3 Integration & Data Flow

Frontend message
  ↓
backend-vitto-api ENGINE (3003)
  → ContextAssemblyService loads:
    - User Dossier + space brief
    - Channel-filtered session history
    - Agent memories from backend-ai-api
    - Persona + behavioral sliders
  → RequestClassifierService → 3-path routing:
    - CONVERSATION → Haiku direct
    - COMMAND → Sonnet
    - COMPLEX TASK → 4-phase pipeline
  → AgentLoopService → 17 tool plugins
  ↓
backend-ai-api WORKLOAD (3001)
  → MCP gateway or module generation
  → LLM calls tracked for cost
  → Episodic memory + outcome feedback
  → Learning pipeline
  ↓
Response → OutcomeFeedback → EpisodicMemory → PatternMining → SkillExtraction

Database Segregation

Database Owner Purpose
gograb-api backend-api:3000 Users, spaces, universes, modules, auth
vitto-mind backend-vitto-api:3003 Dossiers, personas, proactive state, memories
ai-agents backend-ai-api:3001 Decisions, episodes, golden decisions, agent knowledge
ai-memory backend-ai-api:3001 Vector embeddings, semantic search
ai-mcp-dev backend-ai-api:3001 Core rules, learned patterns, trigger states

Rule: No cross-database writes. Vitto → AI handoff is HTTP (idempotent).


2.4 Desktop Control Infrastructure

Three distinct control planes:

Control Plane 1: Remote VM Sandbox (Production Path)

Sandbox Provider Factory (priority selection):

  1. SANDBOX_SERVER_URL → SelfHostedProvider (Hetzner Firecracker)
  2. E2B_API_KEY → E2BCloudProvider (Firecracker microVMs)
  3. Fallback → PlaywrightProvider (headless browser)

Sandbox Orchestrator (APIs/sandbox-orchestrator/):

Control Plane 2: Local Machine Bridge

Two existing implementations:

Implementation Location Capabilities
Node.js CLI apps/vitto-local-bridge/ 5 tools, infinite reconnect, macOS screenshot, single-dep
Tauri v2 Desktop App apps/vitto-desktop/ 13 tools (incl. AppleScript, clipboard, running_apps, system_info), system tray, global hotkey (Cmd+Ctrl+V), OAuth deep link, auto-updater, macOS signed
Angular Browser Bridge libs/libs-ai/vitto-core/ Viewport capture (html2canvas), page actions, Tauri routing

Protocol (LocalBridgeGateway): Socket.IO on /local-bridge namespace. One bridge per user (supersede on reconnect). Timeouts: tool exec 30s, screen capture 15s, page action 10s.

Security: Three-tier model — sandboxed (allowlist: nx, git, yarn, node, etc.), trusted (adds macOS automation), unrestricted (no filtering).

Gap: Neither the Node.js bridge nor the Tauri app are shipped as daemons. No launchd/systemd registration. No heartbeat loop. The Tauri Angular service dies after 10 failed reconnection attempts.

Control Plane 3: Backend Host Desktop

DesktopService executes on the server running backend-ai-api:

8 Service Profiles

Profile Category Start URL
Gmail email mail.google.com
Google Calendar calendar calendar.google.com
GitHub dev github.com
Reddit social reddit.com
LinkedIn social linkedin.com
Twitter social twitter.com
Notion productivity notion.so
Google Search search google.com

Each profile has: credentialFields[], systemPromptHint, suggestedTasks[], uiKnowledge (navigation patterns), persistentByDefault.


3. Competitive Landscape

3.1 OpenClaw Deep Dive

What it is: Open-source, self-hosted, local-first AI agent framework. Created by Peter Steinberger (Nov 2025). Renamed from "Clawdbot" → "Moltbot" → "OpenClaw" (Jan 2026). Steinberger joined OpenAI Feb 2026. Project under 501(c)(3) foundation.

Key stats: 302K+ GitHub stars (fastest-growing OSS project ever). 1,075+ contributors. 54,900+ forks.

Architecture

4-layer stack:

  1. Channel Adapters — 50+ platforms (WhatsApp, Telegram, Discord, Slack, Signal, iMessage, Teams, Matrix, IRC, LINE, WeChat, etc.)
  2. Gateway — Single Node.js process on ws://127.0.0.1:18789
  3. Agent Layer — Multiple isolated agents with SOUL.md + AGENTS.md
  4. Skill Plugins + Memory — 100+ skills on ClawdHub, file-based SQLite memory

What OpenClaw Does Better Than Vitto

Capability OpenClaw Vitto
Messaging channels 50+ out of the box 5 (WhatsApp, Slack, Discord, Telegram, Email)
Observability Built-in OpenTelemetry + Opik (LLM-as-judge) + ClawMetry dashboard Sentry + custom JSON health (no OTEL, no evals)
Skill ecosystem 100+ on ClawdHub, auto-install Internal only, skill composition recently wired (S216)
Local execution IS the local process (zero bridging) WebSocket bridge (requires companion client)
Browser login state Extension Relay preserves existing Chrome sessions Fresh isolated browser every time
Voice Full-duplex Talk Mode (STT + TTS + wake word) Voice note upload only, transcription deferred
Shell PTY Interactive terminal support No PTY mode
Community 302K stars, 1K+ contributors Closed source

What Vitto Does Better Than OpenClaw

Capability Vitto OpenClaw
Multi-tenancy Full user+space scoping, JWT, roles, trust scores None (single-user)
Learning pipeline Intelligence Pyramid (decisions → patterns → rules) File-based memory (no learning)
LLM resilience Smart routing + circuit breakers + 5-provider failover Manual provider config
Agentic workflow 4-phase (ANALYZE → PLAN → EXECUTE → VERIFY) + quality gates Simple LLM ↔ skill loop
Computer Use Full Anthropic CU loop + 3 sandbox providers + supervisor Feature request (Issue #41024)
Code generation 6-phase pipeline + live preview None
Security 6 layers defense-in-depth Default-open, CVE-2026-25253 (CVSS 8.8), 12-20% malicious ClawHub skills
Conversation compression Genome (10-30x reduction) 72-hour in-memory window
VM isolation Firecracker microVMs (hardware-level KVM) Docker containers (optional)

OpenClaw Weaknesses


3.2 Tier 1: Direct Threats

OpenAI — ChatGPT Agent (replaces Operator)

Google — Project Mariner + Chrome Auto Browse

Microsoft — Copilot Studio Computer Use + Work IQ

Simular — OS-Level Desktop Control

Perplexity — Computer + Comet Browser

Rabbit R1 + DLAM


3.3 Tier 2: Partial Overlaps

Player What They Do Strengths Gap vs. Vitto
Claude Cowork (Anthropic) Local file access + Chrome extension. Scheduled tasks. 1M token context, strongest coding agent, safety-first Research preview, no 24/7, no multi-channel, no learning
Devin (Cognition) Autonomous coding agent in sandbox. Goldman Sachs deployed as "Employee #1". $1B+ raised, $73M ARR, SWE-Bench leader Software engineering only, no general purpose
Lindy AI 5,000+ integrations, workflow automation. Phone calls. SOC 2 Type II, HIPAA, GDPR. $54M funded. API/webhook-based, no CU, no desktop
Amazon Nova Act Browser automation, 90% reliability on UI workflows. Bedrock agents with episodic memory. AWS enterprise infra (SOC, HIPAA, FedRAMP) Developer-focused, browser-only, fragmented across 3 products
Relevance AI AI workforce for GTM teams. 2,000+ integrations. Multi-agent orchestration. SOC 2 Type II, RBAC, SSO. $37M funded. Not general-purpose, no CU, business automation only
Induced AI Browser automation from demos. "RPA 3.0". Sam Altman invested. Learns from single demo. 5 employees, beta, no enterprise features
MultiOn Browser agent API. Chrome extension. Developer-friendly, parallel agents, proxy support 5 employees, beta, limited docs
Apple Intelligence Siri overhaul with Gemini. On-device + cloud. Conversational memory (planned). 2B+ device installed base, on-device privacy, free Nothing shipped yet as of March 2026. Keeps slipping.

3.4 Tier 3: Infrastructure Plays (Potential Partners)

Player What They Offer How It Helps Vitto
E2B Firecracker microVM sandbox (<200ms restore) Already integrated
Fly.io Sprites Firecracker + scale-to-zero billing (<1s restore) Alternative to self-hosted orchestrator, cheaper
Blaxel Firecracker, <25ms restore (fastest reported) Lowest-latency sandbox option
Browserbase Managed browser infra, persistent Contexts API (cookie/state persistence) Solves login persistence without VM snapshots
Composio OAuth delegation for 500+ services. Agent never sees tokens. Solves credential management
Nango Open-source, self-hostable OAuth/token management for 700+ APIs Self-hosted alternative to Composio
1Password XAM Agentic AI Credentials Broker. Zero-knowledge. Enterprise-grade credential management
Mem0 Hybrid vector + graph memory (26% accuracy boost, 91% lower latency) Could upgrade Vitto's memory layer
Opik (Comet) LLM observability (traces, LLM-as-judge, hallucination detection) Solves our eval gap
Browser Use Open-source browser agent framework (78K stars, 89.1% WebVoyager) Could augment our browser automation

3.5 Master Comparison Table

Dimension Vitto OpenClaw ChatGPT Agent Google Mariner MS Copilot Simular Perplexity
CU Vision Loop Full (Anthropic) Feature req. CUA model Gemini 2.0 Windows 365 nut-js native Cloud browser
Desktop Control VM (xdotool) + Tauri (local) Skills (nut-js) No No Yes (W365 VM) Yes (OS-level) Yes (Mac mini)
Multi-tenancy Full (user+space) None Per-user Per-user Enterprise None Per-user
Learning Pipeline Intelligence Pyramid None None None Work IQ None None
LLM Routing Smart (5 providers) Manual (20+) Proprietary Gemini only Multi-model Unknown Proprietary
Channels 5 50+ Web only Chrome M365 apps None Comet browser
Voice None (deferred) Full duplex No Astra (research) Yes No No
Eval Framework None Opik (3rd party) Internal Internal Internal None Internal
Memory Tiered vector + genome SQLite + Markdown Session only Session only Work IQ Unknown Workflow
Code Gen 6-phase + live preview None None None None None None
Pricing SaaS (TBD) Free (MIT) $20-200/mo $250/mo $30/user/mo Unknown $200/mo + $599 hw
Security 6-layer depth CVE-2026-25253 Cloud isolated Cloud isolated Azure-grade Unknown Audit + kill switch
Stars/Users Closed source 302K stars 100M+ ChatGPT Billions (Chrome) 400M+ M365 Early Growing

4. Desktop Control — Detailed Comparison

Vitto vs. OpenClaw — Desktop Control Scorecard

Capability Vitto OpenClaw Winner
Computer Use vision loop Full Anthropic CU (75 iter, supervisor) Feature request Vitto
VM sandbox isolation Firecracker microVMs (KVM) Docker (optional) Vitto
VM snapshots/persistence Full lifecycle (suspend → restore) File sync only Vitto
VNC live streaming Yes (/vnc/:sessionId) No Vitto
Stuck detection / supervisor Every 10s, auto-corrective No Vitto
Service profiles (8) Predefined UI knowledge None Vitto
Security / audit trail Defense-in-depth (6 layers) Default-open Vitto
Shell execution (local) Bridge exists but not shipped as daemon Native (IS the local process) OpenClaw
Browser login state reuse No (fresh every session) Extension Relay OpenClaw
Voice → desktop actions None Full-duplex Talk Mode OpenClaw
Accessibility APIs None Clawd Cursor (nut-js) OpenClaw
Desktop GUI control (local) robotjs (server only, useless in prod) nut-js + AppleScript OpenClaw
PTY / interactive terminals No Yes OpenClaw
OCR / template matching No Community skills OpenClaw
macOS native integration Tauri app exists but not shipped Menu bar app + AppleScript OpenClaw

Vitto vs. Industry — CU Benchmarks (Where Available)

Agent WebVoyager OSWorld WebArena
Google Mariner 83.5% — —
Browser Use (OSS) 89.1% — —
Amazon Nova Act — — 90% (UI workflows)
ChatGPT Agent (CUA) 87% 38.1% 58.1%
Vitto Not benchmarked Not benchmarked Not benchmarked

Action needed: Benchmark Vitto's CU loop against WebVoyager and OSWorld to establish competitive position.


5. Where Vitto Has No Competition

These capabilities are genuinely unique in the market as of March 2026:

5.1 Intelligence Pyramid — Continuous Outcome-Based Learning

No competitor learns from its own outcomes. ChatGPT, Mariner, Copilot, OpenClaw — they all have static memory (store facts, recall facts). Vitto mines decisions into patterns, promotes patterns into rules, and injects distilled intelligence into every future interaction. This is compound intelligence.

Metric: 12K+ decisions, 563 learned patterns, auto-promotion at ≥3 uses + ≥80% success rate.

5.2 Smart Model Routing with Failover

No competitor dynamically routes by task complexity AND has circuit-breaker failover across 5 providers. OpenClaw supports 20+ providers but with manual selection. OpenAI/Google/Microsoft use their own models exclusively.

Metric: 60-90% cost savings on simple tasks. Zero downtime during single-provider outages.

5.3 User Dossier + Job Mastery Tracking

Microsoft's Work IQ is the closest, but it's M365-scoped. Vitto builds a comprehensive user intelligence profile (expertise, patterns, avoidances, job competency progression) that crosses all channels and tasks.

5.4 Conversation Genome (10-30x Token Compression)

Google's Always-On Memory Agent uses simple SQLite. Mem0 uses vector+graph. Nobody uses LLM-driven structured extraction (topics, decisions, facts, artifacts) with tiered compression and automatic archival.

5.5 6-Phase Code Generation + Live Preview

Devin operates in a sandbox with no live preview. GitHub Copilot generates snippets, not full modules. Vitto's pipeline (ENHANCE → ANALYZE → DECOMPOSE → PLAN → EXECUTE → INTEGRATE) with in-browser rspack WASM compilation and ESM SDK is unmatched.

5.6 Multi-Channel + CU + Learning in One System

This is the key differentiator. No single competitor combines:


6. Critical Gaps & Technical Debt

6.1 Critical Issues (High Confidence, Real Impact)

GAP-01: Embedding Search Is Broken [Confidence: Certain]

LearningService.isEmbeddingEnabled() in backend-ai-api returns false because setEmbeddingClient() is never called. Embeddings are generated and stored (via DecisionService.generateAndStoreEmbedding()) but never queried semantically. The entire "find similar decisions" falls back to keyword search.

Impact: The Intelligence Pyramid's recall quality is fundamentally degraded. Like building a search engine with only exact string match. Fix: Wire setEmbeddingClient() in module initialization. ~0.5 session.

GAP-02: No Eval Framework [Confidence: Certain]

No golden datasets. No LLM-as-judge. No A/B testing. No regression testing on prompt changes. No quality metrics over time. The Intelligence Pyramid mines patterns from outcomes, but nobody evaluates whether outcomes themselves are correct.

Impact: Every change to prompts, routing, or memory is a coin flip. We can't prove quality is improving. Fix: 50 golden Q&A pairs + nightly Haiku LLM-as-judge scoring + MongoDB storage + regression alerts. ~2 sessions.

GAP-03: Chat Proxy Polling [Confidence: Certain, Severity Revised: Low]

POST /vitto-api/engine/chat uses a 200ms polling loop for up to 60s. However, verification revealed the web UI already uses SSE via POST /engine/message → GET /engine/message/:taskId/stream. The polling loop only affects backend-ai-api's ChatService.proxyToVitto() (feature-flagged via CHAT_USE_VITTO_PROXY=true).

Impact: Low — web UI is unaffected. Only relevant when the ai-api proxy path is enabled. Fix: Replace while loop with lastValueFrom subscription on the existing SSE stream subject. ~0.5 session.

GAP-04: Cost Tracker Shows $0.00 [Confidence: Certain, Root Cause Verified]

Five stacked bugs make cost tracking non-functional (verified 2026-03-17):

  1. vitto.controller.ts:356-360 — Controller returns usage without a cost field. Frontend reads response.usage?.cost which is always undefined.
  2. cost-tracker.service.ts:104-123 — onModuleInit() loads userBudgets from MongoDB but NOT spendingHistory. After restart, getSpendingStats() returns empty.
  3. vitto-engine.service.ts:191-198 — task.cost.provider/model set from static config at creation, never updated after SmartRouter routing. Wrong model in pricing lookup on failover.
  4. phase-executor.service.ts:394-430 — handleTeachIntent() hardcodes tokens: {input:0, output:0} and never calls recordSpending(). All skill generation tasks untracked.
  5. cost-tracker.service.ts:250-310 — getSpendingStats() reads only in-memory array. Fixed by bug 2's rehydration.

Impact: Frontend always shows $0.00. Daily digest shows $0.00 after restart. Dollar-based budgets are theater. Fix: 5 targeted fixes across 4 files. See spec 2026-03-17-vitto-jarvis-evolution-spec.md §1.2. ~1 session.

GAP-05: Local Bridge Not Shipped as Daemon [Confidence: Certain]

Tauri app and Node.js bridge both exist. Neither has launchd/systemd daemon registration. Neither has a heartbeat loop. Angular bridge dies after 10 reconnection attempts.

Impact: The "Jarvis on your machine" story is architecturally ready but not deployable. Fix (verified simpler than expected): Daemon mode is a 1-line config change ("visible": true → false in tauri.conf.json). Close-to-tray and exit prevention already work. AutoStart via LaunchAgent already registered. Needs: infinite reconnection in Angular bridge (change reconnectionAttempts: 10 → Infinity in vitto-screen-capture.service.ts:74), plus heartbeat service. ~1-2 sessions.

GAP-06: No Browser Session Persistence [Confidence: Certain]

Every CU session launches a fresh browser. Users must re-authenticate to every service every time.

Impact: Massive UX friction for any CU task on authenticated services. OpenClaw's Extension Relay and Browserbase Contexts API solve this. Fix: Use VM snapshot to preserve browser profile (Chrome user data directory on persistent rootfs). ~1-2 sessions.

GAP-07: No Voice Pipeline [Confidence: Certain]

Voice note upload exists (S216 Gap 9). Whisper transcription deferred (S216 Gap 10). No STT → agent pipeline. No TTS response. No wake word.

Impact: Voice-to-action is a key "Jarvis" UX differentiator. OpenClaw, Rabbit, and Apple Intelligence all have voice. Fix: Whisper API (STT) + ElevenLabs/browser SpeechSynthesis (TTS) + Tauri push-to-talk. ~2 sessions.

6.2 Non-Critical Improvements (Pragmatic, High Value)

GAP-08: No OpenTelemetry [Confidence: Certain]

No distributed tracing across backends. Custom JSON health endpoint, not Prometheus format. Sandbox orchestrator already has Prometheus — backends don't.

Fix: Add OTEL spans on LLM calls, tool executions, memory recalls. Export to Grafana/Jaeger. ~2 sessions.

GAP-09: No Browser Extension Relay [Confidence: Certain]

Can't control existing Chrome tabs with preserved login state. Always requires fresh isolated browser.

Fix: Build Chrome extension (Manifest V3) connecting to LocalBridgeGateway. Attaches Chrome Debugger API to tabs. ~3 sessions.

GAP-10: No Credential Management [Confidence: Certain]

No OAuth delegation, no password manager integration, no human-takeover UX for password entry.

Industry standard: Composio (500+ services), Nango (700+ APIs, self-hostable), 1Password XAM. Fix: Integrate Composio or Nango for OAuth. Add VNC-based human takeover for password entry. ~3 sessions.

GAP-11: No Action Memory for CU [Confidence: Certain]

CU sessions are fire-and-forget. ComputerUseAuditService keeps an in-memory log (10K entries, no DB). No "last time I logged into Gmail, I clicked X then Y" memory.

Fix: Wire CU completion → VittoMemoryService.remember() with type task_execution. ~1 session.

GAP-12: VittoController God Object [Confidence: Certain]

35+ constructor dependencies. Despite refactoring claims in S161-162, still a monolith.

Fix: Split into domain controllers (chat, tasks, monitoring, admin). ~2 sessions.

GAP-13: Health Check MongoDB Stub [Confidence: Certain]

GET /health/components MongoDB check in backend-ai-api is a stub — no actual ping. latencyMs always ~0.

Fix: Add db.admin().ping() call. ~10 minutes.

GAP-14: Only 5 Messaging Channels [Confidence: Certain]

OpenClaw has 50+. We have WhatsApp, Slack, Discord, Telegram, Email. Missing: Teams, Signal, iMessage, Matrix.

Fix: Teams is the priority for enterprise. @libs-api/messaging-core adapter pattern makes each channel ~1 session.

6.3 Nitpicks


7. Architecture Patterns — Industry State of the Art (2026)

7.1 Persistent Browser Sessions

Pattern Used By How It Works
Ephemeral in-memory OpenAI (OWL) Chromium StoragePartition, discarded on session end
Extension Relay OpenClaw Chrome extension relays CDP to agent, inherits user's cookies/sessions
Persistent Contexts API Browserbase Cookies/state persist across sessions, agent stays logged in
VM Snapshot Manus (E2B) Firecracker sleep/wake, files + browser state survive

Recommendation for Vitto: VM snapshot approach (already have Firecracker infra) + Browserbase Contexts API for lighter-weight persistence.

7.2 Credential Management

Tier Pattern Used By
1 Human-in-the-loop takeover OpenAI Operator
2 OAuth delegation (agent never sees tokens) Composio, Nango, Arcade
3 Password manager bridge 1Password XAM, ChatGPT Atlas

Recommendation for Vitto: Tier 2 (Composio/Nango) for API services + Tier 1 (VNC human takeover) for browser-only services.

7.3 Always-On Agent Architecture

Pattern Used By How It Works
Heartbeat daemon OpenClaw 30-min wake cycle, launchd/systemd, event-driven
Cloud worker Lindy, Relevance Trigger-based (webhook, schedule, email)
Dedicated hardware Perplexity Computer Mac mini running 24/7
Sleep/wake VM Manus E2B Firecracker, hibernate when inactive

Recommendation for Vitto: Heartbeat daemon (Tauri app with launchd) + cloud-side proactive monitoring (ProactiveModule already exists).

7.4 VM Snapshot Technology

Platform Technology Restore Time Persistence
E2B Firecracker <200ms 14-30 days
Fly.io Sprites Firecracker <1s Indefinite
Blaxel Firecracker <25ms Indefinite
Modal gVisor Fast Per-snapshot
Vitto (self-hosted) Firecracker ~500ms (SSH overhead) 7 days

7.5 Memory — Industry Convergence

Two camps:

  1. Vector + Graph hybrid (Mem0, Zep) — 26% accuracy boost, 91% lower latency. Production standard.
  2. LLM-driven over simple storage (Google's SQLite, Manus filesystem) — simpler to operate.

Vitto's position: Closer to camp 1 (vector embeddings in MongoDB) but without graph layer. Adding knowledge graph edges (Vitto already has KnowledgeGraphService) could match Mem0.

7.6 CDP Is Universal

Every browser agent uses Chrome DevTools Protocol: OpenClaw, Browserbase/Stagehand, Chrome DevTools MCP, Nanobrowser. The protocol question is settled.

7.7 Auth Delegation Is Mandatory

Raw credential handling by agents is a non-starter for production. The pattern: OAuth flow handled by platform → tokens in encrypted vault → agent gets reference ID → platform executes on agent's behalf.

Emerging standard: Cross App Access (XAA) — extends OAuth, moves consent to Identity Provider.


8. Strategic Differentiators & Business Moats

8.1 Compound Intelligence (Deepest Moat)

Every Vitto interaction makes future interactions better. This is a network effect at the individual user level and a data advantage at the platform level. No competitor has this.

How to deepen: Close the outcome loop (fix embedding search → improve recall → better pattern mining → stronger core rules → better decisions → better outcomes → repeat).

8.2 Cost Efficiency via Smart Routing

60-90% savings on simple tasks by routing to cheap models. As LLM costs decrease, this advantage compounds — we're always using the cheapest model that's good enough. Competitors using single-model approaches (OpenAI, Google) pay full price for every query.

8.3 Multi-Channel Convergence

User sends a WhatsApp message → Vitto responds with awareness of their web chat history, their CU session on Gmail, and their space context. No competitor bridges channels with shared intelligence.

How to deepen: Cross-channel genome — unified memory that spans WhatsApp + web + desktop interactions for the same user.

8.4 Code Generation Pipeline

Platform generation (NL → full Angular application with CRUD, routing, theming) is a category of its own. No competitor generates and deploys full platforms. Devin writes code in a sandbox; Vitto builds businesses.

8.5 Proactive Intelligence

7 cron-driven monitoring services + space intelligence + anomaly detection + daily digests. Add heartbeat daemon + user-facing proactive actions, and Vitto becomes an always-aware assistant that surfaces insights before the user asks.

8.6 Business Model Advantage

OpenClaw is free (MIT). ChatGPT Agent is $20/mo. Mariner is $250/mo. Vitto can position as:


9. Phased Roadmap to Jarvis-Parity

Phase 1: Ship What Exists (1-2 weeks, 3-4 sessions)

Task Impact Effort Fixes
Wire embedding search (setEmbeddingClient()) Critical — unlocks semantic recall 0.5 session GAP-01
SSE chat responses (replace polling) Critical — UX + perf 1 session GAP-03
Fix cost tracker $0.00 bug High — enables dollar budgets 0.5 session GAP-04
Health check MongoDB ping Low — but stops lying 10 min GAP-13
Harden Tauri app with daemon mode Critical — enables local presence 2 sessions GAP-05

Phase 2: Persistent Desktop Agent (2-3 weeks, 5-6 sessions)

Task Impact Effort Fixes
VM snapshot auth persistence (browser profiles) Critical — login state survives 1-2 sessions GAP-06
CU action memory (loop → VittoMemory) High — "I did this before" 1 session GAP-11
OAuth credential delegation (Composio/Nango) High — autonomous service access 2 sessions GAP-10
Human takeover UX (VNC stream for passwords) Medium — fallback for non-OAuth 1 session GAP-10
Proactive heartbeat (Tauri daemon "anything to do?") High — always-aware 1 session GAP-05

Phase 3: Jarvis Parity (4-6 weeks, 8-10 sessions)

Task Impact Effort Fixes
Voice pipeline (STT + TTS in Tauri) High — voice-to-action 2 sessions GAP-07
Browser extension relay (existing Chrome tabs) High — login state without VMs 3 sessions GAP-09
OpenTelemetry integration (both backends) High — operational visibility 2 sessions GAP-08
Eval framework (golden Q&A + LLM-as-judge) Critical — quality regression detection 2 sessions GAP-02
Microsoft Teams channel Medium — enterprise requirement 1 session GAP-14

Phase 4: Moat Deepening (Ongoing)

Task Impact Effort
Closed-loop skill learning (extraction → composition → execution → feedback) Strategic — compound intelligence 2 sessions
Persona adaptation loop (Dossier observations → behavioral change) Strategic — "it gets me" moment 1 session
Platform intelligence aggregation (cross-user, anonymized) Strategic — network effects 2 sessions
Accessibility API integration (find elements by name) Medium — CU reliability 1-2 sessions
Benchmark on WebVoyager/OSWorld Medium — competitive positioning 1 session
Cross-channel genome (unified memory across all channels) Strategic — convergence moat 2 sessions
VWU outcome-based pricing model Critical — unit economics 1 spec + 3 impl

10. Key Decisions & Open Questions

Decisions Made (This Analysis)

Decision Chosen Reasoning Confidence
Architecture strategy for Jarvis Three-pillar: ship existing Tauri + enhance remote CU + browser relay Tauri app already exists, Firecracker infra is production-grade, browser relay is the missing bridge 0.88
Credential management approach Composio/Nango for OAuth + VNC human takeover for passwords Industry standard (2026). Agent never sees raw tokens. VNC takeover for edge cases. 0.85
Memory upgrade path Fix embedding search first, then add knowledge graph edges Embedding search is broken but infrastructure exists. Graph layer can come later via existing KnowledgeGraphService. 0.90
Eval framework approach Golden Q&A + Haiku LLM-as-judge + regression alerts Minimum viable eval. No need for A/B testing or full OTEL initially. 0.85

Open Questions (Need Verification)

Question Why It Matters How to Verify
Is Tauri app buildable today? S191.3 was last touch — may have stale dependencies Run cd apps/vitto-desktop && cargo build
What's the actual cost tracker bug? $0.00 affects all dollar-based budgets Trace CostTrackerService data flow, check model pricing map
Can Firecracker snapshot preserve Chrome profile? Browser profile on rootfs vs tmpfs determines persistence Test: snapshot VM with Chrome logged into Gmail, restore, check session
Is KnowledgeGraphService production-ready? Graph memory could match Mem0 Review service for completeness, check if indexes exist
How many concurrent CU sessions can we handle? Config says 50 global but untested at scale Load test with 10+ simultaneous sessions
Is SkillCompositionService actually executing? S216 Gap 2 "fixed" but no verification in production Write a test skill with dependencies, verify execution
What's the reconnection behavior of VittoScreenCaptureService after 10 failed attempts? Angular bridge goes silent permanently Verify in production with backend restart

11. Reference: Essential Files

backend-vitto-api (Orchestration Brain)

File Why It Matters
APIs/backend-vitto-api/src/app/app.module.ts Full module inventory
APIs/backend-vitto-api/src/app/engine/vitto-engine.service.ts Core engine
APIs/backend-vitto-api/src/app/engine/phase-executor.service.ts 4-phase workflow
APIs/backend-vitto-api/src/app/engine/agent-loop.service.ts Inner LLM ↔ tools loop
APIs/backend-vitto-api/src/app/llm/smart-router.service.ts Model routing
APIs/backend-vitto-api/src/app/llm/resilient.provider.ts Failover chain
APIs/backend-vitto-api/src/app/llm/provider-factory.service.ts Provider selection
APIs/backend-vitto-api/src/app/mind/vitto-memory.service.ts Tiered memory
APIs/backend-vitto-api/src/app/mind/mind.module.ts 20+ intelligence services
APIs/backend-vitto-api/src/app/tools/plugins/plugins.index.ts 17 tool plugins
APIs/backend-vitto-api/src/app/strategy/cost-guard.service.ts Cost controls
APIs/backend-vitto-api/src/app/security/prompt-injection-guard.service.ts 50+ regex patterns
APIs/backend-vitto-api/src/app/engine/local-bridge.gateway.ts WebSocket bridge protocol
APIs/backend-vitto-api/src/app/engine/computer-use-supervisor.service.ts CU stuck detection

backend-ai-api (Intelligence Layer)

File Why It Matters
APIs/backend-ai-api/src/app/app.module.ts 18 modules with session history
APIs/backend-ai-api/src/app/learning/learning.service.ts Intelligence Pyramid
APIs/backend-ai-api/src/app/learning/jobs/pattern-mining.job.ts 6h cron pattern extraction
APIs/backend-ai-api/src/app/infrastructure/services/decision.service.ts Decision + embedding creation
APIs/backend-ai-api/src/app/infrastructure/services/llm-resilience.service.ts Circuit breaker
APIs/backend-ai-api/src/app/infrastructure/services/computer-use-loop.service.ts Observe-think-act (1,200+ lines)
APIs/backend-ai-api/src/app/infrastructure/services/sandbox/sandbox-provider.factory.ts Provider selection
APIs/backend-ai-api/src/app/sessions/conversation-genome.service.ts Genome compile/hydrate
APIs/backend-ai-api/src/app/chat/chat.service.ts Chat pipeline + memory extraction

Sandbox Orchestrator

File Why It Matters
APIs/sandbox-orchestrator/src/app/sandbox/firecracker.service.ts VM lifecycle
APIs/sandbox-orchestrator/src/app/sandbox/vm-executor.service.ts SSH-based control
APIs/sandbox-orchestrator/src/app/sandbox/sandbox-manager.service.ts Orchestrator facade
APIs/sandbox-orchestrator/src/app/streaming/vnc-proxy.service.ts Live VNC streaming

Desktop Client

File Why It Matters
apps/vitto-local-bridge/index.js Node.js CLI bridge (S139)
apps/vitto-desktop/src-tauri/src/lib.rs Tauri app shell
apps/vitto-desktop/src-tauri/src/local_exec.rs Native command implementations
libs/libs-ai/vitto-core/src/lib/services/vitto-screen-capture.service.ts Angular bridge client

Architecture Documentation

File Why It Matters
docs/english/GoGrab-Space/architecture/backend-vitto-api-architecture.md Vitto architecture source of truth
docs/english/GoGrab-Space/architecture/vitto-memory-architecture.md Memory pipeline design
docs/english/Plans/march-26-plans/GOGRAB-VITTO-COMPREHENSIVE-MASTER-PLAN.md Master plan (BUILT vs PLANNED)
docs/superpowers/specs/2026-03-13-s216-vitto-jarvis-wiring-design.md 10 gap fixes

Last updated: 2026-03-17, Session S225.4 Next review: After Phase 1 completion Owner: Engineering (fguijarro)