Session 150 — Infrastructure, Research & Vision Report

Date: February 27, 2026 Author: Claude (Session 150, Coding Agent) Scope: Self-hosted sandbox infrastructure decisions, cost analysis, vendor comparison, and Vitto "Jarvis" memory architecture analysis

PART 1: Infrastructure Research & Decisions

1.1 Why We're Self-Hosting

E2B Cloud has hard limits that block our product vision:

Break-even: ~8-10 active users. After that, self-hosted saves money every month AND enables capabilities E2B cannot offer at any price.

1.2 What Was Built (Session 150)

32 new files, 1 modified file. All builds pass clean.

Sandbox Orchestrator NestJS App (APIs/sandbox-orchestrator/)

A NestJS API that manages Firecracker microVM lifecycle on bare-metal hardware. This is the "other end" of the SelfHostedProvider HTTP client that was built in Session 144.

VM Rootfs Image (deployment/sandbox-orchestrator/)

Server Scripts (tools/scripts/sandbox/)

Enhanced Existing File

E2B SDK Method Mapping

1.3 VM Model: Per-User Persistent (Recommended) vs Per-Session vs Dual-Layer

History

• Session 139: Per-user (Map<userId, UserSandbox>) — one VM per user
• Session 140: Changed to per-session (Map<sessionId, SessionSandbox>) to solve parallel task conflicts (two tasks fighting over same mouse/focus)
• Session 150: Recommendation to evolve to per-user persistent with Firecracker snapshots for the Jarvis vision

Comparison

Firecracker Snapshot Performance

• VMM restore: ~4-10ms (per Marc Brooker / AWS)
• Full system with app warmup: ~300-500ms
• Snapshot file size: Only dirty pages stored, not full 2GB. Typical ~500MB-1.5GB.
• Memory sharing: Clones share unchanged pages (MAP_PRIVATE demand paging)
• This is how AWS Lambda achieves sub-100ms cold starts.

Recommendation

Start with Per-User Persistent (Model B) on AX42. Tasks queue sequentially on the Home Machine. Add ephemeral Task VM forking (Model C) when parallel execution demand is proven.

1.4 SSH Performance: ControlMaster

What It Does

Normal SSH: every command opens new TCP → TLS handshake → authenticate → execute → close (~50-100ms). ControlMaster: first command opens normally. Every subsequent command reuses the same TCP connection (~2-5ms).

Impact

Cost & Effort

• Monetary cost: Zero
• Implementation: ~30 minutes — add 3 lines to SSH_BASE_ARGS in vm-executor.service.ts
• Risk: Near-zero. ControlMaster socket cleanup on VM destroy.

Recommendation

Implement immediately. Pure upside, no downside.

1.5 Managed Cloud vs Self-Hosted Bare Metal

Can We Use AWS or Google Cloud Instead?

Firecracker needs bare-metal access to /dev/kvm. Regular cloud VMs don't expose this.

AWS i3.metal costs 69x more than Hetzner AX42 for capacity we won't use (488GB when we need 64GB). GCP nested virtualization adds latency and costs 6-8x more.

When Cloud Makes Sense

Only if you need sandbox servers in 10+ global regions simultaneously for latency. For US/EU users, Hetzner Ashburn (VA) and Falkenstein (DE) cover everything.

1.6 Hetzner vs Other Bare Metal Providers

Provider Assessment

Hetzner — Price leader. €39-49 for 64GB NVMe bare metal. Zero setup fees. Excellent docs and rescue system. Limited to 5 locations. 2026 price increases of 3-10% effective April 1.

OVH — 30+ locations worldwide. Unlimited bandwidth. Strong DDoS protection. 2-2.5x more expensive than Hetzner. Mixed support reputation since 2021 Strasbourg fire.

Contabo — 6 regions including Asia-Pacific. Unlimited bandwidth. Network quality noticeably worse than Hetzner/OVH. Slow support. VDS products may not expose KVM.

Hetzner Model Decision Tree

Note: AX42/AX52/AX102 had a motherboard defect in Q1-Q3 2024 production units. Hetzner is replacing affected boards. Orders placed Feb 2026+ get post-fix hardware. The EX44 (Intel) was unaffected.

Migration (AX42 → AX102)

Trivial. Run provision-server.sh on new server, rsync rootfs + snapshots, update DNS, cancel old server. ~30 minutes total, ~5 minutes downtime. Zero code changes needed.

1.7 Mac as Sandbox Server

Why Mac Doesn't Work for This

Firecracker does not run on macOS. It requires Linux KVM. Mac uses Apple's Hypervisor.framework (UTM/QEMU) — different API, different snapshot model, different networking. Using a Mac would require rewriting the entire orchestrator.

Mac Hosting Costs (For Reference)

At $360-399/month for 64GB, a hosted Mac is 7-8x more expensive than Hetzner AX42 ($53) with no advantage for our use case.

When Mac Makes Sense (Future)

If Vitto needs to control macOS-specific apps (Xcode, Keynote, Final Cut) or automate iOS development workflows. That's a genuine future opportunity but not v1.

1.8 Final Infrastructure Decision

PART 2: Can Vitto Be "Jarvis"? — Honest Analysis

2.1 The Vision (Restated for Clarity)

Vitto is a 24/7 AI assistant that:

1. Knows the user — preferences, history, work style, credentials
2. Knows the space — projects, code, data, team members, goals
3. Remembers everything — conversations from 6 months ago, that one time you changed the API key, the fact you prefer dark mode
4. Acts in the real world — browses the web, logs into Gmail, files an LLC, deploys code
5. Gets smarter over time — learns from outcomes, adapts to the user, shares wisdom across the platform
6. Works 24/7 — not per-session, not per-conversation. Vitto is always "on"

Is This Possible?

Yes, with caveats. Let me be precise about what works, what's hard, and what's unsolved.

2.2 What We Already Have (Honestly Assessed)

After exploring the entire codebase, Vitto already has a surprisingly robust memory architecture:

The Three-Tier Memory System

TIER 1: HOT (In-Memory)
├── 50 most recent memories per user
├── Keyword matching, instant retrieval
└── 30-min TTL, evicted on overflow

TIER 2: WARM (MongoDB + Vector Search)
├── vitto-memories: vector-embedded (1536-dim, OpenAI text-embedding-3-small)
├── execution-traces: full task history (90-day TTL)
├── strategy-memories: what approach works best per task type
├── $vectorSearch with cosine similarity, threshold 0.7
└── Text search fallback when vectors unavailable

TIER 3: COLD (Platform Intelligence)
├── golden_decisions: cross-platform decision history
├── core_rules: 10+ distilled, always-apply rules
├── learned_patterns: task-specific patterns with success rates
└── Retrieved via API from backend-ai-api

What Gets Stored Today

What Gets Injected into Every LLM Call

1. User identity (name, email, roles, subscription) — ~100 tokens
2. User memories (preferences, skills, facts) — ~500 tokens
3. Core rules (always-apply patterns) — ~500 tokens
4. Relevant patterns (task-specific) — ~300 tokens
5. Similar past decisions — ~300 tokens
6. Knowledge distribution — variable

Total context overhead: ~2,000-3,000 tokens per call.

Honest Grade: B+

The foundation is solid. Vector search for semantic recall, tiered storage, automatic memory extraction, pattern promotion. But there are real gaps.

2.3 The Hard Problems (Where Jarvis Breaks)

Problem 1: Context Window Is Finite

The fundamental constraint. Claude Opus has a 200K token context window. That sounds large, but:

• A 30-message conversation history: ~5,000-10,000 tokens
• System prompt + context injection: ~3,000 tokens
• User's current request + response: ~2,000 tokens
• Remaining for "memory": ~185,000 tokens

185K tokens is a lot — roughly 300 pages of text. But a user who's worked with Vitto daily for 6 months has generated:

• ~1,000 conversations
• ~500 task executions
• ~5,000 memories
• ~50,000 lines of context

You cannot fit all of this into one prompt. This is the core challenge everyone in the industry faces.

What Competitors Do (And Their Limits)

None of these fully solve the problem. They all trade off between recall accuracy and token efficiency.

Problem 2: Semantic Search Misses Tangential Relevance

Vector search is great for "find memories about TypeScript preferences." It's bad for "I'm building an e-commerce checkout — what did I say 3 months ago about payment processing?"

The second query requires understanding that "checkout" relates to "payment processing" which relates to a conversation about Stripe where you mentioned preferring their newer API. The semantic connection is 3 hops deep. Current vector search catches maybe 1 hop.

Problem 3: Memory Staleness

Vitto remembers you prefer React. Three months later, you switch to Svelte. The old "prefers React" memory is still in the database with high importance. Unless Vitto explicitly detects the contradiction and updates the memory, it will keep injecting stale context.

Our current system has no contradiction detection. Importance never decays. Old memories don't get challenged.

Problem 4: Cross-Space Context Leaking

If a user works on Space A (medical app) and Space B (gaming app), Vitto's memories are partially space-scoped (spaceId field) but many memories are user-scoped. Medical terminology could leak into gaming recommendations.

Problem 5: The "Always On" Illusion

LLMs are stateless. Every API call starts from zero. The "always on" feeling is an illusion built from:

1. Injecting memories into the system prompt (makes it seem like Vitto "remembers")
2. Resuming conversation history (makes it feel continuous)
3. Proactive actions (Vitto checks your inbox even when you're not asking)

The illusion breaks when the context window can't fit everything that matters.

2.4 What Would Actually Make Vitto Groundbreaking

Here's where I'll be inventive. Most competitors do RAG + conversation history. Here's what could genuinely differentiate Vitto:

Idea 1: The "Mind Map" — Structured Memory Graph (Not Just Vectors)

Instead of flat vector embeddings, build a knowledge graph per user where:

User → owns → Space "MyApp"
       │        ├── uses → Tech: Angular, NestJS, PostgreSQL
       │        ├── deployed_on → Railway
       │        └── has_feature → Auth (uses → JWT, preferred → Passport.js)
       │
       ├── prefers → TypeScript over JavaScript
       ├── dislikes → CSS-in-JS
       ├── works_at → peak hours 9am-1pm
       │
       └── did_task → "Set up Stripe checkout" (3 months ago)
                ├── used → Stripe API v2023-10-16
                ├── chose → embedded checkout over hosted
                └── lesson → "Webhooks need idempotency keys"

Why this is better than vectors alone: When Vitto is asked to "add payment processing to Space X," it can traverse the graph:

1. Look up Space X → find tech stack (Angular + NestJS)
2. Find related past task → "Set up Stripe checkout"
3. Pull the lesson → "Webhooks need idempotency keys"
4. Know the user's preference → embedded checkout over hosted

This is 3-hop reasoning that vector search cannot do. It requires structured relationships, not just semantic similarity.

Implementation: Use MongoDB's $graphLookup (already available in our stack) or a dedicated graph layer. Nodes are memories, edges are typed relationships. Build the graph incrementally as memories are created.

Estimated effort: 2-3 weeks for a basic graph with entity extraction + relationship building. Can use Claude itself to extract entities and relationships from conversations.

Idea 2: "Cognitive Load Budgeting" — Smart Context Assembly

Instead of stuffing everything into the context window, Vitto should plan what to remember for each request:

User asks: "Help me add Stripe to my checkout"

Step 1: CLASSIFY the request → payment integration, web development
Step 2: BUDGET the context → 3,000 tokens for memories, 2,000 for patterns
Step 3: RETRIEVE strategically:
   - User's payment preferences (200 tokens)
   - Space tech stack (100 tokens)
   - Past Stripe task summary (300 tokens)
   - Stripe patterns from platform knowledge (500 tokens)
   - Recent conversation context (1,000 tokens)
   - Core rules (500 tokens)
   - SKIP: user's CSS preferences (irrelevant)
   - SKIP: memories about their gaming project (wrong space)
Step 4: ASSEMBLE context with priority ordering (most relevant first)

Why this matters: The key insight from LLM research is that models show a U-shaped performance curve — they attend better to information at the beginning and end of the context, with a "lost in the middle" effect. Smart assembly means putting the most critical memories first.

We partially do this today (tiered recall, importance scoring). The gap is there's no request-aware budgeting — we inject the same user context regardless of whether the user is asking about TypeScript or asking about their weekend.

Idea 3: "Memory Metabolism" — Active Forgetting and Consolidation

Human brains don't store every detail — they consolidate, generalize, and forget. Vitto should do the same:

DAY 1: "User said they prefer tabs over spaces"     → Store as memory (importance 0.7)
DAY 7: Memory consolidation runs. This memory accessed 0 times. → Decay importance to 0.5
DAY 30: Still not accessed. → Decay to 0.3
DAY 90: → Archive (remove from warm tier, keep in cold)

vs.

DAY 1: "User's API key for Stripe is sk_live_xxx"   → Store (importance 0.9)
DAY 7: Accessed 3 times during Stripe integration    → Boost to 0.95
DAY 30: Accessed 1 more time                         → Stays 0.95
DAY 90: Not accessed in 60 days                      → Decay to 0.8 (still high — credentials don't expire)

We have consolidation (7-day conversation merging in MindService.runMemoryConsolidation()). But we don't have importance decay. Every memory stays at its initial importance forever. This means after 6 months, the hot cache is dominated by old high-importance memories that haven't been relevant in months.

Fix: Add an accessedAt timestamp, decay formula: effectiveImportance = baseImportance * (0.95 ^ daysSinceLastAccess). Simple exponential decay. Keeps frequently-accessed memories alive, lets unused ones fade.

Idea 4: "Contextual Persona" — Pre-Computed User Summaries

Instead of assembling context from raw memories on every call, maintain a living document per user that summarizes who they are:

USER BRIEF (auto-generated, refreshed daily)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Name: Paco
Active spaces: MyApp (Angular/NestJS), GameProject (Unity)
Working style: Morning person, peak 9am-1pm. Prefers detailed explanations.
Tech preferences: TypeScript, Angular, NestJS, PostgreSQL. Dislikes CSS-in-JS.
Recent focus: Stripe payment integration (past 2 weeks)
Open tasks: 3 pending code reviews, 1 deployment blocked on env vars
Credentials on file: Gmail, GitHub, Stripe (all stored in VM profile)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

This is ~200 tokens and gives Vitto 80% of the "I know you" feeling. It's regenerated daily (or on significant events) by summarizing the full memory store. The LLM sees this brief + the most relevant specific memories, not a dump of 500 memories.

No one is doing this well. ChatGPT has "Memory" but it's just a flat list of facts. Letta/MemGPT self-edits a "core memory" block but it's limited and can corrupt. A properly maintained User Brief that's regenerated from the full memory graph is more reliable and more comprehensive.

Idea 5: The VM as Memory Extension

This is the truly unique angle that no competitor has:

Vitto's persistent VM IS a form of memory. Chrome cookies are memory. Bookmarks are memory. Browsing history is memory. Saved passwords are memory.

When Vitto logs into Gmail on the user's VM, that login session persists. Next time Vitto needs to check email, there's no "remember the password" step — the browser remembers. This is physical-world memory that exists outside the LLM's context window entirely.

Traditional AI assistant:
  "Check my Gmail" → Look up credentials → Log in → Navigate → Read → Log out

Vitto with persistent VM:
  "Check my Gmail" → Open Chrome tab → Already logged in → Read

  Time: 30 seconds vs 5 seconds
  User friction: Zero vs "enter your 2FA code"

This is what makes the Hetzner + Firecracker + per-user persistent VM decision so important. It's not just about cost savings over E2B. It's about making the VM an extension of Vitto's memory — a physical, browser-based memory layer that doesn't consume context window tokens.

2.5 What's Actually Fine (Don't Over-Engineer)

Being honest about what doesn't need fixing:

1. Vector search for recall is good enough for 90% of queries. Don't build a full knowledge graph before validating that users actually have multi-hop recall needs.

2. 2,000-3,000 tokens of context overhead is acceptable. Claude's 200K window has plenty of room. Don't obsess over token optimization until you're genuinely hitting limits.

3. The Intelligence Pyramid (core rules → patterns → decisions) is a solid architecture. Pattern promotion based on success rate is a genuinely good idea that most competitors don't have.

4. Memory extraction from conversations works. The regex + LLM extraction pipeline catches explicit memories reliably. It misses some implicit ones, but that's acceptable.

5. The tiered recall (hot → warm → cold) with fallback is well-designed. The architecture handles the "I don't remember but let me check" case gracefully.

2.6 Honest Risk Assessment

2.7 Recommended Roadmap

Phase 1: Foundation (Now → 2 weeks)

• Deploy Hetzner AX42 with per-user persistent VMs
• Implement SSH ControlMaster
• Implement importance decay on memories
• Build User Brief (auto-generated daily summary per user)

Phase 2: Intelligence (2-4 weeks)

• Cognitive load budgeting (request-aware context assembly)
• Contradiction detection (new memory conflicts with old → prompt user)
• Proactive memory refresh (re-embed old memories with updated context)

Phase 3: Differentiation (1-2 months)

• Knowledge graph per user (entity extraction + relationship building)
• Multi-hop recall via graph traversal
• VM-as-memory integration (Chrome state as queryable context)
• Cross-space intelligence sharing (with user consent)

Phase 4: Scale (3+ months)

• Dual-layer VMs (Home + ephemeral task forks)
• Multi-server orchestration
• Team memory sharing (shared knowledge graph per space)
• Proactive task suggestions based on patterns

2.8 The Bottom Line

Can Vitto be Jarvis? Yes — with the right memory architecture.

The LLM is not the bottleneck. Claude is smart enough. The bottleneck is what context you give it. The difference between a mediocre AI assistant and Jarvis is not the model — it's the memory system that decides what the model knows about you when it starts thinking.

What we have today is a B+. The gaps are:

1. No importance decay (memories get stale)
2. No request-aware context budgeting (same context injected regardless of task)
3. No multi-hop reasoning (vector search is 1-hop only)
4. No User Brief (no pre-computed "who is this person" summary)
5. No VM-as-memory integration (Chrome state isn't queryable)

Fix those five things, and you have something no competitor offers: an AI that genuinely knows you, remembers your work, has a physical machine ready to act, and gets smarter every day.

The persistent VM on Hetzner is the missing physical layer. MCP + MongoDB + vector search is the digital layer. The knowledge graph would be the reasoning layer. Together, they give Vitto three forms of memory that reinforce each other — and that's architecturally unique in the market.

Sources

• Firecracker Snapshot Documentation
• Marc Brooker on Lambda SnapStart
• Seven Years of Firecracker
• SSH 10x Speedup with ControlMaster
• SSH Connection Multiplexing Guide
• Hetzner AX Server Matrix
• Hetzner AX102
• Hetzner 2026 Price Adjustments
• Achromatic Hetzner Comparison with Benchmarks
• AX42/52/102 Motherboard Issue
• AWS i3.metal Pricing
• GCP Nested Virtualization
• GCP Sole-Tenant Pricing
• OVH Dedicated Server Prices
• Contabo Dedicated Servers
• Scaleway Apple Silicon Pricing
• MacStadium Pricing
• Letta/MemGPT Agent Memory Architecture
• Design Patterns for Long-Term Memory in LLM Architectures
• LLM Context Window Management 2026
• MCP Knowledge Graph Memory Server
• Agent-Recall: SQLite Knowledge Graph for Claude
• Cognee: MCP + LLM Memory
• Top 10 AI Memory Products 2026