GoGrab Unified Master Plan: Infrastructure + Super Agent

Critical Merge of Handoff Plan (S191) + Research Innovations

March 10, 2026 — Senior Engineering Review

Preamble: Honest Assessment of Both Plans

Before merging, let me be blunt about what each plan gets right and wrong.

The Handoff Plan (Infrastructure) is strong. It's grounded in real code, real files, real failure modes. The circuit breaker extraction, retry standardization, and BullMQ alerting are all correct and necessary. The revised positions (keep databases separated, don't split MCP) show intellectual honesty — someone changed their mind when evidence disagreed. That's good engineering.

Weaknesses of the Handoff Plan: It's entirely defensive. Every phase is about not breaking — circuit breakers, retries, caching, monitoring. That's the right foundation, but it contains zero offensive capability. No new intelligence. No competitive differentiation. It's the engineering equivalent of building a fortress with no army inside.

The Research Plan (Super Agent) is ambitious and directionally correct. Semantic caching, model routing, hierarchical memory, and the self-improving skill system are all real innovations with proven implementations elsewhere.

Weaknesses of the Research Plan: It's optimistic. Some of the claimed numbers need scrutiny:

• "40-70% cost reduction from semantic caching" — This assumes high query similarity. Agent tasks are often unique. Real-world hit rates for agent workflows (vs chatbot Q&A) are likely 20-40%, not 60-85%. The cited studies measured customer support bots, not autonomous agents doing novel tasks. Still worth doing, but set expectations at 20-35% cost reduction.
• "40-60% token reduction from delta-only state passing" — Plausible IF sub-agents currently receive full context. Need to verify what Vitto actually sends today. If context is already minimal, the gain is smaller.
• "50+ self-generated skills per month" — Aspirational. Depends entirely on task diversity and quality thresholds. Don't set this as a KPI until we have 3 months of data.
• The TTT-E2E section is honestly filler — no API exists, no timeline, no actionable step. Cut it from the plan.
• A2A protocol adoption is premature. We have zero external agents to interoperate with. Build internal agent communication well; formalize it as A2A later if needed.

The real gap: Neither plan addresses the circular dependency between backend-ai-api and backend-vitto-api, which the Handoff Plan correctly identifies as critical but then only treats with retries and circuit breakers. That's a band-aid. The research plan's "Agent Runtime Layer" is the actual fix, but it's described too abstractly. Below I make it concrete.

Merged Architecture: What We're Actually Building

                    ┌──────────────────────────────────┐
                    │        CLOUDFLARE EDGE            │
                    │  vitto-gateway (webhooks)         │
                    │  vitto-pulse (health monitor)     │
                    └──────────┬───────────────────────┘
                               │
                    ┌──────────▼───────────────────────┐
                    │     backend-vitto-api (3003)      │
                    │                                    │
                    │  ┌─────────────────────────────┐  │
                    │  │   AGENT RUNTIME LAYER (NEW)  │  │
                    │  │  · SessionManager            │  │
                    │  │  · StateGraph (DAG engine)   │  │
                    │  │  · ContextBroker             │  │
                    │  │  · ModelRouter               │  │
                    │  │  · SemanticCache             │  │
                    │  │  · SkillRegistry             │  │
                    │  └─────────────────────────────┘  │
                    │                                    │
                    │  ┌──────────┐  ┌───────────────┐  │
                    │  │ Engine   │  │ Mind/Memory    │  │
                    │  │ (exist.) │  │ (hierarchical) │  │
                    │  └──────────┘  └───────────────┘  │
                    │                                    │
                    │  ┌──────────────────────────────┐  │
                    │  │ RESILIENCE LAYER (Phase 1-2) │  │
                    │  │ · CircuitBreakers (shared)   │  │
                    │  │ · RetryWithBackoff (shared)  │  │
                    │  │ · DeadLetterQueue (shared)   │  │
                    │  └──────────────────────────────┘  │
                    └──┬──────────────┬─────────────────┘
                       │              │
            ┌──────────▼──┐   ┌──────▼─────────────┐
            │ backend-api │   │ backend-ai-api     │
            │ (3000)      │   │ (3001)             │
            │ Platform    │   │ LLM providers      │
            │ CRUD        │   │ Intelligence       │
            └─────────────┘   └────────────────────┘

Unified Phase Plan

I've reordered and merged the phases based on dependency analysis and actual risk. The numbering reflects execution order, not importance.

Phase 0: Baselines (Before Anything Else)

This is missing from both plans and it's a mistake.

You cannot claim "40% cost reduction" or "50% latency improvement" without a baseline. Before implementing anything:

1. Instrument current LLM costs. Add a simple middleware that logs every LLM API call with: model, input tokens, output tokens, latency, caller service, user ID, timestamp. Write to a llm_call_log collection. This takes 2 hours and gives you ground truth.

2. Measure current context sizes. Log what Vitto actually sends to sub-agents today. How many tokens? Is it already minimal or bloated? This determines whether delta-passing is a 5% or 50% win.

3. Measure cache-ability. Log query embeddings for 1 week. Then run offline similarity analysis to determine what percentage of queries would have been cache hits at various thresholds (0.90, 0.92, 0.95). This tells you the REAL semantic cache hit rate for YOUR workload, not someone's chatbot benchmark.

4. Map the actual circular dependency. Trace every HTTP call between backend-ai-api and backend-vitto-api for 24 hours. How often does the cycle actually fire? Is it every request or rare? This determines urgency.

Effort: 1 session. No excuses for skipping this.

Phase 1: Resilience Foundation (Week 1)

Directly from Handoff Plan Phases 1-2, unchanged. This is correct as written.

Why it stays first: nothing innovative matters if WhatsApp messages are being dropped. The Handoff Plan's analysis of the retry table is sobering — channel identity resolution at 0 retries and 0 backoff means a single backend-api hiccup loses a customer message. That's a production defect, not an enhancement request.

Tasks (from Handoff Plan — no changes needed):

1. Extract CircuitBreakerService to @libs-api/backend-core
2. Extract DeadLetterQueueService similarly
3. Add circuit breakers to backend-ai-api LLM calls (replicate Vitto's SmartRouter pattern)
4. Add circuit breakers to backend-api external calls (Stripe, SendGrid, Twilio, Cloudflare, GitLab)
5. Implement provider health endpoints on all 3 APIs
6. Add retries to all inter-service calls per the Handoff Plan's table
7. Add Redis caching for channel identity resolution (5min TTL)
8. Add Redis caching for user preferences (10min TTL)

Critical addition: While implementing retries, also add the instrumentation from Phase 0 to every call. Two birds, one code path.

Effort: 2-3 sessions | Risk: Low | Dependencies: None

Phase 2: Semantic Cache + Model Router (Week 1-2)

Merged from Research Plan 3.1 + 3.2, with critical adjustments

These two go together because they're both interceptors on the LLM call path. Implement them as a pipeline:

User Query → SemanticCache.check() → ModelRouter.classify() → LLM Provider → SemanticCache.store()

Semantic Cache — adjusted design:

The Research Plan correctly identifies this as low-effort/high-impact, but overstates expected gains. Adjustments:

• Use MongoDB Atlas Vector Search (we already have MongoDB clusters) rather than adding Redis vector indexing, which would require Redis Stack (a separate deployment from our standard Redis). Assumption to verify: check if our Railway Redis is Redis Stack or standard Redis. If standard, MongoDB vector search is the pragmatic choice.
• Start with EXACT match caching (hash of normalized prompt + model + temperature). This is free and catches the easy wins.
• Add semantic similarity as Phase 2b ONLY if Phase 0 baselines show >25% semantic similarity in real traffic.
• Cache TTL must account for user context. A cached response for "what's my order status?" is wrong for a different user. Cache key MUST include user_id or space_id.
• Do not cache agentic tool-calling responses. Only cache stateless Q&A-style generations. An agent deciding "which tool to call" should never be cached — the state is different every time.

Model Router — adjusted design:

The Research Plan proposes a classifier-based router. Simpler approach first:

• Rule-based routing by task type. Vitto already knows what task it's executing (chat, code generation, analysis, summarization). Map task types to model tiers statically. No classifier needed initially.
• Tier 1 (Frontier): Planning, complex reasoning, multi-step orchestration → Claude Opus / GPT-5
• Tier 2 (Standard): Chat, summarization, moderate analysis → Claude Sonnet / GPT-4.1
• Tier 3 (Fast): Translation, formatting, simple extraction → Claude Haiku / GPT-4.1 mini
• This already exists partially. The Handoff Plan notes that AI_MODEL_CAPABILITIES and SmartRouterService are centralized (S191). Build on that, don't reinvent.
• Add the classifier-based auto-routing ONLY after you have 1 month of per-task-type cost/quality data from Phase 0 instrumentation.

Effort: 2 sessions | Risk: Low-Medium | Dependencies: Phase 0 baselines

Phase 3: BullMQ Event-Driven Alerting (Week 1-2, parallel)

Directly from Handoff Plan Phase 4, with one addition

The Handoff Plan's design is correct. One addition from the Research Plan:

• Add agent-level metrics to the queue metrics endpoint. Not just BullMQ job counts, but: active agent sessions, tokens consumed in last hour, model distribution (what % of calls go to each tier), cache hit rate. This is the observability foundation for the Agent Runtime Layer that comes in Phase 5.

Effort: 1-2 sessions | Risk: Low | Dependencies: None (parallel with Phase 1)

Phase 4: AI Module Migration (Week 2-3)

Directly from Handoff Plan Phase 3, unchanged

This is correctly scoped. S191 did the hard part (centralizing pricing). What remains is plumbing: proxy controllers, storage unification, module removal.

One risk the Handoff Plan understates: The historical data migration (copying ai_usage and spending_records into llm_usage) is the riskiest task in this phase. If record schemas differ (timestamps, field names, embedded objects), the migration script becomes complex. Verify schema compatibility before estimating effort. If schemas diverge significantly, consider: just start fresh with unified tracking going forward, and keep historical data in its original location for lookback queries.

Effort: 2-3 sessions | Risk: Medium | Dependencies: None (parallel with Phases 1-3)

Phase 5: Agent Runtime Layer (Week 3-4)

NEW — This is the critical merge point between both plans

This is where the Research Plan's "Agent Runtime Layer" meets the Handoff Plan's real infrastructure. Neither plan alone gets this right. Here's what's actually needed:

5.1 SessionManager

Vitto's engine already manages agent sessions. The gap is: there's no formal lifecycle with checkpointing. When a long-running agent task fails halfway through (LLM timeout, Railway restart), the entire task is lost.

Build:

• AgentSession entity with states: CREATED → ACTIVE → PAUSED → COMPLETED | FAILED
• Redis-backed state persistence at each decision boundary (before and after every tool call)
• Resume capability: on failure, reload last checkpoint and retry from that point
• Timeout governance: max 5 minutes per agent turn, max 30 minutes per session, configurable per task type

This directly addresses the circular dependency problem: if backend-ai-api is slow, Vitto can PAUSE the session, checkpoint state, and retry later instead of cascading the failure.

5.2 ContextBroker (Delta-Passing)

This is the Research Plan's "graph-based state orchestration" made concrete:

• When Vitto delegates to a sub-agent, the ContextBroker assembles a minimal context payload:
  • Task description (what to do)
  • Relevant state delta (what changed since the sub-agent's last invocation)
  • User identity + preferences (from Redis cache, Phase 1)
  • Relevant skills (from SkillRegistry, Phase 6)
• It explicitly EXCLUDES: full conversation history, other sub-agents' outputs, raw tool call logs
• On sub-agent completion, the ContextBroker extracts the result delta and merges it into the parent session state

Implementation reality check: This is a significant refactor of how Vitto's engine dispatches work. Don't try to build a generic DAG engine. Start with a simple "parent → child → result" pattern. Add DAG routing later if/when you have 3+ sub-agent types that need to coordinate.

5.3 The Circular Dependency Fix

The Handoff Plan identifies the problem (backend-ai-api ↔ backend-vitto-api). The resilience layer (Phase 1) makes it survivable. The Agent Runtime Layer makes it architecturally correct:

• Vitto (3003) OWNS session orchestration. It decides what to do.
• Backend-ai-api (3001) OWNS LLM access. It executes what Vitto asks.
• The direction is ONE-WAY for any given request: Vitto → AI-API for LLM calls. AI-API → Vitto for task delegation.
• If AI-API needs to delegate a task to Vitto, it does so via BullMQ (already in place) not via synchronous HTTP. This breaks the circular sync dependency.

Effort: 3-4 sessions | Risk: Medium-High | Dependencies: Phase 1 (resilience), Phase 0 (baselines)

Phase 6: Hierarchical Memory + Skills (Week 4-5)

From Research Plan 2.1, 2.2, 4.1 — made concrete

6.1 Memory Tiers

The Research Plan's 4-tier memory model is the right target. But build incrementally:

Tier 1 — Working Memory (already exists): The LLM context window. No changes.

Tier 2 — Short-Term (the Focus pattern, build this): When a sub-agent completes, it calls completeFocus(summary) which generates a compressed summary of its work. This summary replaces the raw interaction history in the parent session's context. Implementation: a FocusService in Vitto's mind module that takes raw agent output, calls a fast model (Haiku) to summarize, and stores the summary in Redis with a session-scoped TTL.

Tier 3 — Contextual (cross-session, build this): User preferences, past interaction patterns, known preferences ("this user prefers concise answers", "this user's space is e-commerce"). Stored in MongoDB vitto-mind. Loaded at session start. Updated at session end. This is NOT full conversation recall — it's a structured profile that grows over time.

Tier 4 — Long-Term / Skills (build this): The Hermes Agent pattern. When Vitto solves a complex task successfully:

1. Detect success (user confirms, task completes without retry)
2. Generate a skill document: what was the task, what approach worked, what tools were used
3. Store in MongoDB with vector embedding for similarity search
4. On future similar tasks, retrieve and inject relevant skills into the sub-agent's context

Critical design decision: Skills must be scoped. A skill learned in Space A should not leak into Space B unless it's a platform-level skill. Multi-tenancy applies to memory too.

Effort: 3-4 sessions | Risk: Medium | Dependencies: Phase 5 (runtime layer)

Phase 7: Cloudflare Edge Layer (Week 4-5, parallel)

From Handoff Plan Phase 5, unchanged

The Handoff Plan's design for vitto-gateway and vitto-pulse is solid. The webhook acknowledgment-at-edge pattern is well-proven. No changes needed.

One consideration: The vitto-gateway Worker introduces a new failure mode — if the Worker acknowledges a webhook but fails to forward to Railway, the message is lost. The Handoff Plan says "use waitUntil() for async forwarding" but doesn't address the case where the Railway backend is completely down. Solution: if forwarding fails after 3 retries, write the payload to Cloudflare Durable Objects as a dead letter, with a cron job that retries forwarding every 5 minutes. This adds modest complexity but prevents message loss.

Effort: 3-4 sessions | Risk: Medium | Dependencies: Phase 3 (metrics endpoint)

Phase 8: MCP Gateway Enhancement (Week 5-6)

From Handoff Plan Phase 6, unchanged

The per-platform tool filtering is a clean win. The tier mapping table in the Handoff Plan is well thought out. No changes needed.

Effort: 2 sessions | Risk: Low | Dependencies: None

Phase 9: Self-Improvement Loop (Week 6-8)

From Research Plan 4.2 — the Karpathy autoresearch pattern

Honest assessment: This is the most exciting idea in the Research Plan and also the hardest to execute correctly. The risk is that Vitto auto-evolves its prompts into something that scores well on a proxy metric but degrades real user experience.

Safe implementation:

1. Do not auto-deploy prompt changes. Ever. The loop generates candidates; a human reviews before promotion.
2. Use A/B testing, not replace-and-measure. 10% of traffic gets the candidate prompt, 90% gets the current prompt. Compare over 48 hours minimum.
3. Metric must be multi-dimensional. Not just "task completion rate" (which can be gamed by the agent lowering task difficulty). Use: completion rate + user rating + token cost + latency. All four must be equal or better.
4. Start with a single, narrow scope. Don't try to auto-optimize Vitto's entire system prompt. Start with ONE specific sub-task: e.g., "how Vitto generates the initial response to a new chat message." Expand scope only after proving the loop works.

Effort: 2-3 sessions for infrastructure, then ongoing | Risk: Medium | Dependencies: Phase 6 (memory/skills for storing results)

Phase 10: Cleanup + Operational Hardening (Week 6-8, parallel)

From Handoff Plan Phase 7, unchanged

This is maintenance work. Correct as written. Do it whenever there's spare capacity between other phases.

What I Cut and Why

What's Actually Missing from Both Plans

1. Rate Limiting Per User

Neither plan addresses what happens when one GoGrab user floods Vitto with requests. Without per-user rate limiting, a single aggressive user can exhaust the entire LLM budget. This should be in Phase 2 alongside the model router.

2. Graceful Degradation UX

Circuit breakers and retries handle the backend side. But what does the USER see when Vitto is degraded? Neither plan specifies the frontend behavior. When the LLM provider is down, Vitto should:

• Acknowledge receipt ("I got your message, but I'm experiencing delays")
• Queue the task for processing when the provider recovers
• Notify the user when the task completes asynchronously

3. Multi-Tenancy in the Memory/Skill System

The Research Plan's skill system doesn't address tenant isolation. A skill learned while serving Space A must not leak into Space B. This is a data privacy issue, not just a nice-to-have. Phase 6 must include tenant-scoped memory from day one.

4. Regression Testing for Agent Behavior

Neither plan includes how to verify that agent behavior doesn't regress after infrastructure changes. You need a suite of "golden path" test scenarios — 20-30 representative user interactions that Vitto should handle correctly. Run them before and after every phase deployment. Without this, you'll ship Phase 5's runtime refactor and silently break 15% of user workflows.

5. Cost Alerting

The Handoff Plan adds health monitoring and queue metrics. The Research Plan adds model routing. Neither adds cost alerting. You need: if LLM spend in the last hour exceeds 2x the trailing 7-day average, alert immediately. A bad prompt loop or stuck agent can burn through hundreds of dollars in an hour.

Revised Timeline

Total: ~8 weeks, ~25-30 sessions

This is 4 weeks longer than the Handoff Plan alone, but delivers a fundamentally different product: not just a stable platform, but an intelligent, self-improving, cost-efficient agent system.

Decision Framework: When to Stop

Not everything in this plan should be built. Here's how to decide what to cut if time or resources constrain:

Must build (Phases 0-1-3): Baselines, resilience, alerting. Without these, production is fragile. Non-negotiable.

Should build (Phases 2-4-5): Cache, routing, migration, runtime layer. These are the cost savings and architectural improvements that make Vitto viable at scale. Skip only if you're not growing.

High-value optional (Phases 6-7-8): Memory, edge layer, MCP tiers. These differentiate Vitto from competitors. Build when the foundation is solid.

Experimental (Phase 9): Self-improvement loop. Build only after Phases 0-6 are deployed and stable. This is the "10x" bet but requires everything else to be working first.

Prepared March 10, 2026 — GoGrab AI Engineering Review This document supersedes both the Handoff Infrastructure Enhancement Plan and the Vitto Super Agent Research Plan.