290.25 Architecture How Should We Fix Vitto Prod Dev Staging Separation
Documentation for Sessions
Session 290.2 — Vitto Production/Development/Staging Separation
Date: 2026-03-31
Type: Infrastructure + Architecture
Status: Complete
Branch: session-290-299
Commits: 8 (2 docs + 6 implementation)
What Was the Problem?
Users on the live GoGrab platform were seeing the error message "Connection to the Vitto engine was lost" when trying to use Vitto (the AI assistant) to generate things like a personal expense tracking system. The platform appeared to be working — all servers showed "healthy" — but Vitto couldn't actually complete any tasks.
Why Was This Happening?
The problem had three layers:
The backend server was lying about its health. When Railway (our cloud hosting) asked "are you healthy?", the Vitto server always said "yes" — even when it was missing critical configuration (like database credentials or connections to other services). It would accept user requests but then fail silently when trying to process them.
Vitto was treated as a second-class citizen. The platform has 4 backend services (Main API, AI API, Docs API, and Vitto API). The first three had a clean, centralized configuration system. Vitto was added later and used a separate, fragile path for figuring out its server address. This meant changes to the configuration system could break Vitto without anyone noticing.
Configuration was fragmented across apps. The platform has 6 frontend applications. Each one had a slightly different way of storing server addresses — some used the centralized system, some had URLs typed directly into code, one was missing URLs entirely. This made it easy for things to fall out of sync.
What Did We Do?
Timeline
| Step | What | Why |
|---|---|---|
| 1. Investigation | Checked all 3 production servers — all healthy and running | Ruled out server downtime |
| 2. Root Cause Analysis | Traced the error from frontend chat UI through SSE streaming to backend task execution | Found 7 distinct issues across backend and frontend |
| 3. Design | Wrote a spec covering all issues with 3 approach options | User approved "Full Prod/Dev Unification" (Approach B) |
| 4. Staging Integration | Incorporated Railway staging environment (production vs staging vs local) | Aligned with existing staging environment spec |
| 5. App Audit | Checked all 6 frontend apps for environment configuration gaps | Found 4 different patterns, 3 apps with issues |
| 6. Implementation | 8 files changed across 6 commits | All TypeScript compilations pass |
| 7. Documentation | Logged 3 decisions, 3 conventions, 5 lessons learned into intelligence system | Prevents recurrence |
Changes Made (in Plain Language)
1. Backend Now Crashes Loud Instead of Failing Silent
Before: Server starts, says "I'm healthy!", then can't actually do anything because it's missing database credentials. After: Server checks for 4 critical settings on startup. If any are missing, it immediately stops with a clear error message telling you exactly what's missing. No more ghost servers.
2. Vitto Promoted to First-Class Citizen
Before: The platform's configuration system knew about 3 of 4 backend services. Vitto used a separate, fragile path. After: Vitto is now configured through the exact same centralized system as the other 3 services. All 4 APIs are treated identically.
3. Staging URLs Updated
Before: Staging server addresses pointed to old, generic Railway URLs (like backend-api-staging.up.railway.app).
After: Staging addresses point to our custom branded domains (staging-api.gograb.space, staging-vitto.gograb.space, etc.).
4. All Frontend Apps Aligned
Before: 6 apps, 4 different patterns:
- gograb-admin & gograb-dev: Used centralized system (correct)
- gograb-ai: Had URLs typed directly as text strings
- space-admin: Legacy code, missing 3 of 4 server addresses entirely
- gograb-space: Own separate config file, missing staging addresses
- docs-angular: Only needs 2 of 4 addresses (docs-focused, fine as-is)
After: All apps that need Vitto now use the centralized configuration system. Missing addresses added. Hardcoded strings replaced.
5. Stale Port Numbers Fixed
Before: Development fallback ports referenced an old workspace (5200, 5100, 5400).
After: Updated to current workspace ports (6200, 6100, 6400). Cosmetic but prevents confusion.
Files Changed
| File | What Changed |
|---|---|
environment-presets.ts (api-config lib) |
Added Vitto to config interface + all 5 environment presets; fixed 3 stale staging URLs |
api-tokens.ts (api-config lib) |
Added Vitto to composite API configuration token |
main.ts (backend-vitto-api) |
Added startup env validation; fixed 3 stale CORS port fallbacks |
environment.shared.ts (gograb-space) |
Added staging URL keys to all 4 URL constants |
environment.railway.ts (gograb-space) |
Added missing Docs API import and URL field |
environment.prod.ts (gograb-ai) |
Replaced 4 hardcoded URL strings with centralized preset references |
environment.ts (space-admin) |
Full rewrite to use centralized presets (was legacy) |
environment.prod.ts (space-admin) |
Full rewrite to use centralized presets (was legacy) |
Three-Environment Model (After Changes)
| Local Development | Staging (Railway) | Production (Railway) | |
|---|---|---|---|
| Main API | localhost:3000 | staging-api.gograb.space | api.gograb.space |
| AI API | localhost:3001 | staging-ai-api.gograb.space | ai-api.gograb.space |
| Docs API | localhost:3002 | staging-docs-api.gograb.space | docs-api.gograb.space |
| Vitto API | localhost:3003 | staging-vitto.gograb.space | vitto-api.gograb.space |
| Detection | window.location = localhost |
NODE_ENV=staging |
NODE_ENV=production |
| Safety | Forgiving (localhost defaults) | Fail-fast (crash if missing) | Fail-fast (crash if missing) |
Verification Results
| Check | Result |
|---|---|
| api-config library build | Pass |
| backend-vitto-api build | Pass |
| gograb-space TypeScript | Pass (0 errors) |
| gograb-ai TypeScript | Pass (0 errors) |
| gograb-admin TypeScript | Pass (0 errors) |
| gograb-dev TypeScript | Pass (0 errors) |
| space-admin TypeScript | Pass (0 errors) |
Note: Some apps have a pre-existing memoryLimit NX schema warning in their project.json — this is unrelated to our changes and existed before this session.
What This Does NOT Include
These items are covered by other specs or sessions:
- Database isolation for staging (DB_ENV suffix) — covered by the Staging Environment Design spec
- Railway staging setup (creating the environment, setting env vars) — documented in staging spec
- Health endpoint overhaul — existing
/engine/statusalready provides dependency info - Frontend staging builds (
environment.staging.tsfiles) — deferred per staging spec Phase 5
Decisions Logged to Intelligence System
| Decision | Type | Confidence | ID |
|---|---|---|---|
| Fail-fast env validation in main.ts | Infrastructure | 0.95 (calibrated: 0.905) | dev_1774983921306_89fj8hi |
| Promote vittoApiUrl to first-class in EnvironmentConfig | Architecture | 0.95 | dev_1774983932233_0bkg358 |
| Unify 5 frontend apps on centralized presets | Refactor | 0.93 | dev_1774983943210_n42ch9a |
Conventions Added
- New API services must be added to all 5 config layers (EnvironmentConfig, presets, ApiConfiguration, API_CONFIG, app environments)
- Railway backends must have fail-fast env var validation before NestFactory.create
- Staging URLs use custom domains (
staging-*.gograb.space), never.up.railway.app
Lessons Learned (Captured in Intelligence System)
- New API services added incrementally drift from centralized patterns — add to all config layers simultaneously
- Railway health checks passing does NOT mean the service is functional — fail-fast validation is cheap and effective
- Staging URLs rot quickly — update URL constants in the same PR as the spec that defines them
- Frontend app environment configs fragment silently — periodic audits prevent drift
- CORS fallback ports are cosmetic but stale values cause debugging confusion
Future Steps
- Deploy to Railway production — push to
deploy/vitto-apibranch to activate the fail-fast validation - Set up Railway staging environment — follow the staging environment design spec to create the staging Railway environment, set env vars, configure DNS
- Create
environment.staging.tsfiles — once staging is live, add staging build configurations to each frontend app (Phase 5 of staging spec) - Fix pre-existing
memoryLimitNX schema issue — removememoryLimitfromproject.jsonbuild targets across affected apps - Audit other backend APIs — apply the same fail-fast env var validation pattern to
backend-api,backend-ai-api, andbackend-docs-api
Related Documents
- Design Spec: docs/superpowers/specs/2026-03-31-vitto-prod-dev-separation-design.md
- Implementation Plan: docs/superpowers/plans/2026-03-31-vitto-prod-dev-separation.md
- Staging Environment Spec: docs/superpowers/specs/2026-03-27-staging-environment-design.md