Infrastructure: Why did AX102 sandbox-orchestrator crash with 502 Bad Gatewa...

Documentation for Sessions

📝 Infrastructure: Why did AX102 sandbox-orchestrator crash with 502 Bad Gatewa...

Date: 2026-03-30
Time: 18:33:09 UTC
Session: gspace1-290.10
Agent: architecture
Confidence: 98%
Status: ✅ Documented

📋 Decision Context

Question:

Why did AX102 sandbox-orchestrator crash with 502 Bad Gateway when Computer Use sessions were requested?

Task: Not specified

🎯 Decision Made

Chosen:

Two fixes: (1) Added firecrackerAvailable guard + spawn error handler in FirecrackerService to prevent process crash on missing binary, (2) Installed Firecracker v1.7.0 binary on AX102 by copying from AX42

Options Considered:

  1. Only install binary (operational fix, no code protection)
  2. Only add code guard (prevents crash but VMs still fail)
  3. Both: code guard + binary install (chosen — defense in depth)

💭 Reasoning

Root cause: Firecracker binary was never installed on AX102. spawn('firecracker') emitted an unhandled 'error' event (ENOENT) on the ChildProcess, crashing the entire Node.js process. systemd restarted it, but next VM creation triggered the same crash. The code had no guard — onModuleInit detected the missing binary (logged a warning) but startVM() proceeded anyway. The spawn error event was unhandled because fcProcess.on('error') was never registered. Two-part fix: code hardening (guard + error handler) prevents crash even if binary is missing in future; binary installation enables actual VM creation.

📁 Files Involved

🔗 Related

Auto-generated by MCP Smart Consolidation