Architecture: How to handle networking for Firecracker VMs (internet acces...

Documentation for Sessions

🏗️ Architecture: How to handle networking for Firecracker VMs (internet acces...

Date: 2026-02-27
Time: 16:24:07 UTC
Session: gspace1-150.5
Agent: angular
Confidence: 93%
Status: ✅ Documented

📋 Decision Context

Question:

How to handle networking for Firecracker VMs (internet access + orchestrator communication)?

Task: Firecracker VM networking architecture

🎯 Decision Made

Chosen:

Linux bridge (br-sandbox, 172.16.0.1/24) with per-VM TAP devices. NetworkService manages IP pool (172.16.0.2-254, 253 addresses), creates TAP at VM creation, attaches to bridge, destroys at VM shutdown. Host does NAT masquerade for internet access via iptables.

Options Considered:

  1. Bridge+TAP with NAT (chosen)
  2. macvlan (direct host network)
  3. User-mode networking (slirp)
  4. No networking (isolated)

💭 Reasoning

ROOT CAUSE: Firecracker VMs need both (a) internet access (to browse Gmail, GitHub, etc.) and (b) orchestrator access (for SSH commands). Bridge+TAP is the standard Firecracker networking pattern — each VM gets a TAP device (virtual network cable) plugged into a bridge (virtual switch). The bridge is NATed to the host's eth0 for internet. SECURITY: VMs can reach internet but iptables blocks access to host management ports (SSH, orchestrator API). Only the orchestrator SSHes INTO VMs, never the reverse. IP POOL: 253 addresses matches MAX_CONTAINERS=30 default with room to grow. Pool is in-memory (no persistence needed — VMs are ephemeral). TAP naming: tap-{first8chars} to stay within Linux IFNAMSIZ 15-char limit. FUTURE RISK: If we need >253 VMs, expand to multiple /24 subnets. EARLY WARNING: Monitor sandbox_ip_pool_available Prometheus metric.

📁 Files Involved

🔗 Related

Auto-generated by MCP Smart Consolidation