Bugfix: Session 177 Bug 3: Why does sshd hang forever in Firecracker...

Documentation for Sessions

🐛 Bugfix: Session 177 Bug 3: Why does sshd hang forever in Firecracker...

Date: 2026-03-05
Time: 17:12:03 UTC
Session: gspace1-178.10
Agent: angular
Confidence: 98%
Status: ✅ Documented


📋 Decision Context

Question:

Session 177 Bug 3: Why does sshd hang forever in Firecracker VM despite desktop (Xvfb/XFCE4/Chrome/VNC) all starting?

Task: Not specified


🎯 Decision Made

Chosen:

Kernel 4.14.174 has zero entropy in Firecracker VMs. getrandom() syscall blocks until CRNG reaches crng_init=2 which never happens without hardware entropy sources. The file was named vmlinux-5.10 but actually contained kernel 4.14.174. Fix: downloaded actual kernel 5.10.0 from Firecracker CI (s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/x86_64/kernels/vmlinux-5.10.bin). Added random.trust_cpu=on boot arg (5.10 supports this, 4.14 does not). Added virtio-rng entropy device via PUT /entropy Firecracker API.

Options Considered:


💭 Reasoning

Deep investigation chain: (1) Hypothesis: rootfs missing — WRONG (rootfs existed at 4GB). (2) Hypothesis: network broken — WRONG (ping worked, VNC on port 5900 was listening). (3) Console showed 'random: dbus-daemon: uninitialized urandom read' and sshd PID existed but port 22 never opened. (4) Tried writing to /dev/urandom — doesn't credit kernel entropy pool in 4.14. (5) Tried Python RNDADDENTROPY ioctl — Python itself blocks on getrandom() during import. (6) Tried /dev/random → /dev/urandom symlink — getrandom() syscall bypasses /dev/random entirely. (7) Added Firecracker virtio-rng device — 204 accepted, kernel showed 'fast init done' (crng_init=1) but OpenSSH needs crng_init=2. (8) FINAL: replaced kernel. 5.10 with random.trust_cpu=on shows 'crng done (trusting CPU manufacturer)' at 29ms. sshd starts immediately. This would be wrong if: CPU lacks RDRAND instruction (all modern x86_64 CPUs have it). Early warning: any kernel version string mismatch between filename and actual version.


📁 Files Involved


🔗 Related


Auto-generated by MCP Smart Consolidation