firecracker-orchestrator/dev-vm-guide.md

Dev VM Setup Guide

Ubuntu 24 setup for running firecracker-orchestrator.

Prerequisites on your hypervisor

Before creating the VM, enable nested virtualization (expose KVM to the guest):

• VMware Fusion: VM Settings → Processors & Memory → Advanced → enable "Enable hypervisor applications in this virtual machine"
• Parallels: VM Config → CPU & Memory → Advanced → enable "Enable nested virtualization"
• UTM / QEMU: Add -cpu host flag or enable "Force multi-core" + "Enable Hypervisor"

The VM must be x86_64 — the orchestrator downloads an x86_64 kernel and Alpine rootfs at init time.

---

Step 1 — Verify KVM is accessible

ls -la /dev/kvm
# Expected: crw-rw---- 1 root kvm ...

If /dev/kvm doesn't exist, nested virtualization isn't enabled on your hypervisor — go back and fix that first.

Add your user to the kvm group so firecracker doesn't require root:

sudo usermod -aG kvm $USER
# Log out and back in, then verify:
groups | grep kvm

---

Step 2 — Install system dependencies

sudo apt update && sudo apt install -y \
  e2fsprogs \
  iproute2 \
  iptables \
  tar \
  wget \
  curl \
  btrfs-progs

Package	Used for
e2fsprogs	mkfs.ext4 — formats the Alpine rootfs image
iproute2	ip — manages TAP devices and bridge
iptables	NAT masquerade for VM networking
tar	Extracts the Alpine minirootfs tarball
btrfs-progs	mkfs.btrfs — enables COW reflink copies (see Step 5)

---

Step 3 — Install Go 1.26.2

wget https://go.dev/dl/go1.26.2.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.26.2.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc
go version
# Expected: go version go1.26.2 linux/amd64

---

Step 4 — Install Firecracker 1.15.1

FIRECRACKER_VERSION=1.15.1
wget https://github.com/firecracker-microvm/firecracker/releases/download/v${FIRECRACKER_VERSION}/firecracker-v${FIRECRACKER_VERSION}-x86_64.tgz
tar -xzf firecracker-v${FIRECRACKER_VERSION}-x86_64.tgz
sudo mv release-v${FIRECRACKER_VERSION}-x86_64/firecracker-v${FIRECRACKER_VERSION}-x86_64 /usr/local/bin/firecracker
sudo chmod +x /usr/local/bin/firecracker
firecracker --version

---

Step 5 — Set up btrfs working directory

The orchestrator uses cp --reflink=always for COW rootfs copies when spawning clones. This only works on btrfs or xfs — on the default ext4 it silently falls back to a full file copy, which wastes disk I/O and space.

Create a btrfs volume mounted at the orchestrator's working directory:

sudo dd if=/dev/zero of=/var/fc-orch.img bs=1M count=8192   # 8 GB
sudo mkfs.btrfs /var/fc-orch.img
sudo mkdir -p /tmp/fc-orch
sudo mount -o loop /var/fc-orch.img /tmp/fc-orch
sudo chown $USER /tmp/fc-orch

To persist across reboots, add to /etc/fstab:

/var/fc-orch.img  /tmp/fc-orch  btrfs  loop  0  0

---

Step 6 — Enable IP forwarding persistently

The orchestrator enables this at runtime, but it resets on reboot:

echo 'net.ipv4.ip_forward=1' | sudo tee /etc/sysctl.d/99-firecracker.conf
sudo sysctl -p /etc/sysctl.d/99-firecracker.conf

---

Step 7 — Build and run

Option A — cross-compile on your Mac and copy over:

# On macOS:
GOOS=linux GOARCH=amd64 go build -o fc-orch .
scp fc-orch user@your-vm:~/

Option B — build directly on the VM:

git clone <your-repo> && cd firecracker-orchestrator
go build -o fc-orch .

Run:

sudo ./fc-orch init      # download kernel + build Alpine rootfs
sudo ./fc-orch golden    # boot golden VM, snapshot it
sudo ./fc-orch spawn 5   # restore 5 clones
sudo ./fc-orch status    # list running clones
sudo ./fc-orch kill      # terminate all VMs
sudo ./fc-orch cleanup   # full teardown

TAP/bridge/iptables operations require root (or CAP_NET_ADMIN). Running via sudo is the simplest approach.

---

Quick checklist

Check	Command	Expected
KVM accessible	ls /dev/kvm	file exists
In kvm group	groups | grep kvm	kvm listed
Go version	go version	go1.26.2
Firecracker	firecracker --version	1.15.1
mkfs.ext4	which mkfs.ext4	path printed
ip / iptables	which ip && which iptables	both paths printed
IP forwarding	sysctl net.ipv4.ip_forward	= 1
btrfs mount	df -T /tmp/fc-orch	type btrfs