# Dev VM Setup Guide

Ubuntu 24 setup for running `firecracker-orchestrator`.

## Prerequisites on your hypervisor

Before creating the VM, **enable nested virtualization** (expose KVM to the guest):

- **VMware Fusion**: VM Settings → Processors & Memory → Advanced → enable "Enable hypervisor applications in this virtual machine"
- **Parallels**: VM Config → CPU & Memory → Advanced → enable "Enable nested virtualization"
- **UTM / QEMU**: Add `-cpu host` flag or enable "Force multi-core" + "Enable Hypervisor"

> The VM must be **x86_64** — the orchestrator downloads an x86_64 kernel and Alpine rootfs at init time.

---

## Step 1 — Verify KVM is accessible

```bash
ls -la /dev/kvm
# Expected: crw-rw---- 1 root kvm ...
```

If `/dev/kvm` doesn't exist, nested virtualization isn't enabled on your hypervisor — go back and fix that first.

Add your user to the `kvm` group so firecracker doesn't require root:

```bash
sudo usermod -aG kvm $USER
# Log out and back in, then verify:
groups | grep kvm
```

---

## Step 2 — Install system dependencies

```bash
sudo apt update && sudo apt install -y \
  e2fsprogs \
  iproute2 \
  iptables \
  tar \
  wget \
  curl \
  btrfs-progs
```

| Package | Used for |
|---|---|
| `e2fsprogs` | `mkfs.ext4` — formats the Alpine rootfs image |
| `iproute2` | `ip` — manages TAP devices and bridge |
| `iptables` | NAT masquerade for VM networking |
| `tar` | Extracts the Alpine minirootfs tarball |
| `btrfs-progs` | `mkfs.btrfs` — enables COW reflink copies (see Step 5) |

---

## Step 3 — Install Go 1.26.2

```bash
wget https://go.dev/dl/go1.26.2.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.26.2.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc
go version
# Expected: go version go1.26.2 linux/amd64
```

---

## Step 4 — Install Firecracker 1.15.1

```bash
FIRECRACKER_VERSION=1.15.1
wget https://github.com/firecracker-microvm/firecracker/releases/download/v${FIRECRACKER_VERSION}/firecracker-v${FIRECRACKER_VERSION}-x86_64.tgz
tar -xzf firecracker-v${FIRECRACKER_VERSION}-x86_64.tgz
sudo mv release-v${FIRECRACKER_VERSION}-x86_64/firecracker-v${FIRECRACKER_VERSION}-x86_64 /usr/local/bin/firecracker
sudo chmod +x /usr/local/bin/firecracker
firecracker --version
```

---

## Step 5 — Set up btrfs working directory

The orchestrator uses `cp --reflink=always` for COW rootfs copies when spawning clones. This only works on **btrfs** or **xfs** — on the default ext4 it silently falls back to a full file copy, which wastes disk I/O and space.

Create a btrfs volume mounted at the orchestrator's working directory:

```bash
sudo dd if=/dev/zero of=/var/fc-orch.img bs=1M count=8192   # 8 GB
sudo mkfs.btrfs /var/fc-orch.img
sudo mkdir -p /tmp/fc-orch
sudo mount -o loop /var/fc-orch.img /tmp/fc-orch
sudo chown $USER /tmp/fc-orch
```

To persist across reboots, add to `/etc/fstab`:

```
/var/fc-orch.img  /tmp/fc-orch  btrfs  loop  0  0
```

---

## Step 6 — Enable IP forwarding persistently

The orchestrator enables this at runtime, but it resets on reboot:

```bash
echo 'net.ipv4.ip_forward=1' | sudo tee /etc/sysctl.d/99-firecracker.conf
sudo sysctl -p /etc/sysctl.d/99-firecracker.conf
```

---

## Step 7 — Build and run

**Option A — cross-compile on your Mac and copy over:**

```bash
# On macOS:
GOOS=linux GOARCH=amd64 go build -o fc-orch .
scp fc-orch user@your-vm:~/
```

**Option B — build directly on the VM:**

```bash
git clone <your-repo> && cd firecracker-orchestrator
go build -o fc-orch .
```

**Run:**

```bash
sudo ./fc-orch init      # download kernel + build Alpine rootfs
sudo ./fc-orch golden    # boot golden VM, snapshot it
sudo ./fc-orch spawn 5   # restore 5 clones
sudo ./fc-orch status    # list running clones
sudo ./fc-orch kill      # terminate all VMs
sudo ./fc-orch cleanup   # full teardown
```

> TAP/bridge/iptables operations require root (or `CAP_NET_ADMIN`). Running via `sudo` is the simplest approach.

---

## Quick checklist

| Check | Command | Expected |
|---|---|---|
| KVM accessible | `ls /dev/kvm` | file exists |
| In kvm group | `groups \| grep kvm` | `kvm` listed |
| Go version | `go version` | `go1.26.2` |
| Firecracker | `firecracker --version` | `1.15.1` |
| mkfs.ext4 | `which mkfs.ext4` | path printed |
| ip / iptables | `which ip && which iptables` | both paths printed |
| IP forwarding | `sysctl net.ipv4.ip_forward` | `= 1` |
| btrfs mount | `df -T /tmp/fc-orch` | type `btrfs` |