# Node VM template with image-builder (Proxmox)

Result: Proxmox template `ubuntu-2404-kube-v1.35.9` on node `pv1-1` — Ubuntu 24.04.4, containerd,
kubeadm/kubelet/kubectl v1.35.9, pre-pulled control-plane images, cloud-init, qemu-guest-agent.
Its VMID is `TEMPLATE_VMID` for CAPMOX.

## Build host prerequisites (Ubuntu 24.04)

```bash
sudo apt install -y make git curl unzip ca-certificates openssl openssh-client \
                    python3 python3-venv python3-pip
python3 -m venv ~/ib-venv && source ~/ib-venv/bin/activate
pip install ansible-core==2.18.18          # avoids PEP 668 / --break-system-packages
git clone https://github.com/kubernetes-sigs/image-builder.git ~/image-builder
cd ~/image-builder/images/capi && git checkout v0.1.55
make deps-proxmox                          # python/ansible/packer 1.9.5 checks + packer init
```

## Build

Overrides live in a var file ([configs/image-builder/proxmox-lab.json.example](../configs/image-builder/proxmox-lab.json.example)),
passed via `PACKER_VAR_FILES` (last on the packer command line → highest precedence).

```bash
export PROXMOX_URL="https://192.168.0.8:8006/api2/json"
export PROXMOX_USERNAME='capmox@pve!capi'
export PROXMOX_TOKEN="<secret>"
export PROXMOX_NODE="pv1-1"               # PVE node *name* (pvesh get /nodes), not IP; template lands here
export PROXMOX_ISO_POOL="local"           # storage with content 'iso'    (pvesm status --content iso)
export PROXMOX_STORAGE_POOL="local-lvm"   # storage with content 'images' (pvesm status --content images)
export PROXMOX_BRIDGE="vmbr0"
export PROXMOX_VLAN="4"
export PACKER_VAR_FILES=~/proxmox-lab.json
unset PACKER_FLAGS ISO_FILE

export SSH_PASSWORD=labpass               # optional: ssh builder@<vm> while provisioning runs
ON_ERROR_ASK=1 PACKER_LOG=1 PACKER_LOG_PATH=packer.log make build-proxmox-ubuntu-2404
```

The build VM (VLAN 4) needs DHCP and must reach the Packer HTTP server on the build host
(ports 8000–9000) to fetch the autoinstall `user-data`.

## What the build does

1. `set-ssh-password` renders the autoinstall `user-data` with a hashed password for user `builder`.
2. Packer creates VM `packer-<uuid>` (2 vCPU, 2 GB, 20 GB), attaches the ISO, serves `http/24.04/`.
3. After `boot_wait` it types the GRUB boot command over VNC: `autoinstall ds=nocloud-net;s=http://<host>:<port>/24.04/`.
4. Subiquity installs Ubuntu and reboots; Packer waits for SSH (up to 2h), IP via guest agent.
5. Provisioners: wait for cloud-init → ansible `firstboot.yml` → reboot → ansible `node.yml`
   (containerd, k8s binaries, images, sysctl, cloud-init config) → goss tests → lock `builder`, shutdown.
6. VM converted to template `ubuntu-2404-kube-v1.35.9`.

Observe it in the PVE noVNC console; `DEBUG=1` steps through each Packer stage; `make -n` prints the
exact `packer build` command. Typical duration 20–40 min.

## Fixes log

| # | Symptom | Root cause | Fix |
| --- | --- | --- | --- |
| 1 | `POST .../storage/local/upload: write: broken pipe` | pveproxy dropped the 3.17 GiB upload mid-stream — missing `Datastore.AllocateTemplate` (token had only `PVEVMAdmin`) or no spool space in `/var/tmp` (exact cause not pinned down) | `PVEDatastoreAdmin` on `/storage`, check `df -h /var/tmp`; then avoid re-uploads with `iso_file` |
| 2 | `net0.bridge: invalid bridge ID 'vmbr0.4@vmbr0'` | `PROXMOX_BRIDGE` set to the `ip link` name of a VLAN sub-interface; PVE wants a bridge name + separate tag | `PROXMOX_BRIDGE=vmbr0`, `PROXMOX_VLAN=4` (→ `bridge=vmbr0,tag=4`) |
| 3 | `403 Permission check failed (/sdn/zones/localnetwork/vmbr0.4, SDN.Use)` | since PVE 8 every Linux bridge is an object in SDN zone `localnetwork`; `PVEVMAdmin` lacks `SDN.Use` | `PVESDNUser` on `/sdn/zones/localnetwork` |
| 4 | `unsupported format 'qcow2' at .../LvmThinPlugin.pm line 102` | image-builder defaults `disk_format` to qcow2; LVM-thin is raw-only. `--var disk_format=raw` in `PACKER_FLAGS` was lost when `PACKER_FLAGS` was re-exported | `"disk_format": "raw"` in the `PACKER_VAR_FILES` JSON |
| 5 | `one of iso_file, iso_url, or ... cd_content must be specified for boot_iso` | packer-plugin-proxmox v1.2.4 requires **exactly one** ISO source; `ubuntu-2404.json` sets `iso_file` from `$ISO_FILE` *and* a hardcoded `iso_url` → count 2. Misleading error; README tip is broken (worth an upstream issue) | set `iso_file` and `"iso_url": ""` in the var file, unset `ISO_FILE` |

Notes carried into CAPMOX (phase 1): same ACL set, `vlan: 4` on the network device, no
`format: qcow2` in `ProxmoxMachineTemplate`s.

## Smoke test of the template

The template has no usable login (`builder` is locked) — access only via cloud-init (NoCloud,
which the PVE Cloud-Init drive provides):

```bash
TPL=<template vmid>; ID=9001
qm clone $TPL $ID --name smoke-k8s --full 1 --storage local-lvm
qm set $ID --ide2 local-lvm:cloudinit
qm set $ID --ciuser jan --sshkeys /root/jan.pub --ipconfig0 ip=dhcp
qm set $ID --net0 virtio,bridge=vmbr0,tag=4
qm start $ID && qm guest cmd $ID network-get-interfaces
# inside: cloud-init status --long; kubeadm version -o short; crictl images
qm stop $ID && qm destroy $ID --purge
```

kubelet crash-looping before `kubeadm init/join` is expected.