KVM/libvirt provider: proxies as VMs on your own hypervisors #5
@@ -41,7 +41,7 @@ providers:
|
|||||||
url: https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2
|
url: https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2
|
||||||
# sha256: "<64 hex>" # optional; verified after download
|
# sha256: "<64 hex>" # optional; verified after download
|
||||||
# firmware: uefi # default; "bios" only for images with no EFI partition
|
# firmware: uefi # default; "bios" only for images with no EFI partition
|
||||||
# # (Debian 13 does not boot via BIOS, see docs/kvm-provider.md)
|
# # (Debian 13 hangs in GRUB under BIOS headless: Debian bug #1111240)
|
||||||
ubuntu-24.04:
|
ubuntu-24.04:
|
||||||
url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
||||||
machineTypes: # placement.machineType names a key here
|
machineTypes: # placement.machineType names a key here
|
||||||
|
|||||||
@@ -455,8 +455,12 @@ they read as choices, not accidents. Chronological by build step.
|
|||||||
state at all; answering that with delete-and-recreate is too drastic,
|
state at all; answering that with delete-and-recreate is too drastic,
|
||||||
so the next poll decides.
|
so the next poll decides.
|
||||||
- **KVM guests boot UEFI by default, secure boot off.** Found on the
|
- **KVM guests boot UEFI by default, secure boot off.** Found on the
|
||||||
first real host: Debian 13's generic image hangs after GRUB under
|
first real host: Debian 13's generic image hangs in GRUB under SeaBIOS
|
||||||
SeaBIOS regardless of CPU model or seed, and boots under OVMF. Every
|
on a headless VM (Debian bug #1111240: `terminal_output gfxterm
|
||||||
|
serial` with a serial console and no video device) regardless of CPU
|
||||||
|
model or seed, and boots under OVMF with the serial console working.
|
||||||
|
The alternatives were a virtio console (loses the kernel's serial
|
||||||
|
output) or a video device the VMs have no use for. Every
|
||||||
current distro cloud image supports UEFI, so it is the default and
|
current distro cloud image supports UEFI, so it is the default and
|
||||||
`firmware: bios` is the per-image escape; libvirt's firmware
|
`firmware: bios` is the per-image escape; libvirt's firmware
|
||||||
autoselection (`<os firmware="efi">`) keeps the OVMF path out of the
|
autoselection (`<os firmware="efi">`) keeps the OVMF path out of the
|
||||||
|
|||||||
+10
-6
@@ -111,7 +111,7 @@ ssh virt@nas1.home.lan ip link show br0
|
|||||||
| `images.<key>.url` | yes per entry | | http(s) URL of a cloud image (qcow2 or raw) |
|
| `images.<key>.url` | yes per entry | | http(s) URL of a cloud image (qcow2 or raw) |
|
||||||
| `images.<key>.sha256` | | | expected digest; verified after download, a change triggers a re-download |
|
| `images.<key>.sha256` | | | expected digest; verified after download, a change triggers a re-download |
|
||||||
| `images.<key>.format` | | `qcow2` | `qcow2` or `raw` |
|
| `images.<key>.format` | | `qcow2` | `qcow2` or `raw` |
|
||||||
| `images.<key>.firmware` | | `uefi` | `uefi` (OVMF, secure boot off) or `bios` (SeaBIOS) for images without an EFI system partition; Debian 13's BIOS path does not boot, see *Known gaps* |
|
| `images.<key>.firmware` | | `uefi` | `uefi` (OVMF, secure boot off) or `bios` (SeaBIOS) for images without an EFI system partition; Debian 13 hangs in GRUB under BIOS on a headless VM, see *Known gaps* |
|
||||||
| `machineTypes.<key>` | ≥ 1 | | `{vcpus, memoryMiB}` size profile |
|
| `machineTypes.<key>` | ≥ 1 | | `{vcpus, memoryMiB}` size profile |
|
||||||
| `hosts[].name` | yes | | the `placement.zone` value that selects this host; unique |
|
| `hosts[].name` | yes | | the `placement.zone` value that selects this host; unique |
|
||||||
| `hosts[].address` | yes | | SSH endpoint, `host:port` |
|
| `hosts[].address` | yes | | SSH endpoint, `host:port` |
|
||||||
@@ -233,11 +233,15 @@ The provider logs through controller-runtime's logger, named `kvm`, with
|
|||||||
- **IP discovery needs either guest traffic or the agent.** A VM on a
|
- **IP discovery needs either guest traffic or the agent.** A VM on a
|
||||||
bridge with no DHCP never gets an address and stays `Provisioning`
|
bridge with no DHCP never gets an address and stays `Provisioning`
|
||||||
forever; check `virsh console`.
|
forever; check `virsh console`.
|
||||||
- **Debian 13 cloud images do not boot via BIOS on the tested host**
|
- **Debian 13 cloud images hang in GRUB under BIOS on a headless VM**
|
||||||
(Ryzen 8845HS, libvirt 11.6, QEMU 10.1): GRUB prints "Booting" and
|
([Debian bug #1111240](https://bugs.debian.org/1111240), open, in
|
||||||
the kernel never reaches the console, with or without a seed, with
|
`grub2`): trixie's image ships `terminal_output gfxterm serial` in its
|
||||||
host-passthrough and host-model CPUs alike. The same image boots
|
GRUB config, and with a serial console but no video device (exactly
|
||||||
under UEFI, which is why `firmware` defaults to `uefi`. Images that
|
the domain this provider defines) GRUB prints "Booting" and never
|
||||||
|
hands over to the kernel. Not host-specific; reproduced by the Debian
|
||||||
|
cloud team. The same image boots under UEFI with the serial console
|
||||||
|
intact, which is why `firmware` defaults to `uefi`. Debian 12
|
||||||
|
(bookworm) images are unaffected and boot either way. Images that
|
||||||
need `firmware: bios` are the exception, not the rule.
|
need `firmware: bios` are the exception, not the rule.
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|||||||
@@ -514,3 +514,21 @@ configured, and the cache reuses a base image by key, so switching
|
|||||||
`KVM_TEST_IMAGE_URL` requires deleting `base-test-image.qcow2` and its
|
`KVM_TEST_IMAGE_URL` requires deleting `base-test-image.qcow2` and its
|
||||||
marker by hand first. That is the reviewer's Step 4 finding showing up
|
marker by hand first. That is the reviewer's Step 4 finding showing up
|
||||||
in practice; still documented rather than fixed.
|
in practice; still documented rather than fixed.
|
||||||
|
|
||||||
|
Follow-up, same day: the cause is known upstream. [Debian bug
|
||||||
|
#1111240](https://bugs.debian.org/1111240) (filed 2025-08-16 against
|
||||||
|
the cloud image, reassigned to `grub2`, still open as of 2026-09-10;
|
||||||
|
trixie ships grub2 2.12-9+deb13u2) reports exactly this with a domain
|
||||||
|
XML almost identical to ours: BIOS, virtio disk, SATA cdrom seed, bridge
|
||||||
|
NIC, `<console><target type='serial'/>`, no `<graphics>`/`<video>`.
|
||||||
|
The cloud team reproduced it: trixie's grub.cfg has `terminal_output
|
||||||
|
gfxterm serial`, and with a serial port present but no video adapter
|
||||||
|
GRUB never hands over to the kernel. Their fix in the image
|
||||||
|
(`terminal_output serial`) is not something an operator can apply;
|
||||||
|
the reporter's workaround (console `target type='virtio'`) boots but
|
||||||
|
takes the kernel's `console=ttyS0` output with it. UEFI sidesteps the
|
||||||
|
gfxterm path and keeps the serial console, so the decision stands, but
|
||||||
|
the docs now name the bug rather than blaming the host, and note that
|
||||||
|
bookworm images are unaffected. `latest` on cloud.debian.org was the
|
||||||
|
2026-08-31 build when this was hit, so every trixie point release to
|
||||||
|
date carries it.
|
||||||
|
|||||||
@@ -37,8 +37,10 @@ type domainParams struct {
|
|||||||
//
|
//
|
||||||
// UEFI is expressed as <os firmware="efi"> with secure boot off, and libvirt
|
// UEFI is expressed as <os firmware="efi"> with secure boot off, and libvirt
|
||||||
// picks the OVMF binary and creates the NVRAM file itself (removed again by
|
// picks the OVMF binary and creates the NVRAM file itself (removed again by
|
||||||
// the nvram undefine flag). Debian 13's BIOS boot path was found to hang
|
// the nvram undefine flag). Under SeaBIOS, Debian 13's GRUB hangs on a
|
||||||
// after GRUB on a real host, while the same image boots under UEFI.
|
// headless domain with a serial console (Debian bug #1111240:
|
||||||
|
// "terminal_output gfxterm serial" with no video device); the same image
|
||||||
|
// boots under UEFI with the serial console intact.
|
||||||
func buildDomainXML(p domainParams) (string, error) {
|
func buildDomainXML(p domainParams) (string, error) {
|
||||||
os := osXML{
|
os := osXML{
|
||||||
Type: osTypeXML{Arch: "x86_64", Machine: "q35", Value: "hvm"},
|
Type: osTypeXML{Arch: "x86_64", Machine: "q35", Value: "hvm"},
|
||||||
|
|||||||
Reference in New Issue
Block a user