kacerrandClaude Opus 5
c10f98a352
baseVolumeName was base-<key>.<format>, byte-identical to what the kvm provider builds in the same pool -- for URL images too, since kvm takes hex(sum[:8]) and kvm2 takes hex(sum[:])[:16], the same eight bytes -- with a completely separate lock table. The shipped config has a live kvm-home entry and a commented-out kvm2-home entry on the same host, the same key debian-13 and the same default pool, and the comment invites the operator to enable it. Both integration tests use key test-image. On beelink both live overlays back onto kvm's base-debian-13.qcow2. Since cachedImage accepts any 64-hex marker when no sha256 is configured and never re-hashes the base, that meant two entries pointing one key at different URLs silently shared a file; a concurrent cold fetch let one entry delete the other's half-uploaded base and boot a guest from the remains; and a sha256 edit deleted the base in place under live overlays, which libvirt permits without complaint. The name now carries the provider entry: base-<entry>-<key>.<format>. imageSource gained an Entry field and resolveImage takes the provider name, so every production call site follows from imageSource.volumeName(). That closes the cross-entry sharing outright, kvm<->kvm2 and kvm2<->kvm2 alike, and turns the per-Provider imageLocks from a latent bug into a correct one: with names unique per entry, two entries cannot contend for a file, so no process-wide lock table is needed. The entry name now reaches a file name, so New checks it against the existing kvmIdentifier rule, exported as provider.IsKVMIdentifier rather than duplicated. Provider names are otherwise validated for all types at once and only for being non-empty and unique, so a name with a slash would have produced a volume nothing could address. One companion fix, because the naming alone does not stop it: cachedImage's digest-mismatch branch deleted the marker and the base and refetched. It now returns ErrPermanent naming both digests and telling the operator what to do. That branch was the remaining path that unlinks a good file under running domains, and the contract already said "add a new key instead" -- the code was contradicting it. The no-marker and non-hex-marker branches keep deleting, since there the base is debris. Not fixed, and said so in the docs rather than left to look closed: editing images.<key>.url is still a silent no-op, since nothing records the URL; and cross-process fencing still needs the deferred atomic publish (upload to a temp name, verify, publish), which lands where H2 put the discard. Migration is a non-event. The base-debian-13.qcow2 on beelink is kvm's, kvm keeps using it untouched, and kvm2 downloads its own copy once per host. Nothing is replaced under a running guest; the pool needs room for both, which the prerequisites now say. The eviction recipe in the guide named base-debian-13.qcow2, which after this change is kvm's file -- following it would have deleted the wrong provider's base. It now greps for the prefix and deletes the marker first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
91 lines
4.3 KiB
YAML
91 lines
4.3 KiB
YAML
# Sample --providers-config file (not a Kubernetes manifest). In-cluster
|
|
# this content lives in the providers-config ConfigMap
|
|
# (config/manager/providers_config.yaml); for `make run-dev` a
|
|
# kubernetes-only variant is at hack/providers-dev.yaml.
|
|
#
|
|
# Named provider instances: "gcp-eu" and "gcp-us" are two configs of the
|
|
# same type. spec.provider on a Proxy refers to the name, not the type.
|
|
providers:
|
|
- name: kubernetes
|
|
type: kubernetes
|
|
# kubernetes:
|
|
# image: ubuntu/squid:6.6-24.04_edge # the default
|
|
- name: gcp-eu
|
|
type: gcp
|
|
gcp:
|
|
project: my-project
|
|
# network: default # VPC network name
|
|
# networkTag: proxy-operator # firewall tag on created instances
|
|
# diskSizeGb: 10
|
|
# auth: Application Default Credentials (workload identity
|
|
# in-cluster, gcloud ADC locally). No key-file plumbing.
|
|
- name: kvm-home
|
|
type: kvm
|
|
kvm:
|
|
# SSH to each host with the libvirt unix socket forwarded through the
|
|
# connection; nothing to open on the hypervisors beyond sshd. Both
|
|
# files come from the optional kvm-ssh Secret mounted by
|
|
# config/manager/manager.yaml — see docs/kvm-provider.md.
|
|
sshKeyFile: /etc/proxy-operator/kvm/id_ed25519
|
|
knownHostsFile: /etc/proxy-operator/kvm/known_hosts
|
|
# insecureIgnoreHostKey: true # skip host-key checks (not recommended)
|
|
# diskSizeGb: 10 # per-VM overlay; cloud-init grows the root fs into it
|
|
# installGuestAgent: false # skip the vendor-data script that installs qemu-guest-agent
|
|
# vendorData: | # extra vendor-data, carried beside the agent script as a
|
|
# #cloud-config # MIME multipart. The Proxy's own cloud-init is never rewritten.
|
|
# ...
|
|
images:
|
|
# placement.image names a key here (or is an https:// URL directly).
|
|
# Fetched into each host's pool on first use and cached there as
|
|
# base-<key>.qcow2; do not change an entry while Proxies use it.
|
|
debian-13:
|
|
url: https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2
|
|
# sha256: "<64 hex>" # optional; verified after download
|
|
# firmware: uefi # default; "bios" only for images with no EFI partition
|
|
# # (Debian 13 hangs in GRUB under BIOS headless: Debian bug #1111240)
|
|
ubuntu-24.04:
|
|
url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
|
machineTypes: # placement.machineType names a key here
|
|
small: {vcpus: 1, memoryMiB: 1024}
|
|
medium: {vcpus: 2, memoryMiB: 2048}
|
|
hosts: # placement.zone names a host; optional with one host
|
|
- name: beelink
|
|
address: beelink:22
|
|
user: novakj # must be in the libvirt group
|
|
bridge: br0 # to a LAN with DHCP; the VM's IP comes from there
|
|
# socket: /var/run/libvirt/libvirt-sock
|
|
# pool: default
|
|
# kvm2 takes the same block shape as kvm: an entry ports by changing type
|
|
# to kvm2 and the block name to kvm2. kvm2 does not adopt domains made by
|
|
# kvm, so delete those Proxies first. It caches its base images under
|
|
# base-<entry>-<key>.<format>, so it downloads its own copy rather than
|
|
# sharing kvm's; budget the pool space. See docs/kvm2-provider.md.
|
|
# - name: kvm2-home
|
|
# type: kvm2
|
|
# kvm2:
|
|
# sshKeyFile: /etc/proxy-operator/kvm/id_ed25519
|
|
# knownHostsFile: /etc/proxy-operator/kvm/known_hosts
|
|
# # insecureIgnoreHostKey: true
|
|
# # diskSizeGb: 10
|
|
# # installGuestAgent: false
|
|
# # vendorData: |
|
|
# # #cloud-config
|
|
# # ...
|
|
# images:
|
|
# debian-13:
|
|
# url: https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2
|
|
# # sha256: "<64 hex>"
|
|
# # firmware: uefi
|
|
# ubuntu-24.04:
|
|
# url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
|
# machineTypes:
|
|
# small: {vcpus: 1, memoryMiB: 1024}
|
|
# medium: {vcpus: 2, memoryMiB: 2048}
|
|
# hosts:
|
|
# - name: beelink
|
|
# address: beelink:22
|
|
# user: novakj
|
|
# bridge: br0
|
|
# # socket: /var/run/libvirt/libvirt-sock
|
|
# # pool: default
|