kvm2: name the cached base image after the provider entry (H4)
baseVolumeName was base-<key>.<format>, byte-identical to what the kvm provider builds in the same pool -- for URL images too, since kvm takes hex(sum[:8]) and kvm2 takes hex(sum[:])[:16], the same eight bytes -- with a completely separate lock table. The shipped config has a live kvm-home entry and a commented-out kvm2-home entry on the same host, the same key debian-13 and the same default pool, and the comment invites the operator to enable it. Both integration tests use key test-image. On beelink both live overlays back onto kvm's base-debian-13.qcow2. Since cachedImage accepts any 64-hex marker when no sha256 is configured and never re-hashes the base, that meant two entries pointing one key at different URLs silently shared a file; a concurrent cold fetch let one entry delete the other's half-uploaded base and boot a guest from the remains; and a sha256 edit deleted the base in place under live overlays, which libvirt permits without complaint. The name now carries the provider entry: base-<entry>-<key>.<format>. imageSource gained an Entry field and resolveImage takes the provider name, so every production call site follows from imageSource.volumeName(). That closes the cross-entry sharing outright, kvm<->kvm2 and kvm2<->kvm2 alike, and turns the per-Provider imageLocks from a latent bug into a correct one: with names unique per entry, two entries cannot contend for a file, so no process-wide lock table is needed. The entry name now reaches a file name, so New checks it against the existing kvmIdentifier rule, exported as provider.IsKVMIdentifier rather than duplicated. Provider names are otherwise validated for all types at once and only for being non-empty and unique, so a name with a slash would have produced a volume nothing could address. One companion fix, because the naming alone does not stop it: cachedImage's digest-mismatch branch deleted the marker and the base and refetched. It now returns ErrPermanent naming both digests and telling the operator what to do. That branch was the remaining path that unlinks a good file under running domains, and the contract already said "add a new key instead" -- the code was contradicting it. The no-marker and non-hex-marker branches keep deleting, since there the base is debris. Not fixed, and said so in the docs rather than left to look closed: editing images.<key>.url is still a silent no-op, since nothing records the URL; and cross-process fencing still needs the deferred atomic publish (upload to a temp name, verify, publish), which lands where H2 put the discard. Migration is a non-event. The base-debian-13.qcow2 on beelink is kvm's, kvm keeps using it untouched, and kvm2 downloads its own copy once per host. Nothing is replaced under a running guest; the pool needs room for both, which the prerequisites now say. The eviction recipe in the guide named base-debian-13.qcow2, which after this change is kvm's file -- following it would have deleted the wrong provider's base. It now greps for the prefix and deletes the marker first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -53,7 +53,9 @@ data:
|
||||
# pool: default
|
||||
# kvm2 takes the same block shape as kvm: an entry ports by changing
|
||||
# type to kvm2 and the block name to kvm2. kvm2 does not adopt domains
|
||||
# made by kvm, so delete those Proxies first. See docs/kvm2-provider.md.
|
||||
# made by kvm, so delete those Proxies first. It caches its base images
|
||||
# under base-<entry>-<key>.<format>, so it downloads its own copy rather
|
||||
# than sharing kvm's; budget the pool space. See docs/kvm2-provider.md.
|
||||
# - name: kvm2-home
|
||||
# type: kvm2
|
||||
# kvm2:
|
||||
|
||||
@@ -57,7 +57,9 @@ providers:
|
||||
# pool: default
|
||||
# kvm2 takes the same block shape as kvm: an entry ports by changing type
|
||||
# to kvm2 and the block name to kvm2. kvm2 does not adopt domains made by
|
||||
# kvm, so delete those Proxies first. See docs/kvm2-provider.md.
|
||||
# kvm, so delete those Proxies first. It caches its base images under
|
||||
# base-<entry>-<key>.<format>, so it downloads its own copy rather than
|
||||
# sharing kvm's; budget the pool space. See docs/kvm2-provider.md.
|
||||
# - name: kvm2-home
|
||||
# type: kvm2
|
||||
# kvm2:
|
||||
|
||||
@@ -74,11 +74,18 @@ field and that catalogue edits affect only new VMs.
|
||||
`sha256`, `format` qcow2 or raw, `firmware` uefi or bios) or a URL given
|
||||
directly in `placement.image`. The provider downloads the image into the
|
||||
host's storage pool on first use and caches it there as a base volume named
|
||||
`base-<key>.<format>` (for a URL: `base-<first 16 hex of sha256(url)>.<format>`),
|
||||
with a sidecar marker volume `base-<key>.<format>.sha256` holding the digest of
|
||||
the verified upload, written last. A base volume without a marker is
|
||||
`base-<entry>-<key>.<format>` (for a URL: `<key>` is the first 16 hex of
|
||||
sha256(url)), where `<entry>` is the provider entry's name, with a sidecar
|
||||
marker volume `base-<entry>-<key>.<format>.sha256` holding the digest of
|
||||
the verified upload, written last. The entry is in the name because the `kvm`
|
||||
provider builds `base-<key>.<format>` in the same pool from the same catalogue
|
||||
shape, and two entries pointing one key at different URLs would otherwise share
|
||||
a file; the cost is one copy per entry per host, and the entry's name is checked
|
||||
against `kvmIdentifier` at construction. A base volume without a marker is
|
||||
incomplete and is deleted and redone. When `sha256` is configured and differs
|
||||
from the marker's content, the image is re-downloaded. The download streams
|
||||
from the marker's content, the image is **not** replaced: that is `ErrPermanent`
|
||||
naming both digests, because overlays back onto the base file and libvirt
|
||||
unlinks it under running domains without complaint. The download streams
|
||||
from the URL through the operator into `StorageVolUpload` with a running
|
||||
sha256; nothing is buffered. `Content-Length` is required because the volume's
|
||||
capacity must be declared before the upload; a response without it is
|
||||
@@ -89,8 +96,9 @@ Proxy cannot cut it off. After a successful upload `StoragePoolRefresh` makes
|
||||
the file usable as a backing image.
|
||||
|
||||
Catalogue edit warning for the guide: overlays back onto the base file, so
|
||||
replacing or evicting a base volume while Proxies use it corrupts those VMs;
|
||||
add a new key instead.
|
||||
evicting a base volume while Proxies use it corrupts those VMs. A `sha256`
|
||||
change is refused for that reason; a `url` change is a silent no-op, since the
|
||||
cache is keyed by entry and key. Add a new key for either.
|
||||
|
||||
**Evidence:** kvm-provider.md "The image cache" and "Known gaps"; execution log
|
||||
of 2026-09-04 (raw qcow2 upload plus pool refresh accepted by libvirt 11.6;
|
||||
@@ -381,7 +389,7 @@ read `kvm2.hosts` and not `kvm.hosts`.
|
||||
| `diskSizeGb` | | 10 | not negative | overlay capacity, quota precheck |
|
||||
| `vendorData` | | | | seed `vendor-data` |
|
||||
| `installGuestAgent` | | true | | seed `vendor-data` |
|
||||
| `images.<key>` | | | key matches `kvmIdentifier` | volume name `base-<key>.<format>` |
|
||||
| `images.<key>` | | | key matches `kvmIdentifier` | volume name `base-<entry>-<key>.<format>` |
|
||||
| `images.<key>.url` | yes per entry | | http(s) URL | download |
|
||||
| `images.<key>.sha256` | | | 64 hex | marker comparison |
|
||||
| `images.<key>.format` | | qcow2 | qcow2 or raw | base volume XML, overlay backing format |
|
||||
|
||||
+40
-12
@@ -37,7 +37,8 @@ Per host, once:
|
||||
API. Nothing on the host filters the port; the provider does not use
|
||||
`spec.port` at all.
|
||||
4. **A storage pool** (default `default`, usually `/var/lib/libvirt/images`)
|
||||
with room for one base image per distro (a few hundred MB) plus the
|
||||
with room for one base image per distro and provider entry (a few
|
||||
hundred MB each) plus the
|
||||
per-VM overlays (`diskSizeGb` each, thin-provisioned). The pool must be
|
||||
running: a stopped pool fails the health check with `PoolNotActive` and
|
||||
fails `Create` permanently.
|
||||
@@ -202,13 +203,22 @@ host's pool:
|
||||
|
||||
| Object | Name | Lifetime |
|
||||
|---|---|---|
|
||||
| base image | `base-<key>.<format>` (for a URL image, `<key>` is the first 16 hex characters of the URL's sha256) | cached; survives Proxy deletion |
|
||||
| image marker | `base-<key>.<format>.sha256`, a 64-byte raw volume holding the digest | with the base image |
|
||||
| base image | `base-<entry>-<key>.<format>`, where `<entry>` is the provider entry's name (for a URL image, `<key>` is the first 16 hex characters of the URL's sha256) | cached; survives Proxy deletion |
|
||||
| image marker | `base-<entry>-<key>.<format>.sha256`, a 64-byte raw volume holding the digest | with the base image |
|
||||
| overlay disk | `proxy-abc….qcow2`, qcow2 backed by the base file, `diskSizeGb` capacity | with the Proxy |
|
||||
| seed ISO | `proxy-abc…-seed.iso`, raw | with the Proxy |
|
||||
| domain | `proxy-abc…` | with the Proxy |
|
||||
|
||||
The names are built in `internal/provider/kvm2/volume.go`.
|
||||
The names are built in `internal/provider/kvm2/volume.go`. The base image
|
||||
carries the **provider entry's name** because the file is shared by every
|
||||
Proxy on that host but must not be shared by two entries: the `kvm`
|
||||
provider builds `base-<key>.<format>` in the same pool from the same
|
||||
catalogue shape, and two kvm2 entries pointing one key at different URLs
|
||||
would otherwise agree on the name and disagree on the contents. The cost
|
||||
is one copy of each image per entry per host. For that reason the entry's
|
||||
name must itself be a usable file name — letters, digits, dots, dashes and
|
||||
underscores — which `New` checks at startup, since provider names are
|
||||
otherwise only required to be non-empty and unique.
|
||||
|
||||
The domain (`buildDomainXML` in `xml.go`) is a `q35` machine with a
|
||||
host-passthrough CPU, ACPI and APIC, memory in MiB and vcpus from the
|
||||
@@ -318,16 +328,27 @@ the rest find the marker.
|
||||
To evict manually:
|
||||
|
||||
```sh
|
||||
ssh novakj@beelink virsh -c qemu:///system vol-delete --pool default base-debian-13.qcow2.sha256
|
||||
ssh novakj@beelink virsh -c qemu:///system vol-delete --pool default base-debian-13.qcow2
|
||||
ssh novakj@beelink virsh -c qemu:///system vol-list --pool default | grep base-kvm2-home-
|
||||
ssh novakj@beelink virsh -c qemu:///system vol-delete --pool default base-kvm2-home-debian-13.qcow2.sha256
|
||||
ssh novakj@beelink virsh -c qemu:///system vol-delete --pool default base-kvm2-home-debian-13.qcow2
|
||||
```
|
||||
|
||||
**Warning: do not change an image entry's `url` or `sha256`, and do not
|
||||
evict its base volume, while Proxies using that image exist on the
|
||||
host.** The overlays back onto the base file; deleting or replacing it
|
||||
corrupts every VM that uses it. Delete those Proxies first, or add the
|
||||
new image under a new key and move Proxies over by changing their
|
||||
`placement.image` (which replaces the VM).
|
||||
Marker first: a base without its marker is refetched, a marker without its
|
||||
base is cleaned up, and neither leaves a Proxy pointing at nothing.
|
||||
|
||||
**Warning: do not evict a base volume while Proxies using that image exist
|
||||
on the host.** The overlays back onto the base file, and libvirt will
|
||||
unlink it under running domains without complaint; the guests keep running
|
||||
on the open file descriptor and fail at their next start. Delete those
|
||||
Proxies first.
|
||||
|
||||
Changing `sha256` on an entry whose base is already cached is **refused**,
|
||||
permanently and by name, rather than resolved by replacing the file — that
|
||||
replacement was the same corruption, just automated. Add the image under a
|
||||
new key and move Proxies over by changing their `placement.image` (which
|
||||
replaces the VM), or delete the Proxies and evict the base by hand.
|
||||
Changing `url` alone does nothing at all: the cache is keyed by entry and
|
||||
key, so the new URL is never fetched. Add a new key for that too.
|
||||
|
||||
## Quota: the pool-space precheck
|
||||
|
||||
@@ -381,6 +402,13 @@ A `kvm` domain left behind on a host is invisible to kvm2 forever; clean
|
||||
it up on the host with `virsh undefine --nvram <name>` and
|
||||
`virsh vol-delete` for its `<name>.qcow2` and `<name>-seed.iso`.
|
||||
|
||||
The base image cache is **not** shared between the two types, and that is
|
||||
deliberate: kvm names its copy `base-<key>.<format>` while kvm2 names its
|
||||
own `base-<entry>-<key>.<format>`, so kvm2 downloads the image once more
|
||||
per host rather than adopting a file another provider maintains. Nothing
|
||||
needs deleting; the `kvm` base stays where it is and keeps serving the
|
||||
overlays that back onto it. Budget the extra few hundred MB in the pool.
|
||||
|
||||
## Logging
|
||||
|
||||
The provider logs through controller-runtime's logger, named `kvm2`, with
|
||||
|
||||
@@ -5,7 +5,7 @@ Plan: [docs/plans/2026-09-17-1515-kvm2-review-fixes.md](../plans/2026-09-17-1515
|
||||
- [x] Step 1 — H1: a stalled write must not delay the teardown
|
||||
- [x] Step 2 — H2: a source error must not reach go-libvirt's stream sender
|
||||
- [x] Step 3 — H3: an unreadable address must not look like a missing one
|
||||
- [ ] Step 4 — H4: the base image cache must not be shared across provider entries
|
||||
- [x] Step 4 — H4: the base image cache must not be shared across provider entries
|
||||
|
||||
Remaining after those: the 48 mediums and lows in `review/REPORT.md`. Each gets its own
|
||||
checklist line here when it is planned, so this list stays a record of decided work rather
|
||||
@@ -283,3 +283,98 @@ fix's clothes.
|
||||
now returns Transient rather than a fake Provisioning — better, and self-correcting on the
|
||||
next poll. Returning NotFound one poll earlier is a two-line addition at the same spot, left
|
||||
for when L6 comes up so this commit stays one idea.
|
||||
|
||||
## Step 4 — H4: the base image cache must not be shared across provider entries
|
||||
|
||||
`baseVolumeName` was `base-<key>.<format>` — byte-identical to what
|
||||
`internal/provider/kvm/image.go` builds in the same pool, for URL images too (kvm takes
|
||||
`hex(sum[:8])`, kvm2 takes `hex(sum[:])[:16]`, the same eight bytes), with a completely
|
||||
separate lock table. The shipped config has a live `kvm-home` entry and a commented-out
|
||||
`kvm2-home` entry on the same host, the same key `debian-13` and the same default pool, and
|
||||
the comment invites the operator to enable it. Both integration tests use key `test-image`.
|
||||
On beelink both live overlays back onto kvm's `base-debian-13.qcow2`.
|
||||
|
||||
Since `cachedImage` accepts any 64-hex marker when no `sha256` is configured and never
|
||||
re-hashes the base, that meant: two entries pointing one key at different URLs silently share
|
||||
a file; a concurrent cold fetch lets one entry delete the other's half-uploaded base and boot
|
||||
a guest from the remains; and a `sha256` edit deletes the base in place under live overlays,
|
||||
which libvirt permits without complaint.
|
||||
|
||||
The name now carries the provider entry: `base-<entry>-<key>.<format>`. `imageSource` gained
|
||||
an `Entry` field and `resolveImage` takes the provider name, so the four production call
|
||||
sites all follow from `imageSource.volumeName()`. That closes the cross-entry sharing
|
||||
outright, kvm↔kvm2 and kvm2↔kvm2 alike, and it turns the existing per-`Provider` `imageLocks`
|
||||
from a latent bug into a correct one — with names unique per entry, no two entries can
|
||||
contend for a file, so a process-wide lock table is not needed.
|
||||
|
||||
**The entry name reaches a file name, so it now has to be a usable one.** Provider names are
|
||||
validated for all types at once and only for being non-empty and unique
|
||||
(`internal/provider/config.go`), so `New` checks its own against the existing `kvmIdentifier`
|
||||
rule, exported as `provider.IsKVMIdentifier` rather than duplicated. A name with a slash
|
||||
fails at startup instead of producing a volume nothing can address.
|
||||
|
||||
**One companion fix, because the naming alone does not stop it.** `cachedImage`'s
|
||||
digest-mismatch branch deleted the marker and the base and refetched. It now returns
|
||||
`ErrPermanent` naming both digests and telling the operator what to do. That branch was the
|
||||
remaining path that unlinks a good file under running domains, and the contract already said
|
||||
"add a new key instead" — the code was contradicting it. The no-marker and non-hex-marker
|
||||
branches keep deleting, since there the base is genuine debris.
|
||||
|
||||
### What this does NOT fix
|
||||
|
||||
Stated plainly rather than left to look closed:
|
||||
|
||||
- **Editing `images.<key>.url` is still a silent no-op.** The cache is keyed by entry and
|
||||
key, and nothing records the URL, so the new one is never fetched. Pre-existing, already
|
||||
documented as "add a new key"; the eviction section now says it outright.
|
||||
- **Cross-process fencing.** Two managers, or a manager plus a hand-run `virsh`, can still
|
||||
meet on one name. E4 refuted the rollout and leader-handover triggers for the shipped
|
||||
manifests, so this is narrow, and its fix is the deferred atomic publish — upload to a temp
|
||||
name, verify, publish — which lands where H2 put the discard.
|
||||
|
||||
### Migration
|
||||
|
||||
Nothing to do. The `base-debian-13.qcow2` on beelink is kvm's, kvm keeps using it untouched,
|
||||
and kvm2 downloads its own copy once per host when `kvm2-home` is enabled. No window in which
|
||||
anything is replaced under a running guest. The pool needs room for both, which the
|
||||
prerequisites now say.
|
||||
|
||||
### Tests
|
||||
|
||||
Mostly literals becoming helpers, plus a new `fx.baseName()` in `provider_test.go` for the
|
||||
three fixtures that asserted `"base-debian-13.qcow2"` by hand. The log and span assertions in
|
||||
`TestEnsureImage_downloadsAndWritesMarkerLast` keep asserting a full name rather than loosening
|
||||
to a prefix, so they still pin that the key stays readable in operator-facing output.
|
||||
|
||||
One test changed behaviour rather than spelling: `TestEnsureImage_sha256ChangeRedownloads`
|
||||
became `TestEnsureImage_sha256ChangeRefusesInsteadOfReplacing`, asserting Permanent, zero
|
||||
server hits, zero deletes, and both volumes untouched. Two added:
|
||||
`TestNew_rejectsUnsafeEntryName`, and a `TestVolumeNames` row for a second entry with the same
|
||||
key.
|
||||
|
||||
Both negative checks were run:
|
||||
|
||||
```bash
|
||||
# baseVolumeName reverted to base-<key>.<format>:
|
||||
go test -count=1 -run TestVolumeNames ./internal/provider/kvm2/
|
||||
# volume_test.go:135: got "base-debian-13.qcow2", want "base-kvm2-lab-debian-13.qcow2"
|
||||
|
||||
# the digest branch reverted to delete-and-refetch:
|
||||
go test -count=1 -run TestEnsureImage_sha256ChangeRefuses ./internal/provider/kvm2/
|
||||
# image_test.go:412: ensureImage() error = <nil>, want ErrPermanent telling the operator what to do
|
||||
```
|
||||
|
||||
`go vet -tags=integration` was run too, since `kvm2_integration_test.go:484` builds the base
|
||||
name and only compiles under that tag.
|
||||
|
||||
### Worth noting
|
||||
|
||||
**The integration test has still never run on beelink.** Step 11 is unchecked in
|
||||
`docs/plans-executions/2026-09-16-1333-kvm2-provider.md`, every `base-*` on the host predates
|
||||
kvm2, and both live domains carry the kvm v1 metadata namespace. After this change it is also
|
||||
the first end-to-end exercise of the new name, and it will do a real cold download under
|
||||
`base-kvm2-integration-test-image.qcow2` beside kvm's untouched `base-test-image.qcow2`.
|
||||
|
||||
**Docs touched beyond the obvious:** the eviction recipe named `base-debian-13.qcow2`, which
|
||||
after this change is *kvm's* file — following it would have deleted the wrong provider's base.
|
||||
It now greps for the prefix and deletes the marker first.
|
||||
|
||||
@@ -108,11 +108,17 @@ var sha256Hex = regexp.MustCompile(`^[0-9a-fA-F]{64}$`)
|
||||
// kvmIdentifier constrains the config names that are pasted into
|
||||
// identifiers elsewhere: a host name goes into the providerID
|
||||
// hosts/<name>/domains/<vm>, which is split on "/" to parse, and an image
|
||||
// key becomes the file name of the cached base volume. A slash in either
|
||||
// produces an ID or a path that never resolves again, so the VM leaks with
|
||||
// nothing able to address it.
|
||||
// key becomes part of the file name of the cached base volume. A slash in
|
||||
// either produces an ID or a path that never resolves again, so the VM
|
||||
// leaks with nothing able to address it.
|
||||
var kvmIdentifier = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]*$`)
|
||||
|
||||
// IsKVMIdentifier reports whether s is safe to paste into a providerID or a
|
||||
// volume name. The provider entry's own name reaches a volume name too, and
|
||||
// that name is validated for every provider type at once (only non-empty
|
||||
// and unique), so the kvm2 provider checks its own at construction.
|
||||
func IsKVMIdentifier(s string) bool { return kvmIdentifier.MatchString(s) }
|
||||
|
||||
func validateKVM(prefix string, c *KVMConfig) error {
|
||||
return validateKVMBlock(prefix, "kvm", c)
|
||||
}
|
||||
@@ -137,7 +143,7 @@ func validateKVMBlock(prefix, block string, c *KVMConfig) error {
|
||||
p := fmt.Sprintf("%s: %s.images[%q]", prefix, block, name)
|
||||
if !kvmIdentifier.MatchString(name) {
|
||||
return fmt.Errorf("%s: key must be letters, digits, dots, dashes or underscores"+
|
||||
" — it becomes the cached volume name base-<key>.<format>", p)
|
||||
" — it becomes part of the cached base volume's name", p)
|
||||
}
|
||||
if !IsHTTPURL(img.URL) {
|
||||
return fmt.Errorf("%s.url must be an http(s) URL (got %q)", p, img.URL)
|
||||
|
||||
@@ -38,7 +38,7 @@ func (p *Provider) Create(ctx context.Context, req provider.CreateRequest) (stri
|
||||
if !ok {
|
||||
return "", p.wrapErr("create", "", permanentf("placement.machineType %q is not in kvm2.machineTypes", req.Placement.MachineType))
|
||||
}
|
||||
src, err := resolveImage(p.cfg, req.Placement.Image)
|
||||
src, err := resolveImage(p.name, p.cfg, req.Placement.Image)
|
||||
if err != nil {
|
||||
return "", p.wrapErr("create", "", err)
|
||||
}
|
||||
|
||||
@@ -61,13 +61,13 @@ func TestCreate_resumesAfterEachPartialStep(t *testing.T) {
|
||||
{
|
||||
name: "base without marker is refetched",
|
||||
setup: func(fx *fixture) {
|
||||
fx.putVol("base-debian-13.qcow2", []byte("truncated"))
|
||||
fx.putVol(fx.baseName(), []byte("truncated"))
|
||||
},
|
||||
check: func(t *testing.T, fx *fixture) {
|
||||
if got := fx.fake.vol("base-debian-13.qcow2"); got == nil || !bytes.Equal(got.content, fx.image) {
|
||||
if got := fx.fake.vol(fx.baseName()); got == nil || !bytes.Equal(got.content, fx.image) {
|
||||
t.Errorf("base volume not refetched")
|
||||
}
|
||||
if fx.fake.vol("base-debian-13.qcow2.sha256") == nil {
|
||||
if fx.fake.vol(fx.baseName()+".sha256") == nil {
|
||||
t.Errorf("marker not written")
|
||||
}
|
||||
},
|
||||
|
||||
@@ -64,7 +64,8 @@ func TestDelete_removesOverlayAndSeed(t *testing.T) {
|
||||
t.Errorf("volume %s still present after Delete", name)
|
||||
}
|
||||
}
|
||||
for _, name := range []string{"base-debian-13.qcow2", "base-debian-13.qcow2.sha256"} {
|
||||
base := "base-" + testProvider + "-debian-13.qcow2"
|
||||
for _, name := range []string{base, base + ".sha256"} {
|
||||
if fx.fake.vol(name) == nil {
|
||||
t.Errorf("shared base volume %s removed by Delete", name)
|
||||
}
|
||||
|
||||
@@ -42,22 +42,25 @@ var hexDigest = regexp.MustCompile(`^[0-9a-f]{64}$`)
|
||||
// catalogue entry, or a URL given directly with the catalogue defaults, plus
|
||||
// the key its base volume is named after.
|
||||
type imageSource struct {
|
||||
Key string
|
||||
Entry string
|
||||
Key string
|
||||
provider.KVMImage
|
||||
}
|
||||
|
||||
func (s imageSource) volumeName() string { return baseVolumeName(s.Key, s.Format) }
|
||||
func (s imageSource) volumeName() string { return baseVolumeName(s.Entry, s.Key, s.Format) }
|
||||
func (s imageSource) markerName() string { return markerVolumeName(s.volumeName()) }
|
||||
|
||||
// resolveImage maps placement.image to a source. cfg is already defaulted,
|
||||
// so a catalogue entry carries its format and firmware; a URL gets qcow2
|
||||
// and uefi.
|
||||
func resolveImage(cfg provider.KVMConfig, image string) (imageSource, error) {
|
||||
// and uefi. entry is the provider entry's name, which the base volume is
|
||||
// named after so two entries never share one file.
|
||||
func resolveImage(entry string, cfg provider.KVMConfig, image string) (imageSource, error) {
|
||||
if img, ok := cfg.Images[image]; ok {
|
||||
return imageSource{Key: image, KVMImage: img}, nil
|
||||
return imageSource{Entry: entry, Key: image, KVMImage: img}, nil
|
||||
}
|
||||
if provider.IsHTTPURL(image) {
|
||||
return imageSource{
|
||||
Entry: entry,
|
||||
Key: urlImageKey(image),
|
||||
KVMImage: provider.KVMImage{URL: image, Format: provider.KVMFormatQcow2, Firmware: provider.KVMFirmwareUEFI},
|
||||
}, nil
|
||||
@@ -127,7 +130,11 @@ func cachedImage(ctx context.Context, log logr.Logger, pool libvirt.StoragePool,
|
||||
case !hexDigest.MatchString(digest):
|
||||
log.V(1).Info("base image marker holds no digest; refetching")
|
||||
case src.SHA256 != "" && !strings.EqualFold(digest, src.SHA256):
|
||||
log.V(1).Info("base image digest differs from the configured sha256; refetching", "cached", digest, "configured", strings.ToLower(src.SHA256))
|
||||
// Refusing rather than replacing: every overlay on this host names
|
||||
// the base as its backing file, and libvirt will unlink it under
|
||||
// running domains without a word.
|
||||
return "", permanentf("base image %s is cached with sha256 %s but %s is configured; add the image under a new key and move Proxies to it, or delete the Proxies using it and evict the base by hand",
|
||||
name, digest, strings.ToLower(src.SHA256))
|
||||
default:
|
||||
base, err := be.StorageVolLookupByName(ctx, pool, name)
|
||||
switch {
|
||||
|
||||
@@ -100,7 +100,7 @@ func (fx *imageFixture) sameConnection() {
|
||||
|
||||
func (fx *imageFixture) src() imageSource {
|
||||
fx.t.Helper()
|
||||
src, err := resolveImage(fx.p.cfg, imageKey)
|
||||
src, err := resolveImage(fx.p.name, fx.p.cfg, imageKey)
|
||||
if err != nil {
|
||||
fx.t.Fatalf("resolveImage(%q) error = %v", imageKey, err)
|
||||
}
|
||||
@@ -180,20 +180,20 @@ func TestResolveImage_table(t *testing.T) {
|
||||
{
|
||||
name: "catalogue key with defaults applied",
|
||||
image: "debian-13",
|
||||
want: imageSource{Key: "debian-13", KVMImage: provider.KVMImage{URL: "https://cloud.debian.org/debian-13.qcow2", SHA256: "ABCD", Format: "qcow2", Firmware: "uefi"}},
|
||||
wantVolume: "base-debian-13.qcow2",
|
||||
want: imageSource{Entry: testProvider, Key: "debian-13", KVMImage: provider.KVMImage{URL: "https://cloud.debian.org/debian-13.qcow2", SHA256: "ABCD", Format: "qcow2", Firmware: "uefi"}},
|
||||
wantVolume: "base-" + testProvider + "-debian-13.qcow2",
|
||||
},
|
||||
{
|
||||
name: "catalogue key with explicit format and firmware",
|
||||
image: "alpine",
|
||||
want: imageSource{Key: "alpine", KVMImage: provider.KVMImage{URL: "https://alpine.example/alpine.img", Format: "raw", Firmware: "bios"}},
|
||||
wantVolume: "base-alpine.raw",
|
||||
want: imageSource{Entry: testProvider, Key: "alpine", KVMImage: provider.KVMImage{URL: "https://alpine.example/alpine.img", Format: "raw", Firmware: "bios"}},
|
||||
wantVolume: "base-" + testProvider + "-alpine.raw",
|
||||
},
|
||||
{
|
||||
name: "http URL used directly with defaults",
|
||||
image: rawURL,
|
||||
want: imageSource{Key: urlImageKey(rawURL), KVMImage: provider.KVMImage{URL: rawURL, Format: "qcow2", Firmware: "uefi"}},
|
||||
wantVolume: "base-" + urlImageKey(rawURL) + ".qcow2",
|
||||
want: imageSource{Entry: testProvider, Key: urlImageKey(rawURL), KVMImage: provider.KVMImage{URL: rawURL, Format: "qcow2", Firmware: "uefi"}},
|
||||
wantVolume: "base-" + testProvider + "-" + urlImageKey(rawURL) + ".qcow2",
|
||||
},
|
||||
{
|
||||
name: "unknown key is permanent and names the value",
|
||||
@@ -214,7 +214,7 @@ func TestResolveImage_table(t *testing.T) {
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
got, err := resolveImage(cfg, tc.image)
|
||||
got, err := resolveImage(testProvider, cfg, tc.image)
|
||||
if tc.wantErr != "" {
|
||||
if err == nil || !errors.Is(err, provider.ErrPermanent) || !strings.Contains(err.Error(), tc.wantErr) {
|
||||
t.Fatalf("resolveImage(%q) = %+v, %v; want ErrPermanent containing %q", tc.image, got, err, tc.wantErr)
|
||||
@@ -278,7 +278,7 @@ func TestEnsureImage_downloadsAndWritesMarkerLast(t *testing.T) {
|
||||
t.Errorf("got %d lines with %s, want 1:\n%s", len(lines), want, strings.Join(fx.logs.all(), "\n"))
|
||||
continue
|
||||
}
|
||||
for _, kv := range []string{`"provider"="kvm2-home"`, `"host"="beelink"`, `"volume"="base-debian-13.qcow2"`, `"bytes"=` + strconv.Itoa(len(body)), `"traceID"=`} {
|
||||
for _, kv := range []string{`"provider"="kvm2-home"`, `"host"="beelink"`, `"volume"="base-` + testProvider + `-debian-13.qcow2"`, `"bytes"=` + strconv.Itoa(len(body)), `"traceID"=`} {
|
||||
if !strings.Contains(lines[0], kv) {
|
||||
t.Errorf("line lacks %s: %s", kv, lines[0])
|
||||
}
|
||||
@@ -295,7 +295,7 @@ func TestEnsureImage_downloadsAndWritesMarkerLast(t *testing.T) {
|
||||
attrs := spans[0].Attributes()
|
||||
for _, want := range []attribute.KeyValue{
|
||||
attribute.String("kvm2.host", "beelink"),
|
||||
attribute.String("kvm2.volume", "base-debian-13.qcow2"),
|
||||
attribute.String("kvm2.volume", "base-"+testProvider+"-debian-13.qcow2"),
|
||||
attribute.String("kvm2.url_host", strings.TrimPrefix(fx.srv.URL, "http://")),
|
||||
attribute.Int64("kvm2.bytes", int64(len(body))),
|
||||
attribute.Bool("kvm2.sha256_configured", false),
|
||||
@@ -391,7 +391,13 @@ func TestEnsureImage_missingMarkerRedownloads(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnsureImage_sha256ChangeRedownloads(t *testing.T) {
|
||||
// TestEnsureImage_sha256ChangeRefusesInsteadOfReplacing pins the rule the
|
||||
// contract already stated and the code contradicted. Every overlay on the
|
||||
// host names the base as its backing file, and libvirt unlinks it under
|
||||
// running domains without complaint, so a digest that no longer matches is
|
||||
// for the operator to resolve, not for the next Create to resolve by
|
||||
// deleting.
|
||||
func TestEnsureImage_sha256ChangeRefusesInsteadOfReplacing(t *testing.T) {
|
||||
t.Parallel()
|
||||
body := randomBody(t, 8192)
|
||||
fx := newImageFixture(t, serveBytes(body))
|
||||
@@ -401,25 +407,21 @@ func TestEnsureImage_sha256ChangeRedownloads(t *testing.T) {
|
||||
fx.putVol(src.volumeName(), []byte("the old image"))
|
||||
fx.putVol(src.markerName(), []byte(digestA))
|
||||
|
||||
if _, err := fx.ensure(fx.ctx, src); err != nil {
|
||||
t.Fatalf("ensureImage() error = %v", err)
|
||||
_, err := fx.ensure(fx.ctx, src)
|
||||
if !errors.Is(err, provider.ErrPermanent) || !strings.Contains(err.Error(), "add the image under a new key") {
|
||||
t.Fatalf("ensureImage() error = %v, want ErrPermanent telling the operator what to do", err)
|
||||
}
|
||||
if fx.hits.Load() != 1 {
|
||||
t.Errorf("server hits = %d, want 1", fx.hits.Load())
|
||||
if fx.hits.Load() != 0 {
|
||||
t.Errorf("server hits = %d, want 0: nothing should be refetched", fx.hits.Load())
|
||||
}
|
||||
if n := fx.pooled.callCount("StorageVolDelete"); n != 2 {
|
||||
t.Errorf("StorageVolDelete called %d times, want 2 (marker and base)", n)
|
||||
if n := fx.pooled.callCount("StorageVolDelete"); n != 0 {
|
||||
t.Errorf("StorageVolDelete called %d times, want 0: the base backs live overlays", n)
|
||||
}
|
||||
base := fx.pooled.vol(src.volumeName())
|
||||
if base == nil || !bytes.Equal(base.content, body) {
|
||||
t.Errorf("base volume was not replaced by the download")
|
||||
if base := fx.pooled.vol(src.volumeName()); base == nil || !bytes.Equal(base.content, []byte("the old image")) {
|
||||
t.Errorf("base volume = %+v, want the cached one untouched", base)
|
||||
}
|
||||
marker := fx.pooled.vol(src.markerName())
|
||||
if marker == nil || string(marker.content) != digestOf(body) {
|
||||
t.Errorf("marker = %+v, want the new digest %s", marker, digestOf(body))
|
||||
}
|
||||
if lines := fx.logs.matching("differs from the configured sha256"); len(lines) != 1 {
|
||||
t.Errorf("got %d refetch lines, want 1", len(lines))
|
||||
if marker := fx.pooled.vol(src.markerName()); marker == nil || string(marker.content) != digestA {
|
||||
t.Errorf("marker = %+v, want the cached digest untouched", marker)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -74,6 +74,9 @@ var _ provider.Provider = (*Provider)(nil)
|
||||
// files are only checked for existence, and a missing one is a warning, not
|
||||
// an error, because the Secret mount is optional and may land later.
|
||||
func New(ctx context.Context, pc provider.ProviderConfig) (provider.Provider, error) {
|
||||
if !provider.IsKVMIdentifier(pc.Name) {
|
||||
return nil, permanentf("provider name %q: kvm2 names its cached base images after the entry, so the name must be letters, digits, dots, dashes or underscores", pc.Name)
|
||||
}
|
||||
cfg := provider.KVMConfig{}
|
||||
if pc.KVM2 != nil {
|
||||
cfg = *pc.KVM2
|
||||
|
||||
@@ -481,7 +481,7 @@ func TestKVM2_lifecycle(t *testing.T) {
|
||||
// 10. The pool keeps the shared base image and its marker, and holds
|
||||
// no per-VM volume.
|
||||
start = time.Now()
|
||||
base := baseVolumeName(integrationImage, provider.KVMFormatQcow2)
|
||||
base := baseVolumeName(integrationProvider, integrationImage, provider.KVMFormatQcow2)
|
||||
marker := markerVolumeName(base)
|
||||
overlay, seed := overlayVolumeName(name), seedVolumeName(name)
|
||||
present := poolVolumes(ctx, t, p, host, base, marker, overlay, seed)
|
||||
|
||||
@@ -110,6 +110,18 @@ func TestNew_nilBlockDoesNotPanic(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestNew_rejectsUnsafeEntryName guards the one config value that reaches a
|
||||
// volume name without passing the kvm validator: the provider entry's own
|
||||
// name, which is checked only for being non-empty and unique.
|
||||
func TestNew_rejectsUnsafeEntryName(t *testing.T) {
|
||||
t.Parallel()
|
||||
ctx, _ := captureContext()
|
||||
_, err := New(ctx, provider.ProviderConfig{Name: "kvm2/home", Type: "kvm2"})
|
||||
if !errors.Is(err, provider.ErrPermanent) || !strings.Contains(err.Error(), "kvm2/home") {
|
||||
t.Fatalf("New() error = %v, want ErrPermanent naming the entry", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHost_table(t *testing.T) {
|
||||
t.Parallel()
|
||||
tests := []struct {
|
||||
|
||||
@@ -129,10 +129,21 @@ func (fx *fixture) putVol(name string, content []byte) *fakeVol {
|
||||
return v
|
||||
}
|
||||
|
||||
// baseName is the cached base image's volume name for this fixture's
|
||||
// provider entry and image key.
|
||||
func (fx *fixture) baseName() string {
|
||||
fx.t.Helper()
|
||||
src, err := resolveImage(fx.p.name, fx.p.cfg, imageKey)
|
||||
if err != nil {
|
||||
fx.t.Fatal(err)
|
||||
}
|
||||
return src.volumeName()
|
||||
}
|
||||
|
||||
// cacheImage places a complete base image and its marker, as a previous
|
||||
// Create on the host would have.
|
||||
func (fx *fixture) cacheImage() {
|
||||
src, err := resolveImage(fx.p.cfg, imageKey)
|
||||
src, err := resolveImage(fx.p.name, fx.p.cfg, imageKey)
|
||||
if err != nil {
|
||||
fx.t.Fatal(err)
|
||||
}
|
||||
@@ -406,7 +417,7 @@ func TestLogging_v0SilentV1OutcomeV2Detail(t *testing.T) {
|
||||
if len(resolved) != 1 {
|
||||
t.Fatalf("V(2) lines matching resolved = %d, want 1", len(resolved))
|
||||
}
|
||||
for _, key := range []string{`"cloudInitBytes"=`, `"image"="base-debian-13.qcow2"`, `"firmware"="uefi"`, `"vcpus"=1`, `"memoryMiB"=1024`, `"machineType"="small"`, `"bridge"="br0"`, `"pool"="default"`} {
|
||||
for _, key := range []string{`"cloudInitBytes"=`, `"image"="base-` + testProvider + `-debian-13.qcow2"`, `"firmware"="uefi"`, `"vcpus"=1`, `"memoryMiB"=1024`, `"machineType"="small"`, `"bridge"="br0"`, `"pool"="default"`} {
|
||||
if !strings.Contains(resolved[0], key) {
|
||||
t.Errorf("resolved line lacks %s: %s", key, resolved[0])
|
||||
}
|
||||
|
||||
@@ -14,7 +14,14 @@ import (
|
||||
func overlayVolumeName(name string) string { return name + ".qcow2" }
|
||||
func seedVolumeName(name string) string { return name + "-seed.iso" }
|
||||
|
||||
func baseVolumeName(key, format string) string { return "base-" + key + "." + format }
|
||||
// baseVolumeName carries the provider entry because the base is shared by
|
||||
// every Proxy on a host but must not be shared by two entries: the kvm
|
||||
// provider builds base-<key>.<format> in the same pool from the same
|
||||
// catalogue shape, and two kvm2 entries pointing one key at different URLs
|
||||
// would otherwise agree on the name and disagree on the contents.
|
||||
func baseVolumeName(entry, key, format string) string {
|
||||
return "base-" + entry + "-" + key + "." + format
|
||||
}
|
||||
func markerVolumeName(baseName string) string { return baseName + ".sha256" }
|
||||
|
||||
// urlImageKey stands in for the catalogue key of an image given as a URL:
|
||||
|
||||
@@ -119,10 +119,14 @@ func TestVolumeNames(t *testing.T) {
|
||||
}{
|
||||
{name: "overlay", got: overlayVolumeName("proxy-abc"), want: "proxy-abc.qcow2"},
|
||||
{name: "seed", got: seedVolumeName("proxy-abc"), want: "proxy-abc-seed.iso"},
|
||||
{name: "base qcow2", got: baseVolumeName("debian-13", provider.KVMFormatQcow2), want: "base-debian-13.qcow2"},
|
||||
{name: "base raw", got: baseVolumeName("alpine", provider.KVMFormatRaw), want: "base-alpine.raw"},
|
||||
{name: "base qcow2", got: baseVolumeName("kvm2-home", "debian-13", provider.KVMFormatQcow2), want: "base-kvm2-home-debian-13.qcow2"},
|
||||
{name: "base raw", got: baseVolumeName("kvm2-home", "alpine", provider.KVMFormatRaw), want: "base-kvm2-home-alpine.raw"},
|
||||
{name: "marker", got: markerVolumeName("base-debian-13.qcow2"), want: "base-debian-13.qcow2.sha256"},
|
||||
{name: "marker of url base", got: markerVolumeName(baseVolumeName(urlImageKey("https://x/y.img"), "qcow2")), want: "base-" + urlImageKey("https://x/y.img") + ".qcow2.sha256"},
|
||||
{name: "marker of url base", got: markerVolumeName(baseVolumeName("kvm2-home", urlImageKey("https://x/y.img"), "qcow2")), want: "base-kvm2-home-" + urlImageKey("https://x/y.img") + ".qcow2.sha256"},
|
||||
// Two entries pointing one key at different images must not agree
|
||||
// on a file name, and neither may collide with the kvm provider's
|
||||
// base-<key>.<format> in the same pool.
|
||||
{name: "another entry, same key", got: baseVolumeName("kvm2-lab", "debian-13", provider.KVMFormatQcow2), want: "base-kvm2-lab-debian-13.qcow2"},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
|
||||
Reference in New Issue
Block a user