2026-09-24 13:12:45 +02:00
28 changed files with 882 additions and 201 deletions
@@ -1,5 +1,24 @@
 # Changelog
 ## 2026-09-24 10:28 UTC — proxmox provider: reviewed, fixed, proven on a real Proxmox VE 9.2 host
 The `proxmox` provider (VMs cloned from a template on Proxmox VE through its REST API)
 finished its last phase: a two-pass review, a real-host run on pv1-1, and captures of the
 three API answers the design had only guessed.
 - Delete now removes the seed ISO **before** destroying the VM. Before, a seed delete that
  failed after the destroy was never retried (the next Get is NotFound), so the ISO leaked.
  A 403 for `Datastore.Allocate` now fails Delete on every path instead of being skipped.
 - Create skips a probe VMID held by someone else's VM at once, instead of waiting up to 10
  minutes on that VM's lock on every retry.
 - The captures corrected two assumptions: a missing ISO storage is a 400 (the health check
  now says `StorageNotFound`, not `APIError`), and a hibernated VM can be destroyed (Delete
  destroys ours instead of latching the Proxy).
 - `TestProxmox_lifecycle` and `TestProxmoxCheck_realHost` passed on PVE 9.2.20: Create 6.2 s,
  an address 50 s later. A kind pass took a sample Proxy to Ready and back.
 - Review record: `docs/reviews/2026-09-23-proxmox-provider-review/REPORT.md`. Timings:
  `docs/testing.md`. Code: `internal/provider/proxmox/{delete,create,check,errors}.go`.
 ## 2026-09-18 07:52 UTC — docs/operating.md, starting with the libvirt-RPC log query
 New home for day-to-day helpers against a running operator, seeded with the `jq` query
@@ -192,7 +192,7 @@ can be two differently-configured instances of the same `gcp` type (see
 | `gcp` | A Compute Engine VM with an ephemeral external IP | `project` (required), `network`, `networkTag`, `diskSizeGb` | The real egress fleet. Deliberately uses only four API calls (Insert / Get / Delete / AggregatedList), all fire-and-forget: `Create` returns as soon as the operation is submitted and the reconciler discovers progress by polling `Get`. Auth is Application Default Credentials — workload identity in-cluster, `gcloud` ADC locally; no key-file plumbing. |
 | `kvm` | A libvirt VM on one of your own hypervisors, reached over SSH | `sshKeyFile`, `knownHostsFile` (unless `insecureIgnoreHostKey`), `machineTypes`, `hosts` (required); `images`, `diskSizeGb`, `vendorData`, `installGuestAgent` | Home-lab or on-prem egress. The base image is downloaded into each host's pool on first use; `Create` waits for the VM to start. See [docs/kvm-provider.md](docs/kvm-provider.md). |
 | `kvm2` | The same: a libvirt VM on one of your own hypervisors, reached over SSH | the same block shape as `kvm`, under `kvm2:` | The second-generation libvirt provider, built from the provider contract. Adds a child span per libvirt RPC, a pool-space precheck reported as quota (the reconciler waits instead of failing), SSH credentials re-read on every dial, and image downloads and health checks on connections of their own. It does not adopt `kvm`'s VMs. See [docs/kvm2-provider.md](docs/kvm2-provider.md). |
-| `proxmox` | A VM on a Proxmox VE cluster: a full clone of a template you prepare, reached through the Proxmox REST API with an API token | `url`, `tokenFile`, `nodes`, `isoStorage`, `templates`, `machineTypes` (required); `caFile` or `insecureSkipTLSVerify`, `storage`, `bridge`, `vlanTag`, `vmidRange`, `requestTimeoutSeconds`, `vendorData`, `installGuestAgent` | Home-lab or on-prem egress without SSH. Cloud-init goes on a seed ISO uploaded through the API; the address comes from the QEMU guest agent, so the proxy network needs a route to a package mirror unless the template ships the agent. `Create` waits for the VM to run. The token is read on every request, from the optional `proxmox-token` Secret (`make proxmox-token-secret`). Built from the provider contract; not yet run against a real host end to end. See [docs/proxmox-provider.md](docs/proxmox-provider.md). |
+| `proxmox` | A VM on a Proxmox VE cluster: a full clone of a template you prepare, reached through the Proxmox REST API with an API token | `url`, `tokenFile`, `nodes`, `isoStorage`, `templates`, `machineTypes` (required); `caFile` or `insecureSkipTLSVerify`, `storage`, `bridge`, `vlanTag`, `vmidRange`, `requestTimeoutSeconds`, `vendorData`, `installGuestAgent` | Home-lab or on-prem egress without SSH. Cloud-init goes on a seed ISO uploaded through the API; the address comes from the QEMU guest agent, so the proxy network needs a route to a package mirror unless the template ships the agent. `Create` waits for the VM to run. The token is read on every request, from the optional `proxmox-token` Secret (`make proxmox-token-secret`). Built from the provider contract and run end to end against Proxmox VE 9.2.20. See [docs/proxmox-provider.md](docs/proxmox-provider.md). |
 ### Adding a provider
@@ -226,8 +226,9 @@ prov.Get    ──► /cluster/resources (absent = NotFound; lock or node unknow
                  + agent network-get-interfaces (first routable IPv4; agent not
                    up yet = Provisioning)
 prov.Delete ──► resources + config: gone, not ours or unmarked = leave it;
-                  hibernated = Permanent; stop → destroy (purge) → delete the seed
+                  stop (unless stopped or hibernated) → delete the seed on the
-                  on the VM's node (or on every node when no VM of ours was left)
+                  VM's node → destroy (purge); with no VM of ours left, the seed
                  is deleted on every node
 prov.ListByTag ─► one /cluster/resources for the whole cluster, one config read per
                    candidate (tag or proxy- name); the description's mark decides
 Check       ──► per node: /version, node status, isoStorage status, templates in
@@ -14,7 +14,7 @@ Pairs with [docs/plans/2026-09-18-1502-proxmox-provider.md](../plans/2026-09-18-
 - [x] Phase 7 — Delete, Checker, CRD comment
 - [x] Phase 8 — Contract and observability sweep, integration test, testing.md
 - [x] Phase 9 — Deployment and documentation
- [ ] Phase 10 — Review, real-host validation, changelog, MR
+- [x] Phase 10 — Review, real-host validation, changelog, MR
 ## Phase 0 — Plan file, branch, host recon, fixtures
@@ -1298,3 +1298,144 @@ Worth noting:
  guide's "Not yet proven end to end" gap and the README's matching
  clause come out once they pass. Then the CHANGELOG entry after the user
  confirms, and the MR.
 ## Phase 10 — Review, real-host validation, changelog, MR
 Where the plan's Phase 10 text and the spec disagreed, the spec won, as in
 every phase since Phase 1. The host is pv1-1 at `https://192.168.0.8:8006`,
 not 192.168.2.8, the template is 9001 and the ISO storage `proxy-seeds`.
 **Review.** Two read-only passes over `git diff main...HEAD`: the
 `go-operator-reviewer` agent, briefed with the spec and the contract
 (P1 to P6, S1 to S3), then `/code-review high` on the branch (CR1 to
 CR10). Of the 19 findings (two found by both passes), 9 were fixed, 3
 accepted as documented limits and 7 rejected with a reason, all recorded in
 [docs/reviews/2026-09-23-proxmox-provider-review/REPORT.md](../reviews/2026-09-23-proxmox-provider-review/REPORT.md).
 The one that mattered was CR1/P1. Delete destroyed the VM and then deleted
 the seed, so a failed seed delete left a seed nothing would ever remove:
 the next `Get` is NotFound, the finalizer goes, and GC lists only VMs.
 Delete now removes the seed between the stop and the destroy, so the VM
 `Get` looks at goes last. A 403 on the every-node path now fails instead
 of being skipped. P2 (CR3) was the other code change: a probe VMID whose
 listed name is someone else's is skipped at once, instead of waiting up to
 the clone budget on its lock on every retry.
 **Lint.** `make lint` still panics inside staticcheck on Go 1.27, so, as
 in Phases 8 and 9:
 ```bash
 bin/golangci-lint run --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
 bin/golangci-lint run --build-tags integration --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
 ```
 Each run was diffed against the pre-fix baseline by file and message,
 with line numbers and occurrence counts stripped. Three new goconst hits
 came in along the way, and all three were fixed rather than accepted. The
 P2 test's lock value made `"backup"`, then `"migrate"`, a repeated
 literal; it now uses `"snapshot-delete"`. The fake's missing-storage error
 repeated `"Parameter verification failed."`; it is now built from the raw
 captured body. Final counts: goconst 224 (231 with the tag), logcheck 12,
 modernize 2, unparam 3, gocyclo 1.
 **Real host.** The user ran the host checklist on pv1-1 and then:
 ```bash
 export PROXMOX_TEST_URL=https://192.168.0.8:8006
 export PROXMOX_TEST_TOKEN_FILE=$HOME/.config/proxmox-token
 export PROXMOX_TEST_CA_FILE=$HOME/.config/proxmox-ca.pem
 export PROXMOX_TEST_NODE=pv1-1
 export PROXMOX_TEST_TEMPLATE_VMID=9001
 export PROXMOX_TEST_ISO_STORAGE=proxy-seeds
 export PROXMOX_TEST_VLAN_TAG=4
 make test-integration-proxmox
 # --- PASS: TestProxmox_lifecycle (59.68s)
 # --- PASS: TestProxmoxCheck_realHost (0.13s)
 ```
 It ran the review fixes, uncommitted, on the first try. Clone task about
 3 s, Create 6.2 s, address 50.2 s after Create returned, Delete plus Get
 plus a second Delete 3.1 s. The Delete deleted the seed while the stopped
 VM's `ide2` still referenced it (imgdel `OK`), then destroyed the VM, and
 step 10 found neither left. That settles the one assumption behind the
 reorder that the fake could not prove. The run confirmed P6 as well: the
 first `Get`, about 6 s after the clone, met `status: unknown` in the
 listing and answered Transient, blaming the node, which was up. The
 message now names both causes.
 **The three answers Phase 0 never captured.** Captured with a scratch
 helper that passes the token to curl through a process substitution and
 writes the Phase 0 fixture shape. Its first offline test caught a bug:
 Python's text mode folds `\r\n`, so the header split found nothing, and
 the file is now opened with `newline=""`. The calls were:
 ```bash
 pvecap storage_status_missing GET /nodes/pv1-1/storage/proxy-missing/status
 pve POST /nodes/pv1-1/qemu/9001/clone -d newid=199999 -d name=proxy-reconaaaaaaaaaaa -d full=1
 pve POST /nodes/pv1-1/qemu/199999/status/start
 pve POST /nodes/pv1-1/qemu/199999/status/suspend                 # pause
 pvecap stop_paused_response POST /nodes/pv1-1/qemu/199999/status/stop
 pvecap task_status_stop_paused GET /nodes/pv1-1/tasks/<upid>/status
 pvecap status_current_after_stop_paused GET /nodes/pv1-1/qemu/199999/status/current
 pve POST /nodes/pv1-1/qemu/199999/status/start
 pve POST /nodes/pv1-1/qemu/199999/status/suspend -d todisk=1     # hibernate: stopped, lock suspended
 pvecap err_destroy_hibernated DELETE '/nodes/pv1-1/qemu/199999?purge=1&destroy-unreferenced-disks=1'
 pvecap task_status_destroy_hibernated GET /nodes/pv1-1/tasks/<upid>/status
 ```
 Two of the three contradicted the code:
 - A missing storage is a **400** with the field error
  `storage: No such storage.`, not a 500, so the Checker reported
  `APIError`. `isNoSuchStorage` matches that exact field error, and a 500
  is still taken as missing (not captured). The matcher matrix caught a
  first version that fired on any `storage` field error, which a clone
  onto a storage without image support also has.
 - The destroy **accepts a hibernated VM**. Its task ended `OK`, removing
  both disks and the `vm-199999-state-suspend-…` volume. Delete refused
  such a VM of ours as Permanent, believing the destroy would, so it now
  destroys it like a stopped one. Create's adoption still refuses one it
  finds before it was ever recorded. The capture was renamed
  `destroy_hibernated_response.json`, because the client test globs
  `err_*.json` as errors.
 - A stop on a paused VM ends `OK` with the VM stopped, as assumed.
 The destroy also cleaned up the scratch VM, and a listing of local-lvm
 content for VMID 199999 came back empty.
 **Kind pass.** The user built and loaded the image, deployed, applied a
 scratch providers config with only `kubernetes` and `proxmox-home`, ran
 the sample Proxy to Ready, ran a `curl -x` through it, deleted it and
 checked pv1-1 for leftovers. Everything passed. The first attempt at the
 override went to a ConfigMap named `providers-config`. `config/default`'s
 `namePrefix` makes the mounted one `egress-proxies-operator-providers-config`,
 so the pod kept the shipped config: no `proxmox-home` ProviderHealth, and
 `kvm-home` and `kvm2-home` failing on the missing SSH key. The corrected
 command replaced the prefixed ConfigMap and deleted the stray one. The
 Secret keeps its plain name (`proxmox-token`), since kustomize renames
 only resources it builds. `make deploy`'s image edit to
 `config/manager/kustomization.yaml` was reverted, not committed.
 Worth noting:
 - **Behaviour changes from Phase 9.** Delete's order is now stop, seed,
  destroy. A hibernated VM of ours is destroyed rather than latched. The
  Checker reports a missing ISO storage as `StorageNotFound`. A
  foreign-named VM on a probe VMID is skipped without a config read.
 - **Accepted, not fixed** (guide Known gaps, spec entry 24):
  - P3: Create adopts a leftover stopped VM whatever the current
    `placement` says.
  - P4: a retry can clone a second VM in a narrow crash window.
 - **Still unproven:**
  - A storage that exists but cannot be used on a node.
  - Every cross-node path. The host has one node.
 - **Flakes.** kvm's `TestKeepalive_keepsHealthyClientOpen` failed once
  under `-race` ("unexpected packet in response to channel open") and
  passed 5 of 5 on rerun. This branch does not touch `internal/provider/kvm`.
  The parked kvm2 items (S5.1, the 200 ms `TestCheck_hostsCheckedConcurrently`)
  did not bite, though the reviewer agent saw the latter fail once.
 - **Docs.** The "not yet run" wording is gone from the README, the
  guide, `testing.md`, spec entry 21 and `providers-existing.md`.
  `testing.md` gained a timings table and the `PROXMOX_TEST_VLAN_TAG`
  export line. The link sweep script from the Phase 9 entry found 0
  broken links over the seven edited docs, and the guide has no `—`.
 - `.claude/settings.json` stays out of the commit.
@@ -34,7 +34,7 @@ and the symbol that shows it.
 |---|---|---|---|---|---|
 | `readiness` | immediate: a Pod is scheduled within seconds (`stateFromPod`) | provisioning-window: minutes of boot (`mapState`) | provisioning-window: boot plus, on first use, an image download (`imageCache.ensure`) | provisioning-window: boot plus, on first use, an image download (`ensureImage`) | provisioning-window: boot plus the guest agent's install at first boot, about 50 s to an address on the test host (`agentAddress`) |
 | `create-return` | submit: returns after `client.Create` | submit: `Insert` is never awaited, `Operation.Wait` is not called | **blocking**: waits through `DomainCreate` (`ensureRunning`), and the first image fetch runs inside (`imageCache.fetch`) | **blocking**: waits through `DomainCreate` and a 15-second poll for running (`ensureRunning`, `waitRunning`), and the first image fetch runs inside (`fetchImage`) | **blocking**: waits through the clone, upload, resize and start tasks and a 60-second poll for running (`provision`, `waitRunning`) |
-| `delete-return` | returns-early: `client.Delete`, Pod terminates on its own | returns-early: `Delete` submits an operation each time | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: the stop, destroy and seed delete tasks complete before return (`destroy`, `deleteSeed`) |
+| `delete-return` | returns-early: `client.Delete`, Pod terminates on its own | returns-early: `Delete` submits an operation each time | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: the stop, seed delete and destroy tasks complete before return (`destroy`, `deleteSeed`) |
 | `ip-source` | assigned-at-create: `PodIP` (`instanceFromPod`) | assigned-at-create: the external NAT IP of the first NIC (`toInstance`) | discovered: ARP table first, then the guest agent, first routable IPv4 (`address`, `firstIPv4`) | discovered: the guest agent first, then the ARP table, first routable IPv4 (`addressSources`, `address`, `firstIPv4`) | discovered: the guest agent only, first routable IPv4 in the agent's order (`agentAddress`, `firstIPv4`) |
 | `gc-mark` | native-labels: Pod labels (`buildPod`) | native-labels: instance labels (`buildInsertRequest`) | written-metadata: a namespaced `<metadata>` element written in `DomainDefineXML` (`metadata.go`) | written-metadata: an element in kvm2's own namespace written in `DomainDefineXML` (`metadataURI`, `newInstanceMetadata`) | written-metadata: three `key=value` lines in the VM's description, passed in the clone call itself (`formatDescription`, `parseDescription`) |
 | `created-at` | backend-timestamp: `CreationTimestamp` | backend-timestamp: `CreationTimestamp`, parse error gives zero | written-at-create: `createdAt` in the metadata element (`newInstanceMetadata`) | written-at-create: `createdAt` in the metadata element, parse error gives zero (`newInstanceMetadata`, `createdAt`) | written-at-create: `proxy-operator-created-at` in the description, parse error gives zero (`parseDescription`) |
@@ -50,7 +50,7 @@ and the symbol that shows it.
 | `timeouts` | ctx-only: no `rest.Config.Timeout` is set | ctx-only: SDK defaults, with SDK retries on Get and List | per-call for the dial only (`defaultHandshakeTimeout` in `dialSSH`); RPCs are ctx-only and a cancelled `call` closes the connection | per-call: a budget around the dial, every RPC, the uploads and the download (`defaultBudgets`, `call`); a cancelled or overlong call kills the pooled connection | per-call: dial and TLS handshake 10 s, each request `requestTimeoutSeconds`, a budget per task wait (`defaultBudgets`); a cancelled request ends only itself |
 | `logging` | none | tiered: V(1) outcomes and `logAPIError`, V(2) detail, V(5) wire log (`wirelog.go`) | tiered: V(1) outcomes and one `libvirt call` line per RPC (`trace.go`), V(2) XML and addresses; no wire tier | tiered: V(1) outcomes and one `libvirt call` line per RPC (`observe.go`), V(2) XML and addresses; no wire tier | tiered: V(1) outcomes and one `proxmox api call` line per request (`observe.go`), V(2) parameters, config summaries and addresses; no wire tier |
 | `child-spans` | transport-wrapper: `tracing.RestConfigWrapper()` on its own `rest.Config`, parent-gated | sdk-instrumentation: the SDK adds `otelhttp` itself; not parent-gated, but every call site has a parent | **none**: `trace.go` is a log decorator, not spans | span-decorator: `kvm2.<Method>` per RPC (`observe`), plus `kvm2.dial` (`libvirtDialer`) and `kvm2.download` (`fetchImage`) | transport-wrapper: `tracing.RestConfigWrapper()` on the client's transport, parent-gated, one `HTTP <METHOD>` span per request (`newHTTPAPI`) |
-| `backend-test` | cluster-e2e: kind (`test/e2e`) | manual-runbook: [gcp-vm-validation.md](gcp-vm-validation.md) | integration-tag: `kvm_integration_test.go`, `make test-integration-kvm` | integration-tag: `kvm2_integration_test.go`, `make test-integration-kvm2` | integration-tag: `proxmox_integration_test.go`, `make test-integration-proxmox`; compiled, not yet run against a host |
+| `backend-test` | cluster-e2e: kind (`test/e2e`) | manual-runbook: [gcp-vm-validation.md](gcp-vm-validation.md) | integration-tag: `kvm_integration_test.go`, `make test-integration-kvm` | integration-tag: `kvm2_integration_test.go`, `make test-integration-kvm2` | integration-tag: `proxmox_integration_test.go`, `make test-integration-proxmox`; passed on a PVE 9.2.20 host |
 ## 2. Other differences
@@ -614,16 +614,18 @@ was written.
 - **Delete.** Under the per-name lock, `destroy`: `resolve`; a lock other
  than `suspended` or `status: unknown` is Transient; the config's `does
  not exist`, another name or no mark leave the VM alone; our VM
-  hibernated is Permanent, because the destroy refuses it and the lock
+  hibernated is destroyed like a stopped one, since the destroy takes it.
-  never clears. Otherwise stop (unless stopped) and destroy with
+  Otherwise stop (unless stopped), `deleteSeed` on the
-  `purge=1&destroy-unreferenced-disks=1`, each a task waited on through
+  VM's node, where a failure fails the call and leaves the VM for the
-  `runTask`, where `does not exist` means gone. Then `deleteSeed` on the
+  next Delete (a 403 for `Datastore.Allocate` is Permanent), then destroy
-  VM's node, where a failure fails the call (a 403 for
+  with `purge=1&destroy-unreferenced-disks=1`. Stop and destroy are tasks
-  `Datastore.Allocate` is Permanent), or, with no VM of ours destroyed,
+  waited on through `runTask`, where `does not exist` means gone. The
-  `deleteSeedEverywhere` over every configured node, where a node that
+  seed goes first because a VM already destroyed is NotFound to `Get`,
-  fails is logged and skipped unless ctx ended. A missing volume still
+  and nothing would call Delete again. With no VM of ours,
-  ends in a successful task, so no message is matched. Nothing is left
+  `deleteSeedEverywhere` tries every configured node, where a node that
-  when nil is returned.
+  fails is logged and skipped unless ctx ended or the failure is
  Permanent. A missing volume still ends in a successful task, so no
  message is matched. Nothing is left when nil is returned.
 - **ListByTag.** One `ClusterResources` covers every node, unreachable
  ones included (their guests are listed with `status: unknown`).
  `isCandidate` keeps QEMU VMs that are not templates and carry the tag
@@ -639,9 +641,9 @@ was written.
  status Transient, including the lock, the taken VMID and pveproxy's 595
  and 596); TLS verification failures (`tls.CertificateVerificationError`
  and the three `x509` types) Permanent; everything else Transient. The
-  500 matchers (`isNotExist`, `isLocked`, `isAlreadyExists`, the three
+  500 matchers (`isNotExist`, `isAlreadyExists`, the two agent ones)
-  agent ones) are anchored to the 500 status and each is backed by a
+  are anchored to the 500 status and each is backed by a captured
-  captured response. `permanentf` and `transientf` classify at the
+  response. `permanentf` and `transientf` classify at the
  source: placement, the template checks, the ownership check, a
  malformed token file or CA file (Permanent); a missing token or CA file,
  an expired wait, a failed task (Transient). Nothing is Quota.
@@ -694,7 +696,7 @@ was written.
  by `PROXMOX_TEST_*` variables and `make test-integration-proxmox`,
  described in
  [testing.md](testing.md#the-proxmox-integration-test-real-proxmox-ve-host);
-  it compiles but has not run against a host yet.
+  it passed against PVE 9.2.20 on 2026-09-24.
 - **Deployment.** The optional `proxmox-token` Secret (keys `token` and
  `ca.pem`) mounted at `/etc/proxy-operator/proxmox`
  (`config/manager/manager.yaml`), `make proxmox-token-secret`, a
@@ -713,11 +715,9 @@ was written.
  - The VMID range is shared with humans; a hand-made VM with a proxy's
    name is Permanent.
  - Cross-node clones (`target`) need shared storage and are untested.
-  - A hibernated VM of ours latches its Proxy in Create and in Delete.
+  - A hibernated VM of ours latches its Proxy when Create's adoption
    finds it; a recorded one is replaced.
  - The CA is cached after first use; the token is not.
  - Never run against a real host yet; three answers were never captured
    (the storage status for a missing storage, a destroy on a hibernated
    VM, a stop on a paused VM).
 ## 9. Gaps shared by the existing providers
@@ -77,7 +77,11 @@ lands on the same number: `vmidFor(uid, attempt)` is
   clears by itself. Ownership is judged at once, as `Get` does (entry 9):
   a foreign hibernated VM is probed past, and ours is `ErrPermanent`
   (resume or destroy it), like kvm2's paused domain. Container and
-   template entries at a VMID are foreign without a config read.
+   template entries at a VMID are foreign without a config read, and so
   is a VM whose listed name is not `req.Name`, whatever its lock: only a
   clone in progress is nameless, and waiting on another VM's `backup`
   lock would hold Create for the whole clone budget on every retry
   (review finding P2).
 The wait in step 3 is what closes the duplicate-VM bug Phase 0 found: after
 a crash mid-clone, the retry's by-name lookup misses the nameless VM, but
@@ -284,7 +288,7 @@ and starts it; the reconciler has no ID for it to `Get`.
 | VMID absent | `/cluster/resources` | `ErrNotFound` |
 | `lock` set, other than `suspended` | resources entry (a clone in progress also has no name) | `ErrTransient` |
 | `lock: suspended` (hibernated to disk) | resources entry | Stopped |
-| `status: unknown` (node not reachable) | resources entry | `ErrTransient` |
+| `status: unknown` (node not reachable, or a VM seconds old the listing has not reported yet: the first `Get` after Create met this on pv1-1) | resources entry | `ErrTransient` |
 | config `does not exist` | `GET …/config` | `ErrNotFound` (deleted between the two calls) |
 | config `lock` set | config | `ErrTransient` |
 | `name` differs from the ID's name | config | `ErrNotFound`, logged: the VMID was reused |
@@ -382,34 +386,48 @@ the seed ISO; the VM's disks are removed by the destroy call itself, with
 1. Parse the ID (`ErrPermanent` if malformed).
 2. `/cluster/resources`. VMID absent, or held by a container: skip to
-   step 7, trying every configured node because the VM's node is no longer
+   step 6, trying every configured node because the VM's node is no longer
   known.
 3. `lock` set, other than `suspended`, or `status: unknown`: `ErrTransient`.
   A clone or another task still running finishes first; the reconciler
   calls Delete again after `DeletionPoll`.
-4. Read the config. `does not exist`: skip to step 7 on every node. A
+4. Read the config. `does not exist`: skip to step 6 on every node. A
   different name, or no mark: the VM is not ours; log it at V(1) and skip
-   to step 7 on every node without touching it. Our VM with a `lock` other
+   to step 6 on every node without touching it. Our VM with a `lock` other
   than `suspended` (taken since the listing): `ErrTransient`. Our VM with
-   `lock: suspended` (hibernated): `ErrPermanent` asking the admin to resume
+   `lock: suspended` (hibernated) is destroyed like a stopped one: the
-   or destroy it, the same answer Create's ownership check gives. The
+   listing shows it `stopped`, so step 5 is skipped, and the destroy
-   destroy refuses a locked VM, and that lock never clears on its own, so
+   accepts it and removes its suspend state volume with the disks
-   Transient would retry forever; Permanent puts the admin action in the
+   (`destroy_hibernated_response.json`, `task_status_destroy_hibernated.json`,
-   Proxy's status, where `reconcileManaged` latches it and the finalizer
+   captured in Phase 10). Until then Delete refused it as `ErrPermanent`
-   path keeps retrying it visibly. A hibernated VM that is not ours is left
+   on the belief that the destroy would. A hibernated VM that is not ours
-   alone like any other.
+   is left alone like any other.
 5. Status (from the listing) other than `stopped`: `POST …/status/stop`,
   wait (stop budget). Stop works on a paused VM too, which the listing
-   shows as `running`. `does not exist` means the VM is gone.
+   shows as `running`: the task ends `OK` and the VM is `stopped`
-6. `DELETE /nodes/{node}/qemu/{vmid}?purge=1&destroy-unreferenced-disks=1`,
+   (`stop_paused_response.json`, `task_status_stop_paused.json`,
-   wait (destroy budget). `does not exist` is success.
+   `status_current_after_stop_paused.json`, captured in Phase 10).
-7. `DELETE /nodes/{node}/storage/{isoStorage}/content/<isoStorage>:iso/<name>-seed.iso`,
+   `does not exist` means the VM is gone.
-   wait (seed-delete budget), on the VM's node when steps 5 and 6 ran on a
+6. `DELETE /nodes/{node}/storage/{isoStorage}/content/<isoStorage>:iso/<name>-seed.iso`,
-   VM of ours, and on every configured node in config order otherwise. A
+   wait (seed-delete budget), on the VM's node when step 5 was reached on
-   missing volume is still a successful task, so no message is matched. On
+   a VM of ours, and on every configured node in config order otherwise
-   the every-node path a node whose request or task fails is logged at V(1)
+   (then step 7 is skipped). A missing volume is still a successful task,
-   and skipped, unless ctx ended, which fails the call; on the VM's node a
+   so no message is matched. The seed goes **before** the destroy: once
-   failure fails the call (a 403 for `Datastore.Allocate` is Permanent).
+   the VM is gone `Get` is `ErrNotFound`, the reconciler releases the
   finalizer and GC no longer lists it, so a seed left by a failed step
   after the destroy would never be deleted (review finding CR1). On the
   VM's node a failure fails the call and leaves the VM stopped for the
   next Delete. On the every-node path a node whose request or task fails
   is logged at V(1) and skipped, unless ctx ended or the failure is
   `ErrPermanent` (a 403 for `Datastore.Allocate`, which every node
   would give), which fail the call, so the missing privilege is visible
   on both paths (review finding P1).
 7. `DELETE /nodes/{node}/qemu/{vmid}?purge=1&destroy-unreferenced-disks=1`,
   wait (destroy budget). `does not exist` is success. The destroy leaves
   the seed alone (Phase 0 deleted it after the destroy), which is why
   step 6 exists. That Proxmox deletes an ISO a stopped VM's `ide2` still
   references was not in the Phase 0 captures; the Phase 10 real-host run
   proved it (imgdel `OK` on the stopped VM's seed, then the destroy).
 8. Return nil.
 Everything is gone when Delete returns nil, so `Get` afterwards is
@@ -474,6 +492,7 @@ per-parameter map. One `classify(op string, err error)`:
 | 401 `Authentication failed!`, 401 `No ticket` | Permanent | the token or its secret is wrong; fixed in the Secret |
 | 403 `Permission check failed (<path>, <privilege>)` | Permanent | the token lacks a privilege; the message names it |
 | 400 `Parameter verification failed.` with `errors` | Permanent | the request is wrong; the message includes the field errors |
 | 400 as above with `storage: No such storage.` | Permanent from a call; the Checker's `StorageNotFound` (entry 16) | the storage does not exist in the cluster |
 | other 4xx | Permanent | a request Proxmox will never accept |
 | 500 `Configuration file '…' does not exist` | NotFound (Transient when `op == "create"`) | the VM is gone |
 | 500 `VM is locked (<lock>)` | Transient | a task holds the VM |
@@ -490,17 +509,24 @@ per-parameter map. One `classify(op string, err error)`:
 Every matched string is backed by a captured response under `testdata/`;
 the table test loads them rather than repeating the text. Matching is on
-fixed substrings anchored to the 500 status: `does not exist`,
+fixed strings anchored to the 500 status: `does not exist`,
-`VM is locked`, `config file already exists`, `QEMU guest agent is not
+`config file already exists`, and the whole messages
-running`, `No QEMU guest agent configured`, and for the stopped VM the whole
+`QEMU guest agent is not running` and `No QEMU guest agent configured`,
-message against `^VM [0-9]+ is not running$`. The agent messages are tested
+plus, on a 400, the `storage` field error `No such storage.` (whole; a
-in that order, because the last is a suffix of the first.
+clone onto a storage without image support has another `storage` error).
 `VM is locked` and the stopped VM's `VM <vmid> is not running` are not
 matched by the provider: both are Transient through the default branch.
 The tests keep a matcher for each, with the whole stopped-VM message
 against `^VM [0-9]+ is not running$` (its tail is a suffix of the
 agent-not-running answer), only to hold the fake's wording to the
 captures (review finding CR5).
 **Evidence:** F1 (error text in the reason phrase and the body; 401 has no
 body); F2, F4, F5, F7 (the 500 messages); F6 (no out-of-space response
 could be produced on thin storage); F22 (both 401 texts); fixtures
 `err_forbidden.json`, `err_bad_param.json`, `err_tag_invalid.json`,
-`err_upload_bad_name.json`; the Phase 0 TLS finding (`192.168.2.8` not in
+`err_upload_bad_name.json`, and from Phase 10 `storage_status_missing.json`
 (a missing storage is a 400, not the 500 the spec had assumed); the Phase 0 TLS finding (`192.168.2.8` not in
 the node certificate); contract §5.
 ## 14. Credentials and TLS (`credentials`)
@@ -611,11 +637,15 @@ an admin:
 - `CredentialsUnavailable`: the token file is missing or malformed. Fix the
  Secret, not Proxmox.
 - `StorageNotFound`: `isoStorage` is missing, inactive, or has no `iso`
-  content. Fix the storage on the node. Any 500 from the storage status
+  content. Fix the storage on the node. A storage the cluster does not
-  call counts as missing, whatever its wording (it does not exist, is not
+  know answers `400 Parameter verification failed.` with the field error
-  enabled on that node, or is offline; no missing-storage response was
+  `storage: No such storage.` (`storage_status_missing.json`, captured in
-  captured, so no text is matched), and the message carries Proxmox's
+  Phase 10); that exact field error is matched, and the message carries
-  text. A 401, 403, 595 or 596 there keeps its shared reason.
+  it. Any 500 from the storage status call counts as missing too,
  whatever its wording: Proxmox knows the storage but cannot use it on
  that node (not enabled there, or offline). That answer was not
  captured, so no text is matched. Any other 4xx, and a 401, 403, 595 or
  596, keeps its shared reason.
 - `TemplateNotFound`: a configured template VMID does not exist or is not a
  template. Fix the template or the catalogue.
@@ -783,8 +813,8 @@ carries the cloud-init text.
 Contract §11 list, with the test for each item. Every name below exists
 (Phase 8 checked each with a grep for `func <name>(`). The two integration
-tests compile under `-tags=integration` but have not run against a host
+tests passed on pv1-1 on 2026-09-24 (timings in
-yet; Phase 10 runs them.
+[testing.md](testing.md#the-proxmox-integration-test-real-proxmox-ve-host)).
 - [x] repeat `Create`: `TestCreate_repeatReturnsSameIDAndCreatesNothing`
 - [x] `Get` after `Delete`: `TestGet_afterDeleteReturnsNotFound`
@@ -802,7 +832,7 @@ yet; Phase 10 runs them.
 - [x] Checker reasons, expired `ctx`, stable order: `TestCheck_reasonsTable`, `TestCheck_expiredContextReturnsPromptly`, `TestCheck_oneEntryPerNodeInConfigOrder`
 - [x] sentinel tests for `CloudInit` and the credential: `TestCreate_cloudInitSentinelNeverLoggedOrInErrors`, `TestClient_tokenNeverInErrorsOrLogs`, and `TestLogging_v0SilentV1OutcomeV2Detail` over a whole lifecycle at every level
 - [x] child spans under the operation span: `TestTracing_httpSpansUnderOperationSpan`, `TestClient_childSpanUnderParent`
- [x] real-backend lifecycle test: `TestProxmox_lifecycle` and `TestProxmoxCheck_realHost` in `proxmox_integration_test.go` (compiled, not yet run)
+- [x] real-backend lifecycle test: `TestProxmox_lifecycle` and `TestProxmoxCheck_realHost` in `proxmox_integration_test.go`, passed on pv1-1
 Tests that cover the Phase 0 corrections specifically, in addition to the
 plan's per-phase lists:
@@ -904,4 +934,28 @@ user confirms.
  VMID range delete each other's VMs), an unconfigured provider latches the
  Proxy.
 Accepted after the Phase 10 review
 ([docs/reviews/2026-09-23-proxmox-provider-review/](reviews/2026-09-23-proxmox-provider-review/REPORT.md),
 findings P3 and P4), and listed in the guide's Known gaps:
 - **A Create that failed permanently after its clone keeps its VM**, and
  the next Create adopts it by name even when `placement` has changed.
  Destroy the VM before changing `placement` on such a Proxy.
 - **A retry can clone a second VM** when a Create is cut off while
  cloning onto a later probe VMID, the VM on the earlier one is removed,
  and the retry comes before the clone ends. The next Create names both
  VMIDs as `ErrPermanent`.
 Changed by the Phase 10 captures of the three answers Phase 0 never saw:
 - **A hibernated VM of ours** latches only Create's adoption
  (`ErrPermanent`, resume or destroy it; entry 2). `Get` reports it
  Stopped, so the reconciler replaces it, and Delete destroys it, since
  the destroy takes a hibernated VM (entry 11). Before the capture, Delete
  refused it as well.
 - **A missing `isoStorage`** is a 400 with `storage: No such storage.`,
  which the Checker reports as `StorageNotFound` (entry 16); a missing
  storage at upload time is Permanent like any 4xx.
 - **A paused VM** stops like a running one (entry 11, step 5), as assumed.
 **Evidence:** contract §13; entries 2, 7, 10, 11, 14 above.
@@ -164,8 +164,14 @@ pve 'cluster/resources?type=vm'              # the template is listed with "temp
   **Rotating the token needs no restart.** The token file is read on
   every request, so the next request after the kubelet refreshes the
-   mount uses the new token. **Rotating the CA does**: it is loaded on
+   mount uses the new token. Keep the old token valid until then: a 401
-   first use and cached for the life of the process. A missing CA file is
+   is a permanent error, so a Proxy whose request lands in the gap
   latches until its spec changes. Add the new token
   (`pveum user token add`), update the Secret, wait a couple of minutes
   for the mount to refresh, and only then remove the old token. A token
   created with `--expire` needs the same rotation before it expires.
   **Rotating the CA does** need a restart: it is loaded on first use and
   cached for the life of the process. A missing CA file is
   retried until it lands; one without a certificate is a permanent error.
 2. Add a provider entry to the providers config
@@ -302,7 +308,9 @@ through an ownership check (`inspect` in `create.go`): a VM with a lock
  naming the VMID and both UIDs. The operator never takes over a VM it did
  not create;
 - our name, our mark, hibernated (`lock: suspended`): **permanent error**,
-  `resume or destroy it in Proxmox` (see *Known gaps*).
+  `resume or destroy it in Proxmox` (see *Known gaps*). This only happens
  when Create has not recorded the VM yet; a recorded VM that gets
  hibernated is reported Stopped and replaced.
 Two VMs with the proxy's name are a permanent error naming every VMID;
 remove all but one.
@@ -392,15 +400,18 @@ pvesm free proxy-seeds:iso/proxy-abc…-seed.iso
 ```
 **Delete** runs under a per-name lock shared with Create: stop (a paused
-VM too), destroy with `purge=1&destroy-unreferenced-disks=1`, then delete
+VM too), delete the seed on the VM's node, then destroy with
-the seed on the VM's node. Each is a task that is waited on. A VM that is
+`purge=1&destroy-unreferenced-disks=1`. Each is a task that is waited on.
-already gone, not ours, or unmarked is left alone, and the seed is then
+The seed goes before the VM because once the VM is gone nothing calls
-looked for on every configured node, where a node that fails is logged
+Delete again; a failed seed delete leaves the VM stopped, and the next
-and skipped. On the VM's own node a failed seed delete fails the call:
+Delete finishes both. A VM that is already gone, not ours, or unmarked is
-**without `Datastore.Allocate` on `isoStorage` the 403 makes Delete
+left alone, and the seed is then looked for on every configured node,
-permanent**, and the Proxy's deletion stays stuck with that message until
+where a node that fails is logged and skipped. A refusal every node would
-the privilege is granted. When Delete returns nil, nothing of the Proxy
+give fails the call instead: **without `Datastore.Allocate` on
-is left.
+`isoStorage` the 403 makes Delete permanent** on either path, and the
 Proxy's deletion stays stuck with that message (a replaced Proxy stays
 failed, its VM stopped) until the privilege is granted. When Delete
 returns nil, nothing of the Proxy is left.
 ## Logging
@@ -554,9 +565,9 @@ transient.
 | Class | What lands there |
 |---|---|
-| `ErrPermanent` | placement values not in the catalogue; a template VMID that is missing or not a template, has a cloud-init drive, or lacks `net0` or a sized boot disk when the config needs them; a malformed providerID; any 4xx (401 `Authentication failed!` or `No ticket`: the token; 403 `Permission check failed (<path>, <privilege>)`: a missing privilege, including `Datastore.Allocate` for the seed delete; 400 `Parameter verification failed.` with the field errors); a TLS verification failure; a token file that exists but is not `<user>@<realm>!<tokenid>=<secret>`; a CA file with no certificate; a VM with the proxy's name that carries no mark or another UID; two VMs with the proxy's name; our VM hibernated (`lock: suspended`), in both Create and Delete |
+| `ErrPermanent` | placement values not in the catalogue; a template VMID that is missing or not a template, has a cloud-init drive, or lacks `net0` or a sized boot disk when the config needs them; a malformed providerID; any 4xx (401 `Authentication failed!` or `No ticket`: the token; 403 `Permission check failed (<path>, <privilege>)`: a missing privilege, including `Datastore.Allocate` for the seed delete; 400 `Parameter verification failed.` with the field errors); a TLS verification failure; a token file that exists but is not `<user>@<realm>!<tokenid>=<secret>`; a CA file with no certificate; a VM with the proxy's name that carries no mark or another UID; two VMs with the proxy's name; our VM hibernated (`lock: suspended`) found by Create's adoption |
 | `ErrNotFound` | from `Get`: the VMID absent from the cluster's resources, the 500 `Configuration file 'nodes/<node>/qemu-server/<vmid>.conf' does not exist`, the VM at the VMID now named differently or unmarked. Inside `Create` the `does not exist` answer is `ErrTransient`, because the contract forbids NotFound from Create; `Delete` treats all of these as done |
-| `ErrTransient` | everything else: a missing token or CA file (the Secret has not landed yet); network errors and cancellation; 500 `VM is locked (<lock>)` and any lock seen in the resources or config; a VM whose node does not answer (`status: unknown`); the agent's answers that `Get` cannot place, such as `VM <vmid> is not running`; any other 5xx, including pveproxy's 595 and 596; a task that fails or runs out of its budget; no free VMID after 32 probes; the wait for running expiring, or a VM that stops right after its start |
+| `ErrTransient` | everything else: a missing token or CA file (the Secret has not landed yet); network errors and cancellation; 500 `VM is locked (<lock>)` and any lock seen in the resources or config; a VM whose node does not answer, or one a few seconds old the listing has not reported yet (`status: unknown`); the agent's answers that `Get` cannot place, such as `VM <vmid> is not running`; any other 5xx, including pveproxy's 595 and 596; a task that fails or runs out of its budget; no free VMID after 32 probes; the wait for running expiring, or a VM that stops right after its start |
 `Get` turns two agent answers into states rather than errors:
 `QEMU guest agent is not running` is Provisioning (booting, or the agent
@@ -613,13 +624,26 @@ They never contain the token, the CA content or cloud-init.
 Beyond the ten limits accepted in the design:
- **A hibernated VM latches its Proxy.** A VM of ours with
+- **A hibernated VM is replaced, or latches a Create.** `Get` reports a
-  `lock: suspended` (hibernated to disk) makes both Create and Delete fail
+  VM of ours with `lock: suspended` (hibernated to disk) as Stopped, so
-  permanently with `resume or destroy it in Proxmox`: the lock never
+  the reconciler deletes it (the destroy removes its suspend state volume
-  clears on its own, and the destroy refuses a locked VM. `Get` reports it
+  too) and creates a new one. Only a Create that finds such a VM before it
-  Stopped. Resume it by starting it (`qm start <vmid>` restores the
+  was ever recorded, by name, fails permanently with
-  hibernated state) or destroy it on the node, then touch the Proxy's
+  `resume or destroy it in Proxmox`: resume it by starting it
-  spec, or let the pending Delete retry.
+  (`qm start <vmid>` restores the hibernated state) or destroy it on the
  node, then touch the Proxy's spec.
 - **A Create that failed permanently after its clone keeps its VM.**
  The stopped VM is adopted by name on the next Create, even when the
  Proxy's `placement` has changed since: the Proxy then runs the old
  template on the old node. After changing `placement` on such a Proxy,
  destroy the VM on the node (`qm destroy <vmid> --purge`) first, or
  delete and recreate the Proxy.
 - **A retry can clone a second VM** in one rare sequence: a Create is
  cut off while cloning onto a later probe VMID (the first was taken by
  someone else's VM), that other VM is removed, and the retry comes
  while the clone is still running, so its VM is still nameless. The
  retry clones onto the now free first VMID. The next Create then fails
  permanently with `VMs … are all named …`; destroy either VM.
 - **The seed upload is one request**, bounded by `requestTimeoutSeconds`.
  A seed is tens of KiB, but a multi-megabyte `vendorData` would need a
  larger timeout.
@@ -627,11 +651,6 @@ Beyond the ten limits accepted in the design:
  `config/samples/proxy_proxmox.yaml` installs Squid with
  `http_access allow all`; restrict it before exposing the proxy network
  to anything but the cluster.
 - **Not yet proven end to end.** The unit tests run over captured
  responses from `pv1-1` and an in-memory fake; the integration tests
  below compile but have not run against a host yet. Three answers were
  never captured: the storage status for a missing storage, a destroy on
  a hibernated VM, and a stop on a paused VM.
 ## Testing
@@ -644,3 +663,5 @@ which also lists the variables it needs; run it with
 `make test-integration-proxmox`. `TestProxmoxCheck_realHost` runs the
 health check against the same host and creates nothing:
 `go test -tags=integration -count=1 -v -run TestProxmoxCheck ./internal/provider/proxmox/`.
 Both passed on `pv1-1` (PVE 9.2.20) on 2026-09-24: Create took 6.2 s,
 and the guest reported an address 50 s after Create returned.
@@ -0,0 +1,286 @@
 # proxmox provider review: final report
 Branch `feat/proxmox-provider` at `7f2073f` (Phases 0 to 9), diffed against
 `main` (`2167819`): 123 files, 49 of them Go. The core is
 `internal/provider/proxmox/`; also in scope were the `nocloud` and
 `keyedlock` extraction from kvm2 (`0a6c862`), the config block in
 `internal/provider/config*.go`, registration and the deploy files.
 **Inputs.** [docs/proxmox-provider-spec.md](../../proxmox-provider-spec.md)
 (the spec the code implements) and
 [docs/provider-contract.md](../../provider-contract.md) (the MUST/SHOULD
 rules). A disagreement with either counts as a finding.
 **Reviewers.** Two independent passes, both read-only:
 1. The `go-operator-reviewer` agent (`.claude/agents/go-operator-reviewer.md`),
   briefed with the spec and the contract, findings **P1 to P6** and
   simplifications **S1 to S3**.
 2. `/code-review high` on the branch, findings **CR1 to CR10**.
 Where both passes found the same thing, the finding is listed once with both
 IDs. Every finding is **fixed** (with a regression test when it is code),
 **accepted** (real, not fixed here, documented as a known limit), or
 **rejected** (with the reason).
 **Toolchain.** `make test` green before and after the fixes. `make lint`
 panics inside staticcheck on the local Go 1.27 toolchain, as it did during
 kvm2, so lint ran without it, over the proxmox package and
 `internal/provider`, with and without the integration tag:
 ```bash
 bin/golangci-lint run --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
 bin/golangci-lint run --build-tags integration --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
 ```
 Before the fixes both runs were at the Phase 8 and 9 baseline: logcheck 12,
 modernize (mapsloop) 2, unparam 3 in `fake_test.go`, gocyclo 1
 (`TestAPITypes_decodeCaptures`), goconst on repeated strings, and the three
 goconst hits in `internal/provider`. After the fixes the counts are the same
 except goconst, one lower (224, 231 with the tag). No hit is new. The first
 version of the P2 test did add one: a third `"backup"` literal in the
 package's tests, then `"migrate"`. It now uses `"snapshot-delete"`.
 ## Summary
 | ID | Severity | Finding | Disposition |
 |---|---|---|---|
 | CR1 + P1 | medium-high | A seed delete that fails after the destroy leaks the seed for good, and the class of the error depends on the attempt | **fixed** |
 | CR2 | medium | A 403 on the seed delete latches a replaced Proxy whose VM is already gone | **fixed** (same change as CR1) |
 | CR3 + P2 | medium | The ownership check waits out any lock, even on a VM whose listed name is someone else's | **fixed** |
 | P3 | low-medium | Create adopts a leftover stopped VM whatever the current placement says | **accepted**, documented |
 | P4 | low | A crash mid-clone plus a freed earlier VMID can yield a second VM | **accepted**, documented |
 | P5 | low | 401/403 are Permanent, which bites during a token rotation | **fixed** in the guide |
 | P6 | low | Get and Delete trust the listing's `status`, which lags | **accepted**, confirmed on the host; error message fixed |
 | CR5 | low | `isLocked` and `isVMNotRunning` are called only from tests | **fixed** |
 | CR8 | low | `TestPermanentfAndTransientf` breaks the `Test<Function>_<scenario>` rule | **fixed** |
 | CR9 | low | `get` repeats `mapState`'s "guest running" rule inline | **fixed** |
 | CR4 + S2 | low | The Checker repeats the cluster-wide `/version` and `/cluster/resources` per node | **rejected** |
 | CR6 | low | `lockName`, `permanentf` and `transientf` are copies of kvm2's | **rejected** |
 | CR7 | low | `readToken`, the CA load and seed staging do file I/O without a ctx | **rejected** |
 | CR10 | low | `New` runs `withDefaults` twice | **rejected** |
 | S1 | simplification | The `observe.go` decorator could be one log line in `send` | **rejected** |
 | S3 | simplification | `nocloud` exports `GuestAgentScript` and `SeedLabel` for tests only | **rejected** |
 ## Fixed
 ### CR1 + P1: the seed outlives a Delete that fails after the destroy
 `internal/provider/proxmox/delete.go`. Delete stopped the VM, destroyed it,
 then deleted the seed. If the seed delete failed (a 595, a task over its
 1 minute budget, a 403), Delete returned an error, but the VM was already
 gone. The reconciler's next `Get` was `ErrNotFound`, so `reconcileDelete`
 released the finalizer without calling Delete again, and GC lists only VMs.
 The seed stayed on `isoStorage` for good. That breaks contract §2.3 ("MUST
 remove everything Create made ... GC sees only what ListByTag returns"). A
 destroy task that outlived its budget but finished in the background had
 the same effect.
 P1 adds the second half. On the finalizer path the next Delete found the
 VMID gone and took `deleteSeedEverywhere`, which logged the same 403 at V(1)
 and returned nil. The seed leaked with nothing visible.
 **Fix.** The seed now goes between the stop and the destroy, so the VM,
 which is what `Get` looks at, is the last thing removed: a failed seed
 delete leaves the VM stopped, `Get` keeps finding it, and the next Delete
 finishes both. `deleteSeedEverywhere` now fails on an `ErrPermanent`
 answer (a 401 or 403 every node would give) instead of skipping it. It
 still skips a node that is down or lacks the storage. `runTask`'s "gone"
 result lost its last reader and was dropped.
 That Proxmox deletes an ISO a stopped VM's `ide2` still references is the
 one assumption the fake cannot prove. From memory of the PVE source, not
 checked against it: the content delete runs `vdisk_free` without looking
 for references to `iso` volumes, and the destroy skips CD-ROM drives.
 Phase 0 only ever deleted the seed after the destroy. **The Phase 10
 real-host run proved it** on 2026-09-24: the imgdel task on the stopped
 VM's seed ended `OK`, the destroy followed, and `TestProxmox_lifecycle`
 step 10 found neither the VM nor the seed.
 **Tests.** `TestDelete_seedDeleteFailureOnVMNodeFails` (403, 596 and a
 failed task on the VM's node) now asserts that no Destroy was sent, that
 `Get` still finds the VM, that a second Delete gives the same class, and
 that a Delete after the fault clears removes both. It failed on the old
 code with `Destroy calls = 1` and `Get = instance not found`.
 `TestDelete_seedDeleteNodeFailureSkippedWhenNodeUnknown/a refused privilege
 fails the call` covers the every-node 403. Updated for the new order:
 `TestDelete_stoppedVMSkipsStop`, `TestDelete_runningVMStoppedFirst`,
 `TestDelete_seedDeleteIsTaskAndIdempotent`,
 `TestDelete_destroyTaskFailureIsTransient` (the seed is now gone and the VM
 kept) and `TestDelete_goneMidwayIsSuccess` (a VM gone at the stop still
 gets a Destroy call, which answers `does not exist`).
 `TestLogging_noOpLinesAtV1`'s skip case now injects a node-level 500,
 since a 403 no longer skips.
 Docs: spec entry 11 (steps 5 to 7 reordered, the Permanent exception),
 the guide's Delete paragraph, `providers-existing.md`'s Delete bullet and
 its `delete-return` cell.
 ### CR2: a 403 on the seed delete latched a Proxy with no VM
 Same code, same fix. On the replace path (`reconcileManaged` sees a Stopped
 VM and calls Delete), the old order destroyed the VM, got a 403 on the
 seed, and returned Permanent. `providerFailure` latched it, so the Proxy
 sat Failed pointing at a VM that no longer existed. The next attempt would
 have returned nil. Now the 403 comes before the destroy: the VM stays
 stopped, and every attempt gives the same Permanent answer naming
 `Datastore.Allocate` until the privilege is granted. The guide says so.
 ### CR3 + P2: the lock wait on a VM that is plainly not ours
 `internal/provider/proxmox/create.go`, `allocate`. Any taken qemu VMID went
 through `inspect`, which first waits for the VM's lock to clear (clone
 budget, 10 minutes) and only then compares names. A VM on one of the
 Proxy's probe VMIDs, listed with another name and holding a long lock (a
 `vzdump` backup, or a stale lock a failed backup left behind), made every
 Create wait 10 minutes and return Transient. `vmidFor` is deterministic,
 so every retry landed on the same VMID. Only a clone in progress is
 nameless (F8), so a listed name other than ours settles the question.
 **Fix.** `allocate` sends a taken entry to `inspect` only when its listed
 name is empty or ours; any other name probes on without a config read.
 Spec entry 2's step 3 says so.
 **Tests.** `TestCreate_takenVMIDInspectedBeforeProbing` gains "foreign VM
 under a lock probes on without waiting" (a `snapshot-delete` lock with a
 2 s clone budget; the old code spent the 2 s and failed Transient), and
 "foreign VM probes on without a read" now expects one config read (the
 template's). `TestCreate_vmidRangeExhaustedIsTransient` expects one
 VMConfig call instead of five.
 ### P5: token rotation order
 `docs/proxmox-provider.md`, cluster-side setup. A 401 is Permanent (the
 same convention as GCP), so a request made with a revoked token before the
 kubelet refreshes the Secret latches that Proxy. The guide now gives the
 order: add the new token, update the Secret, wait for the mount, then
 remove the old one, and the same before an `--expire` date.
 ### CR5: matchers used only by tests
 `isLocked` and `isVMNotRunning` in `errors.go` had no production caller.
 Both answers are Transient through `classify`'s default branch. They moved
 to `fake_test.go`, where `TestFakeAPI_errorsMatchCaptures`,
 `TestAPIError_matchers` and the fake's own checks still use them to hold the
 fake's wording to the captures. Spec entry 13's matching sentence and
 `providers-existing.md` now name only the matchers the provider uses.
 ### CR8: test name
 `TestPermanentfAndTransientf` became `TestPermanentf_wrapsClassWithMessage`
 and `TestTransientf_wrapsClassWithMessage`. kvm2 has the same name; it is
 out of scope here.
 ### CR9: one rule, one place
 `get` asked `status == "running" && qmpstatus in {running, ""}` inline to
 decide whether to query the agent, the same rule `mapState` applies. Both
 now call `guestRunning(status, qmpStatus)` in `state.go`, so a change to it
 cannot leave the two out of step. `TestMapState_*` and the Get tests cover
 it unchanged (`TestMapState_table`, `TestMapState_fromCaptures`).
 ## Accepted
 ### P3: adoption ignores a placement change
 `create.go:94-118`. A Create that fails Permanent after the clone (a 403 on
 the upload, say) leaves a stopped VM. If the user then changes
 `placement.image` or `zone`, the next Create adopts that VM by name, and
 the Proxy runs the old template on the old node. The fix the reviewer
 proposed (record the template in the mark, re-clone a never-started VM
 whose template differs) is a new mark field and a destroy path inside
 Create. That is too much for this MR for a two-step, user-driven sequence.
 Documented in the guide's Known gaps and spec entry 24: destroy the VM
 before changing `placement` on such a Proxy.
 ### P4: a second VM after a crash mid-clone
 The retry misses a nameless in-progress clone on a later probe VMID when
 the VM on the earlier VMID was removed meanwhile, and clones again. It
 takes a crash or cut-off during a clone of a few seconds, a foreign VM on
 the first probe VMID, and that VM's removal in the same window. The
 proposed fix (inspect every nameless `lock: clone` entry in the range
 first) would make every Create wait on every other Proxy's clone. The
 outcome is visible, not silent: the next Create fails Permanent naming
 both VMIDs. Documented in the guide and spec entry 24.
 ### P6: the listing's `status` may lag
 `get.go:51`, `delete.go:60`, `delete.go:90`. This is the reviewer's
 inference, not observed: `/cluster/resources` comes from pvestatd, which
 may lag a state change by seconds. Both cases heal on retry: `Get`
 answering Transient for `unknown` is polled through, and a destroy of a VM
 that started behind the listing's back fails its task as Transient and the
 next Delete stops it first.
 **Confirmed on pv1-1.** The first `Get` came 32 ms after the second Create,
 about 6 s after the clone, and was Transient: the listing still showed the
 VM as `status: unknown`. The next poll 5 s later saw it running. The
 behaviour is right, but the message blamed the node ("its node pv1-1 is not
 answering"), which was up. Get and Delete now say "its node … is not
 answering, or the VM is too new to be reported yet".
 `TestGet_unknownStatusIsTransient` asserts the wording, and spec entry 9
 and the guide's error table name both causes. The Delete side (a stop
 skipped because the listing lags) was not met: the run's Delete came a
 minute after the start.
 ## Rejected
 - **CR4 + S2, Checker repeats cluster-wide calls per node.** Spec entry 16
  makes each node's target self-contained, so a failure is reported
  against the node whose round saw it. The cost is two extra requests per
  extra node per round, and the configured cluster
  has one node. Worth revisiting with a multi-node config, not before.
 - **CR6, helpers copied from kvm2.** `permanentf`/`transientf` are one
  line each; `lockName` is five and wraps `keyedlock.Acquire`'s error in
  `provider.ErrTransient`. Moving that into `keyedlock` would couple a
  generic lock package to the provider sentinels, and putting the helpers
  in `internal/provider` would touch every kvm2 call site in a proxmox MR.
 - **CR7, file I/O without a ctx.** `readToken` and the CA load read a
  mounted Secret with `os.ReadFile`, which cannot be cancelled; a ctx
  parameter would be unused. kvm2's `os.ReadFile` of the SSH key is the
  precedent. Seed staging is `nocloud`'s, shared with kvm2 and unchanged by
  the extraction.
 - **CR10, `withDefaults` twice in `New`.** Startup only, idempotent, a few
  small maps. `newProvider` applies it because the tests build providers
  through it; defaulting once would need a second constructor shape.
 - **S1, replace the `observe.go` decorator with one log line in `send`.**
  The decorator is spec entries 17 and 20's `span-decorator` answer: each
  seam call gets a child span and a V(1)/V(2) line with node, VMID and
  UPID, which `send` does not know. It mirrors kvm2's decorator, and the
  reflection test guarantees every seam method is covered. Swapping the
  design in the review phase is out of proportion.
 - **S3, `nocloud` exports used only by tests.** `SeedLabel` is read by
  both kvm2's and proxmox's Create tests to recognise a real seed, and
  `GuestAgentScript` by proxmox's `TestBuildSeed_table`, which tests
  proxmox's own mapping of its config fields onto the vendor-data, not
  `nocloud`'s composition. The exports are the price of cross-package
  tests.
 ## Checked and fine
 From the reviewer agent's report, verified in code:
 - The kvm2 extraction is a pure move. kvm2 still wraps every seed-build
  error as Permanent, as before, and `keyedlock` is byte-identical to the
  code it replaced.
 - Delete stops or destroys only after the listing shows a qemu entry at
  the VMID and the config carries the name and the mark; a reused VMID or
  an unmarked VM is left alone.
 - A missing VM inside Create is Transient, never NotFound; locks and
  `status: unknown` are Transient, never NotFound.
 - Create resumes at every step: the VMID comes from the UID, a nameless
  locked entry is waited on and adopted, upload/config/resize are
  repeatable, and the mark is written in the clone call.
 - Secrets: `apiError` carries status and message only, the token never
  reaches a `url.Error`, the Checker keeps only the token id, and the
  cloud-init text appears only as `cloudInitBytes`.
 - Every seam call takes the ctx; `pollUntil` honours ctx and budget;
  `keyedlock.Acquire` gives up with ctx.
 - Spec entries 1, 3, 4, 9, 11, 12, 13, 15 and 16 match the code (P2 was the
  one disagreement, and the spec shared the flaw).
 - Every §11 test name in spec entry 21 exists.
 The agent also saw kvm2's `TestCheck_hostsCheckedConcurrently` fail once
 under `-race` (199.9 ms against its 200 ms floor). That file is unchanged
 on this branch; the flake is parked, as before.
@@ -71,9 +71,9 @@ manual runs too.
 | `internal/provider/keyedlock` | `keyedlock_test.go` | the per-name lock kvm2 and proxmox share: the entry is removed once uncontended, waiters share one lock, a waiter gives up with its ctx | pure |
 | `internal/provider/proxmox` | `id_test.go`, `mark_test.go`, `state_test.go`, `seed_test.go`, `errors_test.go`, `api_test.go` | pure pieces: providerID round-trip and malformed forms, the VMID derived from the UID (deterministic, in range, probing forward and wrapping), the description mark (keys, parsed from the captured clone, admin lines kept), state mapping and the IPv4 filter from captures, seed names and building (empty name Permanent without its payload, staging failure Transient); the `classify` table and the 500-message matchers against every captured error; the API types decoded from the captures | pure, over the responses Phase 0 captured from PVE 9.2.20 (`testdata/`) |
 | | `client_test.go` | the thin HTTP client: the token read from its file on every request (missing Transient, unparsable Permanent), `apiError` from status, reason phrase and body, the data envelope, method, path and parameters of every seam call, the clone and destroy query shapes, the volid's slash escaped, HTTP/1.1 only, request timeout and cancel Transient and the client unaffected, a child span under a parent and none without, the token never in errors or logs, the CA file trusted and loaded lazily, an x509 failure Permanent, insecure mode, the upload streamed as multipart | `httptest.NewTLSServer` replaying the captures byte for byte |
-| | `create_test.go`, `get_test.go`, `delete_test.go`, `list_test.go`, `task_test.go`, `proxmox_test.go`, `provider_test.go`, `fake_test.go` | Create: resume after every partial step and after a crash mid-clone, the VMID probe and ownership check, the template's shape refused before any clone, resize as a task, the seed overwritten without a delete, the net0 bridge and VLAN rewrite keeping the MAC, returns only when running, cloud-init sentinel never logged; Get: lock checked before name, the agent's messages, paused from `status/current`, hibernated, a reused VMID, four calls when running; Delete: every already-gone and not-ours case, the seed deleted on the VM's node or on every node, hibernated Permanent, the per-name lock; ListByTag: candidates by tag or name prefix, the mark decides, an unreadable VM skipped unless ctx ended; task and lock waits; `New` makes no request; the contract sweep: every method returns within a second of a cancel (blocked on its first seam call, and mid-operation), every error wrapped with the configured name, op and id, logging tiers V0/V1/V2 with the no-op lines | in-memory `fakeAPI` behind the `api` seam, its error texts checked against the captures |
+| | `create_test.go`, `get_test.go`, `delete_test.go`, `list_test.go`, `task_test.go`, `proxmox_test.go`, `provider_test.go`, `fake_test.go` | Create: resume after every partial step and after a crash mid-clone, the VMID probe and ownership check, the template's shape refused before any clone, resize as a task, the seed overwritten without a delete, the net0 bridge and VLAN rewrite keeping the MAC, returns only when running, cloud-init sentinel never logged; Get: lock checked before name, the agent's messages, paused from `status/current`, hibernated, a reused VMID, four calls when running; Delete: every already-gone and not-ours case, the seed deleted before the destroy on the VM's node or on every node, a hibernated VM destroyed, the per-name lock; ListByTag: candidates by tag or name prefix, the mark decides, an unreadable VM skipped unless ctx ended; task and lock waits; `New` makes no request; the contract sweep: every method returns within a second of a cancel (blocked on its first seam call, and mid-operation), every error wrapped with the configured name, op and id, logging tiers V0/V1/V2 with the no-op lines | in-memory `fakeAPI` behind the `api` seam, its error texts checked against the captures |
 | | `observe_test.go`, `check_test.go`, `provider_test.go` | the decorator: every seam method logs exactly one line (guarded by reflection), starts no span, identifiers at V1, detail at V2, upload body never logged, error fields; a Get through `provider.WithTracing` over the real client makes four `HTTP GET` spans, all children of `provider.get` and none carrying the token; health check: one entry per node in config order, nodes checked concurrently (a barrier, not a timer), expired ctx returns promptly, the reasons table at V2 with a sentinel secret, the transport reasons from real `net/http` and `crypto/tls` errors, detail keys | `fakeAPI` + `funcr` capture + span recorder; the real client over `httptest` for the tracing and transport-reason rows |
-| | `proxmox_integration_test.go` (`-tags=integration`) | one real VM: `New` without a request, `Check` Ready with details, Create, idempotent re-Create with an unchanged VM count, Running with an IPv4 and the mark read back, ListByTag, Delete, NotFound, no VM and no seed left, HTTP spans under `provider.create`; `TestProxmoxCheck_realHost` runs the health check against the host and creates nothing. **Compiles, not yet run against a host** | a **real Proxmox VE host** — see below |
+| | `proxmox_integration_test.go` (`-tags=integration`) | one real VM: `New` without a request, `Check` Ready with details, Create, idempotent re-Create with an unchanged VM count, Running with an IPv4 and the mark read back, ListByTag, Delete, NotFound, no VM and no seed left, HTTP spans under `provider.create`; `TestProxmoxCheck_realHost` runs the health check against the host and creates nothing. **Passed on pv1-1 (PVE 9.2.20), 2026-09-24**: Create 6.2 s, address 50 s later, whole run 61 s | a **real Proxmox VE host** — see below |
 | `internal/health` | `probe_test.go` | probes through a **real CONNECT-capable proxy stub** to a real TLS server: tunnel success, refused CONNECT, wrong status, dead proxy, plain-http forward | `httptest` + hijacked bidirectional tunnel |
 | | `engine_test.go` | thresholds, first-verdict, latency suppression matrix, dropped-event retry, stale-probe UID guard, tick scheduling/seeding/pruning, `Start` end-to-end | fake `client.Reader` + injected `probeFn` |
 | `internal/lease` | `store_test.go` | capacity, `MaxLeases=0`, all three selection tie-breaks, target-scoped vs global cooldowns, expiry via fake clock, report-on-retained-lease, idempotent release, **40 concurrent acquires never exceeding capacity** | injectable clock; the concurrency case is why `-race` matters |
@@ -117,19 +117,21 @@ envtest suite the scaffold generated.
  `httptest` replaying responses captured from PVE 9.2.20. Whether a
  clone, config write, seed upload and start yield a guest that boots and
  reports an address is provable only on a real host (the proxmox
-  integration test below). Two kinds of path have met only the fake:
+  integration test below, which has passed on pv1-1). Two kinds of path
  have met only the fake:
  - **the cross-node paths**: a clone that targets another node, the seed
    deleted on every node when the VM's node is unknown, a Check round over
    several nodes. The test host is a single node, so the integration test
    cannot reach them either
    ([proxmox-provider-spec.md](proxmox-provider-spec.md) entry 24);
-  - **answers never captured**: what the storage status call returns for a
+  - **an answer never captured**: the storage status for a storage the
-    missing or disabled storage (every 500 there is taken as
+    cluster knows but cannot use on a node (disabled there, or offline).
-    `StorageNotFound`), what destroy says for a hibernated VM, and a stop
+    Every 500 there is taken as `StorageNotFound`, and what the fake
-    on a paused VM. The provider does not depend on their wording: any
+    answers is its own guess. The three answers Phase 0 missed were
-    500 from the storage status is `StorageNotFound`, a hibernated VM is
+    captured on pv1-1 on 2026-09-24: a missing storage is a 400 with
-    refused before destroy is called, and a paused VM gets the same stop
+    `storage: No such storage.`, a hibernated VM is destroyed without
-    as a running one. What the fake answers there is its own guess.
+    complaint (its suspend state volume with it), and a stop on a paused
    VM ends `OK` with the VM stopped.
 ## The KVM integration test (real hypervisor)
@@ -303,6 +305,7 @@ export PROXMOX_TEST_CA_FILE=$HOME/.config/proxmox-ca.pem    # the node's pve-roo
 export PROXMOX_TEST_NODE=pv1-1
 export PROXMOX_TEST_TEMPLATE_VMID=9001
 export PROXMOX_TEST_ISO_STORAGE=proxy-seeds                 # holds iso content
 export PROXMOX_TEST_VLAN_TAG=4                              # template 9001's net0 is untagged; proxies live on VLAN 4
 make test-integration-proxmox
 ```
@@ -381,10 +384,26 @@ variables; to run only this test:
 go test -tags=integration -count=1 -v -run TestProxmoxCheck ./internal/provider/proxmox/
 ```
-**Neither test has run against a host yet.** Both compile
+**Last run: 2026-09-24 on pv1-1** (Proxmox VE 9.2.20, kernel
-(`go vet -tags=integration ./...`); Phase 10 of the plan runs them on
+7.0.14-17-pve, template 9001, `proxy-seeds`, VLAN tag 4). Both tests
-pv1-1. Like the kvm tests they are not part of `make test` and not run in
+passed; `go test` reported 60.9 s for the package.
-CI.
+
 | Step | Time |
 |---|---|
 | `Check` | 141 ms (125 ms in `TestProxmoxCheck_realHost`) |
 | clone task, submit to done | about 3 s (seen done at the third 1 s poll) |
 | `Create`, clone to started VM | 6.2 s |
 | second `Create` (adoption, nothing created) | 32 ms |
 | Create's return to the first address | 50.2 s (the agent install from the internet) |
 | `Delete` (stop, seed delete, destroy), `Get`, second `Delete` | 3.1 s |
 | `TestProxmox_lifecycle` | 59.7 s |
 The first `Get`, 32 ms after the second Create, was Transient: the
 listing still showed the new VM with `status: unknown`. The next poll 5 s
 later saw it running. The Delete deleted the seed while the stopped VM's
 `ide2` still referenced it, then destroyed the VM, and step 10 found
 neither left. Like the kvm tests they are not part of `make test` and not
 run in CI.
 ## The scaffolded `make test-e2e` suite
@@ -111,16 +111,22 @@ func (p *Provider) probeNode(ctx context.Context, node string, details map[strin
 	return p.checkTemplates(ctx, details)
 }
-// checkStorage reads isoStorage on the node. A 500 means Proxmox cannot
+// checkStorage reads isoStorage on the node. A storage the cluster does
-// use that storage on this node (it does not exist, is not enabled here,
+// not know is a 400 on the storage parameter (captured). A 500 is taken
-// or is offline), whatever its wording; the message carries Proxmox's
+// the same way, whatever its wording: Proxmox cannot use that storage on
-// text.
+// this node (not enabled here, or offline; not captured). The message
 // carries Proxmox's text.
 func (p *Provider) checkStorage(ctx context.Context, node string, details map[string]string) error {
 	storage := p.cfg.ISOStorage
 	st, err := p.api.StorageStatus(ctx, node, storage)
 	var ae *apiError
-	if errors.As(err, &ae) && ae.Status == http.StatusInternalServerError {
+	if errors.As(err, &ae) {
-		return &checkFailure{reason: reasonStorageNotFound, msg: fmt.Sprintf("storage %s on node %s: %s", storage, node, ae.Message)}
+		switch {
 		case isNoSuchStorage(ae):
 			return &checkFailure{reason: reasonStorageNotFound, msg: fmt.Sprintf("storage %s on node %s: %s", storage, node, ae.Errors["storage"])}
 		case ae.Status == http.StatusInternalServerError:
 			return &checkFailure{reason: reasonStorageNotFound, msg: fmt.Sprintf("storage %s on node %s: %s", storage, node, ae.Message)}
 		}
 	}
 	if err != nil {
 		return fmt.Errorf("reading storage %s on node %s: %w", storage, node, err)
@@ -230,7 +230,21 @@ func TestCheck_reasonsTable(t *testing.T) {
 				fx.fake.mu.Unlock()
 			},
 			wantReason:  reasonStorageNotFound,
-			wantMessage: "storage proxy-seeds on node pv1-1: fake: storage 'proxy-seeds' does not exist",
+			wantMessage: "storage proxy-seeds on node pv1-1: No such storage.",
 		},
 		{
 			// Not captured: a storage Proxmox knows but cannot use here.
 			name: "iso storage unusable on the node",
 			prep: func(fx *fixture) {
 				fx.fake.failWith("StorageStatus", fakeErr(500, "fake: storage 'proxy-seeds' is not online"))
 			},
 			wantReason:  reasonStorageNotFound,
 			wantMessage: "storage proxy-seeds on node pv1-1: fake: storage 'proxy-seeds' is not online",
 		},
 		{
 			name:       "another bad parameter is not a missing storage",
 			prep:       func(fx *fixture) { fx.fake.failWith("StorageStatus", loadAPIError(fx.t, "err_bad_param")) },
 			wantReason: provider.ReasonAPIError,
 		},
 		{
 			name:        "iso storage inactive",
@@ -198,9 +198,10 @@ func (p *Provider) template(ctx context.Context, rs []vmResource, image string,
 }
 // allocate is the probe of spec entry 2: the attempt-th VMID for the UID
-// is cloned onto when the listing shows it free and put through the
+// is cloned onto when the listing shows it free, and put through the
 // ownership check when it is taken, or when the clone finds it taken after
-// all (F4). The boolean reports an adoption rather than a fresh clone.
+// all (F4), by a VM that is nameless or carries our name. The boolean
 // reports an adoption rather than a fresh clone.
 func (p *Provider) allocate(ctx context.Context, log logr.Logger, rs []vmResource, node string, src vmResource, req provider.CreateRequest) (ownedVM, bool, error) {
 	taken := make(map[int]vmResource, len(rs))
 	for _, r := range rs {
@@ -225,7 +226,9 @@ func (p *Provider) allocate(ctx context.Context, log logr.Logger, rs []vmResourc
 				continue
 			}
 		}
-		if r.Type == "qemu" && r.Template == 0 {
+		// A listed name other than ours is someone else's VM whatever its
 		// lock: only a clone in progress is nameless.
 		if r.Type == "qemu" && r.Template == 0 && (r.Name == "" || r.Name == req.Name) {
 			vm, ours, err := p.inspect(ctx, r, req)
 			if err != nil || ours {
 				return vm, ours, err
@@ -248,10 +248,22 @@ func TestCreate_takenVMIDInspectedBeforeProbing(t *testing.T) {
 		wantConfigReads int
 	}{
 		{
-			name:            "foreign VM probes on after its config is read",
+			name:            "foreign VM probes on without a read",
 			setup:           func(fx *fixture) { fx.fake.addVM(slot(0), "someone-else", "running") },
 			wantAttempt:     1,
-			wantConfigReads: 2,
+			wantConfigReads: 1,
 		},
 		{
 			// Review finding P2: another VM's lock (a backup, a snapshot)
 			// can outlast the clone budget, and every retry lands on the
 			// same VMID.
 			name: "foreign VM under a lock probes on without waiting",
 			setup: func(fx *fixture) {
 				fx.fake.addVM(slot(0), "someone-else", "running").lock = "snapshot-delete"
 				fx.p.budgets.clone = 2 * time.Second
 			},
 			wantAttempt:     1,
 			wantConfigReads: 1,
 		},
 		{
 			name: "container probes on without a read",
@@ -581,8 +593,8 @@ func TestCreate_vmidRangeExhaustedIsTransient(t *testing.T) {
 	if got := fx.fake.callCount("Clone"); got != 0 {
 		t.Errorf("Clone calls = %d, want 0", got)
 	}
-	if got := fx.fake.callCount("VMConfig"); got != 5 {
+	if got := fx.fake.callCount("VMConfig"); got != 1 {
-		t.Errorf("VMConfig calls = %d, want the template and each of the 4 VMIDs once", got)
+		t.Errorf("VMConfig calls = %d, want only the template's: the listed names are foreign", got)
 	}
 }
@@ -2,14 +2,17 @@ package proxmox
 import (
 	"context"
 	"errors"
 	"fmt"
 	"time"
 	"github.com/go-logr/logr"
 	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
 )
-// Delete is spec entry 11: under the per-name lock, stop and destroy the VM
+// Delete is spec entry 11: under the per-name lock, stop the VM when it is
-// when it is ours, then delete its seed. Everything already gone is
+// ours, delete its seed, then destroy it. Everything already gone is
 // success, and nothing of ours is left when nil is returned, so Get
 // afterwards is NotFound.
 func (p *Provider) Delete(ctx context.Context, id string) error {
@@ -29,19 +32,17 @@ func (p *Provider) Delete(ctx context.Context, id string) error {
 		return p.wrapErr("delete", id, err)
 	}
 	if node == "" {
-		err = p.deleteSeedEverywhere(ctx, log, name)
+		if err := p.deleteSeedEverywhere(ctx, log, name); err != nil {
 			return p.wrapErr("delete", id, err)
 		}
 	} else {
 		log = log.WithValues("node", node)
 		err = p.deleteSeed(ctx, log, node, name)
 	}
 	if err != nil {
 		return p.wrapErr("delete", id, err)
 	}
 	log.V(1).Info("proxmox vm deleted", "destroyed", node != "")
 	return nil
 }
-// destroy is steps 2 to 6. It returns the node of the VM it destroyed, or
+// destroy is steps 2 to 7. It returns the node of the VM it destroyed, or
 // "" when there was no VM of ours to destroy: then the seed's node is not
 // known, and the seed is looked for on every node.
 func (p *Provider) destroy(ctx context.Context, log logr.Logger, vmid int, name string) (string, error) {
@@ -58,7 +59,7 @@ func (p *Provider) destroy(ctx context.Context, log logr.Logger, vmid int, name
 		return "", transientf("VM %d is locked (%s)", vmid, r.Lock)
 	}
 	if r.Status == "unknown" {
-		return "", transientf("VM %d has status unknown: its node %s is not answering", vmid, r.Node)
+		return "", transientf("VM %d has status unknown: its node %s is not answering, or the VM is too new to be reported yet", vmid, r.Node)
 	}
 	cfg, err := p.api.VMConfig(ctx, r.Node, vmid)
 	if isNotExist(err) {
@@ -76,47 +77,47 @@ func (p *Provider) destroy(ctx context.Context, log logr.Logger, vmid int, name
 		log.V(1).Info("proxmox vm carries no proxy-operator mark, left alone")
 		return "", nil
 	}
-	switch lock := cfg["lock"]; lock {
+	// A hibernated VM is stopped with lock suspended, and the destroy
-	case "":
+	// takes it, state volume included (captured on pv1-1).
-	case lockSuspended:
+	if lock := cfg["lock"]; lock != "" && lock != lockSuspended {
 		// Destroy refuses a locked VM and the lock never clears on its own,
 		// so only an admin can move this on, as in Create's ownership check.
 		return "", permanentf("VM %d named %s is hibernated (lock suspended); resume or destroy it in Proxmox", vmid, name)
 	default:
 		return "", transientf("VM %d is locked (%s)", vmid, lock)
 	}
 	// The listing shows a paused VM as running, and Stop ends it too.
 	if r.Status != "stopped" {
-		gone, err := p.runTask(ctx, log, "stopping", vmid, p.budgets.stop, func() (string, error) { return p.api.Stop(ctx, r.Node, vmid) })
+		if err := p.runTask(ctx, log, "stopping", vmid, p.budgets.stop, func() (string, error) { return p.api.Stop(ctx, r.Node, vmid) }); err != nil {
-		if err != nil || gone {
+			return "", err
 			return r.Node, err
 		}
 	}
-	if _, err := p.runTask(ctx, log, "destroying", vmid, p.budgets.destroy, func() (string, error) { return p.api.Destroy(ctx, r.Node, vmid) }); err != nil {
+	// The seed goes while the VM is still there: once it is gone, Get is
 	// NotFound and nothing calls Delete again to finish the job.
 	if err := p.deleteSeed(ctx, log, r.Node, name); err != nil {
 		return "", err
 	}
 	if err := p.runTask(ctx, log, "destroying", vmid, p.budgets.destroy, func() (string, error) { return p.api.Destroy(ctx, r.Node, vmid) }); err != nil {
 		return "", err
 	}
 	return r.Node, nil
 }
 // runTask submits a task on a VM and waits for it. A VM that is gone by the
-// time the task is submitted reports gone: it is what the task was for.
+// time the task is submitted is success: it is what the task was for.
-func (p *Provider) runTask(ctx context.Context, log logr.Logger, what string, vmid int, budget time.Duration, submit func() (string, error)) (bool, error) {
+func (p *Provider) runTask(ctx context.Context, log logr.Logger, what string, vmid int, budget time.Duration, submit func() (string, error)) error {
 	upid, err := submit()
 	if isNotExist(err) {
 		log.V(1).Info("proxmox vm already gone")
-		return true, nil
+		return nil
 	}
 	if err != nil {
-		return false, fmt.Errorf("%s VM %d: %w", what, vmid, err)
+		return fmt.Errorf("%s VM %d: %w", what, vmid, err)
 	}
 	if err := p.waitTask(ctx, log, upid, budget); err != nil {
-		return false, fmt.Errorf("%s VM %d: %w", what, vmid, err)
+		return fmt.Errorf("%s VM %d: %w", what, vmid, err)
 	}
-	return false, nil
+	return nil
 }
-// deleteSeed is step 7 on one node. A missing volume still ends in a
+// deleteSeed is step 6 on one node. A missing volume still ends in a
 // successful task (F13), so nothing is matched.
 func (p *Provider) deleteSeed(ctx context.Context, log logr.Logger, node, name string) error {
 	volid := seedVolID(p.cfg.ISOStorage, name)
@@ -130,17 +131,19 @@ func (p *Provider) deleteSeed(ctx context.Context, log logr.Logger, node, name s
 	return nil
 }
-// deleteSeedEverywhere is step 7 when the VM's node is not known: every
+// deleteSeedEverywhere is step 6 when the VM's node is not known: every
 // configured node is tried, and one that fails is logged and skipped, so a
 // node that is down or lacks the storage cannot hold the Delete up. An
-// ended ctx fails the call instead.
+// ended ctx fails the call instead, and so does a Permanent refusal such as
 // a 403 for Datastore.Allocate, which every node would give and which
 // would otherwise leave the seed behind unseen.
 func (p *Provider) deleteSeedEverywhere(ctx context.Context, log logr.Logger, name string) error {
 	for _, node := range p.cfg.Nodes {
 		err := p.deleteSeed(ctx, log, node, name)
 		if err == nil {
 			continue
 		}
-		if ctx.Err() != nil {
+		if ctx.Err() != nil || errors.Is(classify("delete", err), provider.ErrPermanent) {
 			return err
 		}
 		log.V(1).Info("proxmox seed delete failed on a node, skipped", "node", node, "reason", err.Error())
@@ -203,6 +203,14 @@ func TestDelete_unmarkedLeavesVMAlone(t *testing.T) {
 // wait for the stop fails.
 func TestDelete_runningVMStoppedFirst(t *testing.T) {
 	t.Parallel()
 	// pv1-1 took a stop on a paused VM as a task that ended OK with the VM
 	// stopped (Phase 10 captures), which is what the fake does for all three.
 	accepted := readCapture(t, "stop_paused_response")
 	done := decodeData[taskStatus](t, "task_status_stop_paused")
 	after := decodeData[vmStatus](t, "status_current_after_stop_paused")
 	if accepted.Status != http.StatusOK || done.Type != "qmstop" || done.ExitStatus != "OK" || after.Status != "stopped" || after.QMPStatus != "stopped" {
 		t.Fatalf("captures = %d, %+v, %+v; want the stop accepted, OK, and the VM stopped", accepted.Status, done, after)
 	}
 	for _, qmp := range []string{"running", "paused", "prelaunch"} {
 		t.Run(qmp, func(t *testing.T) {
 			t.Parallel()
@@ -215,8 +223,8 @@ func TestDelete_runningVMStoppedFirst(t *testing.T) {
 			if stop < 0 || destroy < stop {
 				t.Fatalf("calls = %v, want Stop before Destroy", log)
 			}
-			if n := callsBefore(log[stop:], "TaskStatus", "Destroy"); n != fx.fake.taskPolls+1 {
+			if n := callsBefore(log[stop:], "TaskStatus", "DeleteVolume"); n != fx.fake.taskPolls+1 {
-				t.Errorf("stop task polled %d times before Destroy, want %d", n, fx.fake.taskPolls+1)
+				t.Errorf("stop task polled %d times before the seed delete, want %d", n, fx.fake.taskPolls+1)
 			}
 			if _, ok := fx.fake.vmSnapshot(testVMID); ok {
 				t.Error("VM still exists")
@@ -230,7 +238,7 @@ func TestDelete_stoppedVMSkipsStop(t *testing.T) {
 	fx := newFixture(t, 1)
 	fx.addOurVM(testVMID, testName, "stopped")
 	fx.delete()
-	want := []string{"ClusterResources", "VMConfig", "Destroy", "TaskStatus", "DeleteVolume", "TaskStatus"}
+	want := []string{"ClusterResources", "VMConfig", "DeleteVolume", "TaskStatus", "Destroy", "TaskStatus"}
 	if got := fx.fake.callLog(); !slices.Equal(got, want) {
 		t.Errorf("calls = %v, want %v", got, want)
 	}
@@ -291,26 +299,32 @@ func TestDelete_unknownStatusIsTransient(t *testing.T) {
 	}
 }
-// TestDelete_hibernatedVMIsPermanentIfOurs matches Create's ownership
+// TestDelete_hibernatedVMIsDestroyedIfOurs: on pv1-1 the destroy took a
-// check: Destroy refuses the suspended lock, which never clears on its own,
+// hibernated VM, suspend state volume included, and its task ended OK.
-// so an admin must act. A hibernated VM that is not ours is left alone.
+// Delete used to refuse it as Permanent on the belief that the destroy
-func TestDelete_hibernatedVMIsPermanentIfOurs(t *testing.T) {
+// would. A hibernated VM that is not ours is still left alone.
 func TestDelete_hibernatedVMIsDestroyedIfOurs(t *testing.T) {
 	t.Parallel()
 	t.Run("ours", func(t *testing.T) {
 		t.Parallel()
 		accepted := readCapture(t, "destroy_hibernated_response")
 		done := decodeData[taskStatus](t, "task_status_destroy_hibernated")
 		if accepted.Status != http.StatusOK || done.Type != "qmdestroy" || done.ExitStatus != "OK" {
 			t.Fatalf("captures = %d, %+v; want the destroy accepted and its task OK", accepted.Status, done)
 		}
 		fx := newFixture(t, 1)
 		fx.addOurVM(testVMID, testName, "stopped").lock = lockSuspended
 		fx.putSeed()
-		if msg := fx.deleteFails(provider.ErrPermanent); !strings.Contains(msg, "resume or destroy it in Proxmox") {
+		fx.delete()
-			t.Errorf("error = %s, want the admin action", msg)
+		want := []string{"ClusterResources", "VMConfig", "DeleteVolume", "TaskStatus", "Destroy", "TaskStatus"}
 		if got := fx.fake.callLog()[2:]; !slices.Equal(got, want) {
 			t.Errorf("calls = %v, want %v: no stop, since the VM is not running", got, want)
 		}
-		for _, method := range []string{"Stop", "Destroy", "DeleteVolume"} {
+		if _, ok := fx.fake.vmSnapshot(testVMID); ok {
-			if n := fx.fake.callCount(method); n != 0 {
+			t.Error("hibernated VM still exists")
 				t.Errorf("%s calls = %d, want 0", method, n)
 			}
 		}
-		if _, err := fx.fake.Destroy(fx.ctx, fakeNode, testVMID); !isLocked(err) {
+		if fx.seedPresent() {
-			t.Errorf("fake Destroy on the hibernated VM error = %v, want the lock answer", err)
+			t.Error("seed still exists")
 		}
 	})
 	t.Run("someone else's", func(t *testing.T) {
@@ -367,7 +381,7 @@ func TestDelete_seedDeleteIsTaskAndIdempotent(t *testing.T) {
 			for log[last] != "DeleteVolume" {
 				last--
 			}
-			if n := callsBefore(log[last:], "TaskStatus", ""); n != fx.fake.taskPolls+1 {
+			if n := callsBefore(log[last:], "TaskStatus", "Destroy"); n != fx.fake.taskPolls+1 {
 				t.Errorf("seed delete task polled %d times, want %d: %v", n, fx.fake.taskPolls+1, log)
 			}
 			if fx.seedPresent() {
@@ -377,6 +391,9 @@ func TestDelete_seedDeleteIsTaskAndIdempotent(t *testing.T) {
 	}
 }
 // TestDelete_seedDeleteFailureOnVMNodeFails: the VM outlives a failed seed
 // delete, so Get keeps finding it and the next Delete finishes the job with
 // the same class of answer (review findings CR1, CR2, P1).
 func TestDelete_seedDeleteFailureOnVMNodeFails(t *testing.T) {
 	t.Parallel()
 	tests := []struct {
@@ -403,9 +420,22 @@ func TestDelete_seedDeleteFailureOnVMNodeFails(t *testing.T) {
 			fx := newFixture(t, 1)
 			fx.addOurVM(testVMID, testName, "stopped")
 			tc.setup(fx)
 			fx.putSeed()
 			fx.deleteFails(tc.class)
 			if n := fx.fake.callCount("Destroy"); n != 0 {
 				t.Errorf("Destroy calls = %d, want 0 while the seed is still there", n)
 			}
 			if _, err := fx.p.Get(fx.ctx, fx.id()); errors.Is(err, provider.ErrNotFound) {
 				t.Fatalf("Get = %v, want the VM still found so Delete runs again", err)
 			}
 			fx.deleteFails(tc.class)
 			fx.fake.clearHooks()
 			fx.delete()
 			if _, ok := fx.fake.vmSnapshot(testVMID); ok {
-				t.Error("VM still exists: the destroy should have run before the seed delete")
+				t.Error("VM still exists after the retry")
 			}
 			if fx.seedPresent() {
 				t.Error("seed still exists after the retry")
 			}
 		})
 	}
@@ -436,6 +466,18 @@ func TestDelete_seedDeleteNodeFailureSkippedWhenNodeUnknown(t *testing.T) {
 			t.Error("seed still exists after pv1-2's delete")
 		}
 	})
 	t.Run("a refused privilege fails the call", func(t *testing.T) {
 		t.Parallel()
 		fx := twoNodeFixture(t)
 		fx.putSeed()
 		fx.fake.failWith("DeleteVolume", loadAPIError(t, "err_forbidden_volume_delete"))
 		if msg := fx.deleteFails(provider.ErrPermanent); !strings.Contains(msg, "Datastore.Allocate") {
 			t.Errorf("error = %s, want the missing privilege named", msg)
 		}
 		if n := fx.fake.callCount("DeleteVolume"); n != 1 {
 			t.Errorf("DeleteVolume calls = %d, want 1: every node would refuse the same", n)
 		}
 	})
 	t.Run("ended ctx fails the call", func(t *testing.T) {
 		t.Parallel()
 		fx := twoNodeFixture(t)
@@ -465,8 +507,8 @@ func TestDelete_destroyTaskFailureIsTransient(t *testing.T) {
 	if _, ok := fx.fake.vmSnapshot(testVMID); !ok {
 		t.Error("VM gone after a failed destroy task")
 	}
-	if !fx.seedPresent() {
+	if fx.seedPresent() {
-		t.Error("seed deleted although the VM is still there")
+		t.Error("seed still exists: it goes before the destroy")
 	}
 	if lines := fx.logs.matching(`"msg"="proxmox task failed"`); len(lines) != 1 {
 		t.Errorf("task failed lines = %v, want one", lines)
@@ -485,7 +527,7 @@ func TestDelete_goneMidwayIsSuccess(t *testing.T) {
 		wantNodes   []string
 	}{
 		{name: "before the config read", status: "stopped", method: "VMConfig", wantNodes: []string{fakeNode, "pv1-2"}},
-		{name: "before the stop", status: "running", method: "Stop", wantNodes: []string{fakeNode}},
+		{name: "before the stop", status: "running", method: "Stop", wantDestroy: 1, wantNodes: []string{fakeNode}},
 		{name: "before the destroy", status: "stopped", method: "Destroy", wantDestroy: 1, wantNodes: []string{fakeNode}},
 	}
 	for _, tc := range tests {
@@ -76,14 +76,13 @@ func newAPIError(status int, reason string, body []byte) *apiError {
 // under testdata/.
 var (
 	vmConfigMissing = regexp.MustCompile(`^Configuration file '[^']*' does not exist$`)
 	vmNotRunning    = regexp.MustCompile(`^VM [0-9]+ is not running$`)
 )
 const (
 	msgLocked             = "VM is locked"
 	msgAlreadyExists      = "config file already exists"
 	msgAgentNotRunning    = "QEMU guest agent is not running"
 	msgAgentNotConfigured = "No QEMU guest agent configured"
 	msgNoSuchStorage      = "No such storage."
 )
 // server500 returns the message of err when it is, or wraps, an apiError
@@ -102,10 +101,12 @@ func isNotExist(err error) bool {
 	return ok && vmConfigMissing.MatchString(msg)
 }
-// isLocked reports a VM held by a running task ("VM is locked (clone)").
+// isNoSuchStorage reports a storage the cluster does not know: a 400 whose
-func isLocked(err error) bool {
+// storage field error is exactly msgNoSuchStorage. Other storage field
-	msg, ok := server500(err)
+// errors, such as a clone onto a storage without image support, are not.
-	return ok && strings.HasPrefix(msg, msgLocked)
+func isNoSuchStorage(err error) bool {
 	var ae *apiError
 	return errors.As(err, &ae) && ae.Status == http.StatusBadRequest && ae.Errors["storage"] == msgNoSuchStorage
 }
 // isAlreadyExists reports a clone onto a VMID that is taken.
@@ -127,14 +128,6 @@ func isAgentNotConfigured(err error) bool {
 	return ok && msg == msgAgentNotConfigured
 }
 // isVMNotRunning reports an agent call on a VM that is stopped. It is
 // matched on the whole message because " is not running" alone is also a
 // suffix of the agent-not-running answer.
 func isVMNotRunning(err error) bool {
 	msg, ok := server500(err)
 	return ok && vmNotRunning.MatchString(msg)
 }
 // classify maps any error reaching the provider's boundary to one of the
 // four contract sentinels, per spec entry 13. op is the contract operation:
 // a vanished VM inside Create is Transient, because the contract forbids
@@ -93,6 +93,7 @@ func TestAPIError_matchers(t *testing.T) {
 		"isAlreadyExists":      isAlreadyExists,
 		"isAgentNotRunning":    isAgentNotRunning,
 		"isAgentNotConfigured": isAgentNotConfigured,
 		"isNoSuchStorage":      isNoSuchStorage,
 		"isVMNotRunning":       isVMNotRunning,
 	}
 	tests := []struct {
@@ -104,6 +105,7 @@ func TestAPIError_matchers(t *testing.T) {
 		{"err_destroy_missing", "isNotExist"},
 		{"err_config_while_clone_locked", "isLocked"},
 		{"err_newid_in_use", "isAlreadyExists"},
 		{"storage_status_missing", "isNoSuchStorage"},
 		{"err_agent_not_running", "isAgentNotRunning"},
 		{"err_agent_vm_stopped", "isAgentNotConfigured"},
 		{"err_agent_vm_stopped_agent_enabled", "isVMNotRunning"},
@@ -184,12 +186,16 @@ func TestAPIError_errorString(t *testing.T) {
 	}
 }
-func TestPermanentfAndTransientf(t *testing.T) {
+func TestPermanentf_wrapsClassWithMessage(t *testing.T) {
 	t.Parallel()
 	perm := permanentf("node %q missing", "x")
 	if !errors.Is(perm, provider.ErrPermanent) || perm.Error() != `provider: permanent error: node "x" missing` {
 		t.Errorf("permanentf() = %q, want ErrPermanent with the message", perm)
 	}
 }
 func TestTransientf_wrapsClassWithMessage(t *testing.T) {
 	t.Parallel()
 	trans := transientf("retry %d", 3)
 	if !errors.Is(trans, provider.ErrTransient) || trans.Error() != "provider: transient error: retry 3" {
 		t.Errorf("transientf() = %q, want ErrTransient with the message", trans)
@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"io"
 	"maps"
 	"regexp"
 	"slices"
 	"strconv"
 	"strings"
@@ -140,6 +141,13 @@ func (f *fakeAPI) hookOn(method string, fn func() error) {
 	f.hooks[method] = fn
 }
 // clearHooks drops every hook, so the faults they injected stop.
 func (f *fakeAPI) clearHooks() {
 	f.mu.Lock()
 	defer f.mu.Unlock()
 	clear(f.hooks)
 }
 func (f *fakeAPI) blockOn(method string) chan struct{} {
 	f.mu.Lock()
 	defer f.mu.Unlock()
@@ -191,8 +199,30 @@ func errVMMissing(node string, vmid int) *apiError {
 	return fakeErr(500, fmt.Sprintf("Configuration file 'nodes/%s/qemu-server/%d.conf' does not exist", node, vmid))
 }
 // errNoSuchStorage is the answer for a storage the cluster does not know
 // (Phase 10 capture storage_status_missing).
 func errNoSuchStorage() *apiError {
 	return newAPIError(400, "", []byte(`{"message":"Parameter verification failed.\n","data":null,"errors":{"storage":"No such storage."}}`))
 }
 func errLocked(lock string) *apiError { return fakeErr(500, "VM is locked ("+lock+")") }
 // isLocked and isVMNotRunning recognise two answers the provider leaves to
 // classify's Transient default. They live here only to hold the fake's
 // wording to the captures. The stopped-VM message is matched whole because
 // " is not running" is also a suffix of the agent-not-running answer.
 var vmNotRunning = regexp.MustCompile(`^VM [0-9]+ is not running$`)
 func isLocked(err error) bool {
 	msg, ok := server500(err)
 	return ok && strings.HasPrefix(msg, "VM is locked")
 }
 func isVMNotRunning(err error) bool {
 	msg, ok := server500(err)
 	return ok && vmNotRunning.MatchString(msg)
 }
 func errNewIDInUse(vmid int) *apiError {
 	return fakeErr(500, fmt.Sprintf("unable to create VM %d: config file already exists", vmid))
 }
@@ -272,7 +302,7 @@ func (f *fakeAPI) StorageStatus(ctx context.Context, node, storage string) (stor
 	defer f.mu.Unlock()
 	s, ok := f.storages[storage]
 	if !ok {
-		return storageStatus{}, fakeErr(500, "fake: storage '"+storage+"' does not exist")
+		return storageStatus{}, errNoSuchStorage()
 	}
 	return s, nil
 }
@@ -505,7 +535,9 @@ func (f *fakeAPI) Destroy(ctx context.Context, node string, vmid int) (string, e
 	if err != nil {
 		return "", err
 	}
-	if vm.lock != "" {
+	// A hibernated VM is destroyed like a stopped one (Phase 10 captures
 	// destroy_hibernated_response and task_status_destroy_hibernated).
 	if vm.lock != "" && vm.lock != lockSuspended {
 		return "", errLocked(vm.lock)
 	}
 	if vm.status == "running" {
@@ -666,6 +698,7 @@ func TestFakeAPI_errorsMatchCaptures(t *testing.T) {
 		{capture: "err_destroy_missing", fake: errVMMissing("pv1-1", 199999), matcher: isNotExist},
 		{capture: "err_config_while_clone_locked", fake: errLocked("clone"), matcher: isLocked},
 		{capture: "err_newid_in_use", fake: errNewIDInUse(199999), matcher: isAlreadyExists},
 		{capture: "storage_status_missing", fake: errNoSuchStorage(), matcher: isNoSuchStorage},
 		{capture: "err_agent_vm_stopped_agent_enabled", fake: errVMNotRunning(199999), matcher: isVMNotRunning},
 		{capture: "err_agent_not_running", fake: fakeErr(500, msgAgentNotRunning), matcher: isAgentNotRunning},
 		{capture: "err_agent_vm_stopped", fake: fakeErr(500, msgAgentNotConfigured), matcher: isAgentNotConfigured},
@@ -674,8 +707,8 @@ func TestFakeAPI_errorsMatchCaptures(t *testing.T) {
 		t.Run(tc.capture, func(t *testing.T) {
 			t.Parallel()
 			want := loadAPIError(t, tc.capture)
-			if tc.fake.Status != want.Status || tc.fake.Message != want.Message {
+			if tc.fake.Status != want.Status || tc.fake.Message != want.Message || !maps.Equal(tc.fake.Errors, want.Errors) {
-				t.Errorf("fake error = %d %q, capture = %d %q", tc.fake.Status, tc.fake.Message, want.Status, want.Message)
+				t.Errorf("fake error = %d %q %v, capture = %d %q %v", tc.fake.Status, tc.fake.Message, tc.fake.Errors, want.Status, want.Message, want.Errors)
 			}
 			if !tc.matcher(tc.fake) {
 				t.Errorf("matcher does not accept the fake's %q", tc.fake.Message)
@@ -49,7 +49,7 @@ func (p *Provider) get(ctx context.Context, log logr.Logger, vmid int, name stri
 		return nil, transientf("VM %d is locked (%s)", vmid, r.Lock)
 	}
 	if r.Status == "unknown" {
-		return nil, transientf("VM %d has status unknown: its node %s is not answering", vmid, r.Node)
+		return nil, transientf("VM %d has status unknown: its node %s is not answering, or the VM is too new to be reported yet", vmid, r.Node)
 	}
 	cfg, err := p.api.VMConfig(ctx, r.Node, vmid)
 	if err != nil {
@@ -78,7 +78,7 @@ func (p *Provider) get(ctx context.Context, log logr.Logger, vmid int, name stri
 	if err != nil {
 		return nil, fmt.Errorf("reading status of VM %d: %w", vmid, err)
 	}
-	if st.Status == "running" && (st.QMPStatus == "running" || st.QMPStatus == "") {
+	if guestRunning(st.Status, st.QMPStatus) {
 		ip, state, err := p.agentAddress(ctx, log, r.Node, vmid)
 		if err != nil {
 			return nil, err
@@ -341,8 +341,10 @@ func TestGet_unknownStatusIsTransient(t *testing.T) {
 	fx := newFixture(t, 1)
 	fx.addOurVM(testVMID, testName, "running").status = "unknown"
 	msg := assertGetFails(t, fx, provider.ErrTransient)
-	if !strings.Contains(msg, "pv1-1") {
+	// On pv1-1 the first Get after Create met this with the node up: the
-		t.Errorf("error = %s, want it to name the node", msg)
+	// listing had not reported the new VM yet (Phase 10 run, finding P6).
 	if !strings.Contains(msg, "pv1-1") || !strings.Contains(msg, "too new to be reported yet") {
 		t.Errorf("error = %s, want it to name the node and both causes", msg)
 	}
 	if n := fx.fake.callCount("VMConfig"); n != 0 {
 		t.Errorf("VMConfig calls = %d, want 0 while the node is unknown", n)
@@ -310,13 +310,6 @@ func TestProvider_everyErrorIsWrappedWithConfiguredName(t *testing.T) {
 			op:   "delete", id: "vms/150042", class: provider.ErrPermanent,
 		},
 		{name: "delete waiting on the name lock", setup: holdNameLock, call: del, op: "delete", id: "self", class: provider.ErrTransient},
 		{
 			name: "delete hibernated vm",
 			setup: func(fx *fixture, _ context.CancelFunc) {
 				fx.addOurVM(testVMID, testName, "stopped").lock = lockSuspended
 			},
 			call: del, op: "delete", id: "self", class: provider.ErrPermanent,
 		},
 		{
 			name: "delete destroy task fails",
 			setup: func(fx *fixture, _ context.CancelFunc) {
@@ -579,7 +572,7 @@ func TestLogging_noOpLinesAtV1(t *testing.T) {
 			name: "seed delete skipped on each node",
 			setup: func(fx *fixture) {
 				fx.p.cfg.Nodes = []string{fakeNode, "pv1-2"}
-				fx.fake.failWith("DeleteVolume", loadAPIError(fx.t, "err_forbidden_volume_delete"))
+				fx.fake.failWith("DeleteVolume", fakeErr(500, "fake: storage 'proxy-seeds' is not available on this node"))
 			},
 			call: del, msg: "proxmox seed delete failed on a node, skipped", want: 2,
 		},
@@ -13,22 +13,25 @@ import (
 // is Stopped, which the reconciler answers by replacing the VM. An empty
 // qmpstatus is taken as running: the field is missing, not a state.
 func mapState(status, qmpStatus, ip string) provider.InstanceState {
-	if status != "running" {
+	switch {
-		return provider.StateStopped
+	case guestRunning(status, qmpStatus):
 	}
 	switch qmpStatus {
 	case "running", "":
 		if ip == "" {
 			return provider.StateProvisioning
 		}
 		return provider.StateRunning
-	case "prelaunch":
+	case status == "running" && qmpStatus == "prelaunch":
 		return provider.StateProvisioning
 	default:
 		return provider.StateStopped
 	}
 }
 // guestRunning reports a VM whose guest runs, the only state in which its
 // agent can answer and mapState can call it Running.
 func guestRunning(status, qmpStatus string) bool {
 	return status == "running" && (qmpStatus == "running" || qmpStatus == "")
 }
 // firstIPv4 picks the first usable IPv4 from the guest agent's interfaces,
 // in the agent's order: not on the loopback interface, not loopback,
 // link-local or unspecified. Empty means no address yet.
@@ -0,0 +1,5 @@
 {
  "status": 200,
  "reason": "OK",
  "body": "{\"data\":\"UPID:pv1-1:0014F167:030AA126:6AB4DB09:qmdestroy:199999:proxy-operator@pve!operator:\"}"
 }
@@ -0,0 +1,5 @@
 {
  "status": 200,
  "reason": "OK",
  "body": "{\"data\":{\"name\":\"proxy-reconaaaaaaaaaaa\",\"agent\":1,\"mem\":0,\"maxmem\":2147483648,\"memhost\":0,\"serial\":1,\"maxdisk\":3221225472,\"vmid\":199999,\"ha\":{\"managed\":0},\"status\":\"stopped\",\"qmpstatus\":\"stopped\",\"netin\":0,\"uptime\":0,\"netout\":0,\"disk\":0,\"cpus\":2,\"cpu\":0,\"clipboard\":null}}"
 }
@@ -0,0 +1,5 @@
 {
  "status": 200,
  "reason": "OK",
  "body": "{\"data\":\"UPID:pv1-1:0014F097:030A9433:6AB4DAE8:qmstop:199999:proxy-operator@pve!operator:\"}"
 }
@@ -0,0 +1,5 @@
 {
  "status": 400,
  "reason": "Parameter verification failed.",
  "body": "{\"message\":\"Parameter verification failed.\\n\",\"data\":null,\"errors\":{\"storage\":\"No such storage.\"}}"
 }
@@ -0,0 +1,5 @@
 {
  "status": 200,
  "reason": "OK",
  "body": "{\"data\":{\"status\":\"stopped\",\"exitstatus\":\"OK\",\"upid\":\"UPID:pv1-1:0014F167:030AA126:6AB4DB09:qmdestroy:199999:proxy-operator@pve!operator:\",\"tokenid\":\"operator\",\"id\":\"199999\",\"starttime\":1790237449,\"type\":\"qmdestroy\",\"node\":\"pv1-1\",\"user\":\"proxy-operator@pve\",\"pstart\":51028262,\"pid\":1372519}}"
 }
@@ -0,0 +1,5 @@
 {
  "status": 200,
  "reason": "OK",
  "body": "{\"data\":{\"upid\":\"UPID:pv1-1:0014F097:030A9433:6AB4DAE8:qmstop:199999:proxy-operator@pve!operator:\",\"status\":\"stopped\",\"exitstatus\":\"OK\",\"node\":\"pv1-1\",\"user\":\"proxy-operator@pve\",\"pstart\":51024947,\"pid\":1372311,\"id\":\"199999\",\"tokenid\":\"operator\",\"starttime\":1790237416,\"type\":\"qmstop\"}}"
 }