proxmox: review fixes, real-host validation record, changelog

Phase 10. Two review passes (go-operator-reviewer, /code-review high)
over the branch, recorded in docs/reviews/2026-09-23-proxmox-provider-review/:
9 findings fixed, 3 accepted as documented limits, 7 rejected with a
reason.

- Delete removes the seed between the stop and the destroy, so a failed
  seed delete leaves the VM for the next Delete instead of leaking the
  ISO; a Permanent refusal on the every-node path fails the call.
- Create skips a probe VMID whose listed name is someone else's without
  waiting on its lock.
- The three answers Phase 0 never captured are fixtures now: a missing
  storage is a 400 (the Checker reports StorageNotFound), a hibernated
  VM is destroyable (Delete destroys ours), a paused VM stops.
- TestProxmox_lifecycle and TestProxmoxCheck_realHost passed on pv1-1
  (PVE 9.2.20); the docs record the run and drop "not yet run".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-24 10:28:28 +00:00
co-authored by Claude Opus 5.5
parent 7f2073f94b
commit 3d72507894
28 changed files with 882 additions and 201 deletions
+20 -20
View File
@@ -34,7 +34,7 @@ and the symbol that shows it.
|---|---|---|---|---|---|
| `readiness` | immediate: a Pod is scheduled within seconds (`stateFromPod`) | provisioning-window: minutes of boot (`mapState`) | provisioning-window: boot plus, on first use, an image download (`imageCache.ensure`) | provisioning-window: boot plus, on first use, an image download (`ensureImage`) | provisioning-window: boot plus the guest agent's install at first boot, about 50 s to an address on the test host (`agentAddress`) |
| `create-return` | submit: returns after `client.Create` | submit: `Insert` is never awaited, `Operation.Wait` is not called | **blocking**: waits through `DomainCreate` (`ensureRunning`), and the first image fetch runs inside (`imageCache.fetch`) | **blocking**: waits through `DomainCreate` and a 15-second poll for running (`ensureRunning`, `waitRunning`), and the first image fetch runs inside (`fetchImage`) | **blocking**: waits through the clone, upload, resize and start tasks and a 60-second poll for running (`provision`, `waitRunning`) |
| `delete-return` | returns-early: `client.Delete`, Pod terminates on its own | returns-early: `Delete` submits an operation each time | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: the stop, destroy and seed delete tasks complete before return (`destroy`, `deleteSeed`) |
| `delete-return` | returns-early: `client.Delete`, Pod terminates on its own | returns-early: `Delete` submits an operation each time | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: the stop, seed delete and destroy tasks complete before return (`destroy`, `deleteSeed`) |
| `ip-source` | assigned-at-create: `PodIP` (`instanceFromPod`) | assigned-at-create: the external NAT IP of the first NIC (`toInstance`) | discovered: ARP table first, then the guest agent, first routable IPv4 (`address`, `firstIPv4`) | discovered: the guest agent first, then the ARP table, first routable IPv4 (`addressSources`, `address`, `firstIPv4`) | discovered: the guest agent only, first routable IPv4 in the agent's order (`agentAddress`, `firstIPv4`) |
| `gc-mark` | native-labels: Pod labels (`buildPod`) | native-labels: instance labels (`buildInsertRequest`) | written-metadata: a namespaced `<metadata>` element written in `DomainDefineXML` (`metadata.go`) | written-metadata: an element in kvm2's own namespace written in `DomainDefineXML` (`metadataURI`, `newInstanceMetadata`) | written-metadata: three `key=value` lines in the VM's description, passed in the clone call itself (`formatDescription`, `parseDescription`) |
| `created-at` | backend-timestamp: `CreationTimestamp` | backend-timestamp: `CreationTimestamp`, parse error gives zero | written-at-create: `createdAt` in the metadata element (`newInstanceMetadata`) | written-at-create: `createdAt` in the metadata element, parse error gives zero (`newInstanceMetadata`, `createdAt`) | written-at-create: `proxy-operator-created-at` in the description, parse error gives zero (`parseDescription`) |
@@ -50,7 +50,7 @@ and the symbol that shows it.
| `timeouts` | ctx-only: no `rest.Config.Timeout` is set | ctx-only: SDK defaults, with SDK retries on Get and List | per-call for the dial only (`defaultHandshakeTimeout` in `dialSSH`); RPCs are ctx-only and a cancelled `call` closes the connection | per-call: a budget around the dial, every RPC, the uploads and the download (`defaultBudgets`, `call`); a cancelled or overlong call kills the pooled connection | per-call: dial and TLS handshake 10 s, each request `requestTimeoutSeconds`, a budget per task wait (`defaultBudgets`); a cancelled request ends only itself |
| `logging` | none | tiered: V(1) outcomes and `logAPIError`, V(2) detail, V(5) wire log (`wirelog.go`) | tiered: V(1) outcomes and one `libvirt call` line per RPC (`trace.go`), V(2) XML and addresses; no wire tier | tiered: V(1) outcomes and one `libvirt call` line per RPC (`observe.go`), V(2) XML and addresses; no wire tier | tiered: V(1) outcomes and one `proxmox api call` line per request (`observe.go`), V(2) parameters, config summaries and addresses; no wire tier |
| `child-spans` | transport-wrapper: `tracing.RestConfigWrapper()` on its own `rest.Config`, parent-gated | sdk-instrumentation: the SDK adds `otelhttp` itself; not parent-gated, but every call site has a parent | **none**: `trace.go` is a log decorator, not spans | span-decorator: `kvm2.<Method>` per RPC (`observe`), plus `kvm2.dial` (`libvirtDialer`) and `kvm2.download` (`fetchImage`) | transport-wrapper: `tracing.RestConfigWrapper()` on the client's transport, parent-gated, one `HTTP <METHOD>` span per request (`newHTTPAPI`) |
| `backend-test` | cluster-e2e: kind (`test/e2e`) | manual-runbook: [gcp-vm-validation.md](gcp-vm-validation.md) | integration-tag: `kvm_integration_test.go`, `make test-integration-kvm` | integration-tag: `kvm2_integration_test.go`, `make test-integration-kvm2` | integration-tag: `proxmox_integration_test.go`, `make test-integration-proxmox`; compiled, not yet run against a host |
| `backend-test` | cluster-e2e: kind (`test/e2e`) | manual-runbook: [gcp-vm-validation.md](gcp-vm-validation.md) | integration-tag: `kvm_integration_test.go`, `make test-integration-kvm` | integration-tag: `kvm2_integration_test.go`, `make test-integration-kvm2` | integration-tag: `proxmox_integration_test.go`, `make test-integration-proxmox`; passed on a PVE 9.2.20 host |
## 2. Other differences
@@ -614,16 +614,18 @@ was written.
- **Delete.** Under the per-name lock, `destroy`: `resolve`; a lock other
than `suspended` or `status: unknown` is Transient; the config's `does
not exist`, another name or no mark leave the VM alone; our VM
hibernated is Permanent, because the destroy refuses it and the lock
never clears. Otherwise stop (unless stopped) and destroy with
`purge=1&destroy-unreferenced-disks=1`, each a task waited on through
`runTask`, where `does not exist` means gone. Then `deleteSeed` on the
VM's node, where a failure fails the call (a 403 for
`Datastore.Allocate` is Permanent), or, with no VM of ours destroyed,
`deleteSeedEverywhere` over every configured node, where a node that
fails is logged and skipped unless ctx ended. A missing volume still
ends in a successful task, so no message is matched. Nothing is left
when nil is returned.
hibernated is destroyed like a stopped one, since the destroy takes it.
Otherwise stop (unless stopped), `deleteSeed` on the
VM's node, where a failure fails the call and leaves the VM for the
next Delete (a 403 for `Datastore.Allocate` is Permanent), then destroy
with `purge=1&destroy-unreferenced-disks=1`. Stop and destroy are tasks
waited on through `runTask`, where `does not exist` means gone. The
seed goes first because a VM already destroyed is NotFound to `Get`,
and nothing would call Delete again. With no VM of ours,
`deleteSeedEverywhere` tries every configured node, where a node that
fails is logged and skipped unless ctx ended or the failure is
Permanent. A missing volume still ends in a successful task, so no
message is matched. Nothing is left when nil is returned.
- **ListByTag.** One `ClusterResources` covers every node, unreachable
ones included (their guests are listed with `status: unknown`).
`isCandidate` keeps QEMU VMs that are not templates and carry the tag
@@ -639,9 +641,9 @@ was written.
status Transient, including the lock, the taken VMID and pveproxy's 595
and 596); TLS verification failures (`tls.CertificateVerificationError`
and the three `x509` types) Permanent; everything else Transient. The
500 matchers (`isNotExist`, `isLocked`, `isAlreadyExists`, the three
agent ones) are anchored to the 500 status and each is backed by a
captured response. `permanentf` and `transientf` classify at the
500 matchers (`isNotExist`, `isAlreadyExists`, the two agent ones)
are anchored to the 500 status and each is backed by a captured
response. `permanentf` and `transientf` classify at the
source: placement, the template checks, the ownership check, a
malformed token file or CA file (Permanent); a missing token or CA file,
an expired wait, a failed task (Transient). Nothing is Quota.
@@ -694,7 +696,7 @@ was written.
by `PROXMOX_TEST_*` variables and `make test-integration-proxmox`,
described in
[testing.md](testing.md#the-proxmox-integration-test-real-proxmox-ve-host);
it compiles but has not run against a host yet.
it passed against PVE 9.2.20 on 2026-09-24.
- **Deployment.** The optional `proxmox-token` Secret (keys `token` and
`ca.pem`) mounted at `/etc/proxy-operator/proxmox`
(`config/manager/manager.yaml`), `make proxmox-token-secret`, a
@@ -713,11 +715,9 @@ was written.
- The VMID range is shared with humans; a hand-made VM with a proxy's
name is Permanent.
- Cross-node clones (`target`) need shared storage and are untested.
- A hibernated VM of ours latches its Proxy in Create and in Delete.
- A hibernated VM of ours latches its Proxy when Create's adoption
finds it; a recorded one is replaced.
- The CA is cached after first use; the token is not.
- Never run against a real host yet; three answers were never captured
(the storage status for a missing storage, a destroy on a hibernated
VM, a stop on a paused VM).
## 9. Gaps shared by the existing providers