proxmox: review fixes, real-host validation record, changelog
Phase 10. Two review passes (go-operator-reviewer, /code-review high) over the branch, recorded in docs/reviews/2026-09-23-proxmox-provider-review/: 9 findings fixed, 3 accepted as documented limits, 7 rejected with a reason. - Delete removes the seed between the stop and the destroy, so a failed seed delete leaves the VM for the next Delete instead of leaking the ISO; a Permanent refusal on the every-node path fails the call. - Create skips a probe VMID whose listed name is someone else's without waiting on its lock. - The three answers Phase 0 never captured are fixtures now: a missing storage is a 400 (the Checker reports StorageNotFound), a hibernated VM is destroyable (Delete destroys ours), a paused VM stops. - TestProxmox_lifecycle and TestProxmoxCheck_realHost passed on pv1-1 (PVE 9.2.20); the docs record the run and drop "not yet run". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -226,8 +226,9 @@ prov.Get ──► /cluster/resources (absent = NotFound; lock or node unknow
|
||||
+ agent network-get-interfaces (first routable IPv4; agent not
|
||||
up yet = Provisioning)
|
||||
prov.Delete ──► resources + config: gone, not ours or unmarked = leave it;
|
||||
hibernated = Permanent; stop → destroy (purge) → delete the seed
|
||||
on the VM's node (or on every node when no VM of ours was left)
|
||||
stop (unless stopped or hibernated) → delete the seed on the
|
||||
VM's node → destroy (purge); with no VM of ours left, the seed
|
||||
is deleted on every node
|
||||
prov.ListByTag ─► one /cluster/resources for the whole cluster, one config read per
|
||||
candidate (tag or proxy- name); the description's mark decides
|
||||
Check ──► per node: /version, node status, isoStorage status, templates in
|
||||
|
||||
@@ -14,7 +14,7 @@ Pairs with [docs/plans/2026-09-18-1502-proxmox-provider.md](../plans/2026-09-18-
|
||||
- [x] Phase 7 — Delete, Checker, CRD comment
|
||||
- [x] Phase 8 — Contract and observability sweep, integration test, testing.md
|
||||
- [x] Phase 9 — Deployment and documentation
|
||||
- [ ] Phase 10 — Review, real-host validation, changelog, MR
|
||||
- [x] Phase 10 — Review, real-host validation, changelog, MR
|
||||
|
||||
## Phase 0 — Plan file, branch, host recon, fixtures
|
||||
|
||||
@@ -1298,3 +1298,144 @@ Worth noting:
|
||||
guide's "Not yet proven end to end" gap and the README's matching
|
||||
clause come out once they pass. Then the CHANGELOG entry after the user
|
||||
confirms, and the MR.
|
||||
|
||||
## Phase 10 — Review, real-host validation, changelog, MR
|
||||
|
||||
Where the plan's Phase 10 text and the spec disagreed, the spec won, as in
|
||||
every phase since Phase 1. The host is pv1-1 at `https://192.168.0.8:8006`,
|
||||
not 192.168.2.8, the template is 9001 and the ISO storage `proxy-seeds`.
|
||||
|
||||
**Review.** Two read-only passes over `git diff main...HEAD`: the
|
||||
`go-operator-reviewer` agent, briefed with the spec and the contract
|
||||
(P1 to P6, S1 to S3), then `/code-review high` on the branch (CR1 to
|
||||
CR10). Of the 19 findings (two found by both passes), 9 were fixed, 3
|
||||
accepted as documented limits and 7 rejected with a reason, all recorded in
|
||||
[docs/reviews/2026-09-23-proxmox-provider-review/REPORT.md](../reviews/2026-09-23-proxmox-provider-review/REPORT.md).
|
||||
The one that mattered was CR1/P1. Delete destroyed the VM and then deleted
|
||||
the seed, so a failed seed delete left a seed nothing would ever remove:
|
||||
the next `Get` is NotFound, the finalizer goes, and GC lists only VMs.
|
||||
Delete now removes the seed between the stop and the destroy, so the VM
|
||||
`Get` looks at goes last. A 403 on the every-node path now fails instead
|
||||
of being skipped. P2 (CR3) was the other code change: a probe VMID whose
|
||||
listed name is someone else's is skipped at once, instead of waiting up to
|
||||
the clone budget on its lock on every retry.
|
||||
|
||||
**Lint.** `make lint` still panics inside staticcheck on Go 1.27, so, as
|
||||
in Phases 8 and 9:
|
||||
|
||||
```bash
|
||||
bin/golangci-lint run --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
|
||||
bin/golangci-lint run --build-tags integration --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
|
||||
```
|
||||
|
||||
Each run was diffed against the pre-fix baseline by file and message,
|
||||
with line numbers and occurrence counts stripped. Three new goconst hits
|
||||
came in along the way, and all three were fixed rather than accepted. The
|
||||
P2 test's lock value made `"backup"`, then `"migrate"`, a repeated
|
||||
literal; it now uses `"snapshot-delete"`. The fake's missing-storage error
|
||||
repeated `"Parameter verification failed."`; it is now built from the raw
|
||||
captured body. Final counts: goconst 224 (231 with the tag), logcheck 12,
|
||||
modernize 2, unparam 3, gocyclo 1.
|
||||
|
||||
**Real host.** The user ran the host checklist on pv1-1 and then:
|
||||
|
||||
```bash
|
||||
export PROXMOX_TEST_URL=https://192.168.0.8:8006
|
||||
export PROXMOX_TEST_TOKEN_FILE=$HOME/.config/proxmox-token
|
||||
export PROXMOX_TEST_CA_FILE=$HOME/.config/proxmox-ca.pem
|
||||
export PROXMOX_TEST_NODE=pv1-1
|
||||
export PROXMOX_TEST_TEMPLATE_VMID=9001
|
||||
export PROXMOX_TEST_ISO_STORAGE=proxy-seeds
|
||||
export PROXMOX_TEST_VLAN_TAG=4
|
||||
make test-integration-proxmox
|
||||
# --- PASS: TestProxmox_lifecycle (59.68s)
|
||||
# --- PASS: TestProxmoxCheck_realHost (0.13s)
|
||||
```
|
||||
|
||||
It ran the review fixes, uncommitted, on the first try. Clone task about
|
||||
3 s, Create 6.2 s, address 50.2 s after Create returned, Delete plus Get
|
||||
plus a second Delete 3.1 s. The Delete deleted the seed while the stopped
|
||||
VM's `ide2` still referenced it (imgdel `OK`), then destroyed the VM, and
|
||||
step 10 found neither left. That settles the one assumption behind the
|
||||
reorder that the fake could not prove. The run confirmed P6 as well: the
|
||||
first `Get`, about 6 s after the clone, met `status: unknown` in the
|
||||
listing and answered Transient, blaming the node, which was up. The
|
||||
message now names both causes.
|
||||
|
||||
**The three answers Phase 0 never captured.** Captured with a scratch
|
||||
helper that passes the token to curl through a process substitution and
|
||||
writes the Phase 0 fixture shape. Its first offline test caught a bug:
|
||||
Python's text mode folds `\r\n`, so the header split found nothing, and
|
||||
the file is now opened with `newline=""`. The calls were:
|
||||
|
||||
```bash
|
||||
pvecap storage_status_missing GET /nodes/pv1-1/storage/proxy-missing/status
|
||||
pve POST /nodes/pv1-1/qemu/9001/clone -d newid=199999 -d name=proxy-reconaaaaaaaaaaa -d full=1
|
||||
pve POST /nodes/pv1-1/qemu/199999/status/start
|
||||
pve POST /nodes/pv1-1/qemu/199999/status/suspend # pause
|
||||
pvecap stop_paused_response POST /nodes/pv1-1/qemu/199999/status/stop
|
||||
pvecap task_status_stop_paused GET /nodes/pv1-1/tasks/<upid>/status
|
||||
pvecap status_current_after_stop_paused GET /nodes/pv1-1/qemu/199999/status/current
|
||||
pve POST /nodes/pv1-1/qemu/199999/status/start
|
||||
pve POST /nodes/pv1-1/qemu/199999/status/suspend -d todisk=1 # hibernate: stopped, lock suspended
|
||||
pvecap err_destroy_hibernated DELETE '/nodes/pv1-1/qemu/199999?purge=1&destroy-unreferenced-disks=1'
|
||||
pvecap task_status_destroy_hibernated GET /nodes/pv1-1/tasks/<upid>/status
|
||||
```
|
||||
|
||||
Two of the three contradicted the code:
|
||||
|
||||
- A missing storage is a **400** with the field error
|
||||
`storage: No such storage.`, not a 500, so the Checker reported
|
||||
`APIError`. `isNoSuchStorage` matches that exact field error, and a 500
|
||||
is still taken as missing (not captured). The matcher matrix caught a
|
||||
first version that fired on any `storage` field error, which a clone
|
||||
onto a storage without image support also has.
|
||||
- The destroy **accepts a hibernated VM**. Its task ended `OK`, removing
|
||||
both disks and the `vm-199999-state-suspend-…` volume. Delete refused
|
||||
such a VM of ours as Permanent, believing the destroy would, so it now
|
||||
destroys it like a stopped one. Create's adoption still refuses one it
|
||||
finds before it was ever recorded. The capture was renamed
|
||||
`destroy_hibernated_response.json`, because the client test globs
|
||||
`err_*.json` as errors.
|
||||
- A stop on a paused VM ends `OK` with the VM stopped, as assumed.
|
||||
|
||||
The destroy also cleaned up the scratch VM, and a listing of local-lvm
|
||||
content for VMID 199999 came back empty.
|
||||
|
||||
**Kind pass.** The user built and loaded the image, deployed, applied a
|
||||
scratch providers config with only `kubernetes` and `proxmox-home`, ran
|
||||
the sample Proxy to Ready, ran a `curl -x` through it, deleted it and
|
||||
checked pv1-1 for leftovers. Everything passed. The first attempt at the
|
||||
override went to a ConfigMap named `providers-config`. `config/default`'s
|
||||
`namePrefix` makes the mounted one `egress-proxies-operator-providers-config`,
|
||||
so the pod kept the shipped config: no `proxmox-home` ProviderHealth, and
|
||||
`kvm-home` and `kvm2-home` failing on the missing SSH key. The corrected
|
||||
command replaced the prefixed ConfigMap and deleted the stray one. The
|
||||
Secret keeps its plain name (`proxmox-token`), since kustomize renames
|
||||
only resources it builds. `make deploy`'s image edit to
|
||||
`config/manager/kustomization.yaml` was reverted, not committed.
|
||||
|
||||
Worth noting:
|
||||
|
||||
- **Behaviour changes from Phase 9.** Delete's order is now stop, seed,
|
||||
destroy. A hibernated VM of ours is destroyed rather than latched. The
|
||||
Checker reports a missing ISO storage as `StorageNotFound`. A
|
||||
foreign-named VM on a probe VMID is skipped without a config read.
|
||||
- **Accepted, not fixed** (guide Known gaps, spec entry 24):
|
||||
- P3: Create adopts a leftover stopped VM whatever the current
|
||||
`placement` says.
|
||||
- P4: a retry can clone a second VM in a narrow crash window.
|
||||
- **Still unproven:**
|
||||
- A storage that exists but cannot be used on a node.
|
||||
- Every cross-node path. The host has one node.
|
||||
- **Flakes.** kvm's `TestKeepalive_keepsHealthyClientOpen` failed once
|
||||
under `-race` ("unexpected packet in response to channel open") and
|
||||
passed 5 of 5 on rerun. This branch does not touch `internal/provider/kvm`.
|
||||
The parked kvm2 items (S5.1, the 200 ms `TestCheck_hostsCheckedConcurrently`)
|
||||
did not bite, though the reviewer agent saw the latter fail once.
|
||||
- **Docs.** The "not yet run" wording is gone from the README, the
|
||||
guide, `testing.md`, spec entry 21 and `providers-existing.md`.
|
||||
`testing.md` gained a timings table and the `PROXMOX_TEST_VLAN_TAG`
|
||||
export line. The link sweep script from the Phase 9 entry found 0
|
||||
broken links over the seven edited docs, and the guide has no `—`.
|
||||
- `.claude/settings.json` stays out of the commit.
|
||||
|
||||
+20
-20
@@ -34,7 +34,7 @@ and the symbol that shows it.
|
||||
|---|---|---|---|---|---|
|
||||
| `readiness` | immediate: a Pod is scheduled within seconds (`stateFromPod`) | provisioning-window: minutes of boot (`mapState`) | provisioning-window: boot plus, on first use, an image download (`imageCache.ensure`) | provisioning-window: boot plus, on first use, an image download (`ensureImage`) | provisioning-window: boot plus the guest agent's install at first boot, about 50 s to an address on the test host (`agentAddress`) |
|
||||
| `create-return` | submit: returns after `client.Create` | submit: `Insert` is never awaited, `Operation.Wait` is not called | **blocking**: waits through `DomainCreate` (`ensureRunning`), and the first image fetch runs inside (`imageCache.fetch`) | **blocking**: waits through `DomainCreate` and a 15-second poll for running (`ensureRunning`, `waitRunning`), and the first image fetch runs inside (`fetchImage`) | **blocking**: waits through the clone, upload, resize and start tasks and a 60-second poll for running (`provision`, `waitRunning`) |
|
||||
| `delete-return` | returns-early: `client.Delete`, Pod terminates on its own | returns-early: `Delete` submits an operation each time | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: the stop, destroy and seed delete tasks complete before return (`destroy`, `deleteSeed`) |
|
||||
| `delete-return` | returns-early: `client.Delete`, Pod terminates on its own | returns-early: `Delete` submits an operation each time | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: the stop, seed delete and destroy tasks complete before return (`destroy`, `deleteSeed`) |
|
||||
| `ip-source` | assigned-at-create: `PodIP` (`instanceFromPod`) | assigned-at-create: the external NAT IP of the first NIC (`toInstance`) | discovered: ARP table first, then the guest agent, first routable IPv4 (`address`, `firstIPv4`) | discovered: the guest agent first, then the ARP table, first routable IPv4 (`addressSources`, `address`, `firstIPv4`) | discovered: the guest agent only, first routable IPv4 in the agent's order (`agentAddress`, `firstIPv4`) |
|
||||
| `gc-mark` | native-labels: Pod labels (`buildPod`) | native-labels: instance labels (`buildInsertRequest`) | written-metadata: a namespaced `<metadata>` element written in `DomainDefineXML` (`metadata.go`) | written-metadata: an element in kvm2's own namespace written in `DomainDefineXML` (`metadataURI`, `newInstanceMetadata`) | written-metadata: three `key=value` lines in the VM's description, passed in the clone call itself (`formatDescription`, `parseDescription`) |
|
||||
| `created-at` | backend-timestamp: `CreationTimestamp` | backend-timestamp: `CreationTimestamp`, parse error gives zero | written-at-create: `createdAt` in the metadata element (`newInstanceMetadata`) | written-at-create: `createdAt` in the metadata element, parse error gives zero (`newInstanceMetadata`, `createdAt`) | written-at-create: `proxy-operator-created-at` in the description, parse error gives zero (`parseDescription`) |
|
||||
@@ -50,7 +50,7 @@ and the symbol that shows it.
|
||||
| `timeouts` | ctx-only: no `rest.Config.Timeout` is set | ctx-only: SDK defaults, with SDK retries on Get and List | per-call for the dial only (`defaultHandshakeTimeout` in `dialSSH`); RPCs are ctx-only and a cancelled `call` closes the connection | per-call: a budget around the dial, every RPC, the uploads and the download (`defaultBudgets`, `call`); a cancelled or overlong call kills the pooled connection | per-call: dial and TLS handshake 10 s, each request `requestTimeoutSeconds`, a budget per task wait (`defaultBudgets`); a cancelled request ends only itself |
|
||||
| `logging` | none | tiered: V(1) outcomes and `logAPIError`, V(2) detail, V(5) wire log (`wirelog.go`) | tiered: V(1) outcomes and one `libvirt call` line per RPC (`trace.go`), V(2) XML and addresses; no wire tier | tiered: V(1) outcomes and one `libvirt call` line per RPC (`observe.go`), V(2) XML and addresses; no wire tier | tiered: V(1) outcomes and one `proxmox api call` line per request (`observe.go`), V(2) parameters, config summaries and addresses; no wire tier |
|
||||
| `child-spans` | transport-wrapper: `tracing.RestConfigWrapper()` on its own `rest.Config`, parent-gated | sdk-instrumentation: the SDK adds `otelhttp` itself; not parent-gated, but every call site has a parent | **none**: `trace.go` is a log decorator, not spans | span-decorator: `kvm2.<Method>` per RPC (`observe`), plus `kvm2.dial` (`libvirtDialer`) and `kvm2.download` (`fetchImage`) | transport-wrapper: `tracing.RestConfigWrapper()` on the client's transport, parent-gated, one `HTTP <METHOD>` span per request (`newHTTPAPI`) |
|
||||
| `backend-test` | cluster-e2e: kind (`test/e2e`) | manual-runbook: [gcp-vm-validation.md](gcp-vm-validation.md) | integration-tag: `kvm_integration_test.go`, `make test-integration-kvm` | integration-tag: `kvm2_integration_test.go`, `make test-integration-kvm2` | integration-tag: `proxmox_integration_test.go`, `make test-integration-proxmox`; compiled, not yet run against a host |
|
||||
| `backend-test` | cluster-e2e: kind (`test/e2e`) | manual-runbook: [gcp-vm-validation.md](gcp-vm-validation.md) | integration-tag: `kvm_integration_test.go`, `make test-integration-kvm` | integration-tag: `kvm2_integration_test.go`, `make test-integration-kvm2` | integration-tag: `proxmox_integration_test.go`, `make test-integration-proxmox`; passed on a PVE 9.2.20 host |
|
||||
|
||||
## 2. Other differences
|
||||
|
||||
@@ -614,16 +614,18 @@ was written.
|
||||
- **Delete.** Under the per-name lock, `destroy`: `resolve`; a lock other
|
||||
than `suspended` or `status: unknown` is Transient; the config's `does
|
||||
not exist`, another name or no mark leave the VM alone; our VM
|
||||
hibernated is Permanent, because the destroy refuses it and the lock
|
||||
never clears. Otherwise stop (unless stopped) and destroy with
|
||||
`purge=1&destroy-unreferenced-disks=1`, each a task waited on through
|
||||
`runTask`, where `does not exist` means gone. Then `deleteSeed` on the
|
||||
VM's node, where a failure fails the call (a 403 for
|
||||
`Datastore.Allocate` is Permanent), or, with no VM of ours destroyed,
|
||||
`deleteSeedEverywhere` over every configured node, where a node that
|
||||
fails is logged and skipped unless ctx ended. A missing volume still
|
||||
ends in a successful task, so no message is matched. Nothing is left
|
||||
when nil is returned.
|
||||
hibernated is destroyed like a stopped one, since the destroy takes it.
|
||||
Otherwise stop (unless stopped), `deleteSeed` on the
|
||||
VM's node, where a failure fails the call and leaves the VM for the
|
||||
next Delete (a 403 for `Datastore.Allocate` is Permanent), then destroy
|
||||
with `purge=1&destroy-unreferenced-disks=1`. Stop and destroy are tasks
|
||||
waited on through `runTask`, where `does not exist` means gone. The
|
||||
seed goes first because a VM already destroyed is NotFound to `Get`,
|
||||
and nothing would call Delete again. With no VM of ours,
|
||||
`deleteSeedEverywhere` tries every configured node, where a node that
|
||||
fails is logged and skipped unless ctx ended or the failure is
|
||||
Permanent. A missing volume still ends in a successful task, so no
|
||||
message is matched. Nothing is left when nil is returned.
|
||||
- **ListByTag.** One `ClusterResources` covers every node, unreachable
|
||||
ones included (their guests are listed with `status: unknown`).
|
||||
`isCandidate` keeps QEMU VMs that are not templates and carry the tag
|
||||
@@ -639,9 +641,9 @@ was written.
|
||||
status Transient, including the lock, the taken VMID and pveproxy's 595
|
||||
and 596); TLS verification failures (`tls.CertificateVerificationError`
|
||||
and the three `x509` types) Permanent; everything else Transient. The
|
||||
500 matchers (`isNotExist`, `isLocked`, `isAlreadyExists`, the three
|
||||
agent ones) are anchored to the 500 status and each is backed by a
|
||||
captured response. `permanentf` and `transientf` classify at the
|
||||
500 matchers (`isNotExist`, `isAlreadyExists`, the two agent ones)
|
||||
are anchored to the 500 status and each is backed by a captured
|
||||
response. `permanentf` and `transientf` classify at the
|
||||
source: placement, the template checks, the ownership check, a
|
||||
malformed token file or CA file (Permanent); a missing token or CA file,
|
||||
an expired wait, a failed task (Transient). Nothing is Quota.
|
||||
@@ -694,7 +696,7 @@ was written.
|
||||
by `PROXMOX_TEST_*` variables and `make test-integration-proxmox`,
|
||||
described in
|
||||
[testing.md](testing.md#the-proxmox-integration-test-real-proxmox-ve-host);
|
||||
it compiles but has not run against a host yet.
|
||||
it passed against PVE 9.2.20 on 2026-09-24.
|
||||
- **Deployment.** The optional `proxmox-token` Secret (keys `token` and
|
||||
`ca.pem`) mounted at `/etc/proxy-operator/proxmox`
|
||||
(`config/manager/manager.yaml`), `make proxmox-token-secret`, a
|
||||
@@ -713,11 +715,9 @@ was written.
|
||||
- The VMID range is shared with humans; a hand-made VM with a proxy's
|
||||
name is Permanent.
|
||||
- Cross-node clones (`target`) need shared storage and are untested.
|
||||
- A hibernated VM of ours latches its Proxy in Create and in Delete.
|
||||
- A hibernated VM of ours latches its Proxy when Create's adoption
|
||||
finds it; a recorded one is replaced.
|
||||
- The CA is cached after first use; the token is not.
|
||||
- Never run against a real host yet; three answers were never captured
|
||||
(the storage status for a missing storage, a destroy on a hibernated
|
||||
VM, a stop on a paused VM).
|
||||
|
||||
## 9. Gaps shared by the existing providers
|
||||
|
||||
|
||||
@@ -77,7 +77,11 @@ lands on the same number: `vmidFor(uid, attempt)` is
|
||||
clears by itself. Ownership is judged at once, as `Get` does (entry 9):
|
||||
a foreign hibernated VM is probed past, and ours is `ErrPermanent`
|
||||
(resume or destroy it), like kvm2's paused domain. Container and
|
||||
template entries at a VMID are foreign without a config read.
|
||||
template entries at a VMID are foreign without a config read, and so
|
||||
is a VM whose listed name is not `req.Name`, whatever its lock: only a
|
||||
clone in progress is nameless, and waiting on another VM's `backup`
|
||||
lock would hold Create for the whole clone budget on every retry
|
||||
(review finding P2).
|
||||
|
||||
The wait in step 3 is what closes the duplicate-VM bug Phase 0 found: after
|
||||
a crash mid-clone, the retry's by-name lookup misses the nameless VM, but
|
||||
@@ -284,7 +288,7 @@ and starts it; the reconciler has no ID for it to `Get`.
|
||||
| VMID absent | `/cluster/resources` | `ErrNotFound` |
|
||||
| `lock` set, other than `suspended` | resources entry (a clone in progress also has no name) | `ErrTransient` |
|
||||
| `lock: suspended` (hibernated to disk) | resources entry | Stopped |
|
||||
| `status: unknown` (node not reachable) | resources entry | `ErrTransient` |
|
||||
| `status: unknown` (node not reachable, or a VM seconds old the listing has not reported yet: the first `Get` after Create met this on pv1-1) | resources entry | `ErrTransient` |
|
||||
| config `does not exist` | `GET …/config` | `ErrNotFound` (deleted between the two calls) |
|
||||
| config `lock` set | config | `ErrTransient` |
|
||||
| `name` differs from the ID's name | config | `ErrNotFound`, logged: the VMID was reused |
|
||||
@@ -382,34 +386,48 @@ the seed ISO; the VM's disks are removed by the destroy call itself, with
|
||||
|
||||
1. Parse the ID (`ErrPermanent` if malformed).
|
||||
2. `/cluster/resources`. VMID absent, or held by a container: skip to
|
||||
step 7, trying every configured node because the VM's node is no longer
|
||||
step 6, trying every configured node because the VM's node is no longer
|
||||
known.
|
||||
3. `lock` set, other than `suspended`, or `status: unknown`: `ErrTransient`.
|
||||
A clone or another task still running finishes first; the reconciler
|
||||
calls Delete again after `DeletionPoll`.
|
||||
4. Read the config. `does not exist`: skip to step 7 on every node. A
|
||||
4. Read the config. `does not exist`: skip to step 6 on every node. A
|
||||
different name, or no mark: the VM is not ours; log it at V(1) and skip
|
||||
to step 7 on every node without touching it. Our VM with a `lock` other
|
||||
to step 6 on every node without touching it. Our VM with a `lock` other
|
||||
than `suspended` (taken since the listing): `ErrTransient`. Our VM with
|
||||
`lock: suspended` (hibernated): `ErrPermanent` asking the admin to resume
|
||||
or destroy it, the same answer Create's ownership check gives. The
|
||||
destroy refuses a locked VM, and that lock never clears on its own, so
|
||||
Transient would retry forever; Permanent puts the admin action in the
|
||||
Proxy's status, where `reconcileManaged` latches it and the finalizer
|
||||
path keeps retrying it visibly. A hibernated VM that is not ours is left
|
||||
alone like any other.
|
||||
`lock: suspended` (hibernated) is destroyed like a stopped one: the
|
||||
listing shows it `stopped`, so step 5 is skipped, and the destroy
|
||||
accepts it and removes its suspend state volume with the disks
|
||||
(`destroy_hibernated_response.json`, `task_status_destroy_hibernated.json`,
|
||||
captured in Phase 10). Until then Delete refused it as `ErrPermanent`
|
||||
on the belief that the destroy would. A hibernated VM that is not ours
|
||||
is left alone like any other.
|
||||
5. Status (from the listing) other than `stopped`: `POST …/status/stop`,
|
||||
wait (stop budget). Stop works on a paused VM too, which the listing
|
||||
shows as `running`. `does not exist` means the VM is gone.
|
||||
6. `DELETE /nodes/{node}/qemu/{vmid}?purge=1&destroy-unreferenced-disks=1`,
|
||||
wait (destroy budget). `does not exist` is success.
|
||||
7. `DELETE /nodes/{node}/storage/{isoStorage}/content/<isoStorage>:iso/<name>-seed.iso`,
|
||||
wait (seed-delete budget), on the VM's node when steps 5 and 6 ran on a
|
||||
VM of ours, and on every configured node in config order otherwise. A
|
||||
missing volume is still a successful task, so no message is matched. On
|
||||
the every-node path a node whose request or task fails is logged at V(1)
|
||||
and skipped, unless ctx ended, which fails the call; on the VM's node a
|
||||
failure fails the call (a 403 for `Datastore.Allocate` is Permanent).
|
||||
shows as `running`: the task ends `OK` and the VM is `stopped`
|
||||
(`stop_paused_response.json`, `task_status_stop_paused.json`,
|
||||
`status_current_after_stop_paused.json`, captured in Phase 10).
|
||||
`does not exist` means the VM is gone.
|
||||
6. `DELETE /nodes/{node}/storage/{isoStorage}/content/<isoStorage>:iso/<name>-seed.iso`,
|
||||
wait (seed-delete budget), on the VM's node when step 5 was reached on
|
||||
a VM of ours, and on every configured node in config order otherwise
|
||||
(then step 7 is skipped). A missing volume is still a successful task,
|
||||
so no message is matched. The seed goes **before** the destroy: once
|
||||
the VM is gone `Get` is `ErrNotFound`, the reconciler releases the
|
||||
finalizer and GC no longer lists it, so a seed left by a failed step
|
||||
after the destroy would never be deleted (review finding CR1). On the
|
||||
VM's node a failure fails the call and leaves the VM stopped for the
|
||||
next Delete. On the every-node path a node whose request or task fails
|
||||
is logged at V(1) and skipped, unless ctx ended or the failure is
|
||||
`ErrPermanent` (a 403 for `Datastore.Allocate`, which every node
|
||||
would give), which fail the call, so the missing privilege is visible
|
||||
on both paths (review finding P1).
|
||||
7. `DELETE /nodes/{node}/qemu/{vmid}?purge=1&destroy-unreferenced-disks=1`,
|
||||
wait (destroy budget). `does not exist` is success. The destroy leaves
|
||||
the seed alone (Phase 0 deleted it after the destroy), which is why
|
||||
step 6 exists. That Proxmox deletes an ISO a stopped VM's `ide2` still
|
||||
references was not in the Phase 0 captures; the Phase 10 real-host run
|
||||
proved it (imgdel `OK` on the stopped VM's seed, then the destroy).
|
||||
8. Return nil.
|
||||
|
||||
Everything is gone when Delete returns nil, so `Get` afterwards is
|
||||
@@ -474,6 +492,7 @@ per-parameter map. One `classify(op string, err error)`:
|
||||
| 401 `Authentication failed!`, 401 `No ticket` | Permanent | the token or its secret is wrong; fixed in the Secret |
|
||||
| 403 `Permission check failed (<path>, <privilege>)` | Permanent | the token lacks a privilege; the message names it |
|
||||
| 400 `Parameter verification failed.` with `errors` | Permanent | the request is wrong; the message includes the field errors |
|
||||
| 400 as above with `storage: No such storage.` | Permanent from a call; the Checker's `StorageNotFound` (entry 16) | the storage does not exist in the cluster |
|
||||
| other 4xx | Permanent | a request Proxmox will never accept |
|
||||
| 500 `Configuration file '…' does not exist` | NotFound (Transient when `op == "create"`) | the VM is gone |
|
||||
| 500 `VM is locked (<lock>)` | Transient | a task holds the VM |
|
||||
@@ -490,17 +509,24 @@ per-parameter map. One `classify(op string, err error)`:
|
||||
|
||||
Every matched string is backed by a captured response under `testdata/`;
|
||||
the table test loads them rather than repeating the text. Matching is on
|
||||
fixed substrings anchored to the 500 status: `does not exist`,
|
||||
`VM is locked`, `config file already exists`, `QEMU guest agent is not
|
||||
running`, `No QEMU guest agent configured`, and for the stopped VM the whole
|
||||
message against `^VM [0-9]+ is not running$`. The agent messages are tested
|
||||
in that order, because the last is a suffix of the first.
|
||||
fixed strings anchored to the 500 status: `does not exist`,
|
||||
`config file already exists`, and the whole messages
|
||||
`QEMU guest agent is not running` and `No QEMU guest agent configured`,
|
||||
plus, on a 400, the `storage` field error `No such storage.` (whole; a
|
||||
clone onto a storage without image support has another `storage` error).
|
||||
`VM is locked` and the stopped VM's `VM <vmid> is not running` are not
|
||||
matched by the provider: both are Transient through the default branch.
|
||||
The tests keep a matcher for each, with the whole stopped-VM message
|
||||
against `^VM [0-9]+ is not running$` (its tail is a suffix of the
|
||||
agent-not-running answer), only to hold the fake's wording to the
|
||||
captures (review finding CR5).
|
||||
|
||||
**Evidence:** F1 (error text in the reason phrase and the body; 401 has no
|
||||
body); F2, F4, F5, F7 (the 500 messages); F6 (no out-of-space response
|
||||
could be produced on thin storage); F22 (both 401 texts); fixtures
|
||||
`err_forbidden.json`, `err_bad_param.json`, `err_tag_invalid.json`,
|
||||
`err_upload_bad_name.json`; the Phase 0 TLS finding (`192.168.2.8` not in
|
||||
`err_upload_bad_name.json`, and from Phase 10 `storage_status_missing.json`
|
||||
(a missing storage is a 400, not the 500 the spec had assumed); the Phase 0 TLS finding (`192.168.2.8` not in
|
||||
the node certificate); contract §5.
|
||||
|
||||
## 14. Credentials and TLS (`credentials`)
|
||||
@@ -611,11 +637,15 @@ an admin:
|
||||
- `CredentialsUnavailable`: the token file is missing or malformed. Fix the
|
||||
Secret, not Proxmox.
|
||||
- `StorageNotFound`: `isoStorage` is missing, inactive, or has no `iso`
|
||||
content. Fix the storage on the node. Any 500 from the storage status
|
||||
call counts as missing, whatever its wording (it does not exist, is not
|
||||
enabled on that node, or is offline; no missing-storage response was
|
||||
captured, so no text is matched), and the message carries Proxmox's
|
||||
text. A 401, 403, 595 or 596 there keeps its shared reason.
|
||||
content. Fix the storage on the node. A storage the cluster does not
|
||||
know answers `400 Parameter verification failed.` with the field error
|
||||
`storage: No such storage.` (`storage_status_missing.json`, captured in
|
||||
Phase 10); that exact field error is matched, and the message carries
|
||||
it. Any 500 from the storage status call counts as missing too,
|
||||
whatever its wording: Proxmox knows the storage but cannot use it on
|
||||
that node (not enabled there, or offline). That answer was not
|
||||
captured, so no text is matched. Any other 4xx, and a 401, 403, 595 or
|
||||
596, keeps its shared reason.
|
||||
- `TemplateNotFound`: a configured template VMID does not exist or is not a
|
||||
template. Fix the template or the catalogue.
|
||||
|
||||
@@ -783,8 +813,8 @@ carries the cloud-init text.
|
||||
|
||||
Contract §11 list, with the test for each item. Every name below exists
|
||||
(Phase 8 checked each with a grep for `func <name>(`). The two integration
|
||||
tests compile under `-tags=integration` but have not run against a host
|
||||
yet; Phase 10 runs them.
|
||||
tests passed on pv1-1 on 2026-09-24 (timings in
|
||||
[testing.md](testing.md#the-proxmox-integration-test-real-proxmox-ve-host)).
|
||||
|
||||
- [x] repeat `Create`: `TestCreate_repeatReturnsSameIDAndCreatesNothing`
|
||||
- [x] `Get` after `Delete`: `TestGet_afterDeleteReturnsNotFound`
|
||||
@@ -802,7 +832,7 @@ yet; Phase 10 runs them.
|
||||
- [x] Checker reasons, expired `ctx`, stable order: `TestCheck_reasonsTable`, `TestCheck_expiredContextReturnsPromptly`, `TestCheck_oneEntryPerNodeInConfigOrder`
|
||||
- [x] sentinel tests for `CloudInit` and the credential: `TestCreate_cloudInitSentinelNeverLoggedOrInErrors`, `TestClient_tokenNeverInErrorsOrLogs`, and `TestLogging_v0SilentV1OutcomeV2Detail` over a whole lifecycle at every level
|
||||
- [x] child spans under the operation span: `TestTracing_httpSpansUnderOperationSpan`, `TestClient_childSpanUnderParent`
|
||||
- [x] real-backend lifecycle test: `TestProxmox_lifecycle` and `TestProxmoxCheck_realHost` in `proxmox_integration_test.go` (compiled, not yet run)
|
||||
- [x] real-backend lifecycle test: `TestProxmox_lifecycle` and `TestProxmoxCheck_realHost` in `proxmox_integration_test.go`, passed on pv1-1
|
||||
|
||||
Tests that cover the Phase 0 corrections specifically, in addition to the
|
||||
plan's per-phase lists:
|
||||
@@ -904,4 +934,28 @@ user confirms.
|
||||
VMID range delete each other's VMs), an unconfigured provider latches the
|
||||
Proxy.
|
||||
|
||||
Accepted after the Phase 10 review
|
||||
([docs/reviews/2026-09-23-proxmox-provider-review/](reviews/2026-09-23-proxmox-provider-review/REPORT.md),
|
||||
findings P3 and P4), and listed in the guide's Known gaps:
|
||||
|
||||
- **A Create that failed permanently after its clone keeps its VM**, and
|
||||
the next Create adopts it by name even when `placement` has changed.
|
||||
Destroy the VM before changing `placement` on such a Proxy.
|
||||
- **A retry can clone a second VM** when a Create is cut off while
|
||||
cloning onto a later probe VMID, the VM on the earlier one is removed,
|
||||
and the retry comes before the clone ends. The next Create names both
|
||||
VMIDs as `ErrPermanent`.
|
||||
|
||||
Changed by the Phase 10 captures of the three answers Phase 0 never saw:
|
||||
|
||||
- **A hibernated VM of ours** latches only Create's adoption
|
||||
(`ErrPermanent`, resume or destroy it; entry 2). `Get` reports it
|
||||
Stopped, so the reconciler replaces it, and Delete destroys it, since
|
||||
the destroy takes a hibernated VM (entry 11). Before the capture, Delete
|
||||
refused it as well.
|
||||
- **A missing `isoStorage`** is a 400 with `storage: No such storage.`,
|
||||
which the Checker reports as `StorageNotFound` (entry 16); a missing
|
||||
storage at upload time is Permanent like any 4xx.
|
||||
- **A paused VM** stops like a running one (entry 11, step 5), as assumed.
|
||||
|
||||
**Evidence:** contract §13; entries 2, 7, 10, 11, 14 above.
|
||||
|
||||
+47
-26
@@ -164,8 +164,14 @@ pve 'cluster/resources?type=vm' # the template is listed with "temp
|
||||
|
||||
**Rotating the token needs no restart.** The token file is read on
|
||||
every request, so the next request after the kubelet refreshes the
|
||||
mount uses the new token. **Rotating the CA does**: it is loaded on
|
||||
first use and cached for the life of the process. A missing CA file is
|
||||
mount uses the new token. Keep the old token valid until then: a 401
|
||||
is a permanent error, so a Proxy whose request lands in the gap
|
||||
latches until its spec changes. Add the new token
|
||||
(`pveum user token add`), update the Secret, wait a couple of minutes
|
||||
for the mount to refresh, and only then remove the old token. A token
|
||||
created with `--expire` needs the same rotation before it expires.
|
||||
**Rotating the CA does** need a restart: it is loaded on first use and
|
||||
cached for the life of the process. A missing CA file is
|
||||
retried until it lands; one without a certificate is a permanent error.
|
||||
|
||||
2. Add a provider entry to the providers config
|
||||
@@ -302,7 +308,9 @@ through an ownership check (`inspect` in `create.go`): a VM with a lock
|
||||
naming the VMID and both UIDs. The operator never takes over a VM it did
|
||||
not create;
|
||||
- our name, our mark, hibernated (`lock: suspended`): **permanent error**,
|
||||
`resume or destroy it in Proxmox` (see *Known gaps*).
|
||||
`resume or destroy it in Proxmox` (see *Known gaps*). This only happens
|
||||
when Create has not recorded the VM yet; a recorded VM that gets
|
||||
hibernated is reported Stopped and replaced.
|
||||
|
||||
Two VMs with the proxy's name are a permanent error naming every VMID;
|
||||
remove all but one.
|
||||
@@ -392,15 +400,18 @@ pvesm free proxy-seeds:iso/proxy-abc…-seed.iso
|
||||
```
|
||||
|
||||
**Delete** runs under a per-name lock shared with Create: stop (a paused
|
||||
VM too), destroy with `purge=1&destroy-unreferenced-disks=1`, then delete
|
||||
the seed on the VM's node. Each is a task that is waited on. A VM that is
|
||||
already gone, not ours, or unmarked is left alone, and the seed is then
|
||||
looked for on every configured node, where a node that fails is logged
|
||||
and skipped. On the VM's own node a failed seed delete fails the call:
|
||||
**without `Datastore.Allocate` on `isoStorage` the 403 makes Delete
|
||||
permanent**, and the Proxy's deletion stays stuck with that message until
|
||||
the privilege is granted. When Delete returns nil, nothing of the Proxy
|
||||
is left.
|
||||
VM too), delete the seed on the VM's node, then destroy with
|
||||
`purge=1&destroy-unreferenced-disks=1`. Each is a task that is waited on.
|
||||
The seed goes before the VM because once the VM is gone nothing calls
|
||||
Delete again; a failed seed delete leaves the VM stopped, and the next
|
||||
Delete finishes both. A VM that is already gone, not ours, or unmarked is
|
||||
left alone, and the seed is then looked for on every configured node,
|
||||
where a node that fails is logged and skipped. A refusal every node would
|
||||
give fails the call instead: **without `Datastore.Allocate` on
|
||||
`isoStorage` the 403 makes Delete permanent** on either path, and the
|
||||
Proxy's deletion stays stuck with that message (a replaced Proxy stays
|
||||
failed, its VM stopped) until the privilege is granted. When Delete
|
||||
returns nil, nothing of the Proxy is left.
|
||||
|
||||
## Logging
|
||||
|
||||
@@ -554,9 +565,9 @@ transient.
|
||||
|
||||
| Class | What lands there |
|
||||
|---|---|
|
||||
| `ErrPermanent` | placement values not in the catalogue; a template VMID that is missing or not a template, has a cloud-init drive, or lacks `net0` or a sized boot disk when the config needs them; a malformed providerID; any 4xx (401 `Authentication failed!` or `No ticket`: the token; 403 `Permission check failed (<path>, <privilege>)`: a missing privilege, including `Datastore.Allocate` for the seed delete; 400 `Parameter verification failed.` with the field errors); a TLS verification failure; a token file that exists but is not `<user>@<realm>!<tokenid>=<secret>`; a CA file with no certificate; a VM with the proxy's name that carries no mark or another UID; two VMs with the proxy's name; our VM hibernated (`lock: suspended`), in both Create and Delete |
|
||||
| `ErrPermanent` | placement values not in the catalogue; a template VMID that is missing or not a template, has a cloud-init drive, or lacks `net0` or a sized boot disk when the config needs them; a malformed providerID; any 4xx (401 `Authentication failed!` or `No ticket`: the token; 403 `Permission check failed (<path>, <privilege>)`: a missing privilege, including `Datastore.Allocate` for the seed delete; 400 `Parameter verification failed.` with the field errors); a TLS verification failure; a token file that exists but is not `<user>@<realm>!<tokenid>=<secret>`; a CA file with no certificate; a VM with the proxy's name that carries no mark or another UID; two VMs with the proxy's name; our VM hibernated (`lock: suspended`) found by Create's adoption |
|
||||
| `ErrNotFound` | from `Get`: the VMID absent from the cluster's resources, the 500 `Configuration file 'nodes/<node>/qemu-server/<vmid>.conf' does not exist`, the VM at the VMID now named differently or unmarked. Inside `Create` the `does not exist` answer is `ErrTransient`, because the contract forbids NotFound from Create; `Delete` treats all of these as done |
|
||||
| `ErrTransient` | everything else: a missing token or CA file (the Secret has not landed yet); network errors and cancellation; 500 `VM is locked (<lock>)` and any lock seen in the resources or config; a VM whose node does not answer (`status: unknown`); the agent's answers that `Get` cannot place, such as `VM <vmid> is not running`; any other 5xx, including pveproxy's 595 and 596; a task that fails or runs out of its budget; no free VMID after 32 probes; the wait for running expiring, or a VM that stops right after its start |
|
||||
| `ErrTransient` | everything else: a missing token or CA file (the Secret has not landed yet); network errors and cancellation; 500 `VM is locked (<lock>)` and any lock seen in the resources or config; a VM whose node does not answer, or one a few seconds old the listing has not reported yet (`status: unknown`); the agent's answers that `Get` cannot place, such as `VM <vmid> is not running`; any other 5xx, including pveproxy's 595 and 596; a task that fails or runs out of its budget; no free VMID after 32 probes; the wait for running expiring, or a VM that stops right after its start |
|
||||
|
||||
`Get` turns two agent answers into states rather than errors:
|
||||
`QEMU guest agent is not running` is Provisioning (booting, or the agent
|
||||
@@ -613,13 +624,26 @@ They never contain the token, the CA content or cloud-init.
|
||||
|
||||
Beyond the ten limits accepted in the design:
|
||||
|
||||
- **A hibernated VM latches its Proxy.** A VM of ours with
|
||||
`lock: suspended` (hibernated to disk) makes both Create and Delete fail
|
||||
permanently with `resume or destroy it in Proxmox`: the lock never
|
||||
clears on its own, and the destroy refuses a locked VM. `Get` reports it
|
||||
Stopped. Resume it by starting it (`qm start <vmid>` restores the
|
||||
hibernated state) or destroy it on the node, then touch the Proxy's
|
||||
spec, or let the pending Delete retry.
|
||||
- **A hibernated VM is replaced, or latches a Create.** `Get` reports a
|
||||
VM of ours with `lock: suspended` (hibernated to disk) as Stopped, so
|
||||
the reconciler deletes it (the destroy removes its suspend state volume
|
||||
too) and creates a new one. Only a Create that finds such a VM before it
|
||||
was ever recorded, by name, fails permanently with
|
||||
`resume or destroy it in Proxmox`: resume it by starting it
|
||||
(`qm start <vmid>` restores the hibernated state) or destroy it on the
|
||||
node, then touch the Proxy's spec.
|
||||
- **A Create that failed permanently after its clone keeps its VM.**
|
||||
The stopped VM is adopted by name on the next Create, even when the
|
||||
Proxy's `placement` has changed since: the Proxy then runs the old
|
||||
template on the old node. After changing `placement` on such a Proxy,
|
||||
destroy the VM on the node (`qm destroy <vmid> --purge`) first, or
|
||||
delete and recreate the Proxy.
|
||||
- **A retry can clone a second VM** in one rare sequence: a Create is
|
||||
cut off while cloning onto a later probe VMID (the first was taken by
|
||||
someone else's VM), that other VM is removed, and the retry comes
|
||||
while the clone is still running, so its VM is still nameless. The
|
||||
retry clones onto the now free first VMID. The next Create then fails
|
||||
permanently with `VMs … are all named …`; destroy either VM.
|
||||
- **The seed upload is one request**, bounded by `requestTimeoutSeconds`.
|
||||
A seed is tens of KiB, but a multi-megabyte `vendorData` would need a
|
||||
larger timeout.
|
||||
@@ -627,11 +651,6 @@ Beyond the ten limits accepted in the design:
|
||||
`config/samples/proxy_proxmox.yaml` installs Squid with
|
||||
`http_access allow all`; restrict it before exposing the proxy network
|
||||
to anything but the cluster.
|
||||
- **Not yet proven end to end.** The unit tests run over captured
|
||||
responses from `pv1-1` and an in-memory fake; the integration tests
|
||||
below compile but have not run against a host yet. Three answers were
|
||||
never captured: the storage status for a missing storage, a destroy on
|
||||
a hibernated VM, and a stop on a paused VM.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -644,3 +663,5 @@ which also lists the variables it needs; run it with
|
||||
`make test-integration-proxmox`. `TestProxmoxCheck_realHost` runs the
|
||||
health check against the same host and creates nothing:
|
||||
`go test -tags=integration -count=1 -v -run TestProxmoxCheck ./internal/provider/proxmox/`.
|
||||
Both passed on `pv1-1` (PVE 9.2.20) on 2026-09-24: Create took 6.2 s,
|
||||
and the guest reported an address 50 s after Create returned.
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
# proxmox provider review: final report
|
||||
|
||||
Branch `feat/proxmox-provider` at `7f2073f` (Phases 0 to 9), diffed against
|
||||
`main` (`2167819`): 123 files, 49 of them Go. The core is
|
||||
`internal/provider/proxmox/`; also in scope were the `nocloud` and
|
||||
`keyedlock` extraction from kvm2 (`0a6c862`), the config block in
|
||||
`internal/provider/config*.go`, registration and the deploy files.
|
||||
|
||||
**Inputs.** [docs/proxmox-provider-spec.md](../../proxmox-provider-spec.md)
|
||||
(the spec the code implements) and
|
||||
[docs/provider-contract.md](../../provider-contract.md) (the MUST/SHOULD
|
||||
rules). A disagreement with either counts as a finding.
|
||||
|
||||
**Reviewers.** Two independent passes, both read-only:
|
||||
|
||||
1. The `go-operator-reviewer` agent (`.claude/agents/go-operator-reviewer.md`),
|
||||
briefed with the spec and the contract, findings **P1 to P6** and
|
||||
simplifications **S1 to S3**.
|
||||
2. `/code-review high` on the branch, findings **CR1 to CR10**.
|
||||
|
||||
Where both passes found the same thing, the finding is listed once with both
|
||||
IDs. Every finding is **fixed** (with a regression test when it is code),
|
||||
**accepted** (real, not fixed here, documented as a known limit), or
|
||||
**rejected** (with the reason).
|
||||
|
||||
**Toolchain.** `make test` green before and after the fixes. `make lint`
|
||||
panics inside staticcheck on the local Go 1.27 toolchain, as it did during
|
||||
kvm2, so lint ran without it, over the proxmox package and
|
||||
`internal/provider`, with and without the integration tag:
|
||||
|
||||
```bash
|
||||
bin/golangci-lint run --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
|
||||
bin/golangci-lint run --build-tags integration --disable staticcheck --max-same-issues 0 --max-issues-per-linter 0 ./internal/provider/proxmox/... ./internal/provider/
|
||||
```
|
||||
|
||||
Before the fixes both runs were at the Phase 8 and 9 baseline: logcheck 12,
|
||||
modernize (mapsloop) 2, unparam 3 in `fake_test.go`, gocyclo 1
|
||||
(`TestAPITypes_decodeCaptures`), goconst on repeated strings, and the three
|
||||
goconst hits in `internal/provider`. After the fixes the counts are the same
|
||||
except goconst, one lower (224, 231 with the tag). No hit is new. The first
|
||||
version of the P2 test did add one: a third `"backup"` literal in the
|
||||
package's tests, then `"migrate"`. It now uses `"snapshot-delete"`.
|
||||
|
||||
## Summary
|
||||
|
||||
| ID | Severity | Finding | Disposition |
|
||||
|---|---|---|---|
|
||||
| CR1 + P1 | medium-high | A seed delete that fails after the destroy leaks the seed for good, and the class of the error depends on the attempt | **fixed** |
|
||||
| CR2 | medium | A 403 on the seed delete latches a replaced Proxy whose VM is already gone | **fixed** (same change as CR1) |
|
||||
| CR3 + P2 | medium | The ownership check waits out any lock, even on a VM whose listed name is someone else's | **fixed** |
|
||||
| P3 | low-medium | Create adopts a leftover stopped VM whatever the current placement says | **accepted**, documented |
|
||||
| P4 | low | A crash mid-clone plus a freed earlier VMID can yield a second VM | **accepted**, documented |
|
||||
| P5 | low | 401/403 are Permanent, which bites during a token rotation | **fixed** in the guide |
|
||||
| P6 | low | Get and Delete trust the listing's `status`, which lags | **accepted**, confirmed on the host; error message fixed |
|
||||
| CR5 | low | `isLocked` and `isVMNotRunning` are called only from tests | **fixed** |
|
||||
| CR8 | low | `TestPermanentfAndTransientf` breaks the `Test<Function>_<scenario>` rule | **fixed** |
|
||||
| CR9 | low | `get` repeats `mapState`'s "guest running" rule inline | **fixed** |
|
||||
| CR4 + S2 | low | The Checker repeats the cluster-wide `/version` and `/cluster/resources` per node | **rejected** |
|
||||
| CR6 | low | `lockName`, `permanentf` and `transientf` are copies of kvm2's | **rejected** |
|
||||
| CR7 | low | `readToken`, the CA load and seed staging do file I/O without a ctx | **rejected** |
|
||||
| CR10 | low | `New` runs `withDefaults` twice | **rejected** |
|
||||
| S1 | simplification | The `observe.go` decorator could be one log line in `send` | **rejected** |
|
||||
| S3 | simplification | `nocloud` exports `GuestAgentScript` and `SeedLabel` for tests only | **rejected** |
|
||||
|
||||
## Fixed
|
||||
|
||||
### CR1 + P1: the seed outlives a Delete that fails after the destroy
|
||||
|
||||
`internal/provider/proxmox/delete.go`. Delete stopped the VM, destroyed it,
|
||||
then deleted the seed. If the seed delete failed (a 595, a task over its
|
||||
1 minute budget, a 403), Delete returned an error, but the VM was already
|
||||
gone. The reconciler's next `Get` was `ErrNotFound`, so `reconcileDelete`
|
||||
released the finalizer without calling Delete again, and GC lists only VMs.
|
||||
The seed stayed on `isoStorage` for good. That breaks contract §2.3 ("MUST
|
||||
remove everything Create made ... GC sees only what ListByTag returns"). A
|
||||
destroy task that outlived its budget but finished in the background had
|
||||
the same effect.
|
||||
|
||||
P1 adds the second half. On the finalizer path the next Delete found the
|
||||
VMID gone and took `deleteSeedEverywhere`, which logged the same 403 at V(1)
|
||||
and returned nil. The seed leaked with nothing visible.
|
||||
|
||||
**Fix.** The seed now goes between the stop and the destroy, so the VM,
|
||||
which is what `Get` looks at, is the last thing removed: a failed seed
|
||||
delete leaves the VM stopped, `Get` keeps finding it, and the next Delete
|
||||
finishes both. `deleteSeedEverywhere` now fails on an `ErrPermanent`
|
||||
answer (a 401 or 403 every node would give) instead of skipping it. It
|
||||
still skips a node that is down or lacks the storage. `runTask`'s "gone"
|
||||
result lost its last reader and was dropped.
|
||||
|
||||
That Proxmox deletes an ISO a stopped VM's `ide2` still references is the
|
||||
one assumption the fake cannot prove. From memory of the PVE source, not
|
||||
checked against it: the content delete runs `vdisk_free` without looking
|
||||
for references to `iso` volumes, and the destroy skips CD-ROM drives.
|
||||
Phase 0 only ever deleted the seed after the destroy. **The Phase 10
|
||||
real-host run proved it** on 2026-09-24: the imgdel task on the stopped
|
||||
VM's seed ended `OK`, the destroy followed, and `TestProxmox_lifecycle`
|
||||
step 10 found neither the VM nor the seed.
|
||||
|
||||
**Tests.** `TestDelete_seedDeleteFailureOnVMNodeFails` (403, 596 and a
|
||||
failed task on the VM's node) now asserts that no Destroy was sent, that
|
||||
`Get` still finds the VM, that a second Delete gives the same class, and
|
||||
that a Delete after the fault clears removes both. It failed on the old
|
||||
code with `Destroy calls = 1` and `Get = instance not found`.
|
||||
`TestDelete_seedDeleteNodeFailureSkippedWhenNodeUnknown/a refused privilege
|
||||
fails the call` covers the every-node 403. Updated for the new order:
|
||||
`TestDelete_stoppedVMSkipsStop`, `TestDelete_runningVMStoppedFirst`,
|
||||
`TestDelete_seedDeleteIsTaskAndIdempotent`,
|
||||
`TestDelete_destroyTaskFailureIsTransient` (the seed is now gone and the VM
|
||||
kept) and `TestDelete_goneMidwayIsSuccess` (a VM gone at the stop still
|
||||
gets a Destroy call, which answers `does not exist`).
|
||||
`TestLogging_noOpLinesAtV1`'s skip case now injects a node-level 500,
|
||||
since a 403 no longer skips.
|
||||
|
||||
Docs: spec entry 11 (steps 5 to 7 reordered, the Permanent exception),
|
||||
the guide's Delete paragraph, `providers-existing.md`'s Delete bullet and
|
||||
its `delete-return` cell.
|
||||
|
||||
### CR2: a 403 on the seed delete latched a Proxy with no VM
|
||||
|
||||
Same code, same fix. On the replace path (`reconcileManaged` sees a Stopped
|
||||
VM and calls Delete), the old order destroyed the VM, got a 403 on the
|
||||
seed, and returned Permanent. `providerFailure` latched it, so the Proxy
|
||||
sat Failed pointing at a VM that no longer existed. The next attempt would
|
||||
have returned nil. Now the 403 comes before the destroy: the VM stays
|
||||
stopped, and every attempt gives the same Permanent answer naming
|
||||
`Datastore.Allocate` until the privilege is granted. The guide says so.
|
||||
|
||||
### CR3 + P2: the lock wait on a VM that is plainly not ours
|
||||
|
||||
`internal/provider/proxmox/create.go`, `allocate`. Any taken qemu VMID went
|
||||
through `inspect`, which first waits for the VM's lock to clear (clone
|
||||
budget, 10 minutes) and only then compares names. A VM on one of the
|
||||
Proxy's probe VMIDs, listed with another name and holding a long lock (a
|
||||
`vzdump` backup, or a stale lock a failed backup left behind), made every
|
||||
Create wait 10 minutes and return Transient. `vmidFor` is deterministic,
|
||||
so every retry landed on the same VMID. Only a clone in progress is
|
||||
nameless (F8), so a listed name other than ours settles the question.
|
||||
|
||||
**Fix.** `allocate` sends a taken entry to `inspect` only when its listed
|
||||
name is empty or ours; any other name probes on without a config read.
|
||||
Spec entry 2's step 3 says so.
|
||||
|
||||
**Tests.** `TestCreate_takenVMIDInspectedBeforeProbing` gains "foreign VM
|
||||
under a lock probes on without waiting" (a `snapshot-delete` lock with a
|
||||
2 s clone budget; the old code spent the 2 s and failed Transient), and
|
||||
"foreign VM probes on without a read" now expects one config read (the
|
||||
template's). `TestCreate_vmidRangeExhaustedIsTransient` expects one
|
||||
VMConfig call instead of five.
|
||||
|
||||
### P5: token rotation order
|
||||
|
||||
`docs/proxmox-provider.md`, cluster-side setup. A 401 is Permanent (the
|
||||
same convention as GCP), so a request made with a revoked token before the
|
||||
kubelet refreshes the Secret latches that Proxy. The guide now gives the
|
||||
order: add the new token, update the Secret, wait for the mount, then
|
||||
remove the old one, and the same before an `--expire` date.
|
||||
|
||||
### CR5: matchers used only by tests
|
||||
|
||||
`isLocked` and `isVMNotRunning` in `errors.go` had no production caller.
|
||||
Both answers are Transient through `classify`'s default branch. They moved
|
||||
to `fake_test.go`, where `TestFakeAPI_errorsMatchCaptures`,
|
||||
`TestAPIError_matchers` and the fake's own checks still use them to hold the
|
||||
fake's wording to the captures. Spec entry 13's matching sentence and
|
||||
`providers-existing.md` now name only the matchers the provider uses.
|
||||
|
||||
### CR8: test name
|
||||
|
||||
`TestPermanentfAndTransientf` became `TestPermanentf_wrapsClassWithMessage`
|
||||
and `TestTransientf_wrapsClassWithMessage`. kvm2 has the same name; it is
|
||||
out of scope here.
|
||||
|
||||
### CR9: one rule, one place
|
||||
|
||||
`get` asked `status == "running" && qmpstatus in {running, ""}` inline to
|
||||
decide whether to query the agent, the same rule `mapState` applies. Both
|
||||
now call `guestRunning(status, qmpStatus)` in `state.go`, so a change to it
|
||||
cannot leave the two out of step. `TestMapState_*` and the Get tests cover
|
||||
it unchanged (`TestMapState_table`, `TestMapState_fromCaptures`).
|
||||
|
||||
## Accepted
|
||||
|
||||
### P3: adoption ignores a placement change
|
||||
|
||||
`create.go:94-118`. A Create that fails Permanent after the clone (a 403 on
|
||||
the upload, say) leaves a stopped VM. If the user then changes
|
||||
`placement.image` or `zone`, the next Create adopts that VM by name, and
|
||||
the Proxy runs the old template on the old node. The fix the reviewer
|
||||
proposed (record the template in the mark, re-clone a never-started VM
|
||||
whose template differs) is a new mark field and a destroy path inside
|
||||
Create. That is too much for this MR for a two-step, user-driven sequence.
|
||||
Documented in the guide's Known gaps and spec entry 24: destroy the VM
|
||||
before changing `placement` on such a Proxy.
|
||||
|
||||
### P4: a second VM after a crash mid-clone
|
||||
|
||||
The retry misses a nameless in-progress clone on a later probe VMID when
|
||||
the VM on the earlier VMID was removed meanwhile, and clones again. It
|
||||
takes a crash or cut-off during a clone of a few seconds, a foreign VM on
|
||||
the first probe VMID, and that VM's removal in the same window. The
|
||||
proposed fix (inspect every nameless `lock: clone` entry in the range
|
||||
first) would make every Create wait on every other Proxy's clone. The
|
||||
outcome is visible, not silent: the next Create fails Permanent naming
|
||||
both VMIDs. Documented in the guide and spec entry 24.
|
||||
|
||||
### P6: the listing's `status` may lag
|
||||
|
||||
`get.go:51`, `delete.go:60`, `delete.go:90`. This is the reviewer's
|
||||
inference, not observed: `/cluster/resources` comes from pvestatd, which
|
||||
may lag a state change by seconds. Both cases heal on retry: `Get`
|
||||
answering Transient for `unknown` is polled through, and a destroy of a VM
|
||||
that started behind the listing's back fails its task as Transient and the
|
||||
next Delete stops it first.
|
||||
|
||||
**Confirmed on pv1-1.** The first `Get` came 32 ms after the second Create,
|
||||
about 6 s after the clone, and was Transient: the listing still showed the
|
||||
VM as `status: unknown`. The next poll 5 s later saw it running. The
|
||||
behaviour is right, but the message blamed the node ("its node pv1-1 is not
|
||||
answering"), which was up. Get and Delete now say "its node … is not
|
||||
answering, or the VM is too new to be reported yet".
|
||||
`TestGet_unknownStatusIsTransient` asserts the wording, and spec entry 9
|
||||
and the guide's error table name both causes. The Delete side (a stop
|
||||
skipped because the listing lags) was not met: the run's Delete came a
|
||||
minute after the start.
|
||||
|
||||
## Rejected
|
||||
|
||||
- **CR4 + S2, Checker repeats cluster-wide calls per node.** Spec entry 16
|
||||
makes each node's target self-contained, so a failure is reported
|
||||
against the node whose round saw it. The cost is two extra requests per
|
||||
extra node per round, and the configured cluster
|
||||
has one node. Worth revisiting with a multi-node config, not before.
|
||||
- **CR6, helpers copied from kvm2.** `permanentf`/`transientf` are one
|
||||
line each; `lockName` is five and wraps `keyedlock.Acquire`'s error in
|
||||
`provider.ErrTransient`. Moving that into `keyedlock` would couple a
|
||||
generic lock package to the provider sentinels, and putting the helpers
|
||||
in `internal/provider` would touch every kvm2 call site in a proxmox MR.
|
||||
- **CR7, file I/O without a ctx.** `readToken` and the CA load read a
|
||||
mounted Secret with `os.ReadFile`, which cannot be cancelled; a ctx
|
||||
parameter would be unused. kvm2's `os.ReadFile` of the SSH key is the
|
||||
precedent. Seed staging is `nocloud`'s, shared with kvm2 and unchanged by
|
||||
the extraction.
|
||||
- **CR10, `withDefaults` twice in `New`.** Startup only, idempotent, a few
|
||||
small maps. `newProvider` applies it because the tests build providers
|
||||
through it; defaulting once would need a second constructor shape.
|
||||
- **S1, replace the `observe.go` decorator with one log line in `send`.**
|
||||
The decorator is spec entries 17 and 20's `span-decorator` answer: each
|
||||
seam call gets a child span and a V(1)/V(2) line with node, VMID and
|
||||
UPID, which `send` does not know. It mirrors kvm2's decorator, and the
|
||||
reflection test guarantees every seam method is covered. Swapping the
|
||||
design in the review phase is out of proportion.
|
||||
- **S3, `nocloud` exports used only by tests.** `SeedLabel` is read by
|
||||
both kvm2's and proxmox's Create tests to recognise a real seed, and
|
||||
`GuestAgentScript` by proxmox's `TestBuildSeed_table`, which tests
|
||||
proxmox's own mapping of its config fields onto the vendor-data, not
|
||||
`nocloud`'s composition. The exports are the price of cross-package
|
||||
tests.
|
||||
|
||||
## Checked and fine
|
||||
|
||||
From the reviewer agent's report, verified in code:
|
||||
|
||||
- The kvm2 extraction is a pure move. kvm2 still wraps every seed-build
|
||||
error as Permanent, as before, and `keyedlock` is byte-identical to the
|
||||
code it replaced.
|
||||
- Delete stops or destroys only after the listing shows a qemu entry at
|
||||
the VMID and the config carries the name and the mark; a reused VMID or
|
||||
an unmarked VM is left alone.
|
||||
- A missing VM inside Create is Transient, never NotFound; locks and
|
||||
`status: unknown` are Transient, never NotFound.
|
||||
- Create resumes at every step: the VMID comes from the UID, a nameless
|
||||
locked entry is waited on and adopted, upload/config/resize are
|
||||
repeatable, and the mark is written in the clone call.
|
||||
- Secrets: `apiError` carries status and message only, the token never
|
||||
reaches a `url.Error`, the Checker keeps only the token id, and the
|
||||
cloud-init text appears only as `cloudInitBytes`.
|
||||
- Every seam call takes the ctx; `pollUntil` honours ctx and budget;
|
||||
`keyedlock.Acquire` gives up with ctx.
|
||||
- Spec entries 1, 3, 4, 9, 11, 12, 13, 15 and 16 match the code (P2 was the
|
||||
one disagreement, and the spec shared the flaw).
|
||||
- Every §11 test name in spec entry 21 exists.
|
||||
|
||||
The agent also saw kvm2's `TestCheck_hostsCheckedConcurrently` fail once
|
||||
under `-race` (199.9 ms against its 200 ms floor). That file is unchanged
|
||||
on this branch; the flake is parked, as before.
|
||||
+33
-14
@@ -71,9 +71,9 @@ manual runs too.
|
||||
| `internal/provider/keyedlock` | `keyedlock_test.go` | the per-name lock kvm2 and proxmox share: the entry is removed once uncontended, waiters share one lock, a waiter gives up with its ctx | pure |
|
||||
| `internal/provider/proxmox` | `id_test.go`, `mark_test.go`, `state_test.go`, `seed_test.go`, `errors_test.go`, `api_test.go` | pure pieces: providerID round-trip and malformed forms, the VMID derived from the UID (deterministic, in range, probing forward and wrapping), the description mark (keys, parsed from the captured clone, admin lines kept), state mapping and the IPv4 filter from captures, seed names and building (empty name Permanent without its payload, staging failure Transient); the `classify` table and the 500-message matchers against every captured error; the API types decoded from the captures | pure, over the responses Phase 0 captured from PVE 9.2.20 (`testdata/`) |
|
||||
| | `client_test.go` | the thin HTTP client: the token read from its file on every request (missing Transient, unparsable Permanent), `apiError` from status, reason phrase and body, the data envelope, method, path and parameters of every seam call, the clone and destroy query shapes, the volid's slash escaped, HTTP/1.1 only, request timeout and cancel Transient and the client unaffected, a child span under a parent and none without, the token never in errors or logs, the CA file trusted and loaded lazily, an x509 failure Permanent, insecure mode, the upload streamed as multipart | `httptest.NewTLSServer` replaying the captures byte for byte |
|
||||
| | `create_test.go`, `get_test.go`, `delete_test.go`, `list_test.go`, `task_test.go`, `proxmox_test.go`, `provider_test.go`, `fake_test.go` | Create: resume after every partial step and after a crash mid-clone, the VMID probe and ownership check, the template's shape refused before any clone, resize as a task, the seed overwritten without a delete, the net0 bridge and VLAN rewrite keeping the MAC, returns only when running, cloud-init sentinel never logged; Get: lock checked before name, the agent's messages, paused from `status/current`, hibernated, a reused VMID, four calls when running; Delete: every already-gone and not-ours case, the seed deleted on the VM's node or on every node, hibernated Permanent, the per-name lock; ListByTag: candidates by tag or name prefix, the mark decides, an unreadable VM skipped unless ctx ended; task and lock waits; `New` makes no request; the contract sweep: every method returns within a second of a cancel (blocked on its first seam call, and mid-operation), every error wrapped with the configured name, op and id, logging tiers V0/V1/V2 with the no-op lines | in-memory `fakeAPI` behind the `api` seam, its error texts checked against the captures |
|
||||
| | `create_test.go`, `get_test.go`, `delete_test.go`, `list_test.go`, `task_test.go`, `proxmox_test.go`, `provider_test.go`, `fake_test.go` | Create: resume after every partial step and after a crash mid-clone, the VMID probe and ownership check, the template's shape refused before any clone, resize as a task, the seed overwritten without a delete, the net0 bridge and VLAN rewrite keeping the MAC, returns only when running, cloud-init sentinel never logged; Get: lock checked before name, the agent's messages, paused from `status/current`, hibernated, a reused VMID, four calls when running; Delete: every already-gone and not-ours case, the seed deleted before the destroy on the VM's node or on every node, a hibernated VM destroyed, the per-name lock; ListByTag: candidates by tag or name prefix, the mark decides, an unreadable VM skipped unless ctx ended; task and lock waits; `New` makes no request; the contract sweep: every method returns within a second of a cancel (blocked on its first seam call, and mid-operation), every error wrapped with the configured name, op and id, logging tiers V0/V1/V2 with the no-op lines | in-memory `fakeAPI` behind the `api` seam, its error texts checked against the captures |
|
||||
| | `observe_test.go`, `check_test.go`, `provider_test.go` | the decorator: every seam method logs exactly one line (guarded by reflection), starts no span, identifiers at V1, detail at V2, upload body never logged, error fields; a Get through `provider.WithTracing` over the real client makes four `HTTP GET` spans, all children of `provider.get` and none carrying the token; health check: one entry per node in config order, nodes checked concurrently (a barrier, not a timer), expired ctx returns promptly, the reasons table at V2 with a sentinel secret, the transport reasons from real `net/http` and `crypto/tls` errors, detail keys | `fakeAPI` + `funcr` capture + span recorder; the real client over `httptest` for the tracing and transport-reason rows |
|
||||
| | `proxmox_integration_test.go` (`-tags=integration`) | one real VM: `New` without a request, `Check` Ready with details, Create, idempotent re-Create with an unchanged VM count, Running with an IPv4 and the mark read back, ListByTag, Delete, NotFound, no VM and no seed left, HTTP spans under `provider.create`; `TestProxmoxCheck_realHost` runs the health check against the host and creates nothing. **Compiles, not yet run against a host** | a **real Proxmox VE host** — see below |
|
||||
| | `proxmox_integration_test.go` (`-tags=integration`) | one real VM: `New` without a request, `Check` Ready with details, Create, idempotent re-Create with an unchanged VM count, Running with an IPv4 and the mark read back, ListByTag, Delete, NotFound, no VM and no seed left, HTTP spans under `provider.create`; `TestProxmoxCheck_realHost` runs the health check against the host and creates nothing. **Passed on pv1-1 (PVE 9.2.20), 2026-09-24**: Create 6.2 s, address 50 s later, whole run 61 s | a **real Proxmox VE host** — see below |
|
||||
| `internal/health` | `probe_test.go` | probes through a **real CONNECT-capable proxy stub** to a real TLS server: tunnel success, refused CONNECT, wrong status, dead proxy, plain-http forward | `httptest` + hijacked bidirectional tunnel |
|
||||
| | `engine_test.go` | thresholds, first-verdict, latency suppression matrix, dropped-event retry, stale-probe UID guard, tick scheduling/seeding/pruning, `Start` end-to-end | fake `client.Reader` + injected `probeFn` |
|
||||
| `internal/lease` | `store_test.go` | capacity, `MaxLeases=0`, all three selection tie-breaks, target-scoped vs global cooldowns, expiry via fake clock, report-on-retained-lease, idempotent release, **40 concurrent acquires never exceeding capacity** | injectable clock; the concurrency case is why `-race` matters |
|
||||
@@ -117,19 +117,21 @@ envtest suite the scaffold generated.
|
||||
`httptest` replaying responses captured from PVE 9.2.20. Whether a
|
||||
clone, config write, seed upload and start yield a guest that boots and
|
||||
reports an address is provable only on a real host (the proxmox
|
||||
integration test below). Two kinds of path have met only the fake:
|
||||
integration test below, which has passed on pv1-1). Two kinds of path
|
||||
have met only the fake:
|
||||
- **the cross-node paths**: a clone that targets another node, the seed
|
||||
deleted on every node when the VM's node is unknown, a Check round over
|
||||
several nodes. The test host is a single node, so the integration test
|
||||
cannot reach them either
|
||||
([proxmox-provider-spec.md](proxmox-provider-spec.md) entry 24);
|
||||
- **answers never captured**: what the storage status call returns for a
|
||||
missing or disabled storage (every 500 there is taken as
|
||||
`StorageNotFound`), what destroy says for a hibernated VM, and a stop
|
||||
on a paused VM. The provider does not depend on their wording: any
|
||||
500 from the storage status is `StorageNotFound`, a hibernated VM is
|
||||
refused before destroy is called, and a paused VM gets the same stop
|
||||
as a running one. What the fake answers there is its own guess.
|
||||
- **an answer never captured**: the storage status for a storage the
|
||||
cluster knows but cannot use on a node (disabled there, or offline).
|
||||
Every 500 there is taken as `StorageNotFound`, and what the fake
|
||||
answers is its own guess. The three answers Phase 0 missed were
|
||||
captured on pv1-1 on 2026-09-24: a missing storage is a 400 with
|
||||
`storage: No such storage.`, a hibernated VM is destroyed without
|
||||
complaint (its suspend state volume with it), and a stop on a paused
|
||||
VM ends `OK` with the VM stopped.
|
||||
|
||||
## The KVM integration test (real hypervisor)
|
||||
|
||||
@@ -303,6 +305,7 @@ export PROXMOX_TEST_CA_FILE=$HOME/.config/proxmox-ca.pem # the node's pve-roo
|
||||
export PROXMOX_TEST_NODE=pv1-1
|
||||
export PROXMOX_TEST_TEMPLATE_VMID=9001
|
||||
export PROXMOX_TEST_ISO_STORAGE=proxy-seeds # holds iso content
|
||||
export PROXMOX_TEST_VLAN_TAG=4 # template 9001's net0 is untagged; proxies live on VLAN 4
|
||||
make test-integration-proxmox
|
||||
```
|
||||
|
||||
@@ -381,10 +384,26 @@ variables; to run only this test:
|
||||
go test -tags=integration -count=1 -v -run TestProxmoxCheck ./internal/provider/proxmox/
|
||||
```
|
||||
|
||||
**Neither test has run against a host yet.** Both compile
|
||||
(`go vet -tags=integration ./...`); Phase 10 of the plan runs them on
|
||||
pv1-1. Like the kvm tests they are not part of `make test` and not run in
|
||||
CI.
|
||||
**Last run: 2026-09-24 on pv1-1** (Proxmox VE 9.2.20, kernel
|
||||
7.0.14-17-pve, template 9001, `proxy-seeds`, VLAN tag 4). Both tests
|
||||
passed; `go test` reported 60.9 s for the package.
|
||||
|
||||
| Step | Time |
|
||||
|---|---|
|
||||
| `Check` | 141 ms (125 ms in `TestProxmoxCheck_realHost`) |
|
||||
| clone task, submit to done | about 3 s (seen done at the third 1 s poll) |
|
||||
| `Create`, clone to started VM | 6.2 s |
|
||||
| second `Create` (adoption, nothing created) | 32 ms |
|
||||
| Create's return to the first address | 50.2 s (the agent install from the internet) |
|
||||
| `Delete` (stop, seed delete, destroy), `Get`, second `Delete` | 3.1 s |
|
||||
| `TestProxmox_lifecycle` | 59.7 s |
|
||||
|
||||
The first `Get`, 32 ms after the second Create, was Transient: the
|
||||
listing still showed the new VM with `status: unknown`. The next poll 5 s
|
||||
later saw it running. The Delete deleted the seed while the stopped VM's
|
||||
`ide2` still referenced it, then destroyed the VM, and step 10 found
|
||||
neither left. Like the kvm tests they are not part of `make test` and not
|
||||
run in CI.
|
||||
|
||||
## The scaffolded `make test-e2e` suite
|
||||
|
||||
|
||||
Reference in New Issue
Block a user