# Plan: proxmox, a Proxmox VE provider built from the provider contract

**Created:** 2026-09-18 15:02

## Context

The provider contract (`docs/provider-contract.md`), the spec template
(`docs/provider-spec-template.md`) and the kvm2 build proved the workflow:
spec first, then implementation against the contract, one commit per step.
This plan adds a third VM-style backend, `proxmox`, on the Proxmox VE 9.x
REST API at `https://<host>:8006/api2/json`. The user has a real host at
192.168.2.8 running **9.2.20** (Debian 13 Trixie, QEMU 11) and prepares the
API token and a cloud-init-capable template before Phase 0 (checklist
below). Everything below is written against 9.2; where 8.x differed the
difference is called out, because the appendix and the guide must name the
version they were proven on.

The implementation runs in sessions whose context gets compacted, so every
phase is self-contained: it names what to read first, what it creates or
edits, the test names it must add, a `Verify:` clause, a "Done when" line
and a commit subject. A fresh session reading only this plan, the spec
(`docs/proxmox-provider-spec.md`, from Phase 1 on) and the execution log's
checklist (`docs/plans-executions/2026-09-18-1502-proxmox-provider.md`) must be able to
pick up where the last one stopped.

**Resumption protocol, run at the start of every phase.** Read this plan's
Context and the phase entry; read the spec; read the execution log's
checklist; run `git branch --show-current` (must be `feat/proxmox-provider`
from Phase 0 on), `git status --short` and `git log --oneline -8`. The first
unchecked box is the phase to do. If its files already exist partially, its
"Done when" line says whether it counts as done; finish it, never redo it. A
phase is finished only when its "Done when" holds and its execution-log entry
(a few sentences, exact commands, "worth noting", the box ticked) is
committed together with the code. Follow `CLAUDE.md` for the log format.

## Before Phase 0: what the user prepares on 192.168.2.8

Run as root on the Proxmox node. **`VM.Monitor` was removed in PVE 9.0**, so
the agent read rests on `VM.GuestAgent.Audit` alone and the role below must
not mention `VM.Monitor`. Drop any privilege `pveum` still rejects and note
it for F17.

```bash
pveum user add proxy-operator@pve --comment "egress-proxies-operator"
pveum role add ProxyOperator -privs "VM.Allocate VM.Clone VM.Audit VM.Config.CDROM VM.Config.CPU VM.Config.Memory VM.Config.Disk VM.Config.Network VM.Config.Options VM.PowerMgmt VM.GuestAgent.Audit Datastore.Audit Datastore.AllocateSpace Datastore.AllocateTemplate Sys.Audit SDN.Use"
pveum aclmod / -user proxy-operator@pve -role ProxyOperator
pveum user token add proxy-operator@pve operator --privsep 0
# The secret is printed once. Save it on the workstation as one line:
#   proxy-operator@pve!operator=<secret>
# in ~/.config/proxmox-token (mode 0600). The same file becomes the
# Kubernetes Secret later.
cp /etc/pve/pve-root-ca.pem /tmp/   # copy to the workstation as ~/.config/proxmox-ca.pem, or use insecureSkipTLSVerify
```

A cloud-init template without a Proxmox cloud-init drive (the provider puts
its own NoCloud ISO on `ide2`). Debian 13's BIOS boot path did not boot
under kvm2, so the template boots UEFI:

```bash
wget https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2
qm create 9000 --name debian-13-template --memory 2048 --cores 2 --ostype l26 \
  --machine q35 --bios ovmf --efidisk0 local-lvm:1,efitype=4m,pre-enrolled-keys=0 \
  --scsihw virtio-scsi-pci --net0 virtio,bridge=vmbr0 \
  --serial0 socket --vga serial0 --agent 1
qm set 9000 --scsi0 local-lvm:0,import-from=/root/debian-13-generic-amd64.qcow2
qm set 9000 --boot order=scsi0
qm template 9000
```

Facts the user hands over at Phase 0: the token file path, the CA file path
(or "insecure"), the node name, the template VMID, the ISO-content storage
id (usually `local`), the VM disk storage (`local-lvm`), the bridge, and
whether the operator's cluster can route to that bridge's LAN. Everything
else Phase 0 reads from the API.

## Design decisions made with the user (not re-opened)

- **Backend and client.** Proxmox VE 9.x REST API, JSON, HTTP/1.1 only
  (pveproxy offers no HTTP/2 on any release; only Proxmox Backup Server
  does).
  Thin client on `net/http` (`client-kind: thin-client`), no SDK, no new
  dependency. Auth header
  `Authorization: PVEAPIToken=<user>@<realm>!<tokenid>=<secret>`, read from a
  mounted token file on every request (rotation free). TLS: `caFile` or
  `insecureSkipTLSVerify` (warned at startup). Child spans from a parent-gated
  otelhttp transport: `tracing.RestConfigWrapper()(base)` returns a plain
  `http.RoundTripper` and works as is (`child-spans: transport-wrapper`).
- **Image source.** Clone a prepared template. Config catalogue
  `templates: {<key>: {vmid: 9000}}`; `Placement.Image` is a key.
  `Placement.Zone` is a node name from `nodes: [..]` (optional with one node).
  `Placement.MachineType` is a key of `machineTypes: {<key>: {cores, memoryMiB,
  diskGiB}}`; `diskGiB` (optional) grows the boot disk via `PUT .../resize`.
  `Placement.Region` is ignored (documented).
- **Identity.** `primary-key: allocated-id`. VMID derived deterministically
  from the Proxy UID (FNV-1a 64 into `vmidRange: {min, max}`, default
  100000 to 199999) with forward probing on collision; a clone whose `newid`
  is taken fails with a 500, which is the allocate-then-verify step. Create
  first looks the name up in `GET /cluster/resources?type=vm`; an existing VM
  with that name must carry our mark and a matching or empty UID, else
  Permanent. providerID is location-free: `vms/<vmid>/<name>` (VMs migrate;
  the node is resolved from `/cluster/resources` on every call). Get and
  Delete verify the VM at that VMID has that name and our mark; a different
  name means our VM is gone and the VMID was reused: Get returns NotFound,
  Delete returns nil without touching it.
- **Cloud-init.** NoCloud seed ISO built in-process by reusing kvm2's seed
  code, extracted into a shared package `internal/provider/nocloud` (its own
  phase). Uploaded to `isoStorage` (default `local`) on the target node via
  `POST /nodes/{node}/storage/{storage}/upload` (multipart, `content=iso`,
  `filename=<name>-seed.iso`, streamed through `io.Pipe`, returns a UPID),
  attached as `ide2: <storage>:iso/<name>-seed.iso,media=cdrom`. The template
  must have no Proxmox cloudinit drive on ide2 (documented prerequisite).
  Vendor-data installs qemu-guest-agent like kvm2 (`installGuestAgent`,
  `vendorData`).
- **GC mark.** `gc-mark: written-metadata` in the VM `description`, which the
  clone call accepts (`tags` is not a clone parameter, so tags would be a
  mark window). Content, one per line: `proxy-operator-managed=true`,
  `proxy-operator-uid=<uid>`, `proxy-operator-created-at=<RFC3339>`.
  `created-at: written-at-create`. The config step after the clone also sets
  tag `proxy_operator` for UI filtering only. **Underscore, not hyphen:**
  current PVE tag validation accepts only lowercase letters, digits and
  underscore, and a hyphen is rejected; the underscore form is valid under
  both the strict and the permissive reading, so it is used regardless of
  what F9 finds. ListByTag: one
  `GET /cluster/resources?type=vm`; candidates are entries with tag
  `proxy_operator` or name prefix `proxy-`; a `GET /nodes/{node}/qemu/{vmid}/config`
  per candidate for the description; unmarked skipped; one unreadable VM
  skipped with a V(1) line unless ctx ended; the resources call failing fails
  the whole list (`list-partial: fail-whole`).
- **Create** (`create-return: blocking`, `provisioning-state: create-blocks`,
  `async-ops: task-handle` and `backend-lock`), every step idempotent, in
  this order: validate placement (Permanent before any call); resources
  lookup by name (adopt or allocate); `POST /nodes/{tmplnode}/qemu/{tmplvmid}/clone`
  `{newid, name, description, full=1, target?, storage?}`; wait the task
  (`exitstatus != OK` is Transient with the message; a VM with `lock=clone`
  on retry is waited on); upload the seed (always rebuilt, delete then
  upload; before the config step because `ide2` may refuse a volume that
  does not exist yet); config (`cores`, `memory`, `net0` bridge override if
  configured, `agent=1`, `tags`, `ide2` seed); optional resize;
  `POST .../status/start` and wait; poll `status/current` until `running`
  (capped); return `vms/<vmid>/<name>`. Retry: name found running returns
  the ID; found stopped with our mark continues from the seed step; found
  locked waits.
- **Get.** Resources lookup by vmid: absent is NotFound; name mismatch or no
  mark is NotFound (logged); `lock` set is Transient; `running` reads
  `GET /nodes/{node}/qemu/{vmid}/agent/network-get-interfaces`, first
  routable IPv4 excluding loopback and link-local gives Running, else
  Provisioning (the agent not running is a 500 "QEMU guest agent is not
  running", which is Provisioning, not an error); `stopped` is Stopped;
  `unknown` is Transient; `qmpstatus=paused` is Stopped; anything else is
  Stopped. UID and CreatedAt come from the description on every call, so a
  Get is three calls: resources, config, agent.
- **Delete** (`delete-return: waits`, `secondary-resources: explicit`):
  resolve by vmid via resources; absent or foreign is nil (logged); running
  gets `POST status/stop` and a task wait; then
  `DELETE /nodes/{node}/qemu/{vmid}?purge=1&destroy-unreferenced-disks=1` and
  a task wait; then delete the seed ISO
  `DELETE /nodes/{node}/storage/{storage}/content/{storage}:iso/<name>-seed.iso`
  tolerating absence; a locked VM is Transient. Called again mid-delete it
  repeats the same tolerant steps. GC blind spot: seed ISOs without a VM
  (documented; the ISO name carries the instance name).
- **Errors**, one `classify(op, err)` over `*apiError{Status, Message, Errors}`:
  401, 403, 400 and 404 are Permanent (404 means the path never exists;
  NotFound comes only from the list-and-miss on resources and from the 500
  "does not exist" message in Get and Delete, matched against captured
  fixtures); 500 "does not exist" is NotFound (Transient inside Create); 500
  "VM is locked" is Transient; 500 "already exists" is handled by the
  allocator, otherwise Transient; 500 "not enough free space" or "no space
  left" is QuotaExceeded once a real message is captured, else `quota:
  absent`; other 5xx, 595, 596, network and ctx errors are Transient;
  anything not an apiError is Transient. Error text: Proxmox puts most of it
  in the HTTP status reason phrase (`HTTP/1.1 500 Configuration file '...'
  does not exist`) and newer releases also in the body's `message`; the
  client reads both, body first.
- **Checker.** One target per configured node, in config order, concurrently,
  on the same http client: `GET /version` (Details `pveVersion`),
  `GET /nodes/{node}/status` (Details `kernel`),
  `GET /nodes/{node}/storage/{isoStorage}/status` (Details `isoStorage`,
  `isoStorageAvailGiB`), template existence from `GET /cluster/resources?type=vm`
  (the `template` flag, any node; Details `templates` as `<found>/<configured>`).
  Reasons: the shared ones plus `TemplateNotFound` and `StorageNotFound`.
  The `status.type` comment in `api/v1alpha1/providerhealth_types.go` gains
  `proxmox`; `make manifests`.
- **Timeouts** (`timeouts: per-call`): request timeout 30s
  (`requestTimeoutSeconds`); task-wait caps clone 10m, upload 2m, start and
  stop 2m, destroy 5m, running wait 60s; all in a `budgets` struct
  overridable in tests. A cancelled ctx cancels that request only; the
  connection pool is unaffected. Per-name keyed lock for Create and Delete
  (`internal/provider/keyedlock`, extracted from kvm2); Get and ListByTag
  lock-free.
- **Logging** (`logging: tiered`): `logf.FromContext(ctx).WithName("proxmox").WithValues("provider", name)`,
  `vmid` and `name` on instance lines; V(1) outcomes and one
  `proxmox api call` line per seam call from the `observed` decorator; V(2)
  request params and response summaries; never the token, never cloud-init,
  never the multipart body, never `description` at V(1); no V(5). The
  decorator logs, the transport traces: no `proxmox.<Method>` spans, so
  nothing is double-counted.
- **Config writes** use `PUT .../config` (synchronous; the VM is stopped at
  that point). `resize` is skipped when the boot disk's current `size=` is
  at or above the target. Boot disk: first disk key in `boot: order=...`
  that is not `net*` and not the ide2 cdrom, else `bootdisk`, else the first
  of `scsi0`, `virtio0`, `sata0`, `ide0`; none is Permanent.
- **A UPID names its node** (`UPID:<node>:...`); task polling uses the node
  parsed from the UPID.
- **Two VMs with our name** (Proxmox does not enforce unique names) make
  Create and Get return Permanent naming both VMIDs; an admin resolves it.
- **Seam.** `api` interface with one method per endpoint (table below),
  `httpAPI` production adapter, `fakeAPI` in `fake_test.go` keeping VMs, tasks
  and ISOs in memory with lock simulation, task polls, per-method errors and
  block channels, and a call log.
- **Integration test** `proxmox_integration_test.go` behind
  `//go:build integration`, `PROXMOX_TEST_URL`, `PROXMOX_TEST_TOKEN_FILE`,
  `PROXMOX_TEST_NODE`, `PROXMOX_TEST_TEMPLATE_VMID`, `PROXMOX_TEST_ISO_STORAGE`,
  optional `_CA_FILE`, `_INSECURE`, `_BRIDGE`, `_STORAGE`, `_VMID_MIN`,
  `_VMID_MAX`, `_TIMEOUT`, `_KEEP`, `_LOG_LEVEL`; `make test-integration-proxmox`.
- **Deployment.** Secret `proxmox-token` mounted optional at
  `/etc/proxy-operator/proxmox/token`, `make proxmox-token-secret`, commented
  entries in `config/manager/providers_config.yaml` and
  `config/samples/providers-config.yaml`, `config/samples/proxy_proxmox.yaml`,
  `docs/proxmox-provider.md`, `providers-existing.md` column and section,
  `architecture.md` rows, README row, `testing.md` section, CHANGELOG after
  confirmation. Branch `feat/proxmox-provider`, MR via the Gitea API (no `tea`).

## Backend facts Phase 0 must verify on 192.168.2.8 (numbered for citation)

- F1. Where error text lives: status reason phrase vs. JSON `message`; the
  `errors` map shape on a 400.
- F2. Exact 500 message for a missing VM (`GET /nodes/{node}/qemu/999999/config`).
- F3. Exact 500 message for a locked VM (`POST status/start` on a VM with
  `lock` set), and whether `PUT config` with `lock=backup` sets the lock and
  `delete=lock` clears it through the API, or `qm unlock` is needed.
- F4. Exact 500 message when `newid` is in use on clone.
- F5. Exact 500 message for the agent not running, and for the agent call
  on a stopped VM.
- F6. Exact message for no space, if it can be produced cheaply; otherwise
  `quota: absent`.
- F7. Whether `PUT .../config` is refused while `lock=clone`, and with which
  message.
- F8. `/cluster/resources?type=vm` fields present: `vmid`, `name`, `node`,
  `status` values (`running`, `stopped`, `unknown`), `type` (`qemu`, `lxc`),
  `lock`, `tags` (separator), `template`; whether `lock` appears there during
  a clone.
- F9. Tag rules on 9.2: confirm a hyphen is rejected and `proxy_operator`
  accepted, plus case folding, length, and the `tags` separator on
  `PUT config` and in `/cluster/resources`. The underscore form stays either
  way; this fact only decides what the guide tells users about their own
  tags.
- F10. `description` on `POST .../clone`: accepted, survives verbatim
  (newlines), max length; whether `name` accepts the 22-character
  `proxy-<base32>` form.
- F11. `meta` in the cloned config: is `ctime` the clone time (evidence only;
  `written-at-create` stays the profile).
- F12. Upload endpoint: field names (`content`, `filename`, file part name
  `filename`), the filename rule (`.iso` suffix, no slashes), the response
  (UPID), the privilege it needs, and whether re-uploading an existing name
  is refused (then delete-then-upload) or overwrites. On 9.x `content`
  accepts `iso`, `vztmpl` or `import` and still refuses `snippets`, which is
  why the seed travels as an ISO; confirm `iso` is what an `.iso` needs.
- F13. `DELETE /nodes/{node}/storage/{storage}/content/{volid}`: response
  shape (UPID or null) and the message for a missing volume.
- F14. `status/current` on a stopped VM: is `qmpstatus` present; on a paused
  VM: `status=running`, `qmpstatus=paused`.
- F15. `POST .../clone`: whether a failed clone destroys the partial VM by
  itself, and what a VM interrupted mid-clone looks like (config present,
  `lock=clone`).
- F16. Task status shape: `status` (`running`, `stopped`), `exitstatus`
  (`OK` or the error string), and how long a full clone of the template
  takes.
- F17. Privileges: which of the prepared list the token really needs, in
  particular the exact ACL path that satisfies `SDN.Use` for the `net0`
  write (the 9.x docs describe it as "access SDN vnets and local network
  bridges" without spelling out the path for a plain bridge), and
  `VM.GuestAgent.Audit` for the agent read. Also whether the config step
  needs `VM.Config.Cloudinit` even though only `ide2` is written.
- F18. HTTP/2 is not offered by pveproxy on any release (documented), so the
  client disables it; confirm with `curl --http2 -I` that the connection
  falls back to HTTP/1.1.
- F19. The template's disk key (`scsi0` or `virtio0`), its `boot` line, that
  ide2 is free and no `cloudinit` drive exists.
- F20. `resize` on a stopped VM: request shape, the message when the target
  is smaller or equal, and that it is synchronous.
- F21. Agent response shape: `result[].name`, `result[]["ip-addresses"][]`
  with `ip-address`, `ip-address-type`.
- F22. The 401 body and reason phrase on 9.2. PVE 9.2 unified the
  authentication-failure message so a failed login no longer leaks whether
  the user exists, so any 401 text remembered from 8.x is wrong. Capture it
  fresh into `err_unauthorized.json`.
- F23. Where the upload endpoint stages the request body, given that PVE 9
  moved the node's `/tmp` to a tmpfs capped at half of RAM and cleans `/tmp`
  and `/var/tmp` periodically. A seed ISO is tens of KiB so this cannot
  bite in practice, but the observation belongs in the guide next to the
  size limit.

## Proxmox endpoint map for the seam

| `api` method | HTTP |
|---|---|
| `Version(ctx)` | `GET /version` |
| `NodeStatus(ctx, node)` | `GET /nodes/{node}/status` |
| `StorageStatus(ctx, node, storage)` | `GET /nodes/{node}/storage/{storage}/status` |
| `ClusterResources(ctx)` | `GET /cluster/resources?type=vm` |
| `VMConfig(ctx, node, vmid)` | `GET /nodes/{node}/qemu/{vmid}/config` |
| `SetVMConfig(ctx, node, vmid, params)` | `PUT /nodes/{node}/qemu/{vmid}/config` |
| `Clone(ctx, node, vmid, cloneParams)` | `POST /nodes/{node}/qemu/{vmid}/clone` |
| `Resize(ctx, node, vmid, disk, size)` | `PUT /nodes/{node}/qemu/{vmid}/resize` |
| `Start(ctx, node, vmid)` | `POST /nodes/{node}/qemu/{vmid}/status/start` |
| `Stop(ctx, node, vmid)` | `POST /nodes/{node}/qemu/{vmid}/status/stop` |
| `VMStatus(ctx, node, vmid)` | `GET /nodes/{node}/qemu/{vmid}/status/current` |
| `Destroy(ctx, node, vmid)` | `DELETE /nodes/{node}/qemu/{vmid}?purge=1&destroy-unreferenced-disks=1` |
| `TaskStatus(ctx, node, upid)` | `GET /nodes/{node}/tasks/{upid}/status` |
| `AgentNetworkInterfaces(ctx, node, vmid)` | `GET /nodes/{node}/qemu/{vmid}/agent/network-get-interfaces` |
| `UploadISO(ctx, node, storage, filename, r, size)` | `POST /nodes/{node}/storage/{storage}/upload` (multipart) |
| `DeleteVolume(ctx, node, storage, volid)` | `DELETE /nodes/{node}/storage/{storage}/content/{volid}` |

## Package layout (`internal/provider/proxmox/`)

`proxmox.go` (`Provider`, `New`, `newProvider`, `withDefaults`, `logger`,
`node`, `wrapErr`, `lockName`, `providerOption`, `budgets`), `api.go` (the
`api` interface and DTOs `vmResource`, `vmConfig`, `vmStatus`, `taskStatus`,
`agentInterface`, `storageStatus`, `nodeStatus`, `versionInfo`,
`cloneParams`), `client.go` (`httpAPI`, `newHTTPAPI`, `do`, `readToken`,
`decodeConfig`), `errors.go` (`apiError`, `classify`, `permanentf`,
`transientf`, `isNotExist`, `isLocked`, `isAlreadyExists`,
`isAgentNotRunning`, `isNoSpace`), `id.go` (`formatProviderID`,
`parseProviderID`, `vmidFor`), `mark.go` (`instanceMark`,
`formatDescription`, `parseDescription`), `state.go` (`mapState`,
`firstIPv4`), `observe.go` (`observed`, `observe`), `task.go` (`parseUPID`,
`waitTask`, `waitUnlocked`, `waitRunning`), `seed.go` (`seedISOName`,
`seedVolID`, `uploadSeed`), `create.go`, `get.go`, `delete.go`, `list.go`,
`check.go`, plus `fake_test.go`, `provider_test.go`,
`proxmox_integration_test.go`, `testdata/*.json`. Shared:
`internal/provider/config_proxmox.go`, `internal/provider/nocloud/`,
`internal/provider/keyedlock/`.

Reused as-is: `provider.NameFromUID`, `provider.Wrap`, the `Err*` sentinels,
`provider.Label*` constants, `provider.Reason*` constants,
`tracing.RestConfigWrapper`, `tracing.WithTracerProvider`, kvm2's test
idioms (`fixture`, `assertProviderError`, `logCapture`).

## Steps (one commit per phase, on `feat/proxmox-provider` off `main`)

- **Phase 0 (orchestrator with the user): plan file, branch, host recon,
  fixtures.** Read first: contract §5, §4.9, §7;
  `docs/plans-executions/2026-09-16-1333-kvm2-provider.md` Step 0 entry.
  Do, in this order: `git switch main && git pull --ff-only`; copy this plan
  to `docs/plans/2026-09-18-1502-proxmox-provider.md` with the real timestamp
  (`date "+%Y-%m-%d-%H%M"`), commit it alone on `main`
  (`docs: plan the proxmox provider`); create `feat/proxmox-provider`. Ask
  the user for the hand-over facts listed above. Then, with the user's
  go-ahead (each `curl` is a permission prompt; read-only ones first, one at
  a time), capture F1 to F21 with
  `curl -s --cacert "$CA" -D /tmp/h.txt -o /tmp/b.json -w '%{http_code}\n' -H "Authorization: PVEAPIToken=$(cat "$TOKEN_FILE")" https://192.168.2.8:8006/api2/json/<path>`,
  storing each response as `internal/provider/proxmox/testdata/<name>.json`
  in the shape `{"status": <code>, "reason": "<status line after the
  code>", "body": "<raw body>"}`: `version.json`, `node_status.json`,
  `storage_status.json`, `cluster_resources.json`,
  `vm_config_template.json`, `vm_config_clone.json`,
  `status_current_running.json`, `status_current_stopped.json`,
  `agent_network_get_interfaces.json`, `task_status_running.json`,
  `task_status_stopped_ok.json`, `task_status_stopped_error.json`,
  `upload_response.json`, `content_delete_response.json`,
  `err_vm_missing.json`, `err_locked.json`, `err_newid_in_use.json`,
  `err_agent_not_running.json`, `err_agent_vm_stopped.json`,
  `err_unauthorized.json` (a wrong secret), `err_forbidden.json` (a path
  the token lacks), `err_bad_param.json` (`newid=abc`), `err_no_space.json`
  (only if F6 is cheap), `err_volume_missing.json` (F13). The write
  experiments: one clone of the template into the top of the VMID range
  (`newid=199999`, a multi-line `description`, a `name` of the `proxy-`
  form), a config read of the clone (F10, F11, F19), a `PUT config` while
  it is locked if timing allows (F7), an upload of a 1 KiB
  `recon-seed.iso` (F12) and its delete (F13), a resize attempt (F20), a
  start, an agent read (F5, F21), a stop, then destroy with
  `purge=1&destroy-unreferenced-disks=1`; a final resources read shows
  nothing remains. Redact nothing but the token. Write the answers to F1 to
  F21 and the privilege outcome into the execution log's Phase 0 entry, one
  line per fact; Phase 1 cites them as Evidence. Create the execution log
  `docs/plans-executions/2026-09-18-1502-proxmox-provider.md` with the checklist (Phase
  0 to Phase 10) and this entry.
  Verify: `ls internal/provider/proxmox/testdata | wc -l` is at least 22;
  `grep -rl 'PVEAPIToken' internal/provider/proxmox/testdata` is empty;
  every fixture parses as JSON; the host shows no VM in the range and no
  `recon-seed.iso`.
  Done when: the plan commit is on `main`, the branch exists, the fixtures
  and the execution log with F1 to F21 are committed. Commit: `proxmox:
  captured API fixtures and host facts from 192.168.2.8`.

- **Phase 1 (orchestrator): the spec.** Read first: contract §2, §4, §5,
  §6, §11, §12; `docs/provider-spec-template.md` end to end;
  `docs/kvm2-provider-spec.md` for the level of detail; this plan's
  decisions; the Phase 0 log entry. Create `docs/proxmox-provider-spec.md`
  answering all 24 entries: 1 `vms/<vmid>/<name>`, strict parsing, the
  node-name and storage-id regexps; 2 `allocated-id`, `vmidFor(uid, rng,
  attempt)`, the two-VMs-with-our-name rule, the adoption guard; 3 the
  placement table (Region ignored); 4 the template catalogue and the "add a
  key, never edit" warning; 5 the seed ISO via `nocloud` and the ide2
  prerequisite; 6 `machineTypes` with `diskGiB`; 7 `provisioning-window`,
  `discovered` via the guest agent, the IPv4 filter, `Port` unused; 8
  `written-metadata` in `description` plus the tag, `written-at-create`, F9
  to F11 as Evidence; 9 the state table, `create-blocks`; 10 the ordered
  Create steps with what a retry finds at each, `task-handle` plus
  `backend-lock`, the budgets; 11 the Delete order, the ISO blind spot,
  node-local ISO storage after a migration; 12 `fail-whole`, the candidate
  rule, one config read per candidate; 13 the classify table with the F1 to
  F6 messages, `not-found: error-code` plus list-and-miss on resources
  (allowed because offline nodes' guests appear as `unknown`, not absent);
  14 `mounted-files` (Secret `proxmox-token`, key `token`, path
  `/etc/proxy-operator/proxmox/token`), `ca-file` or
  `insecureSkipTLSVerify`, the privilege list from F17, rotation free; 15
  the config table (Phase 2's struct); 16 the Checker; 17 `thin-client`,
  `transport-wrapper`, the seam's method list; 18 the budgets table; 19 the
  keyed lock per name; 20 the log keys; 21 the §11 list with the test names
  from Phases 2 to 8 (copy from this plan); 22 the deployment list; 23 the
  docs list; 24 the accepted limits (blocking Create up to the clone budget,
  ISO blind spot, VMID range shared with humans, node-local ISO storage, no
  adoption of foreign VMs, two-installs GC caveat, `unknown` status while a
  node is down).
  Verify: `grep -c '^\*\*Answer:\*\*$' docs/proxmox-provider-spec.md` is 0;
  every axis code used appears in contract §4.20; Evidence lines cite F
  numbers; `grep -c '—' docs/proxmox-provider-spec.md` is 0.
  Done when: the spec is committed with all 24 entries and the Phase 1 box
  ticked. Commit: `docs: proxmox provider spec`.

- **Phase 2 (implementer): config block, registration, package skeleton.**
  Read first: spec entries 1, 2, 3, 6, 8, 9, 13, 15; contract §10, §5, §6;
  `internal/provider/config.go`, `config_kvm.go`, `config_test.go`,
  `config_kvm_test.go`; `internal/provider/kvm2/{kvm2.go,id.go,metadata.go,state.go,errors.go}`
  and their tests as the pattern; `cmd/main.go`; `testdata/err_*.json`.
  Create `internal/provider/config_proxmox.go`: `ProxmoxConfig{URL,
  TokenFile, CAFile string; InsecureSkipTLSVerify bool; Nodes []string;
  ISOStorage, Storage, Bridge string; VMIDRange *ProxmoxVMIDRange; Templates
  map[string]ProxmoxTemplate; MachineTypes map[string]ProxmoxMachineType;
  RequestTimeoutSeconds int; VendorData string; InstallGuestAgent *bool}`,
  `ProxmoxVMIDRange{Min, Max int}`, `ProxmoxTemplate{VMID int}`,
  `ProxmoxMachineType{Cores int32; MemoryMiB int64; DiskGiB int64}`,
  `validateProxmox(prefix string, c *ProxmoxConfig) error` (URL http(s)
  with a host; `tokenFile` non-empty; at least one node, unique, matching
  `proxmoxNodeName = ^[a-z0-9][a-z0-9-]*$`; `isoStorage` and `storage`
  matching `proxmoxStorageID = ^[A-Za-z][A-Za-z0-9_.-]*$`; `vmidRange` min
  at least 100, max at most 999999999, min below max; at least one template
  with `vmid` at least 100; at least one machine type with cores and
  memoryMiB above zero and diskGiB not negative; `requestTimeoutSeconds`
  not negative). Edit `config.go`: `Proxmox *ProxmoxConfig` with tag
  `proxmox,omitempty`, `setBlocks`, the allow-list, the dispatch. Edit
  `cmd/main.go`: `"proxmox": proxmox.New`. Create in
  `internal/provider/proxmox/`: `proxmox.go` (`Provider{name, cfg, api,
  now, budgets, nameLocks, tracing}`; until Phase 3 a local `keyedLocks`
  copy in `locks.go` that Phase 3 deletes; `New` never dials: builds the
  client, `os.Stat`s the token file and warns, warns on
  `insecureSkipTLSVerify`; `withDefaults`: `isoStorage` `local`, `vmidRange`
  100000 to 199999, `requestTimeoutSeconds` 30, `installGuestAgent` true;
  `node(zone)` mirrors kvm2's `host`; the four methods stubbed with
  `transientf("not implemented")`), `api.go` (the interface and DTOs;
  `vmConfig` is `map[string]string` with scalars rendered as strings by
  `decodeConfig`), `id.go`, `mark.go` (keys from `provider.LabelManaged`,
  `provider.LabelManagedYes`, `provider.LabelUID` and
  `proxy-operator-created-at`; `parseDescription` tolerates extra lines
  and CRLF), `state.go`, `errors.go` (`apiError{Status int; Message
  string; Errors map[string]string}` whose `Error()` never includes headers
  or URLs; `classify` per spec entry 13; matcher helpers keyed on the exact
  substrings captured in F2 to F6). Tests: `config_proxmox_test.go`
  `TestValidateProxmox_table` (missing url, non-http scheme, missing
  tokenFile, no nodes, duplicate node, node with a slash, bad isoStorage
  id, vmidRange min below 100, min above max, no templates, template vmid
  below 100, no machineTypes, cores zero, negative diskGiB, negative
  timeout, valid full, valid minimal) with a `validProxmoxYAML` const;
  `config_test.go` rows `proxmox type with kvm block`, `kvm type with
  proxmox block`, `proxmox minimal entry loads`. `id_test.go`:
  `TestFormatProviderID_roundTrip`, `TestParseProviderID_malformed`,
  `TestVMIDFor_deterministicAndInRange`, `TestVMIDFor_probesForwardAndWraps`.
  `mark_test.go`: `TestFormatDescription_keys`, `TestParseDescription_table`.
  `state_test.go`: `TestMapState_table`, `TestFirstIPv4_filters`.
  `errors_test.go`: `TestClassify_table` (every class, every matched
  message loaded from `testdata/err_*.json` through `loadFixture(t, name)
  *apiError`, `op == "create"` turning NotFound into Transient,
  `context.Canceled`, a `*net.OpError`, `errors.New("x")`,
  already-classified errors keep their class), `TestAPIError_matchers`,
  `TestPermanentfAndTransientf`. `proxmox_test.go`:
  `TestNew_doesNotDialAndWarnsOnMissingToken`,
  `TestNew_insecureSkipTLSVerifyWarns`, `TestNew_nilBlockDoesNotPanic`,
  `TestWithDefaults_table`, `TestNode_table`,
  `TestWrapErr_carriesProviderNameOpAndID`.
  Verify: `grep -c '"proxmox"' internal/provider/config.go` is 3;
  `go build ./... && go vet ./... && go test -race -count=1 ./internal/provider/... ./cmd/... && go test -short ./...`;
  the fixture loader fails loudly on a missing file.
  Done when: `go test -race ./internal/provider/proxmox/` passes with the
  stubs in place and the constructor is registered. Commit: `proxmox:
  config block, IDs, marks, state mapping, classify, registration`.

- **Phase 3 (implementer): extract `nocloud` and `keyedlock` from kvm2.**
  Read first: `internal/provider/kvm2/{seed.go,seed_test.go,kvm2.go,kvm2_test.go,create.go}`;
  contract §1.4, §2.1; spec entries 5 and 19;
  `grep -n 'seed.go\|keyedLocks\|buildSeedISO' docs/*.md`. Create
  `internal/provider/nocloud/seed.go` exporting `BuildSeedISO(name,
  userData, vendorData string, w io.Writer) error` (plain errors, an
  `ErrEmptyName` sentinel instead of kvm2's `permanentf`),
  `BuildVendorData(configured string, installAgent bool) (string, error)`,
  `GuestAgentScript`, `SeedLabel`; move the seed tests to
  `nocloud/seed_test.go` under their existing names. Create
  `internal/provider/keyedlock/keyedlock.go` exporting `Locks`, `New()
  *Locks`, `(*Locks).Acquire(ctx, key) (release func(), err error)`; move
  the `TestKeyedLocks_*` tests. Edit kvm2: delete `seed.go`, `seed_test.go`,
  the `keyedLocks` type and its tests; `create.go` calls
  `nocloud.BuildVendorData` and `nocloud.BuildSeedISO` and wraps their
  errors with `permanentf`; `kvm2.go` uses `*keyedlock.Locks`. Edit
  `internal/provider/proxmox/proxmox.go` to use `keyedlock.Locks` and delete
  `locks.go`. Fix the doc mentions so they name the new packages.
  Verify: `go test -race -count=1 ./internal/provider/...`; kvm2's test
  count drops by exactly the moved tests and the two new packages carry
  them; `grep -rn 'iso9660' internal/provider/kvm2/` is empty;
  `go vet -tags=integration ./...` still compiles.
  Done when: kvm2 and proxmox import both packages, all tests pass, the docs
  name the new packages. Commit: `provider: extract nocloud seed builder
  and keyedlock from kvm2`.

- **Phase 4 (implementer): seam, HTTP client, decorator, fake.** Read first:
  spec entries 14, 17, 18, 20; contract §4.15, §7, §9;
  `internal/tracing/transport.go`; `internal/provider/kvm2/{observe.go,observe_test.go,fake_test.go}`;
  `internal/provider/proxmox/{api.go,errors.go}`; `testdata/*.json`. Create
  `client.go`: `httpAPI{base *url.URL; http *http.Client; tokenFile
  string}`; `newHTTPAPI(cfg provider.ProxmoxConfig, opts ...tracing.Option)
  (*httpAPI, error)` builds a transport with `DialContext` timeout 10s,
  `TLSHandshakeTimeout` 10s, `ResponseHeaderTimeout` from the request
  timeout, `IdleConnTimeout` 90s, `MaxIdleConnsPerHost` 4,
  `ForceAttemptHTTP2: false`, an empty `TLSNextProto` map (F18), `RootCAs`
  from `caFile` or `InsecureSkipVerify` from the flag, wrapped with
  `tracing.RestConfigWrapper(opts...)(transport)`; `http.Client.Timeout` is
  the request timeout; `do(ctx, method, path string, query, form
  url.Values, out any) error` reads the token file on every call (missing
  file `transientf`, a file without `!` and `=` `permanentf`), sets the
  header, decodes `{"data": ...}` into `out`, turns any non-2xx into
  `*apiError` per F1 with the error body read capped at 64 KiB;
  `UploadISO` streams a multipart body through `io.Pipe` with a goroutine
  writing `content=iso`, `filename` and the file part. Create `observe.go`:
  `observed{inner api; provider string}` writing one V(1) `proxmox api
  call` line per seam call with `call`, `node`, `vmid`, `storage`, `upid`,
  `duration`, `httpStatus` and `error` on failure, V(2) adding request
  params and a response summary; `observe(inner api, providerName string)
  api`; `New` wraps `httpAPI` in it. Create `fake_test.go`: `fakeAPI{mu; vms
  map[int]*fakeVM; tasks map[string]*fakeTask; isos
  map[string]map[string][]byte; hooks; block; calls; log; taskPolls int;
  cloneFailsWith, startFailsWith string; agentNotRunning, cloneLeavesLock
  bool}`, `fakeVM{node, name, status, qmpstatus, lock string; template
  bool; config map[string]string; ifaces []agentInterface}`,
  `fakeTask{node string; polls int; exit string; onDone func()}`; `Clone`
  creates the VM with `lock=clone` and a task that clears the lock or, per
  `cloneFailsWith`, removes the VM (F15); `Start` and `Stop` flip status
  through a task; `Destroy` removes the VM through a task; `UploadISO`
  records bytes; `DeleteVolume` errors with the F13 message when absent;
  `enter(ctx, method)` for blocks and hooks like kvm2; helpers `addVM`,
  `addTemplate`, `failWith`, `blockOn`, `callCount`, `callLog`,
  `fakeErr(status int, msg string) *apiError`. Tests: `client_test.go` over
  `httptest.NewTLSServer`: `TestClient_sendsTokenHeaderFromFileOnEveryRequest`
  (rotate the file between two calls), `TestClient_missingTokenFileIsTransient`,
  `TestClient_unparsableTokenFileIsPermanent`,
  `TestClient_apiErrorFromStatusReasonAndBody` (serve each
  `testdata/err_*.json` verbatim), `TestClient_decodesDataEnvelope`,
  `TestClient_http1Only`, `TestClient_requestTimeoutIsTransient`,
  `TestClient_cancelledContextReturnsPromptly`,
  `TestClient_childSpanUnderParent`, `TestClient_noSpanWithoutParent`,
  `TestClient_tokenNeverInErrorsOrLogs` (sentinel secret echoed by the
  server in a 401 body), `TestClient_caFileTrustsPrivateCA`,
  `TestClient_insecureSkipsVerification`, `TestClient_uploadStreamsMultipart`
  (the server parses the multipart; the client never `io.ReadAll`s the
  reader), `TestClient_destroyQueryFlags`. `observe_test.go`:
  `TestObserve_everySeamMethodLogsOneLine` (reflection over `api`),
  `TestObserve_detailOnlyAtV2`, `TestObserve_uploadBodyNeverLogged`,
  `TestObserve_errorFieldsOnFailure`, `TestObserve_v0Silent`.
  Verify: `go test -race -count=2 ./internal/provider/proxmox/`;
  `grep -n 'ReadAll' internal/provider/proxmox/client.go` shows only the
  bounded error-body read; `grep -n 'context.Background()' internal/provider/proxmox/*.go`
  is empty outside tests.
  Done when: the client passes its tests against the fixtures and the fake
  compiles against the full `api` interface. Commit: `proxmox: thin HTTP
  client, seam, log decorator and in-memory fake`.

- **Phase 5 (implementer): Get, ListByTag, task helpers.** Read first: spec
  entries 7, 8, 9, 12, 13; contract §2.2, §2.4, §4.7, §4.12;
  `internal/provider/kvm2/{get.go,get_test.go,list.go,list_test.go}`; the
  resources, config, status, agent and task fixtures. Create `task.go`:
  `parseUPID(upid) (node string, err error)` (Permanent when malformed),
  `waitTask(ctx, log, api, upid, budget) error` polling `TaskStatus` every
  `budgets.taskPoll` (2s; tests shrink it) until `status == "stopped"`,
  `exitstatus != "OK"` giving `transientf` with the message, the budget
  giving `transientf`, ctx giving Transient; `waitUnlocked(ctx, api, node,
  vmid, budget)` polling `VMConfig` until no `lock`. Create `get.go`:
  `resolve(ctx, api, vmid) (vmResource, bool, error)` (one resources call,
  `Type == "qemu"` and the vmid), the name check, `VMConfig` for the mark,
  `lock` Transient, `unknown` Transient, `running` reads the agent
  (`isAgentNotRunning` gives no IP, other errors Transient), `mapState`.
  Create `list.go` per spec entry 12; `Instance` carries ID, UID, CreatedAt
  only. Replace the two stubs. Tests: `task_test.go` `TestParseUPID_table`,
  `TestWaitTask_table` (ok, error exitstatus, budget exceeded, ctx
  cancelled, poll count), `TestWaitUnlocked_table`. `get_test.go`:
  `TestGet_missingVMIsNotFound`, `TestGet_nameMismatchIsNotFound`,
  `TestGet_unmarkedIsNotFound`, `TestGet_runningWithoutIPIsProvisioning`,
  `TestGet_runningWithIPIsRunning`, `TestGet_agentNotRunningIsProvisioning`,
  `TestGet_agentOtherErrorIsTransient`, `TestGet_ipFilterSkipsLoopbackAndLinkLocal`,
  `TestGet_stoppedIsStopped`, `TestGet_pausedIsStopped`,
  `TestGet_unknownStatusIsTransient`, `TestGet_lockedIsTransientNeverNotFound`,
  `TestGet_unknownStateIsStopped`, `TestGet_readsUIDAndCreatedAtFromBackend`,
  `TestGet_unmanagedDescriptionHasEmptyUID`, `TestGet_malformedIDIsPermanent`,
  `TestGet_resourcesFailureIsTransient`, `TestGet_twoVMsWithOurNameIsPermanent`.
  `list_test.go`: `TestListByTag_excludesUnmarkedVMs`,
  `TestListByTag_emptyUIDStillListed`, `TestListByTag_candidatesByTagOrNamePrefix`,
  `TestListByTag_skipsLXCAndTemplates`, `TestListByTag_resourcesFailureFailsWhole`,
  `TestListByTag_unreadableVMSkipped`, `TestListByTag_unreadableVMWithEndedContextFails`,
  `TestListByTag_unknownStatusStillAttempted`, `TestListByTag_doesNotReadStatusOrAgent`,
  `TestListByTag_idsAcceptedByParse`.
  Verify: `go test -race -count=2 ./internal/provider/proxmox/`;
  `grep -n 'not implemented' internal/provider/proxmox/*.go` lists only
  Create and Delete.
  Done when: Get and ListByTag are real and every listed test passes.
  Commit: `proxmox: Get, ListByTag and task polling`.

- **Phase 6 (implementer): Create.** Read first: spec entries 2, 3, 4, 5, 6,
  10, 19; contract §2.1, §4.2, §4.10, §4.11, §5;
  `internal/provider/kvm2/{create.go,create_test.go}`;
  `internal/provider/nocloud`; F4, F7, F10, F12, F15, F16, F19, F20 in the
  Phase 0 log. Create `seed.go`: `seedISOName(name)` is `<name>-seed.iso`,
  `seedVolID(storage, name)` is `<storage>:iso/<name>-seed.iso`,
  `uploadSeed(ctx, log, api, node, storage, req, cfg)` builds vendor-data
  and the ISO into a `bytes.Buffer` (tens of KiB), deletes an existing
  volume tolerating absence, uploads with the buffer's length, waits the
  task with `budgets.upload`, logs `cloudInitBytes` only. Create `create.go`
  implementing the ordered steps: placement validation (`node`,
  `MachineTypes`, `Templates`, all Permanent naming field and value), the
  per-name lock, `ClusterResources` once for both the name lookup (adoption
  guard, then `waitUnlocked` if locked, then continue from the seed step if
  stopped, return if running) and the template's node (absent template is
  Permanent); allocation loop `for attempt := 0; attempt < maxProbes (32)`
  calling `Clone` with `vmidFor(req.UID, rng, attempt)`, `isAlreadyExists`
  moving to the next attempt, exhaustion Transient; `waitTask` with
  `budgets.clone`; `uploadSeed`; `SetVMConfig` with `cores`, `memory`,
  `agent=1`, `tags=proxy_operator`, `ide2=<volid>,media=cdrom`, and `net0`
  rewritten from the cloned config with `bridge=<cfg.Bridge>` only when
  configured (MAC kept); `bootDisk(cfg)` and `Resize` when `diskGiB` is
  above the current `size=`; `Start` and `waitTask` with `budgets.start`;
  `waitRunning` on `VMStatus` with `budgets.running`; return the ID.
  `isNotExist` inside Create is Transient (classify with `op == "create"`).
  Tests `create_test.go`: `TestCreate_repeatReturnsSameIDAndCreatesNothing`,
  `TestCreate_resumesAfterEachPartialStep` (after clone task, after seed
  upload, after config, after resize, after start, after running),
  `TestCreate_returnsOnlyWhenRunning`, `TestCreate_runningWaitCapIsTransient`,
  `TestCreate_rejectsUnknownPlacement`, `TestCreate_zoneOptionalWithOneNode`,
  `TestCreate_regionIgnored`, `TestCreate_adoptionWithForeignUIDIsPermanent`,
  `TestCreate_adoptsOnlyMarkedVMs`, `TestCreate_duplicateNameIsPermanent`,
  `TestCreate_vmidCollisionProbesForward`, `TestCreate_vmidRangeExhaustedIsTransient`,
  `TestCreate_waitsForCloneLockOnRetry`, `TestCreate_cloneTaskFailureIsTransient`,
  `TestCreate_cloneTaskTimeoutIsTransient`, `TestCreate_markWrittenInCloneCall`,
  `TestCreate_tagSetInConfigStep`, `TestCreate_seedAlwaysRebuilt`,
  `TestCreate_seedUploadedBeforeConfig`, `TestCreate_configSetsCoresMemoryAgentAndSeed`,
  `TestCreate_bridgeOverrideKeepsMAC`, `TestCreate_noBridgeOverrideLeavesNet0`,
  `TestCreate_resizeOnlyWhenLarger`, `TestBootDisk_table`,
  `TestCreate_startTaskFailureIsTransient`, `TestCreate_templateMissingIsPermanent`,
  `TestCreate_notExistInsideCreateIsTransientNotNotFound`,
  `TestCreate_cloudInitSentinelNeverLoggedOrInErrors` (V(2) capture, every
  error path), `TestCreate_concurrentSameNameSerialised`,
  `TestCreate_cancelledMidWaitReturnsPromptly`.
  Verify: `grep -n 'CloudInit' internal/provider/proxmox/create.go internal/provider/proxmox/seed.go`
  shows only `len(req.CloudInit)` and the pass-through into
  `nocloud.BuildSeedISO`; `go test -race -count=2 ./internal/provider/proxmox/`;
  `grep -n 'not implemented'` lists only Delete.
  Done when: Create is real and every listed test passes. Commit:
  `proxmox: Create by cloning a template, seed upload, config, start`.

- **Phase 7 (implementer): Delete, Checker, CRD comment.** Read first: spec
  entries 11, 16; contract §2.3, §4.3, §4.13, §8;
  `internal/provider/kvm2/{delete.go,delete_test.go,check.go,check_test.go}`;
  `internal/provider/check.go`; `api/v1alpha1/providerhealth_types.go`; F13,
  F17 in the Phase 0 log. Create `delete.go`: parse the ID (Permanent), the
  per-name lock, `resolve`; absent is a logged no-op; name mismatch or
  unmarked is a logged no-op; `lock` Transient; `unknown` Transient;
  `running` gets `Stop` and `waitTask` (`budgets.stop`); `Destroy` and
  `waitTask` (`budgets.destroy`), `isNotExist` from `Destroy` being
  success; `DeleteVolume` of the seed on the VM's node tolerating the F13
  absent message; nil. Create `check.go`: `var _ provider.Checker =
  (*Provider)(nil)`; `reasonTemplateNotFound = "TemplateNotFound"`,
  `reasonStorageNotFound = "StorageNotFound"`; `Check` fans out one
  goroutine per node in config order (`wg.Go`), each `checkNode` running
  `Version`, `NodeStatus`, `StorageStatus` (a 500 "does not exist" is
  `StorageNotFound`; a storage whose `content` lacks `iso` is
  `StorageNotFound` with a message saying so), `ClusterResources` for the
  templates; `checkReason(err)`: ctx ended `Timeout`, 401 `Unauthorized`,
  403 `Forbidden`, other `*apiError` `APIError`, network `Unreachable`;
  Details `pveVersion`, `kernel`, `isoStorage`, `isoStorageAvailGiB`,
  `templates`; `Name` is the node, `Address` the base URL. Edit
  `api/v1alpha1/providerhealth_types.go` (`kubernetes, gcp, kvm, kvm2 or
  proxmox`) and run `make manifests`. Tests `delete_test.go`:
  `TestDelete_malformedIDIsPermanent`, `TestDelete_twiceSucceeds`,
  `TestDelete_absentIsNoOp`, `TestDelete_foreignNameIsNoOp`,
  `TestDelete_unmarkedIsNoOp`, `TestDelete_runningVMStoppedFirst`,
  `TestDelete_stoppedVMSkipsStop`, `TestDelete_lockedIsTransient`,
  `TestDelete_unknownStatusIsTransient`, `TestDelete_removesSeedISO`,
  `TestDelete_toleratesMissingISO`, `TestDelete_destroyTaskFailureIsTransient`,
  `TestDelete_usesPurgeAndDestroyUnreferencedDisks`,
  `TestDelete_concurrentSameNameSafe`, `TestDelete_noOpLinesAreSilentAtV0`;
  in `get_test.go`: `TestGet_afterDeleteReturnsNotFound`,
  `TestGet_duringDeleteStillReturnsInstance` (block the fake's `Destroy`).
  `check_test.go`: `TestCheck_oneEntryPerNodeInConfigOrder`,
  `TestCheck_nodesCheckedConcurrently`, `TestCheck_expiredContextReturnsPromptly`,
  `TestCheck_reasonsTable`, `TestCheck_detailsKeys`, `TestCheck_logsVerdictPerNode`,
  `TestCheckReason_table`.
  Verify: `go test -race -count=2 ./internal/provider/proxmox/ ./internal/providerhealth/`;
  `git diff --stat config/` shows only the CRD description line;
  `grep -rn 'not implemented' internal/provider/proxmox/` is empty.
  Done when: all four contract methods and `Check` are real. Commit:
  `proxmox: Delete with seed cleanup, health Checker, CRD type comment`.

- **Phase 8 (implementer): contract sweep, observability sweep, integration
  test, Makefile, testing.md.** Read first: contract §11 and §9; spec
  entries 17, 18, 20, 21; `internal/provider/kvm2/{provider_test.go,kvm2_integration_test.go}`;
  `internal/provider/tracing_test.go`; `Makefile` integration targets;
  `docs/testing.md` kvm2 rows and section. Create `provider_test.go`: the
  shared `fixture` (fake, log capture, budgets shrunk), `assertProviderError`,
  `TestProvider_cancelledContextReturnsPromptly` (each method blocked on
  its first seam call), `TestProvider_everyErrorIsWrappedWithConfiguredName`,
  `TestLogging_v0SilentV1OutcomeV2Detail` (V(1) lines `proxmox vm ensured
  running`, `proxmox vm fetched`, `proxmox vm deleted`, `proxmox vms
  listed`, `proxmox node checked`, every line with `provider`, instance
  lines with `vmid` and `name`; V(2) has `cloudInitBytes` and never the
  text), `TestTracing_httpSpansUnderOperationSpan` (a real `httpAPI`
  against an `httptest` server playing the fixtures for a Get, wrapped in
  `provider.WithTracing`, with a span recorder: every HTTP span is a child
  of `provider.get` and no attribute carries the token). Create
  `proxmox_integration_test.go` (`//go:build integration`):
  `loadIntegrationEnv` with the `PROXMOX_TEST_*` variables, `providerConfig()`,
  the `integrationLog` pattern from kvm2; `TestProxmox_lifecycle` asserting
  in order: `New` returns within 100 ms without a request; `Check` Ready
  with `pveVersion`, `isoStorageAvailGiB` and `templates`; `Create` returns
  `vms/<vmid>/<name>` with the vmid in range; a second `Create` returns the
  same ID and the resources count is unchanged; `Get` reaches Running with
  an IPv4 within the budget, UID and CreatedAt as written; `ListByTag`
  contains it; a TCP connect to the guest's port is logged only; `Delete`
  nil; `Get` NotFound; `Delete` again nil; the ISO storage has no
  `<name>-seed.iso` and resources has no VM with the name; an HTTP span is
  a child of `provider.create` and no attribute carries the cloud-init
  text; `PROXMOX_TEST_KEEP` skips the teardown and prints the `qm destroy`
  and ISO delete commands. `TestProxmoxCheck_realHost` creates nothing. Edit
  `Makefile`: `test-integration-proxmox` (`go test -race -tags=integration
  -count=1 -v -timeout 30m -run TestProxmox ./internal/provider/proxmox/`).
  Edit `docs/testing.md`: rows for the package in the suites table and a
  "The proxmox integration test" section. Update spec entry 21's tick list
  with the final test names.
  Verify: `go test -race -count=2 ./internal/provider/proxmox/`;
  `go vet -tags=integration ./... && go vet -tags=e2e ./...`;
  `make -n test-integration-proxmox`; every §11 item has a test name in
  spec entry 21 that exists.
  Done when: the package covers every §11 item by name, the integration
  test compiles, the target exists. Commit: `proxmox: contract and
  observability tests, integration test, testing docs`.

- **Phase 9 (implementer): deployment and documentation.** Read first:
  contract §12 items 11 to 13; spec entries 22, 23, 24;
  `config/manager/manager.yaml`, `config/manager/providers_config.yaml`,
  `config/samples/{providers-config.yaml,proxy_kvm2.yaml,kustomization.yaml}`;
  `Makefile` `kvm-ssh-secret`; `docs/kvm2-provider.md` headings;
  `docs/providers-existing.md`; `docs/architecture.md` lines mentioning
  kvm2; `README.md`; `docs/providers.md`; `CLAUDE.md` Gotchas. Edit
  `config/manager/manager.yaml`: an optional Secret mount `proxmox-token` at
  `/etc/proxy-operator/proxmox` (key `token`). Edit `Makefile`:
  `PROXMOX_TOKEN_FILE ?= $(HOME)/.config/proxmox-token`,
  `proxmox-token-secret` creating `proxmox-token` from `--from-file=token=`.
  Edit both providers-config files: a commented `proxmox-home` entry (`url:
  https://192.168.2.8:8006`, `tokenFile`, `caFile` or
  `insecureSkipTLSVerify`, `nodes`, `isoStorage`, `storage`, `bridge`,
  `vmidRange`, `templates`, `machineTypes`). Create
  `config/samples/proxy_proxmox.yaml` (provider `proxmox-home`, zone the
  node, machine type `small`, image the template key, the Squid cloud-init
  from `proxy_kvm2.yaml`) and list it in `config/samples/kustomization.yaml`.
  Create `docs/proxmox-provider.md` with sections: Proxmox prerequisites
  (the template commands from this plan, no cloudinit drive on ide2, the
  guest agent or outbound access, the bridge, the privilege list from F17,
  reserving the VMID range), Cluster-side setup (the Secret, the config
  entry), Config reference (spec entry 15's table), Placement (Region
  ignored, catalogue-edit warning), What lands on Proxmox (VMID scheme,
  description mark, tag, seed ISO, config keys, `qm unlock` for a stuck
  clone), Logging, Tracing, Timeouts, Health check, Error classes, Known
  gaps (spec entry 24), Testing. Edit `docs/providers-existing.md`: a fifth
  column in the three tables and a proxmox section mirroring kvm2's. Edit
  `docs/architecture.md`: components row, a proxmox block in §5, the span
  note in §10, a §11 row (and the "All four providers" count), Decisions
  bullets. Edit `README.md`, `docs/providers.md`, `CLAUDE.md` Gotchas
  (`make test-integration-proxmox`).
  Verify: `bin/kustomize build config/default >/dev/null && bin/kustomize build config/samples >/dev/null`;
  a test `TestLoadConfig_proxmoxSampleEntry` loads the uncommented block
  from `config/samples/providers-config.yaml` through `provider.LoadConfig`;
  a link sweep over the edited docs finds no missing target;
  `grep -c '—' docs/proxmox-provider.md` is 0.
  Done when: manifests build, the sample loads, docs are linked and
  link-clean. Commit: `proxmox: deployment files, sample, user guide and
  doc rows`.

- **Phase 10 (orchestrator with the user): review, real-host validation,
  changelog, MR.** Read first: `docs/plans-executions/2026-09-17-1515-kvm2-review-fixes.md`;
  `.claude/agents/go-operator-reviewer.md`; the latest report under
  `docs/reviews/` for the shape; the spec; this plan's Risks. Do: run
  `go-operator-reviewer` on the branch with the spec and the contract as
  inputs, then `/code-review high`, `make lint` (if the local toolchain
  panics in staticcheck as it did during kvm2, record it and run
  `bin/golangci-lint run ./internal/provider/proxmox/...` with staticcheck
  disabled), `make test`. Fix accepted findings with regression tests in
  one commit; record rejected ones under `docs/reviews/<date>-proxmox-provider-review/`.
  Hand the user the host checklist (template ready, no leftover VMs in the
  range, ISO storage has space, the token's privileges per F17) and the
  `PROXMOX_TEST_*` export lines; with the go-ahead run
  `make test-integration-proxmox` and record the clone duration and time to
  IP. Optionally the in-cluster pass on kind: build and load the image,
  `make proxmox-token-secret`, a scratch ConfigMap with the `proxmox-home`
  entry, apply `proxy_proxmox.yaml`, wait for `Ready`, `kubectl get
  providerhealth proxmox-home -o yaml`, a `curl -x` through the proxy,
  delete and confirm the VM and ISO are gone; scratch overrides are not
  committed. After the user confirms: CHANGELOG entry
  (`date "+%Y-%m-%d %H:%M %Z"`), push with `-u` using the token helper from
  memory (`~/.config/gitea-token`), open the MR with the Gitea API
  (`POST /api/v1/repos/kacerr/egress-proxies-operator/pulls` with `head:
  feat/proxmox-provider`, `base: main`, the body ending with the
  attribution line; `pr.json` in the scratchpad, never committed). Never
  merge.
  Verify: the integration run's ordered assertions pass on 192.168.2.8;
  `make test` green; the review report lists every finding as accepted or
  rejected.
  Done when: the MR URL is printed and the execution log's last box is
  ticked. Commit: `proxmox: review fixes, real-host validation record,
  changelog`.

## Verification

- Per phase: the `Verify:` clause, plus `go build ./... && go test -short ./...`
  before every commit and `git diff --stat` reviewed for files outside the
  phase's list.
- Contract mapping: after Phase 8 every rule of contract §2 and §8 has a
  named test in the package, and spec entry 21 maps the §11 list 1:1 to
  existing test names.
- Fixture discipline: every message `classify` or a matcher helper compares
  against is loaded from `testdata/` in `TestClassify_table` or
  `TestAPIError_matchers`; a string literal in `errors.go` without a fixture
  is a review finding.
- Secrets: `TestClient_tokenNeverInErrorsOrLogs` and
  `TestCreate_cloudInitSentinelNeverLoggedOrInErrors` run at V(2), and the
  integration test asserts no span attribute carries the cloud-init text.
- End to end: the integration run (Phase 8's assertions) on 192.168.2.8 and
  the optional kind pass in Phase 10.

## Risks

- **Error text location and exact wording (F1 to F6)** decide the whole
  NotFound path; if messages vary by storage type or PVE minor version, the
  matchers get one fixture per variant and the guide names the tested
  version. This is not hypothetical: 9.2 rewrote the authentication-failure
  message (F22), so fixtures captured from an 8.x host would already be
  wrong. Every matched string comes from 192.168.2.8 at 9.2.20, and the
  guide says so.
- **Clone cleanup (F15).** A failed or interrupted clone may leave a VM with
  `lock=clone` that never clears; Create then waits out `budgets.clone` and
  returns Transient every time. The guide documents `qm unlock`; the
  reconciler keeps retrying rather than latching.
- **`SDN.Use` (F17).** PVE 9 refuses `net0` writes to non-root users without
  it; a missing privilege is a 403 Permanent that latches the Proxy until
  its spec changes. The guide's privilege list is the mitigation.
- **The ide2 seed and a template with a cloudinit drive** both claim
  cloud-init; the Proxy's user-data would be ignored. Documented
  prerequisite; the Checker cannot see it.
- **VMID range shared with humans.** A human VM inside the range costs a
  probe; a human VM carrying our name is Permanent. The guide says to
  reserve the range.
- **Node-local ISO storage after a migration** leaves the seed on the
  original node; Delete tolerates its absence and the ISO is a documented
  blind spot. Shared ISO storage avoids it.
- **Blocking Create** holds a reconcile worker for up to the clone budget;
  the guide sets expectations.
- **The 30s request timeout on the seed upload** is fine for tens of KiB; a
  multi-megabyte `vendorData` would trip it. Documented.
- **Two operator installs** on one Proxmox cluster delete each other's VMs
  (contract §13); documented in entry 24.
- **kvm2 edits in Phase 3** touch a merged provider; the moved tests are the
  guard, and `go vet -tags=integration` keeps its integration test compiling.