diff --git a/docs/plans/2026-09-18-1502-proxmox-provider.md b/docs/plans/2026-09-18-1502-proxmox-provider.md index 9dfed1c..d7c42bd 100644 --- a/docs/plans/2026-09-18-1502-proxmox-provider.md +++ b/docs/plans/2026-09-18-1502-proxmox-provider.md @@ -7,10 +7,13 @@ The provider contract (`docs/provider-contract.md`), the spec template (`docs/provider-spec-template.md`) and the kvm2 build proved the workflow: spec first, then implementation against the contract, one commit per step. -This plan adds a third VM-style backend, `proxmox`, on the Proxmox VE 8.x +This plan adds a third VM-style backend, `proxmox`, on the Proxmox VE 9.x REST API at `https://:8006/api2/json`. The user has a real host at -192.168.2.8 and prepares the API token and a cloud-init-capable template -before Phase 0 (checklist below). +192.168.2.8 running **9.2.20** (Debian 13 Trixie, QEMU 11) and prepares the +API token and a cloud-init-capable template before Phase 0 (checklist +below). Everything below is written against 9.2; where 8.x differed the +difference is called out, because the appendix and the guide must name the +version they were proven on. The implementation runs in sessions whose context gets compacted, so every phase is self-contained: it names what to read first, what it creates or @@ -32,13 +35,14 @@ committed together with the code. Follow `CLAUDE.md` for the log format. ## Before Phase 0: what the user prepares on 192.168.2.8 -Run as root on the Proxmox node. Drop any privilege `pveum` rejects on the -installed version (`VM.GuestAgent.Audit` exists from 8.2; older releases use -`VM.Monitor` for the agent read) and note it for F17. +Run as root on the Proxmox node. **`VM.Monitor` was removed in PVE 9.0**, so +the agent read rests on `VM.GuestAgent.Audit` alone and the role below must +not mention `VM.Monitor`. Drop any privilege `pveum` still rejects and note +it for F17. ```bash pveum user add proxy-operator@pve --comment "egress-proxies-operator" -pveum role add ProxyOperator -privs "VM.Allocate VM.Clone VM.Audit VM.Config.CDROM VM.Config.CPU VM.Config.Memory VM.Config.Disk VM.Config.Network VM.Config.Options VM.PowerMgmt VM.Monitor VM.GuestAgent.Audit Datastore.Audit Datastore.AllocateSpace Datastore.AllocateTemplate Sys.Audit SDN.Use" +pveum role add ProxyOperator -privs "VM.Allocate VM.Clone VM.Audit VM.Config.CDROM VM.Config.CPU VM.Config.Memory VM.Config.Disk VM.Config.Network VM.Config.Options VM.PowerMgmt VM.GuestAgent.Audit Datastore.Audit Datastore.AllocateSpace Datastore.AllocateTemplate Sys.Audit SDN.Use" pveum aclmod / -user proxy-operator@pve -role ProxyOperator pveum user token add proxy-operator@pve operator --privsep 0 # The secret is printed once. Save it on the workstation as one line: @@ -71,7 +75,9 @@ else Phase 0 reads from the API. ## Design decisions made with the user (not re-opened) -- **Backend and client.** Proxmox VE 8.x REST API, JSON, HTTP/1.1 only. +- **Backend and client.** Proxmox VE 9.x REST API, JSON, HTTP/1.1 only + (pveproxy offers no HTTP/2 on any release; only Proxmox Backup Server + does). Thin client on `net/http` (`client-kind: thin-client`), no SDK, no new dependency. Auth header `Authorization: PVEAPIToken=@!=`, read from a @@ -110,9 +116,13 @@ else Phase 0 reads from the API. mark window). Content, one per line: `proxy-operator-managed=true`, `proxy-operator-uid=`, `proxy-operator-created-at=`. `created-at: written-at-create`. The config step after the clone also sets - tag `proxy-operator` for UI filtering only. ListByTag: one + tag `proxy_operator` for UI filtering only. **Underscore, not hyphen:** + current PVE tag validation accepts only lowercase letters, digits and + underscore, and a hyphen is rejected; the underscore form is valid under + both the strict and the permissive reading, so it is used regardless of + what F9 finds. ListByTag: one `GET /cluster/resources?type=vm`; candidates are entries with tag - `proxy-operator` or name prefix `proxy-`; a `GET /nodes/{node}/qemu/{vmid}/config` + `proxy_operator` or name prefix `proxy-`; a `GET /nodes/{node}/qemu/{vmid}/config` per candidate for the description; unmarked skipped; one unreadable VM skipped with a V(1) line unless ctx ended; the resources call failing fails the whole list (`list-partial: fail-whole`). @@ -229,8 +239,11 @@ else Phase 0 reads from the API. `status` values (`running`, `stopped`, `unknown`), `type` (`qemu`, `lxc`), `lock`, `tags` (separator), `template`; whether `lock` appears there during a clone. -- F9. Tag rules: allowed characters, case, length, `tags` separator on - `PUT config` and in `/cluster/resources`. +- F9. Tag rules on 9.2: confirm a hyphen is rejected and `proxy_operator` + accepted, plus case folding, length, and the `tags` separator on + `PUT config` and in `/cluster/resources`. The underscore form stays either + way; this fact only decides what the guide tells users about their own + tags. - F10. `description` on `POST .../clone`: accepted, survives verbatim (newlines), max length; whether `name` accepts the 22-character `proxy-` form. @@ -239,7 +252,9 @@ else Phase 0 reads from the API. - F12. Upload endpoint: field names (`content`, `filename`, file part name `filename`), the filename rule (`.iso` suffix, no slashes), the response (UPID), the privilege it needs, and whether re-uploading an existing name - is refused (then delete-then-upload) or overwrites. + is refused (then delete-then-upload) or overwrites. On 9.x `content` + accepts `iso`, `vztmpl` or `import` and still refuses `snippets`, which is + why the seed travels as an ISO; confirm `iso` is what an `.iso` needs. - F13. `DELETE /nodes/{node}/storage/{storage}/content/{volid}`: response shape (UPID or null) and the message for a missing volume. - F14. `status/current` on a stopped VM: is `qmpstatus` present; on a paused @@ -251,16 +266,29 @@ else Phase 0 reads from the API. (`OK` or the error string), and how long a full clone of the template takes. - F17. Privileges: which of the prepared list the token really needs, in - particular `SDN.Use` on `/sdn/zones/localnetwork/` for the `net0` - write and `VM.GuestAgent.Audit` or `VM.Monitor` for the agent read. -- F18. HTTP/2 is not offered (`curl --http2 -I` falls back or fails), so the - client disables it. + particular the exact ACL path that satisfies `SDN.Use` for the `net0` + write (the 9.x docs describe it as "access SDN vnets and local network + bridges" without spelling out the path for a plain bridge), and + `VM.GuestAgent.Audit` for the agent read. Also whether the config step + needs `VM.Config.Cloudinit` even though only `ide2` is written. +- F18. HTTP/2 is not offered by pveproxy on any release (documented), so the + client disables it; confirm with `curl --http2 -I` that the connection + falls back to HTTP/1.1. - F19. The template's disk key (`scsi0` or `virtio0`), its `boot` line, that ide2 is free and no `cloudinit` drive exists. - F20. `resize` on a stopped VM: request shape, the message when the target is smaller or equal, and that it is synchronous. - F21. Agent response shape: `result[].name`, `result[]["ip-addresses"][]` with `ip-address`, `ip-address-type`. +- F22. The 401 body and reason phrase on 9.2. PVE 9.2 unified the + authentication-failure message so a failed login no longer leaks whether + the user exists, so any 401 text remembered from 8.x is wrong. Capture it + fresh into `err_unauthorized.json`. +- F23. Where the upload endpoint stages the request body, given that PVE 9 + moved the node's `/tmp` to a tmpfs capped at half of RAM and cleans `/tmp` + and `/var/tmp` periodically. A seed ISO is tens of KiB so this cannot + bite in practice, but the observation belongs in the guide next to the + size limit. ## Proxmox endpoint map for the seam @@ -602,7 +630,7 @@ idioms (`fixture`, `assertProviderError`, `logCapture`). calling `Clone` with `vmidFor(req.UID, rng, attempt)`, `isAlreadyExists` moving to the next attempt, exhaustion Transient; `waitTask` with `budgets.clone`; `uploadSeed`; `SetVMConfig` with `cores`, `memory`, - `agent=1`, `tags=proxy-operator`, `ide2=,media=cdrom`, and `net0` + `agent=1`, `tags=proxy_operator`, `ide2=,media=cdrom`, and `net0` rewritten from the cloned config with `bridge=` only when configured (MAC kept); `bootDisk(cfg)` and `Resize` when `diskGiB` is above the current `size=`; `Start` and `waitTask` with `budgets.start`; @@ -817,12 +845,15 @@ idioms (`fixture`, `assertProviderError`, `logCapture`). - **Error text location and exact wording (F1 to F6)** decide the whole NotFound path; if messages vary by storage type or PVE minor version, the matchers get one fixture per variant and the guide names the tested - version. + version. This is not hypothetical: 9.2 rewrote the authentication-failure + message (F22), so fixtures captured from an 8.x host would already be + wrong. Every matched string comes from 192.168.2.8 at 9.2.20, and the + guide says so. - **Clone cleanup (F15).** A failed or interrupted clone may leave a VM with `lock=clone` that never clears; Create then waits out `budgets.clone` and returns Transient every time. The guide documents `qm unlock`; the reconciler keeps retrying rather than latching. -- **`SDN.Use` (F17).** PVE 8 refuses `net0` writes to non-root users without +- **`SDN.Use` (F17).** PVE 9 refuses `net0` writes to non-root users without it; a missing privilege is a 403 Permanent that latches the Proxy until its spec changes. The guide's privilege list is the mitigation. - **The ide2 seed and a template with a cloudinit drive** both claim