kacerr/egress-proxies-operator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

proxmox: a Proxmox VE provider built from the provider contract #9

Merged
kacerr merged 13 commits from feat/proxmox-provider into main 2026-09-24 13:12:45 +02:00
Conversation 0 Commits 13 Files Changed 131
+51 -20
1 changed files with 51 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 2167819157 - Show all commits
docs/plans/2026-09-18-1502-proxmox-provider.md
+51 -20
View File
@@ -7,10 +7,13 @@
The provider contract (`docs/provider-contract.md`), the spec template
(`docs/provider-spec-template.md`) and the kvm2 build proved the workflow:
spec first, then implementation against the contract, one commit per step.
This plan adds a third VM-style backend, `proxmox`, on the Proxmox VE 8.x
This plan adds a third VM-style backend, `proxmox`, on the Proxmox VE 9.x
REST API at `https://<host>:8006/api2/json`. The user has a real host at
192.168.2.8 and prepares the API token and a cloud-init-capable template
before Phase 0 (checklist below).
192.168.2.8 running **9.2.20** (Debian 13 Trixie, QEMU 11) and prepares the
API token and a cloud-init-capable template before Phase 0 (checklist
below). Everything below is written against 9.2; where 8.x differed the
difference is called out, because the appendix and the guide must name the
version they were proven on.
The implementation runs in sessions whose context gets compacted, so every
phase is self-contained: it names what to read first, what it creates or
@@ -32,13 +35,14 @@ committed together with the code. Follow `CLAUDE.md` for the log format.
## Before Phase 0: what the user prepares on 192.168.2.8
Run as root on the Proxmox node. Drop any privilege `pveum` rejects on the
installed version (`VM.GuestAgent.Audit` exists from 8.2; older releases use
`VM.Monitor` for the agent read) and note it for F17.
Run as root on the Proxmox node. **`VM.Monitor` was removed in PVE 9.0**, so
the agent read rests on `VM.GuestAgent.Audit` alone and the role below must
not mention `VM.Monitor`. Drop any privilege `pveum` still rejects and note
it for F17.
```bash
pveum user add proxy-operator@pve --comment "egress-proxies-operator"
pveum role add ProxyOperator -privs "VM.Allocate VM.Clone VM.Audit VM.Config.CDROM VM.Config.CPU VM.Config.Memory VM.Config.Disk VM.Config.Network VM.Config.Options VM.PowerMgmt VM.Monitor VM.GuestAgent.Audit Datastore.Audit Datastore.AllocateSpace Datastore.AllocateTemplate Sys.Audit SDN.Use"
pveum role add ProxyOperator -privs "VM.Allocate VM.Clone VM.Audit VM.Config.CDROM VM.Config.CPU VM.Config.Memory VM.Config.Disk VM.Config.Network VM.Config.Options VM.PowerMgmt VM.GuestAgent.Audit Datastore.Audit Datastore.AllocateSpace Datastore.AllocateTemplate Sys.Audit SDN.Use"
pveum aclmod / -user proxy-operator@pve -role ProxyOperator
pveum user token add proxy-operator@pve operator --privsep 0
# The secret is printed once. Save it on the workstation as one line:
@@ -71,7 +75,9 @@ else Phase 0 reads from the API.
## Design decisions made with the user (not re-opened)
- **Backend and client.** Proxmox VE 8.x REST API, JSON, HTTP/1.1 only.
- **Backend and client.** Proxmox VE 9.x REST API, JSON, HTTP/1.1 only
(pveproxy offers no HTTP/2 on any release; only Proxmox Backup Server
does).
Thin client on `net/http` (`client-kind: thin-client`), no SDK, no new
dependency. Auth header
`Authorization: PVEAPIToken=<user>@<realm>!<tokenid>=<secret>`, read from a
@@ -110,9 +116,13 @@ else Phase 0 reads from the API.
mark window). Content, one per line: `proxy-operator-managed=true`,
`proxy-operator-uid=<uid>`, `proxy-operator-created-at=<RFC3339>`.
`created-at: written-at-create`. The config step after the clone also sets
tag `proxy-operator` for UI filtering only. ListByTag: one
tag `proxy_operator` for UI filtering only. **Underscore, not hyphen:**
current PVE tag validation accepts only lowercase letters, digits and
underscore, and a hyphen is rejected; the underscore form is valid under
both the strict and the permissive reading, so it is used regardless of
what F9 finds. ListByTag: one
`GET /cluster/resources?type=vm`; candidates are entries with tag
`proxy-operator` or name prefix `proxy-`; a `GET /nodes/{node}/qemu/{vmid}/config`
`proxy_operator` or name prefix `proxy-`; a `GET /nodes/{node}/qemu/{vmid}/config`
per candidate for the description; unmarked skipped; one unreadable VM
skipped with a V(1) line unless ctx ended; the resources call failing fails
the whole list (`list-partial: fail-whole`).
@@ -229,8 +239,11 @@ else Phase 0 reads from the API.
`status` values (`running`, `stopped`, `unknown`), `type` (`qemu`, `lxc`),
`lock`, `tags` (separator), `template`; whether `lock` appears there during
a clone.
- F9. Tag rules: allowed characters, case, length, `tags` separator on
`PUT config` and in `/cluster/resources`.
- F9. Tag rules on 9.2: confirm a hyphen is rejected and `proxy_operator`
accepted, plus case folding, length, and the `tags` separator on
`PUT config` and in `/cluster/resources`. The underscore form stays either
way; this fact only decides what the guide tells users about their own
tags.
- F10. `description` on `POST .../clone`: accepted, survives verbatim
(newlines), max length; whether `name` accepts the 22-character
`proxy-<base32>` form.
@@ -239,7 +252,9 @@ else Phase 0 reads from the API.
- F12. Upload endpoint: field names (`content`, `filename`, file part name
`filename`), the filename rule (`.iso` suffix, no slashes), the response
(UPID), the privilege it needs, and whether re-uploading an existing name
is refused (then delete-then-upload) or overwrites.
is refused (then delete-then-upload) or overwrites. On 9.x `content`
accepts `iso`, `vztmpl` or `import` and still refuses `snippets`, which is
why the seed travels as an ISO; confirm `iso` is what an `.iso` needs.
- F13. `DELETE /nodes/{node}/storage/{storage}/content/{volid}`: response
shape (UPID or null) and the message for a missing volume.
- F14. `status/current` on a stopped VM: is `qmpstatus` present; on a paused
@@ -251,16 +266,29 @@ else Phase 0 reads from the API.
(`OK` or the error string), and how long a full clone of the template
takes.
- F17. Privileges: which of the prepared list the token really needs, in
particular `SDN.Use` on `/sdn/zones/localnetwork/<bridge>` for the `net0`
write and `VM.GuestAgent.Audit` or `VM.Monitor` for the agent read.
- F18. HTTP/2 is not offered (`curl --http2 -I` falls back or fails), so the
client disables it.
particular the exact ACL path that satisfies `SDN.Use` for the `net0`
write (the 9.x docs describe it as "access SDN vnets and local network
bridges" without spelling out the path for a plain bridge), and
`VM.GuestAgent.Audit` for the agent read. Also whether the config step
needs `VM.Config.Cloudinit` even though only `ide2` is written.
- F18. HTTP/2 is not offered by pveproxy on any release (documented), so the
client disables it; confirm with `curl --http2 -I` that the connection
falls back to HTTP/1.1.
- F19. The template's disk key (`scsi0` or `virtio0`), its `boot` line, that
ide2 is free and no `cloudinit` drive exists.
- F20. `resize` on a stopped VM: request shape, the message when the target
is smaller or equal, and that it is synchronous.
- F21. Agent response shape: `result[].name`, `result[]["ip-addresses"][]`
with `ip-address`, `ip-address-type`.
- F22. The 401 body and reason phrase on 9.2. PVE 9.2 unified the
authentication-failure message so a failed login no longer leaks whether
the user exists, so any 401 text remembered from 8.x is wrong. Capture it
fresh into `err_unauthorized.json`.
- F23. Where the upload endpoint stages the request body, given that PVE 9
moved the node's `/tmp` to a tmpfs capped at half of RAM and cleans `/tmp`
and `/var/tmp` periodically. A seed ISO is tens of KiB so this cannot
bite in practice, but the observation belongs in the guide next to the
size limit.
## Proxmox endpoint map for the seam
@@ -602,7 +630,7 @@ idioms (`fixture`, `assertProviderError`, `logCapture`).
calling `Clone` with `vmidFor(req.UID, rng, attempt)`, `isAlreadyExists`
moving to the next attempt, exhaustion Transient; `waitTask` with
`budgets.clone`; `uploadSeed`; `SetVMConfig` with `cores`, `memory`,
`agent=1`, `tags=proxy-operator`, `ide2=<volid>,media=cdrom`, and `net0`
`agent=1`, `tags=proxy_operator`, `ide2=<volid>,media=cdrom`, and `net0`
rewritten from the cloned config with `bridge=<cfg.Bridge>` only when
configured (MAC kept); `bootDisk(cfg)` and `Resize` when `diskGiB` is
above the current `size=`; `Start` and `waitTask` with `budgets.start`;
@@ -817,12 +845,15 @@ idioms (`fixture`, `assertProviderError`, `logCapture`).
- **Error text location and exact wording (F1 to F6)** decide the whole
NotFound path; if messages vary by storage type or PVE minor version, the
matchers get one fixture per variant and the guide names the tested
version.
version. This is not hypothetical: 9.2 rewrote the authentication-failure
message (F22), so fixtures captured from an 8.x host would already be
wrong. Every matched string comes from 192.168.2.8 at 9.2.20, and the
guide says so.
- **Clone cleanup (F15).** A failed or interrupted clone may leave a VM with
`lock=clone` that never clears; Create then waits out `budgets.clone` and
returns Transient every time. The guide documents `qm unlock`; the
reconciler keeps retrying rather than latching.
- **`SDN.Use` (F17).** PVE 8 refuses `net0` writes to non-root users without
- **`SDN.Use` (F17).** PVE 9 refuses `net0` writes to non-root users without
it; a missing privilege is a 403 Permanent that latches the Proxy until
its spec changes. The guide's privilege list is the mitigation.
- **The ide2 seed and a template with a cloudinit drive** both claim