Wire the KVM provider: main.go, kvm-ssh Secret mount, samples, docs, CHANGELOG

Registers "kvm" in the provider builtins, mounts an optional kvm-ssh
Secret at /etc/proxy-operator/kvm, adds a kvm-home providers-config
sample and a proxy_kvm.yaml Proxy sample, documents the provider in
docs/kvm-provider.md (prerequisites, config reference, image cache and
its do-not-replace-in-use rule, known gaps), and updates the README
and architecture provider sections and decision log.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-09-10 19:13:51 +00:00
co-authored by Claude Fable 5.1
parent b3984ca69d
commit 791988b5a0
11 changed files with 448 additions and 6 deletions
+44
View File
@@ -9,6 +9,7 @@
| Provider contract | `internal/provider` | library | — | `Provider` interface, error taxonomy, deterministic naming, config, metrics decorator |
| kubernetes provider | `internal/provider/kubernetes` | library | — | real Squid pods in this cluster (local dev/CI) |
| gcp provider | `internal/provider/gcp` | library | — | Compute Engine VMs, four API calls, fire-and-forget ops |
| kvm provider | `internal/provider/kvm` | library | — | VMs on libvirt hosts over SSH-forwarded sockets; image cache, overlay disks, NoCloud seeds |
| Health engine | `internal/health` | Runnable | yes | through-the-proxy probes, thresholds, transition events |
| Lease store | `internal/lease` | Runnable (expiry sweep) | no | in-memory leases + cooldowns, single mutex |
| Discovery API | `internal/discovery` | Runnable | no | HTTP list/lease/release/report on `:8090` |
@@ -147,6 +148,25 @@ prov.ListByTag ─► AggregatedList(label filter, ─┘ prov
ReturnPartialSuccess: true)
providerID = zones/<zone>/instances/<name> — zone-qualified, so Get/Delete
stay correct even mid-replacement after a zone edit
kvm provider (internal/provider/kvm/) — libvirt RPC over an SSH-forwarded unix socket, per host
prov.Create ──► DomainLookupByName: exists → start if needed, done ─┐ every step tolerates
else: ensure base image in pool (download on first use) │ the previous attempt
overlay volume (exists = reuse) │ having got that far;
seed ISO (always rebuilt — a cut-off upload must │ the seed is the one
never be trusted) │ thing redone
DomainDefineXML + DomainCreate ─┘
prov.Get ──► DomainGetState + DomainGetMetadata + DomainInterfaceAddresses
(ARP, then guest agent; first routable IPv4)
running without IP = Provisioning; nostate = Provisioning
prov.Delete ──► destroy (not running = fine) → undefine → delete overlay + seed by name
prov.ListByTag ─► ConnectListAllDomains on every host, keep domains whose
<metadata> carries managed=true; one host down fails the sweep,
one unreadable domain is skipped
providerID = hosts/<host>/domains/<name>; GC identity lives in the domain's
<metadata> element (libvirt has no labels). All RPCs run under a context-
bounded call: go-libvirt never times out on its own, so a cancelled reconcile
closes the SSH connection to unblock it.
```
The reconciler never watches provider-side resources (Pods or GCP VMs).
@@ -415,6 +435,30 @@ they read as choices, not accidents. Chronological by build step.
- **Unknown GCP instance statuses map to `Stopped`:** the reconciler's
answer to Stopped is delete-and-recreate, the always-safe move for
cattle when the API grows a new state.
- **KVM hosts are reached over SSH with the libvirt socket forwarded,**
not libvirtd's TCP/TLS listener: nothing to configure or expose on the
hypervisors beyond sshd, pure Go on the operator side (no cgo, no
libvirt libraries in the image), and the hosts list can grow by a
config edit because the providerID carries the host name.
- **KVM base images are fetched by the operator into the host's pool**
through libvirt's upload stream, rather than expected to be pre-placed:
the operator then needs no filesystem access on the host and a new
host is usable with zero manual staging. The cost is a synchronous
first-time download inside Create.
- **The KVM seed ISO is rebuilt on every Create attempt** before the
domain exists, instead of "exists → skip": a reconcile cancelled mid
upload leaves a truncated seed, and the compensating delete cannot run
because the connection was already closed to unblock the RPC. Skipping
it would boot a VM with no cloud-init and no retry would ever repair it.
- **KVM `nostate` maps to `Provisioning`, not `Stopped`:** unlike GCP's
unknown statuses, `nostate` means libvirt could not determine the
state at all; answering that with delete-and-recreate is too drastic,
so the next poll decides.
- **A KVM domain GC cannot read is skipped, a host it cannot reach fails
the sweep.** GC never deletes what it does not see, so leaving one
domain out is safe; one corrupt domain blocking GC for every other one
is not. A whole host missing from the view would hide many, so that
still aborts.
- **Kubernetes 403s classify as `ErrPermanent`** even though quota
exhaustion also surfaces as 403 (indistinguishable from RBAC denial in
`apierrors`): not hammering an API server that may never allow the