Wire the KVM provider: main.go, kvm-ssh Secret mount, samples, docs, CHANGELOG
Registers "kvm" in the provider builtins, mounts an optional kvm-ssh Secret at /etc/proxy-operator/kvm, adds a kvm-home providers-config sample and a proxy_kvm.yaml Proxy sample, documents the provider in docs/kvm-provider.md (prerequisites, config reference, image cache and its do-not-replace-in-use rule, known gaps), and updates the README and architecture provider sections and decision log. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -9,6 +9,7 @@
|
||||
| Provider contract | `internal/provider` | library | — | `Provider` interface, error taxonomy, deterministic naming, config, metrics decorator |
|
||||
| kubernetes provider | `internal/provider/kubernetes` | library | — | real Squid pods in this cluster (local dev/CI) |
|
||||
| gcp provider | `internal/provider/gcp` | library | — | Compute Engine VMs, four API calls, fire-and-forget ops |
|
||||
| kvm provider | `internal/provider/kvm` | library | — | VMs on libvirt hosts over SSH-forwarded sockets; image cache, overlay disks, NoCloud seeds |
|
||||
| Health engine | `internal/health` | Runnable | yes | through-the-proxy probes, thresholds, transition events |
|
||||
| Lease store | `internal/lease` | Runnable (expiry sweep) | no | in-memory leases + cooldowns, single mutex |
|
||||
| Discovery API | `internal/discovery` | Runnable | no | HTTP list/lease/release/report on `:8090` |
|
||||
@@ -147,6 +148,25 @@ prov.ListByTag ─► AggregatedList(label filter, ─┘ prov
|
||||
ReturnPartialSuccess: true)
|
||||
providerID = zones/<zone>/instances/<name> — zone-qualified, so Get/Delete
|
||||
stay correct even mid-replacement after a zone edit
|
||||
|
||||
kvm provider (internal/provider/kvm/) — libvirt RPC over an SSH-forwarded unix socket, per host
|
||||
prov.Create ──► DomainLookupByName: exists → start if needed, done ─┐ every step tolerates
|
||||
else: ensure base image in pool (download on first use) │ the previous attempt
|
||||
overlay volume (exists = reuse) │ having got that far;
|
||||
seed ISO (always rebuilt — a cut-off upload must │ the seed is the one
|
||||
never be trusted) │ thing redone
|
||||
DomainDefineXML + DomainCreate ─┘
|
||||
prov.Get ──► DomainGetState + DomainGetMetadata + DomainInterfaceAddresses
|
||||
(ARP, then guest agent; first routable IPv4)
|
||||
running without IP = Provisioning; nostate = Provisioning
|
||||
prov.Delete ──► destroy (not running = fine) → undefine → delete overlay + seed by name
|
||||
prov.ListByTag ─► ConnectListAllDomains on every host, keep domains whose
|
||||
<metadata> carries managed=true; one host down fails the sweep,
|
||||
one unreadable domain is skipped
|
||||
providerID = hosts/<host>/domains/<name>; GC identity lives in the domain's
|
||||
<metadata> element (libvirt has no labels). All RPCs run under a context-
|
||||
bounded call: go-libvirt never times out on its own, so a cancelled reconcile
|
||||
closes the SSH connection to unblock it.
|
||||
```
|
||||
|
||||
The reconciler never watches provider-side resources (Pods or GCP VMs).
|
||||
@@ -415,6 +435,30 @@ they read as choices, not accidents. Chronological by build step.
|
||||
- **Unknown GCP instance statuses map to `Stopped`:** the reconciler's
|
||||
answer to Stopped is delete-and-recreate, the always-safe move for
|
||||
cattle when the API grows a new state.
|
||||
- **KVM hosts are reached over SSH with the libvirt socket forwarded,**
|
||||
not libvirtd's TCP/TLS listener: nothing to configure or expose on the
|
||||
hypervisors beyond sshd, pure Go on the operator side (no cgo, no
|
||||
libvirt libraries in the image), and the hosts list can grow by a
|
||||
config edit because the providerID carries the host name.
|
||||
- **KVM base images are fetched by the operator into the host's pool**
|
||||
through libvirt's upload stream, rather than expected to be pre-placed:
|
||||
the operator then needs no filesystem access on the host and a new
|
||||
host is usable with zero manual staging. The cost is a synchronous
|
||||
first-time download inside Create.
|
||||
- **The KVM seed ISO is rebuilt on every Create attempt** before the
|
||||
domain exists, instead of "exists → skip": a reconcile cancelled mid
|
||||
upload leaves a truncated seed, and the compensating delete cannot run
|
||||
because the connection was already closed to unblock the RPC. Skipping
|
||||
it would boot a VM with no cloud-init and no retry would ever repair it.
|
||||
- **KVM `nostate` maps to `Provisioning`, not `Stopped`:** unlike GCP's
|
||||
unknown statuses, `nostate` means libvirt could not determine the
|
||||
state at all; answering that with delete-and-recreate is too drastic,
|
||||
so the next poll decides.
|
||||
- **A KVM domain GC cannot read is skipped, a host it cannot reach fails
|
||||
the sweep.** GC never deletes what it does not see, so leaving one
|
||||
domain out is safe; one corrupt domain blocking GC for every other one
|
||||
is not. A whole host missing from the view would hide many, so that
|
||||
still aborts.
|
||||
- **Kubernetes 403s classify as `ErrPermanent`** even though quota
|
||||
exhaustion also surfaces as 403 (indistinguishable from RBAC denial in
|
||||
`apierrors`): not hammering an API server that may never allow the
|
||||
|
||||
Reference in New Issue
Block a user