Fill the provider blueprint's gaps after a full sweep

A pass over the API types, spec hash, health and discovery call sites,
manager manifests, samples, README, provider docs, plans and the PR review
found rules the blueprint did not state: the IP must be reachable on the
proxy port by the health engine and clients, providers must be safe for
concurrent use, the manager's read-only/128Mi runtime envelope, catalogue
edits bypass the spec hash, Delete is re-called while in progress, and
provider names must be DNS-1123. Adds matrix rows, a deployment checklist
item, and the still-open review findings and broken samples as gaps.

README's "Adding a provider" restated an outdated contract; it now points
to docs/providers.md, and the providers table gains the kvm row.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-16 09:18:40 +00:00
co-authored by Claude Opus 5
parent 01c27b5f35
commit 0bb699bf09
3 changed files with 180 additions and 46 deletions
+9 -38
View File
@@ -104,7 +104,8 @@ make deploy IMG=<registry>/egress-proxies-operator:dev
- Provider config comes from the `providers-config` ConfigMap
([config/manager/providers_config.yaml](config/manager/providers_config.yaml));
the default ships only the kubernetes provider.
the default ships the kubernetes provider plus a `kvm-home` entry that needs
the `kvm-ssh` Secret (without it the manager starts and warns).
- The discovery API is exposed by the
`controller-manager-discovery-service` Service on port 8090.
- Auth: create the token Secret, or the API serves **unauthenticated**
@@ -185,46 +186,16 @@ can be two differently-configured instances of the same `gcp` type (see
| --- | --- | --- | --- |
| `kubernetes` | A real Squid pod (`ubuntu/squid:6.6-24.04_edge` by default) in the same cluster the operator runs in | `image` (optional) | Needs no cloud account — local dev, CI, and the kind quickstart. Pods share the cluster's egress IP, so it exercises the full lifecycle but not distinct egress paths. |
| `gcp` | A Compute Engine VM with an ephemeral external IP | `project` (required), `network`, `networkTag`, `diskSizeGb` | The real egress fleet. Deliberately uses only four API calls (Insert / Get / Delete / AggregatedList), all fire-and-forget: `Create` returns as soon as the operation is submitted and the reconciler discovers progress by polling `Get`. Auth is Application Default Credentials — workload identity in-cluster, `gcloud` ADC locally; no key-file plumbing. |
| `kvm` | A libvirt VM on one of your own hypervisors, reached over SSH | `sshKeyFile`, `knownHostsFile` (unless `insecureIgnoreHostKey`), `machineTypes`, `hosts` (required); `images`, `diskSizeGb`, `vendorData`, `installGuestAgent` | Home-lab or on-prem egress. The base image is downloaded into each host's pool on first use; `Create` waits for the VM to start. See [docs/kvm-provider.md](docs/kvm-provider.md). |
### Adding a provider
A new backend (Hetzner, AWS, ...) is four pieces; the contract lives in
[internal/provider/provider.go](internal/provider/provider.go):
1. **Implement the 4-method `Provider` interface** in a new
`internal/provider/<type>/` package:
- `Create` submits and returns — it never blocks until the VM runs, and
must be idempotent keyed on `req.Name` (a deterministic name derived
from the Proxy's UID), so a repeat call after a crash finds the
existing instance instead of duplicating it.
- `Get` returns `provider.ErrNotFound` as a *normal* outcome — the
reconciler branches on it for replacement and adoption, so don't
treat it as exceptional.
- `Delete` is idempotent: deleting an already-gone instance is not an
error.
- `ListByTag` returns every instance the operator ever tagged, for
orphan GC.
2. **Tag every created resource** with `LabelManaged=true` and
`LabelUID=<Proxy UID>`, and report `CreatedAt` — orphan GC relies on
all three to find owned resources and skip in-flight creates.
3. **Classify every returned error** with `provider.Wrap` into the
four-sentinel taxonomy in
[internal/provider/errors.go](internal/provider/errors.go)
(`ErrNotFound` / `ErrQuotaExceeded` / `ErrTransient` / `ErrPermanent`)
— the reconciler decides retry, slow backoff, or latching `Failed`
purely from that classification, never from provider-specific types.
4. **Wire it up**: add a type-specific config block in
[internal/provider/config.go](internal/provider/config.go), and
register the constructor in the builtins map in
[cmd/main.go](cmd/main.go) (`"<type>": <pkg>.New`). The registry
([internal/provider/registry](internal/provider/registry/registry.go))
handles named instances, and the metrics wrapper is applied
automatically.
Test against a fake API seam rather than the real cloud — see the
`instancesAPI` seam in
[internal/provider/gcp/gcp.go](internal/provider/gcp/gcp.go) for the
pattern.
The contract, the rules each caller depends on, a feature matrix of the
existing providers and a step-by-step checklist are in
[docs/providers.md](docs/providers.md). Start there: several rules (a
self-contained `providerID`, `Get` returning `ErrNotFound` only when the
instance is really gone, `CreatedAt` read back for orphan GC) are enforced by
callers, not by the interface.
## Tracing