Fill the provider blueprint's gaps after a full sweep
A pass over the API types, spec hash, health and discovery call sites, manager manifests, samples, README, provider docs, plans and the PR review found rules the blueprint did not state: the IP must be reachable on the proxy port by the health engine and clients, providers must be safe for concurrent use, the manager's read-only/128Mi runtime envelope, catalogue edits bypass the spec hash, Delete is re-called while in progress, and provider names must be DNS-1123. Adds matrix rows, a deployment checklist item, and the still-open review findings and broken samples as gaps. README's "Adding a provider" restated an outdated contract; it now points to docs/providers.md, and the providers table gains the kvm row. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -104,7 +104,8 @@ make deploy IMG=<registry>/egress-proxies-operator:dev
|
||||
|
||||
- Provider config comes from the `providers-config` ConfigMap
|
||||
([config/manager/providers_config.yaml](config/manager/providers_config.yaml));
|
||||
the default ships only the kubernetes provider.
|
||||
the default ships the kubernetes provider plus a `kvm-home` entry that needs
|
||||
the `kvm-ssh` Secret (without it the manager starts and warns).
|
||||
- The discovery API is exposed by the
|
||||
`controller-manager-discovery-service` Service on port 8090.
|
||||
- Auth: create the token Secret, or the API serves **unauthenticated**
|
||||
@@ -185,46 +186,16 @@ can be two differently-configured instances of the same `gcp` type (see
|
||||
| --- | --- | --- | --- |
|
||||
| `kubernetes` | A real Squid pod (`ubuntu/squid:6.6-24.04_edge` by default) in the same cluster the operator runs in | `image` (optional) | Needs no cloud account — local dev, CI, and the kind quickstart. Pods share the cluster's egress IP, so it exercises the full lifecycle but not distinct egress paths. |
|
||||
| `gcp` | A Compute Engine VM with an ephemeral external IP | `project` (required), `network`, `networkTag`, `diskSizeGb` | The real egress fleet. Deliberately uses only four API calls (Insert / Get / Delete / AggregatedList), all fire-and-forget: `Create` returns as soon as the operation is submitted and the reconciler discovers progress by polling `Get`. Auth is Application Default Credentials — workload identity in-cluster, `gcloud` ADC locally; no key-file plumbing. |
|
||||
| `kvm` | A libvirt VM on one of your own hypervisors, reached over SSH | `sshKeyFile`, `knownHostsFile` (unless `insecureIgnoreHostKey`), `machineTypes`, `hosts` (required); `images`, `diskSizeGb`, `vendorData`, `installGuestAgent` | Home-lab or on-prem egress. The base image is downloaded into each host's pool on first use; `Create` waits for the VM to start. See [docs/kvm-provider.md](docs/kvm-provider.md). |
|
||||
|
||||
### Adding a provider
|
||||
|
||||
A new backend (Hetzner, AWS, ...) is four pieces; the contract lives in
|
||||
[internal/provider/provider.go](internal/provider/provider.go):
|
||||
|
||||
1. **Implement the 4-method `Provider` interface** in a new
|
||||
`internal/provider/<type>/` package:
|
||||
- `Create` submits and returns — it never blocks until the VM runs, and
|
||||
must be idempotent keyed on `req.Name` (a deterministic name derived
|
||||
from the Proxy's UID), so a repeat call after a crash finds the
|
||||
existing instance instead of duplicating it.
|
||||
- `Get` returns `provider.ErrNotFound` as a *normal* outcome — the
|
||||
reconciler branches on it for replacement and adoption, so don't
|
||||
treat it as exceptional.
|
||||
- `Delete` is idempotent: deleting an already-gone instance is not an
|
||||
error.
|
||||
- `ListByTag` returns every instance the operator ever tagged, for
|
||||
orphan GC.
|
||||
2. **Tag every created resource** with `LabelManaged=true` and
|
||||
`LabelUID=<Proxy UID>`, and report `CreatedAt` — orphan GC relies on
|
||||
all three to find owned resources and skip in-flight creates.
|
||||
3. **Classify every returned error** with `provider.Wrap` into the
|
||||
four-sentinel taxonomy in
|
||||
[internal/provider/errors.go](internal/provider/errors.go)
|
||||
(`ErrNotFound` / `ErrQuotaExceeded` / `ErrTransient` / `ErrPermanent`)
|
||||
— the reconciler decides retry, slow backoff, or latching `Failed`
|
||||
purely from that classification, never from provider-specific types.
|
||||
4. **Wire it up**: add a type-specific config block in
|
||||
[internal/provider/config.go](internal/provider/config.go), and
|
||||
register the constructor in the builtins map in
|
||||
[cmd/main.go](cmd/main.go) (`"<type>": <pkg>.New`). The registry
|
||||
([internal/provider/registry](internal/provider/registry/registry.go))
|
||||
handles named instances, and the metrics wrapper is applied
|
||||
automatically.
|
||||
|
||||
Test against a fake API seam rather than the real cloud — see the
|
||||
`instancesAPI` seam in
|
||||
[internal/provider/gcp/gcp.go](internal/provider/gcp/gcp.go) for the
|
||||
pattern.
|
||||
The contract, the rules each caller depends on, a feature matrix of the
|
||||
existing providers and a step-by-step checklist are in
|
||||
[docs/providers.md](docs/providers.md). Start there: several rules (a
|
||||
self-contained `providerID`, `Get` returning `ErrNotFound` only when the
|
||||
instance is really gone, `CreatedAt` read back for orphan GC) are enforced by
|
||||
callers, not by the interface.
|
||||
|
||||
## Tracing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user