egress-proxies-operator/docs/provider-spec-template.md

# Provider spec template

Copy this file to `docs/<type>-provider-spec.md`, fill in every **Answer**
line, and hand the result to whoever implements the provider together with
[provider-contract.md](provider-contract.md) and
[providers-existing.md](providers-existing.md). An unanswered entry blocks
implementation: it is a decision the contract leaves to the provider, and the
implementer must not guess it.

Where an entry names an axis in backticks, the allowed answers are the
profiles of that axis in contract §4, and the entry number matches the last
column of the contract's §4.20 table. **Evidence** is where the answer was
confirmed: a link into the backend's API documentation, or the experiment
that was run. "Verify against the target version" is a valid interim
answer only for the Evidence line, never for the Answer.

## Header

- **Type** (the `type:` value in the providers config, lowercase):
- **Example configured name**:
- **Backend and API version this spec was written against**:
- **Client** (`vendor-sdk` with the module path, or `thin-client`):
- **Author and date**:

## 1. Identity and providerID

**Question.** What is the providerID's shape, which location parts does it
embed, and what validates the config values pasted into it?

**Allowed answers.** `<location>/<name>`, `<location>/<allocated-key>`, or a
location-free `<allocated-key>` (contract §6; required when instances can
migrate, see entry 2).

**Obligations.** Strict parsing; `ErrPermanent` for a malformed ID from `Get`
and `Delete`; a regexp on every config value that becomes an ID segment, in
`config_<type>.go`.

**Answer:**

**Evidence:**

## 2. Primary key (`primary-key`)

**Question.** Is `req.Name` the backend's key, or does the backend allocate an
identifier?

**Allowed answers.** `name-is-key`; `allocated-id`.

**Obligations.** `allocated-id`: `Create` finds an existing instance by name
before allocating; allocation is safe against a concurrent `Create` for
another Proxy (deterministic from `req.UID`, or allocate-then-verify); `Get`
and `Delete` use the allocated key; state how a collision with an instance
the operator does not own is detected and reported.

**Answer:**

**Evidence:**

## 3. Placement mapping

**Question.** What do `Placement.Region`, `Zone`, `MachineType` and `Image`
mean for this backend, which are required, which are ignored, and what
happens when a value is unknown?

**Allowed answers.** Per field: a config catalogue key, a backend identifier
passed through, or ignored (contract §2.1, MAY ignore with documentation).

**Obligations.** Unknown values are rejected with `ErrPermanent` naming the
field and value before any backend call; ignored fields are listed in the
provider's guide because they are still in the spec hash; an empty value with
a single-choice catalogue may default, and the guide says so.

**Answer:**

**Evidence:**

## 4. Image source

**Question.** Where does the boot image come from, and what does a catalogue
edit do to running instances?

**Allowed answers.** A catalogue in the config block (key to URL or backend
image ID); a backend template or image referenced directly by
`Placement.Image`; a URL fetched by the provider into the backend on first
use.

**Obligations.** If the provider fetches or converts images: streaming, no
buffering (contract §1.4), a completeness marker so a cut-off fetch is redone,
concurrency control so two Creates fetch once, and a written warning that
replacing an image in use may corrupt instances that depend on it. If the
backend owns the image: the prerequisite is documented in the guide.

**Answer:**

**Evidence:**

## 5. Cloud-init delivery

**Question.** How does `CreateRequest.CloudInit` reach the guest, byte for
byte?

**Allowed answers.** User-data metadata on the instance; a NoCloud seed
built in-process and attached as media; a file placed on the backend and
referenced from the instance config; not delivered (documented, with how the
proxy gets configured instead).

**Obligations.** Size limits of the channel stated; whether vendor-data or a
second document can ride along; the sentinel test (contract §11); the channel
must be fully reachable through the API the provider uses (no step that needs
shell access on the backend unless the spec says the provider has it).

**Answer:**

**Evidence:**

## 6. Machine type mapping

**Question.** How does `Placement.MachineType` become CPU, memory and disk?

**Allowed answers.** A catalogue of named profiles in the config block; a
backend size name passed through.

**Obligations.** A catalogue needs validation of each profile; disk size
default and its config field named.

**Answer:**

**Evidence:**

## 7. Network and IP discovery (`readiness`, `ip-source`)

**Question.** Which network does the instance attach to, where does its
address come from, how long until it is usable, and what makes it reachable
from the operator pod and from clients on `spec.port`?

**Allowed answers.** `readiness`: `immediate`; `provisioning-window`.
`ip-source`: `assigned-at-create`; `discovered` (name the mechanism: guest
agent, DHCP lease, neighbour table, and the order if several).

**Obligations.** `discovered`: what the guest must run, how it gets
installed, what happens without it; discovery failure is Provisioning, not an
error; only routable IPv4 is reported (state the filter). Reachability:
firewall or routing steps the provider performs, or the prerequisite the
guide states. Whether `Port` is used by the provider at all.

**Answer:**

**Evidence:**

## 8. GC marks and CreatedAt (`gc-mark`, `created-at`)

**Question.** Where do `LabelManaged` and `LabelUID` live, where does
`CreatedAt` come from, and which secondary resources are marked?

**Allowed answers.** `gc-mark`: `native-labels`; `written-metadata`;
`mark-window`. `created-at`: `backend-timestamp`; `written-at-create`.

**Obligations.** `mark-window`: the window's length in seconds, why it is
far below `MinAge`, and how the same Proxy's next reconcile cleans an
unmarked instance. Any value-format constraint of the backend's labels
(allowed characters, length) and how a Proxy UID fits it.

**Answer:**

**Evidence:**

## 9. State mapping (`provisioning-state`)

**Question.** How does every backend state map to `StateProvisioning`,
`StateRunning`, `StateStopped`, `StateTerminated`, and how does `Get` tell
"defined but never started" from "stopped"?

**Allowed answers.** A table of backend state (plus lock or task flags) to
provider state, with the unknown-state row mapping to `StateStopped`.
`provisioning-state`: `native`; `create-blocks`; `marker`.

**Obligations.** Running requires an IP; `marker` names what is recorded at
creation and how `Get` reads it; `create-blocks` names the cap on the wait.

**Answer:**

| Backend state | Lock or task | Provider state |
|---|---|---|
| | | |

**Evidence:**

## 10. Create: blocking and task tracking (`create-return`, `async-ops`)

**Question.** What does `Create` wait for before returning, and how are
asynchronous backend tasks tracked?

**Allowed answers.** `create-return`: `submit`; `blocking`. `async-ops`:
`none`; `task-handle`; `backend-lock`.

**Obligations.** `task-handle`: where the handle is kept and how a failed
task surfaces (in `Get`, as which class, with which message). `backend-lock`:
what `Get` and `Delete` return while locked. Every step of `Create` is
idempotent so a cut-off attempt resumes; list the steps and what each finds
on a retry.

**Answer:**

**Evidence:**

## 11. Delete and secondary resources (`delete-return`, `secondary-resources`)

**Question.** In what order does `Delete` tear things down, what does it do
when called again mid-delete, and what does `Get` return meanwhile?

**Allowed answers.** `delete-return`: `returns-early`; `waits`.
`secondary-resources`: `none`; `cascade` (name the flags set at create);
`explicit` (name each resource and how it is found by name).

**Obligations.** Idempotent at every step; "already deleting" is success and
cheap; a location no longer in the config is a logged no-op success; any
resource that can outlive the instance without being listable is a documented
GC blind spot.

**Answer:**

**Evidence:**

## 12. ListByTag scope and partial failure (`list-partial`)

**Question.** What is the full scope `ListByTag` enumerates, and what happens
when part of it is unreachable?

**Allowed answers.** `fail-whole`; `omit-unreachable`.

**Obligations.** The scope covers everything the provider can create in;
server-side filtering on the mark where the backend allows it; a single
unreadable instance may be skipped with a log line; the cost of one list at
fleet size.

**Answer:**

**Evidence:**

## 13. Error classification (`not-found`, `quota`)

**Question.** Which backend responses map to which of the four classes, and
which signal means "does not exist"?

**Allowed answers.** A table of backend status or code (with message pattern
where needed) to class. `not-found`: `status-404`; `error-code`;
`list-and-miss`. `quota`: `detected`; `absent`.

**Obligations.** One `classify`, a table test with captured real responses
for every matched message; the permission-denied-versus-quota case decided;
authentication failures, host-key or certificate failures, and cancellation
each assigned a class with a reason.

**Answer:**

| Backend response | Class | Why |
|---|---|---|
| | | |

**Evidence:**

## 14. Credentials and TLS (`credentials`)

**Question.** What credential does the operator use, where does it come from,
when is it read, and how is the backend's TLS certificate trusted?

**Allowed answers.** `mounted-files` (Secret name, mount path, file names,
`optional: true`); `environment`; `workload-identity`. TLS: `system-roots`;
`ca-file`; `insecure` (the flag's exact name).

**Obligations.** The minimum privilege set the credential needs, listed;
rotation behaviour (free, or restart needed); the sentinel test for the
credential (contract §7); the constructor never dials.

**Answer:**

**Evidence:**

## 15. Config block

**Question.** What are the fields of `<Type>Config`, their defaults, and their
validation?

**Allowed answers.** A table: field, required, default, validation, and which
identifier or path the value ends up in.

**Obligations.** Contract §10: `setBlocks`, the type allow-list in
`Config.validate` and the constructor map in `cmd/main.go`, `config_<type>.go`
with its tests, regexps for values pasted into IDs or file names.

**Answer:**

| Field | Required | Default | Validation | Ends up in |
|---|---|---|---|---|
| | | | | |

**Evidence:**

## 16. Checker

**Question.** What are the check targets, what does one check do, which
reasons can it report, and what goes into `Details`?

**Allowed answers.** One target per thing that fails independently; the
shared reasons from `internal/provider/check.go` first, provider-specific
reasons only with a stated "what the admin does differently"; `Details` keys
listed.

**Obligations.** Same credential, verb and scope as `ListByTag` and `Create`;
cheap (one page, one object); never disrupts a reconcile that shares a
connection; returns promptly on `ctx`; the `status.type` comment in
`api/v1alpha1/providerhealth_types.go` updated.

**Answer:**

**Evidence:**

## 17. Client and child spans (`client-kind`, `child-spans`)

**Question.** What talks to the backend, and how do its calls appear under the
operation span?

**Allowed answers.** `client-kind`: `vendor-sdk` (module path, and whether it
is maintained); `thin-client`; `rpc-over-ssh`. `child-spans`:
`sdk-instrumentation`; `transport-wrapper`; `span-decorator`; `none`.

**Obligations.** The seam interface's name and the fake that implements it;
`thin-client`: one error type carrying status and body, transport timeouts,
parent-gated `otelhttp`; `none` is a documented gap; the span recorder test
(contract §11).

**Answer:**

**Evidence:**

## 18. Timeouts (`timeouts`)

**Question.** What bounds each backend call, the dial, and any long step such
as a download or a wait loop, and what does a cancelled `ctx` do to a shared
connection?

**Allowed answers.** `per-call` (the budgets, per operation); `ctx-only`
(cite the client's own timeout).

**Obligations.** Contract §1.5: no caller sets a deadline; a hung call pins a
worker or stalls GC. Cancellation behaviour on a shared connection stated
(abandon the call, or close the connection).

**Answer:**

**Evidence:**

## 19. Concurrency

**Question.** What is shared between concurrent calls (connections, caches,
locks), and what happens when two calls for the same instance name overlap?

**Allowed answers.** A description of the shared state and its locking.

**Obligations.** Safe for 3 reconciles, a GC sweep and a check round at once;
two `Delete` calls for one instance overlap safely; one fetch per image per
location when several Creates race.

**Answer:**

**Evidence:**

## 20. Logging (`logging`)

**Question.** Which tiers does the provider log, with which keys on every
line?

**Allowed answers.** `tiered`; `outcome-only`; `none` (cite the client's own
logging).

**Obligations.** Logger from `logf.FromContext(ctx)` on every call; never
`CloudInit` or credentials; `provider` and the instance name on every line
that concerns an instance.

**Answer:**

**Evidence:**

## 21. Tests (`backend-test`)

**Question.** What is the fake, which contract tests exist, and how is the
provider proven against the real backend?

**Allowed answers.** `backend-test`: `cluster-e2e`; `integration-tag` (the
`<TYPE>_TEST_*` variables, the `make test-integration-<type>` target); `manual-runbook`.

**Obligations.** Every item of the contract §11 list, ticked here:

- [ ] repeat `Create`
- [ ] `Get` after `Delete`
- [ ] `Delete` twice
- [ ] malformed ID
- [ ] unknown state maps to `StateStopped`
- [ ] `StateRunning` requires an IP
- [ ] `UID` and `CreatedAt` read back
- [ ] `ListByTag` excludes unmarked instances
- [ ] `Get` during an in-progress delete
- [ ] `Get` during an asynchronous operation or lock
- [ ] never started is never `StateStopped`
- [ ] cancelled `ctx` returns promptly
- [ ] `classify` table with captured responses
- [ ] Checker reasons, expired `ctx`, stable order
- [ ] sentinel tests for `CloudInit` and the credential
- [ ] child spans under the operation span
- [ ] real-backend lifecycle test

**Answer:**

**Evidence:**

## 22. Deployment

**Question.** What changes in `config/` and the Makefile?

**Allowed answers.** The list: the `optional: true` mount in
`config/manager/manager.yaml`; the commented entries in
`config/manager/providers_config.yaml` and
`config/samples/providers-config.yaml`; `config/samples/proxy_<type>.yaml`
with a cloud-init that installs a proxy on the sample image; a Makefile
helper for the Secret; RBAC markers if the provider talks to the cluster's
own API; whether `hack/providers-dev.yaml` gets an entry.

**Answer:**

**Evidence:**

## 23. Docs

**Question.** Which documents are written or updated?

**Allowed answers.** `docs/<type>-provider.md` with its section list; the row
and section in `providers-existing.md`; the `architecture.md` rows (components
table, §5, §11, and §10 if spans are added); the README row; the
`docs/testing.md` section; the `CHANGELOG.md` entry after confirmation.

**Answer:**

**Evidence:**

## 24. Known limits accepted at design time

**Question.** Which of the contract's §13 limits bite this provider in
practice, and which new gaps does this design accept?

**Allowed answers.** A list, each with the consequence for an operator and
the workaround.

**Answer:**

**Evidence:**