# Provider spec template Copy this file to `docs/-provider-spec.md`, fill in every **Answer** line, and hand the result to whoever implements the provider together with [provider-contract.md](provider-contract.md) and [providers-existing.md](providers-existing.md). An unanswered entry blocks implementation: it is a decision the contract leaves to the provider, and the implementer must not guess it. Where an entry names an axis in backticks, the allowed answers are the profiles of that axis in contract §4, and the entry number matches the last column of the contract's §4.20 table. **Evidence** is where the answer was confirmed: a link into the backend's API documentation, or the experiment that was run. "Verify against the target version" is a valid interim answer only for the Evidence line, never for the Answer. ## Header - **Type** (the `type:` value in the providers config, lowercase): - **Example configured name**: - **Backend and API version this spec was written against**: - **Client** (`vendor-sdk` with the module path, or `thin-client`): - **Author and date**: ## 1. Identity and providerID **Question.** What is the providerID's shape, which location parts does it embed, and what validates the config values pasted into it? **Allowed answers.** `/`, `/`, or a location-free `` (contract §6; required when instances can migrate, see entry 2). **Obligations.** Strict parsing; `ErrPermanent` for a malformed ID from `Get` and `Delete`; a regexp on every config value that becomes an ID segment, in `config_.go`. **Answer:** **Evidence:** ## 2. Primary key (`primary-key`) **Question.** Is `req.Name` the backend's key, or does the backend allocate an identifier? **Allowed answers.** `name-is-key`; `allocated-id`. **Obligations.** `allocated-id`: `Create` finds an existing instance by name before allocating; allocation is safe against a concurrent `Create` for another Proxy (deterministic from `req.UID`, or allocate-then-verify); `Get` and `Delete` use the allocated key; state how a collision with an instance the operator does not own is detected and reported. **Answer:** **Evidence:** ## 3. Placement mapping **Question.** What do `Placement.Region`, `Zone`, `MachineType` and `Image` mean for this backend, which are required, which are ignored, and what happens when a value is unknown? **Allowed answers.** Per field: a config catalogue key, a backend identifier passed through, or ignored (contract §2.1, MAY ignore with documentation). **Obligations.** Unknown values are rejected with `ErrPermanent` naming the field and value before any backend call; ignored fields are listed in the provider's guide because they are still in the spec hash; an empty value with a single-choice catalogue may default, and the guide says so. **Answer:** **Evidence:** ## 4. Image source **Question.** Where does the boot image come from, and what does a catalogue edit do to running instances? **Allowed answers.** A catalogue in the config block (key to URL or backend image ID); a backend template or image referenced directly by `Placement.Image`; a URL fetched by the provider into the backend on first use. **Obligations.** If the provider fetches or converts images: streaming, no buffering (contract §1.4), a completeness marker so a cut-off fetch is redone, concurrency control so two Creates fetch once, and a written warning that replacing an image in use may corrupt instances that depend on it. If the backend owns the image: the prerequisite is documented in the guide. **Answer:** **Evidence:** ## 5. Cloud-init delivery **Question.** How does `CreateRequest.CloudInit` reach the guest, byte for byte? **Allowed answers.** User-data metadata on the instance; a NoCloud seed built in-process and attached as media; a file placed on the backend and referenced from the instance config; not delivered (documented, with how the proxy gets configured instead). **Obligations.** Size limits of the channel stated; whether vendor-data or a second document can ride along; the sentinel test (contract §11); the channel must be fully reachable through the API the provider uses (no step that needs shell access on the backend unless the spec says the provider has it). **Answer:** **Evidence:** ## 6. Machine type mapping **Question.** How does `Placement.MachineType` become CPU, memory and disk? **Allowed answers.** A catalogue of named profiles in the config block; a backend size name passed through. **Obligations.** A catalogue needs validation of each profile; disk size default and its config field named. **Answer:** **Evidence:** ## 7. Network and IP discovery (`readiness`, `ip-source`) **Question.** Which network does the instance attach to, where does its address come from, how long until it is usable, and what makes it reachable from the operator pod and from clients on `spec.port`? **Allowed answers.** `readiness`: `immediate`; `provisioning-window`. `ip-source`: `assigned-at-create`; `discovered` (name the mechanism: guest agent, DHCP lease, neighbour table, and the order if several). **Obligations.** `discovered`: what the guest must run, how it gets installed, what happens without it; discovery failure is Provisioning, not an error; only routable IPv4 is reported (state the filter). Reachability: firewall or routing steps the provider performs, or the prerequisite the guide states. Whether `Port` is used by the provider at all. **Answer:** **Evidence:** ## 8. GC marks and CreatedAt (`gc-mark`, `created-at`) **Question.** Where do `LabelManaged` and `LabelUID` live, where does `CreatedAt` come from, and which secondary resources are marked? **Allowed answers.** `gc-mark`: `native-labels`; `written-metadata`; `mark-window`. `created-at`: `backend-timestamp`; `written-at-create`. **Obligations.** `mark-window`: the window's length in seconds, why it is far below `MinAge`, and how the same Proxy's next reconcile cleans an unmarked instance. Any value-format constraint of the backend's labels (allowed characters, length) and how a Proxy UID fits it. **Answer:** **Evidence:** ## 9. State mapping (`provisioning-state`) **Question.** How does every backend state map to `StateProvisioning`, `StateRunning`, `StateStopped`, `StateTerminated`, and how does `Get` tell "defined but never started" from "stopped"? **Allowed answers.** A table of backend state (plus lock or task flags) to provider state, with the unknown-state row mapping to `StateStopped`. `provisioning-state`: `native`; `create-blocks`; `marker`. **Obligations.** Running requires an IP; `marker` names what is recorded at creation and how `Get` reads it; `create-blocks` names the cap on the wait. **Answer:** | Backend state | Lock or task | Provider state | |---|---|---| | | | | **Evidence:** ## 10. Create: blocking and task tracking (`create-return`, `async-ops`) **Question.** What does `Create` wait for before returning, and how are asynchronous backend tasks tracked? **Allowed answers.** `create-return`: `submit`; `blocking`. `async-ops`: `none`; `task-handle`; `backend-lock`. **Obligations.** `task-handle`: where the handle is kept and how a failed task surfaces (in `Get`, as which class, with which message). `backend-lock`: what `Get` and `Delete` return while locked. Every step of `Create` is idempotent so a cut-off attempt resumes; list the steps and what each finds on a retry. **Answer:** **Evidence:** ## 11. Delete and secondary resources (`delete-return`, `secondary-resources`) **Question.** In what order does `Delete` tear things down, what does it do when called again mid-delete, and what does `Get` return meanwhile? **Allowed answers.** `delete-return`: `returns-early`; `waits`. `secondary-resources`: `none`; `cascade` (name the flags set at create); `explicit` (name each resource and how it is found by name). **Obligations.** Idempotent at every step; "already deleting" is success and cheap; a location no longer in the config is a logged no-op success; any resource that can outlive the instance without being listable is a documented GC blind spot. **Answer:** **Evidence:** ## 12. ListByTag scope and partial failure (`list-partial`) **Question.** What is the full scope `ListByTag` enumerates, and what happens when part of it is unreachable? **Allowed answers.** `fail-whole`; `omit-unreachable`. **Obligations.** The scope covers everything the provider can create in; server-side filtering on the mark where the backend allows it; a single unreadable instance may be skipped with a log line; the cost of one list at fleet size. **Answer:** **Evidence:** ## 13. Error classification (`not-found`, `quota`) **Question.** Which backend responses map to which of the four classes, and which signal means "does not exist"? **Allowed answers.** A table of backend status or code (with message pattern where needed) to class. `not-found`: `status-404`; `error-code`; `list-and-miss`. `quota`: `detected`; `absent`. **Obligations.** One `classify`, a table test with captured real responses for every matched message; the permission-denied-versus-quota case decided; authentication failures, host-key or certificate failures, and cancellation each assigned a class with a reason. **Answer:** | Backend response | Class | Why | |---|---|---| | | | | **Evidence:** ## 14. Credentials and TLS (`credentials`) **Question.** What credential does the operator use, where does it come from, when is it read, and how is the backend's TLS certificate trusted? **Allowed answers.** `mounted-files` (Secret name, mount path, file names, `optional: true`); `environment`; `workload-identity`. TLS: `system-roots`; `ca-file`; `insecure` (the flag's exact name). **Obligations.** The minimum privilege set the credential needs, listed; rotation behaviour (free, or restart needed); the sentinel test for the credential (contract §7); the constructor never dials. **Answer:** **Evidence:** ## 15. Config block **Question.** What are the fields of `Config`, their defaults, and their validation? **Allowed answers.** A table: field, required, default, validation, and which identifier or path the value ends up in. **Obligations.** Contract §10: `setBlocks`, the type allow-list in `Config.validate` and the constructor map in `cmd/main.go`, `config_.go` with its tests, regexps for values pasted into IDs or file names. **Answer:** | Field | Required | Default | Validation | Ends up in | |---|---|---|---|---| | | | | | | **Evidence:** ## 16. Checker **Question.** What are the check targets, what does one check do, which reasons can it report, and what goes into `Details`? **Allowed answers.** One target per thing that fails independently; the shared reasons from `internal/provider/check.go` first, provider-specific reasons only with a stated "what the admin does differently"; `Details` keys listed. **Obligations.** Same credential, verb and scope as `ListByTag` and `Create`; cheap (one page, one object); never disrupts a reconcile that shares a connection; returns promptly on `ctx`; the `status.type` comment in `api/v1alpha1/providerhealth_types.go` updated. **Answer:** **Evidence:** ## 17. Client and child spans (`client-kind`, `child-spans`) **Question.** What talks to the backend, and how do its calls appear under the operation span? **Allowed answers.** `client-kind`: `vendor-sdk` (module path, and whether it is maintained); `thin-client`; `rpc-over-ssh`. `child-spans`: `sdk-instrumentation`; `transport-wrapper`; `span-decorator`; `none`. **Obligations.** The seam interface's name and the fake that implements it; `thin-client`: one error type carrying status and body, transport timeouts, parent-gated `otelhttp`; `none` is a documented gap; the span recorder test (contract §11). **Answer:** **Evidence:** ## 18. Timeouts (`timeouts`) **Question.** What bounds each backend call, the dial, and any long step such as a download or a wait loop, and what does a cancelled `ctx` do to a shared connection? **Allowed answers.** `per-call` (the budgets, per operation); `ctx-only` (cite the client's own timeout). **Obligations.** Contract §1.5: no caller sets a deadline; a hung call pins a worker or stalls GC. Cancellation behaviour on a shared connection stated (abandon the call, or close the connection). **Answer:** **Evidence:** ## 19. Concurrency **Question.** What is shared between concurrent calls (connections, caches, locks), and what happens when two calls for the same instance name overlap? **Allowed answers.** A description of the shared state and its locking. **Obligations.** Safe for 3 reconciles, a GC sweep and a check round at once; two `Delete` calls for one instance overlap safely; one fetch per image per location when several Creates race. **Answer:** **Evidence:** ## 20. Logging (`logging`) **Question.** Which tiers does the provider log, with which keys on every line? **Allowed answers.** `tiered`; `outcome-only`; `none` (cite the client's own logging). **Obligations.** Logger from `logf.FromContext(ctx)` on every call; never `CloudInit` or credentials; `provider` and the instance name on every line that concerns an instance. **Answer:** **Evidence:** ## 21. Tests (`backend-test`) **Question.** What is the fake, which contract tests exist, and how is the provider proven against the real backend? **Allowed answers.** `backend-test`: `cluster-e2e`; `integration-tag` (the `_TEST_*` variables, the `make test-integration-` target); `manual-runbook`. **Obligations.** Every item of the contract §11 list, ticked here: - [ ] repeat `Create` - [ ] `Get` after `Delete` - [ ] `Delete` twice - [ ] malformed ID - [ ] unknown state maps to `StateStopped` - [ ] `StateRunning` requires an IP - [ ] `UID` and `CreatedAt` read back - [ ] `ListByTag` excludes unmarked instances - [ ] `Get` during an in-progress delete - [ ] `Get` during an asynchronous operation or lock - [ ] never started is never `StateStopped` - [ ] cancelled `ctx` returns promptly - [ ] `classify` table with captured responses - [ ] Checker reasons, expired `ctx`, stable order - [ ] sentinel tests for `CloudInit` and the credential - [ ] child spans under the operation span - [ ] real-backend lifecycle test **Answer:** **Evidence:** ## 22. Deployment **Question.** What changes in `config/` and the Makefile? **Allowed answers.** The list: the `optional: true` mount in `config/manager/manager.yaml`; the commented entries in `config/manager/providers_config.yaml` and `config/samples/providers-config.yaml`; `config/samples/proxy_.yaml` with a cloud-init that installs a proxy on the sample image; a Makefile helper for the Secret; RBAC markers if the provider talks to the cluster's own API; whether `hack/providers-dev.yaml` gets an entry. **Answer:** **Evidence:** ## 23. Docs **Question.** Which documents are written or updated? **Allowed answers.** `docs/-provider.md` with its section list; the row and section in `providers-existing.md`; the `architecture.md` rows (components table, §5, §11, and §10 if spans are added); the README row; the `docs/testing.md` section; the `CHANGELOG.md` entry after confirmation. **Answer:** **Evidence:** ## 24. Known limits accepted at design time **Question.** Which of the contract's §13 limits bite this provider in practice, and which new gaps does this design accept? **Allowed answers.** A list, each with the consequence for an operator and the workaround. **Answer:** **Evidence:**