egress-proxies-operator/docs/plans-executions/2026-09-11-1526-provider-health-checks.md

Execution: provider health checks

Plan: docs/plans/2026-09-11-1526-provider-health-checks.md

• Step 0 — Save the plan
• Step 1 — ProviderHealth API and CRD
• Step 2 — Contract plus the kubernetes and GCP checks
• Step 3 — KVM check
• Step 4 — Monitor
• Step 5 — Metrics and wiring
• Step 6 — Docs
• Step 7 — Validate, record, push, open the PR

Step 0 — Save the plan

Branched off main (at the PR #5 merge, eb8f075). The plan was copied out of ~/.claude/plans/ and committed on its own, before any code, as the repo's plan rules require:

git checkout -b feat/provider-health main
cp ~/.claude/plans/now-i-would-like-flickering-wall.md docs/plans/2026-09-11-1526-provider-health-checks.md

The uncommitted .claude/settings.json change was left unstaged and stays out of this branch.

Step 1 — ProviderHealth API and CRD

kubebuilder isn't installed here (only bin/controller-gen, bin/kustomize and bin/setup-envtest are), so api/v1alpha1/providerhealth_types.go and the PROJECT entry were written by hand instead of with kubebuilder create api. For a cluster-scoped resource PROJECT simply omits namespaced, which is what kubebuilder itself writes.

The new types are registered in the existing AddKnownTypes call in proxy_types.go, not in a second init().

The deepcopy code, the CRD and the RBAC come from the usual generators:

make generate manifests
go build ./... && go vet ./api/... && go test -short ./api/...

To confirm the new CRD and the viewer role actually reach the rendered install, I built the kustomization and grepped for them:

bin/kustomize build config/default | grep -n providerhealth
# providerhealths.crawl.example.com (scope: Cluster, shortName ph)
# egress-proxies-operator-providerhealth-viewer-role

Worth noting:

• The only role added is the viewer role. The operator is the object's only writer, so the admin and editor roles kubebuilder scaffolds for Proxy were not added.
• The type file has no unit test, since it has no logic. The schema, including the required ready/lastTransitionTime and the list-map keys, is proven by the envtest round-trip in Step 4.
• config/rbac/role.yaml is unchanged so far. The manager's own permissions arrive in Step 4 with the Monitor's RBAC markers.

Step 2 — Contract plus the kubernetes and GCP checks

provider.Checker is an optional interface in internal/provider/check.go, kept out of Provider. Each implementation has a compile-time assertion, because the monitor's type assertion would otherwise fail silently.

• kubernetes: lists at most one managed Pod cluster-wide, using the same client, verb and scope as ListByTag. The target is apiserver, at the restCfg.Host now stored in the new apiHost field.
• GCP: calls a new seam method, Probe. realInstances.Probe lives in check.go, next to its only caller. ListByTag's Sprintf filter became the shared managedFilter constant, so the check and the GC sweep can't drift apart.

Before relying on "MaxResults is only a page size", I checked the SDK's REST AggregatedList in the module cache:

grep -n "func (c \*instancesRESTClient) AggregatedList" -A80 \
  "$(go list -m -f '{{.Dir}}' cloud.google.com/go/compute)"/apiv1/instances_client.go
# InternalFetch: req.MaxResults = pageSize ... return elems, resp.GetNextPageToken(), nil

go build ./... && go vet ./internal/provider/... && go test -race -count=1 ./internal/provider/...

Worth noting:

• The planned TestRealInstances_Probe_fetchesOnePage became a three-case TestRealInstances_Probe table.
  • It keeps the first page's nextPageToken, and asserts exactly one request with maxResults=1 for each case.
  • "empty project" covers iterator.Done.
  • "api error keeps its status" proves the real client returns a *googleapi.Error, so a disabled Compute API is reported as Forbidden rather than Unreachable.
• Once a context has ended, both checks report Timeout whatever the error says. The monitor's timeout is the usual cause, and the underlying error only repeats it.