Provider health checks: ProviderHealth CRD, per-provider checks and gauges #6

Merged
kacerr merged 12 commits from feat/provider-health into main 2026-09-16 10:46:47 +02:00
2 changed files with 35 additions and 1 deletions
Showing only changes of commit e6ed46fd7c - Show all commits
+21
View File
@@ -1,5 +1,26 @@
# Changelog
## 2026-09-16 08:45 UTC — Provider health checks: ProviderHealth CRD, per-provider checks, gauges
- **Each configured provider is now checked on a schedule** and reported in a
cluster-scoped `ProviderHealth` object (`kubectl get ph`), one per provider, with a
`Ready` condition, a reason, and per-target verdicts.
- **Checks:** kubernetes lists one object from the API server; GCP fetches a single
instance page; KVM checks SSH/libvirt reachability and the storage pool on every
host, in parallel. Failures map to fixed reasons (`Unreachable`,
`SSHHandshakeFailed`, `PoolNotFound`, `Timeout`, …).
- **The KVM check can't hurt reconciles:** it uses `hostPool.probe`, which gives up on
a slow RPC without dropping the shared connection, and waiting for a host slot can
be interrupted.
- **Metrics:** `proxy_operator_provider_target_up`, `_check_duration_seconds` and
`_check_timestamp_seconds`. The flags are `--provider-check-interval` and
`--provider-check-timeout` (defaults to `min(15s, interval)`).
- Validated with `make test`, `make test-e2e` and the real-hypervisor KVM run against
beelink. `make lint` is blocked by a Go 1.27 toolchain/staticcheck panic that has
nothing to do with this change.
- Key files: `api/v1alpha1/providerhealth_types.go`, `internal/provider/check.go`,
`internal/provider/kvm/`, `internal/providerhealth/`, `docs/architecture.md`.
## 2026-09-10 22:58 UTC — Code-review fixes on the KVM branch: deletion, validation, lazy SSH, guest agent
Five findings from a review of `feat/kvm-provider`, all fixed on the branch.
@@ -8,7 +8,7 @@ Plan: [docs/plans/2026-09-11-1526-provider-health-checks.md](../plans/2026-09-11
- [x] Step 4 — Monitor
- [x] Step 5 — Metrics and wiring
- [x] Step 6 — Docs
- [ ] Step 7 — Validate, record, push, open the PR
- [x] Step 7 — Validate, record, push, open the PR
## Step 0 — Save the plan
@@ -363,3 +363,16 @@ Worth noting:
the same commit: the old text promised the drop-on-timeout behaviour that is now gone.
- The real-hypervisor commands from the plan's Verification section are still the user's
to run; nothing here replaces them.
### Real-hypervisor validation and PR
The user ran the plan's real-hypervisor checks against beelink (the `TestKVMCheck`
integration test plus the manual `ProviderHealth` and metrics checks) and confirmed
they passed. After that the CHANGELOG entry went in, and the branch was pushed with
the token-file credential helper. The PR against `main` was opened through the Gitea
API because `tea` isn't installed.
Worth noting:
- The branch also carries `docs/plans/2026-09-16-0841-provider-blueprint.md`, on
purpose: that plan reaches `main` with this merge, and the blueprint work starts on a
new branch afterwards.