kacerrandClaude Opus 5
8fa9e8aad9
Four unrelated working-tree changes, none of them touching operator behaviour: - review/*.md -> docs/reviews/2026-09-17-kvm2-provider-review/, byte for byte, so the kvm2 review sits with the rest of the docs under a name that says which review it was. - AGENTS.md: replaced the generic kubebuilder scaffold guide with what is actually true of this repo — the real component layout, the lint/test commands that work here, and the behavioural gotchas (replace-on-change hashing, the cloud-init Secret label, in-memory leases). - .claude/settings.json: the allow list still carried /Users/jan.novak paths from the macOS checkout; re-pointed at /home/novakj and collapsed the one-off entries into prefixes. - config/samples/proxy_kvm2_crawl_only_one.yaml: a one-VM cut of the crawl pool, the rest of the members left commented out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
43 lines
5.8 KiB
Markdown
43 lines
5.8 KiB
Markdown
# egress-proxies-operator
|
|
|
|
## Sources and architecture
|
|
|
|
- Trust `go.mod`, `Makefile`, and current code over version tables or bootstrap prose. `CLAUDE.md`'s opening architecture/commands sections are stale; this is a working operator, not an empty scaffold.
|
|
- `cmd/main.go` assembles the provider registry, reconciler, proxy health engine, in-memory lease store, discovery HTTP API, orphan GC, and provider-health monitor on one controller-runtime manager.
|
|
- `api/v1alpha1` contains namespaced `Proxy` and cluster-scoped `ProviderHealth` resources (`crawl.example.com`). Validation/defaulting uses CRD markers/CEL; there are no admission webhooks.
|
|
- `internal/provider` defines the backend contract independently of CRD types. Constructors are registered in `cmd/main.go`; `spec.provider` selects a configured **instance name**, not a provider type. `kvm` and `kvm2` are separate implementations and do not adopt each other's VMs.
|
|
- Before changing a provider, read `docs/provider-contract.md`; use `docs/provider-spec-template.md` for a new one. Callers rely on self-contained provider IDs, truthful `ErrNotFound`, idempotent create/delete, and persisted UID/creation time for orphan GC. See `docs/providers-existing.md` for implementation-specific exceptions.
|
|
- `docs/architecture.md` explains component ownership and state transitions; `docs/api.md` documents discovery/leases; `docs/testing.md` covers test boundaries and real-infrastructure setup.
|
|
|
|
## Commands and verification
|
|
|
|
- Go version comes from `go.mod` (currently 1.26.0). Make installs development tools into `bin/`.
|
|
- After Go changes: `make lint-fix`, then `make test`. The latter regenerates manifests/DeepCopy, formats, vets, sets up envtest, then runs non-e2e tests with `-race` and writes `cover.out`.
|
|
- Fast checks: `go test -short -race ./...` skips envtest. Focused package/test: `go test -race ./internal/lease/` or `go test -race -run TestAcquire ./internal/lease/`.
|
|
- Envtest needs local API-server/etcd binaries; use `make setup-envtest` before targeted runs. For the controller suite with the current Kubernetes minor:
|
|
```sh
|
|
KUBEBUILDER_ASSETS="$(./bin/setup-envtest use 1.36 --bin-dir "$PWD/bin" -p path)" go test -race ./internal/controller/ -run TestControllers -ginkgo.focus='<spec regex>'
|
|
```
|
|
The Makefile derives the envtest **minor** from `go.mod`; do not replace it with a dependency patch version that may have no published binary bundle.
|
|
- Tests are mostly stdlib `testing`; controller envtest and e2e use Ginkgo/Gomega. Fake clients do not execute CEL or schema defaults: test those with envtest, and use the API's `*OrDefault` helpers in unit tests. Envtest has no kubelet, so it cannot prove Squid starts or serves traffic.
|
|
- `make lint` / `make lint-fix` builds a custom golangci-lint with the `logcheck` plugin from `.custom-gcl.yml`; a stock global binary is insufficient. `.golangci.yml` forbids `sort` (use `slices`). Logging uses controller-runtime/logr, with Kubernetes-style messages.
|
|
- Real-host tests are opt-in: `make test-integration-kvm` / `make test-integration-kvm2` require the corresponding `KVM_TEST_*` / `KVM2_TEST_*` variables from `docs/testing.md`. After provider refactors, compile-check these with `go vet -tags=integration ./...`.
|
|
- E2E requires Docker, kind, kubectl, and an isolated cluster. `make test-e2e` uses `KIND_CLUSTER=egress-proxies-operator-test-e2e`; ensure kubectl targets that cluster, especially if it already exists. On failure, run `make cleanup-test-e2e` explicitly. Compile-check with `go vet -tags=e2e ./...`. Tracing specs additionally require both `TEMPO_URL` and `OTLP_ENDPOINT`.
|
|
- `.gitea/workflows/build.yaml` runs only on tags/manual dispatch; its gate is `go vet ./...`, `go build ./...`, `go test -short ./...`. It does not replace local lint, race, or envtest verification.
|
|
|
|
## Code generation
|
|
|
|
- After API types or Kubebuilder markers change, run `make manifests generate`. Edit source markers, never generated `config/crd/bases/*.yaml`, `config/rbac/role.yaml`, or `**/zz_generated.*.go`.
|
|
- Keep `+kubebuilder:rbac` markers in standalone package-level comment blocks, outside function/type documentation; otherwise controller-gen can silently ignore them. Inspect `git diff config/rbac/role.yaml` after generation.
|
|
- Preserve `+kubebuilder:scaffold:*` markers and the single-group layout. Use `kubebuilder create api` / `kubebuilder create webhook` for new scaffolding; `PROJECT` is CLI-managed.
|
|
|
|
## Behaviour and operational gotchas
|
|
|
|
- Managed proxies are replace-on-change: `internal/controller/spechash.go` hashes placement, resolved cloud-init content, and effective port. Do not hash all of `ProxySpec`: attributes, lease limits, and health settings must not churn VMs.
|
|
- Cloud-init Secrets must carry `crawl.example.com/cloud-init: "true"`; the manager's Secret cache excludes unlabelled Secrets. Rotating labelled Secret content triggers VM replacement.
|
|
- Proxy health checks fetch **through** the proxy. The health engine supplies snapshots/events and the reconciler writes status; status-affecting transitions drive writes, so `lastHealthCheckTime` is not a last-probe heartbeat.
|
|
- Leases/cooldowns are in-memory and disappear on restart. Discovery is not leader-elected; the deployment uses one replica. Adding replicas alone does not provide shared lease capacity.
|
|
- Use `make run-dev` locally: plain `make run` omits the required `--providers-config`. Host-side probes generally cannot route to kind pod IPs; deploy in-cluster using the README quickstart to verify kubernetes-provider health.
|
|
- `make deploy IMG=...` and `make build-installer IMG=...` modify `config/manager/kustomization.yaml` in place. Check that diff before including it in a change.
|
|
- Namespace-restricted mode disables orphan GC by default. `--gc-allow-namespaced` bypasses that guard and can classify instances owned outside the visible namespace as orphans.
|