Four unrelated working-tree changes, none of them touching operator behaviour: - review/*.md -> docs/reviews/2026-09-17-kvm2-provider-review/, byte for byte, so the kvm2 review sits with the rest of the docs under a name that says which review it was. - AGENTS.md: replaced the generic kubebuilder scaffold guide with what is actually true of this repo — the real component layout, the lint/test commands that work here, and the behavioural gotchas (replace-on-change hashing, the cloud-init Secret label, in-memory leases). - .claude/settings.json: the allow list still carried /Users/jan.novak paths from the macOS checkout; re-pointed at /home/novakj and collapsed the one-off entries into prefixes. - config/samples/proxy_kvm2_crawl_only_one.yaml: a one-VM cut of the crawl pool, the rest of the members left commented out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5.8 KiB
5.8 KiB
egress-proxies-operator
Sources and architecture
- Trust
go.mod,Makefile, and current code over version tables or bootstrap prose.CLAUDE.md's opening architecture/commands sections are stale; this is a working operator, not an empty scaffold. cmd/main.goassembles the provider registry, reconciler, proxy health engine, in-memory lease store, discovery HTTP API, orphan GC, and provider-health monitor on one controller-runtime manager.api/v1alpha1contains namespacedProxyand cluster-scopedProviderHealthresources (crawl.example.com). Validation/defaulting uses CRD markers/CEL; there are no admission webhooks.internal/providerdefines the backend contract independently of CRD types. Constructors are registered incmd/main.go;spec.providerselects a configured instance name, not a provider type.kvmandkvm2are separate implementations and do not adopt each other's VMs.- Before changing a provider, read
docs/provider-contract.md; usedocs/provider-spec-template.mdfor a new one. Callers rely on self-contained provider IDs, truthfulErrNotFound, idempotent create/delete, and persisted UID/creation time for orphan GC. Seedocs/providers-existing.mdfor implementation-specific exceptions. docs/architecture.mdexplains component ownership and state transitions;docs/api.mddocuments discovery/leases;docs/testing.mdcovers test boundaries and real-infrastructure setup.
Commands and verification
- Go version comes from
go.mod(currently 1.26.0). Make installs development tools intobin/. - After Go changes:
make lint-fix, thenmake test. The latter regenerates manifests/DeepCopy, formats, vets, sets up envtest, then runs non-e2e tests with-raceand writescover.out. - Fast checks:
go test -short -race ./...skips envtest. Focused package/test:go test -race ./internal/lease/orgo test -race -run TestAcquire ./internal/lease/. - Envtest needs local API-server/etcd binaries; use
make setup-envtestbefore targeted runs. For the controller suite with the current Kubernetes minor:The Makefile derives the envtest minor fromKUBEBUILDER_ASSETS="$(./bin/setup-envtest use 1.36 --bin-dir "$PWD/bin" -p path)" go test -race ./internal/controller/ -run TestControllers -ginkgo.focus='<spec regex>'go.mod; do not replace it with a dependency patch version that may have no published binary bundle. - Tests are mostly stdlib
testing; controller envtest and e2e use Ginkgo/Gomega. Fake clients do not execute CEL or schema defaults: test those with envtest, and use the API's*OrDefaulthelpers in unit tests. Envtest has no kubelet, so it cannot prove Squid starts or serves traffic. make lint/make lint-fixbuilds a custom golangci-lint with thelogcheckplugin from.custom-gcl.yml; a stock global binary is insufficient..golangci.ymlforbidssort(useslices). Logging uses controller-runtime/logr, with Kubernetes-style messages.- Real-host tests are opt-in:
make test-integration-kvm/make test-integration-kvm2require the correspondingKVM_TEST_*/KVM2_TEST_*variables fromdocs/testing.md. After provider refactors, compile-check these withgo vet -tags=integration ./.... - E2E requires Docker, kind, kubectl, and an isolated cluster.
make test-e2eusesKIND_CLUSTER=egress-proxies-operator-test-e2e; ensure kubectl targets that cluster, especially if it already exists. On failure, runmake cleanup-test-e2eexplicitly. Compile-check withgo vet -tags=e2e ./.... Tracing specs additionally require bothTEMPO_URLandOTLP_ENDPOINT. .gitea/workflows/build.yamlruns only on tags/manual dispatch; its gate isgo vet ./...,go build ./...,go test -short ./.... It does not replace local lint, race, or envtest verification.
Code generation
- After API types or Kubebuilder markers change, run
make manifests generate. Edit source markers, never generatedconfig/crd/bases/*.yaml,config/rbac/role.yaml, or**/zz_generated.*.go. - Keep
+kubebuilder:rbacmarkers in standalone package-level comment blocks, outside function/type documentation; otherwise controller-gen can silently ignore them. Inspectgit diff config/rbac/role.yamlafter generation. - Preserve
+kubebuilder:scaffold:*markers and the single-group layout. Usekubebuilder create api/kubebuilder create webhookfor new scaffolding;PROJECTis CLI-managed.
Behaviour and operational gotchas
- Managed proxies are replace-on-change:
internal/controller/spechash.gohashes placement, resolved cloud-init content, and effective port. Do not hash all ofProxySpec: attributes, lease limits, and health settings must not churn VMs. - Cloud-init Secrets must carry
crawl.example.com/cloud-init: "true"; the manager's Secret cache excludes unlabelled Secrets. Rotating labelled Secret content triggers VM replacement. - Proxy health checks fetch through the proxy. The health engine supplies snapshots/events and the reconciler writes status; status-affecting transitions drive writes, so
lastHealthCheckTimeis not a last-probe heartbeat. - Leases/cooldowns are in-memory and disappear on restart. Discovery is not leader-elected; the deployment uses one replica. Adding replicas alone does not provide shared lease capacity.
- Use
make run-devlocally: plainmake runomits the required--providers-config. Host-side probes generally cannot route to kind pod IPs; deploy in-cluster using the README quickstart to verify kubernetes-provider health. make deploy IMG=...andmake build-installer IMG=...modifyconfig/manager/kustomization.yamlin place. Check that diff before including it in a change.- Namespace-restricted mode disables orphan GC by default.
--gc-allow-namespacedbypasses that guard and can classify instances owned outside the visible namespace as orphans.