egress-proxies-operator/docs/plans-executions/2026-08-07-1747-proxy-operator.md

Execution log: proxy-operator

Pairs with docs/plans/2026-08-07-1747-proxy-operator.md.

Status

• Step 0 — Branch and scaffold
• Step 1 — API types (api/v1alpha1/proxy_types.go)
• Step 2 — Provider contract (internal/provider/)
• Step 3 — Kubernetes pod provider (internal/provider/kubernetes/; first built as an in-memory mock, then replaced — see the two Step 3 sections below)
• Step 4 — Reconciler (internal/controller/)
• Step 5 — Health engine (internal/health/)
• Step 6 — Lease store (internal/lease/)
• Step 7 — Discovery API (internal/discovery/)
• Step 8 — GCP provider (internal/provider/gcp/)
• Step 9 — Orphan GC + metrics
• Step 10 — Wiring, config, docs
• Step 11 — Tests
• Verification (vet/test/kind e2e) + commit, push, open MR

Step 0 — Branch and scaffold

Branched off the unborn main:

git checkout -b feat/proxy-operator

Installed kubebuilder v4.15.0 into a scratch GOBIN rather than the default $(go env GOPATH)/bin, since the module's package layout changed and go install .../cmd/kubebuilder@v4.15.0 (the path from the plan) 404s — the binary is now the module root itself:

GOBIN=<scratch>/bin go install sigs.k8s.io/kubebuilder/v4@v4.15.0

Scaffolded in place, with kubebuilder on PATH:

kubebuilder version
# KubeBuilder: v4.15.0, Kubernetes: 1.36.0

kubebuilder init --domain example.com \
  --repo gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator --plugins go/v4
# WARN: target directory not empty (expected — CLAUDE.md, docs/, .claude/ already existed)

kubebuilder create api --group crawl --version v1alpha1 --kind Proxy \
  --resource --controller

create api auto-ran make manifests at the end, which pulled and ran controller-gen itself:

sigs.k8s.io/controller-tools/cmd/controller-gen@v0.21.0
"$(bin)/controller-gen" object:headerFile="hack/boilerplate.go.txt",year=2026 paths="./..."

Confirmed the CRD group landed correctly (no doubling — --domain example.com --group crawl was used specifically to avoid the crawl.crawl.example.com trap called out in the plan):

grep -A2 "GroupVersion =" api/v1alpha1/groupversion_info.go
# SchemeGroupVersion = schema.GroupVersion{Group: "crawl.example.com", Version: "v1alpha1"}

Dropped the scaffolded GitHub Actions workflows (remote is Gitea, not GitHub):

git rm -r --cached .github 2>/dev/null; rm -rf .github

Ran the full manifest/codegen pass once more to confirm the toolchain is reproducible end to end:

make manifests generate
# controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
# controller-gen object:headerFile="hack/boilerplate.go.txt",year=2026 paths="./..."

then go build ./... and go vet ./..., both clean with no output.

Worth noting: CONTROLLER_TOOLS_VERSION in the generated Makefile came out at v0.21.0 by default in this kubebuilder release, so the Makefile edit the plan anticipated wasn't needed. Pre-existing CLAUDE.md/CHANGELOG.md content survived untouched; kubebuilder added its own README.md, AGENTS.md, .golangci.yml, .devcontainer/, Dockerfile on top of them — those get edited or left as-is in later steps. Committed as 076bc66.

Step 1 — API types (api/v1alpha1/proxy_types.go)

Wrote the full ProxySpec/ProxyStatus/Proxy types per the plan, including the four corrections called out there (MaxLeases *int32, HealthCheck with +kubebuilder:default={}, MinLength=1 on Provider/CloudInit.Inline, Conditions with +listType=map), the 7 CEL XValidation rules (6 on ProxySpec, 1 on CloudInitSpec), and pure helpers in helpers.go (EffectivePort/EffectiveHost/HealthCheckOrDefault/MaxLeasesOrDefault) with table-driven tests in helpers_test.go.

Regenerated deepcopy and the CRD:

make manifests generate
# controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
# controller-gen object:headerFile="hack/boilerplate.go.txt",year=2026 paths="./..."

Confirmed all 7 CEL rules and the healthCheck default: {} block landed in the generated CRD as expected:

grep -B1 "rule:" config/crd/bases/crawl.example.com_proxies.yaml
# 7 matches, one per XValidation marker written

Ran the full suite, not just go build/go vet, since this was a good opportunity to confirm envtest itself works end to end for the first time:

make test
# Setting up envtest binaries for Kubernetes version 1.36...
# .../bin/k8s/1.36.2-darwin-arm64   (confirms the plan's envtest version note)

This failed on the first run — not because of anything in the new types, but because kubebuilder's scaffolded placeholder test in proxy_controller_test.go creates a bare Proxy{} with no spec.mode, which our new required/enum field correctly rejects:

Proxy.crawl.example.com "test-resource" is invalid: [spec.mode: Unsupported value: "":
supported values: "Managed", "External", ...]

That's a real envtest apiserver enforcing our schema for the first time, which is useful confirmation on its own. Patched just the resource literal in that scaffold test to a minimal valid spec (Mode: External + Endpoint.Host) rather than rewriting the file — that whole test gets replaced in Step 4 alongside the real reconciler, so a deeper fix now would be thrown away. make test then passed clean: api/v1alpha1 at 20.5% coverage (helpers only — CEL itself isn't unit-testable, it's exercised by the real apiserver as shown above), internal/controller at 66.7%.

Worth noting: the go test ./... command from the plan's own verification section does not work directly for the envtest suite — it needs KUBEBUILDER_ASSETS set, which only make test does via setup-envtest. Plain go test ./... fails the internal/controller package with a /usr/local/kubebuilder/bin/etcd: no such file error that has nothing to do with the code. Use make test, not go test ./..., whenever the controller package is in scope.

Step 2 — Provider contract (internal/provider/)

Wrote the Provider interface (Create/Get/Delete/ListByTag), Instance, Placement, CreateRequest, and the GC-contract label constants (provider.go); the error taxonomy with multi-error Unwrap() []error so errors.Is and errors.As both work off the same wrapped value (errors.go); deterministic instance naming via SHA-256 → base32 → 16 chars (name.go); and --providers-config YAML parsing with fail-fast validation (config.go).

One deliberate deviation from the plan's file layout: the plan listed internal/provider/metrics.go as part of this step, but the provider.WithMetrics decorator it describes is Step 9's concern (it needs the Prometheus vectors that don't exist until the metrics package is built) and nothing in this step depends on it existing yet. Deferred to Step 9 rather than writing a decorator with nowhere to register its metrics.

The registry package (internal/provider/registry/registry.go) came out slightly different from the plan's sketch, and better for it: instead of a package-level var builtin = map[string]Constructor{"mock": mock.New, "gcp": gcp.New} living inside the registry package, Build takes the map[string]Constructor as a parameter. This means registry has zero import on internal/provider/mock or internal/provider/gcp — neither of which exists yet at this point in the plan (mock is Step 3, gcp is Step 8) — so the package compiles today instead of only once both are done, and the explicit wiring lives at the composition root (cmd/main.go, Step 10) rather than being smeared into the registry package itself. Still fully avoids the import-cycle trap the plan called out.

Ran the full suite:

go mod tidy   # sigs.k8s.io/yaml (already an indirect dep of the k8s.io toolchain) promoted to direct
go build ./... && go vet ./...
go test -race -v ./internal/provider/...
make test

internal/provider landed at 96.2% coverage, internal/provider/registry at 100%. make test also ran go fmt ./..., which reformatted errors.go's struct-field comment alignment before its first commit — no logic change, just gofmt on a brand-new file.

Worth noting: sigs.k8s.io/yaml (not gopkg.in/yaml.v3) was picked for --providers-config parsing specifically because it has UnmarshalStrict built in (rejects unknown fields, which is what "fail fast on unknown type" in the plan actually needs) and was already pulled in transitively by the k8s.io toolchain, so no new dependency was added — go mod tidy just promoted it from indirect to direct.

Step 3 — Mock provider (internal/provider/mock/)

Started from the plan's design (state as a pure function of an injectable clock, no background timers) but had to redesign the "real proxy" part before writing any code, once a load-bearing assumption turned out false.

The plan (and the earlier decision to make the mock run a real proxy) assumed each instance could get its own loopback address — 127.0.0.1:<port> per instance, "a fake IP from a private range." Verified that assumption directly before committing to it:

cat <<'EOF' > /tmp/loopbacktest.go
package main
import ("fmt"; "net")
func main() {
	for _, addr := range []string{"127.0.0.2:0", "127.0.0.55:0", "127.1.2.3:0"} {
		l, err := net.Listen("tcp", addr)
		if err != nil { fmt.Printf("%s: FAIL: %v\n", addr, err); continue }
		fmt.Printf("%s: OK\n", addr)
		l.Close()
	}
}
EOF
go run /tmp/loopbacktest.go
# 127.0.0.2:0: FAIL: listen tcp 127.0.0.2:0: bind: can't assign requested address
# 127.0.0.55:0: FAIL: listen tcp 127.0.0.55:0: bind: can't assign requested address
# 127.1.2.3:0: FAIL: listen tcp 127.1.2.3:0: bind: can't assign requested address

Only 127.0.0.1 binds on macOS without sudo ifconfig lo0 alias ... up — Linux routes the whole 127.0.0.0/8 block to loopback by default, macOS doesn't. That's not something the operator can or should do at runtime, so per-instance loopback IPs were out. There's also a second problem the per-instance-IP design didn't solve anyway: EffectivePort() (api/v1alpha1/helpers.go, Step 1) is computed purely from spec.port with no channel for a provider to report back a different port — so whatever a mock instance actually listens on has to be the literal port the reconciler will pass through CreateRequest.Port, not an OS-assigned ephemeral one.

Redesigned around one real listener per port, shared and reference-counted across every instance that uses it, rather than one listener per instance (proxy.go, sharedProxies). This fixes both problems at once: every instance binds the same 127.0.0.1 (no OS issue), and any number of instances can share a port without conflict since it's the exact same underlying listener. Deliberately made the refcounting package-level, not a field on mock.Provider, because a bound TCP port is a genuinely process-global OS resource — two separately configured mock-typed provider entries (e.g. two named "mock" instances in providers-config.yaml) would otherwise both try to bind the same default port and the second one would just fail. This is a case where a package global is the correct model, not a shortcut: it mirrors an OS-level singleton, not application state.

Instance lifecycle otherwise follows the plan exactly: Get/ListByTag derive Provisioning → Running → Terminated → purged (ErrNotFound) from createdAt/deletedAt compared against an injectable clock, with the real proxy listener acquired lazily on the first observed Running and released on Delete (or lazily on purge, so orphaned records can't leak a reference). Create is idempotent by name. Fault injection is wired both ways per the plan: MockConfig.FailNextCreates/FailWith for the demo config, InjectCreateFailures(n, class) for tests.

Tests initially had a real flake, caught by running with -race -count=3 rather than trusting one green run:

go test -race -v ./internal/provider/mock/... 2>&1 | tail -5
# --- FAIL: TestAcquireProxy_sharedAcrossAcquires
#     proxy_test.go:37: port not released after last reference: bind: address already in use

Root cause wasn't the refcounting logic — it was the test helper. freePort asked the OS for a free port by binding to :0 and immediately closing it, which is a classic TOCTOU race under t.Parallel(): two tests can be handed the same "free" port before either actually claims it, since nothing holds it open in between. Fixed by replacing the OS-asks approach with a monotonic counter (20000 + atomic.Int32) — these tests only need a port unique within this test run, not one verified free by the OS at an instant in time, so guaranteeing uniqueness outright is both simpler and correct where the "ask and hope" approach wasn't. Reran -race -count=3 clean afterward.

Added tests beyond the plan's list to close real coverage gaps rather than stopping at "green": config-override branches in New, all four failClassFromString branches, ListByTag's purge-on-list and Running/IP-inclusion paths, and a plain-http:// forwarding test (handleForward) alongside the CONNECT one, since a probeURL override could use either scheme. Landed at 91.1% coverage; the remainder is OS-failure branches (bind errors, hijack failures) not worth simulating for a prototype.

The TestProvider_realProxyTunnelsConnect test is the one that actually matters most here: it opens real sockets end to end — mock Create → Get past provisionDelay → real http.Client with Transport.Proxy dialing through the mock's CONNECT tunnel to a real httptest.NewTLSServer — and gets a real 204 back. That's the concrete proof the "mock runs a real proxy" decision actually delivers a genuine end-to-end healthcheck, not a simulated one.

internal/provider/mock at 91.1% coverage. make test green across the whole repo.

Step 3 (revised) — Kubernetes pod provider replaces the mock

After the mock provider above was built and working, the user pushed back: it felt too far from the real system to build confidence in, and they'd rather have simpler, more "real" code than a fast-but-simulated test double — a kind-based verification pass "once in a while" is an acceptable trade. They proposed replacing it outright with a provider that creates real Pods in the operator's own cluster, rather than keeping mock around as a fallback.

Talked through the trade-off before agreeing to it, since it wasn't free of downsides. Pods sharing a cluster's egress IPs don't solve what this operator actually exists for (routing around IP-based rate limiting needs genuinely distinct egress paths — that's still only gcp), and envtest has no kubelet, so a Pod-based provider can never be exercised by the fast test suite either. Net effect: this is a replacement for the mock's role in local dev/CI confidence-building, not a new alternative to GCP, and the reconciler's own state-machine tests (Step 4) still need a minimal in-test stub Provider — a handful of lines in the test file, not a package with its own config format or fault-injection surface, which is the specific kind of complexity the user was pushing back on.

Picked the proxy software by actually checking what's out there, not by guessing a Docker Hub path:

curl -s "https://hub.docker.com/v2/repositories/vimagick/tinyproxy/" | head -c 400
# last_updated 2021-07-22 — stale
curl -s "https://hub.docker.com/v2/repositories/ubuntu/squid/" | head -c 400
# last_updated 2026-08-07T04:36:14Z — updated the same day, Canonical-published, 50M+ pulls

ubuntu/squid won clearly: actively maintained (rebuilt the same day this check ran), official publisher, and — since it's a public image — kind nodes pull it directly, no build/load step needed for the quickstart. Pinned to 6.6-24.04_edge (Ubuntu 24.04 LTS base) rather than floating latest, for reproducibility.

Design, in internal/provider/kubernetes/:

• pod.go — buildPod is a pure function (mirrors the GCP provider's planned buildInsertRequest): builds a corev1.Pod with one Squid container. Config is generated in Go and passed via a SQUID_CONF env var that the container's command writes to /etc/squid/squid.conf before exec squid — deliberately not a separate ConfigMap, so there's still exactly one Kubernetes object per proxy instance to create, track, and clean up. The config itself is intentionally permissive (http_access allow all, via off, forwarded_for off) — documented in-code as a prototype-for-a-private-cluster choice, the same posture the spec already takes toward cloud-init on the GCP provider (installing/configuring proxy software is explicitly out of scope there; this provider doesn't try to do more).
• kubernetes.go — Create/Get/Delete/ListByTag against a client.Client. providerID is <namespace>/<podName>, parsed with k8s.io/client-go/tools/cache.SplitMetaNamespaceKey — the exact same reasoning as the GCP provider's planned zone-qualified providerID (Step 8): Get/Delete need to be self-contained without re-deriving where the resource lives. State mapping collapses Succeeded/Failed/ Unknown all into Terminated, since the reconciler already treats Stopped and Terminated identically (delete + recreate) — no finer distinction would change any behavior. Running with no PodIP yet maps to Provisioning, not Running, so an empty IP is never published — the same rule the plan already called out for GCP's RUNNING-without-NatIP case.
• Client construction is the one genuinely new pattern this provider needed that GCP/mock didn't: it builds its own client.Client via ctrl.GetConfig(), which auto-detects in-cluster config when running as a Pod and falls back to the local kubeconfig otherwise. That's what makes make run against a local kind cluster and running in-cluster use the exact same code path with zero provider-specific wiring in cmd/main.go. The corresponding risk: New() must never run under go test — it would happily connect to whatever real cluster the developer's kubeconfig points at. Solved the same way mock.Provider solved clock injection: an unexported newWithClient(c, cfg) constructor that tests call directly, bypassing ctrl.GetConfig() entirely. New() sits at 0% test coverage, deliberately — it's the one function that must stay untested by design.
• RBAC implication worth flagging now (implemented in Step 10): ListByTag lists Pods across every namespace, not just the namespace(s) Proxies live in, because orphan GC needs to find every Pod this operator tagged regardless of where it landed. That means the operator's Pod permissions have to be a ClusterRole, not scoped to a single namespace — a real trade-off against the "fleet lives in one namespace, keeps RBAC simple" principle the spec states for the CRD itself. Documented rather than special-cased away.
• One known simplification, documented in a code comment rather than solved: Kubernetes surfaces both RBAC-denied and quota-exceeded as the same 403 Forbidden, and apierrors has no helper to tell them apart. Both classify as ErrPermanent — the safer of the two defaults (stop retrying rather than hammering an API server that will never allow the request), but a real ResourceQuota failure that would clear once other proxies are deleted won't get the 5-minute-backoff-and-retry treatment ErrQuotaExceeded gives on the GCP path.

Testing: unlike the mock, this package's tests use sigs.k8s.io/controller-runtime/pkg/client/fake — real corev1.Pod objects, the real client.Client interface, not a hand-rolled in-memory map. That's a strictly more realistic test double than what mock.Provider was, while still being fast (no cluster, no kubelet) — it just can't prove a container actually starts and serves traffic, which is exactly the gap the kind-based verification pass is for. 77.6% coverage; the only meaningfully uncovered function is New() itself, deliberately.

make test green across the whole repo (go build/go vet clean, internal/provider 96.0%, internal/provider/kubernetes 77.6%, internal/provider/registry 100%, unchanged).

Cleanup — removed unused cert-manager/webhook scaffolding

Not a plan step; the user asked for this directly after I explained what make test-e2e currently does, and didn't want unused scaffold machinery carried forward into later work.

kubebuilder's generic scaffold assumes a project might grow admission webhooks later, so it wires up webhook TLS-cert machinery and an unconditional cert-manager install in the e2e suite defensively. Checked whether any of it was actually load-bearing before touching anything:

grep -rn "cert-manager\|certmanager\|CertManager" config/
# only inside commented-out [CERTMANAGER] blocks in kustomization.yaml —
# the whole block is inert, never uncommented
grep -n "webhook" PROJECT
# no output — kubebuilder create webhook was never run

Confirmed nothing here does anything for this project — no config/webhook/ exists, no +kubebuilder:webhook markers exist anywhere, and the spec's own non-goals explicitly rule out admission webhooks and cert-manager wiring forever, not just "not yet."

Removed:

• cmd/main.go: the webhook import, the three webhook-cert-* flags, the webhookServerOptions/webhookServer construction, and the WebhookServer: field on ctrl.Options — the manager runs with no webhook server at all now, which is correct since nothing registers one. Left the metrics-cert flags alone (--metrics-cert-path etc.) — those let real certs be mounted for the metrics endpoint without cert-manager, which is unrelated to webhooks and still useful. Reworded a comment that said "TODO(user): If you enable certManager..." since that will never happen here.
• test/e2e/e2e_suite_test.go: the unconditional cert-manager install in BeforeSuite and matching uninstall in AfterSuite (setupCertManager/teardownCertManager/shouldCleanupCertManager), and the doc comment claiming the suite "requires Kind and CertManager" (it only requires Kind now).
• test/utils/utils.go: InstallCertManager, UninstallCertManager, IsCertManagerCRDsInstalled, and their now-unused warnError helper and certmanagerVersion/certmanagerURLTmpl constants. Also removed UncommentCode — grepped first and confirmed it had zero callers even before this cleanup; it was dead scaffold code from the start, unrelated to cert-manager, just found while in there.

Left the inert commented-out [WEBHOOK]/[CERTMANAGER] blocks in the config/*/kustomization.yaml files and the +kubebuilder:scaffold:e2e-webhooks-checks-style marker comments in test/e2e/e2e_test.go alone — those are standard kubebuilder codegen anchors and pure comments with no runtime behavior, unlike the cert-manager install this cleanup actually removed. Stripping every trace of "webhook" from every scaffold comment across the tree would be a much bigger, purely cosmetic diff for no behavioral benefit; this cleanup targeted the things that were actually doing something.

Verified with both build tags, since test/e2e only compiles under -tags=e2e:

go build ./... && go vet ./...
go build -tags=e2e ./... && go vet -tags=e2e ./...

Both clean. make test green across the whole repo, unchanged from before the cleanup.

Cleanup — lean-down audit of Steps 0–3

Planned and approved separately in docs/plans/2026-08-08-1335-lean-scaffold-cleanup.md. Prompted by the user asking (a) why webhook machinery existed at all when the spec's §12 says "do NOT build: admission webhooks, cert-manager wiring", and (b) for a full audit of completed work so the project starts as lean as possible.

Why the webhook machinery existed — for the record. kubebuilder init emits it unconditionally: the active webhook-server wiring in cmd/main.go, cert-manager install in the e2e utils, and commented kustomize anchor blocks all arrive with init, not with create webhook (never run here). At Step 0 the scaffold was deliberately committed untouched as a reviewable baseline (only .github/ stripped), and spec §3's "config/ (scaffold-generated, kept working)" was read as license to keep the rest. The process gap: §12's non-goals deserved a pruning pass immediately after the baseline commit, especially for the parts that actually did something. The user caught it, not the build process.

Could init have skipped it? No — verified against the v4.15.0 binary, not from memory:

kubebuilder init --help
# flags: --domain --repo --owner --license(-file) --multigroup --namespaced
#        --fetch-deps --skip-go-version-check --project-version --plugins
# nothing subtracts features; optional plugins (helm, grafana, deploy-image,
# autoupdate) are all additive — there is no "minimal" plugin

Scaffold-then-prune is the only supported path to a lean baseline. Worth knowing at the next project bootstrap: plan the pruning pass as part of scaffolding, not as a later discovery. (--namespaced was the one arguably-applicable flag, but it conflicts with the kubernetes-pod provider's cluster-wide ListByTag for orphan GC — cluster-scoped was correct.)

Audit result for the Go code: clean. Nothing beyond spec that isn't a justified, already-logged deviation (Instance.UID/CreatedAt, MaxLeases *int32, HealthCheck default={}, registry-as-parameter, kubernetes provider per user decision). Trivial extras (shortName=px, MaxProperties=32) stay.

Scope lines drawn by the user during plan review — recorded because they shape what "lean" means for this repo going forward:

• Developer tooling is exempt: .golangci.yml, .custom-gcl.yml, .devcontainer/, AGENTS.md, Makefile lint/docker-buildx/ build-installer targets all stay ("tooling around the project is cool, i just want the application code produced to start as lean as possible").
• Monitoring manifests stay: all of config/prometheus/, plus the paired metrics-TLS-via-cert-manager plumbing (config/default/cert_metrics_manager_patch.yaml and the metrics-certs/ ServiceMonitor halves of the commented replacements block) — removing half of a pair would leave dangling comment references.
• All RBAC manifests stay, including the proxy_admin/editor/viewer helper ClusterRoles whose own headers say "not used by the project itself".
• Network policies go: "we do not need any network policies at the moment".

Removed:

• config/network-policy/ (kustomization.yaml, allow-metrics-traffic.yaml) plus its commented #- ../network-policy line in the default kustomization.
• Webhook-only remnants in config/default/kustomization.yaml: the commented #- ../webhook / #- ../certmanager resource lines, the manager_webhook_patch.yaml patch reference, and the webhook halves of the replacements block (serving-cert → Validating/Mutating WebhookConfiguration cainjection, conversion webhook, and the +kubebuilder:scaffold:crdkustomizecainjection* markers — anchors only for kubebuilder create webhook, permanently a non-goal).
• The two commented [WEBHOOK] blocks in config/crd/kustomization.yaml (conversion patches + the configurations: reference), along with the empty patches: key they lived under and the crdkustomizewebhookpatch marker. Kept the one-line crdkustomizeresource marker — kubebuilder create api could legitimately run again.
• config/crd/kustomizeconfig.yaml — only consumer was the removed configurations: block.

Verified: bin/kustomize build config/default and ... config/crd both render cleanly (no dangling references), go build/go vet clean with and without -tags=e2e, make test green with coverage numbers identical to pre-cleanup.

Step 4 — Reconciler (internal/controller/)

Implemented the state machine per the plan's action table: proxy_controller.go (dispatch + managed/external/delete paths, cloud-init resolution, spec-hash annotation persistence, Secret→Proxy watch mapping), status.go (condition reasons, computePhase, the single deferred patchStatusIfChanged), and spechash.go (explicit {placement, resolved cloud-init, port} hash input, SHA-256 hex). Tests: the action-table suite against a fake client with an in-test stubProvider (reconcile_test.go), computePhase truth table, spec-hash stability/normalization/sensitivity tables, and a rewritten envtest suite (proxy_controller_test.go) driving full lifecycles — provision→Running, spec-change replacement, finalizer deletion, External tracking — against the real apiserver with real CRD defaulting.

Deviation from the plan's Requeue: true rows: ctrl.Result{Requeue} is deprecated in controller-runtime v0.24 (verified in the vendored source, pkg/reconcile/reconcile.go: "Deprecated: Use RequeueAfter instead"), and golangci's staticcheck would flag it. Those rows use a fifth configurable interval instead, RequeueNow (default 1s) — same "process the next state promptly" semantics, still shrinkable in tests like the other four.

A real bug the new tests caught on their first run (both the fake-client and envtest suites, independently): in the replacement path's instance-is-gone branch, the status clear (ProviderID = "") was staged before setSpecHash's metadata Update — and client.Update refreshes the whole object from the server's response, including status, so the staged clear was silently overwritten and the proxy wedged with a stale providerID. Fix: stage status changes only after any metadata Update (the create branch already did it in that order). Worth remembering for every future reconciler: r.Update clobbers in-memory status staged before it.

Two judgment calls the plan left open, now documented in code:

• computePhase maps Provisioned=True with no Healthy verdict yet to Provisioning, not Ready — a proxy nobody has probed shouldn't be advertised as Ready. Health (Step 5) flips it.
• deletionFailure (the finalizer path's error handler) never latches ErrPermanent the way providerFailure does — latching there would wedge the object forever with no retry; it keeps retrying visibly instead.

The permanent-failure latch compares the condition's observedGeneration against the CR generation, so a spec edit automatically clears Failed and retries — no manual annotation-poking needed to recover.

Verification:

make test    # regenerates manifests (role.yaml gains secrets get;list;watch), envtest green
KUBEBUILDER_ASSETS="$PWD/bin/k8s/1.36.2-darwin-arm64" go test -race ./internal/controller/
go test -short ./internal/controller/   # 0.6s — envtest suite correctly skipped
go build -tags=e2e ./... && go vet -tags=e2e ./...

internal/controller at 75.9% coverage; the envtest suite runs in ~6s and is now guarded by testing.Short() per the testing conventions.

Worth noting: the envtest specs simulate instance state by mutating the stub between direct Reconcile calls rather than running the manager — deterministic and fast, at the cost of not exercising watch-driven requeues; Step 11's manager-driven cases cover that. The Secret watch is wired in SetupWithManager but the label-restricted Secret cache it assumes arrives with cmd/main.go in Step 10.

Step 5 — Health engine (internal/health/)

Implemented the engine per the plan's design: probe.go (through-the-proxy probe with the plan's exact transport — fresh per probe, DisableKeepAlives: true so every probe re-exercises CONNECT) and engine.go (leader-elected manager Runnable: 1 s scheduler tick + a pool of 8 workers, per-proxy threshold state under one mutex, transition-only emission over a buffered chan event.GenericEvent). The reconciler side landed in the same step: a HealthSnapshotter interface + applyHealth staging the Healthy condition/latency/lastHealthCheckTime from Engine.Snapshot, and a conditional WatchesRawSource(source.Channel(...)) in SetupWithManager. Everything tolerates nil (engine unwired) until cmd/main.go connects the two in Step 10.

Deviations and judgment calls beyond the plan text:

• First-probe scheduling is split by whether a verdict was seeded. The plan's startup jitter (nextDue = now + rand(0, interval)) applies only to proxies whose state was seeded from an existing Healthy condition — the restart case it exists for. A never-probed proxy is probed on the next tick instead; making a brand-new proxy wait up to a full interval for its first verdict would be pure lag with no thundering-herd benefit.
• The latency-change emission rule only applies while the verdict is healthy. Caught by the first test run, not foreseen: a success streak still below successThreshold (verdict unhealthy, reported unhealthy) satisfied the plan's rule (c) — latency delta vs a stale reported value, rate window open — and emitted a pointless latency-only update for a proxy still reported as unhealthy. Guarded with res.ok && *st.healthy.
• ProbeTLSConfig field added to the engine (nil = system roots). The probe function needs a CA override to be testable against httptest.NewTLSServer, and the same knob is genuinely useful for probing targets signed by a private CA. Not a test-only backdoor.
• State pruning doubles as replacement hygiene: any proxy with no probeable host (provisioning, mid-replacement, deleting) has its state dropped each tick, so a replacement instance always starts with fresh counters. Complementarily, the reconciler's create branch removes the stale Healthy condition and latency fields — a new VM shouldn't wear its predecessor's verdict.

Tests: a real CONNECT-capable proxy stub (hijack + bidirectional io.Copy) probing a real httptest.NewTLSServer — CONNECT success, refused CONNECT, unexpected status, dead proxy, plain-http forwarding; table-driven threshold/suppression/seeding/pruning tests driving record/tick directly; an end-to-end Start test (fake reader, fake probeFn, 5 ms tick) asserting event delivery, snapshot content, and clean shutdown on context cancel; and controller-side tests with a fakeSnapshotter proving Running+healthy ⇒ Ready, Running+unhealthy ⇒ Unhealthy, no-verdict ⇒ no condition, and stale-verdict cleanup on replacement.

Verification (all green):

make test                          # envtest + units; health 93.4%, controller 77.4%
KUBEBUILDER_ASSETS="$PWD/bin/k8s/1.36.2-darwin-arm64" go test -race ./...
go test -race -count=2 ./internal/health/   # shook out the emission-rule bug above

Worth noting: docs/architecture.md (created between Steps 4 and 5 on user request) gained a §6 for the engine and now shows the third workqueue feed (source.Channel). The Healthy condition reasons live in the controller package (ReasonProbeSucceeded/ReasonProbeFailed) — the engine deliberately knows nothing about conditions except reading one at seed time, keeping the state/representation split honest. The hint-driven wg.Go idiom (Go 1.25+) replaced the classic wg.Add/defer wg.Done in the worker pool.

Step 6 — Lease store (internal/lease/)

Implemented store.go per the plan: Acquire takes the whole candidate set so selection and insertion happen under the one store mutex (no overcommit between concurrent requests), selection is a linear scan + slices.SortFunc on (activeLeases asc, latency asc, name asc), AcquireStats{Considered, AtCapacity, InCooldown} feeds Step 7's 409 body, cooldowns live in a map[{proxy, target}]time.Time (empty target = global pool), and expired leases are retained for CooldownWindow past their TTL so a late Report — arriving exactly when a proxy is being rate-limited — still resolves and records its cooldown.

Semantics pinned against the spec (§8) rather than guessed:

• Report results are exactly ok | rate_limited | banned (ParseResult gives the API layer its 400 check). rate_limited and banned both record a cooldown for the same window; ok records nothing. Distinguishing ban duration from rate-limit duration would be a second knob the spec doesn't ask for — noted for the Decisions section.
• Cooldown scoping: the global cooldown (empty target) always applies; a target-scoped cooldown additionally blocks acquisitions for that target; acquisitions without a target see only the global pool ("a proxy rate-limited by one site is still fine for everyone else").
• A Report without a target falls back to the lease's own target before falling back to global — so a client that leased with a target doesn't accidentally poison the whole proxy by omitting it in the report.

Design notes:

• Correctness never depends on the sweep. Every read path (Acquire/ActiveCount/Counts) compares ExpiresAt against the injected clock, so TTL expiry frees capacity immediately even if the background loop hasn't run; the sweep is purely garbage collection. The plan's ExpireLoop became Start(ctx) + NeedLeaderElection() false so the store satisfies manager.Runnable directly — Step 10 just mgr.Add(store)s it. Not leader-elected because lease state is per-process and must expire wherever the discovery API is serving.
• The store knows nothing about Proxy objects — Candidate carries the opaque key, MaxLeases, and latency; the discovery layer does the health/attribute filtering. The spec's LeaseStore interface will be defined consumer-side in internal/discovery (Step 7), per Go idiom; this package exports only the concrete in-memory *Store.
• Lease IDs come from crypto/rand.Text() (Go 1.24+); returned Lease values are copies so callers can't mutate store internals.

Tests (94.8% coverage, -race -count=2 clean): capacity + release freeing slots, MaxLeases=0 unleasable, least-loaded/latency/name selection order, target-scoped vs global cooldown scoping, cooldown expiry via the injected fake clock, TTL freeing capacity with no sweep, report-on-expired-but-retained lease (then ErrUnknownLease after retention), ok recording nothing, idempotent release, ParseResult, 40 concurrent acquires against MaxLeases=5 granting exactly 5, and the Start loop sweeping then stopping cleanly on cancel.

go test -race -count=2 ./internal/lease/
make test    # whole repo green, other packages' coverage unchanged

Worth noting: docs/architecture.md was not extended this step — the lease store is HTTP-driven, not cluster-event-driven, so its diagram belongs with the discovery API and lands in Step 7 (banner updated to say so).

Step 7 — Discovery API (internal/discovery/)

Implemented server.go (Runnable + middleware chain) and handlers.go (the four endpoints + proxyView wire shape) per the plan: stdlib http.ServeMux method+wildcard routing (no third-party router — see the plan clarification commit: this is a stdlib feature since Go 1.22, the project stays on the pinned Go 1.26), middleware outermost-first recover → request-log → MaxBytesReader(64KiB) → constant-time bearer auth with /healthz exempt, empty DISCOVERY_TOKEN serving unauthenticated with a loud startup warning, NeedLeaderElection() = false with the plan's runnable-ordering rationale in the doc comment, and graceful Shutdown with a 10 s grace on ctx cancel.

The LeaseStore interface landed consumer-side in this package (spec §8 wants handlers swappable to a CRD/Redis store); internal/lease.*Store satisfies it without modification.

Judgment calls the plan/spec left open:

• 409 arithmetic: the store only ever sees healthy candidates, so its Considered excludes unhealthy matches. The handler counts unhealthy selector-matches itself and reports considered = healthy + unhealthy, keeping the plan's example arithmetic (7 = 2+2+3) consistent.
• TTL handling: omitted/zero ttlSeconds → 300 s default; negative or above MaxLeaseTTL (default 1h, flag in Step 10) → 400 invalid_ttl rather than silent clamping — a client asking for a week-long lease should find out, not get an hour quietly.
• Grant response includes the fresh activeLeases (the just-granted lease counted), read back via Store.Counts() after the acquire.
• Proxies with a deletionTimestamp are filtered out of both list and candidate selection — a proxy mid-teardown shouldn't be advertised.

Tests (87.3% coverage, -race -count=2 clean, green on first run): httptest over the real handler chain with a fake cache reader and a real lease.Store — auth on/off/wrong-token/healthz-exempt, list filtering (attributes, healthy, combined, empty-is-200), grant shape (201, default TTL, lowest-latency pick, RFC3339 expiresAt, activeLeases=1), the full 409 body arithmetic, invalid TTL/body/result, idempotent 204 release, report→cooldown→409 round-trip, 404 on unknown lease, and a real Start on 127.0.0.1:0 (via the new BoundAddr() accessor) serving healthz then shutting down cleanly on cancel.

go test -race -count=2 ./internal/discovery/
make test    # whole repo green

Worth noting: go mod tidy promoted github.com/go-logr/logr from indirect to direct (the server holds a logr.Logger field). The --discovery-addr, --max-lease-ttl flags and the DISCOVERY_TOKEN Secret mount arrive with cmd/main.go in Step 10. docs/architecture.md gained §7 covering the whole HTTP path and the store's sweep Runnable.

Step 8 — GCP provider (internal/provider/gcp/)

Implemented per the plan: gcp.go (Provider + the flattened instancesAPI test seam + providerID handling + state mapping), insert.go (pure buildInsertRequest + config defaults), errors.go (HTTP-code → taxonomy classification). Only the four calls the spec allows — instances Insert / Get / Delete / AggregatedList — and Operation.Wait is never called: Create/Delete return once the operation is submitted, 409 alreadyExists on Insert and 404 on Delete both count as success, which is what makes repeat calls after a crash correct.

Dependency added (the plan's environment check pinned it):

go get cloud.google.com/go/compute@v1.65.0 google.golang.org/api@latest
# resolved google.golang.org/api v0.292.0; go mod tidy pulled the auth/gax chain

Key shapes, all straight from the plan:

• providerID zones/<zone>/instances/<name> — Get/Delete parse the zone out of the ID instead of re-reading spec.placement.zone, which is wrong exactly when a zone edit is the replacement being processed.
• The seam is not an SDK mirror — verified the plan's premise against the vendored source before designing around it: InstancesScopedListPairIterator has an unexported nextFunc, so a fake cannot construct one. The seam flattens AggregatedList to []*computepb.Instance and returns operations as just their name.
• AggregatedList sets ReturnPartialSuccess: true (one unreachable zone must not fail a GC sweep) and filters by labels.proxy-operator-managed = true.
• RUNNING without a NatIP maps to Provisioning — never publish an empty IP. Unknown/new GCP statuses map to Stopped: the reconciler's response is delete-and-recreate, always safe for cattle.
• Classification: 404→NotFound; 429 and 403-with- quotaExceeded/rateLimitExceeded→Quota; 400/401/403-other→Permanent; everything else (408, 5xx, network, unknown)→Transient.

Judgment call: placement.zone/machineType/image are all required at Create — missing values fail as ErrPermanent with a message naming the empty fields, rather than inventing defaults the spec doesn't define. A wrong guess here would silently create billable VMs of an arbitrary shape; a Failed condition telling the user what to set is strictly better.

Tests (75.2%, -race -count=2 clean, green on first run): the plan's primary field-by-field buildInsertRequest assertion (machine-type URL, boot disk, the exact {External NAT, ONE_TO_ONE_NAT} access config, user-data metadata, GC labels, network tag) plus config overrides and no-metadata-without-cloud-init; the full classification table including errors.Is AND errors.As through the multi-unwrap; and fake-seam tests for zone-qualified IDs, 409-is-success, permanent-on-bad-placement (no API call made), the nine-row state-mapping table, 404 paths, malformed providerIDs, and the ListByTag filter/partial-success assertions. The uncovered remainder is New() (dials real Google with ADC) and the realInstances adapter — the same deliberately-untested posture as the kubernetes provider's New().

Worth noting: gopls suggested replacing proto.String(x) with Go 1.26's new(x) expression; left as proto.String — it is the universal protobuf-construction idiom and matches every example in the SDK docs. Registry wiring ("gcp": gcp.New) happens at the composition root in Step 10, as designed in Step 2. docs/architecture.md §5 now shows both providers' call mappings.

Step 9 — Orphan GC + metrics

Implemented internal/gc/gc.go (the Sweeper Runnable) and internal/metrics/metrics.go (explicit-registration metric set), plus the provider.WithMetrics decorator deferred from Step 2 into internal/provider/metrics.go, and the observation hooks in the health engine and discovery server.

GC, per the plan: NeedLeaderElection() = true (destructive ⇒ single writer), 10 min ticker with the first sweep one full interval after start, per-provider ListByTag with log-and-continue on provider errors, and a kill requires all three of: our UID label present, older than MinAge (10 min), and the UID matching no existing CR — where a CR with a deletionTimestamp still counts as live (its finalizer owns that deletion; GC racing it would double-delete). Two safety behaviors worth naming: a failed Proxy List skips the whole sweep (an unreadable live set proves nothing orphaned), and the namespace-scope guard makes Start refuse to run against a namespace-restricted cache unless --gc-allow-namespaced is explicit, with the flag named in the error. One deviation of record: the plan says kills log "at Warn", but logr has no Warn level — kills log at Info with a WARNING: prefix carrying provider, providerID, and UID, same convention as the discovery server's empty-token warning.

Metrics, per the plan: no init() — Metrics.Register(reg, phases, leases) is called explicitly by the composition root (Step 10), which is also what lets every test use a fresh registry (asserted by a test that registers two sets on two registries). proxy_operator_proxies{phase} and proxy_operator_leases_active are scrape-time collectors fed by closures — a reconcile-incremented gauge drifts and leaks series on delete; reading the source of truth at scrape time cannot. The probe vectors observe every probe (status stays transition-only; metrics carry the high-frequency signal), and the health engine calls ForgetProxy when it prunes a state entry so per-proxy series don't leak. provider_requests_total{provider,op,result} comes from the WithMetrics decorator — the one place Class() is called purely for observability, with results labelled ok / not_found / quota_exceeded / permanent / transient.

Decoupling shape: each consuming package defines its own small recorder interface (health.ProbeMetrics, discovery.LeaseMetrics, provider.RequestRecorder); metrics.Metrics satisfies all of them structurally. Only cmd/main.go will import internal/metrics.

Tests (-race -count=2 clean): GC's true-orphan matrix in one sweep (live kept, deleting-CR kept, young kept, unlabelled kept, orphan deleted), broken-provider isolation, list-failure skips sweep, the namespace guard both ways, and the Start loop sweeping then stopping; metrics via prometheus/testutil — GatherAndCompare on the scrape-time collectors, ForgetProxy dropping series, outcome/result label counts; the decorator's five-way classification table with error passthrough asserted. Coverage: gc 86.1%, metrics 95.0%, provider up to 97.1%; health/discovery re-ran green with the hooks in place.

go test -race -count=2 ./internal/gc/ ./internal/metrics/ ./internal/provider/ ./internal/health/ ./internal/discovery/
make test    # whole repo green

Worth noting: prometheus/client_golang was already in the module via controller-runtime's metrics server, so no new dependency — go mod tidy just promoted it to direct. docs/architecture.md gained §8 (GC sweep) and §9 (metrics shape).

Step 10 — Wiring, config, docs

The composition root and everything around it. cmd/main.go now: parses the plan's flag set (plus --gc-allow-namespaced from Step 9), loads the provider config first and fails fast, builds the registry with {"kubernetes": kubernetes.New, "gcp": gcp.New}, wraps every provider in provider.WithMetrics, then adds the lease store, health engine, discovery server, and GC sweeper to one manager and hands the reconciler its providers + health snapshotter + events channel. Metrics register on controller-runtime's global registry with scrape-time closures (phase counts from the cache, active leases summed from store.Counts()).

Two cache decisions became concrete here:

• The Secret cache is restricted to Secrets labelled crawl.example.com/cloud-init=true (new constant v1alpha1.LabelCloudInit) — the operator holds cluster-wide Secret read RBAC, and an unrestricted cache would hold every Secret in scope. Consequence documented in the README: an unlabelled referenced Secret is invisible → CloudInitError.
• --proxy-namespace restricts the whole cache via DefaultNamespaces and flips the GC sweeper's NamespaceRestricted guard.

Manifests: config/manager/manager.yaml gained the --providers-config arg, the optional DISCOVERY_TOKEN secretKeyRef (optional: true — without the Secret the API runs unauthenticated with its loud warning), the ConfigMap volume mount, and containerPort 8090; new config/manager/providers_config.yaml (kubernetes-only default) and config/default/discovery_service.yaml. RBAC: the pods marker landed in the controller RBAC block (cluster-scoped role — the kubernetes provider's ListByTag spans namespaces). The plan's events RBAC was deliberately omitted: nothing wires an EventRecorder, and unused verbs are lint noise — recorded in Decisions.

Samples: proxy_kubernetes.yaml / proxy_gcp.yaml (with a working Squid-installing cloud-init) / proxy_external.yaml replace the scaffold placeholder; providers-config.yaml documents both provider blocks; hack/providers-dev.yaml + a new run-dev Makefile target run locally with plain-HTTP metrics:

make run-dev
# go run ./cmd/main.go --providers-config hack/providers-dev.yaml \
#   --metrics-bind-address :8080 --metrics-secure=false

Docs: README rewritten per the plan (60-second architecture, kind quickstart, in-cluster deploy incl. token Secret creation, GCP setup with the IAM roles, the two prominent caveats, the no-substitutions version pins note). docs/architecture.md gained the components table and the full Decisions section — the plan's listed decisions plus everything accumulated in this log (RequeueNow, quota≠Failed, 409 arithmetic, TTL-400-not-clamp, GCP required placement, unknown-status→Stopped, banned==rate_limited window, GC logging convention, the Secret label contract, events-RBAC omission, logr-not-slog). CHANGELOG.md got its first entry with a real timestamp.

Verified: make test green across the repo (coverage unchanged), bin/kustomize build config/default and config/samples render clean, make build produces the binary, e2e-tagged build + vet clean. The kind end-to-end run is deliberately still ahead — it is the Verification step's job, after Step 11 closes the remaining test gaps.

Worth noting: make run-dev passes --metrics-secure=false because the scaffold's secure-serving default requires authn/authz reachability that a local process doesn't have; in-cluster deployments keep the secure default from the kustomize patch. The providers map wrapping happens before any consumer sees it, so the reconciler and GC only ever hold instrumented providers.

Step 11 — Tests

Most of the plan's Step 11 list was deliberately front-loaded into the step that built each component (the action-table suite, computePhase and SpecHash tables, lease-store matrix incl. the concurrent -race case, discovery httptest suite, health threshold/CONNECT tests, GCP buildInsertRequest + classification, name-derivation tests from Step 2). This step closed what remained — the envtest-only coverage — and audited the list item by item.

Added to internal/controller/proxy_controller_test.go:

• The CEL cases only a real API server can test (fake clients run neither CEL nor structural defaulting): six invalid-create rejections (Managed-without-provider, External-with-provider, External-without-endpoint, Managed-with-endpoint, cloudInit both/neither), mode-mutation rejection, provider mutation and removal rejection (the has(self.x)==has(oldSelf.x) form exists exactly for the removal case), and the +kubebuilder:default={} assertion — a Proxy created with no healthCheck comes back with every nested default materialized, plus port and maxLeases defaults.
• Ready-through-health: Managed proxy walks to Running (phase still Provisioning — "no health verdict yet must not be Ready"), then a fake HealthSnapshotter supplies a healthy snapshot and the phase flips to Ready with latency in status.
• Quota + permanent, envtest edition: quota → condition QuotaExceeded, RequeueAfter = QuotaRetry, nil error, phase not Failed; then permanent → Failed and the generation latch provably stops further provider calls.
• Adopt: strip the spec-hash annotation off a Running proxy (as an operator upgrade with a changed hash-input struct would), reconcile, and assert the annotation is restored byte-identical, the providerID unchanged, and zero provider deletes.

One repo-wide change: make test now runs with -race (the plan's "everything runs with -race" was previously only true of the manual verification runs, not the canonical target):

go test -race $$(go list ./... | grep -v /e2e) -coverprofile cover.out

Everything green on the first run of the new specs; full suite ~9 s for the controller package under race, -short still skips envtest in 0.6 s.

Worth noting: the provider-removal CEL test has a subtlety worth keeping — removing provider alone would also trip the required-iff rule, so the test flips mode and adds an endpoint in the same update to isolate the immutability rules as the thing that rejects. The plan's remaining checklist item is Verification: the throwaway-kind-cluster run of the README quickstart, then push + MR.

Verification — kind end-to-end

Static checks first (go vet ./..., make build, full make test with -race): all green. Then the real thing, per the spec's §13 "run the kind quickstart yourself and fix what breaks" — and two things broke, both now fixed.

Finding 1 — make run-dev cannot produce a Ready proxy on kind. The operator on the host provisions the pod fine (Provisioned=True, IP published), but the health probe originates on the host, and kind pod IPs (10.244.x.x) are not host-routable — every probe fails by construction and the proxy latches Unhealthy:

Healthy=False: Get "https://www.gstatic.com/generate_204":
  proxyconnect tcp: dial tcp 10.244.0.5:3128: connect: connection refused

Everything around the failure worked exactly as designed (thresholds, condition, phase, and the finalizer delete ran clean from the host). Fix: the README quickstart now deploys the operator in-cluster (docker-build → kind load → deploy → port-forward 8090), with the run-dev limitation documented in both the quickstart and the Development section.

Finding 2 — Squid was OOM-killed at startup in-cluster. With the operator deployed in-cluster the pod crash-looped (OOMKilled, empty logs). Root cause: squid sizes its file-descriptor tables from RLIMIT_NOFILE, and containerd under kind sets that effectively unlimited (~10^9) — squid allocates gigabytes before it ever listens. Fix in the generated config (internal/provider/kubernetes/pod.go): max_filedescriptors 1024 (the load-bearing line) plus cache_mem 16 MB (a crawling forward proxy gains nothing from squid's 256 MB default), with a regression assertion added to pod_test.go.

With both fixes, the full pass:

kind create cluster --name proxy-operator-demo
make install
make docker-build IMG=egress-proxies-operator:dev
kind load docker-image egress-proxies-operator:dev --name proxy-operator-demo
make deploy IMG=egress-proxies-operator:dev
kubectl apply -f config/samples/proxy_kubernetes.yaml
# → Ready 10.244.0.9 lat=89ms Provisioned=True Healthy=True   (~30 s)
kubectl -n egress-proxies-operator-system port-forward svc/...-discovery-service 8090:8090 &
curl -s 'localhost:8090/v1/proxies?healthy=true'          # count:1, latencyMillis:89
curl -s -XPOST localhost:8090/v1/leases -d '{"selector":{"geo":"local"},"ttlSeconds":300}'
#   → 201 {leaseID, proxy(activeLeases:1), expiresAt, ttlSeconds:300}
curl -XPOST .../report -d '{"result":"rate_limited","target":"example.com"}'   # 204
curl -XPOST /v1/leases -d '{...,"target":"example.com"}'  # 409 {inCooldown:1} ✓
curl -XDELETE /v1/leases/<id>   # 204, and 204 again ✓
kubectl delete -f config/samples/proxy_kubernetes.yaml    # finalizer: pod Terminating, CR gone
kind delete cluster --name proxy-operator-demo

A real Squid pod went Ready through a real CONNECT probe, a lease was held on it, the cooldown machinery answered a 409 with correct arithmetic, and the finalizer cleaned up — the plan's success bar, met with the actual product.

Worth noting: make deploy runs kustomize edit set image and mutates config/manager/kustomization.yaml in the working tree — reverted before committing (the repo keeps the pinned stanza). The health probe's ~89 ms latency is gstatic-through-squid from a kind pod on this machine; metrics-side observations were not separately checked in-cluster (covered by unit tests).