egress-proxies-operator/CHANGELOG.md

Changelog

2026-09-10 22:58 UTC — Code-review fixes on the KVM branch: deletion, validation, lazy SSH, guest agent

Five findings from a review of feat/kvm-provider, all fixed on the branch.

• A Proxy can always finish deleting. The delete path used to drop the finalizer only when Get answered not-found, so a permanent error there retried forever — a KVM host removed from the config was enough. It now calls Delete anyway, since that is idempotent and can finish where Get cannot, and crawl.example.com/skip-cleanup: "true" abandons the instance for whatever stays stuck. In the KVM provider, deleting over an unconfigured host is a no-op success; Get still fails permanently there so a typo cannot look like a missing VM.
• The guest agent ships as a vendor-data script, superseding the cloud-config merge in the entry below. Merging could never be made safe: cloud-init drops vendor-data keys the user-data defines, and the strategy cannot be overridden from either document. The Proxy's cloud-init now reaches the seed byte for byte in any format — Jinja, MIME, gzip, scripts — where the round-trip previously re-typed unquoted scalars (version: 1.10 arrived as 1.1), dropped comments and reordered keys. New installGuestAgent setting, default on.
• A missing kvm-ssh Secret no longer crash-loops the manager. The SSH material is read on first use and cached, so only KVM Proxies fail and they recover when the Secret appears. A missing file is transient, an unparseable key permanent, and startup still logs a warning.
• Host names and image keys are validated. Either can end up in a providerID or a volume file name, where a slash made the VM unaddressable for good.
• Verified end to end: unit suite, the integration test against a real hypervisor (22 s, agent reporting the IP), both in-cluster Proxies Ready in 2 min with 204 through squid, the guest showing qemu-guest-agent active with its cloud-config comments intact, and the manager staying up with the Secret pointed away.
• Key files: internal/provider/kvm/{seed.go,conn.go,kvm.go}, internal/controller/proxy_controller.go, internal/provider/config_kvm.go, api/v1alpha1/proxy_types.go.

2026-09-10 21:04 UTC — KVM provider passes in-cluster; guest agent merged into cloud-config

• Two KVM Proxies with real cloud-configs reach Ready from inside the kind cluster (2 min from apply, IP via the guest agent, 204 through squid from the LAN).
• The qemu-guest-agent bootstrap is now merged into each Proxy's cloud-config (packages + runcmd) instead of shipped as vendor-data: cloud-init's default merger drops vendor-data keys the user-data also defines, so any Proxy with its own packages: never got the agent and stayed Provisioning forever. vendorData is now plain extra vendor-data with no default; non-cloud-config user-data (scripts, #include) still gets the bootstrap as vendor-data. A cloud-config that is not a YAML mapping fails the Proxy permanently instead of booting without its config.
• The manager pod mounts a 64 MiB emptyDir on /tmp: the seed ISO writer stages files on disk and the root filesystem is read-only.
• Errors already classified with a provider sentinel no longer drop the cached libvirt SSH connection; only libvirt and transport errors do.
• Key files: internal/provider/kvm/{seed.go,conn.go,errors.go}, config/manager/manager.yaml, docs/kvm-provider.md.

2026-09-10 10:07 UTC — KVM provider validated on a real hypervisor; UEFI default; libvirt call tracing

• make test-integration-kvm passes end to end against a libvirt 11.6 / QEMU 10.1 host (Debian 13 image, UEFI, guest agent reporting the IP after ~20 s). Four defects found only on the real host, all fixed on the branch:
  • domain disks are now type="file" with resolved paths: pool-volume references get no AppArmor rule from virt-aa-helper, so QEMU was denied the disks;
  • domain metadata is parsed by local name: libvirt returns the element without its namespace declaration;
  • guests boot UEFI by default (images.<key>.firmware: uefi|bios, OVMF now a host prerequisite): Debian 13's image hangs in GRUB under SeaBIOS on a headless VM with a serial console (Debian bug #1111240, open); bookworm is unaffected;
  • the image cache's raw-upload path is confirmed to produce a byte-identical file.
• Every libvirt RPC is logged at --zap-log-level=debug as one libvirt call line (rpc=virDomainCreate, identifiers, duration, libvirtError on failure); level 2 adds XML, metadata and addresses; cloud-init is never logged. The integration test shows the trace in -v output (KVM_TEST_LOG_LEVEL, default 1).
• Key files: internal/provider/kvm/{domainxml.go,metadata.go,trace.go,conn.go}, internal/provider/config_kvm.go, docs/kvm-provider.md.

2026-09-08 11:00 UTC — KVM/libvirt provider

• New kvm provider type: Managed proxies as VMs on your own libvirt/KVM hosts. Each host is reached over SSH with the libvirt unix socket forwarded through the connection (pure Go, nothing to expose on the hypervisors beyond sshd); one provider instance can span several hosts, selected by placement.zone.
• Base images (images: map or a direct URL in placement.image) are downloaded by the operator into each host's storage pool on first use, verified by sha256 when configured, and cached there. Per-VM disks are qcow2 overlays; cloud-init arrives on a NoCloud seed ISO built in-process. GC identity lives in the domain's <metadata> element.
• Wiring: "kvm": kvm.New in cmd/main.go; optional kvm-ssh Secret mounted at /etc/proxy-operator/kvm; samples config/samples/proxy_kvm.yaml and a kvm-home block in providers-config.yaml; docs/kvm-provider.md; opt-in integration test (make test-integration-kvm) against a real hypervisor.
• Not validated on a real host at the time of this entry; see 2026-09-10 above.

2026-08-10 09:34 CEST — kind e2e verified; fix Squid OOM in containers; quickstart goes in-cluster

• Full end-to-end pass on a throwaway kind cluster: Squid pod Ready with a real CONNECT probe (89 ms), lease grant/report/cooldown-409/release through the discovery API, finalizer cleanup on delete.
• Fixed the kubernetes provider's generated squid.conf: max_filedescriptors 1024 (squid sizes FD tables from the container's effectively-unlimited RLIMIT_NOFILE and was OOM-killed at startup under kind/containerd) + cache_mem 16 MB.
• README quickstart now deploys the operator in-cluster: make run-dev on a laptop cannot reach kind pod IPs, so health probes fail by construction there (documented).

2026-08-09 17:27 CEST — Operator wired end to end: reconciler, health, leases, discovery, GC, two providers

• cmd/main.go is now the full composition root: --providers-config (required, fail-fast), --discovery-addr, --proxy-namespace, --health-workers, --gc-interval, --gc-min-age, --gc-allow-namespaced, --lease-cooldown, --max-lease-ttl; wires the kubernetes + gcp providers (metrics-instrumented), health engine, lease store, discovery API, orphan GC, and Prometheus metrics onto one manager.
• Deploy manifests: providers ConfigMap mount, optional DISCOVERY_TOKEN Secret env, discovery port 8090 + Service; pods RBAC for the kubernetes provider.
• Samples for all three proxy flavors + providers-config; make run-dev for local development.
• README rewritten (kind quickstart, GCP setup, the two load-bearing caveats); architecture doc completed with components table and the full decision log.