Files
egress-proxies-operator/CHANGELOG.md
T
kacerrandClaude Fable 5.1 366736f53f Merge the qemu-guest-agent bootstrap into each Proxy's cloud-config instead of vendor-data
In the kind cluster two KVM Proxies with real cloud-configs booted fine
but never left Provisioning: the guest agent was not installed, so the
provider could not learn the IP. cloud-init merges vendor-data under
user-data with its default mergers, which keep the user-data's value
for any key both define and never append lists, so a cloud-config with
its own `packages:` silently dropped the vendor-data's packages and
runcmd. The integration test passed only because its user-data was
empty.

withGuestAgent now appends qemu-guest-agent to `packages` and the
enable command to `runcmd` of the Proxy's own cloud-config (skipping
entries already present) and re-renders it. Vendor-data is no longer
defaulted; `vendorData` is plain extra data. User-data that is not a
cloud-config document (script, #include, MIME) still gets the bootstrap
as vendor-data. A cloud-config that is not a YAML mapping, or whose
packages/runcmd is not a list, fails the Proxy as ErrPermanent.

Validated in-cluster: both Proxies Ready and healthy in two minutes, 204
through squid from the LAN. Docs, sample comment, CHANGELOG and the
plan execution log (Run 7) updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 21:06:54 +00:00

5.3 KiB

Changelog

2026-09-10 21:04 UTC — KVM provider passes in-cluster; guest agent merged into cloud-config

  • Two KVM Proxies with real cloud-configs reach Ready from inside the kind cluster (2 min from apply, IP via the guest agent, 204 through squid from the LAN).
  • The qemu-guest-agent bootstrap is now merged into each Proxy's cloud-config (packages + runcmd) instead of shipped as vendor-data: cloud-init's default merger drops vendor-data keys the user-data also defines, so any Proxy with its own packages: never got the agent and stayed Provisioning forever. vendorData is now plain extra vendor-data with no default; non-cloud-config user-data (scripts, #include) still gets the bootstrap as vendor-data. A cloud-config that is not a YAML mapping fails the Proxy permanently instead of booting without its config.
  • The manager pod mounts a 64 MiB emptyDir on /tmp: the seed ISO writer stages files on disk and the root filesystem is read-only.
  • Errors already classified with a provider sentinel no longer drop the cached libvirt SSH connection; only libvirt and transport errors do.
  • Key files: internal/provider/kvm/{seed.go,conn.go,errors.go}, config/manager/manager.yaml, docs/kvm-provider.md.

2026-09-10 10:07 UTC — KVM provider validated on a real hypervisor; UEFI default; libvirt call tracing

  • make test-integration-kvm passes end to end against a libvirt 11.6 / QEMU 10.1 host (Debian 13 image, UEFI, guest agent reporting the IP after ~20 s). Four defects found only on the real host, all fixed on the branch:
    • domain disks are now type="file" with resolved paths: pool-volume references get no AppArmor rule from virt-aa-helper, so QEMU was denied the disks;
    • domain metadata is parsed by local name: libvirt returns the element without its namespace declaration;
    • guests boot UEFI by default (images.<key>.firmware: uefi|bios, OVMF now a host prerequisite): Debian 13's image hangs in GRUB under SeaBIOS on a headless VM with a serial console (Debian bug #1111240, open); bookworm is unaffected;
    • the image cache's raw-upload path is confirmed to produce a byte-identical file.
  • Every libvirt RPC is logged at --zap-log-level=debug as one libvirt call line (rpc=virDomainCreate, identifiers, duration, libvirtError on failure); level 2 adds XML, metadata and addresses; cloud-init is never logged. The integration test shows the trace in -v output (KVM_TEST_LOG_LEVEL, default 1).
  • Key files: internal/provider/kvm/{domainxml.go,metadata.go,trace.go,conn.go}, internal/provider/config_kvm.go, docs/kvm-provider.md.

2026-09-08 11:00 UTC — KVM/libvirt provider

  • New kvm provider type: Managed proxies as VMs on your own libvirt/KVM hosts. Each host is reached over SSH with the libvirt unix socket forwarded through the connection (pure Go, nothing to expose on the hypervisors beyond sshd); one provider instance can span several hosts, selected by placement.zone.
  • Base images (images: map or a direct URL in placement.image) are downloaded by the operator into each host's storage pool on first use, verified by sha256 when configured, and cached there. Per-VM disks are qcow2 overlays; cloud-init arrives on a NoCloud seed ISO built in-process. GC identity lives in the domain's <metadata> element.
  • Wiring: "kvm": kvm.New in cmd/main.go; optional kvm-ssh Secret mounted at /etc/proxy-operator/kvm; samples config/samples/proxy_kvm.yaml and a kvm-home block in providers-config.yaml; docs/kvm-provider.md; opt-in integration test (make test-integration-kvm) against a real hypervisor.
  • Not validated on a real host at the time of this entry; see 2026-09-10 above.

2026-08-10 09:34 CEST — kind e2e verified; fix Squid OOM in containers; quickstart goes in-cluster

  • Full end-to-end pass on a throwaway kind cluster: Squid pod Ready with a real CONNECT probe (89 ms), lease grant/report/cooldown-409/release through the discovery API, finalizer cleanup on delete.
  • Fixed the kubernetes provider's generated squid.conf: max_filedescriptors 1024 (squid sizes FD tables from the container's effectively-unlimited RLIMIT_NOFILE and was OOM-killed at startup under kind/containerd) + cache_mem 16 MB.
  • README quickstart now deploys the operator in-cluster: make run-dev on a laptop cannot reach kind pod IPs, so health probes fail by construction there (documented).

2026-08-09 17:27 CEST — Operator wired end to end: reconciler, health, leases, discovery, GC, two providers

  • cmd/main.go is now the full composition root: --providers-config (required, fail-fast), --discovery-addr, --proxy-namespace, --health-workers, --gc-interval, --gc-min-age, --gc-allow-namespaced, --lease-cooldown, --max-lease-ttl; wires the kubernetes + gcp providers (metrics-instrumented), health engine, lease store, discovery API, orphan GC, and Prometheus metrics onto one manager.
  • Deploy manifests: providers ConfigMap mount, optional DISCOVERY_TOKEN Secret env, discovery port 8090 + Service; pods RBAC for the kubernetes provider.
  • Samples for all three proxy flavors + providers-config; make run-dev for local development.
  • README rewritten (kind quickstart, GCP setup, the two load-bearing caveats); architecture doc completed with components table and the full decision log.