kacerrandClaude Fable 5.1
366736f53f
In the kind cluster two KVM Proxies with real cloud-configs booted fine but never left Provisioning: the guest agent was not installed, so the provider could not learn the IP. cloud-init merges vendor-data under user-data with its default mergers, which keep the user-data's value for any key both define and never append lists, so a cloud-config with its own `packages:` silently dropped the vendor-data's packages and runcmd. The integration test passed only because its user-data was empty. withGuestAgent now appends qemu-guest-agent to `packages` and the enable command to `runcmd` of the Proxy's own cloud-config (skipping entries already present) and re-renders it. Vendor-data is no longer defaulted; `vendorData` is plain extra data. User-data that is not a cloud-config document (script, #include, MIME) still gets the bootstrap as vendor-data. A cloud-config that is not a YAML mapping, or whose packages/runcmd is not a list, fails the Proxy as ErrPermanent. Validated in-cluster: both Proxies Ready and healthy in two minutes, 204 through squid from the LAN. Docs, sample comment, CHANGELOG and the plan execution log (Run 7) updated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
80 lines
5.3 KiB
Markdown
80 lines
5.3 KiB
Markdown
# Changelog
|
|
|
|
## 2026-09-10 21:04 UTC — KVM provider passes in-cluster; guest agent merged into cloud-config
|
|
|
|
- Two KVM Proxies with real cloud-configs reach `Ready` from inside the kind cluster
|
|
(2 min from apply, IP via the guest agent, 204 through squid from the LAN).
|
|
- The `qemu-guest-agent` bootstrap is now merged into each Proxy's cloud-config
|
|
(`packages` + `runcmd`) instead of shipped as vendor-data: cloud-init's default
|
|
merger drops vendor-data keys the user-data also defines, so any Proxy with its own
|
|
`packages:` never got the agent and stayed `Provisioning` forever. `vendorData` is
|
|
now plain extra vendor-data with no default; non-cloud-config user-data (scripts,
|
|
`#include`) still gets the bootstrap as vendor-data. A cloud-config that is not a
|
|
YAML mapping fails the Proxy permanently instead of booting without its config.
|
|
- The manager pod mounts a 64 MiB `emptyDir` on `/tmp`: the seed ISO writer stages
|
|
files on disk and the root filesystem is read-only.
|
|
- Errors already classified with a provider sentinel no longer drop the cached
|
|
libvirt SSH connection; only libvirt and transport errors do.
|
|
- Key files: `internal/provider/kvm/{seed.go,conn.go,errors.go}`,
|
|
`config/manager/manager.yaml`, `docs/kvm-provider.md`.
|
|
|
|
## 2026-09-10 10:07 UTC — KVM provider validated on a real hypervisor; UEFI default; libvirt call tracing
|
|
|
|
- `make test-integration-kvm` passes end to end against a libvirt 11.6 / QEMU 10.1
|
|
host (Debian 13 image, UEFI, guest agent reporting the IP after ~20 s). Four
|
|
defects found only on the real host, all fixed on the branch:
|
|
- domain disks are now `type="file"` with resolved paths: pool-volume references
|
|
get no AppArmor rule from `virt-aa-helper`, so QEMU was denied the disks;
|
|
- domain metadata is parsed by local name: libvirt returns the element without
|
|
its namespace declaration;
|
|
- guests boot **UEFI by default** (`images.<key>.firmware: uefi|bios`, OVMF now a
|
|
host prerequisite): Debian 13's image hangs in GRUB under SeaBIOS on a headless
|
|
VM with a serial console (Debian bug #1111240, open); bookworm is unaffected;
|
|
- the image cache's raw-upload path is confirmed to produce a byte-identical file.
|
|
- Every libvirt RPC is logged at `--zap-log-level=debug` as one `libvirt call` line
|
|
(`rpc=virDomainCreate`, identifiers, duration, `libvirtError` on failure); level 2
|
|
adds XML, metadata and addresses; cloud-init is never logged. The integration test
|
|
shows the trace in `-v` output (`KVM_TEST_LOG_LEVEL`, default 1).
|
|
- Key files: `internal/provider/kvm/{domainxml.go,metadata.go,trace.go,conn.go}`,
|
|
`internal/provider/config_kvm.go`, `docs/kvm-provider.md`.
|
|
|
|
## 2026-09-08 11:00 UTC — KVM/libvirt provider
|
|
|
|
- New `kvm` provider type: Managed proxies as VMs on your own libvirt/KVM hosts. Each
|
|
host is reached over SSH with the libvirt unix socket forwarded through the
|
|
connection (pure Go, nothing to expose on the hypervisors beyond sshd); one provider
|
|
instance can span several hosts, selected by `placement.zone`.
|
|
- Base images (`images:` map or a direct URL in `placement.image`) are downloaded by the
|
|
operator into each host's storage pool on first use, verified by sha256 when configured,
|
|
and cached there. Per-VM disks are qcow2 overlays; cloud-init arrives on a NoCloud seed
|
|
ISO built in-process. GC identity lives in the domain's `<metadata>` element.
|
|
- Wiring: `"kvm": kvm.New` in `cmd/main.go`; optional `kvm-ssh` Secret mounted at
|
|
`/etc/proxy-operator/kvm`; samples `config/samples/proxy_kvm.yaml` and a `kvm-home`
|
|
block in `providers-config.yaml`; `docs/kvm-provider.md`; opt-in integration test
|
|
(`make test-integration-kvm`) against a real hypervisor.
|
|
- Not validated on a real host at the time of this entry; see 2026-09-10 above.
|
|
|
|
## 2026-08-10 09:34 CEST — kind e2e verified; fix Squid OOM in containers; quickstart goes in-cluster
|
|
|
|
- Full end-to-end pass on a throwaway kind cluster: Squid pod Ready with a real CONNECT
|
|
probe (89 ms), lease grant/report/cooldown-409/release through the discovery API,
|
|
finalizer cleanup on delete.
|
|
- Fixed the kubernetes provider's generated squid.conf: `max_filedescriptors 1024`
|
|
(squid sizes FD tables from the container's effectively-unlimited RLIMIT_NOFILE and
|
|
was OOM-killed at startup under kind/containerd) + `cache_mem 16 MB`.
|
|
- README quickstart now deploys the operator in-cluster: `make run-dev` on a laptop
|
|
cannot reach kind pod IPs, so health probes fail by construction there (documented).
|
|
|
|
## 2026-08-09 17:27 CEST — Operator wired end to end: reconciler, health, leases, discovery, GC, two providers
|
|
|
|
- `cmd/main.go` is now the full composition root: `--providers-config` (required, fail-fast),
|
|
`--discovery-addr`, `--proxy-namespace`, `--health-workers`, `--gc-interval`, `--gc-min-age`,
|
|
`--gc-allow-namespaced`, `--lease-cooldown`, `--max-lease-ttl`; wires the kubernetes + gcp
|
|
providers (metrics-instrumented), health engine, lease store, discovery API, orphan GC, and
|
|
Prometheus metrics onto one manager.
|
|
- Deploy manifests: providers ConfigMap mount, optional `DISCOVERY_TOKEN` Secret env,
|
|
discovery port 8090 + Service; pods RBAC for the kubernetes provider.
|
|
- Samples for all three proxy flavors + providers-config; `make run-dev` for local development.
|
|
- README rewritten (kind quickstart, GCP setup, the two load-bearing caveats); architecture doc
|
|
completed with components table and the full decision log.
|