Merge the qemu-guest-agent bootstrap into each Proxy's cloud-config instead of vendor-data
In the kind cluster two KVM Proxies with real cloud-configs booted fine but never left Provisioning: the guest agent was not installed, so the provider could not learn the IP. cloud-init merges vendor-data under user-data with its default mergers, which keep the user-data's value for any key both define and never append lists, so a cloud-config with its own `packages:` silently dropped the vendor-data's packages and runcmd. The integration test passed only because its user-data was empty. withGuestAgent now appends qemu-guest-agent to `packages` and the enable command to `runcmd` of the Proxy's own cloud-config (skipping entries already present) and re-renders it. Vendor-data is no longer defaulted; `vendorData` is plain extra data. User-data that is not a cloud-config document (script, #include, MIME) still gets the bootstrap as vendor-data. A cloud-config that is not a YAML mapping, or whose packages/runcmd is not a list, fails the Proxy as ErrPermanent. Validated in-cluster: both Proxies Ready and healthy in two minutes, 204 through squid from the LAN. Docs, sample comment, CHANGELOG and the plan execution log (Run 7) updated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -557,3 +557,76 @@ on the host it needs `virsh undefine --nvram` plus its two volumes by
|
||||
hand. The `ListByTag` sweep costs one `virDomainGetMetadata` per
|
||||
domain on the host (21 here, ~0.4 ms each over the tunnel), which is
|
||||
fine at this scale but is the call to watch on a busy hypervisor.
|
||||
|
||||
### Run 7: in-cluster, two defects the integration test could not see
|
||||
|
||||
The e2e pass ran the operator inside the kind cluster
|
||||
(`egress-proxies-operator:kvm-dev`, built with `make docker-build`,
|
||||
loaded with `kind load docker-image`, deployed via the kustomize
|
||||
`config/manager` with the `kvm-home` provider pointing at beelink) and
|
||||
two Proxies with real cloud-configs: one plain squid, one with a debug
|
||||
login user. Both defects come from the difference between the test
|
||||
harness and the real thing, not from libvirt.
|
||||
|
||||
**Read-only root filesystem.** The manager runs with
|
||||
`readOnlyRootFilesystem: true`, and `iso9660.NewWriter` stages the seed
|
||||
in a temp directory, so the first Create failed with `creating iso
|
||||
writer: read-only file system`. Fix: a 64 MiB `emptyDir` on `/tmp` in
|
||||
`config/manager/manager.yaml`. Side finding: the error is classified
|
||||
`ErrPermanent` by `writeSeed`, yet `hostPool.call` treated any
|
||||
non-libvirt error as a transport failure and dropped the SSH connection,
|
||||
so every retry redialled. `call` now leaves the connection alone for
|
||||
errors already carrying a provider sentinel (`isClassified`), with a
|
||||
test.
|
||||
|
||||
**The guest agent never got installed.** Both VMs booted (confirmed on
|
||||
the console by hand) but sat in `Provisioning` for 25 minutes: the
|
||||
trace showed `virDomainInterfaceAddresses` from the agent source failing
|
||||
with `ErrAgentUnresponsive` on every poll and the ARP source empty. The
|
||||
integration test had passed with `CloudInit: "#cloud-config\n"`; the
|
||||
real Proxies have `packages: [squid]` and `runcmd:`. cloud-init merges
|
||||
vendor-data under user-data with its default mergers, which keep the
|
||||
user-data's value for any key both define and never append lists, so the
|
||||
vendor-data's `packages`/`runcmd` were dropped wholesale. Confirmed
|
||||
locally with the dev box's cloud-init 26.1 before changing anything:
|
||||
|
||||
```sh
|
||||
python3 - <<'EOF'
|
||||
from cloudinit import mergers
|
||||
user = {"packages": ["squid"], "runcmd": ["systemctl restart squid"]}
|
||||
vendor = {"packages": ["qemu-guest-agent"], "runcmd": [["systemctl", "enable", "--now", "qemu-guest-agent"]]}
|
||||
print(mergers.construct(mergers.default_mergers()).merge(dict(user), dict(vendor)))
|
||||
EOF
|
||||
# {'packages': ['squid'], 'runcmd': ['systemctl restart squid']}
|
||||
```
|
||||
|
||||
Fix: `withGuestAgent` in `seed.go` merges the bootstrap into the Proxy's
|
||||
cloud-config itself (append to `packages` and `runcmd`, skip if present,
|
||||
re-render with `sigs.k8s.io/yaml`), so vendor-data is no longer defaulted
|
||||
and `vendorData` is plain extra data. User-data that is not a cloud-config
|
||||
document (script, `#include`, MIME) still gets the bootstrap as
|
||||
vendor-data since there is nothing to collide with; a cloud-config that
|
||||
does not parse as a mapping fails the Proxy as `ErrPermanent`. The
|
||||
rendered document for the squid Proxy passed `cloud-init schema
|
||||
--config-file`. The logging test's fixture (`#cloud-config` + a bare
|
||||
sentinel line) is not a mapping and had to become `hostname: <sentinel>`.
|
||||
|
||||
Rebuilt, reloaded, restarted, deleted and recreated both Proxies: both
|
||||
`Ready`/`HEALTHY True` in 2 min 1 s from `kubectl apply`, IP via the
|
||||
agent 110 s after `virDomainCreate` (the guest installs squid and the
|
||||
agent from the mirrors first), and `curl -x http://<ip>:3128
|
||||
https://www.gstatic.com/generate_204` from the dev box returns 204
|
||||
through both.
|
||||
|
||||
Worth noting: the `#cloud-config` round-trip drops comments and reorders
|
||||
keys alphabetically; semantics are preserved (numbers, booleans and
|
||||
block scalars survive, checked in tests) but a user diffing the seed
|
||||
against their CR will see a different document. Vendor-data supplied via
|
||||
`vendorData` is subject to the same cloud-init merge rule and is now
|
||||
documented as such. The kind pod also restarted twice during the session
|
||||
from "Pod sandbox changed" (the node briefly lost the API server), which
|
||||
is unrelated to the operator. Still uncommitted on purpose: the
|
||||
`kustomization.yaml` image override (`egress-proxies-operator:kvm-dev`)
|
||||
and the `beelink` host entries in `config/manager/providers_config.yaml`
|
||||
and `config/samples/providers-config.yaml`, which are this machine's dev
|
||||
setup rather than the repo's defaults.
|
||||
|
||||
Reference in New Issue
Block a user