Add KVM provider methods: Create/Get/Delete/ListByTag over the libvirt seam

Create looks the domain up first and only starts it when it exists;
otherwise it caches the image, reuses or creates the overlay, rebuilds
the seed ISO unconditionally (a cancelled upload can leave a truncated
seed that no later attempt would repair), defines and starts the domain.
Get maps libvirt state plus the first routable IPv4 from ARP or the
guest agent onto the provider states. Delete destroys, undefines and
removes the per-VM volumes by name, tolerating every already-gone case.
ListByTag keeps managed domains across all hosts, skipping domains it
cannot read and failing the sweep only for an unreachable host.

Also corrects the Step 2 seed determinism test: the iso9660 writer
stamps the build time into the volume descriptor, so builds differ by a
few bytes; the test now compares label, files and size.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

.claude/settings.json

@@ -83,7 +83,13 @@
       "Bash(kubectl -n egress-proxies-operator-system get pods -o wide)",
       "Bash(kubectl -n egress-proxies-operator-system get deploy egress-proxies-operator-controller-manager -o jsonpath='{.spec.template.spec.containers[0].args}')",
       "Bash(kubectl -n egress-proxies-operator-system logs deploy/egress-proxies-operator-controller-manager)",
-      "Bash(python3 -c \"import json; d=json.load\\(open\\('docs/deploy/sa_key.json'\\)\\); print\\(d.get\\('type'\\), d.get\\('client_email'\\)\\)\")"
+      "Bash(python3 -c \"import json; d=json.load\\(open\\('docs/deploy/sa_key.json'\\)\\); print\\(d.get\\('type'\\), d.get\\('client_email'\\)\\)\")",
+      "Bash(git add *)",
+      "Bash(git commit -q -m 'Add plan: KVM/libvirt provider *)",
+      "Bash(go get *)",
+      "Bash(go build *)",
+      "Bash(go test *)",
+      "Bash(git commit *)"
     ],
     "additionalDirectories": [
       "/Users/jan.novak/srv/go/egress-proxies-operator/.claude",

README.md

@@ -59,7 +59,8 @@ kubectl get px -w
 # proxy-kubernetes-sample   Managed   kubernetes   Ready   10.244.x.x   True
 ```
 
-Once it's `Ready`, port-forward the discovery API and use it:
+Once it's `Ready`, port-forward the discovery API and use it (full
+reference with schemas and error codes: [docs/api.md](docs/api.md)):
 
 ```sh
 kubectl -n egress-proxies-operator-system port-forward \

docs/api.md

@@ -0,0 +1,323 @@
+# Discovery API reference
+
+The operator serves an HTTP API (the *discovery API*) that crawler clients
+use to find and lease egress proxies: list healthy proxies filtered by
+attributes, acquire a TTL-based lease on one, release it early, and report
+how a target site treated the proxy. It is implemented in
+[`internal/discovery`](../internal/discovery/) with lease state in
+[`internal/lease`](../internal/lease/); the only Kubernetes interaction is
+reading `Proxy` resources from the manager's cache.
+
+## Base URL
+
+The API listens on `:8090` (`--discovery-addr`) inside the manager pod and
+is exposed by a Service
+([config/default/discovery_service.yaml](../config/default/discovery_service.yaml)).
+
+In-cluster:
+
+```text
+http://egress-proxies-operator-controller-manager-discovery-service.egress-proxies-operator-system.svc.cluster.local:8090
+```
+
+From a workstation, port-forward:
+
+```sh
+kubectl -n egress-proxies-operator-system port-forward \
+  svc/egress-proxies-operator-controller-manager-discovery-service 8090:8090 &
+```
+
+Set `BASE_URL` to wherever you reach the API; all examples below use it:
+
+```sh
+export BASE_URL=localhost:8090   # via the port-forward above
+# or, from inside the cluster:
+# export BASE_URL=http://egress-proxies-operator-controller-manager-discovery-service.egress-proxies-operator-system.svc.cluster.local:8090
+```
+
+## Authentication
+
+A single static bearer token, read from the `DISCOVERY_TOKEN` environment
+variable at startup. The shipped Deployment populates it from the
+`discovery-token` Secret (key `token`), which is **optional** — if the
+Secret is absent or the token is empty, the API serves **unauthenticated**
+(the manager logs a loud warning at startup). Create the Secret:
+
+```sh
+kubectl -n egress-proxies-operator-system create secret generic discovery-token \
+  --from-literal=token="$(openssl rand -hex 24)"
+```
+
+Send the token on every request:
+
+```sh
+export TOKEN=<the token>
+curl -s -H "Authorization: Bearer $TOKEN" "$BASE_URL/v1/proxies" | jq
+```
+
+A missing or wrong token gets `401 {"error":"unauthorized",...}`.
+`GET /healthz` is always exempt.
+
+The curl examples below omit the `-H "Authorization: Bearer $TOKEN"` flag
+for brevity — add it to every call when auth is enabled.
+
+## Conventions
+
+- Requests and responses are JSON. Errors share one envelope:
+
+  ```json
+  {"error": "<machine_code>", "message": "<human-readable text>"}
+  ```
+
+- Request bodies are capped at **64 KiB** (larger bodies fail the JSON
+  decode with `400 invalid_body`).
+- Proxies with a deletion timestamp (being finalized) are excluded from
+  every response and never offered for lease.
+
+## Configuration
+
+| Setting | Default | Meaning |
+|---|---|---|
+| `--discovery-addr` | `:8090` | Listen address of the API |
+| `--max-lease-ttl` | `1h` | Maximum `ttlSeconds` a client may request |
+| `--lease-cooldown` | `15m` | Cooldown window applied on `rate_limited`/`banned` reports |
+| `DISCOVERY_TOKEN` (env) | empty | Bearer token; empty disables auth |
+
+The shipped Deployment passes none of these flags, so the defaults apply.
+
+## Endpoints
+
+### `GET /healthz`
+
+Liveness check. Unauthenticated, always `200` with body `ok`.
+
+```sh
+curl -s "$BASE_URL/healthz"
+```
+
+### `GET /v1/proxies` — list proxies
+
+Query parameters (all optional):
+
+| Parameter | Values | Effect |
+|---|---|---|
+| `healthy` | `true` \| `false` | Keep only proxies whose `Healthy` condition matches. Any other value → `400 invalid_query`. |
+| `attr.<key>` | any string | Exact match on `spec.attributes[<key>]`. Repeatable; **all** given pairs must match. |
+
+List everything:
+
+```sh
+curl -s "$BASE_URL/v1/proxies" | jq
+```
+
+List healthy proxies in a given geo:
+
+```sh
+curl -s "$BASE_URL/v1/proxies?healthy=true&attr.geo=eu" | jq
+```
+
+Response — `200`, proxies sorted by `id`, an empty match is `200` with
+`"count": 0` (never `404`):
+
+```json
+{
+  "proxies": [
+    {
+      "id": "default/proxy-kubernetes-sample",
+      "ip": "10.244.1.7",
+      "port": 3128,
+      "attributes": {"geo": "local"},
+      "phase": "Ready",
+      "healthy": true,
+      "latencyMillis": 42,
+      "activeLeases": 1,
+      "maxLeases": 5
+    }
+  ],
+  "count": 1
+}
+```
+
+Proxy object fields (the same shape appears inside lease responses):
+
+| Field | Meaning |
+|---|---|
+| `id` | `namespace/name` of the `Proxy` resource; used as the stable key everywhere |
+| `ip` | Effective host — `spec.endpoint.host` for `External` proxies, `status.ip` for `Managed` (empty until the backing VM/pod is up) |
+| `port` | Effective port (default `3128`) |
+| `attributes` | `spec.attributes` — free-form selection labels (`geo`, `asn`, `purpose`, …); omitted when empty |
+| `phase` | `Pending` \| `Provisioning` \| `Ready` \| `Unhealthy` \| `Deleting` \| `Failed` |
+| `healthy` | `true` iff the `Healthy` condition is `True` (the through-the-proxy health probe passes) |
+| `latencyMillis` | Latency of the last status-affecting health probe |
+| `activeLeases` | Currently active leases on this proxy |
+| `maxLeases` | Lease capacity (default `5`; an explicit `0` means unleasable) |
+
+### `POST /v1/leases` — acquire a lease
+
+Picks a healthy proxy with free capacity matching the selector and grants
+an exclusive-slot, TTL-based lease on it.
+
+Request body (every field optional; `{}` is valid):
+
+```json
+{
+  "selector": {"geo": "eu"},
+  "ttlSeconds": 300,
+  "target": "example.com"
+}
+```
+
+| Field | Default | Meaning |
+|---|---|---|
+| `selector` | none | Attribute equality filter, same semantics as `attr.<key>` above |
+| `ttlSeconds` | `300` (5 min) | Lease lifetime; must be ≤ `--max-lease-ttl` (default 1 h), else `400 invalid_ttl` |
+| `target` | none | The site you intend to crawl; enables per-target cooldowns (see below) |
+
+```sh
+curl -s -XPOST "$BASE_URL/v1/leases" \
+  -d '{"selector":{"geo":"eu"},"ttlSeconds":300,"target":"example.com"}' | jq
+```
+
+Success — `201`:
+
+```json
+{
+  "leaseID": "P3X6HHQTPCM5UTGVGE3B5UPS3A",
+  "proxy": {
+    "id": "default/proxy-eu-1",
+    "ip": "34.88.10.20",
+    "port": 3128,
+    "attributes": {"geo": "eu"},
+    "phase": "Ready",
+    "healthy": true,
+    "latencyMillis": 42,
+    "activeLeases": 1,
+    "maxLeases": 5
+  },
+  "expiresAt": "2026-08-11T22:05:00Z",
+  "ttlSeconds": 300
+}
+```
+
+Use `proxy.ip` and `proxy.port` as an HTTP proxy for the lease's lifetime:
+
+```sh
+curl -x http://34.88.10.20:3128 https://example.com
+```
+
+No match — `409` with diagnostic counts explaining why nothing qualified:
+
+```json
+{
+  "error": "no_match",
+  "message": "no healthy proxy with free capacity matched the selector",
+  "considered": 3,
+  "atCapacity": 1,
+  "inCooldown": 1,
+  "unhealthy": 1
+}
+```
+
+| Count | Meaning |
+|---|---|
+| `considered` | Proxies that matched the selector (before health/capacity checks) |
+| `atCapacity` | Skipped because `activeLeases >= maxLeases` |
+| `inCooldown` | Skipped because of an active cooldown for this target (or a global one) |
+| `unhealthy` | Skipped because the `Healthy` condition is not `True` |
+
+Leases expire on their own — releasing is only needed to free the slot
+early. There is no renew/extend endpoint; acquire a new lease instead.
+
+### `DELETE /v1/leases/{id}` — release early
+
+Frees the lease's capacity slot immediately. Idempotent: always `204`,
+including for unknown or already-expired lease IDs.
+
+```sh
+curl -si -XDELETE "$BASE_URL/v1/leases/P3X6HHQTPCM5UTGVGE3B5UPS3A"
+```
+
+### `POST /v1/leases/{id}/report` — report an outcome
+
+Tell the operator how the target site treated the proxy. This is the
+feedback signal that drives cooldowns.
+
+Request body:
+
+```json
+{"result": "rate_limited", "target": "example.com"}
+```
+
+| Field | Values | Meaning |
+|---|---|---|
+| `result` | `ok` \| `rate_limited` \| `banned` | Anything else → `400 invalid_result` |
+| `target` | optional | Which site produced the result; falls back to the lease's `target`, then to global |
+
+```sh
+curl -si -XPOST "$BASE_URL/v1/leases/P3X6HHQTPCM5UTGVGE3B5UPS3A/report" \
+  -d '{"result":"rate_limited","target":"example.com"}'
+```
+
+Responses: `204` on success, `404 unknown_lease` if the lease ID was never
+issued or has aged out.
+
+Semantics:
+
+- `ok` is a pure acknowledgement — nothing is recorded.
+- `rate_limited` and `banned` currently behave **identically**: both put
+  the proxy in one cooldown window (default 15 min, `--lease-cooldown`)
+  for the resolved target.
+- An expired lease remains reportable for one cooldown window past its
+  TTL, so a late "we got rate-limited" still lands.
+
+## Proxy selection and cooldowns
+
+How `POST /v1/leases` picks among eligible proxies (healthy, matching the
+selector, not being deleted, not at capacity, not in cooldown), in order:
+
+1. fewest `activeLeases` (least-loaded),
+2. lowest `latencyMillis`,
+3. lexicographic `id` (deterministic tie-break).
+
+Cooldowns are keyed by **(proxy, target)**:
+
+- A report **with a target** blocks that proxy only for lease requests
+  naming the **same target**. Other targets — and requests with no
+  target — still get the proxy.
+- A report **without a target**, on a lease that also had no target,
+  creates a **global** cooldown: the proxy is blocked for *all* lease
+  requests until the window passes. Always pass `target` on leases and
+  reports unless you really mean "this proxy is bad for everyone".
+
+## End-to-end example
+
+```sh
+# 1. Acquire a lease for crawling example.com through an EU proxy
+LEASE=$(curl -s -XPOST "$BASE_URL/v1/leases" \
+  -H "Authorization: Bearer $TOKEN" \
+  -d '{"selector":{"geo":"eu"},"ttlSeconds":600,"target":"example.com"}')
+LEASE_ID=$(echo "$LEASE" | jq -r .leaseID)
+PROXY=$(echo "$LEASE" | jq -r '"\(.proxy.ip):\(.proxy.port)"')
+
+# 2. Crawl through the leased proxy
+curl -x "http://$PROXY" https://example.com/some/page
+
+# 3. Got a 429? Report it — example.com-bound leases will avoid this
+#    proxy for the next 15 minutes
+curl -s -XPOST "$BASE_URL/v1/leases/$LEASE_ID/report" \
+  -H "Authorization: Bearer $TOKEN" \
+  -d '{"result":"rate_limited","target":"example.com"}'
+
+# 4. Done early? Release the slot (otherwise the TTL frees it)
+curl -s -XDELETE "$BASE_URL/v1/leases/$LEASE_ID" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+## Caveats
+
+- **Lease and cooldown state is in-memory and per-process.** An operator
+  restart drops all active leases and cooldowns. Clients must tolerate a
+  granted lease disappearing (a subsequent report returns `404`).
+- **Run a single replica.** The API is served by every manager replica but
+  is not leader-elected, and lease state is not shared between replicas;
+  the shipped Deployment pins `replicas: 1`.

docs/architecture.md

@@ -213,6 +213,8 @@ Kubernetes interaction is reading Proxies from the manager's cache. The
 server is a non-leader-elected Runnable (all replicas would serve, but the
 deployment ships `replicas: 1` because lease state is per-process — an
 operator restart drops all leases and cooldowns, a documented caveat).
+Client-facing reference with request/response schemas and curl examples:
+[api.md](api.md).
 
 ```text
 crawler client

docs/demo/create-gcp-proxies.sh

@@ -0,0 +1,94 @@
+#!/bin/bash
+# Demo: create N Managed proxies backed by the gcp provider, named
+# proxy-gcp-demo-1 .. proxy-gcp-demo-N, each in a randomly picked EU zone
+# so the fleet gets egress IPs from different locations.
+#
+# Usage:
+#   ./create-gcp-proxies.sh <count>
+#
+# Optional environment:
+#   NAMESPACE     namespace to create the proxies in (default: current context)
+#   GCP_PROVIDER  provider NAME from providers.yaml (default: gcp-eu)
+#   ZONES         space-separated zone list to pick from (default: EU zones below)
+set -euo pipefail
+
+if [ $# -ne 1 ] || ! [[ "$1" =~ ^[1-9][0-9]*$ ]]; then
+    echo "usage: $(basename "$0") <count>  (positive integer)" >&2
+    exit 1
+fi
+count="$1"
+provider="${GCP_PROVIDER:-gcp-eu}"
+
+# GCP zones in the EU where e2-micro is generally available. Override with
+# ZONES="zone1 zone2 ..." if your project has quota only in some of them.
+default_zones=(
+    europe-west1-b europe-west1-c europe-west1-d       # Belgium
+    europe-west2-a europe-west2-b europe-west2-c       # London
+    europe-west3-a europe-west3-b europe-west3-c       # Frankfurt
+    europe-west4-a europe-west4-b europe-west4-c       # Netherlands
+    europe-west6-a europe-west6-b europe-west6-c       # Zurich
+    europe-west8-a europe-west8-b europe-west8-c       # Milan
+    europe-west9-a europe-west9-b europe-west9-c       # Paris
+    europe-central2-a europe-central2-b europe-central2-c   # Warsaw
+    europe-north1-a europe-north1-b europe-north1-c    # Finland
+    europe-southwest1-a europe-southwest1-b europe-southwest1-c   # Madrid
+)
+if [ -n "${ZONES:-}" ]; then
+    read -r -a zones <<< "${ZONES}"
+else
+    zones=("${default_zones[@]}")
+fi
+
+if ! command -v kubectl &> /dev/null; then
+    echo "ERROR: kubectl is required but not installed" >&2
+    exit 1
+fi
+
+ns_args=()
+if [ -n "${NAMESPACE:-}" ]; then
+    ns_args=(-n "${NAMESPACE}")
+fi
+
+for i in $(seq 1 "${count}"); do
+    zone="${zones[RANDOM % ${#zones[@]}]}"
+    echo "Creating proxy-gcp-demo-${i} in ${zone} ..."
+    kubectl apply "${ns_args[@]}" -f - << EOF
+apiVersion: crawl.example.com/v1alpha1
+kind: Proxy
+metadata:
+  name: proxy-gcp-demo-${i}
+spec:
+  mode: Managed
+  provider: ${provider}              # must match a provider NAME in providers.yaml
+  placement:
+    zone: ${zone}
+    machineType: e2-micro
+    # debian-cloud images have no cloud-init, so spec.cloudInit (passed as
+    # user-data metadata) would be silently ignored there. Ubuntu images do.
+    image: projects/ubuntu-os-cloud/global/images/family/ubuntu-2404-lts-amd64
+  port: 3128
+  cloudInit:
+    inline: |
+      #cloud-config
+      package_update: true
+      packages:
+        - squid
+      write_files:
+        - path: /etc/squid/conf.d/proxy-operator.conf
+          content: |
+            http_access allow all
+            via off
+            forwarded_for off
+      runcmd:
+        - systemctl restart squid
+  attributes:
+    geo: eu
+    zone: ${zone}
+    purpose: crawl
+EOF
+done
+
+echo ""
+echo "Created ${count} proxies. VMs take a few minutes to provision and pass"
+echo "the health check. Watch them come up with:"
+echo "  kubectl get px ${ns_args[*]:-} -w"

docs/demo/create-kubernetes-proxies.sh

@@ -0,0 +1,48 @@
+#!/bin/bash
+# Demo: create N Managed proxies backed by the kubernetes-pod provider,
+# named proxy-kubernetes-demo-1 .. proxy-kubernetes-demo-N. Pods share the
+# cluster's egress IP — this exercises the full lifecycle, not distinct
+# egress paths (use create-gcp-proxies.sh for that).
+#
+# Usage:
+#   ./create-kubernetes-proxies.sh <count>
+#
+# Optional environment:
+#   NAMESPACE  namespace to create the proxies in (default: current context)
+set -euo pipefail
+
+if [ $# -ne 1 ] || ! [[ "$1" =~ ^[1-9][0-9]*$ ]]; then
+    echo "usage: $(basename "$0") <count>  (positive integer)" >&2
+    exit 1
+fi
+count="$1"
+
+if ! command -v kubectl &> /dev/null; then
+    echo "ERROR: kubectl is required but not installed" >&2
+    exit 1
+fi
+
+ns_args=()
+if [ -n "${NAMESPACE:-}" ]; then
+    ns_args=(-n "${NAMESPACE}")
+fi
+
+for i in $(seq 1 "${count}"); do
+    echo "Creating proxy-kubernetes-demo-${i} ..."
+    kubectl apply "${ns_args[@]}" -f - << EOF
+apiVersion: crawl.example.com/v1alpha1
+kind: Proxy
+metadata:
+  name: proxy-kubernetes-demo-${i}
+spec:
+  mode: Managed
+  provider: kubernetes
+  attributes:
+    geo: local
+    purpose: crawl
+EOF
+done
+
+echo ""
+echo "Created ${count} proxies. Watch them come up with:"
+echo "  kubectl get px ${ns_args[*]:-} -w"

docs/demo/run-demo.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+# Demo driver: opens a tmux session with a 2x2 pane grid:
+#
+#   top-left:     show-egress-ips-table.sh in a 10s loop, run inside the
+#                 netshoot pod against the in-cluster discovery Service
+#   top-right:    watch -n3 kubectl get px
+#   bottom-left:  create-kubernetes-proxies.sh <count>
+#   bottom-right: create-gcp-proxies.sh <count>
+#
+# Usage:
+#   ./run-demo.sh
+#
+# Optional environment:
+#   COUNT         proxies each create script makes (default: 4)
+#   SESSION       tmux session name (default: proxy-demo; an existing
+#                 session with this name is killed and recreated)
+#   NETSHOOT_POD  pod to exec into for the egress-IP loop (default: netshoot)
+#   DEMO_DIR      where the demo scripts live (default: this script's dir)
+set -euo pipefail
+
+COUNT="${COUNT:-4}"
+SESSION="${SESSION:-proxy-demo}"
+NETSHOOT_POD="${NETSHOOT_POD:-netshoot}"
+DEMO_DIR="${DEMO_DIR:-$(cd "$(dirname "$0")" && pwd)}"
+BASE_URL="http://egress-proxies-operator-controller-manager-discovery-service.egress-proxies-operator-system.svc.cluster.local:8090"
+
+for tool in tmux kubectl; do
+    if ! command -v "${tool}" &> /dev/null; then
+        echo "ERROR: ${tool} is required but not installed" >&2
+        exit 1
+    fi
+done
+
+if ! kubectl get pod "${NETSHOOT_POD}" &> /dev/null; then
+    echo "ERROR: pod ${NETSHOOT_POD} not found — start one with:" >&2
+    echo "  kubectl run netshoot --image=nicolaka/netshoot -- sleep infinity" >&2
+    exit 1
+fi
+
+echo "Copying show-egress-ips-table.sh into pod ${NETSHOOT_POD} ..."
+kubectl cp "${DEMO_DIR}/show-egress-ips-table.sh" "${NETSHOOT_POD}:/tmp/show-egress-ips-table.sh"
+
+if tmux has-session -t "${SESSION}" 2> /dev/null; then
+    echo "Killing existing tmux session ${SESSION}"
+    tmux kill-session -t "${SESSION}"
+fi
+
+# 2x2 grid: after these splits pane indexes are 0 top-left, 1 top-right,
+# 2 bottom-left, 3 bottom-right; tiled layout evens them into quarters.
+tmux new-session -d -s "${SESSION}"
+tmux split-window -h -t "${SESSION}:0"
+tmux split-window -v -t "${SESSION}:0.0"
+tmux split-window -v -t "${SESSION}:0.1"
+tmux select-layout -t "${SESSION}:0" tiled
+
+loop_cmd="BASE_URL=${BASE_URL}; while true; do bash /tmp/show-egress-ips-table.sh \"\$BASE_URL\"; echo; sleep 10; done"
+tmux send-keys -t "${SESSION}:0.0" "kubectl exec -it ${NETSHOOT_POD} -- bash -c '${loop_cmd}'" C-m
+tmux send-keys -t "${SESSION}:0.1" "watch -n3 kubectl get px" C-m
+tmux send-keys -t "${SESSION}:0.2" "bash ${DEMO_DIR}/create-kubernetes-proxies.sh ${COUNT}" C-m
+tmux send-keys -t "${SESSION}:0.3" "bash ${DEMO_DIR}/create-gcp-proxies.sh ${COUNT}" C-m
+
+tmux select-pane -t "${SESSION}:0.2"
+if [ -n "${TMUX:-}" ]; then
+    tmux switch-client -t "${SESSION}"
+else
+    tmux attach-session -t "${SESSION}"
+fi

docs/demo/show-egress-ips-table.sh

@@ -0,0 +1,75 @@
+#!/bin/bash
+# Demo: condensed-table variant of show-egress-ips.sh, made to fit a small
+# tmux pane. One line per proxy: which proxy the request goes through, its
+# endpoint, location (zone/geo attribute), and the egress IP the IP-echo
+# site saw — or unhealthy/FAILED.
+#
+# Usage:
+#   ./show-egress-ips-table.sh <BASE_URL>    e.g. ./show-egress-ips-table.sh localhost:8090
+#
+# Optional environment:
+#   TOKEN        bearer token for the discovery API (see docs/api.md)
+#   IP_ECHO_URL  site that returns the caller's IP as JSON with an "ip" field
+#                (default: https://api.ipify.org?format=json)
+set -euo pipefail
+
+if [ $# -ne 1 ]; then
+    echo "usage: $(basename "$0") <BASE_URL>  (e.g. localhost:8090)" >&2
+    exit 1
+fi
+BASE_URL="$1"
+IP_ECHO_URL="${IP_ECHO_URL:-https://api.ipify.org?format=json}"
+
+for tool in curl jq; do
+    if ! command -v "${tool}" &> /dev/null; then
+        echo "ERROR: ${tool} is required but not installed" >&2
+        exit 1
+    fi
+done
+
+auth_args=()
+if [ -n "${TOKEN:-}" ]; then
+    auth_args=(-H "Authorization: Bearer ${TOKEN}")
+fi
+
+if ! proxies_json=$(curl -sS --fail "${auth_args[@]}" "${BASE_URL}/v1/proxies") \
+    || ! echo "${proxies_json}" | jq -e . > /dev/null 2>&1; then
+    echo "ERROR: could not fetch proxy list from ${BASE_URL}" >&2
+    exit 1
+fi
+
+total=$(echo "${proxies_json}" | jq -r '.count')
+fmt="%-31s %-21s %-21s %s\n"
+
+echo "${total} proxies @ $(date +%H:%M:%S)"
+# shellcheck disable=SC2059
+printf "${fmt}" "PROXY" "ENDPOINT" "LOCATION" "EGRESS-IP"
+
+probed=0
+skipped=0
+failed=0
+while IFS= read -r proxy; do
+    id=$(echo "${proxy}" | jq -r '.id')
+    endpoint=$(echo "${proxy}" | jq -r '"\(.ip):\(.port)"')
+    location=$(echo "${proxy}" | jq -r '.attributes.zone // .attributes.geo // "-"')
+    healthy=$(echo "${proxy}" | jq -r '.healthy')
+
+    if [ "${healthy}" != "true" ]; then
+        # shellcheck disable=SC2059
+        printf "${fmt}" "${id}" "${endpoint}" "${location}" "(unhealthy)"
+        skipped=$((skipped + 1))
+        continue
+    fi
+
+    if response=$(curl -sS --max-time 10 -x "http://${endpoint}" "${IP_ECHO_URL}" 2> /dev/null); then
+        egress=$(echo "${response}" | jq -r '.ip // "?"' 2> /dev/null || echo "?")
+        probed=$((probed + 1))
+    else
+        egress="FAILED"
+        failed=$((failed + 1))
+    fi
+    # shellcheck disable=SC2059
+    printf "${fmt}" "${id}" "${endpoint}" "${location}" "${egress}"
+done < <(echo "${proxies_json}" | jq -c '.proxies[]')
+
+echo "-- ${probed} probed, ${skipped} unhealthy, ${failed} failed --"

docs/demo/show-egress-ips.sh

@@ -0,0 +1,74 @@
+#!/bin/bash
+# Demo: list proxies from the discovery API and show the egress IP each
+# healthy one provides, by calling an IP-echo site through it.
+#
+# Usage:
+#   ./show-egress-ips.sh <BASE_URL>          e.g. ./show-egress-ips.sh localhost:8090
+#
+# Optional environment:
+#   TOKEN        bearer token for the discovery API (see docs/api.md)
+#   IP_ECHO_URL  site that returns the caller's IP as JSON
+#                (default: https://api.ipify.org?format=json)
+set -euo pipefail
+
+if [ $# -ne 1 ]; then
+    echo "usage: $(basename "$0") <BASE_URL>  (e.g. localhost:8090)" >&2
+    exit 1
+fi
+BASE_URL="$1"
+IP_ECHO_URL="${IP_ECHO_URL:-https://api.ipify.org?format=json}"
+
+for tool in curl jq; do
+    if ! command -v "${tool}" &> /dev/null; then
+        echo "ERROR: ${tool} is required but not installed" >&2
+        exit 1
+    fi
+done
+
+auth_args=()
+if [ -n "${TOKEN:-}" ]; then
+    auth_args=(-H "Authorization: Bearer ${TOKEN}")
+fi
+
+echo "Fetching proxies from ${BASE_URL}/v1/proxies ..."
+if ! proxies_json=$(curl -sS --fail "${auth_args[@]}" "${BASE_URL}/v1/proxies"); then
+    echo "ERROR: could not fetch proxy list from ${BASE_URL}" >&2
+    exit 1
+fi
+if ! echo "${proxies_json}" | jq -e . > /dev/null; then
+    echo "ERROR: response from ${BASE_URL}/v1/proxies is not valid JSON" >&2
+    exit 1
+fi
+
+total=$(echo "${proxies_json}" | jq -r '.count')
+echo "Found ${total} proxies"
+echo ""
+
+probed=0
+skipped=0
+failed=0
+while IFS= read -r proxy; do
+    id=$(echo "${proxy}" | jq -r '.id')
+    ip=$(echo "${proxy}" | jq -r '.ip')
+    port=$(echo "${proxy}" | jq -r '.port')
+    healthy=$(echo "${proxy}" | jq -r '.healthy')
+
+    if [ "${healthy}" != "true" ]; then
+        echo "--- skipping ${id} (unhealthy) ---"
+        echo ""
+        skipped=$((skipped + 1))
+        continue
+    fi
+
+    echo "=== via ${id} — http://${ip}:${port} ==="
+    if response=$(curl -sS --max-time 10 -x "http://${ip}:${port}" "${IP_ECHO_URL}"); then
+        echo "${response}" | jq . 2> /dev/null || echo "${response}"
+        probed=$((probed + 1))
+    else
+        echo "WARNING: request through ${id} failed" >&2
+        failed=$((failed + 1))
+    fi
+    echo ""
+done < <(echo "${proxies_json}" | jq -c '.proxies[]')
+
+echo "Done: ${total} proxies — ${probed} probed, ${skipped} skipped (unhealthy), ${failed} failed"

docs/plans-executions/2026-08-11-2152-discovery-api-docs.md

@@ -0,0 +1,38 @@
+# Execution: Discovery API documentation
+
+Plan: [2026-08-11-2152-discovery-api-docs.md](../plans/2026-08-11-2152-discovery-api-docs.md)
+
+- [x] Step 1 — Write `docs/api.md` full API reference
+- [x] Step 2 — Add pointers in README and architecture.md
+
+## Step 1 — docs/api.md
+
+Wrote the full reference: base URL (in-cluster FQDN + port-forward), bearer
+auth, error envelope, configuration table, all five routes with schemas,
+status codes and curl examples, the selection/cooldown semantics section,
+an end-to-end curl walkthrough, and the in-memory/single-replica caveats.
+All facts were taken from the code, not from memory of prior docs.
+
+Worth noting: the doc explicitly calls out two things no earlier doc
+stated for clients — that a report **without** a target on a targetless
+lease creates a *global* cooldown (blocking the proxy for everyone), and
+that `rate_limited` and `banned` currently behave identically. Both came
+straight from `internal/lease/store.go` and are easy to trip over.
+
+## Step 2 — Pointers + verification
+
+Added one-line links to the new doc in README's quickstart (above the curl
+block) and in `docs/architecture.md` §7. Verified the documented behavior
+against the tree rather than trusting the write-up:
+
+```sh
+go build ./... && go test -short ./internal/discovery/ ./internal/lease/
+```
+
+Both pass; a grep of `server_test.go` confirmed every documented status
+code and error code (`invalid_ttl`, `invalid_query`, `invalid_result`,
+`no_match`, `unknown_lease`, 201/204/401/404/409) is asserted by tests.
+
+Worth noting: CHANGELOG entry deliberately deferred until the user
+confirms the docs read well, per the CHANGELOG convention's
+"once the user confirms it works" clause.

docs/plans-executions/2026-08-11-2220-demo-scripts.md

@@ -0,0 +1,55 @@
+# Execution: Demo scripts
+
+Plan: [2026-08-11-2220-demo-scripts.md](../plans/2026-08-11-2220-demo-scripts.md)
+
+- [x] Step 0 — Branch `feat/demo-scripts` + plan commit
+- [x] Step 1 — `docs/demo/show-egress-ips.sh`
+- [x] Extra (added iteratively, not in the original plan) — proxy-creation scripts
+
+## Step 0 + Step 1
+
+Branched off `main`, committed the plan alone, then wrote
+`docs/demo/show-egress-ips.sh`: takes `BASE_URL` as its argument, lists
+`/v1/proxies` (bearer auth via optional `TOKEN` env), probes each healthy
+proxy with `curl -x http://ip:port` against an IP-echo site
+(`IP_ECHO_URL`, default ipify JSON), banners which proxy each request goes
+through, skips unhealthy ones, and ends with a probed/skipped/failed
+summary. Per user request the script was left uncommitted for iteration
+and no verification beyond `bash -n` was run.
+
+## Extra — create-kubernetes-proxies.sh, create-gcp-proxies.sh
+
+Added on the same branch before the first commit:
+
+- `create-kubernetes-proxies.sh <count>` — creates
+  `proxy-kubernetes-demo-1..N` with the kubernetes provider, spec taken
+  from `config/samples/proxy_kubernetes.yaml`, applied via
+  `kubectl apply -f -` heredocs. Optional `NAMESPACE` env.
+- `create-gcp-proxies.sh <count>` — creates `proxy-gcp-demo-1..N` from the
+  user-supplied gcp-eu manifest (e2-micro, Ubuntu 24.04, Squid
+  cloud-init), each with a zone picked randomly from a hardcoded list of
+  27 EU zones so the fleet gets egress IPs from different locations.
+  Env overrides: `ZONES`, `GCP_PROVIDER` (default `gcp-eu`), `NAMESPACE`.
+
+## Extra — run-demo.sh, show-egress-ips-table.sh
+
+Second round of iterative additions:
+
+- `run-demo.sh` — tmux demo driver: 2x2 tiled grid with the egress-IP
+  table looping every 10s inside a netshoot pod (script `kubectl cp`'d
+  into the pod, `BASE_URL` set to the in-cluster Service FQDN),
+  `watch -n3 kubectl get px`, and both create scripts auto-running with
+  `COUNT` proxies each (default 4). Env knobs: `COUNT`, `SESSION`,
+  `NETSHOOT_POD`, `DEMO_DIR` (defaults to the script's own dir).
+- `show-egress-ips-table.sh` — condensed one-line-per-proxy variant of
+  `show-egress-ips.sh` sized for a tmux pane: PROXY / ENDPOINT /
+  LOCATION (zone→geo attribute fallback) / EGRESS-IP columns, with
+  `(unhealthy)` and `FAILED` inline instead of verbose output. The
+  verbose script stays for standalone use; the demo driver uses the
+  table variant.
+
+Worth noting: beyond the user's sample manifest, the gcp script also
+writes the picked zone into `attributes.zone`, so the discovery API
+exposes each proxy's location and leases can select on it. The zone list
+is static — if a project lacks quota in some region, `ZONES` narrows the
+pool; nothing validates zones against the live project.

docs/plans-executions/2026-09-04-1404-kvm-provider.md

@@ -0,0 +1,285 @@
+# Execution: KVM/libvirt provider
+
+Plan: [2026-09-04-1404-kvm-provider.md](../plans/2026-09-04-1404-kvm-provider.md)
+
+- [x] Step 0 — Save plan + branch `feat/kvm-provider`
+- [x] Step 1 — Config (`KVMConfig` & friends, `validate()` case, tests)
+- [x] Step 2 — Pure builders (`domainxml.go`, `volume.go`, `seed.go`, `id.go`, `metadata.go`)
+- [x] Step 3 — Transport + errors (`conn.go`, `errors.go`)
+- [x] Step 3b — Image cache (`image.go`)
+- [x] Step 4 — Provider methods (`kvm.go`) over a fake API
+- [ ] Step 5 — Wiring, manifests, samples, docs, CHANGELOG
+- [ ] Step 6 — Integration test against a real hypervisor
+- [ ] Step 7 — Manual validation + MR
+
+## Step 0 — Save plan + branch
+
+Plan committed alone on `feat/proxy-operator`, then branched:
+
+```bash
+git checkout -b feat/kvm-provider
+```
+
+## Step 1 — Config
+
+The new structs (`KVMConfig`, `KVMHost`, `KVMMachineType`, `KVMImage`) and
+`validateKVM` went into their own file, `internal/provider/config_kvm.go`,
+rather than being appended to `config.go` as the plan said: the KVM block has
+far more fields and rules than the other two providers, and keeping it
+separate leaves `config.go` readable. Tests likewise live in
+`config_kvm_test.go` instead of extending the table in `config_test.go`; the
+invalid-config table covers every required field, duplicate host names, image
+URL scheme, sha256 shape, and the disk format enum.
+
+While adding the `kvm` case to `validate()`, the pairwise "type is X but a Y
+block is set" checks (which would have needed six branches with three types)
+were replaced by a `setBlocks()` helper that lists the non-nil blocks and
+rejects any that don't match `type`. The error strings are unchanged, so the
+existing GCP/kubernetes tests still pass.
+
+Dependencies were fetched up front so the API signatures in the plan could be
+checked with `go doc`; they stay `// indirect` in `go.mod` until Step 2 imports
+them:
+
+```bash
+go get github.com/digitalocean/go-libvirt@latest github.com/kdomanski/iso9660@v0.4.0
+# go-libvirt resolved to v0.0.0-20260814190004-1a83157e1858 (no tagged release)
+# side effect: golang.org/x/crypto 0.54.0 → 0.56.0, x/mod, x/text, x/tools bumped
+```
+
+Worth noting: `go doc` showed that go-libvirt's `New(net.Conn)` is deprecated
+in favour of `NewWithDialer(socket.Dialer)`, where `Dialer` is a one-method
+`Dial() (net.Conn, error)` interface. Step 3 should wrap the SSH client's unix
+forward in that interface instead of using `New`; the plan's transport design
+is otherwise unaffected.
+
+## Step 2 — Pure builders
+
+`internal/provider/kvm/` now has the five builders from the plan plus
+`kvm.go` holding the package doc and `withDefaults` (disk size 10 GiB, the
+default vendor-data, `/var/run/libvirt/libvirt-sock`, pool `default`, image
+format `qcow2`). Everything is a pure function over `encoding/xml` structs or
+strings, unit-tested by unmarshalling the output back and checking fields.
+
+Two design choices differ from the plan text:
+
+- **Domain disks reference pool volumes, not host paths.** The plan had
+  `buildDomainXML` take the overlay and seed paths (needing two
+  `StorageVolGetPath` round-trips per Create). The domain XML now uses
+  `<disk type="volume"><source pool=... volume=.../>`, so only the *base*
+  image path is ever looked up (the overlay's `<backingStore>` needs a real
+  path). Fewer RPCs and the builder's inputs are all names.
+- **Capacities are always `unit="bytes"`.** libvirt's `G` means GiB but
+  `GB` means 10^9; spelling the byte count out removes the ambiguity from
+  both the overlay and the seed/base volumes.
+
+The metadata element is written with a default namespace
+(`<instance xmlns="…/kvm">`) because `encoding/xml` cannot emit prefixes on
+marshal. libvirt may hand it back prefixed; `parseInstanceMetadata` is
+tested against both spellings since `encoding/xml` matches on the namespace
+URI. Whether libvirt accepts the default-namespace form on `DomainDefineXML`
+is confirmed in Step 6 against a real host, not here.
+
+Checked before relying on `github.com/kdomanski/iso9660` for the seed: its
+filename mangling lowercases and keeps `-`, so `user-data`/`meta-data`/
+`vendor-data` come out as `user-data;1` etc. in the directory record, which
+the Linux isofs driver (and therefore cloud-init's NoCloud mount) presents as
+`user-data`. The seed test opens the ISO with the same library's reader and
+asserts label `cidata`, all three files, and byte-identical output for
+identical inputs. `NewWriter()` stages files in a temp directory, so
+`Cleanup()` is deferred.
+
+`go mod tidy` ran as part of this step; it promoted `kdomanski/iso9660` to a
+direct dependency and dropped `go-libvirt` again because nothing imports it
+yet — Step 3 brings it back.
+
+Worth noting: `withDefaults` copies the `Images` map and `Hosts` slice
+instead of mutating the config in place, so the registry's shared
+`ProviderConfig` stays untouched. The default vendor-data installs
+`qemu-guest-agent` via `packages:` + `runcmd`, which means the first boot
+needs network access to the distro mirrors; a Proxy on an isolated bridge
+would still work (ARP discovery), just without the agent.
+
+## Step 3 — Transport + errors
+
+`conn.go` holds the `libvirtAPI` seam (17 methods, signatures as in the
+plan), a per-host connection pool, and the SSH dialer; `errors.go` holds
+`classify`, `isLibvirtError`, `hasCode` and `logAPIError`. Both are
+unit-tested; the whole package still runs in about a second under `-race`.
+
+The one thing the plan did not anticipate: **go-libvirt RPCs take no
+context and never time out**. A hung hypervisor would pin a reconcile worker
+forever. So the pool exposes `call(ctx, host, fn)` rather than handing out
+raw clients: `fn` runs in a goroutine, and if `ctx` ends first the pool
+closes the SSH connection out from under it (which makes the blocked RPC
+return) and reports `ctx.Err()`. The same helper (`awaitOrCancel`) bounds
+`lv.Connect()` during dial. Two more liveness measures for the same reason:
+the SSH handshake gets a deadline (ctx's, or 30 s), and each connection runs
+an OpenSSH `keepalive@openssh.com` loop every 15 s that closes the client
+when a request errors or goes unanswered for one interval.
+
+Connection-drop policy in `call`: any error that is *not* a `libvirt.Error`
+(so io.EOF, "connection closed", SSH failures) drops the cached connection
+and the next call redials; a daemon-reported `libvirt.Error` keeps it. Dials
+are serialised per host (concurrent Creates on one host share a dial) but
+hosts don't block each other.
+
+As Step 1 noted, the dialer uses `libvirt.NewWithDialer` with a
+`socket.Dialer` that wraps `ssh.Client.Dial("unix", socket)` rather than the
+deprecated `New(net.Conn)`.
+
+Tests: the pool is exercised through a fake dialer (reuse, redial after
+`Disconnected`, drop on transport error, keep on libvirt error, cancel
+releases a blocked call, concurrent callers share one dial, hosts
+independent). The SSH layer is tested against an in-process
+`x/crypto/ssh` server that serves `direct-streamlocal@openssh.com` channels
+by dialing the requested unix path (an echo socket standing in for
+`libvirt-sock`), with keys generated per test and a real `known_hosts` file
+written via `knownhosts.Line`. Covered: forward round-trip and the socket
+path the server saw, unknown host key rejected with `knownhosts.KeyError`,
+`insecureIgnoreHostKey`, wrong client key, cancelled ctx, missing key or
+known_hosts file, and keepalive both closing a stalled client and leaving a
+healthy one alone. The libvirt handshake (`Connect()`) itself is *not*
+covered here — faking libvirt's XDR RPC just for that is not worth it; the
+Step 6 integration test covers it against a real host.
+
+`go mod tidy` promoted `go-libvirt` and `golang.org/x/crypto` to direct
+dependencies.
+
+Worth noting: `classify` maps `ErrOperationInvalid` (e.g. destroying a
+domain that is already off) to Transient; Delete in Step 4 must check that
+code itself via `hasCode` and treat it as success rather than relying on
+the classifier. Nothing maps to `ErrQuotaExceeded` — libvirt has no quota
+concept; an out-of-space pool surfaces as a Transient `ErrInternalError`.
+
+## Step 3b — Image cache
+
+`image.go` follows the plan closely: `resolveImage` maps `placement.image`
+to a configured key or a direct http(s) URL (else Permanent);
+`imageCache.ensure` checks base volume + marker (+ marker content when a
+checksum is configured), tears down and re-fetches when anything is off,
+streams the HTTP body through `StorageVolUpload` with a `sha256` tee,
+refreshes the pool, then writes the marker last. A per-`(host, volume)`
+mutex serialises concurrent callers; the pool itself is the only cache
+state.
+
+Errors from this file are pre-tagged with the taxonomy sentinel via
+`fmt.Errorf("%w: …", provider.ErrPermanent, …)` and `classify` now honours
+such tags before looking at libvirt codes. That keeps one `wrapErr` path in
+the provider instead of special-casing image errors.
+
+`fake_test.go` landed with this step rather than Step 4: an in-memory
+`libvirtAPI` with pools, volumes (content, format, capacity, backing path),
+domains (state, metadata, per-source interface addresses), per-method error
+injection (`fail` always / `failOnce`) and an `uploadHook` to break a stream
+mid-way. It models the libvirt error codes the provider branches on
+(`ErrStorageVolExist`, `ErrNoStorageVol`, `ErrNoDomain`,
+`ErrOperationInvalid`, `ErrNoDomainMetadata`, `ErrAgentUnresponsive`).
+
+Tests cover every case in the plan plus: no checksum configured (marker is
+still written with the computed digest; second call needs no marker read),
+403/500 alongside 404/503, missing pool → Permanent, cancelled ctx →
+Transient with nothing created, and different hosts fetching independently.
+
+One test-harness lesson worth keeping: `httptest`'s server drops
+`Content-Length` and switches to chunked encoding for bodies larger than
+its 2 KiB write buffer, so the fake mirror sets the header explicitly.
+Real distro mirrors send it; a server that doesn't is reported as
+Permanent with a message naming the header.
+
+Worth noting: the base volume is created with `capacity = Content-Length`
+and the *image's* format (`qcow2`), and the raw bytes are uploaded over it.
+That relies on libvirt's upload writing the file verbatim (which is how
+`virsh vol-upload` populates volumes); the following `StoragePoolRefresh`
+makes libvirt re-read the real virtual size from the qcow2 header. Step 6
+confirms this on a real host. Also, the whole download runs inside one
+`pool.call`, so cancellation aborts the HTTP body (ctx-bound) and the
+upload stream fails; the half-written base volume is deleted on the spot
+and, failing that, on the next attempt via the missing marker.
+
+## Step 4 — Provider methods
+
+`kvm.go` now holds `Provider`, `New` (loads the SSH key and known_hosts
+up front via `newSSHAuth`, so a bad path fails at startup), the
+`newWithDialer` seam, and `Create`/`Get`/`Delete`/`ListByTag`. Each
+method resolves its host from config, then runs the whole libvirt
+conversation inside one `hostPool.call`, so a cancelled reconcile drops
+the connection and the blocked RPC returns. Errors pass through `classify`
+in one `wrapErr`, as the Step 3b note intended.
+
+Behaviour that goes beyond the plan text, or that a future reader should
+know before touching this file:
+
+- **Create looks up the domain first.** If it exists, only the running
+  check runs and the volumes are never touched. If not, nothing can be
+  using the volumes yet: the overlay is reused when it exists (its
+  creation is atomic; `ErrStorageVolExist` → reuse) but the **seed is
+  always deleted and rebuilt**. The plan said "exists → skip upload"; the
+  reviewer pointed out that a reconcile cancelled mid-upload leaves a
+  truncated seed that the compensating delete cannot remove, because
+  `hostPool.call` has already closed the connection the delete would go
+  over. Skipping such a seed would boot the VM without cloud-init and no
+  retry would ever repair it. Rebuilding costs one ~40 KiB upload per
+  attempt before the domain exists, nothing afterwards. The table test
+  breaks Create after each step and re-runs it, and a separate test
+  starts from a leftover truncated seed.
+- **Get swallows address-source errors.** `DomainInterfaceAddresses` from
+  ARP returns nothing until the guest has sent traffic and the agent
+  source errors with `ErrAgentUnresponsive` until `qemu-guest-agent` is
+  up, so both are logged at V(2) and mean "no IP yet" → `Provisioning`.
+  Loopback, link-local and unspecified addresses are skipped when
+  choosing the first IPv4 (the agent reports `lo` too).
+- **State mapping:** `running`/`blocked` are active; `nostate` →
+  Provisioning (the reconciler answers Stopped with delete-and-recreate,
+  too drastic for a state libvirt could not determine); everything else
+  (paused, pmsuspended, shutdown, shutoff, crashed) → Stopped.
+- **Delete treats a missing pool as "nothing to clean"** rather than
+  Permanent: failing there would pin the Proxy's finalizer on a config
+  edit that renamed the pool. Domain removal is destroy (`ErrOperationInvalid`
+  on a stopped domain is fine) → undefine with managed-save, snapshot
+  metadata and nvram flags → overlay and seed deleted by name.
+- **ListByTag** reads each domain's metadata once (passed into the same
+  `inspect` that Get uses, so no second `DomainGetMetadata` round-trip)
+  and skips domains without the operator's element. A domain that cannot
+  be read at all — vanished mid-sweep, corrupt metadata, state call
+  failing — is *skipped*, not fatal: GC never deletes what it does not
+  see, so leaving one out is safe, whereas one bad domain blocking GC for
+  every other one (the first version's behaviour, caught in review) is
+  not. An unreachable host still fails the whole sweep.
+  `ConnectListAllDomains` is called with flags 0 (active and inactive).
+- A host named in a providerID but no longer in `kvm.hosts` is Permanent
+  for Get and Delete: the operator cannot reach it, and retrying forever
+  would hide the config problem.
+
+Tests run the provider over a `hostDialer` that maps host names to
+`fakeAPI` instances (an unmapped host is "unreachable"). The image is
+pre-seeded in the pool for most tests; one test runs the full first-use
+download through `httptest` and confirms a second Proxy on the same host
+does not download again.
+
+Found while running the package: `TestBuildSeedISO_deterministic` from
+Step 2 was wrong. `iso9660.ImageWriter` stamps `time.Now()` into the
+volume descriptor, so two builds in different seconds differ by six
+timestamp bytes; the test had only passed because both builds landed in
+the same second. Byte identity is nothing the provider depends on (an
+existing seed is never re-uploaded), so the test now compares label,
+files and size, and the comment on `buildSeedISO` no longer claims
+identical output.
+
+The `go-operator-reviewer` agent ran on the first version of this step
+and found the two bugs above (truncated seed, wedged GC sweep) plus the
+double metadata read and the `nostate` mapping; all were fixed before
+committing. One finding it raised is deliberately **not** fixed here
+because it lives in Step 3b's `image.go` and changes the volume naming
+the plan chose: re-fetching a base image (checksum edited in config, or
+upstream content changed) deletes `base-<key>.<fmt>` while existing
+overlays still back onto it, breaking those VMs' disks. Content-addressed
+base names would avoid it. Flagged for the user; until decided, Step 5's
+docs must warn that an image's `sha256`/`url` must not change while
+Proxies using it exist.
+
+Worth noting: `go test -cover` reports 88.2 % for the package; the
+uncovered lines are the real SSH/libvirt dial path and log-only
+branches. `go vet` and `-race` (three runs) are clean. Wiring into
+`cmd/main.go` is Step 5, so `New` is not yet reachable from a running
+operator.

docs/plans/2026-08-11-2152-discovery-api-docs.md

@@ -0,0 +1,58 @@
+# Plan: Discovery API documentation (docs/api.md)
+**Created:** 2026-08-11 21:52
+
+
+## Context
+
+The operator serves an HTTP discovery/lease API on `:8090` ([internal/discovery/](internal/discovery/)) that crawler clients use to list proxies, acquire TTL leases, release them, and report rate-limiting. There is no dedicated API reference today: README has four quickstart curls (no auth header, no schemas), and `docs/architecture.md` §7 has an ASCII route map. The user wants full documentation with curl examples for every feature.
+
+**Decisions made with user:** doc lives in a new `docs/api.md`; commit straight to `main` (no MR).
+
+## Deliverable
+
+### 1. New file `docs/api.md` — full API reference
+
+Content (all facts verified against code during planning):
+
+- **Overview & base URL** — what the API is; in-cluster FQDN `http://egress-proxies-operator-controller-manager-discovery-service.egress-proxies-operator-system.svc.cluster.local:8090` (Service: `config/default/discovery_service.yaml`); local access via `kubectl port-forward svc/egress-proxies-operator-controller-manager-discovery-service 8090:8090`.
+- **Authentication** — static bearer token from `DISCOVERY_TOKEN` env var (populated from the optional `discovery-token` Secret, key `token`; `config/manager/manager.yaml`). Empty token ⇒ auth disabled with startup warning. Curl: `-H "Authorization: Bearer $TOKEN"` on every example. `/healthz` always exempt.
+- **Conventions** — JSON everywhere; error envelope `{"error":"<code>","message":"<text>"}`; request bodies capped at 64 KiB; proxies with a deletion timestamp are excluded from all responses.
+- **Configuration table** — `--discovery-addr` (default `:8090`), `--max-lease-ttl` (default 1h), `--lease-cooldown` (default 15m), `DISCOVERY_TOKEN`. Note the shipped Deployment passes none of these flags, so defaults apply.
+- **Endpoints**, each with request/response schema, status codes, and a copy-pasteable curl example:
+  - `GET /healthz` — liveness, unauthenticated.
+  - `GET /v1/proxies` — filters `healthy=true|false` (else 400 `invalid_query`) and repeatable `attr.<key>=<value>` (verbatim equality on `spec.attributes`, all pairs must match). Response `{"proxies":[proxyView...],"count":N}` sorted by id. Full `proxyView` field table: `id` (ns/name), `ip`, `port`, `attributes`, `phase` (Pending/Provisioning/Ready/Unhealthy/Deleting/Failed), `healthy` (condition `Healthy` == True), `latencyMillis`, `activeLeases`, `maxLeases` (default 5; explicit 0 = unleasable).
+  - `POST /v1/leases` — body `{selector, ttlSeconds, target}` all optional; TTL defaults 5m, capped at max-lease-ttl (else 400 `invalid_ttl`). 201 `{leaseID, proxy, expiresAt, ttlSeconds}`; 409 `no_match` with `considered/atCapacity/inCooldown/unhealthy` counts (documented meanings).
+  - `DELETE /v1/leases/{id}` — early release; always 204, idempotent.
+  - `POST /v1/leases/{id}/report` — body `{result: ok|rate_limited|banned, target}`; 204, 404 `unknown_lease`, 400 `invalid_result`. `ok` is a pure ack; `rate_limited` and `banned` behave identically today (both start one cooldown window).
+- **Selection & cooldown semantics** (short section — this is the non-obvious part clients need):
+  - Selection order: fewest active leases → lowest latency → lexicographic id; deterministic; only healthy, non-deleting proxies with free capacity are candidates.
+  - Cooldown: 15m default (`--lease-cooldown`), keyed `{proxy, target}`. Report with a target blocks only leases requesting that target; report without a target (and lease without one) creates a **global** cooldown blocking all acquisitions of that proxy — call this footgun out explicitly.
+  - Expired leases stay reportable for one cooldown window past TTL.
+- **End-to-end workflow example** — numbered curl walkthrough: acquire → use `proxy.ip:port` as HTTP proxy (`curl -x`) → report `rate_limited` on 429 → release. Using a `jq`-extracted `leaseID`.
+- **Caveats** — lease/cooldown state is in-memory and per-process: single replica only, operator restart drops all leases and cooldowns.
+
+### 2. Pointers to the new doc (small edits)
+
+- `README.md`: one-line link near the quickstart curl section ("full reference: docs/api.md").
+- `docs/architecture.md` §7: one-line link to `docs/api.md` as the detailed reference.
+
+### 3. Housekeeping per CLAUDE.md
+
+- First action post-approval: copy this plan to `docs/plans/<timestamp>-discovery-api-docs.md` (timestamp from `date "+%Y-%m-%d-%H%M"`), commit it alone.
+- Then write the docs, commit to `main` with `Co-Authored-By: Claude <noreply@anthropic.com>` trailer, push.
+- Append execution summary + status checklist to `docs/plans-executions/<same-timestamp>-discovery-api-docs.md` in the docs commit.
+- Add `CHANGELOG.md` entry (timestamp via `date "+%Y-%m-%d %H:%M %Z"`) once the user confirms.
+
+## Key source files (facts source of truth)
+
+- [internal/discovery/handlers.go](internal/discovery/handlers.go), [internal/discovery/server.go](internal/discovery/server.go) — routes, schemas, status codes, auth, limits.
+- [internal/lease/store.go](internal/lease/store.go) — selection order, cooldown/retention, stats.
+- [api/v1alpha1/proxy_types.go](api/v1alpha1/proxy_types.go), [api/v1alpha1/helpers.go](api/v1alpha1/helpers.go) — defaults (port 3128, maxLeases 5), phases, conditions.
+- [cmd/main.go](cmd/main.go) — flags/env defaults.
+- [config/default/discovery_service.yaml](config/default/discovery_service.yaml), [config/manager/manager.yaml](config/manager/manager.yaml) — service DNS, token secret.
+
+## Verification
+
+- Cross-check every documented status code / field name against `internal/discovery/server_test.go` expectations.
+- Sanity-run `go build ./... && go test -short ./internal/discovery/ ./internal/lease/` (no code changes expected — confirms docs match current behavior, not a stale tree).
+- Optionally lint the curl JSON bodies by piping each through `jq .` locally.

docs/plans/2026-08-11-2220-demo-scripts.md

@@ -0,0 +1,33 @@
+# Plan: Demo script — egress IP check through each healthy proxy
+**Created:** 2026-08-11 22:20
+
+
+## Context
+
+First of a planned series of demo scripts under `docs/demo/`. This one showcases the discovery API end to end without leases: list proxies from `$BASE_URL/v1/proxies`, and for each **healthy** one, call an IP-echo site through it (`curl -x`) to show the egress IP that proxy provides — clearly labeling which proxy each request goes through. The user explicitly asked to start by switching to a new branch. The script will be iterated on: **write it but do not commit it** — the user wants to add things to it before anything is committed.
+
+## Steps
+
+### Step 0 — Branch
+
+Create `feat/demo-scripts` off `main`. Commit only the plan copy (`docs/plans/<timestamp>-demo-scripts.md`, timestamp via `date "+%Y-%m-%d-%H%M"`) per CLAUDE.md — nothing else gets committed this round.
+
+### Step 1 — `docs/demo/show-egress-ips.sh` (new file, executable, left uncommitted)
+
+Style: match [.devcontainer/post-install.sh](.devcontainer/post-install.sh) — `#!/bin/bash`, `set -euo pipefail`, `ERROR:`/`WARNING:` messages, `${VAR}` braces.
+
+Behavior:
+
+1. **Args/env:** `BASE_URL` is `$1` (required; missing → usage text + exit 1, e.g. `usage: show-egress-ips.sh <BASE_URL>  (e.g. localhost:8090)`). Optional env: `TOKEN` (bearer token, same name docs/api.md uses; when set, send `Authorization: Bearer $TOKEN`), `IP_ECHO_URL` (default `https://api.ipify.org?format=json` — returns `{"ip":"..."}`).
+2. **Dependency check:** `command -v curl`, `command -v jq` → `ERROR` + exit 1 if missing.
+3. **Fetch** `"$BASE_URL/v1/proxies"` once (no server-side `healthy` filter — fetch all so unhealthy ones can be shown as skipped, which makes the demo more informative). Fail with a clear error if curl or JSON parsing fails.
+4. **Iterate** proxies with `jq -c '.proxies[]'`; for each, extract `id`, `ip`, `port`, `healthy`:
+   - unhealthy → print `--- skipping <id> (unhealthy) ---`
+   - healthy → print a clear banner naming the proxy before the request, e.g. `=== via <id> — http://<ip>:<port> ===`, then `curl -sS --max-time 10 -x "http://${ip}:${port}" "$IP_ECHO_URL"`; print the JSON response. A failed probe prints `WARNING: request through <id> failed` and continues (guard so `set -e` doesn't kill the loop).
+5. Finish with a one-line summary: N proxies, M probed, K skipped/failed.
+
+Reference for API shapes: [docs/api.md](docs/api.md) (`proxies[].id/ip/port/healthy`; `curl -x http://ip:port` usage is already documented there and in README).
+
+### Deliberately deferred (user will iterate on the script first)
+
+- No commit of the script, no push beyond the plan commit, no MR, no execution summary, no CHANGELOG — all wait until the user says the script (or script set) is ready.

docs/plans/2026-09-04-1404-kvm-provider.md

@@ -0,0 +1,313 @@
+# Plan: KVM/libvirt provider
+
+**Created:** 2026-09-04 14:04
+
+## Context
+
+The operator provisions egress proxies through named provider instances (today
+`gcp` and `kubernetes`). The user has several KVM hypervisors on the home
+network and wants proxy VMs created there, using stock Debian/Ubuntu cloud
+images so the VM stays a normal machine they can customise through their own
+cloud-init user-data. Longer term the operator may become a generic "run a VM
+or container with a set of scripts" tool; the KVM provider should not
+foreclose that.
+
+The provider contract (`internal/provider/provider.go`: `Create`/`Get`/`Delete`/
+`ListByTag`, keyed by `NameFromUID`, cloud-init passed verbatim, GC-by-tag) fits
+libvirt with **no CRD or reconciler changes**. All work is a new provider
+package plus config/wiring/docs, following the GCP provider's shape.
+
+Decisions taken with the user (2026-09-04):
+
+- **Transport:** SSH to each hypervisor, forwarding the libvirt unix socket
+  (`ssh.Client.Dial("unix", "/var/run/libvirt/libvirt-sock")` → `libvirt.New(conn)`).
+  Pure Go, no cgo, no libvirt libs in the image. Verified: `x/crypto/ssh`
+  `Client.Dial` has a `"unix"` case (`dialStreamLocal`), go-libvirt `New(net.Conn)`.
+- **Topology:** one provider instance can span several hosts from day one
+  (config has a `hosts` list; providerID encodes the host), so adding hosts
+  later is a config edit with no providerID migration. With one host configured
+  `placement.zone` may be omitted.
+- **Networking:** VM NIC on a host bridge to the LAN, IP from the router's
+  DHCP. IP discovery via `DomainInterfaceAddresses` (ARP source first, then
+  guest agent). Provider-injected `vendor-data` installs `qemu-guest-agent`
+  so agent discovery works on stock images without touching user-data.
+- **Base image:** a stock Debian/Ubuntu cloud image (qcow2, cloud-init
+  preinstalled) that **the operator fetches itself** on first use per host:
+  `placement.image` names an entry in the provider's `images:` map (url +
+  optional sha256) or is an `https://` URL directly. The provider streams the
+  download through libvirt's `StorageVolUpload` into the host's pool, so
+  nothing is pre-placed and the operator needs no filesystem access on the
+  host. Per-VM disk is a qcow2 overlay with the cached base as backing store.
+- **Seed:** NoCloud ISO (`user-data`, `meta-data`, `vendor-data`) built in
+  pure Go with `github.com/kdomanski/iso9660` (`WriteTo(w, "cidata")`) and
+  streamed to the pool with `StorageVolUpload(vol, io.Reader, ...)`. Nothing
+  to install on the hypervisors.
+
+## Design
+
+### Config (`internal/provider/config.go`)
+
+```yaml
+- name: kvm-home
+  type: kvm
+  kvm:
+    sshKeyFile: /etc/proxy-operator/kvm/id_ed25519        # required
+    knownHostsFile: /etc/proxy-operator/kvm/known_hosts   # required unless insecureIgnoreHostKey
+    insecureIgnoreHostKey: false
+    diskSizeGb: 10                    # overlay capacity; cloud-init growpart expands root fs
+    vendorData: |                     # optional override; default installs qemu-guest-agent
+      #cloud-config
+      ...
+    images:                           # optional; placement.image names a key (or is an https URL)
+      debian-13:
+        url: https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2
+        sha256: "…"                   # optional; verified after download, re-fetch when it changes
+        format: qcow2                 # default qcow2
+      ubuntu-24.04:
+        url: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
+    machineTypes:                     # required, ≥1; placement.machineType names a key
+      small:  {vcpus: 1, memoryMiB: 1024}
+      medium: {vcpus: 2, memoryMiB: 2048}
+    hosts:                            # required, ≥1, unique names
+      - name: nas1                    # placement.zone value
+        address: nas1.home.lan:22     # required
+        user: virt                    # required; must be in the libvirt group
+        socket: /var/run/libvirt/libvirt-sock   # default
+        pool: default                 # default "default"
+        bridge: br0                   # required
+```
+
+Add `KVMConfig`, `KVMHost`, `KVMMachineType`, `KVMImage` structs; a `"kvm"`
+case in `validate()` (cross-block checks like the others, required fields
+above, unique host names, image URLs must be `http(s)`, sha256 if set must be
+64 hex chars). Tests in `config_test.go` extend the existing table.
+
+### Placement mapping
+
+| Placement field | KVM meaning |
+|---|---|
+| `zone` | host name from `hosts`; empty → the single configured host, else `ErrPermanent` |
+| `machineType` | key in `machineTypes`; unknown → `ErrPermanent` |
+| `image` | key in `images:`, or an `https://` URL; anything else → `ErrPermanent`. Fetched into the host's pool on first use (see *Image cache*) |
+| `region` | ignored |
+
+### providerID
+
+`hosts/<host>/domains/<name>` with `formatProviderID`/`parseProviderID`
+(same self-contained style as GCP's `zones/<zone>/instances/<name>`).
+
+### GC contract without labels
+
+Domain XML gets a `<metadata>` child in a private namespace, e.g.
+`xmlns="https://gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/kvm"`,
+carrying `<managed>true</managed><uid>…</uid><createdAt>RFC3339</createdAt>`.
+`Get` reads it via `DomainGetMetadata(dom, DomainMetadataElement, uri)`;
+`ListByTag` does `ConnectListAllDomains` on every host and keeps domains where
+that call succeeds and `managed == "true"`. `CreatedAt` comes from the metadata
+(libvirt has no creation time). A host that is unreachable makes `ListByTag`
+return an error (GC skips the sweep that round rather than acting on a partial view).
+
+### Connection management (`internal/provider/kvm/conn.go`)
+
+Per-host lazily dialed `*ssh.Client` + `*libvirt.Libvirt`, cached under a
+mutex, re-dialed when `Disconnected()` is closed or a call fails with a
+transport error. SSH auth: private key from `sshKeyFile`; host key check via
+`knownhosts.New(knownHostsFile)` unless `insecureIgnoreHostKey`. Dial timeout
+from `ctx`. The provider calls libvirt through a narrow interface:
+
+```go
+type libvirtAPI interface {
+    DomainLookupByName(name string) (libvirt.Domain, error)
+    DomainDefineXML(xml string) (libvirt.Domain, error)
+    DomainCreate(d libvirt.Domain) error
+    DomainDestroy(d libvirt.Domain) error
+    DomainUndefineFlags(d libvirt.Domain, f libvirt.DomainUndefineFlagsValues) error
+    DomainGetState(d libvirt.Domain, flags uint32) (int32, int32, error)
+    DomainGetMetadata(d libvirt.Domain, typ int32, uri libvirt.OptString, f libvirt.DomainModificationImpact) (string, error)
+    DomainInterfaceAddresses(d libvirt.Domain, src uint32, flags uint32) ([]libvirt.DomainInterface, error)
+    ConnectListAllDomains(need int32, f libvirt.ConnectListAllDomainsFlags) ([]libvirt.Domain, uint32, error)
+    StoragePoolLookupByName(name string) (libvirt.StoragePool, error)
+    StorageVolLookupByName(p libvirt.StoragePool, name string) (libvirt.StorageVol, error)
+    StorageVolGetPath(v libvirt.StorageVol) (string, error)
+    StorageVolCreateXML(p libvirt.StoragePool, xml string, f libvirt.StorageVolCreateFlags) (libvirt.StorageVol, error)
+    StorageVolUpload(v libvirt.StorageVol, r io.Reader, off, n uint64, f libvirt.StorageVolUploadFlags) error
+    StorageVolDownload(v libvirt.StorageVol, w io.Writer, off, n uint64, f libvirt.StorageVolDownloadFlags) error
+    StorageVolDelete(v libvirt.StorageVol, f libvirt.StorageVolDeleteFlags) error
+    StoragePoolRefresh(p libvirt.StoragePool, flags uint32) error
+}
+```
+
+(All signatures verified against go-libvirt `main` on 2026-09-04.) A
+`hostDialer func(ctx, KVMHost) (libvirtAPI, io.Closer, error)` is the test seam;
+`newWithDialer(cfg, dialer)` mirrors GCP's `newWithAPI`.
+
+### Image cache (`internal/provider/kvm/image.go`)
+
+`ensureImage(ctx, host, api, img) (basePath string, err error)` makes the base
+image present in the host's pool, once per host per image:
+
+- Volume names: `base-<key>.<format>` for an `images:` key, or
+  `base-<sha256(url)[:16]>.<format>` for a URL given directly in
+  `placement.image`. Marker volume `<base>.sha256` (raw, 64 bytes) holds the
+  digest of the verified upload.
+- **Ready check:** base volume exists *and* marker exists *and* (no `sha256`
+  configured, or marker content == configured `sha256`). Read the marker with
+  `StorageVolDownload`. Ready → return `StorageVolGetPath(base)`.
+- **Not ready:** delete both volumes if present (partial upload from a crash,
+  or a changed checksum), then `GET url` with the ctx, require a
+  `Content-Length` (`ErrPermanent` otherwise), `StorageVolCreateXML` the base
+  with `<capacity unit="bytes">Content-Length</capacity>` and the configured
+  format, `StorageVolUpload(base, io.TeeReader(body, sha256), 0, 0, 0)` (length
+  0 = until EOF, so nothing is buffered in the operator), then compare the
+  digest with the configured one (mismatch → delete base, `ErrPermanent`),
+  `StoragePoolRefresh` so libvirt re-reads the real qcow2 capacity, create the
+  marker volume and upload the digest last.
+- HTTP 4xx → `ErrPermanent`; 5xx, network, or upload transport errors →
+  `ErrTransient` (partial volumes are cleaned on the next attempt).
+- A per-`(host, image)` `sync.Mutex` map serialises concurrent Creates for the
+  same image on the same host; the second caller finds it ready.
+- **Judgment call:** the fetch runs synchronously inside `Create`, so the first
+  Proxy for a given image on a given host holds a reconcile worker for the
+  download (LAN speed, a few hundred MB, once). This keeps `Create` idempotent
+  and needs no background state. Async fetch with a `Provisioning`-style
+  progress status is a possible later improvement, noted in the docs.
+
+### Create (idempotent, fire-and-forget after the image is cached)
+
+1. Resolve host, machine type, pool, and image source (`images:` key or URL;
+   else `ErrPermanent`). `ensureImage` → base path.
+2. Overlay volume `<name>.qcow2`: `StorageVolCreateXML` with
+   `<capacity unit="G">diskSizeGb</capacity>`, `<target><format type="qcow2"/></target>`,
+   `<backingStore><path>base</path><format type="qcow2"/></backingStore>`.
+   `ErrStorageVolExist` → reuse.
+3. Seed volume `<name>-seed.iso`: build ISO in memory (`user-data` = req.CloudInit,
+   `meta-data` = `instance-id: <name>\nlocal-hostname: <name>\n`, `vendor-data` =
+   cfg or default), `StorageVolCreateXML` with exact capacity + `<format type="raw"/>`,
+   then `StorageVolUpload`. Exists → skip upload (content is deterministic per spec hash;
+   spec changes recreate the whole instance anyway).
+4. `DomainLookupByName(name)`; if `ErrNoDomain` → `DomainDefineXML(buildDomainXML(...))`.
+5. `DomainGetState`; if not running → `DomainCreate`. Return providerID.
+
+`buildDomainXML` is a pure function over `encoding/xml` structs: `type="kvm"`,
+q35 machine, `host-passthrough` CPU, vcpus/memory from the profile, virtio disk
+(overlay) + sata cdrom (seed), `interface type="bridge"` with virtio model,
+`org.qemu.guest_agent.0` virtio channel, serial console, virtio rng, no
+graphics, and the `<metadata>` block. Unit-tested field by field like
+`buildInsertRequest`.
+
+### Get
+
+`parseProviderID` → host → `DomainLookupByName` (`ErrNoDomain` → `ErrNotFound`)
+→ `DomainGetState` + `DomainGetMetadata` + `DomainInterfaceAddresses`
+(ARP, then agent; first IPv4 wins; agent errors while the agent isn't up yet
+are swallowed). Mapping:
+
+| libvirt state | IP | Instance.State |
+|---|---|---|
+| running | yes | Running |
+| running | no | Provisioning |
+| shutoff / crashed / paused / pmsuspended | – | Stopped |
+| no domain | – | `ErrNotFound` |
+
+### Delete (idempotent)
+
+Destroy if running (`ErrOperationInvalid` on a stopped domain is fine) →
+`DomainUndefineFlags(managed-save | snapshots-metadata | nvram)` → delete
+`<name>.qcow2` and `<name>-seed.iso` (`ErrNoStorageVol` fine). Volumes are
+deleted by name even when the domain is already gone, so a crash between
+volume creation and define still gets cleaned up on the next reconcile.
+Known gap (documented, not solved now): volumes orphaned with no domain *and*
+no CR are invisible to `ListByTag`.
+
+### Errors (`internal/provider/kvm/errors.go`)
+
+`classify(err)` on `libvirt.Error.Code`: `ErrNoDomain`, `ErrNoStorageVol` →
+`ErrNotFound`; `ErrNoStoragePool`, `ErrXMLError`, `ErrConfigUnsupported`,
+`ErrNoSupport`, `ErrAuthFailed`, `ErrInvalidArg` → `ErrPermanent`; SSH dial /
+host-key / transport errors and everything else → `ErrTransient`. Image
+download: HTTP 4xx, missing Content-Length, checksum mismatch → `ErrPermanent`;
+5xx and network errors → `ErrTransient`. Quota is not detectable from libvirt;
+left as transient (revisit with a capacity check later).
+
+### Wiring & manifests
+
+- `cmd/main.go`: add `"kvm": kvm.New` to the builtins map.
+- `config/manager/manager.yaml`: mount Secret `kvm-ssh` (optional) at
+  `/etc/proxy-operator/kvm` (keys `id_ed25519`, `known_hosts`); document the
+  `kubectl create secret generic kvm-ssh --from-file=...` command.
+- `config/samples/providers-config.yaml`: add the `kvm-home` block with the
+  `images:` map (Debian 13 and Ubuntu 24.04 official cloud-image URLs);
+  `config/samples/proxy_kvm.yaml`: Proxy with `placement: {zone: nas1,
+  machineType: small, image: debian-13}` and the same squid cloud-init as the
+  GCP sample plus `ssh_authorized_keys`.
+- `hack/providers-dev.yaml` stays kubernetes-only (needs no secrets).
+- Deps: `github.com/digitalocean/go-libvirt` (latest pseudo-version),
+  `github.com/kdomanski/iso9660 v0.4.0`, promote `golang.org/x/crypto` to direct.
+
+### Docs
+
+- New `docs/kvm-provider.md`: hypervisor prerequisites (user in `libvirt`
+  group, a bridge to the LAN, a storage pool with enough space), the config
+  reference including `images:`, how the image cache works (`base-*` and
+  `*.sha256` volumes in the pool, when a re-download happens, how to evict
+  with `virsh vol-delete`), the placement table, how to log into a VM
+  (`ssh_authorized_keys` in user-data, `virsh console` on the host), and the
+  known gaps (no quota detection, orphan volumes without a domain,
+  synchronous first-time image fetch).
+- Update the provider tables in `README.md`, `docs/architecture.md` §5,
+  `docs/testing.md` (integration test section).
+
+### Towards the generic runner (not built now, just kept open)
+
+Provider knows nothing about squid. Future hooks: `vendor-data` for injecting an
+agent, `QEMUDomainAgentCommand` for reading exit status/output, and a future
+`Placement`-independent "job" mode in the CRD. None of this changes the KVM
+provider's shape.
+
+## Steps
+
+0. **Save plan + branch.** Copy this file to `docs/plans/<ts>-kvm-provider.md`,
+   commit alone, then `git checkout -b feat/kvm-provider` off `feat/proxy-operator`
+   (the working main for this project).
+1. **Config.** `KVMConfig` & friends, `validate()` case, table tests.
+2. **Pure builders.** `internal/provider/kvm/{domainxml.go,volume.go,seed.go,id.go,metadata.go}`
+   + tests: domain XML fields, volume XML, seed contents (unpack the ISO with
+   the same library's reader and assert the three files + label `cidata`),
+   providerID round-trip, metadata marshal/parse.
+3. **Transport + errors.** `conn.go` (SSH dial, host-key policy, per-host cache,
+   reconnect), `errors.go` + classification table test. SSH dialer tested
+   against an in-process `x/crypto/ssh` server that serves a unix socket
+   forward (cheap, keeps the dial code covered).
+3b. **Image cache.** `image.go` + `image_test.go`: `httptest.Server` serving a
+   small fake image over the `fakeAPI` (which records uploaded bytes);
+   cases: fresh fetch, already ready, marker missing (partial upload → redo),
+   checksum changed → redo, checksum mismatch → Permanent + cleanup, 404 →
+   Permanent, 503 → Transient, two concurrent callers → one download.
+4. **Provider methods.** `kvm.go`: `New`, `newWithDialer`, `Create/Get/Delete/ListByTag`
+   over `libvirtAPI`; `kvm_test.go` with a hand-written `fakeAPI` (domains and
+   volumes in maps, per-call error injection) covering idempotent re-Create
+   after each partial step, state/IP mapping, delete-when-gone, ListByTag
+   across two hosts and with one host down.
+5. **Wiring, manifests, samples, docs, CHANGELOG.**
+6. **Integration test** (`//go:build integration`, `kvm_integration_test.go`):
+   skipped unless `KVM_TEST_HOST`, `KVM_TEST_USER`, `KVM_TEST_SSH_KEY`,
+   `KVM_TEST_BRIDGE`, `KVM_TEST_IMAGE_URL` are set; fetches the image (or finds
+   it cached), creates, waits for Running+IP, deletes, asserts per-VM volumes
+   gone and the base image still cached. Deviation from the testcontainers convention,
+   called out in `docs/testing.md`: libvirt+KVM can't run inside a container
+   without `/dev/kvm`, so this targets a real hypervisor.
+7. **Manual validation** on one real hypervisor, results in
+   `docs/plans-executions/` (like `docs/gcp-vm-validation.md`), then MR via `tea`.
+
+## Verification
+
+- `make test` green (`-race`, envtest suite untouched but must still pass).
+- `go test -short ./internal/provider/...` covers builders, fake-API provider, errors, config.
+- Integration: `KVM_TEST_HOST=nas1.home.lan … go test -tags=integration -run TestKVM ./internal/provider/kvm/`.
+- End to end: deploy with the `kvm-ssh` Secret and `kvm-home` config against a
+  host with no cached image, apply `config/samples/proxy_kvm.yaml`, watch the
+  operator log the image fetch and `virsh vol-list` show `base-debian-13.qcow2`
+  plus its `.sha256` marker, watch `kubectl get px -w` reach `Ready` with a LAN
+  IP, apply a second Proxy and confirm no second download, confirm the VM shows up in `GET /v1/proxies` and works with
+  `hack/` demo egress-IP check, delete the Proxy and confirm `virsh list --all`
+  and `virsh vol-list default` are clean.

go.mod

@@ -4,10 +4,13 @@ go 1.26.0
 
 require (
 	cloud.google.com/go/compute v1.65.0
+	github.com/digitalocean/go-libvirt v0.0.0-20260814190004-1a83157e1858
 	github.com/go-logr/logr v1.4.3
+	github.com/kdomanski/iso9660 v0.4.0
 	github.com/onsi/ginkgo/v2 v2.27.4
 	github.com/onsi/gomega v1.39.0
 	github.com/prometheus/client_golang v1.23.2
+	golang.org/x/crypto v0.56.0
 	google.golang.org/api v0.292.0
 	google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af
 	k8s.io/api v0.36.0
@@ -78,17 +81,16 @@ require (
 	go.uber.org/zap v1.27.1 // indirect
 	go.yaml.in/yaml/v2 v2.4.3 // indirect
 	go.yaml.in/yaml/v3 v3.0.4 // indirect
-	golang.org/x/crypto v0.54.0 // indirect
 	golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93 // indirect
-	golang.org/x/mod v0.37.0 // indirect
+	golang.org/x/mod v0.38.0 // indirect
 	golang.org/x/net v0.57.0 // indirect
 	golang.org/x/oauth2 v0.36.0 // indirect
 	golang.org/x/sync v0.22.0 // indirect
 	golang.org/x/sys v0.47.0 // indirect
 	golang.org/x/term v0.45.0 // indirect
-	golang.org/x/text v0.40.0 // indirect
+	golang.org/x/text v0.41.0 // indirect
 	golang.org/x/time v0.15.0 // indirect
-	golang.org/x/tools v0.47.0 // indirect
+	golang.org/x/tools v0.48.0 // indirect
 	gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
 	google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20260630182238-925bb5da69e7 // indirect

go.sum

@@ -28,6 +28,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/digitalocean/go-libvirt v0.0.0-20260814190004-1a83157e1858 h1:8xCFt73OddCD5WXxoYidJBWp3bJMC+CpE7Zwf5tcwk8=
+github.com/digitalocean/go-libvirt v0.0.0-20260814190004-1a83157e1858/go.mod h1:qb0Ofa71d3oXARQf633h2tNaeBxLsVxuDp+jcsVO2+4=
 github.com/emicklei/go-restful/v3 v3.13.0 h1:C4Bl2xDndpU6nJ4bc1jXd+uTmYPVUwkD6bFY/oTyCes=
 github.com/emicklei/go-restful/v3 v3.13.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
 github.com/evanphx/json-patch v0.5.2 h1:xVCHIVMUu1wtM/VkR9jVZ45N3FhZfYMMYGorLCR8P3k=
@@ -96,6 +98,8 @@ github.com/joshdk/go-junit v1.0.0 h1:S86cUKIdwBHWwA6xCmFlf3RTLfVXYQfvanM5Uh+K6GE
 github.com/joshdk/go-junit v1.0.0/go.mod h1:TiiV0PqkaNfFXjEiyjWM3XXrhVyCa1K4Zfga6W52ung=
 github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
+github.com/kdomanski/iso9660 v0.4.0 h1:BPKKdcINz3m0MdjIMwS0wx1nofsOjxOq8TOr45WGHFg=
+github.com/kdomanski/iso9660 v0.4.0/go.mod h1:OxUSupHsO9ceI8lBLPJKWBTphLemjrCQY8LPXM7qSzU=
 github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
 github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
 github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
@@ -198,12 +202,12 @@ go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
 go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
 go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
-golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
-golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
+golang.org/x/crypto v0.56.0 h1:GUh5Ii4J5jtcseSMiRqr1jXCNHoxjeV9Fmekc2oLy6Y=
+golang.org/x/crypto v0.56.0/go.mod h1:OMW5y6CY9l38uPLmxU6l6pwcXp1obtLo3e6gT7gQR2I=
 golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93 h1:fQsdNF2N+/YewlRZiricy4P1iimyPKZ/xwniHj8Q2a0=
 golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93/go.mod h1:EPRbTFwzwjXj9NpYyyrvenVh9Y+GFeEvMNh7Xuz7xgU=
-golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
-golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
+golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk=
+golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40=
 golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
 golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
 golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
@@ -214,12 +218,12 @@ golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
 golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
 golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
 golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
-golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
-golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
+golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
+golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
 golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
 golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
-golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
-golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
+golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=
+golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk=
 gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw=
 gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY=
 gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=

internal/provider/config.go

@@ -21,6 +21,23 @@ type ProviderConfig struct {
 	Type       string            `json:"type"`
 	Kubernetes *KubernetesConfig `json:"kubernetes,omitempty"`
 	GCP        *GCPConfig        `json:"gcp,omitempty"`
+	KVM        *KVMConfig        `json:"kvm,omitempty"`
+}
+
+// setBlocks names the type-specific blocks that are non-nil, so validate can
+// reject a block that doesn't match Type without a pairwise check per type.
+func (p ProviderConfig) setBlocks() []string {
+	var out []string
+	if p.Kubernetes != nil {
+		out = append(out, "kubernetes")
+	}
+	if p.GCP != nil {
+		out = append(out, "gcp")
+	}
+	if p.KVM != nil {
+		out = append(out, "kvm")
+	}
+	return out
 }
 
 // KubernetesConfig configures the kubernetes-pod provider, which creates
@@ -81,22 +98,28 @@ func (c *Config) validate() error {
 		}
 		seen[p.Name] = true
 
+		prefix := fmt.Sprintf("providers[%d] %q", i, p.Name)
 		switch p.Type {
-		case "kubernetes":
-			if p.GCP != nil {
-				return fmt.Errorf("providers[%d] %q: type is kubernetes but a gcp block is set", i, p.Name)
-			}
-		case "gcp":
-			if p.Kubernetes != nil {
-				return fmt.Errorf("providers[%d] %q: type is gcp but a kubernetes block is set", i, p.Name)
-			}
-			if p.GCP == nil || p.GCP.Project == "" {
-				return fmt.Errorf("providers[%d] %q: gcp.project is required", i, p.Name)
-			}
+		case "kubernetes", "gcp", "kvm":
 		case "":
-			return fmt.Errorf("providers[%d] %q: type is required", i, p.Name)
+			return fmt.Errorf("%s: type is required", prefix)
 		default:
-			return fmt.Errorf("providers[%d] %q: unknown provider type %q", i, p.Name, p.Type)
+			return fmt.Errorf("%s: unknown provider type %q", prefix, p.Type)
+		}
+		for _, block := range p.setBlocks() {
+			if block != p.Type {
+				return fmt.Errorf("%s: type is %s but a %s block is set", prefix, p.Type, block)
+			}
+		}
+		switch p.Type {
+		case "gcp":
+			if p.GCP == nil || p.GCP.Project == "" {
+				return fmt.Errorf("%s: gcp.project is required", prefix)
+			}
+		case "kvm":
+			if err := validateKVM(prefix, p.KVM); err != nil {
+				return err
+			}
 		}
 	}
 	return nil

internal/provider/config_kvm.go

@@ -0,0 +1,138 @@
+package provider
+
+import (
+	"fmt"
+	"net/url"
+	"regexp"
+)
+
+// KVMConfig configures a kvm provider instance: one or more libvirt/KVM
+// hypervisors reached over SSH (the libvirt unix socket is forwarded through
+// the SSH connection, so hosts need no libvirtd TCP/TLS listener).
+// placement.zone on a Proxy selects a host by name; with a single host it may
+// be omitted.
+type KVMConfig struct {
+	// SSHKeyFile is the path to the private key used for every host. Required.
+	SSHKeyFile string `json:"sshKeyFile"`
+	// KnownHostsFile is an OpenSSH known_hosts file used to verify host keys.
+	// Required unless InsecureIgnoreHostKey is set.
+	KnownHostsFile string `json:"knownHostsFile,omitempty"`
+	// InsecureIgnoreHostKey skips host-key verification entirely.
+	InsecureIgnoreHostKey bool `json:"insecureIgnoreHostKey,omitempty"`
+	// DiskSizeGB is the capacity of each VM's overlay disk. Default 10. The
+	// guest's cloud-init growpart expands the root filesystem into it.
+	DiskSizeGB int64 `json:"diskSizeGb,omitempty"`
+	// VendorData is cloud-init vendor-data placed on every seed next to the
+	// Proxy's own user-data. Default: installs and starts qemu-guest-agent so
+	// the provider can read the VM's IP even before it shows up in ARP.
+	VendorData string `json:"vendorData,omitempty"`
+	// Images are named base images; placement.image names a key here (or is
+	// an http(s) URL directly). The provider downloads each image into a
+	// host's storage pool on first use and caches it there.
+	Images map[string]KVMImage `json:"images,omitempty"`
+	// MachineTypes are named size profiles; placement.machineType names a
+	// key here. Required, at least one.
+	MachineTypes map[string]KVMMachineType `json:"machineTypes"`
+	// Hosts are the hypervisors. Required, at least one, unique names.
+	Hosts []KVMHost `json:"hosts"`
+}
+
+// KVMImage is one downloadable base image (a distro cloud image with
+// cloud-init preinstalled, e.g. Debian's generic qcow2).
+type KVMImage struct {
+	// URL is where to fetch the image from. Required, http or https.
+	URL string `json:"url"`
+	// SHA256 is the expected hex digest of the file. Optional; when set it is
+	// verified after download and a later change triggers a re-download.
+	SHA256 string `json:"sha256,omitempty"`
+	// Format is the image's disk format, "qcow2" (default) or "raw".
+	Format string `json:"format,omitempty"`
+}
+
+// KVMMachineType is a VM size profile.
+type KVMMachineType struct {
+	VCPUs     int32 `json:"vcpus"`
+	MemoryMiB int64 `json:"memoryMiB"`
+}
+
+// KVMHost is one hypervisor.
+type KVMHost struct {
+	// Name is the placement.zone value that selects this host. Required.
+	Name string `json:"name"`
+	// Address is the SSH endpoint, host:port. Required.
+	Address string `json:"address"`
+	// User is the SSH user; it must be allowed to use the libvirt socket
+	// (typically a member of the libvirt group). Required.
+	User string `json:"user"`
+	// Socket is the libvirt unix socket path on the host.
+	// Default /var/run/libvirt/libvirt-sock.
+	Socket string `json:"socket,omitempty"`
+	// Pool is the libvirt storage pool for base images, overlay disks and
+	// seed ISOs. Default "default".
+	Pool string `json:"pool,omitempty"`
+	// Bridge is the host bridge the VM's NIC attaches to; it must lead to a
+	// network the cluster can reach and that hands out DHCP leases. Required.
+	Bridge string `json:"bridge"`
+}
+
+var sha256Hex = regexp.MustCompile(`^[0-9a-fA-F]{64}$`)
+
+func validateKVM(prefix string, c *KVMConfig) error {
+	if c == nil {
+		return fmt.Errorf("%s: kvm block is required", prefix)
+	}
+	if c.SSHKeyFile == "" {
+		return fmt.Errorf("%s: kvm.sshKeyFile is required", prefix)
+	}
+	if c.KnownHostsFile == "" && !c.InsecureIgnoreHostKey {
+		return fmt.Errorf("%s: kvm.knownHostsFile is required unless kvm.insecureIgnoreHostKey is true", prefix)
+	}
+	if c.DiskSizeGB < 0 {
+		return fmt.Errorf("%s: kvm.diskSizeGb must not be negative", prefix)
+	}
+	for name, img := range c.Images {
+		p := fmt.Sprintf("%s: kvm.images[%q]", prefix, name)
+		u, err := url.Parse(img.URL)
+		if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
+			return fmt.Errorf("%s.url must be an http(s) URL (got %q)", p, img.URL)
+		}
+		if img.SHA256 != "" && !sha256Hex.MatchString(img.SHA256) {
+			return fmt.Errorf("%s.sha256 must be 64 hex characters", p)
+		}
+		if img.Format != "" && img.Format != "qcow2" && img.Format != "raw" {
+			return fmt.Errorf("%s.format must be qcow2 or raw (got %q)", p, img.Format)
+		}
+	}
+	if len(c.MachineTypes) == 0 {
+		return fmt.Errorf("%s: kvm.machineTypes must define at least one profile", prefix)
+	}
+	for name, mt := range c.MachineTypes {
+		if mt.VCPUs <= 0 || mt.MemoryMiB <= 0 {
+			return fmt.Errorf("%s: kvm.machineTypes[%q] needs vcpus > 0 and memoryMiB > 0", prefix, name)
+		}
+	}
+	if len(c.Hosts) == 0 {
+		return fmt.Errorf("%s: kvm.hosts must list at least one host", prefix)
+	}
+	seen := make(map[string]bool, len(c.Hosts))
+	for i, h := range c.Hosts {
+		p := fmt.Sprintf("%s: kvm.hosts[%d]", prefix, i)
+		if h.Name == "" {
+			return fmt.Errorf("%s: name is required", p)
+		}
+		if seen[h.Name] {
+			return fmt.Errorf("%s: duplicate host name %q", p, h.Name)
+		}
+		seen[h.Name] = true
+		if h.Address == "" {
+			return fmt.Errorf("%s %q: address is required", p, h.Name)
+		}
+		if h.User == "" {
+			return fmt.Errorf("%s %q: user is required", p, h.Name)
+		}
+		if h.Bridge == "" {
+			return fmt.Errorf("%s %q: bridge is required", p, h.Name)
+		}
+	}
+	return nil
+}

internal/provider/config_kvm_test.go

@@ -0,0 +1,208 @@
+package provider
+
+import (
+	"strings"
+	"testing"
+)
+
+const validKVMYAML = `
+providers:
+  - name: kvm-home
+    type: kvm
+    kvm:
+      sshKeyFile: /etc/proxy-operator/kvm/id_ed25519
+      knownHostsFile: /etc/proxy-operator/kvm/known_hosts
+      diskSizeGb: 20
+      images:
+        debian-13:
+          url: https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2
+          sha256: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+        ubuntu:
+          url: http://mirror.local/noble.img
+          format: raw
+      machineTypes:
+        small: {vcpus: 1, memoryMiB: 1024}
+        medium: {vcpus: 2, memoryMiB: 2048}
+      hosts:
+        - name: nas1
+          address: nas1.home.lan:22
+          user: virt
+          bridge: br0
+        - name: lab2
+          address: 10.0.0.2:2222
+          user: root
+          socket: /run/libvirt/virtqemud-sock
+          pool: fast
+          bridge: br1
+`
+
+func TestLoadConfig_kvmValid(t *testing.T) {
+	t.Parallel()
+	cfg, err := LoadConfig([]byte(validKVMYAML))
+	if err != nil {
+		t.Fatalf("LoadConfig() error = %v, want nil", err)
+	}
+	k := cfg.Providers[0].KVM
+	if k == nil {
+		t.Fatal("providers[0].kvm = nil")
+	}
+	if k.SSHKeyFile != "/etc/proxy-operator/kvm/id_ed25519" || k.DiskSizeGB != 20 {
+		t.Errorf("kvm = %+v, want sshKeyFile and diskSizeGb parsed", k)
+	}
+	if got := k.Images["debian-13"].SHA256; !strings.HasPrefix(got, "0123") {
+		t.Errorf("images[debian-13].sha256 = %q", got)
+	}
+	if k.Images["ubuntu"].Format != "raw" {
+		t.Errorf("images[ubuntu].format = %q, want raw", k.Images["ubuntu"].Format)
+	}
+	if mt := k.MachineTypes["medium"]; mt.VCPUs != 2 || mt.MemoryMiB != 2048 {
+		t.Errorf("machineTypes[medium] = %+v, want {2 2048}", mt)
+	}
+	if len(k.Hosts) != 2 || k.Hosts[1].Socket != "/run/libvirt/virtqemud-sock" || k.Hosts[1].Pool != "fast" {
+		t.Errorf("hosts = %+v, want two hosts with lab2's socket and pool parsed", k.Hosts)
+	}
+}
+
+func TestLoadConfig_kvmInsecureHostKeyNeedsNoKnownHosts(t *testing.T) {
+	t.Parallel()
+	yaml := strings.Replace(validKVMYAML,
+		"knownHostsFile: /etc/proxy-operator/kvm/known_hosts",
+		"insecureIgnoreHostKey: true", 1)
+	if _, err := LoadConfig([]byte(yaml)); err != nil {
+		t.Fatalf("LoadConfig() error = %v, want nil", err)
+	}
+}
+
+func TestLoadConfig_kvmInvalid(t *testing.T) {
+	t.Parallel()
+	minimal := func(kvmBody string) string {
+		return "providers:\n  - name: kvm-home\n    type: kvm\n    kvm:\n" + indent(kvmBody, 6)
+	}
+	hostsAndTypes := `
+machineTypes:
+  small: {vcpus: 1, memoryMiB: 1024}
+hosts:
+  - {name: nas1, address: nas1:22, user: virt, bridge: br0}
+`
+	tests := []struct {
+		name       string
+		yaml       string
+		wantErrSub string
+	}{
+		{
+			name:       "missing kvm block",
+			yaml:       "providers:\n  - name: kvm-home\n    type: kvm\n",
+			wantErrSub: "kvm block is required",
+		},
+		{
+			name:       "missing sshKeyFile",
+			yaml:       minimal("knownHostsFile: /kh\n" + hostsAndTypes),
+			wantErrSub: "kvm.sshKeyFile is required",
+		},
+		{
+			name:       "missing knownHostsFile without insecure flag",
+			yaml:       minimal("sshKeyFile: /key\n" + hostsAndTypes),
+			wantErrSub: "kvm.knownHostsFile is required unless",
+		},
+		{
+			name:       "negative disk size",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\ndiskSizeGb: -1\n" + hostsAndTypes),
+			wantErrSub: "diskSizeGb must not be negative",
+		},
+		{
+			name:       "no machine types",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nhosts:\n  - {name: nas1, address: nas1:22, user: virt, bridge: br0}\n"),
+			wantErrSub: "machineTypes must define at least one",
+		},
+		{
+			name:       "machine type with zero memory",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nmachineTypes:\n  small: {vcpus: 1}\nhosts:\n  - {name: nas1, address: nas1:22, user: virt, bridge: br0}\n"),
+			wantErrSub: `machineTypes["small"] needs vcpus > 0 and memoryMiB > 0`,
+		},
+		{
+			name:       "no hosts",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nmachineTypes:\n  small: {vcpus: 1, memoryMiB: 1024}\n"),
+			wantErrSub: "kvm.hosts must list at least one host",
+		},
+		{
+			name:       "duplicate host name",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nmachineTypes:\n  small: {vcpus: 1, memoryMiB: 1024}\nhosts:\n  - {name: nas1, address: a:22, user: virt, bridge: br0}\n  - {name: nas1, address: b:22, user: virt, bridge: br0}\n"),
+			wantErrSub: `duplicate host name "nas1"`,
+		},
+		{
+			name:       "host missing address",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nmachineTypes:\n  small: {vcpus: 1, memoryMiB: 1024}\nhosts:\n  - {name: nas1, user: virt, bridge: br0}\n"),
+			wantErrSub: "address is required",
+		},
+		{
+			name:       "host missing user",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nmachineTypes:\n  small: {vcpus: 1, memoryMiB: 1024}\nhosts:\n  - {name: nas1, address: a:22, bridge: br0}\n"),
+			wantErrSub: "user is required",
+		},
+		{
+			name:       "host missing bridge",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nmachineTypes:\n  small: {vcpus: 1, memoryMiB: 1024}\nhosts:\n  - {name: nas1, address: a:22, user: virt}\n"),
+			wantErrSub: "bridge is required",
+		},
+		{
+			name:       "image with non-http url",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nimages:\n  d: {url: file:///tmp/x.qcow2}\n" + hostsAndTypes),
+			wantErrSub: `images["d"].url must be an http(s) URL`,
+		},
+		{
+			name:       "image with bad sha256",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nimages:\n  d: {url: https://x/y.qcow2, sha256: abc}\n" + hostsAndTypes),
+			wantErrSub: `images["d"].sha256 must be 64 hex characters`,
+		},
+		{
+			name:       "image with unknown format",
+			yaml:       minimal("sshKeyFile: /key\nknownHostsFile: /kh\nimages:\n  d: {url: https://x/y.vmdk, format: vmdk}\n" + hostsAndTypes),
+			wantErrSub: `images["d"].format must be qcow2 or raw`,
+		},
+		{
+			name: "kvm type with gcp block",
+			yaml: `
+providers:
+  - name: p1
+    type: kvm
+    gcp:
+      project: my-project`,
+			wantErrSub: "type is kvm but a gcp block is set",
+		},
+		{
+			name: "gcp type with kvm block",
+			yaml: `
+providers:
+  - name: p1
+    type: gcp
+    gcp:
+      project: my-project
+    kvm:
+      sshKeyFile: /key`,
+			wantErrSub: "type is gcp but a kvm block is set",
+		},
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			_, err := LoadConfig([]byte(tc.yaml))
+			if err == nil {
+				t.Fatalf("LoadConfig() error = nil, want error containing %q", tc.wantErrSub)
+			}
+			if !strings.Contains(err.Error(), tc.wantErrSub) {
+				t.Errorf("LoadConfig() error = %q, want substring %q", err.Error(), tc.wantErrSub)
+			}
+		})
+	}
+}
+
+func indent(s string, n int) string {
+	pad := strings.Repeat(" ", n)
+	lines := strings.Split(strings.TrimLeft(s, "\n"), "\n")
+	for i, l := range lines {
+		if l != "" {
+			lines[i] = pad + l
+		}
+	}
+	return strings.Join(lines, "\n")
+}

internal/provider/kvm/conn.go

@@ -0,0 +1,301 @@
+package kvm
+
+import (
+	"context"
+	"fmt"
+	"io"
+	"net"
+	"os"
+	"sync"
+	"time"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+	"golang.org/x/crypto/ssh"
+	"golang.org/x/crypto/ssh/knownhosts"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+const (
+	defaultHandshakeTimeout = 30 * time.Second
+	keepaliveInterval       = 15 * time.Second
+)
+
+// libvirtAPI is the slice of *libvirt.Libvirt the provider uses, and the
+// test seam: the fake in kvm_test.go implements exactly this.
+type libvirtAPI interface {
+	DomainLookupByName(name string) (libvirt.Domain, error)
+	DomainDefineXML(xml string) (libvirt.Domain, error)
+	DomainCreate(d libvirt.Domain) error
+	DomainDestroy(d libvirt.Domain) error
+	DomainUndefineFlags(d libvirt.Domain, flags libvirt.DomainUndefineFlagsValues) error
+	DomainGetState(d libvirt.Domain, flags uint32) (state int32, reason int32, err error)
+	DomainGetMetadata(d libvirt.Domain, typ int32, uri libvirt.OptString, flags libvirt.DomainModificationImpact) (string, error)
+	DomainInterfaceAddresses(d libvirt.Domain, source uint32, flags uint32) ([]libvirt.DomainInterface, error)
+	ConnectListAllDomains(needResults int32, flags libvirt.ConnectListAllDomainsFlags) ([]libvirt.Domain, uint32, error)
+	StoragePoolLookupByName(name string) (libvirt.StoragePool, error)
+	StoragePoolRefresh(pool libvirt.StoragePool, flags uint32) error
+	StorageVolLookupByName(pool libvirt.StoragePool, name string) (libvirt.StorageVol, error)
+	StorageVolGetPath(vol libvirt.StorageVol) (string, error)
+	StorageVolCreateXML(pool libvirt.StoragePool, xml string, flags libvirt.StorageVolCreateFlags) (libvirt.StorageVol, error)
+	StorageVolUpload(vol libvirt.StorageVol, r io.Reader, offset, length uint64, flags libvirt.StorageVolUploadFlags) error
+	StorageVolDownload(vol libvirt.StorageVol, w io.Writer, offset, length uint64, flags libvirt.StorageVolDownloadFlags) error
+	StorageVolDelete(vol libvirt.StorageVol, flags libvirt.StorageVolDeleteFlags) error
+}
+
+// hostClient is one live connection to a hypervisor.
+type hostClient struct {
+	api libvirtAPI
+	// done is closed once the connection is known to be lost. A nil channel
+	// never fires, which suits fakes.
+	done  <-chan struct{}
+	close func()
+}
+
+func (c *hostClient) alive() bool {
+	select {
+	case <-c.done:
+		return false
+	default:
+		return true
+	}
+}
+
+// hostDialer opens a connection to one host. The SSH implementation is the
+// real one; tests inject their own.
+type hostDialer func(ctx context.Context, host provider.KVMHost) (*hostClient, error)
+
+// hostPool caches one connection per host, dialing lazily and redialing
+// after the connection drops. Dials for different hosts don't block each
+// other; concurrent callers for the same host share a single dial.
+type hostPool struct {
+	dial hostDialer
+
+	mu      sync.Mutex
+	entries map[string]*hostEntry
+}
+
+type hostEntry struct {
+	mu     sync.Mutex
+	client *hostClient
+}
+
+func newHostPool(dial hostDialer) *hostPool {
+	return &hostPool{dial: dial, entries: make(map[string]*hostEntry)}
+}
+
+func (p *hostPool) entry(name string) *hostEntry {
+	p.mu.Lock()
+	defer p.mu.Unlock()
+	e, ok := p.entries[name]
+	if !ok {
+		e = &hostEntry{}
+		p.entries[name] = e
+	}
+	return e
+}
+
+func (p *hostPool) get(ctx context.Context, host provider.KVMHost) (*hostClient, error) {
+	e := p.entry(host.Name)
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if e.client != nil {
+		if e.client.alive() {
+			return e.client, nil
+		}
+		e.client.close()
+		e.client = nil
+	}
+	c, err := p.dial(ctx, host)
+	if err != nil {
+		return nil, fmt.Errorf("connecting to host %s (%s): %w", host.Name, host.Address, err)
+	}
+	e.client = c
+	return c, nil
+}
+
+// drop closes and forgets c if it is still the cached client for host, so a
+// connection that was already replaced by a redial is left alone.
+func (p *hostPool) drop(host string, c *hostClient) {
+	e := p.entry(host)
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if e.client == c {
+		e.client = nil
+	}
+	c.close()
+}
+
+// call runs fn against host's connection. go-libvirt RPCs take no context
+// and block until the daemon answers, so when ctx ends first the connection
+// is closed — which unblocks fn with an error — and ctx.Err() is returned.
+// A transport error (anything that is not a libvirt.Error) also drops the
+// cached connection so the next call redials instead of reusing a dead
+// link.
+func (p *hostPool) call(ctx context.Context, host provider.KVMHost, fn func(api libvirtAPI) error) error {
+	c, err := p.get(ctx, host)
+	if err != nil {
+		return err
+	}
+	err = awaitOrCancel(ctx, func() error { return fn(c.api) }, func() { p.drop(host.Name, c) })
+	if err != nil && !isLibvirtError(err) {
+		p.drop(host.Name, c)
+	}
+	return err
+}
+
+// awaitOrCancel runs fn in the background and returns its result, unless ctx
+// ends first: then abort is invoked (it must make fn return eventually, e.g.
+// by closing the connection fn is blocked on) and ctx.Err() is returned. The
+// abandoned fn's result is discarded.
+func awaitOrCancel(ctx context.Context, fn func() error, abort func()) error {
+	errc := make(chan error, 1)
+	go func() { errc <- fn() }()
+	select {
+	case err := <-errc:
+		return err
+	case <-ctx.Done():
+		abort()
+		return ctx.Err()
+	}
+}
+
+// sshAuth holds the pieces of the SSH client config shared by every host:
+// the private key and the host-key policy, both loaded once at startup so a
+// missing or unreadable file fails the operator fast rather than on the
+// first Create.
+type sshAuth struct {
+	signer  ssh.Signer
+	hostKey ssh.HostKeyCallback
+}
+
+func newSSHAuth(cfg provider.KVMConfig) (*sshAuth, error) {
+	key, err := os.ReadFile(cfg.SSHKeyFile)
+	if err != nil {
+		return nil, fmt.Errorf("reading kvm.sshKeyFile: %w", err)
+	}
+	signer, err := ssh.ParsePrivateKey(key)
+	if err != nil {
+		return nil, fmt.Errorf("parsing kvm.sshKeyFile: %w", err)
+	}
+	hostKey := ssh.InsecureIgnoreHostKey() //nolint:gosec // opt-in via config, documented as insecure
+	if !cfg.InsecureIgnoreHostKey {
+		hostKey, err = knownhosts.New(cfg.KnownHostsFile)
+		if err != nil {
+			return nil, fmt.Errorf("loading kvm.knownHostsFile: %w", err)
+		}
+	}
+	return &sshAuth{signer: signer, hostKey: hostKey}, nil
+}
+
+// dialer returns the production hostDialer: SSH to the host, forward its
+// libvirt unix socket through the connection, and open a libvirt session
+// over that.
+func (a *sshAuth) dialer() hostDialer {
+	return func(ctx context.Context, host provider.KVMHost) (*hostClient, error) {
+		client, err := a.dialSSH(ctx, host)
+		if err != nil {
+			return nil, err
+		}
+		hc, err := connectLibvirt(ctx, client, host.Socket)
+		if err != nil {
+			client.Close()
+			return nil, err
+		}
+		return hc, nil
+	}
+}
+
+// dialSSH establishes the SSH connection. The handshake is bounded by ctx's
+// deadline, or defaultHandshakeTimeout when ctx has none, since a
+// half-open hypervisor must not pin a reconcile worker indefinitely.
+func (a *sshAuth) dialSSH(ctx context.Context, host provider.KVMHost) (*ssh.Client, error) {
+	deadline, ok := ctx.Deadline()
+	if !ok {
+		deadline = time.Now().Add(defaultHandshakeTimeout)
+	}
+	var d net.Dialer
+	nc, err := d.DialContext(ctx, "tcp", host.Address)
+	if err != nil {
+		return nil, fmt.Errorf("ssh dial: %w", err)
+	}
+	_ = nc.SetDeadline(deadline)
+	conn, chans, reqs, err := ssh.NewClientConn(nc, host.Address, &ssh.ClientConfig{
+		User:            host.User,
+		Auth:            []ssh.AuthMethod{ssh.PublicKeys(a.signer)},
+		HostKeyCallback: a.hostKey,
+	})
+	if err != nil {
+		nc.Close()
+		return nil, fmt.Errorf("ssh handshake: %w", err)
+	}
+	_ = nc.SetDeadline(time.Time{})
+	return ssh.NewClient(conn, chans, reqs), nil
+}
+
+// unixForward adapts an SSH client to go-libvirt's socket.Dialer: each Dial
+// opens a direct-streamlocal channel to the libvirt socket on the host.
+type unixForward struct {
+	client *ssh.Client
+	path   string
+}
+
+func (u unixForward) Dial() (net.Conn, error) {
+	return u.client.Dial("unix", u.path)
+}
+
+// connectLibvirt opens the libvirt session over the SSH client and starts a
+// keepalive loop so a dead link is noticed within roughly two intervals
+// rather than at the TCP stack's leisure. Closing the returned client tears
+// down the SSH connection, which in turn ends the libvirt session.
+func connectLibvirt(ctx context.Context, client *ssh.Client, socket string) (*hostClient, error) {
+	lv := libvirt.NewWithDialer(unixForward{client: client, path: socket})
+	if err := awaitOrCancel(ctx, lv.Connect, func() { client.Close() }); err != nil {
+		return nil, fmt.Errorf("libvirt connect via %s: %w", socket, err)
+	}
+	stop := make(chan struct{})
+	var once sync.Once
+	go keepalive(client, keepaliveInterval, stop)
+	return &hostClient{
+		api:  lv,
+		done: lv.Disconnected(),
+		close: func() {
+			once.Do(func() {
+				close(stop)
+				client.Close()
+			})
+		},
+	}, nil
+}
+
+// keepalive sends OpenSSH keepalive requests until stop closes; a failed or
+// unanswered request closes the client, which surfaces as Disconnected on
+// the libvirt side and makes the pool redial.
+func keepalive(client *ssh.Client, interval time.Duration, stop <-chan struct{}) {
+	t := time.NewTicker(interval)
+	defer t.Stop()
+	for {
+		select {
+		case <-stop:
+			return
+		case <-t.C:
+		}
+		replied := make(chan error, 1)
+		go func() {
+			_, _, err := client.SendRequest("keepalive@openssh.com", true, nil)
+			replied <- err
+		}()
+		select {
+		case <-stop:
+			return
+		case err := <-replied:
+			if err != nil {
+				client.Close()
+				return
+			}
+		case <-time.After(interval):
+			client.Close()
+			return
+		}
+	}
+}

internal/provider/kvm/conn_test.go

@@ -0,0 +1,569 @@
+package kvm
+
+import (
+	"bytes"
+	"context"
+	"crypto/ed25519"
+	"crypto/rand"
+	"encoding/pem"
+	"errors"
+	"io"
+	"net"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+	"golang.org/x/crypto/ssh"
+	"golang.org/x/crypto/ssh/knownhosts"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+// --- hostPool over a fake dialer ---
+
+type fakeClient struct {
+	done   chan struct{}
+	closed atomic.Bool
+}
+
+func (c *fakeClient) close() {
+	if c.closed.CompareAndSwap(false, true) {
+		close(c.done)
+	}
+}
+
+type fakeDialer struct {
+	mu      sync.Mutex
+	dials   int
+	err     error
+	delay   time.Duration
+	clients []*fakeClient
+}
+
+func (d *fakeDialer) dial(ctx context.Context, _ provider.KVMHost) (*hostClient, error) {
+	if d.delay > 0 {
+		select {
+		case <-time.After(d.delay):
+		case <-ctx.Done():
+			return nil, ctx.Err()
+		}
+	}
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	d.dials++
+	if d.err != nil {
+		return nil, d.err
+	}
+	fc := &fakeClient{done: make(chan struct{})}
+	d.clients = append(d.clients, fc)
+	return &hostClient{api: nil, done: fc.done, close: fc.close}, nil
+}
+
+func (d *fakeDialer) dialCount() int {
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	return d.dials
+}
+
+var testHost = provider.KVMHost{Name: "nas1", Address: "nas1:22", User: "virt", Socket: defaultSocket, Pool: "default", Bridge: "br0"}
+
+func noop(libvirtAPI) error { return nil }
+
+func TestHostPool_call_reusesConnection(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{}
+	p := newHostPool(d.dial)
+	for range 3 {
+		if err := p.call(context.Background(), testHost, noop); err != nil {
+			t.Fatalf("call() error = %v", err)
+		}
+	}
+	if d.dialCount() != 1 {
+		t.Errorf("dials = %d, want 1", d.dialCount())
+	}
+}
+
+func TestHostPool_call_redialsAfterDisconnect(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{}
+	p := newHostPool(d.dial)
+	if err := p.call(context.Background(), testHost, noop); err != nil {
+		t.Fatalf("call() error = %v", err)
+	}
+	d.clients[0].close()
+	if err := p.call(context.Background(), testHost, noop); err != nil {
+		t.Fatalf("second call() error = %v", err)
+	}
+	if d.dialCount() != 2 {
+		t.Errorf("dials = %d, want 2 after the first connection dropped", d.dialCount())
+	}
+}
+
+func TestHostPool_call_transportErrorDropsConnection(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{}
+	p := newHostPool(d.dial)
+	err := p.call(context.Background(), testHost, func(libvirtAPI) error { return io.ErrUnexpectedEOF })
+	if !errors.Is(err, io.ErrUnexpectedEOF) {
+		t.Fatalf("call() error = %v, want the transport error passed through", err)
+	}
+	if !d.clients[0].closed.Load() {
+		t.Error("connection not closed after a transport error")
+	}
+	if err := p.call(context.Background(), testHost, noop); err != nil {
+		t.Fatalf("call() after transport error = %v", err)
+	}
+	if d.dialCount() != 2 {
+		t.Errorf("dials = %d, want 2", d.dialCount())
+	}
+}
+
+func TestHostPool_call_libvirtErrorKeepsConnection(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{}
+	p := newHostPool(d.dial)
+	err := p.call(context.Background(), testHost, func(libvirtAPI) error { return lvErr(libvirt.ErrNoDomain) })
+	if !hasCode(err, libvirt.ErrNoDomain) {
+		t.Fatalf("call() error = %v, want the libvirt error passed through", err)
+	}
+	if err := p.call(context.Background(), testHost, noop); err != nil {
+		t.Fatalf("call() error = %v", err)
+	}
+	if d.dialCount() != 1 {
+		t.Errorf("dials = %d, want 1: a daemon-reported error must not cost the connection", d.dialCount())
+	}
+	if d.clients[0].closed.Load() {
+		t.Error("connection closed after a libvirt error")
+	}
+}
+
+func TestHostPool_call_dialErrorPropagates(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{err: errors.New("ssh: handshake failed")}
+	p := newHostPool(d.dial)
+	err := p.call(context.Background(), testHost, noop)
+	if err == nil || !strings.Contains(err.Error(), "nas1") || !strings.Contains(err.Error(), "handshake failed") {
+		t.Errorf("call() error = %v, want the dial error naming the host", err)
+	}
+}
+
+func TestHostPool_call_cancelAbandonsBlockedCall(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{}
+	p := newHostPool(d.dial)
+	ctx, cancel := context.WithCancel(context.Background())
+	released := make(chan struct{})
+	go func() {
+		time.Sleep(20 * time.Millisecond)
+		cancel()
+	}()
+	err := p.call(ctx, testHost, func(libvirtAPI) error {
+		// Simulates an RPC stuck on a hung hypervisor: it only returns once
+		// the pool closes the connection out from under it.
+		<-d.clients[0].done
+		close(released)
+		return io.EOF
+	})
+	if !errors.Is(err, context.Canceled) {
+		t.Fatalf("call() error = %v, want context.Canceled", err)
+	}
+	select {
+	case <-released:
+	case <-time.After(time.Second):
+		t.Fatal("blocked call was not released by closing the connection")
+	}
+	if err := p.call(context.Background(), testHost, noop); err != nil {
+		t.Fatalf("call() after cancel = %v", err)
+	}
+	if d.dialCount() != 2 {
+		t.Errorf("dials = %d, want 2", d.dialCount())
+	}
+}
+
+func TestHostPool_call_concurrentCallersShareOneDial(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{delay: 30 * time.Millisecond}
+	p := newHostPool(d.dial)
+	var wg sync.WaitGroup
+	for range 8 {
+		wg.Go(func() {
+			if err := p.call(context.Background(), testHost, noop); err != nil {
+				t.Errorf("call() error = %v", err)
+			}
+		})
+	}
+	wg.Wait()
+	if d.dialCount() != 1 {
+		t.Errorf("dials = %d, want 1", d.dialCount())
+	}
+}
+
+func TestHostPool_call_hostsAreIndependent(t *testing.T) {
+	t.Parallel()
+	d := &fakeDialer{}
+	p := newHostPool(d.dial)
+	other := testHost
+	other.Name = "lab2"
+	for _, h := range []provider.KVMHost{testHost, other, testHost, other} {
+		if err := p.call(context.Background(), h, noop); err != nil {
+			t.Fatalf("call(%s) error = %v", h.Name, err)
+		}
+	}
+	if d.dialCount() != 2 {
+		t.Errorf("dials = %d, want one per host", d.dialCount())
+	}
+}
+
+// --- SSH dial against an in-process server ---
+
+func newKeyPair(t *testing.T) (ssh.Signer, []byte) {
+	t.Helper()
+	_, priv, err := ed25519.GenerateKey(rand.Reader)
+	if err != nil {
+		t.Fatal(err)
+	}
+	signer, err := ssh.NewSignerFromKey(priv)
+	if err != nil {
+		t.Fatal(err)
+	}
+	block, err := ssh.MarshalPrivateKey(priv, "")
+	if err != nil {
+		t.Fatal(err)
+	}
+	return signer, pem.EncodeToMemory(block)
+}
+
+// testSSHServer accepts one public key and serves
+// direct-streamlocal@openssh.com channels by dialing the requested unix
+// path, which is exactly what OpenSSH does for ssh.Client.Dial("unix").
+type testSSHServer struct {
+	addr    string
+	hostKey ssh.PublicKey
+
+	mu    sync.Mutex
+	paths []string
+}
+
+func startSSHServer(t *testing.T, clientKey ssh.PublicKey, answerGlobalRequests bool) *testSSHServer {
+	t.Helper()
+	hostSigner, _ := newKeyPair(t)
+	cfg := &ssh.ServerConfig{
+		PublicKeyCallback: func(_ ssh.ConnMetadata, key ssh.PublicKey) (*ssh.Permissions, error) {
+			if bytes.Equal(key.Marshal(), clientKey.Marshal()) {
+				return nil, nil
+			}
+			return nil, errors.New("unknown public key")
+		},
+	}
+	cfg.AddHostKey(hostSigner)
+	ln, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatal(err)
+	}
+	t.Cleanup(func() { ln.Close() })
+	s := &testSSHServer{addr: ln.Addr().String(), hostKey: hostSigner.PublicKey()}
+	go func() {
+		for {
+			nc, err := ln.Accept()
+			if err != nil {
+				return
+			}
+			go s.serve(nc, cfg, answerGlobalRequests)
+		}
+	}()
+	return s
+}
+
+func (s *testSSHServer) serve(nc net.Conn, cfg *ssh.ServerConfig, answerGlobalRequests bool) {
+	_, chans, reqs, err := ssh.NewServerConn(nc, cfg)
+	if err != nil {
+		return
+	}
+	if answerGlobalRequests {
+		go ssh.DiscardRequests(reqs)
+	} else {
+		go func() {
+			for range reqs { //nolint:revive // deliberately never replies, to stall keepalives
+			}
+		}()
+	}
+	for newCh := range chans {
+		if newCh.ChannelType() != "direct-streamlocal@openssh.com" {
+			_ = newCh.Reject(ssh.UnknownChannelType, "test server only forwards unix sockets")
+			continue
+		}
+		var payload struct {
+			Path      string
+			Reserved0 string
+			Reserved1 uint32
+		}
+		if err := ssh.Unmarshal(newCh.ExtraData(), &payload); err != nil {
+			_ = newCh.Reject(ssh.ConnectionFailed, err.Error())
+			continue
+		}
+		s.mu.Lock()
+		s.paths = append(s.paths, payload.Path)
+		s.mu.Unlock()
+		target, err := net.Dial("unix", payload.Path)
+		if err != nil {
+			_ = newCh.Reject(ssh.ConnectionFailed, err.Error())
+			continue
+		}
+		ch, chReqs, err := newCh.Accept()
+		if err != nil {
+			target.Close()
+			continue
+		}
+		go ssh.DiscardRequests(chReqs)
+		go func() { _, _ = io.Copy(ch, target); ch.Close() }()
+		go func() { _, _ = io.Copy(target, ch); target.Close() }()
+	}
+}
+
+func (s *testSSHServer) requestedPaths() []string {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return append([]string(nil), s.paths...)
+}
+
+// startEchoSocket serves a unix socket that echoes everything back, standing
+// in for libvirt-sock.
+func startEchoSocket(t *testing.T) string {
+	t.Helper()
+	path := filepath.Join(t.TempDir(), "libvirt-sock")
+	ln, err := net.Listen("unix", path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	t.Cleanup(func() { ln.Close() })
+	go func() {
+		for {
+			c, err := ln.Accept()
+			if err != nil {
+				return
+			}
+			go func() { _, _ = io.Copy(c, c); c.Close() }()
+		}
+	}()
+	return path
+}
+
+type sshFixture struct {
+	server  *testSSHServer
+	cfg     provider.KVMConfig
+	host    provider.KVMHost
+	keyFile string
+}
+
+func newSSHFixture(t *testing.T, answerGlobalRequests bool) *sshFixture {
+	t.Helper()
+	clientSigner, clientPEM := newKeyPair(t)
+	server := startSSHServer(t, clientSigner.PublicKey(), answerGlobalRequests)
+	dir := t.TempDir()
+	keyFile := filepath.Join(dir, "id_ed25519")
+	if err := os.WriteFile(keyFile, clientPEM, 0o600); err != nil {
+		t.Fatal(err)
+	}
+	khFile := filepath.Join(dir, "known_hosts")
+	line := knownhosts.Line([]string{server.addr}, server.hostKey)
+	if err := os.WriteFile(khFile, []byte(line+"\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	return &sshFixture{
+		server:  server,
+		keyFile: keyFile,
+		cfg:     provider.KVMConfig{SSHKeyFile: keyFile, KnownHostsFile: khFile},
+		host:    provider.KVMHost{Name: "nas1", Address: server.addr, User: "virt", Socket: startEchoSocket(t)},
+	}
+}
+
+func TestSSHAuth_dialSSH_forwardsUnixSocket(t *testing.T) {
+	t.Parallel()
+	f := newSSHFixture(t, true)
+	auth, err := newSSHAuth(f.cfg)
+	if err != nil {
+		t.Fatalf("newSSHAuth() error = %v", err)
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+	defer cancel()
+	client, err := auth.dialSSH(ctx, f.host)
+	if err != nil {
+		t.Fatalf("dialSSH() error = %v", err)
+	}
+	defer client.Close()
+
+	conn, err := unixForward{client: client, path: f.host.Socket}.Dial()
+	if err != nil {
+		t.Fatalf("unixForward.Dial() error = %v", err)
+	}
+	defer conn.Close()
+	if _, err := conn.Write([]byte("ping")); err != nil {
+		t.Fatal(err)
+	}
+	buf := make([]byte, 4)
+	if _, err := io.ReadFull(conn, buf); err != nil || string(buf) != "ping" {
+		t.Fatalf("echo through the forward = %q, %v; want ping", buf, err)
+	}
+	if paths := f.server.requestedPaths(); len(paths) != 1 || paths[0] != f.host.Socket {
+		t.Errorf("server saw forward requests %v, want [%s]", paths, f.host.Socket)
+	}
+}
+
+func TestSSHAuth_dialSSH_rejectsUnknownHostKey(t *testing.T) {
+	t.Parallel()
+	f := newSSHFixture(t, true)
+	impostor, _ := newKeyPair(t)
+	line := knownhosts.Line([]string{f.server.addr}, impostor.PublicKey())
+	if err := os.WriteFile(f.cfg.KnownHostsFile, []byte(line+"\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	auth, err := newSSHAuth(f.cfg)
+	if err != nil {
+		t.Fatalf("newSSHAuth() error = %v", err)
+	}
+	client, err := auth.dialSSH(context.Background(), f.host)
+	if err == nil {
+		client.Close()
+		t.Fatal("dialSSH() succeeded against a host whose key is not in known_hosts")
+	}
+	var keyErr *knownhosts.KeyError
+	if !errors.As(err, &keyErr) {
+		t.Errorf("dialSSH() error = %v, want a knownhosts.KeyError", err)
+	}
+}
+
+func TestSSHAuth_dialSSH_insecureIgnoreHostKey(t *testing.T) {
+	t.Parallel()
+	f := newSSHFixture(t, true)
+	f.cfg.KnownHostsFile = ""
+	f.cfg.InsecureIgnoreHostKey = true
+	auth, err := newSSHAuth(f.cfg)
+	if err != nil {
+		t.Fatalf("newSSHAuth() error = %v", err)
+	}
+	client, err := auth.dialSSH(context.Background(), f.host)
+	if err != nil {
+		t.Fatalf("dialSSH() error = %v", err)
+	}
+	client.Close()
+}
+
+func TestSSHAuth_dialSSH_rejectsWrongClientKey(t *testing.T) {
+	t.Parallel()
+	f := newSSHFixture(t, true)
+	_, otherPEM := newKeyPair(t)
+	if err := os.WriteFile(f.keyFile, otherPEM, 0o600); err != nil {
+		t.Fatal(err)
+	}
+	auth, err := newSSHAuth(f.cfg)
+	if err != nil {
+		t.Fatalf("newSSHAuth() error = %v", err)
+	}
+	client, err := auth.dialSSH(context.Background(), f.host)
+	if err == nil {
+		client.Close()
+		t.Fatal("dialSSH() succeeded with a key the server does not know")
+	}
+	if !strings.Contains(err.Error(), "ssh handshake") {
+		t.Errorf("dialSSH() error = %v, want an auth failure from the handshake", err)
+	}
+}
+
+func TestSSHAuth_dialSSH_cancelledContext(t *testing.T) {
+	t.Parallel()
+	f := newSSHFixture(t, true)
+	auth, err := newSSHAuth(f.cfg)
+	if err != nil {
+		t.Fatalf("newSSHAuth() error = %v", err)
+	}
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel()
+	if client, err := auth.dialSSH(ctx, f.host); err == nil {
+		client.Close()
+		t.Fatal("dialSSH() succeeded with an already-cancelled context")
+	}
+}
+
+func TestNewSSHAuth_missingFiles(t *testing.T) {
+	t.Parallel()
+	_, keyPEM := newKeyPair(t)
+	keyFile := filepath.Join(t.TempDir(), "key")
+	if err := os.WriteFile(keyFile, keyPEM, 0o600); err != nil {
+		t.Fatal(err)
+	}
+	tests := []struct {
+		name    string
+		cfg     provider.KVMConfig
+		wantSub string
+	}{
+		{"key file missing", provider.KVMConfig{SSHKeyFile: "/nonexistent/key", InsecureIgnoreHostKey: true}, "sshKeyFile"},
+		{"key file garbage", provider.KVMConfig{SSHKeyFile: keyFile + ".bad", InsecureIgnoreHostKey: true}, "sshKeyFile"},
+		{"known_hosts missing", provider.KVMConfig{SSHKeyFile: keyFile, KnownHostsFile: "/nonexistent/kh"}, "knownHostsFile"},
+	}
+	if err := os.WriteFile(keyFile+".bad", []byte("not a key"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			_, err := newSSHAuth(tc.cfg)
+			if err == nil || !strings.Contains(err.Error(), tc.wantSub) {
+				t.Errorf("newSSHAuth() error = %v, want one mentioning %s", err, tc.wantSub)
+			}
+		})
+	}
+}
+
+func TestKeepalive_closesClientWhenServerStalls(t *testing.T) {
+	t.Parallel()
+	f := newSSHFixture(t, false)
+	auth, err := newSSHAuth(f.cfg)
+	if err != nil {
+		t.Fatalf("newSSHAuth() error = %v", err)
+	}
+	client, err := auth.dialSSH(context.Background(), f.host)
+	if err != nil {
+		t.Fatalf("dialSSH() error = %v", err)
+	}
+	defer client.Close()
+	stop := make(chan struct{})
+	defer close(stop)
+	go keepalive(client, 20*time.Millisecond, stop)
+
+	closed := make(chan struct{})
+	go func() { _ = client.Wait(); close(closed) }()
+	select {
+	case <-closed:
+	case <-time.After(2 * time.Second):
+		t.Fatal("client still open after keepalives went unanswered")
+	}
+}
+
+func TestKeepalive_keepsHealthyClientOpen(t *testing.T) {
+	t.Parallel()
+	f := newSSHFixture(t, true)
+	auth, err := newSSHAuth(f.cfg)
+	if err != nil {
+		t.Fatalf("newSSHAuth() error = %v", err)
+	}
+	client, err := auth.dialSSH(context.Background(), f.host)
+	if err != nil {
+		t.Fatalf("dialSSH() error = %v", err)
+	}
+	defer client.Close()
+	stop := make(chan struct{})
+	defer close(stop)
+	go keepalive(client, 10*time.Millisecond, stop)
+	time.Sleep(60 * time.Millisecond)
+	fwd := unixForward{client: client, path: f.host.Socket}
+	conn, err := fwd.Dial()
+	if err != nil {
+		t.Fatalf("forward after several keepalives failed: %v", err)
+	}
+	conn.Close()
+}

internal/provider/kvm/domainxml.go

@@ -0,0 +1,224 @@
+package kvm
+
+import (
+	"encoding/xml"
+	"fmt"
+	"time"
+)
+
+const guestAgentChannel = "org.qemu.guest_agent.0"
+
+// domainParams is everything buildDomainXML needs, already resolved from the
+// config and placement so the builder stays a pure function.
+type domainParams struct {
+	Name      string
+	UID       string
+	CreatedAt time.Time
+	VCPUs     int32
+	MemoryMiB int64
+	Pool      string
+	Bridge    string
+	DiskVol   string
+	SeedVol   string
+}
+
+// buildDomainXML renders a q35 KVM guest: host-passthrough CPU, a virtio
+// overlay disk and a SATA cdrom for the seed (both referenced as pool
+// volumes so no host paths are needed), one virtio NIC on the host bridge,
+// a serial console, the guest-agent channel, a virtio RNG, no graphics, and
+// the operator's metadata element.
+func buildDomainXML(p domainParams) (string, error) {
+	d := domainXML{
+		Type:     "kvm",
+		Name:     p.Name,
+		Metadata: domainMetadataXML{Instance: newInstanceMetadata(p.UID, p.CreatedAt)},
+		Memory:   sizeXML{Unit: "MiB", Value: p.MemoryMiB},
+		VCPU:     p.VCPUs,
+		OS: osXML{
+			Type: osTypeXML{Arch: "x86_64", Machine: "q35", Value: "hvm"},
+			Boot: bootXML{Dev: "hd"},
+		},
+		Features:   featuresXML{ACPI: &struct{}{}, APIC: &struct{}{}},
+		CPU:        cpuXML{Mode: "host-passthrough"},
+		Clock:      clockXML{Offset: "utc"},
+		OnPoweroff: "destroy",
+		OnReboot:   "restart",
+		OnCrash:    "destroy",
+		Devices: devicesXML{
+			Disks: []diskXML{
+				{
+					Type:   "volume",
+					Device: "disk",
+					Driver: driverXML{Name: "qemu", Type: formatQcow2},
+					Source: diskSourceXML{Pool: p.Pool, Volume: p.DiskVol},
+					Target: targetXML{Dev: "vda", Bus: "virtio"},
+				},
+				{
+					Type:     "volume",
+					Device:   "cdrom",
+					Driver:   driverXML{Name: "qemu", Type: formatRaw},
+					Source:   diskSourceXML{Pool: p.Pool, Volume: p.SeedVol},
+					Target:   targetXML{Dev: "sda", Bus: "sata"},
+					ReadOnly: &struct{}{},
+				},
+			},
+			Interfaces: []interfaceXML{{
+				Type:   "bridge",
+				Source: ifaceSourceXML{Bridge: p.Bridge},
+				Model:  modelXML{Type: "virtio"},
+			}},
+			Serials:  []serialXML{{Type: "pty", Target: portXML{Port: 0}}},
+			Consoles: []consoleXML{{Type: "pty", Target: consoleTargetXML{Type: "serial", Port: 0}}},
+			Channels: []channelXML{{
+				Type:   "unix",
+				Target: channelTargetXML{Type: "virtio", Name: guestAgentChannel},
+			}},
+			RNG: rngXML{Model: "virtio", Backend: rngBackendXML{Model: "random", Value: "/dev/urandom"}},
+		},
+	}
+	out, err := xml.Marshal(d)
+	if err != nil {
+		return "", fmt.Errorf("marshalling domain xml: %w", err)
+	}
+	return string(out), nil
+}
+
+type domainXML struct {
+	XMLName    xml.Name          `xml:"domain"`
+	Type       string            `xml:"type,attr"`
+	Name       string            `xml:"name"`
+	Metadata   domainMetadataXML `xml:"metadata"`
+	Memory     sizeXML           `xml:"memory"`
+	VCPU       int32             `xml:"vcpu"`
+	OS         osXML             `xml:"os"`
+	Features   featuresXML       `xml:"features"`
+	CPU        cpuXML            `xml:"cpu"`
+	Clock      clockXML          `xml:"clock"`
+	OnPoweroff string            `xml:"on_poweroff"`
+	OnReboot   string            `xml:"on_reboot"`
+	OnCrash    string            `xml:"on_crash"`
+	Devices    devicesXML        `xml:"devices"`
+}
+
+type domainMetadataXML struct {
+	Instance instanceMetadata
+}
+
+type sizeXML struct {
+	Unit  string `xml:"unit,attr"`
+	Value int64  `xml:",chardata"`
+}
+
+type osXML struct {
+	Type osTypeXML `xml:"type"`
+	Boot bootXML   `xml:"boot"`
+}
+
+type osTypeXML struct {
+	Arch    string `xml:"arch,attr"`
+	Machine string `xml:"machine,attr"`
+	Value   string `xml:",chardata"`
+}
+
+type bootXML struct {
+	Dev string `xml:"dev,attr"`
+}
+
+type featuresXML struct {
+	ACPI *struct{} `xml:"acpi"`
+	APIC *struct{} `xml:"apic"`
+}
+
+type cpuXML struct {
+	Mode string `xml:"mode,attr"`
+}
+
+type clockXML struct {
+	Offset string `xml:"offset,attr"`
+}
+
+type devicesXML struct {
+	Disks      []diskXML      `xml:"disk"`
+	Interfaces []interfaceXML `xml:"interface"`
+	Serials    []serialXML    `xml:"serial"`
+	Consoles   []consoleXML   `xml:"console"`
+	Channels   []channelXML   `xml:"channel"`
+	RNG        rngXML         `xml:"rng"`
+}
+
+type diskXML struct {
+	Type     string        `xml:"type,attr"`
+	Device   string        `xml:"device,attr"`
+	Driver   driverXML     `xml:"driver"`
+	Source   diskSourceXML `xml:"source"`
+	Target   targetXML     `xml:"target"`
+	ReadOnly *struct{}     `xml:"readonly"`
+}
+
+type driverXML struct {
+	Name string `xml:"name,attr"`
+	Type string `xml:"type,attr"`
+}
+
+type diskSourceXML struct {
+	Pool   string `xml:"pool,attr"`
+	Volume string `xml:"volume,attr"`
+}
+
+type targetXML struct {
+	Dev string `xml:"dev,attr"`
+	Bus string `xml:"bus,attr"`
+}
+
+type interfaceXML struct {
+	Type   string         `xml:"type,attr"`
+	Source ifaceSourceXML `xml:"source"`
+	Model  modelXML       `xml:"model"`
+}
+
+type ifaceSourceXML struct {
+	Bridge string `xml:"bridge,attr"`
+}
+
+type modelXML struct {
+	Type string `xml:"type,attr"`
+}
+
+type serialXML struct {
+	Type   string  `xml:"type,attr"`
+	Target portXML `xml:"target"`
+}
+
+type portXML struct {
+	Port int `xml:"port,attr"`
+}
+
+type consoleXML struct {
+	Type   string           `xml:"type,attr"`
+	Target consoleTargetXML `xml:"target"`
+}
+
+type consoleTargetXML struct {
+	Type string `xml:"type,attr"`
+	Port int    `xml:"port,attr"`
+}
+
+type channelXML struct {
+	Type   string           `xml:"type,attr"`
+	Target channelTargetXML `xml:"target"`
+}
+
+type channelTargetXML struct {
+	Type string `xml:"type,attr"`
+	Name string `xml:"name,attr"`
+}
+
+type rngXML struct {
+	Model   string        `xml:"model,attr"`
+	Backend rngBackendXML `xml:"backend"`
+}
+
+type rngBackendXML struct {
+	Model string `xml:"model,attr"`
+	Value string `xml:",chardata"`
+}

internal/provider/kvm/domainxml_test.go

@@ -0,0 +1,113 @@
+package kvm
+
+import (
+	"encoding/xml"
+	"strings"
+	"testing"
+	"time"
+)
+
+func testDomainParams() domainParams {
+	return domainParams{
+		Name:      "proxy-abc123def456ghij",
+		UID:       "11111111-2222-3333-4444-555555555555",
+		CreatedAt: time.Date(2026, 9, 4, 12, 0, 0, 0, time.UTC),
+		VCPUs:     2,
+		MemoryMiB: 2048,
+		Pool:      "default",
+		Bridge:    "br0",
+		DiskVol:   "proxy-abc123def456ghij.qcow2",
+		SeedVol:   "proxy-abc123def456ghij-seed.iso",
+	}
+}
+
+func TestBuildDomainXML_fieldByField(t *testing.T) {
+	t.Parallel()
+	raw, err := buildDomainXML(testDomainParams())
+	if err != nil {
+		t.Fatalf("buildDomainXML() error = %v", err)
+	}
+	var d domainXML
+	if err := xml.Unmarshal([]byte(raw), &d); err != nil {
+		t.Fatalf("output is not well-formed XML: %v\n%s", err, raw)
+	}
+
+	if d.Type != "kvm" || d.Name != "proxy-abc123def456ghij" {
+		t.Errorf("type/name = %s/%s", d.Type, d.Name)
+	}
+	if d.VCPU != 2 || d.Memory.Value != 2048 || d.Memory.Unit != "MiB" {
+		t.Errorf("vcpu/memory = %d/%d %s, want 2/2048 MiB", d.VCPU, d.Memory.Value, d.Memory.Unit)
+	}
+	if d.OS.Type.Value != "hvm" || d.OS.Type.Arch != "x86_64" || d.OS.Type.Machine != "q35" || d.OS.Boot.Dev != "hd" {
+		t.Errorf("os = %+v", d.OS)
+	}
+	if d.Features.ACPI == nil || d.Features.APIC == nil {
+		t.Errorf("features = %+v, want acpi and apic", d.Features)
+	}
+	if d.CPU.Mode != "host-passthrough" {
+		t.Errorf("cpu mode = %q", d.CPU.Mode)
+	}
+	if d.OnPoweroff != "destroy" || d.OnReboot != "restart" || d.OnCrash != "destroy" {
+		t.Errorf("lifecycle = %s/%s/%s", d.OnPoweroff, d.OnReboot, d.OnCrash)
+	}
+
+	m := d.Metadata.Instance
+	if !m.managed() || m.UID != "11111111-2222-3333-4444-555555555555" || m.CreatedAt != "2026-09-04T12:00:00Z" {
+		t.Errorf("metadata = %+v", m)
+	}
+	if !strings.Contains(raw, `xmlns="`+metadataURI+`"`) {
+		t.Errorf("metadata element is not namespaced; libvirt rejects that:\n%s", raw)
+	}
+
+	if len(d.Devices.Disks) != 2 {
+		t.Fatalf("disks = %d, want 2", len(d.Devices.Disks))
+	}
+	disk, seed := d.Devices.Disks[0], d.Devices.Disks[1]
+	if disk.Type != "volume" || disk.Device != "disk" || disk.Driver.Type != "qcow2" ||
+		disk.Source.Pool != "default" || disk.Source.Volume != "proxy-abc123def456ghij.qcow2" ||
+		disk.Target.Dev != "vda" || disk.Target.Bus != "virtio" || disk.ReadOnly != nil {
+		t.Errorf("overlay disk = %+v", disk)
+	}
+	if seed.Type != "volume" || seed.Device != "cdrom" || seed.Driver.Type != "raw" ||
+		seed.Source.Pool != "default" || seed.Source.Volume != "proxy-abc123def456ghij-seed.iso" ||
+		seed.Target.Bus != "sata" || seed.ReadOnly == nil {
+		t.Errorf("seed cdrom = %+v", seed)
+	}
+
+	if len(d.Devices.Interfaces) != 1 {
+		t.Fatalf("interfaces = %d, want 1", len(d.Devices.Interfaces))
+	}
+	if nic := d.Devices.Interfaces[0]; nic.Type != "bridge" || nic.Source.Bridge != "br0" || nic.Model.Type != "virtio" {
+		t.Errorf("interface = %+v", nic)
+	}
+
+	if len(d.Devices.Channels) != 1 || d.Devices.Channels[0].Target.Name != guestAgentChannel || d.Devices.Channels[0].Target.Type != "virtio" {
+		t.Errorf("channels = %+v, want one virtio %s channel", d.Devices.Channels, guestAgentChannel)
+	}
+	if len(d.Devices.Serials) != 1 || len(d.Devices.Consoles) != 1 || d.Devices.Consoles[0].Target.Type != "serial" {
+		t.Errorf("serial/console = %+v/%+v", d.Devices.Serials, d.Devices.Consoles)
+	}
+	if d.Devices.RNG.Model != "virtio" || d.Devices.RNG.Backend.Value != "/dev/urandom" {
+		t.Errorf("rng = %+v", d.Devices.RNG)
+	}
+	if strings.Contains(raw, "<graphics") {
+		t.Errorf("domain has a graphics device; headless VMs must not:\n%s", raw)
+	}
+}
+
+func TestBuildDomainXML_escapesUserValues(t *testing.T) {
+	t.Parallel()
+	p := testDomainParams()
+	p.Bridge = `br0" onload="x`
+	raw, err := buildDomainXML(p)
+	if err != nil {
+		t.Fatalf("buildDomainXML() error = %v", err)
+	}
+	var d domainXML
+	if err := xml.Unmarshal([]byte(raw), &d); err != nil {
+		t.Fatalf("output is not well-formed XML: %v", err)
+	}
+	if d.Devices.Interfaces[0].Source.Bridge != p.Bridge {
+		t.Errorf("bridge = %q, want the raw value back", d.Devices.Interfaces[0].Source.Bridge)
+	}
+}

internal/provider/kvm/errors.go

@@ -0,0 +1,72 @@
+package kvm
+
+import (
+	"errors"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+	"github.com/go-logr/logr"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+// classify maps an error onto the provider taxonomy. Errors the package
+// already tagged with a sentinel (image download failures, placement
+// errors) keep their class. For libvirt errors: missing domain or volume →
+// NotFound; anything the host will keep rejecting (bad XML, unsupported
+// config, denied access, missing pool) → Permanent; every other libvirt
+// code, plus SSH/transport failures and cancellation, → Transient, because
+// retrying is safer than latching Failed on an error nobody taught this
+// function to recognise. libvirt exposes no quota signal, so nothing maps
+// to QuotaExceeded.
+func classify(err error) error {
+	for _, class := range []error{provider.ErrNotFound, provider.ErrQuotaExceeded, provider.ErrPermanent, provider.ErrTransient} {
+		if errors.Is(err, class) {
+			return class
+		}
+	}
+	var lerr libvirt.Error
+	if !errors.As(err, &lerr) {
+		return provider.ErrTransient
+	}
+	switch libvirt.ErrorNumber(lerr.Code) {
+	case libvirt.ErrNoDomain, libvirt.ErrNoStorageVol:
+		return provider.ErrNotFound
+	case libvirt.ErrNoStoragePool,
+		libvirt.ErrXMLError, libvirt.ErrXMLDetail, libvirt.ErrXMLInvalidSchema,
+		libvirt.ErrConfigUnsupported, libvirt.ErrNoSupport,
+		libvirt.ErrOperationUnsupported, libvirt.ErrArgumentUnsupported,
+		libvirt.ErrInvalidArg,
+		libvirt.ErrAuthFailed, libvirt.ErrAccessDenied, libvirt.ErrOperationDenied:
+		return provider.ErrPermanent
+	default:
+		return provider.ErrTransient
+	}
+}
+
+// isLibvirtError reports whether err came back from the libvirt daemon (as
+// opposed to the SSH or RPC transport), which is what decides whether the
+// cached connection is still trustworthy.
+func isLibvirtError(err error) bool {
+	var lerr libvirt.Error
+	return errors.As(err, &lerr)
+}
+
+func hasCode(err error, code libvirt.ErrorNumber) bool {
+	var lerr libvirt.Error
+	return errors.As(err, &lerr) && libvirt.ErrorNumber(lerr.Code) == code
+}
+
+// logAPIError records the raw libvirt code and message at V(1); classify
+// collapses them onto the coarser taxonomy, so this is the only place the
+// original detail survives.
+func logAPIError(log logr.Logger, op string, err error) {
+	if !log.V(1).Enabled() {
+		return
+	}
+	kv := []any{"op", op, "error", err.Error()}
+	var lerr libvirt.Error
+	if errors.As(err, &lerr) {
+		kv = append(kv, "libvirtCode", lerr.Code, "libvirtMessage", lerr.Message)
+	}
+	log.V(1).Info("libvirt call failed", kv...)
+}

internal/provider/kvm/errors_test.go

@@ -0,0 +1,69 @@
+package kvm
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"testing"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+func lvErr(code libvirt.ErrorNumber) error {
+	return libvirt.Error{Code: uint32(code), Message: code.String()}
+}
+
+func TestClassify(t *testing.T) {
+	t.Parallel()
+	tests := []struct {
+		name string
+		err  error
+		want error
+	}{
+		{"no domain", lvErr(libvirt.ErrNoDomain), provider.ErrNotFound},
+		{"no storage vol", lvErr(libvirt.ErrNoStorageVol), provider.ErrNotFound},
+		{"no storage pool", lvErr(libvirt.ErrNoStoragePool), provider.ErrPermanent},
+		{"xml error", lvErr(libvirt.ErrXMLError), provider.ErrPermanent},
+		{"xml detail", lvErr(libvirt.ErrXMLDetail), provider.ErrPermanent},
+		{"config unsupported", lvErr(libvirt.ErrConfigUnsupported), provider.ErrPermanent},
+		{"no support", lvErr(libvirt.ErrNoSupport), provider.ErrPermanent},
+		{"invalid arg", lvErr(libvirt.ErrInvalidArg), provider.ErrPermanent},
+		{"auth failed", lvErr(libvirt.ErrAuthFailed), provider.ErrPermanent},
+		{"access denied", lvErr(libvirt.ErrAccessDenied), provider.ErrPermanent},
+		{"operation invalid", lvErr(libvirt.ErrOperationInvalid), provider.ErrTransient},
+		{"operation failed", lvErr(libvirt.ErrOperationFailed), provider.ErrTransient},
+		{"internal error", lvErr(libvirt.ErrInternalError), provider.ErrTransient},
+		{"storage vol exists", lvErr(libvirt.ErrStorageVolExist), provider.ErrTransient},
+		{"wrapped libvirt error", fmt.Errorf("defining: %w", lvErr(libvirt.ErrNoDomain)), provider.ErrNotFound},
+		{"pre-tagged permanent", fmt.Errorf("%w: image returned HTTP 404", provider.ErrPermanent), provider.ErrPermanent},
+		{"pre-tagged transient wrapping libvirt not-found", fmt.Errorf("%w: %w", provider.ErrTransient, lvErr(libvirt.ErrNoDomain)), provider.ErrTransient},
+		{"transport eof", io.EOF, provider.ErrTransient},
+		{"context cancelled", context.Canceled, provider.ErrTransient},
+		{"plain error", errors.New("ssh: handshake failed"), provider.ErrTransient},
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := classify(tc.err); !errors.Is(got, tc.want) {
+				t.Errorf("classify(%v) = %v, want %v", tc.err, got, tc.want)
+			}
+		})
+	}
+}
+
+func TestHasCode(t *testing.T) {
+	t.Parallel()
+	err := fmt.Errorf("lookup: %w", lvErr(libvirt.ErrNoDomain))
+	if !hasCode(err, libvirt.ErrNoDomain) {
+		t.Error("hasCode(ErrNoDomain) = false through a wrap")
+	}
+	if hasCode(err, libvirt.ErrNoStorageVol) {
+		t.Error("hasCode(ErrNoStorageVol) = true for an ErrNoDomain error")
+	}
+	if hasCode(io.EOF, libvirt.ErrNoDomain) || isLibvirtError(io.EOF) {
+		t.Error("io.EOF classified as a libvirt error")
+	}
+}

internal/provider/kvm/fake_test.go

@@ -0,0 +1,408 @@
+package kvm
+
+import (
+	"bytes"
+	"encoding/xml"
+	"fmt"
+	"io"
+	"slices"
+	"sync"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+)
+
+// fakeAPI is an in-memory libvirtAPI: pools of volumes and a set of
+// domains, with per-method error injection. It models just enough libvirt
+// semantics for the provider's decisions to be observable — ErrStorageVolExist
+// on duplicate create, ErrNoDomain/ErrNoStorageVol on missing lookups,
+// ErrOperationInvalid on destroying a stopped domain, and so on.
+type fakeAPI struct {
+	mu       sync.Mutex
+	poolPath string
+	pools    map[string]map[string]*fakeVol
+	domains  map[string]*fakeDomain
+	calls    []string
+	fail     map[string]error
+	failOnce map[string]error
+	// uploadHook, when set, sees every StorageVolUpload before the fake
+	// reads the stream; it can return an error to simulate a broken upload.
+	uploadHook func(vol string, r io.Reader) error
+}
+
+type fakeVol struct {
+	name     string
+	format   string
+	capacity int64
+	backing  string
+	data     []byte
+}
+
+type fakeDomain struct {
+	name     string
+	xml      string
+	metadata string
+	running  bool
+	// addrs is what DomainInterfaceAddresses returns per source.
+	addrs map[uint32][]libvirt.DomainInterface
+}
+
+func newFakeAPI(pools ...string) *fakeAPI {
+	f := &fakeAPI{
+		poolPath: "/var/lib/libvirt/images",
+		pools:    map[string]map[string]*fakeVol{},
+		domains:  map[string]*fakeDomain{},
+		fail:     map[string]error{},
+		failOnce: map[string]error{},
+	}
+	for _, p := range pools {
+		f.pools[p] = map[string]*fakeVol{}
+	}
+	return f
+}
+
+// enter records the call and returns any injected error for method.
+func (f *fakeAPI) enter(method string) error {
+	f.calls = append(f.calls, method)
+	if err, ok := f.failOnce[method]; ok {
+		delete(f.failOnce, method)
+		return err
+	}
+	return f.fail[method]
+}
+
+func (f *fakeAPI) callCount(method string) int {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	n := 0
+	for _, c := range f.calls {
+		if c == method {
+			n++
+		}
+	}
+	return n
+}
+
+func (f *fakeAPI) hasCall(method string) bool { return f.callCount(method) > 0 }
+
+func (f *fakeAPI) vol(pool, name string) *fakeVol {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	return f.pools[pool][name]
+}
+
+func (f *fakeAPI) putVol(pool, name, format string, data []byte) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	f.pools[pool][name] = &fakeVol{name: name, format: format, capacity: int64(len(data)), data: data}
+}
+
+func (f *fakeAPI) volNames(pool string) []string {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	var out []string
+	for n := range f.pools[pool] {
+		out = append(out, n)
+	}
+	slices.Sort(out)
+	return out
+}
+
+func (f *fakeAPI) domain(name string) *fakeDomain {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	return f.domains[name]
+}
+
+func (f *fakeAPI) putDomain(d *fakeDomain) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	f.domains[d.name] = d
+}
+
+func lvError(code libvirt.ErrorNumber, format string, args ...any) error {
+	return libvirt.Error{Code: uint32(code), Message: fmt.Sprintf(format, args...)}
+}
+
+// --- domains ---
+
+func (f *fakeAPI) DomainLookupByName(name string) (libvirt.Domain, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainLookupByName"); err != nil {
+		return libvirt.Domain{}, err
+	}
+	if _, ok := f.domains[name]; !ok {
+		return libvirt.Domain{}, lvError(libvirt.ErrNoDomain, "Domain not found: no domain with matching name '%s'", name)
+	}
+	return libvirt.Domain{Name: name}, nil
+}
+
+func (f *fakeAPI) DomainDefineXML(raw string) (libvirt.Domain, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainDefineXML"); err != nil {
+		return libvirt.Domain{}, err
+	}
+	var d domainXML
+	if err := xml.Unmarshal([]byte(raw), &d); err != nil {
+		return libvirt.Domain{}, lvError(libvirt.ErrXMLError, "%v", err)
+	}
+	meta, err := xml.Marshal(d.Metadata.Instance)
+	if err != nil {
+		return libvirt.Domain{}, lvError(libvirt.ErrXMLError, "%v", err)
+	}
+	existing := f.domains[d.Name]
+	nd := &fakeDomain{name: d.Name, xml: raw, metadata: string(meta)}
+	if existing != nil {
+		nd.running = existing.running
+		nd.addrs = existing.addrs
+	}
+	f.domains[d.Name] = nd
+	return libvirt.Domain{Name: d.Name}, nil
+}
+
+func (f *fakeAPI) DomainCreate(d libvirt.Domain) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainCreate"); err != nil {
+		return err
+	}
+	dom, ok := f.domains[d.Name]
+	if !ok {
+		return lvError(libvirt.ErrNoDomain, "no domain %s", d.Name)
+	}
+	if dom.running {
+		return lvError(libvirt.ErrOperationInvalid, "domain is already running")
+	}
+	dom.running = true
+	return nil
+}
+
+func (f *fakeAPI) DomainDestroy(d libvirt.Domain) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainDestroy"); err != nil {
+		return err
+	}
+	dom, ok := f.domains[d.Name]
+	if !ok {
+		return lvError(libvirt.ErrNoDomain, "no domain %s", d.Name)
+	}
+	if !dom.running {
+		return lvError(libvirt.ErrOperationInvalid, "domain is not running")
+	}
+	dom.running = false
+	return nil
+}
+
+func (f *fakeAPI) DomainUndefineFlags(d libvirt.Domain, _ libvirt.DomainUndefineFlagsValues) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainUndefineFlags"); err != nil {
+		return err
+	}
+	if _, ok := f.domains[d.Name]; !ok {
+		return lvError(libvirt.ErrNoDomain, "no domain %s", d.Name)
+	}
+	delete(f.domains, d.Name)
+	return nil
+}
+
+func (f *fakeAPI) DomainGetState(d libvirt.Domain, _ uint32) (int32, int32, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainGetState"); err != nil {
+		return 0, 0, err
+	}
+	dom, ok := f.domains[d.Name]
+	if !ok {
+		return 0, 0, lvError(libvirt.ErrNoDomain, "no domain %s", d.Name)
+	}
+	if dom.running {
+		return int32(libvirt.DomainRunning), 0, nil
+	}
+	return int32(libvirt.DomainShutoff), 0, nil
+}
+
+func (f *fakeAPI) DomainGetMetadata(d libvirt.Domain, typ int32, uri libvirt.OptString, _ libvirt.DomainModificationImpact) (string, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainGetMetadata"); err != nil {
+		return "", err
+	}
+	dom, ok := f.domains[d.Name]
+	if !ok {
+		return "", lvError(libvirt.ErrNoDomain, "no domain %s", d.Name)
+	}
+	if typ != int32(libvirt.DomainMetadataElement) || len(uri) != 1 || uri[0] != metadataURI || dom.metadata == "" {
+		return "", lvError(libvirt.ErrNoDomainMetadata, "Requested metadata element is not present")
+	}
+	return dom.metadata, nil
+}
+
+func (f *fakeAPI) DomainInterfaceAddresses(d libvirt.Domain, source uint32, _ uint32) ([]libvirt.DomainInterface, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("DomainInterfaceAddresses"); err != nil {
+		return nil, err
+	}
+	dom, ok := f.domains[d.Name]
+	if !ok {
+		return nil, lvError(libvirt.ErrNoDomain, "no domain %s", d.Name)
+	}
+	if !dom.running {
+		return nil, lvError(libvirt.ErrOperationInvalid, "domain is not running")
+	}
+	if source == uint32(libvirt.DomainInterfaceAddressesSrcAgent) && dom.addrs[source] == nil {
+		return nil, lvError(libvirt.ErrAgentUnresponsive, "Guest agent is not responding")
+	}
+	return dom.addrs[source], nil
+}
+
+func (f *fakeAPI) ConnectListAllDomains(_ int32, _ libvirt.ConnectListAllDomainsFlags) ([]libvirt.Domain, uint32, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("ConnectListAllDomains"); err != nil {
+		return nil, 0, err
+	}
+	names := make([]string, 0, len(f.domains))
+	for n := range f.domains {
+		names = append(names, n)
+	}
+	slices.Sort(names)
+	out := make([]libvirt.Domain, 0, len(names))
+	for _, n := range names {
+		out = append(out, libvirt.Domain{Name: n})
+	}
+	return out, uint32(len(out)), nil
+}
+
+// --- storage ---
+
+func (f *fakeAPI) StoragePoolLookupByName(name string) (libvirt.StoragePool, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("StoragePoolLookupByName"); err != nil {
+		return libvirt.StoragePool{}, err
+	}
+	if _, ok := f.pools[name]; !ok {
+		return libvirt.StoragePool{}, lvError(libvirt.ErrNoStoragePool, "Storage pool not found: no storage pool with matching name '%s'", name)
+	}
+	return libvirt.StoragePool{Name: name}, nil
+}
+
+func (f *fakeAPI) StoragePoolRefresh(pool libvirt.StoragePool, _ uint32) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("StoragePoolRefresh"); err != nil {
+		return err
+	}
+	if _, ok := f.pools[pool.Name]; !ok {
+		return lvError(libvirt.ErrNoStoragePool, "no pool %s", pool.Name)
+	}
+	return nil
+}
+
+func (f *fakeAPI) StorageVolLookupByName(pool libvirt.StoragePool, name string) (libvirt.StorageVol, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("StorageVolLookupByName"); err != nil {
+		return libvirt.StorageVol{}, err
+	}
+	if _, ok := f.pools[pool.Name][name]; !ok {
+		return libvirt.StorageVol{}, lvError(libvirt.ErrNoStorageVol, "Storage volume not found: no storage vol with matching name '%s'", name)
+	}
+	return libvirt.StorageVol{Pool: pool.Name, Name: name, Key: f.poolPath + "/" + name}, nil
+}
+
+func (f *fakeAPI) StorageVolGetPath(vol libvirt.StorageVol) (string, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("StorageVolGetPath"); err != nil {
+		return "", err
+	}
+	if _, ok := f.pools[vol.Pool][vol.Name]; !ok {
+		return "", lvError(libvirt.ErrNoStorageVol, "no vol %s", vol.Name)
+	}
+	return f.poolPath + "/" + vol.Name, nil
+}
+
+func (f *fakeAPI) StorageVolCreateXML(pool libvirt.StoragePool, raw string, _ libvirt.StorageVolCreateFlags) (libvirt.StorageVol, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("StorageVolCreateXML"); err != nil {
+		return libvirt.StorageVol{}, err
+	}
+	vols, ok := f.pools[pool.Name]
+	if !ok {
+		return libvirt.StorageVol{}, lvError(libvirt.ErrNoStoragePool, "no pool %s", pool.Name)
+	}
+	var v volumeXML
+	if err := xml.Unmarshal([]byte(raw), &v); err != nil {
+		return libvirt.StorageVol{}, lvError(libvirt.ErrXMLError, "%v", err)
+	}
+	if _, exists := vols[v.Name]; exists {
+		return libvirt.StorageVol{}, lvError(libvirt.ErrStorageVolExist, "storage volume '%s' exists already", v.Name)
+	}
+	fv := &fakeVol{name: v.Name, format: v.Target.Format.Type, capacity: v.Capacity.Value}
+	if v.BackingStore != nil {
+		fv.backing = v.BackingStore.Path
+	}
+	vols[v.Name] = fv
+	return libvirt.StorageVol{Pool: pool.Name, Name: v.Name, Key: f.poolPath + "/" + v.Name}, nil
+}
+
+func (f *fakeAPI) StorageVolUpload(vol libvirt.StorageVol, r io.Reader, _, _ uint64, _ libvirt.StorageVolUploadFlags) error {
+	f.mu.Lock()
+	if err := f.enter("StorageVolUpload"); err != nil {
+		f.mu.Unlock()
+		return err
+	}
+	fv, ok := f.pools[vol.Pool][vol.Name]
+	hook := f.uploadHook
+	f.mu.Unlock()
+	if !ok {
+		return lvError(libvirt.ErrNoStorageVol, "no vol %s", vol.Name)
+	}
+	if hook != nil {
+		if err := hook(vol.Name, r); err != nil {
+			return err
+		}
+	}
+	// Read outside the lock: the stream may be a slow HTTP body.
+	data, err := io.ReadAll(r)
+	if err != nil {
+		return fmt.Errorf("reading upload stream: %w", err)
+	}
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	fv.data = data
+	return nil
+}
+
+func (f *fakeAPI) StorageVolDownload(vol libvirt.StorageVol, w io.Writer, _, _ uint64, _ libvirt.StorageVolDownloadFlags) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("StorageVolDownload"); err != nil {
+		return err
+	}
+	fv, ok := f.pools[vol.Pool][vol.Name]
+	if !ok {
+		return lvError(libvirt.ErrNoStorageVol, "no vol %s", vol.Name)
+	}
+	_, err := io.Copy(w, bytes.NewReader(fv.data))
+	return err
+}
+
+func (f *fakeAPI) StorageVolDelete(vol libvirt.StorageVol, _ libvirt.StorageVolDeleteFlags) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if err := f.enter("StorageVolDelete"); err != nil {
+		return err
+	}
+	if _, ok := f.pools[vol.Pool][vol.Name]; !ok {
+		return lvError(libvirt.ErrNoStorageVol, "no vol %s", vol.Name)
+	}
+	delete(f.pools[vol.Pool], vol.Name)
+	return nil
+}

internal/provider/kvm/id.go

@@ -0,0 +1,21 @@
+package kvm
+
+import (
+	"fmt"
+	"strings"
+)
+
+// The providerID carries the host, so Get/Delete stay correct after a
+// placement.zone edit mid-replacement — re-reading the spec then would look
+// at the wrong hypervisor.
+func formatProviderID(host, name string) string {
+	return fmt.Sprintf("hosts/%s/domains/%s", host, name)
+}
+
+func parseProviderID(id string) (host, name string, err error) {
+	parts := strings.Split(id, "/")
+	if len(parts) != 4 || parts[0] != "hosts" || parts[2] != "domains" || parts[1] == "" || parts[3] == "" {
+		return "", "", fmt.Errorf("malformed kvm providerID %q, want hosts/<host>/domains/<name>", id)
+	}
+	return parts[1], parts[3], nil
+}

internal/provider/kvm/id_test.go

@@ -0,0 +1,41 @@
+package kvm
+
+import "testing"
+
+func TestParseProviderID_roundTrip(t *testing.T) {
+	t.Parallel()
+	id := formatProviderID("nas1", "proxy-abc123def456ghij")
+	if id != "hosts/nas1/domains/proxy-abc123def456ghij" {
+		t.Fatalf("formatProviderID() = %q", id)
+	}
+	host, name, err := parseProviderID(id)
+	if err != nil {
+		t.Fatalf("parseProviderID(%q) error = %v", id, err)
+	}
+	if host != "nas1" || name != "proxy-abc123def456ghij" {
+		t.Errorf("parseProviderID(%q) = %q, %q", id, host, name)
+	}
+}
+
+func TestParseProviderID_malformed(t *testing.T) {
+	t.Parallel()
+	tests := []struct {
+		name string
+		id   string
+	}{
+		{"empty", ""},
+		{"gcp style", "zones/europe-west1-b/instances/proxy-x"},
+		{"missing name", "hosts/nas1/domains/"},
+		{"missing host", "hosts//domains/proxy-x"},
+		{"too many segments", "hosts/nas1/domains/proxy-x/extra"},
+		{"bare name", "proxy-x"},
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if _, _, err := parseProviderID(tc.id); err == nil {
+				t.Errorf("parseProviderID(%q) error = nil, want error", tc.id)
+			}
+		})
+	}
+}

internal/provider/kvm/image.go

@@ -0,0 +1,232 @@
+package kvm
+
+import (
+	"bytes"
+	"context"
+	"crypto/sha256"
+	"encoding/hex"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strings"
+	"sync"
+	"time"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+	"github.com/go-logr/logr"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+// imageSource is a resolved placement.image: either an entry from the
+// provider's images map (Key set) or a URL given directly.
+type imageSource struct {
+	Key    string
+	URL    string
+	SHA256 string
+	Format string
+}
+
+// resolveImage maps placement.image to a source. Anything that is neither a
+// configured key nor an http(s) URL is a spec error the reconciler should
+// surface, not retry.
+func resolveImage(cfg provider.KVMConfig, image string) (imageSource, error) {
+	if img, ok := cfg.Images[image]; ok {
+		return imageSource{Key: image, URL: img.URL, SHA256: img.SHA256, Format: img.Format}, nil
+	}
+	u, err := url.Parse(image)
+	if err == nil && (u.Scheme == "http" || u.Scheme == "https") && u.Host != "" {
+		return imageSource{URL: image, Format: formatQcow2}, nil
+	}
+	return imageSource{}, fmt.Errorf("%w: placement.image %q is neither a key in kvm.images nor an http(s) URL",
+		provider.ErrPermanent, image)
+}
+
+// volumeName is the cached base image's name in the pool: readable for
+// configured keys, a URL hash for ad-hoc URLs.
+func (s imageSource) volumeName() string {
+	if s.Key != "" {
+		return "base-" + s.Key + "." + s.Format
+	}
+	sum := sha256.Sum256([]byte(s.URL))
+	return "base-" + hex.EncodeToString(sum[:8]) + "." + s.Format
+}
+
+// markerName is the sidecar volume holding the sha256 of the verified
+// upload. Its presence is what says "the base volume is complete".
+func (s imageSource) markerName() string {
+	return s.volumeName() + ".sha256"
+}
+
+// imageCache fetches base images into a host's pool on first use. It keeps
+// no state about what is cached — the pool is the source of truth, checked
+// on every call — only a per-(host, image) lock so concurrent Creates for
+// the same image on the same host result in one download.
+type imageCache struct {
+	http *http.Client
+
+	mu    sync.Mutex
+	locks map[string]*sync.Mutex
+}
+
+func newImageCache(client *http.Client) *imageCache {
+	if client == nil {
+		client = http.DefaultClient
+	}
+	return &imageCache{http: client, locks: make(map[string]*sync.Mutex)}
+}
+
+func (c *imageCache) lockFor(key string) *sync.Mutex {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	m, ok := c.locks[key]
+	if !ok {
+		m = &sync.Mutex{}
+		c.locks[key] = m
+	}
+	return m
+}
+
+// ensure makes src present in pool and returns the base volume's path on
+// the host. The image is downloaded synchronously through the libvirt
+// stream when the pool does not already hold a verified copy.
+func (c *imageCache) ensure(ctx context.Context, log logr.Logger, api libvirtAPI, hostName string, pool libvirt.StoragePool, src imageSource) (string, error) {
+	m := c.lockFor(hostName + "/" + src.volumeName())
+	m.Lock()
+	defer m.Unlock()
+
+	base, ready, err := c.lookup(api, pool, src)
+	if err != nil {
+		return "", err
+	}
+	if ready {
+		log.V(1).Info("base image already cached", "host", hostName, "volume", src.volumeName())
+		return api.StorageVolGetPath(base)
+	}
+	if err := deleteIfExists(api, pool, src.markerName()); err != nil {
+		return "", err
+	}
+	if err := deleteIfExists(api, pool, src.volumeName()); err != nil {
+		return "", err
+	}
+	base, err = c.fetch(ctx, log, api, hostName, pool, src)
+	if err != nil {
+		return "", err
+	}
+	return api.StorageVolGetPath(base)
+}
+
+// lookup reports whether the pool holds a complete, verified base volume:
+// the volume and its marker both exist and, when a checksum is configured,
+// the marker records that checksum.
+func (c *imageCache) lookup(api libvirtAPI, pool libvirt.StoragePool, src imageSource) (libvirt.StorageVol, bool, error) {
+	base, err := api.StorageVolLookupByName(pool, src.volumeName())
+	if hasCode(err, libvirt.ErrNoStorageVol) {
+		return libvirt.StorageVol{}, false, nil
+	}
+	if err != nil {
+		return libvirt.StorageVol{}, false, fmt.Errorf("looking up base volume: %w", err)
+	}
+	marker, err := api.StorageVolLookupByName(pool, src.markerName())
+	if hasCode(err, libvirt.ErrNoStorageVol) {
+		return base, false, nil
+	}
+	if err != nil {
+		return libvirt.StorageVol{}, false, fmt.Errorf("looking up image marker: %w", err)
+	}
+	if src.SHA256 == "" {
+		return base, true, nil
+	}
+	var buf bytes.Buffer
+	if err := api.StorageVolDownload(marker, &buf, 0, 0, 0); err != nil {
+		return libvirt.StorageVol{}, false, fmt.Errorf("reading image marker: %w", err)
+	}
+	return base, strings.EqualFold(strings.TrimSpace(buf.String()), src.SHA256), nil
+}
+
+// fetch streams the image from its URL straight into a new base volume,
+// hashing on the way, then writes the marker. Nothing is buffered in the
+// operator beyond the HTTP body's read buffer.
+func (c *imageCache) fetch(ctx context.Context, log logr.Logger, api libvirtAPI, hostName string, pool libvirt.StoragePool, src imageSource) (libvirt.StorageVol, error) {
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, src.URL, nil)
+	if err != nil {
+		return libvirt.StorageVol{}, fmt.Errorf("%w: building image request: %v", provider.ErrPermanent, err)
+	}
+	resp, err := c.http.Do(req)
+	if err != nil {
+		return libvirt.StorageVol{}, fmt.Errorf("%w: fetching image %s: %v", provider.ErrTransient, src.URL, err)
+	}
+	defer resp.Body.Close()
+	switch {
+	case resp.StatusCode >= 200 && resp.StatusCode < 300:
+	case resp.StatusCode >= 400 && resp.StatusCode < 500:
+		return libvirt.StorageVol{}, fmt.Errorf("%w: image %s returned HTTP %d", provider.ErrPermanent, src.URL, resp.StatusCode)
+	default:
+		return libvirt.StorageVol{}, fmt.Errorf("%w: image %s returned HTTP %d", provider.ErrTransient, src.URL, resp.StatusCode)
+	}
+	if resp.ContentLength < 0 {
+		return libvirt.StorageVol{}, fmt.Errorf("%w: image %s has no Content-Length; the base volume needs a known size", provider.ErrPermanent, src.URL)
+	}
+
+	log.Info("fetching base image into host pool",
+		"host", hostName, "pool", pool.Name, "volume", src.volumeName(),
+		"url", src.URL, "bytes", resp.ContentLength)
+	start := time.Now()
+
+	baseXML, err := buildVolumeXML(volumeSpec{Name: src.volumeName(), SizeBytes: resp.ContentLength, Format: src.Format})
+	if err != nil {
+		return libvirt.StorageVol{}, err
+	}
+	base, err := api.StorageVolCreateXML(pool, baseXML, 0)
+	if err != nil {
+		return libvirt.StorageVol{}, fmt.Errorf("creating base volume: %w", err)
+	}
+	hasher := sha256.New()
+	if err := api.StorageVolUpload(base, io.TeeReader(resp.Body, hasher), 0, 0, 0); err != nil {
+		_ = api.StorageVolDelete(base, 0)
+		return libvirt.StorageVol{}, fmt.Errorf("uploading image to base volume: %w", err)
+	}
+	digest := hex.EncodeToString(hasher.Sum(nil))
+	if src.SHA256 != "" && !strings.EqualFold(digest, src.SHA256) {
+		_ = api.StorageVolDelete(base, 0)
+		return libvirt.StorageVol{}, fmt.Errorf("%w: image %s sha256 is %s, config expects %s",
+			provider.ErrPermanent, src.URL, digest, src.SHA256)
+	}
+	// libvirt sized the volume from Content-Length; a refresh makes it
+	// re-read the real virtual size from the uploaded qcow2 header.
+	if err := api.StoragePoolRefresh(pool, 0); err != nil {
+		return libvirt.StorageVol{}, fmt.Errorf("refreshing pool after upload: %w", err)
+	}
+
+	markerXML, err := buildVolumeXML(rawVolume(src.markerName(), int64(len(digest))))
+	if err != nil {
+		return libvirt.StorageVol{}, err
+	}
+	marker, err := api.StorageVolCreateXML(pool, markerXML, 0)
+	if err != nil {
+		return libvirt.StorageVol{}, fmt.Errorf("creating image marker: %w", err)
+	}
+	if err := api.StorageVolUpload(marker, strings.NewReader(digest), 0, 0, 0); err != nil {
+		_ = api.StorageVolDelete(marker, 0)
+		return libvirt.StorageVol{}, fmt.Errorf("writing image marker: %w", err)
+	}
+	log.Info("base image cached",
+		"host", hostName, "volume", src.volumeName(), "sha256", digest,
+		"bytes", resp.ContentLength, "duration", time.Since(start).Round(time.Second))
+	return base, nil
+}
+
+func deleteIfExists(api libvirtAPI, pool libvirt.StoragePool, name string) error {
+	vol, err := api.StorageVolLookupByName(pool, name)
+	if hasCode(err, libvirt.ErrNoStorageVol) {
+		return nil
+	}
+	if err != nil {
+		return fmt.Errorf("looking up volume %s: %w", name, err)
+	}
+	if err := api.StorageVolDelete(vol, 0); err != nil && !hasCode(err, libvirt.ErrNoStorageVol) {
+		return fmt.Errorf("deleting volume %s: %w", name, err)
+	}
+	return nil
+}

internal/provider/kvm/image_test.go

@@ -0,0 +1,385 @@
+package kvm
+
+import (
+	"bytes"
+	"context"
+	"crypto/sha256"
+	"encoding/hex"
+	"errors"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strconv"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/go-logr/logr"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+var testImage = bytes.Repeat([]byte("QFI\xfb fake qcow2 payload "), 1024)
+
+func testImageSHA() string {
+	sum := sha256.Sum256(testImage)
+	return hex.EncodeToString(sum[:])
+}
+
+// imageServer serves testImage and counts hits. Status overrides the
+// response, noLength forces chunked encoding (no Content-Length), delay
+// slows each response so concurrent callers overlap.
+type imageServer struct {
+	*httptest.Server
+	hits     atomic.Int32
+	status   int
+	noLength bool
+	delay    time.Duration
+	body     []byte
+}
+
+func newImageServer(t *testing.T) *imageServer {
+	t.Helper()
+	s := &imageServer{status: http.StatusOK, body: testImage}
+	s.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+		s.hits.Add(1)
+		time.Sleep(s.delay)
+		if s.status != http.StatusOK {
+			w.WriteHeader(s.status)
+			return
+		}
+		// Bodies larger than the server's write buffer would otherwise go
+		// out chunked without a Content-Length, unlike real image mirrors.
+		if s.noLength {
+			w.(http.Flusher).Flush()
+		} else {
+			w.Header().Set("Content-Length", strconv.Itoa(len(s.body)))
+		}
+		_, _ = w.Write(s.body)
+	}))
+	t.Cleanup(s.Close)
+	return s
+}
+
+func testSource(srv *imageServer, sha string) imageSource {
+	return imageSource{Key: "debian-13", URL: srv.URL + "/debian.qcow2", SHA256: sha, Format: formatQcow2}
+}
+
+func ensureTest(t *testing.T, api *fakeAPI, src imageSource) (string, error) {
+	t.Helper()
+	c := newImageCache(nil)
+	pool, err := api.StoragePoolLookupByName("default")
+	if err != nil {
+		t.Fatal(err)
+	}
+	return c.ensure(context.Background(), logr.Discard(), api, "nas1", pool, src)
+}
+
+func TestResolveImage(t *testing.T) {
+	t.Parallel()
+	cfg := withDefaults(provider.KVMConfig{Images: map[string]provider.KVMImage{
+		"debian-13": {URL: "https://cloud.debian.org/d.qcow2", SHA256: "abc"},
+	}})
+	tests := []struct {
+		name    string
+		image   string
+		want    imageSource
+		wantErr bool
+	}{
+		{"configured key", "debian-13", imageSource{Key: "debian-13", URL: "https://cloud.debian.org/d.qcow2", SHA256: "abc", Format: "qcow2"}, false},
+		{"direct https url", "https://x.example/img.qcow2", imageSource{URL: "https://x.example/img.qcow2", Format: "qcow2"}, false},
+		{"direct http url", "http://mirror/img.qcow2", imageSource{URL: "http://mirror/img.qcow2", Format: "qcow2"}, false},
+		{"unknown key", "ubuntu", imageSource{}, true},
+		{"file url", "file:///tmp/x.qcow2", imageSource{}, true},
+		{"empty", "", imageSource{}, true},
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			got, err := resolveImage(cfg, tc.image)
+			if tc.wantErr {
+				if !errors.Is(err, provider.ErrPermanent) {
+					t.Fatalf("resolveImage(%q) error = %v, want ErrPermanent", tc.image, err)
+				}
+				return
+			}
+			if err != nil {
+				t.Fatalf("resolveImage(%q) error = %v", tc.image, err)
+			}
+			if got != tc.want {
+				t.Errorf("resolveImage(%q) = %+v, want %+v", tc.image, got, tc.want)
+			}
+		})
+	}
+}
+
+func TestImageSource_volumeNames(t *testing.T) {
+	t.Parallel()
+	keyed := imageSource{Key: "debian-13", URL: "https://a/b.qcow2", Format: "qcow2"}
+	if keyed.volumeName() != "base-debian-13.qcow2" || keyed.markerName() != "base-debian-13.qcow2.sha256" {
+		t.Errorf("keyed names = %s / %s", keyed.volumeName(), keyed.markerName())
+	}
+	direct := imageSource{URL: "https://a/b.qcow2", Format: "qcow2"}
+	name := direct.volumeName()
+	if !strings.HasPrefix(name, "base-") || !strings.HasSuffix(name, ".qcow2") || len(name) != len("base-")+16+len(".qcow2") {
+		t.Errorf("direct-url name = %q, want base-<16 hex>.qcow2", name)
+	}
+	other := imageSource{URL: "https://a/c.qcow2", Format: "qcow2"}
+	if other.volumeName() == name {
+		t.Error("different URLs produced the same volume name")
+	}
+}
+
+func TestImageCache_ensure_freshFetch(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	path, err := ensureTest(t, api, testSource(srv, testImageSHA()))
+	if err != nil {
+		t.Fatalf("ensure() error = %v", err)
+	}
+	if path != "/var/lib/libvirt/images/base-debian-13.qcow2" {
+		t.Errorf("path = %q", path)
+	}
+	base := api.vol("default", "base-debian-13.qcow2")
+	if base == nil || !bytes.Equal(base.data, testImage) {
+		t.Fatalf("base volume missing or content differs")
+	}
+	if base.format != "qcow2" || base.capacity != int64(len(testImage)) {
+		t.Errorf("base volume format/capacity = %s/%d, want qcow2/%d", base.format, base.capacity, len(testImage))
+	}
+	marker := api.vol("default", "base-debian-13.qcow2.sha256")
+	if marker == nil || string(marker.data) != testImageSHA() {
+		t.Fatalf("marker missing or wrong: %+v", marker)
+	}
+	if !api.hasCall("StoragePoolRefresh") {
+		t.Error("pool was not refreshed after the upload")
+	}
+	if srv.hits.Load() != 1 {
+		t.Errorf("http hits = %d, want 1", srv.hits.Load())
+	}
+}
+
+func TestImageCache_ensure_noChecksumConfigured(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	if _, err := ensureTest(t, api, testSource(srv, "")); err != nil {
+		t.Fatalf("ensure() error = %v", err)
+	}
+	if marker := api.vol("default", "base-debian-13.qcow2.sha256"); marker == nil || string(marker.data) != testImageSHA() {
+		t.Errorf("marker should still record the computed digest, got %+v", marker)
+	}
+	// Second call: ready without a checksum to compare, so no marker read.
+	if _, err := ensureTest(t, api, testSource(srv, "")); err != nil {
+		t.Fatalf("second ensure() error = %v", err)
+	}
+	if srv.hits.Load() != 1 {
+		t.Errorf("http hits = %d, want 1", srv.hits.Load())
+	}
+}
+
+func TestImageCache_ensure_alreadyCached(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	api.putVol("default", "base-debian-13.qcow2", "qcow2", testImage)
+	api.putVol("default", "base-debian-13.qcow2.sha256", "raw", []byte(testImageSHA()))
+	path, err := ensureTest(t, api, testSource(srv, testImageSHA()))
+	if err != nil {
+		t.Fatalf("ensure() error = %v", err)
+	}
+	if path != "/var/lib/libvirt/images/base-debian-13.qcow2" {
+		t.Errorf("path = %q", path)
+	}
+	if srv.hits.Load() != 0 {
+		t.Errorf("http hits = %d, want 0 for a cached image", srv.hits.Load())
+	}
+	if api.hasCall("StorageVolCreateXML") || api.hasCall("StorageVolDelete") {
+		t.Errorf("cached image was touched: calls = %v", api.calls)
+	}
+}
+
+func TestImageCache_ensure_markerMissingRedownloads(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	api.putVol("default", "base-debian-13.qcow2", "qcow2", []byte("half an image"))
+	if _, err := ensureTest(t, api, testSource(srv, testImageSHA())); err != nil {
+		t.Fatalf("ensure() error = %v", err)
+	}
+	if srv.hits.Load() != 1 {
+		t.Errorf("http hits = %d, want 1: a base without its marker is a partial upload", srv.hits.Load())
+	}
+	if base := api.vol("default", "base-debian-13.qcow2"); base == nil || !bytes.Equal(base.data, testImage) {
+		t.Error("partial base volume was not replaced")
+	}
+}
+
+func TestImageCache_ensure_checksumChangedRedownloads(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	api.putVol("default", "base-debian-13.qcow2", "qcow2", []byte("an older release"))
+	api.putVol("default", "base-debian-13.qcow2.sha256", "raw", []byte(strings.Repeat("0", 64)))
+	if _, err := ensureTest(t, api, testSource(srv, testImageSHA())); err != nil {
+		t.Fatalf("ensure() error = %v", err)
+	}
+	if srv.hits.Load() != 1 {
+		t.Errorf("http hits = %d, want 1 after the configured checksum changed", srv.hits.Load())
+	}
+	if marker := api.vol("default", "base-debian-13.qcow2.sha256"); marker == nil || string(marker.data) != testImageSHA() {
+		t.Errorf("marker not updated: %+v", marker)
+	}
+}
+
+func TestImageCache_ensure_checksumMismatchIsPermanent(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	_, err := ensureTest(t, api, testSource(srv, strings.Repeat("a", 64)))
+	if !errors.Is(err, provider.ErrPermanent) {
+		t.Fatalf("ensure() error = %v, want ErrPermanent", err)
+	}
+	if names := api.volNames("default"); len(names) != 0 {
+		t.Errorf("volumes left behind after a checksum mismatch: %v", names)
+	}
+}
+
+func TestImageCache_ensure_httpErrors(t *testing.T) {
+	t.Parallel()
+	tests := []struct {
+		name   string
+		status int
+		want   error
+	}{
+		{"404 is permanent", http.StatusNotFound, provider.ErrPermanent},
+		{"403 is permanent", http.StatusForbidden, provider.ErrPermanent},
+		{"503 is transient", http.StatusServiceUnavailable, provider.ErrTransient},
+		{"500 is transient", http.StatusInternalServerError, provider.ErrTransient},
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			srv := newImageServer(t)
+			srv.status = tc.status
+			api := newFakeAPI("default")
+			_, err := ensureTest(t, api, testSource(srv, ""))
+			if !errors.Is(err, tc.want) {
+				t.Fatalf("ensure() error = %v, want %v", err, tc.want)
+			}
+			if names := api.volNames("default"); len(names) != 0 {
+				t.Errorf("volumes created despite HTTP %d: %v", tc.status, names)
+			}
+		})
+	}
+}
+
+func TestImageCache_ensure_noContentLengthIsPermanent(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	srv.noLength = true
+	api := newFakeAPI("default")
+	_, err := ensureTest(t, api, testSource(srv, ""))
+	if !errors.Is(err, provider.ErrPermanent) || !strings.Contains(err.Error(), "Content-Length") {
+		t.Fatalf("ensure() error = %v, want ErrPermanent about Content-Length", err)
+	}
+}
+
+func TestImageCache_ensure_uploadFailureCleansUp(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	api.uploadHook = func(_ string, r io.Reader) error {
+		_, _ = io.CopyN(io.Discard, r, 100)
+		return io.ErrUnexpectedEOF
+	}
+	_, err := ensureTest(t, api, testSource(srv, testImageSHA()))
+	if !errors.Is(err, provider.ErrTransient) && classify(err) != provider.ErrTransient {
+		t.Fatalf("ensure() error = %v, want a transient failure", err)
+	}
+	if names := api.volNames("default"); len(names) != 0 {
+		t.Errorf("volumes left behind after a failed upload: %v", names)
+	}
+	// Next attempt starts clean and succeeds.
+	api.uploadHook = nil
+	if _, err := ensureTest(t, api, testSource(srv, testImageSHA())); err != nil {
+		t.Fatalf("retry ensure() error = %v", err)
+	}
+}
+
+func TestImageCache_ensure_missingPoolIsPermanent(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	c := newImageCache(nil)
+	pool, _ := api.StoragePoolLookupByName("default")
+	delete(api.pools, "default")
+	_, err := c.ensure(context.Background(), logr.Discard(), api, "nas1", pool, testSource(srv, ""))
+	if classify(err) != provider.ErrPermanent {
+		t.Fatalf("ensure() error = %v, want one classifying as Permanent", err)
+	}
+}
+
+func TestImageCache_ensure_concurrentCallersDownloadOnce(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	srv.delay = 50 * time.Millisecond
+	api := newFakeAPI("default")
+	c := newImageCache(nil)
+	pool, _ := api.StoragePoolLookupByName("default")
+	src := testSource(srv, testImageSHA())
+	var wg sync.WaitGroup
+	paths := make([]string, 4)
+	errs := make([]error, 4)
+	for i := range 4 {
+		wg.Go(func() {
+			paths[i], errs[i] = c.ensure(context.Background(), logr.Discard(), api, "nas1", pool, src)
+		})
+	}
+	wg.Wait()
+	for i := range 4 {
+		if errs[i] != nil || paths[i] != "/var/lib/libvirt/images/base-debian-13.qcow2" {
+			t.Errorf("caller %d: path=%q err=%v", i, paths[i], errs[i])
+		}
+	}
+	if srv.hits.Load() != 1 {
+		t.Errorf("http hits = %d, want 1 for four concurrent callers", srv.hits.Load())
+	}
+}
+
+func TestImageCache_ensure_differentHostsFetchIndependently(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	c := newImageCache(nil)
+	src := testSource(srv, "")
+	for _, host := range []string{"nas1", "lab2"} {
+		api := newFakeAPI("default")
+		pool, _ := api.StoragePoolLookupByName("default")
+		if _, err := c.ensure(context.Background(), logr.Discard(), api, host, pool, src); err != nil {
+			t.Fatalf("ensure(%s) error = %v", host, err)
+		}
+	}
+	if srv.hits.Load() != 2 {
+		t.Errorf("http hits = %d, want one per host", srv.hits.Load())
+	}
+}
+
+func TestImageCache_ensure_cancelledContextIsTransient(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	c := newImageCache(nil)
+	pool, _ := api.StoragePoolLookupByName("default")
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel()
+	_, err := c.ensure(ctx, logr.Discard(), api, "nas1", pool, testSource(srv, ""))
+	if classify(err) != provider.ErrTransient {
+		t.Fatalf("ensure() error = %v, want transient", err)
+	}
+	if names := api.volNames("default"); len(names) != 0 {
+		t.Errorf("volumes created despite cancelled context: %v", names)
+	}
+}

internal/provider/kvm/kvm.go

@@ -0,0 +1,560 @@
+// Package kvm implements the provider contract on libvirt/KVM hypervisors.
+// Each configured host is reached over SSH with the libvirt unix socket
+// forwarded through the connection, so the hypervisors need no libvirtd TCP
+// or TLS listener and the operator image needs no libvirt libraries. VMs
+// boot a stock cloud image (fetched into the host's storage pool on first
+// use) through a qcow2 overlay, with the Proxy's cloud-init delivered on a
+// NoCloud seed ISO built in-process.
+package kvm
+
+import (
+	"bytes"
+	"context"
+	"fmt"
+	"net"
+	"net/http"
+	"time"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+	"github.com/go-logr/logr"
+	logf "sigs.k8s.io/controller-runtime/pkg/log"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+const (
+	defaultDiskSizeGB = 10
+	defaultSocket     = "/var/run/libvirt/libvirt-sock"
+	defaultPool       = "default"
+	formatQcow2       = "qcow2"
+	formatRaw         = "raw"
+
+	// undefineFlags removes everything libvirt may keep alongside a domain
+	// definition, so an undefine never fails on leftover state.
+	undefineFlags = libvirt.DomainUndefineManagedSave |
+		libvirt.DomainUndefineSnapshotsMetadata |
+		libvirt.DomainUndefineNvram
+)
+
+func withDefaults(cfg provider.KVMConfig) provider.KVMConfig {
+	if cfg.DiskSizeGB == 0 {
+		cfg.DiskSizeGB = defaultDiskSizeGB
+	}
+	if cfg.VendorData == "" {
+		cfg.VendorData = defaultVendorData
+	}
+	images := make(map[string]provider.KVMImage, len(cfg.Images))
+	for k, img := range cfg.Images {
+		if img.Format == "" {
+			img.Format = formatQcow2
+		}
+		images[k] = img
+	}
+	cfg.Images = images
+	hosts := make([]provider.KVMHost, len(cfg.Hosts))
+	for i, h := range cfg.Hosts {
+		if h.Socket == "" {
+			h.Socket = defaultSocket
+		}
+		if h.Pool == "" {
+			h.Pool = defaultPool
+		}
+		hosts[i] = h
+	}
+	cfg.Hosts = hosts
+	return cfg
+}
+
+// Provider implements provider.Provider on one or more libvirt hosts.
+type Provider struct {
+	name   string
+	cfg    provider.KVMConfig
+	hosts  *hostPool
+	images *imageCache
+	now    func() time.Time
+}
+
+// New builds a Provider that reaches its hosts over SSH. The key and
+// known_hosts files are read here so a bad path fails at startup rather
+// than on the first Create. Satisfies registry.Constructor.
+func New(_ context.Context, pc provider.ProviderConfig) (provider.Provider, error) {
+	cfg := provider.KVMConfig{}
+	if pc.KVM != nil {
+		cfg = *pc.KVM
+	}
+	auth, err := newSSHAuth(cfg)
+	if err != nil {
+		return nil, fmt.Errorf("kvm provider %q: %w", pc.Name, err)
+	}
+	return newWithDialer(pc, auth.dialer(), nil), nil
+}
+
+// newWithDialer is the test seam: the dialer decides what libvirtAPI each
+// host resolves to, and the HTTP client is what image downloads go through.
+func newWithDialer(pc provider.ProviderConfig, dial hostDialer, httpClient *http.Client) *Provider {
+	cfg := provider.KVMConfig{}
+	if pc.KVM != nil {
+		cfg = *pc.KVM
+	}
+	return &Provider{
+		name:   pc.Name,
+		cfg:    withDefaults(cfg),
+		hosts:  newHostPool(dial),
+		images: newImageCache(httpClient),
+		now:    time.Now,
+	}
+}
+
+func (p *Provider) logger(ctx context.Context) logr.Logger {
+	return logf.FromContext(ctx).WithName("kvm").WithValues("provider", p.name)
+}
+
+func (p *Provider) wrapErr(op, id string, err error) error {
+	return provider.Wrap(classify(err), op, p.name, id, err)
+}
+
+// host resolves placement.zone (or a providerID's host segment) to a
+// configured host. An empty zone is allowed only when exactly one host is
+// configured; anything else is a spec error.
+func (p *Provider) host(zone string) (provider.KVMHost, error) {
+	if zone == "" {
+		switch len(p.cfg.Hosts) {
+		case 0:
+			return provider.KVMHost{}, fmt.Errorf("%w: kvm.hosts is empty", provider.ErrPermanent)
+		case 1:
+			return p.cfg.Hosts[0], nil
+		default:
+			return provider.KVMHost{}, fmt.Errorf("%w: placement.zone is required when kvm.hosts lists more than one host",
+				provider.ErrPermanent)
+		}
+	}
+	for _, h := range p.cfg.Hosts {
+		if h.Name == zone {
+			return h, nil
+		}
+	}
+	return provider.KVMHost{}, fmt.Errorf("%w: host %q is not in kvm.hosts", provider.ErrPermanent, zone)
+}
+
+// Create makes the VM exist and running on the placed host, and is
+// idempotent: a defined domain is only started if needed, its volumes
+// untouched. Before the domain exists nothing can be using the volumes,
+// so an existing overlay is reused (its creation is atomic) but the seed
+// is rebuilt from scratch — a previous attempt may have been cut off mid
+// upload, and a truncated seed would boot a VM without cloud-init. The
+// first Create for an image on a host also downloads that image into the
+// host's pool, synchronously.
+func (p *Provider) Create(ctx context.Context, req provider.CreateRequest) (string, error) {
+	pl := req.Placement
+	host, err := p.host(pl.Zone)
+	if err != nil {
+		return "", p.wrapErr("create", "", err)
+	}
+	mt, ok := p.cfg.MachineTypes[pl.MachineType]
+	if !ok {
+		return "", p.wrapErr("create", "", fmt.Errorf("%w: placement.machineType %q is not in kvm.machineTypes",
+			provider.ErrPermanent, pl.MachineType))
+	}
+	src, err := resolveImage(p.cfg, pl.Image)
+	if err != nil {
+		return "", p.wrapErr("create", "", err)
+	}
+	id := formatProviderID(host.Name, req.Name)
+	log := p.logger(ctx).WithValues("host", host.Name, "name", req.Name)
+	log.V(2).Info("KVM create resolved",
+		"pool", host.Pool, "bridge", host.Bridge, "machineType", pl.MachineType,
+		"vcpus", mt.VCPUs, "memoryMiB", mt.MemoryMiB, "diskSizeGb", p.cfg.DiskSizeGB,
+		"image", src.volumeName(), "cloudInitBytes", len(req.CloudInit))
+
+	err = p.hosts.call(ctx, host, func(api libvirtAPI) error {
+		dom, err := api.DomainLookupByName(req.Name)
+		if err == nil {
+			log.V(1).Info("domain already defined")
+			return p.ensureRunning(log, api, dom)
+		}
+		if !hasCode(err, libvirt.ErrNoDomain) {
+			return fmt.Errorf("looking up domain: %w", err)
+		}
+		pool, err := api.StoragePoolLookupByName(host.Pool)
+		if err != nil {
+			return fmt.Errorf("looking up pool %s: %w", host.Pool, err)
+		}
+		basePath, err := p.images.ensure(ctx, log, api, host.Name, pool, src)
+		if err != nil {
+			return err
+		}
+		if err := p.ensureOverlay(log, api, pool, req.Name, basePath, src.Format); err != nil {
+			return err
+		}
+		if err := p.writeSeed(log, api, pool, req.Name, req.CloudInit); err != nil {
+			return err
+		}
+		dom, err = p.defineDomain(log, api, host, req, mt)
+		if err != nil {
+			return err
+		}
+		return p.ensureRunning(log, api, dom)
+	})
+	if err != nil {
+		logAPIError(log, "create", err)
+		return "", p.wrapErr("create", id, err)
+	}
+	log.V(1).Info("KVM domain ensured running", "id", id)
+	return id, nil
+}
+
+func (p *Provider) ensureOverlay(log logr.Logger, api libvirtAPI, pool libvirt.StoragePool, name, basePath, baseFormat string) error {
+	xml, err := buildVolumeXML(overlayVolume(name, p.cfg.DiskSizeGB, basePath, baseFormat))
+	if err != nil {
+		return err
+	}
+	_, err = api.StorageVolCreateXML(pool, xml, 0)
+	switch {
+	case err == nil:
+		log.V(1).Info("overlay volume created", "volume", overlayVolumeName(name), "base", basePath)
+	case hasCode(err, libvirt.ErrStorageVolExist):
+		log.V(1).Info("overlay volume already exists", "volume", overlayVolumeName(name))
+	default:
+		return fmt.Errorf("creating overlay volume: %w", err)
+	}
+	return nil
+}
+
+// writeSeed replaces the seed volume with a freshly built NoCloud ISO.
+// Only called while no domain references it.
+func (p *Provider) writeSeed(log logr.Logger, api libvirtAPI, pool libvirt.StoragePool, name, userData string) error {
+	seedName := seedVolumeName(name)
+	if err := deleteIfExists(api, pool, seedName); err != nil {
+		return err
+	}
+	iso, err := buildSeedISO(name, userData, p.cfg.VendorData)
+	if err != nil {
+		return fmt.Errorf("%w: %v", provider.ErrPermanent, err)
+	}
+	xml, err := buildVolumeXML(rawVolume(seedName, int64(len(iso))))
+	if err != nil {
+		return err
+	}
+	vol, err := api.StorageVolCreateXML(pool, xml, 0)
+	if err != nil {
+		return fmt.Errorf("creating seed volume: %w", err)
+	}
+	if err := api.StorageVolUpload(vol, bytes.NewReader(iso), 0, 0, 0); err != nil {
+		return fmt.Errorf("uploading seed volume: %w", err)
+	}
+	log.V(1).Info("seed volume uploaded", "volume", seedName, "bytes", len(iso))
+	return nil
+}
+
+func (p *Provider) defineDomain(log logr.Logger, api libvirtAPI, host provider.KVMHost, req provider.CreateRequest, mt provider.KVMMachineType) (libvirt.Domain, error) {
+	xml, err := buildDomainXML(domainParams{
+		Name:      req.Name,
+		UID:       req.UID,
+		CreatedAt: p.now(),
+		VCPUs:     mt.VCPUs,
+		MemoryMiB: mt.MemoryMiB,
+		Pool:      host.Pool,
+		Bridge:    host.Bridge,
+		DiskVol:   overlayVolumeName(req.Name),
+		SeedVol:   seedVolumeName(req.Name),
+	})
+	if err != nil {
+		return libvirt.Domain{}, err
+	}
+	dom, err := api.DomainDefineXML(xml)
+	if err != nil {
+		return libvirt.Domain{}, fmt.Errorf("defining domain: %w", err)
+	}
+	log.V(1).Info("domain defined", "uid", req.UID)
+	return dom, nil
+}
+
+func (p *Provider) ensureRunning(log logr.Logger, api libvirtAPI, dom libvirt.Domain) error {
+	state, _, err := api.DomainGetState(dom, 0)
+	if err != nil {
+		return fmt.Errorf("reading domain state: %w", err)
+	}
+	if isActive(libvirt.DomainState(state)) {
+		return nil
+	}
+	err = api.DomainCreate(dom)
+	switch {
+	case err == nil:
+		log.V(1).Info("domain started")
+	case hasCode(err, libvirt.ErrOperationInvalid):
+		log.V(1).Info("domain already running, start treated as success")
+	default:
+		return fmt.Errorf("starting domain: %w", err)
+	}
+	return nil
+}
+
+// Get reads the domain's state, metadata and addresses. The providerID
+// carries the host, so this looks at the right hypervisor even after a
+// placement.zone edit.
+func (p *Provider) Get(ctx context.Context, providerID string) (*provider.Instance, error) {
+	hostName, name, err := parseProviderID(providerID)
+	if err != nil {
+		return nil, provider.Wrap(provider.ErrPermanent, "get", p.name, providerID, err)
+	}
+	host, err := p.host(hostName)
+	if err != nil {
+		return nil, p.wrapErr("get", providerID, err)
+	}
+	log := p.logger(ctx).WithValues("host", host.Name, "name", name)
+	var inst *provider.Instance
+	err = p.hosts.call(ctx, host, func(api libvirtAPI) error {
+		dom, err := api.DomainLookupByName(name)
+		if err != nil {
+			return fmt.Errorf("looking up domain: %w", err)
+		}
+		meta, err := p.metadata(api, dom)
+		if err != nil {
+			return err
+		}
+		inst, err = p.inspect(log, api, host.Name, dom, meta)
+		return err
+	})
+	if err != nil {
+		logAPIError(log, "get", err)
+		return nil, p.wrapErr("get", providerID, err)
+	}
+	log.V(1).Info("KVM domain fetched", "state", inst.State, "ip", inst.IP, "uid", inst.UID)
+	return inst, nil
+}
+
+// inspect assembles an Instance from a domain that is known to exist and
+// whose metadata has already been read (the zero value for a domain
+// without the operator's element, which reports an empty UID as the
+// contract requires).
+func (p *Provider) inspect(log logr.Logger, api libvirtAPI, hostName string, dom libvirt.Domain, meta instanceMetadata) (*provider.Instance, error) {
+	state, _, err := api.DomainGetState(dom, 0)
+	if err != nil {
+		return nil, fmt.Errorf("reading domain state: %w", err)
+	}
+	var ip string
+	if isActive(libvirt.DomainState(state)) {
+		ip = p.address(log, api, dom)
+	}
+	return &provider.Instance{
+		ID:        formatProviderID(hostName, dom.Name),
+		IP:        ip,
+		State:     mapState(libvirt.DomainState(state), ip),
+		UID:       meta.UID,
+		CreatedAt: meta.createdAt(),
+	}, nil
+}
+
+// metadata returns the operator's element, or the zero value when the
+// domain has none (it was not created by this operator).
+func (p *Provider) metadata(api libvirtAPI, dom libvirt.Domain) (instanceMetadata, error) {
+	raw, err := api.DomainGetMetadata(dom, int32(libvirt.DomainMetadataElement), libvirt.OptString{metadataURI}, libvirt.DomainAffectCurrent)
+	if hasCode(err, libvirt.ErrNoDomainMetadata) {
+		return instanceMetadata{}, nil
+	}
+	if err != nil {
+		return instanceMetadata{}, fmt.Errorf("reading domain metadata: %w", err)
+	}
+	meta, err := parseInstanceMetadata(raw)
+	if err != nil {
+		return instanceMetadata{}, fmt.Errorf("%w: %v", provider.ErrPermanent, err)
+	}
+	return meta, nil
+}
+
+// address returns the domain's first routable IPv4, asking the host's ARP
+// table first and the guest agent second. Both sources fail routinely while
+// the guest is still booting (no traffic seen yet, agent not installed
+// yet), so their errors mean "no address yet", not "Get failed".
+func (p *Provider) address(log logr.Logger, api libvirtAPI, dom libvirt.Domain) string {
+	for _, source := range []libvirt.DomainInterfaceAddressesSource{
+		libvirt.DomainInterfaceAddressesSrcArp,
+		libvirt.DomainInterfaceAddressesSrcAgent,
+	} {
+		ifaces, err := api.DomainInterfaceAddresses(dom, uint32(source), 0)
+		if err != nil {
+			log.V(2).Info("no addresses from source", "source", source.String(), "error", err.Error())
+			continue
+		}
+		if ip := firstIPv4(ifaces); ip != "" {
+			return ip
+		}
+	}
+	return ""
+}
+
+func firstIPv4(ifaces []libvirt.DomainInterface) string {
+	for _, iface := range ifaces {
+		for _, a := range iface.Addrs {
+			if a.Type != int32(libvirt.IPAddrTypeIpv4) {
+				continue
+			}
+			ip := net.ParseIP(a.Addr)
+			if ip == nil || ip.IsLoopback() || ip.IsLinkLocalUnicast() || ip.IsUnspecified() {
+				continue
+			}
+			return a.Addr
+		}
+	}
+	return ""
+}
+
+// isActive reports whether the guest is executing (or blocked on the
+// hypervisor, which is still "up" from the outside).
+func isActive(s libvirt.DomainState) bool {
+	return s == libvirt.DomainRunning || s == libvirt.DomainBlocked
+}
+
+// mapState collapses libvirt's states onto the provider's. An active
+// domain without an address is still Provisioning: an empty IP must never
+// be published as Running. "No state" is also Provisioning — the
+// reconciler answers Stopped with delete-and-recreate, too drastic for a
+// state libvirt could not determine; the next poll decides. Everything
+// else — paused, suspended, shut off, crashed, shutting down — is Stopped.
+func mapState(s libvirt.DomainState, ip string) provider.InstanceState {
+	if s == libvirt.DomainNostate {
+		return provider.StateProvisioning
+	}
+	if !isActive(s) {
+		return provider.StateStopped
+	}
+	if ip == "" {
+		return provider.StateProvisioning
+	}
+	return provider.StateRunning
+}
+
+// Delete tears the VM down and is idempotent at every step: a stopped
+// domain is not destroyed again, a missing domain is not undefined, and
+// the per-VM volumes are removed by name even when the domain is already
+// gone, so a crash between volume creation and define is cleaned up on the
+// next reconcile.
+func (p *Provider) Delete(ctx context.Context, providerID string) error {
+	hostName, name, err := parseProviderID(providerID)
+	if err != nil {
+		return provider.Wrap(provider.ErrPermanent, "delete", p.name, providerID, err)
+	}
+	host, err := p.host(hostName)
+	if err != nil {
+		return p.wrapErr("delete", providerID, err)
+	}
+	log := p.logger(ctx).WithValues("host", host.Name, "name", name)
+	err = p.hosts.call(ctx, host, func(api libvirtAPI) error {
+		if err := p.removeDomain(log, api, name); err != nil {
+			return err
+		}
+		return p.removeVolumes(log, api, host.Pool, name)
+	})
+	if err != nil {
+		logAPIError(log, "delete", err)
+		return p.wrapErr("delete", providerID, err)
+	}
+	log.V(1).Info("KVM domain deleted")
+	return nil
+}
+
+func (p *Provider) removeDomain(log logr.Logger, api libvirtAPI, name string) error {
+	dom, err := api.DomainLookupByName(name)
+	if hasCode(err, libvirt.ErrNoDomain) {
+		log.V(1).Info("domain already gone")
+		return nil
+	}
+	if err != nil {
+		return fmt.Errorf("looking up domain: %w", err)
+	}
+	err = api.DomainDestroy(dom)
+	switch {
+	case err == nil:
+		log.V(1).Info("domain destroyed")
+	case hasCode(err, libvirt.ErrOperationInvalid):
+		log.V(1).Info("domain not running, destroy skipped")
+	case hasCode(err, libvirt.ErrNoDomain):
+		return nil
+	default:
+		return fmt.Errorf("destroying domain: %w", err)
+	}
+	if err := api.DomainUndefineFlags(dom, undefineFlags); err != nil && !hasCode(err, libvirt.ErrNoDomain) {
+		return fmt.Errorf("undefining domain: %w", err)
+	}
+	log.V(1).Info("domain undefined")
+	return nil
+}
+
+// removeVolumes deletes the overlay and seed. A missing pool means there
+// is nothing left to clean, not a failure: refusing to finish a Delete
+// over a pool that no longer exists would pin the Proxy's finalizer.
+func (p *Provider) removeVolumes(log logr.Logger, api libvirtAPI, poolName, name string) error {
+	pool, err := api.StoragePoolLookupByName(poolName)
+	if hasCode(err, libvirt.ErrNoStoragePool) {
+		log.Info("pool not found, no volumes to delete", "pool", poolName)
+		return nil
+	}
+	if err != nil {
+		return fmt.Errorf("looking up pool %s: %w", poolName, err)
+	}
+	for _, vol := range []string{overlayVolumeName(name), seedVolumeName(name)} {
+		if err := deleteIfExists(api, pool, vol); err != nil {
+			return err
+		}
+	}
+	log.V(1).Info("per-VM volumes deleted")
+	return nil
+}
+
+// ListByTag inspects every domain on every host and keeps those carrying
+// the operator's managed metadata. One unreachable host fails the whole
+// sweep, so orphan GC never acts on a partial view of the fleet; a single
+// domain that cannot be read is skipped instead, which is safe in the
+// same direction — a domain missing from the list is never collected.
+func (p *Provider) ListByTag(ctx context.Context) ([]provider.Instance, error) {
+	log := p.logger(ctx)
+	var out []provider.Instance
+	for _, host := range p.cfg.Hosts {
+		hlog := log.WithValues("host", host.Name)
+		err := p.hosts.call(ctx, host, func(api libvirtAPI) error {
+			doms, _, err := api.ConnectListAllDomains(1, 0)
+			if err != nil {
+				return fmt.Errorf("listing domains: %w", err)
+			}
+			for _, dom := range doms {
+				if inst := p.managedInstance(hlog, api, host.Name, dom); inst != nil {
+					out = append(out, *inst)
+				}
+			}
+			return nil
+		})
+		if err != nil {
+			logAPIError(hlog, "list", err)
+			return nil, p.wrapErr("list", "", err)
+		}
+	}
+	log.V(1).Info("KVM domains listed", "hosts", len(p.cfg.Hosts), "managed", len(out))
+	for _, inst := range out {
+		log.V(2).Info("KVM listed instance", "id", inst.ID, "state", inst.State, "uid", inst.UID, "createdAt", inst.CreatedAt)
+	}
+	return out, nil
+}
+
+// managedInstance returns nil for domains the operator does not own, for
+// domains that vanished between the listing and the inspection, and for
+// domains whose metadata or state cannot be read (corrupt element, mid
+// teardown): one bad domain must not block GC for every other one. A
+// transport failure would also land here, but it fails the next RPC in
+// the same sweep too, so the host still counts as unreachable.
+func (p *Provider) managedInstance(log logr.Logger, api libvirtAPI, hostName string, dom libvirt.Domain) *provider.Instance {
+	log = log.WithValues("name", dom.Name)
+	meta, err := p.metadata(api, dom)
+	if err != nil {
+		log.V(1).Info("skipping domain with unreadable metadata", "error", err.Error())
+		return nil
+	}
+	if !meta.managed() {
+		return nil
+	}
+	inst, err := p.inspect(log, api, hostName, dom, meta)
+	if err != nil {
+		log.V(1).Info("skipping domain that could not be inspected", "error", err.Error())
+		return nil
+	}
+	return inst
+}

internal/provider/kvm/kvm_test.go

@@ -0,0 +1,880 @@
+package kvm
+
+import (
+	"context"
+	"encoding/xml"
+	"errors"
+	"fmt"
+	"io"
+	"strings"
+	"testing"
+	"time"
+
+	libvirt "github.com/digitalocean/go-libvirt"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+func TestWithDefaults(t *testing.T) {
+	t.Parallel()
+	cfg := withDefaults(provider.KVMConfig{
+		Images: map[string]provider.KVMImage{
+			"debian": {URL: "https://x/d.qcow2"},
+			"raw":    {URL: "https://x/r.img", Format: "raw"},
+		},
+		Hosts: []provider.KVMHost{
+			{Name: "a", Address: "a:22", User: "virt", Bridge: "br0"},
+			{Name: "b", Address: "b:22", User: "virt", Bridge: "br1", Socket: "/run/s", Pool: "fast"},
+		},
+	})
+	if cfg.DiskSizeGB != defaultDiskSizeGB {
+		t.Errorf("diskSizeGb = %d, want %d", cfg.DiskSizeGB, defaultDiskSizeGB)
+	}
+	if cfg.VendorData != defaultVendorData {
+		t.Errorf("vendorData = %q, want the default", cfg.VendorData)
+	}
+	if cfg.Images["debian"].Format != "qcow2" || cfg.Images["raw"].Format != "raw" {
+		t.Errorf("image formats = %q/%q, want qcow2 default and raw kept", cfg.Images["debian"].Format, cfg.Images["raw"].Format)
+	}
+	if a := cfg.Hosts[0]; a.Socket != defaultSocket || a.Pool != defaultPool {
+		t.Errorf("host a = %+v, want default socket and pool", a)
+	}
+	if b := cfg.Hosts[1]; b.Socket != "/run/s" || b.Pool != "fast" {
+		t.Errorf("host b = %+v, want explicit socket and pool kept", b)
+	}
+}
+
+func TestWithDefaults_keepsExplicitValues(t *testing.T) {
+	t.Parallel()
+	cfg := withDefaults(provider.KVMConfig{DiskSizeGB: 42, VendorData: "#cloud-config\n"})
+	if cfg.DiskSizeGB != 42 || cfg.VendorData != "#cloud-config\n" {
+		t.Errorf("cfg = %+v, want explicit values kept", cfg)
+	}
+}
+
+// --- provider over fake hosts ---
+
+const (
+	testName      = "proxy-abc123def456ghij"
+	testUID       = "11111111-2222-3333-4444-555555555555"
+	testImageURL  = "https://images.example/debian-13.qcow2"
+	testCloudInit = "#cloud-config\npackages: [squid]\n"
+)
+
+var (
+	testNow    = time.Date(2026, 9, 8, 10, 0, 0, 0, time.UTC)
+	errHostOff = errors.New("ssh dial: connection refused")
+)
+
+// hostFakes routes each configured host name to its own fakeAPI; a host
+// with no entry is unreachable.
+type hostFakes map[string]*fakeAPI
+
+func (h hostFakes) dial(_ context.Context, host provider.KVMHost) (*hostClient, error) {
+	api, ok := h[host.Name]
+	if !ok {
+		return nil, errHostOff
+	}
+	return &hostClient{api: api, close: func() {}}, nil
+}
+
+func testConfig(hosts ...string) provider.ProviderConfig {
+	cfg := &provider.KVMConfig{
+		SSHKeyFile:     "/k",
+		KnownHostsFile: "/kh",
+		Images:         map[string]provider.KVMImage{"debian-13": {URL: testImageURL}},
+		MachineTypes:   map[string]provider.KVMMachineType{"small": {VCPUs: 2, MemoryMiB: 2048}},
+	}
+	for _, h := range hosts {
+		cfg.Hosts = append(cfg.Hosts, provider.KVMHost{Name: h, Address: h + ":22", User: "virt", Bridge: "br0"})
+	}
+	return provider.ProviderConfig{Name: "kvm-home", Type: "kvm", KVM: cfg}
+}
+
+func newTestProvider(fakes hostFakes, hosts ...string) *Provider {
+	p := newWithDialer(testConfig(hosts...), fakes.dial, nil)
+	p.now = func() time.Time { return testNow }
+	return p
+}
+
+// cacheImage pre-populates the pool with the configured image and its
+// marker so Create never touches the network.
+func cacheImage(api *fakeAPI) {
+	src := imageSource{Key: "debian-13", URL: testImageURL, Format: formatQcow2}
+	api.putVol("default", src.volumeName(), formatQcow2, []byte("qcow2"))
+	api.putVol("default", src.markerName(), formatRaw, []byte(testImageSHA()))
+}
+
+func readyHost() *fakeAPI {
+	api := newFakeAPI("default")
+	cacheImage(api)
+	return api
+}
+
+func testRequest() provider.CreateRequest {
+	return provider.CreateRequest{
+		Name:      testName,
+		UID:       testUID,
+		Namespace: "default",
+		ProxyName: "home-proxy-1",
+		Placement: provider.Placement{Zone: "nas1", MachineType: "small", Image: "debian-13"},
+		CloudInit: testCloudInit,
+		Port:      3128,
+	}
+}
+
+func metadataXML(t *testing.T, uid string, created time.Time) string {
+	t.Helper()
+	out, err := xml.Marshal(newInstanceMetadata(uid, created))
+	if err != nil {
+		t.Fatal(err)
+	}
+	return string(out)
+}
+
+func ipv4(addr string) libvirt.DomainIPAddr {
+	return libvirt.DomainIPAddr{Type: int32(libvirt.IPAddrTypeIpv4), Addr: addr, Prefix: 24}
+}
+
+func ipv6(addr string) libvirt.DomainIPAddr {
+	return libvirt.DomainIPAddr{Type: int32(libvirt.IPAddrTypeIpv6), Addr: addr, Prefix: 64}
+}
+
+// managedDomain seeds a domain as Create would have left it.
+func managedDomain(t *testing.T, api *fakeAPI, name, uid string, running bool, addrs map[uint32][]libvirt.DomainInterface) {
+	t.Helper()
+	api.putDomain(&fakeDomain{name: name, metadata: metadataXML(t, uid, testNow), running: running, addrs: addrs})
+	api.putVol("default", overlayVolumeName(name), formatQcow2, nil)
+	api.putVol("default", seedVolumeName(name), formatRaw, []byte("iso"))
+}
+
+func wantClass(t *testing.T, err error, class error) {
+	t.Helper()
+	if err == nil {
+		t.Fatalf("got nil error, want %v", class)
+	}
+	if !errors.Is(err, class) {
+		t.Fatalf("error class = %v, want %v (err: %v)", provider.Class(err), class, err)
+	}
+}
+
+func assertCreated(t *testing.T, api *fakeAPI) {
+	t.Helper()
+	dom := api.domain(testName)
+	if dom == nil {
+		t.Fatal("domain not defined")
+	}
+	if !dom.running {
+		t.Error("domain not running")
+	}
+	overlay := api.vol("default", overlayVolumeName(testName))
+	if overlay == nil {
+		t.Fatal("overlay volume missing")
+	}
+	if overlay.backing != "/var/lib/libvirt/images/base-debian-13.qcow2" || overlay.format != formatQcow2 {
+		t.Errorf("overlay = %+v, want qcow2 backed by the cached base", overlay)
+	}
+	if overlay.capacity != defaultDiskSizeGB*bytesPerGiB {
+		t.Errorf("overlay capacity = %d, want %d", overlay.capacity, defaultDiskSizeGB*bytesPerGiB)
+	}
+	seed := api.vol("default", seedVolumeName(testName))
+	if seed == nil {
+		t.Fatal("seed volume missing")
+	}
+	if seed.format != formatRaw || len(seed.data) == 0 || int64(len(seed.data)) != seed.capacity {
+		t.Errorf("seed = format %s, %d bytes, capacity %d; want raw with capacity == uploaded size", seed.format, len(seed.data), seed.capacity)
+	}
+}
+
+func TestCreate_freshHost(t *testing.T) {
+	t.Parallel()
+	api := readyHost()
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+	id, err := p.Create(context.Background(), testRequest())
+	if err != nil {
+		t.Fatalf("Create: %v", err)
+	}
+	if want := "hosts/nas1/domains/" + testName; id != want {
+		t.Errorf("providerID = %s, want %s", id, want)
+	}
+	assertCreated(t, api)
+
+	var d domainXML
+	if err := xml.Unmarshal([]byte(api.domain(testName).xml), &d); err != nil {
+		t.Fatal(err)
+	}
+	if d.VCPU != 2 || d.Memory.Value != 2048 || d.Memory.Unit != "MiB" {
+		t.Errorf("domain size = %d vcpu / %d %s, want 2 / 2048 MiB", d.VCPU, d.Memory.Value, d.Memory.Unit)
+	}
+	if d.Devices.Interfaces[0].Source.Bridge != "br0" {
+		t.Errorf("bridge = %q, want br0", d.Devices.Interfaces[0].Source.Bridge)
+	}
+	if m := d.Metadata.Instance; m.UID != testUID || !m.managed() || !m.createdAt().Equal(testNow) {
+		t.Errorf("metadata = %+v, want managed, uid %s, createdAt %s", m, testUID, testNow)
+	}
+	if api.hasCall("StorageVolDownload") || api.callCount("StorageVolUpload") != 1 {
+		t.Errorf("calls = %v, want no image download and exactly one upload (the seed)", api.calls)
+	}
+}
+
+func TestCreate_fetchesImageOnFirstUse(t *testing.T) {
+	t.Parallel()
+	srv := newImageServer(t)
+	api := newFakeAPI("default")
+	cfg := testConfig("nas1")
+	cfg.KVM.Images["debian-13"] = provider.KVMImage{URL: srv.URL + "/debian.qcow2", SHA256: testImageSHA()}
+	p := newWithDialer(cfg, hostFakes{"nas1": api}.dial, nil)
+
+	if _, err := p.Create(context.Background(), testRequest()); err != nil {
+		t.Fatalf("Create: %v", err)
+	}
+	if srv.hits.Load() != 1 {
+		t.Errorf("image server hits = %d, want 1", srv.hits.Load())
+	}
+	base := api.vol("default", "base-debian-13.qcow2")
+	if base == nil || string(base.data) != string(testImage) {
+		t.Fatal("base image not cached in the pool")
+	}
+	if api.domain(testName) == nil || !api.domain(testName).running {
+		t.Error("domain not running after image fetch")
+	}
+
+	// A second Proxy with the same image on the same host reuses the cache.
+	req := testRequest()
+	req.Name, req.UID = "proxy-second", "uid-2"
+	if _, err := p.Create(context.Background(), req); err != nil {
+		t.Fatalf("second Create: %v", err)
+	}
+	if srv.hits.Load() != 1 {
+		t.Errorf("image server hits after second Create = %d, want still 1", srv.hits.Load())
+	}
+}
+
+func TestCreate_isIdempotentWhenComplete(t *testing.T) {
+	t.Parallel()
+	api := readyHost()
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+	if _, err := p.Create(context.Background(), testRequest()); err != nil {
+		t.Fatal(err)
+	}
+	before := len(api.calls)
+
+	id, err := p.Create(context.Background(), testRequest())
+	if err != nil {
+		t.Fatalf("second Create: %v", err)
+	}
+	if id != "hosts/nas1/domains/"+testName {
+		t.Errorf("providerID = %s", id)
+	}
+	for _, m := range []string{"DomainDefineXML", "DomainCreate", "StorageVolUpload"} {
+		if n := api.callCount(m); n != 1 {
+			t.Errorf("%s called %d times across two Creates, want 1", m, n)
+		}
+	}
+	if len(api.calls) == before {
+		t.Error("second Create made no calls at all; expected lookups")
+	}
+}
+
+// Each case breaks Create after a different step, then re-runs it and
+// expects the second attempt to pick up where the first stopped without
+// duplicating what already exists.
+func TestCreate_resumesAfterPartialFailure(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name     string
+		sabotage func(api *fakeAPI)
+		class    error
+		check    func(t *testing.T, api *fakeAPI)
+	}{
+		{
+			name: "seed upload fails after overlay exists",
+			sabotage: func(api *fakeAPI) {
+				api.uploadHook = func(vol string, _ io.Reader) error {
+					api.uploadHook = nil
+					return io.ErrUnexpectedEOF
+				}
+			},
+			class: provider.ErrTransient,
+			check: func(t *testing.T, api *fakeAPI) {
+				if n := api.callCount("StorageVolUpload"); n != 2 {
+					t.Errorf("seed uploads = %d, want 2 (failed, then redone)", n)
+				}
+				if n := api.callCount("StorageVolCreateXML"); n != 4 {
+					t.Errorf("volume creates = %d, want 4 (overlay, seed; overlay rejected as existing, seed again)", n)
+				}
+				if names := api.volNames("default"); len(names) != 4 {
+					t.Errorf("pool holds %v, want base, marker, one overlay, one seed", names)
+				}
+			},
+		},
+		{
+			name:     "define fails after volumes exist",
+			sabotage: func(api *fakeAPI) { api.failOnce["DomainDefineXML"] = lvError(libvirt.ErrInternalError, "boom") },
+			class:    provider.ErrTransient,
+			check: func(t *testing.T, api *fakeAPI) {
+				if n := api.callCount("StorageVolUpload"); n != 2 {
+					t.Errorf("seed uploads = %d, want 2 (seed rebuilt while no domain exists)", n)
+				}
+				if n := api.callCount("DomainDefineXML"); n != 2 {
+					t.Errorf("defines = %d, want 2", n)
+				}
+			},
+		},
+		{
+			name:     "start fails after define",
+			sabotage: func(api *fakeAPI) { api.failOnce["DomainCreate"] = lvError(libvirt.ErrInternalError, "boom") },
+			class:    provider.ErrTransient,
+			check: func(t *testing.T, api *fakeAPI) {
+				if n := api.callCount("DomainDefineXML"); n != 1 {
+					t.Errorf("defines = %d, want 1 (existing domain reused)", n)
+				}
+				if n := api.callCount("DomainCreate"); n != 2 {
+					t.Errorf("starts = %d, want 2", n)
+				}
+			},
+		},
+		{
+			name:     "transport drops mid-create",
+			sabotage: func(api *fakeAPI) { api.failOnce["DomainLookupByName"] = io.EOF },
+			class:    provider.ErrTransient,
+			check: func(t *testing.T, api *fakeAPI) {
+				if n := api.callCount("DomainDefineXML"); n != 1 {
+					t.Errorf("defines = %d, want 1", n)
+				}
+			},
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			api := readyHost()
+			tc.sabotage(api)
+			p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+			_, err := p.Create(context.Background(), testRequest())
+			wantClass(t, err, tc.class)
+			if _, err := p.Create(context.Background(), testRequest()); err != nil {
+				t.Fatalf("retry Create: %v", err)
+			}
+			assertCreated(t, api)
+			tc.check(t, api)
+		})
+	}
+}
+
+// A reconcile cancelled mid-upload leaves a seed volume whose content is
+// short; the compensating delete cannot run because the pool has already
+// closed the connection. The next Create must not trust that volume.
+func TestCreate_replacesLeftoverSeed(t *testing.T) {
+	t.Parallel()
+	api := readyHost()
+	api.putVol("default", seedVolumeName(testName), formatRaw, []byte("trunc"))
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+	if _, err := p.Create(context.Background(), testRequest()); err != nil {
+		t.Fatal(err)
+	}
+	assertCreated(t, api)
+	if seed := api.vol("default", seedVolumeName(testName)); string(seed.data) == "trunc" {
+		t.Error("leftover seed was kept")
+	}
+	if n := api.callCount("StorageVolUpload"); n != 1 {
+		t.Errorf("uploads = %d, want 1 (the rebuilt seed)", n)
+	}
+}
+
+func TestCreate_startsStoppedDomain(t *testing.T) {
+	t.Parallel()
+	api := readyHost()
+	managedDomain(t, api, testName, testUID, false, nil)
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+	if _, err := p.Create(context.Background(), testRequest()); err != nil {
+		t.Fatal(err)
+	}
+	if !api.domain(testName).running {
+		t.Error("domain still stopped")
+	}
+	if api.hasCall("DomainDefineXML") || api.hasCall("StorageVolUpload") || api.hasCall("StorageVolDelete") {
+		t.Errorf("calls = %v, want no define, upload or volume delete for an existing domain", api.calls)
+	}
+}
+
+func TestCreate_singleHostNeedsNoZone(t *testing.T) {
+	t.Parallel()
+	api := readyHost()
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+	req := testRequest()
+	req.Placement.Zone = ""
+
+	id, err := p.Create(context.Background(), req)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if id != "hosts/nas1/domains/"+testName {
+		t.Errorf("providerID = %s, want the single host filled in", id)
+	}
+}
+
+func TestCreate_placementErrors(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name  string
+		hosts []string
+		edit  func(*provider.CreateRequest)
+		want  string
+	}{
+		{"unknown host", []string{"nas1"}, func(r *provider.CreateRequest) { r.Placement.Zone = "nas9" }, "not in kvm.hosts"},
+		{"empty zone with two hosts", []string{"nas1", "nas2"}, func(r *provider.CreateRequest) { r.Placement.Zone = "" }, "placement.zone is required"},
+		{"empty zone with no hosts", nil, func(r *provider.CreateRequest) { r.Placement.Zone = "" }, "kvm.hosts is empty"},
+		{"unknown machine type", []string{"nas1"}, func(r *provider.CreateRequest) { r.Placement.MachineType = "huge" }, "not in kvm.machineTypes"},
+		{"empty machine type", []string{"nas1"}, func(r *provider.CreateRequest) { r.Placement.MachineType = "" }, "not in kvm.machineTypes"},
+		{"unknown image", []string{"nas1"}, func(r *provider.CreateRequest) { r.Placement.Image = "windows" }, "neither a key"},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			fakes := hostFakes{}
+			for _, h := range tc.hosts {
+				fakes[h] = readyHost()
+			}
+			p := newTestProvider(fakes, tc.hosts...)
+			req := testRequest()
+			tc.edit(&req)
+
+			_, err := p.Create(context.Background(), req)
+			wantClass(t, err, provider.ErrPermanent)
+			if !strings.Contains(err.Error(), tc.want) {
+				t.Errorf("error = %q, want it to mention %q", err, tc.want)
+			}
+			for _, api := range fakes {
+				if len(api.calls) != 0 {
+					t.Errorf("host was called: %v", api.calls)
+				}
+			}
+		})
+	}
+}
+
+func TestCreate_hostFailures(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name  string
+		setup func() (hostFakes, string)
+		class error
+	}{
+		{
+			name:  "host unreachable",
+			setup: func() (hostFakes, string) { return hostFakes{}, "nas1" },
+			class: provider.ErrTransient,
+		},
+		{
+			name: "pool missing",
+			setup: func() (hostFakes, string) {
+				api := newFakeAPI("other")
+				return hostFakes{"nas1": api}, "nas1"
+			},
+			class: provider.ErrPermanent,
+		},
+		{
+			name: "domain xml rejected",
+			setup: func() (hostFakes, string) {
+				api := readyHost()
+				api.fail["DomainDefineXML"] = lvError(libvirt.ErrConfigUnsupported, "no q35 here")
+				return hostFakes{"nas1": api}, "nas1"
+			},
+			class: provider.ErrPermanent,
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			fakes, host := tc.setup()
+			p := newTestProvider(fakes, host)
+			_, err := p.Create(context.Background(), testRequest())
+			wantClass(t, err, tc.class)
+		})
+	}
+}
+
+func TestCreate_cancelledContext(t *testing.T) {
+	t.Parallel()
+	api := readyHost()
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel()
+
+	_, err := p.Create(ctx, testRequest())
+	wantClass(t, err, provider.ErrTransient)
+	if !errors.Is(err, context.Canceled) {
+		t.Errorf("error = %v, want it to unwrap to context.Canceled", err)
+	}
+}
+
+func TestGet(t *testing.T) {
+	t.Parallel()
+	arp := map[uint32][]libvirt.DomainInterface{
+		uint32(libvirt.DomainInterfaceAddressesSrcArp): {{Name: "vnet0", Addrs: []libvirt.DomainIPAddr{ipv4("192.168.1.50")}}},
+	}
+	agentOnly := map[uint32][]libvirt.DomainInterface{
+		uint32(libvirt.DomainInterfaceAddressesSrcAgent): {
+			{Name: "lo", Addrs: []libvirt.DomainIPAddr{ipv4("127.0.0.1"), ipv6("::1")}},
+			{Name: "enp1s0", Addrs: []libvirt.DomainIPAddr{ipv6("fe80::1"), ipv4("192.168.1.51")}},
+		},
+	}
+	linkLocalOnly := map[uint32][]libvirt.DomainInterface{
+		uint32(libvirt.DomainInterfaceAddressesSrcAgent): {
+			{Name: "enp1s0", Addrs: []libvirt.DomainIPAddr{ipv4("169.254.10.10")}},
+		},
+	}
+	cases := []struct {
+		name    string
+		running bool
+		addrs   map[uint32][]libvirt.DomainInterface
+		state   provider.InstanceState
+		ip      string
+	}{
+		{"running with arp address", true, arp, provider.StateRunning, "192.168.1.50"},
+		{"running, agent only, skips loopback and v6", true, agentOnly, provider.StateRunning, "192.168.1.51"},
+		{"running, no address yet", true, nil, provider.StateProvisioning, ""},
+		{"running, only link-local", true, linkLocalOnly, provider.StateProvisioning, ""},
+		{"shut off", false, arp, provider.StateStopped, ""},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			api := readyHost()
+			managedDomain(t, api, testName, testUID, tc.running, tc.addrs)
+			p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+			inst, err := p.Get(context.Background(), "hosts/nas1/domains/"+testName)
+			if err != nil {
+				t.Fatalf("Get: %v", err)
+			}
+			want := provider.Instance{ID: "hosts/nas1/domains/" + testName, IP: tc.ip, State: tc.state, UID: testUID, CreatedAt: testNow}
+			if *inst != want {
+				t.Errorf("instance = %+v, want %+v", *inst, want)
+			}
+		})
+	}
+}
+
+func TestGet_unmanagedDomainHasNoUID(t *testing.T) {
+	t.Parallel()
+	api := readyHost()
+	api.putDomain(&fakeDomain{name: "hand-made", running: true})
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+	inst, err := p.Get(context.Background(), "hosts/nas1/domains/hand-made")
+	if err != nil {
+		t.Fatalf("Get: %v", err)
+	}
+	if inst.UID != "" || !inst.CreatedAt.IsZero() {
+		t.Errorf("instance = %+v, want empty UID and zero CreatedAt", inst)
+	}
+}
+
+func TestGet_errors(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name  string
+		id    string
+		setup func(t *testing.T, api *fakeAPI)
+		class error
+	}{
+		{"missing domain", "hosts/nas1/domains/" + testName, func(*testing.T, *fakeAPI) {}, provider.ErrNotFound},
+		{"malformed id", "zones/x/instances/y", func(*testing.T, *fakeAPI) {}, provider.ErrPermanent},
+		{"host not configured", "hosts/nas9/domains/" + testName, func(*testing.T, *fakeAPI) {}, provider.ErrPermanent},
+		{
+			"state read fails", "hosts/nas1/domains/" + testName,
+			func(t *testing.T, api *fakeAPI) {
+				managedDomain(t, api, testName, testUID, true, nil)
+				api.fail["DomainGetState"] = lvError(libvirt.ErrInternalError, "boom")
+			},
+			provider.ErrTransient,
+		},
+		{
+			"corrupt metadata", "hosts/nas1/domains/" + testName,
+			func(_ *testing.T, api *fakeAPI) {
+				api.putDomain(&fakeDomain{name: testName, metadata: "<instance", running: true})
+			},
+			provider.ErrPermanent,
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			api := readyHost()
+			tc.setup(t, api)
+			p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+			_, err := p.Get(context.Background(), tc.id)
+			wantClass(t, err, tc.class)
+		})
+	}
+}
+
+func TestGet_hostUnreachable(t *testing.T) {
+	t.Parallel()
+	p := newTestProvider(hostFakes{}, "nas1")
+	_, err := p.Get(context.Background(), "hosts/nas1/domains/"+testName)
+	wantClass(t, err, provider.ErrTransient)
+}
+
+func assertGone(t *testing.T, api *fakeAPI) {
+	t.Helper()
+	if api.domain(testName) != nil {
+		t.Error("domain still defined")
+	}
+	if names := api.volNames("default"); len(names) != 2 {
+		t.Errorf("pool holds %v, want only the cached base image and its marker", names)
+	}
+}
+
+func TestDelete(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name  string
+		setup func(t *testing.T, api *fakeAPI)
+		check func(t *testing.T, api *fakeAPI)
+	}{
+		{
+			name:  "running domain",
+			setup: func(t *testing.T, api *fakeAPI) { managedDomain(t, api, testName, testUID, true, nil) },
+			check: func(t *testing.T, api *fakeAPI) {
+				if !api.hasCall("DomainDestroy") || !api.hasCall("DomainUndefineFlags") {
+					t.Errorf("calls = %v, want destroy and undefine", api.calls)
+				}
+			},
+		},
+		{
+			name:  "stopped domain",
+			setup: func(t *testing.T, api *fakeAPI) { managedDomain(t, api, testName, testUID, false, nil) },
+			check: func(t *testing.T, api *fakeAPI) {
+				if !api.hasCall("DomainUndefineFlags") {
+					t.Errorf("calls = %v, want undefine despite destroy being invalid", api.calls)
+				}
+			},
+		},
+		{
+			name: "domain gone, volumes left",
+			setup: func(_ *testing.T, api *fakeAPI) {
+				api.putVol("default", overlayVolumeName(testName), formatQcow2, nil)
+				api.putVol("default", seedVolumeName(testName), formatRaw, nil)
+			},
+			check: func(t *testing.T, api *fakeAPI) {
+				if api.hasCall("DomainDestroy") || api.hasCall("DomainUndefineFlags") {
+					t.Errorf("calls = %v, want no domain operations", api.calls)
+				}
+			},
+		},
+		{
+			name:  "nothing exists",
+			setup: func(*testing.T, *fakeAPI) {},
+			check: func(t *testing.T, api *fakeAPI) {
+				if api.hasCall("StorageVolDelete") {
+					t.Errorf("calls = %v, want no deletes", api.calls)
+				}
+			},
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			api := readyHost()
+			tc.setup(t, api)
+			p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+			if err := p.Delete(context.Background(), "hosts/nas1/domains/"+testName); err != nil {
+				t.Fatalf("Delete: %v", err)
+			}
+			assertGone(t, api)
+			tc.check(t, api)
+			if err := p.Delete(context.Background(), "hosts/nas1/domains/"+testName); err != nil {
+				t.Fatalf("second Delete: %v", err)
+			}
+		})
+	}
+}
+
+func TestDelete_missingPoolIsNotAnError(t *testing.T) {
+	t.Parallel()
+	api := newFakeAPI("other")
+	api.putDomain(&fakeDomain{name: testName, metadata: metadataXML(t, testUID, testNow), running: true})
+	p := newTestProvider(hostFakes{"nas1": api}, "nas1")
+
+	if err := p.Delete(context.Background(), "hosts/nas1/domains/"+testName); err != nil {
+		t.Fatalf("Delete: %v", err)
+	}
+	if api.domain(testName) != nil {
+		t.Error("domain still defined")
+	}
+}
+
+func TestDelete_errors(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name  string
+		id    string
+		fakes func(t *testing.T) hostFakes
+		class error
+	}{
+		{"malformed id", "nope", func(*testing.T) hostFakes { return hostFakes{"nas1": readyHost()} }, provider.ErrPermanent},
+		{"host not configured", "hosts/nas9/domains/x", func(*testing.T) hostFakes { return hostFakes{"nas1": readyHost()} }, provider.ErrPermanent},
+		{"host unreachable", "hosts/nas1/domains/x", func(*testing.T) hostFakes { return hostFakes{} }, provider.ErrTransient},
+		{
+			"undefine fails", "hosts/nas1/domains/" + testName,
+			func(t *testing.T) hostFakes {
+				api := readyHost()
+				managedDomain(t, api, testName, testUID, true, nil)
+				api.fail["DomainUndefineFlags"] = lvError(libvirt.ErrInternalError, "boom")
+				return hostFakes{"nas1": api}
+			},
+			provider.ErrTransient,
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			p := newTestProvider(tc.fakes(t), "nas1")
+			wantClass(t, p.Delete(context.Background(), tc.id), tc.class)
+		})
+	}
+}
+
+func TestListByTag_acrossHosts(t *testing.T) {
+	t.Parallel()
+	nas1, nas2 := readyHost(), readyHost()
+	managedDomain(t, nas1, "proxy-one", "uid-1", true, map[uint32][]libvirt.DomainInterface{
+		uint32(libvirt.DomainInterfaceAddressesSrcArp): {{Addrs: []libvirt.DomainIPAddr{ipv4("192.168.1.10")}}},
+	})
+	nas1.putDomain(&fakeDomain{name: "hand-made", running: true})
+	managedDomain(t, nas2, "proxy-two", "uid-2", false, nil)
+	nas2.putDomain(&fakeDomain{name: "foreign", metadata: `<instance xmlns="` + metadataURI + `"><managed>false</managed></instance>`})
+	p := newTestProvider(hostFakes{"nas1": nas1, "nas2": nas2}, "nas1", "nas2")
+
+	got, err := p.ListByTag(context.Background())
+	if err != nil {
+		t.Fatalf("ListByTag: %v", err)
+	}
+	want := []provider.Instance{
+		{ID: "hosts/nas1/domains/proxy-one", IP: "192.168.1.10", State: provider.StateRunning, UID: "uid-1", CreatedAt: testNow},
+		{ID: "hosts/nas2/domains/proxy-two", State: provider.StateStopped, UID: "uid-2", CreatedAt: testNow},
+	}
+	if len(got) != len(want) {
+		t.Fatalf("got %d instances %+v, want %d", len(got), got, len(want))
+	}
+	for i := range want {
+		if got[i] != want[i] {
+			t.Errorf("instance[%d] = %+v, want %+v", i, got[i], want[i])
+		}
+	}
+}
+
+func TestListByTag_oneHostDownFailsSweep(t *testing.T) {
+	t.Parallel()
+	nas1 := readyHost()
+	managedDomain(t, nas1, "proxy-one", "uid-1", true, nil)
+	p := newTestProvider(hostFakes{"nas1": nas1}, "nas1", "nas2")
+
+	got, err := p.ListByTag(context.Background())
+	wantClass(t, err, provider.ErrTransient)
+	if got != nil {
+		t.Errorf("got %+v alongside the error, want nil (no partial view)", got)
+	}
+}
+
+// A domain that cannot be read is left out of the sweep rather than
+// failing it: GC never deletes what it does not see, so skipping is the
+// safe direction, while one bad domain blocking GC for all others is not.
+func TestListByTag_skipsUnreadableDomain(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name  string
+		setup func(t *testing.T, api *fakeAPI)
+	}{
+		{
+			"deleted mid-sweep",
+			func(t *testing.T, api *fakeAPI) {
+				managedDomain(t, api, "proxy-one", "uid-1", true, nil)
+				api.failOnce["DomainGetMetadata"] = lvError(libvirt.ErrNoDomain, "gone")
+			},
+		},
+		{
+			"corrupt metadata",
+			func(_ *testing.T, api *fakeAPI) {
+				api.putDomain(&fakeDomain{name: "proxy-one", metadata: "<instance", running: true})
+			},
+		},
+		{
+			"state unreadable",
+			func(t *testing.T, api *fakeAPI) {
+				managedDomain(t, api, "proxy-one", "uid-1", true, nil)
+				api.failOnce["DomainGetState"] = lvError(libvirt.ErrInternalError, "boom")
+			},
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			nas1 := readyHost()
+			tc.setup(t, nas1)
+			managedDomain(t, nas1, "proxy-two", "uid-2", false, nil)
+			p := newTestProvider(hostFakes{"nas1": nas1}, "nas1")
+
+			got, err := p.ListByTag(context.Background())
+			if err != nil {
+				t.Fatalf("ListByTag: %v", err)
+			}
+			if len(got) != 1 || got[0].UID != "uid-2" {
+				t.Errorf("got %+v, want only proxy-two", got)
+			}
+		})
+	}
+}
+
+func TestListByTag_noHosts(t *testing.T) {
+	t.Parallel()
+	p := newTestProvider(hostFakes{})
+	got, err := p.ListByTag(context.Background())
+	if err != nil || len(got) != 0 {
+		t.Errorf("got %v, %v; want empty, nil", got, err)
+	}
+}
+
+func TestMapState(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		state libvirt.DomainState
+		ip    string
+		want  provider.InstanceState
+	}{
+		{libvirt.DomainRunning, "10.0.0.1", provider.StateRunning},
+		{libvirt.DomainRunning, "", provider.StateProvisioning},
+		{libvirt.DomainBlocked, "10.0.0.1", provider.StateRunning},
+		{libvirt.DomainPaused, "10.0.0.1", provider.StateStopped},
+		{libvirt.DomainShutdown, "", provider.StateStopped},
+		{libvirt.DomainShutoff, "", provider.StateStopped},
+		{libvirt.DomainCrashed, "", provider.StateStopped},
+		{libvirt.DomainPmsuspended, "", provider.StateStopped},
+		{libvirt.DomainNostate, "", provider.StateProvisioning},
+	}
+	for _, tc := range cases {
+		t.Run(fmt.Sprintf("%s/%q", tc.state, tc.ip), func(t *testing.T) {
+			t.Parallel()
+			if got := mapState(tc.state, tc.ip); got != tc.want {
+				t.Errorf("mapState(%s, %q) = %s, want %s", tc.state, tc.ip, got, tc.want)
+			}
+		})
+	}
+}
+
+func TestNew_badKeyPathFailsFast(t *testing.T) {
+	t.Parallel()
+	cfg := testConfig("nas1")
+	cfg.KVM.SSHKeyFile = t.TempDir() + "/missing"
+	if _, err := New(context.Background(), cfg); err == nil {
+		t.Fatal("New succeeded with a missing key file")
+	}
+}

internal/provider/kvm/metadata.go

@@ -0,0 +1,57 @@
+package kvm
+
+import (
+	"encoding/xml"
+	"fmt"
+	"time"
+
+	"gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/internal/provider"
+)
+
+// metadataURI namespaces the operator's element inside the domain's
+// <metadata>. libvirt has no labels or tags, but it stores any namespaced XML
+// there and hands it back through DomainGetMetadata, which is how the GC
+// contract (managed flag + owning UID) and the creation time survive across
+// operator restarts.
+const metadataURI = "https://gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/kvm"
+
+type instanceMetadata struct {
+	XMLName   xml.Name `xml:"https://gitea.home.hrajfrisbee.cz/kacerr/egress-proxies-operator/kvm instance"`
+	Managed   string   `xml:"managed"`
+	UID       string   `xml:"uid"`
+	CreatedAt string   `xml:"createdAt"`
+}
+
+func newInstanceMetadata(uid string, createdAt time.Time) instanceMetadata {
+	return instanceMetadata{
+		Managed:   provider.LabelManagedYes,
+		UID:       uid,
+		CreatedAt: createdAt.UTC().Format(time.RFC3339),
+	}
+}
+
+// parseInstanceMetadata decodes the element as DomainGetMetadata returns it.
+// libvirt may re-serialise it with a namespace prefix rather than the default
+// namespace it was defined with; encoding/xml matches on namespace URI, so
+// both spellings decode.
+func parseInstanceMetadata(s string) (instanceMetadata, error) {
+	var m instanceMetadata
+	if err := xml.Unmarshal([]byte(s), &m); err != nil {
+		return instanceMetadata{}, fmt.Errorf("parsing domain metadata: %w", err)
+	}
+	return m, nil
+}
+
+func (m instanceMetadata) managed() bool {
+	return m.Managed == provider.LabelManagedYes
+}
+
+// createdAt returns the zero time on a malformed timestamp: orphan GC treats
+// zero as "old", so a corrupt value can never protect a domain forever.
+func (m instanceMetadata) createdAt() time.Time {
+	t, err := time.Parse(time.RFC3339, m.CreatedAt)
+	if err != nil {
+		return time.Time{}
+	}
+	return t
+}

internal/provider/kvm/metadata_test.go

@@ -0,0 +1,79 @@
+package kvm
+
+import (
+	"encoding/xml"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestInstanceMetadata_roundTrip(t *testing.T) {
+	t.Parallel()
+	created := time.Date(2026, 9, 4, 12, 30, 0, 0, time.FixedZone("CEST", 2*3600))
+	raw, err := xml.Marshal(newInstanceMetadata("uid-1", created))
+	if err != nil {
+		t.Fatalf("Marshal() error = %v", err)
+	}
+	if !strings.Contains(string(raw), `xmlns="`+metadataURI+`"`) {
+		t.Errorf("marshalled metadata lacks the namespace declaration: %s", raw)
+	}
+	m, err := parseInstanceMetadata(string(raw))
+	if err != nil {
+		t.Fatalf("parseInstanceMetadata() error = %v", err)
+	}
+	if !m.managed() || m.UID != "uid-1" {
+		t.Errorf("parsed = %+v, want managed with uid-1", m)
+	}
+	if got := m.createdAt(); !got.Equal(created) || got.Location() != time.UTC {
+		t.Errorf("createdAt() = %v, want %v in UTC", got, created)
+	}
+}
+
+func TestParseInstanceMetadata_libvirtPrefixedForm(t *testing.T) {
+	t.Parallel()
+	// libvirt re-serialises the stored element with whatever prefix it
+	// chooses; only the namespace URI is contractual.
+	in := `<op:instance xmlns:op="` + metadataURI + `">` +
+		`<op:managed>true</op:managed><op:uid>uid-2</op:uid>` +
+		`<op:createdAt>2026-09-04T10:00:00Z</op:createdAt></op:instance>`
+	m, err := parseInstanceMetadata(in)
+	if err != nil {
+		t.Fatalf("parseInstanceMetadata() error = %v", err)
+	}
+	if !m.managed() || m.UID != "uid-2" || m.createdAt().Hour() != 10 {
+		t.Errorf("parsed = %+v", m)
+	}
+}
+
+func TestParseInstanceMetadata_notManaged(t *testing.T) {
+	t.Parallel()
+	tests := []struct {
+		name    string
+		in      string
+		wantErr bool
+	}{
+		{"managed false", `<instance xmlns="` + metadataURI + `"><managed>false</managed><uid>u</uid></instance>`, false},
+		{"foreign element", `<foo xmlns="urn:other"><managed>true</managed></foo>`, true},
+		{"garbage", `not xml`, true},
+	}
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			m, err := parseInstanceMetadata(tc.in)
+			if (err != nil) != tc.wantErr {
+				t.Fatalf("error = %v, wantErr %v", err, tc.wantErr)
+			}
+			if m.managed() {
+				t.Errorf("managed() = true for %q", tc.in)
+			}
+		})
+	}
+}
+
+func TestInstanceMetadata_createdAtMalformedIsZero(t *testing.T) {
+	t.Parallel()
+	m := instanceMetadata{CreatedAt: "yesterday"}
+	if !m.createdAt().IsZero() {
+		t.Errorf("createdAt() = %v, want zero time", m.createdAt())
+	}
+}

internal/provider/kvm/seed.go

@@ -0,0 +1,53 @@
+package kvm
+
+import (
+	"bytes"
+	"fmt"
+	"strings"
+
+	"github.com/kdomanski/iso9660"
+)
+
+// seedLabel is the volume label cloud-init's NoCloud datasource looks for.
+const seedLabel = "cidata"
+
+// defaultVendorData is the cloud-init vendor-data shipped on every seed
+// unless the config overrides it. cloud-init merges it under the Proxy's own
+// user-data, so stock images grow a guest agent (which lets the provider
+// read the VM's IP before ARP sees traffic) without the user's cloud-init
+// having to know about it.
+const defaultVendorData = `#cloud-config
+packages:
+  - qemu-guest-agent
+runcmd:
+  - [systemctl, enable, --now, qemu-guest-agent]
+`
+
+// buildSeedISO renders a NoCloud seed: user-data verbatim, vendor-data as
+// given, and a meta-data whose instance-id is the deterministic VM name, so
+// cloud-init never mistakes a reboot for a new instance. The ISO's volume
+// descriptor carries the build time, so two builds differ in a few
+// timestamp bytes; the files and label are identical.
+func buildSeedISO(name, userData, vendorData string) ([]byte, error) {
+	w, err := iso9660.NewWriter()
+	if err != nil {
+		return nil, fmt.Errorf("creating iso writer: %w", err)
+	}
+	defer w.Cleanup() //nolint:errcheck // temp-dir cleanup; nothing to do on failure
+
+	files := []struct{ name, body string }{
+		{"user-data", userData},
+		{"meta-data", fmt.Sprintf("instance-id: %s\nlocal-hostname: %s\n", name, name)},
+		{"vendor-data", vendorData},
+	}
+	for _, f := range files {
+		if err := w.AddFile(strings.NewReader(f.body), f.name); err != nil {
+			return nil, fmt.Errorf("adding %s to seed iso: %w", f.name, err)
+		}
+	}
+	var buf bytes.Buffer
+	if err := w.WriteTo(&buf, seedLabel); err != nil {
+		return nil, fmt.Errorf("writing seed iso: %w", err)
+	}
+	return buf.Bytes(), nil
+}

internal/provider/kvm/seed_test.go

@@ -0,0 +1,100 @@
+package kvm
+
+import (
+	"bytes"
+	"io"
+	"maps"
+	"strings"
+	"testing"
+
+	"github.com/kdomanski/iso9660"
+)
+
+func readSeed(t *testing.T, iso []byte) (label string, files map[string]string) {
+	t.Helper()
+	img, err := iso9660.OpenImage(bytes.NewReader(iso))
+	if err != nil {
+		t.Fatalf("OpenImage() error = %v", err)
+	}
+	label, err = img.Label()
+	if err != nil {
+		t.Fatalf("Label() error = %v", err)
+	}
+	root, err := img.RootDir()
+	if err != nil {
+		t.Fatalf("RootDir() error = %v", err)
+	}
+	children, err := root.GetChildren()
+	if err != nil {
+		t.Fatalf("GetChildren() error = %v", err)
+	}
+	files = make(map[string]string, len(children))
+	for _, c := range children {
+		body, err := io.ReadAll(c.Reader())
+		if err != nil {
+			t.Fatalf("reading %s: %v", c.Name(), err)
+		}
+		files[c.Name()] = string(body)
+	}
+	return label, files
+}
+
+func TestBuildSeedISO_contents(t *testing.T) {
+	t.Parallel()
+	userData := "#cloud-config\npackages: [squid]\n"
+	iso, err := buildSeedISO("proxy-abc123def456ghij", userData, defaultVendorData)
+	if err != nil {
+		t.Fatalf("buildSeedISO() error = %v", err)
+	}
+	label, files := readSeed(t, iso)
+	if !strings.EqualFold(label, seedLabel) {
+		t.Errorf("label = %q, want %q", label, seedLabel)
+	}
+	if len(files) != 3 {
+		t.Errorf("files = %v, want exactly user-data, meta-data, vendor-data", files)
+	}
+	if files["user-data"] != userData {
+		t.Errorf("user-data = %q, want verbatim cloud-init", files["user-data"])
+	}
+	if files["vendor-data"] != defaultVendorData {
+		t.Errorf("vendor-data = %q", files["vendor-data"])
+	}
+	if got, want := files["meta-data"], "instance-id: proxy-abc123def456ghij\nlocal-hostname: proxy-abc123def456ghij\n"; got != want {
+		t.Errorf("meta-data = %q, want %q", got, want)
+	}
+}
+
+// The writer stamps the build time into the volume descriptor, so two
+// builds are not byte-identical; what must not change between them is
+// what cloud-init reads: the label and the three files.
+func TestBuildSeedISO_deterministicContent(t *testing.T) {
+	t.Parallel()
+	a, err := buildSeedISO("proxy-x", "#cloud-config\n", "")
+	if err != nil {
+		t.Fatalf("first buildSeedISO() error = %v", err)
+	}
+	b, err := buildSeedISO("proxy-x", "#cloud-config\n", "")
+	if err != nil {
+		t.Fatalf("second buildSeedISO() error = %v", err)
+	}
+	labelA, filesA := readSeed(t, a)
+	labelB, filesB := readSeed(t, b)
+	if labelA != labelB || !maps.Equal(filesA, filesB) {
+		t.Errorf("two builds for the same inputs differ: %q %v vs %q %v", labelA, filesA, labelB, filesB)
+	}
+	if len(a) != len(b) {
+		t.Errorf("sizes differ: %d vs %d; the seed volume is created with the ISO's exact size", len(a), len(b))
+	}
+}
+
+func TestBuildSeedISO_emptyUserData(t *testing.T) {
+	t.Parallel()
+	iso, err := buildSeedISO("proxy-x", "", defaultVendorData)
+	if err != nil {
+		t.Fatalf("buildSeedISO() error = %v", err)
+	}
+	_, files := readSeed(t, iso)
+	if body, ok := files["user-data"]; !ok || body != "" {
+		t.Errorf("user-data = %q (present=%v), want an empty file to still exist", body, ok)
+	}
+}

internal/provider/kvm/volume.go

@@ -0,0 +1,78 @@
+package kvm
+
+import (
+	"encoding/xml"
+	"fmt"
+)
+
+const bytesPerGiB = 1 << 30
+
+func overlayVolumeName(name string) string { return name + ".qcow2" }
+
+func seedVolumeName(name string) string { return name + "-seed.iso" }
+
+// volumeSpec describes a storage volume to create. A non-empty BackingPath
+// makes it a copy-on-write overlay of that file.
+type volumeSpec struct {
+	Name          string
+	SizeBytes     int64
+	Format        string
+	BackingPath   string
+	BackingFormat string
+}
+
+func overlayVolume(name string, sizeGB int64, basePath, baseFormat string) volumeSpec {
+	return volumeSpec{
+		Name:          overlayVolumeName(name),
+		SizeBytes:     sizeGB * bytesPerGiB,
+		Format:        formatQcow2,
+		BackingPath:   basePath,
+		BackingFormat: baseFormat,
+	}
+}
+
+func rawVolume(name string, sizeBytes int64) volumeSpec {
+	return volumeSpec{Name: name, SizeBytes: sizeBytes, Format: formatRaw}
+}
+
+// buildVolumeXML renders the volume for StorageVolCreateXML. Capacity is
+// always given in bytes so no libvirt unit-suffix ambiguity (G vs GiB) can
+// creep in.
+func buildVolumeXML(v volumeSpec) (string, error) {
+	x := volumeXML{
+		Type:     "file",
+		Name:     v.Name,
+		Capacity: sizeXML{Unit: "bytes", Value: v.SizeBytes},
+		Target:   volumeTargetXML{Format: formatXML{Type: v.Format}},
+	}
+	if v.BackingPath != "" {
+		x.BackingStore = &backingStoreXML{Path: v.BackingPath, Format: formatXML{Type: v.BackingFormat}}
+	}
+	out, err := xml.Marshal(x)
+	if err != nil {
+		return "", fmt.Errorf("marshalling volume xml: %w", err)
+	}
+	return string(out), nil
+}
+
+type volumeXML struct {
+	XMLName      xml.Name         `xml:"volume"`
+	Type         string           `xml:"type,attr"`
+	Name         string           `xml:"name"`
+	Capacity     sizeXML          `xml:"capacity"`
+	Target       volumeTargetXML  `xml:"target"`
+	BackingStore *backingStoreXML `xml:"backingStore"`
+}
+
+type volumeTargetXML struct {
+	Format formatXML `xml:"format"`
+}
+
+type formatXML struct {
+	Type string `xml:"type,attr"`
+}
+
+type backingStoreXML struct {
+	Path   string    `xml:"path"`
+	Format formatXML `xml:"format"`
+}

internal/provider/kvm/volume_test.go

@@ -0,0 +1,62 @@
+package kvm
+
+import (
+	"encoding/xml"
+	"strings"
+	"testing"
+)
+
+func TestVolumeNames(t *testing.T) {
+	t.Parallel()
+	if got := overlayVolumeName("proxy-x"); got != "proxy-x.qcow2" {
+		t.Errorf("overlayVolumeName = %q", got)
+	}
+	if got := seedVolumeName("proxy-x"); got != "proxy-x-seed.iso" {
+		t.Errorf("seedVolumeName = %q", got)
+	}
+}
+
+func TestBuildVolumeXML_overlay(t *testing.T) {
+	t.Parallel()
+	raw, err := buildVolumeXML(overlayVolume("proxy-x", 10, "/var/lib/libvirt/images/base-debian-13.qcow2", "qcow2"))
+	if err != nil {
+		t.Fatalf("buildVolumeXML() error = %v", err)
+	}
+	var v volumeXML
+	if err := xml.Unmarshal([]byte(raw), &v); err != nil {
+		t.Fatalf("output is not well-formed XML: %v", err)
+	}
+	if v.Type != "file" || v.Name != "proxy-x.qcow2" {
+		t.Errorf("type/name = %s/%s", v.Type, v.Name)
+	}
+	if v.Capacity.Unit != "bytes" || v.Capacity.Value != 10*bytesPerGiB {
+		t.Errorf("capacity = %d %s, want %d bytes", v.Capacity.Value, v.Capacity.Unit, 10*bytesPerGiB)
+	}
+	if v.Target.Format.Type != "qcow2" {
+		t.Errorf("format = %q", v.Target.Format.Type)
+	}
+	if v.BackingStore == nil {
+		t.Fatalf("backingStore missing:\n%s", raw)
+	}
+	if v.BackingStore.Path != "/var/lib/libvirt/images/base-debian-13.qcow2" || v.BackingStore.Format.Type != "qcow2" {
+		t.Errorf("backingStore = %+v", v.BackingStore)
+	}
+}
+
+func TestBuildVolumeXML_rawHasNoBackingStore(t *testing.T) {
+	t.Parallel()
+	raw, err := buildVolumeXML(rawVolume("proxy-x-seed.iso", 372736))
+	if err != nil {
+		t.Fatalf("buildVolumeXML() error = %v", err)
+	}
+	if strings.Contains(raw, "backingStore") {
+		t.Errorf("raw volume must not declare a backing store:\n%s", raw)
+	}
+	var v volumeXML
+	if err := xml.Unmarshal([]byte(raw), &v); err != nil {
+		t.Fatalf("output is not well-formed XML: %v", err)
+	}
+	if v.Name != "proxy-x-seed.iso" || v.Capacity.Value != 372736 || v.Target.Format.Type != "raw" {
+		t.Errorf("volume = %+v", v)
+	}
+}