egress-proxies-operator/docs/operating.md

Operating the operator

Helpers for running the operator day to day: log queries, one-off inspections, and the small recipes that are easier to look up than to reconstruct. Not a runbook — it assumes you already know what you are looking for.

Log queries

The manager logs one JSON object per line, so jq is the tool. The .msg field is the discriminator: it is a fixed string per event, and the variable parts live in their own fields rather than being interpolated into the message.

Libvirt RPCs, with timestamps

Both libvirt providers log one "libvirt call" per RPC, so this catches kvm and kvm2 alike:

jq -r 'select(.msg == "libvirt call") | "\(.ts) \(.rpc)"'

2026-09-18T07:48:29Z virStoragePoolGetInfo
2026-09-18T07:48:29Z ConnectGetLibVersion
2026-09-18T07:48:29Z ConnectGetHostname
2026-09-18T07:48:29Z StoragePoolLookupByName
2026-09-18T07:48:29Z StoragePoolGetInfo

These lines are debug level, which is the binary's default (zap.Options{Development: true} in cmd/main.go) — nothing to turn on. They disappear if the Deployment has been given --zap-log-level=info, which is the switch for outcomes only.

Piping from a live manager:

kubectl logs -n egress-proxies-operator-system \
  deploy/egress-proxies-operator-controller-manager -f \
  | jq -r 'select(.msg == "libvirt call") | "\(.ts) \(.rpc)"'

jq aborts on the first line that is not JSON, which happens when something bypasses the structured logger — a panic trace, or output from a tool wrapping the manager. Reading line-by-line with fromjson? skips those instead:

jq -rR 'fromjson? | select(.msg == "libvirt call") | [.ts, .rpc] | @tsv'

Note that the connection-level events use their own .msg values ("libvirt host connected", "kvm2 host checked"), so they drop out of the filters above. Widen to select(.msg | startswith("libvirt")) to keep the connect lines — usually what you want when chasing a latency spike rather than a call count.