kacerrandClaude Opus 5
1fa3e55b9c
A place for helpers against a running operator, as opposed to the design docs. First entry is the jq filter for timestamp + rpc: both libvirt providers log the same "libvirt call" msg, so one select covers kvm and kvm2, and a fromjson? variant survives a stream with non-JSON lines in it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
59 lines
2.0 KiB
Markdown
59 lines
2.0 KiB
Markdown
# Operating the operator
|
|
|
|
Helpers for running the operator day to day: log queries, one-off
|
|
inspections, and the small recipes that are easier to look up than to
|
|
reconstruct. Not a runbook — it assumes you already know what you are
|
|
looking for.
|
|
|
|
## Log queries
|
|
|
|
The manager logs one JSON object per line, so `jq` is the tool. The
|
|
`.msg` field is the discriminator: it is a fixed string per event, and
|
|
the variable parts live in their own fields rather than being
|
|
interpolated into the message.
|
|
|
|
### Libvirt RPCs, with timestamps
|
|
|
|
Both libvirt providers log one `"libvirt call"` per RPC, so this catches
|
|
`kvm` and `kvm2` alike:
|
|
|
|
```sh
|
|
jq -r 'select(.msg == "libvirt call") | "\(.ts) \(.rpc)"'
|
|
```
|
|
|
|
```text
|
|
2026-09-18T07:48:29Z virStoragePoolGetInfo
|
|
2026-09-18T07:48:29Z ConnectGetLibVersion
|
|
2026-09-18T07:48:29Z ConnectGetHostname
|
|
2026-09-18T07:48:29Z StoragePoolLookupByName
|
|
2026-09-18T07:48:29Z StoragePoolGetInfo
|
|
```
|
|
|
|
These lines are debug level, which is the binary's default
|
|
(`zap.Options{Development: true}` in `cmd/main.go`) — nothing to turn on.
|
|
They disappear if the Deployment has been given `--zap-log-level=info`,
|
|
which is the switch for outcomes only.
|
|
|
|
Piping from a live manager:
|
|
|
|
```sh
|
|
kubectl logs -n egress-proxies-operator-system \
|
|
deploy/egress-proxies-operator-controller-manager -f \
|
|
| jq -r 'select(.msg == "libvirt call") | "\(.ts) \(.rpc)"'
|
|
```
|
|
|
|
`jq` aborts on the first line that is not JSON, which happens when
|
|
something bypasses the structured logger — a panic trace, or output from
|
|
a tool wrapping the manager. Reading line-by-line with `fromjson?` skips
|
|
those instead:
|
|
|
|
```sh
|
|
jq -rR 'fromjson? | select(.msg == "libvirt call") | [.ts, .rpc] | @tsv'
|
|
```
|
|
|
|
Note that the connection-level events use their own `.msg` values
|
|
(`"libvirt host connected"`, `"kvm2 host checked"`), so they drop out of
|
|
the filters above. Widen to `select(.msg | startswith("libvirt"))` to
|
|
keep the connect lines — usually what you want when chasing a latency
|
|
spike rather than a call count.
|