docs: start docs/operating.md with the libvirt-RPC log query
A place for helpers against a running operator, as opposed to the design docs. First entry is the jq filter for timestamp + rpc: both libvirt providers log the same "libvirt call" msg, so one select covers kvm and kvm2, and a fromjson? variant survives a stream with non-JSON lines in it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,12 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 2026-09-18 07:52 UTC — docs/operating.md, starting with the libvirt-RPC log query
|
||||||
|
|
||||||
|
New home for day-to-day helpers against a running operator, seeded with the `jq` query
|
||||||
|
that pulls timestamp + rpc out of the manager's JSON logs (both libvirt providers log the
|
||||||
|
same `"libvirt call"` msg, so one filter covers `kvm` and `kvm2`), plus the `fromjson?`
|
||||||
|
variant for streams carrying non-JSON lines. Linked from the README's doc index.
|
||||||
|
|
||||||
## 2026-09-18 07:44 UTC — a kvm2 crawl-pool sample
|
## 2026-09-18 07:44 UTC — a kvm2 crawl-pool sample
|
||||||
|
|
||||||
Until now the only kvm2 manifest was the one-VM `config/samples/proxy_kvm2.yaml`, which
|
Until now the only kvm2 manifest was the one-VM `config/samples/proxy_kvm2.yaml`, which
|
||||||
|
|||||||
@@ -310,4 +310,6 @@ and the manual kind verification procedure — is in
|
|||||||
|
|
||||||
Project layout, reconcile-loop diagrams, and the decision log are in
|
Project layout, reconcile-loop diagrams, and the decision log are in
|
||||||
[docs/architecture.md](docs/architecture.md); the build history is in
|
[docs/architecture.md](docs/architecture.md); the build history is in
|
||||||
[docs/plans-executions/](docs/plans-executions/).
|
[docs/plans-executions/](docs/plans-executions/). Day-to-day helpers for a
|
||||||
|
running operator — log queries and one-off inspections — are collected in
|
||||||
|
[docs/operating.md](docs/operating.md).
|
||||||
|
|||||||
@@ -0,0 +1,58 @@
|
|||||||
|
# Operating the operator
|
||||||
|
|
||||||
|
Helpers for running the operator day to day: log queries, one-off
|
||||||
|
inspections, and the small recipes that are easier to look up than to
|
||||||
|
reconstruct. Not a runbook — it assumes you already know what you are
|
||||||
|
looking for.
|
||||||
|
|
||||||
|
## Log queries
|
||||||
|
|
||||||
|
The manager logs one JSON object per line, so `jq` is the tool. The
|
||||||
|
`.msg` field is the discriminator: it is a fixed string per event, and
|
||||||
|
the variable parts live in their own fields rather than being
|
||||||
|
interpolated into the message.
|
||||||
|
|
||||||
|
### Libvirt RPCs, with timestamps
|
||||||
|
|
||||||
|
Both libvirt providers log one `"libvirt call"` per RPC, so this catches
|
||||||
|
`kvm` and `kvm2` alike:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
jq -r 'select(.msg == "libvirt call") | "\(.ts) \(.rpc)"'
|
||||||
|
```
|
||||||
|
|
||||||
|
```text
|
||||||
|
2026-09-18T07:48:29Z virStoragePoolGetInfo
|
||||||
|
2026-09-18T07:48:29Z ConnectGetLibVersion
|
||||||
|
2026-09-18T07:48:29Z ConnectGetHostname
|
||||||
|
2026-09-18T07:48:29Z StoragePoolLookupByName
|
||||||
|
2026-09-18T07:48:29Z StoragePoolGetInfo
|
||||||
|
```
|
||||||
|
|
||||||
|
These lines are debug level, which is the binary's default
|
||||||
|
(`zap.Options{Development: true}` in `cmd/main.go`) — nothing to turn on.
|
||||||
|
They disappear if the Deployment has been given `--zap-log-level=info`,
|
||||||
|
which is the switch for outcomes only.
|
||||||
|
|
||||||
|
Piping from a live manager:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
kubectl logs -n egress-proxies-operator-system \
|
||||||
|
deploy/egress-proxies-operator-controller-manager -f \
|
||||||
|
| jq -r 'select(.msg == "libvirt call") | "\(.ts) \(.rpc)"'
|
||||||
|
```
|
||||||
|
|
||||||
|
`jq` aborts on the first line that is not JSON, which happens when
|
||||||
|
something bypasses the structured logger — a panic trace, or output from
|
||||||
|
a tool wrapping the manager. Reading line-by-line with `fromjson?` skips
|
||||||
|
those instead:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
jq -rR 'fromjson? | select(.msg == "libvirt call") | [.ts, .rpc] | @tsv'
|
||||||
|
```
|
||||||
|
|
||||||
|
Note that the connection-level events use their own `.msg` values
|
||||||
|
(`"libvirt host connected"`, `"kvm2 host checked"`), so they drop out of
|
||||||
|
the filters above. Widen to `select(.msg | startswith("libvirt"))` to
|
||||||
|
keep the connect lines — usually what you want when chasing a latency
|
||||||
|
spike rather than a call count.
|
||||||
Reference in New Issue
Block a user