diff --git a/.claude/agents/cluster-inspector.md b/.claude/agents/cluster-inspector.md index 0099a03..725fdc7 100644 --- a/.claude/agents/cluster-inspector.md +++ b/.claude/agents/cluster-inspector.md @@ -24,7 +24,8 @@ Triage order: 2. Objects + conditions: `kubectl get cluster,kcp,md,ms,machines,proxmoxmachines -A -o wide`, `kubectl describe` the first non-Ready object. 3. Controller logs: `capi-system/capi-controller-manager`, `capmox-system/capmox-controller-manager`, - `capi-kubeadm-*-system`, IPAM (`kubectl get ipaddressclaims,ipaddresses -A`). + `capi-kubeadm-*-system`, IPAM (`kubectl get ipaddressclaims,ipaddresses.ipam.cluster.x-k8s.io -A`; plain + `ipaddresses` is the built-in networking.k8s.io type). 4. Node side: Node vs Machine providerID match (`kubectl get nodes -o custom-columns=N:.metadata.name,P:.spec.providerID`), kubelet/cloud-init status if SSH access is given. diff --git a/.claude/cluster-inspector/rbac.yaml b/.claude/cluster-inspector/rbac.yaml index 6deaa1c..53519d9 100644 --- a/.claude/cluster-inspector/rbac.yaml +++ b/.claude/cluster-inspector/rbac.yaml @@ -33,6 +33,10 @@ rules: - apiGroups: [""] resources: ["nodes", "namespaces"] verbs: ["get", "list", "watch"] + # clusterctl describe checks the CAPI CRD storage version + - apiGroups: ["apiextensions.k8s.io"] + resources: ["customresourcedefinitions"] + verbs: ["get", "list", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding diff --git a/docs/phase-1-proxmox-capi.md b/docs/phase-1-proxmox-capi.md index 2769964..18769b7 100644 --- a/docs/phase-1-proxmox-capi.md +++ b/docs/phase-1-proxmox-capi.md @@ -19,7 +19,7 @@ Versions: see [versions.md](versions.md). Template build: see [image-builder-pro - [x] Step 0 — Proxmox prep (API token, ACLs, bridge/VLAN, IP ranges) - [x] Step 1 — VM template `ubuntu-2404-kube-v1.35.9` built and smoke-tested - [x] Step 2 — management cluster (`ansible/mgmt.yml`: k3s + cluster-api-operator + providers) -- [ ] Step 3 — workload cluster `lab1` +- [x] Step 3 — workload cluster `lab1` (VMs 102/103, `.31`/`.32`, VIP `.30`) - [ ] Step 4 — CNI (Cilium) - [ ] Step 5 — MachineDeployment autoscaling annotations - [ ] Step 6 — cluster-autoscaler @@ -117,7 +117,8 @@ kubectl get proxmoxmachines,machines -w clusterctl get kubeconfig lab1 > lab1.kubeconfig ``` -Node IPs come from the IPAM pool (`kubectl get ipaddressclaims,ipaddresses`). providerID is set by +Node IPs come from the IPAM pool (`kubectl get ipaddressclaims,ipaddresses.ipam.cluster.x-k8s.io`; +plain `ipaddresses` resolves to the built-in `networking.k8s.io` type on k8s ≥ 1.33). providerID is set by the CAPMOX template as `proxmox://{{ ds.meta_data.instance_id }}` — don't touch it, the autoscaler matches Nodes to Machines by it. diff --git a/docs/sessions/2026-09-27-1112-lab1-first-review.md b/docs/sessions/2026-09-27-1112-lab1-first-review.md new file mode 100644 index 0000000..aecb02f --- /dev/null +++ b/docs/sessions/2026-09-27-1112-lab1-first-review.md @@ -0,0 +1,43 @@ +# Session: lab1 created — first review by the cluster-inspector agent + +**Created:** 2026-09-27 11:12 CEST + +## Question + +> lab cluster looks good, can you look at it with inspector agent ? + +`lab1` had just been applied from `configs/capi/lab1.yaml` (phase-1 Step 3). Only the +management-side read-only kubeconfig existed, so the review covered what the management cluster sees. + +## What was checked + +`cluster-inspector` agent with `lab-mgmt-ro`: Cluster / KCP / MD / MS / Machines / ProxmoxMachines / +ProxmoxCluster conditions, IPAM claims, MD field managers, logs of all five CAPI controllers since +creation, Warning events in `default`. `clusterctl describe` failed (403 on CRDs). + +## Findings + +- CAPI / CAPMOX side healthy: VMs 102 (CP, 192.168.4.31) and 103 (worker, .32) cloned from + template 101 with VLAN 4, raw disk, planned sizes; control plane initialised and Available, + etcd/certs healthy, VIP 192.168.4.30 reachable; providerIDs match across Machine / + ProxmoxMachine / Node. First real use of the `capmox@pve!capi` token: no Proxmox API errors. +- Controller errors only in the first ~60 s after creation (no route to VIP, `lab1-kubeconfig` + not found, `vm not initialized`) — normal bootstrap noise, nothing after 09:01 UTC. +- MachineDeployment `replicas` defaulted to 1 and is owned by no field manager, so re-applying + `lab1.yaml` won't fight cluster-autoscaler. +- Both nodes NotReady (`cni plugin not initialized`) as of ~09:09 UTC → Cilium (Step 4) not + installed or not starting; the owner's "looks good" didn't hold for node readiness. +- IPAM pool 2/9 used; 1 CP + 5 workers + 1 surge = 7 fits. +- On k8s 1.35, `kubectl get ipaddresses` hits the built-in `networking.k8s.io` IPAddress, not + CAPI's — runbook and agent now use `ipaddresses.ipam.cluster.x-k8s.io`. + +## Open questions + +- Why the CNI wasn't up (not installed yet vs. image pull / egress from VLAN 4) — needs the lab1 + read-only kubeconfig or the owner's check. + +## Follow-ups + +- `.claude/cluster-inspector/rbac.yaml`: CRD read added (for `clusterctl describe`); applied by the + next `ansible-playbook mgmt.yml --tags inspector`. +- Phase-1 checklist: Step 3 ticked; next Step 4 (Cilium), then Step 5 annotations in `lab1.yaml`.