lab1: pin cilium-operator to the control plane; Step 5 verified

Re-check of lab1 after Steps 4-5 (inspector on lab-mgmt, read-only gets
on lab1):
- Cluster/KCP/MD Available for 13h; no controller errors since bootstrap.
- All six autoscaler annotations on lab1-workers (copied to the
  MachineSet). spec.replicas has no owner: kubectl-client-side-apply owns
  only clusterName/selector/template, so re-applying lab1.yaml won't
  fight cluster-autoscaler.
- Nodes Ready on v1.35.9; Node/Machine providerIDs match.
- Cilium, envoy and CoreDNS healthy. No PDBs in the cluster.

Found: cilium-operator (replicas: 1) was scheduled onto the only worker.
It is a non-DaemonSet kube-system pod without a PDB, so with CA's default
skip-nodes-with-system-pods=true the last worker could never scale to
zero (Step 7). The chart already tolerates the control-plane taint, so a
nodeSelector on node-role.kubernetes.io/control-plane pins it there.

- configs/cilium/values.yaml: operator.nodeSelector.
- phase-1 doc: Step 5 ticked, Step 4 explains why replicas: 1 isn't
  enough, new troubleshooting row.
- Session note extended with the re-check.

Not applied yet: the owner runs helm upgrade with the new values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/phase-1-proxmox-capi.md

@@ -21,7 +21,7 @@ Versions: see [versions.md](versions.md). Template build: see [image-builder-pro
 - [x] Step 2 — management cluster (`ansible/mgmt.yml`: k3s + cluster-api-operator + providers)
 - [x] Step 3 — workload cluster `lab1` (VMs 102/103, `.31`/`.32`, VIP `.30`)
 - [x] Step 4 — CNI (Cilium 1.20.2, `configs/cilium/values.yaml`)
-- [ ] Step 5 — MachineDeployment autoscaling annotations
+- [x] Step 5 — MachineDeployment autoscaling annotations
 - [ ] Step 6 — cluster-autoscaler
 - [ ] Step 7 — scale tests
 
@@ -142,8 +142,10 @@ kubectl -n kube-system get pods -o wide                    # cilium DaemonSet 2/
 ```
 
 - `ipam.mode=kubernetes` uses the pod CIDR from `clusterNetwork` instead of Cilium's `10.0.0.0/8` default.
-- `operator.replicas: 1`: the default 2 replicas with node anti-affinity put one on a worker, and a
-  kube-system pod without a PDB blocks scale-down to zero.
+- `operator.replicas: 1` + `operator.nodeSelector` on `node-role.kubernetes.io/control-plane`: the
+  default 2 replicas with node anti-affinity put one on a worker, and a kube-system pod without a
+  PDB blocks scale-down to zero. `replicas: 1` alone isn't enough: the single replica was scheduled
+  onto the worker (the chart tolerates the control-plane taint, so the selector pins it there).
 - kube-proxy stays (kubeadm default); no kube-proxy replacement in this phase.
 
 ## Step 5 — autoscaling annotations
@@ -208,6 +210,7 @@ CPU-bound at 2 pods per 4-core / 4 GiB worker).
 | CA never scales down | Node `providerID` ≠ Machine `providerID` | compare `kubectl get nodes -o custom-columns=N:.metadata.name,P:.spec.providerID` with `kubectl get machines` |
 | MD jumps back to old count | `spec.replicas` in manifest re-applied | remove `replicas` |
 | last worker won't go to 0 | kube-system pods without PDB (`skip-nodes-with-system-pods=true`) | PDB for CoreDNS or `skip-nodes-with-system-pods: false` |
+| `cilium-operator` runs on a worker despite `operator.replicas: 1` | chart tolerates the CP taint but doesn't prefer the CP; the scheduler picked the worker | `operator.nodeSelector: {node-role.kubernetes.io/control-plane: ""}` in `configs/cilium/values.yaml`, `helm upgrade` |
 | scale-from-zero does nothing | capacity annotations missing/wrong | fix cpu/memory/maximum-pods |
 | clone fails `unsupported format 'qcow2'` | qcow2 on LVM-thin | remove `format` |
 | ProxmoxMachine stuck without IP | IPAM pool exhausted | `kubectl get ipaddressclaims`, widen range |

docs/sessions/2026-09-27-1112-lab1-first-review.md

@@ -41,3 +41,34 @@ creation, Warning events in `default`. `clusterctl describe` failed (403 on CRDs
 - `.claude/cluster-inspector/rbac.yaml`: CRD read added (for `clusterctl describe`); applied by the
   next `ansible-playbook mgmt.yml --tags inspector`.
 - Phase-1 checklist: Step 3 ticked; next Step 4 (Cilium), then Step 5 annotations in `lab1.yaml`.
+
+## 2026-09-28 00:55 CEST — re-check after Steps 4–5
+
+**What was checked:** the `cluster-inspector` agent with `lab-mgmt-ro` covered the MD annotations,
+field managers, Cluster/KCP/MD conditions, IPAM and controller logs. For the workload side, Claude
+ran read-only `kubectl get` with the owner's admin `~/.kube/lab1.kubeconfig`, because no `lab1-ro`
+kubeconfig exists yet. The agent refuses admin kubeconfigs when permission is relayed through
+another agent.
+
+**Findings:**
+
+- `lab1` has been stable for 13h. Cluster, KCP and MD are Available, and the controllers have
+  logged no errors since the ~09:00 UTC bootstrap noise.
+- All six Step 5 annotations are on `lab1-workers` and copied to its MachineSet.
+- Nothing owns `spec.replicas`: `kubectl-client-side-apply` owns only
+  clusterName/selector/template.
+- Both nodes are Ready on v1.35.9, and Node providerIDs match the Machines.
+- Cilium 2/2, envoy 2/2, operator 1/1 and CoreDNS 2/2 (both replicas on the CP) are healthy. No
+  pods are outside Running, and there are no PDBs anywhere.
+- **`cilium-operator` runs on the worker** despite `operator.replicas: 1`. With CA's default
+  `skip-nodes-with-system-pods=true` and no PDB, it would block Step 7's scale-to-zero. Fixed in
+  `configs/cilium/values.yaml` with a `nodeSelector` on the control-plane role; the owner still
+  has to apply it with `helm upgrade`.
+
+**Follow-ups:**
+
+- Owner: `helm upgrade` Cilium with the new values, then confirm the operator runs on
+  `lab1-control-plane-*`.
+- Owner: mint a `lab1-ro` kubeconfig with `.claude/cluster-inspector/make-kubeconfig.sh`, so that
+  workload checks can go through the agent.
+- After Step 6: check that only CA / the scale subresource owns `spec.replicas`.