lab1 workers on both PVE nodes: per-node template + MD, balanced CA

PVE is now the 2-node cluster homelab (pv1-1 + pv1-2), no shared storage:
- template 101 copied to pv1-2 as 9101; MDs lab1-workers-pv1-1 (max 6) and
  lab1-workers-pv1-2 (max 12), PMTs pinned with allowedNodes; pool .31-.50;
  lab1-workers renamed for symmetry (decision #18, #7 superseded)
- CA: balancing-label node.cluster.x-k8s.io/pool (+ pool label, capacity labels
  with os/arch, maxPods on the MDs), scan-interval 1m, v 2, default QPS (#19).
  Scale test: 16 pods -> 4 + 4 across both nodes.
- runbook, image-builder copy procedure, fixes tables, future task to report
  three CA clusterapi bugs upstream, lab facts, changelog, execution log

Plan: docs/plans/2026-09-29-2315-pv1-2-workers.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-30 11:07:17 +02:00
co-authored by Claude Opus 5.5
parent 5e5b6c506a
commit 6ef55778ff
14 changed files with 476 additions and 50 deletions
+35 -1
View File
@@ -3,6 +3,40 @@
Deferred work, newest first. When a task is picked up it gets a plan in `docs/plans/`; then tick
it here and link the plan.
## - [ ] Report three cluster-autoscaler clusterapi-provider bugs upstream
**Added:** 2026-09-30 09:28 CEST · **Why deferred:** worked around in config (decision #19);
reporting is not blocking.
**Problem.** Three bugs in CA 1.35.2 `cloudprovider/clusterapi`, all still on master and 1.36.1 as
of 2026-09-29, found in [the pv1-2 execution log](plans-executions/2026-09-29-2315-pv1-2-workers.md):
1. **Uncached discovery.**
- `clusterapi_provider.go:195` builds a `memory.NewMemCacheClient`, but passes the raw
discovery client to the controller (`:209`).
- `getKindPreferredVersion` therefore does 3 GETs per MD at 0 replicas on every `NodeGroups()`
call, about 15× per loop. The loop took 18 s at 5 QPS.
2. **`Labels()` reads the wrong path.** It reads `spec.template.spec.metadata.labels`
(`clusterapi_unstructured.go:207`), which doesn't exist in CAPI v1beta2, so MD template labels
never reach the from-zero template node. The unit tests use the same wrong path.
3. **Empty os/arch labels.** For an infra template without `status`, as with CAPMOX,
`InstanceSystemInfo()` returns an empty struct. The template node then gets
`kubernetes.io/os=""` and `arch=""`, which override the generic `linux`/`amd64`.
**Options:**
- Issues only.
- Issues plus small PRs: pass the cached client, fix the path, skip empty nodeInfo fields.
- Carry a patched CA image: rejected, versions must stay upstream and pinned.
**Cost.** About 1–2 h for issues with source references. PRs would add the CLA and a test for each.
**Changes when done:**
- Link the issues/PRs in `docs/phase-1-proxmox-capi.md` troubleshooting.
- Once fixed in a 1.35.x patch release: bump CA in `docs/versions.md`, then revisit
`scan-interval` and the `capacity…/labels` annotation.
**Run by the owner:** nothing on the lab. Filing on GitHub is the owner's call, since it goes out
under their account.
## - [ ] Namespace per workload cluster (`lab1` out of `default`)
**Added:** 2026-09-28 09:29 CEST · **Why deferred:** keep phase 1 moving; `default` works for now.
@@ -36,7 +70,7 @@ next cluster.
(`helm uninstall ca -n default` first if it was installed in `default`).
- `ansible/group_vars/all.yml`: `inspector_workload_clusters` entry `{ name: lab1, namespace: lab1 }`.
- `docs/phase-1-proxmox-capi.md`: Steps 3, 5 and 6 (namespace in commands and log expectations,
`MachineDeployment/lab1/lab1-workers`).
`MachineDeployment/lab1/lab1-workers-pv1-{1,2}`).
- `.claude/agents/cluster-inspector.md`: the "Where things live" table says CA runs in `default`.
- `docs/decisions.md`: new row (namespace per workload cluster).