kacerrandClaude Opus 5.5
5e5b6c506a
Owner's pending work, committed ahead of the pv1-2 change it underlies: - lab1.yaml: worker template lab1-worker-2g (2048 MiB, decision #16), MD rollout maxSurge 0 / maxUnavailable 1, ProxmoxCluster schedulerHints.memoryAdjustment 160 (#17) - cluster-autoscaler values: RBAC for proxmoxmachinetemplates, aggressive scale-down profile, Cilium startup-taint; runbook Steps 4-7 and troubleshooting rows - inflate test pods 512Mi; decisions #16/#17; session note on the memory overcommit - docs/future-tasks.md convention (CLAUDE.md, README) with the namespace-per-cluster task - AGENTS.md; allowlist entries in .claude/settings.json Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5.7 KiB
5.7 KiB
Decisions
Every direction chosen, rejected or reversed, newest at the bottom. Rows are never rewritten:
a reversal is a new row, and the old row's status becomes superseded by #N. Status is
active, proposed (not yet confirmed by lab work) or superseded by #N.
Rows 1–9 were back-filled on 2026-09-26 from the README, versions.md, the phase docs and the
image-builder fixes log; their dates are when the decision was recorded, not necessarily made.
| # | Date | Decision | Alternatives considered | Why | Status |
|---|---|---|---|---|---|
| 1 | 2026-09-24 | Pin CAPI core to 1.12.x (versions.md) | latest CAPI 1.14.x | CAPMOX v0.9.x is only tested against CAPI 1.11/1.12; caps workload Kubernetes and cluster-autoscaler at 1.35 | active |
| 2 | 2026-09-24 | Management cluster is k3s on a Proxmox VM | kind on a workstation | CAPI controllers and the autoscaler must run permanently | active |
| 3 | 2026-09-24 | Node images built with kubernetes-sigs/image-builder (Ubuntu 24.04, Proxmox builder) | — | upstream-maintained CAPI images | active |
| 4 | 2026-09-24 | image-builder overrides go in a PACKER_VAR_FILES JSON |
PACKER_FLAGS |
flags were silently lost when PACKER_FLAGS was re-exported (fixes log #4) |
active |
| 5 | 2026-09-24 | Cilium installed by Helm with ipam.mode=kubernetes, pod CIDR 10.244.0.0/16 |
ClusterResourceSet CNI flavors; pod CIDR 192.168.0.0/16 |
keeps the CNI version under our control; 192.168.0.0/16 overlaps the lab LAN |
active |
| 6 | 2026-09-24 | Scale-from-zero via capacity.cluster-autoscaler.kubernetes.io/* annotations on the MachineDeployment |
infrastructure provider publishing status.capacity |
CAPMOX does not publish status.capacity on ProxmoxMachineTemplate |
active |
| 7 | 2026-09-24 | Template only on pv1-1, ALLOWED_NODES=[pv1-1] |
template on every node; shared storage | local-lvm is node-local and there is no shared storage yet |
active |
| 8 | 2026-09-26 | Phase 2: spike CAPLV first; own provider if insufficient; KubeVirt as fallback (phase-2) | CAPK + KubeVirt, Metal3, own provider from the start | CAPLV shares CAPI 1.12 with CAPMOX and fits plain libvirt hosts directly | proposed |
| 9 | 2026-09-26 | Phases 2 and 3 are independent and can be done in either order | strict sequence | no technical dependency between the KVM provider and hosted control planes | active |
| 10 | 2026-09-26 | Keep project history in the repo: plans, execution logs, session notes, decisions, prompts, reviews, changelog (session) | chat history only; raw transcripts committed | research repo, reasoning must outlive sessions; transcripts are noisy and may contain secrets | active |
| 11 | 2026-09-27 | Cluster inspection by agents only via read-only ServiceAccount kubeconfigs minted by .claude/cluster-inspector/make-kubeconfig.sh (short-lived TokenRequest tokens, gitignored) |
admin kubeconfig + permission allowlist; long-lived SA token Secret; RBAC under configs/ |
RBAC (view, no Secrets) enforces read-only regardless of which kubectl verb an agent runs — also closes the kubectl get secret gap from the history-conventions session; short-lived tokens limit leak impact |
active |
| 12 | 2026-09-27 | Management cluster built by one idempotent Ansible playbook, ansible/mgmt.yml, run from the laptop (plan) |
cloud-init + k3s auto-deploy manifests; bash scripts with guards; OpenTofu + Helm | owner's choice; idempotent modules, one tool; only ansible.builtin, charts through k3s's helm-controller so nothing extra on the VM |
active |
| 13 | 2026-09-27 | CAPI providers installed by cluster-api-operator 0.28.0 (Provider CRs), cert-manager v1.21.2 pre-installed | guarded clusterctl init |
declarative; re-apply is a no-op; upgrade = spec.version bump; 0.28.0 built against CAPI 1.12.10. Consequence: no clusterctl init/upgrade/move/delete |
active |
| 14 | 2026-09-27 | Management VM created by the playbook from the pinned Ubuntu 24.04 cloud image (VM 201, 192.168.4.21) | hand-made VM; the image-builder node template | reproducible from the repo; the image-builder template is a kubeadm node image, not meant for k3s | active |
| 15 | 2026-09-27 | lab1 workers 4 vCPU / 4 GiB (not 8 GiB); inflate test pods request 1500m / 1Gi so they stay CPU-bound at 2 per worker |
8 GiB workers with 2Gi inflate pods | owner's choice to save hypervisor RAM; scale tests keep the 1 → 3 worker shape (configs/capi/lab1.yaml) | superseded by #16 |
| 16 | 2026-09-28 | lab1 workers 4 vCPU / 2 GiB (new ProxmoxMachineTemplate/lab1-worker-2g, rolled in via the MD); inflate pods 1500m / 512Mi, still CPU-bound at 2 per worker |
keep 4 GiB and delete stopped VMs; CAPMOX memoryAdjustment overcommit |
CAPMOX reserves the configured memory of every VM, stopped ones included, so 4 GiB workers ran out after the first scale-up (troubleshooting); 2 GiB (~1.78 GiB allocatable) still leaves room for kubelet/containerd plus real pods, and max 5 workers now fits the host | proposed |
| 17 | 2026-09-28 | CAPMOX memory overcommit on lab1: ProxmoxCluster.spec.schedulerHints.memoryAdjustment: 160 (46.7 GiB reservable on the 29.16 GiB pv1-1); MD rollout maxSurge 0 / maxUnavailable 1 |
100% (default); 120% (minimum sizing for max 5 × 2 GiB workers); 0 (memory check off); deleting stopped VMs | CAPMOX counts the configured memory of stopped VMs (12 GiB here), and no release can use real free memory; 160% keeps a bound, and host-side protection is KSM + swap, not ballooning (session) | proposed |