lab1: 2 GiB workers, CAPMOX memory overcommit, CA install (Step 6), future-tasks

Owner's pending work, committed ahead of the pv1-2 change it underlies:
- lab1.yaml: worker template lab1-worker-2g (2048 MiB, decision #16), MD rollout
  maxSurge 0 / maxUnavailable 1, ProxmoxCluster schedulerHints.memoryAdjustment 160 (#17)
- cluster-autoscaler values: RBAC for proxmoxmachinetemplates, aggressive scale-down
  profile, Cilium startup-taint; runbook Steps 4-7 and troubleshooting rows
- inflate test pods 512Mi; decisions #16/#17; session note on the memory overcommit
- docs/future-tasks.md convention (CLAUDE.md, README) with the namespace-per-cluster task
- AGENTS.md; allowlist entries in .claude/settings.json

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/sessions/2026-09-28-1007-capmox-memory-overcommit.md

@@ -0,0 +1,64 @@
+# Session: CAPMOX memory accounting and overcommit on pv1-1
+
+**Created:** 2026-09-28 10:07 CEST
+
+## Question
+
+> can we configure capmox so that it works with real available memory instead of calculation what
+> stopped vms would use if somebody would start them? also can we configure memory overprovisioning
+> on proxmox node?
+
+Context: during Step 7, new workers failed with `cannot reserve …B of memory on node pv1-1`. The
+host had about 12.8 GiB actually available, but CAPMOX counted 168 MB. Later this deadlocked the
+2 GiB worker rollout (see the phase-1 troubleshooting table).
+
+## What was checked (2026-09-28)
+
+- CAPMOX source for v0.9.1 (the latest tag) and `main`:
+  - `pkg/proxmox/goproxmox/api_client.go` `GetReservableMemoryBytes`;
+  - `internal/service/scheduler/vmscheduler.go`;
+  - `api/v1alpha2/proxmoxcluster_types.go` `SchedulerHints`;
+  - `api/v1alpha2/proxmoxmachine_types.go`;
+  - `internal/webhook/proxmoxcluster_webhook.go`.
+- `pv1-1` (read-only):
+  - `qm list`, node memory status;
+  - KSM: `ksmtuned` active, `/sys/kernel/mm/ksm/run` = 0, `pages_sharing` = 0;
+  - 8 GiB swap;
+  - VM 103 has no `balloon:` line.
+
+## Findings
+
+- **No "real available memory" mode in any CAPMOX release.** `main` is identical to v0.9.1. The
+  reservable memory is `node total × memoryAdjustment/100` minus the configured `maxmem` of every
+  non-template VM (running or stopped) and every container. That's deliberate: real usage is
+  racy, because VMs grow into their configured memory later. Using free memory would need an
+  upstream change.
+- **The only knob is `ProxmoxCluster.spec.schedulerHints.memoryAdjustment`** (percent, default 100):
+  - `> 100` allows bounded overcommit;
+  - `0` turns the memory check off completely (it then only checks request ≤ host total).
+  - The ProxmoxCluster update webhook only validates the control-plane endpoint, so the hint can
+    be changed on a live cluster. The scheduler reads it whenever it places a VM, and the stuck
+    ProxmoxMachine retries.
+- **Sizing for `lab1`:** everything except the workers is 21 GiB configured (VMs 100, 102, 201,
+  1001, 2001, 160549). 5 workers × 2 GiB brings that to 31 GiB, and the host has 29.16 GiB, so
+  `memoryAdjustment ≥ 107`. At 120 there's 35 GiB reservable, which fits max 5 workers with about
+  4 GiB to spare.
+- **Proxmox itself enforces no memory overcommit limit.** VMs can be started past physical RAM; CAPMOX
+  was the only admission control. On the PVE side, "overprovisioning" is about how the host copes:
+  - **KSM:** `ksmtuned` is active but only merges below 20% free (`KSM_THRES_COEF`). Identical
+    Ubuntu clones deduplicate well. It's currently idle because the host has about 44% free.
+  - **Host swap (8 GiB):** a safety net, but slow.
+  - **Ballooning** (`balloon: <min>` below `memory`, auto-balloon above 80% host use): not
+    exposed by CAPMOX v0.9.1 (`ProxmoxMachineSpec` has no balloon field). It's also a poor fit
+    for Kubernetes nodes: the guest keeps reporting full capacity while the balloon takes memory
+    away, so the kubelet schedules against memory that isn't there.
+
+## Open questions
+
+- ~~120 or 0?~~ The owner chose **160** (decision #17): 46.7 GiB reservable, about 17.7 GiB headroom with today's VMs.
+- Is an upstream feature request worth filing (a free-memory mode, or excluding stopped VMs)?
+
+## Follow-ups
+
+- Done: `schedulerHints.memoryAdjustment: 160` on `ProxmoxCluster/lab1` in `configs/capi/lab1.yaml`,
+  decisions #17. Watch real host memory and KSM `pages_sharing` during scale tests.