lab1 workers on both PVE nodes: per-node template + MD, balanced CA
PVE is now the 2-node cluster homelab (pv1-1 + pv1-2), no shared storage: - template 101 copied to pv1-2 as 9101; MDs lab1-workers-pv1-1 (max 6) and lab1-workers-pv1-2 (max 12), PMTs pinned with allowedNodes; pool .31-.50; lab1-workers renamed for symmetry (decision #18, #7 superseded) - CA: balancing-label node.cluster.x-k8s.io/pool (+ pool label, capacity labels with os/arch, maxPods on the MDs), scan-interval 1m, v 2, default QPS (#19). Scale test: 16 pods -> 4 + 4 across both nodes. - runbook, image-builder copy procedure, fixes tables, future task to report three CA clusterapi bugs upstream, lab facts, changelog, execution log Plan: docs/plans/2026-09-29-2315-pv1-2-workers.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -18,7 +18,8 @@ rbac:
|
||||
verbs: [get, list, watch]
|
||||
extraArgs:
|
||||
expander: least-waste
|
||||
# Aggressive lab scale-down profile (CA 1.35 defaults in brackets): idle worker gone in ~3 min.
|
||||
# Aggressive lab scale-down profile (CA 1.35 defaults in brackets): idle worker gone in ~3-4 min
|
||||
# (scan-interval 1m adds up to a minute to the 1m unneeded time).
|
||||
# Keep delay-after-add >= node startup (~2-3 min here) or bursts flap up/down.
|
||||
scale-down-unneeded-time: 1m # [10m]
|
||||
scale-down-delay-after-add: 2m # [10m] cluster-wide cooldown after any scale-up
|
||||
@@ -27,7 +28,15 @@ extraArgs:
|
||||
scale-down-utilization-threshold: "0.6" # [0.5]
|
||||
max-graceful-termination-sec: "60" # [600] drain wait per pod
|
||||
balance-similar-node-groups: true
|
||||
# One worker MD per PVE node. Compare groups only on this label: the generic comparator fails when
|
||||
# one group has a real-node template (cached, 2Gi VM = ~1.78Gi allocatable, os/arch set) and the
|
||||
# other a from-zero one. Disables the resource checks, so keep the MDs' capacity identical.
|
||||
balancing-label: node.cluster.x-k8s.io/pool
|
||||
# Fewer API requests instead of a higher kube-client-qps: every MD at 0 replicas costs ~3 uncached
|
||||
# discovery GETs per NodeGroups() call (~15x per loop). Unschedulable pods and scale-ups/downs still
|
||||
# trigger a loop immediately; only idle loops (scale-down evaluation) are spaced out.
|
||||
scan-interval: 1m # [10s]
|
||||
# Cilium taints new nodes NoSchedule until its agent is up. Without this, CA simulates the freshly
|
||||
# registered (still tainted) node, finds the pending pods don't fit and adds one node too many.
|
||||
startup-taint: node.cilium.io/agent-not-ready
|
||||
v: 4
|
||||
v: 2 # [chart 4] 4 logs every throttled request and MD lookup
|
||||
|
||||
Reference in New Issue
Block a user