cks: swap kind CNI to Cilium, fix broken ingress-nginx setup

kindnet's NetworkPolicy enforcement isn't guaranteed, undermining the
Task 1 lab; Cilium (with kube-proxy replacement) enforces it
deterministically. Also fixes ingress-nginx, which was silently broken:
missing ingress-ready node label left the controller Pending forever,
and hostPorts 80/443 were never mapped in kind-config.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

kubernetes/cks/exam-by-claude/README.md

@@ -3,8 +3,8 @@
 Timed practice that mirrors the current CKS (17 tasks, 2h, six domains). Runs on kind (k8s v1.35.5) with the host-side / kernel bits on your Ubuntu VM.
 
 ## Prereqs (Ubuntu VM)
-- docker, `kind >= v0.32.0`, `kubectl` v1.35, `helm`
-- host tools installed latest: `kube-bench`, `trivy`, `kubesec`, `falco` (v0.44.x), optional `cosign`
+- docker, `kind >= v0.32.0`, `kubectl` v1.35, `helm` (used to install Cilium — load-bearing, not optional)
+- host tools installed latest: `kube-bench`, `trivy`, `kubesec`, `falco` (v0.44.x), optional `cosign`, optional `cilium-cli` for `cilium status`/`cilium connectivity test`
 - host must have AppArmor enabled (default on Ubuntu) for Task 7
 
 ## Quickstart
@@ -32,6 +32,9 @@ kind delete cluster --name cks && ./bootstrap.sh && ./seed.sh
 
 ## Notes / kind caveats
 - **Version:** exam is on v1.35 per the Linux Foundation page. k8s 1.36 shipped ~May 2026; the exam env aligns "within 4–8 weeks" of a release, so it *may* have rolled to 1.36 by your date — verify on the LF exam page. Nothing in this set changes between 1.35/1.36. To bump: swap the digest in `kind-config.yaml` for a `kindest/node:v1.36.x` one.
+- **CNI is Cilium**, not kindnet — `disableDefaultCNI: true` in `kind-config.yaml`, installed by `bootstrap.sh` via Helm. It runs with the **kube-proxy replacement** (`kubeProxyMode: none`), so there is no `kube-proxy` DaemonSet in `kube-system`; kube-bench's kube-proxy CIS checks are N/A (Task 3's four target checks — kubelet ×3, apiserver ×1 — are unaffected). This also means Task 1's NetworkPolicy is genuinely enforced, not a maybe.
+- Between `kind create cluster` and the Cilium install finishing, nodes sit **NotReady** and CoreDNS is **Pending** — expected, not a failure; `bootstrap.sh` waits it out.
+- Ingress reaches the cluster on host ports **80/443** (mapped in `kind-config.yaml`) via the control-plane node, which is labelled `ingress-ready=true` so the ingress-nginx kind manifest schedules onto it.
 - **AppArmor** needs a real AppArmor host; works on Ubuntu, not on Docker Desktop/macOS.
 - **gVisor (Task 11)** pod stays Pending in kind (no `runsc` in kind's containerd) — the config is the graded artifact.
 - **ImagePolicyWebhook / audit / encryption** edit the live apiserver: always `cp` the manifest first so you can revert fast.

kubernetes/cks/exam-by-claude/bootstrap.sh

@@ -1,32 +1,60 @@
 #!/usr/bin/env bash
 # CKS mock — cluster bootstrap. Run on your Ubuntu VM (needs docker, kind >= v0.32.0, kubectl v1.35, helm).
+# CNI is Cilium (kube-proxy replacement, no kindnet, no kube-proxy) — see kind-config.yaml's networking: block.
 set -euo pipefail
 
 HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 cd "$HERE"
 
+# ---- pinned versions ----
+CILIUM_VERSION="1.20.0"
+INGRESS_NGINX_REF="controller-v1.15.1"
+
 mkdir -p exam-files
 
 echo "[*] Creating kind cluster 'cks' (1 cp + 2 workers, k8s v1.35.5)..."
+# No --wait here: with disableDefaultCNI, nodes are legitimately NotReady (and CoreDNS Pending)
+# until Cilium is installed below — waiting now would just time out.
 kind create cluster --config kind-config.yaml
 kubectl config use-context kind-cks
 
+echo "[*] Installing Cilium v${CILIUM_VERSION} (CNI + kube-proxy replacement)..."
+CP_IP="$(docker inspect -f '{{.NetworkSettings.Networks.kind.IPAddress}}' cks-control-plane)"
+helm repo add cilium https://helm.cilium.io/ >/dev/null
+helm repo update cilium >/dev/null
+helm upgrade --install cilium cilium/cilium \
+  --version "$CILIUM_VERSION" --namespace kube-system \
+  --set ipam.mode=kubernetes \
+  --set kubeProxyReplacement=true \
+  --set k8sServiceHost="$CP_IP" --set k8sServicePort=6443 \
+  --set operator.replicas=1 \
+  --set image.pullPolicy=IfNotPresent \
+  --wait --timeout=5m
+
+echo "[*] Waiting for Cilium and nodes to be Ready..."
+kubectl -n kube-system rollout status ds/cilium --timeout=300s
+kubectl wait --for=condition=Ready nodes --all --timeout=300s
+command -v cilium >/dev/null 2>&1 && cilium status --wait || true
+
 echo "[*] Installing an ingress controller (needed for the TLS task)..."
-kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
-kubectl -n ingress-nginx wait --for=condition=Ready pod \
-  -l app.kubernetes.io/component=controller --timeout=180s || true
+kubectl label node cks-control-plane ingress-ready=true --overwrite
+kubectl apply -f "https://raw.githubusercontent.com/kubernetes/ingress-nginx/${INGRESS_NGINX_REF}/deploy/static/provider/kind/deploy.yaml"
+kubectl -n ingress-nginx rollout status deploy/ingress-nginx-controller --timeout=300s
 
 cat <<'EOF'
 
 [*] Cluster up. Sanity check:
     kubectl get nodes -o wide
+    cilium status
+    kubectl -n kube-system get ds kube-proxy   # expected: not found — Cilium replaces it
 
 [*] Host-side tooling you want on the VM (install latest):
     - kube-bench   : https://github.com/aquasecurity/kube-bench/releases  (or run as a Job in-cluster)
     - trivy        : https://github.com/aquasecurity/trivy   (apt: aquasecurity repo)
     - kubesec      : https://github.com/controlplaneio/kubesec/releases
     - falco        : apt install falco   (v0.44.1) — used in the runtime task on the VM
-    - helm         : for Kyverno / Falco chart installs
+    - helm         : for Kyverno / Falco chart installs / Cilium (already used above)
+    - cilium-cli   : optional, https://github.com/cilium/cilium-cli/releases — `cilium status`/`cilium connectivity test`
     - cosign       : optional, image signing task
 
 [*] Now run ./seed.sh to plant the vulnerable/target objects for the kubectl-only tasks.

kubernetes/cks/exam-by-claude/exam.md

@@ -43,7 +43,7 @@ Expose the `hello` Service in namespace `web` over HTTPS.
 1. Create a TLS Secret `hello-tls` in `web` from a self-signed cert for host `hello.cks.local` (you generate the cert/key on the VM).
 2. Create an Ingress `hello` in `web` routing host `hello.cks.local` path `/` to Service `hello:80`, using `hello-tls` for TLS.
 
-> Verify: `curl -k --resolve hello.cks.local:443:127.0.0.1 https://hello.cks.local/` returns `hello-tls` (adjust port to the ingress-nginx NodePort/hostPort in your setup).
+> Verify: `curl -k --resolve hello.cks.local:443:127.0.0.1 https://hello.cks.local/` returns `hello-tls`.
 
 ---
 

kubernetes/cks/exam-by-claude/kind-config.yaml

@@ -1,9 +1,16 @@
 # CKS mock cluster — 1 control-plane + 2 workers, k8s v1.35
 # Pinned digest from kind v0.32.0 (default 1.36) — we force 1.35.5 to match the exam.
 # If `kind load`/image pull complains, use kind >= v0.32.0.
+#
+# CNI is Cilium (installed by bootstrap.sh), not kindnet — see networking: below.
 kind: Cluster
 apiVersion: kind.x-k8s.io/v1alpha4
 name: cks
+networking:
+  disableDefaultCNI: true     # no kindnet — Cilium takes over, and it enforces NetworkPolicy deterministically
+  kubeProxyMode: none         # Cilium eBPF kube-proxy replacement (no kube-proxy DaemonSet)
+  podSubnet: "10.244.0.0/16"  # Cilium runs with ipam.mode=kubernetes, honoring this
+  serviceSubnet: "10.96.0.0/16"
 nodes:
   - role: control-plane
     image: kindest/node:v1.35.5@sha256:ce977ae6d65918d0b58a5f8b5e940429c2ce42fa3a5619ec2bbc60b949c0ac95
@@ -12,7 +19,14 @@ nodes:
       - hostPath: ./exam-files
         containerPath: /etc/kubernetes/exam
     extraPortMappings:
-      # so you can curl the Ingress task from the VM
+      # ingress-nginx's kind provider manifest hostPorts — Task 2 (TLS Ingress) needs 80/443 reachable
+      - containerPort: 80
+        hostPort: 80
+        protocol: TCP
+      - containerPort: 443
+        hostPort: 443
+        protocol: TCP
+      # so you can curl the Ingress task from the VM via NodePort too, if you set one up
       - containerPort: 30443
         hostPort: 30443
         protocol: TCP

kubernetes/cks/exam-by-claude/solutions.md

@@ -34,7 +34,11 @@ Verify:
 kubectl -n prod exec deploy/frontend -- curl -s --max-time 3 db:5432   # -> db-ok
 kubectl -n prod exec deploy/attacker -- curl -s --max-time 3 db:5432   # -> timeout
 ```
-Gotcha: kind's default CNI (kindnetd) enforces NetworkPolicy in recent versions; if your kindnet build doesn't, swap to Calico. On the exam CNI always enforces.
+Gotcha: this cluster runs Cilium (not kindnetd), which enforces NetworkPolicy deterministically — no "if your build doesn't enforce it" caveat. If the attacker curl doesn't actually time out, don't blame the CNI; debug it:
+```bash
+cilium status                                              # agents healthy on all 3 nodes?
+kubectl -n kube-system exec ds/cilium -- cilium-dbg endpoint list   # policy enforcement column for the db/attacker/frontend endpoints
+```
 
 ---
 
@@ -64,9 +68,8 @@ spec:
             pathType: Prefix
             backend: {service: {name: hello, port: {number: 80}}}
 ```
-Verify (port depends on how ingress-nginx is exposed in kind — NodePort or the hostPort mapping):
+Verify:
 ```bash
-kubectl -n ingress-nginx get svc ingress-nginx-controller
 curl -k --resolve hello.cks.local:443:127.0.0.1 https://hello.cks.local/
 ```