kindnet's NetworkPolicy enforcement isn't guaranteed, undermining the Task 1 lab; Cilium (with kube-proxy replacement) enforces it deterministically. Also fixes ingress-nginx, which was silently broken: missing ingress-ready node label left the controller Pending forever, and hostPorts 80/443 were never mapped in kind-config. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
16 KiB
CKS Mock — Solutions (k8s v1.35)
Reference answers. There's usually more than one valid path; these are the exam-fast ones. Control-plane file edits assume docker exec -it cks-control-plane bash (or cks-worker for node tasks).
Before ANY apiserver edit:
cp /etc/kubernetes/manifests/kube-apiserver.yaml ~/kube-apiserver.yaml.bakinside the node. If the pod won't come back,cpit back.
Task 1 — NetworkPolicy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: default-deny, namespace: prod}
spec:
podSelector: {}
policyTypes: [Ingress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: allow-frontend, namespace: prod}
spec:
podSelector: {matchLabels: {app: db}}
policyTypes: [Ingress]
ingress:
- from:
- podSelector: {matchLabels: {app: frontend}}
ports:
- {protocol: TCP, port: 5432}
Verify:
kubectl -n prod exec deploy/frontend -- curl -s --max-time 3 db:5432 # -> db-ok
kubectl -n prod exec deploy/attacker -- curl -s --max-time 3 db:5432 # -> timeout
Gotcha: this cluster runs Cilium (not kindnetd), which enforces NetworkPolicy deterministically — no "if your build doesn't enforce it" caveat. If the attacker curl doesn't actually time out, don't blame the CNI; debug it:
cilium status # agents healthy on all 3 nodes?
kubectl -n kube-system exec ds/cilium -- cilium-dbg endpoint list # policy enforcement column for the db/attacker/frontend endpoints
Task 2 — Ingress TLS
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout tls.key -out tls.crt -subj "/CN=hello.cks.local/O=cks"
kubectl -n web create secret tls hello-tls --cert=tls.crt --key=tls.key
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello
namespace: web
spec:
ingressClassName: nginx
tls:
- hosts: [hello.cks.local]
secretName: hello-tls
rules:
- host: hello.cks.local
http:
paths:
- path: /
pathType: Prefix
backend: {service: {name: hello, port: {number: 80}}}
Verify:
curl -k --resolve hello.cks.local:443:127.0.0.1 https://hello.cks.local/
Task 3 — kube-bench remediation
Run it (Job is easiest inside kind):
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml
kubectl logs -f job/kube-bench | less
Or download the binary and kube-bench run --targets master,node on the node.
Kubelet — edit on the node /var/lib/kubelet/config.yaml:
authentication:
anonymous:
enabled: false
webhook:
enabled: true
authorization:
mode: Webhook
readOnlyPort: 0
systemctl restart kubelet
API server — /etc/kubernetes/manifests/kube-apiserver.yaml, add to command:
- --profiling=false
Re-run kube-bench; those checks flip to PASS.
Task 4 — RBAC least privilege
Replace the Role (binding already points at ci-runner-role):
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: {name: ci-runner-role, namespace: dev}
rules:
- apiGroups: [""]
resources: ["pods", "configmaps"]
verbs: ["get", "list", "watch"]
kubectl -n dev replace -f role.yaml # or edit
kubectl auth can-i get pods --as=system:serviceaccount:dev:ci-runner -n dev # yes
kubectl auth can-i delete pods --as=system:serviceaccount:dev:ci-runner -n dev # no
kubectl auth can-i '*' secrets --as=system:serviceaccount:dev:ci-runner -n dev # no
Task 5 — API server hardening
/etc/kubernetes/manifests/kube-apiserver.yaml, in command:
- --anonymous-auth=false
- --profiling=false
- --enable-admission-plugins=NodeRestriction # merge with existing list, comma-separated
If --enable-admission-plugins already exists, append ,NodeRestriction to it — don't add a second flag.
Wait for restart:
kubectl -n kube-system get pod -l component=kube-apiserver -w
Note: with --anonymous-auth=false, /healthz and /livez still work for the kubelet because it authenticates; unauthenticated curl gets 401.
Task 6 — automountServiceAccountToken
kubectl -n apps patch sa web-sa -p '{"automountServiceAccountToken": false}'
kubectl -n apps rollout restart deploy/web
kubectl -n apps exec deploy/web -- ls /var/run/secrets/kubernetes.io/serviceaccount 2>&1 # No such file
(Alternatively set automountServiceAccountToken: false in the pod template — SA-level is cleaner here and is what the task asks.)
Task 7 — AppArmor (GA securityContext API)
Load profile on the worker:
docker exec cks-worker apparmor_parser -q /root/deny-write
docker exec cks-worker aa-status | grep k8s-deny-write # enforce
Pod — use the appArmorProfile field, not the beta annotation:
apiVersion: v1
kind: Pod
metadata: {name: locked, namespace: sysh}
spec:
nodeName: cks-worker
securityContext:
appArmorProfile:
type: Localhost
localhostProfile: k8s-deny-write
containers:
- name: c
image: busybox:1.36
command: ["sh","-c","sleep 1h"]
kubectl -n sysh exec locked -- sh -c 'echo x > /tmp/x' # Permission denied
Gotcha: the profile only exists on cks-worker, so you must pin the pod there or the load must be on every schedulable node.
Task 8 — seccomp
apiVersion: v1
kind: Pod
metadata: {name: traced, namespace: sysh}
spec:
nodeName: cks-worker
securityContext:
seccompProfile:
type: Localhost
localhostProfile: profiles/audit.json
containers:
- {name: c, image: busybox:1.36, command: ["sh","-c","sleep 1h"]}
---
apiVersion: v1
kind: Pod
metadata: {name: defaulted, namespace: sysh}
spec:
nodeName: cks-worker
securityContext:
seccompProfile: {type: RuntimeDefault}
containers:
- {name: c, image: busybox:1.36, command: ["sh","-c","sleep 1h"]}
localhostProfile is relative to the kubelet seccomp root (/var/lib/kubelet/seccomp), so profiles/audit.json resolves to /var/lib/kubelet/seccomp/profiles/audit.json.
Task 9 — Pod Security Admission (restricted)
Label the namespace:
kubectl label ns restricted-ns \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/enforce-version=latest \
pod-security.kubernetes.io/warn=restricted \
pod-security.kubernetes.io/audit=restricted --overwrite
Fix the Deployment pod template to satisfy restricted:
spec:
securityContext:
runAsNonRoot: true
seccompProfile: {type: RuntimeDefault}
containers:
- name: c
image: nginx:1.27
securityContext:
allowPrivilegeEscalation: false
privileged: false
runAsNonRoot: true
capabilities: {drop: ["ALL"]}
nginx:1.27 wants to bind :80 — under runAsNonRoot it'll fail unless you use an unprivileged image/port. For the exam-grade "complies + rolls out," use nginxinc/nginx-unprivileged:1.27 (listens on 8080) or set runAsUser: 101. The graded bit is restricted-compliance; the unprivileged image makes it actually run.
Task 10 — Encryption at rest
Key + config on the control-plane:
mkdir -p /etc/kubernetes/enc
head -c 32 /dev/urandom | base64 # copy the value
cat > /etc/kubernetes/enc/enc.yaml <<EOF
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources: ["secrets"]
providers:
- aescbc:
keys:
- name: key1
secret: <PASTE_BASE64_32B_KEY>
- identity: {}
EOF
apiserver manifest — add flag + volume + mount:
# command:
- --encryption-provider-config=/etc/kubernetes/enc/enc.yaml
# volumes:
- name: enc
hostPath: {path: /etc/kubernetes/enc, type: DirectoryOrCreate}
# volumeMounts:
- name: enc
mountPath: /etc/kubernetes/enc
readOnly: true
After apiserver is back, rewrite all secrets:
kubectl get secrets -A -o json | kubectl replace -f -
Verify with etcdctl inside the control-plane:
ETCDCTL_API=3 etcdctl \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
get /registry/secrets/vault-ns/pre-existing | hexdump -C | head
# expect: k8s:enc:aescbc:v1:key1:... and NO plaintext SUPERSECRET
Task 11 — RuntimeClass
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata: {name: gvisor}
handler: runsc
---
apiVersion: v1
kind: Pod
metadata: {name: sandboxed, namespace: runtime}
spec:
runtimeClassName: gvisor
containers:
- {name: c, image: nginx:1.27}
Stays Pending in kind (no runsc handler in containerd) — expected. Config is the deliverable.
Task 12 — ValidatingAdmissionPolicy
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata: {name: pod-hardening}
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE","UPDATE"]
resources: ["pods"]
validations:
- expression: >-
!object.spec.containers.exists(c,
has(c.securityContext) && has(c.securityContext.privileged)
&& c.securityContext.privileged == true)
message: "privileged containers are not allowed"
- expression: >-
!has(object.spec.volumes) ||
!object.spec.volumes.exists(v, has(v.hostPath))
message: "hostPath volumes are not allowed"
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata: {name: pod-hardening-binding}
spec:
policyName: pod-hardening
validationActions: ["Deny"]
matchResources: {} # all namespaces
Also check initContainers/ephemeralContainers if you want it airtight; for the task, containers is what's graded. Test:
kubectl run bad --image=nginx:1.27 --privileged # denied
kubectl run ok --image=nginx:1.27 # admitted
Task 13 — Trivy scan & evict
for img in nginx:1.19.0 debian:10 nginx:1.27; do
echo "== $img =="
trivy image --severity HIGH,CRITICAL --quiet --scanners vuln "$img" | tail -5
done
nginx:1.19.0 and debian:10 will show HIGH/CRITICAL; delete those pods:
kubectl -n images delete pod legacy-app old-debian
kubectl -n images get pods # clean-app remains
Exam tip: map image→pod first (kubectl -n images get pods -o custom-columns=POD:.metadata.name,IMG:.spec.containers[*].image), scan each, delete by HIGH/CRITICAL count > 0.
Task 14 — Kyverno policies
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: {name: disallow-latest-tag}
spec:
validationFailureAction: Enforce
background: false
rules:
- name: require-explicit-tag
match: {any: [{resources: {kinds: [Pod]}}]}
validate:
message: "images must not use :latest or an empty tag"
pattern:
spec:
containers:
- image: "!*:latest"
- name: require-tag-present
match: {any: [{resources: {kinds: [Pod]}}]}
validate:
message: "image tag is required"
pattern:
spec:
containers:
- image: "*:*"
---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: {name: allowed-registries}
spec:
validationFailureAction: Enforce
background: false
rules:
- name: only-cks-registry
match: {any: [{resources: {kinds: [Pod]}}]}
validate:
message: "images must come from registry.cks.local"
pattern:
spec:
containers:
- image: "registry.cks.local/*"
Test:
kubectl -n team-a run bad --image=nginx:latest # denied (latest)
kubectl -n team-a run bad2 --image=docker.io/nginx:1.27 # denied (registry)
kubectl -n team-a run good --image=registry.cks.local/nginx:1.27 # admitted
Note: recent Kyverno also exposes the newer validate.foreach/CEL syntax; the pattern form above is the fastest to write under time.
Task 15 — ImagePolicyWebhook
/etc/kubernetes/admission/admission-config.yaml:
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
configuration:
imagePolicy:
kubeConfigFile: /etc/kubernetes/admission/imagepolicy-kubeconfig.yaml
allowTTL: 50
denyTTL: 50
retryBackoff: 500
defaultAllow: false
/etc/kubernetes/admission/imagepolicy-kubeconfig.yaml:
apiVersion: v1
kind: Config
clusters:
- name: image-checker
cluster:
server: https://image-checker.local/check # no live backend needed for the drill
users:
- name: apiserver
contexts:
- name: default
context: {cluster: image-checker, user: apiserver}
current-context: default
apiserver manifest:
# command:
- --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook # merge with existing
- --admission-control-config-file=/etc/kubernetes/admission/admission-config.yaml
# volume + mount for /etc/kubernetes/admission (readOnly)
With defaultAllow: false and no reachable backend, kubectl run t --image=nginx:1.27 → rejected by ImagePolicyWebhook. Flip to defaultAllow: true afterwards if you want the cluster usable.
Task 16 — Falco custom rule
/etc/falco/falco_rules.local.yaml:
- rule: Shell in container
desc: Detect a shell spawned inside a container
condition: >
spawned_process and container
and proc.name in (sh, bash)
output: >
Shell in container (container_id=%container.id container_name=%container.name
proc=%proc.name user=%user.name)
priority: WARNING
tags: [container, shell, mitre_execution]
systemctl restart falco # or: falco -r /etc/falco/falco_rules.yaml -r /etc/falco/falco_rules.local.yaml
# trigger:
kubectl run trigger --image=busybox:1.36 -- sh -c 'sleep 1h'
kubectl exec -it trigger -- sh
# capture:
journalctl -u falco | grep "Shell in container" | tail -1 > /opt/course/falco-hits.txt
Gotchas: container macro excludes host processes; spawned_process = execve. If running Falco via systemd, alerts land in journalctl -u falco (and/or /var/log/syslog) — grab from whichever your output channel is. Check the rule loaded: falco --list | grep -i "Shell in container" or watch startup logs for parse errors.
Task 17 — Audit logging
/etc/kubernetes/audit/policy.yaml:
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages: ["RequestReceived"]
rules:
- level: RequestResponse
resources:
- group: ""
resources: ["pods"]
verbs: ["create","update","patch","delete"]
- level: Metadata
resources:
- group: ""
resources: ["secrets","configmaps"]
- level: None
apiserver manifest — flags:
- --audit-policy-file=/etc/kubernetes/audit/policy.yaml
- --audit-log-path=/var/log/kubernetes/audit/audit.log
- --audit-log-maxage=7
- --audit-log-maxbackup=2
- --audit-log-maxsize=50
volumes + mounts:
# volumes:
- name: audit-policy
hostPath: {path: /etc/kubernetes/audit, type: DirectoryOrCreate}
- name: audit-logs
hostPath: {path: /var/log/kubernetes/audit, type: DirectoryOrCreate}
# volumeMounts:
- name: audit-policy
mountPath: /etc/kubernetes/audit
readOnly: true
- name: audit-logs
mountPath: /var/log/kubernetes/audit
readOnly: false
Verify inside the control-plane:
kubectl -n vault-ns get secret pre-existing
tail -f /var/log/kubernetes/audit/audit.log | grep pre-existing # Metadata level, no body
Big gotcha: the audit-logs mount must be readOnly: false or the apiserver crashloops silently. And the log dir hostPath is inside the control-plane container — that's fine here.
Reset between attempts
kind delete cluster --name cks
./bootstrap.sh && ./seed.sh