docs: add Kubernetes CKS study notes
This commit is contained in:
545
kubernetes/cks/exam-by-claude/solutions.md
Normal file
545
kubernetes/cks/exam-by-claude/solutions.md
Normal file
@@ -0,0 +1,545 @@
|
||||
# CKS Mock — Solutions (k8s v1.35)
|
||||
|
||||
Reference answers. There's usually more than one valid path; these are the exam-fast ones. Control-plane file edits assume `docker exec -it cks-control-plane bash` (or `cks-worker` for node tasks).
|
||||
|
||||
> **Before ANY apiserver edit:** `cp /etc/kubernetes/manifests/kube-apiserver.yaml ~/kube-apiserver.yaml.bak` inside the node. If the pod won't come back, `cp` it back.
|
||||
|
||||
---
|
||||
|
||||
## Task 1 — NetworkPolicy
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata: {name: default-deny, namespace: prod}
|
||||
spec:
|
||||
podSelector: {}
|
||||
policyTypes: [Ingress]
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata: {name: allow-frontend, namespace: prod}
|
||||
spec:
|
||||
podSelector: {matchLabels: {app: db}}
|
||||
policyTypes: [Ingress]
|
||||
ingress:
|
||||
- from:
|
||||
- podSelector: {matchLabels: {app: frontend}}
|
||||
ports:
|
||||
- {protocol: TCP, port: 5432}
|
||||
```
|
||||
|
||||
Verify:
|
||||
```bash
|
||||
kubectl -n prod exec deploy/frontend -- curl -s --max-time 3 db:5432 # -> db-ok
|
||||
kubectl -n prod exec deploy/attacker -- curl -s --max-time 3 db:5432 # -> timeout
|
||||
```
|
||||
Gotcha: kind's default CNI (kindnetd) enforces NetworkPolicy in recent versions; if your kindnet build doesn't, swap to Calico. On the exam CNI always enforces.
|
||||
|
||||
---
|
||||
|
||||
## Task 2 — Ingress TLS
|
||||
|
||||
```bash
|
||||
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
|
||||
-keyout tls.key -out tls.crt -subj "/CN=hello.cks.local/O=cks"
|
||||
kubectl -n web create secret tls hello-tls --cert=tls.crt --key=tls.key
|
||||
```
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: hello
|
||||
namespace: web
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts: [hello.cks.local]
|
||||
secretName: hello-tls
|
||||
rules:
|
||||
- host: hello.cks.local
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend: {service: {name: hello, port: {number: 80}}}
|
||||
```
|
||||
Verify (port depends on how ingress-nginx is exposed in kind — NodePort or the hostPort mapping):
|
||||
```bash
|
||||
kubectl -n ingress-nginx get svc ingress-nginx-controller
|
||||
curl -k --resolve hello.cks.local:443:127.0.0.1 https://hello.cks.local/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3 — kube-bench remediation
|
||||
|
||||
Run it (Job is easiest inside kind):
|
||||
```bash
|
||||
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml
|
||||
kubectl logs -f job/kube-bench | less
|
||||
```
|
||||
Or download the binary and `kube-bench run --targets master,node` on the node.
|
||||
|
||||
Kubelet — edit on the node `/var/lib/kubelet/config.yaml`:
|
||||
```yaml
|
||||
authentication:
|
||||
anonymous:
|
||||
enabled: false
|
||||
webhook:
|
||||
enabled: true
|
||||
authorization:
|
||||
mode: Webhook
|
||||
readOnlyPort: 0
|
||||
```
|
||||
```bash
|
||||
systemctl restart kubelet
|
||||
```
|
||||
API server — `/etc/kubernetes/manifests/kube-apiserver.yaml`, add to `command`:
|
||||
```
|
||||
- --profiling=false
|
||||
```
|
||||
Re-run kube-bench; those checks flip to PASS.
|
||||
|
||||
---
|
||||
|
||||
## Task 4 — RBAC least privilege
|
||||
|
||||
Replace the Role (binding already points at `ci-runner-role`):
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata: {name: ci-runner-role, namespace: dev}
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["pods", "configmaps"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
```
|
||||
```bash
|
||||
kubectl -n dev replace -f role.yaml # or edit
|
||||
kubectl auth can-i get pods --as=system:serviceaccount:dev:ci-runner -n dev # yes
|
||||
kubectl auth can-i delete pods --as=system:serviceaccount:dev:ci-runner -n dev # no
|
||||
kubectl auth can-i '*' secrets --as=system:serviceaccount:dev:ci-runner -n dev # no
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5 — API server hardening
|
||||
|
||||
`/etc/kubernetes/manifests/kube-apiserver.yaml`, in `command`:
|
||||
```
|
||||
- --anonymous-auth=false
|
||||
- --profiling=false
|
||||
- --enable-admission-plugins=NodeRestriction # merge with existing list, comma-separated
|
||||
```
|
||||
If `--enable-admission-plugins` already exists, append `,NodeRestriction` to it — don't add a second flag.
|
||||
Wait for restart:
|
||||
```bash
|
||||
kubectl -n kube-system get pod -l component=kube-apiserver -w
|
||||
```
|
||||
Note: with `--anonymous-auth=false`, `/healthz` and `/livez` still work for the kubelet because it authenticates; unauthenticated curl gets 401.
|
||||
|
||||
---
|
||||
|
||||
## Task 6 — automountServiceAccountToken
|
||||
|
||||
```bash
|
||||
kubectl -n apps patch sa web-sa -p '{"automountServiceAccountToken": false}'
|
||||
kubectl -n apps rollout restart deploy/web
|
||||
kubectl -n apps exec deploy/web -- ls /var/run/secrets/kubernetes.io/serviceaccount 2>&1 # No such file
|
||||
```
|
||||
(Alternatively set `automountServiceAccountToken: false` in the pod template — SA-level is cleaner here and is what the task asks.)
|
||||
|
||||
---
|
||||
|
||||
## Task 7 — AppArmor (GA securityContext API)
|
||||
|
||||
Load profile on the worker:
|
||||
```bash
|
||||
docker exec cks-worker apparmor_parser -q /root/deny-write
|
||||
docker exec cks-worker aa-status | grep k8s-deny-write # enforce
|
||||
```
|
||||
Pod — **use the appArmorProfile field, not the beta annotation**:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata: {name: locked, namespace: sysh}
|
||||
spec:
|
||||
nodeName: cks-worker
|
||||
securityContext:
|
||||
appArmorProfile:
|
||||
type: Localhost
|
||||
localhostProfile: k8s-deny-write
|
||||
containers:
|
||||
- name: c
|
||||
image: busybox:1.36
|
||||
command: ["sh","-c","sleep 1h"]
|
||||
```
|
||||
```bash
|
||||
kubectl -n sysh exec locked -- sh -c 'echo x > /tmp/x' # Permission denied
|
||||
```
|
||||
Gotcha: the profile only exists on `cks-worker`, so you must pin the pod there or the load must be on every schedulable node.
|
||||
|
||||
---
|
||||
|
||||
## Task 8 — seccomp
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata: {name: traced, namespace: sysh}
|
||||
spec:
|
||||
nodeName: cks-worker
|
||||
securityContext:
|
||||
seccompProfile:
|
||||
type: Localhost
|
||||
localhostProfile: profiles/audit.json
|
||||
containers:
|
||||
- {name: c, image: busybox:1.36, command: ["sh","-c","sleep 1h"]}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata: {name: defaulted, namespace: sysh}
|
||||
spec:
|
||||
nodeName: cks-worker
|
||||
securityContext:
|
||||
seccompProfile: {type: RuntimeDefault}
|
||||
containers:
|
||||
- {name: c, image: busybox:1.36, command: ["sh","-c","sleep 1h"]}
|
||||
```
|
||||
`localhostProfile` is relative to the kubelet seccomp root (`/var/lib/kubelet/seccomp`), so `profiles/audit.json` resolves to `/var/lib/kubelet/seccomp/profiles/audit.json`.
|
||||
|
||||
---
|
||||
|
||||
## Task 9 — Pod Security Admission (restricted)
|
||||
|
||||
Label the namespace:
|
||||
```bash
|
||||
kubectl label ns restricted-ns \
|
||||
pod-security.kubernetes.io/enforce=restricted \
|
||||
pod-security.kubernetes.io/enforce-version=latest \
|
||||
pod-security.kubernetes.io/warn=restricted \
|
||||
pod-security.kubernetes.io/audit=restricted --overwrite
|
||||
```
|
||||
Fix the Deployment pod template to satisfy `restricted`:
|
||||
```yaml
|
||||
spec:
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
seccompProfile: {type: RuntimeDefault}
|
||||
containers:
|
||||
- name: c
|
||||
image: nginx:1.27
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
privileged: false
|
||||
runAsNonRoot: true
|
||||
capabilities: {drop: ["ALL"]}
|
||||
```
|
||||
nginx:1.27 wants to bind :80 — under `runAsNonRoot` it'll fail unless you use an unprivileged image/port. For the exam-grade "complies + rolls out," use `nginxinc/nginx-unprivileged:1.27` (listens on 8080) or set `runAsUser: 101`. The graded bit is restricted-compliance; the unprivileged image makes it actually run.
|
||||
|
||||
---
|
||||
|
||||
## Task 10 — Encryption at rest
|
||||
|
||||
Key + config on the control-plane:
|
||||
```bash
|
||||
mkdir -p /etc/kubernetes/enc
|
||||
head -c 32 /dev/urandom | base64 # copy the value
|
||||
cat > /etc/kubernetes/enc/enc.yaml <<EOF
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: EncryptionConfiguration
|
||||
resources:
|
||||
- resources: ["secrets"]
|
||||
providers:
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: <PASTE_BASE64_32B_KEY>
|
||||
- identity: {}
|
||||
EOF
|
||||
```
|
||||
apiserver manifest — add flag + volume + mount:
|
||||
```yaml
|
||||
# command:
|
||||
- --encryption-provider-config=/etc/kubernetes/enc/enc.yaml
|
||||
# volumes:
|
||||
- name: enc
|
||||
hostPath: {path: /etc/kubernetes/enc, type: DirectoryOrCreate}
|
||||
# volumeMounts:
|
||||
- name: enc
|
||||
mountPath: /etc/kubernetes/enc
|
||||
readOnly: true
|
||||
```
|
||||
After apiserver is back, rewrite all secrets:
|
||||
```bash
|
||||
kubectl get secrets -A -o json | kubectl replace -f -
|
||||
```
|
||||
Verify with etcdctl inside the control-plane:
|
||||
```bash
|
||||
ETCDCTL_API=3 etcdctl \
|
||||
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
|
||||
--cert=/etc/kubernetes/pki/etcd/server.crt \
|
||||
--key=/etc/kubernetes/pki/etcd/server.key \
|
||||
get /registry/secrets/vault-ns/pre-existing | hexdump -C | head
|
||||
# expect: k8s:enc:aescbc:v1:key1:... and NO plaintext SUPERSECRET
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 11 — RuntimeClass
|
||||
|
||||
```yaml
|
||||
apiVersion: node.k8s.io/v1
|
||||
kind: RuntimeClass
|
||||
metadata: {name: gvisor}
|
||||
handler: runsc
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata: {name: sandboxed, namespace: runtime}
|
||||
spec:
|
||||
runtimeClassName: gvisor
|
||||
containers:
|
||||
- {name: c, image: nginx:1.27}
|
||||
```
|
||||
Stays Pending in kind (no `runsc` handler in containerd) — expected. Config is the deliverable.
|
||||
|
||||
---
|
||||
|
||||
## Task 12 — ValidatingAdmissionPolicy
|
||||
|
||||
```yaml
|
||||
apiVersion: admissionregistration.k8s.io/v1
|
||||
kind: ValidatingAdmissionPolicy
|
||||
metadata: {name: pod-hardening}
|
||||
spec:
|
||||
failurePolicy: Fail
|
||||
matchConstraints:
|
||||
resourceRules:
|
||||
- apiGroups: [""]
|
||||
apiVersions: ["v1"]
|
||||
operations: ["CREATE","UPDATE"]
|
||||
resources: ["pods"]
|
||||
validations:
|
||||
- expression: >-
|
||||
!object.spec.containers.exists(c,
|
||||
has(c.securityContext) && has(c.securityContext.privileged)
|
||||
&& c.securityContext.privileged == true)
|
||||
message: "privileged containers are not allowed"
|
||||
- expression: >-
|
||||
!has(object.spec.volumes) ||
|
||||
!object.spec.volumes.exists(v, has(v.hostPath))
|
||||
message: "hostPath volumes are not allowed"
|
||||
---
|
||||
apiVersion: admissionregistration.k8s.io/v1
|
||||
kind: ValidatingAdmissionPolicyBinding
|
||||
metadata: {name: pod-hardening-binding}
|
||||
spec:
|
||||
policyName: pod-hardening
|
||||
validationActions: ["Deny"]
|
||||
matchResources: {} # all namespaces
|
||||
```
|
||||
Also check `initContainers`/`ephemeralContainers` if you want it airtight; for the task, `containers` is what's graded. Test:
|
||||
```bash
|
||||
kubectl run bad --image=nginx:1.27 --privileged # denied
|
||||
kubectl run ok --image=nginx:1.27 # admitted
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 13 — Trivy scan & evict
|
||||
|
||||
```bash
|
||||
for img in nginx:1.19.0 debian:10 nginx:1.27; do
|
||||
echo "== $img =="
|
||||
trivy image --severity HIGH,CRITICAL --quiet --scanners vuln "$img" | tail -5
|
||||
done
|
||||
```
|
||||
`nginx:1.19.0` and `debian:10` will show HIGH/CRITICAL; delete those pods:
|
||||
```bash
|
||||
kubectl -n images delete pod legacy-app old-debian
|
||||
kubectl -n images get pods # clean-app remains
|
||||
```
|
||||
Exam tip: map image→pod first (`kubectl -n images get pods -o custom-columns=POD:.metadata.name,IMG:.spec.containers[*].image`), scan each, delete by HIGH/CRITICAL count > 0.
|
||||
|
||||
---
|
||||
|
||||
## Task 14 — Kyverno policies
|
||||
|
||||
```yaml
|
||||
apiVersion: kyverno.io/v1
|
||||
kind: ClusterPolicy
|
||||
metadata: {name: disallow-latest-tag}
|
||||
spec:
|
||||
validationFailureAction: Enforce
|
||||
background: false
|
||||
rules:
|
||||
- name: require-explicit-tag
|
||||
match: {any: [{resources: {kinds: [Pod]}}]}
|
||||
validate:
|
||||
message: "images must not use :latest or an empty tag"
|
||||
pattern:
|
||||
spec:
|
||||
containers:
|
||||
- image: "!*:latest"
|
||||
- name: require-tag-present
|
||||
match: {any: [{resources: {kinds: [Pod]}}]}
|
||||
validate:
|
||||
message: "image tag is required"
|
||||
pattern:
|
||||
spec:
|
||||
containers:
|
||||
- image: "*:*"
|
||||
---
|
||||
apiVersion: kyverno.io/v1
|
||||
kind: ClusterPolicy
|
||||
metadata: {name: allowed-registries}
|
||||
spec:
|
||||
validationFailureAction: Enforce
|
||||
background: false
|
||||
rules:
|
||||
- name: only-cks-registry
|
||||
match: {any: [{resources: {kinds: [Pod]}}]}
|
||||
validate:
|
||||
message: "images must come from registry.cks.local"
|
||||
pattern:
|
||||
spec:
|
||||
containers:
|
||||
- image: "registry.cks.local/*"
|
||||
```
|
||||
Test:
|
||||
```bash
|
||||
kubectl -n team-a run bad --image=nginx:latest # denied (latest)
|
||||
kubectl -n team-a run bad2 --image=docker.io/nginx:1.27 # denied (registry)
|
||||
kubectl -n team-a run good --image=registry.cks.local/nginx:1.27 # admitted
|
||||
```
|
||||
Note: recent Kyverno also exposes the newer `validate.foreach`/CEL syntax; the pattern form above is the fastest to write under time.
|
||||
|
||||
---
|
||||
|
||||
## Task 15 — ImagePolicyWebhook
|
||||
|
||||
`/etc/kubernetes/admission/admission-config.yaml`:
|
||||
```yaml
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: ImagePolicyWebhook
|
||||
configuration:
|
||||
imagePolicy:
|
||||
kubeConfigFile: /etc/kubernetes/admission/imagepolicy-kubeconfig.yaml
|
||||
allowTTL: 50
|
||||
denyTTL: 50
|
||||
retryBackoff: 500
|
||||
defaultAllow: false
|
||||
```
|
||||
`/etc/kubernetes/admission/imagepolicy-kubeconfig.yaml`:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Config
|
||||
clusters:
|
||||
- name: image-checker
|
||||
cluster:
|
||||
server: https://image-checker.local/check # no live backend needed for the drill
|
||||
users:
|
||||
- name: apiserver
|
||||
contexts:
|
||||
- name: default
|
||||
context: {cluster: image-checker, user: apiserver}
|
||||
current-context: default
|
||||
```
|
||||
apiserver manifest:
|
||||
```yaml
|
||||
# command:
|
||||
- --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook # merge with existing
|
||||
- --admission-control-config-file=/etc/kubernetes/admission/admission-config.yaml
|
||||
# volume + mount for /etc/kubernetes/admission (readOnly)
|
||||
```
|
||||
With `defaultAllow: false` and no reachable backend, `kubectl run t --image=nginx:1.27` → rejected by ImagePolicyWebhook. Flip to `defaultAllow: true` afterwards if you want the cluster usable.
|
||||
|
||||
---
|
||||
|
||||
## Task 16 — Falco custom rule
|
||||
|
||||
`/etc/falco/falco_rules.local.yaml`:
|
||||
```yaml
|
||||
- rule: Shell in container
|
||||
desc: Detect a shell spawned inside a container
|
||||
condition: >
|
||||
spawned_process and container
|
||||
and proc.name in (sh, bash)
|
||||
output: >
|
||||
Shell in container (container_id=%container.id container_name=%container.name
|
||||
proc=%proc.name user=%user.name)
|
||||
priority: WARNING
|
||||
tags: [container, shell, mitre_execution]
|
||||
```
|
||||
```bash
|
||||
systemctl restart falco # or: falco -r /etc/falco/falco_rules.yaml -r /etc/falco/falco_rules.local.yaml
|
||||
# trigger:
|
||||
kubectl run trigger --image=busybox:1.36 -- sh -c 'sleep 1h'
|
||||
kubectl exec -it trigger -- sh
|
||||
# capture:
|
||||
journalctl -u falco | grep "Shell in container" | tail -1 > /opt/course/falco-hits.txt
|
||||
```
|
||||
Gotchas: `container` macro excludes host processes; `spawned_process` = execve. If running Falco via systemd, alerts land in `journalctl -u falco` (and/or `/var/log/syslog`) — grab from whichever your output channel is. Check the rule loaded: `falco --list | grep -i "Shell in container"` or watch startup logs for parse errors.
|
||||
|
||||
---
|
||||
|
||||
## Task 17 — Audit logging
|
||||
|
||||
`/etc/kubernetes/audit/policy.yaml`:
|
||||
```yaml
|
||||
apiVersion: audit.k8s.io/v1
|
||||
kind: Policy
|
||||
omitStages: ["RequestReceived"]
|
||||
rules:
|
||||
- level: RequestResponse
|
||||
resources:
|
||||
- group: ""
|
||||
resources: ["pods"]
|
||||
verbs: ["create","update","patch","delete"]
|
||||
- level: Metadata
|
||||
resources:
|
||||
- group: ""
|
||||
resources: ["secrets","configmaps"]
|
||||
- level: None
|
||||
```
|
||||
apiserver manifest — flags:
|
||||
```yaml
|
||||
- --audit-policy-file=/etc/kubernetes/audit/policy.yaml
|
||||
- --audit-log-path=/var/log/kubernetes/audit/audit.log
|
||||
- --audit-log-maxage=7
|
||||
- --audit-log-maxbackup=2
|
||||
- --audit-log-maxsize=50
|
||||
```
|
||||
volumes + mounts:
|
||||
```yaml
|
||||
# volumes:
|
||||
- name: audit-policy
|
||||
hostPath: {path: /etc/kubernetes/audit, type: DirectoryOrCreate}
|
||||
- name: audit-logs
|
||||
hostPath: {path: /var/log/kubernetes/audit, type: DirectoryOrCreate}
|
||||
# volumeMounts:
|
||||
- name: audit-policy
|
||||
mountPath: /etc/kubernetes/audit
|
||||
readOnly: true
|
||||
- name: audit-logs
|
||||
mountPath: /var/log/kubernetes/audit
|
||||
readOnly: false
|
||||
```
|
||||
Verify inside the control-plane:
|
||||
```bash
|
||||
kubectl -n vault-ns get secret pre-existing
|
||||
tail -f /var/log/kubernetes/audit/audit.log | grep pre-existing # Metadata level, no body
|
||||
```
|
||||
Big gotcha: the audit-logs mount must be `readOnly: false` or the apiserver crashloops silently. And the log dir hostPath is *inside the control-plane container* — that's fine here.
|
||||
|
||||
---
|
||||
|
||||
### Reset between attempts
|
||||
```bash
|
||||
kind delete cluster --name cks
|
||||
./bootstrap.sh && ./seed.sh
|
||||
```
|
||||
Reference in New Issue
Block a user