143 lines
6.2 KiB
Markdown
143 lines
6.2 KiB
Markdown
# Assignment
|
|
|
|
One of the DevOps team members was trying to install a WordPress website on a LAMP stack, which is deployed on a Kubernetes cluster. It was working well, and we could see the installation page a few hours ago. However, something seems to have gone wrong with the stack after the website went down. Please look into the issue and fix it:
|
|
|
|
|
|
|
|
FYI, the deployment name is lamp-wp and it is using a service named lamp-service. Apache is using the default HTTP port, and the NodePort is 30008. From the application logs, it has been identified that the application is facing some issues connecting to the database, in addition to other problems. Additionally, there are some environment variables associated with the pods, such as MYSQL_ROOT_PASSWORD, MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD, and MYSQL_HOST
|
|
|
|
Also, do not attempt to delete or modify any other existing components, such as deployment names, service names, types, labels, secrets and so on.
|
|
|
|
|
|
Note: The kubectl utility on the jump-host has been configured to work with the Kubernetes cluster.
|
|
|
|
# Solution
|
|
|
|
# Troubleshoot LAMP WordPress — `lamp-wp` / `lamp-service`
|
|
|
|
The WordPress site is down after a bad change. The hints point to **two+ problems**: a DB-connection
|
|
issue (env vars) and "other problems" (typically the service port). This diagnoses, then fixes
|
|
surgically — **without** touching deployment/service names, types, labels, or secrets, as required.
|
|
|
|
## Step 1 — Diagnose
|
|
|
|
```bash
|
|
# Pod + container state
|
|
kubectl get pods -l app=lamp-wp # adjust selector to match
|
|
kubectl describe pod -l app=lamp-wp
|
|
|
|
# App logs (the DB connection error shows here)
|
|
kubectl logs -l app=lamp-wp -c <httpd-or-php-container>
|
|
kubectl logs -l app=lamp-wp -c <mysql-container>
|
|
|
|
# Full specs — find the mismatches
|
|
kubectl get deployment lamp-wp -o yaml
|
|
kubectl get service lamp-service -o yaml
|
|
kubectl get secrets # note the secret + keys the env vars use
|
|
```
|
|
|
|
Look for these specific mismatches:
|
|
|
|
1. **Service port** — Apache serves on **80** (default HTTP). The Service's `targetPort` must be
|
|
`80`; if the bad edit set it to something else (e.g. `8080`), the NodePort routes to a dead
|
|
port. `nodePort` stays `30008`.
|
|
2. **`MYSQL_HOST`** — in the WordPress/PHP container this must equal the **DB service name** (check
|
|
`kubectl get svc` for the MySQL service). If it points at the wrong host/IP, WordPress can't
|
|
reach the database — the logged connection error.
|
|
3. **DB credential env consistency** — the WordPress container's `MYSQL_ROOT_PASSWORD`,
|
|
`MYSQL_DATABASE`, `MYSQL_USER`, `MYSQL_PASSWORD` must match what the **MySQL** container was
|
|
initialized with (usually both pull from the same Secret keys). A key mismatch means WordPress
|
|
authenticates with wrong credentials.
|
|
|
|
## Step 2 — Fix (surgical; names/types/labels/secrets untouched)
|
|
|
|
### Service targetPort → 80
|
|
|
|
```bash
|
|
kubectl patch service lamp-service \
|
|
--type=json \
|
|
-p='[{"op":"replace","path":"/spec/ports/0/targetPort","value":80}]'
|
|
```
|
|
|
|
### `MYSQL_HOST` → the correct DB service name
|
|
|
|
```bash
|
|
# Find the MySQL service name first
|
|
kubectl get svc
|
|
|
|
# Set MYSQL_HOST on the WordPress/PHP container (use its real container name)
|
|
kubectl set env deployment/lamp-wp \
|
|
--containers='<httpd-or-php-container>' \
|
|
MYSQL_HOST=<mysql-service-name>
|
|
```
|
|
|
|
### Any wrong credential env / secret key reference
|
|
|
|
If an env var references a wrong Secret key, correct the **reference** (not the Secret) in place:
|
|
|
|
```bash
|
|
kubectl edit deployment lamp-wp
|
|
# fix the mistyped valueFrom.secretKeyRef.key / .name to match `kubectl get secret <name> -o yaml`
|
|
```
|
|
|
|
Each change updates the pod template and triggers a fresh rollout:
|
|
|
|
```bash
|
|
kubectl rollout status deployment/lamp-wp
|
|
```
|
|
|
|
## How it works
|
|
|
|
### Two independent failure planes
|
|
|
|
This stack breaks in two places that must both be right:
|
|
|
|
- **Network path** — `<node-ip>:30008` (nodePort) → Service `port` → **`targetPort` 80** → Apache.
|
|
If `targetPort` doesn't match Apache's listen port, the page is unreachable even when the pod is
|
|
perfectly healthy. That's the "other problem" beyond the DB.
|
|
- **App→DB path** — WordPress connects to MySQL over the cluster network using `MYSQL_HOST` plus the
|
|
credential env vars. `MYSQL_HOST` must resolve to the MySQL **Service** (stable DNS name), and the
|
|
credentials must match what MySQL was initialized with. A wrong host or mismatched credential is
|
|
the logged "can't connect to database" error.
|
|
|
|
Fixing one without the other leaves the site down, which is why the task hints at multiple issues.
|
|
|
|
### Why surgical edits, not re-apply
|
|
|
|
The task forbids changing deployment/service names, types, labels, and secrets. Re-applying a
|
|
hand-built manifest risks altering those by omission. `kubectl patch` (targetPort), `kubectl set
|
|
env` (one env var on one container), and `kubectl edit` (a single reference) each touch exactly the
|
|
broken field and leave every protected component intact. This is the safe way to honor the "don't
|
|
modify other components" rule.
|
|
|
|
### Why `MYSQL_HOST` is a Service name
|
|
|
|
Pod IPs are ephemeral; a Service gives MySQL a stable DNS name inside the cluster. WordPress must
|
|
target that name so it keeps resolving across pod restarts — hardcoding an IP or using a wrong name
|
|
breaks on the first reschedule, which is exactly the kind of thing a bad manual edit introduces.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
# Pods running
|
|
kubectl get pods -l app=lamp-wp
|
|
|
|
# Service targetPort is 80, nodePort 30008
|
|
kubectl get service lamp-service \
|
|
-o jsonpath='{.spec.ports[0].targetPort}{" "}{.spec.ports[0].nodePort}{"\n"}'
|
|
|
|
# WordPress env has correct MYSQL_HOST
|
|
kubectl set env deployment/lamp-wp --list | grep MYSQL_HOST
|
|
|
|
# App reachable / DB connected (no more DB error in logs)
|
|
kubectl logs -l app=lamp-wp -c <httpd-or-php-container> | tail
|
|
curl -sI http://<node-ip>:30008 | head -n1
|
|
```
|
|
|
|
Expected — pods `Running`, service `targetPort 80` / `nodePort 30008`, `MYSQL_HOST` set to the
|
|
MySQL service name, no DB-connection errors in the logs, and the WordPress installation/login page
|
|
loading at `<node-ip>:30008`.
|
|
|
|
> Paste `kubectl get deployment lamp-wp -o yaml` and `kubectl get service lamp-service -o yaml`
|
|
> (plus `kubectl get secret <name> -o yaml`) and I'll give you the exact patches — the precise fix
|
|
> depends on which values were changed. |