Compare commits
1 Commits
main
...
974c3e8b00
| Author | SHA1 | Date | |
|---|---|---|---|
|
974c3e8b00 |
@@ -1,14 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(for f:*)",
|
||||
"Bash(do echo:*)",
|
||||
"Read(//Users/jan.novak/srv/personal/home-kubernetes/**)",
|
||||
"Bash(done)",
|
||||
"Bash(ssh docker-30 *)",
|
||||
"Bash(git add *)",
|
||||
"WebSearch",
|
||||
"WebFetch(domain:cert-manager.io)"
|
||||
]
|
||||
}
|
||||
}
|
||||
14
.gitignore
vendored
14
.gitignore
vendored
@@ -1,15 +1 @@
|
||||
.DS_Store
|
||||
|
||||
.terraform/
|
||||
.terraform.lock.hcl
|
||||
|
||||
kubernetes-kvm-terraform/join-command.txt
|
||||
kubernetes-kvm-terraform/kubeconfig
|
||||
|
||||
tmp/
|
||||
|
||||
vms/utility-101-shadow/docker/monitoring/smtp_password
|
||||
|
||||
docker-30/zot/sync-credentials.json
|
||||
vms-home/docker-30/zot/sync-credentials.json
|
||||
kubernetes-kvm-terraform/gke_gcloud_auth_plugin_cache
|
||||
|
||||
105
CLAUDE.md
105
CLAUDE.md
@@ -1,105 +0,0 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Repository Purpose
|
||||
|
||||
Home Kubernetes lab infrastructure-as-code. Manages VM provisioning (Terraform/KVM), Kubernetes cluster configuration, and application deployments via Flux GitOps — all pointing at a self-hosted Gitea instance.
|
||||
|
||||
## Common Commands
|
||||
|
||||
### Terraform (kubernetes-kvm-terraform/)
|
||||
```bash
|
||||
cd kubernetes-kvm-terraform
|
||||
tofu init
|
||||
tofu plan
|
||||
tofu apply
|
||||
tofu destroy
|
||||
```
|
||||
|
||||
### Flux GitOps
|
||||
```bash
|
||||
# Check reconciliation status
|
||||
kubectl get kustomizations -A
|
||||
kubectl get helmreleases -A
|
||||
|
||||
# Force reconcile
|
||||
flux reconcile kustomization flux-system --with-source
|
||||
flux reconcile helmrelease <name> -n <namespace>
|
||||
|
||||
# Watch logs
|
||||
flux logs --follow
|
||||
|
||||
# Check source sync
|
||||
flux get sources git
|
||||
```
|
||||
|
||||
### kubectl — common ops
|
||||
```bash
|
||||
export KUBECONFIG=kubernetes-kvm-terraform/kubeconfig
|
||||
kubectl get nodes
|
||||
kubectl get pods -A
|
||||
kubectl get secrets -A
|
||||
```
|
||||
|
||||
### Docker Compose (docker-30/ services)
|
||||
```bash
|
||||
# These run on 192.168.0.30 (docker-30), accessed via SSH
|
||||
ssh novakj@192.168.0.30
|
||||
cd /path/to/service && docker compose up -d
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Infrastructure Layer
|
||||
- **Hypervisors**: homer (192.168.0.7) and beelink (192.168.0.6) run KVM/libvirt
|
||||
- **Terraform provider**: primary `qemu+ssh://novakj@192.168.0.7/system`, secondary alias `kvm-beelink` for 192.168.0.6
|
||||
- **Kubernetes master**: kube-master-31 @ 192.168.0.31:6443, bootstrapped via kubeadm
|
||||
- **OS**: Ubuntu 24.04 Noble cloud images, network bridge `br0`, subnet 192.168.0.0/24
|
||||
|
||||
### GitOps Layer (`gitops/home-kubernetes/`)
|
||||
Flux syncs from Gitea (`https://gitea.home.hrajfrisbee.cz`, main branch, every 10 minutes). The reconciliation order is enforced via `dependsOn` in `flux-system/extra-kustomizations.yaml`:
|
||||
|
||||
```
|
||||
00-crds → 00-rbac → cilium → cert-manager → external-secrets → everything else
|
||||
```
|
||||
|
||||
Each application lives in its own subdirectory under `gitops/home-kubernetes/` and is referenced as a Flux `Kustomization` resource. Prune is enabled — removing a manifest from git removes it from the cluster.
|
||||
|
||||
### Secrets Flow
|
||||
Vault (docker-30) → External-Secrets controller (in-cluster) → Kubernetes `Secret` objects. Applications reference `ExternalSecret` CRs that pull from Vault paths. Do not put real secrets in git.
|
||||
|
||||
### Networking
|
||||
- **CNI**: Cilium 1.19.x with Gateway API and Hubble UI enabled
|
||||
- **L2 LB**: Cilium `CiliumL2AnnouncementPolicy` + `CiliumLoadBalancerIPPool` for bare-metal load-balancer IPs (defined in `gitops/home-kubernetes/cilium/`)
|
||||
- **Ingress**: ingress-nginx for HTTP(S) workloads; Gateway API for newer apps
|
||||
- **TLS**: cert-manager issues wildcard cert (`*.home.hrajfrisbee.cz`) referenced by apps
|
||||
|
||||
### Storage
|
||||
- **democratic-CSI**: iSCSI volumes backed by FreeNAS at 192.168.0.40
|
||||
- **Longhorn**: configured but disabled in Flux (directory present, not in `extra-kustomizations.yaml`)
|
||||
|
||||
### Supporting Services on docker-30 (192.168.0.30)
|
||||
All managed with Docker Compose:
|
||||
- **Gitea** — Git server + act_runner (GitHub Actions-compatible CI)
|
||||
- **Vault** — secrets backend for External Secrets
|
||||
- **Zot** — private OCI/container registry
|
||||
- **Kanidm** — identity management / OIDC provider
|
||||
- **nginx** — reverse proxy for docker-30 services
|
||||
|
||||
## Key Conventions
|
||||
|
||||
- **New application**: create a directory under `gitops/home-kubernetes/<app-name>/`, add a `Kustomization` entry in `flux-system/extra-kustomizations.yaml` with appropriate `dependsOn`.
|
||||
- **Helm apps**: use a `HelmRepository` + `HelmRelease` pair; pin chart versions explicitly.
|
||||
- **Secrets**: add an `ExternalSecret` CR pointing to the Vault path; never commit actual secret values.
|
||||
- **Terraform state**: `kubernetes-kvm-terraform/terraform.tfstate*` is gitignored/sensitive; treat it carefully.
|
||||
- **kubeconfig**: `kubernetes-kvm-terraform/kubeconfig` is gitignored; obtain it from the master node after provisioning.
|
||||
|
||||
|
||||
## Plans
|
||||
|
||||
When Claude Code's plan mode is used, save the plan file inside the repo at
|
||||
`docs/plans/YYYY-MM-DD-HHMM-<slug>.md` instead of the default `~/.claude/plans/`
|
||||
location. Get the timestamp with `date "+%Y-%m-%d-%H%M"` (matches the changelog
|
||||
convention). The `<slug>` should be a short kebab-case summary of the plan's topic.
|
||||
@@ -1,7 +0,0 @@
|
||||
# How i did have some fun with gatewayApi
|
||||
|
||||
As nginx-ingress will be no longer maintained it looks like the time to upgrade is coming even for the lazy old schoolers who were very happy with it (even though annotations and custom snippets might not be the cleanest way, people experienced with nginx might have liked it for a reason). Although I tried to pretend that there is no gatewayApi and ingress is the thing for quite some time recently everything aligned for me to give it a go.
|
||||
|
||||
## Which implementation to choose?
|
||||
|
||||
There is a number of options and I'm only vaguely aware of them as I was not interested in it for a long time. But i have a tendency of trying to do something with Envoy (personal feeling that it is the right next thing here?) and I'm using the Cilium network plugin in my homelab kubernetes cluster. All this considered answer is obvious: use envoy gateway which is part of cilium.
|
||||
@@ -57,15 +57,6 @@ services:
|
||||
- GITEA__server__ROOT_URL=https://gitea.home.hrajfrisbee.cz
|
||||
- GITEA__security__SECRET_KEY=${GITEA_SECRET_KEY}
|
||||
- GITEA__security__INTERNAL_TOKEN=${INTERNAL_TOKEN}
|
||||
- GITEA__mailer__ENABLED=true
|
||||
- GITEA__mailer__PROTOCOL=smtps
|
||||
- GITEA__mailer__SMTP_ADDR=smtp.gmail.com
|
||||
- GITEA__mailer__SMTP_PORT=465
|
||||
- GITEA__mailer__USER=kacerr.cz@gmail.com
|
||||
- GITEA__mailer__PASSWD=${GMAIL_GITEA_APP_PASSWORD}
|
||||
- GITEA__mailer__FROM=kacerr.cz+gitea@gmail.com
|
||||
- GITEA__packages__ENABLED=true
|
||||
|
||||
#- GITEA__storage__STORAGE_TYPE=minio
|
||||
#- GITEA__storage__MINIO_ENDPOINT=minio:9000
|
||||
#- GITEA__storage__MINIO_ACCESS_KEY_ID=gitea
|
||||
@@ -92,12 +83,10 @@ services:
|
||||
depends_on:
|
||||
- gitea
|
||||
environment:
|
||||
GITEA_INSTANCE_URL: https://gitea.home.hrajfrisbee.cz/
|
||||
GITEA_INSTANCE_URL: http://gitea:3000
|
||||
GITEA_RUNNER_REGISTRATION_TOKEN: ${RUNNER_TOKEN}
|
||||
CONFIG_FILE: /config/config.yaml
|
||||
volumes:
|
||||
- ./runner-data:/data
|
||||
- ./runner-config.yaml:/config/config.yaml:ro
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
networks:
|
||||
- gitea-network
|
||||
@@ -1,14 +0,0 @@
|
||||
## kubectl
|
||||
|
||||
```bash
|
||||
# condensed -o wide
|
||||
kubectl get pods -A -o wide --watch | awk '
|
||||
/NOMINATED NODE/ { sub(/[[:space:]]+NOMINATED NODE[[:space:]]+READINESS GATES[[:space:]]*$/,""); print; fflush(); next }
|
||||
{ sub(/[[:space:]]+[^[:space:]]+[[:space:]]+[^[:space:]]+[[:space:]]*$/,""); print; fflush() }
|
||||
'
|
||||
|
||||
# condensed -o wide
|
||||
kubectl get pods -A --watch -o custom-columns=\
|
||||
'NAMESPACE:.metadata.namespace,NAME:.metadata.name,STATUS:.status.phase,RESTARTS:.status.containerStatuses[0].restartCount,IP:.status.podIP,NODE:.spec.nodeName'
|
||||
|
||||
```
|
||||
@@ -1,180 +0,0 @@
|
||||
# Kubernetes component tracing
|
||||
|
||||
OpenTelemetry tracing for kube-apiserver and kubelet, shipping OTLP spans to Jaeger on docker-29. Useful for measuring pod-creation latency broken down by component.
|
||||
|
||||
Related: [plans/k8s-pod-creation-tracing.md](../plans/2026-05-21%2020%3A15%20-%20k8s-pod-creation-tracing.md) — setup rationale, storage sizing, etcd tracing notes.
|
||||
|
||||
---
|
||||
|
||||
## Where the toggles live
|
||||
|
||||
| Component | Config on node | Restart trigger |
|
||||
|---|---|---|
|
||||
| kube-apiserver | `/etc/kubernetes/manifests/kube-apiserver.yaml` (flag) + `/etc/kubernetes/tracing-config.yaml` | kubelet detects manifest change, auto-restarts static pod (~5–10 s) |
|
||||
| kubelet | `/var/lib/kubelet/config.yaml` (`tracing:` block) + `kube-system/kubelet-config` CM | `systemctl restart kubelet` |
|
||||
|
||||
Nodes: kube-master-31 @ 192.168.0.31, kube-node-32 @ 192.168.0.32, kube-node-33 @ 192.168.0.33.
|
||||
|
||||
---
|
||||
|
||||
## Quick suppress (no component restart)
|
||||
|
||||
Stop the Jaeger backend — components keep building spans but have nowhere to send them. Lowest-risk; use this between measurement sessions.
|
||||
|
||||
```bash
|
||||
ssh novakj@192.168.0.29 'cd ~/docker-29/tracing && docker compose stop jaeger'
|
||||
```
|
||||
|
||||
Resume:
|
||||
|
||||
```bash
|
||||
ssh novakj@192.168.0.29 'cd ~/docker-29/tracing && docker compose start jaeger'
|
||||
```
|
||||
|
||||
Caveat: components still pay the (small) cost of building spans. For true zero-overhead, use **Full disable** below.
|
||||
|
||||
---
|
||||
|
||||
## Full enable (live cluster)
|
||||
|
||||
### 1. Start Jaeger on docker-30
|
||||
|
||||
```bash
|
||||
ssh novakj@192.168.0.29 'cd ~/docker-29/tracing && docker compose up -d'
|
||||
```
|
||||
|
||||
### 2. kube-apiserver (master only — 192.168.0.31)
|
||||
|
||||
```bash
|
||||
ssh novakj@192.168.0.31 bash -s <<'EOF'
|
||||
# TracingConfiguration file
|
||||
sudo tee /etc/kubernetes/tracing-config.yaml > /dev/null <<'YAML'
|
||||
apiVersion: apiserver.config.k8s.io/v1beta1
|
||||
kind: TracingConfiguration
|
||||
endpoint: 192.168.0.29:4317
|
||||
samplingRatePerMillion: 1000000
|
||||
YAML
|
||||
|
||||
# Back up the manifest before touching it
|
||||
sudo cp /etc/kubernetes/manifests/kube-apiserver.yaml /root/kube-apiserver.yaml.pre-tracing
|
||||
|
||||
# Add the flag (after --tls-private-key-file line)
|
||||
sudo sed -i '/--tls-private-key-file=/a\ - --tracing-config-file=/etc/kubernetes/tracing-config.yaml' \
|
||||
/etc/kubernetes/manifests/kube-apiserver.yaml
|
||||
|
||||
# Wait for static pod to restart
|
||||
sleep 10
|
||||
sudo crictl ps | grep apiserver
|
||||
kubectl get --raw /livez
|
||||
EOF
|
||||
```
|
||||
|
||||
### 3. kubelet (all three nodes)
|
||||
|
||||
Run on each node in turn — wait for `Ready` before moving to the next to avoid simultaneous NotReady:
|
||||
|
||||
```bash
|
||||
for NODE in 192.168.0.31 192.168.0.32 192.168.0.33; do
|
||||
echo "==> $NODE"
|
||||
ssh ubuntu@$NODE bash -s <<'EOF'
|
||||
grep -q '^tracing:' /var/lib/kubelet/config.yaml && echo "already enabled, skipping" && exit 0
|
||||
sudo tee -a /var/lib/kubelet/config.yaml > /dev/null <<'YAML'
|
||||
tracing:
|
||||
endpoint: 192.168.0.29:4317
|
||||
samplingRatePerMillion: 1000000
|
||||
YAML
|
||||
sudo systemctl restart kubelet
|
||||
EOF
|
||||
# wait for node Ready before proceeding
|
||||
kubectl wait node --for=condition=Ready --timeout=60s $(kubectl get node -o wide | awk "/$NODE/{print \$1}")
|
||||
done
|
||||
```
|
||||
|
||||
### 4. Update the cluster ConfigMap (so future joins/reboots pick it up)
|
||||
|
||||
```bash
|
||||
kubectl -n kube-system edit cm kubelet-config
|
||||
# under the `kubelet:` key, add:
|
||||
# tracing:
|
||||
# endpoint: 192.168.0.29:4317
|
||||
# samplingRatePerMillion: 1000000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Full disable (live cluster)
|
||||
|
||||
### kube-apiserver (master only)
|
||||
|
||||
```bash
|
||||
ssh novakj@192.168.0.31 bash -s <<'EOF'
|
||||
# Restore the pre-tracing manifest (exact revert)
|
||||
sudo cp /root/kube-apiserver.yaml.pre-tracing /etc/kubernetes/manifests/kube-apiserver.yaml
|
||||
# Optional: remove the config file
|
||||
sudo rm -f /etc/kubernetes/tracing-config.yaml
|
||||
EOF
|
||||
```
|
||||
|
||||
Static pod restarts automatically. If the backup wasn't taken, remove the flag in-place instead:
|
||||
|
||||
```bash
|
||||
sudo sed -i '/--tracing-config-file=/d' /etc/kubernetes/manifests/kube-apiserver.yaml
|
||||
```
|
||||
|
||||
### kubelet (all three nodes)
|
||||
|
||||
```bash
|
||||
for NODE in 192.168.0.31 192.168.0.32 192.168.0.33; do
|
||||
echo "==> $NODE"
|
||||
ssh novakj@$NODE bash -s <<'EOF'
|
||||
sudo sed -i '/^tracing:/,/samplingRatePerMillion:/d' /var/lib/kubelet/config.yaml
|
||||
sudo systemctl restart kubelet
|
||||
EOF
|
||||
kubectl wait node --for=condition=Ready --timeout=60s $(kubectl get node -o wide | awk "/$NODE/{print \$1}")
|
||||
done
|
||||
```
|
||||
|
||||
Remove the tracing block from the cluster ConfigMap too:
|
||||
|
||||
```bash
|
||||
kubectl -n kube-system edit cm kubelet-config
|
||||
# remove the `tracing:` block under `kubelet:`
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
### After enable
|
||||
|
||||
1. Open Jaeger UI: `http://192.168.0.29:16686` — Services dropdown should show `apiserver` and `kubelet` (appear after first traced request).
|
||||
|
||||
2. Generate a full pod-lifecycle trace:
|
||||
|
||||
```bash
|
||||
kubectl run trace-probe --image=registry.k8s.io/pause:3.10 --restart=Never
|
||||
kubectl wait --for=condition=Ready pod/trace-probe --timeout=60s
|
||||
kubectl delete pod trace-probe
|
||||
```
|
||||
|
||||
3. In Jaeger UI, search:
|
||||
- Service `apiserver` — look for spans covering the create/watch flow for `trace-probe`.
|
||||
- Service `kubelet` — look for `syncPod` spans tied to the same pod.
|
||||
|
||||
4. The gap between apiserver spans and kubelet spans is scheduler time (kube-scheduler has no OTLP tracing in 1.32). Cross-check with `kubectl get events` or pod condition timestamps.
|
||||
|
||||
### After disable
|
||||
|
||||
Repeat step 2 above, then confirm no new spans appear in Jaeger for `apiserver` or `kubelet` within ~1 minute.
|
||||
|
||||
---
|
||||
|
||||
## Sampling rate cheatsheet
|
||||
|
||||
Change `samplingRatePerMillion` in `/etc/kubernetes/tracing-config.yaml` (apiserver) and `/var/lib/kubelet/config.yaml` (kubelet) then restart each component.
|
||||
|
||||
| Value | Effective rate | Use case |
|
||||
|---|---|---|
|
||||
| `1000000` | 100% | Active measurement |
|
||||
| `10000` | 1% | Steady-state background |
|
||||
| `0` | off | Disable without removing config |
|
||||
@@ -1,142 +0,0 @@
|
||||
# Alloy: deployment & operations
|
||||
|
||||
Three Grafana Alloy instances run in the `monitoring` namespace, all deployed via
|
||||
the Grafana `alloy` Helm chart (`>=1.8.0 <2.0.0`, image `v1.17.1`) and managed by
|
||||
Flux with `driftDetection: enabled` (manual `kubectl edit`s get reverted — always
|
||||
change config via git). Logs and events ship to Loki; metrics ship to Mimir. Every
|
||||
instance stamps `cluster="homelab"` as an external label.
|
||||
|
||||
---
|
||||
|
||||
## Instances
|
||||
|
||||
| Instance | Controller | Collects | Sink |
|
||||
|---|---|---|---|
|
||||
| `alloy-logs` | DaemonSet (`runAsUser: 0`, one pod per node) | pod logs + node journald | Loki `http://192.168.0.30:3100/loki/api/v1/push` |
|
||||
| `alloy-events` | Deployment (1 replica) | Kubernetes events | Loki (same endpoint) |
|
||||
| `alloy-metrics` | Deployment (1 replica) | all ServiceMonitors + PodMonitors cluster-wide | Mimir `http://192.168.0.30:9009/api/v1/push` |
|
||||
|
||||
`alloy-logs` and `alloy-events` are one-per-node / cluster-scoped respectively —
|
||||
`alloy-metrics` is also a single Deployment, never a DaemonSet, since its
|
||||
Prometheus-Operator discovery is already cluster-wide (a DaemonSet would scrape
|
||||
every target once per node).
|
||||
|
||||
---
|
||||
|
||||
## Files & Flux wiring
|
||||
|
||||
All manifests live in `gitops/home-kubernetes/alloy/`:
|
||||
|
||||
- `helmrepository_grafana.yaml` — the shared `grafana` HelmRepository.
|
||||
- `helmrelease_alloy-logs.yaml` — logs DaemonSet.
|
||||
- `helmrelease_allow-events.yaml` — events Deployment (note the `allow` filename typo, kept as-is).
|
||||
- `helmrelease_alloy-metrics.yaml` — metrics Deployment.
|
||||
- `rbac_log-collector.yaml` — ClusterRole `alloy-log-reader`, bound to the `alloy-logs`/`alloy-events` ServiceAccounts.
|
||||
- `rbac_metrics-collector.yaml` — ClusterRole `alloy-metrics-reader`, bound to the `alloy-metrics` ServiceAccount.
|
||||
|
||||
Each HelmRelease sets `rbac.create: false` — RBAC is supplied manually by the two
|
||||
files above rather than by the chart, since the chart's default ClusterRole
|
||||
doesn't cover everything each pipeline needs (journald/events reads, or
|
||||
ServiceMonitor/PodMonitor/kubelet-metrics reads).
|
||||
|
||||
Reconciliation: a Flux `Kustomization` named `alloy` in
|
||||
`gitops/home-kubernetes/flux-system/extra-kustomizations.yaml`, pointing at
|
||||
`./gitops/home-kubernetes/alloy`, with `dependsOn: kube-prometheus` — it needs
|
||||
the `monitoring` namespace and the Prometheus Operator CRDs (ServiceMonitor/
|
||||
PodMonitor) to exist first.
|
||||
|
||||
---
|
||||
|
||||
## How metrics discovery works
|
||||
|
||||
`alloy-metrics` runs two Alloy components — `prometheus.operator.servicemonitors`
|
||||
and `prometheus.operator.podmonitors` — which discover and scrape **every**
|
||||
ServiceMonitor/PodMonitor in the cluster, then forward to
|
||||
`prometheus.remote_write` targeting Mimir. There's no bespoke scrape config for
|
||||
"standard Kubernetes metrics": kube-prometheus-stack already ships
|
||||
ServiceMonitors for kubelet, cAdvisor, kube-state-metrics, node-exporter,
|
||||
apiserver, and coredns, so those are picked up automatically. Any future
|
||||
ServiceMonitor/PodMonitor added anywhere in the cluster is picked up the same way
|
||||
with no Alloy config change.
|
||||
|
||||
This is **additive** — kube-prometheus-stack's bundled Prometheus keeps scraping
|
||||
the same targets into its own local 60d TSDB. Alloy scraping the same targets a
|
||||
second time and shipping to Mimir is expected double-collection, not a bug.
|
||||
|
||||
---
|
||||
|
||||
## Operating
|
||||
|
||||
Status:
|
||||
|
||||
```bash
|
||||
flux get helmreleases -n monitoring
|
||||
kubectl get pods -n monitoring -l app.kubernetes.io/name=alloy -o wide
|
||||
```
|
||||
|
||||
Force reconcile:
|
||||
|
||||
```bash
|
||||
flux reconcile kustomization alloy --with-source
|
||||
flux reconcile helmrelease alloy-metrics -n monitoring
|
||||
```
|
||||
|
||||
Logs:
|
||||
|
||||
```bash
|
||||
kubectl -n monitoring logs deploy/alloy-metrics
|
||||
kubectl -n monitoring logs ds/alloy-logs
|
||||
kubectl -n monitoring logs deploy/alloy-events
|
||||
```
|
||||
|
||||
Alloy UI (component graph, target health, remote_write queue status):
|
||||
|
||||
```bash
|
||||
kubectl -n monitoring port-forward deploy/alloy-metrics 12345:12345
|
||||
# open http://localhost:12345
|
||||
```
|
||||
|
||||
Editing config: change the inline `alloy.configMap.content` block in the
|
||||
relevant `helmrelease_*.yaml`, commit, push, then force-reconcile (above).
|
||||
Do not `kubectl edit` the generated ConfigMap directly — `driftDetection` will
|
||||
revert it on the next reconcile.
|
||||
|
||||
---
|
||||
|
||||
## Verification / smoke tests
|
||||
|
||||
**Metrics reaching Mimir:**
|
||||
|
||||
```bash
|
||||
curl -s 'http://192.168.0.30:9009/prometheus/api/v1/query?query=up' | jq '.data.result | length'
|
||||
# expect > 0
|
||||
```
|
||||
|
||||
Spot-check a few standard series exist and carry the cluster label:
|
||||
|
||||
```bash
|
||||
curl -s 'http://192.168.0.30:9009/prometheus/api/v1/query?query=kube_pod_info' | jq '.data.result[0].metric'
|
||||
curl -s 'http://192.168.0.30:9009/prometheus/api/v1/query?query=node_cpu_seconds_total' | jq '.data.result | length'
|
||||
curl -s 'http://192.168.0.30:9009/prometheus/api/v1/query?query=container_cpu_usage_seconds_total' | jq '.data.result | length'
|
||||
# confirm "cluster": "homelab" is present in the returned label sets
|
||||
```
|
||||
|
||||
**Logs reaching Loki:**
|
||||
|
||||
```bash
|
||||
curl -s http://192.168.0.30:3100/ready
|
||||
# then in Grafana / logcli, query {cluster="homelab"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause |
|
||||
|---|---|
|
||||
| `forbidden` errors in `alloy-metrics` logs | `alloy-metrics-reader` ClusterRole missing a resource/verb — check `rbac_metrics-collector.yaml` against what the component is trying to read |
|
||||
| No targets shown in the Alloy UI | Prometheus Operator CRDs not installed, or `kube-prometheus` Flux Kustomization not Ready yet (metrics depends on it) |
|
||||
| `prometheus.remote_write` shows failed sends / 4xx-5xx | Mimir down, or wrong endpoint — must be `http://192.168.0.30:9009/api/v1/push` exactly |
|
||||
| Loki push failing | Loki down at `192.168.0.30:3100`, check with `curl http://192.168.0.30:3100/ready` |
|
||||
| Same series appearing to be scraped twice | Expected — kube-prometheus-stack's bundled Prometheus and `alloy-metrics` both scrape the same ServiceMonitors by design (additive, not deduplicated) |
|
||||
| Manual `kubectl edit` on the Alloy ConfigMap reverts itself | `driftDetection: enabled` — edit the HelmRelease in git instead |
|
||||
@@ -1,140 +0,0 @@
|
||||
# Tracing enable/disable runbook for K8s components
|
||||
|
||||
## Context
|
||||
|
||||
The pod-creation tracing plan ([plans/2026-05-21 20:15 - k8s-pod-creation-tracing.md](../../plans/2026-05-21%2020%3A15%20-%20k8s-pod-creation-tracing.md)) covers the *one-time setup* of OTLP tracing across kube-apiserver and kubelet, plus the Jaeger backend on docker-29. What's missing is a short operational doc for **toggling tracing on/off as quickly as possible** on the running cluster — useful for short measurement experiments where we want overhead only when we're actively recording.
|
||||
|
||||
The Terraform/cloud-init changes already in flight (`master.tf`, `files/manifests/kube-apiserver.yaml`, `files/tracing-config.yaml`) cover *rebuilds*. This runbook covers the *live* cluster: kube-master-31 + kube-node-32 + kube-node-33, K8s 1.32.
|
||||
|
||||
Output of this plan: a single new file `docs/kubernetes-tracing.md` containing three runbooks (quick suppress, full enable, full disable) and a verification section. No code or manifest changes — pure documentation.
|
||||
|
||||
## What "as quickly as possible" means
|
||||
|
||||
There are two speeds of toggling:
|
||||
|
||||
1. **Quick suppress** — leave the tracing flags/config in place on the components; just stop Jaeger so span exports go nowhere. No K8s component restart, takes seconds, but components still pay the (small) cost of building spans and attempting to export.
|
||||
2. **Full enable / full disable** — edit configs and restart components. apiserver static-pod restart is automatic via kubelet manifest watch (~5–10s); kubelet itself needs `systemctl restart kubelet` on each node (brief NotReady blip per node, no pod evictions). This is what we'd use to truly remove overhead between experiments.
|
||||
|
||||
The runbook will present both, with quick-suppress as the default for "I'm done measuring for now" and full-disable for "we don't need this on the cluster anymore".
|
||||
|
||||
## Deliverable: `docs/kubernetes-tracing.md`
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
# Kubernetes component tracing
|
||||
|
||||
## Components and where the toggles live
|
||||
- table: component | config file on node | restart trigger
|
||||
- kube-apiserver | /etc/kubernetes/manifests/kube-apiserver.yaml (flag) +
|
||||
/etc/kubernetes/tracing-config.yaml (config) | kubelet auto-restarts static pod
|
||||
- kubelet | /var/lib/kubelet/config.yaml (`tracing:` block) +
|
||||
kube-system/kubelet-config CM (cluster-wide source of truth) | systemctl restart kubelet
|
||||
|
||||
## Quick suppress (no component restart)
|
||||
1. `ssh novakj@192.168.0.29 'cd /path/to/tracing && docker compose stop jaeger'`
|
||||
2. Components continue running with tracing wired up but spans go nowhere.
|
||||
3. To resume: `docker compose start jaeger`.
|
||||
Caveat: components still build spans and attempt OTLP export — small CPU/network overhead. For a real "off", use full disable.
|
||||
|
||||
## Full enable (live cluster)
|
||||
|
||||
Prereqs: Jaeger is up on docker-29 (`docker compose up -d` in `docker-29/tracing/`).
|
||||
|
||||
### kube-apiserver (master only)
|
||||
On 192.168.0.31:
|
||||
```bash
|
||||
# 1. Drop the TracingConfiguration file (same content as files/tracing-config.yaml)
|
||||
sudo tee /etc/kubernetes/tracing-config.yaml > /dev/null <<'EOF'
|
||||
apiVersion: apiserver.config.k8s.io/v1beta1
|
||||
kind: TracingConfiguration
|
||||
endpoint: 192.168.0.29:4317
|
||||
samplingRatePerMillion: 1000000
|
||||
EOF
|
||||
|
||||
# 2. Back up the apiserver manifest
|
||||
sudo cp /etc/kubernetes/manifests/kube-apiserver.yaml /root/kube-apiserver.yaml.pre-tracing
|
||||
|
||||
# 3. Add the flag (insert after --tls-private-key-file line)
|
||||
sudo sed -i '/--tls-private-key-file=/a\ - --tracing-config-file=/etc/kubernetes/tracing-config.yaml' /etc/kubernetes/manifests/kube-apiserver.yaml
|
||||
|
||||
# 4. Wait for kubelet to restart the static pod (~5-10s)
|
||||
sudo crictl ps | grep apiserver # new container id == restarted
|
||||
kubectl get --raw /livez # should return "ok"
|
||||
```
|
||||
|
||||
### kubelet (every node: master + workers)
|
||||
|
||||
Per-node edit:
|
||||
```bash
|
||||
# Append the tracing block under kubelet config (idempotent: check first)
|
||||
grep -q '^tracing:' /var/lib/kubelet/config.yaml || sudo tee -a /var/lib/kubelet/config.yaml > /dev/null <<'EOF'
|
||||
tracing:
|
||||
endpoint: 192.168.0.29:4317
|
||||
samplingRatePerMillion: 1000000
|
||||
EOF
|
||||
|
||||
sudo systemctl restart kubelet
|
||||
```
|
||||
|
||||
Stagger across nodes (one at a time) so we don't NotReady the whole cluster at once.
|
||||
|
||||
Then make it survive node rebuilds/joins by updating the cluster ConfigMap:
|
||||
```bash
|
||||
kubectl -n kube-system edit cm kubelet-config
|
||||
# add the same `tracing:` block under the `kubelet:` key
|
||||
```
|
||||
|
||||
## Full disable (live cluster)
|
||||
|
||||
### kube-apiserver
|
||||
```bash
|
||||
# Restore the pre-tracing manifest (preferred — exact revert)
|
||||
sudo cp /root/kube-apiserver.yaml.pre-tracing /etc/kubernetes/manifests/kube-apiserver.yaml
|
||||
# OR remove just the flag in place:
|
||||
sudo sed -i '/--tracing-config-file=/d' /etc/kubernetes/manifests/kube-apiserver.yaml
|
||||
|
||||
# Optional: remove the now-unused config
|
||||
sudo rm /etc/kubernetes/tracing-config.yaml
|
||||
```
|
||||
kubelet restarts the static pod automatically.
|
||||
|
||||
### kubelet (every node)
|
||||
```bash
|
||||
sudo sed -i '/^tracing:/,/samplingRatePerMillion:/d' /var/lib/kubelet/config.yaml
|
||||
sudo systemctl restart kubelet
|
||||
```
|
||||
Then strip the `tracing:` block from `kube-system/kubelet-config` CM so node rejoins don't reintroduce it.
|
||||
|
||||
## Verification (after enable)
|
||||
1. Open `http://192.168.0.29:16686` — Services dropdown shows `apiserver` (appears after the first traced request) and `kubelet`.
|
||||
2. Generate end-to-end traffic:
|
||||
```bash
|
||||
kubectl run trace-probe --image=registry.k8s.io/pause:3.10 --restart=Never
|
||||
kubectl wait --for=condition=Ready pod/trace-probe --timeout=60s
|
||||
kubectl delete pod trace-probe
|
||||
```
|
||||
3. In Jaeger: filter service `apiserver`, look for spans tagged with the probe pod name; filter service `kubelet`, look for `syncPod` spans.
|
||||
4. To verify *disable*: after running disable steps, repeat (2). Within a minute, no new spans for `apiserver`/`kubelet` should appear in Jaeger.
|
||||
|
||||
## Notes
|
||||
- 100% sampling is fine for short experiments; for steady-state leave tracing off (quick suppress is enough between sessions).
|
||||
- The flag/config are GA in K8s 1.32 — no `--feature-gates` needed.
|
||||
- This runbook only covers apiserver + kubelet. etcd and kube-scheduler tracing are documented as future steps in the parent plan.
|
||||
```
|
||||
|
||||
## Files to change
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `docs/kubernetes-tracing.md` | **new** — the runbook described above |
|
||||
|
||||
No other files modified. The Terraform/cloud-init side stays as already drafted in the parent plan.
|
||||
|
||||
## Verification of this plan's output
|
||||
|
||||
After the runbook is written:
|
||||
|
||||
1. Skim `docs/kubernetes-tracing.md` end-to-end and check that someone unfamiliar with the setup can follow it without opening the parent plan.
|
||||
2. Cross-check the sed/tee commands against the actual files on the live master via SSH — confirm line patterns match (e.g., the `--tls-private-key-file=` line is present so the sed insertion lands in the right spot).
|
||||
3. Dry-run the quick-suppress flow first (it's the lowest-risk path) and confirm Jaeger UI stops receiving new spans within ~30 s.
|
||||
@@ -1,91 +0,0 @@
|
||||
# Move Jaeger/OTLP host from 192.168.0.30 (docker-30) → 192.168.0.29 (docker-29)
|
||||
|
||||
## Context
|
||||
|
||||
The Jaeger backend that receives OTLP spans from kube-apiserver and kubelet has been relocated from the `docker-30` host (192.168.0.30) to `docker-29` (192.168.0.29). The new compose stack is already present at [vms-home/docker-29/tracing/](../../vms-home/docker-29/tracing/) and binds `0.0.0.0:4317`/`0.0.0.0:4318`, so the backend side requires no changes.
|
||||
|
||||
The OTLP exporter endpoints on the Kubernetes side, plus the runbooks/plans that document them, still point at `192.168.0.30:4317`. They need to be repointed to `192.168.0.29:4317`. Without this change, apiserver and kubelet spans go nowhere once docker-30's Jaeger is shut down.
|
||||
|
||||
The other docker-30 references in the repo (Vault, Gitea, Zot registry mirror, Kanidm, nginx) are unrelated to tracing and must be left alone.
|
||||
|
||||
## Scope
|
||||
|
||||
In-repo file edits only. Live cluster updates are documented as a follow-up checklist; no live execution is part of this plan.
|
||||
|
||||
## File-level changes
|
||||
|
||||
### 1. Live OTEL config (load-bearing)
|
||||
|
||||
**[kubernetes-kvm-terraform/files/tracing-config.yaml:3](../../kubernetes-kvm-terraform/files/tracing-config.yaml#L3)**
|
||||
- `endpoint: 192.168.0.30:4317` → `endpoint: 192.168.0.29:4317`
|
||||
|
||||
**[kubernetes-kvm-terraform/master.tf](../../kubernetes-kvm-terraform/master.tf)** — two occurrences inside cloud-init heredocs:
|
||||
- Line 122 (apiserver `TracingConfiguration` written to `/etc/kubernetes/tracing-config.yaml`): `192.168.0.30:4317` → `192.168.0.29:4317`
|
||||
- Line 169 (`KubeletConfiguration.tracing.endpoint` in the kubeadm config): `192.168.0.30:4317` → `192.168.0.29:4317`
|
||||
- **Do NOT touch line 8** (`zot_registry_ip = "192.168.0.30"`) — that's the Zot container-registry mirror, not tracing.
|
||||
|
||||
### 2. Runbook docs
|
||||
|
||||
**[docs/kubernetes-tracing.md](../kubernetes-tracing.md)** — replace tracing-related references throughout (≈10 lines):
|
||||
- All occurrences of `192.168.0.30:4317` → `192.168.0.29:4317` (lines 54, 83, 99)
|
||||
- `http://192.168.0.30:16686` → `http://192.168.0.29:16686` (line 150)
|
||||
- SSH targets `novakj@192.168.0.30` → `novakj@192.168.0.29` (lines 25, 31, 43)
|
||||
- Remote directory `~/docker-30/tracing` → `~/docker-29/tracing` (lines 25, 31, 43)
|
||||
- Prose "Jaeger on docker-30" → "Jaeger on docker-29" (lines 3, 40)
|
||||
|
||||
### 3. Plan files (historical record — also requested)
|
||||
|
||||
**[plans/2026-05-21 20:15 - k8s-pod-creation-tracing.md](../../plans/2026-05-21%2020%3A15%20-%20k8s-pod-creation-tracing.md)** — ~12 references:
|
||||
- `192.168.0.30:4317` (lines 38, 76, 101, 111) and `http://192.168.0.30:16686` (line 141) → `.29`
|
||||
- `docker-30/tracing/` directory path (lines 13, 16, 117, 128, 155) → `docker-29/tracing/`
|
||||
- Prose mentions of "docker-30" in tracing context (lines 5, 11) → "docker-29"
|
||||
|
||||
**[docs/plans/2026-05-21-2035-tracing-toggle-runbook.md](2026-05-21-2035-tracing-toggle-runbook.md)** — ~6 references:
|
||||
- `192.168.0.30:4317` (lines 51, 73) and `http://192.168.0.30:16686` (line 110) → `.29`
|
||||
- SSH target `novakj@192.168.0.30` (line 35) → `novakj@192.168.0.29`
|
||||
- Prose mentions of "docker-30" in tracing context (lines 5, 42) → "docker-29"
|
||||
|
||||
### Search safety
|
||||
|
||||
Use a per-file targeted replace with surrounding context (e.g. `endpoint: 192.168.0.30:4317`, `novakj@192.168.0.30`, `docker-30/tracing`), not a blanket repo-wide `sed`. The leave-alone list is large (Zot, Vault, Gitea, Kanidm, nginx, gitignore, claude settings, generated tfstate, commented-out blackbox/prometheus examples) — see Phase 1 exploration for the full inventory.
|
||||
|
||||
## Verification
|
||||
|
||||
### In-repo
|
||||
```bash
|
||||
# Confirm no tracing-related .30 references remain
|
||||
grep -nE '192\.168\.0\.30:(4317|16686)' -r .
|
||||
grep -nE 'docker-30/tracing' -r .
|
||||
# Confirm new IP is in place
|
||||
grep -nE '192\.168\.0\.29:4317' kubernetes-kvm-terraform/
|
||||
```
|
||||
Both `grep` calls in step 1 should return empty. The leave-alone Zot/Vault/Gitea/nginx hits at port 3000/8200/9443/etc. should still be present (sanity check that the surgical replace didn't over-match).
|
||||
|
||||
Re-run `tofu plan` in [kubernetes-kvm-terraform/](../../kubernetes-kvm-terraform/) to confirm the diff only shows the two endpoint changes and no unrelated drift.
|
||||
|
||||
### Live cluster follow-up (out of scope for this plan's execution, documented for the operator)
|
||||
|
||||
The in-repo changes only affect newly-bootstrapped nodes. The existing master + workers still have the old endpoint baked into their live files. After merging the repo changes, run:
|
||||
|
||||
1. **kube-apiserver** (master, 192.168.0.31):
|
||||
```bash
|
||||
ssh novakj@192.168.0.31 'sudo sed -i s/192.168.0.30:4317/192.168.0.29:4317/ /etc/kubernetes/tracing-config.yaml'
|
||||
# kubelet auto-restarts the static pod within ~10s
|
||||
```
|
||||
2. **kubelet** on each node (192.168.0.31, .32, .33), serially:
|
||||
```bash
|
||||
ssh novakj@$NODE 'sudo sed -i s/192.168.0.30:4317/192.168.0.29:4317/ /var/lib/kubelet/config.yaml && sudo systemctl restart kubelet'
|
||||
kubectl wait node --for=condition=Ready --timeout=60s <node-name>
|
||||
```
|
||||
3. **Cluster ConfigMap** so future joins/reboots pick it up:
|
||||
```bash
|
||||
kubectl -n kube-system edit cm kubelet-config
|
||||
# update the tracing.endpoint value under kubelet:
|
||||
```
|
||||
4. **End-to-end smoke test** (from [docs/kubernetes-tracing.md](../kubernetes-tracing.md) "Verification" section):
|
||||
```bash
|
||||
kubectl run trace-probe --image=registry.k8s.io/pause:3.10 --restart=Never
|
||||
kubectl wait --for=condition=Ready pod/trace-probe --timeout=60s
|
||||
kubectl delete pod trace-probe
|
||||
```
|
||||
Then open `http://192.168.0.29:16686` and confirm `apiserver` and `kubelet` services show recent spans for `trace-probe`.
|
||||
@@ -1,165 +0,0 @@
|
||||
# Deploy Mimir on docker-30 (mirror the Loki setup)
|
||||
|
||||
## Context
|
||||
|
||||
Loki is already deployed on the docker host (`192.168.0.30`) under
|
||||
`vms-home/docker-30/loki/`, storing chunks in Garage S3 and queried through a
|
||||
shared Grafana. The user wants Mimir deployed for **metrics** in the same style:
|
||||
self-contained compose dir, Garage S3 backend, `.env`-supplied credentials,
|
||||
env-expanded config.
|
||||
|
||||
Decisions made:
|
||||
- **Mimir only for now.** No producer is wired up in this plan; Mimir simply
|
||||
exposes its remote_write endpoint (`http://192.168.0.30:9009/api/v1/push`) for
|
||||
a Prometheus/Alloy to point at later.
|
||||
- **Reuse the existing Grafana** (the one in the Loki compose) by adding a
|
||||
Prometheus-type datasource — no second Grafana.
|
||||
|
||||
Mimir runs in **monolithic mode** (`-target=all`, single binary), which is the
|
||||
right shape for a single-node home lab and matches Loki's single-process model.
|
||||
|
||||
## Reference: how Loki does it (the pattern to copy)
|
||||
|
||||
`vms-home/docker-30/loki/loki-config.yaml` + `docker-compose.yaml`:
|
||||
- Garage S3 at `192.168.0.30:3900`, `region: garage`, `s3forcepathstyle: true`,
|
||||
`insecure: true`, bucket `loki-chunks`.
|
||||
- Creds via `${GARAGE_ACCESS_KEY}` / `${GARAGE_SECRET_KEY}` from `.env`,
|
||||
expanded with `-config.expand-env=true`.
|
||||
- `obs` bridge network; Grafana published on `3001`.
|
||||
|
||||
## Files to create (new dir `vms-home/docker-30/mimir/`)
|
||||
|
||||
### 1. `mimir-config.yaml` — monolithic + Garage S3
|
||||
|
||||
```yaml
|
||||
multitenancy_enabled: false # single-tenant; Loki mirrors this (auth_enabled:false)
|
||||
|
||||
server:
|
||||
http_listen_port: 9009
|
||||
grpc_listen_port: 9095
|
||||
log_level: info
|
||||
|
||||
common:
|
||||
storage:
|
||||
backend: s3
|
||||
s3:
|
||||
endpoint: 192.168.0.30:3900 # no scheme; insecure toggles http
|
||||
region: garage # MUST byte-match s3_region in garage.toml
|
||||
access_key_id: ${GARAGE_ACCESS_KEY}
|
||||
secret_access_key: ${GARAGE_SECRET_KEY}
|
||||
insecure: true # http on LAN
|
||||
bucket_lookup_type: path # path-style — Garage requirement (== s3forcepathstyle)
|
||||
|
||||
blocks_storage:
|
||||
s3: { bucket_name: mimir-blocks }
|
||||
tsdb: { dir: /data/tsdb }
|
||||
bucket_store:{ sync_dir: /data/tsdb-sync }
|
||||
|
||||
ruler_storage:
|
||||
s3: { bucket_name: mimir-ruler }
|
||||
|
||||
alertmanager_storage:
|
||||
s3: { bucket_name: mimir-alertmanager }
|
||||
|
||||
compactor:
|
||||
data_dir: /data/compactor
|
||||
|
||||
ruler:
|
||||
rule_path: /data/ruler
|
||||
|
||||
alertmanager:
|
||||
data_dir: /data/alertmanager
|
||||
|
||||
limits:
|
||||
compactor_blocks_retention_period: 744h # 31d, matches Loki's retention
|
||||
ingestion_rate: 50000 # samples/s per tenant; bump on 429s
|
||||
ingestion_burst_size: 100000
|
||||
```
|
||||
|
||||
Note: the three storage components inherit endpoint/creds from `common.storage`;
|
||||
only `bucket_name` is overridden per component. Buckets can be collapsed to one
|
||||
if preferred, but three is cleaner and Garage bucket creation is cheap.
|
||||
|
||||
### 2. `docker-compose.yaml` — copy Loki's, swap image/ports/paths
|
||||
|
||||
```yaml
|
||||
services:
|
||||
mimir:
|
||||
image: grafana/mimir:2.16.1 # pin latest stable — verify tag before apply
|
||||
container_name: mimir
|
||||
restart: unless-stopped
|
||||
command:
|
||||
- -config.file=/etc/mimir/config.yaml
|
||||
- -config.expand-env=true # REQUIRED — expands ${GARAGE_*}
|
||||
- -target=all # monolithic single-binary mode
|
||||
ports:
|
||||
- "9009:9009" # push (/api/v1/push) + query (/prometheus)
|
||||
volumes:
|
||||
- ./mimir-config.yaml:/etc/mimir/config.yaml:ro
|
||||
- ./mimir-data:/data
|
||||
env_file: [.env]
|
||||
networks: [obs]
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -qO- http://localhost:9009/ready || exit 1"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
start_period: 30s
|
||||
logging:
|
||||
driver: json-file
|
||||
options: { max-size: "10m", max-file: "3" }
|
||||
|
||||
networks:
|
||||
obs: { driver: bridge }
|
||||
```
|
||||
|
||||
### 3. `.env` — reuse the same Garage credentials as Loki
|
||||
|
||||
```
|
||||
GARAGE_ACCESS_KEY=<same key id Loki uses>
|
||||
GARAGE_SECRET_KEY=<same secret>
|
||||
```
|
||||
|
||||
## Manual steps on docker-30 (execution)
|
||||
|
||||
1. **Create Garage buckets** and grant the existing Loki key access:
|
||||
```bash
|
||||
ssh novakj@192.168.0.30
|
||||
for b in mimir-blocks mimir-ruler mimir-alertmanager; do
|
||||
docker exec garage /garage bucket create "$b"
|
||||
done
|
||||
docker exec garage /garage key list # get the key name Loki already uses
|
||||
for b in mimir-blocks mimir-ruler mimir-alertmanager; do
|
||||
docker exec garage /garage bucket allow --read --write "$b" --key <keyname>
|
||||
done
|
||||
```
|
||||
2. Copy the `mimir/` dir to the host (or `git pull` if this repo is checked out
|
||||
there), fill `.env`, then `cd mimir && docker compose up -d`.
|
||||
3. **Add Mimir datasource to the existing Grafana** (`:3001`):
|
||||
- Type: **Prometheus**
|
||||
- URL: `http://192.168.0.30:9009/prometheus`
|
||||
- No `X-Scope-OrgID` header needed (multitenancy disabled).
|
||||
|
||||
## Verification
|
||||
|
||||
- `curl http://192.168.0.30:9009/ready` → `ready` (also gates the healthcheck).
|
||||
- `docker compose ps` shows `mimir` healthy; `docker compose logs -f mimir`
|
||||
clean (no S3 auth / bucket errors — confirms Garage wiring).
|
||||
- Smoke-test the write path without a producer:
|
||||
```bash
|
||||
curl http://192.168.0.30:9009/api/v1/push # expect 4xx (proto body), NOT conn-refused
|
||||
```
|
||||
- In Grafana → Explore → Mimir datasource, run `up` or
|
||||
`count({__name__!=""})`. Empty until a producer remote_writes — expected.
|
||||
- Confirm blocks land in Garage after ~2h (first block flush):
|
||||
`docker exec garage /garage bucket info mimir-blocks`.
|
||||
|
||||
## Later (out of scope here): wiring a producer
|
||||
|
||||
Point any Prometheus/Alloy at:
|
||||
```yaml
|
||||
remote_write:
|
||||
- url: http://192.168.0.30:9009/api/v1/push
|
||||
```
|
||||
The existing `vms/utility-101-shadow/docker/monitoring/prometheus.yml` is the
|
||||
natural first candidate.
|
||||
@@ -1,116 +0,0 @@
|
||||
# Alloy → Mimir: scrape Kubernetes metrics + Prometheus Operator monitors
|
||||
|
||||
> On execution, also copy this file into the repo per CLAUDE.md convention:
|
||||
> `docs/plans/2026-07-08-HHMM-alloy-metrics-to-mimir.md` (get `HHMM` via `date "+%H%M"`).
|
||||
|
||||
## Context
|
||||
|
||||
Mimir is now up on docker-30 (`http://192.168.0.30:9009`, single-tenant, plain
|
||||
HTTP over LAN) but nothing in the cluster pushes metrics to it. Grafana Alloy
|
||||
already exists in the repo but is **logs-only** (ships pod/journal logs + k8s
|
||||
events to Loki) and — importantly — the entire `gitops/home-kubernetes/alloy/`
|
||||
directory is **not wired into Flux** yet.
|
||||
|
||||
kube-prometheus-stack is deployed and active: it provides the Prometheus
|
||||
Operator CRDs (ServiceMonitor / PodMonitor), kube-state-metrics, node-exporter,
|
||||
and ServiceMonitors for all standard targets (kubelet, cAdvisor, apiserver,
|
||||
coredns, etc.). Its bundled Prometheus scrapes everything into a local 60d TSDB
|
||||
but does **not** remote_write anywhere.
|
||||
|
||||
**Goal:** add a dedicated `alloy-metrics` instance that scrapes standard
|
||||
Kubernetes metrics and reads ServiceMonitors/PodMonitors from the Prometheus
|
||||
Operator, remote-writing to Mimir — additive alongside the existing Prometheus
|
||||
(no changes to kube-prometheus-stack). Because kube-prometheus-stack already
|
||||
ships ServiceMonitors for every standard target, a single
|
||||
`prometheus.operator.servicemonitors` + `prometheus.operator.podmonitors` pair
|
||||
satisfies **both** requirements at once. As part of wiring the `alloy/` dir into
|
||||
Flux, the existing `alloy-logs` and `alloy-events` releases also come under
|
||||
GitOps management (confirmed desired).
|
||||
|
||||
**Decisions confirmed:** additive/coexist with bundled Prometheus; activate all
|
||||
three Alloy instances (logs, events, metrics).
|
||||
|
||||
## Design
|
||||
|
||||
- Metrics scraping runs as a **single Deployment** (`controller.type: deployment`,
|
||||
1 replica) — never a DaemonSet, or every cluster-wide target would be scraped
|
||||
once per node. This is why it must be a separate release from `alloy-logs`
|
||||
(which is a DaemonSet).
|
||||
- Mirror the conventions already in `helmrelease_alloy-logs.yaml`: chart `alloy`
|
||||
`>=1.8.0 <2.0.0`, image `tag: v1.17.1`, `rbac.create: false`,
|
||||
`alloy.enableReporting: false`, `driftDetection: enabled`, install/upgrade
|
||||
`retries: 3`, `interval: 30m`, namespace `monitoring`.
|
||||
- River config = three components:
|
||||
1. `prometheus.operator.servicemonitors "sm"` — discovers/scrapes **all**
|
||||
ServiceMonitors cluster-wide (covers kubelet, cAdvisor, kube-state-metrics,
|
||||
node-exporter, apiserver, coredns, controller-manager, scheduler — i.e. the
|
||||
"standard kubernetes metrics").
|
||||
2. `prometheus.operator.podmonitors "pm"` — discovers/scrapes all PodMonitors.
|
||||
3. `prometheus.remote_write "mimir"` → `http://192.168.0.30:9009/api/v1/push`,
|
||||
`external_labels = { cluster = "homelab" }` (matches the logs config).
|
||||
Both operator components `forward_to = [prometheus.remote_write.mimir.receiver]`.
|
||||
- Reuse `cluster = "homelab"` external label for consistency with the Loki side.
|
||||
|
||||
## Files
|
||||
|
||||
### New — `gitops/home-kubernetes/alloy/helmrelease_alloy-metrics.yaml`
|
||||
HelmRelease `alloy-metrics` (ns `monitoring`), Deployment, serviceAccount
|
||||
`alloy-metrics`, inline `alloy.configMap.content` with the three components
|
||||
above. Modest resources (e.g. requests cpu 100m / mem 256Mi, limit mem 768Mi —
|
||||
operator discovery + scrape buffers use more than the log shipper).
|
||||
|
||||
### New — `gitops/home-kubernetes/alloy/rbac_metrics-collector.yaml`
|
||||
The `prometheus.operator.*` components need read access beyond the existing
|
||||
`alloy-log-reader` role. ClusterRole `alloy-metrics-reader` + ClusterRoleBinding
|
||||
to SA `alloy-metrics` (monitoring), granting:
|
||||
- core: `namespaces, nodes, nodes/metrics, nodes/proxy, services, endpoints, pods` — get/list/watch
|
||||
- `discovery.k8s.io`: `endpointslices` — get/list/watch
|
||||
- `networking.k8s.io`: `ingresses` — get/list/watch
|
||||
- `monitoring.coreos.com`: `servicemonitors, podmonitors, probes, scrapeconfigs` — get/list/watch
|
||||
- nonResourceURLs `/metrics`, `/metrics/cadvisor` — get (kubelet scraping)
|
||||
|
||||
(Kept as a separate file so the metrics RBAC is self-contained; the existing
|
||||
`rbac_log-collector.yaml` stays untouched.)
|
||||
|
||||
### New — `gitops/home-kubernetes/alloy/kustomization.yaml`
|
||||
`kind: Kustomization` listing all manifests in the dir: `helmrepository_grafana.yaml`,
|
||||
`helmrelease_alloy-logs.yaml`, `helmrelease_allow-events.yaml`,
|
||||
`helmrelease_alloy-metrics.yaml`, `rbac_log-collector.yaml`,
|
||||
`rbac_metrics-collector.yaml`. Makes the Flux build explicit/deterministic.
|
||||
|
||||
### Modified — `gitops/home-kubernetes/flux-system/extra-kustomizations.yaml`
|
||||
Append an `alloy` Flux Kustomization:
|
||||
```yaml
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: alloy
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/alloy
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
dependsOn:
|
||||
- name: kube-prometheus # needs the monitoring ns + operator CRDs/ServiceMonitors
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
1. Commit + push; force reconcile: `flux reconcile kustomization flux-system --with-source`.
|
||||
2. `kubectl get kustomizations -A` — `alloy` Ready=True.
|
||||
3. `kubectl get helmreleases -n monitoring` — `alloy-metrics` (+ logs/events) Ready.
|
||||
4. `kubectl -n monitoring get pods -l app.kubernetes.io/instance=alloy-metrics`
|
||||
Running; `kubectl -n monitoring logs deploy/alloy-metrics` shows no RBAC
|
||||
`forbidden` errors and targets discovered.
|
||||
5. Alloy UI: `kubectl -n monitoring port-forward deploy/alloy-metrics 12345:12345`
|
||||
→ open `http://localhost:12345`; confirm the two operator components list
|
||||
healthy targets and `prometheus.remote_write.mimir` shows successful sends.
|
||||
6. Confirm Mimir is receiving: query it directly, e.g.
|
||||
`curl -s 'http://192.168.0.30:9009/prometheus/api/v1/query?query=up' | jq '.data.result | length'`
|
||||
returns > 0 (or check the Grafana Mimir datasource for `up`,
|
||||
`kube_pod_info`, `node_cpu_seconds_total`, `container_cpu_usage_seconds_total`).
|
||||
7. Sanity: a `cluster="homelab"` label is present on the ingested series.
|
||||
@@ -1,391 +0,0 @@
|
||||
# E2B Dev VM Setup on KVM Homelab
|
||||
|
||||
## Context
|
||||
|
||||
The user wants to run the E2B infrastructure dev stack on a KVM virtual machine in their homelab. E2B uses Firecracker microVMs (which need `/dev/kvm`), so the guest VM needs **nested virtualization** (KVM-in-KVM). This guide covers VM creation, OS configuration, toolchain installation, and running the full dev stack.
|
||||
|
||||
> **Note from upstream**: [DEV-LOCAL.md](DEV-LOCAL.md) says "Linux is required. This is a work in progress. Not everything will function as expected."
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Create the KVM VM on the Hypervisor Host
|
||||
|
||||
### 1.1 Enable nested virtualization on the host
|
||||
|
||||
**Intel:**
|
||||
|
||||
```bash
|
||||
cat /sys/module/kvm_intel/parameters/nested # check
|
||||
sudo modprobe -r kvm_intel && sudo modprobe kvm_intel nested=1
|
||||
echo "options kvm_intel nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf
|
||||
```
|
||||
|
||||
**AMD:**
|
||||
|
||||
```bash
|
||||
cat /sys/module/kvm_amd/parameters/nested
|
||||
sudo modprobe -r kvm_amd && sudo modprobe kvm_amd nested=1
|
||||
echo "options kvm_amd nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf
|
||||
|
||||
# get details about loaded kernel module
|
||||
systool -v -m kvm_amd
|
||||
```
|
||||
|
||||
### 1.2 Create the VM
|
||||
|
||||
```bash
|
||||
virt-install \
|
||||
--name e2b-dev \
|
||||
--ram 16384 \
|
||||
--vcpus 8 \
|
||||
--cpu host-passthrough \
|
||||
--os-variant ubuntu24.04 \
|
||||
--disk path=/var/lib/libvirt/images/e2b-dev.qcow2,size=100,format=qcow2,bus=virtio \
|
||||
--network bridge=virbr0,model=virtio \
|
||||
--graphics none \
|
||||
--console pty,target_type=serial \
|
||||
--cdrom /path/to/ubuntu-24.04-live-server-amd64.iso \
|
||||
--extra-args 'console=ttyS0,115200n8'
|
||||
```
|
||||
|
||||
**Why these specs:**
|
||||
| Resource | Value | Rationale |
|
||||
|----------|-------|-----------|
|
||||
| RAM | 16 GB (24-32 better) | 4 GB for huge pages (2048 x 2MB), ~4 GB for 10 Docker containers, rest for Go services + Firecracker VMs |
|
||||
| vCPUs | 8 | Firecracker VMs consume vCPUs; Go services are concurrent |
|
||||
| Disk | 100 GB | Docker images, FC binaries, kernels, rootfs, Go cache, build artifacts |
|
||||
| CPU | `host-passthrough` | **Mandatory** -- exposes VMX/SVM to guest so `/dev/kvm` works inside the VM |
|
||||
|
||||
### 1.3 Alternative: libvirt XML
|
||||
|
||||
If you manage VMs declaratively, the critical part is:
|
||||
|
||||
```xml
|
||||
<cpu mode='host-passthrough' check='none' migratable='off'/>
|
||||
```
|
||||
|
||||
### 1.4 Verify nested KVM works (after OS install)
|
||||
|
||||
```bash
|
||||
ls -la /dev/kvm # must exist
|
||||
lsmod | grep kvm # kvm + kvm_intel/kvm_amd
|
||||
grep -cE '(vmx|svm)' /proc/cpuinfo # must be > 0
|
||||
```
|
||||
|
||||
If `/dev/kvm` is missing, go back to 1.1.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: OS Configuration (inside the guest VM)
|
||||
|
||||
**Recommended OS:** Ubuntu 24.04 LTS Server (matches CI, well-tested with Firecracker)
|
||||
|
||||
### 2.1 Base packages
|
||||
|
||||
```bash
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
sudo apt install -y \
|
||||
build-essential git curl wget unzip jq make gcc pkg-config \
|
||||
iptables iproute2 net-tools ca-certificates gnupg \
|
||||
lsb-release software-properties-common gettext-base
|
||||
```
|
||||
|
||||
### 2.2 Kernel modules
|
||||
|
||||
```bash
|
||||
# Load now
|
||||
sudo modprobe nbd nbds_max=64
|
||||
sudo modprobe kvm
|
||||
sudo modprobe kvm_intel # or kvm_amd
|
||||
sudo modprobe tun
|
||||
sudo modprobe veth
|
||||
sudo modprobe nf_tables
|
||||
sudo modprobe nft_nat
|
||||
|
||||
# Persist across reboots
|
||||
cat <<'EOF' | sudo tee /etc/modules-load.d/e2b.conf
|
||||
nbd
|
||||
kvm
|
||||
kvm_intel
|
||||
tun
|
||||
veth
|
||||
nf_tables
|
||||
nft_nat
|
||||
EOF
|
||||
|
||||
echo "options nbd nbds_max=64" | sudo tee /etc/modprobe.d/nbd.conf
|
||||
```
|
||||
|
||||
### 2.3 Sysctl
|
||||
|
||||
```bash
|
||||
cat <<'EOF' | sudo tee /etc/sysctl.d/99-e2b.conf
|
||||
vm.nr_hugepages=2048
|
||||
vm.max_map_count=1048576
|
||||
vm.swappiness=10
|
||||
vm.vfs_cache_pressure=50
|
||||
net.ipv4.ip_forward=1
|
||||
net.core.somaxconn=65535
|
||||
net.core.netdev_max_backlog=65535
|
||||
net.ipv4.tcp_max_syn_backlog=65535
|
||||
EOF
|
||||
|
||||
sudo sysctl --system
|
||||
```
|
||||
|
||||
### 2.4 Udev rules (suppress NBD inotify noise)
|
||||
|
||||
```bash
|
||||
cat <<'EOF' | sudo tee /etc/udev/rules.d/99-e2b-nbd.rules
|
||||
KERNEL=="nbd*", OPTIONS+="nowatch"
|
||||
EOF
|
||||
sudo udevadm control --reload-rules && sudo udevadm trigger
|
||||
```
|
||||
|
||||
### 2.5 File descriptor limits
|
||||
|
||||
```bash
|
||||
cat <<'EOF' | sudo tee /etc/security/limits.d/e2b.conf
|
||||
* soft nofile 1048576
|
||||
* hard nofile 1048576
|
||||
root soft nofile 1048576
|
||||
root hard nofile 1048576
|
||||
EOF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Install Toolchain
|
||||
|
||||
### 3.1 Docker
|
||||
|
||||
```bash
|
||||
curl -fsSL https://get.docker.com | sh
|
||||
sudo usermod -aG docker $USER
|
||||
# Log out and back in
|
||||
docker --version && docker compose version
|
||||
```
|
||||
|
||||
### 3.2 mise (manages all tools from `.tool-versions`)
|
||||
|
||||
```bash
|
||||
curl https://mise.run | sh
|
||||
echo 'eval "$(~/.local/bin/mise activate bash)"' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
### 3.3 Install project tools
|
||||
|
||||
The repo's [.tool-versions](.tool-versions) pins:
|
||||
|
||||
| Tool | Version |
|
||||
| --- | --- |
|
||||
| golang | 1.25.4 |
|
||||
| buf | 1.28.1 |
|
||||
| bun | 1.3.2 |
|
||||
| protoc | 29.3 |
|
||||
| protoc-gen-go | 1.28.1 |
|
||||
| protoc-gen-go-grpc | 1.6.1 |
|
||||
| protoc-gen-connect-go | 1.18.1 |
|
||||
| golangci-lint | 2.8.0 |
|
||||
| terraform | 1.5.7 |
|
||||
| packer | 1.13.1 |
|
||||
| python | 3.13.11 |
|
||||
| gcloud | 534.0.0 |
|
||||
|
||||
```bash
|
||||
cd ~/e2b-infra # after cloning
|
||||
mise install # installs everything from .tool-versions
|
||||
```
|
||||
|
||||
> **Note:** `gcloud` is only needed for `gsutil` to download prebuilt artifacts. If you don't have GCP credentials, you can download them via HTTPS instead (see Phase 5).
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Clone Repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/e2b-dev/infra.git ~/e2b-infra
|
||||
cd ~/e2b-infra
|
||||
go work sync
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Download Prebuilt Artifacts
|
||||
|
||||
Firecracker binaries and Linux kernels must be downloaded from the public GCS bucket.
|
||||
|
||||
**With gsutil (if gcloud is configured):**
|
||||
|
||||
```bash
|
||||
make download-public-kernels
|
||||
make download-public-firecrackers
|
||||
```
|
||||
|
||||
**Without GCP credentials (HTTPS alternative):**
|
||||
|
||||
The bucket `e2b-prod-public-builds` is publicly accessible. You'll need to browse/list it to find available versions, then download manually:
|
||||
|
||||
```bash
|
||||
# Install gsutil standalone (no GCP project needed for public buckets)
|
||||
# OR use curl/wget against:
|
||||
# https://storage.googleapis.com/e2b-prod-public-builds/
|
||||
# to discover and download kernel and firecracker builds
|
||||
```
|
||||
|
||||
The simplest path is to install just the `gcloud` CLI (via mise) and run the make targets -- no GCP project or auth is needed for public bucket reads.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Prepare Local Environment
|
||||
|
||||
### 6.1 Start infrastructure containers
|
||||
|
||||
```bash
|
||||
make local-infra
|
||||
# Starts: PostgreSQL 17.4, Redis 7.4.2, ClickHouse 25.4.5.24,
|
||||
# Grafana 12.0.0, Loki 3.4.1, Tempo 2.8.2, Mimir 2.17.1,
|
||||
# OTEL Collector 0.146.0, Vector, Memcached 1.6.38
|
||||
```
|
||||
|
||||
Wait for all containers to be healthy:
|
||||
|
||||
```bash
|
||||
docker compose -f packages/local-dev/docker-compose.yaml ps
|
||||
```
|
||||
|
||||
### 6.2 Initialize databases
|
||||
|
||||
```bash
|
||||
make -C packages/db migrate-local # PostgreSQL
|
||||
make -C packages/clickhouse migrate-local # ClickHouse
|
||||
```
|
||||
|
||||
### 6.3 Build envd (in-VM daemon)
|
||||
|
||||
```bash
|
||||
make -C packages/envd build
|
||||
```
|
||||
|
||||
### 6.4 Seed database with dev credentials
|
||||
|
||||
```bash
|
||||
make -C packages/local-dev seed-database
|
||||
```
|
||||
|
||||
Creates test user, team, API key, and access token for local development.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Run the Dev Stack
|
||||
|
||||
Each service runs in the foreground. Use **tmux**, **screen**, or separate SSH sessions.
|
||||
|
||||
| Terminal | Command | Listens on |
|
||||
| --- | --- | --- |
|
||||
| 1 | `make local-infra` | (Docker containers) |
|
||||
| 2 | `make -C packages/api run-local` | `:3000` |
|
||||
| 3 | `make -C packages/orchestrator build-debug && sudo make -C packages/orchestrator run-local` | `:5008` |
|
||||
| 4 | `make -C packages/client-proxy run-local` | `:3002` |
|
||||
|
||||
> The orchestrator **requires sudo** -- Firecracker needs root for `/dev/kvm`, network namespaces, veth pairs, nftables rules, and NBD devices.
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Verify
|
||||
|
||||
### 8.1 Health checks
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:3000/health # API
|
||||
curl -s -o /dev/null -w "%{http_code}" http://localhost:53000 # Grafana (expect 302)
|
||||
curl -s 'http://localhost:8123/?query=SELECT%201' # ClickHouse
|
||||
redis-cli -h localhost -p 6379 ping # Redis
|
||||
```
|
||||
|
||||
### 8.2 Build the base template
|
||||
|
||||
```bash
|
||||
make -C packages/shared/scripts local-build-base-template
|
||||
```
|
||||
|
||||
### 8.3 Test with E2B client SDK
|
||||
|
||||
```bash
|
||||
export E2B_API_KEY=e2b_53ae1fed82754c17ad8077fbc8bcdd90
|
||||
export E2B_ACCESS_TOKEN=sk_e2b_89215020937a4c989cde33d7bc647715
|
||||
export E2B_API_URL=http://localhost:3000
|
||||
export E2B_SANDBOX_URL=http://localhost:3002
|
||||
# Use E2B SDK/CLI to create a sandbox
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Access from Host (Optional)
|
||||
|
||||
### SSH port forwarding
|
||||
|
||||
```bash
|
||||
ssh -N \
|
||||
-L 3000:localhost:3000 \
|
||||
-L 3002:localhost:3002 \
|
||||
-L 53000:localhost:53000 \
|
||||
-L 5432:localhost:5432 \
|
||||
-L 8123:localhost:8123 \
|
||||
user@<vm-ip>
|
||||
```
|
||||
|
||||
### Or use bridged networking
|
||||
|
||||
If the VM has a routable IP on your LAN, services are directly accessible (Docker binds to `0.0.0.0`, Go services listen on all interfaces).
|
||||
|
||||
---
|
||||
|
||||
## Service Endpoints Reference
|
||||
|
||||
| Service | URL |
|
||||
| --- | --- |
|
||||
| E2B API | `http://localhost:3000` |
|
||||
| E2B Client Proxy | `http://localhost:3002` |
|
||||
| E2B Orchestrator | `http://localhost:5008` |
|
||||
| Grafana | `http://localhost:53000` |
|
||||
| PostgreSQL | `postgres://postgres:postgres@localhost:5432` |
|
||||
| ClickHouse (HTTP) | `http://localhost:8123` |
|
||||
| ClickHouse (native) | `localhost:9000` |
|
||||
| Redis | `localhost:6379` |
|
||||
| OTEL Collector (gRPC) | `localhost:4317` |
|
||||
| OTEL Collector (HTTP) | `localhost:4318` |
|
||||
| Loki | `http://localhost:3100` |
|
||||
| Vector | `localhost:30006` |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Fix |
|
||||
| --- | --- |
|
||||
| `/dev/kvm` missing in guest | Enable nested virt on host (Phase 1.1), use `host-passthrough` CPU |
|
||||
| `modprobe nbd` fails | Check kernel has NBD support: `modinfo nbd` |
|
||||
| Orchestrator permission errors | Must run with `sudo` |
|
||||
| Huge pages < 2048 | Not enough contiguous memory; increase VM RAM or set earlier in boot |
|
||||
| Docker containers won't start | Check `systemctl status docker`, port conflicts with `ss -tlnp` |
|
||||
| `gsutil` not found | Install via `mise install gcloud` or download artifacts via HTTPS |
|
||||
|
||||
---
|
||||
|
||||
## After VM Reboot Checklist
|
||||
|
||||
```bash
|
||||
# 1. Verify kernel modules and sysctl (should be persistent)
|
||||
lsmod | grep nbd
|
||||
cat /proc/sys/vm/nr_hugepages # expect 2048
|
||||
|
||||
# 2. Start infra
|
||||
cd ~/e2b-infra && make local-infra
|
||||
|
||||
# 3. Start services (separate terminals)
|
||||
make -C packages/api run-local
|
||||
sudo make -C packages/orchestrator run-local
|
||||
make -C packages/client-proxy run-local
|
||||
```
|
||||
@@ -1,95 +0,0 @@
|
||||
## dev-vm on beelink
|
||||
|
||||
```bash
|
||||
virt-install \
|
||||
--name e2b-dev \
|
||||
--ram 16384 \
|
||||
--vcpus 8 \
|
||||
--cpu host-passthrough \
|
||||
--os-variant ubuntu24.04 \
|
||||
--disk path=/srv/vms/e2b-dev.qcow2,size=100,format=qcow2,bus=virtio \
|
||||
--network bridge=br0,model=virtio \
|
||||
--graphics none \
|
||||
--console pty,target_type=serial \
|
||||
--location /srv/vms/isos/ubuntu-24.04.3-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
|
||||
--extra-args 'console=ttyS0,115200n8'
|
||||
|
||||
|
||||
# base packages
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
sudo apt install -y \
|
||||
build-essential git curl wget unzip jq make gcc pkg-config \
|
||||
iptables iproute2 net-tools ca-certificates gnupg \
|
||||
lsb-release software-properties-common gettext-base
|
||||
|
||||
|
||||
# kernel modules
|
||||
# Load now
|
||||
sudo modprobe nbd nbds_max=64
|
||||
sudo modprobe kvm
|
||||
sudo modprobe kvm_amd # or kvm_amd
|
||||
sudo modprobe tun
|
||||
sudo modprobe veth
|
||||
sudo modprobe nf_tables
|
||||
sudo modprobe nft_nat
|
||||
|
||||
# Persist across reboots
|
||||
cat <<'EOF' | sudo tee /etc/modules-load.d/e2b.conf
|
||||
nbd
|
||||
kvm
|
||||
kvm_amd
|
||||
tun
|
||||
veth
|
||||
nf_tables
|
||||
nft_nat
|
||||
EOF
|
||||
|
||||
echo "options nbd nbds_max=64" | sudo tee /etc/modprobe.d/nbd.conf
|
||||
|
||||
# sysctl
|
||||
cat <<'EOF' | sudo tee /etc/sysctl.d/99-e2b.conf
|
||||
vm.nr_hugepages=2048
|
||||
vm.max_map_count=1048576
|
||||
vm.swappiness=10
|
||||
vm.vfs_cache_pressure=50
|
||||
net.ipv4.ip_forward=1
|
||||
net.core.somaxconn=65535
|
||||
net.core.netdev_max_backlog=65535
|
||||
net.ipv4.tcp_max_syn_backlog=65535
|
||||
EOF
|
||||
|
||||
sudo sysctl --system
|
||||
|
||||
# udev rules
|
||||
cat <<'EOF' | sudo tee /etc/udev/rules.d/99-e2b-nbd.rules
|
||||
KERNEL=="nbd*", OPTIONS+="nowatch"
|
||||
EOF
|
||||
sudo udevadm control --reload-rules && sudo udevadm trigger
|
||||
|
||||
# file descriptor limits
|
||||
cat <<'EOF' | sudo tee /etc/security/limits.d/e2b.conf
|
||||
* soft nofile 1048576
|
||||
* hard nofile 1048576
|
||||
root soft nofile 1048576
|
||||
root hard nofile 1048576
|
||||
EOF
|
||||
```
|
||||
|
||||
## install toolchain
|
||||
|
||||
```bash
|
||||
# docker
|
||||
curl -fsSL https://get.docker.com | sh
|
||||
sudo usermod -aG docker $USER
|
||||
# Log out and back in
|
||||
docker --version && docker compose version
|
||||
|
||||
# mise
|
||||
curl https://mise.run | sh
|
||||
echo 'eval "$(~/.local/bin/mise activate bash)"' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
|
||||
```
|
||||
|
||||
@@ -1,280 +0,0 @@
|
||||
# E2B Sandbox Usage Guide
|
||||
|
||||
A practical guide for creating, observing, and using E2B sandboxes via the API.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Local Dev Setup
|
||||
|
||||
Seed the database to create a test team and API key:
|
||||
|
||||
```bash
|
||||
make -C packages/local-dev seed-database
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
E2B_API_KEY=e2b_53ae1fed82754c17ad8077fbc8bcdd90
|
||||
E2B_ACCESS_TOKEN=sk_e2b_89215020937a4c989cde33d7bc647715
|
||||
E2B_API_URL=http://localhost:3000
|
||||
E2B_SANDBOX_URL=http://localhost:3002
|
||||
```
|
||||
|
||||
All examples below use `$E2B_API_URL` and `$E2B_API_KEY` — export them in your shell for convenience:
|
||||
|
||||
```bash
|
||||
export E2B_API_URL=http://localhost:3000
|
||||
export E2B_API_KEY=e2b_53ae1fed82754c17ad8077fbc8bcdd90
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 1. Create a Sandbox
|
||||
|
||||
```bash
|
||||
curl -X POST $E2B_API_URL/sandboxes \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"templateID": "base",
|
||||
"timeout": 300
|
||||
}'
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"sandboxID": "i1234567890abcdef",
|
||||
"templateID": "base",
|
||||
"envdVersion": "0.5.8",
|
||||
"domain": "i1234567890abcdef.your-domain"
|
||||
}
|
||||
```
|
||||
|
||||
Save the `sandboxID` and `domain` from the response for subsequent commands.
|
||||
|
||||
### Optional Parameters
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `secure` | bool | Returns an `envdAccessToken` for authenticated envd access |
|
||||
| `envVars` | object | Inject environment variables into the VM |
|
||||
| `metadata` | object | Tag the sandbox for filtering (e.g. `{"purpose": "test"}`) |
|
||||
| `autoPause` | bool | Pause instead of kill on timeout |
|
||||
| `allow_internet_access` | bool | Allow internet egress from the VM |
|
||||
| `network.allowPublicTraffic` | bool | Allow inbound public traffic |
|
||||
| `network.allowOut` | array | Allowed egress destinations (IPs, CIDRs, domains) |
|
||||
| `network.denyOut` | array | Denied egress destinations (IPs/CIDRs only) |
|
||||
| `volumeMounts` | array | Mount persistent volumes (`{"name": "vol", "path": "/data"}`) |
|
||||
|
||||
**Example with options:**
|
||||
|
||||
```bash
|
||||
curl -X POST $E2B_API_URL/sandboxes \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"templateID": "base",
|
||||
"timeout": 300,
|
||||
"secure": true,
|
||||
"envVars": {"MY_VAR": "hello"},
|
||||
"metadata": {"purpose": "demo"}
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Observe Sandbox State
|
||||
|
||||
### Get details of a specific sandbox
|
||||
|
||||
```bash
|
||||
curl $E2B_API_URL/sandboxes/{sandboxID} \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
```
|
||||
|
||||
Returns state (`running`/`paused`), CPU/memory/disk config, network settings, and TTL (`endAt`).
|
||||
|
||||
### List all sandboxes
|
||||
|
||||
```bash
|
||||
# Running only
|
||||
curl "$E2B_API_URL/v2/sandboxes?state=running" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
|
||||
# Running and paused
|
||||
curl "$E2B_API_URL/v2/sandboxes?state=running&state=paused" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
|
||||
# Filter by metadata
|
||||
curl "$E2B_API_URL/v2/sandboxes?metadata=purpose%3Ddemo" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
```
|
||||
|
||||
### Get resource metrics (CPU, memory, disk)
|
||||
|
||||
```bash
|
||||
curl "$E2B_API_URL/sandboxes/{sandboxID}/metrics?start=$(date -v-5M +%s)&end=$(date +%s)" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"timestampUnix": 1234567890,
|
||||
"cpuCount": 2,
|
||||
"cpuUsedPct": 25.5,
|
||||
"memUsed": 268435456,
|
||||
"memTotal": 536870912,
|
||||
"diskUsed": 1073741824,
|
||||
"diskTotal": 5368709120
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Get sandbox logs
|
||||
|
||||
```bash
|
||||
curl "$E2B_API_URL/v2/sandboxes/{sandboxID}/logs?limit=100&direction=backward" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
```
|
||||
|
||||
Optional query parameters: `cursor` (ms timestamp), `level` (min log level), `search` (substring match).
|
||||
|
||||
---
|
||||
|
||||
## 3. Use the Sandbox
|
||||
|
||||
The in-VM daemon (**envd**) runs on port 49983 inside each sandbox, exposed via the sandbox's `domain`.
|
||||
|
||||
### Upload a file
|
||||
|
||||
```bash
|
||||
curl -X POST https://{domain}/files \
|
||||
-H "Content-Type: multipart/form-data" \
|
||||
-F "file=@script.py" \
|
||||
-F "path=/home/user/script.py"
|
||||
```
|
||||
|
||||
### Using the E2B Python SDK
|
||||
|
||||
Point the SDK at your local API:
|
||||
|
||||
```python
|
||||
from e2b import Sandbox
|
||||
|
||||
sbx = Sandbox("base", api_url="http://localhost:3000")
|
||||
|
||||
# Run a command
|
||||
result = sbx.commands.run("echo 'Hello from Firecracker VM!'")
|
||||
print(result.stdout)
|
||||
|
||||
# Write and execute a file
|
||||
sbx.files.write("/home/user/hello.py", "print('Hello world')")
|
||||
result = sbx.commands.run("python3 /home/user/hello.py")
|
||||
print(result.stdout)
|
||||
|
||||
# List files
|
||||
files = sbx.files.list("/home/user")
|
||||
for f in files:
|
||||
print(f.name)
|
||||
|
||||
sbx.kill()
|
||||
```
|
||||
|
||||
### Envd Connect RPC API
|
||||
|
||||
The envd daemon exposes Connect RPC services for programmatic access:
|
||||
|
||||
**Process Service:**
|
||||
- `Start(ProcessConfig)` — start a new process
|
||||
- `List()` — list running processes
|
||||
- `Connect(ProcessSelector)` — connect to process stdio
|
||||
- `Signal(ProcessSelector, Signal)` — send signal to process
|
||||
|
||||
**Filesystem Service:**
|
||||
- `ListDir(Path)` — list directory contents
|
||||
- `Stat(Path)` — get file metadata
|
||||
- `WatchDir(Path)` — watch for changes
|
||||
- `Move(Source, Dest)` — move/rename file
|
||||
- `RemoveDir(Path)` — remove directory
|
||||
|
||||
---
|
||||
|
||||
## 4. Lifecycle Management
|
||||
|
||||
### Extend timeout
|
||||
|
||||
```bash
|
||||
# Add 60 seconds to the TTL
|
||||
curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/refreshes \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"duration": 60}'
|
||||
|
||||
# Or set an absolute timeout (seconds from now)
|
||||
curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/timeout \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"timeout": 120}'
|
||||
```
|
||||
|
||||
### Pause (snapshot to disk)
|
||||
|
||||
```bash
|
||||
curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/pause \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
```
|
||||
|
||||
### Resume / reconnect
|
||||
|
||||
```bash
|
||||
curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/connect \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"timeout": 300}'
|
||||
```
|
||||
|
||||
Returns `200` if already running, `201` if resumed from paused state.
|
||||
|
||||
### Kill
|
||||
|
||||
```bash
|
||||
curl -X DELETE $E2B_API_URL/sandboxes/{sandboxID} \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
```
|
||||
|
||||
### Create a snapshot (template from running sandbox)
|
||||
|
||||
```bash
|
||||
curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/snapshots \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name": "my-snapshot"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Network Configuration
|
||||
|
||||
Update network rules on a running sandbox:
|
||||
|
||||
```bash
|
||||
curl -X PUT $E2B_API_URL/sandboxes/{sandboxID}/network \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"allowOut": ["8.8.8.8", "example.com"],
|
||||
"denyOut": ["0.0.0.0/0"]
|
||||
}'
|
||||
```
|
||||
|
||||
**Rules:**
|
||||
- If `allowOut` contains domain names, `denyOut` must include `0.0.0.0/0`
|
||||
- Domain names are not supported in `denyOut` (IPs/CIDRs only)
|
||||
- `allowOut` entries take precedence over `denyOut`
|
||||
- Omitting both fields clears all rules
|
||||
@@ -1,46 +0,0 @@
|
||||
## VMs
|
||||
|
||||
```bash
|
||||
|
||||
E2B_API_URL=http://192.168.0.61:3000
|
||||
|
||||
# create vm
|
||||
curl -X POST $E2B_API_URL/sandboxes \
|
||||
-H "X-API-Key: $E2B_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"templateID": "base",
|
||||
"timeout": 300
|
||||
}'
|
||||
|
||||
# response
|
||||
{"alias":"base","clientID":"6532622b","domain":null,"envdVersion":"0.5.8","sandboxID":"in3h60s6h0ie3kigrcls4","templateID":"pz1l1owhmy0w84e12eqv","trafficAccessToken":null}
|
||||
|
||||
sandboxID=in3h60s6h0ie3kigrcls4
|
||||
|
||||
# create sandboxVM and capture id
|
||||
sandboxID=$(curl -s -X POST http://localhost:3000/sandboxes -H "X-API-Key: $E2B_API_KEY" -H "Content-Type: application/json" -d '{
|
||||
"templateID": "base",
|
||||
"timeout": 300
|
||||
}' | jq -r .sandboxID
|
||||
)
|
||||
|
||||
|
||||
# get state
|
||||
curl http://localhost:3000/sandboxes/${sandboxID} \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
|
||||
# list running sandboxes
|
||||
curl "http://localhost:3000/v2/sandboxes?state=running" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
|
||||
# sandbox resource metrics
|
||||
curl "http://localhost:3000/sandboxes/{sandboxID}/metrics?start=$(date -v-5M +%s)&end=$(date +%s)" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
|
||||
# sandbox logs
|
||||
curl "http://localhost:3000/v2/sandboxes/{sandboxID}/logs?limit=100&direction=backward" \
|
||||
-H "X-API-Key: $E2B_API_KEY"
|
||||
|
||||
|
||||
```
|
||||
@@ -1,22 +0,0 @@
|
||||
# Disable the default all-in-one installation
|
||||
shims:
|
||||
disableAll: true
|
||||
firecracker:
|
||||
enabled: true
|
||||
|
||||
# In this chart version, 'env' must be a map, not a list.
|
||||
# The template converts these keys into Environment Variables for you.
|
||||
env:
|
||||
kataArtifacts: "firecracker"
|
||||
multiInstallSuffix: "" # This fixes the specific error you saw
|
||||
installArtifactsOnly: false
|
||||
|
||||
# Automatically create the RuntimeClass
|
||||
runtimeClass:
|
||||
create: true
|
||||
name: kata-fc
|
||||
handler: kata-fc
|
||||
|
||||
# Set Firecracker as the default for amd64 nodes
|
||||
defaultShim:
|
||||
amd64: firecracker
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: node.k8s.io/v1
|
||||
kind: RuntimeClass
|
||||
metadata:
|
||||
name: kata-fc
|
||||
handler: kata-fc
|
||||
scheduling:
|
||||
nodeSelector:
|
||||
# Ensure these pods only land on your specialized node
|
||||
runtime: kata-fc
|
||||
tolerations:
|
||||
- key: "dedicated"
|
||||
operator: "Equal"
|
||||
value: "kata-fc"
|
||||
effect: "NoSchedule"
|
||||
@@ -1,198 +0,0 @@
|
||||
## Node: kube-node-34
|
||||
|
||||
```bash
|
||||
apt install cpu-checker
|
||||
|
||||
# check kvm availability
|
||||
kvm-ok
|
||||
cat /sys/module/kvm_amd/parameters/nested
|
||||
lsmod | grep kvm
|
||||
|
||||
# install kata containers
|
||||
# using kata-deploy ?
|
||||
|
||||
export VERSION=$(curl -sSL https://api.github.com/repos/kata-containers/kata-containers/releases/latest | jq .tag_name | tr -d '"')
|
||||
export CHART="oci://ghcr.io/kata-containers/kata-deploy-charts/kata-deploy"
|
||||
|
||||
helm upgrade --install kata-deploy "${CHART}" --version "${VERSION}" --values experiments/kata_and_fc_on_kube/manifests/kata-deploy-values.yaml
|
||||
|
||||
# label only specific nodes to be used with kata-fc
|
||||
kubectl label node kube-node-34 runtime=kata-fc
|
||||
# taint node with NoSchedule
|
||||
kubectl taint nodes kube-node-34 dedicated=kata-fc:NoSchedule
|
||||
|
||||
# configure runtime class
|
||||
k apply -f experiments/kata_and_fc_on_kube/manifests//runtime-class_kata-fc.yaml
|
||||
|
||||
# devmapper configuration for firecracker in containerd
|
||||
ctr plugins ls | grep devmapper
|
||||
|
||||
# create data and metadata files
|
||||
# might want to replace this with LVM !!!!
|
||||
# sudo mkdir -p /var/lib/containerd/devmapper
|
||||
# sudo truncate -s 10G /var/lib/containerd/devmapper/data
|
||||
# sudo truncate -s 1G /var/lib/containerd/devmapper/init_metadata
|
||||
|
||||
# # Associate the files with Loop Devices
|
||||
# sudo losetup /dev/loop10 /var/lib/containerd/devmapper/data
|
||||
# sudo losetup /dev/loop11 /var/lib/containerd/devmapper/init_metadata
|
||||
|
||||
# # Use dmsetup to create the pool:
|
||||
# # This command creates the mapping.
|
||||
# # The numbers '0 20971520' represent the size in 512-byte sectors (for a 10GB file).
|
||||
# sudo dmsetup create containerd-pool --table "0 20971520 thin-pool /dev/loop11 /dev/loop10 128 32768 1"
|
||||
|
||||
lvcreate -L 10G -T kata-vg/kata-pool
|
||||
|
||||
# Add the configuration to /etc/containerd/config.toml:
|
||||
# Find the [plugins."io.containerd.snapshotter.v1.devmapper"] section and update it:
|
||||
|
||||
[plugins."io.containerd.snapshotter.v1.devmapper"]
|
||||
# LVM uses a specific naming convention in /dev/mapper/
|
||||
# It is VolumeGroupName-LogicalVolumeName
|
||||
pool_name = "kata--vg-kata--pool"
|
||||
root_path = "/var/lib/containerd/devmapper"
|
||||
base_image_size = "10GB"
|
||||
discard_blocks = true
|
||||
|
||||
|
||||
# restart containerd
|
||||
systemctl restart containerd
|
||||
|
||||
|
||||
|
||||
```
|
||||
|
||||
|
||||
### kube node preparation & deployment
|
||||
|
||||
```bash
|
||||
# ---------------------------------------------------------------------------
|
||||
# Packages
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
apt-get update
|
||||
apt-get install -y \
|
||||
qemu-guest-agent \
|
||||
openssh-server \
|
||||
apt-transport-https \
|
||||
ca-certificates \
|
||||
curl \
|
||||
gnupg \
|
||||
nvme-cli
|
||||
|
||||
systemctl enable --now qemu-guest-agent
|
||||
systemctl enable --now ssh
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# nvme-tcp
|
||||
# ---------------------------------------------------------------------------
|
||||
apt-get install -y linux-modules-extra-$(uname -r)
|
||||
modprobe nvme-tcp
|
||||
echo "nvme-tcp" >> /etc/modules-load.d/nvme-tcp.conf
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Kernel modules for Kubernetes
|
||||
# ---------------------------------------------------------------------------
|
||||
cat > /etc/modules-load.d/k8s.conf <<'EOF'
|
||||
overlay
|
||||
br_netfilter
|
||||
EOF
|
||||
|
||||
modprobe overlay
|
||||
modprobe br_netfilter
|
||||
|
||||
cat > /etc/sysctl.d/k8s.conf <<'EOF'
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.ipv4.ip_forward = 1
|
||||
EOF
|
||||
|
||||
sysctl --system
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# containerd
|
||||
# ---------------------------------------------------------------------------
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
||||
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" \
|
||||
> /etc/apt/sources.list.d/docker.list
|
||||
apt-get update && apt-get install -y containerd.io
|
||||
|
||||
cat > /etc/containerd/config.toml <<'EOF'
|
||||
version = 2
|
||||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
|
||||
runtime_type = "io.containerd.runc.v2"
|
||||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
|
||||
SystemdCgroup = true
|
||||
[plugins."io.containerd.grpc.v1.cri".registry]
|
||||
config_path = "/etc/containerd/certs.d"
|
||||
EOF
|
||||
|
||||
# Registry mirrors pointing to Zot at 192.168.0.30:5000
|
||||
mkdir -p \
|
||||
/etc/containerd/certs.d/docker.io \
|
||||
/etc/containerd/certs.d/registry.k8s.io \
|
||||
/etc/containerd/certs.d/ghcr.io \
|
||||
/etc/containerd/certs.d/quay.io
|
||||
|
||||
cat > /etc/containerd/certs.d/docker.io/hosts.toml <<'EOF'
|
||||
server = "https://registry-1.docker.io"
|
||||
[host."http://192.168.0.30:5000/v2/docker.io"]
|
||||
capabilities = ["pull", "resolve"]
|
||||
skip_verify = true
|
||||
override_path = true
|
||||
EOF
|
||||
|
||||
cat > /etc/containerd/certs.d/registry.k8s.io/hosts.toml <<'EOF'
|
||||
server = "https://registry.k8s.io"
|
||||
[host."http://192.168.0.30:5000/v2/registry.k8s.io"]
|
||||
capabilities = ["pull", "resolve"]
|
||||
skip_verify = true
|
||||
override_path = true
|
||||
EOF
|
||||
|
||||
cat > /etc/containerd/certs.d/ghcr.io/hosts.toml <<'EOF'
|
||||
server = "https://ghcr.io"
|
||||
[host."http://192.168.0.30:5000/v2/ghcr.io"]
|
||||
capabilities = ["pull", "resolve"]
|
||||
skip_verify = true
|
||||
override_path = true
|
||||
EOF
|
||||
|
||||
cat > /etc/containerd/certs.d/quay.io/hosts.toml <<'EOF'
|
||||
server = "https://quay.io"
|
||||
[host."http://192.168.0.30:5000/v2/quay.io"]
|
||||
capabilities = ["pull", "resolve"]
|
||||
skip_verify = true
|
||||
override_path = true
|
||||
EOF
|
||||
|
||||
systemctl restart containerd
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# kubelet systemd drop-in
|
||||
# ---------------------------------------------------------------------------
|
||||
mkdir -p /etc/systemd/system/kubelet.service.d
|
||||
cat > /etc/systemd/system/kubelet.service.d/10-containerd.conf <<'EOF'
|
||||
[Unit]
|
||||
After=containerd.service
|
||||
Requires=containerd.service
|
||||
|
||||
[Service]
|
||||
ExecStartPre=/bin/bash -c 'until [ -S /var/run/containerd/containerd.sock ]; do sleep 1; done'
|
||||
ExecStartPre=/usr/bin/crictl info
|
||||
EOF
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# kubeadm / kubelet / kubectl v1.32
|
||||
# ---------------------------------------------------------------------------
|
||||
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key \
|
||||
| gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /" \
|
||||
> /etc/apt/sources.list.d/kubernetes.list
|
||||
apt-get update && apt-get install -y kubelet kubeadm kubectl
|
||||
apt-mark hold kubelet kubeadm kubectl
|
||||
|
||||
|
||||
# join kube cluster
|
||||
JOIN_COMMAND="kubeadm join 192.168.0.31:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>"
|
||||
@@ -1,14 +0,0 @@
|
||||
```bash
|
||||
virt-install \
|
||||
--name microsandbox \
|
||||
--ram 8192\
|
||||
--vcpus 4 \
|
||||
--cpu host-passthrough \
|
||||
--os-variant ubuntu24.04 \
|
||||
--disk path=/srv/vms/microsandbox.qcow2,size=100,format=qcow2,bus=virtio \
|
||||
--network bridge=br0,model=virtio \
|
||||
--graphics none \
|
||||
--console pty,target_type=serial \
|
||||
--location /srv/vms/isos/ubuntu-24.04.3-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
|
||||
--extra-args 'console=ttyS0,115200n8'
|
||||
```
|
||||
@@ -1,27 +0,0 @@
|
||||
#cloud-config
|
||||
hostname: ${hostname}
|
||||
manage_etc_hosts: true
|
||||
|
||||
# Kernel modules for container networking
|
||||
write_files:
|
||||
- path: /etc/modules-load.d/k8s.conf
|
||||
content: |
|
||||
overlay
|
||||
br_netfilter
|
||||
- path: /etc/sysctl.d/k8s.conf
|
||||
content: |
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.ipv4.ip_forward = 1
|
||||
|
||||
packages:
|
||||
- curl
|
||||
- gpg
|
||||
- apt-transport-https
|
||||
|
||||
runcmd:
|
||||
- modprobe overlay
|
||||
- modprobe br_netfilter
|
||||
- sysctl --system
|
||||
- swapoff -a
|
||||
- sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
|
||||
@@ -1,74 +0,0 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
orbstack = {
|
||||
source = "robertdebock/orbstack"
|
||||
version = "~> 3.0"
|
||||
}
|
||||
cloudinit = {
|
||||
source = "hashicorp/cloudinit"
|
||||
version = "~> 2.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
variable "node_count" {
|
||||
description = "Number of nodes to deploy"
|
||||
type = number
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "name" {
|
||||
description = "Base name for the machines"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "distro" {
|
||||
description = "OS distribution"
|
||||
type = string
|
||||
default = "ubuntu"
|
||||
}
|
||||
|
||||
variable "distro_version" {
|
||||
description = "OS distribution version/codename"
|
||||
type = string
|
||||
default = "noble"
|
||||
}
|
||||
|
||||
variable "extra_cloud_init_parts" {
|
||||
description = "Additional cloud-init parts to layer on top of the base config"
|
||||
type = list(object({ content = string, content_type = string }))
|
||||
default = []
|
||||
}
|
||||
|
||||
data "cloudinit_config" "this" {
|
||||
count = var.node_count
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = templatefile("${path.module}/cloud-init-base.yaml", {
|
||||
hostname = "${var.name}-${count.index + 1}"
|
||||
})
|
||||
}
|
||||
|
||||
dynamic "part" {
|
||||
for_each = var.extra_cloud_init_parts
|
||||
content {
|
||||
content_type = part.value.content_type
|
||||
content = part.value.content
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "orbstack_machine" "this" {
|
||||
count = var.node_count
|
||||
|
||||
name = "${var.name}-${count.index + 1}"
|
||||
distro = var.distro
|
||||
region = var.distro_version
|
||||
|
||||
user_data = data.cloudinit_config.this[count.index].rendered
|
||||
}
|
||||
|
||||
output "machines" {
|
||||
value = { for m in orbstack_machine.this : m.name => m.ip_address }
|
||||
}
|
||||
@@ -1,11 +0,0 @@
|
||||
#cloud-config
|
||||
merge_how:
|
||||
- name: list
|
||||
settings: [append]
|
||||
- name: dict
|
||||
settings: [recurse_array]
|
||||
|
||||
runcmd:
|
||||
- curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" K3S_TOKEN="${k3s_token}" sh -
|
||||
- until kubectl get nodes; do sleep 2; done
|
||||
- cp /etc/rancher/k3s/k3s.yaml /root/kubeconfig.yaml
|
||||
@@ -1,9 +0,0 @@
|
||||
#cloud-config
|
||||
merge_how:
|
||||
- name: list
|
||||
settings: [append]
|
||||
- name: dict
|
||||
settings: [recurse_array]
|
||||
|
||||
runcmd:
|
||||
- curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent" K3S_URL="https://${cp_ip}:6443" K3S_TOKEN="${join_token}" sh -
|
||||
@@ -1,39 +0,0 @@
|
||||
variable "k3s_token" {
|
||||
description = "Shared secret for k3s cluster join (set via TF_VAR or tfvars)"
|
||||
type = string
|
||||
sensitive = true
|
||||
}
|
||||
|
||||
module "control_plane" {
|
||||
source = "../../modules/base-template"
|
||||
name = "k3s-cp"
|
||||
node_count = 1
|
||||
|
||||
extra_cloud_init_parts = [{
|
||||
content_type = "text/cloud-config"
|
||||
content = templatefile("${path.module}/cloud-init-cp.yaml", {
|
||||
k3s_token = var.k3s_token
|
||||
})
|
||||
}]
|
||||
}
|
||||
|
||||
module "workers" {
|
||||
source = "../../modules/base-template"
|
||||
name = "k3s-worker"
|
||||
node_count = 2
|
||||
|
||||
extra_cloud_init_parts = [{
|
||||
content_type = "text/cloud-config"
|
||||
content = templatefile("${path.module}/cloud-init-worker.yaml", {
|
||||
cp_ip = values(module.control_plane.machines)[0]
|
||||
join_token = var.k3s_token
|
||||
})
|
||||
}]
|
||||
}
|
||||
|
||||
output "cluster" {
|
||||
value = {
|
||||
control_plane = module.control_plane.machines
|
||||
workers = module.workers.machines
|
||||
}
|
||||
}
|
||||
@@ -1,219 +0,0 @@
|
||||
## install
|
||||
|
||||
```bash
|
||||
# master node
|
||||
curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=v1.32.12+rke2r1 sh -
|
||||
systemctl enable rke2-server.service
|
||||
systemctl start rke2-server.service
|
||||
journalctl -u rke2-server -f
|
||||
|
||||
# open firewalld
|
||||
sudo firewall-cmd --permanent --add-port=9345/tcp
|
||||
sudo firewall-cmd --permanent --add-port=6443/tcp
|
||||
sudo firewall-cmd --permanent --add-port=10250/tcp # Kubelet
|
||||
sudo firewall-cmd --reload
|
||||
|
||||
# install nerdctl
|
||||
# Set the version
|
||||
VERSION="2.2.1" # Check GitHub for the latest version
|
||||
|
||||
# Download the tarball
|
||||
wget https://github.com/containerd/nerdctl/releases/download/v${VERSION}/nerdctl-${VERSION}-linux-arm64.tar.gz
|
||||
|
||||
# Extract to your path
|
||||
sudo tar -C /usr/local/bin -xzvf nerdctl-${VERSION}-linux-arm64.tar.gz nerdctl
|
||||
|
||||
# configure nerdctl
|
||||
sudo mkdir -p /etc/nerdctl
|
||||
sudo tee /etc/nerdctl/nerdctl.toml <<EOF
|
||||
address = "unix:///run/k3s/containerd/containerd.sock"
|
||||
namespace = "k8s.io"
|
||||
EOF
|
||||
|
||||
# install buildkit
|
||||
# Set current stable version
|
||||
BK_VER="0.28.0"
|
||||
|
||||
# Download arm64 binary
|
||||
wget https://github.com/moby/buildkit/releases/download/v${BK_VER}/buildkit-v${BK_VER}.linux-arm64.tar.gz
|
||||
|
||||
# Extract only the binaries to /usr/local/bin
|
||||
sudo tar -C /usr/local/bin -xzvf buildkit-v${BK_VER}.linux-arm64.tar.gz --strip-components=1 bin/
|
||||
|
||||
# Create the service file
|
||||
sudo tee /etc/systemd/system/buildkit.service <<EOF
|
||||
[Unit]
|
||||
Description=BuildKit
|
||||
Documentation=https://github.com/moby/buildkit
|
||||
|
||||
[Service]
|
||||
ExecStart=/usr/local/bin/buildkitd --addr unix:///run/buildkit/buildkitd.sock
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
# Reload and Start
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now buildkit
|
||||
|
||||
# ---------------------------------------------
|
||||
|
||||
|
||||
# agent/worker node
|
||||
curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" INSTALL_RKE2_VERSION=v1.32.12+rke2r1 sh -
|
||||
systemctl enable rke2-agent.service
|
||||
mkdir -p /etc/rancher/rke2/
|
||||
# token from master node
|
||||
# cat /var/lib/rancher/rke2/server/node-token
|
||||
cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
|
||||
server: https://192.168.64.3:9345
|
||||
token: K107618960f87b9efb3a3255ce00a9743d29f1db9376820c9144cb85fa3c554dc69::server:06b2effdf0c9ce3952efc8a5d80bf084
|
||||
EOF
|
||||
systemctl start rke2-agent.service
|
||||
journalctl -u rke2-agent -f
|
||||
|
||||
|
||||
# Set up kubectl on the server node
|
||||
echo 'export KUBECONFIG=/etc/rancher/rke2/rke2.yaml' >> ~/.bashrc
|
||||
echo 'export PATH=$PATH:/var/lib/rancher/rke2/bin' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
|
||||
|
||||
## build and deploy application
|
||||
|
||||
```bash
|
||||
# build container with nerdctl
|
||||
nerdctl --namespace k8s.io build --tag hello-world:latest .
|
||||
|
||||
# export image as tar on master node
|
||||
nerdctl save hello-world:latest -o hello-world.tar
|
||||
# copy it over to worker node
|
||||
scp hello-world.tar novakj@192.168.64.4:~/
|
||||
# import image on the agent node
|
||||
sudo /var/lib/rancher/rke2/bin/ctr --address /run/k3s/containerd/containerd.sock -n k8s.io images import hello-world.tar
|
||||
|
||||
kubectl create namespace rke2-apps
|
||||
|
||||
cat <<EOF > deployment.yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-world-deployment
|
||||
namespace: rke2-apps
|
||||
labels:
|
||||
type: staticwebapp
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
type: staticwebapp
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
type: staticwebapp
|
||||
spec:
|
||||
containers:
|
||||
- name: staticwebapp
|
||||
image: hello-world:latest
|
||||
imagePullPolicy: Never
|
||||
ports:
|
||||
- containerPort: 80
|
||||
resources:
|
||||
requests:
|
||||
memory: "32Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "64Mi"
|
||||
cpu: "300m"
|
||||
EOF
|
||||
|
||||
kubectl create -f deployment.yaml
|
||||
|
||||
# expose deployment
|
||||
kubectl expose deployment hello-world-deployment --name hello-world-service --port=8080 --target-port=80 -n rke2-apps
|
||||
|
||||
# install ingress-nginx (even though i thought that there is ingress controller already deployed)
|
||||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.2/deploy/static/provider/cloud/deploy.yaml
|
||||
|
||||
# create ingress with "localhost" as host
|
||||
kubectl create ingress hello-world-ingress --class=nginx --rule="test-host/*=hello-world-service:8080" -n rke2-apps
|
||||
kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 8081:80
|
||||
|
||||
|
||||
|
||||
# incomplete completion configuration ;-)
|
||||
dnf install bash-completion -y
|
||||
alias 'k=kubectl'
|
||||
|
||||
# ~/.bashrc
|
||||
|
||||
# 1. Load the main bash-completion package first
|
||||
# On Rocky/RHEL, it's usually at this path:
|
||||
[[ -r "/usr/share/bash-completion/bash_completion" ]] && . "/usr/share/bash-completion/bash_completion"
|
||||
|
||||
# Enable kubectl bash completion
|
||||
source <(kubectl completion bash)
|
||||
|
||||
# Set up the alias
|
||||
alias k=kubectl
|
||||
|
||||
# Link the kubectl completion logic to the 'k' alias
|
||||
complete -o default -F __start_kubectl k
|
||||
```
|
||||
|
||||
|
||||
|
||||
## upgrading RKE2
|
||||
|
||||
```bash
|
||||
# install upgrade controller
|
||||
kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/download/v0.9.1/system-upgrade-controller.yaml
|
||||
|
||||
# server upgrade
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: upgrade.cattle.io/v1
|
||||
kind: Plan
|
||||
metadata:
|
||||
name: rke2-server-upgrade
|
||||
namespace: system-upgrade
|
||||
spec:
|
||||
concurrency: 1
|
||||
cordon: true
|
||||
nodeSelector:
|
||||
matchExpressions:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: In
|
||||
values: ["true"]
|
||||
serviceAccountName: system-upgrade
|
||||
upgrade:
|
||||
image: rancher/rke2-upgrade
|
||||
version: v1.33.9+rke2r1
|
||||
EOF
|
||||
|
||||
# agent upgrade
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: upgrade.cattle.io/v1
|
||||
kind: Plan
|
||||
metadata:
|
||||
name: rke2-agent-upgrade
|
||||
namespace: system-upgrade
|
||||
spec:
|
||||
concurrency: 1
|
||||
cordon: true
|
||||
nodeSelector:
|
||||
matchExpressions:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: DoesNotExist
|
||||
prepare:
|
||||
# Logic: "Don't start workers until servers are done"
|
||||
args: ["wait-for-plan", "rke2-server-upgrade"]
|
||||
image: rancher/rke2-upgrade
|
||||
serviceAccountName: system-upgrade
|
||||
upgrade:
|
||||
image: rancher/rke2-upgrade
|
||||
version: v1.33.9+rke2r1
|
||||
EOF
|
||||
```
|
||||
@@ -1,215 +0,0 @@
|
||||
## VMS creation
|
||||
|
||||
|
||||
|
||||
```bash
|
||||
# hypervisor: beelink (192.168.0.6)
|
||||
|
||||
vms_path=/srv/vms/images
|
||||
isos_path=/srv/vms/isos
|
||||
cd $isos_path
|
||||
# Grab Ubuntu 24.04 cloud image
|
||||
wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
||||
|
||||
# Create a base disk from the cloud image (repeat per node)
|
||||
for NODE in rke2-server rke2-agent1 rke2-agent2; do
|
||||
DISK_SIZE="30G"
|
||||
[[ "$NODE" == rke2-agent* ]] && DISK_SIZE="50G"
|
||||
|
||||
qemu-img create -f qcow2 -F qcow2 -b $isos_path/noble-server-cloudimg-amd64.img $vms_path/${NODE}.qcow2
|
||||
qemu-img resize $vms_path/${NODE}.qcow2 ${DISK_SIZE}
|
||||
done
|
||||
```
|
||||
|
||||
|
||||
```bash
|
||||
# prepare cloud-init/user-data
|
||||
apt install cloud-image-utils
|
||||
|
||||
# prepare cloud-init and launch VMs
|
||||
declare -A nodes=(
|
||||
[rke2-server]="192.168.0.51"
|
||||
[rke2-agent1]="192.168.0.52"
|
||||
[rke2-agent2]="192.168.0.53"
|
||||
)
|
||||
|
||||
for node in "${!nodes[@]}"; do
|
||||
ip="${nodes[$node]}"
|
||||
|
||||
# user-data
|
||||
cat <<EOF > user-data-${node}
|
||||
#cloud-config
|
||||
hostname: ${node}
|
||||
manage_etc_hosts: false
|
||||
users:
|
||||
- name: sre
|
||||
sudo: ALL=(ALL) NOPASSWD:ALL
|
||||
shell: /bin/bash
|
||||
ssh_authorized_keys:
|
||||
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINQxxkqmvtVI+8c5BkTaJ5c2HfBFRXJWMmEcevvfP9tV jan.novak@Jans-MacBook-Air.local
|
||||
write_files:
|
||||
- path: /etc/hosts
|
||||
append: true
|
||||
content: |
|
||||
192.168.0.51 rke2-server
|
||||
192.168.0.52 rke2-agent1
|
||||
192.168.0.53 rke2-agent2
|
||||
- path: /etc/modules-load.d/rke2.conf
|
||||
content: |
|
||||
br_netfilter
|
||||
overlay
|
||||
- path: /etc/sysctl.d/99-rke2.conf
|
||||
content: |
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.ipv4.ip_forward = 1
|
||||
swap:
|
||||
filename: /swap.img
|
||||
size: 0
|
||||
maxsize: 0
|
||||
runcmd:
|
||||
- swapoff -a
|
||||
- sed -i '/swap/d' /etc/fstab
|
||||
- modprobe br_netfilter
|
||||
- modprobe overlay
|
||||
- sysctl --system
|
||||
package_update: true
|
||||
packages:
|
||||
- qemu-guest-agent
|
||||
- nfs-common
|
||||
- open-iscsi
|
||||
power_state:
|
||||
mode: reboot
|
||||
EOF
|
||||
|
||||
# network config
|
||||
cat <<EOF > network-config-${node}
|
||||
network:
|
||||
version: 2
|
||||
ethernets:
|
||||
eth0:
|
||||
match:
|
||||
driver: "virtio_net"
|
||||
addresses:
|
||||
- "${ip}/24"
|
||||
nameservers:
|
||||
addresses:
|
||||
- 8.8.8.8
|
||||
routes:
|
||||
- to: "default"
|
||||
via: "192.168.0.4"
|
||||
EOF
|
||||
|
||||
cloud-localds --network-config=./network-config-${node} \
|
||||
$vms_path/${node}-seed.iso ./user-data-${node}
|
||||
done
|
||||
|
||||
# Launch VMs
|
||||
for node in "${!nodes[@]}"; do
|
||||
virt-install \
|
||||
--name ${node} \
|
||||
--ram 4096 --vcpus 2 \
|
||||
--os-variant ubuntu24.04 \
|
||||
--disk $vms_path/${node}.qcow2,bus=virtio \
|
||||
--disk $vms_path/${node}-seed.iso,device=cdrom \
|
||||
--network bridge=br0,model=virtio \
|
||||
--graphics none \
|
||||
--console pty,target_type=serial \
|
||||
--noautoconsole \
|
||||
--import
|
||||
done
|
||||
|
||||
```
|
||||
|
||||
|
||||
## RKE2 installation
|
||||
|
||||
```bash
|
||||
# there are no .deb packages - only rpm or tarball
|
||||
# "magic" install script can handle that
|
||||
curl -sfL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.32 sudo sh -
|
||||
|
||||
# Create config directory
|
||||
sudo mkdir -p /etc/rancher/rke2
|
||||
|
||||
# Server configuration
|
||||
cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
|
||||
# Bind the API to the node's IP (not 127.0.0.1)
|
||||
tls-san:
|
||||
- rke2-server
|
||||
- 192.168.0.51
|
||||
# Write kubeconfig readable by non-root
|
||||
write-kubeconfig-mode: "0644"
|
||||
# CNI - canal is default and fine for home lab
|
||||
# Alternatives: cilium, calico, multus
|
||||
cni:
|
||||
- canal
|
||||
# Disable servicelb if you plan to use metallb
|
||||
disable:
|
||||
- rke2-service-lb
|
||||
EOF
|
||||
|
||||
# Enable and start
|
||||
sudo systemctl enable rke2-server.service
|
||||
sudo systemctl start rke2-server.service
|
||||
|
||||
# Watch bootstrap (takes 2-3 min on first run)
|
||||
sudo journalctl -u rke2-server -f
|
||||
|
||||
```
|
||||
|
||||
|
||||
## Grab the join token and kubeconfig
|
||||
|
||||
```bash
|
||||
|
||||
# Token for agents to join
|
||||
sudo cat /var/lib/rancher/rke2/server/node-token
|
||||
# Save this somewhere - you'll need it on every agent
|
||||
|
||||
# Set up kubectl on the server node
|
||||
echo 'export KUBECONFIG=/etc/rancher/rke2/rke2.yaml' >> ~/.bashrc
|
||||
echo 'export PATH=$PATH:/var/lib/rancher/rke2/bin' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
|
||||
# Verify
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
|
||||
## Install RKE2 Agents (Workers)
|
||||
|
||||
```bash
|
||||
# Install RKE2 agent - same channel as server
|
||||
curl -sfL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.32 INSTALL_RKE2_TYPE=agent sudo sh -
|
||||
|
||||
# Create config
|
||||
sudo mkdir -p /etc/rancher/rke2
|
||||
|
||||
cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
|
||||
server: https://192.168.0.51:9345
|
||||
token: K10dba0bfff01d610ffed41c6b82a1b8861ee19e5af34a3bdd21970936823a846da::server:1f02fb36a288dcd06770cab28b015bd7
|
||||
EOF
|
||||
|
||||
# Enable and start
|
||||
sudo systemctl enable rke2-agent.service
|
||||
sudo systemctl start rke2-agent.service
|
||||
|
||||
# Watch join
|
||||
sudo journalctl -u rke2-agent -f
|
||||
|
||||
```
|
||||
|
||||
|
||||
## Copy Kubeconfig to Your Workstation
|
||||
|
||||
```bash
|
||||
# On your workstation (not the VMs)
|
||||
scp sre@192.168.0.51:/etc/rancher/rke2/rke2.yaml ~/.kube/rke2-homelab.yaml
|
||||
|
||||
# Fix the server address (it'll say 127.0.0.1)
|
||||
gsed -i 's/127.0.0.1/192.168.0.51/' ~/.kube/rke2-homelab.yaml
|
||||
|
||||
export KUBECONFIG=~/.kube/rke2-homelab.yaml
|
||||
kubectl get nodes
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,28 +0,0 @@
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: external-secrets-crds
|
||||
namespace: external-secrets
|
||||
spec:
|
||||
interval: 1h
|
||||
chart:
|
||||
spec:
|
||||
chart: external-secrets
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: external-secrets
|
||||
namespace: flux-system
|
||||
version: "1.2.1"
|
||||
values:
|
||||
installCRDs: true
|
||||
webhook:
|
||||
create: false
|
||||
certController:
|
||||
create: false
|
||||
serviceAccount:
|
||||
create: false
|
||||
resources: {}
|
||||
crds:
|
||||
createClusterExternalSecret: true
|
||||
createClusterSecretStore: true
|
||||
createPushSecret: true
|
||||
@@ -1,8 +0,0 @@
|
||||
apiVersion: source.toolkit.fluxcd.io/v1
|
||||
kind: HelmRepository
|
||||
metadata:
|
||||
name: external-secrets
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 1h
|
||||
url: https://charts.external-secrets.io
|
||||
@@ -1,20 +0,0 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: gitea-ci-deploy
|
||||
subjects:
|
||||
- kind: User
|
||||
name: "gitea_ci@idm.home.hrajfrisbee.cz" # matches preferred_username claim
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
roleRef:
|
||||
# kind: ClusterRole
|
||||
# name: edit # scope down as needed
|
||||
# apiGroup: rbac.authorization.k8s.io
|
||||
|
||||
# this is obviously too much permissions
|
||||
# but we can live with it for homelab
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: oidc-cluster-admin
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: User
|
||||
name: https://idm.home.hrajfrisbee.cz/oauth2/openid/k8s#35842461-a1c4-4ad6-8b29-697c5ddbfe84
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: User
|
||||
name: novakj@idm.home.hrajfrisbee.cz
|
||||
@@ -1,42 +0,0 @@
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: alloy-events
|
||||
namespace: monitoring
|
||||
spec:
|
||||
interval: 30m
|
||||
chart:
|
||||
spec:
|
||||
chart: alloy
|
||||
version: ">=1.8.0 <2.0.0"
|
||||
sourceRef: { kind: HelmRepository, name: grafana, namespace: monitoring }
|
||||
interval: 12h
|
||||
install: { remediation: { retries: 3 } }
|
||||
upgrade: { remediation: { retries: 3 } }
|
||||
driftDetection: { mode: enabled }
|
||||
values:
|
||||
controller:
|
||||
type: deployment
|
||||
replicas: 1 # cluster-scoped → exactly one
|
||||
image:
|
||||
tag: v1.17.1
|
||||
rbac: { create: false }
|
||||
serviceAccount: { create: true, name: alloy-events }
|
||||
alloy:
|
||||
enableReporting: false
|
||||
resources:
|
||||
requests: { cpu: 50m, memory: 96Mi }
|
||||
limits: { memory: 256Mi }
|
||||
configMap:
|
||||
content: |-
|
||||
loki.source.kubernetes_events "events" {
|
||||
forward_to = [loki.process.events.receiver]
|
||||
}
|
||||
loki.process "events" {
|
||||
stage.static_labels { values = { source = "kubernetes-events" } }
|
||||
forward_to = [loki.write.loki.receiver]
|
||||
}
|
||||
loki.write "loki" {
|
||||
endpoint { url = "http://192.168.0.30:3100/loki/api/v1/push" }
|
||||
external_labels = { cluster = "homelab" }
|
||||
}
|
||||
@@ -1,106 +0,0 @@
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: alloy-logs
|
||||
namespace: monitoring
|
||||
spec:
|
||||
interval: 30m
|
||||
chart:
|
||||
spec:
|
||||
chart: alloy
|
||||
version: ">=1.8.0 <2.0.0" # newest 1.x each reconcile; pin exact if you want determinism
|
||||
sourceRef: { kind: HelmRepository, name: grafana, namespace: monitoring }
|
||||
interval: 12h # poll cadence for new chart versions
|
||||
install: { remediation: { retries: 3 } }
|
||||
upgrade: { remediation: { retries: 3 } }
|
||||
driftDetection: { mode: enabled } # revert manual kubectl edits
|
||||
values:
|
||||
controller:
|
||||
type: daemonset
|
||||
image:
|
||||
tag: v1.17.1
|
||||
rbac: { create: false }
|
||||
serviceAccount: { create: true, name: alloy-logs }
|
||||
alloy:
|
||||
enableReporting: false
|
||||
securityContext: { runAsUser: 0, runAsGroup: 0 } # journald read
|
||||
mounts:
|
||||
varlog: true
|
||||
resources:
|
||||
requests: { cpu: 100m, memory: 128Mi }
|
||||
limits: { memory: 512Mi }
|
||||
extraEnv:
|
||||
- name: NODE_NAME
|
||||
valueFrom: { fieldRef: { fieldPath: spec.nodeName } }
|
||||
configMap:
|
||||
content: |-
|
||||
discovery.kubernetes "pods" {
|
||||
role = "pod"
|
||||
selectors {
|
||||
role = "pod"
|
||||
field = "spec.nodeName=" + sys.env("NODE_NAME")
|
||||
}
|
||||
}
|
||||
discovery.relabel "pods" {
|
||||
targets = discovery.kubernetes.pods.targets
|
||||
rule {
|
||||
source_labels = ["__meta_kubernetes_namespace"]
|
||||
target_label = "namespace"
|
||||
}
|
||||
rule {
|
||||
source_labels = ["__meta_kubernetes_pod_label_app"]
|
||||
target_label = "app"
|
||||
}
|
||||
rule {
|
||||
source_labels = ["__meta_kubernetes_pod_container_name"]
|
||||
target_label = "container"
|
||||
}
|
||||
rule {
|
||||
source_labels = ["__meta_kubernetes_pod_node_name"]
|
||||
target_label = "node"
|
||||
}
|
||||
rule {
|
||||
source_labels = ["__meta_kubernetes_pod_name"]
|
||||
target_label = "pod"
|
||||
}
|
||||
}
|
||||
loki.source.kubernetes "pods" {
|
||||
targets = discovery.relabel.pods.output
|
||||
forward_to = [loki.process.pods.receiver]
|
||||
}
|
||||
loki.process "pods" {
|
||||
stage.static_labels {
|
||||
values = { source = "kubernetes" }
|
||||
}
|
||||
forward_to = [loki.write.loki.receiver]
|
||||
}
|
||||
|
||||
discovery.relabel "journal" {
|
||||
targets = []
|
||||
rule {
|
||||
source_labels = ["__journal__systemd_unit"]
|
||||
target_label = "unit"
|
||||
}
|
||||
rule {
|
||||
source_labels = ["__journal__hostname"]
|
||||
target_label = "node"
|
||||
}
|
||||
rule {
|
||||
source_labels = ["__journal_priority_keyword"]
|
||||
target_label = "level"
|
||||
}
|
||||
}
|
||||
loki.source.journal "node" {
|
||||
path = "/var/log/journal"
|
||||
max_age = "12h"
|
||||
labels = { source = "node-journal" }
|
||||
relabel_rules = discovery.relabel.journal.rules
|
||||
forward_to = [loki.write.loki.receiver]
|
||||
}
|
||||
|
||||
loki.write "loki" {
|
||||
endpoint {
|
||||
url = "http://192.168.0.30:3100/loki/api/v1/push"
|
||||
}
|
||||
external_labels = { cluster = "homelab" }
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: alloy-metrics
|
||||
namespace: monitoring
|
||||
spec:
|
||||
interval: 30m
|
||||
chart:
|
||||
spec:
|
||||
chart: alloy
|
||||
version: ">=1.8.0 <2.0.0"
|
||||
sourceRef: { kind: HelmRepository, name: grafana, namespace: monitoring }
|
||||
interval: 12h
|
||||
install: { remediation: { retries: 3 } }
|
||||
upgrade: { remediation: { retries: 3 } }
|
||||
driftDetection: { mode: enabled }
|
||||
values:
|
||||
controller:
|
||||
type: deployment
|
||||
replicas: 1 # cluster-scoped scrape target discovery → exactly one
|
||||
image:
|
||||
tag: v1.17.1
|
||||
rbac: { create: false }
|
||||
serviceAccount: { create: true, name: alloy-metrics }
|
||||
alloy:
|
||||
enableReporting: false
|
||||
resources:
|
||||
requests: { cpu: 100m, memory: 256Mi }
|
||||
limits: { memory: 768Mi }
|
||||
configMap:
|
||||
content: |-
|
||||
// Discovers every ServiceMonitor cluster-wide. kube-prometheus-stack already ships
|
||||
// ServiceMonitors for kubelet/cAdvisor, kube-state-metrics, node-exporter, apiserver,
|
||||
// coredns, controller-manager and scheduler — so this alone covers "standard k8s metrics".
|
||||
prometheus.operator.servicemonitors "sm" {
|
||||
forward_to = [prometheus.remote_write.mimir.receiver]
|
||||
}
|
||||
|
||||
// Discovers every PodMonitor cluster-wide, same selector scope as above.
|
||||
prometheus.operator.podmonitors "pm" {
|
||||
forward_to = [prometheus.remote_write.mimir.receiver]
|
||||
}
|
||||
|
||||
prometheus.remote_write "mimir" {
|
||||
endpoint {
|
||||
url = "http://192.168.0.30:9009/api/v1/push"
|
||||
}
|
||||
external_labels = { cluster = "homelab" }
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
apiVersion: source.toolkit.fluxcd.io/v1
|
||||
kind: HelmRepository
|
||||
metadata:
|
||||
name: grafana
|
||||
namespace: monitoring
|
||||
spec:
|
||||
interval: 1h
|
||||
url: https://grafana.github.io/helm-charts
|
||||
@@ -1,15 +0,0 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata: { name: alloy-log-reader }
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["pods", "pods/log", "namespaces", "nodes", "nodes/proxy", "events"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata: { name: alloy-log-reader }
|
||||
roleRef: { apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: alloy-log-reader }
|
||||
subjects:
|
||||
- { kind: ServiceAccount, name: alloy-logs, namespace: monitoring }
|
||||
- { kind: ServiceAccount, name: alloy-events, namespace: monitoring }
|
||||
@@ -1,25 +0,0 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata: { name: alloy-metrics-reader }
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["namespaces", "nodes", "nodes/metrics", "nodes/proxy", "services", "endpoints", "pods"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
- apiGroups: ["discovery.k8s.io"]
|
||||
resources: ["endpointslices"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
- apiGroups: ["networking.k8s.io"]
|
||||
resources: ["ingresses"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
- apiGroups: ["monitoring.coreos.com"]
|
||||
resources: ["servicemonitors", "podmonitors", "probes", "scrapeconfigs"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
- nonResourceURLs: ["/metrics", "/metrics/cadvisor"]
|
||||
verbs: ["get"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata: { name: alloy-metrics-reader }
|
||||
roleRef: { apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: alloy-metrics-reader }
|
||||
subjects:
|
||||
- { kind: ServiceAccount, name: alloy-metrics, namespace: monitoring }
|
||||
@@ -1,10 +0,0 @@
|
||||
## debugging alloy
|
||||
|
||||
```bash
|
||||
POD=$(kubectl -n monitoring get pod -l app.kubernetes.io/name=alloy -o name | head -1)
|
||||
kubectl -n monitoring debug -it $POD --image=nicolaka/netshoot --target=alloy -- bash
|
||||
|
||||
|
||||
curl -v http://192.168.0.30:3100/ready
|
||||
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,22 +0,0 @@
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: ClusterIssuer
|
||||
metadata:
|
||||
name: letsencrypt-prod-dns
|
||||
spec:
|
||||
acme:
|
||||
server: https://acme-v02.api.letsencrypt.org/directory
|
||||
email: kacerr.cz@gmail.com
|
||||
privateKeySecretRef:
|
||||
name: letsencrypt-dns-account-key
|
||||
solvers:
|
||||
- dns01:
|
||||
rfc2136:
|
||||
nameserver: dns-update-proxy.cert-manager.svc.cluster.local:53
|
||||
tsigKeyName: acme-update-key
|
||||
tsigAlgorithm: HMACSHA512
|
||||
tsigSecretSecretRef:
|
||||
name: acme-update-key
|
||||
key: acme-update-key
|
||||
selector:
|
||||
dnsZones:
|
||||
- "lab.home.hrajfrisbee.cz"
|
||||
@@ -1,33 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: dns-update-proxy
|
||||
namespace: cert-manager
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: dns-update-proxy
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: dns-update-proxy
|
||||
spec:
|
||||
containers:
|
||||
- name: socat-tcp
|
||||
image: alpine/socat
|
||||
args:
|
||||
- TCP-LISTEN:53,fork,reuseaddr
|
||||
- TCP:87.236.195.209:5353
|
||||
ports:
|
||||
- containerPort: 53
|
||||
protocol: TCP
|
||||
- name: socat-udp
|
||||
image: alpine/socat
|
||||
args:
|
||||
- -T5
|
||||
- UDP-RECVFROM:53,fork,reuseaddr
|
||||
- UDP:87.236.195.209:5353
|
||||
ports:
|
||||
- containerPort: 53
|
||||
protocol: UDP
|
||||
@@ -1,18 +0,0 @@
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: acme-update-key
|
||||
namespace: cert-manager
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
name: vault-backend # or your store
|
||||
kind: ClusterSecretStore
|
||||
target:
|
||||
name: acme-update-key
|
||||
creationPolicy: Owner
|
||||
data:
|
||||
- secretKey: acme-update-key
|
||||
remoteRef:
|
||||
key: k8s_home/cert-manager
|
||||
property: acme-update-key
|
||||
@@ -1,62 +0,0 @@
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: cert-manager
|
||||
namespace: cert-manager
|
||||
spec:
|
||||
interval: 1h
|
||||
chart:
|
||||
spec:
|
||||
chart: cert-manager
|
||||
version: "v1.20.2"
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: cert-manager
|
||||
namespace: flux-system
|
||||
install:
|
||||
createNamespace: true
|
||||
crds: CreateReplace
|
||||
upgrade:
|
||||
crds: CreateReplace
|
||||
values:
|
||||
global:
|
||||
logLevel: 6
|
||||
crds:
|
||||
enabled: false
|
||||
config:
|
||||
apiVersion: controller.config.cert-manager.io/v1alpha1
|
||||
kind: ControllerConfiguration
|
||||
enableGatewayAPI: true
|
||||
prometheus:
|
||||
enabled: true
|
||||
extraObjects:
|
||||
- |
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: ClusterIssuer
|
||||
metadata:
|
||||
name: letsencrypt-staging
|
||||
spec:
|
||||
acme:
|
||||
server: https://acme-staging-v02.api.letsencrypt.org/directory
|
||||
email: kacerr.cz+lets-encrypt@gmail.com
|
||||
privateKeySecretRef:
|
||||
name: letsencrypt-staging-account-key
|
||||
solvers:
|
||||
- http01:
|
||||
ingress:
|
||||
ingressClassName: nginx
|
||||
- |
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: ClusterIssuer
|
||||
metadata:
|
||||
name: letsencrypt-prod
|
||||
spec:
|
||||
acme:
|
||||
server: https://acme-v02.api.letsencrypt.org/directory
|
||||
email: kacerr.cz+lets-encrypt@gmail.com
|
||||
privateKeySecretRef:
|
||||
name: letsencrypt-prod-account-key
|
||||
solvers:
|
||||
- http01:
|
||||
ingress:
|
||||
ingressClassName: nginx
|
||||
@@ -1,8 +0,0 @@
|
||||
apiVersion: source.toolkit.fluxcd.io/v1
|
||||
kind: HelmRepository
|
||||
metadata:
|
||||
name: cert-manager
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 1h
|
||||
url: https://charts.jetstack.io
|
||||
@@ -1,4 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: cert-manager
|
||||
@@ -1,17 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: dns-update-proxy
|
||||
namespace: cert-manager
|
||||
spec:
|
||||
selector:
|
||||
app: dns-update-proxy
|
||||
ports:
|
||||
- name: dns-tcp
|
||||
port: 53
|
||||
targetPort: 53
|
||||
protocol: TCP
|
||||
- name: dns-udp
|
||||
port: 53
|
||||
targetPort: 53
|
||||
protocol: UDP
|
||||
@@ -1,10 +0,0 @@
|
||||
apiVersion: cilium.io/v2alpha1
|
||||
kind: CiliumL2AnnouncementPolicy
|
||||
metadata:
|
||||
name: default
|
||||
spec:
|
||||
interfaces:
|
||||
- ^en.* # Match your interfaces
|
||||
loadBalancerIPs: true
|
||||
serviceSelector:
|
||||
matchLabels: {}
|
||||
@@ -1,12 +0,0 @@
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: Certificate
|
||||
metadata:
|
||||
name: wildcard-lab-home-hrajfrisbee
|
||||
namespace: kube-system
|
||||
spec:
|
||||
secretName: wildcard-lab-home-hrajfrisbee-tls
|
||||
issuerRef:
|
||||
name: letsencrypt-prod-dns
|
||||
kind: ClusterIssuer
|
||||
dnsNames:
|
||||
- "*.lab.home.hrajfrisbee.cz"
|
||||
@@ -1,27 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: Gateway
|
||||
metadata:
|
||||
name: cilium-gateway
|
||||
namespace: kube-system
|
||||
spec:
|
||||
gatewayClassName: cilium
|
||||
listeners:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: HTTP
|
||||
allowedRoutes:
|
||||
namespaces:
|
||||
from: All
|
||||
- name: lab-home-hrajfrisbee-https-wildcard
|
||||
hostname: "*.lab.home.hrajfrisbee.cz"
|
||||
port: 443
|
||||
protocol: HTTPS
|
||||
tls:
|
||||
mode: Terminate
|
||||
certificateRefs:
|
||||
- kind: Secret
|
||||
name: wildcard-lab-home-hrajfrisbee-tls
|
||||
allowedRoutes:
|
||||
namespaces:
|
||||
from: All
|
||||
@@ -1,47 +0,0 @@
|
||||
---
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: cilium
|
||||
namespace: kube-system
|
||||
spec:
|
||||
chart:
|
||||
spec:
|
||||
chart: cilium
|
||||
reconcileStrategy: ChartVersion
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: cilium
|
||||
namespace: flux-system
|
||||
version: 1.19.1
|
||||
interval: 5m0s
|
||||
values:
|
||||
cluster:
|
||||
name: "home-kube"
|
||||
devices: "eth+ bond+ en+"
|
||||
hubble:
|
||||
relay:
|
||||
enabled: true
|
||||
ui:
|
||||
enabled: true
|
||||
ingressController:
|
||||
enabled: true
|
||||
ipam:
|
||||
mode: cluster-pool
|
||||
operator:
|
||||
clusterPoolIPv4MaskSize: 24
|
||||
clusterPoolIPv4PodCIDRList: "10.96.0.0/16"
|
||||
l2announcements:
|
||||
enabled: true
|
||||
gatewayAPI:
|
||||
enabled: true
|
||||
gatewayClass:
|
||||
create: "true"
|
||||
kubeProxyReplacement: true
|
||||
k8sServiceHost: 192.168.0.31 # or LB IP
|
||||
k8sServicePort: 6443
|
||||
|
||||
# disable envoy daemonset - i guess that is stupid idea anyway
|
||||
# envoy:
|
||||
# enabled: false
|
||||
# l7Proxy: false
|
||||
@@ -1,9 +0,0 @@
|
||||
apiVersion: "cilium.io/v2alpha1"
|
||||
kind: CiliumLoadBalancerIPPool
|
||||
metadata:
|
||||
name: cilium-lb-ipam
|
||||
namespace: kube-system
|
||||
spec:
|
||||
blocks:
|
||||
- start: "192.168.0.35"
|
||||
stop: "192.168.0.39"
|
||||
16
gitops/home-kubernetes/cilium/release.yaml
Normal file
16
gitops/home-kubernetes/cilium/release.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
---
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: cilium
|
||||
namespace: kube-system
|
||||
spec:
|
||||
chart:
|
||||
spec:
|
||||
chart: cilium
|
||||
reconcileStrategy: ChartVersion
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: cilium
|
||||
version: 1.16.5
|
||||
interval: 5m0s
|
||||
@@ -1,17 +0,0 @@
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: Certificate
|
||||
metadata:
|
||||
name: operator-test-lab-home-hrajfrisbee-2
|
||||
namespace: default
|
||||
annotations:
|
||||
gateway-cert-operator.io/gateway-name: "cilium-gateway"
|
||||
gateway-cert-operator.io/gateway-namespace: "kube-system"
|
||||
|
||||
spec:
|
||||
secretName: operator-test-lab-home-hrajfrisbee-2
|
||||
issuerRef:
|
||||
group: cert-manager.io
|
||||
kind: ClusterIssuer
|
||||
name: letsencrypt-prod
|
||||
dnsNames:
|
||||
- "operator-test-2.lab.home.hrajfrisbee.cz"
|
||||
@@ -1,17 +0,0 @@
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: Certificate
|
||||
metadata:
|
||||
name: operator-test-lab-home-hrajfrisbee
|
||||
namespace: default
|
||||
annotations:
|
||||
gateway-cert-operator.io/gateway-name: "cilium-gateway"
|
||||
gateway-cert-operator.io/gateway-namespace: "kube-system"
|
||||
|
||||
spec:
|
||||
secretName: operator-test-lab-home-hrajfrisbee
|
||||
issuerRef:
|
||||
group: cert-manager.io
|
||||
kind: ClusterIssuer
|
||||
name: letsencrypt-prod
|
||||
dnsNames:
|
||||
- "operator-test.lab.home.hrajfrisbee.cz"
|
||||
@@ -1,30 +0,0 @@
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: operator-test
|
||||
namespace: default
|
||||
labels:
|
||||
app: operator-test
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: operator-test
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: operator-test
|
||||
spec:
|
||||
containers:
|
||||
- name: whoami
|
||||
image: traefik/whoami
|
||||
ports:
|
||||
- containerPort: 80
|
||||
name: http
|
||||
resources:
|
||||
requests:
|
||||
cpu: 10m
|
||||
memory: 16Mi
|
||||
limits:
|
||||
memory: 32Mi
|
||||
@@ -1,25 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: operator-test-redirect
|
||||
namespace: default
|
||||
labels:
|
||||
app: operator-test
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: cilium-gateway
|
||||
namespace: kube-system
|
||||
sectionName: http
|
||||
hostnames:
|
||||
- operator-test.lab.home.hrajfrisbee.cz
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
filters:
|
||||
- type: RequestRedirect
|
||||
requestRedirect:
|
||||
scheme: https
|
||||
statusCode: 301
|
||||
@@ -1,24 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: operator-test
|
||||
namespace: default
|
||||
labels:
|
||||
app: operator-test
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: cilium-gateway
|
||||
namespace: kube-system
|
||||
sectionName: auto-operator-test-lab-home-hrajfrisbee-operator-test-lab-home-hrajfrisbee-cz
|
||||
hostnames:
|
||||
- operator-test.lab.home.hrajfrisbee.cz
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
backendRefs:
|
||||
- name: operator-test
|
||||
namespace: default
|
||||
port: 80
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: gateway.networking.k8s.io/v1beta1
|
||||
kind: ReferenceGrant
|
||||
metadata:
|
||||
name: allow-kube-system-gateway
|
||||
namespace: default
|
||||
spec:
|
||||
from:
|
||||
- group: gateway.networking.k8s.io
|
||||
kind: Gateway
|
||||
namespace: kube-system
|
||||
to:
|
||||
- group: ""
|
||||
kind: Secret
|
||||
name: operator-test-lab-home-hrajfrisbee
|
||||
@@ -1,15 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: operator-test
|
||||
namespace: default
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: operator-test
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 80
|
||||
protocol: TCP
|
||||
name: http
|
||||
@@ -1,19 +0,0 @@
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ClusterSecretStore
|
||||
metadata:
|
||||
name: vault-backend
|
||||
namespace: external-secrets
|
||||
spec:
|
||||
provider:
|
||||
vault:
|
||||
server: "https://vault.hrajfrisbee.cz"
|
||||
path: "secret"
|
||||
version: "v2"
|
||||
auth:
|
||||
appRole:
|
||||
path: "approle"
|
||||
roleId: "864e352d-2064-2bf9-2c73-dbd676a95368" # or reference a secret
|
||||
secretRef:
|
||||
name: vault-approle
|
||||
key: secret-id
|
||||
namespace: external-secrets
|
||||
@@ -1,63 +0,0 @@
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: external-secrets
|
||||
namespace: external-secrets
|
||||
spec:
|
||||
interval: 30m
|
||||
chart:
|
||||
spec:
|
||||
chart: external-secrets
|
||||
version: "1.2.1" # latest stable 1.x
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: external-secrets
|
||||
namespace: flux-system
|
||||
install:
|
||||
createNamespace: true
|
||||
remediation:
|
||||
retries: 3
|
||||
upgrade:
|
||||
remediation:
|
||||
retries: 3
|
||||
values:
|
||||
replicaCount: 1
|
||||
leaderElect: true
|
||||
|
||||
# Resources (adjust to your cluster)
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
memory: 256Mi
|
||||
|
||||
webhook:
|
||||
replicaCount: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 25m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
memory: 128Mi
|
||||
podDisruptionBudget:
|
||||
enabled: true
|
||||
minAvailable: 1
|
||||
|
||||
certController:
|
||||
replicaCount: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 25m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
memory: 128Mi
|
||||
|
||||
# Metrics (enable if prometheus-operator is present)
|
||||
serviceMonitor:
|
||||
enabled: false
|
||||
|
||||
# Pod disruption budgets
|
||||
podDisruptionBudget:
|
||||
enabled: true
|
||||
minAvailable: 1
|
||||
@@ -1,4 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: external-secrets
|
||||
@@ -1,10 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: vault-approle
|
||||
namespace: external-secrets
|
||||
annotations:
|
||||
kustomize.toolkit.fluxcd.io/reconcile: disabled
|
||||
type: Opaque
|
||||
stringData:
|
||||
secret-id: --- fill in the secret_id ---
|
||||
@@ -1,32 +1,6 @@
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: 00-crds
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/00-crds
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: 00-rbac
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/00-rbac
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: cilium
|
||||
namespace: flux-system
|
||||
@@ -40,62 +14,6 @@ spec:
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: cert-manager
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/cert-manager
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: external-secrets
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/external-secrets
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
dependsOn:
|
||||
- name: 00-crds
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: kube-prometheus
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/kube-prometheus
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: alloy
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/alloy
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
dependsOn:
|
||||
- name: kube-prometheus # needs the monitoring ns + operator CRDs/ServiceMonitors
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
namespace: flux-system
|
||||
@@ -110,63 +28,11 @@ spec:
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: oauth-proxy
|
||||
name: kuard
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/oauth-proxy
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: podinfo
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/podinfo
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: plane
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/plane
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: experiments
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/experiments
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: fuj
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./gitops/home-kubernetes/fuj
|
||||
path: ./gitops/home-kubernetes/kuard
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
|
||||
@@ -11,7 +11,7 @@ spec:
|
||||
branch: main
|
||||
secretRef:
|
||||
name: flux-system
|
||||
url: https://gitea.home.hrajfrisbee.cz/kacerr/home-kubernetes
|
||||
url: https://gitlab.hrajfrisbee.cz/infrastructure/home-kubernetes.git
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
name: flux-web
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 30m
|
||||
releaseName: flux-web
|
||||
chartRef:
|
||||
kind: OCIRepository
|
||||
name: flux-operator
|
||||
values:
|
||||
fullnameOverride: flux-web
|
||||
installCRDs: true
|
||||
web:
|
||||
serverOnly: true
|
||||
@@ -1,13 +0,0 @@
|
||||
apiVersion: source.toolkit.fluxcd.io/v1
|
||||
kind: OCIRepository
|
||||
metadata:
|
||||
name: flux-operator
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 30m
|
||||
url: oci://ghcr.io/controlplaneio-fluxcd/charts/flux-operator
|
||||
layerSelector:
|
||||
mediaType: "application/vnd.cncf.helm.chart.content.v1.tar+gzip"
|
||||
operation: copy
|
||||
ref:
|
||||
semver: "0.x"
|
||||
@@ -1,63 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: fuj-management
|
||||
namespace: fuj
|
||||
labels:
|
||||
app: fuj-management
|
||||
spec:
|
||||
replicas: 1
|
||||
strategy:
|
||||
type: Recreate
|
||||
selector:
|
||||
matchLabels:
|
||||
app: fuj-management
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: fuj-management
|
||||
spec:
|
||||
imagePullSecrets:
|
||||
- name: gitea-registry
|
||||
containers:
|
||||
- name: fuj-management
|
||||
image: gitea.home.hrajfrisbee.cz/kacerr/fuj-management:0.39-go
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
name: http
|
||||
env:
|
||||
- name: PORT
|
||||
value: "8080"
|
||||
- name: TZ
|
||||
value: Europe/Prague
|
||||
- name: CREDENTIALS_PATH
|
||||
value: /secrets/credentials.json
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 128Mi
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 8080
|
||||
initialDelaySeconds: 2
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 8080
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 30
|
||||
timeoutSeconds: 5
|
||||
volumeMounts:
|
||||
- name: bot-credentials
|
||||
mountPath: /secrets
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: bot-credentials
|
||||
secret:
|
||||
secretName: fuj-management-bot-credentials
|
||||
@@ -1,60 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: fujarna
|
||||
namespace: fuj
|
||||
labels:
|
||||
app: fujarna
|
||||
spec:
|
||||
replicas: 1
|
||||
strategy:
|
||||
type: Recreate
|
||||
selector:
|
||||
matchLabels:
|
||||
app: fujarna
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: fujarna
|
||||
spec:
|
||||
imagePullSecrets:
|
||||
- name: gitea-registry
|
||||
containers:
|
||||
- name: fujarna
|
||||
image: gitea.home.hrajfrisbee.cz/kacerr/fujarna:v0.0.2
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
name: http
|
||||
env:
|
||||
- name: PORT
|
||||
value: "8080"
|
||||
- name: TZ
|
||||
value: Europe/Prague
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 128Mi
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /api/tournaments
|
||||
port: 8080
|
||||
initialDelaySeconds: 2
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /api/tournaments
|
||||
port: 8080
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 30
|
||||
timeoutSeconds: 5
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: tournament-data
|
||||
@@ -1,22 +0,0 @@
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: gitea-registry
|
||||
namespace: fuj
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
name: vault-backend
|
||||
kind: ClusterSecretStore
|
||||
target:
|
||||
name: gitea-registry
|
||||
creationPolicy: Owner
|
||||
template:
|
||||
type: kubernetes.io/dockerconfigjson
|
||||
data:
|
||||
.dockerconfigjson: "{{ .token }}"
|
||||
data:
|
||||
- secretKey: token
|
||||
remoteRef:
|
||||
key: k8s_home/gitea/container-registry
|
||||
property: token
|
||||
@@ -1,30 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: fujarna-redirect
|
||||
namespace: fuj
|
||||
labels:
|
||||
app: fujarna
|
||||
app.kubernetes.io/name: fujarna-httproute
|
||||
app.kubernetes.io/instance: fujarna
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: httproute
|
||||
app.kubernetes.io/part-of: fujarna
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: cilium-gateway
|
||||
namespace: kube-system
|
||||
sectionName: http
|
||||
hostnames:
|
||||
- fujarna.lab.home.hrajfrisbee.cz
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
filters:
|
||||
- type: RequestRedirect
|
||||
requestRedirect:
|
||||
scheme: https
|
||||
statusCode: 301
|
||||
@@ -1,29 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: fujarna
|
||||
namespace: fuj
|
||||
labels:
|
||||
app: fujarna
|
||||
app.kubernetes.io/name: fujarna-httproute
|
||||
app.kubernetes.io/instance: fujarna
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: httproute
|
||||
app.kubernetes.io/part-of: fujarna
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: cilium-gateway
|
||||
namespace: kube-system
|
||||
sectionName: lab-home-hrajfrisbee-https-wildcard
|
||||
hostnames:
|
||||
- fujarna.lab.home.hrajfrisbee.cz
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
backendRefs:
|
||||
- name: fujarna
|
||||
namespace: fuj
|
||||
port: 80
|
||||
@@ -1,28 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: fuj-management
|
||||
namespace: fuj
|
||||
labels:
|
||||
app: fuj-management
|
||||
app.kubernetes.io/name: fuj-management-httproute
|
||||
app.kubernetes.io/instance: fuj-management
|
||||
app.kubernetes.io/component: httproute
|
||||
app.kubernetes.io/part-of: fuj-management
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: cilium-gateway
|
||||
namespace: kube-system
|
||||
sectionName: lab-home-hrajfrisbee-https-wildcard
|
||||
hostnames:
|
||||
- fuj-management.lab.home.hrajfrisbee.cz
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
backendRefs:
|
||||
- name: fuj-management
|
||||
namespace: fuj
|
||||
port: 80
|
||||
@@ -1,30 +0,0 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: fujarna
|
||||
namespace: fuj
|
||||
annotations:
|
||||
# WebSocket support - increase timeouts for live scoring connections
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
||||
# Uncomment for cert-manager TLS:
|
||||
# cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
spec:
|
||||
ingressClassName: nginx # change to "traefik" for k3s default
|
||||
rules:
|
||||
- host: hrajfrisbee.cz # change to your domain
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: fujarna
|
||||
port:
|
||||
number: 80
|
||||
# Uncomment for TLS:
|
||||
# tls:
|
||||
# - hosts:
|
||||
# - hrajfrisbee.cz
|
||||
# secretName: fujarna-tls
|
||||
@@ -1,4 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: fuj
|
||||
@@ -1,12 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: tournament-data
|
||||
namespace: fuj
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
# storageClassName: local-path # uncomment to override cluster default
|
||||
@@ -1,8 +0,0 @@
|
||||
apiVersion: v1
|
||||
data:
|
||||
credentials.json: ewogICAgInR5cGUiOiAic2VydmljZV9hY2NvdW50IiwKICAgICJwcm9qZWN0X2lkIjogInBsYXRlZC1tZXNoLTQ4OTAxMC1zMSIsCiAgICAicHJpdmF0ZV9rZXlfaWQiOiAiYzQxM2VhYWUzMzk0NGFmZDljZjkyZjgzNGYzYmM1NDM5OGI1ZGY5YyIsCiAgICAicHJpdmF0ZV9rZXkiOiAiLS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tXG5NSUlFdmdJQkFEQU5CZ2txaGtpRzl3MEJBUUVGQUFTQ0JLZ3dnZ1NrQWdFQUFvSUJBUURuTzBJQ3p2NDNaMWFKXG5MYi9NSkFZbld2U2NWM2tlelBGSHpHNVd2WUE3SXBxM2ZkcUp6ZWYxSzcxeGJodGY5NFB6T2RNVGdrYXcxcmZYXG40QjhOMS9laWFZaWUyZkkwenA4emptejl0YnpDUjVwM0hiK3M0eHROR3hVUDJBWWQzVmRrbE5TRFlacGJ5Qm8rXG5qNWl1cy9BOXdYV1ZkQ2EvNnA1MmtYSU1Ld3o4b05rbmpMRTlzV21ZV1kvQmJZT1o0UExPSjNTWEdEMFJNQzlGXG4wbjBLUDJTRWxvaTFPc2wvV2s0Q21WazJFc0lNSjBiMHA1Wm1GZUZ6SlAzSFZUS0VtOExHam5vWjVwQmwzWGM0XG5GOThKSVFFQzZQallOcXQvSFlxdWRLb2hrelVSS2h5ekVJdHp3NXBrUnk3d1l1OGJHSGJOMzF0dWFpeDRMdFNjXG5zYU5oOHJhVEFnTUJBQUVDZ2dFQWFXZ2U4RTdSbmdueWJiZzRMV1BpbGtBbEw3dkVUK0VXd0NjVnlWNjdQTG5LXG5rNGpBZlg4cWxSMnFUekhsTXJzUElHb2txVWtDMW93YTVFS2JoV0VFMXJtSytQYmJMVzFmTDA1bXFzVVUxZTkwXG5INGFsSUxlcWMxeThIaXZZcGZhSFp5ZGROTUxpYmFKckxFZWpUMGhoWkpWeTkzMHFTT1EzOWs3WkVXanlrNW9WXG5WcUk2SE4yVGUrRTN0aklmNFcrN0lKN0c3RU9ZVG4zZHlrQnZCN1QwLzR2M0pUeGMzTk1IN0RZYzFyd1dicEdKXG5JcmNidndOa3VyVHhsdWg1VGtvSkZjd3ZXdEl4NE52MFFPNkk5TlpjMllPakZUMncwMUFrKzNDSTJLNmJOVE9qXG43WWFxYlRoNDk2cU1Md2NtRmFUZy9tbXEzMGZycEVsMWdQTWJWZFJqTVFLQmdRRDY3SDMvb0pGU1ZDWnFYQjFEXG5tdHZRblVMSWUvZkU5TXc4T2VrcUdMclQxbGlUTllhTks4YnRsOGFrbFlZY3FXU09zWk53ZkdyNmNhOGhUSEwyXG5pWXZNbkR4aVJPWkVkeElPUFJGY2l4SlNpdG5hb3kzSmtMMml0K2cxMUVGdUEyS3dGd29jc0RrRkQ1QTh0ZEFpXG5UOGhEeEc4dE5saFFrTUY0ekRMNTE2cHRLUUtCZ1FEcjZNZitLelBoNHM2Qm1GVzkrUzNtZEFZZkJETkJsZHJrXG56L1hrY2NteHNWeUpNNTErR2MzY3kzYmhIRDU4aGVLb3U1ZHdYanNqZkRCZlZKcjBpTHM0R3dGbjNlWlJYNmluXG5NVmlycmpyUWt5WnlSVDJkcHl5ODQ5MGN1WC9xQ01hbTVRYzY0UmpJN3JhRTM5ZDBRcnRRd0hWbnZCN1RHaEcvXG50WElXeThMQld3S0JnRGRtMi9NQyt0TVlyVnF5ZCt1alY0RmFGNUs0ZkY1S1JvWE5xNE9WN2pvemkxOEZaRzNyXG5Db01QRGRidEJLMXpZVFo0VXJ0MVhSRnE1R1lQd1JwYUNEbm5EUllOYkdJVmdERVFucksyS2hkN2ZOcGZTUTdHXG5ISGkrN2xCa1N3WUZUVmJmMzJXeUt3VEpDT20rb3ZQUWRjRXdyL05sbWw0L2Q5ZzZEYzlJSk9MeEFvR0JBTi9yXG5QaGwwN0hRV0VpbFViTjQ3TEdRMzhLOTBEeXJMa00vcXpOMGZYSEM3aHJmOE5CdERadkdTZHE1SlhzNVhUc01WXG4vREorWHVmWnN4aHBlQU52M1RUdC9UR1hvRXZRM29hRUFrUkN4alMvMTVoVFBKN3QxVHNkbGZ6ejNnUklmRFpoXG40YkR6ZzlFRk5GMS9Fa1NhS3E0Rlgrc3MxcDFOZTFkYVpJWmF0ZXROQW9HQkFLeStZTGRjQVh6dElaMTkvS0wwXG5sWTBPZi92azVJNUtKNlNiVUVheU9HTFJkWTJBdGk1R1QwQlpjVEV3QjYzWTNyWEVNQ3BsM25Ca1pVRklRczNEXG5wSE9DUDdDMXQ3STRSZlBGaGlSTzEvQnVsajZjUXNrUHFJWlpadGxOQjdROG1BVEttMnpXVThZTjRNbFhZSVBVXG5naGRrZHVQRWIyRC9ueDl3U3F2NzdUeWJcbi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS1cbiIsCiAgICAiY2xpZW50X2VtYWlsIjogImZ1ai1tYW5hZ2VtZW50LWJvdEBwbGF0ZWQtbWVzaC00ODkwMTAtczEuaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20iLAogICAgImNsaWVudF9pZCI6ICIxMDM0ODY2MjM3NTQxOTMzODgxMzQiLAogICAgImF1dGhfdXJpIjogImh0dHBzOi8vYWNjb3VudHMuZ29vZ2xlLmNvbS9vL29hdXRoMi9hdXRoIiwKICAgICJ0b2tlbl91cmkiOiAiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLAogICAgImF1dGhfcHJvdmlkZXJfeDUwOV9jZXJ0X3VybCI6ICJodHRwczovL3d3dy5nb29nbGVhcGlzLmNvbS9vYXV0aDIvdjEvY2VydHMiLAogICAgImNsaWVudF94NTA5X2NlcnRfdXJsIjogImh0dHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL3JvYm90L3YxL21ldGFkYXRhL3g1MDkvZnVqLW1hbmFnZW1lbnQtYm90JTQwcGxhdGVkLW1lc2gtNDg5MDEwLXMxLmlhbS5nc2VydmljZWFjY291bnQuY29tIiwKICAgICJ1bml2ZXJzZV9kb21haW4iOiAiZ29vZ2xlYXBpcy5jb20iCn0=
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: fuj-management-bot-credentials
|
||||
namespace: fuj
|
||||
type: Opaque
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: fuj-management
|
||||
namespace: fuj
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: fuj-management
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 8080
|
||||
protocol: TCP
|
||||
name: http
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: fujarna
|
||||
namespace: fuj
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: fujarna
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 8080
|
||||
protocol: TCP
|
||||
name: http
|
||||
@@ -1,11 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: ghost-on-kubernetes
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: namespace
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
@@ -1,17 +0,0 @@
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: ghost-config
|
||||
namespace: ghost-on-kubernetes
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
name: vault-backend
|
||||
kind: ClusterSecretStore
|
||||
target:
|
||||
name: ghost-config
|
||||
data:
|
||||
- secretKey: gmail-app-password
|
||||
remoteRef:
|
||||
key: k8s_home/ghost # Vault path (without 'data/' prefix)
|
||||
property: gmail-app-password
|
||||
@@ -1,42 +0,0 @@
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-mysql-env
|
||||
namespace: ghost-on-kubernetes
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
name: vault-backend
|
||||
kind: ClusterSecretStore
|
||||
target:
|
||||
name: ghost-on-kubernetes-mysql-env # resulting K8s secret name
|
||||
data:
|
||||
- secretKey: MYSQL_DATABASE # key in K8s secret
|
||||
remoteRef:
|
||||
key: k8s_home/ghost # Vault path (without 'data/' prefix)
|
||||
property: mysql-db-name # field within Vault secret
|
||||
- secretKey: MYSQL_USER # key in K8s secret
|
||||
remoteRef:
|
||||
key: k8s_home/ghost
|
||||
property: mysql-db-user
|
||||
- secretKey: MYSQL_PASSWORD
|
||||
remoteRef:
|
||||
key: k8s_home/ghost
|
||||
property: mysql-db-password
|
||||
- secretKey: MYSQL_ROOT_PASSWORD
|
||||
remoteRef:
|
||||
key: k8s_home/ghost
|
||||
property: mysql-db-root-password
|
||||
- secretKey: MYSQL_HOST
|
||||
remoteRef:
|
||||
key: k8s_home/ghost
|
||||
property: mysql-host
|
||||
|
||||
|
||||
# type: Opaque
|
||||
# stringData:
|
||||
# MYSQL_DATABASE: mysql-db-name # Same as in config.production.json
|
||||
# MYSQL_USER: mysql-db-user # Same as in config.production.json
|
||||
# MYSQL_PASSWORD: mysql-db-password # Same as in config.production.json
|
||||
# MYSQL_ROOT_PASSWORD: mysql-db-root-password # Same as in config.production.json
|
||||
# MYSQL_HOST: '%' # Same as in config.production.json
|
||||
@@ -1,21 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-mysql-env
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes-mysql
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-mysql-env
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: database-secret
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
type: Opaque
|
||||
stringData:
|
||||
MYSQL_DATABASE: mysql-db-name # Same as in config.production.json
|
||||
MYSQL_USER: mysql-db-user # Same as in config.production.json
|
||||
MYSQL_PASSWORD: mysql-db-password # Same as in config.production.json
|
||||
MYSQL_ROOT_PASSWORD: mysql-db-root-password # Same as in config.production.json
|
||||
MYSQL_HOST: '%' # Same as in config.production.json
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: tls-secret
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: tls-secret
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: tls-secret
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
|
||||
type: kubernetes.io/tls
|
||||
stringData:
|
||||
tls.crt: content-tls-crt-base64 # Optional, if you want to use your own TLS certificate
|
||||
tls.key: content-tls-key-base64 # Optional, if you want to use your own TLS certificate
|
||||
@@ -1,49 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: k8s-ghost-content
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: k8s-ghost-content
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: storage
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
spec:
|
||||
# Change this to your storageClassName, we suggest using a storageClassName that supports ReadWriteMany for production.
|
||||
storageClassName: freenas-iscsi
|
||||
volumeMode: Filesystem
|
||||
# Change this to your accessModes. We suggest ReadWriteMany for production, ReadWriteOnce for development.
|
||||
# With ReadWriteMany, you can have multiple replicas of Ghost, so you can achieve high availability.
|
||||
# Note that ReadWriteMany is not supported by all storage providers and may require additional configuration.
|
||||
# Ghost officialy doesn't support HA, they suggest using a CDN or caching. Info: https://ghost.org/docs/faq/clustering-sharding-multi-server/
|
||||
accessModes:
|
||||
- ReadWriteOnce # Change this to your accessModes if needed, we suggest ReadWriteMany so we can scale the deployment later.
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-mysql-pvc
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes-mysql
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-mysql-pvc
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: database-storage
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
|
||||
spec:
|
||||
storageClassName: freenas-iscsi
|
||||
volumeMode: Filesystem
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
@@ -1,49 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-service
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-service
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: service-frontend
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
|
||||
spec:
|
||||
ports:
|
||||
- port: 2368
|
||||
protocol: TCP
|
||||
targetPort: ghk8s
|
||||
name: ghk8s
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: ghost-on-kubernetes
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-mysql-service
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes-mysql
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-mysql-service
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: service-database
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
spec:
|
||||
ports:
|
||||
- port: 3306
|
||||
protocol: TCP
|
||||
targetPort: mysqlgh
|
||||
name: mysqlgh
|
||||
type: ClusterIP
|
||||
clusterIP: None
|
||||
selector:
|
||||
app: ghost-on-kubernetes-mysql
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: ghost-config-prod
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: ghost-config-prod
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: ghost-config
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
type: Opaque
|
||||
stringData:
|
||||
config.production.json: |-
|
||||
{
|
||||
"url": "https://ghost.lab.home.hrajfrisbee.cz",
|
||||
"admin": {
|
||||
"url": "https://ghost.lab.home.hrajfrisbee.cz"
|
||||
},
|
||||
"server": {
|
||||
"port": 2368,
|
||||
"host": "0.0.0.0"
|
||||
},
|
||||
"mail": {
|
||||
"transport": "SMTP",
|
||||
"from": "user@server.com",
|
||||
"options": {
|
||||
"service": "Google",
|
||||
"host": "smtp.gmail.com",
|
||||
"port": 465,
|
||||
"secure": true,
|
||||
"auth": {
|
||||
"user": "user@server.com",
|
||||
"pass": "passsword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"logging": {
|
||||
"transports": [
|
||||
"stdout"
|
||||
]
|
||||
},
|
||||
"database": {
|
||||
"client": "mysql",
|
||||
"connection":
|
||||
{
|
||||
"host": "ghost-on-kubernetes-mysql-service",
|
||||
"user": "mysql-db-user",
|
||||
"password": "mysql-db-password",
|
||||
"database": "mysql-db-name",
|
||||
"port": "3306"
|
||||
}
|
||||
},
|
||||
"process": "local",
|
||||
"paths": {
|
||||
"contentPath": "/home/nonroot/app/ghost/content"
|
||||
}
|
||||
}
|
||||
@@ -1,134 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-mysql
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes-mysql
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-mysql
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: database
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
spec:
|
||||
serviceName: ghost-on-kubernetes-mysql-service
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ghost-on-kubernetes-mysql
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: ghost-on-kubernetes-mysql
|
||||
spec:
|
||||
initContainers:
|
||||
- name: ghost-on-kubernetes-mysql-init
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
privileged: false
|
||||
readOnlyRootFilesystem: true
|
||||
image: docker.io/busybox:stable-musl
|
||||
imagePullPolicy: Always # You can change this value according to your needs
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
echo 'Changing ownership of mysql mount directory to 65534:65534'
|
||||
chown -Rfv 65534:65534 /mnt/mysql || echo 'Error changing ownership of mysql mount directory to 65534:65534'
|
||||
echo 'Changing ownership of tmp mount directory to 65534:65534'
|
||||
chown -Rfv 65534:65534 /mnt/tmp || echo 'Error changing ownership of tmp mount directory to 65534:65534'
|
||||
echo 'Changing ownership of socket mount directory to 65534:65534'
|
||||
chown -Rfv 65534:65534 /mnt/var/run/mysqld || echo 'Error changing ownership of socket mount directory to 65534:65534'
|
||||
|
||||
|
||||
volumeMounts:
|
||||
- name: ghost-on-kubernetes-mysql-volume
|
||||
mountPath: /mnt/mysql
|
||||
subPath: mysql-empty-subdir
|
||||
readOnly: false
|
||||
|
||||
- name: ghost-on-kubernetes-mysql-tmp
|
||||
mountPath: /mnt/tmp
|
||||
readOnly: false
|
||||
|
||||
- name: ghost-on-kubernetes-mysql-socket
|
||||
mountPath: /mnt/var/run/mysqld
|
||||
readOnly: false
|
||||
|
||||
# YOu can ajust the resources according to your needs
|
||||
resources:
|
||||
requests:
|
||||
memory: 0Mi
|
||||
cpu: 0m
|
||||
limits:
|
||||
memory: 1Gi
|
||||
cpu: 900m
|
||||
|
||||
containers:
|
||||
- name: ghost-on-kubernetes-mysql
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
privileged: false
|
||||
readOnlyRootFilesystem: true
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65534
|
||||
|
||||
image: docker.io/mysql:8.4
|
||||
imagePullPolicy: Always # You can change this value according to your needs
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: ghost-on-kubernetes-mysql-env
|
||||
resources:
|
||||
requests:
|
||||
memory: 500Mi # You can change this value according to your needs
|
||||
cpu: 300m # You can change this value according to your needs
|
||||
limits:
|
||||
memory: 1Gi # You can change this value according to your needs
|
||||
cpu: 900m # You can change this value according to your needs
|
||||
ports:
|
||||
- containerPort: 3306
|
||||
protocol: TCP
|
||||
name: mysqlgh
|
||||
volumeMounts:
|
||||
- name: ghost-on-kubernetes-mysql-volume
|
||||
mountPath: /var/lib/mysql
|
||||
subPath: mysql-empty-subdir
|
||||
readOnly: false
|
||||
|
||||
- name: ghost-on-kubernetes-mysql-tmp
|
||||
mountPath: /tmp
|
||||
readOnly: false
|
||||
|
||||
- name: ghost-on-kubernetes-mysql-socket
|
||||
mountPath: /var/run/mysqld
|
||||
readOnly: false
|
||||
|
||||
automountServiceAccountToken: false
|
||||
|
||||
# Optional: Uncomment the following to specify node selectors
|
||||
# affinity:
|
||||
# nodeAffinity:
|
||||
# requiredDuringSchedulingIgnoredDuringExecution:
|
||||
# nodeSelectorTerms:
|
||||
# - matchExpressions:
|
||||
# - key: node-role.kubernetes.io/worker
|
||||
# operator: In
|
||||
# values:
|
||||
# - 'true'
|
||||
|
||||
securityContext:
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
volumes:
|
||||
- name: ghost-on-kubernetes-mysql-volume
|
||||
persistentVolumeClaim:
|
||||
claimName: ghost-on-kubernetes-mysql-pvc
|
||||
- name: ghost-on-kubernetes-mysql-tmp
|
||||
emptyDir:
|
||||
sizeLimit: 128Mi
|
||||
- name: ghost-on-kubernetes-mysql-socket
|
||||
emptyDir:
|
||||
sizeLimit: 128Mi
|
||||
@@ -1,214 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ghost-on-kubernetes
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: ghost-on-kubernetes
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: ghost
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
|
||||
spec:
|
||||
# If you want HA for your Ghost instance, you can increase the number of replicas AFTER creation and you need to adjust the storage class. See 02-pvc.yaml for more information.
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ghost-on-kubernetes
|
||||
minReadySeconds: 5
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxUnavailable: 0
|
||||
maxSurge: 3
|
||||
revisionHistoryLimit: 4
|
||||
progressDeadlineSeconds: 600
|
||||
template:
|
||||
metadata:
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
spec:
|
||||
automountServiceAccountToken: false # Disable automounting of service account token
|
||||
volumes:
|
||||
- name: k8s-ghost-content
|
||||
persistentVolumeClaim:
|
||||
claimName: k8s-ghost-content
|
||||
|
||||
- name: ghost-config-prod
|
||||
secret:
|
||||
secretName: ghost-config-prod
|
||||
defaultMode: 420
|
||||
|
||||
- name: tmp
|
||||
emptyDir:
|
||||
sizeLimit: 64Mi
|
||||
|
||||
initContainers:
|
||||
- name: permissions-fix
|
||||
imagePullPolicy: Always
|
||||
image: docker.io/busybox:stable-musl
|
||||
env:
|
||||
- name: GHOST_INSTALL
|
||||
value: /home/nonroot/app/ghost
|
||||
- name: GHOST_CONTENT
|
||||
value: /home/nonroot/app/ghost/content
|
||||
- name: NODE_ENV
|
||||
value: production
|
||||
securityContext:
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
resources:
|
||||
limits:
|
||||
cpu: 900m
|
||||
memory: 1000Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
command:
|
||||
- /bin/sh
|
||||
- '-c'
|
||||
- |
|
||||
set -e
|
||||
|
||||
export DIRS='files logs apps themes data public settings images media'
|
||||
echo 'Check if base dirs exists, if not, create them'
|
||||
echo "Directories to check: $DIRS"
|
||||
for dir in $DIRS; do
|
||||
if [ ! -d $GHOST_CONTENT/$dir ]; then
|
||||
echo "Creating $GHOST_CONTENT/$dir directory"
|
||||
mkdir -pv $GHOST_CONTENT/$dir || echo "Error creating $GHOST_CONTENT/$dir directory"
|
||||
fi
|
||||
chown -Rfv 65532:65532 $GHOST_CONTENT/$dir && echo "chown ok on $dir" || echo "Error changing ownership of $GHOST_CONTENT/$dir directory"
|
||||
done
|
||||
exit 0
|
||||
|
||||
|
||||
volumeMounts:
|
||||
- name: k8s-ghost-content
|
||||
mountPath: /home/nonroot/app/ghost/content
|
||||
readOnly: false
|
||||
|
||||
containers:
|
||||
- name: ghost-on-kubernetes
|
||||
# For development, you can use the following image:
|
||||
# image: ghcr.io/sredevopsorg/ghost-on-kubernetes:latest-dev
|
||||
# image: ghcr.io/sredevopsorg/ghost-on-kubernetes:main
|
||||
image: ghost:bookworm
|
||||
imagePullPolicy: Always
|
||||
ports:
|
||||
- name: ghk8s
|
||||
containerPort: 2368
|
||||
protocol: TCP
|
||||
|
||||
# You should uncomment the following lines in production. Change the values according to your environment.
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ghost/api/v4/admin/site/
|
||||
port: ghk8s
|
||||
httpHeaders:
|
||||
- name: X-Forwarded-Proto
|
||||
value: https
|
||||
- name: Host
|
||||
value: ghost.lab.home.hrajfrisbee.cz
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
initialDelaySeconds: 10
|
||||
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /ghost/api/v4/admin/site/
|
||||
port: ghk8s
|
||||
httpHeaders:
|
||||
- name: X-Forwarded-Proto
|
||||
value: https
|
||||
- name: Host
|
||||
value: ghost.lab.home.hrajfrisbee.cz
|
||||
periodSeconds: 300
|
||||
timeoutSeconds: 3
|
||||
successThreshold: 1
|
||||
failureThreshold: 1
|
||||
initialDelaySeconds: 30
|
||||
|
||||
env:
|
||||
- name: NODE_ENV
|
||||
value: production
|
||||
- name: url
|
||||
value: "https://ghost.lab.home.hrajfrisbee.cz"
|
||||
- name: database__client
|
||||
value: "mysql"
|
||||
- name: database__connection__host
|
||||
value: "ghost-on-kubernetes-mysql-service"
|
||||
- name: database__connection__port
|
||||
value: "3306"
|
||||
- name: database__connection__user
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ghost-on-kubernetes-mysql-env
|
||||
key: MYSQL_USER
|
||||
- name: database__connection__password
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ghost-on-kubernetes-mysql-env
|
||||
key: MYSQL_PASSWORD
|
||||
- name: database__connection__database
|
||||
value: "ghost"
|
||||
- name: mail__transport
|
||||
value: "SMTP"
|
||||
- name: mail__options__service
|
||||
value: "Gmail"
|
||||
- name: mail__options__auth__user
|
||||
value: "kacerr.cz@gmail.com"
|
||||
- name: mail__options__auth__pass
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ghost-config
|
||||
key: gmail-app-password
|
||||
- name: mail__from
|
||||
value: "'Kacerr's Blog' <kacerr.cz@gmail.com>"
|
||||
|
||||
resources:
|
||||
limits:
|
||||
cpu: 800m
|
||||
memory: 800Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
|
||||
volumeMounts:
|
||||
- name: k8s-ghost-content
|
||||
mountPath: /home/nonroot/app/ghost/content
|
||||
readOnly: false
|
||||
- name: ghost-config-prod
|
||||
readOnly: true
|
||||
mountPath: /home/nonroot/app/ghost/config.production.json
|
||||
subPath: config.production.json
|
||||
- name: tmp # This is the temporary volume mount to allow loading themes
|
||||
mountPath: /tmp
|
||||
readOnly: false
|
||||
securityContext:
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65532
|
||||
|
||||
|
||||
restartPolicy: Always
|
||||
terminationGracePeriodSeconds: 15
|
||||
dnsPolicy: ClusterFirst
|
||||
# Optional: Uncomment the following to specify node selectors
|
||||
# affinity:
|
||||
# nodeAffinity:
|
||||
# requiredDuringSchedulingIgnoredDuringExecution:
|
||||
# nodeSelectorTerms:
|
||||
# - matchExpressions:
|
||||
# - key: node-role.kubernetes.io/worker
|
||||
# operator: In
|
||||
# values:
|
||||
# - 'true'
|
||||
securityContext: {}
|
||||
@@ -1,30 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-redirect
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-httproute
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: httproute
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: cilium-gateway
|
||||
namespace: kube-system
|
||||
sectionName: http
|
||||
hostnames:
|
||||
- ghost.lab.home.hrajfrisbee.cz
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
filters:
|
||||
- type: RequestRedirect
|
||||
requestRedirect:
|
||||
scheme: https
|
||||
statusCode: 301
|
||||
@@ -1,29 +0,0 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: ghost-on-kubernetes
|
||||
namespace: ghost-on-kubernetes
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-httproute
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: httproute
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: cilium-gateway
|
||||
namespace: kube-system
|
||||
sectionName: lab-home-hrajfrisbee-https-wildcard
|
||||
hostnames:
|
||||
- ghost.lab.home.hrajfrisbee.cz
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
backendRefs:
|
||||
- name: ghost-on-kubernetes-service
|
||||
namespace: ghost-on-kubernetes
|
||||
port: 2368
|
||||
@@ -1,33 +0,0 @@
|
||||
# Optional: If you have a domain name, you can create an Ingress resource to expose your Ghost blog to the internet.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: ghost-on-kubernetes-ingress
|
||||
namespace: ghost-on-kubernetes
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
labels:
|
||||
app: ghost-on-kubernetes
|
||||
app.kubernetes.io/name: ghost-on-kubernetes-ingress
|
||||
app.kubernetes.io/instance: ghost-on-kubernetes
|
||||
app.kubernetes.io/version: '6.0'
|
||||
app.kubernetes.io/component: ingress
|
||||
app.kubernetes.io/part-of: ghost-on-kubernetes
|
||||
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- ghost.lab.home.hrajfrisbee.cz
|
||||
secretName: tls-secret
|
||||
rules:
|
||||
- host: ghost.lab.home.hrajfrisbee.cz
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: ghost-on-kubernetes-service
|
||||
port:
|
||||
name: ghk8s
|
||||
@@ -11,13 +11,11 @@ spec:
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: ingress-nginx
|
||||
version: 4.14.1
|
||||
version: 4.12.0
|
||||
values:
|
||||
controller:
|
||||
admissionWebhooks:
|
||||
enabled: false
|
||||
patch:
|
||||
enabled: false
|
||||
config:
|
||||
annotations-risk-level: "Critical"
|
||||
interval: 5m0s
|
||||
29
gitops/home-kubernetes/kuard/deployment_kuard.yaml
Normal file
29
gitops/home-kubernetes/kuard/deployment_kuard.yaml
Normal file
@@ -0,0 +1,29 @@
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: kuard
|
||||
namespace: kuard
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: kuard
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: kuard
|
||||
spec:
|
||||
containers:
|
||||
- image: gcr.io/kuar-demo/kuard-amd64:1
|
||||
imagePullPolicy: Always
|
||||
name: kuard
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
resources:
|
||||
limits:
|
||||
cpu: 100m
|
||||
memory: 100Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 100Mi
|
||||
29
gitops/home-kubernetes/kuard/ingress_kuard.yaml
Normal file
29
gitops/home-kubernetes/kuard/ingress_kuard.yaml
Normal file
@@ -0,0 +1,29 @@
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: kuard
|
||||
namespace: kuard
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
rules:
|
||||
- host: test.kuard.dev
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: kuard
|
||||
port:
|
||||
number: 80
|
||||
- host: kuard.home.lab
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: kuard
|
||||
port:
|
||||
number: 80
|
||||
@@ -1,4 +1,5 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: plane
|
||||
name: kuard
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user