gitops: update fuj-management

Co-Authored-By: Claude <noreply@anthropic.com>
gitops/fuj: split into per-app manifests, add fuj-management deployment
2026-06-12 18:01:46 +00:00 · 2026-06-11 20:59:14 +02:00 · 2026-06-11 13:04:21 +02:00 · 2026-05-01 18:12:38 +02:00 · 2026-04-01 22:23:12 +02:00 · 2026-03-07 23:09:52 +01:00
85 changed files with 4822 additions and 10 deletions
--- a/.claude/settings.json
+++ b/.claude/settings.json
@@ -0,0 +1,10 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(for f:*)",
+      "Bash(do echo:*)",
+      "Read(//Users/jan.novak/srv/personal/home-kubernetes/**)",
+      "Bash(done)"
+    ]
+  }
+}
--- a/.gitignore
+++ b/.gitignore
@@ -5,3 +5,10 @@

 kubernetes-kvm-terraform/join-command.txt
 kubernetes-kvm-terraform/kubeconfig
+
+tmp/
+
+vms/utility-101-shadow/docker/monitoring/smtp_password
+
+docker-30/zot/sync-credentials.json
+kubernetes-kvm-terraform/gke_gcloud_auth_plugin_cache
--- a/blog/content/001-having-fun-with-gatewayApi.md
+++ b/blog/content/001-having-fun-with-gatewayApi.md
@@ -0,0 +1,7 @@
+# How i did have some fun with gatewayApi
+
+As nginx-ingress will be no longer maintained it looks like the time to upgrade is coming even for the lazy old schoolers who were very happy with it (even though annotations and custom snippets might not be the cleanest way, people experienced with nginx might have liked it for a reason). Although I tried to pretend that there is no gatewayApi and ingress is the thing for quite some time recently everything aligned for me to give it a go.
+
+## Which implementation to choose?
+
+There is a number of options and I'm only vaguely aware of them as I was not interested in it for a long time. But i have a tendency of trying to do something with Envoy (personal feeling that it is the right next thing here?) and I'm using the Cilium network plugin in my homelab kubernetes cluster. All this considered answer is obvious: use envoy gateway which is part of cilium.
--- a/docker-30/fuj-management/run.sh
+++ b/docker-30/fuj-management/run.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+docker rm -f fuj-management
+
+# gitea registry login with kacerr / token
+docker run -d --name fuj-management \
+  --restart=always \
+  -p 8081:5001 \
+  -v /srv/fuj-management/data:/app/data \
+  gitea.home.hrajfrisbee.cz/kacerr/fuj-management:latest
--- a/docker-30/gitea/docker-compose.yaml
+++ b/docker-30/gitea/docker-compose.yaml
@@ -94,8 +94,10 @@ services:
    environment:
      GITEA_INSTANCE_URL: https://gitea.home.hrajfrisbee.cz/
      GITEA_RUNNER_REGISTRATION_TOKEN: ${RUNNER_TOKEN}
+      CONFIG_FILE: /config/config.yaml
    volumes:
      - ./runner-data:/data
+      - ./runner-config.yaml:/config/config.yaml:ro
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - gitea-network
--- a/docker-30/kanidm/gitea-action-kubernetes-access.md
+++ b/docker-30/kanidm/gitea-action-kubernetes-access.md
@@ -0,0 +1,84 @@
+## 1. Create Kanidm service account + OAuth2 client
+
+```bash
+# Create a service account for CI
+kanidm service-account create gitea_ci "Gitea CI Deploy" idm_admins --name idm_admin
+
+# Create a group and add the service account
+kanidm group create k8s_deployers
+kanidm group add-members k8s_deployers gitea_ci
+
+# Create the OAuth2 client (or reuse existing k8s one)
+# If you already have a k8s OIDC client, just add scope maps:
+kanidm system oauth2 update-scope-map k8s k8s_deployers openid groups
+
+# Generate an API token for the service account
+kanidm service-account api-token generate --name idm_admin gitea_ci "gitea-ci-token"
+# ⚠️ Save the output token — this is the subject_token for exchange
+```
+
+## 2. RBAC in Kubernetes
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: gitea-ci-deploy
+subjects:
+  - kind: User
+    name: "gitea_ci@idm.home.hrajfrisbee.cz"  # matches preferred_username claim
+    apiGroup: rbac.authorization.k8s.io
+roleRef:
+  kind: ClusterRole
+  name: edit  # scope down as needed
+  apiGroup: rbac.authorization.k8s.io
+
+```
+
+## 3. Token exchange + kubeconfig setup (test in bash)
+
+```bash
+vault-login # prepared alias
+
+#!/usr/bin/env bash
+# set -euo pipefail
+
+KANIDM_URL="https://idm.home.hrajfrisbee.cz"
+OAUTH2_CLIENT_ID="k8s"          # your k8s OIDC client name in Kanidm
+API_TOKEN=$(vault kv get -format=json -mount="secret" "k8s_home/gitea/gitea-ci-token" |jq -r .data.data.token)
+K8S_API="https://192.168.0.31:6443"
+
+# Exchange the API token for an OIDC token via RFC 8693
+RESPONSE=$(curl -sf -X POST "${KANIDM_URL}/oauth2/token" \
+  -d "grant_type=urn:ietf:params:oauth:grant-type:token-exchange" \
+  -d "client_id=${OAUTH2_CLIENT_ID}" \
+  -d "subject_token=${API_TOKEN}" \
+  -d "subject_token_type=urn:ietf:params:oauth:token-type:access_token" \
+  -d "audience=${OAUTH2_CLIENT_ID}" \
+  -d "scope=openid groups")
+
+ID_TOKEN=$(echo "$RESPONSE" | jq -r '.id_token')
+
+# Inspect claims (sanity check)
+echo "$ID_TOKEN" | cut -d. -f2 | base64 -d 2>/dev/null | jq .
+
+# Build kubeconfig
+export KUBECONFIG=$(mktemp)
+kubectl config set-cluster mycluster \
+  --server="${K8S_API}" \
+  --certificate-authority=/path/to/ca.crt
+
+kubectl config set-credentials gitea-ci \
+  --token="${ID_TOKEN}"
+
+kubectl config set-context gitea-ci \
+  --cluster=mycluster \
+  --user=gitea-ci
+
+kubectl config use-context gitea-ci
+
+# Test
+kubectl auth whoami
+kubectl get ns
+
+```
--- a/docker-30/kanidm/readme.md
+++ b/docker-30/kanidm/readme.md
@@ -1,3 +1,27 @@
+## Upgrade
+
+```bash
+docker exec -it kanidmd kanidmd domain upgrade-check
+# make sure backup exists: /srv/docker/kanidm/data/kanidm/backups
+
+# change container image in: /srv/docker/kanidm/run.sh
+
+# kanidm data restore
+docker stop kanidmd
+docker run --rm -it \
+  -v kanidmd:/data \
+  -v kanidmd_backups:/backup \
+  kanidm/server:latest \
+  /sbin/kanidmd database restore -c /data/server.toml /backup/kanidm.backup.json
+docker start kanidmd
+```
+
+## Recover passwords from kanidm instance
+
+```bash
+docker exec -i -t kanidmd  kanidmd recover-account idm_admin
+```
+
 ## add user to k8s group

 based on: https://blog.kammel.dev/post/k8s_home_lab_2025_06/
@@ -115,4 +139,31 @@ docker run --rm -i -t -v --restart=always \
  docker.io/kanidm/server:latest \
  kanidmd cert-generate

+```
+
+
+## Service account for gitea runner
+
+```bash
+# create service account
+#kanidm service-account create \
+#  gitea_ci \              # account name
+#  "Gitea CI Deploy" \     # display name
+#  idm_admins \            # entry-managed-by (delegation group)
+#  --name idm_admin        # authenticate as this user
+
+kanidm service-account create gitea_ci "Gitea CI Deploy" idm_admins --name idm_admin
+
+# Create a group and add the service account
+kanidm group create k8s_deployers
+kanidm group add-members k8s_deployers gitea_ci
+
+# Create the OAuth2 client (or reuse existing k8s one)
+# If you already have a k8s OIDC client, just add scope maps:
+kanidm system oauth2 update-scope-map k8s k8s_deployers openid groups
+
+# Generate an API token for the service account
+kanidm service-account api-token generate --name idm_admin gitea_ci "gitea-ci-token"
+# ⚠️ Save the output token — this is the subject_token for exchange
+
 ```
--- a/docker-30/kanidm/run.sh
+++ b/docker-30/kanidm/run.sh
@@ -0,0 +1,9 @@
+docker rm -f kanidmd
+
+docker run -d --name=kanidmd --restart=always \
+  -p '8443:8443' \
+  -p '3636:3636' \
+  --volume /srv/docker/kanidm/data:/data \
+  docker.io/kanidm/server:1.9.1
+
+# previous version: 1.8.5
--- a/docker-30/maru-hleda-byt/run.sh
+++ b/docker-30/maru-hleda-byt/run.sh
@@ -4,6 +4,7 @@ docker rm -f maru-hleda-byt

 # gitea registry login with kacerr / token
 docker run -d --name maru-hleda-byt \
+  --restart=always \
  -p 8080:8080 \
  -v /srv/maru-hleda-byt/data:/app/data \
  gitea.home.hrajfrisbee.cz/littlemeat/maru-hleda-byt:0.01
--- a/docker-30/vault/gitea-access-into-vault.md
+++ b/docker-30/vault/gitea-access-into-vault.md
@@ -0,0 +1,10 @@
+## 1. Enable & configure JWT auth in Vault
+
+```bash
+vault auth enable jwt
+
+vault write auth/jwt/config \
+  bound_issuer="https://gitea.home.hrajfrisbee.cz" \
+  jwks_url="https://gitea.home.hrajfrisbee.cz/login/oauth/keys"
+
+```
--- a/docker-30/vault/terraform/main.tf
+++ b/docker-30/vault/terraform/main.tf
@@ -4,6 +4,13 @@ resource "vault_mount" "kv" {
  description = "KV v2 secrets engine"
 }

+resource "vault_jwt_auth_backend" "gitea" {
+  path               = "jwt"
+  type               = "jwt"
+  bound_issuer       = "https://gitea.home.hrajfrisbee.cz"
+  jwks_url           = "https://gitea.home.hrajfrisbee.cz/login/oauth/keys"
+}
+
 resource "vault_policy" "eso_read" {
  name   = "external-secrets-read"
  policy = <<-EOT
@@ -16,6 +23,37 @@ resource "vault_policy" "eso_read" {
  EOT
 }

+# for now i allow my gitea to read everything in /v1/secret/data/gitea
+resource "vault_policy" "gitea_ci_read" {
+  name   = "gitea-ci-read"
+  policy = <<-EOT
+    path "${vault_mount.kv.path}/data/gitea/*" {
+      capabilities = ["read"]
+    }
+    path "${vault_mount.kv.path}/metadata/gitea/*" {
+      capabilities = ["read", "list"]
+    }
+  EOT
+}
+
+resource "vault_jwt_auth_backend_role" "gitea_ci" {
+  backend        = vault_jwt_auth_backend.gitea.path
+  role_name      = "gitea-ci"
+  role_type      = "jwt"
+  token_policies = [vault_policy.gitea_ci_read.name]
+
+  user_claim      = "sub"
+  bound_audiences = ["https://gitea.home.hrajfrisbee.cz"]
+
+  # allow any valid jwt token when commented out
+  # bound_claims = {
+  #   repository = "myorg/repo1,myorg/repo3"
+  # }
+
+  token_ttl     = 600
+  token_max_ttl = 1200
+}
+
 resource "vault_auth_backend" "approle" {
  type = "approle"
 }
@@ -46,4 +84,32 @@ output "role_id" {
 output "secret_id" {
  value     = vault_approle_auth_backend_role_secret_id.eso.secret_id
  sensitive = true
+}
+
+resource "vault_approle_auth_backend_role" "gitea_ci" {
+  backend        = vault_auth_backend.approle.path
+  role_name      = "gitea-ci"
+  token_policies = [vault_policy.gitea_ci_read.name]
+  token_ttl      = 600
+  token_max_ttl  = 1200
+}
+
+data "vault_approle_auth_backend_role_id" "gitea_ci" {
+  backend   = vault_auth_backend.approle.path
+  role_name = vault_approle_auth_backend_role.gitea_ci.role_name
+}
+
+resource "vault_approle_auth_backend_role_secret_id" "gitea_ci" {
+  backend   = vault_auth_backend.approle.path
+  role_name = vault_approle_auth_backend_role.gitea_ci.role_name
+}
+
+output "gitea_ci_role_id" {
+  value     = data.vault_approle_auth_backend_role_id.gitea_ci.role_id
+  sensitive = true
+}
+
+output "gitea_ci_secret_id" {
+  value     = vault_approle_auth_backend_role_secret_id.gitea_ci.secret_id
+  sensitive = true
 }
--- a/docker-30/vault/terraform/terraform.tfstate
+++ b/docker-30/vault/terraform/terraform.tfstate
--- a/docker-30/vault/terraform/terraform.tfstate.backup
+++ b/docker-30/vault/terraform/terraform.tfstate.backup
--- a/docker-30/zot/config.yaml
+++ b/docker-30/zot/config.yaml
@@ -0,0 +1,52 @@
+distSpecVersion: "1.1.0"
+storage:
+  rootDirectory: /var/lib/zot
+  gc: true
+  gcDelay: "24h"
+  dedupe: true
+http:
+  address: 0.0.0.0
+  port: 5000
+  compat:
+  - docker2s2
+log:
+  # level: info
+  level: debug
+extensions:
+  ui:
+    enable: true
+  search:
+    enable: true
+  sync:
+    enable: true
+    credentialsFile: "/etc/zot/sync-credentials.json"
+    registries:
+      - urls: ["https://registry-1.docker.io"]
+        onDemand: true
+        tlsVerify: true
+        content:
+          - prefix: "library/**"
+            destination: "/docker.io/library"
+          - prefix: "democraticcsi/**"
+            destination: "/democraticcsi"
+          - prefix: "**"
+            destination: "/docker.io"
+
+      - urls: ["https://registry.k8s.io"]
+        onDemand: true
+        tlsVerify: true
+        content:
+          - prefix: "**"
+            destination: "/registry.k8s.io"
+      - urls: ["https://ghcr.io"]
+        onDemand: true
+        tlsVerify: true
+        content:
+          - prefix: "**"
+            destination: "/ghcr.io"
+      - urls: ["https://quay.io"]
+        onDemand: true
+        tlsVerify: true
+        content:
+          - prefix: "**"
+            destination: "/quay.io"
--- a/docker-30/zot/docker-compose.yaml
+++ b/docker-30/zot/docker-compose.yaml
@@ -0,0 +1,12 @@
+services:
+  zot:
+    image: ghcr.io/project-zot/zot-linux-amd64:latest
+    container_name: zot
+    restart: unless-stopped
+    command: serve /etc/zot/config.yaml
+    ports:
+      - "5000:5000"
+    volumes:
+      - ./config.yaml:/etc/zot/config.yaml:ro
+      - ./sync-credentials.json:/etc/zot/sync-credentials.json:ro
+      - /srv/container-registry-data:/var/lib/zot
--- a/experiments/e2b/dev-vm-deployment-plan.md
+++ b/experiments/e2b/dev-vm-deployment-plan.md
@@ -0,0 +1,391 @@
+# E2B Dev VM Setup on KVM Homelab
+
+## Context
+
+The user wants to run the E2B infrastructure dev stack on a KVM virtual machine in their homelab. E2B uses Firecracker microVMs (which need `/dev/kvm`), so the guest VM needs **nested virtualization** (KVM-in-KVM). This guide covers VM creation, OS configuration, toolchain installation, and running the full dev stack.
+
+> **Note from upstream**: [DEV-LOCAL.md](DEV-LOCAL.md) says "Linux is required. This is a work in progress. Not everything will function as expected."
+
+---
+
+## Phase 1: Create the KVM VM on the Hypervisor Host
+
+### 1.1 Enable nested virtualization on the host
+
+**Intel:**
+
+```bash
+cat /sys/module/kvm_intel/parameters/nested   # check
+sudo modprobe -r kvm_intel && sudo modprobe kvm_intel nested=1
+echo "options kvm_intel nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf
+```
+
+**AMD:**
+
+```bash
+cat /sys/module/kvm_amd/parameters/nested
+sudo modprobe -r kvm_amd && sudo modprobe kvm_amd nested=1
+echo "options kvm_amd nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf
+
+# get details about loaded kernel module
+systool -v -m kvm_amd
+```
+
+### 1.2 Create the VM
+
+```bash
+virt-install \
+  --name e2b-dev \
+  --ram 16384 \
+  --vcpus 8 \
+  --cpu host-passthrough \
+  --os-variant ubuntu24.04 \
+  --disk path=/var/lib/libvirt/images/e2b-dev.qcow2,size=100,format=qcow2,bus=virtio \
+  --network bridge=virbr0,model=virtio \
+  --graphics none \
+  --console pty,target_type=serial \
+  --cdrom /path/to/ubuntu-24.04-live-server-amd64.iso \
+  --extra-args 'console=ttyS0,115200n8'
+```
+
+**Why these specs:**
+| Resource | Value | Rationale |
+|----------|-------|-----------|
+| RAM | 16 GB (24-32 better) | 4 GB for huge pages (2048 x 2MB), ~4 GB for 10 Docker containers, rest for Go services + Firecracker VMs |
+| vCPUs | 8 | Firecracker VMs consume vCPUs; Go services are concurrent |
+| Disk | 100 GB | Docker images, FC binaries, kernels, rootfs, Go cache, build artifacts |
+| CPU | `host-passthrough` | **Mandatory** -- exposes VMX/SVM to guest so `/dev/kvm` works inside the VM |
+
+### 1.3 Alternative: libvirt XML
+
+If you manage VMs declaratively, the critical part is:
+
+```xml
+<cpu mode='host-passthrough' check='none' migratable='off'/>
+```
+
+### 1.4 Verify nested KVM works (after OS install)
+
+```bash
+ls -la /dev/kvm                                    # must exist
+lsmod | grep kvm                                   # kvm + kvm_intel/kvm_amd
+grep -cE '(vmx|svm)' /proc/cpuinfo                # must be > 0
+```
+
+If `/dev/kvm` is missing, go back to 1.1.
+
+---
+
+## Phase 2: OS Configuration (inside the guest VM)
+
+**Recommended OS:** Ubuntu 24.04 LTS Server (matches CI, well-tested with Firecracker)
+
+### 2.1 Base packages
+
+```bash
+sudo apt update && sudo apt upgrade -y
+sudo apt install -y \
+  build-essential git curl wget unzip jq make gcc pkg-config \
+  iptables iproute2 net-tools ca-certificates gnupg \
+  lsb-release software-properties-common gettext-base
+```
+
+### 2.2 Kernel modules
+
+```bash
+# Load now
+sudo modprobe nbd nbds_max=64
+sudo modprobe kvm
+sudo modprobe kvm_intel   # or kvm_amd
+sudo modprobe tun
+sudo modprobe veth
+sudo modprobe nf_tables
+sudo modprobe nft_nat
+
+# Persist across reboots
+cat <<'EOF' | sudo tee /etc/modules-load.d/e2b.conf
+nbd
+kvm
+kvm_intel
+tun
+veth
+nf_tables
+nft_nat
+EOF
+
+echo "options nbd nbds_max=64" | sudo tee /etc/modprobe.d/nbd.conf
+```
+
+### 2.3 Sysctl
+
+```bash
+cat <<'EOF' | sudo tee /etc/sysctl.d/99-e2b.conf
+vm.nr_hugepages=2048
+vm.max_map_count=1048576
+vm.swappiness=10
+vm.vfs_cache_pressure=50
+net.ipv4.ip_forward=1
+net.core.somaxconn=65535
+net.core.netdev_max_backlog=65535
+net.ipv4.tcp_max_syn_backlog=65535
+EOF
+
+sudo sysctl --system
+```
+
+### 2.4 Udev rules (suppress NBD inotify noise)
+
+```bash
+cat <<'EOF' | sudo tee /etc/udev/rules.d/99-e2b-nbd.rules
+KERNEL=="nbd*", OPTIONS+="nowatch"
+EOF
+sudo udevadm control --reload-rules && sudo udevadm trigger
+```
+
+### 2.5 File descriptor limits
+
+```bash
+cat <<'EOF' | sudo tee /etc/security/limits.d/e2b.conf
+*    soft    nofile    1048576
+*    hard    nofile    1048576
+root soft    nofile    1048576
+root hard    nofile    1048576
+EOF
+```
+
+---
+
+## Phase 3: Install Toolchain
+
+### 3.1 Docker
+
+```bash
+curl -fsSL https://get.docker.com | sh
+sudo usermod -aG docker $USER
+# Log out and back in
+docker --version && docker compose version
+```
+
+### 3.2 mise (manages all tools from `.tool-versions`)
+
+```bash
+curl https://mise.run | sh
+echo 'eval "$(~/.local/bin/mise activate bash)"' >> ~/.bashrc
+source ~/.bashrc
+```
+
+### 3.3 Install project tools
+
+The repo's [.tool-versions](.tool-versions) pins:
+
+| Tool | Version |
+| --- | --- |
+| golang | 1.25.4 |
+| buf | 1.28.1 |
+| bun | 1.3.2 |
+| protoc | 29.3 |
+| protoc-gen-go | 1.28.1 |
+| protoc-gen-go-grpc | 1.6.1 |
+| protoc-gen-connect-go | 1.18.1 |
+| golangci-lint | 2.8.0 |
+| terraform | 1.5.7 |
+| packer | 1.13.1 |
+| python | 3.13.11 |
+| gcloud | 534.0.0 |
+
+```bash
+cd ~/e2b-infra   # after cloning
+mise install     # installs everything from .tool-versions
+```
+
+> **Note:** `gcloud` is only needed for `gsutil` to download prebuilt artifacts. If you don't have GCP credentials, you can download them via HTTPS instead (see Phase 5).
+
+---
+
+## Phase 4: Clone Repository
+
+```bash
+git clone https://github.com/e2b-dev/infra.git ~/e2b-infra
+cd ~/e2b-infra
+go work sync
+```
+
+---
+
+## Phase 5: Download Prebuilt Artifacts
+
+Firecracker binaries and Linux kernels must be downloaded from the public GCS bucket.
+
+**With gsutil (if gcloud is configured):**
+
+```bash
+make download-public-kernels
+make download-public-firecrackers
+```
+
+**Without GCP credentials (HTTPS alternative):**
+
+The bucket `e2b-prod-public-builds` is publicly accessible. You'll need to browse/list it to find available versions, then download manually:
+
+```bash
+# Install gsutil standalone (no GCP project needed for public buckets)
+# OR use curl/wget against:
+#   https://storage.googleapis.com/e2b-prod-public-builds/
+# to discover and download kernel and firecracker builds
+```
+
+The simplest path is to install just the `gcloud` CLI (via mise) and run the make targets -- no GCP project or auth is needed for public bucket reads.
+
+---
+
+## Phase 6: Prepare Local Environment
+
+### 6.1 Start infrastructure containers
+
+```bash
+make local-infra
+# Starts: PostgreSQL 17.4, Redis 7.4.2, ClickHouse 25.4.5.24,
+#          Grafana 12.0.0, Loki 3.4.1, Tempo 2.8.2, Mimir 2.17.1,
+#          OTEL Collector 0.146.0, Vector, Memcached 1.6.38
+```
+
+Wait for all containers to be healthy:
+
+```bash
+docker compose -f packages/local-dev/docker-compose.yaml ps
+```
+
+### 6.2 Initialize databases
+
+```bash
+make -C packages/db migrate-local              # PostgreSQL
+make -C packages/clickhouse migrate-local      # ClickHouse
+```
+
+### 6.3 Build envd (in-VM daemon)
+
+```bash
+make -C packages/envd build
+```
+
+### 6.4 Seed database with dev credentials
+
+```bash
+make -C packages/local-dev seed-database
+```
+
+Creates test user, team, API key, and access token for local development.
+
+---
+
+## Phase 7: Run the Dev Stack
+
+Each service runs in the foreground. Use **tmux**, **screen**, or separate SSH sessions.
+
+| Terminal | Command | Listens on |
+| --- | --- | --- |
+| 1   | `make local-infra` | (Docker containers) |
+| 2   | `make -C packages/api run-local` | `:3000` |
+| 3   | `make -C packages/orchestrator build-debug && sudo make -C packages/orchestrator run-local` | `:5008` |
+| 4   | `make -C packages/client-proxy run-local` | `:3002` |
+
+> The orchestrator **requires sudo** -- Firecracker needs root for `/dev/kvm`, network namespaces, veth pairs, nftables rules, and NBD devices.
+
+---
+
+## Phase 8: Verify
+
+### 8.1 Health checks
+
+```bash
+curl -s http://localhost:3000/health              # API
+curl -s -o /dev/null -w "%{http_code}" http://localhost:53000   # Grafana (expect 302)
+curl -s 'http://localhost:8123/?query=SELECT%201'  # ClickHouse
+redis-cli -h localhost -p 6379 ping                # Redis
+```
+
+### 8.2 Build the base template
+
+```bash
+make -C packages/shared/scripts local-build-base-template
+```
+
+### 8.3 Test with E2B client SDK
+
+```bash
+export E2B_API_KEY=e2b_53ae1fed82754c17ad8077fbc8bcdd90
+export E2B_ACCESS_TOKEN=sk_e2b_89215020937a4c989cde33d7bc647715
+export E2B_API_URL=http://localhost:3000
+export E2B_SANDBOX_URL=http://localhost:3002
+# Use E2B SDK/CLI to create a sandbox
+```
+
+---
+
+## Phase 9: Access from Host (Optional)
+
+### SSH port forwarding
+
+```bash
+ssh -N \
+  -L 3000:localhost:3000 \
+  -L 3002:localhost:3002 \
+  -L 53000:localhost:53000 \
+  -L 5432:localhost:5432 \
+  -L 8123:localhost:8123 \
+  user@<vm-ip>
+```
+
+### Or use bridged networking
+
+If the VM has a routable IP on your LAN, services are directly accessible (Docker binds to `0.0.0.0`, Go services listen on all interfaces).
+
+---
+
+## Service Endpoints Reference
+
+| Service | URL |
+| --- | --- |
+| E2B API | `http://localhost:3000` |
+| E2B Client Proxy | `http://localhost:3002` |
+| E2B Orchestrator | `http://localhost:5008` |
+| Grafana | `http://localhost:53000` |
+| PostgreSQL | `postgres://postgres:postgres@localhost:5432` |
+| ClickHouse (HTTP) | `http://localhost:8123` |
+| ClickHouse (native) | `localhost:9000` |
+| Redis | `localhost:6379` |
+| OTEL Collector (gRPC) | `localhost:4317` |
+| OTEL Collector (HTTP) | `localhost:4318` |
+| Loki | `http://localhost:3100` |
+| Vector | `localhost:30006` |
+
+---
+
+## Troubleshooting
+
+| Problem | Fix |
+| --- | --- |
+| `/dev/kvm` missing in guest | Enable nested virt on host (Phase 1.1), use `host-passthrough` CPU |
+| `modprobe nbd` fails | Check kernel has NBD support: `modinfo nbd` |
+| Orchestrator permission errors | Must run with `sudo` |
+| Huge pages < 2048 | Not enough contiguous memory; increase VM RAM or set earlier in boot |
+| Docker containers won't start | Check `systemctl status docker`, port conflicts with `ss -tlnp` |
+| `gsutil` not found | Install via `mise install gcloud` or download artifacts via HTTPS |
+
+---
+
+## After VM Reboot Checklist
+
+```bash
+# 1. Verify kernel modules and sysctl (should be persistent)
+lsmod | grep nbd
+cat /proc/sys/vm/nr_hugepages   # expect 2048
+
+# 2. Start infra
+cd ~/e2b-infra && make local-infra
+
+# 3. Start services (separate terminals)
+make -C packages/api run-local
+sudo make -C packages/orchestrator run-local
+make -C packages/client-proxy run-local
+```
--- a/experiments/e2b/installation.md
+++ b/experiments/e2b/installation.md
@@ -0,0 +1,95 @@
+## dev-vm on beelink
+
+```bash
+virt-install \
+  --name e2b-dev \
+  --ram 16384 \
+  --vcpus 8 \
+  --cpu host-passthrough \
+  --os-variant ubuntu24.04 \
+  --disk path=/srv/vms/e2b-dev.qcow2,size=100,format=qcow2,bus=virtio \
+  --network bridge=br0,model=virtio \
+  --graphics none \
+  --console pty,target_type=serial \
+  --location /srv/vms/isos/ubuntu-24.04.3-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
+  --extra-args 'console=ttyS0,115200n8'
+
+
+# base packages
+sudo apt update && sudo apt upgrade -y
+sudo apt install -y \
+  build-essential git curl wget unzip jq make gcc pkg-config \
+  iptables iproute2 net-tools ca-certificates gnupg \
+  lsb-release software-properties-common gettext-base
+
+
+# kernel modules
+# Load now
+sudo modprobe nbd nbds_max=64
+sudo modprobe kvm
+sudo modprobe kvm_amd   # or kvm_amd
+sudo modprobe tun
+sudo modprobe veth
+sudo modprobe nf_tables
+sudo modprobe nft_nat
+
+# Persist across reboots
+cat <<'EOF' | sudo tee /etc/modules-load.d/e2b.conf
+nbd
+kvm
+kvm_amd
+tun
+veth
+nf_tables
+nft_nat
+EOF
+
+echo "options nbd nbds_max=64" | sudo tee /etc/modprobe.d/nbd.conf
+
+# sysctl
+cat <<'EOF' | sudo tee /etc/sysctl.d/99-e2b.conf
+vm.nr_hugepages=2048
+vm.max_map_count=1048576
+vm.swappiness=10
+vm.vfs_cache_pressure=50
+net.ipv4.ip_forward=1
+net.core.somaxconn=65535
+net.core.netdev_max_backlog=65535
+net.ipv4.tcp_max_syn_backlog=65535
+EOF
+
+sudo sysctl --system
+
+# udev rules
+cat <<'EOF' | sudo tee /etc/udev/rules.d/99-e2b-nbd.rules
+KERNEL=="nbd*", OPTIONS+="nowatch"
+EOF
+sudo udevadm control --reload-rules && sudo udevadm trigger
+
+# file descriptor limits
+cat <<'EOF' | sudo tee /etc/security/limits.d/e2b.conf
+*    soft    nofile    1048576
+*    hard    nofile    1048576
+root soft    nofile    1048576
+root hard    nofile    1048576
+EOF
+```
+
+## install toolchain
+
+```bash
+# docker 
+curl -fsSL https://get.docker.com | sh
+sudo usermod -aG docker $USER
+# Log out and back in
+docker --version && docker compose version
+
+# mise
+curl https://mise.run | sh
+echo 'eval "$(~/.local/bin/mise activate bash)"' >> ~/.bashrc
+source ~/.bashrc
+```
+
+
+```
+
--- a/experiments/e2b/sanbox-usage-generated-from-infra-repo.md
+++ b/experiments/e2b/sanbox-usage-generated-from-infra-repo.md
@@ -0,0 +1,280 @@
+# E2B Sandbox Usage Guide
+
+A practical guide for creating, observing, and using E2B sandboxes via the API.
+
+## Prerequisites
+
+### Local Dev Setup
+
+Seed the database to create a test team and API key:
+
+```bash
+make -C packages/local-dev seed-database
+```
+
+### Environment Variables
+
+```bash
+E2B_API_KEY=e2b_53ae1fed82754c17ad8077fbc8bcdd90
+E2B_ACCESS_TOKEN=sk_e2b_89215020937a4c989cde33d7bc647715
+E2B_API_URL=http://localhost:3000
+E2B_SANDBOX_URL=http://localhost:3002
+```
+
+All examples below use `$E2B_API_URL` and `$E2B_API_KEY` — export them in your shell for convenience:
+
+```bash
+export E2B_API_URL=http://localhost:3000
+export E2B_API_KEY=e2b_53ae1fed82754c17ad8077fbc8bcdd90
+```
+
+---
+
+## 1. Create a Sandbox
+
+```bash
+curl -X POST $E2B_API_URL/sandboxes \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "templateID": "base",
+    "timeout": 300
+  }'
+```
+
+**Response:**
+
+```json
+{
+  "sandboxID": "i1234567890abcdef",
+  "templateID": "base",
+  "envdVersion": "0.5.8",
+  "domain": "i1234567890abcdef.your-domain"
+}
+```
+
+Save the `sandboxID` and `domain` from the response for subsequent commands.
+
+### Optional Parameters
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `secure` | bool | Returns an `envdAccessToken` for authenticated envd access |
+| `envVars` | object | Inject environment variables into the VM |
+| `metadata` | object | Tag the sandbox for filtering (e.g. `{"purpose": "test"}`) |
+| `autoPause` | bool | Pause instead of kill on timeout |
+| `allow_internet_access` | bool | Allow internet egress from the VM |
+| `network.allowPublicTraffic` | bool | Allow inbound public traffic |
+| `network.allowOut` | array | Allowed egress destinations (IPs, CIDRs, domains) |
+| `network.denyOut` | array | Denied egress destinations (IPs/CIDRs only) |
+| `volumeMounts` | array | Mount persistent volumes (`{"name": "vol", "path": "/data"}`) |
+
+**Example with options:**
+
+```bash
+curl -X POST $E2B_API_URL/sandboxes \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "templateID": "base",
+    "timeout": 300,
+    "secure": true,
+    "envVars": {"MY_VAR": "hello"},
+    "metadata": {"purpose": "demo"}
+  }'
+```
+
+---
+
+## 2. Observe Sandbox State
+
+### Get details of a specific sandbox
+
+```bash
+curl $E2B_API_URL/sandboxes/{sandboxID} \
+  -H "X-API-Key: $E2B_API_KEY"
+```
+
+Returns state (`running`/`paused`), CPU/memory/disk config, network settings, and TTL (`endAt`).
+
+### List all sandboxes
+
+```bash
+# Running only
+curl "$E2B_API_URL/v2/sandboxes?state=running" \
+  -H "X-API-Key: $E2B_API_KEY"
+
+# Running and paused
+curl "$E2B_API_URL/v2/sandboxes?state=running&state=paused" \
+  -H "X-API-Key: $E2B_API_KEY"
+
+# Filter by metadata
+curl "$E2B_API_URL/v2/sandboxes?metadata=purpose%3Ddemo" \
+  -H "X-API-Key: $E2B_API_KEY"
+```
+
+### Get resource metrics (CPU, memory, disk)
+
+```bash
+curl "$E2B_API_URL/sandboxes/{sandboxID}/metrics?start=$(date -v-5M +%s)&end=$(date +%s)" \
+  -H "X-API-Key: $E2B_API_KEY"
+```
+
+**Response:**
+
+```json
+[
+  {
+    "timestampUnix": 1234567890,
+    "cpuCount": 2,
+    "cpuUsedPct": 25.5,
+    "memUsed": 268435456,
+    "memTotal": 536870912,
+    "diskUsed": 1073741824,
+    "diskTotal": 5368709120
+  }
+]
+```
+
+### Get sandbox logs
+
+```bash
+curl "$E2B_API_URL/v2/sandboxes/{sandboxID}/logs?limit=100&direction=backward" \
+  -H "X-API-Key: $E2B_API_KEY"
+```
+
+Optional query parameters: `cursor` (ms timestamp), `level` (min log level), `search` (substring match).
+
+---
+
+## 3. Use the Sandbox
+
+The in-VM daemon (**envd**) runs on port 49983 inside each sandbox, exposed via the sandbox's `domain`.
+
+### Upload a file
+
+```bash
+curl -X POST https://{domain}/files \
+  -H "Content-Type: multipart/form-data" \
+  -F "file=@script.py" \
+  -F "path=/home/user/script.py"
+```
+
+### Using the E2B Python SDK
+
+Point the SDK at your local API:
+
+```python
+from e2b import Sandbox
+
+sbx = Sandbox("base", api_url="http://localhost:3000")
+
+# Run a command
+result = sbx.commands.run("echo 'Hello from Firecracker VM!'")
+print(result.stdout)
+
+# Write and execute a file
+sbx.files.write("/home/user/hello.py", "print('Hello world')")
+result = sbx.commands.run("python3 /home/user/hello.py")
+print(result.stdout)
+
+# List files
+files = sbx.files.list("/home/user")
+for f in files:
+    print(f.name)
+
+sbx.kill()
+```
+
+### Envd Connect RPC API
+
+The envd daemon exposes Connect RPC services for programmatic access:
+
+**Process Service:**
+- `Start(ProcessConfig)` — start a new process
+- `List()` — list running processes
+- `Connect(ProcessSelector)` — connect to process stdio
+- `Signal(ProcessSelector, Signal)` — send signal to process
+
+**Filesystem Service:**
+- `ListDir(Path)` — list directory contents
+- `Stat(Path)` — get file metadata
+- `WatchDir(Path)` — watch for changes
+- `Move(Source, Dest)` — move/rename file
+- `RemoveDir(Path)` — remove directory
+
+---
+
+## 4. Lifecycle Management
+
+### Extend timeout
+
+```bash
+# Add 60 seconds to the TTL
+curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/refreshes \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"duration": 60}'
+
+# Or set an absolute timeout (seconds from now)
+curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/timeout \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"timeout": 120}'
+```
+
+### Pause (snapshot to disk)
+
+```bash
+curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/pause \
+  -H "X-API-Key: $E2B_API_KEY"
+```
+
+### Resume / reconnect
+
+```bash
+curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/connect \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"timeout": 300}'
+```
+
+Returns `200` if already running, `201` if resumed from paused state.
+
+### Kill
+
+```bash
+curl -X DELETE $E2B_API_URL/sandboxes/{sandboxID} \
+  -H "X-API-Key: $E2B_API_KEY"
+```
+
+### Create a snapshot (template from running sandbox)
+
+```bash
+curl -X POST $E2B_API_URL/sandboxes/{sandboxID}/snapshots \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"name": "my-snapshot"}'
+```
+
+---
+
+## 5. Network Configuration
+
+Update network rules on a running sandbox:
+
+```bash
+curl -X PUT $E2B_API_URL/sandboxes/{sandboxID}/network \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "allowOut": ["8.8.8.8", "example.com"],
+    "denyOut": ["0.0.0.0/0"]
+  }'
+```
+
+**Rules:**
+- If `allowOut` contains domain names, `denyOut` must include `0.0.0.0/0`
+- Domain names are not supported in `denyOut` (IPs/CIDRs only)
+- `allowOut` entries take precedence over `denyOut`
+- Omitting both fields clears all rules
--- a/experiments/e2b/using-e2b-infra.md
+++ b/experiments/e2b/using-e2b-infra.md
@@ -0,0 +1,46 @@
+## VMs
+
+```bash
+
+E2B_API_URL=http://192.168.0.61:3000
+
+# create vm
+curl -X POST $E2B_API_URL/sandboxes \
+  -H "X-API-Key: $E2B_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "templateID": "base",
+    "timeout": 300
+  }'
+
+# response
+{"alias":"base","clientID":"6532622b","domain":null,"envdVersion":"0.5.8","sandboxID":"in3h60s6h0ie3kigrcls4","templateID":"pz1l1owhmy0w84e12eqv","trafficAccessToken":null}
+
+sandboxID=in3h60s6h0ie3kigrcls4
+
+# create sandboxVM and capture id
+sandboxID=$(curl -s -X POST http://localhost:3000/sandboxes   -H "X-API-Key: $E2B_API_KEY"   -H "Content-Type: application/json"   -d '{
+    "templateID": "base",
+    "timeout": 300
+  }' | jq -r .sandboxID
+)
+
+
+# get state
+curl http://localhost:3000/sandboxes/${sandboxID} \
+  -H "X-API-Key: $E2B_API_KEY"
+
+# list running sandboxes
+curl "http://localhost:3000/v2/sandboxes?state=running" \
+  -H "X-API-Key: $E2B_API_KEY"
+
+# sandbox resource metrics
+curl "http://localhost:3000/sandboxes/{sandboxID}/metrics?start=$(date -v-5M +%s)&end=$(date +%s)" \
+  -H "X-API-Key: $E2B_API_KEY"
+
+# sandbox logs
+curl "http://localhost:3000/v2/sandboxes/{sandboxID}/logs?limit=100&direction=backward" \
+  -H "X-API-Key: $E2B_API_KEY"
+
+
+```
--- a/experiments/kata_and_fc_on_kube/manifests/kata-deploy-values.yaml
+++ b/experiments/kata_and_fc_on_kube/manifests/kata-deploy-values.yaml
@@ -0,0 +1,22 @@
+# Disable the default all-in-one installation
+shims:
+  disableAll: true
+  firecracker:
+    enabled: true
+  
+# In this chart version, 'env' must be a map, not a list.
+# The template converts these keys into Environment Variables for you.
+env:
+  kataArtifacts: "firecracker"
+  multiInstallSuffix: ""  # This fixes the specific error you saw
+  installArtifactsOnly: false
+
+# Automatically create the RuntimeClass
+runtimeClass:
+  create: true
+  name: kata-fc
+  handler: kata-fc
+
+# Set Firecracker as the default for amd64 nodes
+defaultShim:
+  amd64: firecracker
--- a/experiments/kata_and_fc_on_kube/manifests/runtime-class_kata-fc.yaml
+++ b/experiments/kata_and_fc_on_kube/manifests/runtime-class_kata-fc.yaml
@@ -0,0 +1,14 @@
+apiVersion: node.k8s.io/v1
+kind: RuntimeClass
+metadata:
+  name: kata-fc
+handler: kata-fc
+scheduling:
+  nodeSelector:
+    # Ensure these pods only land on your specialized node
+    runtime: kata-fc
+  tolerations:
+    - key: "dedicated"
+      operator: "Equal"
+      value: "kata-fc"
+      effect: "NoSchedule"    
--- a/experiments/kata_and_fc_on_kube/readme.md
+++ b/experiments/kata_and_fc_on_kube/readme.md
@@ -0,0 +1,198 @@
+## Node: kube-node-34
+
+```bash
+apt install cpu-checker
+
+# check kvm availability
+kvm-ok
+cat /sys/module/kvm_amd/parameters/nested
+lsmod | grep kvm
+
+# install kata containers
+# using kata-deploy ? 
+
+export VERSION=$(curl -sSL https://api.github.com/repos/kata-containers/kata-containers/releases/latest | jq .tag_name | tr -d '"')
+export CHART="oci://ghcr.io/kata-containers/kata-deploy-charts/kata-deploy"
+
+helm upgrade --install  kata-deploy "${CHART}" --version "${VERSION}" --values experiments/kata_and_fc_on_kube/manifests/kata-deploy-values.yaml
+
+# label only specific nodes to be used with kata-fc
+kubectl label node kube-node-34 runtime=kata-fc
+# taint node with NoSchedule
+kubectl taint nodes kube-node-34 dedicated=kata-fc:NoSchedule
+
+# configure runtime class
+k apply -f experiments/kata_and_fc_on_kube/manifests//runtime-class_kata-fc.yaml
+
+# devmapper configuration for firecracker in containerd
+ctr plugins ls | grep devmapper
+
+# create data and metadata files
+# might want to replace this with LVM !!!!
+# sudo mkdir -p /var/lib/containerd/devmapper
+# sudo truncate -s 10G /var/lib/containerd/devmapper/data
+# sudo truncate -s 1G /var/lib/containerd/devmapper/init_metadata
+
+# # Associate the files with Loop Devices
+# sudo losetup /dev/loop10 /var/lib/containerd/devmapper/data
+# sudo losetup /dev/loop11 /var/lib/containerd/devmapper/init_metadata
+
+# # Use dmsetup to create the pool:
+# # This command creates the mapping. 
+# # The numbers '0 20971520' represent the size in 512-byte sectors (for a 10GB file).
+# sudo dmsetup create containerd-pool --table "0 20971520 thin-pool /dev/loop11 /dev/loop10 128 32768 1"
+
+lvcreate -L 10G -T kata-vg/kata-pool
+
+# Add the configuration to /etc/containerd/config.toml:
+# Find the [plugins."io.containerd.snapshotter.v1.devmapper"] section and update it:
+
+[plugins."io.containerd.snapshotter.v1.devmapper"]
+  # LVM uses a specific naming convention in /dev/mapper/
+  # It is VolumeGroupName-LogicalVolumeName
+  pool_name = "kata--vg-kata--pool"
+  root_path = "/var/lib/containerd/devmapper"
+  base_image_size = "10GB"
+  discard_blocks = true
+
+  
+# restart containerd
+systemctl restart containerd
+
+
+
+```
+
+
+### kube node preparation & deployment
+
+```bash
+# ---------------------------------------------------------------------------
+# Packages
+# ---------------------------------------------------------------------------
+
+apt-get update
+apt-get install -y \
+  qemu-guest-agent \
+  openssh-server \
+  apt-transport-https \
+  ca-certificates \
+  curl \
+  gnupg \
+  nvme-cli
+
+systemctl enable --now qemu-guest-agent
+systemctl enable --now ssh
+
+# ---------------------------------------------------------------------------
+# nvme-tcp
+# ---------------------------------------------------------------------------
+apt-get install -y linux-modules-extra-$(uname -r)
+modprobe nvme-tcp
+echo "nvme-tcp" >> /etc/modules-load.d/nvme-tcp.conf
+
+# ---------------------------------------------------------------------------
+# Kernel modules for Kubernetes
+# ---------------------------------------------------------------------------
+cat > /etc/modules-load.d/k8s.conf <<'EOF'
+overlay
+br_netfilter
+EOF
+
+modprobe overlay
+modprobe br_netfilter
+
+cat > /etc/sysctl.d/k8s.conf <<'EOF'
+net.bridge.bridge-nf-call-iptables = 1
+net.bridge.bridge-nf-call-ip6tables = 1
+net.ipv4.ip_forward = 1
+EOF
+
+sysctl --system
+
+# ---------------------------------------------------------------------------
+# containerd
+# ---------------------------------------------------------------------------
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
+echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" \
+  > /etc/apt/sources.list.d/docker.list
+apt-get update && apt-get install -y containerd.io
+
+cat > /etc/containerd/config.toml <<'EOF'
+version = 2
+[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
+  runtime_type = "io.containerd.runc.v2"
+  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
+    SystemdCgroup = true
+[plugins."io.containerd.grpc.v1.cri".registry]
+  config_path = "/etc/containerd/certs.d"
+EOF
+
+# Registry mirrors pointing to Zot at 192.168.0.30:5000
+mkdir -p \
+  /etc/containerd/certs.d/docker.io \
+  /etc/containerd/certs.d/registry.k8s.io \
+  /etc/containerd/certs.d/ghcr.io \
+  /etc/containerd/certs.d/quay.io
+
+cat > /etc/containerd/certs.d/docker.io/hosts.toml <<'EOF'
+server = "https://registry-1.docker.io"
+[host."http://192.168.0.30:5000/v2/docker.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+cat > /etc/containerd/certs.d/registry.k8s.io/hosts.toml <<'EOF'
+server = "https://registry.k8s.io"
+[host."http://192.168.0.30:5000/v2/registry.k8s.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+cat > /etc/containerd/certs.d/ghcr.io/hosts.toml <<'EOF'
+server = "https://ghcr.io"
+[host."http://192.168.0.30:5000/v2/ghcr.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+cat > /etc/containerd/certs.d/quay.io/hosts.toml <<'EOF'
+server = "https://quay.io"
+[host."http://192.168.0.30:5000/v2/quay.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+systemctl restart containerd
+
+# ---------------------------------------------------------------------------
+# kubelet systemd drop-in
+# ---------------------------------------------------------------------------
+mkdir -p /etc/systemd/system/kubelet.service.d
+cat > /etc/systemd/system/kubelet.service.d/10-containerd.conf <<'EOF'
+[Unit]
+After=containerd.service
+Requires=containerd.service
+
+[Service]
+ExecStartPre=/bin/bash -c 'until [ -S /var/run/containerd/containerd.sock ]; do sleep 1; done'
+ExecStartPre=/usr/bin/crictl info
+EOF
+
+# ---------------------------------------------------------------------------
+# kubeadm / kubelet / kubectl v1.32
+# ---------------------------------------------------------------------------
+curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key \
+  | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
+echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /" \
+  > /etc/apt/sources.list.d/kubernetes.list
+apt-get update && apt-get install -y kubelet kubeadm kubectl
+apt-mark hold kubelet kubeadm kubectl
+
+
+# join kube cluster
+JOIN_COMMAND="kubeadm join 192.168.0.31:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>"
--- a/experiments/microsandbox/install.md
+++ b/experiments/microsandbox/install.md
@@ -0,0 +1,14 @@
+```bash
+virt-install \
+  --name microsandbox \
+  --ram 8192\
+  --vcpus 4 \
+  --cpu host-passthrough \
+  --os-variant ubuntu24.04 \
+  --disk path=/srv/vms/microsandbox.qcow2,size=100,format=qcow2,bus=virtio \
+  --network bridge=br0,model=virtio \
+  --graphics none \
+  --console pty,target_type=serial \
+  --location /srv/vms/isos/ubuntu-24.04.3-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
+  --extra-args 'console=ttyS0,115200n8'
+```
--- a/experiments/orb-stack/modules/base-template/cloud-init-base.yaml
+++ b/experiments/orb-stack/modules/base-template/cloud-init-base.yaml
@@ -0,0 +1,27 @@
+#cloud-config
+hostname: ${hostname}
+manage_etc_hosts: true
+
+# Kernel modules for container networking
+write_files:
+  - path: /etc/modules-load.d/k8s.conf
+    content: |
+      overlay
+      br_netfilter
+  - path: /etc/sysctl.d/k8s.conf
+    content: |
+      net.bridge.bridge-nf-call-iptables  = 1
+      net.bridge.bridge-nf-call-ip6tables = 1
+      net.ipv4.ip_forward                 = 1
+
+packages:
+  - curl
+  - gpg
+  - apt-transport-https
+
+runcmd:
+  - modprobe overlay
+  - modprobe br_netfilter
+  - sysctl --system
+  - swapoff -a
+  - sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
--- a/experiments/orb-stack/modules/base-template/main.tf
+++ b/experiments/orb-stack/modules/base-template/main.tf
@@ -0,0 +1,74 @@
+terraform {
+  required_providers {
+    orbstack = {
+      source  = "robertdebock/orbstack"
+      version = "~> 3.0"
+    }
+    cloudinit = {
+      source  = "hashicorp/cloudinit"
+      version = "~> 2.0"
+    }
+  }
+}
+
+variable "node_count" {
+  description = "Number of nodes to deploy"
+  type        = number
+  default     = 3
+}
+
+variable "name" {
+  description = "Base name for the machines"
+  type        = string
+}
+
+variable "distro" {
+  description = "OS distribution"
+  type        = string
+  default     = "ubuntu"
+}
+
+variable "distro_version" {
+  description = "OS distribution version/codename"
+  type        = string
+  default     = "noble"
+}
+
+variable "extra_cloud_init_parts" {
+  description = "Additional cloud-init parts to layer on top of the base config"
+  type        = list(object({ content = string, content_type = string }))
+  default     = []
+}
+
+data "cloudinit_config" "this" {
+  count = var.node_count
+
+  part {
+    content_type = "text/cloud-config"
+    content = templatefile("${path.module}/cloud-init-base.yaml", {
+      hostname = "${var.name}-${count.index + 1}"
+    })
+  }
+
+  dynamic "part" {
+    for_each = var.extra_cloud_init_parts
+    content {
+      content_type = part.value.content_type
+      content      = part.value.content
+    }
+  }
+}
+
+resource "orbstack_machine" "this" {
+  count = var.node_count
+
+  name   = "${var.name}-${count.index + 1}"
+  distro = var.distro
+  region = var.distro_version
+
+  user_data = data.cloudinit_config.this[count.index].rendered
+}
+
+output "machines" {
+  value = { for m in orbstack_machine.this : m.name => m.ip_address }
+}
--- a/experiments/orb-stack/stacks/k3s/cloud-init-cp.yaml
+++ b/experiments/orb-stack/stacks/k3s/cloud-init-cp.yaml
@@ -0,0 +1,11 @@
+#cloud-config
+merge_how:
+  - name: list
+    settings: [append]
+  - name: dict
+    settings: [recurse_array]
+
+runcmd:
+  - curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" K3S_TOKEN="${k3s_token}" sh -
+  - until kubectl get nodes; do sleep 2; done
+  - cp /etc/rancher/k3s/k3s.yaml /root/kubeconfig.yaml
--- a/experiments/orb-stack/stacks/k3s/cloud-init-worker.yaml
+++ b/experiments/orb-stack/stacks/k3s/cloud-init-worker.yaml
@@ -0,0 +1,9 @@
+#cloud-config
+merge_how:
+  - name: list
+    settings: [append]
+  - name: dict
+    settings: [recurse_array]
+
+runcmd:
+  - curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent" K3S_URL="https://${cp_ip}:6443" K3S_TOKEN="${join_token}" sh -
--- a/experiments/orb-stack/stacks/k3s/main.tf
+++ b/experiments/orb-stack/stacks/k3s/main.tf
@@ -0,0 +1,39 @@
+variable "k3s_token" {
+  description = "Shared secret for k3s cluster join (set via TF_VAR or tfvars)"
+  type        = string
+  sensitive   = true
+}
+
+module "control_plane" {
+  source     = "../../modules/base-template"
+  name       = "k3s-cp"
+  node_count = 1
+
+  extra_cloud_init_parts = [{
+    content_type = "text/cloud-config"
+    content = templatefile("${path.module}/cloud-init-cp.yaml", {
+      k3s_token = var.k3s_token
+    })
+  }]
+}
+
+module "workers" {
+  source     = "../../modules/base-template"
+  name       = "k3s-worker"
+  node_count = 2
+
+  extra_cloud_init_parts = [{
+    content_type = "text/cloud-config"
+    content = templatefile("${path.module}/cloud-init-worker.yaml", {
+      cp_ip      = values(module.control_plane.machines)[0]
+      join_token = var.k3s_token
+    })
+  }]
+}
+
+output "cluster" {
+  value = {
+    control_plane = module.control_plane.machines
+    workers       = module.workers.machines
+  }
+}
--- a/experiments/rke2/rke2-basics.md
+++ b/experiments/rke2/rke2-basics.md
@@ -0,0 +1,219 @@
+## install
+
+```bash
+# master node
+curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=v1.32.12+rke2r1 sh -
+systemctl enable rke2-server.service
+systemctl start rke2-server.service
+journalctl -u rke2-server -f
+
+# open firewalld
+sudo firewall-cmd --permanent --add-port=9345/tcp
+sudo firewall-cmd --permanent --add-port=6443/tcp
+sudo firewall-cmd --permanent --add-port=10250/tcp # Kubelet
+sudo firewall-cmd --reload
+
+# install nerdctl
+# Set the version
+VERSION="2.2.1" # Check GitHub for the latest version
+
+# Download the tarball
+wget https://github.com/containerd/nerdctl/releases/download/v${VERSION}/nerdctl-${VERSION}-linux-arm64.tar.gz
+
+# Extract to your path
+sudo tar -C /usr/local/bin -xzvf nerdctl-${VERSION}-linux-arm64.tar.gz nerdctl
+
+# configure nerdctl
+sudo mkdir -p /etc/nerdctl
+sudo tee /etc/nerdctl/nerdctl.toml <<EOF
+address   = "unix:///run/k3s/containerd/containerd.sock"
+namespace = "k8s.io"
+EOF
+
+# install buildkit
+# Set current stable version
+BK_VER="0.28.0"
+
+# Download arm64 binary
+wget https://github.com/moby/buildkit/releases/download/v${BK_VER}/buildkit-v${BK_VER}.linux-arm64.tar.gz
+
+# Extract only the binaries to /usr/local/bin
+sudo tar -C /usr/local/bin -xzvf buildkit-v${BK_VER}.linux-arm64.tar.gz --strip-components=1 bin/
+
+# Create the service file
+sudo tee /etc/systemd/system/buildkit.service <<EOF
+[Unit]
+Description=BuildKit
+Documentation=https://github.com/moby/buildkit
+
+[Service]
+ExecStart=/usr/local/bin/buildkitd --addr unix:///run/buildkit/buildkitd.sock
+
+[Install]
+WantedBy=multi-user.target
+EOF
+
+# Reload and Start
+sudo systemctl daemon-reload
+sudo systemctl enable --now buildkit
+
+# ---------------------------------------------
+
+
+# agent/worker node
+curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" INSTALL_RKE2_VERSION=v1.32.12+rke2r1 sh -
+systemctl enable rke2-agent.service
+mkdir -p /etc/rancher/rke2/
+# token from master node
+# cat /var/lib/rancher/rke2/server/node-token
+cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
+server: https://192.168.64.3:9345
+token: K107618960f87b9efb3a3255ce00a9743d29f1db9376820c9144cb85fa3c554dc69::server:06b2effdf0c9ce3952efc8a5d80bf084
+EOF
+systemctl start rke2-agent.service
+journalctl -u rke2-agent -f
+
+
+# Set up kubectl on the server node
+echo 'export KUBECONFIG=/etc/rancher/rke2/rke2.yaml' >> ~/.bashrc
+echo 'export PATH=$PATH:/var/lib/rancher/rke2/bin' >> ~/.bashrc
+source ~/.bashrc
+```
+
+
+
+## build and deploy application
+
+```bash
+# build container with nerdctl
+nerdctl --namespace k8s.io build --tag hello-world:latest .
+
+# export image as tar on master node
+nerdctl save hello-world:latest -o hello-world.tar
+# copy it over to worker node
+scp hello-world.tar novakj@192.168.64.4:~/
+# import image on the agent node
+sudo /var/lib/rancher/rke2/bin/ctr --address /run/k3s/containerd/containerd.sock -n k8s.io images import hello-world.tar
+
+kubectl create namespace rke2-apps
+
+cat <<EOF > deployment.yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: hello-world-deployment
+  namespace: rke2-apps
+  labels:
+    type: staticwebapp
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      type: staticwebapp
+  template:
+    metadata:
+      labels:
+        type: staticwebapp
+    spec:
+      containers:
+      - name: staticwebapp
+        image: hello-world:latest
+        imagePullPolicy: Never
+        ports:
+        - containerPort: 80
+        resources:
+          requests:
+            memory: "32Mi"
+            cpu: "200m"
+          limits:
+            memory: "64Mi"
+            cpu: "300m"
+EOF
+
+kubectl create -f deployment.yaml
+
+# expose deployment
+kubectl expose deployment hello-world-deployment --name hello-world-service --port=8080 --target-port=80 -n rke2-apps
+
+# install ingress-nginx (even though i thought that there is ingress controller already deployed)
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.2/deploy/static/provider/cloud/deploy.yaml
+
+# create ingress with "localhost" as host
+kubectl create ingress hello-world-ingress --class=nginx --rule="test-host/*=hello-world-service:8080" -n rke2-apps
+kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 8081:80
+
+
+
+# incomplete completion configuration ;-)
+dnf install bash-completion -y
+alias 'k=kubectl'
+
+# ~/.bashrc
+
+# 1. Load the main bash-completion package first
+# On Rocky/RHEL, it's usually at this path:
+[[ -r "/usr/share/bash-completion/bash_completion" ]] && . "/usr/share/bash-completion/bash_completion"
+
+# Enable kubectl bash completion
+source <(kubectl completion bash)
+
+# Set up the alias
+alias k=kubectl
+
+# Link the kubectl completion logic to the 'k' alias
+complete -o default -F __start_kubectl k
+```
+
+
+
+## upgrading RKE2
+
+```bash
+# install upgrade controller
+kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/download/v0.9.1/system-upgrade-controller.yaml
+
+# server upgrade
+cat <<EOF | kubectl apply -f -
+apiVersion: upgrade.cattle.io/v1
+kind: Plan
+metadata:
+  name: rke2-server-upgrade
+  namespace: system-upgrade
+spec:
+  concurrency: 1
+  cordon: true
+  nodeSelector:
+    matchExpressions:
+      - key: node-role.kubernetes.io/control-plane
+        operator: In
+        values: ["true"]
+  serviceAccountName: system-upgrade
+  upgrade:
+    image: rancher/rke2-upgrade
+  version: v1.33.9+rke2r1
+EOF
+
+# agent upgrade
+cat <<EOF | kubectl apply -f -
+apiVersion: upgrade.cattle.io/v1
+kind: Plan
+metadata:
+  name: rke2-agent-upgrade
+  namespace: system-upgrade
+spec:
+  concurrency: 1
+  cordon: true
+  nodeSelector:
+    matchExpressions:
+      - key: node-role.kubernetes.io/control-plane
+        operator: DoesNotExist
+  prepare:
+    # Logic: "Don't start workers until servers are done"
+    args: ["wait-for-plan", "rke2-server-upgrade"]
+    image: rancher/rke2-upgrade
+  serviceAccountName: system-upgrade
+  upgrade:
+    image: rancher/rke2-upgrade
+  version: v1.33.9+rke2r1
+EOF
+```
--- a/experiments/rke2/step-by-step.md
+++ b/experiments/rke2/step-by-step.md
@@ -0,0 +1,215 @@
+## VMS creation
+
+
+
+```bash
+# hypervisor: beelink (192.168.0.6)
+
+vms_path=/srv/vms/images
+isos_path=/srv/vms/isos
+cd $isos_path
+# Grab Ubuntu 24.04 cloud image
+wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
+
+# Create a base disk from the cloud image (repeat per node)
+for NODE in rke2-server rke2-agent1 rke2-agent2; do
+  DISK_SIZE="30G"
+  [[ "$NODE" == rke2-agent* ]] && DISK_SIZE="50G"
+
+  qemu-img create -f qcow2 -F qcow2 -b $isos_path/noble-server-cloudimg-amd64.img $vms_path/${NODE}.qcow2
+  qemu-img resize $vms_path/${NODE}.qcow2 ${DISK_SIZE}
+done
+```
+
+
+```bash
+# prepare cloud-init/user-data
+apt install cloud-image-utils
+
+# prepare cloud-init and launch VMs
+declare -A nodes=(
+  [rke2-server]="192.168.0.51"
+  [rke2-agent1]="192.168.0.52"
+  [rke2-agent2]="192.168.0.53"
+)
+
+for node in "${!nodes[@]}"; do
+  ip="${nodes[$node]}"
+
+  # user-data
+  cat <<EOF > user-data-${node}
+#cloud-config
+hostname: ${node}
+manage_etc_hosts: false
+users:
+  - name: sre
+    sudo: ALL=(ALL) NOPASSWD:ALL
+    shell: /bin/bash
+    ssh_authorized_keys:
+      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINQxxkqmvtVI+8c5BkTaJ5c2HfBFRXJWMmEcevvfP9tV jan.novak@Jans-MacBook-Air.local
+write_files:
+  - path: /etc/hosts
+    append: true
+    content: |
+      192.168.0.51 rke2-server
+      192.168.0.52 rke2-agent1
+      192.168.0.53 rke2-agent2
+  - path: /etc/modules-load.d/rke2.conf
+    content: |
+      br_netfilter
+      overlay
+  - path: /etc/sysctl.d/99-rke2.conf
+    content: |
+      net.bridge.bridge-nf-call-iptables  = 1
+      net.bridge.bridge-nf-call-ip6tables = 1
+      net.ipv4.ip_forward                 = 1
+swap:
+  filename: /swap.img
+  size: 0
+  maxsize: 0
+runcmd:
+  - swapoff -a
+  - sed -i '/swap/d' /etc/fstab
+  - modprobe br_netfilter
+  - modprobe overlay
+  - sysctl --system
+package_update: true
+packages:
+  - qemu-guest-agent
+  - nfs-common
+  - open-iscsi
+power_state:
+  mode: reboot
+EOF
+
+  # network config
+  cat <<EOF > network-config-${node}
+network:
+  version: 2
+  ethernets:
+    eth0:
+      match:
+        driver: "virtio_net"
+      addresses:
+      - "${ip}/24"
+      nameservers:
+        addresses:
+        - 8.8.8.8
+      routes:
+      - to: "default"
+        via: "192.168.0.4"
+EOF
+
+  cloud-localds --network-config=./network-config-${node} \
+    $vms_path/${node}-seed.iso ./user-data-${node}
+done
+
+# Launch VMs
+for node in "${!nodes[@]}"; do
+  virt-install \
+    --name ${node} \
+    --ram 4096 --vcpus 2 \
+    --os-variant ubuntu24.04 \
+    --disk $vms_path/${node}.qcow2,bus=virtio \
+    --disk $vms_path/${node}-seed.iso,device=cdrom \
+    --network bridge=br0,model=virtio \
+    --graphics none \
+    --console pty,target_type=serial \
+    --noautoconsole \
+    --import
+done
+
+```
+
+
+## RKE2 installation
+
+```bash
+# there are no .deb packages - only rpm or tarball
+# "magic" install script can handle that
+curl -sfL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.32 sudo sh -
+
+# Create config directory
+sudo mkdir -p /etc/rancher/rke2
+
+# Server configuration
+cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
+# Bind the API to the node's IP (not 127.0.0.1)
+tls-san:
+  - rke2-server
+  - 192.168.0.51
+# Write kubeconfig readable by non-root
+write-kubeconfig-mode: "0644"
+# CNI - canal is default and fine for home lab
+# Alternatives: cilium, calico, multus
+cni:
+  - canal
+# Disable servicelb if you plan to use metallb
+disable:
+  - rke2-service-lb
+EOF
+
+# Enable and start
+sudo systemctl enable rke2-server.service
+sudo systemctl start rke2-server.service
+
+# Watch bootstrap (takes 2-3 min on first run)
+sudo journalctl -u rke2-server -f
+
+```
+
+
+## Grab the join token and kubeconfig
+
+```bash
+
+# Token for agents to join
+sudo cat /var/lib/rancher/rke2/server/node-token
+# Save this somewhere - you'll need it on every agent
+
+# Set up kubectl on the server node
+echo 'export KUBECONFIG=/etc/rancher/rke2/rke2.yaml' >> ~/.bashrc
+echo 'export PATH=$PATH:/var/lib/rancher/rke2/bin' >> ~/.bashrc
+source ~/.bashrc
+
+# Verify
+kubectl get nodes
+```
+
+
+## Install RKE2 Agents (Workers)
+
+```bash
+# Install RKE2 agent - same channel as server
+curl -sfL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.32 INSTALL_RKE2_TYPE=agent sudo sh -
+
+# Create config
+sudo mkdir -p /etc/rancher/rke2
+
+cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
+server: https://192.168.0.51:9345
+token: K10dba0bfff01d610ffed41c6b82a1b8861ee19e5af34a3bdd21970936823a846da::server:1f02fb36a288dcd06770cab28b015bd7
+EOF
+
+# Enable and start
+sudo systemctl enable rke2-agent.service
+sudo systemctl start rke2-agent.service
+
+# Watch join
+sudo journalctl -u rke2-agent -f
+
+```
+
+
+## Copy Kubeconfig to Your Workstation
+
+```bash
+# On your workstation (not the VMs)
+scp sre@192.168.0.51:/etc/rancher/rke2/rke2.yaml ~/.kube/rke2-homelab.yaml
+
+# Fix the server address (it'll say 127.0.0.1)
+gsed -i 's/127.0.0.1/192.168.0.51/' ~/.kube/rke2-homelab.yaml
+
+export KUBECONFIG=~/.kube/rke2-homelab.yaml
+kubectl get nodes
+```
--- a/gitops/home-kubernetes/00-rbac/clusterRoleBinding_gitea-ci.yaml
+++ b/gitops/home-kubernetes/00-rbac/clusterRoleBinding_gitea-ci.yaml
@@ -0,0 +1,20 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: gitea-ci-deploy
+subjects:
+  - kind: User
+    name: "gitea_ci@idm.home.hrajfrisbee.cz"  # matches preferred_username claim
+    apiGroup: rbac.authorization.k8s.io
+roleRef:
+  # kind: ClusterRole
+  # name: edit  # scope down as needed
+  # apiGroup: rbac.authorization.k8s.io
+
+  # this is obviously too much permissions
+  # but we can live with it for homelab 
+  kind: ClusterRole
+  name: cluster-admin
+  apiGroup: rbac.authorization.k8s.io
+
+  
--- a/gitops/home-kubernetes/00-rbac/clusterrolebinding_oidc-admin.yaml
+++ b/gitops/home-kubernetes/00-rbac/clusterrolebinding_oidc-admin.yaml
@@ -0,0 +1,15 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: oidc-cluster-admin
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: cluster-admin
+subjects:
+- apiGroup: rbac.authorization.k8s.io
+  kind: User
+  name: https://idm.home.hrajfrisbee.cz/oauth2/openid/k8s#35842461-a1c4-4ad6-8b29-697c5ddbfe84
+- apiGroup: rbac.authorization.k8s.io
+  kind: User
+  name: novakj@idm.home.hrajfrisbee.cz
--- a/gitops/home-kubernetes/cilium/certificate_wildcard-lab-home-hrajfrisbee.yaml
+++ b/gitops/home-kubernetes/cilium/certificate_wildcard-lab-home-hrajfrisbee.yaml
@@ -0,0 +1,12 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: wildcard-lab-home-hrajfrisbee
+  namespace: kube-system
+spec:
+  secretName: wildcard-lab-home-hrajfrisbee-tls
+  issuerRef:
+    name: letsencrypt-prod-dns
+    kind: ClusterIssuer
+  dnsNames:
+    - "*.lab.home.hrajfrisbee.cz"
--- a/gitops/home-kubernetes/cilium/gateway.yaml
+++ b/gitops/home-kubernetes/cilium/gateway.yaml
@@ -13,14 +13,15 @@ spec:
      allowedRoutes:
        namespaces:
          from: All
-    - name: https
+    - name: lab-home-hrajfrisbee-https-wildcard
+      hostname: "*.lab.home.hrajfrisbee.cz"
      port: 443
      protocol: HTTPS
-      allowedRoutes:
-        namespaces:
-          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
-            name: gateway-tls
+            name: wildcard-lab-home-hrajfrisbee-tls
+      allowedRoutes:
+        namespaces:
+          from: All
--- a/gitops/home-kubernetes/cilium/helmrelelase_cilium.yaml
+++ b/gitops/home-kubernetes/cilium/helmrelelase_cilium.yaml
@@ -13,7 +13,7 @@ spec:
        kind: HelmRepository
        name: cilium
        namespace: flux-system
-      version: 1.18.5
+      version: 1.19.1
  interval: 5m0s
  values:
    cluster:
@@ -24,6 +24,8 @@ spec:
        enabled: true
      ui:
        enabled: true
+    ingressController:
+      enabled: true
    ipam:
      mode: cluster-pool
      operator:
@@ -33,6 +35,8 @@ spec:
      enabled: true
    gatewayAPI:
      enabled: true
+      gatewayClass:
+        create: "true"
    kubeProxyReplacement: true
    k8sServiceHost: 192.168.0.31  # or LB IP
    k8sServicePort: 6443
--- a/gitops/home-kubernetes/experiments/certificate_operator-test-2.yaml
+++ b/gitops/home-kubernetes/experiments/certificate_operator-test-2.yaml
@@ -0,0 +1,17 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: operator-test-lab-home-hrajfrisbee-2
+  namespace: default
+  annotations:
+    gateway-cert-operator.io/gateway-name: "cilium-gateway"
+    gateway-cert-operator.io/gateway-namespace: "kube-system"
+
+spec:
+  secretName: operator-test-lab-home-hrajfrisbee-2
+  issuerRef:
+    group: cert-manager.io
+    kind: ClusterIssuer
+    name: letsencrypt-prod
+  dnsNames:
+    - "operator-test-2.lab.home.hrajfrisbee.cz"
--- a/gitops/home-kubernetes/experiments/certificate_operator-test.yaml
+++ b/gitops/home-kubernetes/experiments/certificate_operator-test.yaml
@@ -0,0 +1,17 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: operator-test-lab-home-hrajfrisbee
+  namespace: default
+  annotations:
+    gateway-cert-operator.io/gateway-name: "cilium-gateway"
+    gateway-cert-operator.io/gateway-namespace: "kube-system"
+
+spec:
+  secretName: operator-test-lab-home-hrajfrisbee
+  issuerRef:
+    group: cert-manager.io
+    kind: ClusterIssuer
+    name: letsencrypt-prod
+  dnsNames:
+    - "operator-test.lab.home.hrajfrisbee.cz"
--- a/gitops/home-kubernetes/experiments/deployment_operator-test.yaml
+++ b/gitops/home-kubernetes/experiments/deployment_operator-test.yaml
@@ -0,0 +1,30 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: operator-test
+  namespace: default
+  labels:
+    app: operator-test
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: operator-test
+  template:
+    metadata:
+      labels:
+        app: operator-test
+    spec:
+      containers:
+        - name: whoami
+          image: traefik/whoami
+          ports:
+            - containerPort: 80
+              name: http
+          resources:
+            requests:
+              cpu: 10m
+              memory: 16Mi
+            limits:
+              memory: 32Mi
--- a/gitops/home-kubernetes/experiments/httproute_operator-test-notls.yaml
+++ b/gitops/home-kubernetes/experiments/httproute_operator-test-notls.yaml
@@ -0,0 +1,25 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: operator-test-redirect
+  namespace: default
+  labels:
+    app: operator-test
+spec:
+  parentRefs:
+    - name: cilium-gateway
+      namespace: kube-system
+      sectionName: http
+  hostnames:
+    - operator-test.lab.home.hrajfrisbee.cz
+  rules:
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /
+      filters:
+        - type: RequestRedirect
+          requestRedirect:
+            scheme: https
+            statusCode: 301
--- a/gitops/home-kubernetes/experiments/httproute_operator-test.yaml
+++ b/gitops/home-kubernetes/experiments/httproute_operator-test.yaml
@@ -0,0 +1,24 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: operator-test
+  namespace: default
+  labels:
+    app: operator-test
+spec:
+  parentRefs:
+    - name: cilium-gateway
+      namespace: kube-system
+      sectionName: auto-operator-test-lab-home-hrajfrisbee-operator-test-lab-home-hrajfrisbee-cz
+  hostnames:
+    - operator-test.lab.home.hrajfrisbee.cz
+  rules:
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /
+      backendRefs:
+        - name: operator-test
+          namespace: default
+          port: 80
--- a/gitops/home-kubernetes/experiments/referencegrant_default.yaml
+++ b/gitops/home-kubernetes/experiments/referencegrant_default.yaml
@@ -0,0 +1,14 @@
+apiVersion: gateway.networking.k8s.io/v1beta1
+kind: ReferenceGrant
+metadata:
+  name: allow-kube-system-gateway
+  namespace: default
+spec:
+  from:
+  - group: gateway.networking.k8s.io
+    kind: Gateway
+    namespace: kube-system
+  to:
+  - group: ""
+    kind: Secret
+    name: operator-test-lab-home-hrajfrisbee
--- a/gitops/home-kubernetes/experiments/service_operator-test.yaml
+++ b/gitops/home-kubernetes/experiments/service_operator-test.yaml
@@ -0,0 +1,15 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: operator-test
+  namespace: default
+spec:
+  type: ClusterIP
+  selector:
+    app: operator-test
+  ports:
+    - port: 80
+      targetPort: 80
+      protocol: TCP
+      name: http
--- a/gitops/home-kubernetes/flux-system/extra-kustomizations.yaml
+++ b/gitops/home-kubernetes/flux-system/extra-kustomizations.yaml
@@ -14,6 +14,19 @@ spec:
 ---
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
+metadata:
+  name: 00-rbac
+  namespace: flux-system
+spec:
+  interval: 10m0s
+  path: ./gitops/home-kubernetes/00-rbac
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+---
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
 metadata:
  name: cilium
  namespace: flux-system
@@ -117,6 +130,19 @@ spec:
  sourceRef:
    kind: GitRepository
    name: flux-system
+---
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: experiments
+  namespace: flux-system
+spec:
+  interval: 10m0s
+  path: ./gitops/home-kubernetes/experiments
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
 # ---
 # apiVersion: kustomize.toolkit.fluxcd.io/v1
 # kind: Kustomization
--- a/gitops/home-kubernetes/flux-system/helmrelease_flux-web.yaml
+++ b/gitops/home-kubernetes/flux-system/helmrelease_flux-web.yaml
@@ -0,0 +1,16 @@
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: flux-web
+  namespace: flux-system
+spec:
+  interval: 30m
+  releaseName: flux-web
+  chartRef:
+    kind: OCIRepository
+    name: flux-operator
+  values:
+    fullnameOverride: flux-web
+    installCRDs: true
+    web:
+      serverOnly: true
--- a/gitops/home-kubernetes/flux-system/ocirepository_control-plane.yaml
+++ b/gitops/home-kubernetes/flux-system/ocirepository_control-plane.yaml
@@ -0,0 +1,13 @@
+apiVersion: source.toolkit.fluxcd.io/v1
+kind: OCIRepository
+metadata:
+  name: flux-operator
+  namespace: flux-system
+spec:
+  interval: 30m
+  url: oci://ghcr.io/controlplaneio-fluxcd/charts/flux-operator
+  layerSelector:
+    mediaType: "application/vnd.cncf.helm.chart.content.v1.tar+gzip"
+    operation: copy
+  ref:
+    semver: "0.x"
--- a/gitops/home-kubernetes/fuj/deployment_fuj-management.yaml
+++ b/gitops/home-kubernetes/fuj/deployment_fuj-management.yaml
@@ -0,0 +1,63 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: fuj-management
+  namespace: fuj
+  labels:
+    app: fuj-management
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: fuj-management
+  template:
+    metadata:
+      labels:
+        app: fuj-management
+    spec:
+      imagePullSecrets:
+        - name: gitea-registry
+      containers:
+        - name: fuj-management
+          image: gitea.home.hrajfrisbee.cz/kacerr/fuj-management:0.39-go
+          ports:
+            - containerPort: 8080
+              name: http
+          env:
+            - name: PORT
+              value: "8080"
+            - name: TZ
+              value: Europe/Prague
+            - name: CREDENTIALS_PATH
+              value: /secrets/credentials.json              
+          resources:
+            requests:
+              cpu: 50m
+              memory: 64Mi
+            limits:
+              cpu: 200m
+              memory: 128Mi
+          readinessProbe:
+            httpGet:
+              path: /
+              port: 8080
+            initialDelaySeconds: 2
+            periodSeconds: 10
+            timeoutSeconds: 3
+          livenessProbe:
+            httpGet:
+              path: /
+              port: 8080
+            initialDelaySeconds: 5
+            periodSeconds: 30
+            timeoutSeconds: 5
+          volumeMounts:
+            - name: bot-credentials
+              mountPath: /secrets
+              readOnly: true            
+      volumes:
+        - name: bot-credentials
+          secret:
+            secretName: fuj-management-bot-credentials
--- a/gitops/home-kubernetes/fuj/deployment_fujarna.yaml
+++ b/gitops/home-kubernetes/fuj/deployment_fujarna.yaml
@@ -0,0 +1,60 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: fujarna
+  namespace: fuj
+  labels:
+    app: fujarna
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: fujarna
+  template:
+    metadata:
+      labels:
+        app: fujarna
+    spec:
+      imagePullSecrets:
+        - name: gitea-registry
+      containers:
+        - name: fujarna
+          image: gitea.home.hrajfrisbee.cz/kacerr/fujarna:v0.0.2
+          ports:
+            - containerPort: 8080
+              name: http
+          env:
+            - name: PORT
+              value: "8080"
+            - name: TZ
+              value: Europe/Prague
+          volumeMounts:
+            - name: data
+              mountPath: /app/data
+          resources:
+            requests:
+              cpu: 50m
+              memory: 64Mi
+            limits:
+              cpu: 200m
+              memory: 128Mi
+          readinessProbe:
+            httpGet:
+              path: /api/tournaments
+              port: 8080
+            initialDelaySeconds: 2
+            periodSeconds: 10
+            timeoutSeconds: 3
+          livenessProbe:
+            httpGet:
+              path: /api/tournaments
+              port: 8080
+            initialDelaySeconds: 5
+            periodSeconds: 30
+            timeoutSeconds: 5
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: tournament-data
--- a/gitops/home-kubernetes/fuj/externalsecret_gitea-registry.yaml
+++ b/gitops/home-kubernetes/fuj/externalsecret_gitea-registry.yaml
@@ -0,0 +1,22 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: gitea-registry
+  namespace: fuj
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    name: vault-backend
+    kind: ClusterSecretStore
+  target:
+    name: gitea-registry
+    creationPolicy: Owner
+    template:
+      type: kubernetes.io/dockerconfigjson
+      data:
+        .dockerconfigjson: "{{ .token }}"
+  data:
+    - secretKey: token
+      remoteRef:
+        key: k8s_home/gitea/container-registry
+        property: token
--- a/gitops/home-kubernetes/fuj/httproute-notls.yaml
+++ b/gitops/home-kubernetes/fuj/httproute-notls.yaml
@@ -0,0 +1,30 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: fujarna-redirect
+  namespace: fuj
+  labels:
+    app: fujarna
+    app.kubernetes.io/name: fujarna-httproute
+    app.kubernetes.io/instance: fujarna
+    app.kubernetes.io/version: '6.0'
+    app.kubernetes.io/component: httproute
+    app.kubernetes.io/part-of: fujarna
+spec:
+  parentRefs:
+    - name: cilium-gateway
+      namespace: kube-system
+      sectionName: http
+  hostnames:
+    - fujarna.lab.home.hrajfrisbee.cz
+  rules:
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /
+      filters:
+        - type: RequestRedirect
+          requestRedirect:
+            scheme: https
+            statusCode: 301
--- a/gitops/home-kubernetes/fuj/httproute_fujarna.yaml
+++ b/gitops/home-kubernetes/fuj/httproute_fujarna.yaml
@@ -0,0 +1,29 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: fujarna
+  namespace: fuj
+  labels:
+    app: fujarna
+    app.kubernetes.io/name: fujarna-httproute
+    app.kubernetes.io/instance: fujarna
+    app.kubernetes.io/version: '6.0'
+    app.kubernetes.io/component: httproute
+    app.kubernetes.io/part-of: fujarna
+spec:
+  parentRefs:
+    - name: cilium-gateway
+      namespace: kube-system
+      sectionName: lab-home-hrajfrisbee-https-wildcard
+  hostnames:
+    - fujarna.lab.home.hrajfrisbee.cz
+  rules:
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /
+      backendRefs:
+        - name: fujarna
+          namespace: fuj
+          port: 80
--- a/gitops/home-kubernetes/fuj/httproute_management.yaml
+++ b/gitops/home-kubernetes/fuj/httproute_management.yaml
@@ -0,0 +1,28 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: fuj-management
+  namespace: fuj
+  labels:
+    app: fuj-management
+    app.kubernetes.io/name: fuj-management-httproute
+    app.kubernetes.io/instance: fuj-management
+    app.kubernetes.io/component: httproute
+    app.kubernetes.io/part-of: fuj-management
+spec:
+  parentRefs:
+    - name: cilium-gateway
+      namespace: kube-system
+      sectionName: lab-home-hrajfrisbee-https-wildcard
+  hostnames:
+    - fuj-management.lab.home.hrajfrisbee.cz
+  rules:
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /
+      backendRefs:
+        - name: fuj-management
+          namespace: fuj
+          port: 80
--- a/gitops/home-kubernetes/fuj/ingress.yaml
+++ b/gitops/home-kubernetes/fuj/ingress.yaml
@@ -0,0 +1,30 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: fujarna
+  namespace: fuj
+  annotations:
+    # WebSocket support - increase timeouts for live scoring connections
+    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
+    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
+    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
+    # Uncomment for cert-manager TLS:
+    # cert-manager.io/cluster-issuer: letsencrypt-prod
+spec:
+  ingressClassName: nginx  # change to "traefik" for k3s default
+  rules:
+    - host: hrajfrisbee.cz  # change to your domain
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: fujarna
+                port:
+                  number: 80
+  # Uncomment for TLS:
+  # tls:
+  #   - hosts:
+  #       - hrajfrisbee.cz
+  #     secretName: fujarna-tls
--- a/gitops/home-kubernetes/fuj/namespace.yaml
+++ b/gitops/home-kubernetes/fuj/namespace.yaml
@@ -0,0 +1,4 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: fuj
--- a/gitops/home-kubernetes/fuj/pvc.yaml
+++ b/gitops/home-kubernetes/fuj/pvc.yaml
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: tournament-data
+  namespace: fuj
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: 1Gi
+  # storageClassName: local-path  # uncomment to override cluster default
--- a/gitops/home-kubernetes/fuj/secret_fuj-management-bot-credentials.yaml
+++ b/gitops/home-kubernetes/fuj/secret_fuj-management-bot-credentials.yaml
@@ -0,0 +1,8 @@
+apiVersion: v1
+data:
+  credentials.json: ewogICAgInR5cGUiOiAic2VydmljZV9hY2NvdW50IiwKICAgICJwcm9qZWN0X2lkIjogInBsYXRlZC1tZXNoLTQ4OTAxMC1zMSIsCiAgICAicHJpdmF0ZV9rZXlfaWQiOiAiYzQxM2VhYWUzMzk0NGFmZDljZjkyZjgzNGYzYmM1NDM5OGI1ZGY5YyIsCiAgICAicHJpdmF0ZV9rZXkiOiAiLS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tXG5NSUlFdmdJQkFEQU5CZ2txaGtpRzl3MEJBUUVGQUFTQ0JLZ3dnZ1NrQWdFQUFvSUJBUURuTzBJQ3p2NDNaMWFKXG5MYi9NSkFZbld2U2NWM2tlelBGSHpHNVd2WUE3SXBxM2ZkcUp6ZWYxSzcxeGJodGY5NFB6T2RNVGdrYXcxcmZYXG40QjhOMS9laWFZaWUyZkkwenA4emptejl0YnpDUjVwM0hiK3M0eHROR3hVUDJBWWQzVmRrbE5TRFlacGJ5Qm8rXG5qNWl1cy9BOXdYV1ZkQ2EvNnA1MmtYSU1Ld3o4b05rbmpMRTlzV21ZV1kvQmJZT1o0UExPSjNTWEdEMFJNQzlGXG4wbjBLUDJTRWxvaTFPc2wvV2s0Q21WazJFc0lNSjBiMHA1Wm1GZUZ6SlAzSFZUS0VtOExHam5vWjVwQmwzWGM0XG5GOThKSVFFQzZQallOcXQvSFlxdWRLb2hrelVSS2h5ekVJdHp3NXBrUnk3d1l1OGJHSGJOMzF0dWFpeDRMdFNjXG5zYU5oOHJhVEFnTUJBQUVDZ2dFQWFXZ2U4RTdSbmdueWJiZzRMV1BpbGtBbEw3dkVUK0VXd0NjVnlWNjdQTG5LXG5rNGpBZlg4cWxSMnFUekhsTXJzUElHb2txVWtDMW93YTVFS2JoV0VFMXJtSytQYmJMVzFmTDA1bXFzVVUxZTkwXG5INGFsSUxlcWMxeThIaXZZcGZhSFp5ZGROTUxpYmFKckxFZWpUMGhoWkpWeTkzMHFTT1EzOWs3WkVXanlrNW9WXG5WcUk2SE4yVGUrRTN0aklmNFcrN0lKN0c3RU9ZVG4zZHlrQnZCN1QwLzR2M0pUeGMzTk1IN0RZYzFyd1dicEdKXG5JcmNidndOa3VyVHhsdWg1VGtvSkZjd3ZXdEl4NE52MFFPNkk5TlpjMllPakZUMncwMUFrKzNDSTJLNmJOVE9qXG43WWFxYlRoNDk2cU1Md2NtRmFUZy9tbXEzMGZycEVsMWdQTWJWZFJqTVFLQmdRRDY3SDMvb0pGU1ZDWnFYQjFEXG5tdHZRblVMSWUvZkU5TXc4T2VrcUdMclQxbGlUTllhTks4YnRsOGFrbFlZY3FXU09zWk53ZkdyNmNhOGhUSEwyXG5pWXZNbkR4aVJPWkVkeElPUFJGY2l4SlNpdG5hb3kzSmtMMml0K2cxMUVGdUEyS3dGd29jc0RrRkQ1QTh0ZEFpXG5UOGhEeEc4dE5saFFrTUY0ekRMNTE2cHRLUUtCZ1FEcjZNZitLelBoNHM2Qm1GVzkrUzNtZEFZZkJETkJsZHJrXG56L1hrY2NteHNWeUpNNTErR2MzY3kzYmhIRDU4aGVLb3U1ZHdYanNqZkRCZlZKcjBpTHM0R3dGbjNlWlJYNmluXG5NVmlycmpyUWt5WnlSVDJkcHl5ODQ5MGN1WC9xQ01hbTVRYzY0UmpJN3JhRTM5ZDBRcnRRd0hWbnZCN1RHaEcvXG50WElXeThMQld3S0JnRGRtMi9NQyt0TVlyVnF5ZCt1alY0RmFGNUs0ZkY1S1JvWE5xNE9WN2pvemkxOEZaRzNyXG5Db01QRGRidEJLMXpZVFo0VXJ0MVhSRnE1R1lQd1JwYUNEbm5EUllOYkdJVmdERVFucksyS2hkN2ZOcGZTUTdHXG5ISGkrN2xCa1N3WUZUVmJmMzJXeUt3VEpDT20rb3ZQUWRjRXdyL05sbWw0L2Q5ZzZEYzlJSk9MeEFvR0JBTi9yXG5QaGwwN0hRV0VpbFViTjQ3TEdRMzhLOTBEeXJMa00vcXpOMGZYSEM3aHJmOE5CdERadkdTZHE1SlhzNVhUc01WXG4vREorWHVmWnN4aHBlQU52M1RUdC9UR1hvRXZRM29hRUFrUkN4alMvMTVoVFBKN3QxVHNkbGZ6ejNnUklmRFpoXG40YkR6ZzlFRk5GMS9Fa1NhS3E0Rlgrc3MxcDFOZTFkYVpJWmF0ZXROQW9HQkFLeStZTGRjQVh6dElaMTkvS0wwXG5sWTBPZi92azVJNUtKNlNiVUVheU9HTFJkWTJBdGk1R1QwQlpjVEV3QjYzWTNyWEVNQ3BsM25Ca1pVRklRczNEXG5wSE9DUDdDMXQ3STRSZlBGaGlSTzEvQnVsajZjUXNrUHFJWlpadGxOQjdROG1BVEttMnpXVThZTjRNbFhZSVBVXG5naGRrZHVQRWIyRC9ueDl3U3F2NzdUeWJcbi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS1cbiIsCiAgICAiY2xpZW50X2VtYWlsIjogImZ1ai1tYW5hZ2VtZW50LWJvdEBwbGF0ZWQtbWVzaC00ODkwMTAtczEuaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20iLAogICAgImNsaWVudF9pZCI6ICIxMDM0ODY2MjM3NTQxOTMzODgxMzQiLAogICAgImF1dGhfdXJpIjogImh0dHBzOi8vYWNjb3VudHMuZ29vZ2xlLmNvbS9vL29hdXRoMi9hdXRoIiwKICAgICJ0b2tlbl91cmkiOiAiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLAogICAgImF1dGhfcHJvdmlkZXJfeDUwOV9jZXJ0X3VybCI6ICJodHRwczovL3d3dy5nb29nbGVhcGlzLmNvbS9vYXV0aDIvdjEvY2VydHMiLAogICAgImNsaWVudF94NTA5X2NlcnRfdXJsIjogImh0dHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL3JvYm90L3YxL21ldGFkYXRhL3g1MDkvZnVqLW1hbmFnZW1lbnQtYm90JTQwcGxhdGVkLW1lc2gtNDg5MDEwLXMxLmlhbS5nc2VydmljZWFjY291bnQuY29tIiwKICAgICJ1bml2ZXJzZV9kb21haW4iOiAiZ29vZ2xlYXBpcy5jb20iCn0=
+kind: Secret
+metadata:
+  name: fuj-management-bot-credentials
+  namespace: fuj
+type: Opaque
--- a/gitops/home-kubernetes/fuj/service_fuj-management.yaml
+++ b/gitops/home-kubernetes/fuj/service_fuj-management.yaml
@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: fuj-management
+  namespace: fuj
+spec:
+  type: ClusterIP
+  selector:
+    app: fuj-management
+  ports:
+    - port: 80
+      targetPort: 8080
+      protocol: TCP
+      name: http
--- a/gitops/home-kubernetes/fuj/service_fujarna.yaml
+++ b/gitops/home-kubernetes/fuj/service_fujarna.yaml
@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: fujarna
+  namespace: fuj
+spec:
+  type: ClusterIP
+  selector:
+    app: fujarna
+  ports:
+    - port: 80
+      targetPort: 8080
+      protocol: TCP
+      name: http
--- a/gitops/home-kubernetes/oauth-proxy/helmrelease.yaml
+++ b/gitops/home-kubernetes/oauth-proxy/helmrelease.yaml
@@ -39,6 +39,7 @@ spec:
    ingress:
      enabled: true
      className: nginx
+      pathType: Prefix
      annotations:
        cert-manager.io/cluster-issuer: letsencrypt-prod
      hosts:
--- a/gitops/home-kubernetes/podinfo/helmRelease.yaml
+++ b/gitops/home-kubernetes/podinfo/helmRelease.yaml
@@ -24,7 +24,7 @@ spec:
        - host: podinfo.lab.home.hrajfrisbee.cz
          paths:
            - path: /
-              pathType: ImplementationSpecific
+              pathType: Prefix
      tls: []
      #  - secretName: chart-example-tls
      #    hosts:
--- a/gitops/home-kubernetes/seafile/conf/seahub_settings.py
+++ b/gitops/home-kubernetes/seafile/conf/seahub_settings.py
@@ -0,0 +1,12 @@
+EMAIL_USE_TLS = True
+EMAIL_HOST = 'smtp.gmail.com'
+EMAIL_HOST_USER = 'kacerr.cz@gmail.com'
+EMAIL_HOST_PASSWORD = 'zeyd ppmy gfqu gaws'  # App Password, not your login password
+EMAIL_PORT = 587
+DEFAULT_FROM_EMAIL = EMAIL_HOST_USER
+SERVER_EMAIL = EMAIL_HOST_USER
+
+
+FILE_SERVER_ROOT = 'https://seafile.lab.home.hrajfrisbee.cz/seafhttp'
+
+DEBUG = True
--- a/kubernetes-kvm-terraform/files/auth-config.yaml
+++ b/kubernetes-kvm-terraform/files/auth-config.yaml
@@ -0,0 +1,14 @@
+apiVersion: apiserver.config.k8s.io/v1beta1
+kind: AuthenticationConfiguration
+jwt:
+  - issuer:
+      url: https://idm.home.hrajfrisbee.cz/oauth2/openid/k8s
+      audiences:
+        - k8s
+    claimMappings:
+      username:
+        claim: preferred_username
+        prefix: ""
+      groups:
+        claim: groups
+        prefix: ""
--- a/kubernetes-kvm-terraform/files/join-node-02.sh
+++ b/kubernetes-kvm-terraform/files/join-node-02.sh
@@ -0,0 +1,155 @@
+#!/usr/bin/env bash
+# Manual equivalent of the cloud-init user_data for kube-node-33 (node_02).
+# Run as root on the target VM.
+#
+# Before running, generate a fresh join command on the control plane:
+#   kubeadm token create --print-join-command
+# Then paste it below.
+
+set -euo pipefail
+
+JOIN_COMMAND="kubeadm join 192.168.0.31:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>"
+
+# ---------------------------------------------------------------------------
+# Hostname
+# ---------------------------------------------------------------------------
+hostnamectl set-hostname kube-node-33
+
+# ---------------------------------------------------------------------------
+# Packages
+# ---------------------------------------------------------------------------
+apt-get update
+apt-get install -y \
+  qemu-guest-agent \
+  openssh-server \
+  apt-transport-https \
+  ca-certificates \
+  curl \
+  gnupg \
+  nvme-cli
+
+systemctl enable --now qemu-guest-agent
+systemctl enable --now ssh
+
+# ---------------------------------------------------------------------------
+# nvme-tcp
+# ---------------------------------------------------------------------------
+apt-get install -y linux-modules-extra-$(uname -r)
+modprobe nvme-tcp
+echo "nvme-tcp" >> /etc/modules-load.d/nvme-tcp.conf
+
+# ---------------------------------------------------------------------------
+# Kernel modules for Kubernetes
+# ---------------------------------------------------------------------------
+cat > /etc/modules-load.d/k8s.conf <<'EOF'
+overlay
+br_netfilter
+EOF
+
+modprobe overlay
+modprobe br_netfilter
+
+cat > /etc/sysctl.d/k8s.conf <<'EOF'
+net.bridge.bridge-nf-call-iptables = 1
+net.bridge.bridge-nf-call-ip6tables = 1
+net.ipv4.ip_forward = 1
+EOF
+
+sysctl --system
+
+# ---------------------------------------------------------------------------
+# containerd
+# ---------------------------------------------------------------------------
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
+echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" \
+  > /etc/apt/sources.list.d/docker.list
+apt-get update && apt-get install -y containerd.io
+
+cat > /etc/containerd/config.toml <<'EOF'
+version = 2
+[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
+  runtime_type = "io.containerd.runc.v2"
+  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
+    SystemdCgroup = true
+[plugins."io.containerd.grpc.v1.cri".registry]
+  config_path = "/etc/containerd/certs.d"
+EOF
+
+# Registry mirrors pointing to Zot at 192.168.0.30:5000
+mkdir -p \
+  /etc/containerd/certs.d/docker.io \
+  /etc/containerd/certs.d/registry.k8s.io \
+  /etc/containerd/certs.d/ghcr.io \
+  /etc/containerd/certs.d/quay.io
+
+cat > /etc/containerd/certs.d/docker.io/hosts.toml <<'EOF'
+server = "https://registry-1.docker.io"
+[host."http://192.168.0.30:5000/v2/docker.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+cat > /etc/containerd/certs.d/registry.k8s.io/hosts.toml <<'EOF'
+server = "https://registry.k8s.io"
+[host."http://192.168.0.30:5000/v2/registry.k8s.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+cat > /etc/containerd/certs.d/ghcr.io/hosts.toml <<'EOF'
+server = "https://ghcr.io"
+[host."http://192.168.0.30:5000/v2/ghcr.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+cat > /etc/containerd/certs.d/quay.io/hosts.toml <<'EOF'
+server = "https://quay.io"
+[host."http://192.168.0.30:5000/v2/quay.io"]
+  capabilities = ["pull", "resolve"]
+  skip_verify = true
+  override_path = true
+EOF
+
+systemctl restart containerd
+
+# ---------------------------------------------------------------------------
+# kubelet systemd drop-in
+# ---------------------------------------------------------------------------
+mkdir -p /etc/systemd/system/kubelet.service.d
+cat > /etc/systemd/system/kubelet.service.d/10-containerd.conf <<'EOF'
+[Unit]
+After=containerd.service
+Requires=containerd.service
+
+[Service]
+ExecStartPre=/bin/bash -c 'until [ -S /var/run/containerd/containerd.sock ]; do sleep 1; done'
+ExecStartPre=/usr/bin/crictl info
+EOF
+
+# ---------------------------------------------------------------------------
+# kubeadm / kubelet / kubectl v1.32
+# ---------------------------------------------------------------------------
+curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key \
+  | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
+echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /" \
+  > /etc/apt/sources.list.d/kubernetes.list
+apt-get update && apt-get install -y kubelet kubeadm kubectl
+apt-mark hold kubelet kubeadm kubectl
+
+# ---------------------------------------------------------------------------
+# kubectl shell helpers (available after next login / source)
+# ---------------------------------------------------------------------------
+cat > /etc/profile.d/kubectl.sh <<'EOF'
+alias k='kubectl'
+source <(kubectl completion bash)
+complete -o default -F __start_kubectl k
+EOF
+
+# ---------------------------------------------------------------------------
+# Join the cluster
+# ---------------------------------------------------------------------------
+$JOIN_COMMAND
--- a/kubernetes-kvm-terraform/files/manifests/kube-apiserver.yaml
+++ b/kubernetes-kvm-terraform/files/manifests/kube-apiserver.yaml
@@ -0,0 +1,132 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  annotations:
+    kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.0.31:6443
+  creationTimestamp: null
+  labels:
+    component: kube-apiserver
+    tier: control-plane
+  name: kube-apiserver
+  namespace: kube-system
+spec:
+  containers:
+  - command:
+    - kube-apiserver
+    - --advertise-address=192.168.0.31
+    - --allow-privileged=true
+    - --authorization-mode=Node,RBAC
+    - --client-ca-file=/etc/kubernetes/pki/ca.crt
+    - --enable-admission-plugins=NodeRestriction
+    - --enable-bootstrap-token-auth=true
+    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
+    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
+    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
+    - --etcd-servers=https://127.0.0.1:2379
+    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
+    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
+    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
+    #- --oidc-issuer-url=https://idm.home.hrajfrisbee.cz/oauth2/openid/k8s
+    #- --oidc-client-id=k8s
+    #- --oidc-signing-algs=ES256
+    - --authentication-config=/etc/kubernetes/auth-config.yaml
+    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
+    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
+    - --requestheader-allowed-names=front-proxy-client
+    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
+    - --requestheader-extra-headers-prefix=X-Remote-Extra-
+    - --requestheader-group-headers=X-Remote-Group
+    - --requestheader-username-headers=X-Remote-User
+    - --secure-port=6443
+    - --service-account-issuer=https://kubernetes.default.svc.cluster.local
+    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
+    - --service-account-signing-key-file=/etc/kubernetes/pki/sa.key
+    - --service-cluster-ip-range=10.96.0.0/12
+    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
+    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
+    image: registry.k8s.io/kube-apiserver:v1.32.11
+    imagePullPolicy: IfNotPresent
+    livenessProbe:
+      failureThreshold: 8
+      httpGet:
+        host: 192.168.0.31
+        path: /livez
+        port: 6443
+        scheme: HTTPS
+      initialDelaySeconds: 10
+      periodSeconds: 10
+      timeoutSeconds: 15
+    name: kube-apiserver
+    readinessProbe:
+      failureThreshold: 3
+      httpGet:
+        host: 192.168.0.31
+        path: /readyz
+        port: 6443
+        scheme: HTTPS
+      periodSeconds: 1
+      timeoutSeconds: 15
+    resources:
+      requests:
+        cpu: 250m
+    startupProbe:
+      failureThreshold: 24
+      httpGet:
+        host: 192.168.0.31
+        path: /livez
+        port: 6443
+        scheme: HTTPS
+      initialDelaySeconds: 10
+      periodSeconds: 10
+      timeoutSeconds: 15
+    volumeMounts:
+    - mountPath: /etc/ssl/certs
+      name: ca-certs
+      readOnly: true
+    - mountPath: /etc/ca-certificates
+      name: etc-ca-certificates
+      readOnly: true
+    - mountPath: /etc/kubernetes/pki
+      name: k8s-certs
+      readOnly: true
+    - mountPath: /usr/local/share/ca-certificates
+      name: usr-local-share-ca-certificates
+      readOnly: true
+    - mountPath: /usr/share/ca-certificates
+      name: usr-share-ca-certificates
+      readOnly: true
+    - mountPath: /etc/kubernetes
+      name: k8s-config
+      readOnly: true      
+  hostNetwork: true
+  priority: 2000001000
+  priorityClassName: system-node-critical
+  securityContext:
+    seccompProfile:
+      type: RuntimeDefault
+  volumes:
+  - hostPath:
+      path: /etc/ssl/certs
+      type: DirectoryOrCreate
+    name: ca-certs
+  - hostPath:
+      path: /etc/ca-certificates
+      type: DirectoryOrCreate
+    name: etc-ca-certificates
+  - hostPath:
+      path: /etc/kubernetes/pki
+      type: DirectoryOrCreate
+    name: k8s-certs
+  - hostPath:
+      path: /usr/local/share/ca-certificates
+      type: DirectoryOrCreate
+    name: usr-local-share-ca-certificates
+  - hostPath:
+      path: /usr/share/ca-certificates
+      type: DirectoryOrCreate
+    name: usr-share-ca-certificates
+  - hostPath:
+      path: /etc/kubernetes
+      type: DirectoryOrCreate
+    name: k8s-config    
+status: {}
--- a/kubernetes-kvm-terraform/master.tf
+++ b/kubernetes-kvm-terraform/master.tf
@@ -115,6 +115,23 @@ locals {
            skip_verify = true
            override_path = true              

+      - path: /etc/kubernetes/auth-config.yaml
+        content: |
+          apiVersion: apiserver.config.k8s.io/v1beta1
+          kind: AuthenticationConfiguration
+          jwt:
+            - issuer:
+                url: https://idm.home.hrajfrisbee.cz/oauth2/openid/k8s
+                audiences:
+                  - k8s
+              claimMappings:
+                username:
+                  claim: preferred_username
+                  prefix: ""
+                groups:
+                  claim: groups
+                  prefix: ""
+      
      - path: /root/kubeadm-config.yaml
        content: |
          apiVersion: kubeadm.k8s.io/v1beta3
@@ -130,6 +147,13 @@ locals {
            oidc-signing-algs: "ES256"          
          networking:
            podSubnet: "10.244.0.0/16"
+          # /etc/kubernetes/auth-config.yaml has to be created on master node somehow
+          extraVolumes:
+            - name: auth-config
+              hostPath: /etc/kubernetes/auth-config.yaml
+              mountPath: /etc/kubernetes/auth-config.yaml
+              readOnly: true
+              pathType: File            
          ---
          apiVersion: kubelet.config.k8s.io/v1beta1
          kind: KubeletConfiguration
--- a/servers/saint/export-vms.md
+++ b/servers/saint/export-vms.md
@@ -0,0 +1,49 @@
+## export VMs
+
+```bash
+# vms need to get exported so that i can do something with outdated hypervisor
+
+vm=utility-101
+virsh dumpxml $vm > $vm-saint.xml
+virsh domblklist $vm
+# /srv/vms-normal-hdd/utility-101.qcow2
+
+tar -cvzf ${vm}-saint-backup.tar.gz ${vm}-saint.xml /srv/vms-normal-hdd/utility-101.qcow2
+
+vm=storage
+h=saint
+virsh shutdown $vm
+virsh dumpxml $vm > $vm-$h.xml
+virsh domblklist $vm
+# /srv/vms-normal-hdd/storage.qcow2
+
+tar -cvzf ${vm}-${h}-backup.tar.gz ${vm}-${h}.xml /srv/vms-normal-hdd/storage.qcow2
+
+# ----
+vm=docker
+h=saint
+virsh shutdown $vm
+virsh dumpxml $vm > $vm-$h.xml
+virsh domblklist $vm
+# /srv/vms-ssd/docker.qcow2
+# /srv/vms-normal-hdd/docker-srv.qcow2
+# /srv/vms-normal-hdd/docker-vgroot-2.qcow2
+
+tar -cvzf ${vm}-${h}-backup.tar.gz ${vm}-${h}.xml /srv/vms-ssd/docker.qcow2 /srv/vms-normal-hdd/docker-srv.qcow2 /srv/vms-normal-hdd/docker-vgroot-2.qcow2
+
+
+# ----
+vm=demon
+h=agent
+virsh shutdown $vm
+virsh dumpxml $vm > $vm-$h.xml
+virsh domblklist $vm
+# /srv/nfs/data/vms-moved-from-failing-disk/demon_1.img
+# /srv/nfs/data/vms-moved-from-failing-disk/demon_2nd_disk.img
+
+tar -cvzf ${vm}-${h}-backup.tar.gz ${vm}-${h}.xml /srv/nfs/data/vms-moved-from-failing-disk/demon_1.img /srv/nfs/data/vms-moved-from-failing-disk/demon_2nd_disk.img
+
+
+
+
+```
--- a/servers/saint/files/nginx/104_docker_zlutazimnice.cz.conf
+++ b/servers/saint/files/nginx/104_docker_zlutazimnice.cz.conf
@@ -0,0 +1,378 @@
+server {
+  server_name
+        api.psmf.hrajfrisbee.cz
+        ;
+  location / {
+          proxy_set_header        Host            $host;
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_set_header        X-Forwarded-Proto $scheme;
+    proxy_pass http://192.168.122.25:5001;
+  }
+
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/api.psmf.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/api.psmf.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+server {
+  #  if ($host = api-sync.psmf.hrajfrisbee.cz) {
+  #      return 301 https://$host$request_uri;
+  #  } # managed by Certbot
+
+
+  listen 80;
+  client_max_body_size 15m;                                                                                                                                                              proxy_read_timeout 600;
+  server_name
+        api-sync.psmf.hrajfrisbee.cz
+        ;
+  location / {
+    proxy_pass http://192.168.122.25:8003;
+  }
+}
+
+server {
+  server_name
+        api-sync.psmf.hrajfrisbee.cz
+        ;
+  location / {
+    proxy_pass http://192.168.122.25:8003;
+  }
+
+  client_max_body_size 15m;
+  proxy_read_timeout 600;
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/api-sync.psmf.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/api-sync.psmf.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+}
+
+server {
+  server_name api.evidence.cald.cz
+              api.evidence.czechultimate.cz
+                ;
+  location / {
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_pass http://192.168.122.25:8001;
+  }
+
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/api.evidence.czechultimate.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/api.evidence.czechultimate.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+server {
+  server_name evidence.cald.cz
+              evidence.czechultimate.cz
+                ;
+  location / {
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_pass http://192.168.122.25:8002;
+  }
+
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/evidence.czechultimate.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/evidence.czechultimate.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+server {
+  server_name
+    hrajfrisbee.cz
+    www.hrajfrisbee.cz;
+  location / {
+          proxy_set_header        Host            $host;
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_set_header        X-Forwarded-Proto $scheme;
+    proxy_pass http://192.168.122.25:5002;
+  }
+
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+server {
+    if ($host = beta.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+        listen 80;
+        server_name  beta.hrajfrisbee.cz;
+        location / {
+          proxy_set_header        Host            $host;
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_set_header        X-Forwarded-Proto $scheme;
+          proxy_pass http://192.168.122.25:5003;
+        }
+}
+
+server {
+  server_name
+    hrajfrisbee.cz
+    www.hrajfrisbee.cz;
+  location / {
+          proxy_set_header        Host            $host;
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_set_header        X-Forwarded-Proto $scheme;
+    proxy_pass http://192.168.122.25:5002;
+  }
+
+    listen 443 ssl; # managed by Certbot
+ssl_certificate /etc/letsencrypt/live/www.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+ssl_certificate_key /etc/letsencrypt/live/www.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+server {
+        listen 80;
+        server_name  beta.hrajfrisbee.cz;
+        location / {
+          proxy_set_header        Host            $host;
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_set_header        X-Forwarded-Proto $scheme;
+          proxy_pass http://192.168.122.25:5003;
+        }
+
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+
+server {
+
+        server_name
+                gitlab.hrajfrisbee.cz
+        ;
+
+          location / {
+            proxy_pass http://192.168.122.25:80;
+            proxy_set_header        X-Real-IP       $remote_addr;
+            proxy_set_header        Host            $host;
+            proxy_redirect          off;
+            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+            proxy_connect_timeout 90;
+            proxy_send_timeout 90;
+            proxy_read_timeout 90;
+            client_max_body_size 10m;
+            client_body_buffer_size 128k;
+            proxy_buffer_size 4k;
+            proxy_buffers 4 32k;
+            proxy_busy_buffers_size 64k;
+
+          }
+
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/gitlab.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/gitlab.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+server {
+        server_name
+                registry.gitlab.hrajfrisbee.cz
+        ;
+
+        location / {
+                proxy_pass       http://192.168.122.25:5005;
+                proxy_set_header Host            $host;
+                proxy_set_header X-Forwarded-For $remote_addr;
+                proxy_set_header X-Forwarded-Proto https;
+                proxy_set_header X-Forwarded-Ssl on;
+
+                client_max_body_size 3000M;
+                client_body_buffer_size 200000k;
+            #auth_basic "Restricted Content";
+            #auth_basic_user_file /etc/nginx/.htpasswd;
+          }
+
+
+    listen 443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/registry.gitlab.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/registry.gitlab.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+
+server {
+    if ($host = hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+    if ($host = www.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+  listen 80;
+  server_name
+    hrajfrisbee.cz
+    www.hrajfrisbee.cz;
+    return 404; # managed by Certbot
+}
+
+server {
+    if ($host = hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+  listen 80;
+  server_name
+    hrajfrisbee.cz
+    www.hrajfrisbee.cz;
+    return 404; # managed by Certbot
+}
+
+
+server {
+    if ($host = gitlab.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+        server_name
+                gitlab.hrajfrisbee.cz
+        ;
+
+        listen 80;
+    return 404; # managed by Certbot
+
+
+}
+
+server {
+    if ($host = evidence.cald.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+  listen 80;
+  server_name evidence.cald.cz;
+    return 404; # managed by Certbot
+}
+
+server {
+    if ($host = evidence.czechultimate.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+  listen 80;
+  server_name evidence.czechultimate.cz;
+    return 404; # managed by Certbot
+}
+
+server {
+    if ($host = api.evidence.cald.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+  listen 80;
+  server_name api.evidence.cald.cz;
+    return 404; # managed by Certbot
+}
+
+server {
+    if ($host = api.evidence.czechultimate.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+  listen 80;
+  server_name api.evidence.czechultimate.cz;
+    return 404; # managed by Certbot
+}
+
+server {
+    if ($host = api.psmf.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+  listen 80;
+  server_name api.psmf.hrajfrisbee.cz;
+    return 404; # managed by Certbot
+}
+
+
+#server {
+#    if ($host = api-sync.psmf.hrajfrisbee.cz) {
+#        return 301 https://$host$request_uri;
+#    } # managed by Certbot
+#
+#
+#  listen 80;
+#  server_name
+#       api-sync.psmf.hrajfrisbee.cz
+#       ;
+#    return 404; # managed by Certbot
+#}
+
+
+server {
+#    if ($host = api-sync.psmf.hrajfrisbee.cz) {
+#        return 301 https://$host$request_uri;
+#    } # managed by Certbot
+
+
+  server_name
+        api-sync.psmf.hrajfrisbee.cz
+        ;
+
+    listen 80;
+    return 404; # managed by Certbot
+
+
+}
--- a/servers/saint/files/utility-101-saint.xml
+++ b/servers/saint/files/utility-101-saint.xml
@@ -0,0 +1,111 @@
+<domain type='kvm'>
+  <name>utility-101</name>
+  <uuid>82f853d8-9997-4b6d-96b0-629482f2c61e</uuid>
+  <memory unit='KiB'>2097152</memory>
+  <currentMemory unit='KiB'>2097152</currentMemory>
+  <vcpu placement='static'>2</vcpu>
+  <os>
+    <type arch='x86_64' machine='pc-i440fx-rhel7.0.0'>hvm</type>
+    <boot dev='hd'/>
+  </os>
+  <features>
+    <acpi/>
+    <apic/>
+  </features>
+  <cpu mode='custom' match='exact' check='partial'>
+    <model fallback='allow'>Haswell-noTSX</model>
+  </cpu>
+  <clock offset='utc'>
+    <timer name='rtc' tickpolicy='catchup'/>
+    <timer name='pit' tickpolicy='delay'/>
+    <timer name='hpet' present='no'/>
+  </clock>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <pm>
+    <suspend-to-mem enabled='no'/>
+    <suspend-to-disk enabled='no'/>
+  </pm>
+  <devices>
+    <emulator>/usr/libexec/qemu-kvm</emulator>
+    <disk type='file' device='disk'>
+      <driver name='qemu' type='qcow2'/>
+      <source file='/srv/vms-normal-hdd/utility-101.qcow2'/>
+      <target dev='hda' bus='ide'/>
+      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
+    </disk>
+    <disk type='file' device='cdrom'>
+      <driver name='qemu' type='raw'/>
+      <target dev='hdb' bus='ide'/>
+      <readonly/>
+      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
+    </disk>
+    <controller type='usb' index='0' model='ich9-ehci1'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
+    </controller>
+    <controller type='usb' index='0' model='ich9-uhci1'>
+      <master startport='0'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
+    </controller>
+    <controller type='usb' index='0' model='ich9-uhci2'>
+      <master startport='2'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
+    </controller>
+    <controller type='usb' index='0' model='ich9-uhci3'>
+      <master startport='4'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
+    </controller>
+    <controller type='pci' index='0' model='pci-root'/>
+    <controller type='ide' index='0'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    </controller>
+    <controller type='virtio-serial' index='0'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
+    </controller>
+    <interface type='network'>
+      <mac address='52:54:00:cd:83:a4'/>
+      <source network='default'/>
+      <model type='rtl8139'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
+    </interface>
+    <interface type='network'>
+      <mac address='52:54:00:cd:66:dc'/>
+      <source network='network-4'/>
+      <model type='rtl8139'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
+    </interface>
+    <serial type='pty'>
+      <target port='0'/>
+    </serial>
+    <console type='pty'>
+      <target type='serial' port='0'/>
+    </console>
+    <channel type='spicevmc'>
+      <target type='virtio' name='com.redhat.spice.0'/>
+      <address type='virtio-serial' controller='0' bus='0' port='1'/>
+    </channel>
+    <input type='mouse' bus='ps2'/>
+    <input type='keyboard' bus='ps2'/>
+    <graphics type='spice' autoport='yes'>
+      <listen type='address'/>
+      <image compression='off'/>
+    </graphics>
+    <sound model='ich6'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
+    </sound>
+    <video>
+      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
+    </video>
+    <redirdev bus='usb' type='spicevmc'>
+      <address type='usb' bus='0' port='1'/>
+    </redirdev>
+    <redirdev bus='usb' type='spicevmc'>
+      <address type='usb' bus='0' port='2'/>
+    </redirdev>
+    <memballoon model='virtio'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
+    </memballoon>
+  </devices>
+</domain>
--- a/servers/shadow/iptables/rules.v4
+++ b/servers/shadow/iptables/rules.v4
@@ -0,0 +1,134 @@
+# Generated by iptables-save v1.8.10 (nf_tables) on Sun Nov 17 01:37:49 2024
+*mangle
+:PREROUTING ACCEPT [756:126788]
+:INPUT ACCEPT [715:122089]
+:FORWARD ACCEPT [40:4623]
+:OUTPUT ACCEPT [420:58795]
+:POSTROUTING ACCEPT [460:63418]
+:LIBVIRT_PRT - [0:0]
+-A POSTROUTING -j LIBVIRT_PRT
+-A LIBVIRT_PRT -o virbr100 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+COMMIT
+# Completed on Sun Nov 17 01:37:49 2024
+# Generated by iptables-save v1.8.10 (nf_tables) on Sun Nov 17 01:37:49 2024
+*filter
+:INPUT DROP [387:104781]
+:FORWARD DROP [0:0]
+:OUTPUT ACCEPT [42:5859]
+:DOCKER - [0:0]
+:DOCKER-ISOLATION-STAGE-1 - [0:0]
+:DOCKER-ISOLATION-STAGE-2 - [0:0]
+:DOCKER-USER - [0:0]
+:LIBVIRT_FWI - [0:0]
+:LIBVIRT_FWO - [0:0]
+:LIBVIRT_FWX - [0:0]
+:LIBVIRT_INP - [0:0]
+:LIBVIRT_OUT - [0:0]
+:f2b-sshd - [0:0]
+-A INPUT -j LIBVIRT_INP
+-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
+-A INPUT -p icmp -j ACCEPT
+-A INPUT -i virbr100 -j ACCEPT
+-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
+-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
+-A INPUT -i virbr0 -p udp -m udp --dport 5353 -j ACCEPT
+-A INPUT -i virbr0 -p tcp -m tcp --dport 5353 -j ACCEPT
+-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
+-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 1022 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 2022 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -i lo -j ACCEPT
+-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -i eno1 -p tcp -m tcp --dport 53 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 53 -j ACCEPT
+-A FORWARD -i eno1 -p tcp -m tcp --dport 5353 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 5353 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 51820 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 1194 -j ACCEPT
+-A FORWARD -j DOCKER-USER
+-A FORWARD -j DOCKER-ISOLATION-STAGE-1
+-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -o docker0 -j DOCKER
+-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
+-A FORWARD -i docker0 -o docker0 -j ACCEPT
+-A FORWARD -j LIBVIRT_FWX
+-A FORWARD -j LIBVIRT_FWI
+-A FORWARD -j LIBVIRT_FWO
+-A FORWARD -o br-8be00fb1442a -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -o br-8be00fb1442a -j DOCKER
+-A FORWARD -i br-8be00fb1442a ! -o br-8be00fb1442a -j ACCEPT
+-A FORWARD -i br-8be00fb1442a -o br-8be00fb1442a -j ACCEPT
+-A FORWARD -d 192.168.123.141/32 -p tcp -m tcp --dport 80 -j ACCEPT
+-A OUTPUT -j LIBVIRT_OUT
+-A OUTPUT -p tcp -m tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT
+-A OUTPUT -o lo -j ACCEPT
+-A OUTPUT -o virbr100 -j ACCEPT
+-A OUTPUT -m conntrack --ctstate ESTABLISHED -j ACCEPT
+-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
+-A DOCKER-ISOLATION-STAGE-1 -i br-8be00fb1442a ! -o br-8be00fb1442a -j DOCKER-ISOLATION-STAGE-2
+-A DOCKER-ISOLATION-STAGE-1 -j RETURN
+-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
+-A DOCKER-ISOLATION-STAGE-2 -o br-8be00fb1442a -j DROP
+-A DOCKER-ISOLATION-STAGE-2 -j RETURN
+-A DOCKER-USER -j RETURN
+-A LIBVIRT_FWI -d 192.168.123.0/24 -o virbr100 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A LIBVIRT_FWI -o virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWI -o virbr1 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWO -s 192.168.123.0/24 -i virbr100 -j ACCEPT
+-A LIBVIRT_FWO -i virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWO -i virbr1 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
+-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWX -i virbr100 -o virbr100 -j ACCEPT
+-A LIBVIRT_FWX -i virbr1 -o virbr1 -j ACCEPT
+-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
+-A LIBVIRT_INP -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -p udp -m udp --dport 5353 -j ACCEPT
+-A LIBVIRT_INP -p tcp -m tcp --dport 5353 -j ACCEPT
+-A LIBVIRT_INP -p udp -m udp --dport 67 -j ACCEPT
+-A LIBVIRT_INP -p tcp -m tcp --dport 67 -j ACCEPT
+-A LIBVIRT_OUT -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -p udp -m udp --dport 5353 -j ACCEPT
+-A LIBVIRT_OUT -p tcp -m tcp --dport 5353 -j ACCEPT
+-A LIBVIRT_OUT -p udp -m udp --dport 68 -j ACCEPT
+-A LIBVIRT_OUT -p tcp -m tcp --dport 68 -j ACCEPT
+-A f2b-sshd -j RETURN
+COMMIT
+# Completed on Sun Nov 17 01:37:49 2024
+# Generated by iptables-save v1.8.10 (nf_tables) on Sun Nov 17 01:37:49 2024
+*nat
+:PREROUTING ACCEPT [409:105569]
+:INPUT ACCEPT [22:1288]
+:OUTPUT ACCEPT [1:76]
+:POSTROUTING ACCEPT [12:818]
+:DOCKER - [0:0]
+:LIBVIRT_PRT - [0:0]
+-A PREROUTING -i eno1 -p tcp -m tcp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p tcp -m tcp --dport 5353 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 5353 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 51820 -j DNAT --to-destination 192.168.123.101:51820
+-A PREROUTING -i eno1 -p udp -m udp --dport 1194 -j DNAT --to-destination 192.168.123.101:1194
+-A PREROUTING -i eno1 -p tcp -m tcp --dport 21080 -j DNAT --to-destination 192.168.123.141:80
+-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
+-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
+-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
+-A POSTROUTING -j LIBVIRT_PRT
+-A POSTROUTING -s 172.18.0.0/16 ! -o br-8be00fb1442a -j MASQUERADE
+-A DOCKER -i docker0 -j RETURN
+-A DOCKER -i br-8be00fb1442a -j RETURN
+-A LIBVIRT_PRT -s 192.168.123.0/24 -d 224.0.0.0/24 -j RETURN
+-A LIBVIRT_PRT -s 192.168.123.0/24 -d 255.255.255.255/32 -j RETURN
+-A LIBVIRT_PRT -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
+-A LIBVIRT_PRT -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
+-A LIBVIRT_PRT -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE
+COMMIT
+# Completed on Sun Nov 17 01:37:49 2024
--- a/servers/shadow/iptables/rules.v4.backup
+++ b/servers/shadow/iptables/rules.v4.backup
@@ -0,0 +1,248 @@
+# Generated by iptables-save v1.8.10 (nf_tables) on Sun Nov 17 01:37:49 2024
+*mangle
+:PREROUTING ACCEPT [756:126788]
+:INPUT ACCEPT [715:122089]
+:FORWARD ACCEPT [40:4623]
+:OUTPUT ACCEPT [420:58795]
+:POSTROUTING ACCEPT [460:63418]
+:LIBVIRT_PRT - [0:0]
+-A POSTROUTING -j LIBVIRT_PRT
+-A POSTROUTING -o virbr100 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A POSTROUTING -o virbr100 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A POSTROUTING -o virbr100 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A POSTROUTING -o virbr100 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A LIBVIRT_PRT -o virbr100 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
+COMMIT
+# Completed on Sun Nov 17 01:37:49 2024
+# Generated by iptables-save v1.8.10 (nf_tables) on Sun Nov 17 01:37:49 2024
+*filter
+:INPUT DROP [387:104781]
+:FORWARD DROP [0:0]
+:OUTPUT ACCEPT [42:5859]
+:DOCKER - [0:0]
+:DOCKER-ISOLATION-STAGE-1 - [0:0]
+:DOCKER-ISOLATION-STAGE-2 - [0:0]
+:DOCKER-USER - [0:0]
+:LIBVIRT_FWI - [0:0]
+:LIBVIRT_FWO - [0:0]
+:LIBVIRT_FWX - [0:0]
+:LIBVIRT_INP - [0:0]
+:LIBVIRT_OUT - [0:0]
+:f2b-sshd - [0:0]
+-A INPUT -j LIBVIRT_INP
+-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
+-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
+-A INPUT -p icmp -j ACCEPT
+-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
+-A INPUT -i virbr100 -p udp -m udp --dport 53 -j ACCEPT
+-A INPUT -i virbr100 -p tcp -m tcp --dport 53 -j ACCEPT
+-A INPUT -i virbr100 -p udp -m udp --dport 67 -j ACCEPT
+-A INPUT -i virbr100 -p tcp -m tcp --dport 67 -j ACCEPT
+-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
+-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
+-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
+-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
+-A INPUT -i virbr100 -j ACCEPT
+-A INPUT -i virbr100 -p udp -m udp --dport 53 -j ACCEPT
+-A INPUT -i virbr100 -p tcp -m tcp --dport 53 -j ACCEPT
+-A INPUT -i virbr100 -p udp -m udp --dport 67 -j ACCEPT
+-A INPUT -i virbr100 -p tcp -m tcp --dport 67 -j ACCEPT
+-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
+-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
+-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
+-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
+-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
+-A INPUT -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 1022 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 2022 -m state --state NEW,ESTABLISHED -j ACCEPT
+-A INPUT -i lo -j ACCEPT
+-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A INPUT -i virbr100 -j ACCEPT
+-A FORWARD -i eno1 -p tcp -m tcp --dport 53 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 53 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 51820 -j ACCEPT
+-A FORWARD -j DOCKER-USER
+-A FORWARD -j DOCKER-ISOLATION-STAGE-1
+-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -o docker0 -j DOCKER
+-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
+-A FORWARD -i docker0 -o docker0 -j ACCEPT
+-A FORWARD -j LIBVIRT_FWX
+-A FORWARD -j LIBVIRT_FWI
+-A FORWARD -j LIBVIRT_FWO
+-A FORWARD -i eno1 -p tcp -m tcp --dport 53 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 53 -j ACCEPT
+-A FORWARD -d 192.168.123.0/24 -o virbr100 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -s 192.168.123.0/24 -i virbr100 -j ACCEPT
+-A FORWARD -i virbr100 -o virbr100 -j ACCEPT
+-A FORWARD -o virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -i virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
+-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
+-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -i eno1 -p udp -m udp --dport 1194 -j ACCEPT
+-A FORWARD -i eno1 -p tcp -m tcp --dport 53 -j ACCEPT
+-A FORWARD -i eno1 -p udp -m udp --dport 53 -j ACCEPT
+-A FORWARD -d 192.168.123.0/24 -o virbr100 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -s 192.168.123.0/24 -i virbr100 -j ACCEPT
+-A FORWARD -i virbr100 -o virbr100 -j ACCEPT
+-A FORWARD -o virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -i virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
+-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
+-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A FORWARD -o br-8be00fb1442a -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -o br-8be00fb1442a -j DOCKER
+-A FORWARD -i br-8be00fb1442a ! -o br-8be00fb1442a -j ACCEPT
+-A FORWARD -i br-8be00fb1442a -o br-8be00fb1442a -j ACCEPT
+-A FORWARD -d 192.168.123.141/32 -p tcp -m tcp --dport 80 -j ACCEPT
+-A OUTPUT -j LIBVIRT_OUT
+-A OUTPUT -o virbr100 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -o virbr100 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -o virbr100 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -p tcp -m tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT
+-A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -o virbr100 -p udp -m udp --dport 68 -j ACCEPT
+-A OUTPUT -o lo -j ACCEPT
+-A OUTPUT -o virbr100 -j ACCEPT
+-A OUTPUT -m conntrack --ctstate ESTABLISHED -j ACCEPT
+-A OUTPUT -o virbr100 -j ACCEPT
+-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
+-A DOCKER-ISOLATION-STAGE-1 -i br-8be00fb1442a ! -o br-8be00fb1442a -j DOCKER-ISOLATION-STAGE-2
+-A DOCKER-ISOLATION-STAGE-1 -j RETURN
+-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
+-A DOCKER-ISOLATION-STAGE-2 -o br-8be00fb1442a -j DROP
+-A DOCKER-ISOLATION-STAGE-2 -j RETURN
+-A DOCKER-USER -j RETURN
+-A LIBVIRT_FWI -d 192.168.123.0/24 -o virbr100 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A LIBVIRT_FWI -o virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWI -o virbr1 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWO -s 192.168.123.0/24 -i virbr100 -j ACCEPT
+-A LIBVIRT_FWO -i virbr100 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWO -i virbr1 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
+-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
+-A LIBVIRT_FWX -i virbr100 -o virbr100 -j ACCEPT
+-A LIBVIRT_FWX -i virbr1 -o virbr1 -j ACCEPT
+-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
+-A LIBVIRT_INP -i virbr100 -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -i virbr100 -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -i virbr100 -p udp -m udp --dport 67 -j ACCEPT
+-A LIBVIRT_INP -i virbr100 -p tcp -m tcp --dport 67 -j ACCEPT
+-A LIBVIRT_INP -i virbr1 -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -i virbr1 -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -i virbr1 -p udp -m udp --dport 67 -j ACCEPT
+-A LIBVIRT_INP -i virbr1 -p tcp -m tcp --dport 67 -j ACCEPT
+-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
+-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
+-A LIBVIRT_OUT -o virbr100 -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -o virbr100 -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -o virbr100 -p udp -m udp --dport 68 -j ACCEPT
+-A LIBVIRT_OUT -o virbr100 -p tcp -m tcp --dport 68 -j ACCEPT
+-A LIBVIRT_OUT -o virbr1 -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -o virbr1 -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -o virbr1 -p udp -m udp --dport 68 -j ACCEPT
+-A LIBVIRT_OUT -o virbr1 -p tcp -m tcp --dport 68 -j ACCEPT
+-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
+-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
+-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
+-A f2b-sshd -s 222.187.254.41/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 207.46.227.197/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 125.77.23.30/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 222.186.175.216/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 94.200.202.26/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 103.80.36.218/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 62.234.126.132/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 106.52.248.175/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 104.248.5.69/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 129.211.49.227/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 112.85.42.176/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 222.186.15.62/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 222.186.30.112/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 222.186.175.167/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 222.186.52.39/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 207.154.215.119/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 36.91.76.171/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 134.175.19.71/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 144.217.243.216/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 210.206.92.137/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 222.186.30.76/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 49.51.90.173/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -s 222.186.190.2/32 -j REJECT --reject-with icmp-port-unreachable
+-A f2b-sshd -j RETURN
+-A f2b-sshd -j RETURN
+-A f2b-sshd -j RETURN
+-A f2b-sshd -j RETURN
+COMMIT
+# Completed on Sun Nov 17 01:37:49 2024
+# Generated by iptables-save v1.8.10 (nf_tables) on Sun Nov 17 01:37:49 2024
+*nat
+:PREROUTING ACCEPT [409:105569]
+:INPUT ACCEPT [22:1288]
+:OUTPUT ACCEPT [1:76]
+:POSTROUTING ACCEPT [12:818]
+:DOCKER - [0:0]
+:LIBVIRT_PRT - [0:0]
+-A PREROUTING -i eno1 -p tcp -m tcp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 51820 -j DNAT --to-destination 192.168.123.101:51820
+-A PREROUTING -i eno1 -p tcp -m tcp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 1194 -j DNAT --to-destination 192.168.123.101:1194
+-A PREROUTING -i eno1 -p tcp -m tcp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p udp -m udp --dport 53 -j DNAT --to-destination 192.168.123.101:53
+-A PREROUTING -i eno1 -p tcp -m tcp --dport 21080 -j DNAT --to-destination 192.168.123.141:80
+-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
+-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
+-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
+-A POSTROUTING -j LIBVIRT_PRT
+-A POSTROUTING -s 192.168.123.0/24 -d 224.0.0.0/24 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 -d 255.255.255.255/32 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE
+-A POSTROUTING -s 192.168.123.0/24 -d 224.0.0.0/24 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 -d 255.255.255.255/32 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE
+-A POSTROUTING -s 172.18.0.0/16 ! -o br-8be00fb1442a -j MASQUERADE
+-A POSTROUTING -s 192.168.123.0/24 -d 224.0.0.0/24 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 -d 255.255.255.255/32 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE
+-A POSTROUTING -s 192.168.123.0/24 -d 224.0.0.0/24 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 -d 255.255.255.255/32 -j RETURN
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
+-A POSTROUTING -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE
+-A DOCKER -i docker0 -j RETURN
+-A DOCKER -i br-8be00fb1442a -j RETURN
+-A LIBVIRT_PRT -s 192.168.123.0/24 -d 224.0.0.0/24 -j RETURN
+-A LIBVIRT_PRT -s 192.168.123.0/24 -d 255.255.255.255/32 -j RETURN
+-A LIBVIRT_PRT -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
+-A LIBVIRT_PRT -s 192.168.123.0/24 ! -d 192.168.123.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
+-A LIBVIRT_PRT -s 192.168.123.0/24 ! -d 192.168.123.0/24 -j MASQUERADE
+COMMIT
+# Completed on Sun Nov 17 01:37:49 2024
--- a/servers/shadow/nginx/21_docker_hrajfrisbee.cz.conf
+++ b/servers/shadow/nginx/21_docker_hrajfrisbee.cz.conf
@@ -0,0 +1,136 @@
+server {
+  server_name
+        api.psmf.hrajfrisbee.cz
+        ;
+  location / {
+          proxy_set_header        Host            $host;
+          proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+          proxy_set_header        X-Forwarded-Proto $scheme;
+    proxy_pass http://192.168.123.21:5001;
+  }
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+    listen 8443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/api.psmf.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/api.psmf.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+}
+
+server {
+    if ($host = api.psmf.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+  listen 80;
+  server_name api.psmf.hrajfrisbee.cz;
+    return 404; # managed by Certbot
+}
+
+# ----------------
+# api-sync.psmf.hrajfrisbee.cz
+server {
+  #  if ($host = api-sync.psmf.hrajfrisbee.cz) {
+  #      return 301 https://$host$request_uri;
+  #  } # managed by Certbot
+
+
+  listen 80;
+  client_max_body_size 15m;                                                                                                                                                              proxy_read_timeout 600;
+  server_name
+        api-sync.psmf.hrajfrisbee.cz
+        ;
+  location / {
+    proxy_pass http://192.168.123.21:8003;
+  }
+}
+server {
+    if ($host = api.psmf.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+  server_name
+        api.psmf.hrajfrisbee.cz
+        ;
+    listen 80;
+    return 404; # managed by Certbot
+
+
+}
+
+# ----------------
+# gitlab & friends
+# ----------------
+
+server {
+
+        server_name
+                gitlab.hrajfrisbee.cz
+        ;
+
+          location / {
+            proxy_pass http://192.168.123.21:80;
+            proxy_set_header        X-Real-IP       $remote_addr;
+            proxy_set_header        Host            $host;
+            proxy_redirect          off;
+            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
+            proxy_connect_timeout 90;
+            proxy_send_timeout 90;
+            proxy_read_timeout 90;
+            client_max_body_size 10m;
+            client_body_buffer_size 128k;
+            proxy_buffer_size 4k;
+            proxy_buffers 4 32k;
+            proxy_busy_buffers_size 64k;
+
+          }
+
+    #listen 443 ssl; # managed by Certbot
+    #ssl_certificate /etc/letsencrypt/live/gitlab.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    #ssl_certificate_key /etc/letsencrypt/live/gitlab.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    #include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    #ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
+
+server {
+        server_name
+                registry.gitlab.hrajfrisbee.cz
+        ;
+
+        location / {
+                proxy_pass       http://192.168.123.21:5005;
+                proxy_set_header Host            $host;
+                proxy_set_header X-Forwarded-For $remote_addr;
+                proxy_set_header X-Forwarded-Proto https;
+                proxy_set_header X-Forwarded-Ssl on;
+
+                client_max_body_size 3000M;
+                client_body_buffer_size 200000k;
+            #auth_basic "Restricted Content";
+            #auth_basic_user_file /etc/nginx/.htpasswd;
+          }
+
+
+    #listen 443 ssl; # managed by Certbot
+    #ssl_certificate /etc/letsencrypt/live/registry.gitlab.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    #ssl_certificate_key /etc/letsencrypt/live/registry.gitlab.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    #include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    #ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+
+    if ($scheme != "https") {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+}
--- a/servers/shadow/nginx/nginx-sites-enabled-default
+++ b/servers/shadow/nginx/nginx-sites-enabled-default
@@ -212,6 +212,88 @@ server {
    return 404; # managed by Certbot
 }

+server {
+    server_name jellyfin.home.hrajfrisbee.cz; # managed by Certbot
+
+
+    # Security headers for media streaming
+    add_header X-Frame-Options "SAMEORIGIN";
+    add_header X-XSS-Protection "1; mode=block";
+    add_header X-Content-Type-Options "nosniff";
+
+    # Increase body size for high-res movie posters
+    client_max_body_size 20M;
+
+    location / {
+        # Proxy to your Synology or VM IP and Jellyfin port (default 8096)
+        proxy_pass https://docker-30:443;
+        
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_set_header X-Forwarded-Protocol $scheme;
+        proxy_set_header X-Forwarded-Host $http_host;
+
+        # Disable buffering for smoother streaming
+        proxy_buffering off;
+    }
+
+    listen 8443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/jellyfin.home.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/jellyfin.home.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+}
+server {
+    if ($host = jellyfin.home.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+
+        listen 80 ;
+    server_name jellyfin.home.hrajfrisbee.cz;
+    return 404; # managed by Certbot
+
+
+}
+
+
+server {
+
+	root /srv/webs/random-shit;
+	server_name random-shit.hrajfrisbee.cz; # managed by Certbot
+	
+	# Enable directory browsing
+	autoindex on;
+
+	# Optional: Show file sizes in MB/GB instead of bytes
+	autoindex_exact_size off;
+
+	# Optional: Show file timestamps in your local server time
+	autoindex_localtime on;
+	
+	# Optional: Choose format (html, xml, json, or jsonp)
+	autoindex_format html;
+
+
+    listen 8443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/random-shit.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/random-shit.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+}
+server {
+    if ($host = random-shit.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+	listen 80 ;
+    server_name random-shit.hrajfrisbee.cz;
+    return 404; # managed by Certbot
+}

 server {

@@ -240,8 +322,8 @@ server {
    ssl_certificate_key /etc/letsencrypt/live/vault.hrajfrisbee.cz/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
-
 }
+
 server {
    if ($host = vault.hrajfrisbee.cz) {
        return 301 https://$host$request_uri;
@@ -250,6 +332,60 @@ server {
 	listen 80 ;
    server_name vault.hrajfrisbee.cz;
    return 404; # managed by Certbot
+}

+server {
+    server_name maru-hleda-byt.home.hrajfrisbee.cz; # managed by Certbot

+	location / {
+		proxy_pass http://docker-30:8080;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+	}
+
+    listen 8443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/maru-hleda-byt.home.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/maru-hleda-byt.home.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+}
+
+server {
+    if ($host = maru-hleda-byt.home.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+	listen 80 ;
+    server_name maru-hleda-byt.home.hrajfrisbee.cz;
+    return 404; # managed by Certbot
+}
+
+server {
+    server_name fuj-management.home.hrajfrisbee.cz; # managed by Certbot
+
+	location / {
+		proxy_pass http://docker-30:8081;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+	}
+
+    listen 8443 ssl; # managed by Certbot
+    ssl_certificate /etc/letsencrypt/live/fuj-management.home.hrajfrisbee.cz/fullchain.pem; # managed by Certbot
+    ssl_certificate_key /etc/letsencrypt/live/fuj-management.home.hrajfrisbee.cz/privkey.pem; # managed by Certbot
+    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+}
+
+server {
+    if ($host = fuj-management.home.hrajfrisbee.cz) {
+        return 301 https://$host$request_uri;
+    } # managed by Certbot
+
+	listen 80 ;
+    server_name fuj-management.home.hrajfrisbee.cz;
+    return 404; # managed by Certbot
 }
--- a/servers/shadow/nginx/nginx.conf
+++ b/servers/shadow/nginx/nginx.conf
@@ -64,7 +64,12 @@ http {
 stream {
    map $ssl_preread_server_name $backend {
        # Passthrough to K8s
+		ghost.lab.home.hrajfrisbee.cz   k8s_gatewayapi;
+		fujarna.lab.home.hrajfrisbee.cz   k8s_gatewayapi;
+		operator-test.lab.home.hrajfrisbee.cz   k8s_gatewayapi;
+
        ~^.+\.lab\.home\.hrajfrisbee\.cz$    k8s_ingress;
+        lab\.home\.hrajfrisbee\.cz$    		 k8s_ingress;

        default                               local_https;
    }
@@ -73,6 +78,10 @@ stream {
        server docker-30:9443;
    }

+    upstream k8s_gatewayapi {
+        server docker-30:9444;
+    }
+
    upstream local_https {
        server 127.0.0.1:8443;  # Loop back to http block
    }
--- a/servers/shadow/storage-23/readme.md
+++ b/servers/shadow/storage-23/readme.md
@@ -0,0 +1,13 @@
+```bash
+# install postgres + create users
+apt install postgresql postgresql-contrib -y
+
+# su - postgres
+# psql
+#   create database psmfapi owner psmf;
+# create users from other database with pg_dumpall --globals-only
+
+# install redis
+apt install redis-server -y
+
+```
--- a/vms/docker/install.md
+++ b/vms/docker/install.md
@@ -0,0 +1,32 @@
+## expected services
+
+- tailscaled
+- node-exporter
+
+```bash
+# tailscale installation
+# add repo and public keys
+curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/$(lsb_release -cs).noarmor.gpg | sudo tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null
+curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/$(lsb_release -cs).tailscale-keyring.list | sudo tee /etc/apt/sources.list.d/tailscale.list
+
+# install
+sudo apt update
+sudo apt install tailscale -y
+
+# enable & start
+systemctl enable tailscaled
+systemctl start tailscaled
+
+# authenticate
+tailscale up
+```
+
+### deploy node-exporter
+
+```bash
+mkdir -p /srv/docker/node-exporter
+cp docker-compose.yaml /srv/docker/node-exporter/
+cp node-exporter.service /etc/systemd/system/
+systemctl daemon-reload
+systemctl enable --now node-exporter
+```
--- a/vms/docker/node-exporter/docker-compose.yaml
+++ b/vms/docker/node-exporter/docker-compose.yaml
@@ -0,0 +1,19 @@
+services:
+  node-exporter:
+    image: prom/node-exporter:latest
+    container_name: node-exporter
+    restart: unless-stopped
+    pid: host
+    network_mode: host
+    volumes:
+      - /proc:/host/proc:ro
+      - /sys:/host/sys:ro
+      - /:/host/root:ro,rslave
+      - /etc/timezone:/etc/timezone:ro
+      - /etc/localtime:/etc/localtime:ro
+    command:
+      - '--path.procfs=/host/proc'
+      - '--path.sysfs=/host/sys'
+      - '--path.rootfs=/host/root'
+      - '--collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|var/lib/docker/.+)($|/)'
+      - '--collector.systemd'
--- a/vms/docker/node-exporter/node-exporter.service
+++ b/vms/docker/node-exporter/node-exporter.service
@@ -0,0 +1,15 @@
+[Unit]
+Description=Node Exporter
+After=docker.service network-online.target
+Requires=docker.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+WorkingDirectory=/srv/docker/node-exporter
+ExecStart=/usr/bin/docker compose up -d --remove-orphans
+ExecStop=/usr/bin/docker compose down
+TimeoutStartSec=120
+
+[Install]
+WantedBy=multi-user.target
--- a/vms/utility-101-shadow/docker/monitoring/alertmanager.yml
+++ b/vms/utility-101-shadow/docker/monitoring/alertmanager.yml
@@ -0,0 +1,68 @@
+global:
+  resolve_timeout: 5m
+
+  # ── Email (SMTP) defaults ──────────────────────────────────────────────────
+  smtp_smarthost: 'smtp.gmail.com:587'
+  smtp_from: 'kacerr.cz+utility-101-shadow@gmail.com'
+  smtp_auth_username: 'kacerr.cz@gmail.com'
+  smtp_auth_password_file: '/run/secrets/smtp_password'
+  smtp_require_tls: true
+
+route:
+  receiver: default
+  group_by: [alertname, instance]
+  group_wait: 30s       # wait before sending the first alert in a group
+  group_interval: 5m    # wait before sending alerts for a group that has been updated
+  repeat_interval: 4h   # resend still-firing alerts after this interval
+
+  routes:
+    # warnings only go to slack + discord, not email
+    - matchers:
+        - severity = warning
+      receiver: non-critical
+
+    # critical alerts go to all channels
+    - matchers:
+        - severity = critical
+      receiver: default
+
+receivers:
+
+  # Sends to all three channels
+  - name: default
+    email_configs:
+      - to: 'kacerr.cz@gmail.com'
+        send_resolved: true
+    slack_configs:
+      - api_url: 'https://hooks.slack.com/services/REPLACE/WITH/YOUR_WEBHOOK'
+        channel: '#alerts'
+        send_resolved: true
+        title: '{{ if eq .Status "firing" }}:red_circle:{{ else }}:large_green_circle:{{ end }} {{ .CommonAnnotations.summary }}'
+        text: '{{ range .Alerts }}{{ .Annotations.description }}{{ "\n" }}{{ end }}'
+    discord_configs:
+      - webhook_url: 'https://discord.com/api/webhooks/REPLACE/WITH_YOUR_WEBHOOK'
+        send_resolved: true
+        title: '{{ if eq .Status "firing" }}🔴{{ else }}🟢{{ end }} {{ .CommonAnnotations.summary }}'
+        message: '{{ range .Alerts }}{{ .Annotations.description }}{{ "\n" }}{{ end }}'
+
+  # Warnings: slack + discord only
+  - name: non-critical
+    slack_configs:
+      - api_url: 'https://hooks.slack.com/services/REPLACE/WITH/YOUR_WEBHOOK'
+        channel: '#alerts'
+        send_resolved: true
+        title: '{{ if eq .Status "firing" }}:warning:{{ else }}:large_green_circle:{{ end }} {{ .CommonAnnotations.summary }}'
+        text: '{{ range .Alerts }}{{ .Annotations.description }}{{ "\n" }}{{ end }}'
+    discord_configs:
+      - webhook_url: 'https://discord.com/api/webhooks/REPLACE/WITH_YOUR_WEBHOOK'
+        send_resolved: true
+        title: '{{ if eq .Status "firing" }}⚠️{{ else }}🟢{{ end }} {{ .CommonAnnotations.summary }}'
+        message: '{{ range .Alerts }}{{ .Annotations.description }}{{ "\n" }}{{ end }}'
+
+inhibit_rules:
+  # Suppress SSLCertExpiringSoon if SSLCertExpired is already firing for the same instance
+  - source_matchers:
+      - alertname = SSLCertExpired
+    target_matchers:
+      - alertname = SSLCertExpiringSoon
+    equal: [instance]
--- a/vms/utility-101-shadow/docker/monitoring/alerts.yml
+++ b/vms/utility-101-shadow/docker/monitoring/alerts.yml
@@ -0,0 +1,146 @@
+groups:
+  - name: node-exporter
+    rules:
+
+      # node_exporter unreachable
+      - alert: HostDown
+        expr: up{job=~"node-exporter.*"} == 0
+        for: 2m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Host down: {{ $labels.instance }}"
+          description: "node_exporter on {{ $labels.instance }} has been unreachable for more than 2 minutes."
+
+      # CPU > 85% for 5m
+      - alert: HighCPULoad
+        expr: avg by(instance) (rate(node_cpu_seconds_total{mode!="idle"}[5m])) > 0.85
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High CPU load: {{ $labels.instance }}"
+          description: "CPU usage on {{ $labels.instance }} is {{ $value | humanizePercentage }} (threshold: 85%)."
+
+      # CPU > 95% for 5m
+      - alert: CriticalCPULoad
+        expr: avg by(instance) (rate(node_cpu_seconds_total{mode!="idle"}[5m])) > 0.95
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Critical CPU load: {{ $labels.instance }}"
+          description: "CPU usage on {{ $labels.instance }} is {{ $value | humanizePercentage }} (threshold: 95%)."
+
+      # RAM used > 90% for 5m
+      - alert: HighMemoryUsage
+        expr: (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) > 0.90
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High memory usage: {{ $labels.instance }}"
+          description: "Memory usage on {{ $labels.instance }} is {{ $value | humanizePercentage }} (threshold: 90%)."
+
+      # RAM used > 95% for 5m
+      - alert: CriticalMemoryUsage
+        expr: (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) > 0.95
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Critical memory usage: {{ $labels.instance }}"
+          description: "Memory usage on {{ $labels.instance }} is {{ $value | humanizePercentage }} (threshold: 95%)."
+
+      # Disk used > 85% for 5m
+      - alert: DiskSpaceLow
+        expr: (1 - node_filesystem_avail_bytes{fstype!~"tmpfs|overlay|squashfs"} / node_filesystem_size_bytes) > 0.85
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "Low disk space: {{ $labels.instance }} {{ $labels.mountpoint }}"
+          description: "Disk usage on {{ $labels.instance }}:{{ $labels.mountpoint }} is {{ $value | humanizePercentage }} (threshold: 85%)."
+
+      # Disk used > 95% for 5m
+      - alert: DiskSpaceCritical
+        expr: (1 - node_filesystem_avail_bytes{fstype!~"tmpfs|overlay|squashfs"} / node_filesystem_size_bytes) > 0.95
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Critical disk space: {{ $labels.instance }} {{ $labels.mountpoint }}"
+          description: "Disk usage on {{ $labels.instance }}:{{ $labels.mountpoint }} is {{ $value | humanizePercentage }} (threshold: 95%)."
+
+      # Disk predicted to fill within 4h
+      - alert: DiskWillFillIn4h
+        expr: predict_linear(node_filesystem_avail_bytes{fstype!~"tmpfs|overlay|squashfs"}[1h], 4 * 3600) < 0
+        for: 30m
+        labels:
+          severity: warning
+        annotations:
+          summary: "Disk filling up: {{ $labels.instance }} {{ $labels.mountpoint }}"
+          description: "Disk on {{ $labels.instance }}:{{ $labels.mountpoint }} is projected to run out of space within 4 hours."
+
+      # iowait > 20% for 10m
+      - alert: HighDiskIOWait
+        expr: avg by(instance) (rate(node_cpu_seconds_total{mode="iowait"}[5m])) > 0.20
+        for: 10m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High disk I/O wait: {{ $labels.instance }}"
+          description: "I/O wait on {{ $labels.instance }} is {{ $value | humanizePercentage }} (threshold: 20%)."
+
+      # OOM kill detected
+      - alert: OOMKillDetected
+        expr: increase(node_vmstat_oom_kill[5m]) > 0
+        for: 0m
+        labels:
+          severity: critical
+        annotations:
+          summary: "OOM kill detected: {{ $labels.instance }}"
+          description: "An OOM kill event was detected on {{ $labels.instance }}."
+
+      # systemd unit in failed state (requires --collector.systemd)
+      - alert: SystemdServiceFailed
+        expr: node_systemd_unit_state{state="failed"} == 1
+        for: 2m
+        labels:
+          severity: warning
+        annotations:
+          summary: "Systemd service failed: {{ $labels.instance }}"
+          description: "Service {{ $labels.name }} on {{ $labels.instance }} has been in failed state for more than 2 minutes."
+
+  - name: blackbox-https
+    rules:
+
+      # Probe returned non-2xx or timed out
+      - alert: ProbeFailed
+        expr: probe_success == 0
+        for: 2m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Probe failed: {{ $labels.instance }}"
+          description: "Endpoint {{ $labels.instance }} is unreachable or returned a non-2xx response for more than 2 minutes."
+
+      # SSL certificate expires within 21 days
+      - alert: SSLCertExpiringSoon
+        expr: (probe_ssl_earliest_cert_expiry - time()) / 86400 < 21
+        for: 1h
+        labels:
+          severity: warning
+        annotations:
+          summary: "SSL cert expiring soon: {{ $labels.instance }}"
+          description: "SSL certificate for {{ $labels.instance }} expires in {{ $value | printf \"%.0f\" }} days."
+
+      # SSL certificate already expired
+      - alert: SSLCertExpired
+        expr: (probe_ssl_earliest_cert_expiry - time()) / 86400 < 0
+        for: 0m
+        labels:
+          severity: critical
+        annotations:
+          summary: "SSL cert expired: {{ $labels.instance }}"
+          description: "SSL certificate for {{ $labels.instance }} has expired."
--- a/vms/utility-101-shadow/docker/monitoring/blackbox.yml
+++ b/vms/utility-101-shadow/docker/monitoring/blackbox.yml
@@ -0,0 +1,113 @@
+modules:
+
+  # ── HTTP ──────────────────────────────────────────────────────────────────
+
+  # Plain HTTP GET, expects 2xx response
+  http_2xx:
+    prober: http
+    timeout: 10s
+    http:
+      method: GET
+      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
+      valid_status_codes: []  # defaults to 2xx
+      follow_redirects: true
+      preferred_ip_protocol: "ip4"
+
+  # HTTP POST with JSON body, expects 2xx response
+  http_post_2xx:
+    prober: http
+    timeout: 10s
+    http:
+      method: POST
+      headers:
+        Content-Type: application/json
+      body: '{}'
+      valid_status_codes: []  # defaults to 2xx
+      follow_redirects: true
+      preferred_ip_protocol: "ip4"
+
+  # HTTP GET for endpoints that must return 401 (e.g. unauthenticated API)
+  http_401_expected:
+    prober: http
+    timeout: 10s
+    http:
+      method: GET
+      valid_status_codes: [401]
+      follow_redirects: false
+      preferred_ip_protocol: "ip4"
+
+  # ── HTTPS ─────────────────────────────────────────────────────────────────
+
+  # HTTPS GET, TLS certificate verified (chain + expiry)
+  # Use this for production endpoints — probe_ssl_earliest_cert_expiry is populated
+  https_2xx:
+    prober: http
+    timeout: 10s
+    http:
+      method: GET
+      valid_status_codes: []  # defaults to 2xx
+      follow_redirects: true
+      preferred_ip_protocol: "ip4"
+      tls_config:
+        insecure_skip_verify: false
+
+  # HTTPS GET, skip TLS certificate verification
+  # Use for self-signed certs or internal CAs not trusted by the exporter
+  https_insecure:
+    prober: http
+    timeout: 10s
+    http:
+      method: GET
+      valid_status_codes: []  # defaults to 2xx
+      follow_redirects: true
+      preferred_ip_protocol: "ip4"
+      tls_config:
+        insecure_skip_verify: true
+
+  # ── TCP ───────────────────────────────────────────────────────────────────
+
+  # Raw TCP connect — checks port is open and accepting connections
+  # Target format: host:port  (e.g. "192.168.0.30:5432")
+  tcp_connect:
+    prober: tcp
+    timeout: 10s
+    tcp:
+      preferred_ip_protocol: "ip4"
+
+  # ── ICMP ──────────────────────────────────────────────────────────────────
+
+  # ICMP ping — checks host reachability and latency
+  # Note: requires NET_RAW capability or running as root
+  icmp:
+    prober: icmp
+    timeout: 10s
+    icmp:
+      preferred_ip_protocol: "ip4"
+
+  # ── DNS ───────────────────────────────────────────────────────────────────
+
+  # DNS lookup via UDP (standard)
+  # Target: resolver IP  (e.g. "8.8.8.8")
+  # Query name and type set in prometheus.yml per target via params
+  dns_udp:
+    prober: dns
+    timeout: 10s
+    dns:
+      transport_protocol: "udp"
+      preferred_ip_protocol: "ip4"
+      query_name: "example.com"
+      query_type: "A"
+      valid_rcodes:
+        - NOERROR
+
+  # DNS lookup via TCP
+  dns_tcp:
+    prober: dns
+    timeout: 10s
+    dns:
+      transport_protocol: "tcp"
+      preferred_ip_protocol: "ip4"
+      query_name: "example.com"
+      query_type: "A"
+      valid_rcodes:
+        - NOERROR
--- a/vms/utility-101-shadow/docker/monitoring/docker-compose.yaml
+++ b/vms/utility-101-shadow/docker/monitoring/docker-compose.yaml
@@ -0,0 +1,57 @@
+version: '3.8'
+
+services:
+  prometheus:
+    image: prom/prometheus:latest
+    container_name: prometheus
+    restart: unless-stopped
+    ports:
+      - "9090:9090"
+    volumes:
+      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
+      - ./alerts.yml:/etc/prometheus/alerts.yml:ro
+      - ./data:/prometheus
+      - /etc/timezone:/etc/timezone:ro
+      - /etc/localtime:/etc/localtime:ro
+    command:
+      - '--config.file=/etc/prometheus/prometheus.yml'
+      - '--storage.tsdb.path=/prometheus'
+      - '--storage.tsdb.retention.time=60d'
+      - '--web.enable-lifecycle'
+    networks:
+      - monitoring-network
+
+  blackbox-exporter:
+    image: prom/blackbox-exporter:latest
+    container_name: blackbox-exporter
+    restart: unless-stopped
+    ports:
+      - "9115:9115"
+    volumes:
+      - ./blackbox.yml:/etc/blackbox_exporter/config.yml:ro
+      - /etc/timezone:/etc/timezone:ro
+      - /etc/localtime:/etc/localtime:ro
+    networks:
+      - monitoring-network
+
+  alertmanager:
+    image: prom/alertmanager:latest
+    container_name: alertmanager
+    restart: unless-stopped
+    ports:
+      - "9093:9093"
+    volumes:
+      - ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
+      - ./alertmanager-data:/alertmanager
+      - ./smtp_password:/run/secrets/smtp_password:ro
+      - /etc/timezone:/etc/timezone:ro
+      - /etc/localtime:/etc/localtime:ro
+    command:
+      - '--config.file=/etc/alertmanager/alertmanager.yml'
+      - '--storage.path=/alertmanager'
+    networks:
+      - monitoring-network
+
+networks:
+  monitoring-network:
+    driver: bridge
--- a/vms/utility-101-shadow/docker/monitoring/install.md
+++ b/vms/utility-101-shadow/docker/monitoring/install.md
@@ -0,0 +1,20 @@
+## extra installation needs
+
+```bash
+mkdir -p /srv/docker/monitoring/data
+chown 65534:65534 /srv/docker/monitoring/data
+
+# google smtp password
+echo -n 'correct google app password' > smtp_password
+chmod 600 smtp_password
+
+chown 65534:65534 /srv/docker/monitoring/smtp_password
+
+# enable systemd unit
+cp /srv/docker/monitoring/monitoring.service /etc/systemd/system/monitoring.service
+systemctl daemon-reload
+systemctl enable --now monitoring
+
+
+
+```
--- a/vms/utility-101-shadow/docker/monitoring/monitoring.service
+++ b/vms/utility-101-shadow/docker/monitoring/monitoring.service
@@ -0,0 +1,15 @@
+[Unit]
+Description=Monitoring Stack (Prometheus + Blackbox Exporter + Alertmanager)
+After=docker.service network-online.target
+Requires=docker.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+WorkingDirectory=/srv/docker/monitoring
+ExecStart=/usr/bin/docker compose up -d --remove-orphans
+ExecStop=/usr/bin/docker compose down
+TimeoutStartSec=300
+
+[Install]
+WantedBy=multi-user.target
--- a/vms/utility-101-shadow/docker/monitoring/prometheus.yml
+++ b/vms/utility-101-shadow/docker/monitoring/prometheus.yml
@@ -0,0 +1,142 @@
+global:
+  scrape_interval: 60s
+  evaluation_interval: 60s
+  scrape_timeout: 15s
+
+alerting:
+  alertmanagers:
+    - static_configs:
+        - targets:
+            - alertmanager:9093
+
+rule_files:
+  - /etc/prometheus/alerts.yml
+
+scrape_configs:
+
+  # ── Prometheus itself ──────────────────────────────────────────────────────
+  - job_name: prometheus
+    static_configs:
+      - targets:
+          - localhost:9090
+
+  # ── Blackbox Exporter own metrics ─────────────────────────────────────────
+  - job_name: blackbox
+    static_configs:
+      - targets:
+          - blackbox-exporter:9115
+
+  # ── Node Exporter (host: docker) ──────────────────────────────────────────
+  - job_name: node-exporter-docker
+    static_configs:
+      - targets:
+          - docker:9100
+        labels:
+          instance: docker
+
+  # ── HTTPS probes (TLS verified) ───────────────────────────────────────────
+  - job_name: blackbox-https
+    metrics_path: /probe
+    params:
+      module: [https_2xx]
+    static_configs:
+      - targets:
+          - https://fuj-management.home.hrajfrisbee.cz/
+          - https://gitea.home.hrajfrisbee.cz/
+          - https://vault.hrajfrisbee.cz/
+          - https://idm.home.hrajfrisbee.cz/
+          - https://maru-hleda-byt.home.hrajfrisbee.cz/mapa_bytu.html
+          # - https://nonexistent.home.hrajfrisbee.cz/
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: blackbox-exporter:9115
+
+  # ── HTTP probes (plain HTTP, no TLS) ──────────────────────────────────────
+  - job_name: blackbox-http
+    metrics_path: /probe
+    params:
+      module: [http_2xx]
+    static_configs:
+      - targets:
+          # - http://192.168.0.30:8080/
+          # - http://some-internal-service:port/healthz
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: blackbox-exporter:9115
+
+  # ── HTTP POST probes ───────────────────────────────────────────────────────
+  - job_name: blackbox-http-post
+    metrics_path: /probe
+    params:
+      module: [http_post_2xx]
+    static_configs:
+      - targets:
+          # - http://some-api/endpoint
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: blackbox-exporter:9115
+
+  # ── TCP port probes ────────────────────────────────────────────────────────
+  - job_name: blackbox-tcp
+    metrics_path: /probe
+    params:
+      module: [tcp_connect]
+    static_configs:
+      - targets:
+          # - 192.168.0.30:5432   # postgres
+          # - 192.168.0.30:6379   # redis
+          # - 192.168.0.30:22     # ssh
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: blackbox-exporter:9115
+
+  # ── ICMP ping probes ───────────────────────────────────────────────────────
+  - job_name: blackbox-icmp
+    metrics_path: /probe
+    params:
+      module: [icmp]
+    static_configs:
+      - targets:
+          # - 192.168.0.30
+          # - 192.168.0.1    # gateway
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: blackbox-exporter:9115
+
+  # ── DNS probes ─────────────────────────────────────────────────────────────
+  - job_name: blackbox-dns
+    metrics_path: /probe
+    params:
+      module: [dns_udp]
+    static_configs:
+      - targets:
+          # - 8.8.8.8           # Google DNS
+          # - 1.1.1.1           # Cloudflare DNS
+          # - 192.168.0.1       # local resolver
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: blackbox-exporter:9115
--- a/vms/utility-101-shadow/named.conf.local
+++ b/vms/utility-101-shadow/named.conf.local
@@ -0,0 +1,54 @@
+//
+// Do any local configuration here
+//
+
+// Consider adding the 1918 zones here, if they are not used in your
+// organization
+//include "/etc/bind/zones.rfc1918";
+
+key "acme-update-key" {
+	algorithm hmac-sha512;
+	secret "T6R1TpLGegHwFWO/I1LwtdGePRD+w00Oe4mJECW7qfheKJ/7FxlINH+Yk2vMvJCVNojj8BWoFAyEFCwGBpGROQ==";
+};
+
+zone "czechultimate.cz" {
+        type master;
+        file "/etc/bind/zones/czechultimate.cz.dns";
+        inline-signing yes;
+        auto-dnssec maintain;
+        key-directory "/etc/bind/keys";
+        allow-transfer {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+        also-notify {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+};
+
+zone "hrajfrisbee.cz" {
+        type master;
+        file "/etc/bind/zones/hrajfrisbee.cz.dns";
+        allow-transfer {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+        also-notify {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+
+        update-policy {
+            // Allow ACME challenges only for lab.home subdomain
+            grant acme-update-key name _acme-challenge.lab.home.hrajfrisbee.cz. TXT;
+
+            // If you need wildcards under lab.home (e.g. _acme-challenge.foo.lab.home.hrajfrisbee.cz):
+            grant acme-update-key subdomain _acme-challenge.lab.home.hrajfrisbee.cz. TXT;
+        };        
+};
+
+// points at zlutazimnice nameservers @nic.cz - cannot be working
+zone "fraktalbar.cz" {
+        type master;
+        file "/etc/bind/zones/fraktalbar.cz.dns";
+        allow-transfer {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+        also-notify {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+};
+
+// points at zlutazimnice nameservers @nic.cz - cannot be working
+zone "vegtral.cz" {
+        type master;
+        file "/etc/bind/zones/vegtral.cz.dns";
+        allow-transfer {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+        also-notify {87.236.197.83; 89.187.144.180; 87.236.196.85; };
+};
+
--- a/vms/utility-101-shadow/readme.md
+++ b/vms/utility-101-shadow/readme.md
@@ -0,0 +1,41 @@
+## Services
+
+- wireguard
+- openvpn server
+- dns server (bind)
+
+## Docker
+
+```bash
+# installation
+apt-get install -y ca-certificates curl
+
+# 3. Add Docker's official GPG key
+sudo install -m 0755 -d /etc/apt/keyrings
+sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
+sudo chmod a+r /etc/apt/keyrings/docker.asc
+
+# 4. Add Docker apt repository
+echo \
+  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
+  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
+  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+
+# 5. Install Docker Engine
+sudo apt-get update
+sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
+
+# 6. Enable and start Docker
+sudo systemctl enable docker
+sudo systemctl start docker
+
+
+```
+
+## named tweaks
+
+1. Generate TSIG key
+
+```bash
+tsig-keygen -a hmac-sha512 acme-update-key
+```
Author	SHA1	Message	Date
uh-cli bot	77ea1dc571	gitops: update fuj-management Co-Authored-By: Claude <noreply@anthropic.com>	2026-06-12 18:01:46 +00:00
Jan Novak	512b3cc2a6	gitops/fuj: split into per-app manifests, add fuj-management deployment Rename generic deployment/service/httproute to per-app suffixes (_fujarna, _fuj-management) and add fuj-management deployment, service, httproute, and bot-credentials secret. Namespace name corrected to fuj. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-06-11 20:59:14 +02:00
Jan Novak	3299373f3d	gitops: rename namespace fujarna to fuj	2026-06-11 13:04:21 +02:00
Jan Novak	80d0cc1168	misc: zot registry, k8s OIDC, server configs, sandbox experiments, and notes - docker-30/zot: add Zot OCI registry with on-demand sync to docker.io, registry.k8s.io, ghcr.io, quay.io - kubernetes-kvm-terraform: wire Kanidm OIDC via structured AuthenticationConfiguration; add reference apiserver manifest and join-node-02 helper - servers: reorganize shadow/ under servers/, add saint vhost config and utility-101 VM definition, add shadow hrajfrisbee.cz vhost and storage-23 notes - experiments: add notes and configs for e2b dev VM, kata + firecracker on kube, microsandbox, orb-stack k3s (terraform + cloud-init), rke2 - vms/docker: document tailscale + node-exporter setup - blog: stub post on Gateway API - chore: gitignore tmp/, smtp_password, and the two local-only credential caches; add per-project .claude/settings.json Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-01 18:12:38 +02:00
Jan Novak	5ca27a832b	gitops: upgrade Cilium to 1.19.1, add fujarna app, flux web UI, OIDC RBAC, and experiments - Upgrade Cilium helm release from 1.18.5 to 1.19.1 with gatewayClass creation enabled - Escalate gitea CI service account to cluster-admin, add OIDC cluster-admin binding - Deploy fujarna app with full manifest set (deployment, service, PVC, httproutes, external secret) - Add Flux web UI via flux-operator OCI repository and helm release - Add experiments kustomization with test resources for gateway API and certificates Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-04-01 22:23:12 +02:00
Jan Novak	96ba77a606	docker-30: gitea CI/CD integration with Vault and Kanidm, misc updates vault: - Add JWT auth backend bound to Gitea (jwks_url from gitea OIDC keys) - Add gitea-ci-read policy scoped to secret/data/gitea/* - Add JWT role gitea-ci (sub claim, bound to Gitea audience, 10m TTL) - Add AppRole gitea-ci as alternative auth method for the same policy - Add gitea-access-into-vault.md documenting the setup end-to-end - Update terraform.tfstate (OpenTofu 1.11.5, new gitea-ci resources) kanidm: - Add run.sh with docker run command (pinned to v1.9.1) - Add gitea-action-kubernetes-access.md documenting how to set up a Kanidm service account and OAuth2 client for Gitea CI k8s access - readme: add upgrade procedure, recover-account command, and service account + API token setup for gitea-ci-token maru-hleda-byt: - Add --restart=always to docker run command fuj-management: - Add run.sh (new service config) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-07 23:09:52 +01:00
Jan Novak	dda6a9d032	vms: add monitoring stack and node-exporter for docker host utility-101-shadow: - Add full monitoring stack (Prometheus + Blackbox Exporter + Alertmanager) with Docker Compose and a systemd unit (monitoring.service) - Prometheus scrapes: itself, blackbox-exporter, and node-exporter on the docker host (docker:9100); blackbox probes cover HTTPS endpoints with TLS cert monitoring - Alertmanager routes warnings to Slack/Discord, critical alerts also to email (Gmail SMTP); inhibit rule suppresses SSLCertExpiringSoon when SSLCertExpired already fires - Alert rules: 11 node-exporter alerts (host down, CPU, memory, disk fill/prediction, iowait, OOM kill, systemd failed units) + 3 blackbox alerts (probe failed, SSL expiring, SSL expired) - readme: add services list and Docker Engine installation steps docker host: - Add node-exporter container running with host pid/network and read-only mounts of /proc, /sys, / for full host metrics visibility - Enable --collector.systemd for systemd unit state metrics - Add systemd unit (node-exporter.service) to manage the container Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-07 23:07:44 +01:00
Jan Novak	1b6015f732	gitops: fix kustomization: 00-rbac	2026-03-01 14:39:36 +01:00
Jan Novak	9877b093e8	gitops: add rbac kustomization + store some forgotten older changes in repo	2026-03-01 14:33:56 +01:00
Jan Novak	0eab64c954	hosting: some config files for host: shadow, some named conf for utility-101-shadow vm	2026-02-20 02:16:16 +01:00
Jan Novak	be362a5ab7	gitops/cilium: configure gateway and wildcard certificate it needs	2026-02-20 02:15:02 +01:00