Files
kacerrandClaude Opus 5 4e5b5d62df Probe hosts for health instead of risking the shared connection
The health check shares each host's libvirt connection with the reconciles,
so it must not be able to take that connection away from them.

- A timed-out check no longer drops the link. The check goes through the new
  hostPool.probe, which abandons its RPCs when the context ends and leaves
  the connection alone, so an image upload or a Create running on the same
  link is not cut off. A link that really died is closed by the keepalive.
  One abandoned probe per host may stay waiting; until it returns, the next
  check reports Timeout at once (errProbeBusy).
- Waiting for a host's connection slot now ends with the caller's context.
  hostEntry used a mutex, so a check could wait out a reconcile redialing a
  dead host — minutes — past its own timeout. The slot is a one-slot channel
  and acquire bounds only the wait, never the uncontended take.
- --provider-check-timeout defaults to 0 and derives min(15s, interval).
  The flat 15s default contradicted the validation that the timeout may not
  exceed the interval, so --provider-check-interval=12s alone failed to
  start.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 05:04:45 +00:00

179 lines
6.6 KiB
YAML

apiVersion: v1
kind: Namespace
metadata:
labels:
control-plane: controller-manager
app.kubernetes.io/name: egress-proxies-operator
app.kubernetes.io/managed-by: kustomize
name: system
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
labels:
control-plane: controller-manager
app.kubernetes.io/name: egress-proxies-operator
app.kubernetes.io/managed-by: kustomize
spec:
selector:
matchLabels:
control-plane: controller-manager
app.kubernetes.io/name: egress-proxies-operator
replicas: 1
template:
metadata:
annotations:
kubectl.kubernetes.io/default-container: manager
labels:
control-plane: controller-manager
app.kubernetes.io/name: egress-proxies-operator
spec:
# TODO(user): Uncomment the following code to configure the nodeAffinity expression
# according to the platforms which are supported by your solution.
# It is considered best practice to support multiple architectures. You can
# build your manager image using the makefile target docker-buildx.
# affinity:
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/arch
# operator: In
# values:
# - amd64
# - arm64
# - ppc64le
# - s390x
# - key: kubernetes.io/os
# operator: In
# values:
# - linux
securityContext:
# Projects are configured by default to adhere to the "restricted" Pod Security Standards.
# This ensures that deployments meet the highest security requirements for Kubernetes.
# For more details, see: https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- command:
- /manager
args:
- --leader-elect
- --health-probe-bind-address=:8081
- --providers-config=/etc/proxy-operator/providers.yaml
# Add --trace-health-probes to emit one span per health probe
# (high volume: ~1/s per proxy).
# Provider health checks (kubectl get providerhealth) run every
# minute; tune with --provider-check-interval (0 turns them off,
# otherwise at least 10s) and --provider-check-timeout (15s, or
# the interval if that is shorter).
env:
# Bearer token for the discovery API. Optional: without the
# Secret the API serves unauthenticated (with a loud warning).
# Create it with:
# kubectl -n egress-proxies-operator-system create secret \
# generic discovery-token --from-literal=token=<your-token>
- name: DISCOVERY_TOKEN
valueFrom:
secretKeyRef:
name: discovery-token
key: token
optional: true
# --- OpenTelemetry tracing ---
# Tracing is OFF until OTEL_EXPORTER_OTLP_ENDPOINT (or
# OTEL_TRACES_EXPORTER) is set; without it the operator runs
# exactly as before. The downward-API vars must stay listed
# before OTEL_RESOURCE_ATTRIBUTES: $(VAR) expansion only sees
# earlier-listed vars.
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OTEL_SERVICE_NAME
value: egress-proxies-operator
- name: OTEL_RESOURCE_ATTRIBUTES
value: k8s.pod.name=$(POD_NAME),k8s.namespace.name=$(POD_NAMESPACE)
# Point at an OTLP receiver (Tempo, Jaeger, otel-collector) to
# enable tracing; http/protobuf (4318) is the default protocol,
# set OTEL_EXPORTER_OTLP_PROTOCOL=grpc for 4317.
# - name: OTEL_EXPORTER_OTLP_ENDPOINT
# value: http://jaeger-collector.observability:4318
# - name: OTEL_TRACES_SAMPLER
# value: parentbased_traceidratio
# - name: OTEL_TRACES_SAMPLER_ARG
# value: "0.1"
image: controller:latest
name: manager
ports:
- containerPort: 8081
name: health
protocol: TCP
- containerPort: 8090
name: discovery
protocol: TCP
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- "ALL"
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: 8081
initialDelaySeconds: 5
periodSeconds: 10
# TODO(user): Configure the resources accordingly based on the project requirements.
# More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
resources:
limits:
cpu: 500m
memory: 128Mi
requests:
cpu: 10m
memory: 64Mi
volumeMounts:
- name: providers-config
mountPath: /etc/proxy-operator
readOnly: true
# SSH material for kvm providers (sshKeyFile / knownHostsFile in the
# providers config point here). Optional: the mount is empty when the
# Secret does not exist, and only kvm entries need it. Create it with:
# kubectl -n egress-proxies-operator-system create secret generic kvm-ssh \
# --from-file=id_ed25519=<private key> \
# --from-file=known_hosts=<known_hosts with every host's key>
- name: kvm-ssh
mountPath: /etc/proxy-operator/kvm
readOnly: true
# Scratch space: the root filesystem is read-only, and the kvm
# provider builds each VM's cloud-init seed ISO through a temp
# directory (a few hundred KB, deleted right after upload).
- name: tmp
mountPath: /tmp
volumes:
- name: providers-config
configMap:
name: providers-config
- name: kvm-ssh
secret:
secretName: kvm-ssh
optional: true
- name: tmp
emptyDir:
sizeLimit: 64Mi
serviceAccountName: controller-manager
terminationGracePeriodSeconds: 10