# A pool of Managed crawl proxies on the kvm2 provider — the multi-proxy # counterpart to the single proxy_kvm2.yaml sample. Everything here boots on # the one host the kvm2 entry configures (beelink); the proxies differ in the # attributes crawlers select on, in size, and in image. # # Requires the same prerequisites as proxy_kvm2.yaml: a providers-config # entry named "kvm2-home" and the kvm-ssh Secret that the kvm and kvm2 # providers share (docs/kvm2-provider.md). # # Deliberately NOT in kustomization.yaml: `kubectl apply -k config/samples` # would boot every VM in here at once. Apply it on its own when you want the # pool: # # kubectl apply -f config/samples/proxy_kvm2_crawl_pool.yaml # # Crawlers reach these through the discovery API by attribute, e.g. # `GET /v1/proxies?attr.purpose=crawl&healthy=true`, or by leasing one with # the same selector. Attribute matching is exact equality, so keep the keys # consistent across the pool and vary only the values you want to select on. # # The first Proxy for an image on the host waits for that image to download; # the rest reuse the cached copy. debian-13 and ubuntu-24.04 are separate # downloads, so the ubuntu member below is the slow one on a cold host. --- # One cloud-init Secret shared by the whole pool: the proxies below point at # it with secretRef instead of each carrying its own copy of the squid # config. The crawl.example.com/cloud-init label is mandatory — the manager's # cache only holds Secrets that carry it, so an unlabelled Secret is # invisible to the operator and the Proxy never resolves. # # Editing this Secret changes the resolved cloud-init of every Proxy that # references it, and that replaces each of their VMs. That is the trade for # sharing one definition: convenient for a config rollout, disruptive if you # only meant to touch one member. Give a member its own Secret (or an inline # block) when it needs to diverge. apiVersion: v1 kind: Secret metadata: name: crawl-squid-cloud-init labels: crawl.example.com/cloud-init: "true" type: Opaque stringData: # The default key name; SecretKeySelector.key defaults to user-data, so # the proxies below omit it. user-data: | #cloud-config packages: - squid write_files: - path: /etc/squid/conf.d/proxy-operator.conf content: | http_port 3128 http_access allow all via off forwarded_for off runcmd: - systemctl restart squid # Optional: log in with `ssh debian@` (or `ubuntu@` on the # ubuntu-24.04 member) to look around; `virsh console ` on the # host works too. # ssh_authorized_keys: # - ssh-ed25519 AAAA... you@laptop --- apiVersion: crawl.example.com/v1alpha1 kind: Proxy metadata: name: crawl-home-1 spec: mode: Managed provider: kvm2-home placement: zone: beelink machineType: small image: debian-13 port: 3128 cloudInit: secretRef: name: crawl-squid-cloud-init attributes: geo: home purpose: crawl tier: general # --- # apiVersion: crawl.example.com/v1alpha1 # kind: Proxy # metadata: # name: crawl-home-2 # spec: # mode: Managed # provider: kvm2-home # placement: # zone: beelink # machineType: small # image: debian-13 # port: 3128 # cloudInit: # secretRef: # name: crawl-squid-cloud-init # attributes: # geo: home # purpose: crawl # tier: general