egress-proxies-operator/docs/gcp-vm-validation.md

Validating real VM creation on GCP

Recipe for wiring the GCP provider into a live cluster and watching a Proxy CR create a real Compute Engine VM. Angle brackets mark values you supply: <PROJECT_ID>, <SA_KEY_PATH>, <ZONE>, <CLUSTER_EGRESS_IP>, <REGISTRY_IMAGE>.

The one important fact up front: the operator takes no GCP credentials through its own config. The client is built with Application Default Credentials (internal/provider/gcp/gcp.go, New()); there is no key-file field in the providers config. The only secret to prepare is a service-account JSON key, injected via the standard GOOGLE_APPLICATION_CREDENTIALS mechanism. On GKE you would use workload identity instead and skip the key entirely.

1. GCP-side prerequisites (prepared outside the cluster)

1. A project — <PROJECT_ID> — with the Compute Engine API enabled.

2. A service account with roles/compute.instanceAdmin.v1 on the project. The operator only calls instances Insert/Get/Delete/AggregatedList and does not attach a service account to the VMs it creates, so no iam.serviceAccountUser is needed.

3. A JSON key for that service account, saved at <SA_KEY_PATH>.

4. A firewall rule: created VMs get network tag proxy-operator (the default; configurable as gcp.networkTag), an ephemeral external IP, and Squid listening on 3128.

   gcloud compute firewall-rules create allow-proxy-operator \
     --project <PROJECT_ID> \
     --network default \
     --allow tcp:3128 \
     --target-tags proxy-operator \
     --source-ranges <CLUSTER_EGRESS_IP>/32
   

   The source range must cover the cluster's egress IP — the operator's CONNECT health probes originate there, and without the rule the Proxy hangs at Running/unhealthy instead of reaching Ready. ⚠️ The sample cloud-init configures http_access allow all, so on a public IP this is an open proxy — keep the source ranges tight.

2. Create the credentials Secret

Namespace is egress-proxies-operator-system after kustomize prefixing:

kubectl -n egress-proxies-operator-system create secret generic gcp-credentials \
  --from-file=key.json=<SA_KEY_PATH>

3. Add a GCP entry to the providers ConfigMap

Edit config/manager/providers_config.yaml (mounted at /etc/proxy-operator/providers.yaml):

providers:
  - name: kubernetes
    type: kubernetes
  - name: gcp-eu          # spec.provider on a Proxy refers to this NAME, not the type
    type: gcp
    gcp:
      project: <PROJECT_ID>
      # network: default            # these three default as shown
      # networkTag: proxy-operator
      # diskSizeGb: 10

The config is validated fail-fast at startup — a typo shows up immediately in the manager log, not on first use.

4. Mount the Secret and point ADC at it

In config/manager/manager.yaml, add to the manager container:

env:
  - name: GOOGLE_APPLICATION_CREDENTIALS
    value: /var/secrets/gcp/key.json
volumeMounts:
  - name: gcp-credentials
    mountPath: /var/secrets/gcp
    readOnly: true
volumes:
  - name: gcp-credentials
    secret:
      secretName: gcp-credentials

(volumeMounts merges into the existing container list; volumes into the existing pod-level list.)

5. Deploy and create the Proxy

make deploy IMG=<REGISTRY_IMAGE>

config/samples/proxy_gcp.yaml is usable as-is once spec.provider matches the name from step 3. All three placement fields are mandatory for GCP — a missing one sets the Proxy to Failed with a message naming it:

spec:
  mode: Managed
  provider: gcp-eu
  placement:
    zone: <ZONE>                     # e.g. europe-west1-b
    machineType: e2-micro
    image: projects/debian-cloud/global/images/family/debian-12

kubectl apply -f config/samples/proxy_gcp.yaml

6. What you should see

kubectl get proxy -w

Provisioning → Running (VM's external IP published in status) → Ready (CONNECT health probe succeeded through the public IP). Then:

# the VM exists and carries the GC labels
gcloud compute instances list --project <PROJECT_ID> \
  --filter 'labels.proxy-operator-managed=yes'

# the proxy actually tunnels — should print the VM's external IP
curl -x http://<EXTERNAL_IP>:3128 https://ifconfig.me

Cleanup — the finalizer deletes the VM:

kubectl delete proxy proxy-gcp-sample
gcloud compute instances list --project <PROJECT_ID>   # should be empty again

Gotchas

• The orphan GC sweeps the whole project: any VM labeled proxy-operator-managed=yes whose UID does not match a live Proxy CR in this cluster is deleted once past the age threshold. Do not point two operator installs at the same project, and do not hand-create VMs with that label.
• VM creation is fire-and-forget — the provider never waits on the insert operation; progress is discovered by polling Get. A quota error or bad image name surfaces on the Proxy's status/conditions a reconcile later, not synchronously. kubectl describe proxy is the place to look when something stalls.
• e2-micro costs pennies but is not free everywhere — remember to delete the CR (or check gcloud compute instances list) when done.