egress-proxies-operator/docs/gcp-vm-validation.md

# Validating real VM creation on GCP

Recipe for wiring the GCP provider into a live cluster and watching a
`Proxy` CR create a real Compute Engine VM. Angle brackets mark values you
supply: `<PROJECT_ID>`, `<SA_KEY_PATH>`, `<ZONE>`, `<CLUSTER_EGRESS_IP>`,
`<REGISTRY_IMAGE>`.

The one important fact up front: **the operator takes no GCP credentials
through its own config.** The client is built with Application Default
Credentials (`internal/provider/gcp/gcp.go`, `New()`); there is no
key-file field in the providers config. The only secret to prepare is a
service-account JSON key, injected via the standard
`GOOGLE_APPLICATION_CREDENTIALS` mechanism. On GKE you would use workload
identity instead and skip the key entirely.

## 1. GCP-side prerequisites (prepared outside the cluster)

1. A project — `<PROJECT_ID>` — with the **Compute Engine API enabled**.
2. A **service account** with `roles/compute.instanceAdmin.v1` on the
   project. The operator only calls instances
   `Insert`/`Get`/`Delete`/`AggregatedList` and does not attach a service
   account to the VMs it creates, so no `iam.serviceAccountUser` is
   needed.
3. A **JSON key** for that service account, saved at `<SA_KEY_PATH>`.
4. A **firewall rule**: created VMs get network tag `proxy-operator` (the
   default; configurable as `gcp.networkTag`), an ephemeral external IP,
   and Squid listening on 3128.

   ```sh
   gcloud compute firewall-rules create allow-proxy-operator \
     --project <PROJECT_ID> \
     --network default \
     --allow tcp:3128 \
     --target-tags proxy-operator \
     --source-ranges <CLUSTER_EGRESS_IP>/32
   ```

   The source range must cover the cluster's egress IP — the operator's
   CONNECT health probes originate there, and without the rule the Proxy
   hangs at `Running`/unhealthy instead of reaching `Ready`. ⚠️ The
   sample cloud-init configures `http_access allow all`, so on a public
   IP this is an open proxy — keep the source ranges tight.

## 2. Create the credentials Secret

Namespace is `egress-proxies-operator-system` after kustomize prefixing:

```sh
kubectl -n egress-proxies-operator-system create secret generic gcp-credentials \
  --from-file=key.json=<SA_KEY_PATH>
```

## 3. Add a GCP entry to the providers ConfigMap

Edit `config/manager/providers_config.yaml` (mounted at
`/etc/proxy-operator/providers.yaml`):

```yaml
providers:
  - name: kubernetes
    type: kubernetes
  - name: gcp-eu          # spec.provider on a Proxy refers to this NAME, not the type
    type: gcp
    gcp:
      project: <PROJECT_ID>
      # network: default            # these three default as shown
      # networkTag: proxy-operator
      # diskSizeGb: 10
```

The config is validated fail-fast at startup — a typo shows up
immediately in the manager log, not on first use.

## 4. Mount the Secret and point ADC at it

In `config/manager/manager.yaml`, add to the manager container:

```yaml
env:
  - name: GOOGLE_APPLICATION_CREDENTIALS
    value: /var/secrets/gcp/key.json
volumeMounts:
  - name: gcp-credentials
    mountPath: /var/secrets/gcp
    readOnly: true
volumes:
  - name: gcp-credentials
    secret:
      secretName: gcp-credentials
```

(`volumeMounts` merges into the existing container list; `volumes` into
the existing pod-level list.)

## 5. Deploy and create the Proxy

```sh
make deploy IMG=<REGISTRY_IMAGE>
```

`config/samples/proxy_gcp.yaml` is usable as-is once `spec.provider`
matches the name from step 3. All three placement fields are mandatory
for GCP — a missing one sets the Proxy to `Failed` with a message naming
it:

```yaml
spec:
  mode: Managed
  provider: gcp-eu
  placement:
    zone: <ZONE>                     # e.g. europe-west1-b
    machineType: e2-micro
    image: projects/debian-cloud/global/images/family/debian-12
```

```sh
kubectl apply -f config/samples/proxy_gcp.yaml
```

## 6. What you should see

```sh
kubectl get proxy -w
```

`Provisioning` → `Running` (VM's external IP published in status) →
`Ready` (CONNECT health probe succeeded through the public IP). Then:

```sh
# the VM exists and carries the GC labels
gcloud compute instances list --project <PROJECT_ID> \
  --filter 'labels.proxy-operator-managed=yes'

# the proxy actually tunnels — should print the VM's external IP
curl -x http://<EXTERNAL_IP>:3128 https://ifconfig.me
```

Cleanup — the finalizer deletes the VM:

```sh
kubectl delete proxy proxy-gcp-sample
gcloud compute instances list --project <PROJECT_ID>   # should be empty again
```

## Gotchas

- **The orphan GC sweeps the whole project**: any VM labeled
  `proxy-operator-managed=yes` whose UID does not match a live Proxy CR
  in *this* cluster is deleted once past the age threshold. Do not point
  two operator installs at the same project, and do not hand-create VMs
  with that label.
- **VM creation is fire-and-forget** — the provider never waits on the
  insert operation; progress is discovered by polling `Get`. A quota
  error or bad image name surfaces on the Proxy's status/conditions a
  reconcile later, not synchronously. `kubectl describe proxy` is the
  place to look when something stalls.
- **e2-micro costs pennies but is not free everywhere** — remember to
  delete the CR (or check `gcloud compute instances list`) when done.