egress-proxies-operator/docs/api.md

# Discovery API reference

The operator serves an HTTP API (the *discovery API*) that crawler clients
use to find and lease egress proxies: list healthy proxies filtered by
attributes, acquire a TTL-based lease on one, release it early, and report
how a target site treated the proxy. It is implemented in
[`internal/discovery`](../internal/discovery/) with lease state in
[`internal/lease`](../internal/lease/); the only Kubernetes interaction is
reading `Proxy` resources from the manager's cache.

## Base URL

The API listens on `:8090` (`--discovery-addr`) inside the manager pod and
is exposed by a Service
([config/default/discovery_service.yaml](../config/default/discovery_service.yaml)).

In-cluster:

```text
http://egress-proxies-operator-controller-manager-discovery-service.egress-proxies-operator-system.svc.cluster.local:8090
```

From a workstation, port-forward:

```sh
kubectl -n egress-proxies-operator-system port-forward \
  svc/egress-proxies-operator-controller-manager-discovery-service 8090:8090 &
```

Set `BASE_URL` to wherever you reach the API; all examples below use it:

```sh
export BASE_URL=localhost:8090   # via the port-forward above
# or, from inside the cluster:
# export BASE_URL=http://egress-proxies-operator-controller-manager-discovery-service.egress-proxies-operator-system.svc.cluster.local:8090
```

## Authentication

A single static bearer token, read from the `DISCOVERY_TOKEN` environment
variable at startup. The shipped Deployment populates it from the
`discovery-token` Secret (key `token`), which is **optional** — if the
Secret is absent or the token is empty, the API serves **unauthenticated**
(the manager logs a loud warning at startup). Create the Secret:

```sh
kubectl -n egress-proxies-operator-system create secret generic discovery-token \
  --from-literal=token="$(openssl rand -hex 24)"
```

Send the token on every request:

```sh
export TOKEN=<the token>
curl -s -H "Authorization: Bearer $TOKEN" "$BASE_URL/v1/proxies" | jq
```

A missing or wrong token gets `401 {"error":"unauthorized",...}`.
`GET /healthz` is always exempt.

The curl examples below omit the `-H "Authorization: Bearer $TOKEN"` flag
for brevity — add it to every call when auth is enabled.

## Conventions

- Requests and responses are JSON. Errors share one envelope:

  ```json
  {"error": "<machine_code>", "message": "<human-readable text>"}
  ```

- Request bodies are capped at **64 KiB** (larger bodies fail the JSON
  decode with `400 invalid_body`).
- Proxies with a deletion timestamp (being finalized) are excluded from
  every response and never offered for lease.

## Configuration

| Setting | Default | Meaning |
|---|---|---|
| `--discovery-addr` | `:8090` | Listen address of the API |
| `--max-lease-ttl` | `1h` | Maximum `ttlSeconds` a client may request |
| `--lease-cooldown` | `15m` | Cooldown window applied on `rate_limited`/`banned` reports |
| `DISCOVERY_TOKEN` (env) | empty | Bearer token; empty disables auth |

The shipped Deployment passes none of these flags, so the defaults apply.

## Endpoints

### `GET /healthz`

Liveness check. Unauthenticated, always `200` with body `ok`.

```sh
curl -s "$BASE_URL/healthz"
```

### `GET /v1/proxies` — list proxies

Query parameters (all optional):

| Parameter | Values | Effect |
|---|---|---|
| `healthy` | `true` \| `false` | Keep only proxies whose `Healthy` condition matches. Any other value → `400 invalid_query`. |
| `attr.<key>` | any string | Exact match on `spec.attributes[<key>]`. Repeatable; **all** given pairs must match. |

List everything:

```sh
curl -s "$BASE_URL/v1/proxies" | jq
```

List healthy proxies in a given geo:

```sh
curl -s "$BASE_URL/v1/proxies?healthy=true&attr.geo=eu" | jq
```

Response — `200`, proxies sorted by `id`, an empty match is `200` with
`"count": 0` (never `404`):

```json
{
  "proxies": [
    {
      "id": "default/proxy-kubernetes-sample",
      "ip": "10.244.1.7",
      "port": 3128,
      "attributes": {"geo": "local"},
      "phase": "Ready",
      "healthy": true,
      "latencyMillis": 42,
      "activeLeases": 1,
      "maxLeases": 5
    }
  ],
  "count": 1
}
```

Proxy object fields (the same shape appears inside lease responses):

| Field | Meaning |
|---|---|
| `id` | `namespace/name` of the `Proxy` resource; used as the stable key everywhere |
| `ip` | Effective host — `spec.endpoint.host` for `External` proxies, `status.ip` for `Managed` (empty until the backing VM/pod is up) |
| `port` | Effective port (default `3128`) |
| `attributes` | `spec.attributes` — free-form selection labels (`geo`, `asn`, `purpose`, …); omitted when empty |
| `phase` | `Pending` \| `Provisioning` \| `Ready` \| `Unhealthy` \| `Deleting` \| `Failed` |
| `healthy` | `true` iff the `Healthy` condition is `True` (the through-the-proxy health probe passes) |
| `latencyMillis` | Latency of the last status-affecting health probe |
| `activeLeases` | Currently active leases on this proxy |
| `maxLeases` | Lease capacity (default `5`; an explicit `0` means unleasable) |

### `POST /v1/leases` — acquire a lease

Picks a healthy proxy with free capacity matching the selector and grants
an exclusive-slot, TTL-based lease on it.

Request body (every field optional; `{}` is valid):

```json
{
  "selector": {"geo": "eu"},
  "ttlSeconds": 300,
  "target": "example.com"
}
```

| Field | Default | Meaning |
|---|---|---|
| `selector` | none | Attribute equality filter, same semantics as `attr.<key>` above |
| `ttlSeconds` | `300` (5 min) | Lease lifetime; must be ≤ `--max-lease-ttl` (default 1 h), else `400 invalid_ttl` |
| `target` | none | The site you intend to crawl; enables per-target cooldowns (see below) |

```sh
curl -s -XPOST "$BASE_URL/v1/leases" \
  -d '{"selector":{"geo":"eu"},"ttlSeconds":300,"target":"example.com"}' | jq
```

Success — `201`:

```json
{
  "leaseID": "P3X6HHQTPCM5UTGVGE3B5UPS3A",
  "proxy": {
    "id": "default/proxy-eu-1",
    "ip": "34.88.10.20",
    "port": 3128,
    "attributes": {"geo": "eu"},
    "phase": "Ready",
    "healthy": true,
    "latencyMillis": 42,
    "activeLeases": 1,
    "maxLeases": 5
  },
  "expiresAt": "2026-08-11T22:05:00Z",
  "ttlSeconds": 300
}
```

Use `proxy.ip` and `proxy.port` as an HTTP proxy for the lease's lifetime:

```sh
curl -x http://34.88.10.20:3128 https://example.com
```

No match — `409` with diagnostic counts explaining why nothing qualified:

```json
{
  "error": "no_match",
  "message": "no healthy proxy with free capacity matched the selector",
  "considered": 3,
  "atCapacity": 1,
  "inCooldown": 1,
  "unhealthy": 1
}
```

| Count | Meaning |
|---|---|
| `considered` | Proxies that matched the selector (before health/capacity checks) |
| `atCapacity` | Skipped because `activeLeases >= maxLeases` |
| `inCooldown` | Skipped because of an active cooldown for this target (or a global one) |
| `unhealthy` | Skipped because the `Healthy` condition is not `True` |

Leases expire on their own — releasing is only needed to free the slot
early. There is no renew/extend endpoint; acquire a new lease instead.

### `DELETE /v1/leases/{id}` — release early

Frees the lease's capacity slot immediately. Idempotent: always `204`,
including for unknown or already-expired lease IDs.

```sh
curl -si -XDELETE "$BASE_URL/v1/leases/P3X6HHQTPCM5UTGVGE3B5UPS3A"
```

### `POST /v1/leases/{id}/report` — report an outcome

Tell the operator how the target site treated the proxy. This is the
feedback signal that drives cooldowns.

Request body:

```json
{"result": "rate_limited", "target": "example.com"}
```

| Field | Values | Meaning |
|---|---|---|
| `result` | `ok` \| `rate_limited` \| `banned` | Anything else → `400 invalid_result` |
| `target` | optional | Which site produced the result; falls back to the lease's `target`, then to global |

```sh
curl -si -XPOST "$BASE_URL/v1/leases/P3X6HHQTPCM5UTGVGE3B5UPS3A/report" \
  -d '{"result":"rate_limited","target":"example.com"}'
```

Responses: `204` on success, `404 unknown_lease` if the lease ID was never
issued or has aged out.

Semantics:

- `ok` is a pure acknowledgement — nothing is recorded.
- `rate_limited` and `banned` currently behave **identically**: both put
  the proxy in one cooldown window (default 15 min, `--lease-cooldown`)
  for the resolved target.
- An expired lease remains reportable for one cooldown window past its
  TTL, so a late "we got rate-limited" still lands.

## Proxy selection and cooldowns

How `POST /v1/leases` picks among eligible proxies (healthy, matching the
selector, not being deleted, not at capacity, not in cooldown), in order:

1. fewest `activeLeases` (least-loaded),
2. lowest `latencyMillis`,
3. lexicographic `id` (deterministic tie-break).

Cooldowns are keyed by **(proxy, target)**:

- A report **with a target** blocks that proxy only for lease requests
  naming the **same target**. Other targets — and requests with no
  target — still get the proxy.
- A report **without a target**, on a lease that also had no target,
  creates a **global** cooldown: the proxy is blocked for *all* lease
  requests until the window passes. Always pass `target` on leases and
  reports unless you really mean "this proxy is bad for everyone".

## End-to-end example

```sh
# 1. Acquire a lease for crawling example.com through an EU proxy
LEASE=$(curl -s -XPOST "$BASE_URL/v1/leases" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"selector":{"geo":"eu"},"ttlSeconds":600,"target":"example.com"}')
LEASE_ID=$(echo "$LEASE" | jq -r .leaseID)
PROXY=$(echo "$LEASE" | jq -r '"\(.proxy.ip):\(.proxy.port)"')

# 2. Crawl through the leased proxy
curl -x "http://$PROXY" https://example.com/some/page

# 3. Got a 429? Report it — example.com-bound leases will avoid this
#    proxy for the next 15 minutes
curl -s -XPOST "$BASE_URL/v1/leases/$LEASE_ID/report" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"result":"rate_limited","target":"example.com"}'

# 4. Done early? Release the slot (otherwise the TTL frees it)
curl -s -XDELETE "$BASE_URL/v1/leases/$LEASE_ID" \
  -H "Authorization: Bearer $TOKEN"
```

## Caveats

- **Lease and cooldown state is in-memory and per-process.** An operator
  restart drops all active leases and cooldowns. Clients must tolerate a
  granted lease disappearing (a subsequent report returns `404`).
- **Run a single replica.** The API is served by every manager replica but
  is not leader-elected, and lease state is not shared between replicas;
  the shipped Deployment pins `replicas: 1`.