Add the discovery HTTP API: list, lease, release, report over the manager cache

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-09 15:26:11 +02:00
parent f6d50e4744
commit 4aa3d47e3c
6 changed files with 950 additions and 8 deletions

View File

@@ -1,12 +1,10 @@
# Architecture
> **Status:** the operator is built through Step 6 (lease store) of
> **Status:** the operator is built through Step 7 (discovery API) of
> [docs/plans/2026-08-07-1747-proxy-operator.md](plans/2026-08-07-1747-proxy-operator.md).
> This document currently covers the event/reconcile flow; the components
> table and the Decisions section arrive with Step 10. The lease store
> (`internal/lease/`) is HTTP-driven, not cluster-event-driven, so its
> diagram lands together with the discovery API in Step 7; the orphan-GC
> flow lands with Step 9.
> This document currently covers the event/reconcile flow and the
> HTTP-driven lease/discovery path; the components table and the Decisions
> section arrive with Step 10, and the orphan-GC flow lands with Step 9.
## Event flow: cluster events → reconciler functions
@@ -185,3 +183,48 @@ Consequence worth knowing: `status.lastHealthCheckTime` is the time of the
last *status-affecting* probe, not the most recent probe — suppressed
probes deliberately never write status. True probe recency will live in
metrics (Step 9).
### 7. Discovery + lease API (`internal/discovery/`, `internal/lease/`)
HTTP-driven, not cluster-event-driven: crawler clients call in; the only
Kubernetes interaction is reading Proxies from the manager's cache. The
server is a non-leader-elected Runnable (all replicas would serve, but the
deployment ships `replicas: 1` because lease state is per-process — an
operator restart drops all leases and cooldowns, a documented caveat).
```text
crawler client
│ Authorization: Bearer $DISCOVERY_TOKEN (empty token = auth disabled, loud startup warning)
Server.handler() middleware, outermost first (server.go)
recover → request-log → MaxBytesReader(64KiB) → bearer auth (constant-time; /healthz exempt)
├─ GET /healthz ──► 200 ok (unauthenticated)
├─ GET /v1/proxies?attr.k=v&healthy=true (handlers.go)
│ Reader.List(Proxies) ── manager cache
│ filter: attributes equality + Healthy condition
│ + Store.Counts() for activeLeases
│ ──► 200 {"proxies":[...], "count":N} (empty list is 200, not 404)
├─ POST /v1/leases {"selector":{...},"ttlSeconds":300,"target":"..."}
│ Reader.List → filter selector; unhealthy matches counted, not offered
│ Store.Acquire(healthy candidates, target, ttl) ── one lock: select+insert
│ │ selection: fewest active leases, then latency, then name
│ ├─ granted ──► 201 {leaseID, proxy:{...}, expiresAt, ttlSeconds}
│ └─ ErrNoMatch ──► 409 {"error":"no_match", considered, atCapacity,
│ inCooldown, unhealthy}
├─ DELETE /v1/leases/{id} ──► Store.Release ──► always 204 (idempotent)
└─ POST /v1/leases/{id}/report {"result":"ok|rate_limited|banned","target":"..."}
Store.Report ── rate_limited/banned ⇒ cooldown[{proxy,target}] for
│ CooldownWindow (target falls back: report → lease → global)
├─ 204 │ 400 invalid_result │ 404 unknown_lease
└─ an expired lease still resolves for CooldownWindow past its TTL —
a late report lands exactly when the proxy is being rate-limited
Store.Start(ctx) ── manager Runnable, NOT leader-elected: sweeps expired
leases + cooldowns; correctness never depends on the
sweep (every read checks ExpiresAt against the clock)
```

View File

@@ -11,7 +11,7 @@ Pairs with [docs/plans/2026-08-07-1747-proxy-operator.md](../plans/2026-08-07-17
- [x] Step 4 — Reconciler (`internal/controller/`)
- [x] Step 5 — Health engine (`internal/health/`)
- [x] Step 6 — Lease store (`internal/lease/`)
- [ ] Step 7 — Discovery API (`internal/discovery/`)
- [x] Step 7 — Discovery API (`internal/discovery/`)
- [ ] Step 8 — GCP provider (`internal/provider/gcp/`)
- [ ] Step 9 — Orphan GC + metrics
- [ ] Step 10 — Wiring, config, docs
@@ -758,3 +758,56 @@ Worth noting: `docs/architecture.md` was not extended this step — the
lease store is HTTP-driven, not cluster-event-driven, so its diagram
belongs with the discovery API and lands in Step 7 (banner updated to say
so).
## Step 7 — Discovery API (`internal/discovery/`)
Implemented `server.go` (Runnable + middleware chain) and `handlers.go`
(the four endpoints + `proxyView` wire shape) per the plan: stdlib
`http.ServeMux` method+wildcard routing (no third-party router — see the
plan clarification commit: this is a stdlib feature since Go 1.22, the
project stays on the pinned Go 1.26), middleware outermost-first recover →
request-log → `MaxBytesReader(64KiB)` → constant-time bearer auth with
`/healthz` exempt, empty `DISCOVERY_TOKEN` serving unauthenticated with a
loud startup warning, `NeedLeaderElection() = false` with the plan's
runnable-ordering rationale in the doc comment, and graceful `Shutdown`
with a 10 s grace on ctx cancel.
The `LeaseStore` interface landed consumer-side in this package (spec §8
wants handlers swappable to a CRD/Redis store); `internal/lease.*Store`
satisfies it without modification.
Judgment calls the plan/spec left open:
- **409 arithmetic:** the store only ever sees healthy candidates, so its
`Considered` excludes unhealthy matches. The handler counts unhealthy
selector-matches itself and reports `considered = healthy + unhealthy`,
keeping the plan's example arithmetic (7 = 2+2+3) consistent.
- **TTL handling:** omitted/zero `ttlSeconds` → 300 s default; negative or
above `MaxLeaseTTL` (default 1h, flag in Step 10) → 400 `invalid_ttl`
rather than silent clamping — a client asking for a week-long lease
should find out, not get an hour quietly.
- **Grant response includes the fresh `activeLeases`** (the just-granted
lease counted), read back via `Store.Counts()` after the acquire.
- Proxies with a deletionTimestamp are filtered out of both list and
candidate selection — a proxy mid-teardown shouldn't be advertised.
Tests (87.3% coverage, `-race -count=2` clean, green on first run):
httptest over the real handler chain with a fake cache reader and a real
`lease.Store` — auth on/off/wrong-token/healthz-exempt, list filtering
(attributes, healthy, combined, empty-is-200), grant shape (201, default
TTL, lowest-latency pick, RFC3339 expiresAt, activeLeases=1), the full
409 body arithmetic, invalid TTL/body/result, idempotent 204 release,
report→cooldown→409 round-trip, 404 on unknown lease, and a real
`Start` on `127.0.0.1:0` (via the new `BoundAddr()` accessor) serving
healthz then shutting down cleanly on cancel.
```bash
go test -race -count=2 ./internal/discovery/
make test # whole repo green
```
Worth noting: `go mod tidy` promoted `github.com/go-logr/logr` from
indirect to direct (the server holds a `logr.Logger` field). The
`--discovery-addr`, `--max-lease-ttl` flags and the `DISCOVERY_TOKEN`
Secret mount arrive with `cmd/main.go` in Step 10. `docs/architecture.md`
gained §7 covering the whole HTTP path and the store's sweep Runnable.