Add the discovery HTTP API: list, lease, release, report over the manager cache
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -1,12 +1,10 @@
|
||||
# Architecture
|
||||
|
||||
> **Status:** the operator is built through Step 6 (lease store) of
|
||||
> **Status:** the operator is built through Step 7 (discovery API) of
|
||||
> [docs/plans/2026-08-07-1747-proxy-operator.md](plans/2026-08-07-1747-proxy-operator.md).
|
||||
> This document currently covers the event/reconcile flow; the components
|
||||
> table and the Decisions section arrive with Step 10. The lease store
|
||||
> (`internal/lease/`) is HTTP-driven, not cluster-event-driven, so its
|
||||
> diagram lands together with the discovery API in Step 7; the orphan-GC
|
||||
> flow lands with Step 9.
|
||||
> This document currently covers the event/reconcile flow and the
|
||||
> HTTP-driven lease/discovery path; the components table and the Decisions
|
||||
> section arrive with Step 10, and the orphan-GC flow lands with Step 9.
|
||||
|
||||
## Event flow: cluster events → reconciler functions
|
||||
|
||||
@@ -185,3 +183,48 @@ Consequence worth knowing: `status.lastHealthCheckTime` is the time of the
|
||||
last *status-affecting* probe, not the most recent probe — suppressed
|
||||
probes deliberately never write status. True probe recency will live in
|
||||
metrics (Step 9).
|
||||
|
||||
### 7. Discovery + lease API (`internal/discovery/`, `internal/lease/`)
|
||||
|
||||
HTTP-driven, not cluster-event-driven: crawler clients call in; the only
|
||||
Kubernetes interaction is reading Proxies from the manager's cache. The
|
||||
server is a non-leader-elected Runnable (all replicas would serve, but the
|
||||
deployment ships `replicas: 1` because lease state is per-process — an
|
||||
operator restart drops all leases and cooldowns, a documented caveat).
|
||||
|
||||
```text
|
||||
crawler client
|
||||
│ Authorization: Bearer $DISCOVERY_TOKEN (empty token = auth disabled, loud startup warning)
|
||||
▼
|
||||
Server.handler() middleware, outermost first (server.go)
|
||||
recover → request-log → MaxBytesReader(64KiB) → bearer auth (constant-time; /healthz exempt)
|
||||
│
|
||||
├─ GET /healthz ──► 200 ok (unauthenticated)
|
||||
│
|
||||
├─ GET /v1/proxies?attr.k=v&healthy=true (handlers.go)
|
||||
│ Reader.List(Proxies) ── manager cache
|
||||
│ filter: attributes equality + Healthy condition
|
||||
│ + Store.Counts() for activeLeases
|
||||
│ ──► 200 {"proxies":[...], "count":N} (empty list is 200, not 404)
|
||||
│
|
||||
├─ POST /v1/leases {"selector":{...},"ttlSeconds":300,"target":"..."}
|
||||
│ Reader.List → filter selector; unhealthy matches counted, not offered
|
||||
│ Store.Acquire(healthy candidates, target, ttl) ── one lock: select+insert
|
||||
│ │ selection: fewest active leases, then latency, then name
|
||||
│ ├─ granted ──► 201 {leaseID, proxy:{...}, expiresAt, ttlSeconds}
|
||||
│ └─ ErrNoMatch ──► 409 {"error":"no_match", considered, atCapacity,
|
||||
│ inCooldown, unhealthy}
|
||||
│
|
||||
├─ DELETE /v1/leases/{id} ──► Store.Release ──► always 204 (idempotent)
|
||||
│
|
||||
└─ POST /v1/leases/{id}/report {"result":"ok|rate_limited|banned","target":"..."}
|
||||
Store.Report ── rate_limited/banned ⇒ cooldown[{proxy,target}] for
|
||||
│ CooldownWindow (target falls back: report → lease → global)
|
||||
├─ 204 │ 400 invalid_result │ 404 unknown_lease
|
||||
└─ an expired lease still resolves for CooldownWindow past its TTL —
|
||||
a late report lands exactly when the proxy is being rate-limited
|
||||
|
||||
Store.Start(ctx) ── manager Runnable, NOT leader-elected: sweeps expired
|
||||
leases + cooldowns; correctness never depends on the
|
||||
sweep (every read checks ExpiresAt against the clock)
|
||||
```
|
||||
|
||||
@@ -11,7 +11,7 @@ Pairs with [docs/plans/2026-08-07-1747-proxy-operator.md](../plans/2026-08-07-17
|
||||
- [x] Step 4 — Reconciler (`internal/controller/`)
|
||||
- [x] Step 5 — Health engine (`internal/health/`)
|
||||
- [x] Step 6 — Lease store (`internal/lease/`)
|
||||
- [ ] Step 7 — Discovery API (`internal/discovery/`)
|
||||
- [x] Step 7 — Discovery API (`internal/discovery/`)
|
||||
- [ ] Step 8 — GCP provider (`internal/provider/gcp/`)
|
||||
- [ ] Step 9 — Orphan GC + metrics
|
||||
- [ ] Step 10 — Wiring, config, docs
|
||||
@@ -758,3 +758,56 @@ Worth noting: `docs/architecture.md` was not extended this step — the
|
||||
lease store is HTTP-driven, not cluster-event-driven, so its diagram
|
||||
belongs with the discovery API and lands in Step 7 (banner updated to say
|
||||
so).
|
||||
|
||||
## Step 7 — Discovery API (`internal/discovery/`)
|
||||
|
||||
Implemented `server.go` (Runnable + middleware chain) and `handlers.go`
|
||||
(the four endpoints + `proxyView` wire shape) per the plan: stdlib
|
||||
`http.ServeMux` method+wildcard routing (no third-party router — see the
|
||||
plan clarification commit: this is a stdlib feature since Go 1.22, the
|
||||
project stays on the pinned Go 1.26), middleware outermost-first recover →
|
||||
request-log → `MaxBytesReader(64KiB)` → constant-time bearer auth with
|
||||
`/healthz` exempt, empty `DISCOVERY_TOKEN` serving unauthenticated with a
|
||||
loud startup warning, `NeedLeaderElection() = false` with the plan's
|
||||
runnable-ordering rationale in the doc comment, and graceful `Shutdown`
|
||||
with a 10 s grace on ctx cancel.
|
||||
|
||||
The `LeaseStore` interface landed consumer-side in this package (spec §8
|
||||
wants handlers swappable to a CRD/Redis store); `internal/lease.*Store`
|
||||
satisfies it without modification.
|
||||
|
||||
Judgment calls the plan/spec left open:
|
||||
|
||||
- **409 arithmetic:** the store only ever sees healthy candidates, so its
|
||||
`Considered` excludes unhealthy matches. The handler counts unhealthy
|
||||
selector-matches itself and reports `considered = healthy + unhealthy`,
|
||||
keeping the plan's example arithmetic (7 = 2+2+3) consistent.
|
||||
- **TTL handling:** omitted/zero `ttlSeconds` → 300 s default; negative or
|
||||
above `MaxLeaseTTL` (default 1h, flag in Step 10) → 400 `invalid_ttl`
|
||||
rather than silent clamping — a client asking for a week-long lease
|
||||
should find out, not get an hour quietly.
|
||||
- **Grant response includes the fresh `activeLeases`** (the just-granted
|
||||
lease counted), read back via `Store.Counts()` after the acquire.
|
||||
- Proxies with a deletionTimestamp are filtered out of both list and
|
||||
candidate selection — a proxy mid-teardown shouldn't be advertised.
|
||||
|
||||
Tests (87.3% coverage, `-race -count=2` clean, green on first run):
|
||||
httptest over the real handler chain with a fake cache reader and a real
|
||||
`lease.Store` — auth on/off/wrong-token/healthz-exempt, list filtering
|
||||
(attributes, healthy, combined, empty-is-200), grant shape (201, default
|
||||
TTL, lowest-latency pick, RFC3339 expiresAt, activeLeases=1), the full
|
||||
409 body arithmetic, invalid TTL/body/result, idempotent 204 release,
|
||||
report→cooldown→409 round-trip, 404 on unknown lease, and a real
|
||||
`Start` on `127.0.0.1:0` (via the new `BoundAddr()` accessor) serving
|
||||
healthz then shutting down cleanly on cancel.
|
||||
|
||||
```bash
|
||||
go test -race -count=2 ./internal/discovery/
|
||||
make test # whole repo green
|
||||
```
|
||||
|
||||
Worth noting: `go mod tidy` promoted `github.com/go-logr/logr` from
|
||||
indirect to direct (the server holds a `logr.Logger` field). The
|
||||
`--discovery-addr`, `--max-lease-ttl` flags and the `DISCOVERY_TOKEN`
|
||||
Secret mount arrive with `cmd/main.go` in Step 10. `docs/architecture.md`
|
||||
gained §7 covering the whole HTTP path and the store's sweep Runnable.
|
||||
|
||||
Reference in New Issue
Block a user