Add the in-memory lease store: least-loaded selection, cooldowns, TTL retention
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -10,7 +10,7 @@ Pairs with [docs/plans/2026-08-07-1747-proxy-operator.md](../plans/2026-08-07-17
|
||||
- [x] Step 3 — Kubernetes pod provider (`internal/provider/kubernetes/`; first built as an in-memory mock, then replaced — see the two Step 3 sections below)
|
||||
- [x] Step 4 — Reconciler (`internal/controller/`)
|
||||
- [x] Step 5 — Health engine (`internal/health/`)
|
||||
- [ ] Step 6 — Lease store (`internal/lease/`)
|
||||
- [x] Step 6 — Lease store (`internal/lease/`)
|
||||
- [ ] Step 7 — Discovery API (`internal/discovery/`)
|
||||
- [ ] Step 8 — GCP provider (`internal/provider/gcp/`)
|
||||
- [ ] Step 9 — Orphan GC + metrics
|
||||
@@ -694,3 +694,67 @@ engine deliberately knows nothing about conditions except reading one at
|
||||
seed time, keeping the state/representation split honest. The
|
||||
`hint`-driven `wg.Go` idiom (Go 1.25+) replaced the classic
|
||||
`wg.Add/defer wg.Done` in the worker pool.
|
||||
|
||||
## Step 6 — Lease store (`internal/lease/`)
|
||||
|
||||
Implemented `store.go` per the plan: `Acquire` takes the whole candidate
|
||||
set so selection and insertion happen under the one store mutex (no
|
||||
overcommit between concurrent requests), selection is a linear scan +
|
||||
`slices.SortFunc` on `(activeLeases asc, latency asc, name asc)`,
|
||||
`AcquireStats{Considered, AtCapacity, InCooldown}` feeds Step 7's 409
|
||||
body, cooldowns live in a `map[{proxy, target}]time.Time` (empty target =
|
||||
global pool), and expired leases are retained for `CooldownWindow` past
|
||||
their TTL so a late `Report` — arriving exactly when a proxy is being
|
||||
rate-limited — still resolves and records its cooldown.
|
||||
|
||||
Semantics pinned against the spec (§8) rather than guessed:
|
||||
|
||||
- Report results are exactly `ok | rate_limited | banned` (`ParseResult`
|
||||
gives the API layer its 400 check). `rate_limited` and `banned` both
|
||||
record a cooldown for the same window; `ok` records nothing.
|
||||
Distinguishing ban duration from rate-limit duration would be a second
|
||||
knob the spec doesn't ask for — noted for the Decisions section.
|
||||
- Cooldown scoping: the global cooldown (empty target) always applies; a
|
||||
target-scoped cooldown additionally blocks acquisitions for that target;
|
||||
acquisitions without a target see only the global pool ("a proxy
|
||||
rate-limited by one site is still fine for everyone else").
|
||||
- A `Report` without a target falls back to the lease's own target before
|
||||
falling back to global — so a client that leased with a target doesn't
|
||||
accidentally poison the whole proxy by omitting it in the report.
|
||||
|
||||
Design notes:
|
||||
|
||||
- **Correctness never depends on the sweep.** Every read path
|
||||
(`Acquire`/`ActiveCount`/`Counts`) compares `ExpiresAt` against the
|
||||
injected clock, so TTL expiry frees capacity immediately even if the
|
||||
background loop hasn't run; the sweep is purely garbage collection. The
|
||||
plan's `ExpireLoop` became `Start(ctx)` + `NeedLeaderElection() false`
|
||||
so the store satisfies `manager.Runnable` directly — Step 10 just
|
||||
`mgr.Add(store)`s it. Not leader-elected because lease state is
|
||||
per-process and must expire wherever the discovery API is serving.
|
||||
- The store knows nothing about Proxy objects — `Candidate` carries the
|
||||
opaque key, `MaxLeases`, and latency; the discovery layer does the
|
||||
health/attribute filtering. The spec's `LeaseStore` interface will be
|
||||
defined consumer-side in `internal/discovery` (Step 7), per Go idiom;
|
||||
this package exports only the concrete in-memory `*Store`.
|
||||
- Lease IDs come from `crypto/rand.Text()` (Go 1.24+); returned `Lease`
|
||||
values are copies so callers can't mutate store internals.
|
||||
|
||||
Tests (94.8% coverage, `-race -count=2` clean): capacity + release
|
||||
freeing slots, `MaxLeases=0` unleasable, least-loaded/latency/name
|
||||
selection order, target-scoped vs global cooldown scoping, cooldown
|
||||
expiry via the injected fake clock, TTL freeing capacity with no sweep,
|
||||
report-on-expired-but-retained lease (then `ErrUnknownLease` after
|
||||
retention), `ok` recording nothing, idempotent release, `ParseResult`,
|
||||
40 concurrent acquires against `MaxLeases=5` granting exactly 5, and the
|
||||
`Start` loop sweeping then stopping cleanly on cancel.
|
||||
|
||||
```bash
|
||||
go test -race -count=2 ./internal/lease/
|
||||
make test # whole repo green, other packages' coverage unchanged
|
||||
```
|
||||
|
||||
Worth noting: `docs/architecture.md` was not extended this step — the
|
||||
lease store is HTTP-driven, not cluster-event-driven, so its diagram
|
||||
belongs with the discovery API and lands in Step 7 (banner updated to say
|
||||
so).
|
||||
|
||||
Reference in New Issue
Block a user