kacerrandClaude Fable 5
7219eb5a23
Includes solved 02-worker-pool and 09-safe-counter (mutex, atomic, sharded). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnJj9ZqEx9nUvzkjdwfMq
03 — Token Bucket Rate Limiter
Rate limiting is the SRE conversation topic: it appears in load shedding, API quotas, retry storms, and every postmortem featuring the phrase "cascading failure."
Why lazy refill beats a ticker
The tempting design — a goroutine adding tokens on a time.Ticker — has real
problems:
- a goroutine + timer per limiter (per-user limiters → thousands of tickers),
- refill granularity = tick period (bursty admission at tick boundaries),
- lifecycle: who stops the ticker when the limiter is GC'd? (Nobody. It leaks.)
Lazy refill computes tokens += elapsed * rate on demand: zero background
work, nanosecond-precision, and the limiter is just a struct — GC handles it.
This "compute state from timestamps instead of maintaining it" trick
generalizes (TTL caches, session expiry, leases).
Token bucket vs the alternatives (know this table)
| Algorithm | Bursts? | Memory | Famous gotcha |
|---|---|---|---|
| Token bucket | yes, up to burst | O(1) | float drift over long uptimes |
| Leaky bucket | no (smooths) | O(1) | adds latency by design |
| Fixed window | 2x at boundary | O(1) | 100 req at 0:59 + 100 at 1:01 = "fine" |
| Sliding window | accurate | O(events) | memory scales with rate |
The fixed-window boundary burst is a favorite interview probe: a limit of "100/min" admits 200 requests in 2 seconds across a window edge.
Monotonic time, or: why your limiter survives NTP
time.Now() in Go carries both wall and monotonic clock readings, and
t2.Sub(t1) uses the monotonic part. So an NTP step-back won't mint you
negative elapsed time or a token jackpot. This is why you store last time.Time and subtract — never store unix floats and diff them yourself.
(Kernel connection: CLOCK_MONOTONIC vs CLOCK_REALTIME.)
Production answer
golang.org/x/time/rate is the canonical implementation — token bucket with
Allow, blocking Wait(ctx), and Reserve. Say you'd use it in prod; the
interview wants the from-scratch version to see if you understand what's
inside. Follow-up they love: "make it distributed" → local buckets w/ divided
quota, or centralized counters (Redis + Lua for atomicity), and the CAP-ish
trade-off between accuracy and availability.