learning-go-for-sre-interview/03-rate-limiter/README.md

03 — Token Bucket Rate Limiter

Rate limiting is the SRE conversation topic: it appears in load shedding, API quotas, retry storms, and every postmortem featuring the phrase "cascading failure."

Why lazy refill beats a ticker

The tempting design — a goroutine adding tokens on a time.Ticker — has real problems:

• a goroutine + timer per limiter (per-user limiters → thousands of tickers),
• refill granularity = tick period (bursty admission at tick boundaries),
• lifecycle: who stops the ticker when the limiter is GC'd? (Nobody. It leaks.)

Lazy refill computes tokens += elapsed * rate on demand: zero background work, nanosecond-precision, and the limiter is just a struct — GC handles it. This "compute state from timestamps instead of maintaining it" trick generalizes (TTL caches, session expiry, leases).

Token bucket vs the alternatives (know this table)

Algorithm	Bursts?	Memory	Famous gotcha
Token bucket	yes, up to burst	O(1)	float drift over long uptimes
Leaky bucket	no (smooths)	O(1)	adds latency by design
Fixed window	2x at boundary	O(1)	100 req at 0:59 + 100 at 1:01 = "fine"
Sliding window	accurate	O(events)	memory scales with rate

The fixed-window boundary burst is a favorite interview probe: a limit of "100/min" admits 200 requests in 2 seconds across a window edge.

Monotonic time, or: why your limiter survives NTP

time.Now() in Go carries both wall and monotonic clock readings, and t2.Sub(t1) uses the monotonic part. So an NTP step-back won't mint you negative elapsed time or a token jackpot. This is why you store last time.Time and subtract — never store unix floats and diff them yourself. (Kernel connection: CLOCK_MONOTONIC vs CLOCK_REALTIME.)

Production answer

golang.org/x/time/rate is the canonical implementation — token bucket with Allow, blocking Wait(ctx), and Reserve. Say you'd use it in prod; the interview wants the from-scratch version to see if you understand what's inside. Follow-up they love: "make it distributed" → local buckets w/ divided quota, or centralized counters (Redis + Lua for atomicity), and the CAP-ish trade-off between accuracy and availability.