kacerrandClaude Fable 5
7219eb5a23
Includes solved 02-worker-pool and 09-safe-counter (mutex, atomic, sharded). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnJj9ZqEx9nUvzkjdwfMq
49 lines
2.4 KiB
Markdown
49 lines
2.4 KiB
Markdown
# 03 — Token Bucket Rate Limiter
|
|
|
|
Rate limiting is *the* SRE conversation topic: it appears in load shedding,
|
|
API quotas, retry storms, and every postmortem featuring the phrase
|
|
"cascading failure."
|
|
|
|
## Why lazy refill beats a ticker
|
|
|
|
The tempting design — a goroutine adding tokens on a `time.Ticker` — has real
|
|
problems:
|
|
|
|
- a goroutine + timer **per limiter** (per-user limiters → thousands of tickers),
|
|
- refill granularity = tick period (bursty admission at tick boundaries),
|
|
- lifecycle: who stops the ticker when the limiter is GC'd? (Nobody. It leaks.)
|
|
|
|
Lazy refill computes `tokens += elapsed * rate` on demand: zero background
|
|
work, nanosecond-precision, and the limiter is just a struct — GC handles it.
|
|
This "compute state from timestamps instead of maintaining it" trick
|
|
generalizes (TTL caches, session expiry, leases).
|
|
|
|
## Token bucket vs the alternatives (know this table)
|
|
|
|
| Algorithm | Bursts? | Memory | Famous gotcha |
|
|
|-----------------|------------------|----------|----------------------------------------|
|
|
| Token bucket | yes, up to burst | O(1) | float drift over long uptimes |
|
|
| Leaky bucket | no (smooths) | O(1) | adds latency by design |
|
|
| Fixed window | 2x at boundary | O(1) | 100 req at 0:59 + 100 at 1:01 = "fine" |
|
|
| Sliding window | accurate | O(events)| memory scales with rate |
|
|
|
|
The fixed-window boundary burst is a favorite interview probe: a limit of
|
|
"100/min" admits 200 requests in 2 seconds across a window edge.
|
|
|
|
## Monotonic time, or: why your limiter survives NTP
|
|
|
|
`time.Now()` in Go carries **both** wall and monotonic clock readings, and
|
|
`t2.Sub(t1)` uses the monotonic part. So an NTP step-back won't mint you
|
|
negative elapsed time or a token jackpot. This is why you store `last
|
|
time.Time` and subtract — never store unix floats and diff them yourself.
|
|
(Kernel connection: `CLOCK_MONOTONIC` vs `CLOCK_REALTIME`.)
|
|
|
|
## Production answer
|
|
|
|
`golang.org/x/time/rate` is the canonical implementation — token bucket with
|
|
`Allow`, blocking `Wait(ctx)`, and `Reserve`. Say you'd use it in prod; the
|
|
interview wants the from-scratch version to see if you understand what's
|
|
inside. Follow-up they love: "make it distributed" → local buckets w/ divided
|
|
quota, or centralized counters (Redis + Lua for atomicity), and the CAP-ish
|
|
trade-off between accuracy and availability.
|