# Experiment — a global rate limiter as a service

Exercise 03 built an in-process token bucket. That limits **one process**.
When N replicas of a service must share one quota ("tenant-a gets 10 req/s,
total"), the bucket has to live somewhere they can all reach. This is the
smallest honest version of that: an HTTP service holding per-key buckets, and
a demo client that plays one replica.

```
99-experiments/global-ratelimit/
  limiter/   token bucket (ex03) + sharded keyed store (ex09) + idle eviction   go test -race
  server/    HTTP service: /allow, /stats, /healthz
  client/    demo replica: asks /allow before each unit of work
```

## Run it

```
# terminal 1 — the limiter
go run ./99-experiments/global-ratelimit/server -rate 10 -burst 10

# quick check
for i in $(seq 12); do curl -s -o /dev/null -w '%{http_code} ' 'localhost:8100/allow?key=t'; done
#   200 200 200 200 200 200 200 200 200 200 429 429
curl -si 'localhost:8100/allow?key=t' | grep -iE 'HTTP/|retry|ratelimit'

# terminals 2–4 — three replicas, each WANTS 20/s, all on key tenant-a
go run ./99-experiments/global-ratelimit/client -id inst-1 -rps 20
go run ./99-experiments/global-ratelimit/client -id inst-2 -rps 20
go run ./99-experiments/global-ratelimit/client -id inst-3 -rps 20

curl -s localhost:8100/stats
```

Observed (3 × 20 rps attempted, limit 10/s, 8 s):

```
inst-1 summary: attempts=159 allowed=31 denied=127 → work done=32
inst-2 summary: attempts=159 allowed=28 denied=130 → work done=29
inst-3 summary: attempts=160 allowed=29 denied=130 → work done=30
                                     ───
                                      88 ≈ 10/s × 8 s + burst 10
inst-1 /allow latency: p50=0.6ms p99=11ms
```

The **sum** stays at the limit, not each instance — that's the whole point of
centralising it. Each replica gets roughly a third, with no coordination
between replicas and no per-replica configuration.

## API

```
POST|GET /allow?key=K[&n=1]
  200 {"allowed":true,"remaining":9,"limit":10,"rate":10}
  429 {"allowed":false,"remaining":0.3,"limit":10,"rate":10,"retry_after_ms":70}
  headers: X-RateLimit-Limit, X-RateLimit-Remaining, Retry-After (429; whole seconds, rounded up)
  400 missing key / bad n
GET /stats    {"keys":2,"allowed_total":99,"denied_total":391}
GET /healthz  ok
```

Server flags: `-addr`, `-rate`/`-burst` (defaults for unknown keys),
`-limits "tenant-a=100:200,tenant-b=5:5"` (per-key overrides), `-sweep 1m`.

Client flags: `-id`, `-limiter`, `-key`, `-rps`, `-duration`, `-timeout 200ms`,
`-fail open|closed`.

## What to observe

**The hop.** Every decision is an HTTP round trip: p50 ≈ 0.6 ms on loopback,
1–2 ms across an AZ. The in-process limiter is ~100 ns. That's the price of
"global", and why you only pay it at the edge (per-tenant quota), not in inner
loops.

**Fail-open vs fail-closed.** Start a client with each mode, then Ctrl-C the
server mid-run:

```
open    allowed=0 denied=0 errors=10   → work done keeps going (49 of 59)
closed  allowed=0 denied=0 errors=10   → work stops             (18 of 59)
```

The limiter is now a dependency on the request path. Fail-open protects your
users and risks overloading what the limiter guards; fail-closed protects the
backend and turns a limiter outage into *your* outage. Pick deliberately, per
call site, and alert on the error rate either way. `-timeout` matters for the
same reason: a *slow* limiter must not make you slow.

**Idle eviction.** `-sweep` removes buckets untouched for that long. An idle
bucket is full anyway (once `idle ≥ burst/rate`), so recreating it later is
indistinguishable from keeping it — `last` doubles as the idle timestamp,
nothing extra to track. Without this, one bucket per API key ever seen lives
forever.

**Burst semantics across replicas.** The first second admits ~20 (burst 10 +
10/s). Replicas that start first eat the burst. If that's unfair for your use
case, lower `burst` — global buckets make the burst global too.

## What this is not

- **Not HA.** One process, in memory. Restart = all buckets full again
  (a free burst for everyone) and, for the duration, fail-open or fail-closed
  as configured. Real deployments put the bucket in Redis and run the
  refill+consume as one Lua script so it's atomic — same arithmetic as
  `limiter.Bucket.AllowN`, different storage.
- **Not accurate under Redis Cluster / multiple limiter nodes** unless keys are
  routed consistently to one node. Keys → node is what makes a "global" bucket
  global.
- **No local tier.** Production usually also keeps a coarse in-process limiter
  (ex03) as a pre-filter, so a flooding client is cut off before it costs a
  network hop per request, and caches "denied until T" locally.

## References

- Stripe, "Scaling your API with rate limiters" — the token bucket + Redis
  pattern and why to shed load with it
- Envoy `ratelimit` service / `local_ratelimit` filter — the two-tier design
- `golang.org/x/time/rate` — the in-process reference implementation