# 06 — Retry with Backoff + Jitter

Retries are the most dangerous reliability primitive: applied naively they
convert partial outages into total ones. Interviewers use this task to check
whether you've internalized that or just memorized the loop.

## Why jitter is non-negotiable (the thundering herd)

A dependency blips for 2 seconds. 10,000 clients fail *simultaneously*.
Without jitter they all retry at t+1s, t+3s, t+7s — as one synchronized
battering ram. The service recovers, gets hit by a perfectly-phased wave of
10k requests, dies again. Congratulations, your retry policy is now a DDoS
with extra steps — this is a **metastable failure**, and the system can stay
wedged even after the original trigger is gone.

The AWS Architecture Blog's classic analysis ("Exponential Backoff and
Jitter") compared strategies and **full jitter** — `sleep = rand(0, backoff)`
— came out best: maximal desynchronization, nearly minimal total work. That's
what you implement here. "Equal jitter" (`backoff/2 + rand(0, backoff/2)`) is
the runner-up if you want a floor.

## `time.Sleep` vs `select` — the test that catches everyone

```go
select {
case <-time.After(sleep):
case <-ctx.Done():
    return ctx.Err()
}
```

`time.Sleep` is uninterruptible. In a shutdown, a worker mid-backoff on a
10-minute sleep holds your rolling deploy hostage until SIGKILL. Selecting on
`ctx.Done()` is what makes retries composable with graceful shutdown
(exercise 08). This single line is the difference between "knows Go" and
"has been paged."

## Error classification — retrying the unretryable

Retrying a 400 or a permission error is pure waste; retrying a *non-idempotent*
operation (POST /charge) is a career event. The pattern here uses a sentinel
(`errors.Is(err, ErrRetryable)`); production code often classifies by type
(`errors.As(&netErr)`), by status code, or via an `interface{ Temporary() bool }`.
Bring up **idempotency keys** as the real fix for retrying writes.

## The complete picture (say this at the whiteboard)

Retries alone are not a resilience strategy. The full stack:

1. **Cap the backoff** (`min(backoff, maxBackoff)`) — 2^20 seconds is not a plan.
2. **Retry budgets** (e.g., retries ≤ 10% of requests, à la the SRE book / Finagle)
   beat per-request counts — they bound *aggregate* amplification.
3. **Circuit breaker** — stop calling a dead dependency entirely; probe
   half-open. Backoff protects *them*, the breaker protects *you*.
4. **Deadline propagation** — retrying past the caller's deadline serves
   responses nobody is waiting for. Notice the ctx check does this for free.
5. **Only retry at one layer.** Client retries × mesh retries × app retries =
   exponential amplification. Pick an owner.