Add body-drain experiment: observe HTTP body drain/close/leak effects

Local counting server + httptrace/goroutine metrics showing connection
reuse (drain), pool bypass (close only) and fd/goroutine leak (no close).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GbnJj9ZqEx9nUvzkjdwfMq

99-experiments/body-drain/README.md

@@ -0,0 +1,110 @@
+# Experiment — not draining / closing HTTP response bodies
+
+Companion to exercise 01, assignment point 5 ("reuse a single http.Client and
+properly drain+close response bodies so connections are reused").
+
+```
+go run ./99-experiments/body-drain -mode all
+go run ./99-experiments/body-drain -mode leak -n 200
+ulimit -n 64 && go run ./99-experiments/body-drain -mode leak -n 200   # watch it die
+```
+
+## What it does
+
+A local `httptest.Server` returns a 1 MiB body and counts accepted TCP
+connections via `Server.ConnState`. The client does N sequential GETs in one of
+three modes, with a fresh `Transport` per mode:
+
+| mode    | client does                                   |
+|---------|-----------------------------------------------|
+| `drain` | `io.Copy(io.Discard, body)` then `body.Close()` |
+| `close` | `body.Close()` only                             |
+| `leak`  | nothing                                         |
+
+## What to watch
+
+- **`server new conns`** — TCP connections the server accepted. Ideal: 1.
+- **`httptrace GotConn reused/fresh`** — the client's own view: did `client.Do`
+  pull a conn from the idle pool or dial a new one.
+- **`goroutines delta`** — every connection the transport owns runs two
+  goroutines (`readLoop` + `writeLoop`). Pooled conns show up here too, but
+  they disappear after `CloseIdleConnections()`. Leaked ones don't.
+- **`server active conns`** — conns the server still considers open after the
+  client is "done".
+
+## Typical output
+
+```
+== mode=drain ==
+  httptrace GotConn  : reused=19 fresh=1
+  goroutines         : before=2 after=5 (delta +3)
+  after CloseIdleConnections: goroutines=2
+  server new conns   : 1 (ideal: 1)
+  server active conns: 0
+
+== mode=close ==
+  httptrace GotConn  : reused=0 fresh=20
+  goroutines         : before=2 after=2
+  server new conns   : 20 (ideal: 1)
+  server active conns: 0
+
+== mode=leak ==
+  httptrace GotConn  : reused=0 fresh=20
+  goroutines         : before=2 after=73 (delta +71)
+  after CloseIdleConnections: goroutines=73        <- nothing reclaimed
+  server new conns   : 20 (ideal: 1)
+  server active conns: 20                          <- server stuck too
+```
+
+## Reading it
+
+**`close` without drain is not a leak, but it defeats the pool.** The transport
+sees unread bytes on the wire; reusing that TCP stream would corrupt the next
+response, so it closes the conn. Every request pays DNS + TCP + TLS again. On
+loopback that's microseconds; against a real HTTPS host it's 3–4 RTTs per
+request and a TLS handshake the server has to do CPU work for. Correct output,
+silently several times slower, and a `TIME_WAIT` pile-up on your host.
+
+**`leak` leaks three things per request:** a file descriptor, two goroutines,
+and the read buffer. `CloseIdleConnections` can't help because the conn is not
+idle — the transport thinks a response is still in progress. Under any real
+request rate this ends in `dial tcp: socket: too many open files`.
+
+**The server suffers as well.** Its handler is blocked in `Write()` because the
+client never reads and the kernel socket buffers are full. Without a
+`WriteTimeout` those handler goroutines live until the client process dies.
+This is why `httptest.Server.Close` hangs and logs "blocked in Close after 5
+seconds" if you remove the `CloseClientConnections()` call — try it.
+
+**Why doesn't a `Client.Timeout` hide this?** It does, partially: when the timer
+fires, the transport force-closes the conn and the goroutines go away. So a
+leaky client with a 5 s timeout leaks for 5 s per request instead of forever —
+enough to exhaust fds at a few hundred req/s, and no connection reuse either
+way. Add `Timeout: 5 * time.Second` to the client in `run()` and re-run `leak`
+with a `time.Sleep(6 * time.Second)` before the goroutine count to see it.
+
+## Rules that follow
+
+```go
+resp, err := client.Do(req)
+if err != nil { return err }
+defer resp.Body.Close()
+io.Copy(io.Discard, resp.Body)   // or io.CopyN(..., 64<<10) if bodies may be huge
+```
+
+- Always `Close()`, always via `defer` right after the error check.
+- Read to EOF if you want the conn back. If you don't need the body and it may
+  be large, cap the drain — past some size a new handshake is cheaper than the
+  download.
+- Consider `HEAD` for health checks: nothing to drain.
+- `Transport.MaxIdleConnsPerHost` defaults to 2; with `limit` concurrent
+  checkers to one host, raise it to `limit` or the pool thrashes anyway.
+
+## References
+
+- `net/http` docs, `Client.Do`: "If the Body is not both read to EOF and closed,
+  the Client's underlying RoundTripper may not be able to re-use a persistent
+  TCP connection" — https://pkg.go.dev/net/http#Client.Do
+- `net/http/httptrace` — https://pkg.go.dev/net/http/httptrace
+- `http.Server.ConnState` — https://pkg.go.dev/net/http#Server.ConnState
+- golang/go#4677 — the issue where this behaviour was settled.