diff --git a/98-http-server-experiments/README.md b/98-http-server-experiments/README.md new file mode 100644 index 0000000..0e33b52 --- /dev/null +++ b/98-http-server-experiments/README.md @@ -0,0 +1,283 @@ +# 98 — HTTP server connection experiments + +How many connections can a Go HTTP server hold, and what gives out first? This +folder finds the **order of magnitude** (not a portable constant) for four +different failure modes, and — just as important — teaches you to tell a real +server ceiling from your own load generator running out first. + +One simple server is the subject under test; one load client drives it in four +modes. Run them in two terminals so each side has its own fd budget. + +``` +98-http-server-experiments/ + server/main.go subject under test: /static /compute /sleep /stats, ConnState + in_flight counters + client/main.go load generator: -mode throughput | idle | slow | slowloris + loadlib/ tested core: Burn (CPU), Percentile, Classify (which side ran out) + README.md this file — decisions log + pasted evidence +``` + +Pure stdlib, zero dependencies. Part of the `gosre-exercises` module, so: + +```sh +go run ./98-http-server-experiments/server # terminal 1 +go run ./98-http-server-experiments/client -mode idle -n 2000 # terminal 2 +go test -race ./98-http-server-experiments/loadlib/ +``` + +## Run it + +**Terminal 1 — the server.** Give it its own fd budget so *you* decide when it +runs out, not the OS default. bash sets both soft and hard limits: + +```sh +ulimit -Hn 4096; ulimit -Sn 4096 +go run ./98-http-server-experiments/server # naive: no timeouts +go run ./98-http-server-experiments/server -read-header-timeout 3s -idle-timeout 30s # hardened +``` + +(Since Go 1.19 the runtime raises the soft `RLIMIT_NOFILE` to the hard limit at +startup, so only the **hard** limit really caps you. bash's `ulimit -n` sets +both; zsh's sets only the soft one — use `-Hn`.) + +The server prints one line per second: + +``` +19:12:20 accepted_total=10001 active=10001 in_flight=1 goroutines=10006 heap=107MiB +``` + +Three numbers that are easy to conflate: + +| number | meaning | via | +|---|---|---| +| `accepted_total` | every TCP conn ever accepted (only grows) | `ConnState` StateNew | +| `active` | conns open **right now** — idle keep-alive *or* busy | `ConnState` new − closed | +| `in_flight` | handlers **actually running** right now | middleware around the mux | + +An idle keep-alive counts in `active` but not `in_flight`; a running `/sleep` +counts in both. That gap is the whole point of the idle-vs-slow contrast. + +**Terminal 2 — the client.** Give it *its own* fd budget, separate from the server: + +```sh +ulimit -Hn 64; ulimit -Sn 64 +go run ./98-http-server-experiments/client -mode idle -n 200 +``` + +## The four dimensions + +### 1. Idle connection ceiling — `-mode idle` + +Open `-n` keep-alive conns and hold them idle. Uses **raw `net.Conn`**, not an +`http.Client` — with a fast endpoint the client's connection pool reuses conns +faster than you dial, so N concurrent GETs do *not* reliably produce N +simultaneous conns. Dialing raw and keeping each fd open guarantees exactly N. + +```sh +go run ./98-http-server-experiments/client -mode idle -n 10000 -duration 30s +``` + +``` +== mode=idle target=http://127.0.0.1:8098 == + server before : active=1 in_flight=1 goroutines=6 heap=0MiB + opened : 10000/10000 idle conns held, 0 failed + client goroutines : before=3 after=3 (delta +0; raw fds, no per-conn goroutine) + server after open : active=10001 in_flight=1 goroutines=10006 heap=107MiB ← active≈n, in_flight≈0 +``` + +**Reading it:** 10 000 idle conns cost the server ~107 MiB (from a <1 MiB +baseline) and one goroutine each → **order of magnitude ~10 KiB per idle +connection**. A million idle keep-alives would be ~10 GiB and a million +goroutines — the server holds them fine until it runs out of fds or RAM, not CPU. +Note the client's goroutine delta is **0**: raw fds have no per-conn goroutine, +unlike `http.Transport`'s pool (2 goroutines per pooled conn). + +### 2. Active slow requests — `-mode slow` + +Fire `-c` concurrent `/sleep` requests and hold them *all in flight* for the +duration. Same conn count as idle, but now each pins a **running** server +goroutine. Prints the server's `in_flight` each second. + +```sh +go run ./98-http-server-experiments/client -mode slow -c 500 -duration 10s +``` + +``` +== mode=slow target=http://127.0.0.1:8098 == + firing 500 concurrent /sleep?d=10s ... + server 01:13:13 active=501 in_flight=501 goroutines=1006 heap=25MiB + requests : 500 ok, 0 failed (of 500 held in flight) + errors : none +``` + +**Reading it:** `in_flight=501` (the 500 sleepers + the stats poll) vs idle's +`in_flight=1` — that is the difference between "conn open" and "handler running." +Each in-flight request is one goroutine on a goroutine-per-request server; 500 +held costs ~1000 goroutines and a few MiB. Push `-c` up in orders of magnitude to +find where the *server* (goroutine stacks, fds) — not the client — gives out. +`/sleep` honors `r.Context()`, so a client that hangs up or a server timeout +frees the goroutine immediately rather than sleeping to term. + +### 3. Timeouts & slowloris — `-mode slowloris` + +Open `-c` raw TCP conns, send a request line + one header on each, then dribble a +bogus header every `-drip` and **never** send the terminating blank line. The +handler never starts. The only thing that reclaims these conns is the server's +`ReadHeaderTimeout`. + +**Naive server** (`go run .../server`, no timeouts): + +``` +== mode=slowloris target=http://127.0.0.1:8098 == + opening 200 slowloris conns ... + server 01:13:26 active=201 in_flight=1 goroutines=206 heap=26MiB ← stuck, handler never runs + reclaimed by server: 0 + (0 reclaimed: the server has no -read-header-timeout — these conns would stick until fds run out) +``` + +**Hardened server** (`go run .../server -read-header-timeout 3s`): + +``` + server 01:13:40 active=201 in_flight=1 goroutines=206 heap=2MiB + server 01:13:42 active=1 in_flight=1 goroutines=6 heap=3MiB ← reclaimed ~3s after opening + reclaimed by server: 200 (server closed the conn mid-headers → -read-header-timeout worked) +``` + +**Reading it:** with no `ReadHeaderTimeout`, 200 clients doing *nothing* consume +200 connections indefinitely — a handful of hosts can exhaust a server's fds +without ever completing a request. Setting `ReadHeaderTimeout=3s` drops each conn +~3 s after its last progress and `active` stays flat. The same logic applies to +`IdleTimeout` for *completed* keep-alives that then go quiet. **Every production +`http.Server` should set these; the stdlib defaults are all zero (off).** + +### 4. Client-side limits — when the measurement is lying + +The load generator hits its own walls, often *before* the server does. Push idle +past the ephemeral-port ceiling and watch the **client** fail while the server is +fine: + +```sh +go run ./98-http-server-experiments/client -mode idle -n 20000 -duration 5s +``` + +``` + opened : 16312/20000 idle conns held, 3688 failed + errors : client-port-limit=3688 + server after open : active=16313 in_flight=1 goroutines=16318 heap=199MiB +``` + +**Reading it:** this host has 16 384 ephemeral ports (`49152–65535`), and every +conn to the same server `ip:port` needs a distinct one. At 16 312 the client got +`connect: can't assign requested address` — a **client-side** wall. The server +was untroubled (it held 16 313 conns on ~199 MiB). If you didn't cross-check the +server's `/stats`, you might wrongly conclude "the server maxes out at ~16 k." +Every mode fetches `/stats` at start and end for exactly this reason. + +## What to watch + +- **`active` vs `in_flight`** — open connections vs running handlers (dimensions 1 & 2). +- **`goroutines`** — ~1 per server conn, ~2 per pooled *client* conn, ~0 per raw client conn. +- **`heap`** — server memory; ~10 KiB per idle conn on this host. +- **`reused` vs `fresh`** (throughput) — is the pool actually reusing conns, or dialing every time? +- **`p50/p99/max`** latency and **error histogram** classified by which side ran out. + +The throughput control shows how fast it goes when nothing is broken, and how a +CPU-bound endpoint plateaus near `GOMAXPROCS`: + +```sh +go run ./98-http-server-experiments/client -mode throughput -path /static -c 50 -duration 3s +# requests : 445013 ok, 0 failed → 148338 req/s latency p50=259µs p99=1.4ms reused=444971 fresh=42 + +go run ./98-http-server-experiments/client -mode throughput -path "/compute?difficulty=200" -c 50 -duration 3s +# requests : 370 ok, 0 failed → 123 req/s latency p50=393ms p99=1.24s +``` + +150 k req/s on `/static` collapses to ~120 req/s on `/compute?difficulty=200`: +that's CPU-bound, not connection-bound — 8 cores fully busy, more concurrency +just deepens the queue. + +## Reading it — symptom → cause + +| symptom | cause | +|---|---| +| client `dial: socket: too many open files` | **client** fd/ulimit — raise the client's limit; the measurement was client-bound | +| client `connect: can't assign requested address` | **client** ephemeral ports exhausted — reuse conns, or use multiple client IPs | +| server log `accept: too many open files` | **server** fd ceiling — the real thing | +| `timeout awaiting response headers` / `connection reset` | server can't accept/service; client is fine | +| slowloris conns stick forever | server has no `ReadHeaderTimeout` | +| idle keep-alives never drop | server has no `IdleTimeout` | +| throughput plateaus near `GOMAXPROCS` on `/compute` | CPU-bound, not conn-bound | + +The `Classify` helper in `loadlib` turns these strings into counted classes so +the client's error histogram tells you which side gave out. + +## Client-side pitfalls (know when the number is yours, not the server's) + +- **Ephemeral port exhaustion.** One client → one server `ip:port` gets ~16 k–28 k + ephemeral ports (`net.inet.ip.portrange.first/last` on macOS, + `net.ipv4.ip_local_port_range` on Linux). Above that, `connect` fails. Fix for + the *test*: reuse keep-alive conns (don't open a fresh 4-tuple per request), or + drive from several client IPs/hosts. +- **TIME_WAIT.** Every actively-closed conn sits in TIME_WAIT ~60 s holding its + 4-tuple; a close-per-request pattern piles them up + (`netstat -an | grep -c TIME_WAIT`) and starves ephemeral ports. Keep-alive + reuse avoids it. `SO_REUSEADDR` does not help the client here. +- **`MaxIdleConnsPerHost` (default 2).** If it's below your concurrency, the + client dials fresh every round and you measure TCP handshakes, not the server. + The client auto-sizes it to `-c`; override with `-idle-per-host`. +- **`Client.Timeout` / `ResponseHeaderTimeout`.** A timeout force-closes a stuck + conn, which *hides* a hold or a leak. The client defaults `-timeout 0`; `slow` + mode disables the header timeout entirely because its responses are slow **by + design** (an early version misreported held requests as "server-cant-accept"). +- **Always cross-check `/stats`.** If the client says "failed" but the server + never saw the conn, the client ran out first. + +## Host caveats (numbers here are not portable) + +Measured on: Apple M-series (Mac15,12), 8 cores, macOS (darwin/arm64), go 1.27. +`kern.maxfilesperproc=92160`, ephemeral ports `49152–65535` (16 384). + +- **macOS vs Linux differ.** fd caps (`kern.maxfilesperproc` / `ulimit`), TIME_WAIT + and port-range sysctls, and even `accept`/`dial` error strings are not the same. +- **Listen backlog** caps the accept queue independently of your handler: + `kern.ipc.somaxconn` (macOS) / `net.core.somaxconn` (Linux). +- **One process, shared fd limit.** If you run both roles in one process they + share the fd budget and the server usually dies first — use two terminals. + +## Decisions log (what we built, how, and why) + +- **One server binary + one client binary; scenarios are `-mode`, not separate + binaries.** The four dimensions stress the *same* server; splitting them would + duplicate the transport/httptrace/reporting scaffolding four times. Two-process + split so each side gets its own fd/ulimit (same rationale as `99-experiments/body-drain`'s `-role`). +- **Three live counters, not one.** `active` (ConnState) and `in_flight` + (middleware) are tracked separately because "conn open" ≠ "handler running" — + that distinction *is* the idle-vs-slow experiment. +- **`idle` mode uses raw `net.Conn`, not `http.Client`.** The pool reuses conns + faster than we dial against a fast endpoint, so a pool-based idle mode + under-counts non-deterministically (20 000 GETs held only ~8 700). Raw dials + give exactly N — and, as a bonus, expose ephemeral-port exhaustion cleanly. +- **`slowloris` uses raw `net.Conn`.** The stdlib client won't send a deliberately + incomplete request, so we speak HTTP by hand: request line + headers, never the + terminating blank line. +- **`Burn` is deterministic and non-elidable.** A data-dependent integer hash of + `difficulty × 200k` iterations; the handler writes the checksum so the compiler + can't drop the loop. No `runtime.Gosched`, so it genuinely holds a P. Tested by + iteration count, not wall-clock, to stay non-flaky. +- **Zero dependencies.** No `x/net/netutil.LimitListener` for `-max-conns`; a real + server caps concurrency with a semaphore-guarded handler or a wrapped listener, + and we'd rather observe the OS ceiling than hide it. +- **Server timeouts default to OFF.** So the slowloris vulnerability is + demonstrable out of the box; turn `-read-header-timeout` / `-idle-timeout` on to + watch the server reclaim. +- **Network ceilings are not unit-tested.** They're environmental (ulimit, sysctl, + host); only `Burn`, `Percentile`, and `Classify` are tested. The ceilings live + here as pasted evidence. + +## References + +- `net/http.Server` timeout fields — https://pkg.go.dev/net/http#Server +- Cloudflare, "The complete guide to Go net/http timeouts" — https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/ +- `net/http/httptrace` — https://pkg.go.dev/net/http/httptrace +- `http.ConnState` — https://pkg.go.dev/net/http#ConnState +- Slowloris (CWE-400 / low-and-slow) — https://en.wikipedia.org/wiki/Slowloris_(cyber_attack) +- Ephemeral ports & TIME_WAIT — https://www.rfc-editor.org/rfc/rfc6335, `sysctl net.inet.ip.portrange` / `net.ipv4.ip_local_port_range` diff --git a/98-http-server-experiments/client/main.go b/98-http-server-experiments/client/main.go new file mode 100644 index 0000000..dc5d476 --- /dev/null +++ b/98-http-server-experiments/client/main.go @@ -0,0 +1,487 @@ +// Load generator for the experiment server. One binary, four modes, each a +// different way to ask "how many connections, and what gives out first?". +// +// Modes (-mode): +// +// throughput closed loop: -c workers hammer -path for -duration, reusing conns. +// The control — how fast when nothing is broken. Point -path at +// /compute?difficulty=N to watch it plateau near GOMAXPROCS. +// idle open -n keep-alive conns to /static and HOLD them (don't close). +// Measures the cost of an idle connection: fds, goroutines, memory. +// slow fire -c concurrent /sleep requests and hold them all in flight. +// Same conn count as idle, but each pins a running server goroutine. +// slowloris -c raw TCP conns that dribble headers forever and never finish +// the request. Defeated only by the server's -read-header-timeout. +// +// Every mode cross-checks the server's own /stats so you can tell a real server +// ceiling from your own client running out first. Which error you get tells you +// which side gave out: +// +// dial ...: socket: too many open files → THIS process (raise the client's ulimit) +// timeout awaiting response headers → the server can't accept +// connection reset by peer → the server closed on you (or a timeout fired) +// connection refused → nothing listening +// +// Give the client its own fd budget, separate from the server's: +// +// ulimit -Hn 64; ulimit -Sn 64; go run ./98-http-server-experiments/client -mode idle -n 200 +// +// (bash sets soft+hard; zsh only soft — use -Hn. Since Go 1.19 only the hard +// limit really caps you.) +package main + +import ( + "bufio" + "context" + "encoding/json" + "flag" + "fmt" + "io" + "net" + "net/http" + "net/http/httptrace" + "os" + "os/signal" + "runtime" + "strings" + "sync" + "sync/atomic" + "time" + + "gosre-exercises/98-http-server-experiments/loadlib" +) + +// serverStats mirrors the server's /stats JSON for the cross-check. +type serverStats struct { + AcceptedTotal int64 `json:"accepted_total"` + ActiveConns int64 `json:"active_conns"` + InFlight int64 `json:"in_flight"` + Goroutines int `json:"goroutines"` + HeapBytes uint64 `json:"heap_bytes"` +} + +func (s serverStats) String() string { + return fmt.Sprintf("active=%d in_flight=%d goroutines=%d heap=%dMiB", + s.ActiveConns, s.InFlight, s.Goroutines, s.HeapBytes>>20) +} + +// fetchStats reads the server's /stats over a throwaway conn (own transport so it +// never borrows from or perturbs the pool we're measuring). +func fetchStats(base string) (serverStats, error) { + var st serverStats + c := &http.Client{Timeout: 2 * time.Second} + resp, err := c.Get(base + "/stats") + if err != nil { + return st, err + } + defer resp.Body.Close() + err = json.NewDecoder(resp.Body).Decode(&st) + return st, err +} + +// opts bundle the CLI knobs the modes read. +type opts struct { + url, path string + n, c int + duration time.Duration + idlePerHost int + headerTimeout time.Duration + clientTimeout time.Duration + drip time.Duration +} + +// newTransport builds a fresh pool so runs don't share state. idlePerHost must be +// >= the concurrency you want reused, or conns get dropped instead of pooled. +func newTransport(o opts) *http.Transport { + tr := http.DefaultTransport.(*http.Transport).Clone() + tr.MaxIdleConns = o.n + o.c + tr.MaxIdleConnsPerHost = o.idlePerHost + tr.ResponseHeaderTimeout = o.headerTimeout + return tr +} + +// errHistogram counts failures by which side they blame. +type errHistogram struct { + mu sync.Mutex + m map[loadlib.ErrClass]int +} + +func newErrHistogram() *errHistogram { return &errHistogram{m: map[loadlib.ErrClass]int{}} } +func (h *errHistogram) add(err error) { + h.mu.Lock() + h.m[loadlib.Classify(err)]++ + h.mu.Unlock() +} +func (h *errHistogram) String() string { + h.mu.Lock() + defer h.mu.Unlock() + if len(h.m) == 0 { + return "none" + } + s := "" + for cls, n := range h.m { + s += fmt.Sprintf("%s=%d ", cls, n) + } + return s +} + +// ---- mode: throughput ---- + +// runThroughput drives a closed loop: c workers each loop Do()+drain for the +// duration. Conns are reused, so this measures service speed, not dial cost. +func runThroughput(o opts) { + tr := newTransport(o) + defer tr.CloseIdleConnections() + + var reused, fresh, ok, failed atomic.Int64 + trace := &httptrace.ClientTrace{GotConn: func(i httptrace.GotConnInfo) { + if i.Reused { + reused.Add(1) + } else { + fresh.Add(1) + } + }} + ctx := httptrace.WithClientTrace(context.Background(), trace) + deadline := time.Now().Add(o.duration) + client := &http.Client{Transport: tr, Timeout: o.clientTimeout} + hist := newErrHistogram() + target := o.url + o.path + + var wg sync.WaitGroup + var lats []loadlib.Latencies + lats = make([]loadlib.Latencies, o.c) + for w := 0; w < o.c; w++ { + wg.Add(1) + go func(w int) { + defer wg.Done() + for time.Now().Before(deadline) { + start := time.Now() + req, _ := http.NewRequestWithContext(ctx, http.MethodGet, target, nil) + resp, err := client.Do(req) + if err != nil { + failed.Add(1) + hist.add(err) + continue + } + io.Copy(io.Discard, resp.Body) + resp.Body.Close() + lats[w].Add(time.Since(start)) + ok.Add(1) + } + }(w) + } + wg.Wait() + + var all loadlib.Latencies + for i := range lats { + all.Merge(&lats[i]) + } + p50, p99, max := all.Snapshot() + rps := float64(ok.Load()) / o.duration.Seconds() + fmt.Printf(" target : %s (concurrency %d, %v)\n", target, o.c, o.duration) + fmt.Printf(" requests : %d ok, %d failed → %.0f req/s\n", ok.Load(), failed.Load(), rps) + fmt.Printf(" latency : p50=%v p99=%v max=%v\n", p50.Round(time.Microsecond), p99.Round(time.Microsecond), max.Round(time.Microsecond)) + fmt.Printf(" conns : reused=%d fresh=%d\n", reused.Load(), fresh.Load()) + fmt.Printf(" errors : %s\n", hist) +} + +// ---- mode: idle ---- + +// runIdle opens n keep-alive conns and HOLDS them idle, then reports the server's +// view. It uses raw net.Conn rather than http.Client on purpose: with a fast +// endpoint the client's connection pool reuses conns faster than we dial, so n +// concurrent GETs do NOT reliably produce n simultaneous conns. Dialing raw and +// keeping each fd open guarantees exactly n idle keep-alives — the number we want +// to be honest about. Each does one real GET /static (so the server completes a +// request and parks the conn as idle: active=n, in_flight≈0), then we sit on it. +func runIdle(o opts) { + before, _ := fetchStats(o.url) + fmt.Printf(" server before : %s\n", before) + + host := hostPort(o.url) + grBefore := runtime.NumGoroutine() + + var opened, failed atomic.Int64 + hist := newErrHistogram() + conns := make([]net.Conn, 0, o.n) + var mu sync.Mutex + + // Bounded dial concurrency: fast enough to open thousands quickly, without a + // thundering herd of goroutines. Each conn, once its response is drained, + // just sits open — no per-conn goroutine, unlike the transport's 2-per-conn. + sem := make(chan struct{}, 256) + var wg sync.WaitGroup + for i := 0; i < o.n; i++ { + sem <- struct{}{} + wg.Add(1) + go func() { + defer wg.Done() + defer func() { <-sem }() + conn, err := net.DialTimeout("tcp", host, 5*time.Second) + if err != nil { + failed.Add(1) + hist.add(err) + return + } + // One real keep-alive request, response drained, conn left open. + if _, err := fmt.Fprintf(conn, "GET /static HTTP/1.1\r\nHost: %s\r\nConnection: keep-alive\r\n\r\n", host); err != nil { + failed.Add(1) + hist.add(err) + conn.Close() + return + } + br := bufio.NewReader(conn) + resp, err := http.ReadResponse(br, nil) + if err != nil { + failed.Add(1) + hist.add(err) + conn.Close() + return + } + io.Copy(io.Discard, resp.Body) + resp.Body.Close() + mu.Lock() + conns = append(conns, conn) + mu.Unlock() + opened.Add(1) + }() + } + wg.Wait() + time.Sleep(100 * time.Millisecond) // let the server finish parking them as idle + + peak, _ := fetchStats(o.url) + fmt.Printf(" opened : %d/%d idle conns held, %d failed\n", opened.Load(), o.n, failed.Load()) + fmt.Printf(" errors : %s\n", hist) + fmt.Printf(" client goroutines : before=%d after=%d (delta %+d; raw fds, no per-conn goroutine)\n", + grBefore, runtime.NumGoroutine(), runtime.NumGoroutine()-grBefore) + fmt.Printf(" server after open : %s ← active≈n, in_flight≈0: these conns are OPEN but no handler runs\n", peak) + fmt.Printf(" holding %d idle conns for %v — watch the server's active= stay put. Ctrl-C to release early.\n", opened.Load(), o.duration) + + holdFor(o.url, o.duration) + + for _, c := range conns { + c.Close() + } + time.Sleep(200 * time.Millisecond) + after, _ := fetchStats(o.url) + fmt.Printf(" server after close : %s (active should fall back toward its baseline)\n", after) +} + +// ---- mode: slow ---- + +// runSlow fires c concurrent /sleep requests that each stay in flight for the +// whole duration, so c server goroutines are pinned at once. Prints the server's +// in_flight/goroutines each second while they're held. +func runSlow(o opts) { + before, _ := fetchStats(o.url) + fmt.Printf(" server before : %s\n", before) + + // By design every response is slow (it arrives only after the sleep), so we + // must NOT apply a response-header timeout here — it would abort legitimate + // slow responses and lie that the server "can't accept". Instead we bound + // each request with a context deadline a bit past the sleep, so a truly + // stuck request still unblocks wg.Wait() and gets reported. + tr := newTransport(o) + tr.ResponseHeaderTimeout = 0 + defer tr.CloseIdleConnections() + client := &http.Client{Transport: tr} + target := fmt.Sprintf("%s/sleep?d=%s", o.url, o.duration) + reqTimeout := o.duration + 10*time.Second + + var ok, failed atomic.Int64 + hist := newErrHistogram() + + stopPoll := make(chan struct{}) + go pollStats(o.url, stopPoll) + + fmt.Printf(" firing %d concurrent /sleep?d=%v ...\n", o.c, o.duration) + var wg sync.WaitGroup + for i := 0; i < o.c; i++ { + wg.Add(1) + go func() { + defer wg.Done() + ctx, cancel := context.WithTimeout(context.Background(), reqTimeout) + defer cancel() + req, _ := http.NewRequestWithContext(ctx, http.MethodGet, target, nil) + resp, err := client.Do(req) + if err != nil { + failed.Add(1) + hist.add(err) + return + } + io.Copy(io.Discard, resp.Body) + resp.Body.Close() + ok.Add(1) + }() + } + wg.Wait() + close(stopPoll) + + fmt.Printf(" requests : %d ok, %d failed (of %d held in flight)\n", ok.Load(), failed.Load(), o.c) + fmt.Printf(" errors : %s\n", hist) +} + +// ---- mode: slowloris ---- + +// runSlowloris opens c raw TCP conns, sends a request line + one header on each, +// then dribbles a bogus header every -drip and NEVER sends the terminating blank +// line. The handler never starts. A server with -read-header-timeout closes the +// conn after that timeout; we detect it as a read returning EOF/reset and count +// how many the server reclaimed. +func runSlowloris(o opts) { + host := hostPort(o.url) + before, _ := fetchStats(o.url) + fmt.Printf(" server before : %s\n", before) + fmt.Printf(" opening %d slowloris conns to %s (drip every %v) ...\n", o.c, host, o.drip) + + var held, reclaimed, dialFailed atomic.Int64 + hist := newErrHistogram() + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt) + defer stop() + + var wg sync.WaitGroup + for i := 0; i < o.c; i++ { + conn, err := net.DialTimeout("tcp", host, 5*time.Second) + if err != nil { + dialFailed.Add(1) + hist.add(err) + continue + } + held.Add(1) + wg.Add(1) + go func(conn net.Conn) { + defer wg.Done() + defer conn.Close() + // Incomplete request: a valid start line and Host, but no blank line. + fmt.Fprintf(conn, "GET /static HTTP/1.1\r\nHost: %s\r\n", host) + dripTick := time.NewTicker(o.drip) + defer dripTick.Stop() + // A reader goroutine turns a server-side close into a signal. + closed := make(chan struct{}) + go func() { + buf := make([]byte, 1) + conn.Read(buf) // blocks until the server closes → EOF/reset + close(closed) + }() + for { + select { + case <-ctx.Done(): + return + case <-closed: + reclaimed.Add(1) + return + case <-dripTick.C: + // One more bogus header — still no terminating blank line. + if _, err := fmt.Fprint(conn, "X-Keep: alive\r\n"); err != nil { + reclaimed.Add(1) + return + } + } + } + }(conn) + } + + // Report the server's view periodically while conns are held. + stopPoll := make(chan struct{}) + go pollStats(o.url, stopPoll) + select { + case <-ctx.Done(): + case <-time.After(o.duration): + } + stop() // stop the drip loops + close(stopPoll) // stop polling + wg.Wait() + + after, _ := fetchStats(o.url) + fmt.Printf(" held : %d conns opened, %d dial-failed\n", held.Load(), dialFailed.Load()) + fmt.Printf(" reclaimed by server: %d (server closed the conn mid-headers → -read-header-timeout worked)\n", reclaimed.Load()) + fmt.Printf(" errors : %s\n", hist) + fmt.Printf(" server after : %s\n", after) + if reclaimed.Load() == 0 { + fmt.Println(" (0 reclaimed: the server has no -read-header-timeout — these conns would stick until fds run out)") + } +} + +// ---- helpers ---- + +// pollStats prints the server's /stats once a second until stop is closed. +func pollStats(base string, stop <-chan struct{}) { + tick := time.NewTicker(time.Second) + defer tick.Stop() + for { + select { + case <-stop: + return + case <-tick.C: + if st, err := fetchStats(base); err == nil { + fmt.Printf(" server %s %s\n", time.Now().Format("15:04:05"), st) + } + } + } +} + +// holdFor blocks for d or until Ctrl-C, printing the server's view each second so +// you can watch the held conns sit there. +func holdFor(base string, d time.Duration) { + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt) + defer stop() + stopPoll := make(chan struct{}) + go pollStats(base, stopPoll) + select { + case <-ctx.Done(): + case <-time.After(d): + } + close(stopPoll) +} + +// hostPort turns http://host:port[/...] into host:port for net.Dial. +func hostPort(rawurl string) string { + s := strings.TrimPrefix(rawurl, "http://") + if i := strings.IndexByte(s, '/'); i >= 0 { + s = s[:i] + } + return s +} + +func main() { + mode := flag.String("mode", "throughput", "throughput | idle | slow | slowloris") + url := flag.String("url", "http://127.0.0.1:8098", "server base URL") + path := flag.String("path", "/static", "throughput: path to hit, e.g. /static or /compute?difficulty=50") + n := flag.Int("n", 1000, "idle: number of keep-alive conns to open and hold") + c := flag.Int("c", 50, "throughput/slow/slowloris: concurrency (workers / held conns)") + duration := flag.Duration("duration", 10*time.Second, "how long to run / hold") + idlePerHost := flag.Int("idle-per-host", 0, "Transport.MaxIdleConnsPerHost (0 = auto: n for idle, c otherwise)") + headerTimeout := flag.Duration("header-timeout", 3*time.Second, "Transport.ResponseHeaderTimeout (0 = none)") + clientTimeout := flag.Duration("timeout", 0, "http.Client.Timeout (0 = none; a timeout force-closes stuck conns and hides holds)") + drip := flag.Duration("drip", 10*time.Second, "slowloris: interval between dribbled header bytes") + flag.Parse() + + o := opts{ + url: *url, path: *path, n: *n, c: *c, duration: *duration, + idlePerHost: *idlePerHost, headerTimeout: *headerTimeout, + clientTimeout: *clientTimeout, drip: *drip, + } + // Auto-size the idle pool so conns are actually reused/held, not dropped. + if o.idlePerHost == 0 { + if *mode == "idle" { + o.idlePerHost = o.n + } else { + o.idlePerHost = o.c + } + } + + fmt.Printf("== mode=%s target=%s ==\n", *mode, o.url) + switch *mode { + case "throughput": + runThroughput(o) + case "idle": + runIdle(o) + case "slow": + runSlow(o) + case "slowloris": + runSlowloris(o) + default: + fmt.Fprintf(os.Stderr, "unknown -mode %q (throughput|idle|slow|slowloris)\n", *mode) + os.Exit(2) + } +} diff --git a/98-http-server-experiments/loadlib/loadlib.go b/98-http-server-experiments/loadlib/loadlib.go new file mode 100644 index 0000000..67f1d03 --- /dev/null +++ b/98-http-server-experiments/loadlib/loadlib.go @@ -0,0 +1,164 @@ +// Package loadlib holds the small, pure pieces the server and load client share +// and that are worth a unit test: a deterministic CPU burn, a percentile helper, +// and an error classifier that says which side of a connection ran out. +// +// Everything network-shaped lives in the two binaries; those ceilings are +// environmental (ulimit/sysctl/host) and are documented as pasted evidence in +// the folder README rather than asserted in a test. +package loadlib + +import ( + "errors" + "net" + "sort" + "strings" + "time" +) + +// burnIters is the inner-loop count per unit of difficulty. Kept as a named +// constant so the test can reason about total iterations without a wall clock. +const burnIters = 200_000 + +// Iterations returns how many inner-loop steps Burn(difficulty) will run. +// It is the seam the test asserts on: cost is linear in difficulty, no timing. +func Iterations(difficulty int) int { + if difficulty < 1 { + difficulty = 1 + } + return difficulty * burnIters +} + +// Burn occupies a CPU for an amount of work proportional to difficulty and +// returns a checksum. The caller must use the return value (write it to the +// response) so the compiler cannot elide the loop. +// +// The mix is a cheap integer hash with a data dependency across iterations, so +// each step depends on the last and the optimizer can't collapse the loop or +// vectorize it away. There is deliberately no runtime.Gosched inside: Burn is +// meant to genuinely hold a P, which is the whole point of the /compute probe +// (how many CPU-bound requests before GOMAXPROCS saturates). +func Burn(difficulty int) uint64 { + var x uint64 = 1469598103934665603 // FNV-1a offset basis, just a seed + n := Iterations(difficulty) + for i := 0; i < n; i++ { + x ^= uint64(i) + x *= 1099511628211 // FNV-1a prime + x = x<<13 | x>>51 // rotate so high bits feed back in + } + return x +} + +// Percentile returns the p-th percentile (p in [0,1]) of an already-sorted +// slice. Empty → 0, p<=0 → first, p>=1 → last. Nearest-rank, no interpolation — +// good enough for "orders of magnitude" latency reporting. +func Percentile(sorted []time.Duration, p float64) time.Duration { + if len(sorted) == 0 { + return 0 + } + if p < 0 { + p = 0 + } + if p > 1 { + p = 1 + } + i := int(float64(len(sorted)-1) * p) + return sorted[i] +} + +// Latencies accumulates request durations and reports percentiles, keeping the +// client's main loop thin. Not safe for concurrent use — each worker should own +// one and they get merged, or a single collector goroutine owns it. +type Latencies struct { + d []time.Duration +} + +func (l *Latencies) Add(d time.Duration) { l.d = append(l.d, d) } +func (l *Latencies) Len() int { return len(l.d) } + +// Merge folds another accumulator's samples in (for per-worker collection). +func (l *Latencies) Merge(o *Latencies) { l.d = append(l.d, o.d...) } + +// Snapshot sorts and returns p50, p99, and max. Sorts in place; call once at end. +func (l *Latencies) Snapshot() (p50, p99, max time.Duration) { + sort.Slice(l.d, func(i, j int) bool { return l.d[i] < l.d[j] }) + return Percentile(l.d, 0.50), Percentile(l.d, 0.99), Percentile(l.d, 1) +} + +// ErrClass names which side of the connection gave out. It turns the body-drain +// "which error string means what" table into something the client can count. +type ErrClass int + +const ( + ErrNone ErrClass = iota // no error + ErrClientFdLimit // this process ran out of fds (dial: too many open files) + ErrClientPortLimit // this process ran out of ephemeral ports (can't assign requested address) + ErrServerCantAccept // server can't accept (out of fds / backlog full) — we time out + ErrTimeout // request/response timed out (ambiguous, but usually the server) + ErrConnReset // peer reset/closed the connection + ErrRefused // nothing listening / server gone + ErrOther +) + +func (c ErrClass) String() string { + switch c { + case ErrNone: + return "none" + case ErrClientFdLimit: + return "client-fd-limit" + case ErrClientPortLimit: + return "client-port-limit" + case ErrServerCantAccept: + return "server-cant-accept" + case ErrTimeout: + return "timeout" + case ErrConnReset: + return "conn-reset" + case ErrRefused: + return "conn-refused" + default: + return "other" + } +} + +// Classify maps a net/http client error to the side that most likely caused it. +// It leans on error strings because the kernel/runtime messages are the only +// distinguishing signal for several of these; the checks are ordered most- to +// least-specific so, e.g., "too many open files" wins over the generic timeout. +func Classify(err error) ErrClass { + if err == nil { + return ErrNone + } + msg := strings.ToLower(err.Error()) + + switch { + case strings.Contains(msg, "too many open files"): + // A dial failing this way is THIS process out of fds. (A server out of + // fds fails on accept and never produces this string on the client.) + return ErrClientFdLimit + case strings.Contains(msg, "assign requested address"): + // EADDRNOTAVAIL: no ephemeral port left for a fresh 4-tuple to this + // server. A client-side wall, often hit before the fd limit. + return ErrClientPortLimit + case strings.Contains(msg, "connection refused"): + return ErrRefused + case strings.Contains(msg, "connection reset") || strings.Contains(msg, "broken pipe") || + strings.Contains(msg, "eof"): + return ErrConnReset + } + + // "awaiting response headers" / a dial timeout usually means the server + // accepted us into its backlog but can't service (or can't accept at all), + // so we sit waiting. Treat a timeout as a server-can't-keep-up signal, but + // keep a generic ErrTimeout for the plain deadline case. + var ne net.Error + if errors.As(err, &ne) && ne.Timeout() { + if strings.Contains(msg, "awaiting response headers") { + return ErrServerCantAccept + } + return ErrTimeout + } + if strings.Contains(msg, "timeout") || strings.Contains(msg, "deadline exceeded") { + return ErrTimeout + } + return ErrOther +} diff --git a/98-http-server-experiments/loadlib/loadlib_test.go b/98-http-server-experiments/loadlib/loadlib_test.go new file mode 100644 index 0000000..1c99c09 --- /dev/null +++ b/98-http-server-experiments/loadlib/loadlib_test.go @@ -0,0 +1,147 @@ +package loadlib + +import ( + "errors" + "fmt" + "net" + "testing" + "time" +) + +func TestBurnDeterministic(t *testing.T) { + // Same difficulty must give the same checksum: the /compute endpoint returns + // it, and a flaky checksum would mean a non-deterministic loop. + for _, d := range []int{1, 3, 10} { + if a, b := Burn(d), Burn(d); a != b { + t.Errorf("Burn(%d) not deterministic: %d != %d", d, a, b) + } + } +} + +func TestBurnDifficultyChangesResult(t *testing.T) { + // Different difficulties should not collide (they run different iteration + // counts); a collision here would suggest the loop body ignores i. + if Burn(1) == Burn(2) { + t.Error("Burn(1) == Burn(2): difficulty has no effect on the checksum") + } +} + +func TestIterationsLinearAndFloored(t *testing.T) { + if got := Iterations(2); got != 2*burnIters { + t.Errorf("Iterations(2) = %d, want %d", got, 2*burnIters) + } + // difficulty < 1 is clamped to 1 so /compute?difficulty=0 still does work. + for _, d := range []int{0, -5} { + if got := Iterations(d); got != burnIters { + t.Errorf("Iterations(%d) = %d, want %d (floor of 1)", d, got, burnIters) + } + } + // Cost is monotonic in difficulty. + if Iterations(3) <= Iterations(2) { + t.Error("Iterations not increasing with difficulty") + } +} + +func TestPercentileEdges(t *testing.T) { + if got := Percentile(nil, 0.5); got != 0 { + t.Errorf("Percentile(empty) = %v, want 0", got) + } + s := []time.Duration{10, 20, 30, 40, 50} + cases := []struct { + p float64 + want time.Duration + }{ + {-1, 10}, // clamped to first + {0, 10}, // first + {0.5, 30}, // middle + {1, 50}, // last + {2, 50}, // clamped to last + } + for _, c := range cases { + if got := Percentile(s, c.p); got != c.want { + t.Errorf("Percentile(%v, %v) = %v, want %v", s, c.p, got, c.want) + } + } + if got := Percentile([]time.Duration{99}, 0.9); got != 99 { + t.Errorf("Percentile(single) = %v, want 99", got) + } +} + +func TestLatenciesSnapshot(t *testing.T) { + var l Latencies + // Add out of order; Snapshot must sort before reporting. + for _, d := range []time.Duration{50, 10, 30, 40, 20} { + l.Add(d * time.Millisecond) + } + if l.Len() != 5 { + t.Fatalf("Len = %d, want 5", l.Len()) + } + p50, p99, max := l.Snapshot() + if p50 != 30*time.Millisecond { + t.Errorf("p50 = %v, want 30ms", p50) + } + if max != 50*time.Millisecond { + t.Errorf("max = %v, want 50ms", max) + } + // Nearest-rank: index int(4*0.99)=3 → 40ms for this 5-sample set. + if p99 != 40*time.Millisecond { + t.Errorf("p99 = %v, want 40ms", p99) + } +} + +func TestLatenciesMerge(t *testing.T) { + var a, b Latencies + a.Add(10 * time.Millisecond) + b.Add(20 * time.Millisecond) + a.Merge(&b) + if a.Len() != 2 { + t.Fatalf("after merge Len = %d, want 2", a.Len()) + } +} + +func TestClassify(t *testing.T) { + timeoutErr := &net.OpError{ + Op: "dial", + Err: timeoutError{}, + } + headerTimeout := fmt.Errorf("net/http: timeout awaiting response headers") + + cases := []struct { + name string + err error + want ErrClass + }{ + {"nil", nil, ErrNone}, + {"client fds", errors.New("dial tcp 127.0.0.1:8098: socket: too many open files"), ErrClientFdLimit}, + {"client ports", errors.New("dial tcp 127.0.0.1:8098: connect: can't assign requested address"), ErrClientPortLimit}, + {"refused", errors.New("dial tcp 127.0.0.1:8098: connect: connection refused"), ErrRefused}, + {"reset", errors.New("read tcp: connection reset by peer"), ErrConnReset}, + {"eof", errors.New("unexpected EOF"), ErrConnReset}, + {"plain net timeout", timeoutErr, ErrTimeout}, + {"server cant accept", headerTimeoutNetErr{}, ErrServerCantAccept}, + {"header timeout string", headerTimeout, ErrTimeout}, // not a net.Error, but has "timeout" + {"deadline", errors.New("context deadline exceeded"), ErrTimeout}, + {"other", errors.New("something weird"), ErrOther}, + } + for _, c := range cases { + if got := Classify(c.err); got != c.want { + t.Errorf("%s: Classify(%v) = %v, want %v", c.name, c.err, got, c.want) + } + } +} + +// timeoutError is a minimal net.Error whose Timeout() reports true, used to build +// a synthetic *net.OpError the way the runtime would on a real dial timeout. +type timeoutError struct{} + +func (timeoutError) Error() string { return "i/o timeout" } +func (timeoutError) Timeout() bool { return true } +func (timeoutError) Temporary() bool { return true } + +// headerTimeoutNetErr is a net.Error whose message is the exact one net/http +// produces when ResponseHeaderTimeout fires — the "server can't accept" signal. +type headerTimeoutNetErr struct{} + +func (headerTimeoutNetErr) Error() string { return "net/http: timeout awaiting response headers" } +func (headerTimeoutNetErr) Timeout() bool { return true } +func (headerTimeoutNetErr) Temporary() bool { return true } diff --git a/98-http-server-experiments/server/main.go b/98-http-server-experiments/server/main.go new file mode 100644 index 0000000..c13d809 --- /dev/null +++ b/98-http-server-experiments/server/main.go @@ -0,0 +1,219 @@ +// Experiment subject — a deliberately simple HTTP server whose only job is to +// be measured. How many connections can it hold, and what breaks first? +// +// Endpoints: +// +// GET /static fixed tiny body, no per-request work (the baseline) +// GET /compute?difficulty=N burns CPU ∝ N, returns the checksum (CPU-bound probe) +// GET /sleep?d=500ms sleeps d, honoring client disconnect (in-flight holder) +// GET /healthz liveness +// GET /stats JSON: accepted_total, active_conns, in_flight, goroutines, heap_bytes +// +// Observability. We separate three numbers that are easy to conflate: +// +// accepted_total every TCP conn ever accepted (only grows) +// active_conns conns open right now (idle keep-alive OR busy) — via ConnState +// in_flight handlers actually running right now — via middleware +// +// An idle keep-alive conn counts in active_conns but not in_flight; a running +// /sleep counts in both. That gap is the whole point of the idle-vs-slow contrast. +// +// Timeout flags are the naive-vs-hardened knobs. They default to OFF so the +// slowloris experiment works out of the box; turn them on to watch the server +// reclaim connections: +// +// -read-header-timeout 3s drop a conn that dawdles sending headers (kills slowloris) +// -idle-timeout 30s drop a keep-alive conn that goes quiet +// -read-timeout / -write-timeout whole-request deadlines +// +// Run (terminal 1), then drive it with ./client (terminal 2): +// +// go run ./98-http-server-experiments/server +// go run ./98-http-server-experiments/server -read-header-timeout 3s -idle-timeout 30s # hardened +// +// Give the server its own fd budget so YOU decide when it runs out (not the OS +// default of a few hundred / a few thousand). bash sets both soft and hard: +// +// ulimit -Hn 4096; ulimit -Sn 4096; go run ./98-http-server-experiments/server +// +// (Since Go 1.19 the runtime raises the soft RLIMIT_NOFILE to the hard limit at +// startup, so only the hard limit really caps you. zsh's `ulimit -n` sets only +// the soft one — use `-Hn`.) +package main + +import ( + "context" + "encoding/json" + "flag" + "fmt" + "net" + "net/http" + "os" + "os/signal" + "runtime" + "strconv" + "sync/atomic" + "time" + + "gosre-exercises/98-http-server-experiments/loadlib" +) + +// server is the subject under test: an http.Server plus the three live counters. +type server struct { + srv *http.Server + ln net.Listener + acceptedTot atomic.Int64 // every conn ever accepted + activeConns atomic.Int64 // conns open right now + inFlight atomic.Int64 // handlers running right now +} + +func (s *server) handleStatic(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "text/plain") + fmt.Fprint(w, "ok\n") +} + +// handleCompute burns CPU proportional to ?difficulty= and writes the checksum +// back. Writing it matters: it forces the burn to actually happen (the compiler +// can't prove the result is unused) and lets the client confirm the work ran. +func (s *server) handleCompute(w http.ResponseWriter, r *http.Request) { + difficulty := 1 + if v := r.URL.Query().Get("difficulty"); v != "" { + if n, err := strconv.Atoi(v); err == nil { + difficulty = n + } + } + sum := loadlib.Burn(difficulty) + w.Header().Set("Content-Type", "text/plain") + fmt.Fprintf(w, "difficulty=%d checksum=%d\n", difficulty, sum) +} + +// handleSleep keeps the request in flight for ?d= but returns early if the +// client disconnects or a server timeout fires — otherwise a client that hangs +// up would leave the goroutine sleeping to full term, hiding the reclaim. +func (s *server) handleSleep(w http.ResponseWriter, r *http.Request) { + d := 500 * time.Millisecond + if v := r.URL.Query().Get("d"); v != "" { + if parsed, err := time.ParseDuration(v); err == nil { + d = parsed + } + } + select { + case <-time.After(d): + w.Header().Set("Content-Type", "text/plain") + fmt.Fprintf(w, "slept %v\n", d) + case <-r.Context().Done(): + // Client went away or a timeout fired; nothing to write. + } +} + +type statsSnapshot struct { + AcceptedTotal int64 `json:"accepted_total"` + ActiveConns int64 `json:"active_conns"` + InFlight int64 `json:"in_flight"` + Goroutines int `json:"goroutines"` + HeapBytes uint64 `json:"heap_bytes"` +} + +func (s *server) snapshot() statsSnapshot { + var m runtime.MemStats + runtime.ReadMemStats(&m) + return statsSnapshot{ + AcceptedTotal: s.acceptedTot.Load(), + ActiveConns: s.activeConns.Load(), + InFlight: s.inFlight.Load(), + Goroutines: runtime.NumGoroutine(), + HeapBytes: m.HeapInuse, + } +} + +func (s *server) handleStats(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(s.snapshot()) +} + +// countInFlight wraps a handler so in_flight tracks handlers actually running, +// as distinct from conns merely open (which ConnState already counts). +func (s *server) countInFlight(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + s.inFlight.Add(1) + defer s.inFlight.Add(-1) + next.ServeHTTP(w, r) + }) +} + +func main() { + addr := flag.String("addr", "127.0.0.1:8098", "address to listen on") + readHeaderTimeout := flag.Duration("read-header-timeout", 0, "Server.ReadHeaderTimeout (0 = off; set 3s to defeat slowloris)") + readTimeout := flag.Duration("read-timeout", 0, "Server.ReadTimeout (0 = off; whole-request read deadline)") + writeTimeout := flag.Duration("write-timeout", 0, "Server.WriteTimeout (0 = off)") + idleTimeout := flag.Duration("idle-timeout", 0, "Server.IdleTimeout (0 = off; set 30s to reclaim quiet keep-alives)") + maxHeaderBytes := flag.Int("max-header-bytes", http.DefaultMaxHeaderBytes, "Server.MaxHeaderBytes (1<<20 default)") + flag.Parse() + + // NOTE: no -max-conns. Capping concurrent connections in the stdlib means + // wrapping the Listener (x/net/netutil.LimitListener) or a semaphore in the + // handler; we keep this zero-dependency and instead observe the ceiling the + // OS imposes (fds/backlog). See the README for the stdlib-only pattern. + + ln, err := net.Listen("tcp", *addr) + if err != nil { + fmt.Fprintln(os.Stderr, "listen:", err) + os.Exit(1) + } + s := &server{ln: ln} + + mux := http.NewServeMux() + mux.HandleFunc("/static", s.handleStatic) + mux.HandleFunc("/compute", s.handleCompute) + mux.HandleFunc("/sleep", s.handleSleep) + mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "ok") }) + mux.HandleFunc("/stats", s.handleStats) + + s.srv = &http.Server{ + Handler: s.countInFlight(mux), + ReadHeaderTimeout: *readHeaderTimeout, + ReadTimeout: *readTimeout, + WriteTimeout: *writeTimeout, + IdleTimeout: *idleTimeout, + MaxHeaderBytes: *maxHeaderBytes, + ConnState: func(c net.Conn, st http.ConnState) { + switch st { + case http.StateNew: + s.acceptedTot.Add(1) + s.activeConns.Add(1) + case http.StateClosed, http.StateHijacked: + s.activeConns.Add(-1) + } + }, + } + + go func() { + if err := s.srv.Serve(ln); err != nil && err != http.ErrServerClosed { + fmt.Fprintln(os.Stderr, "serve:", err) + } + }() + + fmt.Printf("serving on http://%s (timeouts: readHeader=%v read=%v write=%v idle=%v)\n", + ln.Addr(), *readHeaderTimeout, *readTimeout, *writeTimeout, *idleTimeout) + fmt.Println("point the client at it: go run ./98-http-server-experiments/client -mode idle -n 2000") + + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt) + defer stop() + tick := time.NewTicker(time.Second) + defer tick.Stop() + for { + select { + case <-ctx.Done(): + fmt.Println("\nshutting down") + shutCtx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + s.srv.Shutdown(shutCtx) + return + case <-tick.C: + snap := s.snapshot() + fmt.Printf("%s accepted_total=%d active=%d in_flight=%d goroutines=%d heap=%dMiB\n", + time.Now().Format("15:04:05"), snap.AcceptedTotal, snap.ActiveConns, + snap.InFlight, snap.Goroutines, snap.HeapBytes>>20) + } + } +}