Add URL-level concurrency (-c flag, Step 7)
Probe multiple URLs in parallel with a bounded worker pool; the N samples of each URL remain sequential to preserve accurate min/avg/max statistics. Default auto-concurrency is min(numURLs, 8); -c 1 restores serial mode. Output is always buffered and printed in original input order. Verified clean with go test -race. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
76
docs/usage/step-7-concurrency.md
Normal file
76
docs/usage/step-7-concurrency.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# Step 7 — Concurrency
|
||||
|
||||
`latprobe` now probes multiple URLs in parallel, cutting wall-clock time for
|
||||
large runs without sacrificing measurement accuracy.
|
||||
|
||||
## How it works
|
||||
|
||||
| Axis | Behaviour |
|
||||
|------|-----------|
|
||||
| Across URLs | **Parallel** — up to `-c` workers run simultaneously |
|
||||
| Samples of one URL (`-n`) | **Sequential** — always serial within a URL |
|
||||
|
||||
Keeping samples sequential ensures that min/avg/max statistics for a single URL
|
||||
reflect genuine latency variability, not artificial load created by firing
|
||||
multiple requests at the same server at once.
|
||||
|
||||
Output is always printed in **input order**, regardless of which URL finishes
|
||||
first.
|
||||
|
||||
## Flag
|
||||
|
||||
```
|
||||
-c, --concurrency int Max URLs probed in parallel (0 = auto, default)
|
||||
```
|
||||
|
||||
| Value | Behaviour |
|
||||
|-------|-----------|
|
||||
| `0` (default) | Auto: `min(numURLs, 8)` — scales with workload, caps at 8 |
|
||||
| `1` | Fully serial — identical to the old behaviour; best for precision |
|
||||
| `N > 1` | Explicit worker cap |
|
||||
|
||||
## When to use serial mode
|
||||
|
||||
For the most accurate latency numbers — especially when comparing sites —
|
||||
use `-c 1`. Concurrent probes share your local NIC, DNS resolver, and CPU,
|
||||
which can inflate timings on slower machines or fast batch runs.
|
||||
|
||||
```sh
|
||||
# High fidelity: one URL at a time
|
||||
./go/latprobe -c 1 -n 10 https://www.google.com https://www.bbc.co.uk
|
||||
|
||||
# Speed: all four probed in parallel (auto default would do this anyway)
|
||||
./go/latprobe -c 4 -n 10 https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.abc.net.au
|
||||
```
|
||||
|
||||
## Example: speedup in practice
|
||||
|
||||
Five sites, 5 samples each — sequential vs parallel:
|
||||
|
||||
```sh
|
||||
# Serial (-c 1): ~wall time ≈ sum of all RTTs × 5
|
||||
time ./go/latprobe -c 1 -n 5 \
|
||||
https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
|
||||
# real ~1m15s (5 sites × ~15s sequential)
|
||||
|
||||
# Parallel (default): ~wall time ≈ slowest single URL × 5
|
||||
time ./go/latprobe -n 5 \
|
||||
https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
|
||||
# real ~18s (all 5 measured at once, gated by the slowest)
|
||||
```
|
||||
|
||||
## Exit codes
|
||||
|
||||
Exit codes work exactly as before: when multiple failure types occur the
|
||||
**highest** code wins. Concurrency does not change this — exit codes are
|
||||
accumulated after all workers complete.
|
||||
|
||||
## Race safety
|
||||
|
||||
The implementation uses a semaphore channel (`chan struct{}`) and
|
||||
`sync.WaitGroup` with each goroutine writing only its own indexed result slot,
|
||||
so there is no shared mutable state. `http.DefaultClient` is safe for concurrent
|
||||
use. Verified clean with `go test -race` (`make go-test-race`).
|
||||
Reference in New Issue
Block a user