Files

Jan Novak 9a69ba946f Add URL-level concurrency (-c flag, Step 7)

Probe multiple URLs in parallel with a bounded worker pool; the N samples
of each URL remain sequential to preserve accurate min/avg/max statistics.
Default auto-concurrency is min(numURLs, 8); -c 1 restores serial mode.
Output is always buffered and printed in original input order. Verified
clean with go test -race.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-07-01 01:33:03 +02:00

2.6 KiB

Raw Blame History

Step 7 — Concurrency

latprobe now probes multiple URLs in parallel, cutting wall-clock time for large runs without sacrificing measurement accuracy.

How it works

Axis	Behaviour
Across URLs	Parallel — up to `-c` workers run simultaneously
Samples of one URL (`-n`)	Sequential — always serial within a URL

Keeping samples sequential ensures that min/avg/max statistics for a single URL reflect genuine latency variability, not artificial load created by firing multiple requests at the same server at once.

Output is always printed in input order, regardless of which URL finishes first.

Flag

-c, --concurrency int   Max URLs probed in parallel (0 = auto, default)

Value	Behaviour
`0` (default)	Auto: `min(numURLs, 8)` — scales with workload, caps at 8
`1`	Fully serial — identical to the old behaviour; best for precision
`N > 1`	Explicit worker cap

When to use serial mode

For the most accurate latency numbers — especially when comparing sites — use -c 1. Concurrent probes share your local NIC, DNS resolver, and CPU, which can inflate timings on slower machines or fast batch runs.

# High fidelity: one URL at a time
./go/latprobe -c 1 -n 10 https://www.google.com https://www.bbc.co.uk

# Speed: all four probed in parallel (auto default would do this anyway)
./go/latprobe -c 4 -n 10 https://www.google.com https://www.bbc.co.uk \
  https://www.lemonde.fr https://www.abc.net.au

Example: speedup in practice

Five sites, 5 samples each — sequential vs parallel:

# Serial (-c 1): ~wall time ≈ sum of all RTTs × 5
time ./go/latprobe -c 1 -n 5 \
  https://www.google.com https://www.bbc.co.uk \
  https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
# real ~1m15s  (5 sites × ~15s sequential)

# Parallel (default): ~wall time ≈ slowest single URL × 5
time ./go/latprobe -n 5 \
  https://www.google.com https://www.bbc.co.uk \
  https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
# real ~18s  (all 5 measured at once, gated by the slowest)

Exit codes

Exit codes work exactly as before: when multiple failure types occur the highest code wins. Concurrency does not change this — exit codes are accumulated after all workers complete.

Race safety

The implementation uses a semaphore channel (chan struct{}) and sync.WaitGroup with each goroutine writing only its own indexed result slot, so there is no shared mutable state. http.DefaultClient is safe for concurrent use. Verified clean with go test -race (make go-test-race).

2.6 KiB Raw Blame History Unescape Escape