Add URL-level concurrency (-c flag, Step 7)
Probe multiple URLs in parallel with a bounded worker pool; the N samples of each URL remain sequential to preserve accurate min/avg/max statistics. Default auto-concurrency is min(numURLs, 8); -c 1 restores serial mode. Output is always buffered and printed in original input order. Verified clean with go test -race. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
18
CHANGELOG.md
18
CHANGELOG.md
@@ -4,6 +4,24 @@ All completed features are logged here in reverse-chronological order.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-01 01:23 — Concurrency (Go, Step 7)
|
||||
|
||||
- Added `-c`/`--concurrency` flag: max URLs probed in parallel (0 = auto)
|
||||
- Default auto-concurrency: `min(numURLs, 8)` — scales with workload, caps at 8
|
||||
- Parallelism is **across URLs only**; the N samples of each URL stay sequential
|
||||
to preserve the accuracy of per-URL min/avg/max statistics
|
||||
- Output buffered and printed in original input order (deterministic for both
|
||||
text and JSON), exit-code accumulation unchanged
|
||||
- Implementation: semaphore channel + `sync.WaitGroup`, each goroutine writes
|
||||
only its own indexed result slot — verified clean with `go test -race`
|
||||
- Added `make go-test-race` target to the Makefile
|
||||
- 3 new tests: `TestRunConcurrentOrder`, `TestConcurrentWorstCode`,
|
||||
`TestConcurrentJSONOrder`
|
||||
- Updated runtime estimates in `docs/custom-usage-examples.md`
|
||||
- User doc: `docs/usage/step-7-concurrency.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-01 01:05 — Root Makefile
|
||||
|
||||
- Namespaced targets: `go-build`, `go-run`, `go-test`, `go-test-verbose`, `go-check`, `go-cover`, `go-fmt`, `go-vet`, `go-tidy`, `go-install`, `go-lint`, `go-clean`
|
||||
|
||||
4
Makefile
4
Makefile
@@ -53,6 +53,10 @@ go-test: ## go: run all tests
|
||||
go-test-verbose: ## go: run all tests with per-case output
|
||||
go -C $(GO_DIR) test -v ./...
|
||||
|
||||
.PHONY: go-test-race
|
||||
go-test-race: ## go: run tests with the race detector enabled
|
||||
go -C $(GO_DIR) test -race ./...
|
||||
|
||||
.PHONY: go-check
|
||||
go-check: go-fmt go-vet go-test ## go: fmt + vet + test (pre-commit gate)
|
||||
|
||||
|
||||
16
README.md
16
README.md
@@ -28,12 +28,16 @@ Both implementations produce identical CLI behaviour and output formats.
|
||||
latprobe [flags] <url> [url ...]
|
||||
|
||||
Flags:
|
||||
-n, --count int Number of requests per URL (default 1)
|
||||
--json Output results as JSON instead of text
|
||||
-n, --count int Number of requests per URL (default 1)
|
||||
-c, --concurrency int Max URLs probed in parallel, 0 = auto (default min(numURLs,8))
|
||||
--timeout duration Request timeout, e.g. 10s, 500ms (default 10s)
|
||||
--fail Exit non-zero on HTTP status >= 400 (exit code 6)
|
||||
--json Output results as JSON instead of text
|
||||
|
||||
Examples:
|
||||
latprobe https://example.com
|
||||
latprobe -n 5 https://example.com https://www.google.com
|
||||
latprobe -c 1 -n 10 https://example.com # serial, most accurate
|
||||
latprobe --json https://example.com | jq .
|
||||
```
|
||||
|
||||
@@ -96,6 +100,7 @@ https://example.com (5 samples)
|
||||
| 4 | `--json` output flag | ✅ Done |
|
||||
| 5 | Failure handling — `--timeout`, `--fail`, distinct exit codes, partial timing | ✅ Done |
|
||||
| 6 | Integration tests — in-process matrix + subprocess smoke tests | ✅ Done |
|
||||
| 7 | Concurrency — `-c` worker pool across URLs; samples stay serial per URL | ✅ Done |
|
||||
|
||||
### Python
|
||||
|
||||
@@ -116,9 +121,10 @@ make test # run all tests
|
||||
make check # fmt + vet + test (pre-commit gate)
|
||||
make go-run ARGS="https://example.com"
|
||||
make go-test-verbose
|
||||
make go-cover # coverage report → go/coverage.html
|
||||
make go-lint # golangci-lint
|
||||
make clean # remove build artifacts
|
||||
make go-cover # coverage report → go/coverage.html
|
||||
make go-test-race # run tests with race detector
|
||||
make go-lint # golangci-lint
|
||||
make clean # remove build artifacts
|
||||
```
|
||||
|
||||
See [`docs/usage/makefile.md`](docs/usage/makefile.md) for the full target reference.
|
||||
|
||||
@@ -7,7 +7,11 @@ make build
|
||||
# binary is now at go/latprobe
|
||||
```
|
||||
|
||||
Estimated runtimes assume a typical home broadband connection.
|
||||
**Concurrency note:** Since Step 7, `latprobe` probes URLs in parallel by default
|
||||
(`min(numURLs, 8)` workers). Estimated runtimes below reflect this — they are
|
||||
roughly _slowest-single-URL × samples_ rather than the sum across all URLs.
|
||||
Use `-c 1` to restore serial execution for the most accurate per-phase numbers.
|
||||
|
||||
The timeout flag is shortened to `--timeout 3s` wherever non-routable IPs are
|
||||
used, so each timed-out sample fails in 3 s instead of the default 10 s.
|
||||
|
||||
@@ -32,7 +36,7 @@ used, so each timed-out sample fails in 3 s instead of the default 10 s.
|
||||
https://www.timeslive.co.za
|
||||
```
|
||||
|
||||
*Estimated runtime: ~30 s — shows per-phase breakdown once per site.*
|
||||
*Estimated runtime: ~5–8 s — all 10 sites measured in parallel; shows per-phase breakdown once per site.*
|
||||
|
||||
---
|
||||
|
||||
@@ -55,7 +59,7 @@ Same 10 sites, 10 requests each — reveals real latency distribution (min/avg/m
|
||||
https://www.timeslive.co.za
|
||||
```
|
||||
|
||||
*Estimated runtime: ~3–5 min — best for spotting jitter and TTFB variance.*
|
||||
*Estimated runtime: ~20–30 s — 8 workers run in parallel; wall time ≈ slowest single URL × 10 samples.*
|
||||
|
||||
---
|
||||
|
||||
@@ -86,7 +90,7 @@ Same as Example 2, machine-readable. Pipe into `jq` to extract specific phases.
|
||||
| jq '.[] | {url, avg_total_ms: .phases.total.avg_ms}'
|
||||
```
|
||||
|
||||
*Estimated runtime: ~3–5 min.*
|
||||
*Estimated runtime: ~20–30 s.*
|
||||
|
||||
---
|
||||
|
||||
@@ -110,7 +114,7 @@ The `.invalid` TLD is guaranteed NXDOMAIN by RFC 6761.
|
||||
https://nonexistent-host-two.invalid
|
||||
```
|
||||
|
||||
*Estimated runtime: ~2–3 min — DNS failures resolve near-instantly.*
|
||||
*Estimated runtime: ~20–30 s — DNS failures resolve near-instantly; 8 workers run in parallel.*
|
||||
|
||||
---
|
||||
|
||||
@@ -133,7 +137,7 @@ The `.invalid` TLD is guaranteed NXDOMAIN by RFC 6761.
|
||||
http://127.0.0.1:19999
|
||||
```
|
||||
|
||||
*Estimated runtime: ~2–3 min — refused connections fail immediately.*
|
||||
*Estimated runtime: ~20–30 s — refused connections fail immediately; 8 workers run in parallel.*
|
||||
|
||||
---
|
||||
|
||||
@@ -158,7 +162,7 @@ sample to 3 s instead of the default 10 s.
|
||||
https://203.0.113.1
|
||||
```
|
||||
|
||||
*Estimated runtime: ~3–4 min — timeout IPs add 3 s × 10 samples = 30 s each.*
|
||||
*Estimated runtime: ~35–40 s — all 10 URLs measured in parallel; the 2 timeout IPs each add 3 s × 10 samples = 30 s and are the bottleneck.*
|
||||
|
||||
---
|
||||
|
||||
@@ -182,7 +186,7 @@ would exit 0; with it, exit code becomes 6.
|
||||
https://httpbin.org/status/503
|
||||
```
|
||||
|
||||
*Estimated runtime: ~2–3 min — full timing captured even for error responses.*
|
||||
*Estimated runtime: ~20–30 s — full timing captured even for error responses; all 10 URLs measured in parallel.*
|
||||
|
||||
---
|
||||
|
||||
@@ -206,7 +210,7 @@ would exit 0; with it, exit code becomes 6.
|
||||
https://expired.badssl.com
|
||||
```
|
||||
|
||||
*Estimated runtime: ~2–3 min — TLS failures surface DNS + connect timing.*
|
||||
*Estimated runtime: ~20–30 s — TLS failures surface DNS + connect timing; all 10 URLs measured in parallel.*
|
||||
|
||||
---
|
||||
|
||||
@@ -237,7 +241,7 @@ HTTP error with `--fail` (exit 6). Highest code wins → **exit 5** (TLS is
|
||||
https://self-signed.badssl.com
|
||||
```
|
||||
|
||||
*Estimated runtime: ~5–7 min — the comprehensive stress test.*
|
||||
*Estimated runtime: ~35–45 s — 15 URLs measured with 8 workers; the 2 timeout IPs (30 s × each) are the bottleneck.*
|
||||
|
||||
---
|
||||
|
||||
@@ -276,7 +280,7 @@ JSON output lets you pipe results to `jq` for filtering.
|
||||
| jq '.[] | select(.failed > 0 or .status >= 400)'
|
||||
```
|
||||
|
||||
*Estimated runtime: ~30–60 s — fastest end-to-end check of all error paths.*
|
||||
*Estimated runtime: ~4–6 s — 15 URLs probed in parallel with `-n 1`; only the 2 s timeout IPs add meaningful delay.*
|
||||
|
||||
---
|
||||
|
||||
|
||||
140
docs/plans/2026-07-01-01-23-concurrency.md
Normal file
140
docs/plans/2026-07-01-01-23-concurrency.md
Normal file
@@ -0,0 +1,140 @@
|
||||
# Plan: Concurrency (Go, Step 7)
|
||||
|
||||
## Context
|
||||
|
||||
Large probe runs are slow. The example matrices in `docs/custom-usage-examples.md`
|
||||
are 15 URLs × 10 samples = 150 sequential HTTP requests, dominated by summed
|
||||
network round-trips, so a single run takes minutes. Execution is currently
|
||||
strictly sequential: `run()` loops over URLs and, for each, `runSamples()` loops
|
||||
`-n` times calling `probe.Measure()`.
|
||||
|
||||
We want to cut wall-clock time by probing multiple **URLs** in parallel, while
|
||||
keeping measurement honest. Concurrency and latency accuracy are in tension:
|
||||
firing many requests at once makes them contend for the local NIC/CPU/resolver
|
||||
and inflates timings. The decision (confirmed with the user) is to parallelize
|
||||
**only across URLs** — the samples of a single URL stay sequential so each URL's
|
||||
min/avg/max remains a clean, self-consistent measurement.
|
||||
|
||||
Confirmed decisions:
|
||||
- **Granularity:** across URLs only; samples sequential per URL. Unit of work =
|
||||
one URL with its full `-n` sample set.
|
||||
- **Default concurrency:** auto when `-c` is not given → `min(numURLs, 8)`.
|
||||
`-c 1` forces fully sequential (most accurate); `-c N` sets an explicit cap.
|
||||
- **Output:** buffered and printed in original input order (deterministic;
|
||||
identical layout to today, for both text and JSON).
|
||||
|
||||
## Files to modify
|
||||
|
||||
### `go/main.go` — core change
|
||||
|
||||
1. **New flag** (mirror the `-n`/`--count` pattern at lines 78–79):
|
||||
```go
|
||||
conc := fs.Int("concurrency", 0, "max URLs probed in parallel (0 = auto)")
|
||||
fs.IntVar(conc, "c", 0, "max URLs probed in parallel (shorthand)")
|
||||
```
|
||||
Add `-c, --concurrency` to `usageText` (around lines 22–27).
|
||||
|
||||
2. **Resolve effective concurrency** after the empty-URL check (line 96):
|
||||
```go
|
||||
const defaultMaxConc = 8
|
||||
workers := *conc
|
||||
if workers <= 0 {
|
||||
workers = min(len(urls), defaultMaxConc) // auto
|
||||
}
|
||||
workers = min(workers, len(urls)) // never exceed work units
|
||||
if workers < 1 {
|
||||
workers = 1
|
||||
}
|
||||
```
|
||||
(Go 1.26 has builtin `min`.)
|
||||
|
||||
3. **Split `run()` into measure phase (concurrent) + render phase (sequential).**
|
||||
Introduce a small result holder:
|
||||
```go
|
||||
type urlResult struct {
|
||||
url string
|
||||
succeeded, failed []probe.Result
|
||||
}
|
||||
```
|
||||
Measure phase — bounded worker pool via a semaphore channel + `sync.WaitGroup`,
|
||||
each goroutine writing only its own preallocated slot (no shared mutable state,
|
||||
so no mutex / data race):
|
||||
```go
|
||||
results := make([]urlResult, len(urls))
|
||||
sem := make(chan struct{}, workers)
|
||||
var wg sync.WaitGroup
|
||||
for i, rawURL := range urls {
|
||||
wg.Add(1)
|
||||
go func(i int, rawURL string) {
|
||||
defer wg.Done()
|
||||
sem <- struct{}{}
|
||||
defer func() { <-sem }()
|
||||
s, f := runSamples(rawURL, *count, opts)
|
||||
results[i] = urlResult{rawURL, s, f}
|
||||
}(i, rawURL)
|
||||
}
|
||||
wg.Wait()
|
||||
```
|
||||
Render phase — the existing per-URL loop body (lines 102–127), unchanged in
|
||||
behaviour, now reading from `results[i]` instead of calling `runSamples`
|
||||
inline. `worstCode` accumulation, JSON entry building, and ordered text
|
||||
printing all stay identical. JSON encode block (129–136) and `return worstCode`
|
||||
unchanged.
|
||||
|
||||
`runSamples`, `printURL`, `buildJSONEntry`, and all output helpers stay as-is.
|
||||
|
||||
Concurrency safety: `probe.Measure` uses `http.DefaultClient`, which is safe for
|
||||
concurrent use; distinct URLs hit distinct hosts so the shared transport pool is
|
||||
not a correctness concern. Output order is deterministic because rendering reads
|
||||
the indexed `results` slice after `wg.Wait()`.
|
||||
|
||||
### `go/run_test.go` — add coverage
|
||||
|
||||
- `TestRunConcurrentOrder`: spin up several `httptest` servers (reuse existing
|
||||
`statusSrv` helper), pass them with `-c` larger than 1, and assert the printed
|
||||
blocks appear in **input order** and the exit code matches the sequential run.
|
||||
- Add a mixed success/failure case under high `-c` to confirm `worstCode`
|
||||
aggregation is unaffected by parallelism.
|
||||
|
||||
### `Makefile` + `docs/usage/makefile.md`
|
||||
|
||||
- Add `go-test-race`: `go -C $(GO_DIR) test -race ./...` (with `##` help comment,
|
||||
added to `.PHONY`). Document it in the Go targets table. Leave `check` as-is
|
||||
(race run kept as an explicit opt-in target).
|
||||
|
||||
### Docs & conventions
|
||||
|
||||
- `docs/usage/step-7-concurrency.md` (new): explain the `-c`/`--concurrency`
|
||||
flag, the auto default (`min(numURLs, 8)`), the across-URLs-only model, the
|
||||
accuracy trade-off (use `-c 1` for the most precise numbers), and a worked
|
||||
example with before/after timing.
|
||||
- `README.md`: add `-c, --concurrency` to the Usage flags block; add **Step 7 —
|
||||
Concurrency** row (✅) to the Go roadmap table.
|
||||
- `CHANGELOG.md`: new top entry, `2026-07-01 01:23 — Concurrency (Go, Step 7)`.
|
||||
- `docs/custom-usage-examples.md`: note near the top that runs now parallelize
|
||||
across URLs by default and that `-c 1` restores serial timing; revise the
|
||||
Example 2/3/9 runtime estimates to reflect the speedup.
|
||||
- `docs/plans/2026-07-01-01-23-concurrency.md`: copy of this plan (per CLAUDE.md).
|
||||
|
||||
## Verification
|
||||
|
||||
```sh
|
||||
make build
|
||||
# Auto concurrency (defaults to min(numURLs,8)) — should be much faster than before:
|
||||
time ./go/latprobe -n 5 https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
|
||||
# Forced serial for comparison / accuracy:
|
||||
time ./go/latprobe -c 1 -n 5 https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
|
||||
# Output order is deterministic regardless of -c:
|
||||
./go/latprobe -c 8 https://www.google.com https://www.bbc.co.uk https://www.abc.net.au
|
||||
# JSON array order also matches input order:
|
||||
./go/latprobe --json -c 8 https://www.google.com https://www.bbc.co.uk | jq '.[].url'
|
||||
|
||||
make test # full suite incl. new ordering tests
|
||||
make go-test-race # data-race detector must report clean
|
||||
make check # fmt + vet + test gate
|
||||
```
|
||||
|
||||
Confirm: identical output (modulo timing numbers) between `-c 1` and default for
|
||||
the same URL set; exit codes unchanged; `-race` clean.
|
||||
@@ -32,6 +32,7 @@ make help # list all targets with descriptions
|
||||
| `go-run` | Build and run (pass flags via `ARGS=`) |
|
||||
| `go-test` | `go test ./...` |
|
||||
| `go-test-verbose` | `go test -v ./...` — per-case PASS/FAIL output |
|
||||
| `go-test-race` | `go test -race ./...` — run tests with the data-race detector |
|
||||
| `go-check` | `go-fmt` + `go-vet` + `go-test` |
|
||||
| `go-cover` | Coverage report → `go/coverage.html` |
|
||||
| `go-fmt` | `gofmt -w go/` |
|
||||
@@ -54,6 +55,9 @@ make test
|
||||
# Verbose output showing each test case
|
||||
make go-test-verbose
|
||||
|
||||
# Run tests with the race detector (verify concurrency safety)
|
||||
make go-test-race
|
||||
|
||||
# Pre-commit gate (format, vet, test all in one)
|
||||
make check
|
||||
|
||||
|
||||
76
docs/usage/step-7-concurrency.md
Normal file
76
docs/usage/step-7-concurrency.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# Step 7 — Concurrency
|
||||
|
||||
`latprobe` now probes multiple URLs in parallel, cutting wall-clock time for
|
||||
large runs without sacrificing measurement accuracy.
|
||||
|
||||
## How it works
|
||||
|
||||
| Axis | Behaviour |
|
||||
|------|-----------|
|
||||
| Across URLs | **Parallel** — up to `-c` workers run simultaneously |
|
||||
| Samples of one URL (`-n`) | **Sequential** — always serial within a URL |
|
||||
|
||||
Keeping samples sequential ensures that min/avg/max statistics for a single URL
|
||||
reflect genuine latency variability, not artificial load created by firing
|
||||
multiple requests at the same server at once.
|
||||
|
||||
Output is always printed in **input order**, regardless of which URL finishes
|
||||
first.
|
||||
|
||||
## Flag
|
||||
|
||||
```
|
||||
-c, --concurrency int Max URLs probed in parallel (0 = auto, default)
|
||||
```
|
||||
|
||||
| Value | Behaviour |
|
||||
|-------|-----------|
|
||||
| `0` (default) | Auto: `min(numURLs, 8)` — scales with workload, caps at 8 |
|
||||
| `1` | Fully serial — identical to the old behaviour; best for precision |
|
||||
| `N > 1` | Explicit worker cap |
|
||||
|
||||
## When to use serial mode
|
||||
|
||||
For the most accurate latency numbers — especially when comparing sites —
|
||||
use `-c 1`. Concurrent probes share your local NIC, DNS resolver, and CPU,
|
||||
which can inflate timings on slower machines or fast batch runs.
|
||||
|
||||
```sh
|
||||
# High fidelity: one URL at a time
|
||||
./go/latprobe -c 1 -n 10 https://www.google.com https://www.bbc.co.uk
|
||||
|
||||
# Speed: all four probed in parallel (auto default would do this anyway)
|
||||
./go/latprobe -c 4 -n 10 https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.abc.net.au
|
||||
```
|
||||
|
||||
## Example: speedup in practice
|
||||
|
||||
Five sites, 5 samples each — sequential vs parallel:
|
||||
|
||||
```sh
|
||||
# Serial (-c 1): ~wall time ≈ sum of all RTTs × 5
|
||||
time ./go/latprobe -c 1 -n 5 \
|
||||
https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
|
||||
# real ~1m15s (5 sites × ~15s sequential)
|
||||
|
||||
# Parallel (default): ~wall time ≈ slowest single URL × 5
|
||||
time ./go/latprobe -n 5 \
|
||||
https://www.google.com https://www.bbc.co.uk \
|
||||
https://www.lemonde.fr https://www.spiegel.de https://www.abc.net.au
|
||||
# real ~18s (all 5 measured at once, gated by the slowest)
|
||||
```
|
||||
|
||||
## Exit codes
|
||||
|
||||
Exit codes work exactly as before: when multiple failure types occur the
|
||||
**highest** code wins. Concurrency does not change this — exit codes are
|
||||
accumulated after all workers complete.
|
||||
|
||||
## Race safety
|
||||
|
||||
The implementation uses a semaphore channel (`chan struct{}`) and
|
||||
`sync.WaitGroup` with each goroutine writing only its own indexed result slot,
|
||||
so there is no shared mutable state. `http.DefaultClient` is safe for concurrent
|
||||
use. Verified clean with `go test -race` (`make go-test-race`).
|
||||
48
go/main.go
48
go/main.go
@@ -9,6 +9,7 @@ import (
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"latprobe/internal/probe"
|
||||
@@ -20,11 +21,12 @@ Usage:
|
||||
latprobe [flags] <url> [url ...]
|
||||
|
||||
Flags:
|
||||
-n, --count int Number of requests per URL (default 1)
|
||||
--timeout duration Request timeout, e.g. 10s, 500ms (default 10s)
|
||||
--fail Exit non-zero on HTTP status >= 400 (exit code 6)
|
||||
--json Output results as JSON instead of text
|
||||
-h, --help Show this help
|
||||
-n, --count int Number of requests per URL (default 1)
|
||||
-c, --concurrency int Max URLs probed in parallel, 0 = auto (default min(numURLs,8))
|
||||
--timeout duration Request timeout, e.g. 10s, 500ms (default 10s)
|
||||
--fail Exit non-zero on HTTP status >= 400 (exit code 6)
|
||||
--json Output results as JSON instead of text
|
||||
-h, --help Show this help
|
||||
|
||||
Exit codes:
|
||||
0 All probes succeeded
|
||||
@@ -77,6 +79,8 @@ func run(args []string, stdout, stderr io.Writer) int {
|
||||
|
||||
count := fs.Int("count", 1, "number of requests per URL")
|
||||
fs.IntVar(count, "n", 1, "number of requests per URL (shorthand)")
|
||||
conc := fs.Int("concurrency", 0, "max URLs probed in parallel (0 = auto)")
|
||||
fs.IntVar(conc, "c", 0, "max URLs probed in parallel (shorthand)")
|
||||
timeout := fs.Duration("timeout", 10*time.Second, "request timeout per sample")
|
||||
fail := fs.Bool("fail", false, "exit non-zero on HTTP status >= 400")
|
||||
jsonOut := fs.Bool("json", false, "output results as JSON instead of text")
|
||||
@@ -96,11 +100,43 @@ func run(args []string, stdout, stderr io.Writer) int {
|
||||
}
|
||||
|
||||
opts := probe.Options{Timeout: *timeout}
|
||||
|
||||
// Resolve effective worker count.
|
||||
const defaultMaxConc = 8
|
||||
workers := *conc
|
||||
if workers <= 0 {
|
||||
workers = min(len(urls), defaultMaxConc) // auto
|
||||
}
|
||||
workers = min(workers, len(urls)) // never more goroutines than work units
|
||||
workers = max(workers, 1)
|
||||
|
||||
// ── Measure phase (concurrent) ────────────────────────────────────────────
|
||||
// Each goroutine writes only its own indexed slot — no shared mutable state.
|
||||
type urlResult struct {
|
||||
succeeded, failed []probe.Result
|
||||
}
|
||||
results := make([]urlResult, len(urls))
|
||||
sem := make(chan struct{}, workers)
|
||||
var wg sync.WaitGroup
|
||||
for i, rawURL := range urls {
|
||||
wg.Add(1)
|
||||
go func(i int, rawURL string) {
|
||||
defer wg.Done()
|
||||
sem <- struct{}{}
|
||||
defer func() { <-sem }()
|
||||
s, f := runSamples(rawURL, *count, opts)
|
||||
results[i] = urlResult{s, f}
|
||||
}(i, rawURL)
|
||||
}
|
||||
wg.Wait()
|
||||
|
||||
// ── Render phase (sequential, input order) ────────────────────────────────
|
||||
worstCode := exitOK
|
||||
var jsonEntries []jsonEntry
|
||||
|
||||
for i, rawURL := range urls {
|
||||
succeeded, failed := runSamples(rawURL, *count, opts)
|
||||
succeeded := results[i].succeeded
|
||||
failed := results[i].failed
|
||||
|
||||
for _, r := range failed {
|
||||
if c := failPhaseCode(r.FailPhase); c > worstCode {
|
||||
|
||||
@@ -250,6 +250,90 @@ func TestJSONDNSFailure(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// ── concurrency tests ─────────────────────────────────────────────────────────
|
||||
|
||||
// TestRunConcurrentOrder verifies that with -c > 1 the URL blocks are printed
|
||||
// in the original input order and exit codes are accumulated correctly.
|
||||
func TestRunConcurrentOrder(t *testing.T) {
|
||||
a := statusSrv(200)
|
||||
defer a.Close()
|
||||
b := statusSrv(200)
|
||||
defer b.Close()
|
||||
c := statusSrv(200)
|
||||
defer c.Close()
|
||||
|
||||
// Run with explicit high concurrency — all three URLs measured in parallel.
|
||||
stdout, _, code := invoke("-c", "3", a.URL, b.URL, c.URL)
|
||||
if code != exitOK {
|
||||
t.Fatalf("exit code = %d, want 0\nstdout:\n%s", code, stdout)
|
||||
}
|
||||
|
||||
// Each URL block must appear and in the correct order.
|
||||
posA := strings.Index(stdout, a.URL)
|
||||
posB := strings.Index(stdout, b.URL)
|
||||
posC := strings.Index(stdout, c.URL)
|
||||
if posA < 0 || posB < 0 || posC < 0 {
|
||||
t.Fatalf("one or more URLs missing from stdout:\n%s", stdout)
|
||||
}
|
||||
if !(posA < posB && posB < posC) {
|
||||
t.Errorf("URLs out of order: posA=%d posB=%d posC=%d\nstdout:\n%s",
|
||||
posA, posB, posC, stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConcurrentWorstCode confirms that worstCode aggregation is correct when
|
||||
// both successes and failures run concurrently.
|
||||
func TestConcurrentWorstCode(t *testing.T) {
|
||||
good := statusSrv(200)
|
||||
defer good.Close()
|
||||
|
||||
// DNS failure mixed with a successful URL — highest code (2) must win.
|
||||
stdout, _, code := invoke("-c", "2",
|
||||
good.URL,
|
||||
"https://totally.bogus.domain.for.concurrency.test.invalid",
|
||||
)
|
||||
if code != exitDNS {
|
||||
t.Errorf("exit code = %d, want %d (DNS)\nstdout:\n%s", code, exitDNS, stdout)
|
||||
}
|
||||
// Both URLs must appear in output (good one before the failing one).
|
||||
if !strings.Contains(stdout, good.URL) {
|
||||
t.Errorf("stdout missing good URL\nstdout:\n%s", stdout)
|
||||
}
|
||||
if !strings.Contains(stdout, "✗ dns:") {
|
||||
t.Errorf("stdout missing DNS failure marker\nstdout:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConcurrentJSONOrder verifies that JSON entries preserve input order under
|
||||
// parallel execution.
|
||||
func TestConcurrentJSONOrder(t *testing.T) {
|
||||
a := statusSrv(200)
|
||||
defer a.Close()
|
||||
b := statusSrv(200)
|
||||
defer b.Close()
|
||||
|
||||
stdout, _, code := invoke("--json", "-c", "2", a.URL, b.URL)
|
||||
if code != exitOK {
|
||||
t.Fatalf("exit code = %d, want 0\nstdout:\n%s", code, stdout)
|
||||
}
|
||||
|
||||
var results []struct {
|
||||
URL string `json:"url"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(stdout), &results); err != nil {
|
||||
t.Fatalf("invalid JSON: %v\nstdout: %s", err, stdout)
|
||||
}
|
||||
if len(results) != 2 {
|
||||
t.Fatalf("expected 2 results, got %d", len(results))
|
||||
}
|
||||
if results[0].URL != a.URL {
|
||||
t.Errorf("results[0].url = %q, want %q", results[0].URL, a.URL)
|
||||
}
|
||||
if results[1].URL != b.URL {
|
||||
t.Errorf("results[1].url = %q, want %q", results[1].URL, b.URL)
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONSampling(t *testing.T) {
|
||||
srv := statusSrv(200)
|
||||
defer srv.Close()
|
||||
|
||||
Reference in New Issue
Block a user