http-latency-prober/CHANGELOG.md

Changelog

All completed features are logged here in reverse-chronological order.

---

2026-07-01 12:55 — --verbose / -v flag for latprobe (Python)

• -v/--verbose flag added to the latprobe CLI
• probe.py: new CertInfo + VerboseDetail dataclasses; Options.verbose; Result.detail; TTFB loop now accumulates until \r\n\r\n (unchanged TTFB semantics — t_first_byte stamped on first recv(), not when headers complete); captures resolved IP (from getaddrinfo), TLS version/cipher/bits (from sock.version()/sock.cipher()), verified cert (from getpeercert()), and all response headers
• cli.py: verbose text block appended after timing rows in all four output branches (single, aggregate, all-failed, mixed); _VERBOSE_HEADERS priority list controls which headers appear in text mode; JSON "verbose" object includes all parsed headers, full cert fields, TLS metadata; "verbose" key omitted when flag is absent
• 9 new hermetic probe tests, 15 new CLI tests covering verbose text and JSON across success, connect-fail, and DNS-fail scenarios
• 15 new integration tests for real TLS cert fields (CN, expiry, issuer), IP format, TLS version string, header presence, and verbose text/JSON output
• python/configs/usage-latprobe.md: runnable reference with real output for all features including verbose and --verbose --json
• docs/usage/py-latprobe.md: updated with --verbose flag and examples
• Plan: docs/plans/2026-07-01-12-55-py-latprobe-verbose.md

---

2026-07-01 12:23 — Full latprobe Python package (Python, Step 3)

• python/latprobe/ package: full port of the Go CLI, runnable as python -m latprobe
• probe.py: raw-socket HTTP measurement with Options(timeout) parameter; mirrors phases.py technique; partial phases preserved on failure
• aggregate.py: summarize(List[Result]) -> Aggregate with per-phase PhaseStats(min_ms, avg_ms, max_ms)
• duration.py: parses Go-style duration strings (10s, 500ms, 2m, bare seconds)
• cli.py: injectable run(args, stdout, stderr) -> int; argparse with injected streams (_Parser subclass); ThreadPoolExecutor concurrency across URLs; four text-rendering branches (single, aggregate, all-failed, mixed); JSON output; worst-exit-code accumulation
• Exit codes: 0 ok, 1 usage, 2 dns, 3 connect, 4 timeout, 5 tls, 6 http≥400 (--fail)
• python/tests/test_probe.py: 9 tests — success, 404, DNS fail, connect refused, TTFB timeout, bad scheme, partial phase invariants; uses http.server + daemon threads
• python/tests/test_cli.py: 23 tests — drives cli.run() in-process; covers usage errors, single/aggregate text, failure exit codes, worst-code, --fail, JSON schema, JSON ordering, JSON error grouping
• Makefile py-test: added PYTHONPATH=$(PY_DIR), removed || true
• User doc: docs/usage/py-latprobe.md

---

2026-07-01 12:04 — Per-phase latency measurement (Python, Step 2)

• python/phases.py: self-contained script; hand-drives raw sockets to time each HTTP phase individually — DNS (getaddrinfo), TCP connect, TLS handshake (ssl.wrap_socket, HTTPS only), TTFB (sendall → first recv), Transfer, Total
• Partial phases preserved on failure (same invariant as Go's probe.go)
• Error classification mirrors Go's priority: dns → timeout → tls → connect
• Output format matches Go's single-sample text layout (14-char labels, ─ separator, %8.2f ms alignment)
• Input: bare URL args or plain-text config file (same format as simple.py)
• Exits 0 if all URLs complete without network error; 1 if any failed
• User doc: docs/usage/py-phases.md

---

2026-07-01 11:30 — Error-path example configs for simple.py (Python, Step 1 refinement)

• python/configs/ directory with 7 purpose-built config files, one per error class: all-ok.txt, dns-failure.txt, connection-refused.txt, timeout.txt, tls-errors.txt (badssl.com), http-errors.txt (httpstat.us), mixed.txt
• python/configs/usage.md: runnable shell commands + expected output for every config
• Fixed docs/usage/py-simple.md: 4xx/5xx responses are reported as FAIL (not OK), because urlopen raises HTTPError for non-2xx; added pointer to the example configs

---

2026-07-01 10:39 — Simple reachability checker (Python, Step 1)

• python/simple.py: reads a plain-text site list (one URL per line, # comments), issues a GET to each, prints aligned OK / FAIL + elapsed ms per site
• Plain-text config format forward-compatible with future key=value annotations
• Exits 0 if all sites responded, 1 if any failed or config is missing
• python/sites.txt: committed example config
• Makefile py-* targets added: py-simple-run, py-phases-run, py-run, py-test, py-check, py-clean; umbrella test, check, clean now include Python
• User doc: docs/usage/py-simple.md

---

2026-07-01 01:23 — Concurrency (Go, Step 7)

• Added -c/--concurrency flag: max URLs probed in parallel (0 = auto)
• Default auto-concurrency: min(numURLs, 8) — scales with workload, caps at 8
• Parallelism is across URLs only; the N samples of each URL stay sequential to preserve the accuracy of per-URL min/avg/max statistics
• Output buffered and printed in original input order (deterministic for both text and JSON), exit-code accumulation unchanged
• Implementation: semaphore channel + sync.WaitGroup, each goroutine writes only its own indexed result slot — verified clean with go test -race
• Added make go-test-race target to the Makefile
• 3 new tests: TestRunConcurrentOrder, TestConcurrentWorstCode, TestConcurrentJSONOrder
• Updated runtime estimates in docs/custom-usage-examples.md
• User doc: docs/usage/step-7-concurrency.md

---

2026-07-01 01:05 — Root Makefile

• Namespaced targets: go-build, go-run, go-test, go-test-verbose, go-check, go-cover, go-fmt, go-vet, go-tidy, go-install, go-lint, go-clean
• Umbrella targets (build, test, check, fmt, vet, clean, all) delegate to Go now; py-* will slot in during the Python port
• help is the default target; auto-generated from ## comments
• go-run accepts ARGS= for passing flags, go-lint guards for golangci-lint
• .gitignore updated with coverage artifacts

---

2026-07-01 00:49 — Integration tests (Go, Step 6)

• Extracted run(args, stdout, stderr) int from main() to make the CLI testable in-process
• Fixed TLS classification bug: TLSHandshakeDone fires with the error on cert rejection, so tlsErr is now captured and checked before the stale tlsStart.IsZero() guard
• run_test.go: 14 in-process tests covering exit codes 0–6, multi-URL, sampling, and JSON structure assertions
• cli_test.go: TestMain builds the real binary; 3 subprocess smoke tests exercise the actual os.Exit path
• All test servers use httptest + stdlib; .invalid TLD for deterministic DNS failures; no external network dependency

---

2026-07-01 00:38 — Failure handling (Go, Step 5)

• Classified network failures into dns / connect / timeout / tls with distinct exit codes (2–5)
• Partial timing preserved up to the failure point (e.g. DNS phase shown on NXDOMAIN)
• --timeout flag (default 10s) applied via context.WithTimeout
• --fail flag: HTTP status ≥ 400 → exit code 6 (curl-style)
• -n sampling continues on network failure; aggregates successes, reports fail count + cause
• JSON output extended with succeeded, failed, errors[] fields
• Highest exit code across all URLs/failure types is used as the process exit

---

2026-07-01 00:08 — JSON output flag (Go, Step 4)

• --json flag emits a JSON array with one entry per URL
• Schema always uses min/avg/max shape (consistent regardless of -n)
• Phases absent from the request (e.g. TLS on HTTP) are omitted from the JSON object
• Failed URLs are excluded from JSON output and reported to stderr
• Text output unchanged and remains the default

---

2026-07-01 00:08 — Multiple URLs + sampling (Go, Step 3)

• -n/--count flag repeats each URL N times and reports min/avg/max per phase
• probe.Summarize aggregates a slice of Results into per-phase PhaseStats
• Multi-sample output shows an aligned min/avg/max table with a header row
• Single-sample output (n=1) is unchanged from Step 2

---

2026-07-01 00:08 — Per-phase breakdown (Go, Step 2)

• Instrumented requests with net/http/httptrace.ClientTrace
• DNS lookup, TCP connect, TLS handshake, Server/TTFB, Transfer, Total phases
• TLS row omitted automatically for plain http:// URLs
• Aligned text output with separator before Total

---

2026-07-01 00:08 — Simple total latency (Go, Step 1)

• probe.Measure performs an HTTP GET and records wall-clock total time
• main.go prints URL, HTTP status code, and total duration for each URL
• Exits with code 1 if any URL fails
• Multiple URLs accepted as positional arguments

---

2026-07-01 00:08 — Project scaffold (Go, Step 0)

• Created project structure: go/, python/, docs/plans/, docs/usage/
• Initialised Go module latprobe (go/go.mod)
• Minimal go/main.go that prints usage and exits cleanly when no URL is given
• Stub go/internal/probe/probe.go defining the Result type and Measure signature
• Foundation files: README.md, CLAUDE.md, CHANGELOG.md
• Saved initial plan to docs/plans/2026-07-01-00-08-go-latency-tool.md