http-latency-prober/CHANGELOG.md

# Changelog

All completed features are logged here in reverse-chronological order.

---

## 2026-07-01 12:55 — `--verbose` / `-v` flag for `latprobe` (Python)

- `-v`/`--verbose` flag added to the `latprobe` CLI
- `probe.py`: new `CertInfo` + `VerboseDetail` dataclasses; `Options.verbose`;
  `Result.detail`; TTFB loop now accumulates until `\r\n\r\n` (unchanged TTFB
  semantics — `t_first_byte` stamped on first `recv()`, not when headers complete);
  captures resolved IP (from `getaddrinfo`), TLS version/cipher/bits
  (from `sock.version()`/`sock.cipher()`), verified cert (from `getpeercert()`),
  and all response headers
- `cli.py`: verbose text block appended after timing rows in all four output
  branches (single, aggregate, all-failed, mixed); `_VERBOSE_HEADERS` priority
  list controls which headers appear in text mode; JSON `"verbose"` object
  includes all parsed headers, full cert fields, TLS metadata; `"verbose"` key
  omitted when flag is absent
- 9 new hermetic probe tests, 15 new CLI tests covering verbose text and JSON
  across success, connect-fail, and DNS-fail scenarios
- 15 new integration tests for real TLS cert fields (CN, expiry, issuer),
  IP format, TLS version string, header presence, and verbose text/JSON output
- `python/configs/usage-latprobe.md`: runnable reference with real output for
  all features including verbose and `--verbose --json`
- `docs/usage/py-latprobe.md`: updated with `--verbose` flag and examples
- Plan: `docs/plans/2026-07-01-12-55-py-latprobe-verbose.md`

---

## 2026-07-01 12:23 — Full `latprobe` Python package (Python, Step 3)

- `python/latprobe/` package: full port of the Go CLI, runnable as `python -m latprobe`
- `probe.py`: raw-socket HTTP measurement with `Options(timeout)` parameter; mirrors
  `phases.py` technique; partial phases preserved on failure
- `aggregate.py`: `summarize(List[Result]) -> Aggregate` with per-phase `PhaseStats(min_ms, avg_ms, max_ms)`
- `duration.py`: parses Go-style duration strings (`10s`, `500ms`, `2m`, bare seconds)
- `cli.py`: injectable `run(args, stdout, stderr) -> int`; argparse with injected streams
  (`_Parser` subclass); `ThreadPoolExecutor` concurrency across URLs; four text-rendering
  branches (single, aggregate, all-failed, mixed); JSON output; worst-exit-code accumulation
- Exit codes: 0 ok, 1 usage, 2 dns, 3 connect, 4 timeout, 5 tls, 6 http≥400 (`--fail`)
- `python/tests/test_probe.py`: 9 tests — success, 404, DNS fail, connect refused, TTFB
  timeout, bad scheme, partial phase invariants; uses `http.server` + daemon threads
- `python/tests/test_cli.py`: 23 tests — drives `cli.run()` in-process; covers usage
  errors, single/aggregate text, failure exit codes, worst-code, `--fail`, JSON schema,
  JSON ordering, JSON error grouping
- Makefile `py-test`: added `PYTHONPATH=$(PY_DIR)`, removed `|| true`
- User doc: `docs/usage/py-latprobe.md`

---

## 2026-07-01 12:04 — Per-phase latency measurement (Python, Step 2)

- `python/phases.py`: self-contained script; hand-drives raw sockets to time
  each HTTP phase individually — DNS (`getaddrinfo`), TCP connect, TLS handshake
  (`ssl.wrap_socket`, HTTPS only), TTFB (sendall → first recv), Transfer, Total
- Partial phases preserved on failure (same invariant as Go's `probe.go`)
- Error classification mirrors Go's priority: dns → timeout → tls → connect
- Output format matches Go's single-sample text layout (14-char labels, `─`
  separator, `%8.2f ms` alignment)
- Input: bare URL args or plain-text config file (same format as `simple.py`)
- Exits 0 if all URLs complete without network error; 1 if any failed
- User doc: `docs/usage/py-phases.md`

---

## 2026-07-01 11:30 — Error-path example configs for simple.py (Python, Step 1 refinement)

- `python/configs/` directory with 7 purpose-built config files, one per error class:
  `all-ok.txt`, `dns-failure.txt`, `connection-refused.txt`, `timeout.txt`,
  `tls-errors.txt` (badssl.com), `http-errors.txt` (httpstat.us), `mixed.txt`
- `python/configs/usage.md`: runnable shell commands + expected output for every config
- Fixed `docs/usage/py-simple.md`: 4xx/5xx responses are reported as `FAIL` (not `OK`),
  because `urlopen` raises `HTTPError` for non-2xx; added pointer to the example configs

---

## 2026-07-01 10:39 — Simple reachability checker (Python, Step 1)

- `python/simple.py`: reads a plain-text site list (one URL per line, `#` comments),
  issues a GET to each, prints aligned `OK / FAIL + elapsed ms` per site
- Plain-text config format forward-compatible with future `key=value` annotations
- Exits 0 if all sites responded, 1 if any failed or config is missing
- `python/sites.txt`: committed example config
- Makefile `py-*` targets added: `py-simple-run`, `py-phases-run`, `py-run`,
  `py-test`, `py-check`, `py-clean`; umbrella `test`, `check`, `clean` now include Python
- User doc: `docs/usage/py-simple.md`

---

## 2026-07-01 01:23 — Concurrency (Go, Step 7)

- Added `-c`/`--concurrency` flag: max URLs probed in parallel (0 = auto)
- Default auto-concurrency: `min(numURLs, 8)` — scales with workload, caps at 8
- Parallelism is **across URLs only**; the N samples of each URL stay sequential
  to preserve the accuracy of per-URL min/avg/max statistics
- Output buffered and printed in original input order (deterministic for both
  text and JSON), exit-code accumulation unchanged
- Implementation: semaphore channel + `sync.WaitGroup`, each goroutine writes
  only its own indexed result slot — verified clean with `go test -race`
- Added `make go-test-race` target to the Makefile
- 3 new tests: `TestRunConcurrentOrder`, `TestConcurrentWorstCode`,
  `TestConcurrentJSONOrder`
- Updated runtime estimates in `docs/custom-usage-examples.md`
- User doc: `docs/usage/step-7-concurrency.md`

---

## 2026-07-01 01:05 — Root Makefile

- Namespaced targets: `go-build`, `go-run`, `go-test`, `go-test-verbose`, `go-check`, `go-cover`, `go-fmt`, `go-vet`, `go-tidy`, `go-install`, `go-lint`, `go-clean`
- Umbrella targets (`build`, `test`, `check`, `fmt`, `vet`, `clean`, `all`) delegate to Go now; `py-*` will slot in during the Python port
- `help` is the default target; auto-generated from `##` comments
- `go-run` accepts `ARGS=` for passing flags, `go-lint` guards for `golangci-lint`
- `.gitignore` updated with coverage artifacts

---

## 2026-07-01 00:49 — Integration tests (Go, Step 6)

- Extracted `run(args, stdout, stderr) int` from `main()` to make the CLI testable in-process
- Fixed TLS classification bug: `TLSHandshakeDone` fires with the error on cert rejection, so `tlsErr` is now captured and checked before the stale `tlsStart.IsZero()` guard
- `run_test.go`: 14 in-process tests covering exit codes 0–6, multi-URL, sampling, and JSON structure assertions
- `cli_test.go`: `TestMain` builds the real binary; 3 subprocess smoke tests exercise the actual `os.Exit` path
- All test servers use `httptest` + stdlib; `.invalid` TLD for deterministic DNS failures; no external network dependency

---

## 2026-07-01 00:38 — Failure handling (Go, Step 5)

- Classified network failures into `dns` / `connect` / `timeout` / `tls` with distinct exit codes (2–5)
- Partial timing preserved up to the failure point (e.g. DNS phase shown on NXDOMAIN)
- `--timeout` flag (default 10s) applied via `context.WithTimeout`
- `--fail` flag: HTTP status ≥ 400 → exit code 6 (curl-style)
- `-n` sampling continues on network failure; aggregates successes, reports fail count + cause
- JSON output extended with `succeeded`, `failed`, `errors[]` fields
- Highest exit code across all URLs/failure types is used as the process exit

---

## 2026-07-01 00:08 — JSON output flag (Go, Step 4)

- `--json` flag emits a JSON array with one entry per URL
- Schema always uses min/avg/max shape (consistent regardless of `-n`)
- Phases absent from the request (e.g. TLS on HTTP) are omitted from the JSON object
- Failed URLs are excluded from JSON output and reported to stderr
- Text output unchanged and remains the default

---

## 2026-07-01 00:08 — Multiple URLs + sampling (Go, Step 3)

- `-n`/`--count` flag repeats each URL N times and reports min/avg/max per phase
- `probe.Summarize` aggregates a slice of Results into per-phase PhaseStats
- Multi-sample output shows an aligned min/avg/max table with a header row
- Single-sample output (n=1) is unchanged from Step 2

---

## 2026-07-01 00:08 — Per-phase breakdown (Go, Step 2)

- Instrumented requests with `net/http/httptrace.ClientTrace`
- DNS lookup, TCP connect, TLS handshake, Server/TTFB, Transfer, Total phases
- TLS row omitted automatically for plain `http://` URLs
- Aligned text output with separator before Total

---

## 2026-07-01 00:08 — Simple total latency (Go, Step 1)

- `probe.Measure` performs an HTTP GET and records wall-clock total time
- `main.go` prints URL, HTTP status code, and total duration for each URL
- Exits with code 1 if any URL fails
- Multiple URLs accepted as positional arguments

---

## 2026-07-01 00:08 — Project scaffold (Go, Step 0)

- Created project structure: `go/`, `python/`, `docs/plans/`, `docs/usage/`
- Initialised Go module `latprobe` (`go/go.mod`)
- Minimal `go/main.go` that prints usage and exits cleanly when no URL is given
- Stub `go/internal/probe/probe.go` defining the `Result` type and `Measure` signature
- Foundation files: `README.md`, `CLAUDE.md`, `CHANGELOG.md`
- Saved initial plan to `docs/plans/2026-07-01-00-08-go-latency-tool.md`