http-latency-prober/docs/plans/2026-07-02-14-05-hxprobe-run-summary-footer.md

hxprobe: end-of-run summary footer for multi-URL runs

Context

A single scalar exit code is inherently lossy when probing multiple URLs with different failure classes: today run() returns the numeric max across all URLs (e.g. DNS-fail on one URL + TLS-fail on another → exit 5, and the DNS failure is invisible in the code). The failures are all printed per-URL, but there's no consolidated view — for many URLs you must scroll and eyeball each block to know what happened overall.

The exit code's job is a coarse pass/fail + severity hint for scripts, and it's a deliberate cross-implementation contract shared with latprobe and the Go version (documented in hxprobe/USAGE.md:475-488, asserted by 13 tests in tests/test_cli.py). So we keep the worst-code exit unchanged and instead give humans the full picture the scalar can't: an end-of-run summary footer that tallies every URL's outcome and shows how the exit code was derived.

Decisions (confirmed with the user):

• Exit code: unchanged (worst/highest severity across URLs).
• Summary: text footer, multi-URL only (len(urls) > 1). Single-URL text output stays byte-identical; JSON output stays a bare array (unchanged).

Design

Per-URL classification (reuses existing helpers)

Each URL gets one "worst outcome code" using the existing mapping — no new severity scheme:

• start at EXIT_OK;
• for each failed sample: code = max(code, _phase_code(r.fail_phase)) (_phase_code at cli.py:31);
• if --fail: for each succeeded sample with status_code >= 400: code = max(code, EXIT_HTTP).

A URL is "ok" iff its code is EXIT_OK, else "failed" and bucketed by its code. A partially-failed URL (some samples ok, some failed) classifies by its worst sample — consistent with how its own block and the global exit code already behave.

Footer format (text, only when len(urls) > 1)

Printed once after the last URL block, before return worst. Failure classes use the same ✗ bullet style as _print_failure_summary (cli.py:219-224). The final → exit N (label) line explicitly ties the tally to the returned code — directly answering "why is the exit code what it is". Example (3 URLs):

https://example.com  (200)
  ... phase rows ...

http://no.such.host.invalid  (FAILED)
  ✗ dns: [Errno 8] nodename nor servname provided

https://self-signed.badssl.com  (FAILED)
  ✗ tls: certificate verify failed

  ─────────────────────────────────────────────────
  Summary: 3 URLs — 1 ok, 2 failed
  ✗ dns  : 1
  ✗ tls  : 1
  → exit 5 (tls)

All-ok multi-URL run → Summary: 3 URLs — 3 ok, no ✗ lines, → exit 0 (ok).

Changes

All in hxprobe/hxprobe/cli.py unless noted.

1. Reverse label map next to the exit-code constants (cli.py:16-32): a small _EXIT_LABELS: dict[int, str] mapping EXIT_DNS→"dns", EXIT_CONNECT→"connect", EXIT_TIMEOUT→"timeout", EXIT_TLS→"tls", EXIT_HTTP→"http", EXIT_OK→"ok". (Inverse of the existing forward mapping; kept explicit for readability, matching the codebase's style.)

2. Refactor the accumulation loop (cli.py:454-462) to compute a per-URL code and fold it into worst, collecting url_codes: list[int] (one per URL, index-aligned with urls). This also unifies the two "running max" idioms flagged in docs/explanations/2026-07-02-13-25-hxprobe-worst-exit-code-and-render-loop.md (if c > worst vs max(...)) into one — a small simplification bonus.

3. New _print_run_summary(urls, url_codes, worst, out) helper (near the other _print_* renderers): builds the ok/failed tally + per-class counts from url_codes and writes the footer. No-op guard is the caller's len(urls) > 1 check.

4. Call site after the loop (cli.py:470-475): in text mode only (if not ns.json_out and len(urls) > 1:), call _print_run_summary(...) before return worst. JSON path untouched — still json.dumps(json_items) as a bare array.

Tests (hxprobe/tests/test_cli.py)

Add hermetic tests (reuse the existing _OKHandler/_start_server/_invoke harness and _free_port for a refused connection):

• multi-URL mixed — one OK server URL + one http://127.0.0.1:<free> (connection refused): assert code == EXIT_CONNECT, and the footer strings are present ("Summary: 2 URLs", "1 ok", "1 failed", "connect : 1", "→ exit 3").
• multi-URL all ok — two OK URLs: assert code == EXIT_OK and "Summary: 2 URLs — 2 ok" present; verify no "✗" in the footer region.
• single URL has NO footer — one OK URL: assert "Summary:" NOT in out (locks the multi-URL-only rule).

Backward-compat guard: the footer contains no "200" substring, so the existing test_reads_urls_from_file assertion out.count("200") == 2 still holds; all 13 existing exit-code tests are unaffected (worst-code unchanged).

Docs / project conventions (per CLAUDE.md)

• Copy this approved plan into docs/plans/<yyyy-mm-dd-hh-mm>-hxprobe-run-summary-footer.md as the first implementation step (before code) — see the feedback_plan_mode_docs_plans memory note.
• hxprobe/USAGE.md: add a short "Multi-URL summary footer" subsection with a real captured example; add a sentence to the Exit-codes section noting the footer shows the per-class breakdown behind the scalar. Exit-code table itself is unchanged.
• docs/usage/hxprobe.md: one-line mention if it lists features.
• New docs/explanations/<ts>-hxprobe-run-summary-footer.md is optional; the existing worst-exit-code explanation can get a short "Update:" pointer.
• CHANGELOG.md: new timestamped entry.
• docs/summaries/<ts>-hxprobe-run-summary-footer.md: implementation summary.

Verification

1. cd hxprobe && .venv/bin/python -m pytest tests/test_cli.py tests/test_probe.py -q — all existing + new tests pass.
2. .venv/bin/ruff check hxprobe/ — clean.
3. Manual, capturing exit codes:

   .venv/bin/python -m hxprobe https://example.com https://example.org   # footer, exit 0
   .venv/bin/python -m hxprobe https://example.com http://no.such.host.invalid; echo $?   # footer w/ dns:1, exit 2
   .venv/bin/python -m hxprobe https://example.com   # single URL: NO footer, unchanged
   .venv/bin/python -m hxprobe --json https://example.com https://example.org   # bare JSON array, NO footer