Jan Novak
f487a4b1bd
Plans (docs/plans/): - 2026-07-01-23-47-py-hxprobe-httpx.md — initial httpx probe design - 2026-07-02-09-32 through 14-05 — standalone project, toolchain, usage doc + Makefile, file input (-f), simplification pass, run-summary footer Summaries (docs/summaries/): one per completed feature, recording what was actually built, deviations from the plan, and verification steps Explanations (docs/explanations/): two deep-dives written during review — hxprobe concurrency model and worst-exit-code + render-loop analysis Usage (docs/usage/hxprobe.md): overview with pointer to hxprobe/USAGE.md for the full runnable reference Walkthrough (docs/py-latprobe-walkthrough.md): narrative tour of the latprobe Python package for interview / code-review context CHANGELOG.md: entries for all hxprobe features (toolchain, usage doc, file input, simplification, run-summary footer) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
137 lines
6.4 KiB
Markdown
137 lines
6.4 KiB
Markdown
# hxprobe: end-of-run summary footer for multi-URL runs
|
|
|
|
## Context
|
|
|
|
A single scalar exit code is inherently lossy when probing multiple URLs with
|
|
different failure classes: today `run()` returns the numeric max across all
|
|
URLs (e.g. DNS-fail on one URL + TLS-fail on another → exit `5`, and the DNS
|
|
failure is invisible in the code). The failures *are* all printed per-URL, but
|
|
there's no consolidated view — for many URLs you must scroll and eyeball each
|
|
block to know what happened overall.
|
|
|
|
The exit code's job is a coarse pass/fail + severity hint for scripts, and it's
|
|
a **deliberate cross-implementation contract** shared with `latprobe` and the
|
|
Go version (documented in `hxprobe/USAGE.md:475-488`, asserted by 13 tests in
|
|
`tests/test_cli.py`). So we keep the worst-code exit unchanged and instead give
|
|
humans the full picture the scalar can't: an **end-of-run summary footer** that
|
|
tallies every URL's outcome and shows how the exit code was derived.
|
|
|
|
Decisions (confirmed with the user):
|
|
- Exit code: **unchanged** (worst/highest severity across URLs).
|
|
- Summary: **text footer, multi-URL only** (`len(urls) > 1`). Single-URL text
|
|
output stays byte-identical; JSON output stays a bare array (unchanged).
|
|
|
|
## Design
|
|
|
|
### Per-URL classification (reuses existing helpers)
|
|
Each URL gets one "worst outcome code" using the *existing* mapping — no new
|
|
severity scheme:
|
|
- start at `EXIT_OK`;
|
|
- for each failed sample: `code = max(code, _phase_code(r.fail_phase))`
|
|
(`_phase_code` at `cli.py:31`);
|
|
- if `--fail`: for each succeeded sample with `status_code >= 400`:
|
|
`code = max(code, EXIT_HTTP)`.
|
|
|
|
A URL is "ok" iff its code is `EXIT_OK`, else "failed" and bucketed by its code.
|
|
A partially-failed URL (some samples ok, some failed) classifies by its worst
|
|
sample — consistent with how its own block and the global exit code already
|
|
behave.
|
|
|
|
### Footer format (text, only when `len(urls) > 1`)
|
|
Printed once after the last URL block, before `return worst`. Failure classes
|
|
use the same `✗` bullet style as `_print_failure_summary` (`cli.py:219-224`).
|
|
The final `→ exit N (label)` line explicitly ties the tally to the returned
|
|
code — directly answering "why is the exit code what it is". Example (3 URLs):
|
|
|
|
```
|
|
https://example.com (200)
|
|
... phase rows ...
|
|
|
|
http://no.such.host.invalid (FAILED)
|
|
✗ dns: [Errno 8] nodename nor servname provided
|
|
|
|
https://self-signed.badssl.com (FAILED)
|
|
✗ tls: certificate verify failed
|
|
|
|
─────────────────────────────────────────────────
|
|
Summary: 3 URLs — 1 ok, 2 failed
|
|
✗ dns : 1
|
|
✗ tls : 1
|
|
→ exit 5 (tls)
|
|
```
|
|
|
|
All-ok multi-URL run → `Summary: 3 URLs — 3 ok`, no `✗` lines, `→ exit 0 (ok)`.
|
|
|
|
## Changes
|
|
|
|
All in `hxprobe/hxprobe/cli.py` unless noted.
|
|
|
|
1. **Reverse label map** next to the exit-code constants (`cli.py:16-32`): a
|
|
small `_EXIT_LABELS: dict[int, str]` mapping `EXIT_DNS→"dns"`,
|
|
`EXIT_CONNECT→"connect"`, `EXIT_TIMEOUT→"timeout"`, `EXIT_TLS→"tls"`,
|
|
`EXIT_HTTP→"http"`, `EXIT_OK→"ok"`. (Inverse of the existing forward mapping;
|
|
kept explicit for readability, matching the codebase's style.)
|
|
|
|
2. **Refactor the accumulation loop** (`cli.py:454-462`) to compute a per-URL
|
|
code and fold it into `worst`, collecting `url_codes: list[int]` (one per
|
|
URL, index-aligned with `urls`). This also unifies the two "running max"
|
|
idioms flagged in
|
|
`docs/explanations/2026-07-02-13-25-hxprobe-worst-exit-code-and-render-loop.md`
|
|
(`if c > worst` vs `max(...)`) into one — a small simplification bonus.
|
|
|
|
3. **New `_print_run_summary(urls, url_codes, worst, out)` helper** (near the
|
|
other `_print_*` renderers): builds the ok/failed tally + per-class counts
|
|
from `url_codes` and writes the footer. No-op guard is the caller's
|
|
`len(urls) > 1` check.
|
|
|
|
4. **Call site** after the loop (`cli.py:470-475`): in text mode only
|
|
(`if not ns.json_out and len(urls) > 1:`), call `_print_run_summary(...)`
|
|
before `return worst`. JSON path untouched — still `json.dumps(json_items)`
|
|
as a bare array.
|
|
|
|
## Tests (`hxprobe/tests/test_cli.py`)
|
|
|
|
Add hermetic tests (reuse the existing `_OKHandler`/`_start_server`/`_invoke`
|
|
harness and `_free_port` for a refused connection):
|
|
- **multi-URL mixed** — one OK server URL + one `http://127.0.0.1:<free>`
|
|
(connection refused): assert `code == EXIT_CONNECT`, and the footer strings
|
|
are present (`"Summary: 2 URLs"`, `"1 ok"`, `"1 failed"`, `"connect : 1"`,
|
|
`"→ exit 3"`).
|
|
- **multi-URL all ok** — two OK URLs: assert `code == EXIT_OK` and
|
|
`"Summary: 2 URLs — 2 ok"` present; verify no `"✗"` in the footer region.
|
|
- **single URL has NO footer** — one OK URL: assert `"Summary:"` NOT in `out`
|
|
(locks the multi-URL-only rule).
|
|
|
|
Backward-compat guard: the footer contains no `"200"` substring, so the
|
|
existing `test_reads_urls_from_file` assertion `out.count("200") == 2` still
|
|
holds; all 13 existing exit-code tests are unaffected (worst-code unchanged).
|
|
|
|
## Docs / project conventions (per CLAUDE.md)
|
|
|
|
- **Copy this approved plan** into
|
|
`docs/plans/<yyyy-mm-dd-hh-mm>-hxprobe-run-summary-footer.md` as the first
|
|
implementation step (before code) — see the `feedback_plan_mode_docs_plans`
|
|
memory note.
|
|
- `hxprobe/USAGE.md`: add a short "Multi-URL summary footer" subsection with a
|
|
real captured example; add a sentence to the Exit-codes section noting the
|
|
footer shows the per-class breakdown behind the scalar. Exit-code table
|
|
itself is unchanged.
|
|
- `docs/usage/hxprobe.md`: one-line mention if it lists features.
|
|
- New `docs/explanations/<ts>-hxprobe-run-summary-footer.md` is optional; the
|
|
existing worst-exit-code explanation can get a short "Update:" pointer.
|
|
- `CHANGELOG.md`: new timestamped entry.
|
|
- `docs/summaries/<ts>-hxprobe-run-summary-footer.md`: implementation summary.
|
|
|
|
## Verification
|
|
|
|
1. `cd hxprobe && .venv/bin/python -m pytest tests/test_cli.py tests/test_probe.py -q`
|
|
— all existing + new tests pass.
|
|
2. `.venv/bin/ruff check hxprobe/` — clean.
|
|
3. Manual, capturing exit codes:
|
|
```
|
|
.venv/bin/python -m hxprobe https://example.com https://example.org # footer, exit 0
|
|
.venv/bin/python -m hxprobe https://example.com http://no.such.host.invalid; echo $? # footer w/ dns:1, exit 2
|
|
.venv/bin/python -m hxprobe https://example.com # single URL: NO footer, unchanged
|
|
.venv/bin/python -m hxprobe --json https://example.com https://example.org # bare JSON array, NO footer
|
|
```
|