docs: hxprobe plans, summaries, explanations, usage, and changelog
Plans (docs/plans/): - 2026-07-01-23-47-py-hxprobe-httpx.md — initial httpx probe design - 2026-07-02-09-32 through 14-05 — standalone project, toolchain, usage doc + Makefile, file input (-f), simplification pass, run-summary footer Summaries (docs/summaries/): one per completed feature, recording what was actually built, deviations from the plan, and verification steps Explanations (docs/explanations/): two deep-dives written during review — hxprobe concurrency model and worst-exit-code + render-loop analysis Usage (docs/usage/hxprobe.md): overview with pointer to hxprobe/USAGE.md for the full runnable reference Walkthrough (docs/py-latprobe-walkthrough.md): narrative tour of the latprobe Python package for interview / code-review context CHANGELOG.md: entries for all hxprobe features (toolchain, usage doc, file input, simplification, run-summary footer) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
247
CHANGELOG.md
247
CHANGELOG.md
@@ -4,6 +4,253 @@ All completed features are logged here in reverse-chronological order.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-02 14:05 — `hxprobe`: end-of-run summary footer for multi-URL runs
|
||||
|
||||
- `hxprobe/hxprobe/cli.py`: when more than one URL is probed (positional args
|
||||
or `-f`), a footer is now appended after the last URL block — a tally of
|
||||
every URL's outcome (`N ok`, `M failed` broken down by class) plus the
|
||||
`→ exit N (label)` line tying it to the process exit code. Single-URL text
|
||||
output and JSON output (still a bare array) are both byte-identical to
|
||||
before — new `_print_run_summary()`, gated on `len(urls) > 1` and text mode
|
||||
only
|
||||
- The process **exit code itself is unchanged**: still the highest severity
|
||||
across all URLs (worst-code wins), matching `latprobe`/Go and the 13
|
||||
existing exit-code tests — a deliberate decision, since that scalar is a
|
||||
documented cross-implementation contract (`hxprobe/USAGE.md`'s Exit codes
|
||||
table). The footer exists to give humans the per-URL breakdown the scalar
|
||||
can't show, not to change what gets returned
|
||||
- New `_EXIT_LABELS` (inverse of `_PHASE_EXIT`) for rendering exit codes as
|
||||
short labels (`dns`, `tls`, `http`, …) in the footer
|
||||
- Refactored the per-URL accumulation loop to compute one worst-code per URL
|
||||
first, then fold into the global `worst` — this also unified the two
|
||||
"running max" idioms (`if c > worst: worst = c` vs `max(worst, ...)`) that
|
||||
`docs/explanations/2026-07-02-13-25-hxprobe-worst-exit-code-and-render-loop.md`
|
||||
had flagged as a stylistic wrinkle into a single `max(...)` call
|
||||
- `hxprobe/tests/test_cli.py`: 3 new hermetic tests (`TestCLIRunSummary`) —
|
||||
multi-URL mixed outcome, multi-URL all-ok, and single-URL-has-no-footer;
|
||||
all 33 pre-existing tests pass unedited
|
||||
- Triggered by a follow-up question: does a single worst-code exit even make
|
||||
sense across multiple URLs with different error classes? Answer: keep the
|
||||
scalar (parity contract) but add the missing visibility as output, not by
|
||||
changing the exit code's semantics
|
||||
- `hxprobe/USAGE.md`: new "Multi-URL summary footer" section with real
|
||||
captured output; refreshed the "Multiple URLs" and `-f` multi-URL examples
|
||||
to show the footer (previously stale — captured before this feature
|
||||
existed); added a sentence to the Exit codes section
|
||||
- `docs/usage/hxprobe.md`: one-line mention pointing at the new section
|
||||
- Plan: `docs/plans/2026-07-02-14-05-hxprobe-run-summary-footer.md`
|
||||
- Summary: `docs/summaries/2026-07-02-14-05-hxprobe-run-summary-footer.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-02 12:05 — `hxprobe` simplification pass (no behavior change)
|
||||
|
||||
- `hxprobe/hxprobe/cli.py`: deleted the `_Parser`/`_ArgExit` scaffolding
|
||||
(~32 lines) that hand-reimplemented what `argparse.ArgumentParser` already
|
||||
does (help→stdout, usage/errors→stderr, raise instead of hard-exit);
|
||||
replaced with a plain `ArgumentParser` wrapped in
|
||||
`contextlib.redirect_stdout`/`redirect_stderr`, catching `SystemExit` at one
|
||||
site
|
||||
- `hxprobe/hxprobe/probe.py`: trimmed `_TimingStream.get_extra_info` to the
|
||||
one branch (`ssl_object`) actually consumed on hxprobe's request path — the
|
||||
`server_addr`/`client_addr` branches were dead (only `httpx`'s own CLI
|
||||
queries them); removed `_dns_set`/`_connect_set`/`_tls_set` from `_Trace`,
|
||||
redundant with the `Phase.present` flag already on `self.dns`/`connect`/`tls`
|
||||
- `hxprobe/hxprobe/cli.py`: `_load_urls` no longer takes only the first
|
||||
whitespace token per line ("forward-compatible with future `key=value`
|
||||
annotations" that never materialized); extracted `_summarize_failures()`
|
||||
to remove a duplicated dedup loop shared by `_print_failure_summary` and
|
||||
`_build_json_entry`
|
||||
- Triggered by a question about a dead `file=` parameter on
|
||||
`_Parser.print_help`/`print_usage`; verified no behavior changed — all 33
|
||||
existing tests pass unedited, plus manual smoke tests of `-h`, no-args
|
||||
(stdout/stderr routing double-checked via separate file redirection), and a
|
||||
live verbose request
|
||||
- Plan: `docs/plans/2026-07-02-12-05-hxprobe-simplification.md`
|
||||
- Summary: `docs/summaries/2026-07-02-12-05-hxprobe-simplification.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-02 11:14 — `hxprobe`: read target URLs from a file (`-f`/`--file`)
|
||||
|
||||
- `hxprobe/hxprobe/cli.py`: new `-f`/`--file PATH` flag reads URLs from a
|
||||
plain-text file (one per line, `#` comments, blank lines skipped, first
|
||||
whitespace-separated token per line) — same format as `simple.py`'s
|
||||
`load_sites()`, reimplemented locally as `_load_urls()` rather than
|
||||
imported (hxprobe still imports nothing outside its own directory)
|
||||
- `-f`/`--file` is **mutually exclusive** with positional `url` args (both
|
||||
or neither given → usage error); `urls` positional changed from
|
||||
`nargs="+"` to `nargs="*"` to allow the file-only case
|
||||
- Missing file / unreadable file / empty file all produce a clear usage
|
||||
error (`EXIT_USAGE`) rather than a traceback
|
||||
- `hxprobe/configs/*.txt`: 7 new fixtures mirroring `python/configs/`'s set
|
||||
(all-ok, dns-failure, connection-refused, timeout, tls-errors,
|
||||
http-errors, mixed), each with a verified expected exit code in its
|
||||
header comment — actually run against `hxprobe -f ...` during
|
||||
implementation, not assumed from the `simple.py` originals (whose
|
||||
blanket 0/1 exit scheme differs from hxprobe's per-failure-class 0–6)
|
||||
- `hxprobe/tests/test_cli.py`: 5 new hermetic tests (`TestCLIFileInput`) —
|
||||
successful multi-URL read from a temp file, missing file, empty file,
|
||||
both-sources-given, and neither-given error paths
|
||||
- `hxprobe/USAGE.md`: new "Reading URLs from a file (`-f`)" section with
|
||||
real captured output, placed after "Multiple URLs"
|
||||
- Plan: `docs/plans/2026-07-02-11-14-hxprobe-file-input.md`
|
||||
- Summary: `docs/summaries/2026-07-02-11-14-hxprobe-file-input.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-02 10:22 — `hxprobe` usage reference doc + standalone Makefile
|
||||
|
||||
- `hxprobe/USAGE.md`: new "Runnable Usage Reference" matching the depth of
|
||||
`python/configs/usage-latprobe.md` — 16 cases with real captured output
|
||||
(basic, verbose × HTTPS/plain-HTTP/redirect-followed/`--no-follow-
|
||||
redirects`/`--no-http2`/TLS-failure/DNS-failure, sampling, multi-URL,
|
||||
`--fail`, JSON, JSON+verbose, timeout, exit-codes table, Makefile
|
||||
shortcuts). Placed inside `hxprobe/` itself (not `python/configs/`) so it
|
||||
travels with the project if extracted to its own repo
|
||||
- Timeout example uses `192.0.2.1` (RFC 5737 TEST-NET-1) instead of
|
||||
`10.255.255.1` — the latter resolves to an immediate "connection refused"
|
||||
in this dev sandbox rather than a genuine timeout; `192.0.2.1` reproduces
|
||||
a real ~500ms timeout reliably
|
||||
- `docs/usage/hxprobe.md`: added a one-line pointer to `hxprobe/USAGE.md` at
|
||||
the top; no other changes — it stays the CLAUDE.md-mandated summary doc
|
||||
- `hxprobe/Makefile`: new, fully standalone (`help`, `deps`, `run`, `lint`,
|
||||
`fmt`, `test`, `test-integration`, `check`, `clean`) — same auto-generated
|
||||
`## comment` help style as the root Makefile, short target names (no
|
||||
`hx-` prefix needed inside `hxprobe/`'s own scope). Deliberately
|
||||
independent from the root Makefile's `hx-*` targets — neither calls into
|
||||
the other, per explicit user decision (avoids one Makefile's changes
|
||||
silently breaking the other, at the cost of some duplicated `uv`/`pytest`
|
||||
invocation logic)
|
||||
- No changes to the root `Makefile` — verified its `hx-*` section is
|
||||
byte-for-byte unchanged
|
||||
- Plan: `docs/plans/2026-07-02-10-22-hxprobe-usage-doc-and-makefile.md`
|
||||
- Summary: `docs/summaries/2026-07-02-10-22-hxprobe-usage-doc-and-makefile.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-02 09:57 — `hxprobe` toolchain modernization (uv, ruff, pytest)
|
||||
|
||||
- Adopted `uv` for environment/dependency management: `hxprobe/pyproject.toml`
|
||||
gets a `[dependency-groups] dev = ["pytest>=8.0", "ruff>=0.8"]` section;
|
||||
`hxprobe/.python-version` pins Python 3.14; `hxprobe/uv.lock` (new, 18
|
||||
packages resolved) pins every transitive dependency (`httpx`, `httpcore`,
|
||||
`h2`, `certifi`, etc.) for reproducible installs — previously nothing was
|
||||
pinned beyond `httpx[http2]>=0.28`
|
||||
- Added `ruff` for linting + formatting: `[tool.ruff]` config
|
||||
(`target-version = "py311"`, `line-length = 100`); fixed the one real
|
||||
finding (`import sys` unused in `cli.py`, inherited from the original
|
||||
`latprobe/cli.py`); ran `ruff format` across the project (6 files
|
||||
reformatted — mostly collapsing the hand-aligned `=`/dict-key columns to
|
||||
single-space, no semantic changes; verified via full syntax check + test
|
||||
run before and after)
|
||||
- Swapped the test runner from stdlib `unittest discover` to `pytest`.
|
||||
Existing `unittest.TestCase` classes run unchanged (pytest is a superset
|
||||
runner) — no test-code rewrite. Replaced the `test_[!i]*.py` filename-glob
|
||||
hermetic/integration split with a proper `pytest.mark.integration` marker
|
||||
(`pytestmark = pytest.mark.integration` in `tests/test_integration.py`);
|
||||
registered in `[tool.pytest.ini_options]` to avoid unknown-marker warnings
|
||||
- Skipped mypy for now (type hints stay as documentation only) — explicit
|
||||
user decision, not an oversight
|
||||
- `hxprobe/README.md`: new — quick start (`uv sync`, `uv run ...`) for the
|
||||
standalone project, independent of the parent repo's docs
|
||||
- Makefile: `hx-deps`/`hx-run`/`hx-test`/`hx-test-integration` now shell out
|
||||
to `uv sync`/`uv run` instead of hand-managed `python -m venv` + `pip
|
||||
install -e .` (removed `HX_VENV`/`HX_VENV_PYTHON` vars — uv owns this now);
|
||||
added `hx-lint`/`hx-fmt`; `hx-check` now runs lint + hermetic tests
|
||||
(mirrors `go-check`'s fmt+vet+test bundling, previously only ran tests)
|
||||
- `docs/usage/hxprobe.md`: updated Setup and Makefile-targets sections for
|
||||
the new toolchain
|
||||
- `.gitignore`: added `.pytest_cache/`/`.ruff_cache/`
|
||||
- No behavior change to the probe/CLI itself — confirmed identical output
|
||||
before/after, `python/latprobe` untouched, zero `latprobe` imports remain
|
||||
in `hxprobe/`
|
||||
- Plan: `docs/plans/2026-07-02-09-57-hxprobe-toolchain-modernization.md`
|
||||
- Summary: `docs/summaries/2026-07-02-09-57-hxprobe-toolchain-modernization.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-02 09:32 — `hxprobe` extracted into a standalone top-level project
|
||||
|
||||
- Moved `hxprobe` out of `python/` into a new top-level `hxprobe/` directory
|
||||
(sibling of `go/` and `python/`), with its own `pyproject.toml`, its own
|
||||
venv (`hxprobe/.venv`), and its own `hxprobe/tests/` — structured so it
|
||||
could be `cp -r`'d into a separate repo and work unchanged
|
||||
- Removed every `from latprobe import ...` in `hxprobe/`: `probe.py` now
|
||||
defines its own `Options`/`Phase`/`Result`/`VerboseDetail`/`CertInfo`
|
||||
dataclasses and its own `_parse_cert`/`_parse_cert_date` helpers (copied,
|
||||
not shared); `aggregate.py`/`duration.py` are verbatim copies (their
|
||||
imports were already package-relative, so no edits needed);
|
||||
`cli.py` is now a full standalone implementation (own exit codes, argparse,
|
||||
text/JSON rendering) instead of delegating to `latprobe.cli.run()` via an
|
||||
injected `measure_fn`
|
||||
- Reverted `python/latprobe/{cli.py,probe.py}` to their pre-`hxprobe` state
|
||||
(`git checkout --`) — the `measure_fn`/`prog`/`description`/`protocol_flags`
|
||||
injection points, `Options.follow_redirects`/`http2`, and
|
||||
`VerboseDetail.http_version`/`redirect_count` only existed to support the
|
||||
now-removed sharing; `python/` is back to zero third-party dependencies, no
|
||||
`pyproject.toml`, no venv
|
||||
- Makefile: removed `py-deps` and the `PY_VENV*` variables; `py-test`/
|
||||
`py-check` reverted to running directly against `$(PYTHON)`; added a new
|
||||
standalone `hx-deps`/`hx-run`/`hx-test`/`hx-test-integration`/`hx-check`/
|
||||
`hx-clean` section using `HX_DIR`/`HX_VENV*`; umbrella `test`/`check`/
|
||||
`clean` now include the hxprobe targets alongside Go and Python
|
||||
(`go-*`/`py-*`/`hx-*`)
|
||||
- Tests moved and renamed to drop the now-redundant `hx_`/`_hx` segments:
|
||||
`test_hx_probe.py` → `hxprobe/tests/test_probe.py`,
|
||||
`test_hx_cli.py` → `hxprobe/tests/test_cli.py`,
|
||||
`test_integration_hx.py` → `hxprobe/tests/test_integration.py` (same
|
||||
`test_[!i]*.py` hermetic-vs-integration exclusion convention as `latprobe`)
|
||||
- `docs/usage/py-hxprobe.md` renamed to `docs/usage/hxprobe.md` and rewritten
|
||||
for the new standalone structure and Makefile targets; the TCP_NODELAY/
|
||||
Nagle TTFB-accuracy finding is preserved
|
||||
- No behavior change — same CLI flags, same output, same six-phase timing,
|
||||
same HTTP/2/redirect handling as before; this was a pure decoupling/
|
||||
restructuring
|
||||
- Plan: `docs/plans/2026-07-02-09-32-hxprobe-standalone-project.md`
|
||||
- Summary: `docs/summaries/2026-07-02-09-32-hxprobe-standalone-project.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-02 00:24 — `hxprobe`: httpx-based Python probe, Go-client parity
|
||||
|
||||
- New sibling package `python/hxprobe/`, built on `httpx` instead of raw
|
||||
sockets, so the client matches Go's `http.DefaultClient`: HTTP/2 negotiated
|
||||
via ALPN, redirects followed by default, connection pooling, default TLS
|
||||
verification — while still reporting the full six-phase breakdown (DNS, TCP
|
||||
connect, TLS, TTFB, Transfer, Total)
|
||||
- `hxprobe/probe.py`: DNS/TCP connect/TLS timed by instrumenting a custom
|
||||
httpcore `NetworkBackend`/`NetworkStream` (`_TimingBackend`/`_TimingStream`);
|
||||
HTTP framing (HTTP/1.1 or HTTP/2), redirects, and keep-alive stay entirely
|
||||
owned by httpx; `measure()` returns the same `latprobe.probe.Result`
|
||||
dataclass, so aggregation/rendering are reused unchanged
|
||||
- `latprobe/probe.py`: added `Options.follow_redirects`/`Options.http2`
|
||||
(ignored by the raw-socket `measure()`) and
|
||||
`VerboseDetail.http_version`/`redirect_count` (always `""`/`0` there)
|
||||
- `latprobe/cli.py`: `run()`/`_run_samples()` take an injectable `measure_fn`
|
||||
(default: the existing socket `measure`), plus `prog`/`description`/
|
||||
`protocol_flags` overrides — `hxprobe.cli.run()` reuses the entire argparse,
|
||||
concurrency, exit-code, and text/JSON rendering pipeline unchanged
|
||||
- New CLI flags (hxprobe only, via `protocol_flags=True`): `--no-http2`,
|
||||
`--no-follow-redirects`
|
||||
- `python/pyproject.toml`: first third-party dependency in this repo
|
||||
(`httpx[http2]`); `make py-deps` provisions `python/.venv`
|
||||
- Verified experimentally that `latprobe`'s raw socket — which never sets
|
||||
`TCP_NODELAY` — pays a real ~40-50ms Nagle/delayed-ACK penalty on its TTFB
|
||||
phase; `hxprobe` sets `TCP_NODELAY` (matching httpcore's default and Go's
|
||||
`net.Dialer`) and does not. Documented in `docs/usage/py-hxprobe.md` as a
|
||||
known divergence — `hxprobe`'s TTFB is the more accurate of the two, not
|
||||
just different
|
||||
- 17 new hermetic tests (`tests/test_hx_probe.py`), 11 new hermetic CLI tests
|
||||
(`tests/test_hx_cli.py`), 9 new live integration tests
|
||||
(`tests/test_integration_hx.py`, excluded from the default gate)
|
||||
- Makefile: `py-deps`, `hx-run`, `hx-test-integration`; `py-test`/`py-check`
|
||||
now run via the venv and include the new hermetic hxprobe tests
|
||||
- User doc: `docs/usage/py-hxprobe.md`
|
||||
- Plan: `docs/plans/2026-07-01-23-47-py-hxprobe-httpx.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-01 12:55 — `--verbose` / `-v` flag for `latprobe` (Python)
|
||||
|
||||
- `-v`/`--verbose` flag added to the `latprobe` CLI
|
||||
|
||||
Reference in New Issue
Block a user