http-latency-prober/docs/plans/2026-07-01-23-47-py-hxprobe-httpx.md

Plan: hxprobe — httpx-based Python probe (Go-client parity)

Context

The existing Python latprobe package measures per-phase HTTP latency with raw sockets. That gives an excellent DNS/TCP/TLS/TTFB/Transfer breakdown, but the cost is that it always speaks HTTP/1.1 and does not follow redirects — diverging from the Go implementation, whose http.DefaultClient negotiates HTTP/2 via ALPN, follows redirects (up to 10), pools connections, and verifies TLS by default (confirmed: go/internal/probe/probe.go uses http.DefaultClient.Do with no custom transport/CheckRedirect).

Goal: add a second Python implementation, hxprobe, built on the httpx library so it matches the Go client's protocol capabilities (HTTP/2, redirects, pooling, TLS verification) while preserving the full 6-phase timing that is latprobe's whole point. Python has no equivalent of Go's net/http/httptrace, so the phase breakdown is recovered by instrumenting httpx's network backend.

This is an additive, Python-only experiment — intentionally outside the CLAUDE.md "Go first, then Python port" flow, since the user explicitly asked for a library-based Python variant. No Go change is required.

Approach

New sibling package python/hxprobe/, reusing everything reusable from latprobe (dataclasses, aggregation, duration parsing, CLI rendering) so the only genuinely new code is the httpx probe backend.

Key design: instrumented httpx transport

httpx (sync httpx.Client) runs on httpcore. To recover per-phase timing we subclass httpcore's sync network backend and time the phases at the socket level, letting httpx own HTTP framing, HTTP/2, redirects, and keep-alive:

• connect_tcp(...) — reimplement DNS + TCP as separate steps (port the socket.getaddrinfo → socket.connect split already in latprobe/probe.py:167-203), timestamping DNS and TCP connect independently, and capturing the resolved IP.
• start_tls(...) — timestamp the TLS handshake; pull negotiated version / cipher / peer cert from the SSL object for verbose mode.
• TTFB = headers-received minus end-of-TLS (server processing), measured via client.stream("GET", ...) (the stream() context yields once response headers arrive).
• Transfer = iterating resp.iter_raw() to EOF, minus headers-received.
• Total = wraps the whole measure() call.

Each measure() call uses a fresh httpx.Client (no cross-sample pooling) so every -n sample yields a full phase breakdown — matching the current raw-socket latprobe behavior rather than Go's pool-reuse quirk.

A per-call trace object (held by the backend instance) records timings and the fail_phase at the exact point a phase raises, giving precise error classification (dns/connect/timeout/tls/transfer/request) without guessing from httpx exception types.

Redirects (followed by default, Go parity): DNS/connect/TLS are reported from the first connection (mirrors Go's connectStart.IsZero() guard); TTFB/Transfer/Total span the full followed chain. redirect_count and the negotiated http_version (h2 vs http/1.1) are surfaced as new verbose fields — a genuine capability the socket version lacks.

Files

New:

• python/pyproject.toml — project metadata; dependency httpx[http2] (pulls h2). Makes latprobe + hxprobe pip install -e .-able; tests still run via PYTHONPATH.
• python/hxprobe/__init__.py
• python/hxprobe/probe.py — measure(url, opts) -> latprobe.probe.Result (imports & returns the same Result so aggregation/rendering just work); _TimingBackend, the per-call trace, error classification, verbose capture.
• python/hxprobe/cli.py — thin: delegates to latprobe.cli.run(...) passing measure_fn=hxprobe.probe.measure (see reuse edit below).
• python/hxprobe/__main__.py — sys.exit(cli.run(sys.argv[1:], ...)).
• python/tests/test_hx_probe.py — hermetic, local http.server: phase presence, redirect following (302 handler — validates the Go-parity feature), connection-refused → connect, black-hole port → timeout.
• python/tests/test_hx_cli.py — hermetic CLI via run() with io.StringIO (mirrors tests/test_cli.py:73-76).
• python/tests/test_integration_hx.py — live, excluded from default gate (filename starts test_i…, so the test_[!i]*.py glob skips it): probe a real HTTP/2 host and assert negotiated http_version == "h2"; guard with a @skipUnless(_online()) like tests/test_integration.py:29-40.
• docs/usage/py-hxprobe.md — usage doc (what it does, flags, example with expected output, and an explicit socket-vs-httpx capability comparison table), per CLAUDE.md.

Edited (small, backward-compatible):

• python/latprobe/cli.py — parameterize run() and _run_samples() with an injectable measure_fn (default = current latprobe.probe.measure), so hxprobe reuses all argparse, concurrency, exit-code, text/JSON rendering logic. Extend _print_verbose_block and _build_json_entry to show http_version / redirect_count when present (existing socket path never sets them → output unchanged).
• python/latprobe/probe.py — add optional fields with safe defaults: Options.follow_redirects=True, Options.http2=True (ignored by the socket measure); VerboseDetail.http_version="", VerboseDetail.redirect_count=0.
• Makefile — add py-deps (create .venv, pip install -e python), hx-run (cd python && python -m hxprobe $(ARGS)), and fold test_hx_* into the existing py-test gate; add hx-test-integration for the live h2 check.
• CHANGELOG.md — append a timestamped one-line entry on completion.

Reused as-is: latprobe.aggregate.summarize, latprobe.duration.parse_duration, latprobe.probe.{Result,Phase,CertInfo}, and the entire latprobe.cli renderer via the measure_fn injection.

Flags / behavior

Same surface as latprobe (-n/--count, -c/--concurrency, --timeout, --fail, --json, -v/--verbose) via the reused parser, plus two new opt-outs for the Go-like defaults: --no-http2 and --no-follow-redirects. Exit codes 0–6 stay identical to Go/latprobe.

Verification

1. make py-deps — create venv, install httpx[http2].
2. make hx-run ARGS="https://example.com" — full 6-phase text output renders.
3. make hx-run ARGS="-v https://www.cloudflare.com" — verbose block shows http_version: h2 and TLS/cert details.
4. Go-parity spot checks:
   • HTTP/2: python -m hxprobe -v <h2-host> reports h2 where python -m latprobe -v <h2-host> reports HTTP/1.1.
   • Redirects: python -m hxprobe http://github.com follows to https and shows redirect_count > 0 (socket latprobe shows a raw 301).
5. make py-test — hermetic suite (now including test_hx_probe.py, test_hx_cli.py) is green; latprobe's existing tests still pass (proves the measure_fn/Options/VerboseDetail edits are backward-compatible).
6. make hx-test-integration — live test confirms real h2 negotiation.
7. Confirm --json output for hxprobe matches the latprobe schema plus the optional verbose.http_version / verbose.redirect_count keys.