# Summary: `hxprobe` — httpx-based Python probe

Plan: [docs/plans/2026-07-01-23-47-py-hxprobe-httpx.md](../plans/2026-07-01-23-47-py-hxprobe-httpx.md)

## What was built

A new sibling package, `python/hxprobe/`, alongside the existing raw-socket
`latprobe`. It uses `httpx` so the client matches Go's `http.DefaultClient`:
HTTP/2 negotiated via ALPN, redirects followed by default, connection
pooling, and default TLS verification — while still reporting the full
six-phase breakdown (DNS, TCP connect, TLS, TTFB, Transfer, Total).

Files:
- `python/pyproject.toml` — first third-party dependency in this repo
  (`httpx[http2]`)
- `python/hxprobe/probe.py` — the core: `_Trace`, `_TimingStream`,
  `_TimingBackend`, `_TimingTransport`, `measure()`
- `python/hxprobe/cli.py`, `__main__.py`, `__init__.py` — thin wrappers
- `python/latprobe/probe.py` (edited) — added `Options.follow_redirects`/
  `Options.http2` (ignored by the socket `measure()`) and
  `VerboseDetail.http_version`/`redirect_count` (always `""`/`0` there)
- `python/latprobe/cli.py` (edited) — `run()`/`_run_samples()` take an
  injectable `measure_fn`, plus `prog`/`description`/`protocol_flags`
  overrides, so `hxprobe.cli.run()` reuses the entire argparse/concurrency/
  exit-code/rendering pipeline unchanged
- `python/tests/test_hx_probe.py` (17 tests), `test_hx_cli.py` (11 tests),
  `test_integration_hx.py` (9 live tests, excluded from the default gate via
  the existing `test_i*` naming convention)
- `Makefile` — `py-deps` (creates `python/.venv`, installs `httpx[http2]`),
  `hx-run`, `hx-test-integration`; `py-test`/`py-check` now run through the
  venv and include the new hermetic hxprobe tests
- `docs/usage/py-hxprobe.md` — usage doc with real captured output
- `.gitignore` — added `*.egg-info/` (editable-install artifact)

## Key design decisions

- **Instrument the transport, don't hand-roll HTTP.** Subclassed
  `httpcore.NetworkBackend`/`NetworkStream` to time DNS/TCP connect/TLS at
  the socket level, letting httpx own HTTP/1.1 vs HTTP/2 framing, redirects,
  and keep-alive. This was the reason to use httpx at all — get the protocol
  behavior of a real client while keeping latprobe's phase granularity.
- **First-hop-wins for dns/connect/tls; last-hop-wins for ttfb/transfer.**
  When redirects are followed, connection-identity fields (dns/connect/tls
  timing, resolved IP, TLS/cert info) reflect the *first* connection.
  `wrote_request`/`first_byte` are simply overwritten on every write/read, so
  they naturally end up reflecting the *last* hop — which mirrors how Go's
  own unguarded `httptrace.ClientTrace` hooks behave for a followed redirect.
- **Fresh `httpx.Client` per `measure()` call, no cross-sample pooling** —
  matches latprobe's per-call socket creation so every `-n` sample gets a
  full phase breakdown.
- **`measure_fn` injection over subclassing/duplication** in `latprobe.cli`,
  so hxprobe reuses argparse, concurrency, exit codes, and text/JSON
  rendering with zero duplicated logic — the socket and httpx probes only
  differ in `probe.py`.
- **New CLI flags gated behind `protocol_flags=True`** so `latprobe`'s own
  `--help` output stays byte-for-byte unchanged (verified) — `--no-http2`/
  `--no-follow-redirects` only appear for `hxprobe`.

## Notable finding (not part of the original plan)

While comparing `hxprobe` and `latprobe` timings against the same live host,
`hxprobe`'s TTFB was consistently ~40-50ms *lower*. Verified experimentally
(not just assumed) that this is a real effect, not noise: `latprobe`'s raw
socket never sets `TCP_NODELAY`, so its request write is subject to Nagle's
algorithm interacting with the server's delayed-ACK timer — a well-known
artifact. Forcing `TCP_NODELAY` onto `latprobe`'s socket collapsed its TTFB
to match `hxprobe`'s. `hxprobe` sets `TCP_NODELAY` (matching httpcore's own
default backend and Go's `net.Dialer`), so its TTFB numbers are the more
accurate of the two — not just different. Did not change `latprobe` itself
(out of scope for this task); documented the divergence in
`docs/usage/py-hxprobe.md`, in the CHANGELOG, and inline in
`hxprobe/probe.py`.

## Deviations from the plan

- Plan sketched `TTFB = headers-received minus end-of-TLS`; implemented as
  `TTFB = first-byte minus wrote-request` instead (matches both Go and the
  existing `latprobe` definition — the plan's phrasing was an approximation).
- Plan said verbose TLS/cert metadata could follow "last hop"; implemented
  as first-hop-wins uniformly across dns/connect/tls/ip/tls-info for
  simplicity and consistency (only `ttfb`/`transfer` are last-hop).
- Everything else (library choice, transport-instrumentation approach,
  pyproject.toml, sibling-package placement, flag surface) matches the
  approved plan as written.

## Verification

- `make check` (Go + Python full gate): exit 0, 83 hermetic Python tests
  (72 pre-existing + 11 new hermetic CLI + hxprobe's share of the 17 probe
  tests already counted), zero regressions to latprobe's original 39.
- `make hx-test-integration`: 9/9 live tests pass, including real HTTP/2
  negotiation against example.com (Cloudflare) and a real http→https redirect
  follow against github.com.
- Manual spot checks: `--json` schema, `--fail` exit code 6, DNS failure
  (exit 2), connection-refused (exit 3), `--no-http2`/`--no-follow-redirects`
  flag behavior, `latprobe --help` output diffed byte-for-byte against
  pre-change output to confirm no regression.
- Caught and reverted an incidental `gofmt` whitespace diff in
  `go/internal/probe/probe.go` that `make check`'s `go-fmt` step produced —
  unrelated to this change, out of scope, not committed.