Files

Jan Novak 404b372070 feat(hxprobe): httpx-based HTTP probe — full standalone package

Third Python implementation in the latency-tool progression.
Mirrors latprobe's feature set but replaces raw sockets with httpx,
gaining HTTP/2 support and redirect following.

Package layout (hxprobe/hxprobe/):
- probe.py: asyncio + httpx.AsyncClient with a _TimingStream transport
  wrapper that captures DNS/connect/TLS/TTFB/transfer phase timings via
  httpx event hooks (get_connection_stats, request_started, etc.).
  VerboseDetail captures resolved IP, TLS version/cipher/bits, cert CN/
  expiry/issuer (from httpx's SSLObject), and response headers.
  Options: timeout, verbose, follow_redirects, http2
- aggregate.py: summarize() → per-phase min/avg/max (same schema as latprobe)
- cli.py: run(args,stdout,stderr)->int; argparse with redirect_stdout/
  redirect_stderr + SystemExit catch; -n/--count, -c/--concurrency,
  --timeout, --fail, --json, -v/--verbose, --no-follow-redirects,
  --no-http2, -f/--file (URL list from file, mutually exclusive with args);
  multi-URL summary footer (tally + exit label) when len(urls) > 1;
  worst-exit-code logic mirrors Go/latprobe; JSON output is bare array
- duration.py: same parse_duration() as latprobe
- Exit codes: 0 ok, 1 usage, 2 dns, 3 connect, 4 timeout, 5 tls, 6 http≥400

Tests (hxprobe/tests/):
- test_probe.py: 18 hermetic tests using anyio + in-process ASGI servers
- test_cli.py: 36 hermetic tests (success, failures, JSON, verbose, -f flag,
  run-summary footer, worst-code accumulation)
- test_integration.py: pytest-marked @integration (excluded from hx-test)

Toolchain: uv + ruff + pytest; pyproject.toml with [dependency-groups];
hxprobe/Makefile standalone (help, deps, run, lint, fmt, test,
test-integration, check, clean); .python-version pins 3.14
Configs: 8 fixture files mirroring python/configs/ (all-ok through mixed)
USAGE.md: 16 runnable examples with real captured output

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-07-02 13:22:31 +02:00

22 KiB

Raw Blame History

`hxprobe` — Runnable Usage Reference

All commands below assume you're inside this directory (cd hxprobe) with dependencies installed (make deps or uv sync). Timings will differ on your machine and network; the output structure is stable. Every example below was run against the live internet.

Basic — single URL

make run ARGS="https://example.com"
# or directly:
uv run python -m hxprobe https://example.com

https://example.com  (200)
  DNS lookup     :     17.98 ms
  TCP connect    :      8.18 ms
  TLS handshake  :     15.41 ms
  Server (TTFB)  :      3.38 ms
  Transfer       :      7.99 ms
  ─────────────────────────────
  Total          :     61.93 ms

Verbose mode — IP, protocol, TLS, certificate

uv run python -m hxprobe --verbose https://example.com

https://example.com  (200)
  DNS lookup     :      2.33 ms
  TCP connect    :     14.73 ms
  TLS handshake  :     15.22 ms
  Server (TTFB)  :      5.63 ms
  Transfer       :      0.57 ms
  ─────────────────────────────
  Total          :     49.16 ms
  IP             :  104.20.23.154
  Protocol       :  HTTP/2
  TLS            :  TLSv1.3  TLS_AES_256_GCM_SHA384  256 bit
  Cert           :  CN=example.com  valid until 2026-08-29  SSL Corporation

Protocol is the one row latprobe (the raw-socket sibling implementation) never prints — it's the ALPN-negotiated HTTP version, only meaningful for a client that can actually speak more than one.

Verbose — plain HTTP (no TLS block)

uv run python -m hxprobe --verbose http://example.com

http://example.com  (200)
  DNS lookup     :      2.15 ms
  TCP connect    :      8.52 ms
  Server (TTFB)  :     16.33 ms
  Transfer       :      0.42 ms
  ─────────────────────────────
  Total          :     27.80 ms
  IP             :  104.20.23.154
  Protocol       :  HTTP/1.1

No TLS or Cert rows for http:// URLs. Protocol still shows — HTTP/2 is not attempted over cleartext (see Limitations in docs/usage/hxprobe.md), so this is always HTTP/1.1.

Verbose — redirect followed by default

uv run python -m hxprobe --verbose http://github.com

http://github.com  (200)
  DNS lookup     :     14.34 ms
  TCP connect    :     20.88 ms
  TLS handshake  :     25.91 ms
  Server (TTFB)  :      3.44 ms
  Transfer       :     63.33 ms
  ─────────────────────────────
  Total          :    197.21 ms
  IP             :  140.82.121.3
  Protocol       :  HTTP/2  (1 redirect)
  TLS            :  TLSv1.3  TLS_AES_128_GCM_SHA256  128 bit
  Cert           :  CN=github.com  valid until 2026-08-02  Sectigo Limited

http://github.com 301-redirects to https://github.com; hxprobe follows it by default (matching Go's http.DefaultClient) and reports the final response. latprobe has no equivalent — it would print the bare 301 and stop. DNS/TCP connect/TLS are timed from the first connection only; Server (TTFB)/Transfer reflect the final hop (see "Divergence from latprobe" in docs/usage/hxprobe.md for why).

`--no-follow-redirects` — report the raw redirect instead

uv run python -m hxprobe --no-follow-redirects http://github.com

http://github.com  (301)
  DNS lookup     :      2.85 ms
  TCP connect    :     24.16 ms
  Server (TTFB)  :     24.50 ms
  Transfer       :      0.59 ms
  ─────────────────────────────
  Total          :     52.58 ms

Same shape latprobe would show for any redirect — hxprobe just makes it opt-in rather than the default.

`--no-http2` — force HTTP/1.1

uv run python -m hxprobe --verbose --no-http2 https://example.com

https://example.com  (200)
  DNS lookup     :      2.31 ms
  TCP connect    :      8.75 ms
  TLS handshake  :     13.61 ms
  Server (TTFB)  :     13.51 ms
  Transfer       :      0.44 ms
  ─────────────────────────────
  Total          :     39.07 ms
  IP             :  104.20.23.154
  Protocol       :  HTTP/1.1
  TLS            :  TLSv1.3  TLS_AES_256_GCM_SHA384  256 bit
  Cert           :  CN=example.com  valid until 2026-08-29  SSL Corporation

Useful for isolating whether a latency difference is due to protocol version rather than network conditions.

Verbose — TLS failure (certificate expired)

uv run python -m hxprobe --verbose https://expired.badssl.com/
echo "exit: $?"

https://expired.badssl.com/  (FAILED)
  DNS lookup     :     33.34 ms
  TCP connect    :   1129.53 ms
  TLS handshake  :    318.54 ms
  ─────────────────────────────
  Total          :   1484.12 ms
  ✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1081)
  IP             :  104.154.89.105
exit: 5

The IP is shown even on TLS failure (DNS and TCP both succeeded). badssl.com is a shared, often-loaded demo host — TCP connect here is unusually slow; that's the host, not hxprobe.

Verbose — DNS failure (no IP to show)

uv run python -m hxprobe --verbose http://no.such.host.invalid
echo "exit: $?"

http://no.such.host.invalid  (FAILED)
  Total          :     13.22 ms
  ✗ dns: [Errno 8] nodename nor servname provided, or not known
exit: 2

The verbose block is empty (IP never resolved), so it's suppressed entirely — same behavior as latprobe.

Sampling (`-n`) — min / avg / max table

uv run python -m hxprobe -n 3 https://example.com

https://example.com  (200, 3 samples)
                          min        avg        max
  DNS lookup     :    1.23 ms    2.29 ms    3.02 ms
  TCP connect    :    8.05 ms    8.73 ms    9.81 ms
  TLS handshake  :   12.99 ms   15.65 ms   17.48 ms
  Server (TTFB)  :    0.01 ms    7.23 ms   11.95 ms
  Transfer       :    0.63 ms    2.18 ms    5.15 ms
  ─────────────────────────────────────────────────
  Total          :   38.56 ms   49.16 ms   54.70 ms

With --verbose, the block is appended below the table using the last successful sample's detail:

uv run python -m hxprobe --verbose -n 3 https://example.com

https://example.com  (200, 3 samples)
                          min        avg        max
  ...
  Total          :   39.95 ms   42.55 ms   44.66 ms
  IP             :  104.20.23.154
  Protocol       :  HTTP/2
  TLS            :  TLSv1.3  TLS_AES_256_GCM_SHA384  256 bit
  Cert           :  CN=example.com  valid until 2026-08-29  SSL Corporation

Multiple URLs (probed in parallel)

uv run python -m hxprobe https://example.com https://www.iana.org

https://example.com  (200)
  DNS lookup     :      3.96 ms
  TCP connect    :     13.13 ms
  TLS handshake  :     15.31 ms
  Server (TTFB)  :      0.01 ms
  Transfer       :      7.20 ms
  ─────────────────────────────
  Total          :     50.16 ms

https://www.iana.org  (200)
  DNS lookup     :     26.67 ms
  TCP connect    :      9.05 ms
  TLS handshake  :     14.73 ms
  Server (TTFB)  :      3.37 ms
  Transfer       :      0.63 ms
  ─────────────────────────────
  Total          :     70.42 ms

  ─────────────────────────────────────────────────
  Summary: 2 URLs — 2 ok
  → exit 0 (ok)

Output for each URL is separated by a blank line. Exit code is the worst across all — see "Multi-URL summary footer" below for how that scalar breaks down when URLs have different outcomes.

Multi-URL summary footer

Whenever more than one URL is probed (positional args or -f), a footer is appended after the last URL block: a per-outcome tally plus the exit code it produced. Single-URL runs never show it — text output is otherwise unchanged.

uv run python -m hxprobe https://example.com http://no.such.host.invalid https://self-signed.badssl.com
echo "exit: $?"

https://example.com  (200)
  DNS lookup     :      4.65 ms
  TCP connect    :      9.56 ms
  TLS handshake  :     14.90 ms
  Server (TTFB)  :      0.01 ms
  Transfer       :      8.84 ms
  ─────────────────────────────
  Total          :     47.28 ms

http://no.such.host.invalid  (FAILED)
  Total          :      4.54 ms
  ✗ dns: [Errno 8] nodename nor servname provided, or not known

https://self-signed.badssl.com  (FAILED)
  DNS lookup     :     24.75 ms
  TCP connect    :   1126.05 ms
  TLS handshake  :    327.05 ms
  ─────────────────────────────
  Total          :   1478.35 ms
  ✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1081)

  ─────────────────────────────────────────────────
  Summary: 3 URLs — 1 ok, 2 failed
  ✗ dns     : 1
  ✗ tls     : 1
  → exit 5 (tls)
exit: 5

Each URL is classified by its own worst sample (same _phase_code mapping the overall exit code uses), then tallied. The → exit N (label) line ties the tally directly to the process exit code, since the code alone can't show which URLs failed how — here a DNS failure (would be exit 2 alone) is outranked by the TLS failure (5), and the footer is what makes that visible. JSON output (--json) is unaffected — it stays a bare array; per-URL failures are already in each entry's errors[].

Reading URLs from a file (`-f`)

uv run python -m hxprobe -f configs/all-ok.txt

https://example.com  (200)
  DNS lookup     :      3.60 ms
  TCP connect    :     17.61 ms
  TLS handshake  :     16.39 ms
  Server (TTFB)  :      4.06 ms
  Transfer       :      0.60 ms
  ─────────────────────────────
  Total          :     56.24 ms

https://www.google.com  (200)
  DNS lookup     :      4.18 ms
  TCP connect    :     17.23 ms
  TLS handshake  :     30.62 ms
  Server (TTFB)  :      2.68 ms
  Transfer       :     69.83 ms
  ─────────────────────────────
  Total          :    142.95 ms

https://www.iana.org  (200)
  DNS lookup     :      4.25 ms
  TCP connect    :     17.18 ms
  TLS handshake  :     18.87 ms
  Server (TTFB)  :      8.13 ms
  Transfer       :      0.50 ms
  ─────────────────────────────
  Total          :     60.76 ms

  ─────────────────────────────────────────────────
  Summary: 3 URLs — 3 ok
  → exit 0 (ok)

-f/--file reads a plain-text URL list — one per line, blank lines and #-prefixed comment lines skipped — the same format simple.py/phases.py use elsewhere in this repo. It's mutually exclusive with positional URL arguments: pass one or the other, not both.

uv run python -m hxprobe -f configs/all-ok.txt https://extra.example.com

usage: hxprobe [-h] [-f PATH] [-n N] [-c N] [--timeout DURATION] [--fail]
               [--json] [-v] [--no-http2] [--no-follow-redirects]
               [url ...]
hxprobe: error: cannot combine positional url arguments with -f/--file

configs/ ships one fixture per failure class, each self-documenting its expected exit code in a header comment (verified by actually running it, not just asserted):

File	Demonstrates	Exit code
`configs/all-ok.txt`	Everything succeeds	0
`configs/dns-failure.txt`	Unresolvable hostnames	2
`configs/connection-refused.txt`	Loopback ports with no listener	3
`configs/timeout.txt`	Non-routable IPs (RFC 5737 TEST-NET-1)	4
`configs/tls-errors.txt`	badssl.com cert failures	5
`configs/http-errors.txt`	404s, run with `--fail`	6
`configs/mixed.txt`	One of each class, run with `--fail`	6
`configs/large-mixed.txt`	30 URLs, ≥10 failing across all 5 classes — good demo of the multi-URL summary footer	6

uv run python -m hxprobe -f configs/dns-failure.txt
echo "exit: $?"

https://this-host-does-not-exist.invalid  (FAILED)
  Total          :     16.91 ms
  ✗ dns: [Errno 8] nodename nor servname provided, or not known

http://no.such.host.invalid  (FAILED)
  Total          :      4.56 ms
  ✗ dns: [Errno 8] nodename nor servname provided, or not known

  ─────────────────────────────────────────────────
  Summary: 2 URLs — 0 ok, 2 failed
  ✗ dns     : 2
  → exit 2 (dns)
exit: 2

configs/large-mixed.txt scales this up to 30 URLs so the summary footer has something substantial to tally — 20 real sites expected to succeed plus 10 deliberately broken across all five failure classes at once:

uv run python -m hxprobe --fail --timeout 2s -f configs/large-mixed.txt
echo "exit: $?"

https://example.com  (200)
  DNS lookup     :      3.96 ms
  TCP connect    :     10.24 ms
  TLS handshake  :     14.76 ms
  Server (TTFB)  :      4.14 ms
  Transfer       :      3.71 ms
  ─────────────────────────────
  Total          :     49.54 ms

... 18 more successful URLs ...

https://stackoverflow.com  (200)
  DNS lookup     :     12.99 ms
  TCP connect    :      9.00 ms
  TLS handshake  :     16.58 ms
  Server (TTFB)  :    197.72 ms
  Transfer       :    289.92 ms
  ─────────────────────────────
  Total          :    684.24 ms

https://this-host-does-not-exist.invalid  (FAILED)
  Total          :      1.25 ms
  ✗ dns: [Errno 8] nodename nor servname provided, or not known

http://no.such.host.invalid  (FAILED)
  Total          :      0.92 ms
  ✗ dns: [Errno 8] nodename nor servname provided, or not known

http://127.0.0.1:9999  (FAILED)
  DNS lookup     :      0.01 ms
  TCP connect    :      0.09 ms
  ─────────────────────────────
  Total          :      0.26 ms
  ✗ connect: [Errno 61] Connection refused

http://127.0.0.1:19999  (FAILED)
  DNS lookup     :      0.01 ms
  TCP connect    :      0.09 ms
  ─────────────────────────────
  Total          :      0.23 ms
  ✗ connect: [Errno 61] Connection refused

http://10.255.255.1/  (FAILED)
  DNS lookup     :      0.03 ms
  TCP connect    :     16.50 ms
  ─────────────────────────────
  Total          :     16.64 ms
  ✗ connect: [Errno 61] Connection refused

http://192.0.2.1/  (FAILED)
  DNS lookup     :      0.01 ms
  TCP connect    :   2001.27 ms
  ─────────────────────────────
  Total          :   2001.56 ms
  ✗ timeout: timed out

https://expired.badssl.com/  (FAILED)
  DNS lookup     :     16.37 ms
  TCP connect    :   1125.42 ms
  TLS handshake  :    329.12 ms
  ─────────────────────────────
  Total          :   1471.48 ms
  ✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1081)

https://self-signed.badssl.com/  (FAILED)
  DNS lookup     :     13.27 ms
  TCP connect    :   1127.27 ms
  TLS handshake  :    325.84 ms
  ─────────────────────────────
  Total          :   1466.90 ms
  ✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1081)

https://www.google.com/this-page-does-not-exist-at-all-1234567890  (404 ✗)
  DNS lookup     :      0.96 ms
  TCP connect    :      7.61 ms
  TLS handshake  :     25.25 ms
  Server (TTFB)  :      0.01 ms
  Transfer       :     99.63 ms
  ─────────────────────────────
  Total          :    142.38 ms

https://github.com/this-repo-does-not-exist-abcxyz123/no-way  (404 ✗)
  DNS lookup     :      1.23 ms
  TCP connect    :     19.57 ms
  TLS handshake  :     22.02 ms
  Server (TTFB)  :    209.47 ms
  Transfer       :     67.20 ms
  ─────────────────────────────
  Total          :    339.82 ms

  ─────────────────────────────────────────────────
  Summary: 30 URLs — 20 ok, 10 failed
  ✗ dns     : 2
  ✗ connect : 3
  ✗ timeout : 1
  ✗ tls     : 2
  ✗ http    : 2
  → exit 6 (http)
exit: 6

The individual URL blocks above are unchanged in shape from every other example on this page — this is purely a matter of scale. The summary footer is where scale actually pays off: instead of scanning 30 blocks to count outcomes, Summary: 30 URLs — 20 ok, 10 failed plus the per-class breakdown answers "what happened" at a glance, and → exit 6 (http) explains why that particular exit code came back (the two 404s under --fail, at EXIT_HTTP = 6, outrank every other class present).

`--fail` flag — exit non-zero on HTTP 4xx/5xx

uv run python -m hxprobe --fail https://www.google.com/this-page-does-not-exist-at-all-1234567890
echo "exit: $?"

https://www.google.com/this-page-does-not-exist-at-all-1234567890  (404 ✗)
  DNS lookup     :      2.96 ms
  TCP connect    :     11.57 ms
  TLS handshake  :     24.95 ms
  Server (TTFB)  :      0.01 ms
  Transfer       :    104.22 ms
  ─────────────────────────────
  Total          :    152.77 ms
exit: 6

Without --fail, 4xx/5xx responses are shown normally and the exit code is 0.

JSON output

uv run python -m hxprobe --json https://example.com | python3.14 -m json.tool

[
  {
    "url": "https://example.com",
    "status": 200,
    "succeeded": 1,
    "failed": 0,
    "phases": {
      "dns":      {"min_ms": 2.80, "avg_ms": 2.80, "max_ms": 2.80},
      "connect":  {"min_ms": 11.68, "avg_ms": 11.68, "max_ms": 11.68},
      "tls":      {"min_ms": 14.52, "avg_ms": 14.52, "max_ms": 14.52},
      "ttfb":     {"min_ms": 6.49, "avg_ms": 6.49, "max_ms": 6.49},
      "transfer": {"min_ms": 0.81, "avg_ms": 0.81, "max_ms": 0.81},
      "total":    {"min_ms": 45.59, "avg_ms": 45.59, "max_ms": 45.59}
    }
  }
]

Same shape as latprobe's JSON — phases is omitted when all samples failed, tls is omitted for http:// URLs.

JSON + verbose

uv run python -m hxprobe --verbose --json https://example.com

[
  {
    "url": "https://example.com",
    "status": 200,
    "succeeded": 1,
    "failed": 0,
    "phases": { "...": "..." },
    "verbose": {
      "ip": "104.20.23.154",
      "http_version": "HTTP/2",
      "redirect_count": 0,
      "tls_version": "TLSv1.3",
      "tls_cipher": "TLS_AES_256_GCM_SHA384",
      "tls_bits": 256,
      "cert": {
        "cn": "example.com",
        "sans": ["example.com", "*.example.com"],
        "expiry": "2026-08-29",
        "issuer_cn": "SSL Corporation",
        "verified": true
      },
      "headers": {
        "content-type": "text/html",
        "server": "cloudflare",
        "cf-cache-status": "HIT"
      }
    }
  }
]

"http_version"/"redirect_count" are the two keys latprobe's JSON never has (its VerboseDetail.http_version stays ""). "headers" contains all parsed response headers (the text view shows only a priority list).

Timeout

uv run python -m hxprobe --timeout 500ms http://192.0.2.1/
echo "exit: $?"

http://192.0.2.1/  (FAILED)
  DNS lookup     :      3.11 ms
  TCP connect    :    501.51 ms
  ─────────────────────────────
  Total          :    505.20 ms
  ✗ timeout: timed out
exit: 4

192.0.2.1 is in RFC 5737's TEST-NET-1 range — reserved for documentation, guaranteed unreachable, and the kernel gets no reply so the connect phase runs the full --timeout before giving up. --timeout accepts ms, s, m suffixes or a bare number of seconds.

Exit codes

Code	Meaning
0	All probes succeeded (or HTTP 4xx without `--fail`)
1	Usage / argument error
2	DNS failure
3	TCP connect failure
4	Timeout
5	TLS error
6	HTTP status ≥ 400 with `--fail`

The highest exit code across all URLs is returned as the process exit — identical scheme to latprobe and the Go implementation. This scalar is kept unchanged for compatibility; for multi-URL runs, the "Multi-URL summary footer" section above shows the full per-URL breakdown behind it.

Makefile shortcuts

From inside this directory:

make run ARGS="--verbose https://example.com"   # run hxprobe
make test                                        # hermetic tests only
make test-integration                            # live internet tests
make lint                                        # ruff check
make fmt                                         # ruff format
make check                                       # lint + hermetic tests

From the parent repo's root, the equivalent shortcuts are prefixed hx-:

make hx-run ARGS="--verbose https://example.com"
make hx-test
make hx-test-integration
make hx-check

Both Makefiles are independent — neither calls into the other — so either works whether you're inside a checkout of the full monorepo or a standalone copy of just hxprobe/.

22 KiB Raw Blame History

hxprobe — Runnable Usage Reference