Files
http-latency-prober/python/configs/usage-latprobe.md
Jan Novak 24ea9c9e71 feat(py): step 3 — full latprobe package with --verbose flag
Adds the complete Python port of the Go latprobe tool plus diagnostic
verbose output that the Go version does not yet have.

Package (python/latprobe/):
- probe.py: raw-socket HTTP timing with CertInfo/VerboseDetail dataclasses;
  TTFB loop accumulates to \r\n\r\n so headers are parseable without changing
  t_first_byte semantics; captures resolved IP, TLS version/cipher/bits,
  verified cert (via getpeercert()), and all response headers when verbose=True
- aggregate.py: summarize() → per-phase min/avg/max PhaseStats
- cli.py: injectable run(args,stdout,stderr)->int; argparse with injected
  streams; ThreadPoolExecutor concurrency across URLs; four text-rendering
  branches; JSON output; worst-exit-code accumulation; -v/--verbose flag
  appends IP/TLS/cert/header block after every timing table; JSON extended
  with "verbose" object (omitted when flag absent)
- duration.py: parse Go-style duration strings (10s, 500ms, 2m, bare seconds)
- Exit codes: 0 ok, 1 usage, 2 dns, 3 connect, 4 timeout, 5 tls, 6 http≥400

Tests:
- test_probe.py: 18 hermetic tests (success, 404, DNS/connect/timeout/scheme
  failures, partial-phase invariants, verbose detail fields)
- test_cli.py: 38 hermetic tests (usage errors, single/aggregate/failure text,
  worst-code, --fail, JSON schema/ordering/grouping, verbose text and JSON)
- test_integration.py: 45 tests against live internet services (badssl.com for
  TLS errors, real cert/IP/header validation in verbose mode)

Makefile: PYTHONPATH fix, fnmatch pattern test_[!i]*.py to exclude
integration tests from py-test, new py-test-integration target

Docs: docs/usage/py-latprobe.md, python/configs/usage-latprobe.md (runnable
reference with captured output), plans for both the package and verbose feature

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-01 13:45:56 +02:00

7.8 KiB

latprobe — Runnable Usage Reference

All commands run from the repository root. Timings will differ on your machine and network; the output structure is stable.


Basic — single URL

make py-run ARGS="https://example.com"
# or directly:
python3.14 -m latprobe https://example.com
https://example.com  (200)
  DNS lookup     :     16.47 ms
  TCP connect    :      9.76 ms
  TLS handshake  :     13.09 ms
  Server (TTFB)  :     60.31 ms
  Transfer       :      0.20 ms
  ─────────────────────────────
  Total          :    112.76 ms

Verbose mode — IP, TLS, certificate, headers

python3.14 -m latprobe --verbose https://example.com
https://example.com  (200)
  DNS lookup     :     16.47 ms
  TCP connect    :      9.76 ms
  TLS handshake  :     13.09 ms
  Server (TTFB)  :     60.31 ms
  Transfer       :      0.20 ms
  ─────────────────────────────
  Total          :    112.76 ms
  IP             :  104.20.23.154
  TLS            :  TLSv1.3  TLS_AES_256_GCM_SHA384  256 bit
  Cert           :  CN=example.com  valid until 2026-08-29  SSL Corporation
  Server         :  cloudflare
  Content-Type   :  text/html

The verbose block shows:

  • IP — first resolved address (useful when DNS round-robins across IPs)
  • TLS — protocol version, cipher suite, and key bits
  • Cert — common name, expiry date (prefixed EXPIRED if past), and issuer
  • Response headers from the priority list: Location, Server, Content-Type, X-Cache, CF-Cache-Status, Cache-Control, Via, X-Powered-By

Verbose — plain HTTP (no TLS block)

python3.14 -m latprobe --verbose http://example.com
http://example.com  (301)
  DNS lookup     :     18.22 ms
  TCP connect    :     10.01 ms
  Server (TTFB)  :     65.40 ms
  Transfer       :      0.08 ms
  ─────────────────────────────
  Total          :     96.10 ms
  IP             :  104.20.23.154
  Location       :  https://www.example.com/
  Server         :  cloudflare
  Content-Type   :  text/html

No TLS or Cert rows for http:// URLs.


Verbose — TLS failure (certificate expired)

python3.14 -m latprobe --verbose https://expired.badssl.com/
echo "exit: $?"
https://expired.badssl.com/  (FAILED)
  DNS lookup     :     30.98 ms
  TCP connect    :    126.58 ms
  TLS handshake  :    293.58 ms
  ─────────────────────────────
  Total          :    463.37 ms
  ✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1082)
  IP             :  104.154.89.105
exit: 5

The IP is shown even on TLS failure (DNS and TCP both succeeded). The error message identifies the cause; certificate details are unavailable because Python's stdlib does not expose the rejected cert.


Verbose — DNS failure (no IP to show)

python3.14 -m latprobe --verbose http://no.such.host.invalid
echo "exit: $?"
http://no.such.host.invalid  (FAILED)
  Total          :      2.65 ms
  ✗ dns: [Errno 8] nodename nor servname provided, or not known
exit: 2

The verbose block is empty (IP never resolved), so it is suppressed entirely.


Sampling (-n) — min / avg / max table

python3.14 -m latprobe -n 3 https://example.com
https://example.com  (200, 3 samples)
                          min        avg        max
  DNS lookup     :    1.15 ms    2.09 ms    3.45 ms
  TCP connect    :    8.76 ms    9.50 ms   10.21 ms
  TLS handshake  :   14.48 ms   16.63 ms   20.56 ms
  Server (TTFB)  :   65.44 ms   68.22 ms   70.58 ms
  Transfer       :    0.15 ms    0.28 ms    0.42 ms
  ─────────────────────────────────────────────────
  Total          :  101.64 ms  106.31 ms  115.47 ms

With --verbose, the verbose block is appended below the table using the last successful sample's detail:

python3.14 -m latprobe --verbose -n 3 https://example.com
https://example.com  (200, 3 samples)
                          min        avg        max
  ...
  Total          :  101.64 ms  106.31 ms  115.47 ms
  IP             :  104.20.23.154
  TLS            :  TLSv1.3  TLS_AES_256_GCM_SHA384  256 bit
  Cert           :  CN=example.com  valid until 2026-08-29  SSL Corporation
  Server         :  cloudflare
  Content-Type   :  text/html

Multiple URLs (probed in parallel)

python3.14 -m latprobe https://example.com https://www.iana.org

Output for each URL is separated by a blank line. Exit code = worst across all.


--fail flag — exit non-zero on HTTP 4xx/5xx

python3.14 -m latprobe --fail https://www.google.com/this-page-does-not-exist
echo "exit: $?"
https://www.google.com/this-page-does-not-exist  (404 ✗)
  ...
  Total          :    145.50 ms
exit: 6

Without --fail, HTTP 4xx/5xx responses are shown normally and the exit code is 0.


JSON output

python3.14 -m latprobe --json https://example.com | python3.14 -m json.tool
[
  {
    "url": "https://example.com",
    "status": 200,
    "succeeded": 1,
    "failed": 0,
    "phases": {
      "dns":      {"min_ms": 16.47, "avg_ms": 16.47, "max_ms": 16.47},
      "connect":  {"min_ms":  9.76, "avg_ms":  9.76, "max_ms":  9.76},
      "tls":      {"min_ms": 13.09, "avg_ms": 13.09, "max_ms": 13.09},
      "ttfb":     {"min_ms": 60.31, "avg_ms": 60.31, "max_ms": 60.31},
      "transfer": {"min_ms":  0.20, "avg_ms":  0.20, "max_ms":  0.20},
      "total":    {"min_ms": 112.76, "avg_ms": 112.76, "max_ms": 112.76}
    }
  }
]

phases is omitted when all samples failed (only errors is present). tls key is omitted for http:// URLs.


JSON + verbose

python3.14 -m latprobe --verbose --json https://example.com
[
  {
    "url": "https://example.com",
    "status": 200,
    "succeeded": 1,
    "failed": 0,
    "phases": { "..." },
    "verbose": {
      "ip": "104.20.23.154",
      "tls_version": "TLSv1.3",
      "tls_cipher": "TLS_AES_256_GCM_SHA384",
      "tls_bits": 256,
      "cert": {
        "cn": "example.com",
        "sans": ["example.com", "*.example.com"],
        "expiry": "2026-08-29",
        "issuer_cn": "SSL Corporation",
        "verified": true
      },
      "headers": {
        "Content-Type": "text/html",
        "Server": "cloudflare",
        "cf-cache-status": "HIT"
      }
    }
  }
]

"verbose" is omitted when --verbose is not set. "cert" is omitted for http:// URLs and when TLS fails. "headers" contains all parsed response headers (the text block shows only the priority list).


Timeout

python3.14 -m latprobe --timeout 500ms http://10.255.255.1/
echo "exit: $?"
http://10.255.255.1/  (FAILED)
  DNS lookup     :      0.20 ms
  TCP connect    :    500.18 ms
  ─────────────────────────────
  Total          :    500.40 ms
  ✗ timeout: timed out
exit: 4

--timeout accepts ms, s, m suffixes or bare seconds.


Exit codes

Code Meaning
0 All probes succeeded (or HTTP 4xx without --fail)
1 Usage / argument error
2 DNS failure
3 TCP connect failure
4 Timeout
5 TLS error
6 HTTP status ≥ 400 with --fail

The highest exit code across all URLs is returned as the process exit.


Makefile shortcuts

make py-run ARGS="--verbose https://example.com"   # run latprobe
make py-test                                        # hermetic tests only
make py-test-integration                            # live internet tests (~30 s)
make py-check                                       # alias for py-test