feat(py): step 2 — per-phase latency measurement (phases.py)

Hand-drives raw sockets to time each HTTP phase individually — DNS (getaddrinfo), TCP connect, TLS handshake (ssl.wrap_socket, https only), TTFB (sendall → first recv), Transfer (first byte → EOF), Total. Partial phases are preserved on failure (same invariant as Go's probe.go). Error classification mirrors Go's priority: dns → timeout → tls → connect. Output format matches Go's single-sample text layout. Input: bare URL args or plain-text config file (same format as simple.py). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 12:07:44 +02:00
parent 6db2abb713
commit f609d8124f
4 changed files with 520 additions and 0 deletions
--- a/docs/plans/2026-07-01-12-04-py-phases.md
+++ b/docs/plans/2026-07-01-12-04-py-phases.md
@@ -0,0 +1,68 @@
+# Python Port — Step 2: `phases.py`
+
+## Goal
+
+Introduce the manual per-phase timing technique — the Python answer to Go's
+`net/http/httptrace`. A single self-contained script that hand-drives a raw
+socket for each URL and times each phase individually with `time.perf_counter()`.
+
+## Why raw sockets
+
+Python's `urllib` / `httpx` / `requests` give no per-phase callbacks, unlike
+Go's `httptrace.ClientTrace`. The only way to time DNS, TCP connect, TLS
+handshake, TTFB, and transfer independently is to drive the connection at the
+socket level:
+- `socket.getaddrinfo()` → DNS
+- `sock.connect()` → TCP
+- `ssl.SSLContext.wrap_socket()` → TLS (HTTPS only)
+- `sock.sendall(request)` + `sock.recv()` → TTFB
+- drain to EOF → Transfer
+
+## Input
+
+- Bare URL(s) as positional args: `python phases.py https://example.com`
+- Or a plain-text config file: `python phases.py configs/all-ok.txt`
+  (detected by whether the first arg starts with `http://` / `https://`)
+
+## Output
+
+Mirrors Go's single-sample text layout for each URL:
+
+```
+https://example.com  (200)
+  DNS lookup     :     18.21 ms
+  TCP connect    :     10.12 ms
+  TLS handshake  :     36.11 ms
+  Server (TTFB)  :     21.95 ms
+  Transfer       :      0.18 ms
+  ─────────────────────────────
+  Total          :     88.00 ms
+```
+
+- TLS row omitted for `http://` URLs.
+- On failure: header shows `(FAILED)`, partial phases shown, error at the end.
+- Multiple URLs separated by a blank line.
+
+## Error classification
+
+Mirrors Go's priority order (dns → timeout → tls → connect):
+- `socket.gaierror` → `"dns"`
+- `socket.timeout` / `TimeoutError` → `"timeout"` (regardless of phase)
+- `ssl.SSLError` or `OSError` during TLS wrap → `"tls"`
+- `ConnectionRefusedError` / other `OSError` during connect → `"connect"`
+- Error after first byte → `"transfer"`
+
+Partial phases are preserved on failure (same invariant as Go).
+
+## Known limitations (documented in usage doc)
+
+- No redirect following (3xx responses are reported with their raw status code).
+- `Connection: close` + read-to-EOF; no keep-alive, no HTTP/2.
+- Timeout hard-coded at 10 s (no `--timeout` flag — that's in the full version).
+- Bodies fully drained to get accurate Transfer timing.
+
+## Files
+
+- `python/phases.py` — the script
+- `docs/usage/py-phases.md` — user-facing doc
+- CHANGELOG.md entry
--- a/docs/usage/py-phases.md
+++ b/docs/usage/py-phases.md
@@ -0,0 +1,151 @@
+# `phases.py` — Per-Phase HTTP Latency Measurement
+
+## What it does
+
+Measures the latency of each phase of an HTTP request by hand-driving a raw
+socket, timing each step individually with `time.perf_counter()`. This is the
+Python answer to Go's `net/http/httptrace` — there is no equivalent callback
+API in Python's stdlib, so we instrument at the socket level.
+
+Phases measured:
+
+| Phase | What is timed |
+|-------|---------------|
+| DNS lookup | `socket.getaddrinfo()` — hostname resolution |
+| TCP connect | `sock.connect()` — SYN to connection established |
+| TLS handshake | `ssl.wrap_socket()` — full handshake (HTTPS only) |
+| Server (TTFB) | `sendall()` return → first `recv()` byte — server processing time |
+| Transfer | First byte → EOF — body download time |
+| Total | DNS start → body EOF — wall-clock end-to-end |
+
+The TLS row is omitted automatically for `http://` URLs.
+
+## Flags / arguments
+
+```
+python phases.py <url> [url ...]
+python phases.py <config_file>
+```
+
+| Argument | Meaning |
+|----------|---------|
+| `<url> [url …]` | One or more URLs starting with `http://` or `https://` |
+| `<config_file>` | Path to a plain-text URL list (same format as `simple.py`) |
+
+Detection is automatic: if the first argument starts with `http://` or
+`https://`, all arguments are treated as URLs; otherwise the single argument
+is treated as a config file path.
+
+**Exit codes:**
+
+| Code | Meaning |
+|------|---------|
+| 0 | All URLs completed without a network error |
+| 1 | One or more URLs failed, or a usage/config error |
+
+HTTP error statuses (4xx, 5xx) do **not** set exit code 1 — the request
+completed successfully at the network level. The status code is visible in
+the output header.
+
+## Example — single URL
+
+```sh
+python phases.py https://example.com
+```
+
+```
+https://example.com  (200)
+  DNS lookup     :     21.74 ms
+  TCP connect    :     10.79 ms
+  TLS handshake  :     18.48 ms
+  Server (TTFB)  :     73.72 ms
+  Transfer       :      0.13 ms
+  ─────────────────────────────
+  Total          :    132.95 ms
+```
+
+## Example — HTTP URL (no TLS row)
+
+```sh
+python phases.py http://example.com
+```
+
+```
+http://example.com  (200)
+  DNS lookup     :      3.37 ms
+  TCP connect    :     13.51 ms
+  Server (TTFB)  :     20.01 ms
+  Transfer       :      2.85 ms
+  ─────────────────────────────
+  Total          :     39.77 ms
+```
+
+## Example — DNS failure (partial phases)
+
+```sh
+python phases.py https://no.such.host.invalid
+```
+
+```
+https://no.such.host.invalid  (FAILED)
+  ─────────────────────────────
+  Total          :      0.67 ms
+  ✗ dns: [Errno 8] nodename nor servname provided, or not known
+```
+
+## Example — TLS failure (partial phases preserved)
+
+```sh
+python phases.py https://expired.badssl.com/
+```
+
+```
+https://expired.badssl.com/  (FAILED)
+  DNS lookup     :     28.32 ms
+  TCP connect    :    129.01 ms
+  TLS handshake  :    305.30 ms
+  ─────────────────────────────
+  Total          :    473.17 ms
+  ✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired …
+```
+
+Note: DNS and TCP phases are populated even though the overall request failed.
+This is the same behaviour as the Go tool — partial timing is preserved up to
+the point of failure.
+
+## Example — config file
+
+```sh
+python phases.py configs/all-ok.txt
+```
+
+Multiple URLs are separated by a blank line, matching Go's text output style.
+
+## Using the error-path example configs
+
+The `python/configs/` files from `simple.py` work identically with `phases.py`.
+Compare the error handling between the two tools:
+
+```sh
+# DNS failure
+python phases.py configs/dns-failure.txt
+
+# Connection refused — note DNS phase is present, TCP is partial
+python phases.py configs/connection-refused.txt
+
+# TLS errors — all three phases up to TLS are present
+python phases.py configs/tls-errors.txt
+```
+
+## Limitations
+
+- **No redirect following.** 3xx responses are reported with their raw status
+  code; the redirect target is not probed. Use `latprobe/` (the full version)
+  or `simple.py` (which uses `urllib`, which follows redirects) if you need
+  the final destination's timing.
+- **HTTP 1.1 + `Connection: close` only.** No keep-alive, no HTTP/2, no auth,
+  no custom headers beyond `Host` and `User-Agent`.
+- **Timeout hard-coded at 10 s.** Use `latprobe/` for a `--timeout` flag.
+- **No sampling.** Each URL is probed once. Use `latprobe/` for `-n` (min/avg/max).
+- **Body is fully drained.** Transfer time includes reading the entire response
+  body, so it is real wall-clock transfer time.