Jan Novak
f609d8124f
Hand-drives raw sockets to time each HTTP phase individually — DNS (getaddrinfo), TCP connect, TLS handshake (ssl.wrap_socket, https only), TTFB (sendall → first recv), Transfer (first byte → EOF), Total. Partial phases are preserved on failure (same invariant as Go's probe.go). Error classification mirrors Go's priority: dns → timeout → tls → connect. Output format matches Go's single-sample text layout. Input: bare URL args or plain-text config file (same format as simple.py). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
152 lines
4.7 KiB
Markdown
152 lines
4.7 KiB
Markdown
# `phases.py` — Per-Phase HTTP Latency Measurement
|
|
|
|
## What it does
|
|
|
|
Measures the latency of each phase of an HTTP request by hand-driving a raw
|
|
socket, timing each step individually with `time.perf_counter()`. This is the
|
|
Python answer to Go's `net/http/httptrace` — there is no equivalent callback
|
|
API in Python's stdlib, so we instrument at the socket level.
|
|
|
|
Phases measured:
|
|
|
|
| Phase | What is timed |
|
|
|-------|---------------|
|
|
| DNS lookup | `socket.getaddrinfo()` — hostname resolution |
|
|
| TCP connect | `sock.connect()` — SYN to connection established |
|
|
| TLS handshake | `ssl.wrap_socket()` — full handshake (HTTPS only) |
|
|
| Server (TTFB) | `sendall()` return → first `recv()` byte — server processing time |
|
|
| Transfer | First byte → EOF — body download time |
|
|
| Total | DNS start → body EOF — wall-clock end-to-end |
|
|
|
|
The TLS row is omitted automatically for `http://` URLs.
|
|
|
|
## Flags / arguments
|
|
|
|
```
|
|
python phases.py <url> [url ...]
|
|
python phases.py <config_file>
|
|
```
|
|
|
|
| Argument | Meaning |
|
|
|----------|---------|
|
|
| `<url> [url …]` | One or more URLs starting with `http://` or `https://` |
|
|
| `<config_file>` | Path to a plain-text URL list (same format as `simple.py`) |
|
|
|
|
Detection is automatic: if the first argument starts with `http://` or
|
|
`https://`, all arguments are treated as URLs; otherwise the single argument
|
|
is treated as a config file path.
|
|
|
|
**Exit codes:**
|
|
|
|
| Code | Meaning |
|
|
|------|---------|
|
|
| 0 | All URLs completed without a network error |
|
|
| 1 | One or more URLs failed, or a usage/config error |
|
|
|
|
HTTP error statuses (4xx, 5xx) do **not** set exit code 1 — the request
|
|
completed successfully at the network level. The status code is visible in
|
|
the output header.
|
|
|
|
## Example — single URL
|
|
|
|
```sh
|
|
python phases.py https://example.com
|
|
```
|
|
|
|
```
|
|
https://example.com (200)
|
|
DNS lookup : 21.74 ms
|
|
TCP connect : 10.79 ms
|
|
TLS handshake : 18.48 ms
|
|
Server (TTFB) : 73.72 ms
|
|
Transfer : 0.13 ms
|
|
─────────────────────────────
|
|
Total : 132.95 ms
|
|
```
|
|
|
|
## Example — HTTP URL (no TLS row)
|
|
|
|
```sh
|
|
python phases.py http://example.com
|
|
```
|
|
|
|
```
|
|
http://example.com (200)
|
|
DNS lookup : 3.37 ms
|
|
TCP connect : 13.51 ms
|
|
Server (TTFB) : 20.01 ms
|
|
Transfer : 2.85 ms
|
|
─────────────────────────────
|
|
Total : 39.77 ms
|
|
```
|
|
|
|
## Example — DNS failure (partial phases)
|
|
|
|
```sh
|
|
python phases.py https://no.such.host.invalid
|
|
```
|
|
|
|
```
|
|
https://no.such.host.invalid (FAILED)
|
|
─────────────────────────────
|
|
Total : 0.67 ms
|
|
✗ dns: [Errno 8] nodename nor servname provided, or not known
|
|
```
|
|
|
|
## Example — TLS failure (partial phases preserved)
|
|
|
|
```sh
|
|
python phases.py https://expired.badssl.com/
|
|
```
|
|
|
|
```
|
|
https://expired.badssl.com/ (FAILED)
|
|
DNS lookup : 28.32 ms
|
|
TCP connect : 129.01 ms
|
|
TLS handshake : 305.30 ms
|
|
─────────────────────────────
|
|
Total : 473.17 ms
|
|
✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired …
|
|
```
|
|
|
|
Note: DNS and TCP phases are populated even though the overall request failed.
|
|
This is the same behaviour as the Go tool — partial timing is preserved up to
|
|
the point of failure.
|
|
|
|
## Example — config file
|
|
|
|
```sh
|
|
python phases.py configs/all-ok.txt
|
|
```
|
|
|
|
Multiple URLs are separated by a blank line, matching Go's text output style.
|
|
|
|
## Using the error-path example configs
|
|
|
|
The `python/configs/` files from `simple.py` work identically with `phases.py`.
|
|
Compare the error handling between the two tools:
|
|
|
|
```sh
|
|
# DNS failure
|
|
python phases.py configs/dns-failure.txt
|
|
|
|
# Connection refused — note DNS phase is present, TCP is partial
|
|
python phases.py configs/connection-refused.txt
|
|
|
|
# TLS errors — all three phases up to TLS are present
|
|
python phases.py configs/tls-errors.txt
|
|
```
|
|
|
|
## Limitations
|
|
|
|
- **No redirect following.** 3xx responses are reported with their raw status
|
|
code; the redirect target is not probed. Use `latprobe/` (the full version)
|
|
or `simple.py` (which uses `urllib`, which follows redirects) if you need
|
|
the final destination's timing.
|
|
- **HTTP 1.1 + `Connection: close` only.** No keep-alive, no HTTP/2, no auth,
|
|
no custom headers beyond `Host` and `User-Agent`.
|
|
- **Timeout hard-coded at 10 s.** Use `latprobe/` for a `--timeout` flag.
|
|
- **No sampling.** Each URL is probed once. Use `latprobe/` for `-n` (min/avg/max).
|
|
- **Body is fully drained.** Transfer time includes reading the entire response
|
|
body, so it is real wall-clock transfer time.
|