docs: hxprobe plans, summaries, explanations, usage, and changelog
Plans (docs/plans/): - 2026-07-01-23-47-py-hxprobe-httpx.md — initial httpx probe design - 2026-07-02-09-32 through 14-05 — standalone project, toolchain, usage doc + Makefile, file input (-f), simplification pass, run-summary footer Summaries (docs/summaries/): one per completed feature, recording what was actually built, deviations from the plan, and verification steps Explanations (docs/explanations/): two deep-dives written during review — hxprobe concurrency model and worst-exit-code + render-loop analysis Usage (docs/usage/hxprobe.md): overview with pointer to hxprobe/USAGE.md for the full runnable reference Walkthrough (docs/py-latprobe-walkthrough.md): narrative tour of the latprobe Python package for interview / code-review context CHANGELOG.md: entries for all hxprobe features (toolchain, usage doc, file input, simplification, run-summary footer) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
139
docs/py-latprobe-walkthrough.md
Normal file
139
docs/py-latprobe-walkthrough.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# latprobe (Python) — code walkthrough
|
||||
|
||||
A guided tour of `python/latprobe/` for anyone about to edit the code by hand.
|
||||
For CLI usage/flags, see [docs/usage/py-latprobe.md](usage/py-latprobe.md).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
python/
|
||||
├── simple.py step 1 (standalone) — reachability + total elapsed time
|
||||
├── phases.py step 2 (standalone) — per-phase timing, single file
|
||||
├── sites.txt sample config for simple.py / phases.py
|
||||
├── configs/ fixture configs (all-ok, dns-failure, timeout, tls-errors, ...)
|
||||
├── latprobe/ step 3 — the real package
|
||||
│ ├── __main__.py
|
||||
│ ├── probe.py
|
||||
│ ├── aggregate.py
|
||||
│ ├── duration.py
|
||||
│ └── cli.py
|
||||
└── tests/
|
||||
├── test_probe.py
|
||||
├── test_cli.py
|
||||
└── test_integration.py
|
||||
```
|
||||
|
||||
`simple.py` and `phases.py` are earlier milestones, kept as reference (mirrors
|
||||
the Go step-by-step convention: simple → phases → full package). They are not
|
||||
imported by the package.
|
||||
|
||||
## The package: `python/latprobe/`
|
||||
|
||||
### `probe.py` — the engine
|
||||
|
||||
No argparse, no printing. `measure(url, opts) -> Result` does one HTTP(S)
|
||||
request over raw sockets (not `requests`/`urllib`) so it can time each phase
|
||||
itself:
|
||||
|
||||
- DNS (`socket.getaddrinfo`)
|
||||
- TCP connect
|
||||
- TLS handshake (`ssl.wrap_socket`, HTTPS only)
|
||||
- TTFB (write request → first byte, reading until `\r\n\r\n`)
|
||||
- Transfer (drain the rest of the body)
|
||||
|
||||
Each phase is timed with `time.perf_counter()` and wrapped by the `_p()`
|
||||
helper into a `Phase(ms, present)`.
|
||||
|
||||
Every failure path (DNS error, connect timeout, TLS error, etc.) returns
|
||||
early with a **partially-filled** `Result` — `fail_phase` says where it died,
|
||||
`err` holds the exception, and whatever phases completed before the failure
|
||||
are preserved. This is why the CLI can still print e.g. "DNS: 12ms" even if
|
||||
TCP connect then failed.
|
||||
|
||||
`Options(timeout, verbose)` goes in, `Result` comes out. When `verbose=True`,
|
||||
`Result.detail` (`VerboseDetail`) is also filled in: resolved IP, TLS
|
||||
version/cipher, parsed certificate (`CertInfo`), and all response headers.
|
||||
|
||||
This is the only file with actual measurement logic — change *how* something
|
||||
is measured here (e.g. add a redirect-follow phase, redefine what counts as
|
||||
TTFB).
|
||||
|
||||
### `aggregate.py` — pure math, no I/O
|
||||
|
||||
`summarize(results: list[Result]) -> Aggregate` takes a list of successful
|
||||
`Result`s (from repeated `-n` sampling of the same URL) and computes
|
||||
min/avg/max per phase into `PhaseStats`. `Aggregate.status_code` uses the
|
||||
*last* result's status. Small and self-contained — safe to extend (e.g. add
|
||||
p50/p95) without touching anything else.
|
||||
|
||||
### `duration.py`
|
||||
|
||||
One function: `parse_duration("500ms" | "10s" | "2m") -> float` seconds, used
|
||||
for `--timeout`.
|
||||
|
||||
### `cli.py` — everything else
|
||||
|
||||
Argument parsing, orchestration, and both text/JSON rendering. The file
|
||||
you'll touch most for UX changes.
|
||||
|
||||
- `_Parser` subclasses `argparse.ArgumentParser` to redirect all output
|
||||
through injected `stdout`/`stderr` streams and raise `_ArgExit` instead of
|
||||
calling `sys.exit` — this is what makes `run()` fully testable without
|
||||
subprocess (tests just pass in `io.StringIO()`).
|
||||
- Exit codes (`EXIT_DNS=2`, `EXIT_CONNECT=3`, etc.) are commented as
|
||||
mirroring the Go version. `_phase_code()` maps a `fail_phase` string to the
|
||||
matching code, and `run()` tracks the *worst* code across all URLs/samples.
|
||||
- `run(args, stdout, stderr) -> int` is the entry point:
|
||||
1. parse args → build `Options`
|
||||
2. run `_run_samples()` per URL concurrently via `ThreadPoolExecutor`
|
||||
(`--concurrency`, defaulting to `min(len(urls), 8)`)
|
||||
3. for each URL, pick one of four print paths in `_print_url`
|
||||
(`_print_single` / `_print_aggregate` / `_print_all_failed`), based on
|
||||
success/failure counts and whether `-n` > 1
|
||||
4. or, if `--json`, build dict entries via `_build_json_entry` and dump
|
||||
them all at the end.
|
||||
- Verbose rendering (`_print_verbose_block`) is shared between the single
|
||||
and aggregate text paths; JSON verbose data is built separately in
|
||||
`_build_json_entry`.
|
||||
|
||||
### `__main__.py`
|
||||
|
||||
Trivial shim: `sys.exit(run(sys.argv[1:], sys.stdout, sys.stderr))`, letting
|
||||
you run `python -m latprobe <url>`.
|
||||
|
||||
## Tests (`python/tests/`)
|
||||
|
||||
- `test_probe.py` — unit tests against `measure()` directly (against the
|
||||
`configs/*.txt` failure-mode fixtures: DNS failure, connection refused,
|
||||
TLS errors, timeouts).
|
||||
- `test_cli.py` — drives `run()` with injected `io.StringIO` streams,
|
||||
checking text/JSON output and exit codes.
|
||||
- `test_integration.py` — end-to-end, against the `configs/*.txt` files
|
||||
(`all-ok.txt`, `mixed.txt`, `http-errors.txt`, etc.).
|
||||
|
||||
## How it connects, end to end
|
||||
|
||||
```
|
||||
__main__.py
|
||||
→ cli.run()
|
||||
→ parses flags
|
||||
→ for each URL: probe.measure() (× count, across a thread pool)
|
||||
→ collects Result objects
|
||||
→ if count > 1: aggregate.summarize() → Aggregate
|
||||
→ cli.py's _print_* / _build_json_entry render text or JSON
|
||||
→ returns the worst exit code seen
|
||||
```
|
||||
|
||||
## Where to make changes
|
||||
|
||||
| Change you want | File |
|
||||
|-------------------------------------------|------------------|
|
||||
| Timing/measurement behavior | `probe.py` |
|
||||
| min/avg/max or new stats | `aggregate.py` |
|
||||
| Flags, output formatting, exit-code logic | `cli.py` |
|
||||
| Duration string parsing | `duration.py` |
|
||||
|
||||
The dataclasses (`Result`, `Phase`, `Options`, `VerboseDetail`, `Aggregate`,
|
||||
`PhaseStats`) are the contracts between these files — adding a field to
|
||||
`Result` in `probe.py` typically means threading it through `aggregate.py`
|
||||
(if it should be averaged) and `cli.py` (if it should be printed/JSON-encoded).
|
||||
Reference in New Issue
Block a user