Compare commits

...

3 Commits

Author SHA1 Message Date
f487a4b1bd docs: hxprobe plans, summaries, explanations, usage, and changelog
Plans (docs/plans/):
- 2026-07-01-23-47-py-hxprobe-httpx.md — initial httpx probe design
- 2026-07-02-09-32 through 14-05 — standalone project, toolchain, usage
  doc + Makefile, file input (-f), simplification pass, run-summary footer

Summaries (docs/summaries/): one per completed feature, recording what
was actually built, deviations from the plan, and verification steps

Explanations (docs/explanations/): two deep-dives written during review —
hxprobe concurrency model and worst-exit-code + render-loop analysis

Usage (docs/usage/hxprobe.md): overview with pointer to hxprobe/USAGE.md
for the full runnable reference

Walkthrough (docs/py-latprobe-walkthrough.md): narrative tour of the
latprobe Python package for interview / code-review context

CHANGELOG.md: entries for all hxprobe features (toolchain, usage doc,
file input, simplification, run-summary footer)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-02 13:22:59 +02:00
13a562e966 chore: wire hxprobe into root project (Makefile, CLAUDE.md, .gitignore)
Makefile: add hx-* targets (deps, run, lint, fmt, test, test-integration,
check, clean) that delegate into hxprobe/ via uv; fold hx-test and hx-check
into the umbrella test/check/clean targets; add HX_DIR variable and a note
that hxprobe is intentionally standalone (no cross-imports from python/).

CLAUDE.md: add "Implementation Summaries" convention — after each feature,
save a summary under docs/summaries/ with the same timestamp+slug naming as
plans. Add note about copying plan-mode scratch files into docs/plans/ before
starting implementation.

.gitignore: add *.egg-info/, .pytest_cache/, .ruff_cache/ for the hxprobe
uv+ruff+pytest toolchain.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-02 13:22:43 +02:00
404b372070 feat(hxprobe): httpx-based HTTP probe — full standalone package
Third Python implementation in the latency-tool progression.
Mirrors latprobe's feature set but replaces raw sockets with httpx,
gaining HTTP/2 support and redirect following.

Package layout (hxprobe/hxprobe/):
- probe.py: asyncio + httpx.AsyncClient with a _TimingStream transport
  wrapper that captures DNS/connect/TLS/TTFB/transfer phase timings via
  httpx event hooks (get_connection_stats, request_started, etc.).
  VerboseDetail captures resolved IP, TLS version/cipher/bits, cert CN/
  expiry/issuer (from httpx's SSLObject), and response headers.
  Options: timeout, verbose, follow_redirects, http2
- aggregate.py: summarize() → per-phase min/avg/max (same schema as latprobe)
- cli.py: run(args,stdout,stderr)->int; argparse with redirect_stdout/
  redirect_stderr + SystemExit catch; -n/--count, -c/--concurrency,
  --timeout, --fail, --json, -v/--verbose, --no-follow-redirects,
  --no-http2, -f/--file (URL list from file, mutually exclusive with args);
  multi-URL summary footer (tally + exit label) when len(urls) > 1;
  worst-exit-code logic mirrors Go/latprobe; JSON output is bare array
- duration.py: same parse_duration() as latprobe
- Exit codes: 0 ok, 1 usage, 2 dns, 3 connect, 4 timeout, 5 tls, 6 http≥400

Tests (hxprobe/tests/):
- test_probe.py: 18 hermetic tests using anyio + in-process ASGI servers
- test_cli.py: 36 hermetic tests (success, failures, JSON, verbose, -f flag,
  run-summary footer, worst-code accumulation)
- test_integration.py: pytest-marked @integration (excluded from hx-test)

Toolchain: uv + ruff + pytest; pyproject.toml with [dependency-groups];
hxprobe/Makefile standalone (help, deps, run, lint, fmt, test,
test-integration, check, clean); .python-version pins 3.14
Configs: 8 fixture files mirroring python/configs/ (all-ok through mixed)
USAGE.md: 16 runnable examples with real captured output

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-02 13:22:31 +02:00
46 changed files with 5361 additions and 9 deletions

3
.gitignore vendored
View File

@@ -8,6 +8,9 @@ __pycache__/
*.pyc
*.pyo
.venv/
*.egg-info/
.pytest_cache/
.ruff_cache/
# OS
.DS_Store

View File

@@ -4,6 +4,253 @@ All completed features are logged here in reverse-chronological order.
---
## 2026-07-02 14:05 — `hxprobe`: end-of-run summary footer for multi-URL runs
- `hxprobe/hxprobe/cli.py`: when more than one URL is probed (positional args
or `-f`), a footer is now appended after the last URL block — a tally of
every URL's outcome (`N ok`, `M failed` broken down by class) plus the
`→ exit N (label)` line tying it to the process exit code. Single-URL text
output and JSON output (still a bare array) are both byte-identical to
before — new `_print_run_summary()`, gated on `len(urls) > 1` and text mode
only
- The process **exit code itself is unchanged**: still the highest severity
across all URLs (worst-code wins), matching `latprobe`/Go and the 13
existing exit-code tests — a deliberate decision, since that scalar is a
documented cross-implementation contract (`hxprobe/USAGE.md`'s Exit codes
table). The footer exists to give humans the per-URL breakdown the scalar
can't show, not to change what gets returned
- New `_EXIT_LABELS` (inverse of `_PHASE_EXIT`) for rendering exit codes as
short labels (`dns`, `tls`, `http`, …) in the footer
- Refactored the per-URL accumulation loop to compute one worst-code per URL
first, then fold into the global `worst` — this also unified the two
"running max" idioms (`if c > worst: worst = c` vs `max(worst, ...)`) that
`docs/explanations/2026-07-02-13-25-hxprobe-worst-exit-code-and-render-loop.md`
had flagged as a stylistic wrinkle into a single `max(...)` call
- `hxprobe/tests/test_cli.py`: 3 new hermetic tests (`TestCLIRunSummary`) —
multi-URL mixed outcome, multi-URL all-ok, and single-URL-has-no-footer;
all 33 pre-existing tests pass unedited
- Triggered by a follow-up question: does a single worst-code exit even make
sense across multiple URLs with different error classes? Answer: keep the
scalar (parity contract) but add the missing visibility as output, not by
changing the exit code's semantics
- `hxprobe/USAGE.md`: new "Multi-URL summary footer" section with real
captured output; refreshed the "Multiple URLs" and `-f` multi-URL examples
to show the footer (previously stale — captured before this feature
existed); added a sentence to the Exit codes section
- `docs/usage/hxprobe.md`: one-line mention pointing at the new section
- Plan: `docs/plans/2026-07-02-14-05-hxprobe-run-summary-footer.md`
- Summary: `docs/summaries/2026-07-02-14-05-hxprobe-run-summary-footer.md`
---
## 2026-07-02 12:05 — `hxprobe` simplification pass (no behavior change)
- `hxprobe/hxprobe/cli.py`: deleted the `_Parser`/`_ArgExit` scaffolding
(~32 lines) that hand-reimplemented what `argparse.ArgumentParser` already
does (help→stdout, usage/errors→stderr, raise instead of hard-exit);
replaced with a plain `ArgumentParser` wrapped in
`contextlib.redirect_stdout`/`redirect_stderr`, catching `SystemExit` at one
site
- `hxprobe/hxprobe/probe.py`: trimmed `_TimingStream.get_extra_info` to the
one branch (`ssl_object`) actually consumed on hxprobe's request path — the
`server_addr`/`client_addr` branches were dead (only `httpx`'s own CLI
queries them); removed `_dns_set`/`_connect_set`/`_tls_set` from `_Trace`,
redundant with the `Phase.present` flag already on `self.dns`/`connect`/`tls`
- `hxprobe/hxprobe/cli.py`: `_load_urls` no longer takes only the first
whitespace token per line ("forward-compatible with future `key=value`
annotations" that never materialized); extracted `_summarize_failures()`
to remove a duplicated dedup loop shared by `_print_failure_summary` and
`_build_json_entry`
- Triggered by a question about a dead `file=` parameter on
`_Parser.print_help`/`print_usage`; verified no behavior changed — all 33
existing tests pass unedited, plus manual smoke tests of `-h`, no-args
(stdout/stderr routing double-checked via separate file redirection), and a
live verbose request
- Plan: `docs/plans/2026-07-02-12-05-hxprobe-simplification.md`
- Summary: `docs/summaries/2026-07-02-12-05-hxprobe-simplification.md`
---
## 2026-07-02 11:14 — `hxprobe`: read target URLs from a file (`-f`/`--file`)
- `hxprobe/hxprobe/cli.py`: new `-f`/`--file PATH` flag reads URLs from a
plain-text file (one per line, `#` comments, blank lines skipped, first
whitespace-separated token per line) — same format as `simple.py`'s
`load_sites()`, reimplemented locally as `_load_urls()` rather than
imported (hxprobe still imports nothing outside its own directory)
- `-f`/`--file` is **mutually exclusive** with positional `url` args (both
or neither given → usage error); `urls` positional changed from
`nargs="+"` to `nargs="*"` to allow the file-only case
- Missing file / unreadable file / empty file all produce a clear usage
error (`EXIT_USAGE`) rather than a traceback
- `hxprobe/configs/*.txt`: 7 new fixtures mirroring `python/configs/`'s set
(all-ok, dns-failure, connection-refused, timeout, tls-errors,
http-errors, mixed), each with a verified expected exit code in its
header comment — actually run against `hxprobe -f ...` during
implementation, not assumed from the `simple.py` originals (whose
blanket 0/1 exit scheme differs from hxprobe's per-failure-class 06)
- `hxprobe/tests/test_cli.py`: 5 new hermetic tests (`TestCLIFileInput`) —
successful multi-URL read from a temp file, missing file, empty file,
both-sources-given, and neither-given error paths
- `hxprobe/USAGE.md`: new "Reading URLs from a file (`-f`)" section with
real captured output, placed after "Multiple URLs"
- Plan: `docs/plans/2026-07-02-11-14-hxprobe-file-input.md`
- Summary: `docs/summaries/2026-07-02-11-14-hxprobe-file-input.md`
---
## 2026-07-02 10:22 — `hxprobe` usage reference doc + standalone Makefile
- `hxprobe/USAGE.md`: new "Runnable Usage Reference" matching the depth of
`python/configs/usage-latprobe.md` — 16 cases with real captured output
(basic, verbose × HTTPS/plain-HTTP/redirect-followed/`--no-follow-
redirects`/`--no-http2`/TLS-failure/DNS-failure, sampling, multi-URL,
`--fail`, JSON, JSON+verbose, timeout, exit-codes table, Makefile
shortcuts). Placed inside `hxprobe/` itself (not `python/configs/`) so it
travels with the project if extracted to its own repo
- Timeout example uses `192.0.2.1` (RFC 5737 TEST-NET-1) instead of
`10.255.255.1` — the latter resolves to an immediate "connection refused"
in this dev sandbox rather than a genuine timeout; `192.0.2.1` reproduces
a real ~500ms timeout reliably
- `docs/usage/hxprobe.md`: added a one-line pointer to `hxprobe/USAGE.md` at
the top; no other changes — it stays the CLAUDE.md-mandated summary doc
- `hxprobe/Makefile`: new, fully standalone (`help`, `deps`, `run`, `lint`,
`fmt`, `test`, `test-integration`, `check`, `clean`) — same auto-generated
`## comment` help style as the root Makefile, short target names (no
`hx-` prefix needed inside `hxprobe/`'s own scope). Deliberately
independent from the root Makefile's `hx-*` targets — neither calls into
the other, per explicit user decision (avoids one Makefile's changes
silently breaking the other, at the cost of some duplicated `uv`/`pytest`
invocation logic)
- No changes to the root `Makefile` — verified its `hx-*` section is
byte-for-byte unchanged
- Plan: `docs/plans/2026-07-02-10-22-hxprobe-usage-doc-and-makefile.md`
- Summary: `docs/summaries/2026-07-02-10-22-hxprobe-usage-doc-and-makefile.md`
---
## 2026-07-02 09:57 — `hxprobe` toolchain modernization (uv, ruff, pytest)
- Adopted `uv` for environment/dependency management: `hxprobe/pyproject.toml`
gets a `[dependency-groups] dev = ["pytest>=8.0", "ruff>=0.8"]` section;
`hxprobe/.python-version` pins Python 3.14; `hxprobe/uv.lock` (new, 18
packages resolved) pins every transitive dependency (`httpx`, `httpcore`,
`h2`, `certifi`, etc.) for reproducible installs — previously nothing was
pinned beyond `httpx[http2]>=0.28`
- Added `ruff` for linting + formatting: `[tool.ruff]` config
(`target-version = "py311"`, `line-length = 100`); fixed the one real
finding (`import sys` unused in `cli.py`, inherited from the original
`latprobe/cli.py`); ran `ruff format` across the project (6 files
reformatted — mostly collapsing the hand-aligned `=`/dict-key columns to
single-space, no semantic changes; verified via full syntax check + test
run before and after)
- Swapped the test runner from stdlib `unittest discover` to `pytest`.
Existing `unittest.TestCase` classes run unchanged (pytest is a superset
runner) — no test-code rewrite. Replaced the `test_[!i]*.py` filename-glob
hermetic/integration split with a proper `pytest.mark.integration` marker
(`pytestmark = pytest.mark.integration` in `tests/test_integration.py`);
registered in `[tool.pytest.ini_options]` to avoid unknown-marker warnings
- Skipped mypy for now (type hints stay as documentation only) — explicit
user decision, not an oversight
- `hxprobe/README.md`: new — quick start (`uv sync`, `uv run ...`) for the
standalone project, independent of the parent repo's docs
- Makefile: `hx-deps`/`hx-run`/`hx-test`/`hx-test-integration` now shell out
to `uv sync`/`uv run` instead of hand-managed `python -m venv` + `pip
install -e .` (removed `HX_VENV`/`HX_VENV_PYTHON` vars — uv owns this now);
added `hx-lint`/`hx-fmt`; `hx-check` now runs lint + hermetic tests
(mirrors `go-check`'s fmt+vet+test bundling, previously only ran tests)
- `docs/usage/hxprobe.md`: updated Setup and Makefile-targets sections for
the new toolchain
- `.gitignore`: added `.pytest_cache/`/`.ruff_cache/`
- No behavior change to the probe/CLI itself — confirmed identical output
before/after, `python/latprobe` untouched, zero `latprobe` imports remain
in `hxprobe/`
- Plan: `docs/plans/2026-07-02-09-57-hxprobe-toolchain-modernization.md`
- Summary: `docs/summaries/2026-07-02-09-57-hxprobe-toolchain-modernization.md`
---
## 2026-07-02 09:32 — `hxprobe` extracted into a standalone top-level project
- Moved `hxprobe` out of `python/` into a new top-level `hxprobe/` directory
(sibling of `go/` and `python/`), with its own `pyproject.toml`, its own
venv (`hxprobe/.venv`), and its own `hxprobe/tests/` — structured so it
could be `cp -r`'d into a separate repo and work unchanged
- Removed every `from latprobe import ...` in `hxprobe/`: `probe.py` now
defines its own `Options`/`Phase`/`Result`/`VerboseDetail`/`CertInfo`
dataclasses and its own `_parse_cert`/`_parse_cert_date` helpers (copied,
not shared); `aggregate.py`/`duration.py` are verbatim copies (their
imports were already package-relative, so no edits needed);
`cli.py` is now a full standalone implementation (own exit codes, argparse,
text/JSON rendering) instead of delegating to `latprobe.cli.run()` via an
injected `measure_fn`
- Reverted `python/latprobe/{cli.py,probe.py}` to their pre-`hxprobe` state
(`git checkout --`) — the `measure_fn`/`prog`/`description`/`protocol_flags`
injection points, `Options.follow_redirects`/`http2`, and
`VerboseDetail.http_version`/`redirect_count` only existed to support the
now-removed sharing; `python/` is back to zero third-party dependencies, no
`pyproject.toml`, no venv
- Makefile: removed `py-deps` and the `PY_VENV*` variables; `py-test`/
`py-check` reverted to running directly against `$(PYTHON)`; added a new
standalone `hx-deps`/`hx-run`/`hx-test`/`hx-test-integration`/`hx-check`/
`hx-clean` section using `HX_DIR`/`HX_VENV*`; umbrella `test`/`check`/
`clean` now include the hxprobe targets alongside Go and Python
(`go-*`/`py-*`/`hx-*`)
- Tests moved and renamed to drop the now-redundant `hx_`/`_hx` segments:
`test_hx_probe.py``hxprobe/tests/test_probe.py`,
`test_hx_cli.py``hxprobe/tests/test_cli.py`,
`test_integration_hx.py``hxprobe/tests/test_integration.py` (same
`test_[!i]*.py` hermetic-vs-integration exclusion convention as `latprobe`)
- `docs/usage/py-hxprobe.md` renamed to `docs/usage/hxprobe.md` and rewritten
for the new standalone structure and Makefile targets; the TCP_NODELAY/
Nagle TTFB-accuracy finding is preserved
- No behavior change — same CLI flags, same output, same six-phase timing,
same HTTP/2/redirect handling as before; this was a pure decoupling/
restructuring
- Plan: `docs/plans/2026-07-02-09-32-hxprobe-standalone-project.md`
- Summary: `docs/summaries/2026-07-02-09-32-hxprobe-standalone-project.md`
---
## 2026-07-02 00:24 — `hxprobe`: httpx-based Python probe, Go-client parity
- New sibling package `python/hxprobe/`, built on `httpx` instead of raw
sockets, so the client matches Go's `http.DefaultClient`: HTTP/2 negotiated
via ALPN, redirects followed by default, connection pooling, default TLS
verification — while still reporting the full six-phase breakdown (DNS, TCP
connect, TLS, TTFB, Transfer, Total)
- `hxprobe/probe.py`: DNS/TCP connect/TLS timed by instrumenting a custom
httpcore `NetworkBackend`/`NetworkStream` (`_TimingBackend`/`_TimingStream`);
HTTP framing (HTTP/1.1 or HTTP/2), redirects, and keep-alive stay entirely
owned by httpx; `measure()` returns the same `latprobe.probe.Result`
dataclass, so aggregation/rendering are reused unchanged
- `latprobe/probe.py`: added `Options.follow_redirects`/`Options.http2`
(ignored by the raw-socket `measure()`) and
`VerboseDetail.http_version`/`redirect_count` (always `""`/`0` there)
- `latprobe/cli.py`: `run()`/`_run_samples()` take an injectable `measure_fn`
(default: the existing socket `measure`), plus `prog`/`description`/
`protocol_flags` overrides — `hxprobe.cli.run()` reuses the entire argparse,
concurrency, exit-code, and text/JSON rendering pipeline unchanged
- New CLI flags (hxprobe only, via `protocol_flags=True`): `--no-http2`,
`--no-follow-redirects`
- `python/pyproject.toml`: first third-party dependency in this repo
(`httpx[http2]`); `make py-deps` provisions `python/.venv`
- Verified experimentally that `latprobe`'s raw socket — which never sets
`TCP_NODELAY` — pays a real ~40-50ms Nagle/delayed-ACK penalty on its TTFB
phase; `hxprobe` sets `TCP_NODELAY` (matching httpcore's default and Go's
`net.Dialer`) and does not. Documented in `docs/usage/py-hxprobe.md` as a
known divergence — `hxprobe`'s TTFB is the more accurate of the two, not
just different
- 17 new hermetic tests (`tests/test_hx_probe.py`), 11 new hermetic CLI tests
(`tests/test_hx_cli.py`), 9 new live integration tests
(`tests/test_integration_hx.py`, excluded from the default gate)
- Makefile: `py-deps`, `hx-run`, `hx-test-integration`; `py-test`/`py-check`
now run via the venv and include the new hermetic hxprobe tests
- User doc: `docs/usage/py-hxprobe.md`
- Plan: `docs/plans/2026-07-01-23-47-py-hxprobe-httpx.md`
---
## 2026-07-01 12:55 — `--verbose` / `-v` flag for `latprobe` (Python)
- `-v`/`--verbose` flag added to the `latprobe` CLI

View File

@@ -10,6 +10,23 @@ a short kebab-case description.
Example: `docs/plans/2026-07-01-00-08-go-latency-tool.md`
If a plan is drafted via plan-mode tooling that restricts writes to a separate
scratch file, copy the approved plan verbatim into `docs/plans/` as the first
implementation step, before making any code changes. The scratch file existing
elsewhere does not satisfy this convention.
## Implementation Summaries
After finishing the implementation of a feature, save a summary under
`docs/summaries/` with a filename that starts with the **current timestamp in
`yyyy-mm-dd-hh-mm` format** followed by a short kebab-case description (same
convention as plans). The summary covers what was actually built (as opposed
to the plan, which covers what was intended): files added/changed, key design
decisions, any deviations from the plan, notable findings made along the way,
and how it was verified.
Example: `docs/summaries/2026-07-02-00-29-py-hxprobe-httpx.md`
## Changelog
Every completed feature is appended to `CHANGELOG.md` at the project root with

View File

@@ -1,9 +1,14 @@
GO_DIR := go
PY_DIR := python
BINARY := latprobe
PYTHON := python3.14
ARGS ?=
SITES ?= $(PY_DIR)/sites.txt
GO_DIR := go
PY_DIR := python
BINARY := latprobe
PYTHON := python3.14
ARGS ?=
SITES ?= $(PY_DIR)/sites.txt
# hxprobe is a fully standalone project (own pyproject.toml, own uv-managed
# venv/lockfile) — it imports nothing from python/latprobe, and could be
# `cp -r`'d into its own repo as-is.
HX_DIR := hxprobe
.DEFAULT_GOAL := help
@@ -24,10 +29,10 @@ all: check build ## Run checks then build
build: go-build ## Build binary (delegates to go-build)
.PHONY: test
test: go-test py-test ## Run all tests (Go + Python)
test: go-test py-test hx-test ## Run all tests (Go + Python + hxprobe)
.PHONY: check
check: go-check py-check ## Run fmt + vet + test gate (Go + Python)
check: go-check py-check hx-check ## Run fmt + vet + test gate (Go + Python + hxprobe)
.PHONY: fmt
fmt: go-fmt ## Format source code (delegates to go-fmt)
@@ -36,7 +41,7 @@ fmt: go-fmt ## Format source code (delegates to go-fmt)
vet: go-vet ## Run go vet (delegates to go-vet)
.PHONY: clean
clean: go-clean py-clean ## Remove build and coverage artifacts
clean: go-clean py-clean hx-clean ## Remove build and coverage artifacts
# ── Go targets ────────────────────────────────────────────────────────────────
@@ -126,3 +131,39 @@ py-check: py-test ## py: run Python test gate (hermetic only)
py-clean: ## py: remove Python bytecode and __pycache__ dirs
@find $(PY_DIR) -type d -name '__pycache__' -exec rm -rf {} + 2>/dev/null; \
find $(PY_DIR) -name '*.pyc' -delete 2>/dev/null; true
# ── hxprobe (standalone httpx-based probe; own pyproject.toml/uv.lock, no
# imports from python/latprobe — see hxprobe/pyproject.toml) ─────────────────
.PHONY: hx-deps
hx-deps: ## hx: sync hxprobe's uv-managed environment from its lockfile; idempotent
cd $(HX_DIR) && uv sync
.PHONY: hx-run
hx-run: hx-deps ## hx: run hxprobe (pass flags via ARGS="…")
cd $(HX_DIR) && uv run python -m hxprobe $(ARGS)
.PHONY: hx-lint
hx-lint: hx-deps ## hx: lint hxprobe with ruff
cd $(HX_DIR) && uv run ruff check .
.PHONY: hx-fmt
hx-fmt: hx-deps ## hx: format hxprobe with ruff
cd $(HX_DIR) && uv run ruff format .
.PHONY: hx-test
hx-test: hx-deps ## hx: run hxprobe unit tests (hermetic)
cd $(HX_DIR) && uv run pytest tests -m "not integration" -v $(ARGS)
.PHONY: hx-test-integration
hx-test-integration: hx-deps ## hx: run hxprobe integration tests against live internet services (HTTP/2, redirects)
cd $(HX_DIR) && uv run pytest tests -m integration -v $(ARGS)
.PHONY: hx-check
hx-check: hx-lint hx-test ## hx: run hxprobe test gate (lint + hermetic tests)
.PHONY: hx-clean
hx-clean: ## hx: remove hxprobe bytecode, caches, and egg-info
@find $(HX_DIR) -type d -name '__pycache__' -exec rm -rf {} + 2>/dev/null; \
find $(HX_DIR) -name '*.pyc' -delete 2>/dev/null; \
rm -rf $(HX_DIR)/*.egg-info $(HX_DIR)/.pytest_cache $(HX_DIR)/.ruff_cache; true

View File

@@ -0,0 +1,56 @@
# How does concurrency work in hxprobe?
## Question
Selection in `hxprobe/hxprobe/cli.py:448-449`:
```python
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as ex:
all_results = list(ex.map(_probe, urls))
```
> explain how concurrency works in hxprobe
## Answer
There are two independent levels of repetition:
1. **Across URLs** — parallel, via a thread pool.
2. **Across samples of the same URL** (`-n`/`--count`) — sequential, within a single thread.
### Worker count
`cli.py:440-443`
```python
workers = ns.concurrency
if workers <= 0:
workers = min(len(urls), 8)
workers = max(1, min(workers, len(urls)))
```
`-c`/`--concurrency` picks the pool size; `0` (default) means "auto" → `min(len(urls), 8)`. The final clamp guarantees `1 ≤ workers ≤ len(urls)` — never more threads than there are URLs to probe, never zero.
### The pool itself
`cli.py:445-449`
```python
def _probe(url: str) -> tuple[list[Result], list[Result]]:
return _run_samples(url, count, opts)
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as ex:
all_results = list(ex.map(_probe, urls))
```
One task per URL is submitted to the pool. `_probe` delegates to `_run_samples`, `cli.py:64-69`:
```python
def _run_samples(url: str, count: int, opts: Options) -> tuple[list[Result], list[Result]]:
succeeded, failed = [], []
for _ in range(count):
r = measure(url, opts)
(failed if r.err else succeeded).append(r)
return succeeded, failed
```
So within one URL's thread, the `count` samples run **one at a time** — never concurrently. This is deliberate: if N samples fired at the same host in parallel, they'd contend for the same TCP/TLS handshake path and connection setup, and the resulting min/avg/max per phase would reflect that contention rather than the host's actual latency. Keeping samples sequential is what makes `-n`'s statistics meaningful.
### Why threads work here despite the GIL
`measure()` does blocking socket/SSL I/O — `socket.connect()`, `.recv()`, `ssl.wrap_socket()`, etc. (see the custom `_TimingBackend`/`_TimingStream` in `probe.py`). CPython releases the GIL around blocking syscalls, so N threads genuinely overlap in wall-clock time waiting on the network, even though only one thread executes Python bytecode at once. This is I/O-bound concurrency, not CPU parallelism — threads are the right tool, not `asyncio` or multiprocessing.
### Ordering guarantee
`ex.map(_probe, urls)` returns results in the *same order as the input `urls`*, regardless of which thread finishes first — that's a documented property of `Executor.map`. That's why `list(ex.map(...))` can be zipped directly against `urls` afterward (`cli.py:454`) to produce deterministic text/JSON output order, even though the underlying probes complete out of order.
### No shared mutable state / no locks needed
Each thread's `_probe` call returns its own `(succeeded, failed)` tuple; nothing is written to a shared structure until back in the main thread after the `with` block exits (which also blocks until every submitted task completes, since `ThreadPoolExecutor.__exit__` calls `shutdown(wait=True)`). The worst-exit-code accumulation and JSON building (`cli.py:451-472`) then run single-threaded over `all_results`.

View File

@@ -0,0 +1,144 @@
# What does the per-URL worst-exit-code / render loop do, and does it make sense?
## Question
Selection in `hxprobe/hxprobe/cli.py:454-472`:
```python
for i, (url, (succeeded, failed)) in enumerate(zip(urls, all_results)):
for r in failed:
c = _phase_code(r.fail_phase)
if c > worst:
worst = c
if ns.fail:
for r in succeeded:
if r.status_code >= 400:
worst = max(worst, EXIT_HTTP)
last_detail = succeeded[-1].detail if succeeded else (failed[-1].detail if failed else None)
if ns.json_out:
json_items.append(_build_json_entry(url, succeeded, failed, last_detail))
continue
if i > 0:
stdout.write("\n")
_print_url(url, succeeded, failed, count, ns.fail, stdout)
```
> explain following block and if it makes sense, other related stuff, and save it in explanations
## What it does
This is the single pass over per-URL results that runs after the thread pool
(`cli.py:448-449`, see [2026-07-02-13-10-hxprobe-concurrency.md](2026-07-02-13-10-hxprobe-concurrency.md))
finishes. It does two jobs in one loop: compute the process's final exit code,
and render output (text or accumulate JSON) — one URL at a time, in input
order (guaranteed by `zip(urls, all_results)` since `ex.map` preserves order).
**1. Network-failure exit code (`cli.py:455-458`)**
```python
for r in failed:
c = _phase_code(r.fail_phase)
if c > worst:
worst = c
```
Every failed sample (across every URL, since `worst` is declared once before
the loop) is mapped to an exit code via `_phase_code` / `_PHASE_EXIT`
(`cli.py:24-32`):
```python
_PHASE_EXIT: dict[str, int] = {
"dns": EXIT_DNS, # 2
"timeout": EXIT_TIMEOUT, # 4
"tls": EXIT_TLS, # 5
}
def _phase_code(fail_phase: str) -> int:
return _PHASE_EXIT.get(fail_phase, EXIT_CONNECT) # 3, the fallback
```
Any `fail_phase` not in the table — `"connect"`, `"transfer"`, `"request"`
falls back to `EXIT_CONNECT` (3). `worst` tracks the running max across all
URLs/samples, so the process exit code always reflects the single worst
failure class seen, per the exit-code table (0 ok … 6 http via `--fail`).
**2. `--fail` (HTTP status ≥ 400) exit code (`cli.py:459-462`)**
```python
if ns.fail:
for r in succeeded:
if r.status_code >= 400:
worst = max(worst, EXIT_HTTP)
```
Only runs when `--fail` is passed. Note this scans `succeeded` — a 404 is not
a network failure, so those `Result`s land in `succeeded` with a populated
`status_code`; `--fail` is what turns "successfully got a bad status" into a
non-zero exit, curl-style.
**3. Verbose-detail selection (`cli.py:464`)**
```python
last_detail = succeeded[-1].detail if succeeded else (failed[-1].detail if failed else None)
```
Prefers the last successful sample's `detail` (freshest full picture: IP,
protocol, TLS, headers); falls back to the last *failed* sample's `detail` if
nothing succeeded (e.g. resolved IP is still known even on a connection
refusal); `None` if there's nothing to show. Reasonable design — surfaces
partial diagnostic info even on total failure.
**4. JSON accumulation vs. text rendering (`cli.py:466-472`)**
```python
if ns.json_out:
json_items.append(_build_json_entry(url, succeeded, failed, last_detail))
continue
if i > 0:
stdout.write("\n")
_print_url(url, succeeded, failed, count, ns.fail, stdout)
```
JSON mode builds up `json_items` (dumped once after the loop) and skips
straight to the next URL via `continue`. Text mode writes a blank-line
separator before every URL block except the first (`i > 0`), then delegates
actual formatting to `_print_url`, which picks one of four branches
(single / aggregate / all-failed / mixed) based on `n_ok`/`n_fail`/`total_count`.
## Does it make sense?
**Yes, structurally.** Combining exit-code accumulation and rendering into one
O(n) pass is reasonable for a CLI at this scale — no need to split into two
loops. The `last_detail` fallback logic is a genuinely good touch. The one
stylistic wrinkle — the DNS/connect/timeout/tls loop uses
`if c > worst: worst = c` while the `--fail` branch uses `worst = max(worst,
EXIT_HTTP)` for the same "keep the running max" purpose — is harmless
inconsistency, not a bug.
**Update (2026-07-02):** the "does a single worst-code exit even make sense
across multiple URLs with different errors" question came back as a follow-up
and led to a real feature — see
[2026-07-02-14-05-hxprobe-run-summary-footer.md](2026-07-02-14-05-hxprobe-run-summary-footer.md).
Short version: the scalar exit code is kept (it's a documented cross-
implementation contract with `latprobe`/Go), but multi-URL runs now get an
end-of-run summary footer tallying every URL's outcome, so the "worst code"
is no longer the only visibility into what happened. That change also
unified the two idioms noted above into one `max(...)` call.
**One real gap, found while checking this: `-n`/`--count` is unvalidated.**
`cli.py:349-355` declares `--count` as `type=int, default=1` with no minimum.
`_run_samples` (`cli.py:64-69`) does `for _ in range(count): ...`, so
`--count 0` (or any negative value) makes the loop body never execute, and
both `succeeded` and `failed` come back empty for that URL. Confirmed live:
```
$ hxprobe --count 0 https://example.com
(0, 0 samples)
exit=0
$ hxprobe --count -2 https://example.com
(0, 0 samples)
exit=0
```
The URL itself is missing from the header, `status` reads `0`, and the exit
code is `0` (success) — because `_print_url` falls into the `elif n_fail == 0`
aggregate branch with `summarize([])`, which returns a bare
`Aggregate()` (all defaults, `url=""`) rather than anything referencing the
actual `url` variable. This is silent garbage output instead of a clear
usage error, and it's inconsistent with how the rest of `run()` already
validates arguments (e.g. the `parser.error(...)` calls for the
`urls`/`--file` mutual-exclusion checks at `cli.py:407-410`). Worth a
`parser.error("count must be >= 1")`-style guard if this is ever picked up —
not fixed here since it wasn't asked for, just flagged as a finding.

View File

@@ -0,0 +1,130 @@
# Plan: `hxprobe` — httpx-based Python probe (Go-client parity)
## Context
The existing Python `latprobe` package measures per-phase HTTP latency with raw
sockets. That gives an excellent DNS/TCP/TLS/TTFB/Transfer breakdown, but the
cost is that it always speaks **HTTP/1.1** and **does not follow redirects**
diverging from the Go implementation, whose `http.DefaultClient` negotiates
**HTTP/2** via ALPN, **follows redirects** (up to 10), pools connections, and
verifies TLS by default (confirmed: `go/internal/probe/probe.go` uses
`http.DefaultClient.Do` with no custom transport/`CheckRedirect`).
Goal: add a **second Python implementation, `hxprobe`**, built on the `httpx`
library so it matches the Go client's protocol capabilities (HTTP/2, redirects,
pooling, TLS verification) **while preserving the full 6-phase timing** that is
latprobe's whole point. Python has no equivalent of Go's `net/http/httptrace`,
so the phase breakdown is recovered by instrumenting httpx's network backend.
This is an **additive, Python-only experiment** — intentionally outside the
CLAUDE.md "Go first, then Python port" flow, since the user explicitly asked for
a library-based Python variant. No Go change is required.
## Approach
New sibling package `python/hxprobe/`, reusing everything reusable from
`latprobe` (dataclasses, aggregation, duration parsing, CLI rendering) so the
only genuinely new code is the httpx probe backend.
### Key design: instrumented httpx transport
`httpx` (sync `httpx.Client`) runs on `httpcore`. To recover per-phase timing we
subclass httpcore's sync network backend and time the phases at the socket
level, letting httpx own HTTP framing, HTTP/2, redirects, and keep-alive:
- `connect_tcp(...)` — reimplement DNS + TCP as separate steps (port the
`socket.getaddrinfo``socket.connect` split already in
`latprobe/probe.py:167-203`), timestamping **DNS** and **TCP connect**
independently, and capturing the resolved IP.
- `start_tls(...)` — timestamp the **TLS handshake**; pull negotiated version /
cipher / peer cert from the SSL object for verbose mode.
- **TTFB** = headers-received minus end-of-TLS (server processing), measured via
`client.stream("GET", ...)` (the `stream()` context yields once response
headers arrive).
- **Transfer** = iterating `resp.iter_raw()` to EOF, minus headers-received.
- **Total** = wraps the whole `measure()` call.
Each `measure()` call uses a **fresh `httpx.Client` (no cross-sample pooling)** so
every `-n` sample yields a full phase breakdown — matching the current
raw-socket `latprobe` behavior rather than Go's pool-reuse quirk.
A per-call trace object (held by the backend instance) records timings and the
`fail_phase` at the exact point a phase raises, giving precise error
classification (`dns`/`connect`/`timeout`/`tls`/`transfer`/`request`) without
guessing from httpx exception types.
**Redirects (followed by default, Go parity):** DNS/connect/TLS are reported
from the **first** connection (mirrors Go's `connectStart.IsZero()` guard);
TTFB/Transfer/Total span the full followed chain. `redirect_count` and the
negotiated `http_version` (`h2` vs `http/1.1`) are surfaced as new verbose
fields — a genuine capability the socket version lacks.
## Files
**New:**
- `python/pyproject.toml` — project metadata; dependency `httpx[http2]` (pulls
`h2`). Makes `latprobe` + `hxprobe` `pip install -e .`-able; tests still run
via `PYTHONPATH`.
- `python/hxprobe/__init__.py`
- `python/hxprobe/probe.py``measure(url, opts) -> latprobe.probe.Result`
(imports & returns the **same `Result`** so aggregation/rendering just work);
`_TimingBackend`, the per-call trace, error classification, verbose capture.
- `python/hxprobe/cli.py` — thin: delegates to `latprobe.cli.run(...)` passing
`measure_fn=hxprobe.probe.measure` (see reuse edit below).
- `python/hxprobe/__main__.py``sys.exit(cli.run(sys.argv[1:], ...))`.
- `python/tests/test_hx_probe.py` — hermetic, local `http.server`: phase
presence, **redirect following** (302 handler — validates the Go-parity
feature), connection-refused → `connect`, black-hole port → `timeout`.
- `python/tests/test_hx_cli.py` — hermetic CLI via `run()` with `io.StringIO`
(mirrors `tests/test_cli.py:73-76`).
- `python/tests/test_integration_hx.py` — live, **excluded from default gate**
(filename starts `test_i…`, so the `test_[!i]*.py` glob skips it): probe a
real HTTP/2 host and assert negotiated `http_version == "h2"`; guard with a
`@skipUnless(_online())` like `tests/test_integration.py:29-40`.
- `docs/usage/py-hxprobe.md` — usage doc (what it does, flags, example with
expected output, and an explicit socket-vs-httpx capability comparison table),
per CLAUDE.md.
**Edited (small, backward-compatible):**
- `python/latprobe/cli.py` — parameterize `run()` and `_run_samples()` with an
injectable `measure_fn` (default = current `latprobe.probe.measure`), so
`hxprobe` reuses all argparse, concurrency, exit-code, text/JSON rendering
logic. Extend `_print_verbose_block` and `_build_json_entry` to show
`http_version` / `redirect_count` **when present** (existing socket path never
sets them → output unchanged).
- `python/latprobe/probe.py` — add optional fields with safe defaults:
`Options.follow_redirects=True`, `Options.http2=True` (ignored by the socket
measure); `VerboseDetail.http_version=""`, `VerboseDetail.redirect_count=0`.
- `Makefile` — add `py-deps` (create `.venv`, `pip install -e python`),
`hx-run` (`cd python && python -m hxprobe $(ARGS)`), and fold `test_hx_*` into
the existing `py-test` gate; add `hx-test-integration` for the live h2 check.
- `CHANGELOG.md` — append a timestamped one-line entry on completion.
**Reused as-is:** `latprobe.aggregate.summarize`,
`latprobe.duration.parse_duration`, `latprobe.probe.{Result,Phase,CertInfo}`,
and the entire `latprobe.cli` renderer via the `measure_fn` injection.
## Flags / behavior
Same surface as `latprobe` (`-n/--count`, `-c/--concurrency`, `--timeout`,
`--fail`, `--json`, `-v/--verbose`) via the reused parser, plus two new opt-outs
for the Go-like defaults: `--no-http2` and `--no-follow-redirects`. Exit codes
06 stay identical to Go/`latprobe`.
## Verification
1. `make py-deps` — create venv, install `httpx[http2]`.
2. `make hx-run ARGS="https://example.com"` — full 6-phase text output renders.
3. `make hx-run ARGS="-v https://www.cloudflare.com"` — verbose block shows
`http_version: h2` and TLS/cert details.
4. **Go-parity spot checks:**
- HTTP/2: `python -m hxprobe -v <h2-host>` reports `h2` where
`python -m latprobe -v <h2-host>` reports HTTP/1.1.
- Redirects: `python -m hxprobe http://github.com` follows to https and shows
`redirect_count > 0` (socket `latprobe` shows a raw 301).
5. `make py-test` — hermetic suite (now including `test_hx_probe.py`,
`test_hx_cli.py`) is green; `latprobe`'s existing tests still pass (proves the
`measure_fn`/`Options`/`VerboseDetail` edits are backward-compatible).
6. `make hx-test-integration` — live test confirms real `h2` negotiation.
7. Confirm `--json` output for `hxprobe` matches the `latprobe` schema plus the
optional `verbose.http_version` / `verbose.redirect_count` keys.

View File

@@ -0,0 +1,151 @@
# Plan: extract `hxprobe` into a fully standalone top-level project
## Context
`hxprobe` currently lives at `python/hxprobe/` and imports shared code from
`latprobe` (`latprobe.probe.{Options,Phase,Result,VerboseDetail,_parse_cert}`,
`latprobe.cli.run`). That coupling was a deliberate reuse choice at the time,
but the user now wants `hxprobe` to be a genuinely independent project — no
`from latprobe import ...` anywhere, duplication accepted — and, per their
follow-up answer, independent enough that it could be `cp -r`'d into its own
repo tomorrow: own `pyproject.toml`, own venv, own Makefile section, own
top-level directory (sibling to `go/` and `python/`), not nested under `python/`.
Confirmed via `git status`/`git log`: everything hxprobe-related
(`python/hxprobe/`, `python/pyproject.toml`, `python/tests/test_hx_*.py`,
`docs/usage/py-hxprobe.md`, the two `docs/plans`/`docs/summaries` entries) is
still uncommitted from this session, and `python/latprobe/cli.py` /
`python/latprobe/probe.py` only diverge from the last commit (`24ea9c9`) by
the reuse-oriented additions made to support hxprobe. So this is a clean,
low-risk restructuring: move already-debugged code, revert latprobe with
`git checkout --`, no git history surgery needed.
## Target layout
```
hxprobe/ (new, top-level, sibling of go/ and python/)
├── pyproject.toml (own manifest: httpx[http2] dependency)
├── hxprobe/
│ ├── __init__.py
│ ├── probe.py (own Options/Phase/Result/VerboseDetail/CertInfo
│ │ + own _parse_cert/_parse_cert_date + existing
│ │ _Trace/_TimingStream/_TimingBackend/
│ │ _TimingTransport/measure() — unchanged logic)
│ ├── aggregate.py (verbatim copy of latprobe/aggregate.py —
│ │ its `from .probe import Result` is already
│ │ package-relative, needs zero edits)
│ ├── duration.py (verbatim copy of latprobe/duration.py — no
│ │ imports at all)
│ ├── cli.py (full standalone CLI — see below)
│ └── __main__.py (unchanged: `from .cli import run`)
└── tests/
├── __init__.py (empty, matches python/tests/__init__.py)
├── test_probe.py (moved from python/tests/test_hx_probe.py)
├── test_cli.py (moved from python/tests/test_hx_cli.py)
└── test_integration.py (moved from python/tests/test_integration_hx.py)
```
`python/` reverts to containing only `latprobe` — zero third-party deps, no
`pyproject.toml`, no venv, exactly its pre-hxprobe state.
## Step-by-step
**1. Move already-debugged files (preserve the bug fixes already made):**
`git mv`/`mv` (untracked, so plain `mv` is fine) `python/hxprobe/{probe.py,__init__.py,__main__.py}`
to `hxprobe/hxprobe/`, and the three `python/tests/test_hx_*.py` /
`test_integration_hx.py` files to `hxprobe/tests/` with the `hx_`/`_hx` name
segments dropped (`test_probe.py`, `test_cli.py`, `test_integration.py`).
Do **not** rewrite these from scratch — they already have the mark_dns /
verbose-on-failure-path / Content-Length-on-keep-alive fixes found during
the original implementation.
**2. Inline the dataclasses into `hxprobe/hxprobe/probe.py`:**
Replace `from latprobe.probe import Options, Phase, Result, VerboseDetail, _parse_cert`
with local definitions copied verbatim from `python/latprobe/probe.py`:
`CertInfo`, `VerboseDetail` (its `http_version`/`redirect_count` fields are
now simply always-meaningful, no more "populated only by hxprobe" caveat
comment needed), `Options` (with `follow_redirects`/`http2` as normal fields,
no more "ignored by socket measure()" caveat), `Phase`, `Result`,
`_parse_cert`, `_parse_cert_date`. Everything else in the file (`_Trace`,
`_TimingStream`, `_TimingBackend`, `_TimingTransport`, `measure()`,
`_classify`, `_unwrap`, `_fill_phases`, `_fill_verbose`) is untouched.
**3. Create `hxprobe/hxprobe/aggregate.py` and `duration.py`:**
Verbatim copies of `python/latprobe/aggregate.py` and `duration.py`.
**4. Write `hxprobe/hxprobe/cli.py` as a full standalone CLI:**
Start from the *current* `python/latprobe/cli.py` (it already has 100% of
the needed logic, including the `--no-http2`/`--no-follow-redirects` flags,
the verbose "Protocol" row, and the JSON `http_version`/`redirect_count`
keys — all added earlier specifically for hxprobe). Strip the
generalization scaffolding that only existed to let `latprobe` share this
code:
- Remove the `MeasureFn` type alias and the `measure_fn` parameter from
`_run_samples`/`run` — call `measure` directly (module-level import from
`.probe`).
- Remove `run()`'s `prog`/`description`/`protocol_flags` parameters —
hardcode `prog="hxprobe"` and the httpx-specific description.
- Make the `--no-http2`/`--no-follow-redirects` `add_argument` calls
unconditional (drop the `if protocol_flags:` guard).
- Everything else (exit codes, `_ArgExit`/`_Parser`, phase-label/verbose
constants, all `_print_*`/`_build_json_entry` rendering) carries over
unchanged — it's already correct standalone logic.
**5. Revert `python/latprobe/` to its pre-hxprobe state:**
`git checkout -- python/latprobe/cli.py python/latprobe/probe.py` (safe:
confirmed these are the only diffs since the last commit, and both diffs
are exactly the reuse scaffolding being removed here).
**6. Clean up the old shared-package artifacts:**
Delete `python/hxprobe/`, `python/pyproject.toml`, `python/.venv/`, and the
three `python/tests/test_hx_*`/`test_integration_hx.py` files (now moved).
**7. Makefile:**
- Remove `PY_VENV`/`PY_VENV_PYTHON` vars and the `py-deps` target; revert
`py-test`/`py-test-integration`/`py-check` to invoke `$(PYTHON)` directly
with no `py-deps` prerequisite (their pre-hxprobe form). Remove `hx-run`/
`hx-test-integration` from the Python section (moving out).
- Add `HX_DIR := hxprobe`, `HX_VENV := $(HX_DIR)/.venv`,
`HX_VENV_PYTHON := $(HX_VENV)/bin/$(PYTHON)` and a new "── hxprobe
(standalone) ──" section: `hx-deps` (idempotent venv+pip install, mirrors
the removed `py-deps`), `hx-run`, `hx-test` (hermetic, `test_[!i]*.py`
glob), `hx-test-integration` (`test_integration.py` pattern), `hx-check`,
`hx-clean`.
- Fold into the umbrella targets: `test: go-test py-test hx-test`,
`check: go-check py-check hx-check`, `clean: go-clean py-clean hx-clean`.
**8. Docs:**
- Rename `docs/usage/py-hxprobe.md``docs/usage/hxprobe.md` (drop the
`py-` prefix — it's no longer part of the Python port). Update: remove the
"reuses `latprobe.cli.run()` in full" claim (now false — say it's a fully
standalone implementation sharing only the *design*, not the code, with
`latprobe`); update setup/Makefile-target sections to `make hx-deps`/
`hx-run`/`hx-test`/`hx-test-integration`; update file paths from
`python/hxprobe/` to `hxprobe/hxprobe/`. Keep the TCP_NODELAY/Nagle finding
and the example transcripts (still accurate — behavior is unchanged, only
location/packaging changed).
- `CHANGELOG.md`: new entry describing the extraction.
- `docs/summaries/`: new dated summary per the CLAUDE.md convention
(leave the original `2026-07-02-00-29-py-hxprobe-httpx.md` as-is — it's a
historical record of that implementation; this is a follow-up).
- `docs/plans/`: save this plan as
`docs/plans/<yyyy-mm-dd-hh-mm>-hxprobe-standalone-project.md` at
implementation start, per CLAUDE.md.
## Verification
1. `rm -rf hxprobe/.venv python/.venv` (fresh state) then `make hx-deps`
creates `hxprobe/.venv`, installs `httpx[http2]` from `hxprobe/pyproject.toml`.
2. `grep -rn "latprobe" hxprobe/` — must return nothing (proves the
decoupling).
3. `make hx-run ARGS="-v http://github.com"` — same output as before (HTTP/2,
1 redirect followed, IP/TLS/cert shown).
4. `make hx-test` — all hermetic hxprobe tests pass standalone.
5. `make hx-test-integration` — live HTTP/2 + redirect tests still pass.
6. `make py-test` — confirms `latprobe`'s own suite is back to its original,
dependency-free form and still green (no `py-deps` needed to run it).
7. `make check` (top-level) — Go + latprobe + hxprobe all green in one gate.
8. `diff <(python -m latprobe --help) <(git show 24ea9c9:python/latprobe/cli.py | ...)`
or simpler: confirm `python -m latprobe --help` output is byte-identical
to before this whole feature existed (no leftover `--no-http2` etc.).

View File

@@ -0,0 +1,115 @@
# Plan: modernize hxprobe's Python toolchain
## Context
`hxprobe` (`hxprobe/`) is a fully standalone Python project (own
`pyproject.toml`, own venv, own tests — see
`docs/summaries/2026-07-02-09-32-hxprobe-standalone-project.md`). Its
toolchain is currently bare-minimum: hand-rolled `python -m venv` + `pip
install -e .`, no linter/formatter, no type checker, no dependency lockfile,
and stdlib `unittest` with a filename-glob convention
(`test_[!i]*.py`/`test_integration.py`) to separate hermetic from live tests.
User confirmed direction (via AskUserQuestion): adopt **uv** for env/deps
(with a lockfile), add **ruff** for lint+format, **skip mypy** for now, and
**swap the test runner to pytest** (existing `unittest.TestCase` classes run
unchanged under pytest — no test-code rewrite) using a proper
`@pytest.mark.integration` marker instead of the filename-glob trick.
Confirmed via inspection: `uv` (v0.11.19) is already installed globally
(Homebrew); `ruff`/`mypy`/`pytest` only exist under an unrelated pyenv 3.12.3
shim, not on the 3.14 interpreter this project targets — `uv` sidesteps that
mismatch by installing everything into the project's own venv. Git remote is
a self-hosted Gitea instance, not GitHub, so no CI is being set up here.
## Changes
**`hxprobe/pyproject.toml`:**
- Add `[dependency-groups]` with `dev = ["pytest>=8.0", "ruff>=0.8"]` (PEP
735, the current uv-native way to declare dev-only deps — keeps the
install-as-a-library `dependencies` list clean).
- Add `[tool.pytest.ini_options]`: `testpaths = ["tests"]` and a registered
`integration` marker (avoids `PytestUnknownMarkWarning`).
- Add `[tool.ruff]`: `target-version = "py311"` (matches `requires-python`)
and `line-length = 100` (close to the codebase's existing longest lines,
~103 chars, to minimize reformatting churn).
- Leave `[build-system]`/`[tool.setuptools]` untouched — build backend
wasn't part of the discussion, setuptools works fine here.
**`hxprobe/.python-version`:** new file, `3.14`, so `uv sync`/`uv run` pin the
same interpreter the rest of the repo uses without relying on `$PATH` order.
**`hxprobe/uv.lock`:** generated by `uv lock` — first real lockfile pinning
`httpx`, `httpcore`, `h2`, `certifi`, and friends. Committed (not gitignored;
lockfiles belong in version control).
**Test changes (runner swap only, no rewrite):**
- `hxprobe/tests/test_integration.py`: add `pytestmark = pytest.mark.integration`
at module level. This replaces the "named `test_integration.py` so the
`test_[!i]*.py` glob skips it" convention — selection becomes `-m
"not integration"` / `-m integration`, independent of filename.
- `test_probe.py`/`test_cli.py`: no changes — they're already hermetic
(local `http.server` fixtures only) and need no marker.
- `unittest.TestCase` classes, `_NEEDS_NET = unittest.skipUnless(...)`, and
the `if __name__ == "__main__": unittest.main()` guards all stay exactly
as-is — pytest natively discovers and runs unittest-style tests and
respects `unittest.skip*` decorators with zero changes required.
**Lint fixes:** run `ruff check --fix` / `ruff format` over `hxprobe/` and
review the diff. One known pre-existing issue it will flag: `import sys` in
`cli.py` is unused (inherited from the original `latprobe/cli.py`) — remove
it. Otherwise expect mostly whitespace/quote-style normalization.
**`Makefile`:** replace the `hx-*` section to run through `uv` instead of a
hand-managed venv:
```makefile
HX_DIR := hxprobe # (drop HX_VENV / HX_VENV_PYTHON — uv owns this now)
hx-deps: cd $(HX_DIR) && uv sync
hx-run: (deps: hx-deps) cd $(HX_DIR) && uv run python -m hxprobe $(ARGS)
hx-lint: (deps: hx-deps) cd $(HX_DIR) && uv run ruff check .
hx-fmt: (deps: hx-deps) cd $(HX_DIR) && uv run ruff format .
hx-test: (deps: hx-deps) cd $(HX_DIR) && uv run pytest tests -m "not integration" -v $(ARGS)
hx-test-integration: (deps: hx-deps) cd $(HX_DIR) && uv run pytest tests -m integration -v $(ARGS)
hx-check: hx-lint hx-test (test gate now includes lint, mirroring go-check's fmt+vet+test bundling)
hx-clean: also removes .pytest_cache / .ruff_cache alongside __pycache__/egg-info
```
Umbrella `test`/`check`/`clean` targets keep delegating to `hx-test`/
`hx-check`/`hx-clean` unchanged.
**`hxprobe/README.md`:** new, minimal — since this project is meant to be
`cp -r`-able to its own repo, it should carry its own quick-start
(`uv sync`, `uv run python -m hxprobe <url>`, `uv run pytest`) rather than
relying on the monorepo's root docs.
**Docs:** update `docs/usage/hxprobe.md`'s "Setup" section (`uv sync`
instead of manual venv+pip) and "Makefile targets" section (add
`hx-lint`/`hx-fmt`, update test invocation description). New
`docs/plans/<timestamp>-hxprobe-toolchain-modernization.md` and
`docs/summaries/<timestamp>-hxprobe-toolchain-modernization.md` per
CLAUDE.md convention. New `CHANGELOG.md` entry.
**`.gitignore`:** already covers `.venv/`/`__pycache__/`/`*.egg-info/`
unanchored; add `.pytest_cache/` and `.ruff_cache/` (new caches these tools
create).
## Verification
1. `cd hxprobe && uv sync` — creates `.venv`, generates/uses `uv.lock`,
installs `httpx[http2]` + dev deps (`pytest`, `ruff`).
2. `make hx-run ARGS="-v http://github.com"` — same output as before
(HTTP/2, 1 redirect, IP/TLS/cert shown) — confirms the runtime behavior
is untouched by the toolchain swap.
3. `make hx-lint` — clean (after fixing whatever `ruff check` surfaces,
including the unused `import sys`).
4. `make hx-test` — all hermetic tests pass under pytest; confirm the
`integration`-marked tests are excluded (test count matches the current
28 hermetic tests).
5. `make hx-test-integration` — the 9 live tests run and pass under the
`integration` marker selection.
6. `make check` (top-level) — Go + latprobe + hxprobe (lint + test) all
green in one gate.
7. `grep -rn "latprobe" hxprobe/` — still zero matches (toolchain change
must not reintroduce coupling).
8. Confirm `python/latprobe/` is untouched (`git diff --stat python/` empty)
— this is a hxprobe-only change.

View File

@@ -0,0 +1,112 @@
# Plan: hxprobe-specific usage doc (case-by-case) + hxprobe-specific Makefile
## Context
`hxprobe` is a fully standalone project (own `pyproject.toml`, `uv.lock`,
`README.md` — see `docs/summaries/2026-07-02-09-32-hxprobe-standalone-project.md`
and `...-toolchain-modernization.md`). Two gaps remain versus the other
Python implementations and versus hxprobe's own "could be `cp -r`'d to its
own repo" design goal:
1. **Docs**: `latprobe`/`phases.py`/`simple.py` each have two usage docs —
the CLAUDE.md-mandated `docs/usage/py-<name>.md` (what/flags/one example)
_and_ a much richer `python/configs/usage-<name>.md` "Runnable Usage
Reference" walking through a full set of concrete cases with real
captured output (confirmed by reading `python/configs/usage-latprobe.md`,
313 lines: basic, verbose × 4 variants, sampling, multi-URL, `--fail`,
JSON × 2, timeout, exit-codes table, Makefile shortcuts). `hxprobe` only
has the first kind. User confirmed: add the second kind, placed _inside_
`hxprobe/` itself (not under `python/configs/`, since hxprobe no longer
lives there) so the doc travels with the project if extracted.
2. **Makefile**: hxprobe currently has no `Makefile` of its own — the only
way to run/test/lint it is through the parent repo's root `Makefile`.
Extracted to its own repo, there'd be no `make` interface left. User
confirmed: add `hxprobe/Makefile`, fully independent from the root
Makefile's existing `hx-*` targets (no delegation either direction —
both keep their own complete logic, at the cost of some duplication).
Confirmed: neither `latprobe` nor `hxprobe` accept a config file (both take
URLs as positional CLI args — only `simple.py`/`phases.py` read the
`python/configs/*.txt` files), so no `.txt`-config-file equivalent is needed
for hxprobe; this is a docs+Makefile-only task.
## Changes
**`hxprobe/USAGE.md`** (new) — modeled directly on
`python/configs/usage-latprobe.md`'s structure and tone (concrete `sh`
command blocks immediately followed by real captured output, real IPs/certs/
timings, brief explanatory notes, `---` section separators). Cases, in order:
1. Basic — single URL
2. Verbose — HTTPS site (shows `Protocol: HTTP/2`, TLS, cert)
3. Verbose — plain HTTP (no TLS block)
4. Verbose — redirect followed by default (`http://github.com` → 200,
`Protocol: HTTP/2 (1 redirect)`) — **hxprobe-specific**, latprobe has no
equivalent
5. `--no-follow-redirects` — same URL, raw `301` instead — **hxprobe-specific**
6. `--no-http2` — forces `Protocol: HTTP/1.1`**hxprobe-specific**
7. Verbose — TLS failure (expired cert, badssl.com)
8. Verbose — DNS failure (empty verbose block, suppressed)
9. Sampling (`-n`) — min/avg/max table, plus verbose+sampling
10. Multiple URLs (parallel probing)
11. `--fail` flag — exit 6 on HTTP 4xx
12. JSON output
13. JSON + verbose (includes `http_version`/`redirect_count` keys)
14. Timeout
15. Exit codes table (same 06 scheme as `latprobe`/Go)
16. Makefile shortcuts — both the new `hxprobe/Makefile` (`make run`,
`make test`, etc., run from inside `hxprobe/`) and the parent repo's
root shortcuts (`make hx-run`, run from the repo root)
Cases 1, 2, 3, 4, 5, 6, 8, 11 can reuse real output already captured earlier
in this session (still accurate — no code changed since). Cases 7, 9, 10,
12, 13, 14 need fresh live runs during implementation to get real numbers
(same standard the other `usage-*.md` docs hold themselves to — no
fabricated timings).
**`docs/usage/hxprobe.md`** (edit) — add a one-line pointer near the top:
"For a full case-by-case runnable reference, see `hxprobe/USAGE.md`." No
other changes; it stays the CLAUDE.md-mandated summary doc.
**`hxprobe/Makefile`** (new) — fully self-sufficient, same auto-generated
`## comment` help style as the root Makefile, short target names (no `hx-`
prefix needed since it's already scoped by being inside `hxprobe/`):
```makefile
PYTHON ?= python3.14
ARGS ?=
.DEFAULT_GOAL := help
help # auto-generated from ## comments, same style as root Makefile
deps # uv sync
run # uv run python -m hxprobe $(ARGS) (deps: deps)
lint # uv run ruff check . (deps: deps)
fmt # uv run ruff format . (deps: deps)
test # uv run pytest tests -m "not integration" -v $(ARGS) (deps: deps)
test-integration # uv run pytest tests -m integration -v $(ARGS) (deps: deps)
check # lint + test
clean # remove __pycache__/*.pyc/*.egg-info/.pytest_cache/.ruff_cache
```
No changes to the root `Makefile` — its existing `hx-*` targets are left
exactly as-is per the "keep both independent" decision.
**Docs housekeeping** (per CLAUDE.md convention): save this plan to
`docs/plans/<timestamp>-hxprobe-usage-doc-and-makefile.md`, write a summary
to `docs/summaries/<timestamp>-hxprobe-usage-doc-and-makefile.md` after
implementation, and append a `CHANGELOG.md` entry.
## Verification
1. `cd hxprobe && make help` — lists all targets with descriptions, works
with zero dependency on the parent repo's Makefile.
2. `cd hxprobe && make run ARGS="-v https://example.com"` — same output as
`make hx-run ARGS="-v https://example.com"` from the repo root (proves
the two Makefiles agree, without one calling the other).
3. `cd hxprobe && make check` — lint + hermetic tests pass (28 tests).
4. `cd hxprobe && make test-integration` — 9 live tests pass.
5. Re-run every command block in `hxprobe/USAGE.md` and confirm the
captured output matches what's printed in the doc (structure must be
stable even if exact millisecond timings drift).
6. Confirm the root `Makefile`'s `hx-*` targets are byte-for-byte unchanged
(`git diff Makefile` shows no `hx-*` section changes from this task).

View File

@@ -0,0 +1,102 @@
# Plan: read target URLs from a file for hxprobe
## Context
`hxprobe` currently only accepts URLs as positional CLI arguments
(`hxprobe/hxprobe/cli.py:368`, `nargs="+"`). The user wants a file-based
input mode too, matching the existing convention `simple.py`/`phases.py`
already use (`python/simple.py:20-30`'s `load_sites()`: plain text, one URL
per line, `#`-comments and blank lines skipped, first whitespace-separated
token taken per line).
User-confirmed decisions:
- **Mutually exclusive** with positional URL args (either pass URLs on the
command line, or `-f FILE`, never both).
- **hxprobe only** — `latprobe` is intentionally left untouched.
- **Add example fixture files** (`hxprobe/configs/*.txt`, mirroring
`python/configs/*.txt`'s exact set: all-ok, dns-failure,
connection-refused, timeout, tls-errors, http-errors, mixed) plus one new
section in the existing `hxprobe/USAGE.md` demonstrating the flag.
## Implementation
**`hxprobe/hxprobe/cli.py`:**
- `urls` positional becomes `nargs="*"` (was `nargs="+"`) — no longer
required on its own, since `-f` is now a second valid source.
- New flag: `-f, --file PATH` — "read URLs from a file, one per line,
`#` comments allowed (mutually exclusive with positional url args)".
Placed right after the `urls` positional definition in the argparse
block, since the two are the two ways of specifying what to probe.
- New helper `_load_urls(path: str) -> list[str]`, duplicating (not
importing) `simple.py`'s `load_sites()` logic — consistent with hxprobe's
established "imports nothing outside its own directory" rule from the
standalone-extraction work.
- After `parser.parse_args()`, manual validation (mirrors the existing
`--timeout` invalid-value handling style — write to the injected
`stderr`, `return EXIT_USAGE`, rather than routing through
`argparse`'s mutually-exclusive-group machinery, which doesn't mix
cleanly with a variadic positional):
- both `ns.urls` and `ns.file` given → `parser.error(...)` (usage error,
consistent with how `_Parser.error()` already handles bad usage)
- neither given → `parser.error(...)`
- `ns.file` given but unreadable (`FileNotFoundError`/`OSError`) →
`stderr.write(...)`; `return EXIT_USAGE`
- `ns.file` given but yields zero URLs → same treatment
- otherwise `urls = _load_urls(ns.file)` or `urls = ns.urls`
**`hxprobe/tests/test_cli.py`:** new hermetic tests — successful multi-URL
run from a file, missing-file error, empty-file error, and the
both-sources-given usage error. Uses a temp file (`tempfile`), no network
needed for the parsing-error cases.
**`hxprobe/configs/*.txt`** (new directory) — same 7 fixtures as
`python/configs/`, adapted:
- `all-ok.txt`, `dns-failure.txt`, `connection-refused.txt`,
`tls-errors.txt` — same URLs, same behavior (DNS/TCP/TLS failures are
identical regardless of HTTP client sophistication); only the header
comments change (`hxprobe -f configs/<name>.txt` instead of
`python3.14 python/simple.py ...`).
- `timeout.txt` — same two targets (`10.255.255.1`, `192.0.2.1`, RFC 5737
TEST-NET-1) as the original; "expected exit code" documents normal-network
behavior (exit 4), same caveat the original file already carries about
network-dependent behavior.
- `http-errors.txt` — same 404 URLs; header comment updated to show the
demo command with `--fail` (hxprobe treats 4xx as success without
`--fail`, unlike `simple.py`, which always raises on HTTPError) —
"expected exit code" becomes 6, not `simple.py`'s blanket 1.
- `mixed.txt` — same mixed set; demo command includes `--fail`; expected
exit code recalculated as the worst code across the included classes
(dns=2, connect=3, tls=5, http=6 with `--fail`) → 6.
- Each header's "Expected exit code" will be verified by actually running
the fixture through `hxprobe -f ...` during implementation, not assumed
from the `simple.py` originals — hxprobe's worst-code-wins exit scheme
(`hxprobe/hxprobe/cli.py:455-481`) differs fundamentally from
`simple.py`'s blanket 0/1.
**`hxprobe/USAGE.md`:** new section "Reading URLs from a file (`-f`)",
placed after the "Multiple URLs" case (same family of "what to probe"
examples) — command + real captured output using `configs/all-ok.txt` or
`configs/mixed.txt`, plus a short list of the other fixture files available
and what each demonstrates.
**Docs housekeeping** (per CLAUDE.md convention): save this plan under
`docs/plans/`, write a summary under `docs/summaries/` after implementation,
append a `CHANGELOG.md` entry.
## Verification
1. `cd hxprobe && uv run python -m hxprobe -f configs/all-ok.txt` — probes
all 3 URLs, exit 0.
2. `cd hxprobe && uv run python -m hxprobe -f configs/dns-failure.txt`
exit 2; `connection-refused.txt` → exit 3; `tls-errors.txt` → exit 5;
`--fail -f configs/http-errors.txt` → exit 6; `--fail -f configs/mixed.txt`
→ exit 6 (confirms the worst-code documented in each header is accurate).
3. `uv run python -m hxprobe https://example.com -f configs/all-ok.txt`
usage error (both sources given).
4. `uv run python -m hxprobe -f /no/such/file` — usage error, clear message.
5. `cd hxprobe && make check` — new hermetic tests pass alongside the
existing 28.
6. Re-run the new `hxprobe/USAGE.md` section's command and confirm captured
output matches what's printed in the doc.
7. `grep -n "import" hxprobe/hxprobe/cli.py` — confirm no new import from
`python/simple.py` or anywhere outside `hxprobe/`.

View File

@@ -0,0 +1,174 @@
# hxprobe simplification plan
## Context
The `hxprobe` Python package works and is well-tested, but it carries several
pieces of **speculative scaffolding** — generality added for things that "might
happen later" — plus some **redundant state** and **duplicated logic**. The
user's explicit directive: keep the code simple, with no unnecessary
abstractions or scaffolding for hypothetical future features.
This plan removes that overhead without changing any observable behavior. All
existing tests in `tests/test_cli.py` and `tests/test_probe.py` must continue
to pass unchanged (they are the behavioral contract). Net effect: ~4050 fewer
lines, fewer moving parts, no new abstractions.
The trigger was a question about the `_Parser._print_message`/`print_help`/
`print_usage` overrides, whose `file=` parameter is accepted but never used —
the classic "kept for a future that never came" smell. Investigation showed the
whole `_Parser` subclass re-implements behavior the standard library already
provides.
---
## Changes
### 1. Delete the `_Parser` subclass — use stdlib stream redirection
**File:** `hxprobe/hxprobe/cli.py` (lines 6094, and the `run()` wiring ~375455)
The `_Parser` class + `_ArgExit` exception (~32 lines) exist only to (a) route
argparse's help/usage/error output to the injected `stdout`/`stderr` streams and
(b) raise instead of calling `sys.exit()`. The standard library already does
both:
- `argparse.ArgumentParser` writes help to `sys.stdout` and usage/errors to
`sys.stderr` by default, and already raises `SystemExit` (not a hard exit) —
so it is already testable.
- `contextlib.redirect_stdout(stdout)` / `redirect_stderr(stderr)` patch the
streams argparse writes to.
**Do:**
- Remove `class _ArgExit`, `class _Parser`, and all four overrides
(`_print_message`, `print_help`, `print_usage`, `error`, `exit`).
- In `run()`, build a plain `argparse.ArgumentParser(prog="hxprobe", ...)` (drop
the `out=`/`err=` kwargs).
- Wrap the parse + the two manual validations in redirection and catch
`SystemExit`:
```python
import contextlib
...
try:
with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr):
ns = parser.parse_args(args)
if ns.urls and ns.file:
parser.error("cannot combine positional url arguments with -f/--file")
if not ns.urls and not ns.file:
parser.error("no URLs given (pass as arguments or with -f/--file)")
except SystemExit as exc:
return EXIT_OK if not exc.code else EXIT_USAGE
```
**Behavior parity (verified against tests):**
- `-h` → argparse prints help to `stdout`, raises `SystemExit(0)` → returns
`EXIT_OK`. (`test_help_shows_hxprobe_prog_name`, `test_help_lists_protocol_flags`)
- `parser.error(...)` → prints `prog: error: msg` + usage to `stderr`, raises
`SystemExit(2)` → mapped to `EXIT_USAGE`. (`test_*_is_usage_error`)
- Note: stdlib `error()` exits with code 2; we normalise any non-zero parse exit
to `EXIT_USAGE` (1) at the single catch site, replacing the per-method code
baked into the old override.
Only the parse/validate block needs redirection; the rest of `run()` keeps
writing directly to the passed `stdout`/`stderr`.
### 2. Trim `_TimingStream.get_extra_info` to the branch that's actually used
**File:** `hxprobe/hxprobe/probe.py` (lines 262275)
`server_addr` and `client_addr` are only ever queried by `httpx/_main.py` (the
`httpx` CLI command), never by the request path hxprobe drives — verified by
grepping the installed `httpcore`/`httpx`. The only branch httpcore's sync
connection path calls is `ssl_object` (and `is_readable`, which we intentionally
leave unhandled → `None`).
**Do:** reduce the method to:
```python
def get_extra_info(self, info: str):
if info == "ssl_object" and isinstance(self._sock, ssl.SSLSocket):
return self._sock
return None
```
### 3. Drop the `_load_urls` "future annotations" scaffolding
**File:** `hxprobe/hxprobe/cli.py` (lines 358369)
The `line.split()[0]` + docstring ("forward-compatible with future
'url key=value' annotations") is scaffolding for a feature that doesn't exist.
Use the stripped line directly and simplify the docstring to describe what it
actually does (one URL per line, `#` comments and blank lines skipped).
`test_reads_urls_from_file` (comments + blanks) still passes.
### 4. Remove redundant `_set` flags in `_Trace`
**File:** `hxprobe/hxprobe/probe.py` (lines 126191)
`_dns_set`, `_connect_set`, `_tls_set` duplicate information already carried by
`Phase.present` on the corresponding `self.dns` / `self.connect` / `self.tls`.
The initial `Phase()` has `present=False`, so the first-hop-wins guard is
identical.
**Do:** delete the three boolean fields; replace each guard, e.g.
`if not self._dns_set:``if not self.dns.present:` (same for connect/tls).
Keeps the redirect first-hop-wins semantics documented at probe.py:114123.
### 5. De-duplicate the failure counting logic
**File:** `hxprobe/hxprobe/cli.py`
The identical "dedupe failures into ordered (phase, message, count)" loop appears
twice: `_print_failure_summary` (lines 242258) and `_build_json_entry`
(lines 314325). Extract one helper next to the other rendering helpers:
```python
def _summarize_failures(failed: list[Result]) -> list[tuple[str, str, int]]:
counts: dict[tuple[str, str], int] = {}
order: list[tuple[str, str]] = []
for r in failed:
key = (r.fail_phase, str(r.err))
if key not in counts:
order.append(key)
counts[key] = 0
counts[key] += 1
return [(ph, msg, counts[(ph, msg)]) for ph, msg in order]
```
Rewrite both call sites to consume it (text side keeps the `1 ×` vs `N ×`
formatting; JSON side maps to `{"phase", "count", "message"}`).
---
## Explicitly NOT changing (considered, kept)
- **The custom httpcore backend** (`_TimingBackend`/`_TimingStream`/
`_TimingTransport`) — this *is* the tool's reason to exist (splitting DNS/TCP,
timing TLS). Not scaffolding.
- **`ThreadPoolExecutor` concurrency** — backs the shipped `-c/--concurrency`
and multi-URL/`-f` features. Real, not speculative.
- **Explicit per-phase dataclass fields** in `probe.py`/`aggregate.py` — a loop
would be shorter but less readable; explicit is clearer and matches the text/
JSON renderers. Leave as-is.
- **`CertInfo.sans`** — not shown in the text block but *is* emitted in JSON
verbose output; it's a real feature, not dead.
---
## Verification
1. **Unit/CLI tests (primary contract):**
```
cd hxprobe && .venv/bin/python -m pytest tests/test_cli.py tests/test_probe.py -q
```
All must pass with no edits to the test files. These already cover: `-h`
help→stdout + exit 0, the four usage errors→stderr + exit 1, DNS/connect/
timeout failures, `--fail`, `--json`, redirects, `--no-http2`,
`--no-follow-redirects`, `-f` file input, and verbose/cert/TLS detail.
2. **Lint:** `cd hxprobe && .venv/bin/ruff check hxprobe/`
3. **Manual smoke (help + error routing, since #1 rewrites that path):**
```
.venv/bin/python -m hxprobe -h # help on stdout, exit 0
.venv/bin/python -m hxprobe # "no URLs given" on stderr, exit 1
.venv/bin/python -m hxprobe https://example.com -v # phases + verbose block
```
4. **Per-CLAUDE.md project conventions:** after implementing, add a
`docs/summaries/<yyyy-mm-dd-hh-mm>-hxprobe-simplification.md` summary and a
`CHANGELOG.md` entry. (No `docs/usage/` change — behavior is unchanged.)

View File

@@ -0,0 +1,136 @@
# hxprobe: end-of-run summary footer for multi-URL runs
## Context
A single scalar exit code is inherently lossy when probing multiple URLs with
different failure classes: today `run()` returns the numeric max across all
URLs (e.g. DNS-fail on one URL + TLS-fail on another → exit `5`, and the DNS
failure is invisible in the code). The failures *are* all printed per-URL, but
there's no consolidated view — for many URLs you must scroll and eyeball each
block to know what happened overall.
The exit code's job is a coarse pass/fail + severity hint for scripts, and it's
a **deliberate cross-implementation contract** shared with `latprobe` and the
Go version (documented in `hxprobe/USAGE.md:475-488`, asserted by 13 tests in
`tests/test_cli.py`). So we keep the worst-code exit unchanged and instead give
humans the full picture the scalar can't: an **end-of-run summary footer** that
tallies every URL's outcome and shows how the exit code was derived.
Decisions (confirmed with the user):
- Exit code: **unchanged** (worst/highest severity across URLs).
- Summary: **text footer, multi-URL only** (`len(urls) > 1`). Single-URL text
output stays byte-identical; JSON output stays a bare array (unchanged).
## Design
### Per-URL classification (reuses existing helpers)
Each URL gets one "worst outcome code" using the *existing* mapping — no new
severity scheme:
- start at `EXIT_OK`;
- for each failed sample: `code = max(code, _phase_code(r.fail_phase))`
(`_phase_code` at `cli.py:31`);
- if `--fail`: for each succeeded sample with `status_code >= 400`:
`code = max(code, EXIT_HTTP)`.
A URL is "ok" iff its code is `EXIT_OK`, else "failed" and bucketed by its code.
A partially-failed URL (some samples ok, some failed) classifies by its worst
sample — consistent with how its own block and the global exit code already
behave.
### Footer format (text, only when `len(urls) > 1`)
Printed once after the last URL block, before `return worst`. Failure classes
use the same `✗` bullet style as `_print_failure_summary` (`cli.py:219-224`).
The final `→ exit N (label)` line explicitly ties the tally to the returned
code — directly answering "why is the exit code what it is". Example (3 URLs):
```
https://example.com (200)
... phase rows ...
http://no.such.host.invalid (FAILED)
✗ dns: [Errno 8] nodename nor servname provided
https://self-signed.badssl.com (FAILED)
✗ tls: certificate verify failed
─────────────────────────────────────────────────
Summary: 3 URLs — 1 ok, 2 failed
✗ dns : 1
✗ tls : 1
→ exit 5 (tls)
```
All-ok multi-URL run → `Summary: 3 URLs — 3 ok`, no `✗` lines, `→ exit 0 (ok)`.
## Changes
All in `hxprobe/hxprobe/cli.py` unless noted.
1. **Reverse label map** next to the exit-code constants (`cli.py:16-32`): a
small `_EXIT_LABELS: dict[int, str]` mapping `EXIT_DNS→"dns"`,
`EXIT_CONNECT→"connect"`, `EXIT_TIMEOUT→"timeout"`, `EXIT_TLS→"tls"`,
`EXIT_HTTP→"http"`, `EXIT_OK→"ok"`. (Inverse of the existing forward mapping;
kept explicit for readability, matching the codebase's style.)
2. **Refactor the accumulation loop** (`cli.py:454-462`) to compute a per-URL
code and fold it into `worst`, collecting `url_codes: list[int]` (one per
URL, index-aligned with `urls`). This also unifies the two "running max"
idioms flagged in
`docs/explanations/2026-07-02-13-25-hxprobe-worst-exit-code-and-render-loop.md`
(`if c > worst` vs `max(...)`) into one — a small simplification bonus.
3. **New `_print_run_summary(urls, url_codes, worst, out)` helper** (near the
other `_print_*` renderers): builds the ok/failed tally + per-class counts
from `url_codes` and writes the footer. No-op guard is the caller's
`len(urls) > 1` check.
4. **Call site** after the loop (`cli.py:470-475`): in text mode only
(`if not ns.json_out and len(urls) > 1:`), call `_print_run_summary(...)`
before `return worst`. JSON path untouched — still `json.dumps(json_items)`
as a bare array.
## Tests (`hxprobe/tests/test_cli.py`)
Add hermetic tests (reuse the existing `_OKHandler`/`_start_server`/`_invoke`
harness and `_free_port` for a refused connection):
- **multi-URL mixed** — one OK server URL + one `http://127.0.0.1:<free>`
(connection refused): assert `code == EXIT_CONNECT`, and the footer strings
are present (`"Summary: 2 URLs"`, `"1 ok"`, `"1 failed"`, `"connect : 1"`,
`"→ exit 3"`).
- **multi-URL all ok** — two OK URLs: assert `code == EXIT_OK` and
`"Summary: 2 URLs — 2 ok"` present; verify no `"✗"` in the footer region.
- **single URL has NO footer** — one OK URL: assert `"Summary:"` NOT in `out`
(locks the multi-URL-only rule).
Backward-compat guard: the footer contains no `"200"` substring, so the
existing `test_reads_urls_from_file` assertion `out.count("200") == 2` still
holds; all 13 existing exit-code tests are unaffected (worst-code unchanged).
## Docs / project conventions (per CLAUDE.md)
- **Copy this approved plan** into
`docs/plans/<yyyy-mm-dd-hh-mm>-hxprobe-run-summary-footer.md` as the first
implementation step (before code) — see the `feedback_plan_mode_docs_plans`
memory note.
- `hxprobe/USAGE.md`: add a short "Multi-URL summary footer" subsection with a
real captured example; add a sentence to the Exit-codes section noting the
footer shows the per-class breakdown behind the scalar. Exit-code table
itself is unchanged.
- `docs/usage/hxprobe.md`: one-line mention if it lists features.
- New `docs/explanations/<ts>-hxprobe-run-summary-footer.md` is optional; the
existing worst-exit-code explanation can get a short "Update:" pointer.
- `CHANGELOG.md`: new timestamped entry.
- `docs/summaries/<ts>-hxprobe-run-summary-footer.md`: implementation summary.
## Verification
1. `cd hxprobe && .venv/bin/python -m pytest tests/test_cli.py tests/test_probe.py -q`
— all existing + new tests pass.
2. `.venv/bin/ruff check hxprobe/` — clean.
3. Manual, capturing exit codes:
```
.venv/bin/python -m hxprobe https://example.com https://example.org # footer, exit 0
.venv/bin/python -m hxprobe https://example.com http://no.such.host.invalid; echo $? # footer w/ dns:1, exit 2
.venv/bin/python -m hxprobe https://example.com # single URL: NO footer, unchanged
.venv/bin/python -m hxprobe --json https://example.com https://example.org # bare JSON array, NO footer
```

View File

@@ -0,0 +1,139 @@
# latprobe (Python) — code walkthrough
A guided tour of `python/latprobe/` for anyone about to edit the code by hand.
For CLI usage/flags, see [docs/usage/py-latprobe.md](usage/py-latprobe.md).
## Layout
```
python/
├── simple.py step 1 (standalone) — reachability + total elapsed time
├── phases.py step 2 (standalone) — per-phase timing, single file
├── sites.txt sample config for simple.py / phases.py
├── configs/ fixture configs (all-ok, dns-failure, timeout, tls-errors, ...)
├── latprobe/ step 3 — the real package
│ ├── __main__.py
│ ├── probe.py
│ ├── aggregate.py
│ ├── duration.py
│ └── cli.py
└── tests/
├── test_probe.py
├── test_cli.py
└── test_integration.py
```
`simple.py` and `phases.py` are earlier milestones, kept as reference (mirrors
the Go step-by-step convention: simple → phases → full package). They are not
imported by the package.
## The package: `python/latprobe/`
### `probe.py` — the engine
No argparse, no printing. `measure(url, opts) -> Result` does one HTTP(S)
request over raw sockets (not `requests`/`urllib`) so it can time each phase
itself:
- DNS (`socket.getaddrinfo`)
- TCP connect
- TLS handshake (`ssl.wrap_socket`, HTTPS only)
- TTFB (write request → first byte, reading until `\r\n\r\n`)
- Transfer (drain the rest of the body)
Each phase is timed with `time.perf_counter()` and wrapped by the `_p()`
helper into a `Phase(ms, present)`.
Every failure path (DNS error, connect timeout, TLS error, etc.) returns
early with a **partially-filled** `Result``fail_phase` says where it died,
`err` holds the exception, and whatever phases completed before the failure
are preserved. This is why the CLI can still print e.g. "DNS: 12ms" even if
TCP connect then failed.
`Options(timeout, verbose)` goes in, `Result` comes out. When `verbose=True`,
`Result.detail` (`VerboseDetail`) is also filled in: resolved IP, TLS
version/cipher, parsed certificate (`CertInfo`), and all response headers.
This is the only file with actual measurement logic — change *how* something
is measured here (e.g. add a redirect-follow phase, redefine what counts as
TTFB).
### `aggregate.py` — pure math, no I/O
`summarize(results: list[Result]) -> Aggregate` takes a list of successful
`Result`s (from repeated `-n` sampling of the same URL) and computes
min/avg/max per phase into `PhaseStats`. `Aggregate.status_code` uses the
*last* result's status. Small and self-contained — safe to extend (e.g. add
p50/p95) without touching anything else.
### `duration.py`
One function: `parse_duration("500ms" | "10s" | "2m") -> float` seconds, used
for `--timeout`.
### `cli.py` — everything else
Argument parsing, orchestration, and both text/JSON rendering. The file
you'll touch most for UX changes.
- `_Parser` subclasses `argparse.ArgumentParser` to redirect all output
through injected `stdout`/`stderr` streams and raise `_ArgExit` instead of
calling `sys.exit` — this is what makes `run()` fully testable without
subprocess (tests just pass in `io.StringIO()`).
- Exit codes (`EXIT_DNS=2`, `EXIT_CONNECT=3`, etc.) are commented as
mirroring the Go version. `_phase_code()` maps a `fail_phase` string to the
matching code, and `run()` tracks the *worst* code across all URLs/samples.
- `run(args, stdout, stderr) -> int` is the entry point:
1. parse args → build `Options`
2. run `_run_samples()` per URL concurrently via `ThreadPoolExecutor`
(`--concurrency`, defaulting to `min(len(urls), 8)`)
3. for each URL, pick one of four print paths in `_print_url`
(`_print_single` / `_print_aggregate` / `_print_all_failed`), based on
success/failure counts and whether `-n` > 1
4. or, if `--json`, build dict entries via `_build_json_entry` and dump
them all at the end.
- Verbose rendering (`_print_verbose_block`) is shared between the single
and aggregate text paths; JSON verbose data is built separately in
`_build_json_entry`.
### `__main__.py`
Trivial shim: `sys.exit(run(sys.argv[1:], sys.stdout, sys.stderr))`, letting
you run `python -m latprobe <url>`.
## Tests (`python/tests/`)
- `test_probe.py` — unit tests against `measure()` directly (against the
`configs/*.txt` failure-mode fixtures: DNS failure, connection refused,
TLS errors, timeouts).
- `test_cli.py` — drives `run()` with injected `io.StringIO` streams,
checking text/JSON output and exit codes.
- `test_integration.py` — end-to-end, against the `configs/*.txt` files
(`all-ok.txt`, `mixed.txt`, `http-errors.txt`, etc.).
## How it connects, end to end
```
__main__.py
→ cli.run()
→ parses flags
→ for each URL: probe.measure() (× count, across a thread pool)
→ collects Result objects
→ if count > 1: aggregate.summarize() → Aggregate
→ cli.py's _print_* / _build_json_entry render text or JSON
→ returns the worst exit code seen
```
## Where to make changes
| Change you want | File |
|-------------------------------------------|------------------|
| Timing/measurement behavior | `probe.py` |
| min/avg/max or new stats | `aggregate.py` |
| Flags, output formatting, exit-code logic | `cli.py` |
| Duration string parsing | `duration.py` |
The dataclasses (`Result`, `Phase`, `Options`, `VerboseDetail`, `Aggregate`,
`PhaseStats`) are the contracts between these files — adding a field to
`Result` in `probe.py` typically means threading it through `aggregate.py`
(if it should be averaged) and `cli.py` (if it should be printed/JSON-encoded).

View File

@@ -0,0 +1,100 @@
# Summary: `hxprobe` — httpx-based Python probe
Plan: [docs/plans/2026-07-01-23-47-py-hxprobe-httpx.md](../plans/2026-07-01-23-47-py-hxprobe-httpx.md)
## What was built
A new sibling package, `python/hxprobe/`, alongside the existing raw-socket
`latprobe`. It uses `httpx` so the client matches Go's `http.DefaultClient`:
HTTP/2 negotiated via ALPN, redirects followed by default, connection
pooling, and default TLS verification — while still reporting the full
six-phase breakdown (DNS, TCP connect, TLS, TTFB, Transfer, Total).
Files:
- `python/pyproject.toml` — first third-party dependency in this repo
(`httpx[http2]`)
- `python/hxprobe/probe.py` — the core: `_Trace`, `_TimingStream`,
`_TimingBackend`, `_TimingTransport`, `measure()`
- `python/hxprobe/cli.py`, `__main__.py`, `__init__.py` — thin wrappers
- `python/latprobe/probe.py` (edited) — added `Options.follow_redirects`/
`Options.http2` (ignored by the socket `measure()`) and
`VerboseDetail.http_version`/`redirect_count` (always `""`/`0` there)
- `python/latprobe/cli.py` (edited) — `run()`/`_run_samples()` take an
injectable `measure_fn`, plus `prog`/`description`/`protocol_flags`
overrides, so `hxprobe.cli.run()` reuses the entire argparse/concurrency/
exit-code/rendering pipeline unchanged
- `python/tests/test_hx_probe.py` (17 tests), `test_hx_cli.py` (11 tests),
`test_integration_hx.py` (9 live tests, excluded from the default gate via
the existing `test_i*` naming convention)
- `Makefile``py-deps` (creates `python/.venv`, installs `httpx[http2]`),
`hx-run`, `hx-test-integration`; `py-test`/`py-check` now run through the
venv and include the new hermetic hxprobe tests
- `docs/usage/py-hxprobe.md` — usage doc with real captured output
- `.gitignore` — added `*.egg-info/` (editable-install artifact)
## Key design decisions
- **Instrument the transport, don't hand-roll HTTP.** Subclassed
`httpcore.NetworkBackend`/`NetworkStream` to time DNS/TCP connect/TLS at
the socket level, letting httpx own HTTP/1.1 vs HTTP/2 framing, redirects,
and keep-alive. This was the reason to use httpx at all — get the protocol
behavior of a real client while keeping latprobe's phase granularity.
- **First-hop-wins for dns/connect/tls; last-hop-wins for ttfb/transfer.**
When redirects are followed, connection-identity fields (dns/connect/tls
timing, resolved IP, TLS/cert info) reflect the *first* connection.
`wrote_request`/`first_byte` are simply overwritten on every write/read, so
they naturally end up reflecting the *last* hop — which mirrors how Go's
own unguarded `httptrace.ClientTrace` hooks behave for a followed redirect.
- **Fresh `httpx.Client` per `measure()` call, no cross-sample pooling** —
matches latprobe's per-call socket creation so every `-n` sample gets a
full phase breakdown.
- **`measure_fn` injection over subclassing/duplication** in `latprobe.cli`,
so hxprobe reuses argparse, concurrency, exit codes, and text/JSON
rendering with zero duplicated logic — the socket and httpx probes only
differ in `probe.py`.
- **New CLI flags gated behind `protocol_flags=True`** so `latprobe`'s own
`--help` output stays byte-for-byte unchanged (verified) — `--no-http2`/
`--no-follow-redirects` only appear for `hxprobe`.
## Notable finding (not part of the original plan)
While comparing `hxprobe` and `latprobe` timings against the same live host,
`hxprobe`'s TTFB was consistently ~40-50ms *lower*. Verified experimentally
(not just assumed) that this is a real effect, not noise: `latprobe`'s raw
socket never sets `TCP_NODELAY`, so its request write is subject to Nagle's
algorithm interacting with the server's delayed-ACK timer — a well-known
artifact. Forcing `TCP_NODELAY` onto `latprobe`'s socket collapsed its TTFB
to match `hxprobe`'s. `hxprobe` sets `TCP_NODELAY` (matching httpcore's own
default backend and Go's `net.Dialer`), so its TTFB numbers are the more
accurate of the two — not just different. Did not change `latprobe` itself
(out of scope for this task); documented the divergence in
`docs/usage/py-hxprobe.md`, in the CHANGELOG, and inline in
`hxprobe/probe.py`.
## Deviations from the plan
- Plan sketched `TTFB = headers-received minus end-of-TLS`; implemented as
`TTFB = first-byte minus wrote-request` instead (matches both Go and the
existing `latprobe` definition — the plan's phrasing was an approximation).
- Plan said verbose TLS/cert metadata could follow "last hop"; implemented
as first-hop-wins uniformly across dns/connect/tls/ip/tls-info for
simplicity and consistency (only `ttfb`/`transfer` are last-hop).
- Everything else (library choice, transport-instrumentation approach,
pyproject.toml, sibling-package placement, flag surface) matches the
approved plan as written.
## Verification
- `make check` (Go + Python full gate): exit 0, 83 hermetic Python tests
(72 pre-existing + 11 new hermetic CLI + hxprobe's share of the 17 probe
tests already counted), zero regressions to latprobe's original 39.
- `make hx-test-integration`: 9/9 live tests pass, including real HTTP/2
negotiation against example.com (Cloudflare) and a real http→https redirect
follow against github.com.
- Manual spot checks: `--json` schema, `--fail` exit code 6, DNS failure
(exit 2), connection-refused (exit 3), `--no-http2`/`--no-follow-redirects`
flag behavior, `latprobe --help` output diffed byte-for-byte against
pre-change output to confirm no regression.
- Caught and reverted an incidental `gofmt` whitespace diff in
`go/internal/probe/probe.go` that `make check`'s `go-fmt` step produced —
unrelated to this change, out of scope, not committed.

View File

@@ -0,0 +1,117 @@
# Summary: extract `hxprobe` into a standalone top-level project
Plan: [docs/plans/2026-07-02-09-32-hxprobe-standalone-project.md](../plans/2026-07-02-09-32-hxprobe-standalone-project.md)
## What was built
`hxprobe` moved from `python/hxprobe/` to a new top-level `hxprobe/`
directory (sibling of `go/` and `python/`), structured as a genuinely
independent project: own `pyproject.toml`, own venv (`hxprobe/.venv`), own
`tests/` directory. It imports nothing from `python/latprobe` — verified with
`grep -rn "latprobe" hxprobe/` returning zero matches in code (a few comments
mention Go's `http.DefaultClient` for context, which is fine; no comment
references `latprobe` as a concrete path/module anymore either).
Files:
- `hxprobe/pyproject.toml` — own manifest, `httpx[http2]` dependency,
`packages = ["hxprobe"]`
- `hxprobe/hxprobe/probe.py` — moved from `python/hxprobe/probe.py`, with
`Options`, `Phase`, `Result`, `VerboseDetail`, `CertInfo` dataclasses and
`_parse_cert`/`_parse_cert_date` helpers now defined locally (copied from
`latprobe/probe.py`) instead of imported; all the httpx-instrumentation
logic (`_Trace`, `_TimingStream`, `_TimingBackend`, `_TimingTransport`,
`measure()`) is unchanged
- `hxprobe/hxprobe/aggregate.py`, `duration.py` — verbatim copies of
`latprobe`'s; their imports were already package-relative (`from .probe
import Result`), so copying required zero edits
- `hxprobe/hxprobe/cli.py` — rewritten as a full standalone CLI. Previously a
27-line wrapper delegating to `latprobe.cli.run()` via an injected
`measure_fn`; now has its own exit codes, argparse, and text/JSON
rendering (copied from the shared `latprobe/cli.py`, then stripped of the
`measure_fn`/`prog`/`description`/`protocol_flags` parameterization that
only existed to let two packages share one `run()`)
- `hxprobe/tests/{test_probe,test_cli,test_integration}.py` — moved from
`python/tests/test_hx_*.py` / `test_integration_hx.py`, `hx`-prefix
dropped, imports repointed from `latprobe.*` to `hxprobe.*`
- `python/latprobe/{cli.py,probe.py}` — reverted via `git checkout --` to
their exact pre-`hxprobe` committed state (confirmed zero diff afterward)
- `Makefile``py-deps`/`PY_VENV*`/shared `hx-run`/`hx-test-integration`
removed from the Python section; new standalone `hx-deps`/`hx-run`/
`hx-test`/`hx-test-integration`/`hx-check`/`hx-clean` targets under
`HX_DIR`/`HX_VENV*`; umbrella `test`/`check`/`clean` now run all three
(`go-*`/`py-*`/`hx-*`)
- `docs/usage/py-hxprobe.md``docs/usage/hxprobe.md` (renamed + rewritten
for the new structure)
- Old `python/hxprobe/`, `python/pyproject.toml`, `python/.venv`,
`python/latprobe_python.egg-info/` deleted
## Key design decisions
- **Move, don't rewrite, the already-debugged files.** `probe.py` and the
three test files carry real bug fixes found during the original
implementation (the `mark_dns` present-on-failure bug, verbose detail not
populated on the failure path, `Content-Length` needed once the test
fixtures switched to HTTP/1.1 keep-alive). Rewriting from scratch would
have risked reintroducing them.
- **`aggregate.py`/`duration.py` needed zero import changes** — their
existing relative imports (`from .probe import ...`) already resolve
correctly once copied into a new package with its own `probe.py`. Not
every file needed the same treatment as `probe.py`/`cli.py`.
- **`cli.py`'s starting point was the *current* shared `latprobe/cli.py`**,
not a from-scratch rewrite — it already contained 100% of the needed
logic (including the `--no-http2`/`--no-follow-redirects` flags and the
verbose Protocol row/JSON keys, both added earlier specifically for
hxprobe). The only work was deleting the generalization scaffolding
(`measure_fn`, `MeasureFn`, `prog`/`description`/`protocol_flags` params)
that existed solely to let two packages share one `run()`.
- **Reverting `latprobe` via `git checkout --` rather than hand-editing** —
confirmed first via `git log`/`git diff --stat` that `cli.py`/`probe.py`
had no changes besides the hxprobe-sharing scaffolding since the last
commit, making this a safe, exact, zero-risk revert.
- **Comments referencing `latprobe` by file path were reworded**, not just
the imports. A comment like "already used by latprobe/probe.py" becomes a
dangling reference once this directory is genuinely portable to another
repo. Reworded ~5 comments/docstrings to describe the technique generically
(e.g., "ports the getaddrinfo → connect split" → "splits DNS and TCP
connect into two timed steps") instead of naming the other project.
## Deviations from the plan
None of substance. One judgment call not spelled out in the plan: the old
`test_cli.py`'s docstring said its tests "focus on what's different... since
run() delegates almost entirely to latprobe.cli.run()" — no longer true now
that `cli.py` is a full standalone reimplementation. Rewrote that docstring
for accuracy rather than leaving a stale claim, without expanding the test
suite itself (that would be a larger, separate scope-creep beyond what was
asked — see the coverage note below).
## Notable follow-up worth flagging
`hxprobe/tests/test_cli.py` has ~11 tests versus `latprobe/tests/test_cli.py`'s
~23. That gap was fine when hxprobe's `cli.py` was a thin wrapper around
already-tested shared code; it's a real gap now that `cli.py` is an
independent ~440-line reimplementation with its own copy of every rendering
branch. Existing tests do exercise the core paths (single URL, `--fail`,
JSON, DNS/connect failures) so this isn't uncovered, but reaching
`latprobe`-level depth (multi-URL separator, sampling aggregate output, JSON
error grouping, all-failed multi-header) would be a reasonable next step if
full independent confidence in the standalone project matters. Not done here
— out of scope for a decoupling/move task.
## Verification
- `grep -rn "latprobe" hxprobe/` — zero matches in code; comment mentions
reworded to be self-contained
- `git diff --stat python/latprobe/` — empty (exact revert to last commit)
- `make hx-deps` — creates `hxprobe/.venv`, installs `httpx[http2]` from
`hxprobe/pyproject.toml`
- `make hx-run ARGS="-v http://github.com"` — same output as before the
move (HTTP/2, 1 redirect followed, IP/TLS/cert shown)
- `make hx-test` — all hermetic hxprobe tests pass standalone (own venv,
own `PYTHONPATH=hxprobe`)
- `make hx-test-integration` — live HTTP/2 + redirect tests still pass
- `make py-test``latprobe`'s own suite passes with zero setup (no
`py-deps`/venv needed), confirming the revert didn't leave residue
- `make check` (top-level) — Go + latprobe + hxprobe all green in one gate
- `python -m latprobe --help` output confirmed unchanged from before the
whole hxprobe feature existed (no leftover `--no-http2` etc.)

View File

@@ -0,0 +1,73 @@
# Summary: hxprobe toolchain modernization (uv, ruff, pytest)
Plan: [docs/plans/2026-07-02-09-57-hxprobe-toolchain-modernization.md](../plans/2026-07-02-09-57-hxprobe-toolchain-modernization.md)
## What was built
Modernized `hxprobe`'s Python toolchain per the user's confirmed choices
(uv, ruff, pytest; mypy skipped). No behavior change to the probe/CLI
itself — this is purely tooling.
- `hxprobe/pyproject.toml`: added `[dependency-groups] dev = ["pytest>=8.0",
"ruff>=0.8"]` (PEP 735), `[tool.pytest.ini_options]` (testpaths, a
registered `integration` marker), `[tool.ruff]` (`target-version =
"py311"`, `line-length = 100`)
- `hxprobe/.python-version`: new, pins `3.14`
- `hxprobe/uv.lock`: new, 18 packages resolved and pinned (`httpx`,
`httpcore`, `h2`, `certifi`, `pytest`, `ruff`, and their transitive deps)
- `hxprobe/tests/test_integration.py`: added `pytestmark =
pytest.mark.integration`, replacing the `test_[!i]*.py` filename-glob
convention with a real pytest marker
- Ran `ruff check --fix` (one fix: removed unused `import sys` in `cli.py`,
inherited from the original `latprobe/cli.py`) and `ruff format .`
(6 files reformatted — collapsed the hand-aligned `=`/dict-key columns to
single-space; no semantic changes)
- `hxprobe/README.md`: new, standalone quick-start
- `Makefile`: `hx-deps`/`hx-run`/`hx-test`/`hx-test-integration` now shell
out to `uv sync`/`uv run` instead of manual venv+pip; added `hx-lint`/
`hx-fmt`; `hx-check` now bundles lint + hermetic tests (mirrors
`go-check`'s fmt+vet+test pattern)
- `docs/usage/hxprobe.md`: Setup and Makefile-targets sections updated
- `.gitignore`: added `.pytest_cache/`, `.ruff_cache/`
## Key design decisions
- **`[dependency-groups]` (PEP 735) over `[tool.uv.dev-dependencies]`** —
the standardized, current uv-recommended way to declare dev-only deps,
keeps them out of the installable package's `dependencies` list.
- **Runner swap only, no test rewrite.** `unittest.TestCase` classes,
`unittest.skipUnless` decorators, and `if __name__ == "__main__":
unittest.main()` guards are untouched — pytest is a superset runner for
unittest-style tests. Only the marker-based selection mechanism changed.
- **`hx-check` now includes lint**, not just tests — matches how
`go-check: go-fmt go-vet go-test` already bundles static checks with
tests in this repo, rather than treating lint as a separate, easy-to-skip
step.
- **Verified the `ruff format` diff was purely cosmetic** before accepting
it: reviewed the actual diff (whitespace/alignment only, no reordering or
logic changes), then confirmed with `ast.parse` on every file post-format
and a full pytest run afterward (37 tests: 28 hermetic + 9 integration,
all passing) — did not run the suite pre-format, so this confirms the
post-format state is correct rather than a strict before/after diff.
## Deviations from the plan
None. Implemented exactly as planned; the `import sys` removal and the
~6-file reformatting were both explicitly anticipated in the plan text
("Lint fixes" section) rather than being surprises.
## Verification
- `cd hxprobe && uv sync` — creates `.venv`, installs from `uv.lock`
(`httpx[http2]`, `pytest`, `ruff` + transitive deps)
- `make hx-run ARGS="-v http://github.com"` — unchanged output (HTTP/2,
1 redirect, IP/TLS/cert)
- `make hx-lint` — `ruff check .` clean (`All checks passed!`)
- `make hx-test` — `pytest tests -m "not integration"`: 28 passed, 9
deselected (matches the pre-toolchain-change hermetic test count exactly)
- `make hx-test-integration` — `pytest tests -m integration`: 9 passed, 28
deselected
- `grep -rn "latprobe" hxprobe/` — zero matches (toolchain change didn't
reintroduce coupling)
- `git diff --stat python/` — empty (hxprobe-only change, `latprobe`
untouched)

View File

@@ -0,0 +1,74 @@
# Summary: hxprobe usage reference doc + standalone Makefile
Plan: [docs/plans/2026-07-02-10-22-hxprobe-usage-doc-and-makefile.md](../plans/2026-07-02-10-22-hxprobe-usage-doc-and-makefile.md)
## What was built
- **`hxprobe/USAGE.md`** (new): a "Runnable Usage Reference" matching the
depth/format of `python/configs/usage-latprobe.md` (concrete `sh` command
→ real captured output, brief explanatory notes, `---` section
separators). 16 cases: basic, verbose (HTTPS, plain HTTP, redirect
followed, `--no-follow-redirects`, `--no-http2`, TLS failure, DNS
failure), sampling, multi-URL, `--fail`, JSON, JSON+verbose, timeout,
exit-codes table, Makefile shortcuts (both `hxprobe/Makefile`'s own and
the parent repo's `hx-*` ones). Placed inside `hxprobe/` per the user's
explicit choice, so the doc travels with the project if it's ever
extracted to its own repo.
- **`docs/usage/hxprobe.md`** (edit): one-line pointer added at the top to
`hxprobe/USAGE.md`. No other changes.
- **`hxprobe/Makefile`** (new): standalone, `help`/`deps`/`run`/`lint`/
`fmt`/`test`/`test-integration`/`check`/`clean`, same auto-generated
`## comment` help style as the root Makefile. Deliberately independent
from the root Makefile's `hx-*` targets (neither calls into the other) —
explicit user decision over the "delegate" alternative.
## Key design decisions
- **Every output in `USAGE.md` is real, freshly captured this session** —
10 of 16 cases were run live during implementation specifically for this
doc (TLS failure, sampling ×2, multi-URL, `--fail`, JSON ×2, timeout,
basic, plain-HTTP), the rest reused real captures from earlier in the
same session (redirect-follow, `--no-follow-redirects`, `--no-http2`, DNS
failure) since the code hadn't changed since those were taken. No numbers
were fabricated or extrapolated.
- **Timeout example uses `192.0.2.1`, not `10.255.255.1`.** Tested both:
`10.255.255.1` resolves to an immediate `connect: Connection refused` in
this dev sandbox (the sandbox's network layer actively rejects the
packet rather than dropping it silently), which would misrepresent the
timeout path. `192.0.2.1` (RFC 5737 TEST-NET-1, reserved/unreachable)
reliably produces a genuine ~500ms timeout here, so that's what the doc
uses and explains.
- **No delegation between the two Makefiles**, per explicit user
instruction — both have complete, independent implementations of the
same `uv sync`/`uv run pytest`/`ruff` commands. This is deliberate
duplication: a change to one Makefile's command flags won't silently
break the other, at the cost of needing to update both if the underlying
`uv run ...` invocations ever change.
- **`hxprobe/Makefile` target names have no `hx-` prefix** (`run`, `test`,
not `hx-run`, `hx-test`) since the prefix's whole purpose — disambiguating
from `go-*`/`py-*` targets in the same file — doesn't apply once you're
already inside `hxprobe/`'s own Makefile.
## Deviations from the plan
One, driven by what the live network in this sandbox actually does: the
plan didn't anticipate the timeout target needing to change from
`10.255.255.1` (used in `latprobe`'s own timeout example) to `192.0.2.1`.
Discovered and resolved by testing both live before writing the doc, rather
than assuming `latprobe`'s example target would work identically for
hxprobe.
## Verification
- `cd hxprobe && make help` — lists all 9 targets, no dependency on the
root Makefile
- `cd hxprobe && make run ARGS="-v https://example.com"` — matches
`make hx-run ARGS="-v https://example.com"` output shape from the repo
root (both Makefiles agree independently)
- `cd hxprobe && make check` — 28 hermetic tests pass (lint + test)
- `cd hxprobe && make test-integration` — 9 live tests pass
- Every command block in `USAGE.md` was actually executed during
authoring; output pasted directly from the terminal, not hand-edited
beyond JSON float-precision rounding (documented as such in the doc)
- `git diff Makefile` — confirms the root Makefile's `hx-*` section is
unchanged by this task

View File

@@ -0,0 +1,77 @@
# Summary: hxprobe reads target URLs from a file
Plan: [docs/plans/2026-07-02-11-14-hxprobe-file-input.md](../plans/2026-07-02-11-14-hxprobe-file-input.md)
## What was built
- **`hxprobe/hxprobe/cli.py`**: new `-f`/`--file PATH` flag, mutually
exclusive with positional `url` args. `urls` positional changed
`nargs="+"``nargs="*"`. New `_load_urls()` helper (same format as
`simple.py`'s `load_sites()`: one URL per line, `#` comments, blank
lines skipped, first token per line) — reimplemented locally rather than
imported, keeping hxprobe's "no imports outside its own directory" rule
intact. Validation added right after `parser.parse_args()`, inside the
same `try/except _ArgExit` block: both-given and neither-given are usage
errors via `parser.error()`; missing/unreadable/empty file are usage
errors via direct `stderr.write()` + `return EXIT_USAGE` (mirroring the
existing `--timeout` invalid-value handling style already in the file).
- **`hxprobe/tests/test_cli.py`**: new `TestCLIFileInput` class, 5 tests —
reads URLs from a temp file successfully, missing file, empty file,
both-sources error, neither-given error.
- **`hxprobe/configs/*.txt`**: 7 new fixtures mirroring
`python/configs/`'s exact set. Each header comment states an "Expected
exit code" that was verified by actually running the fixture through
`hxprobe -f ...` during implementation (not assumed from the
`simple.py` originals, whose blanket 0/1 exit scheme is fundamentally
different from hxprobe's per-failure-class 06 worst-code-wins scheme).
- **`hxprobe/USAGE.md`**: new "Reading URLs from a file (`-f`)" section,
placed after "Multiple URLs", with real captured output for the
successful case, the mutually-exclusive error case, and one failure
fixture (`dns-failure.txt`), plus a table listing all 7 fixtures and
their expected exit codes.
## Key design decisions
- **Manual post-parse validation instead of `argparse`'s
`add_mutually_exclusive_group`** — a variadic positional (`nargs="*"`)
doesn't mix cleanly with argparse's built-in mutually-exclusive-group
machinery. Manual checks after `parser.parse_args()` (still inside the
same `try/except _ArgExit`, using `parser.error()`) give the same
usage-error behavior with full control over the message text.
- **File I/O errors don't go through `parser.error()`** — they use the
same direct `stderr.write()` + `return EXIT_USAGE` pattern already
established for `--timeout` parsing failures, since they're discovered
after parsing succeeds, not during it.
- **Every fixture's exit code was verified live, not assumed.** Two
required real judgment calls the `simple.py` originals didn't need:
`http-errors.txt` and `mixed.txt` both needed `--fail` added to their
demo command (hxprobe treats 4xx as success without it, unlike
`simple.py` which always raises on `HTTPError`) to actually demonstrate
a failure — without `--fail` both would silently show exit 0.
## Deviations from the plan
None of substance. The plan anticipated needing to verify exit codes
live rather than assume them; that anticipation paid off exactly as
expected for `http-errors.txt`/`mixed.txt` (needed `--fail` added) and
`timeout.txt` (needed `--timeout 2s` added to keep the demo fast, and a
note added about this sandbox occasionally short-circuiting one of the two
timeout targets to an immediate "connection refused" — observed directly:
in this session's test run, `10.255.255.1` genuinely timed out; in an
earlier, unrelated test earlier in the session it instead got refused
immediately. The fixture keeps both targets so at least one demonstrates
the real timeout path regardless).
## Verification
- `all-ok.txt` → exit 0, `dns-failure.txt` → 2, `connection-refused.txt`
→ 3, `tls-errors.txt` → 5, `timeout.txt` (with `--timeout 2s`) → 4,
`http-errors.txt` (with `--fail`) → 6, `mixed.txt` (with `--fail`) → 6 —
all confirmed via real `$?` checks (not through a `grep` pipe, which
masks the real exit code — caught and corrected this during testing)
- `uv run pytest tests/test_cli.py -m "not integration"` — 16/16 pass
(11 pre-existing + 5 new)
- `uv run ruff check .` / `uv run ruff format --check .` — clean
- Both-sources-given and missing-file error messages verified against the
doc's pasted output, including the `usage:` block that's actually
printed (an early draft of the doc omitted it — caught on review)

View File

@@ -0,0 +1,83 @@
# hxprobe simplification — summary
Plan: `docs/plans/2026-07-02-12-05-hxprobe-simplification.md` (five targeted
removals of speculative scaffolding / redundant state / duplicated logic, no
behavior change). Trigger: a question about a dead `file=` parameter on
`_Parser.print_help`/`print_usage` in `hxprobe/hxprobe/cli.py`, which led to a
broader pass over the whole package for the same pattern.
## What changed
1. **`hxprobe/hxprobe/cli.py` — deleted the `_Parser` subclass and `_ArgExit`
exception** (~32 lines). It re-implemented, by hand, behavior the standard
library already provides: `argparse.ArgumentParser` already writes help to
`sys.stdout` and usage/errors to `sys.stderr`, and already raises
`SystemExit` rather than hard-exiting. `run()` now builds a plain
`argparse.ArgumentParser`, wraps the parse + validation calls in
`contextlib.redirect_stdout(stdout)` / `redirect_stderr(stderr)`, and
catches `SystemExit` at a single site:
```python
except SystemExit as exc:
return EXIT_OK if not exc.code else EXIT_USAGE
```
Deviation from the original plan sketch: the plan's snippet used
`EXIT_OK if not exc.code else EXIT_USAGE`, matching what was actually
implemented (equivalent to, but slightly more defensive than, the
`exc.code == 0` check first drafted, since argparse can in principle pass
`None`).
2. **`hxprobe/hxprobe/probe.py` — trimmed `_TimingStream.get_extra_info`** to
the single branch actually consumed on the request path (`ssl_object`).
Verified by grepping the installed `httpcore`/`httpx` packages: the
`server_addr`/`client_addr` branches were only ever queried by `httpx`'s own
`_main.py` (the `httpx` CLI command), never by anything hxprobe's request
path touches.
3. **`hxprobe/hxprobe/cli.py` — simplified `_load_urls`**: dropped the
`line.split()[0]` "forward-compatible with future `url key=value`
annotations" scaffolding; a stripped line is used directly. Docstring
updated to describe only what the function does today.
4. **`hxprobe/hxprobe/probe.py` — removed `_dns_set`/`_connect_set`/
`_tls_set`** from `_Trace`. These booleans duplicated `Phase.present` on
`self.dns`/`self.connect`/`self.tls` (each starts as `Phase()`, i.e.
`present=False`). Guards rewritten from `if not self._dns_set:` to
`if not self.dns.present:` (and analogously for connect/tls) — identical
first-hop-wins semantics for redirects, one less piece of parallel state.
5. **`hxprobe/hxprobe/cli.py` — extracted `_summarize_failures`**, a shared
helper that dedupes `failed: list[Result]` into `[(phase, message, count),
...]` in first-seen order. `_print_failure_summary` (text rendering) and
`_build_json_entry` (JSON rendering) previously each carried an identical
~12-line dedup loop; both now call the helper and only differ in how they
format the tuple.
## Not changed (considered, kept — per the plan)
- The custom httpcore backend (`_TimingBackend`/`_TimingStream`/
`_TimingTransport`) — this is the tool's actual reason to exist.
- `ThreadPoolExecutor` concurrency — backs the shipped `-c/--concurrency` and
multi-URL/`-f` features.
- Explicit per-phase dataclass fields in `probe.py`/`aggregate.py`.
- `CertInfo.sans` — unused in text output but real (emitted in JSON verbose
output).
## Verification
- `hxprobe/.venv/bin/python -m pytest tests/test_cli.py tests/test_probe.py -q`
→ **33 passed**, no test file edits.
- `hxprobe/.venv/bin/ruff check hxprobe/` → **all checks passed**.
- Manual smoke tests (`python -m hxprobe -h`, no-args, and a live
`https://example.com -v` request):
- `-h` → help text on stdout, exit `0`.
- No args → usage + `no URLs given` error on **stderr only** (confirmed via
separate stdout/stderr redirection to files — stdout was empty), exit `1`.
- Live verbose request → correct 6-phase timing, resolved IP, HTTP/2
protocol, TLS version/cipher, and certificate detail — confirms the
`get_extra_info` trim didn't break `ssl_object` retrieval and the
`_Trace` refactor didn't break phase/first-hop-wins tracking.
Net: `cli.py` and `probe.py` are shorter and carry less parallel/duplicated
state; no observable behavior changed.

View File

@@ -0,0 +1,105 @@
# hxprobe: end-of-run summary footer — summary
Plan: `docs/plans/2026-07-02-14-05-hxprobe-run-summary-footer.md`.
Trigger: a follow-up to
`docs/explanations/2026-07-02-13-25-hxprobe-worst-exit-code-and-render-loop.md`
— does a single worst-code exit even make sense across multiple URLs with
different possible errors? The answer landed on: keep the scalar exit code
(it's a documented cross-implementation contract with `latprobe`/Go — see
`hxprobe/USAGE.md`'s Exit codes table, asserted by 13 existing tests), and
instead add the missing visibility as an end-of-run summary footer.
## What changed
All in `hxprobe/hxprobe/cli.py` unless noted, matching the plan exactly (no
deviations):
1. **`_EXIT_LABELS`** (new, next to `_PHASE_EXIT`): inverse mapping from exit
code → short label (`ok`, `dns`, `connect`, `timeout`, `tls`, `http`), used
to render the footer's `→ exit N (label)` line.
2. **Accumulation loop refactor**: previously computed a single running
`worst` directly inside the per-URL loop via two different idioms
(`if c > worst: worst = c` for network failures, `max(worst, EXIT_HTTP)`
for `--fail`). Now each URL first gets its own `code` (`EXIT_OK` folded up
via `max()` across its failed samples and, if `--fail`, its ≥400 successes),
appended to a new `url_codes: list[int]` (index-aligned with `urls`), and
*then* folded into `worst = max(worst, code)`. Unifies both idioms into one.
3. **`_print_run_summary(urls, url_codes, worst, out)`** (new helper, next to
`_print_failure_summary`): writes a separator line, an
`N URLs — X ok[, Y failed]` header, one `✗ {label:<8}: {count}` line per
non-OK class present (first-seen order, same dedup style as
`_summarize_failures`), and the closing `→ exit N (label)` line.
4. **Call site**: `elif len(urls) > 1: _print_run_summary(urls, url_codes,
worst, stdout)` — added after the per-URL loop, in the `else` branch of the
existing `if ns.json_out:` check (so it's text-mode-only), right before
`return worst`. JSON path (`json.dumps(json_items, ...)`) is completely
untouched — still a bare array, no top-level summary object, preserving the
documented "same shape as `latprobe`'s JSON" contract.
## Design decisions (confirmed with user before implementing)
- **Exit code stays a scalar** (worst/highest severity across URLs) — not
count-of-failed-URLs, not binary 0/1. Both alternatives were presented and
rejected because they'd break the documented 06 table, Go/`latprobe`
parity, and the 13 existing exit-code tests.
- **Summary is a text footer, multi-URL only** (`len(urls) > 1`) — not always
shown, and not also duplicated into JSON as a top-level object (which would
turn the JSON array into an object and break the documented array-shape
parity). Single-URL text output is untouched; JSON output is untouched.
## Tests
`hxprobe/tests/test_cli.py`: new `TestCLIRunSummary` class, 3 tests, reusing
the existing `_OKHandler`/`_start_server`/`_free_port`/`_invoke` harness:
- `test_multi_url_mixed_shows_summary` — one OK URL + one connection-refused
URL: asserts `EXIT_CONNECT`, and the footer strings (`"Summary: 2 URLs"`,
`"1 ok"`, `"1 failed"`, `"connect : 1"`, `"→ exit 3"`).
- `test_multi_url_all_ok_summary` — two OK URLs: asserts `EXIT_OK`,
`"Summary: 2 URLs — 2 ok"`, and no `"✗"` anywhere in output.
- `test_single_url_has_no_summary` — one OK URL: asserts `"Summary:"` is
absent (locks the multi-URL-only rule).
All 33 pre-existing tests pass unedited (33 + 3 new = 36 total).
## Verification
- `hxprobe/.venv/bin/python -m pytest tests/test_cli.py tests/test_probe.py -q`
→ **36 passed**.
- `hxprobe/.venv/bin/ruff check hxprobe/ tests/` → **all checks passed**.
- Manual smoke tests, all matching the plan's expected behavior exactly:
- `hxprobe https://example.com https://example.org` → footer
`Summary: 2 URLs — 2 ok` / `→ exit 0 (ok)`.
- `hxprobe https://example.com http://no.such.host.invalid` → footer
`Summary: 2 URLs — 1 ok, 1 failed` / `✗ dns : 1` / `→ exit 2 (dns)`;
process exit code confirmed `2` via `echo $?`.
- `hxprobe https://example.com` (single URL) → **no** footer, output
byte-for-byte the same shape as before this change.
- `hxprobe --json https://example.com https://example.org` → still a bare
JSON array, no summary object.
- Also captured a 3-URL mixed run (`example.com` ok, DNS failure, TLS
failure against `self-signed.badssl.com`) to confirm severity ordering in
the footer: DNS (2) and TLS (5) both counted, exit reported as `5 (tls)`
— the higher-severity class correctly wins the scalar while the footer
still shows the DNS failure that the scalar alone would hide.
## Docs updated (per CLAUDE.md conventions)
- `hxprobe/USAGE.md`: new "Multi-URL summary footer" section with a real
3-URL mixed-outcome capture; refreshed the pre-existing "Multiple URLs" and
`-f configs/all-ok.txt` / `-f configs/dns-failure.txt` examples, which were
captured before this feature existed and were now stale (missing the
footer) — replaced with fresh live captures; added a sentence to the Exit
codes section pointing at the new section.
- `docs/usage/hxprobe.md`: one-line addition to the exit-codes bullet
pointing at `hxprobe/USAGE.md`'s new section.
- `docs/explanations/2026-07-02-13-25-hxprobe-worst-exit-code-and-render-loop.md`:
appended an "Update (2026-07-02)" paragraph pointing forward to this work,
per the plan's "optional — pointer instead of a new file" option.
- `CHANGELOG.md`: new entry at the top.
- This file.
No deviations from the approved plan.

232
docs/usage/hxprobe.md Normal file
View File

@@ -0,0 +1,232 @@
# `hxprobe/` — httpx-based HTTP latency probe (Go-client parity)
> For a full case-by-case runnable reference (every flag, every failure
> mode, real captured output), see [`hxprobe/USAGE.md`](../../hxprobe/USAGE.md).
## What it does
A standalone Python CLI, independent of anything else in this repo (own
`pyproject.toml`, own venv — see [Setup](#setup)). It measures per-phase HTTP
latency like [`python/latprobe`](py-latprobe.md), but is built on the
[httpx](https://www.python-httpx.org/) library instead of raw sockets, so the
client matches Go's `http.DefaultClient`: **HTTP/2 negotiated via ALPN**,
**redirects followed by default**, connection pooling, and default TLS
verification — while still reporting the same six-phase breakdown (DNS, TCP
connect, TLS, TTFB, Transfer, Total).
Python has no equivalent of Go's `net/http/httptrace`, so the phase timing is
recovered by instrumenting httpx's network backend directly
(`hxprobe/probe.py`): a custom `NetworkBackend`/`NetworkStream` pair times
DNS, TCP connect, and TLS at the socket level, while HTTP framing (HTTP/1.1
or HTTP/2), redirect-following, and keep-alive stay entirely owned by httpx.
Run it from the `hxprobe/` directory with `uv run python -m hxprobe`, or via
`make hx-run ARGS="…"` from the repo root.
## Setup
`hxprobe` is a fully self-contained project — it could be copied out of this
repo into its own tomorrow and still work, with its own `pyproject.toml`,
lockfile (`uv.lock`), and [uv](https://docs.astral.sh/uv/)-managed venv
(`hxprobe/.venv`), separate from anything under `python/`. Install once:
```sh
make hx-deps
```
This runs `uv sync` inside `hxprobe/`, creating `.venv` and installing
`httpx` (plus `h2` for HTTP/2), `pytest`, and `ruff` from `uv.lock` — pinned,
reproducible versions, not whatever the resolver happens to pick at install
time. It's a prerequisite of `make hx-run`/`make hx-test`/etc., so those
targets set it up automatically on first run — `make hx-deps` is only needed
if you want to call `uv run python -m hxprobe` directly from inside
`hxprobe/`. See [hxprobe/README.md](../../hxprobe/README.md) for the
from-inside-the-directory quick start.
## Flags / arguments
Same surface as `latprobe`, plus two opt-outs for the Go-like defaults:
```
python -m hxprobe [flags] <url> [url ...]
```
| Flag | Default | Description |
|------|---------|-------------|
| `url …` (positional) | required | One or more URLs to probe |
| `-n N`, `--count N` | `1` | Number of requests per URL |
| `-c N`, `--concurrency N` | `0` (auto) | Max parallel URLs; `0` = `min(len(urls), 8)` |
| `--timeout DURATION` | `10s` | Per-request timeout; supports `ms`, `s`, `m`, or bare seconds |
| `--fail` | off | Exit non-zero when any HTTP status ≥ 400 |
| `--json` | off | Output as JSON array instead of text |
| `-v`, `--verbose` | off | Show resolved IP, negotiated protocol/redirects, TLS info, certificate, response headers |
| `--no-http2` | off (HTTP/2 on) | Disable HTTP/2 negotiation, force HTTP/1.1 |
| `--no-follow-redirects` | off (follow on) | Report the raw redirect response instead of following it |
| `-h`, `--help` | — | Show help and exit 0 |
**Exit codes:** identical to `latprobe` (0 ok, 1 usage, 2 dns, 3 connect,
4 timeout, 5 tls, 6 http≥400 with `--fail`); the highest code across all URLs
is returned. When more than one URL is probed, a summary footer is appended
after the last URL block tallying every URL's outcome (ok / dns / connect /
timeout / tls / http) and the resulting exit code — see "Multi-URL summary
footer" in [`hxprobe/USAGE.md`](../../hxprobe/USAGE.md) for a real example.
## Examples
### Single URL — negotiates HTTP/2 by default
```sh
make hx-run ARGS="https://example.com"
```
```
https://example.com (200)
DNS lookup : 2.81 ms
TCP connect : 8.28 ms
TLS handshake : 16.13 ms
Server (TTFB) : 1.04 ms
Transfer : 0.62 ms
─────────────────────────────
Total : 42.53 ms
```
### Verbose — shows the negotiated protocol
```sh
make hx-run ARGS="-v https://example.com"
```
```
https://example.com (200)
DNS lookup : 2.93 ms
TCP connect : 8.58 ms
TLS handshake : 17.71 ms
Server (TTFB) : 4.35 ms
Transfer : 0.84 ms
─────────────────────────────
Total : 44.06 ms
IP : 104.20.23.154
Protocol : HTTP/2
TLS : TLSv1.3 TLS_AES_256_GCM_SHA384 256 bit
Cert : CN=example.com valid until 2026-08-29 SSL Corporation
```
### Redirects followed by default
```sh
make hx-run ARGS="-v http://github.com"
```
```
http://github.com (200)
DNS lookup : 14.01 ms
TCP connect : 19.42 ms
TLS handshake : 22.72 ms
Server (TTFB) : 0.01 ms
Transfer : 65.66 ms
─────────────────────────────
Total : 196.72 ms
IP : 140.82.121.4
Protocol : HTTP/2 (1 redirect)
TLS : TLSv1.3 TLS_AES_128_GCM_SHA256 128 bit
Cert : CN=github.com valid until 2026-08-02 Sectigo Limited
```
Pass `--no-follow-redirects` to get the raw redirect response instead:
```sh
make hx-run ARGS="--no-follow-redirects http://github.com"
```
```
http://github.com (301)
DNS lookup : 2.85 ms
TCP connect : 24.16 ms
Server (TTFB) : 24.50 ms
Transfer : 0.59 ms
─────────────────────────────
Total : 52.58 ms
```
### Forcing HTTP/1.1
```sh
make hx-run ARGS="-v --no-http2 https://example.com"
```
```
https://example.com (200)
DNS lookup : 2.31 ms
TCP connect : 8.75 ms
TLS handshake : 13.61 ms
Server (TTFB) : 13.51 ms
Transfer : 0.44 ms
─────────────────────────────
Total : 39.07 ms
IP : 104.20.23.154
Protocol : HTTP/1.1
TLS : TLSv1.3 TLS_AES_256_GCM_SHA384 256 bit
Cert : CN=example.com valid until 2026-08-29 SSL Corporation
```
### Sampling, JSON, `--fail`, DNS failure
Same shape as `latprobe`'s own examples — see
[py-latprobe.md](py-latprobe.md#examples) for `-n`, `--json`, multi-URL,
`--fail`, and timeout output. hxprobe's JSON verbose object adds two keys:
```json
"verbose": {
"ip": "104.20.23.154",
"http_version": "HTTP/2",
"redirect_count": 1,
"tls_version": "TLSv1.3",
...
}
```
`http_version`/`redirect_count` are omitted from the text view when
`http_version` is unknown (only happens on failures before headers arrived).
## HTTP/2, redirects, and a TTFB accuracy note
**HTTP/2 and redirects** are the headline reasons this project exists
alongside the raw-socket `latprobe` — see [py-latprobe.md](py-latprobe.md) for
what that implementation does instead (always HTTP/1.1, never follows
redirects).
**A TTFB finding worth knowing, if you've compared numbers against another
raw-socket HTTP client:** during development, this implementation's TTFB
came in consistently ~40-50ms *lower* than a comparable raw-socket
implementation that didn't set `TCP_NODELAY` on its connect socket. That gap
wasn't noise — a socket that never sets `TCP_NODELAY` is subject to Nagle's
algorithm interacting with the server's delayed-ACK timer, a well-known
~40ms artifact. This implementation sets `TCP_NODELAY` on every connection
(matching both httpcore's own default backend and Go's `net.Dialer`),
avoiding that penalty. Forcing `TCP_NODELAY` onto the other socket
experimentally collapsed its TTFB to match this one's — confirming the
cause. If you're comparing hxprobe's numbers against some other HTTP/1.1
client that doesn't set `TCP_NODELAY`, expect this implementation's TTFB to
read lower, and correctly so.
**Redirect semantics for `dns`/`connect`/`tls` vs `ttfb`/`transfer`:** when
redirects are followed, `dns`/`connect`/`tls` (and the verbose IP/TLS/cert
fields) reflect the **first** connection only — "cost of reaching the origin
server." `ttfb`/`transfer` reflect the **last** hop, because each write/read
call overwrites them — which mirrors how Go's own `httptrace.ClientTrace`
hooks behave for a followed redirect (they aren't guarded either, so the last
hop wins there too).
## Limitations
- Always `GET`, no custom headers/body/auth — matches Go's `http.DefaultClient`.
- `Options.timeout` applies uniformly to connect/read/write/pool phases (a
single value); it is not split into separate per-phase budgets.
- No UNIX socket support.
- HTTP/2 requires TLS (`https://`) in practice — cleartext `h2c` is not
attempted for `http://` URLs (matches nearly every real HTTP/2 deployment).
## Makefile targets
```sh
make hx-deps # one-time: uv sync (venv + lockfile install)
make hx-run ARGS="-v -n 3 https://example.com" # run it
make hx-test # hermetic unit tests (pytest -m "not integration")
make hx-test-integration # live tests: real HTTP/2 negotiation, redirects
make hx-lint # ruff check
make hx-fmt # ruff format
make hx-check # lint + hermetic tests (the pre-commit-style gate)
```
`hx-check` (lint + hermetic tests) also runs as part of the repo-root
`make check`; `hx-test` runs as part of `make test`.

1
hxprobe/.python-version Normal file
View File

@@ -0,0 +1 @@
3.14

55
hxprobe/Makefile Normal file
View File

@@ -0,0 +1,55 @@
# hxprobe — standalone Makefile. Requires uv (https://docs.astral.sh/uv/).
#
# This project is fully self-contained (own pyproject.toml, own uv.lock) and
# this Makefile works the same whether run from inside a parent monorepo or
# after `cp -r`'ing hxprobe/ into its own repo. It does not depend on, and is
# not depended on by, the parent repo's root Makefile — the two are kept
# independent on purpose so a change to one can't break the other.
PYTHON ?= python3.14
ARGS ?=
.DEFAULT_GOAL := help
# ── help ──────────────────────────────────────────────────────────────────────
.PHONY: help
help: ## Show this help
@grep -E '^[a-zA-Z_-]+:.*##' $(MAKEFILE_LIST) \
| awk 'BEGIN {FS = ":.*##"}; {printf " \033[36m%-20s\033[0m %s\n", $$1, $$2}' \
| sort
# ── targets ───────────────────────────────────────────────────────────────────
.PHONY: deps
deps: ## sync the venv from uv.lock; idempotent
uv sync
.PHONY: run
run: deps ## run hxprobe (pass flags via ARGS="…")
uv run $(PYTHON) -m hxprobe $(ARGS)
.PHONY: lint
lint: deps ## lint with ruff
uv run ruff check .
.PHONY: fmt
fmt: deps ## format with ruff
uv run ruff format .
.PHONY: test
test: deps ## run hermetic unit tests
uv run pytest tests -m "not integration" -v $(ARGS)
.PHONY: test-integration
test-integration: deps ## run integration tests against live internet services (HTTP/2, redirects)
uv run pytest tests -m integration -v $(ARGS)
.PHONY: check
check: lint test ## run the full gate: lint + hermetic tests
.PHONY: clean
clean: ## remove bytecode, caches, and egg-info
@find . -type d -name '__pycache__' -exec rm -rf {} + 2>/dev/null; \
find . -name '*.pyc' -delete 2>/dev/null; \
rm -rf *.egg-info .pytest_cache .ruff_cache; true

38
hxprobe/README.md Normal file
View File

@@ -0,0 +1,38 @@
# hxprobe
Per-phase HTTP latency probe built on [httpx](https://www.python-httpx.org/),
matching Go's `http.DefaultClient`: HTTP/2 negotiated via ALPN, redirects
followed by default, connection pooling, and default TLS verification —
while reporting a six-phase timing breakdown (DNS, TCP connect, TLS, TTFB,
Transfer, Total).
This is a fully self-contained project: no imports outside this directory,
own `pyproject.toml`, own lockfile (`uv.lock`). It can be copied out of its
parent repo and used standalone.
See [docs/usage/hxprobe.md](../docs/usage/hxprobe.md) in the parent repo for
full flag reference, example output, and design notes (redirect semantics,
a TTFB-accuracy finding worth knowing about).
## Quick start
Requires [uv](https://docs.astral.sh/uv/).
```sh
uv sync # create .venv, install deps + lockfile
uv run python -m hxprobe https://example.com
uv run python -m hxprobe -v http://github.com # verbose: shows HTTP/2, redirects, TLS, cert
```
## Development
```sh
uv run pytest tests -m "not integration" -v # hermetic unit tests
uv run pytest tests -m integration -v # live tests (hits the real internet)
uv run ruff check . # lint
uv run ruff format . # format
```
From the parent repo's root, the equivalent Makefile targets are
`make hx-run`, `make hx-test`, `make hx-test-integration`, `make hx-lint`,
`make hx-fmt`, and `make hx-check` (lint + hermetic tests).

703
hxprobe/USAGE.md Normal file
View File

@@ -0,0 +1,703 @@
# `hxprobe` — Runnable Usage Reference
All commands below assume you're inside this directory (`cd hxprobe`) with
dependencies installed (`make deps` or `uv sync`). Timings will differ on
your machine and network; the output structure is stable. Every example
below was run against the live internet.
---
## Basic — single URL
```sh
make run ARGS="https://example.com"
# or directly:
uv run python -m hxprobe https://example.com
```
```
https://example.com (200)
DNS lookup : 17.98 ms
TCP connect : 8.18 ms
TLS handshake : 15.41 ms
Server (TTFB) : 3.38 ms
Transfer : 7.99 ms
─────────────────────────────
Total : 61.93 ms
```
---
## Verbose mode — IP, protocol, TLS, certificate
```sh
uv run python -m hxprobe --verbose https://example.com
```
```
https://example.com (200)
DNS lookup : 2.33 ms
TCP connect : 14.73 ms
TLS handshake : 15.22 ms
Server (TTFB) : 5.63 ms
Transfer : 0.57 ms
─────────────────────────────
Total : 49.16 ms
IP : 104.20.23.154
Protocol : HTTP/2
TLS : TLSv1.3 TLS_AES_256_GCM_SHA384 256 bit
Cert : CN=example.com valid until 2026-08-29 SSL Corporation
```
`Protocol` is the one row `latprobe` (the raw-socket sibling implementation)
never prints — it's the ALPN-negotiated HTTP version, only meaningful for a
client that can actually speak more than one.
---
## Verbose — plain HTTP (no TLS block)
```sh
uv run python -m hxprobe --verbose http://example.com
```
```
http://example.com (200)
DNS lookup : 2.15 ms
TCP connect : 8.52 ms
Server (TTFB) : 16.33 ms
Transfer : 0.42 ms
─────────────────────────────
Total : 27.80 ms
IP : 104.20.23.154
Protocol : HTTP/1.1
```
No `TLS` or `Cert` rows for `http://` URLs. `Protocol` still shows —
HTTP/2 is not attempted over cleartext (see Limitations in
[`docs/usage/hxprobe.md`](../docs/usage/hxprobe.md)), so this is always
`HTTP/1.1`.
---
## Verbose — redirect followed by default
```sh
uv run python -m hxprobe --verbose http://github.com
```
```
http://github.com (200)
DNS lookup : 14.34 ms
TCP connect : 20.88 ms
TLS handshake : 25.91 ms
Server (TTFB) : 3.44 ms
Transfer : 63.33 ms
─────────────────────────────
Total : 197.21 ms
IP : 140.82.121.3
Protocol : HTTP/2 (1 redirect)
TLS : TLSv1.3 TLS_AES_128_GCM_SHA256 128 bit
Cert : CN=github.com valid until 2026-08-02 Sectigo Limited
```
`http://github.com` 301-redirects to `https://github.com`; hxprobe follows
it by default (matching Go's `http.DefaultClient`) and reports the final
response. `latprobe` has no equivalent — it would print the bare `301` and
stop. `DNS`/`TCP connect`/`TLS` are timed from the *first* connection only;
`Server (TTFB)`/`Transfer` reflect the final hop (see "Divergence from
`latprobe`" in [`docs/usage/hxprobe.md`](../docs/usage/hxprobe.md) for why).
---
## `--no-follow-redirects` — report the raw redirect instead
```sh
uv run python -m hxprobe --no-follow-redirects http://github.com
```
```
http://github.com (301)
DNS lookup : 2.85 ms
TCP connect : 24.16 ms
Server (TTFB) : 24.50 ms
Transfer : 0.59 ms
─────────────────────────────
Total : 52.58 ms
```
Same shape `latprobe` would show for any redirect — hxprobe just makes it
opt-in rather than the default.
---
## `--no-http2` — force HTTP/1.1
```sh
uv run python -m hxprobe --verbose --no-http2 https://example.com
```
```
https://example.com (200)
DNS lookup : 2.31 ms
TCP connect : 8.75 ms
TLS handshake : 13.61 ms
Server (TTFB) : 13.51 ms
Transfer : 0.44 ms
─────────────────────────────
Total : 39.07 ms
IP : 104.20.23.154
Protocol : HTTP/1.1
TLS : TLSv1.3 TLS_AES_256_GCM_SHA384 256 bit
Cert : CN=example.com valid until 2026-08-29 SSL Corporation
```
Useful for isolating whether a latency difference is due to protocol version
rather than network conditions.
---
## Verbose — TLS failure (certificate expired)
```sh
uv run python -m hxprobe --verbose https://expired.badssl.com/
echo "exit: $?"
```
```
https://expired.badssl.com/ (FAILED)
DNS lookup : 33.34 ms
TCP connect : 1129.53 ms
TLS handshake : 318.54 ms
─────────────────────────────
Total : 1484.12 ms
✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1081)
IP : 104.154.89.105
exit: 5
```
The IP is shown even on TLS failure (DNS and TCP both succeeded). badssl.com
is a shared, often-loaded demo host — `TCP connect` here is unusually slow;
that's the host, not hxprobe.
---
## Verbose — DNS failure (no IP to show)
```sh
uv run python -m hxprobe --verbose http://no.such.host.invalid
echo "exit: $?"
```
```
http://no.such.host.invalid (FAILED)
Total : 13.22 ms
✗ dns: [Errno 8] nodename nor servname provided, or not known
exit: 2
```
The verbose block is empty (IP never resolved), so it's suppressed entirely
— same behavior as `latprobe`.
---
## Sampling (`-n`) — min / avg / max table
```sh
uv run python -m hxprobe -n 3 https://example.com
```
```
https://example.com (200, 3 samples)
min avg max
DNS lookup : 1.23 ms 2.29 ms 3.02 ms
TCP connect : 8.05 ms 8.73 ms 9.81 ms
TLS handshake : 12.99 ms 15.65 ms 17.48 ms
Server (TTFB) : 0.01 ms 7.23 ms 11.95 ms
Transfer : 0.63 ms 2.18 ms 5.15 ms
─────────────────────────────────────────────────
Total : 38.56 ms 49.16 ms 54.70 ms
```
With `--verbose`, the block is appended below the table using the last
successful sample's detail:
```sh
uv run python -m hxprobe --verbose -n 3 https://example.com
```
```
https://example.com (200, 3 samples)
min avg max
...
Total : 39.95 ms 42.55 ms 44.66 ms
IP : 104.20.23.154
Protocol : HTTP/2
TLS : TLSv1.3 TLS_AES_256_GCM_SHA384 256 bit
Cert : CN=example.com valid until 2026-08-29 SSL Corporation
```
---
## Multiple URLs (probed in parallel)
```sh
uv run python -m hxprobe https://example.com https://www.iana.org
```
```
https://example.com (200)
DNS lookup : 3.96 ms
TCP connect : 13.13 ms
TLS handshake : 15.31 ms
Server (TTFB) : 0.01 ms
Transfer : 7.20 ms
─────────────────────────────
Total : 50.16 ms
https://www.iana.org (200)
DNS lookup : 26.67 ms
TCP connect : 9.05 ms
TLS handshake : 14.73 ms
Server (TTFB) : 3.37 ms
Transfer : 0.63 ms
─────────────────────────────
Total : 70.42 ms
─────────────────────────────────────────────────
Summary: 2 URLs — 2 ok
→ exit 0 (ok)
```
Output for each URL is separated by a blank line. Exit code is the worst
across all — see "Multi-URL summary footer" below for how that scalar breaks
down when URLs have different outcomes.
---
## Multi-URL summary footer
Whenever more than one URL is probed (positional args or `-f`), a footer is
appended after the last URL block: a per-outcome tally plus the exit code it
produced. Single-URL runs never show it — text output is otherwise unchanged.
```sh
uv run python -m hxprobe https://example.com http://no.such.host.invalid https://self-signed.badssl.com
echo "exit: $?"
```
```
https://example.com (200)
DNS lookup : 4.65 ms
TCP connect : 9.56 ms
TLS handshake : 14.90 ms
Server (TTFB) : 0.01 ms
Transfer : 8.84 ms
─────────────────────────────
Total : 47.28 ms
http://no.such.host.invalid (FAILED)
Total : 4.54 ms
✗ dns: [Errno 8] nodename nor servname provided, or not known
https://self-signed.badssl.com (FAILED)
DNS lookup : 24.75 ms
TCP connect : 1126.05 ms
TLS handshake : 327.05 ms
─────────────────────────────
Total : 1478.35 ms
✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1081)
─────────────────────────────────────────────────
Summary: 3 URLs — 1 ok, 2 failed
✗ dns : 1
✗ tls : 1
→ exit 5 (tls)
exit: 5
```
Each URL is classified by its own worst sample (same `_phase_code` mapping the
overall exit code uses), then tallied. The `→ exit N (label)` line ties the
tally directly to the process exit code, since the code alone can't show
*which* URLs failed *how* — here a DNS failure (would be exit `2` alone) is
outranked by the TLS failure (`5`), and the footer is what makes that visible.
JSON output (`--json`) is unaffected — it stays a bare array; per-URL failures
are already in each entry's `errors[]`.
---
## Reading URLs from a file (`-f`)
```sh
uv run python -m hxprobe -f configs/all-ok.txt
```
```
https://example.com (200)
DNS lookup : 3.60 ms
TCP connect : 17.61 ms
TLS handshake : 16.39 ms
Server (TTFB) : 4.06 ms
Transfer : 0.60 ms
─────────────────────────────
Total : 56.24 ms
https://www.google.com (200)
DNS lookup : 4.18 ms
TCP connect : 17.23 ms
TLS handshake : 30.62 ms
Server (TTFB) : 2.68 ms
Transfer : 69.83 ms
─────────────────────────────
Total : 142.95 ms
https://www.iana.org (200)
DNS lookup : 4.25 ms
TCP connect : 17.18 ms
TLS handshake : 18.87 ms
Server (TTFB) : 8.13 ms
Transfer : 0.50 ms
─────────────────────────────
Total : 60.76 ms
─────────────────────────────────────────────────
Summary: 3 URLs — 3 ok
→ exit 0 (ok)
```
`-f`/`--file` reads a plain-text URL list — one per line, blank lines and
`#`-prefixed comment lines skipped — the same format `simple.py`/`phases.py`
use elsewhere in this repo. It's **mutually exclusive** with positional URL
arguments: pass one or the other, not both.
```sh
uv run python -m hxprobe -f configs/all-ok.txt https://extra.example.com
```
```
usage: hxprobe [-h] [-f PATH] [-n N] [-c N] [--timeout DURATION] [--fail]
[--json] [-v] [--no-http2] [--no-follow-redirects]
[url ...]
hxprobe: error: cannot combine positional url arguments with -f/--file
```
`configs/` ships one fixture per failure class, each self-documenting its
expected exit code in a header comment (verified by actually running it,
not just asserted):
| File | Demonstrates | Exit code |
|------|---------------|-----------|
| `configs/all-ok.txt` | Everything succeeds | 0 |
| `configs/dns-failure.txt` | Unresolvable hostnames | 2 |
| `configs/connection-refused.txt` | Loopback ports with no listener | 3 |
| `configs/timeout.txt` | Non-routable IPs (RFC 5737 TEST-NET-1) | 4 |
| `configs/tls-errors.txt` | badssl.com cert failures | 5 |
| `configs/http-errors.txt` | 404s, run with `--fail` | 6 |
| `configs/mixed.txt` | One of each class, run with `--fail` | 6 |
| `configs/large-mixed.txt` | 30 URLs, ≥10 failing across all 5 classes — good demo of the multi-URL summary footer | 6 |
```sh
uv run python -m hxprobe -f configs/dns-failure.txt
echo "exit: $?"
```
```
https://this-host-does-not-exist.invalid (FAILED)
Total : 16.91 ms
✗ dns: [Errno 8] nodename nor servname provided, or not known
http://no.such.host.invalid (FAILED)
Total : 4.56 ms
✗ dns: [Errno 8] nodename nor servname provided, or not known
─────────────────────────────────────────────────
Summary: 2 URLs — 0 ok, 2 failed
✗ dns : 2
→ exit 2 (dns)
exit: 2
```
`configs/large-mixed.txt` scales this up to 30 URLs so the summary footer has
something substantial to tally — 20 real sites expected to succeed plus 10
deliberately broken across all five failure classes at once:
```sh
uv run python -m hxprobe --fail --timeout 2s -f configs/large-mixed.txt
echo "exit: $?"
```
```
https://example.com (200)
DNS lookup : 3.96 ms
TCP connect : 10.24 ms
TLS handshake : 14.76 ms
Server (TTFB) : 4.14 ms
Transfer : 3.71 ms
─────────────────────────────
Total : 49.54 ms
... 18 more successful URLs ...
https://stackoverflow.com (200)
DNS lookup : 12.99 ms
TCP connect : 9.00 ms
TLS handshake : 16.58 ms
Server (TTFB) : 197.72 ms
Transfer : 289.92 ms
─────────────────────────────
Total : 684.24 ms
https://this-host-does-not-exist.invalid (FAILED)
Total : 1.25 ms
✗ dns: [Errno 8] nodename nor servname provided, or not known
http://no.such.host.invalid (FAILED)
Total : 0.92 ms
✗ dns: [Errno 8] nodename nor servname provided, or not known
http://127.0.0.1:9999 (FAILED)
DNS lookup : 0.01 ms
TCP connect : 0.09 ms
─────────────────────────────
Total : 0.26 ms
✗ connect: [Errno 61] Connection refused
http://127.0.0.1:19999 (FAILED)
DNS lookup : 0.01 ms
TCP connect : 0.09 ms
─────────────────────────────
Total : 0.23 ms
✗ connect: [Errno 61] Connection refused
http://10.255.255.1/ (FAILED)
DNS lookup : 0.03 ms
TCP connect : 16.50 ms
─────────────────────────────
Total : 16.64 ms
✗ connect: [Errno 61] Connection refused
http://192.0.2.1/ (FAILED)
DNS lookup : 0.01 ms
TCP connect : 2001.27 ms
─────────────────────────────
Total : 2001.56 ms
✗ timeout: timed out
https://expired.badssl.com/ (FAILED)
DNS lookup : 16.37 ms
TCP connect : 1125.42 ms
TLS handshake : 329.12 ms
─────────────────────────────
Total : 1471.48 ms
✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1081)
https://self-signed.badssl.com/ (FAILED)
DNS lookup : 13.27 ms
TCP connect : 1127.27 ms
TLS handshake : 325.84 ms
─────────────────────────────
Total : 1466.90 ms
✗ tls: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1081)
https://www.google.com/this-page-does-not-exist-at-all-1234567890 (404 ✗)
DNS lookup : 0.96 ms
TCP connect : 7.61 ms
TLS handshake : 25.25 ms
Server (TTFB) : 0.01 ms
Transfer : 99.63 ms
─────────────────────────────
Total : 142.38 ms
https://github.com/this-repo-does-not-exist-abcxyz123/no-way (404 ✗)
DNS lookup : 1.23 ms
TCP connect : 19.57 ms
TLS handshake : 22.02 ms
Server (TTFB) : 209.47 ms
Transfer : 67.20 ms
─────────────────────────────
Total : 339.82 ms
─────────────────────────────────────────────────
Summary: 30 URLs — 20 ok, 10 failed
✗ dns : 2
✗ connect : 3
✗ timeout : 1
✗ tls : 2
✗ http : 2
→ exit 6 (http)
exit: 6
```
The individual URL blocks above are unchanged in shape from every other
example on this page — this is purely a matter of scale. The summary footer
is where scale actually pays off: instead of scanning 30 blocks to count
outcomes, `Summary: 30 URLs — 20 ok, 10 failed` plus the per-class breakdown
answers "what happened" at a glance, and `→ exit 6 (http)` explains *why*
that particular exit code came back (the two 404s under `--fail`, at
`EXIT_HTTP = 6`, outrank every other class present).
---
## `--fail` flag — exit non-zero on HTTP 4xx/5xx
```sh
uv run python -m hxprobe --fail https://www.google.com/this-page-does-not-exist-at-all-1234567890
echo "exit: $?"
```
```
https://www.google.com/this-page-does-not-exist-at-all-1234567890 (404 ✗)
DNS lookup : 2.96 ms
TCP connect : 11.57 ms
TLS handshake : 24.95 ms
Server (TTFB) : 0.01 ms
Transfer : 104.22 ms
─────────────────────────────
Total : 152.77 ms
exit: 6
```
Without `--fail`, 4xx/5xx responses are shown normally and the exit code is `0`.
---
## JSON output
```sh
uv run python -m hxprobe --json https://example.com | python3.14 -m json.tool
```
```json
[
{
"url": "https://example.com",
"status": 200,
"succeeded": 1,
"failed": 0,
"phases": {
"dns": {"min_ms": 2.80, "avg_ms": 2.80, "max_ms": 2.80},
"connect": {"min_ms": 11.68, "avg_ms": 11.68, "max_ms": 11.68},
"tls": {"min_ms": 14.52, "avg_ms": 14.52, "max_ms": 14.52},
"ttfb": {"min_ms": 6.49, "avg_ms": 6.49, "max_ms": 6.49},
"transfer": {"min_ms": 0.81, "avg_ms": 0.81, "max_ms": 0.81},
"total": {"min_ms": 45.59, "avg_ms": 45.59, "max_ms": 45.59}
}
}
]
```
Same shape as `latprobe`'s JSON — `phases` is omitted when all samples
failed, `tls` is omitted for `http://` URLs.
---
## JSON + verbose
```sh
uv run python -m hxprobe --verbose --json https://example.com
```
```json
[
{
"url": "https://example.com",
"status": 200,
"succeeded": 1,
"failed": 0,
"phases": { "...": "..." },
"verbose": {
"ip": "104.20.23.154",
"http_version": "HTTP/2",
"redirect_count": 0,
"tls_version": "TLSv1.3",
"tls_cipher": "TLS_AES_256_GCM_SHA384",
"tls_bits": 256,
"cert": {
"cn": "example.com",
"sans": ["example.com", "*.example.com"],
"expiry": "2026-08-29",
"issuer_cn": "SSL Corporation",
"verified": true
},
"headers": {
"content-type": "text/html",
"server": "cloudflare",
"cf-cache-status": "HIT"
}
}
}
]
```
`"http_version"`/`"redirect_count"` are the two keys `latprobe`'s JSON never
has (its `VerboseDetail.http_version` stays `""`). `"headers"` contains
**all** parsed response headers (the text view shows only a priority list).
---
## Timeout
```sh
uv run python -m hxprobe --timeout 500ms http://192.0.2.1/
echo "exit: $?"
```
```
http://192.0.2.1/ (FAILED)
DNS lookup : 3.11 ms
TCP connect : 501.51 ms
─────────────────────────────
Total : 505.20 ms
✗ timeout: timed out
exit: 4
```
`192.0.2.1` is in RFC 5737's TEST-NET-1 range — reserved for documentation,
guaranteed unreachable, and the kernel gets no reply so the connect phase
runs the full `--timeout` before giving up. `--timeout` accepts `ms`, `s`,
`m` suffixes or a bare number of seconds.
---
## Exit codes
| Code | Meaning |
|------|---------|
| 0 | All probes succeeded (or HTTP 4xx without `--fail`) |
| 1 | Usage / argument error |
| 2 | DNS failure |
| 3 | TCP connect failure |
| 4 | Timeout |
| 5 | TLS error |
| 6 | HTTP status ≥ 400 with `--fail` |
The **highest** exit code across all URLs is returned as the process exit —
identical scheme to `latprobe` and the Go implementation. This scalar is kept
unchanged for compatibility; for multi-URL runs, the "Multi-URL summary
footer" section above shows the full per-URL breakdown behind it.
---
## Makefile shortcuts
From inside this directory:
```sh
make run ARGS="--verbose https://example.com" # run hxprobe
make test # hermetic tests only
make test-integration # live internet tests
make lint # ruff check
make fmt # ruff format
make check # lint + hermetic tests
```
From the parent repo's root, the equivalent shortcuts are prefixed `hx-`:
```sh
make hx-run ARGS="--verbose https://example.com"
make hx-test
make hx-test-integration
make hx-check
```
Both Makefiles are independent — neither calls into the other — so either
works whether you're inside a checkout of the full monorepo or a standalone
copy of just `hxprobe/`.

View File

@@ -0,0 +1,7 @@
# All sites are expected to respond successfully.
# Run: uv run python -m hxprobe -f configs/all-ok.txt
# Expected exit code: 0
https://example.com
https://www.google.com
https://www.iana.org

View File

@@ -0,0 +1,7 @@
# Connection-refused errors — loopback addresses with no server on those ports.
# The OS rejects the TCP SYN immediately, so these fail in milliseconds.
# Run: uv run python -m hxprobe -f configs/connection-refused.txt
# Expected exit code: 3
http://127.0.0.1:9999
http://127.0.0.1:19999

View File

@@ -0,0 +1,9 @@
# DNS resolution failures — hostnames that cannot be resolved.
# .invalid is an IANA-reserved TLD guaranteed never to resolve (RFC 2606).
# Run: uv run python -m hxprobe -f configs/dns-failure.txt
# Expected exit code: 2
https://this-host-does-not-exist.invalid
http://no.such.host.invalid
# Also exercises the blank-line and comment-line parser paths.

View File

@@ -0,0 +1,22 @@
# HTTP error status codes — without --fail, hxprobe treats 4xx/5xx as a
# normal (successful) probe outcome, matching Go's http.DefaultClient
# semantics. --fail opts back into "HTTP error = failure", which is what
# this fixture demonstrates. The URLs below are real paths that reliably
# return 404 on stable public servers.
#
# Note: a genuine 500 from a well-known server is hard to provoke on demand.
# These 404s are sufficient to demonstrate --fail's effect on the exit code.
#
# Requires internet access.
#
# Run: uv run python -m hxprobe --fail -f configs/http-errors.txt
# Expected exit code: 6 (0 without --fail — the requests still succeed)
# 404 from Google
https://www.google.com/this-page-does-not-exist-at-all-1234567890
# 404 from GitHub
https://github.com/this-repo-does-not-exist-abcxyz123/no-way
# 404 from IANA
https://www.iana.org/this-page-does-not-exist-either

View File

@@ -0,0 +1,67 @@
# Large mixed run — 30 URLs, 20 expected to succeed and 10 to fail, covering
# all five failure classes (dns/connect/timeout/tls/http) at once. A bigger
# sibling of mixed.txt, and a good demo of the multi-URL summary footer
# (only shown when more than one URL is probed) — the footer tallies exactly
# how many of the 10 fall into each class, since the single worst exit code
# can't show that on its own.
#
# --timeout 2s shortens the two timeout entries from the 10s default; --fail
# is required for the two 404s to count as failures (without it they're
# "successful" 404 responses, per Go http.DefaultClient semantics, and only
# 8 of the 30 would fail).
#
# Requires internet access.
#
# Run: uv run python -m hxprobe --fail --timeout 2s -f configs/large-mixed.txt
# Expected exit code: 6 (http wins — the highest class present)
# Expected: at least 10 of 30 fail, spanning all 5 classes. The *exact*
# per-class mix can shift with network conditions — same caveat as
# tls-errors.txt (badssl.com may reset/timeout instead of a clean TLS error)
# and timeout.txt (a sandboxed network may refuse instantly instead of
# timing out); a couple of the "OK" sites were swapped out during testing
# for being unreliable in some sandboxes (Wikipedia's bot detection,
# httpbin.org's frequent overload). Verified by an actual run:
# 30 URLs — 20 ok, 10 failed (dns: 2, connect: 3, timeout: 1, tls: 2,
# http: 2) → exit 6.
# ── OK (20) ──────────────────────────────────────────────────────────────
https://example.com
https://example.org
https://example.net
https://www.google.com
https://www.iana.org
https://github.com
https://www.debian.org
https://www.postgresql.org
https://www.mozilla.org
https://developer.mozilla.org
https://www.python.org
https://pypi.org
https://www.cloudflare.com
https://www.apache.org
https://www.rust-lang.org
https://go.dev
https://nodejs.org
https://www.w3.org
https://www.ietf.org
https://stackoverflow.com
# ── DNS failure (2) — .invalid is IANA-reserved, RFC 2606 ──────────────────
https://this-host-does-not-exist.invalid
http://no.such.host.invalid
# ── Connection refused (2) — loopback ports with no listener ───────────────
http://127.0.0.1:9999
http://127.0.0.1:19999
# ── Timeout (2) — RFC 5737 TEST-NET-1, non-routable ─────────────────────────
http://10.255.255.1/
http://192.0.2.1/
# ── TLS certificate errors (2) — badssl.com ─────────────────────────────────
https://expired.badssl.com/
https://self-signed.badssl.com/
# ── HTTP errors (2) — real 404s, need --fail to count as failures ──────────
https://www.google.com/this-page-does-not-exist-at-all-1234567890
https://github.com/this-repo-does-not-exist-abcxyz123/no-way

23
hxprobe/configs/mixed.txt Normal file
View File

@@ -0,0 +1,23 @@
# Mixed — one entry from each error class alongside a successful site.
# Shows that OK and FAIL lines can interleave in the same run, and that the
# exit code is the *worst* code across every URL (2=dns, 3=connect, 5=tls,
# 6=http-with-fail — 6 wins here). Timeout is omitted so the run completes
# in a few seconds; --fail is needed for the 404 line to count as a failure.
#
# Run: uv run python -m hxprobe --fail -f configs/mixed.txt
# Expected exit code: 6
# Success
https://example.com
# DNS failure
https://no.such.host.invalid
# Connection refused (loopback, no server)
http://127.0.0.1:9999
# TLS certificate error
https://self-signed.badssl.com/
# HTTP error (server responded with 404; needs --fail to count as a failure)
https://github.com/this-repo-does-not-exist-abcxyz123/no-way

View File

@@ -0,0 +1,16 @@
# Timeout — non-routable IP addresses that accept no TCP traffic.
# The kernel sends a SYN but never gets a reply; hxprobe waits the full
# --timeout per host before giving up. Default --timeout is 10s (~20s total
# for two hosts); --timeout 2s below shortens the demo.
#
# 10.255.255.1 and 192.0.2.1 (RFC 5737 TEST-NET-1, documentation-only range)
# are guaranteed to be unreachable on any normal network. (In some sandboxed
# dev environments one of these may instead get an immediate "connection
# refused" from the network layer rather than a true timeout — if that
# happens here, the other target still demonstrates the timeout path.)
#
# Run: uv run python -m hxprobe --timeout 2s -f configs/timeout.txt
# Expected exit code: 4
http://10.255.255.1/
http://192.0.2.1/

View File

@@ -0,0 +1,22 @@
# TLS certificate errors — badssl.com provides endpoints with intentionally
# broken certificates. hxprobe uses the same stdlib ssl verification as
# latprobe and rejects them by default.
#
# Note: badssl.com may intermittently reset the connection instead of
# completing the TLS handshake. The error message will then read
# "[Errno 54] Connection reset by peer" rather than CERTIFICATE_VERIFY_FAILED,
# but hxprobe still correctly reports a tls/connect failure in either case.
#
# Requires internet access.
#
# Run: uv run python -m hxprobe -f configs/tls-errors.txt
# Expected exit code: 5
# Certificate has expired
https://expired.badssl.com/
# Certificate is self-signed (not trusted by the system CA store)
https://self-signed.badssl.com/
# Certificate chain is incomplete (intermediate CA missing)
https://incomplete-chain.badssl.com/

View File

@@ -0,0 +1,7 @@
"""hxprobe — httpx-based per-phase HTTP latency probe.
Uses the httpx library so the client matches Go's http.DefaultClient: HTTP/2
via ALPN, redirects followed by default, and connection pooling — while
still reporting a six-phase breakdown (DNS, TCP connect, TLS, TTFB, Transfer,
Total) by instrumenting httpx's network backend directly.
"""

View File

@@ -0,0 +1,5 @@
import sys
from .cli import run
sys.exit(run(sys.argv[1:], sys.stdout, sys.stderr))

View File

@@ -0,0 +1,53 @@
from __future__ import annotations
from dataclasses import dataclass, field
from .probe import Result
@dataclass
class PhaseStats:
min_ms: float = 0.0
avg_ms: float = 0.0
max_ms: float = 0.0
present: bool = False
@dataclass
class Aggregate:
url: str = ""
count: int = 0
status_code: int = 0
dns: PhaseStats = field(default_factory=PhaseStats)
connect: PhaseStats = field(default_factory=PhaseStats)
tls: PhaseStats = field(default_factory=PhaseStats)
ttfb: PhaseStats = field(default_factory=PhaseStats)
transfer: PhaseStats = field(default_factory=PhaseStats)
total: PhaseStats = field(default_factory=PhaseStats)
def summarize(results: list[Result]) -> Aggregate:
"""Compute per-phase min/avg/max over a list of succeeded Results."""
if not results:
return Aggregate()
a = Aggregate(url=results[0].url, count=len(results))
a.status_code = results[-1].status_code
def _stats(attr: str) -> PhaseStats:
vals = [getattr(r, attr).ms for r in results if getattr(r, attr).present]
if not vals:
return PhaseStats()
return PhaseStats(
min_ms=min(vals),
avg_ms=sum(vals) / len(vals),
max_ms=max(vals),
present=True,
)
a.dns = _stats("dns")
a.connect = _stats("connect")
a.tls = _stats("tls")
a.ttfb = _stats("ttfb")
a.transfer = _stats("transfer")
a.total = _stats("total")
return a

523
hxprobe/hxprobe/cli.py Normal file
View File

@@ -0,0 +1,523 @@
from __future__ import annotations
import argparse
import concurrent.futures
import contextlib
import datetime
import json
from typing import IO
from .aggregate import Aggregate, PhaseStats, summarize
from .duration import parse_duration
from .probe import Options, Result, VerboseDetail, measure
# ── exit codes ────────────────────────────────────────────────────────────────
EXIT_OK = 0
EXIT_USAGE = 1
EXIT_DNS = 2
EXIT_CONNECT = 3
EXIT_TIMEOUT = 4
EXIT_TLS = 5
EXIT_HTTP = 6
_PHASE_EXIT: dict[str, int] = {
"dns": EXIT_DNS,
"timeout": EXIT_TIMEOUT,
"tls": EXIT_TLS,
}
def _phase_code(fail_phase: str) -> int:
return _PHASE_EXIT.get(fail_phase, EXIT_CONNECT)
# Labels for the end-of-run summary footer — inverse of _PHASE_EXIT plus the
# two codes it doesn't cover (EXIT_OK, EXIT_HTTP via --fail).
_EXIT_LABELS: dict[int, str] = {
EXIT_OK: "ok",
EXIT_DNS: "dns",
EXIT_CONNECT: "connect",
EXIT_TIMEOUT: "timeout",
EXIT_TLS: "tls",
EXIT_HTTP: "http",
}
# ── display constants ─────────────────────────────────────────────────────────
_SINGLE_SEP = " " + "" * 29
_AGG_SEP = " " + "" * 49
_PHASE_LABELS = [
("dns", "DNS lookup "),
("connect", "TCP connect "),
("tls", "TLS handshake "),
("ttfb", "Server (TTFB) "),
("transfer", "Transfer "),
]
# Response headers shown in text verbose block, in priority order.
# In JSON verbose mode all parsed headers are included.
_VERBOSE_HEADERS = [
"Location",
"Server",
"Content-Type",
"X-Cache",
"CF-Cache-Status",
"Cache-Control",
"Via",
"X-Powered-By",
]
# ── sampling ──────────────────────────────────────────────────────────────────
def _run_samples(url: str, count: int, opts: Options) -> tuple[list[Result], list[Result]]:
succeeded, failed = [], []
for _ in range(count):
r = measure(url, opts)
(failed if r.err else succeeded).append(r)
return succeeded, failed
# ── verbose rendering ─────────────────────────────────────────────────────────
def _cert_expired(expiry: str) -> bool:
try:
return datetime.date.fromisoformat(expiry) < datetime.date.today()
except (ValueError, TypeError):
return False
def _print_verbose_block(detail: VerboseDetail, out: IO) -> None:
"""Write the verbose detail block (IP, protocol, TLS, cert, headers) after timing rows."""
def _row(label: str, value: str) -> None:
out.write(f" {label:<14} : {value}\n")
if detail.resolved_ip:
_row("IP", detail.resolved_ip)
if detail.http_version:
proto = detail.http_version
if detail.redirect_count:
proto += (
f" ({detail.redirect_count} redirect{'s' if detail.redirect_count != 1 else ''})"
)
_row("Protocol", proto)
if detail.tls_version:
parts = [detail.tls_version]
if detail.tls_cipher:
parts.append(detail.tls_cipher)
if detail.tls_bits:
parts.append(f"{detail.tls_bits} bit")
_row("TLS", " ".join(parts))
if detail.cert:
c = detail.cert
label = "Cert (unvrf.)" if not c.verified else "Cert"
cert_parts = []
if c.cn:
cert_parts.append(f"CN={c.cn}")
if c.expiry:
tag = "EXPIRED" if _cert_expired(c.expiry) else "valid until"
cert_parts.append(f"{tag} {c.expiry}")
if c.issuer_cn:
cert_parts.append(c.issuer_cn)
_row(label, " ".join(cert_parts))
for name in _VERBOSE_HEADERS:
value = detail.headers.get(name)
if value:
_row(name, value)
# ── text rendering ────────────────────────────────────────────────────────────
def _status_str(status_code: int, fail_flag: bool) -> str:
s = str(status_code)
if fail_flag and status_code >= 400:
s += ""
return s
def _print_single(r: Result, fail_flag: bool, out: IO) -> None:
status = _status_str(r.status_code, fail_flag)
out.write(f"{r.url} ({status})\n")
for attr, label in _PHASE_LABELS:
ph = getattr(r, attr)
if ph.present:
out.write(f" {label} : {ph.ms:8.2f} ms\n")
out.write(_SINGLE_SEP + "\n")
if r.total.present:
out.write(f" {'Total '} : {r.total.ms:8.2f} ms\n")
if r.err is not None:
out.write(f"{r.fail_phase}: {r.err}\n")
if r.detail is not None:
_print_verbose_block(r.detail, out)
def _print_aggregate(
agg: Aggregate,
failed: list[Result],
fail_flag: bool,
out: IO,
detail: VerboseDetail | None = None,
) -> None:
status = _status_str(agg.status_code, fail_flag)
header = f"{agg.url} ({status}, {agg.count} samples"
if failed:
header += f", {len(failed)} failed"
out.write(header + ")\n")
if agg.total.present:
out.write(f" {'':14} {'min':>9} {'avg':>9} {'max':>9}\n")
for attr, label in _PHASE_LABELS:
ps: PhaseStats = getattr(agg, attr)
if ps.present:
out.write(
f" {label} : {ps.min_ms:6.2f} ms {ps.avg_ms:6.2f} ms {ps.max_ms:6.2f} ms\n"
)
out.write(_AGG_SEP + "\n")
out.write(
f" {'Total '} : {agg.total.min_ms:6.2f} ms"
f" {agg.total.avg_ms:6.2f} ms"
f" {agg.total.max_ms:6.2f} ms\n"
)
_print_failure_summary(failed, out)
if detail is not None:
_print_verbose_block(detail, out)
def _print_all_failed(url: str, failed: list[Result], total_count: int, out: IO) -> None:
header = f"{url} (FAILED"
if total_count > 1:
header += f", 0/{total_count} succeeded"
out.write(header + ")\n")
last = failed[-1]
any_phase = False
for attr, label in _PHASE_LABELS:
ph = getattr(last, attr)
if ph.present:
out.write(f" {label} : {ph.ms:8.2f} ms\n")
any_phase = True
if last.total.present:
if any_phase:
out.write(_SINGLE_SEP + "\n")
out.write(f" {'Total '} : {last.total.ms:8.2f} ms\n")
_print_failure_summary(failed, out)
if last.detail is not None:
_print_verbose_block(last.detail, out)
def _summarize_failures(failed: list[Result]) -> list[tuple[str, str, int]]:
"""Dedupe failures into (phase, message, count), preserving first-seen order."""
counts: dict[tuple[str, str], int] = {}
order: list[tuple[str, str]] = []
for r in failed:
key = (r.fail_phase, str(r.err))
if key not in counts:
order.append(key)
counts[key] = 0
counts[key] += 1
return [(phase, msg, counts[(phase, msg)]) for phase, msg in order]
def _print_failure_summary(failed: list[Result], out: IO) -> None:
for phase, msg, n in _summarize_failures(failed):
if n == 1:
out.write(f"{phase}: {msg}\n")
else:
out.write(f"{n} × {phase}: {msg}\n")
def _print_run_summary(urls: list[str], url_codes: list[int], worst: int, out: IO) -> None:
"""End-of-run footer tallying every URL's outcome — only called for
multi-URL runs, where the single worst-code exit can't show the mix of
failure classes behind it."""
n_ok = sum(1 for c in url_codes if c == EXIT_OK)
n_failed = len(urls) - n_ok
out.write("\n" + _AGG_SEP + "\n")
header = f" Summary: {len(urls)} URLs — {n_ok} ok"
if n_failed:
header += f", {n_failed} failed"
out.write(header + "\n")
if n_failed:
counts: dict[int, int] = {}
order: list[int] = []
for c in url_codes:
if c == EXIT_OK:
continue
if c not in counts:
order.append(c)
counts[c] = 0
counts[c] += 1
for c in order:
label = _EXIT_LABELS.get(c, str(c))
out.write(f"{label:<8}: {counts[c]}\n")
out.write(f" → exit {worst} ({_EXIT_LABELS.get(worst, str(worst))})\n")
def _print_url(
url: str,
succeeded: list[Result],
failed: list[Result],
total_count: int,
fail_flag: bool,
out: IO,
) -> None:
n_ok = len(succeeded)
n_fail = len(failed)
last_detail = succeeded[-1].detail if succeeded else None
if n_fail == 0 and total_count == 1:
_print_single(succeeded[0], fail_flag, out)
elif n_fail == 0:
_print_aggregate(summarize(succeeded), [], fail_flag, out, last_detail)
elif n_ok == 0:
_print_all_failed(url, failed, total_count, out)
else:
_print_aggregate(summarize(succeeded), failed, fail_flag, out, last_detail)
# ── JSON rendering ────────────────────────────────────────────────────────────
def _build_json_entry(
url: str,
succeeded: list[Result],
failed: list[Result],
detail: VerboseDetail | None = None,
) -> dict:
entry: dict = {
"url": url,
"status": 0,
"succeeded": len(succeeded),
"failed": len(failed),
}
if succeeded:
agg = summarize(succeeded)
entry["status"] = agg.status_code
phases: dict = {}
for attr in ("dns", "connect", "tls", "ttfb", "transfer", "total"):
ps: PhaseStats = getattr(agg, attr)
if ps.present:
phases[attr] = {
"min_ms": ps.min_ms,
"avg_ms": ps.avg_ms,
"max_ms": ps.max_ms,
}
if phases:
entry["phases"] = phases
if failed:
entry["errors"] = [
{"phase": phase, "count": n, "message": msg}
for phase, msg, n in _summarize_failures(failed)
]
if detail is not None:
v: dict = {}
if detail.resolved_ip:
v["ip"] = detail.resolved_ip
if detail.http_version:
v["http_version"] = detail.http_version
v["redirect_count"] = detail.redirect_count
if detail.tls_version:
v["tls_version"] = detail.tls_version
v["tls_cipher"] = detail.tls_cipher
v["tls_bits"] = detail.tls_bits
if detail.cert:
c = detail.cert
v["cert"] = {
"cn": c.cn,
"sans": c.sans,
"expiry": c.expiry,
"issuer_cn": c.issuer_cn,
"verified": c.verified,
}
if detail.headers:
v["headers"] = dict(detail.headers)
if v:
entry["verbose"] = v
return entry
# ── URL sources ───────────────────────────────────────────────────────────────
def _load_urls(path: str) -> list[str]:
"""Read URLs from a plain-text file: one per line, '#' comments and blank
lines skipped."""
urls: list[str] = []
with open(path) as f:
for raw in f:
line = raw.strip()
if not line or line.startswith("#"):
continue
urls.append(line)
return urls
# ── entry point ───────────────────────────────────────────────────────────────
def run(args: list[str], stdout: IO, stderr: IO) -> int:
parser = argparse.ArgumentParser(
prog="hxprobe",
description=(
"measure per-phase HTTP request latency via httpx "
"(HTTP/2, redirects, connection pooling — matches Go's http.DefaultClient)"
),
)
parser.add_argument("urls", nargs="*", metavar="url")
parser.add_argument(
"-f",
"--file",
metavar="PATH",
help="read URLs from a file, one per line, '#' comments allowed "
"(mutually exclusive with positional url args)",
)
parser.add_argument(
"-n",
"--count",
type=int,
default=1,
metavar="N",
help="number of requests per URL (default: 1)",
)
parser.add_argument(
"-c",
"--concurrency",
type=int,
default=0,
metavar="N",
help="max parallel URLs, 0=auto (default: 0)",
)
parser.add_argument(
"--timeout",
default="10s",
metavar="DURATION",
help="per-request timeout, e.g. 10s, 500ms (default: 10s)",
)
parser.add_argument(
"--fail",
action="store_true",
help="exit non-zero on HTTP status >= 400",
)
parser.add_argument(
"--json",
action="store_true",
dest="json_out",
help="output results as JSON instead of text",
)
parser.add_argument(
"-v",
"--verbose",
action="store_true",
help="show resolved IP, negotiated protocol/redirects, TLS version/cipher, "
"certificate, and response headers",
)
parser.add_argument(
"--no-http2",
action="store_false",
dest="http2",
default=True,
help="disable HTTP/2 negotiation, force HTTP/1.1",
)
parser.add_argument(
"--no-follow-redirects",
action="store_false",
dest="follow_redirects",
default=True,
help="do not follow HTTP redirects",
)
try:
with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr):
ns = parser.parse_args(args)
if ns.urls and ns.file:
parser.error("cannot combine positional url arguments with -f/--file")
if not ns.urls and not ns.file:
parser.error("no URLs given (pass as arguments or with -f/--file)")
except SystemExit as exc:
return EXIT_OK if not exc.code else EXIT_USAGE
try:
timeout_secs = parse_duration(ns.timeout)
except ValueError:
stderr.write(f"hxprobe: error: invalid timeout: {ns.timeout!r}\n")
return EXIT_USAGE
if ns.file:
try:
urls = _load_urls(ns.file)
except OSError as exc:
stderr.write(f"hxprobe: error: cannot read {ns.file!r}: {exc}\n")
return EXIT_USAGE
if not urls:
stderr.write(f"hxprobe: error: no URLs found in {ns.file!r}\n")
return EXIT_USAGE
else:
urls = ns.urls
opts = Options(
timeout=timeout_secs,
verbose=ns.verbose,
follow_redirects=ns.follow_redirects,
http2=ns.http2,
)
count = ns.count
workers = ns.concurrency
if workers <= 0:
workers = min(len(urls), 8)
workers = max(1, min(workers, len(urls)))
def _probe(url: str) -> tuple[list[Result], list[Result]]:
return _run_samples(url, count, opts)
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as ex:
all_results = list(ex.map(_probe, urls))
worst = EXIT_OK
json_items = []
url_codes: list[int] = []
for i, (url, (succeeded, failed)) in enumerate(zip(urls, all_results)):
code = EXIT_OK
for r in failed:
code = max(code, _phase_code(r.fail_phase))
if ns.fail:
for r in succeeded:
if r.status_code >= 400:
code = max(code, EXIT_HTTP)
url_codes.append(code)
worst = max(worst, code)
last_detail = succeeded[-1].detail if succeeded else (failed[-1].detail if failed else None)
if ns.json_out:
json_items.append(_build_json_entry(url, succeeded, failed, last_detail))
continue
if i > 0:
stdout.write("\n")
_print_url(url, succeeded, failed, count, ns.fail, stdout)
if ns.json_out:
stdout.write(json.dumps(json_items, indent=2) + "\n")
elif len(urls) > 1:
_print_run_summary(urls, url_codes, worst, stdout)
return worst

View File

@@ -0,0 +1,14 @@
def parse_duration(s: str) -> float:
"""Parse a duration string to seconds.
Suffixes: ms (milliseconds), s (seconds), m (minutes).
A bare number is treated as seconds.
"""
s = s.strip()
if s.endswith("ms"):
return float(s[:-2]) / 1000
if s.endswith("s"):
return float(s[:-1])
if s.endswith("m"):
return float(s[:-1]) * 60
return float(s)

458
hxprobe/hxprobe/probe.py Normal file
View File

@@ -0,0 +1,458 @@
from __future__ import annotations
import datetime
import socket
import ssl
import time
import urllib.parse
from dataclasses import dataclass, field
import httpcore
import httpx
# ── verbose detail dataclasses ────────────────────────────────────────────────
@dataclass
class CertInfo:
cn: str = ""
sans: list[str] = field(default_factory=list)
expiry: str = "" # ISO date "YYYY-MM-DD"
issuer_cn: str = ""
verified: bool = False # True = TLS handshake passed verification
@dataclass
class VerboseDetail:
resolved_ip: str = ""
tls_version: str = ""
tls_cipher: str = ""
tls_bits: int = 0
cert: CertInfo | None = None
headers: dict[str, str] = field(default_factory=dict) # all parsed response headers
http_version: str = ""
redirect_count: int = 0
# ── core dataclasses ──────────────────────────────────────────────────────────
@dataclass
class Options:
timeout: float = 10.0
verbose: bool = False
follow_redirects: bool = True
http2: bool = True
@dataclass
class Phase:
ms: float = 0.0
present: bool = False
@dataclass
class Result:
url: str
dns: Phase = field(default_factory=Phase)
connect: Phase = field(default_factory=Phase)
tls: Phase = field(default_factory=Phase)
ttfb: Phase = field(default_factory=Phase)
transfer: Phase = field(default_factory=Phase)
total: Phase = field(default_factory=Phase)
status_code: int = 0
fail_phase: str = ""
err: Exception | None = None
detail: VerboseDetail | None = None # populated only when opts.verbose=True
# ── cert parsing helpers ──────────────────────────────────────────────────────
def _parse_cert_date(s: str) -> str:
"""Convert SSL cert date 'Jun 14 00:00:00 2025 GMT''2025-06-14'."""
if not s:
return ""
s = " ".join(s.split()) # collapse double-spaces ("May 5 ..." → "May 5 ...")
for fmt in ("%b %d %H:%M:%S %Y %Z", "%b %d %H:%M:%S %Y"):
try:
return datetime.datetime.strptime(s, fmt).strftime("%Y-%m-%d")
except ValueError:
pass
return ""
def _parse_cert(peer: dict, *, verified: bool) -> CertInfo:
"""Build CertInfo from the dict returned by SSLSocket.getpeercert()."""
def _attr(rdns, key: str) -> str:
for rdn in rdns:
for k, v in rdn:
if k == key:
return v
return ""
issuer = peer.get("issuer", ())
# Prefer organizationName for issuer (more human-readable than CA CN)
issuer_cn = _attr(issuer, "organizationName") or _attr(issuer, "commonName")
return CertInfo(
cn=_attr(peer.get("subject", ()), "commonName"),
sans=[v for k, v in peer.get("subjectAltName", ()) if k == "DNS"],
expiry=_parse_cert_date(peer.get("notAfter", "")),
issuer_cn=issuer_cn,
verified=verified,
)
# ── per-call trace ────────────────────────────────────────────────────────────
#
# httpx has no equivalent of Go's net/http/httptrace, so per-phase timing is
# recovered by instrumenting httpcore's NetworkBackend/NetworkStream directly
# (see _TimingBackend/_TimingStream below). One _Trace instance is created per
# measure() call and threaded through the custom backend.
#
# Semantics when redirects are followed:
# - dns / connect / tls / resolved_ip / tls_* / cert — reported from the
# FIRST connection only (first-hop-wins), mirroring the "cost of reaching
# the origin server" that Go's connectStart.IsZero() guard preserves.
# - wrote_request / first_byte (and therefore ttfb / transfer) — reflect the
# LAST hop, since each write()/read() call overwrites them. This matches
# Go's own httptrace.ClientTrace behavior: WroteRequest/GotFirstResponseByte
# fire on every redirect hop and the last one wins, because Go's hooks
# aren't guarded either.
class _Trace:
def __init__(self) -> None:
self.dns = Phase()
self.connect = Phase()
self.tls = Phase()
self.resolved_ip = ""
self.tls_version = ""
self.tls_cipher = ""
self.tls_bits = 0
self.cert_peer: dict | None = None
self.wrote_request: float | None = None
self.first_byte: float | None = None
self.fail_phase = ""
self._tls_t0 = 0.0
def mark_dns(self, start: float, end: float, *, fail: bool = False) -> None:
# On failure, no phase is "present" for a lookup that never resolved —
# only total gets set (dns.present stays False).
if fail:
self.fail_phase = "dns"
return
if not self.dns.present:
self.dns = Phase(ms=(end - start) * 1000, present=True)
def set_resolved_ip(self, ip: str) -> None:
if not self.resolved_ip:
self.resolved_ip = ip
def mark_connect(self, start: float, end: float, *, fail: str | None = None) -> None:
if not self.connect.present:
self.connect = Phase(ms=(end - start) * 1000, present=True)
if fail:
self.fail_phase = fail
def mark_tls_start(self) -> None:
self._tls_t0 = time.perf_counter()
def finish_tls(self, ssl_sock: ssl.SSLSocket | None, *, fail: str | None = None) -> None:
if not self.tls.present:
self.tls = Phase(ms=(time.perf_counter() - self._tls_t0) * 1000, present=True)
if ssl_sock is not None:
self.tls_version = ssl_sock.version() or ""
cipher = ssl_sock.cipher()
if cipher:
self.tls_cipher, _, self.tls_bits = cipher
try:
peer = ssl_sock.getpeercert()
except Exception:
peer = None
if peer:
self.cert_peer = peer
if fail:
self.fail_phase = fail
def mark_wrote_request(self) -> None:
self.wrote_request = time.perf_counter()
def mark_first_byte(self) -> None:
self.first_byte = time.perf_counter()
# ── instrumented httpcore network backend ────────────────────────────────────
class _TimingStream(httpcore.NetworkStream):
"""Wraps a raw (or TLS) socket, stamping the trace on write/read/start_tls.
httpcore's own h11/h2 connection objects call read()/write()/start_tls()
regardless of HTTP version, so this wrapper is protocol-agnostic — HTTP/2
framing, redirects, and keep-alive are still entirely owned by httpx.
"""
def __init__(self, sock: socket.socket, trace: _Trace) -> None:
self._sock = sock
self._trace = trace
self._awaiting_first_byte = False
def read(self, max_bytes: int, timeout: float | None = None) -> bytes:
self._sock.settimeout(timeout)
try:
data = self._sock.recv(max_bytes)
except socket.timeout as exc:
raise httpcore.ReadTimeout(str(exc)) from exc
except OSError as exc:
raise httpcore.ReadError(str(exc)) from exc
if self._awaiting_first_byte:
self._awaiting_first_byte = False
self._trace.mark_first_byte()
return data
def write(self, buffer: bytes, timeout: float | None = None) -> None:
if not buffer:
return
self._sock.settimeout(timeout)
try:
self._sock.sendall(buffer)
except socket.timeout as exc:
raise httpcore.WriteTimeout(str(exc)) from exc
except OSError as exc:
raise httpcore.WriteError(str(exc)) from exc
self._trace.mark_wrote_request()
self._awaiting_first_byte = True
def close(self) -> None:
try:
self._sock.close()
except OSError:
pass
def start_tls(
self,
ssl_context: ssl.SSLContext,
server_hostname: str | None = None,
timeout: float | None = None,
) -> httpcore.NetworkStream:
# httpcore already set ALPN protocols (["http/1.1", "h2"] or just
# ["http/1.1"]) on ssl_context before calling start_tls — we just wrap.
self._trace.mark_tls_start()
self._sock.settimeout(timeout)
try:
tls_sock = ssl_context.wrap_socket(self._sock, server_hostname=server_hostname)
except socket.timeout as exc:
self._trace.finish_tls(None, fail="timeout")
raise httpcore.ConnectTimeout(str(exc)) from exc
except (ssl.SSLError, OSError) as exc:
self._trace.finish_tls(None, fail="tls")
raise httpcore.ConnectError(str(exc)) from exc
self._trace.finish_tls(tls_sock)
return _TimingStream(tls_sock, self._trace)
def get_extra_info(self, info: str):
if info == "ssl_object" and isinstance(self._sock, ssl.SSLSocket):
return self._sock
return None
class _TimingBackend(httpcore.NetworkBackend):
"""Splits DNS and TCP connect into two timed steps: httpcore's own
SyncBackend fuses them by calling socket.create_connection(), which does
getaddrinfo() and connect() internally with no way to time them apart."""
def __init__(self, trace: _Trace) -> None:
self._trace = trace
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
) -> httpcore.NetworkStream:
trace = self._trace
t0 = time.perf_counter()
try:
infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
except socket.gaierror as exc:
trace.mark_dns(t0, time.perf_counter(), fail=True)
raise httpcore.ConnectError(str(exc)) from exc
trace.mark_dns(t0, time.perf_counter())
addr = infos[0][4]
family = infos[0][0]
trace.set_resolved_ip(str(addr[0]))
sock = socket.socket(family, socket.SOCK_STREAM)
if local_address is not None:
sock.bind((local_address, 0))
sock.settimeout(timeout)
t1 = time.perf_counter()
try:
sock.connect(addr)
except socket.timeout as exc:
sock.close()
trace.mark_connect(t1, time.perf_counter(), fail="timeout")
raise httpcore.ConnectTimeout(str(exc)) from exc
except OSError as exc:
sock.close()
trace.mark_connect(t1, time.perf_counter(), fail="connect")
raise httpcore.ConnectError(str(exc)) from exc
trace.mark_connect(t1, time.perf_counter())
# TCP_NODELAY (matches httpcore's own SyncBackend and Go's net.Dialer
# default): disables Nagle's algorithm. Verified experimentally (by
# comparing against a plain socket that omits this) that skipping it
# costs a real ~40-50ms on TTFB against a live server — the classic
# Nagle/delayed-ACK interaction — not just noise. See docs/usage/hxprobe.md.
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
for option in socket_options or ():
sock.setsockopt(*option)
return _TimingStream(sock, trace)
def connect_unix_socket(self, path, timeout=None, socket_options=None):
raise NotImplementedError("hxprobe does not support UNIX sockets")
class _TimingTransport(httpx.HTTPTransport):
"""Subclasses httpx.HTTPTransport but replaces its connection pool's
network_backend, so handle_request()/close() (inherited, unchanged) still
get httpx's own httpcore-exception -> httpx-exception mapping for free."""
def __init__(
self, backend: httpcore.NetworkBackend, *, ssl_context: ssl.SSLContext, http2: bool
) -> None:
self._pool = httpcore.ConnectionPool(
ssl_context=ssl_context,
http1=True,
http2=http2,
network_backend=backend,
)
# ── measure ───────────────────────────────────────────────────────────────────
def _classify(exc: httpx.HTTPError) -> str:
"""Fallback classification for errors not raised by our own backend (e.g.
HTTP/2 stream resets, malformed responses) — "transfer" is the catch-all
bucket for post-connect failures that aren't a timeout or connect error."""
if isinstance(exc, httpx.TimeoutException):
return "timeout"
if isinstance(exc, httpx.ConnectError):
return "connect"
return "transfer"
def _unwrap(exc: BaseException) -> BaseException:
return exc.__cause__ if exc.__cause__ is not None else exc
def _fill_phases(r: Result, trace: _Trace, t_start: float, t_end: float) -> None:
r.dns = trace.dns
r.connect = trace.connect
r.tls = trace.tls
if trace.wrote_request is not None and trace.first_byte is not None:
r.ttfb = Phase(ms=(trace.first_byte - trace.wrote_request) * 1000, present=True)
if trace.first_byte is not None:
r.transfer = Phase(ms=(t_end - trace.first_byte) * 1000, present=True)
r.total = Phase(ms=(t_end - t_start) * 1000, present=True)
def _fill_verbose(
r: Result,
trace: _Trace,
*,
http_version: str = "",
redirect_count: int = 0,
headers: dict[str, str] | None = None,
) -> None:
"""Populate r.detail from whatever the trace captured. Called on both the
success and failure paths — e.g. resolved_ip/TLS info are known even when
a later phase (TTFB, transfer) is what actually failed."""
if r.detail is None:
return
d = r.detail
d.resolved_ip = trace.resolved_ip
d.http_version = http_version
d.redirect_count = redirect_count
if trace.tls.present:
d.tls_version = trace.tls_version
d.tls_cipher = trace.tls_cipher
d.tls_bits = trace.tls_bits
if trace.cert_peer:
d.cert = _parse_cert(trace.cert_peer, verified=True)
if headers:
d.headers = headers
def measure(raw_url: str, opts: Options | None = None) -> Result:
"""Probe raw_url via httpx and return a Result with per-phase timings.
Partial phases are preserved on failure; fail_phase is one of
dns/connect/timeout/tls/transfer/request. The underlying client
negotiates HTTP/2 (opts.http2, default True) and follows redirects
(opts.follow_redirects, default True) — matching Go's http.DefaultClient.
"""
if opts is None:
opts = Options()
r = Result(url=raw_url)
if opts.verbose:
r.detail = VerboseDetail()
parsed = urllib.parse.urlparse(raw_url)
scheme = parsed.scheme.lower()
if scheme not in ("http", "https"):
r.fail_phase = "request"
r.err = ValueError(f"unsupported scheme: {scheme!r}")
return r
trace = _Trace()
ssl_context = ssl.create_default_context()
transport = _TimingTransport(_TimingBackend(trace), ssl_context=ssl_context, http2=opts.http2)
status_code = 0
redirect_count = 0
http_version = ""
headers: dict[str, str] = {}
t_start = time.perf_counter()
try:
with httpx.Client(
transport=transport,
timeout=opts.timeout,
follow_redirects=opts.follow_redirects,
) as client:
with client.stream("GET", raw_url) as resp:
for _ in resp.iter_raw():
pass
status_code = resp.status_code
redirect_count = len(resp.history)
http_version = resp.http_version
if opts.verbose:
headers = dict(resp.headers)
except httpx.InvalidURL as exc:
r.fail_phase = "request"
r.err = exc
return r
except httpx.HTTPError as exc:
t_end = time.perf_counter()
_fill_phases(r, trace, t_start, t_end)
_fill_verbose(r, trace)
r.fail_phase = trace.fail_phase or _classify(exc)
r.err = _unwrap(exc)
return r
t_end = time.perf_counter()
_fill_phases(r, trace, t_start, t_end)
_fill_verbose(
r, trace, http_version=http_version, redirect_count=redirect_count, headers=headers
)
r.status_code = status_code
return r

31
hxprobe/pyproject.toml Normal file
View File

@@ -0,0 +1,31 @@
[project]
name = "hxprobe"
version = "0.1.0"
description = "Per-phase HTTP latency probe built on httpx — matches Go's http.DefaultClient (HTTP/2, redirects, connection pooling)"
requires-python = ">=3.11"
dependencies = [
"httpx[http2]>=0.28",
]
[dependency-groups]
dev = [
"pytest>=8.0",
"ruff>=0.8",
]
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
packages = ["hxprobe"]
[tool.pytest.ini_options]
testpaths = ["tests"]
markers = [
"integration: tests that hit the live internet (excluded from the default run)",
]
[tool.ruff]
target-version = "py311"
line-length = 100

View File

260
hxprobe/tests/test_cli.py Normal file
View File

@@ -0,0 +1,260 @@
"""Hermetic tests for hxprobe.cli.run() — a standalone CLI (its own argparse,
exit codes, and text/JSON rendering, not shared with any other package).
These tests spot-check the core rendering paths (single URL, --fail, JSON,
DNS/connect failures) plus what's specific to hxprobe: the --no-http2/
--no-follow-redirects flags and that redirects/http_version are really
wired through end to end."""
import http.server
import io
import json
import os
import socket
import tempfile
import threading
import unittest
from hxprobe.cli import EXIT_CONNECT, EXIT_DNS, EXIT_HTTP, EXIT_OK, EXIT_USAGE, run
def _free_port() -> int:
with socket.socket() as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
class _OKHandler(http.server.BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1" # keep-alive is in play — always send Content-Length
def do_GET(self):
body = b"hello hxprobe"
self.send_response(200)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
class _NotFoundHandler(http.server.BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def do_GET(self):
body = b"not found"
self.send_response(404)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
class _RedirectHandler(http.server.BaseHTTPRequestHandler):
"""/redirect -> 302 to /landed; /landed -> 200."""
protocol_version = "HTTP/1.1"
def do_GET(self):
if self.path == "/redirect":
self.send_response(302)
self.send_header("Location", "/landed")
self.send_header("Content-Length", "0")
self.end_headers()
else:
body = b"landed"
self.send_response(200)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
def _start_server(handler_class):
server = http.server.HTTPServer(("127.0.0.1", 0), handler_class)
t = threading.Thread(target=server.serve_forever)
t.daemon = True
t.start()
return server, server.server_address[1]
def _invoke(args: list[str]) -> tuple[int, str, str]:
out, err = io.StringIO(), io.StringIO()
code = run(args, out, err)
return code, out.getvalue(), err.getvalue()
class TestCLIBasics(unittest.TestCase):
def test_help_shows_hxprobe_prog_name(self):
code, out, err = _invoke(["-h"])
self.assertEqual(code, EXIT_OK)
self.assertIn("hxprobe", out)
def test_help_lists_protocol_flags(self):
code, out, err = _invoke(["-h"])
self.assertIn("--no-http2", out)
self.assertIn("--no-follow-redirects", out)
class TestCLISuccess(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.ok_server, cls.ok_port = _start_server(_OKHandler)
cls.nf_server, cls.nf_port = _start_server(_NotFoundHandler)
cls.redirect_server, cls.redirect_port = _start_server(_RedirectHandler)
@classmethod
def tearDownClass(cls):
cls.ok_server.shutdown()
cls.nf_server.shutdown()
cls.redirect_server.shutdown()
def _url(self, port=None, path=""):
return f"http://127.0.0.1:{port or self.ok_port}{path}"
def test_single_url_success(self):
code, out, _ = _invoke([self._url()])
self.assertEqual(code, EXIT_OK)
self.assertIn("200", out)
self.assertIn("DNS lookup", out)
def test_fail_flag_on_404(self):
code, out, _ = _invoke(["--fail", self._url(self.nf_port)])
self.assertEqual(code, EXIT_HTTP)
self.assertIn("", out)
def test_redirect_followed_by_default(self):
code, out, _ = _invoke(["-v", self._url(self.redirect_port, "/redirect")])
self.assertEqual(code, EXIT_OK)
self.assertIn("200", out)
self.assertIn("1 redirect", out)
def test_no_follow_redirects_flag(self):
code, out, _ = _invoke(
["--no-follow-redirects", self._url(self.redirect_port, "/redirect")]
)
self.assertEqual(code, EXIT_OK)
self.assertIn("302", out)
def test_verbose_shows_protocol_row(self):
code, out, _ = _invoke(["-v", self._url()])
self.assertIn("Protocol", out)
self.assertIn("HTTP/1.1", out)
def test_json_includes_http_version_and_redirect_count(self):
code, out, _ = _invoke(["--json", "-v", self._url(self.redirect_port, "/redirect")])
data = json.loads(out)
verbose = data[0]["verbose"]
self.assertEqual(verbose["redirect_count"], 1)
self.assertIn("http_version", verbose)
def test_no_http2_flag_forces_http1(self):
code, out, _ = _invoke(["--no-http2", "-v", self._url()])
self.assertEqual(code, EXIT_OK)
self.assertIn("HTTP/1.1", out)
class TestCLIFileInput(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.ok_server, cls.ok_port = _start_server(_OKHandler)
@classmethod
def tearDownClass(cls):
cls.ok_server.shutdown()
def _url(self):
return f"http://127.0.0.1:{self.ok_port}"
def test_reads_urls_from_file(self):
with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as f:
f.write(f"# comment\n\n{self._url()}\n{self._url()}\n")
path = f.name
try:
code, out, _ = _invoke(["-f", path])
finally:
os.unlink(path)
self.assertEqual(code, EXIT_OK)
self.assertEqual(out.count("200"), 2)
def test_missing_file_is_usage_error(self):
code, out, err = _invoke(["-f", "/no/such/file/hxprobe-test"])
self.assertEqual(code, EXIT_USAGE)
self.assertIn("cannot read", err)
def test_empty_file_is_usage_error(self):
with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as f:
path = f.name
try:
code, out, err = _invoke(["-f", path])
finally:
os.unlink(path)
self.assertEqual(code, EXIT_USAGE)
self.assertIn("no URLs found", err)
def test_both_sources_given_is_usage_error(self):
code, out, err = _invoke(["-f", "/tmp/whatever", self._url()])
self.assertEqual(code, EXIT_USAGE)
self.assertIn("cannot combine", err)
def test_no_sources_given_is_usage_error(self):
code, out, err = _invoke([])
self.assertEqual(code, EXIT_USAGE)
self.assertIn("no URLs given", err)
class TestCLIFailures(unittest.TestCase):
def test_dns_failure(self):
code, out, _ = _invoke(["http://no.such.host.invalid"])
self.assertEqual(code, EXIT_DNS)
self.assertIn("FAILED", out)
def test_connection_refused(self):
port = _free_port()
code, out, _ = _invoke([f"http://127.0.0.1:{port}"])
self.assertEqual(code, EXIT_CONNECT)
self.assertIn("FAILED", out)
class TestCLIRunSummary(unittest.TestCase):
"""The end-of-run footer: only shown for multi-URL runs, since the single
worst-code exit can't show the mix of failure classes behind it."""
@classmethod
def setUpClass(cls):
cls.ok_server, cls.ok_port = _start_server(_OKHandler)
@classmethod
def tearDownClass(cls):
cls.ok_server.shutdown()
def _url(self):
return f"http://127.0.0.1:{self.ok_port}"
def test_multi_url_mixed_shows_summary(self):
port = _free_port()
code, out, _ = _invoke([self._url(), f"http://127.0.0.1:{port}"])
self.assertEqual(code, EXIT_CONNECT)
self.assertIn("Summary: 2 URLs", out)
self.assertIn("1 ok", out)
self.assertIn("1 failed", out)
self.assertIn("connect : 1", out)
self.assertIn("→ exit 3", out)
def test_multi_url_all_ok_summary(self):
code, out, _ = _invoke([self._url(), self._url()])
self.assertEqual(code, EXIT_OK)
self.assertIn("Summary: 2 URLs — 2 ok", out)
self.assertNotIn("", out)
def test_single_url_has_no_summary(self):
code, out, _ = _invoke([self._url()])
self.assertEqual(code, EXIT_OK)
self.assertNotIn("Summary:", out)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,122 @@
"""Integration tests against live internet services for hxprobe.
Run with:
make hx-test-integration
cd hxprobe && uv run pytest tests -m integration -v
Marked with `pytest.mark.integration` (see module-level `pytestmark` below)
so it's excluded from the default `hx-test`/`make test` gate, which runs
`pytest -m "not integration"` — these hit the real internet.
The point of this file is specifically hxprobe's headline capabilities: real
HTTP/2 negotiation and redirect-following.
"""
import io
import socket
import unittest
import pytest
from hxprobe.cli import EXIT_DNS, EXIT_OK, run
from hxprobe.probe import Options, measure
pytestmark = pytest.mark.integration
def _online() -> bool:
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(3)
s.connect(("8.8.8.8", 53))
s.close()
return True
except OSError:
return False
_NEEDS_NET = unittest.skipUnless(_online(), "no internet connectivity")
def _run(args: list[str]) -> tuple[int, str, str]:
out, err = io.StringIO(), io.StringIO()
code = run(args, out, err)
return code, out.getvalue(), err.getvalue()
@_NEEDS_NET
class TestHTTP2Negotiation(unittest.TestCase):
"""The headline Go-parity feature: real ALPN HTTP/2, not forced HTTP/1.1."""
def test_negotiates_h2_against_cloudflare(self):
r = measure("https://example.com", Options(verbose=True))
self.assertIsNone(r.err, r.err)
self.assertEqual(r.detail.http_version, "HTTP/2")
def test_no_http2_flag_forces_http1(self):
code, out, _ = _run(["--no-http2", "-v", "https://example.com"])
self.assertEqual(code, EXIT_OK)
self.assertIn("HTTP/1.1", out)
self.assertNotIn("HTTP/2", out)
@_NEEDS_NET
class TestRedirectsLive(unittest.TestCase):
"""The other headline Go-parity feature: redirects followed by default."""
def test_follows_http_to_https_redirect(self):
code, out, _ = _run(["-v", "http://github.com"])
self.assertEqual(code, EXIT_OK)
self.assertIn("200", out)
self.assertIn("redirect", out)
def test_no_follow_redirects_reports_redirect_status(self):
code, out, _ = _run(["--no-follow-redirects", "http://github.com"])
self.assertEqual(code, EXIT_OK)
self.assertNotIn("redirect_count", out) # text mode never shows the raw key
self.assertTrue(any(code_str in out for code_str in ("301", "302", "307", "308")))
@_NEEDS_NET
class TestSuccess(unittest.TestCase):
def test_https_all_phases_present(self):
r = measure("https://example.com")
self.assertIsNone(r.err, r.err)
self.assertEqual(r.status_code, 200)
self.assertTrue(r.dns.present)
self.assertTrue(r.connect.present)
self.assertTrue(r.tls.present)
self.assertTrue(r.ttfb.present)
self.assertTrue(r.transfer.present)
self.assertTrue(r.total.present)
def test_http_no_tls_phase(self):
r = measure("http://example.com")
self.assertIsNone(r.err, r.err)
self.assertFalse(r.tls.present)
def test_timings_are_positive(self):
r = measure("https://example.com")
for attr in ("dns", "connect", "tls", "ttfb", "transfer", "total"):
ph = getattr(r, attr)
if ph.present:
self.assertGreater(ph.ms, 0, f"{attr}.ms should be > 0")
@_NEEDS_NET
class TestDNSFailure(unittest.TestCase):
def test_invalid_tld_phase(self):
r = measure("http://no.such.host.invalid")
self.assertEqual(r.fail_phase, "dns")
self.assertIsNotNone(r.err)
self.assertFalse(r.dns.present)
self.assertFalse(r.connect.present)
self.assertTrue(r.total.present)
def test_cli_dns_failure_exit_code(self):
code, out, _ = _run(["http://no.such.host.invalid"])
self.assertEqual(code, EXIT_DNS)
if __name__ == "__main__":
unittest.main()

256
hxprobe/tests/test_probe.py Normal file
View File

@@ -0,0 +1,256 @@
"""Hermetic tests for hxprobe.probe.measure(), including a redirect test
covering the Go-http.DefaultClient-parity feature (redirects followed by
default) that a plain raw-socket HTTP/1.1 client would not have."""
import http.server
import socket
import threading
import unittest
from hxprobe.probe import Options, VerboseDetail, measure
class _OKHandler(http.server.BaseHTTPRequestHandler):
# HTTP/1.1 (BaseHTTPRequestHandler defaults to 1.0) so hxprobe negotiates
# http_version="HTTP/1.1" instead of "HTTP/1.0". Keep-alive is then in
# play, so every response below sends an explicit Content-Length —
# without it, h11 has no way to detect body-end short of connection
# close, and the client hangs until it hits the request timeout.
protocol_version = "HTTP/1.1"
def _send_body(self, status: int, body: bytes) -> None:
self.send_response(status)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self):
self._send_body(200, b"hello hxprobe")
def log_message(self, *args):
pass
class _NotFoundHandler(http.server.BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def do_GET(self):
body = b"not found"
self.send_response(404)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
class _RedirectHandler(http.server.BaseHTTPRequestHandler):
"""/redirect -> 302 to /landed; /landed -> 200."""
protocol_version = "HTTP/1.1"
def do_GET(self):
if self.path == "/redirect":
self.send_response(302)
self.send_header("Location", "/landed")
self.send_header("Content-Length", "0")
self.end_headers()
else:
body = b"landed"
self.send_response(200)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
def _start_server(handler_class):
server = http.server.HTTPServer(("127.0.0.1", 0), handler_class)
t = threading.Thread(target=server.serve_forever)
t.daemon = True
t.start()
return server, server.server_address[1]
def _free_port() -> int:
with socket.socket() as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
def _black_hole_port() -> int:
"""Bind a port that accepts TCP but never sends any data (triggers a read timeout)."""
srv = socket.socket()
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("127.0.0.1", 0))
srv.listen(10)
port = srv.getsockname()[1]
conns: list = []
def _serve():
while True:
try:
conn, _ = srv.accept()
conns.append(conn)
except OSError:
break
threading.Thread(target=_serve, daemon=True).start()
return port
class TestMeasureSuccess(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.ok_server, cls.ok_port = _start_server(_OKHandler)
cls.nf_server, cls.nf_port = _start_server(_NotFoundHandler)
@classmethod
def tearDownClass(cls):
cls.ok_server.shutdown()
cls.nf_server.shutdown()
def test_all_phases_present_on_success(self):
r = measure(f"http://127.0.0.1:{self.ok_port}")
self.assertIsNone(r.err)
self.assertEqual(r.fail_phase, "")
self.assertEqual(r.status_code, 200)
self.assertTrue(r.dns.present)
self.assertTrue(r.connect.present)
self.assertFalse(r.tls.present) # http — no TLS
self.assertTrue(r.ttfb.present)
self.assertTrue(r.transfer.present)
self.assertTrue(r.total.present)
def test_status_code_404(self):
r = measure(f"http://127.0.0.1:{self.nf_port}")
self.assertIsNone(r.err)
self.assertEqual(r.status_code, 404)
self.assertTrue(r.total.present)
def test_options_default_follows_redirects_and_http2(self):
opts = Options()
self.assertTrue(opts.follow_redirects)
self.assertTrue(opts.http2)
class TestMeasureFailures(unittest.TestCase):
def test_dns_failure(self):
r = measure("http://no.such.host.invalid")
self.assertEqual(r.fail_phase, "dns")
self.assertIsNotNone(r.err)
self.assertFalse(r.dns.present)
self.assertFalse(r.connect.present)
self.assertTrue(r.total.present)
def test_connection_refused(self):
port = _free_port()
r = measure(f"http://127.0.0.1:{port}")
self.assertEqual(r.fail_phase, "connect")
self.assertIsNotNone(r.err)
self.assertTrue(r.dns.present)
self.assertTrue(r.connect.present)
self.assertFalse(r.ttfb.present)
self.assertTrue(r.total.present)
def test_ttfb_timeout(self):
port = _black_hole_port()
r = measure(f"http://127.0.0.1:{port}", Options(timeout=0.2))
self.assertEqual(r.fail_phase, "timeout")
self.assertIsNotNone(r.err)
self.assertTrue(r.dns.present)
self.assertTrue(r.connect.present)
self.assertTrue(r.total.present)
def test_bad_scheme(self):
r = measure("ftp://example.com")
self.assertEqual(r.fail_phase, "request")
self.assertIsNotNone(r.err)
self.assertFalse(r.total.present)
class TestRedirects(unittest.TestCase):
"""The Go-http.DefaultClient-parity feature: redirects followed by
default, with dns/connect/tls timed from the first hop only."""
@classmethod
def setUpClass(cls):
cls.server, cls.port = _start_server(_RedirectHandler)
@classmethod
def tearDownClass(cls):
cls.server.shutdown()
def _url(self, path: str) -> str:
return f"http://127.0.0.1:{self.port}{path}"
def test_follows_redirect_by_default(self):
r = measure(self._url("/redirect"), Options(verbose=True))
self.assertIsNone(r.err)
self.assertEqual(r.status_code, 200)
self.assertEqual(r.detail.redirect_count, 1)
def test_no_follow_redirects_reports_302(self):
r = measure(self._url("/redirect"), Options(follow_redirects=False, verbose=True))
self.assertIsNone(r.err)
self.assertEqual(r.status_code, 302)
self.assertEqual(r.detail.redirect_count, 0)
def test_dns_and_connect_present_across_redirect(self):
r = measure(self._url("/redirect"))
self.assertTrue(r.dns.present)
self.assertTrue(r.connect.present)
self.assertTrue(r.ttfb.present)
self.assertTrue(r.total.present)
class TestVerboseDetail(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.ok_server, cls.ok_port = _start_server(_OKHandler)
@classmethod
def tearDownClass(cls):
cls.ok_server.shutdown()
def _url(self):
return f"http://127.0.0.1:{self.ok_port}"
def test_detail_none_without_verbose(self):
r = measure(self._url())
self.assertIsNone(r.detail)
def test_detail_present_with_verbose(self):
r = measure(self._url(), Options(verbose=True))
self.assertIsInstance(r.detail, VerboseDetail)
def test_resolved_ip_set_on_success(self):
r = measure(self._url(), Options(verbose=True))
self.assertEqual(r.detail.resolved_ip, "127.0.0.1")
def test_http_version_set_on_success(self):
r = measure(self._url(), Options(verbose=True))
self.assertIn(r.detail.http_version, ("HTTP/1.1", "HTTP/2"))
def test_no_tls_fields_for_http(self):
r = measure(self._url(), Options(verbose=True))
self.assertEqual(r.detail.tls_version, "")
self.assertIsNone(r.detail.cert)
def test_headers_populated_on_success(self):
r = measure(self._url(), Options(verbose=True))
self.assertIsInstance(r.detail.headers, dict)
self.assertTrue(len(r.detail.headers) > 0)
def test_ip_set_on_connect_fail(self):
port = _free_port()
r = measure(f"http://127.0.0.1:{port}", Options(verbose=True))
self.assertEqual(r.fail_phase, "connect")
self.assertEqual(r.detail.resolved_ip, "127.0.0.1")
if __name__ == "__main__":
unittest.main()

225
hxprobe/uv.lock generated Normal file
View File

@@ -0,0 +1,225 @@
version = 1
revision = 3
requires-python = ">=3.11"
[[package]]
name = "anyio"
version = "4.14.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "idna" },
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/3b/72/5562aabb8dd7181e8e860622a38bea08d17842b99ecd4c91f84ac95251b0/anyio-4.14.1.tar.gz", hash = "sha256:8d648a3544c1a700e3ff78615cd679e4c5c3f149904287e73687b2596963629e", size = 254831, upload-time = "2026-06-24T20:56:06.017Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b0/7b/90df4a0a816d98d6ea26f559d87836d494a2cf1fcf063be67df50a7bcc30/anyio-4.14.1-py3-none-any.whl", hash = "sha256:4e5533c5b8ff0a24f5d7a176cbe6877129cd183893f66b537f8f227d10527d72", size = 124875, upload-time = "2026-06-24T20:56:04.413Z" },
]
[[package]]
name = "certifi"
version = "2026.6.17"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/c9/c7/424b75da314c1045981bd9777432fad05a9e0c69daa4ed7e308bbaffe405/certifi-2026.6.17.tar.gz", hash = "sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432", size = 134594, upload-time = "2026-06-17T10:31:07.894Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ef/2f/c5464532e965badff2f4c4c1a3a83f5697f0d7c407ed0cda44aaa99bb451/certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db", size = 133289, upload-time = "2026-06-17T10:31:06.348Z" },
]
[[package]]
name = "colorama"
version = "0.4.6"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
]
[[package]]
name = "h11"
version = "0.16.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
]
[[package]]
name = "h2"
version = "4.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "hpack" },
{ name = "hyperframe" },
]
sdist = { url = "https://files.pythonhosted.org/packages/1d/17/afa56379f94ad0fe8defd37d6eb3f89a25404ffc71d4d848893d270325fc/h2-4.3.0.tar.gz", hash = "sha256:6c59efe4323fa18b47a632221a1888bd7fde6249819beda254aeca909f221bf1", size = 2152026, upload-time = "2025-08-23T18:12:19.778Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/69/b2/119f6e6dcbd96f9069ce9a2665e0146588dc9f88f29549711853645e736a/h2-4.3.0-py3-none-any.whl", hash = "sha256:c438f029a25f7945c69e0ccf0fb951dc3f73a5f6412981daee861431b70e2bdd", size = 61779, upload-time = "2025-08-23T18:12:17.779Z" },
]
[[package]]
name = "hpack"
version = "4.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/26/5b/fcabf6028144a8723726318b07a32c2f3314acdff6265743cf08a344b18e/hpack-4.2.0.tar.gz", hash = "sha256:0895cfa3b5531fc65fe439c05eb65144f123bf7a394fcaa56aa423548d8e45c0", size = 51300, upload-time = "2026-06-23T18:34:46.667Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/71/b4/4a9fcfb2aef6ba44d9073ecd301443aa00b3dac95de5619f2a7de7ec8a91/hpack-4.2.0-py3-none-any.whl", hash = "sha256:858ac0b02280fa582b5080d68db0899c62a80375e0e5413a74970c5e518b6986", size = 34246, upload-time = "2026-06-23T18:34:45.472Z" },
]
[[package]]
name = "httpcore"
version = "1.0.9"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "certifi" },
{ name = "h11" },
]
sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484, upload-time = "2025-04-24T22:06:22.219Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },
]
[[package]]
name = "httpx"
version = "0.28.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "anyio" },
{ name = "certifi" },
{ name = "httpcore" },
{ name = "idna" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406, upload-time = "2024-12-06T15:37:23.222Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
]
[package.optional-dependencies]
http2 = [
{ name = "h2" },
]
[[package]]
name = "hxprobe"
version = "0.1.0"
source = { editable = "." }
dependencies = [
{ name = "httpx", extra = ["http2"] },
]
[package.dev-dependencies]
dev = [
{ name = "pytest" },
{ name = "ruff" },
]
[package.metadata]
requires-dist = [{ name = "httpx", extras = ["http2"], specifier = ">=0.28" }]
[package.metadata.requires-dev]
dev = [
{ name = "pytest", specifier = ">=8.0" },
{ name = "ruff", specifier = ">=0.8" },
]
[[package]]
name = "hyperframe"
version = "6.1.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/02/e7/94f8232d4a74cc99514c13a9f995811485a6903d48e5d952771ef6322e30/hyperframe-6.1.0.tar.gz", hash = "sha256:f630908a00854a7adeabd6382b43923a4c4cd4b821fcb527e6ab9e15382a3b08", size = 26566, upload-time = "2025-01-22T21:41:49.302Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/48/30/47d0bf6072f7252e6521f3447ccfa40b421b6824517f82854703d0f5a98b/hyperframe-6.1.0-py3-none-any.whl", hash = "sha256:b03380493a519fce58ea5af42e4a42317bf9bd425596f7a0835ffce80f1a42e5", size = 13007, upload-time = "2025-01-22T21:41:47.295Z" },
]
[[package]]
name = "idna"
version = "3.18"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
]
[[package]]
name = "iniconfig"
version = "2.3.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
]
[[package]]
name = "packaging"
version = "26.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
]
[[package]]
name = "pluggy"
version = "1.6.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
]
[[package]]
name = "pygments"
version = "2.20.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
]
[[package]]
name = "pytest"
version = "9.1.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "iniconfig" },
{ name = "packaging" },
{ name = "pluggy" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
]
[[package]]
name = "ruff"
version = "0.15.20"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/43/dc/35b341fc554ba02f217fc10da57d1a75168cfbcf75b0ef2202176d4c4f2d/ruff-0.15.20.tar.gz", hash = "sha256:1416eb04349192646b54de98f146c4f59afe37d0decfc02c3cbbf396f3a28566", size = 4755489, upload-time = "2026-06-25T17:20:37.578Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/94/d9/2d5014f0253ba541d2061d9fa7193f48e941c8b21bb88a7ff9bbe0bd0596/ruff-0.15.20-py3-none-linux_armv6l.whl", hash = "sha256:00e188c53e499c3c1637f73c91dcf2fb56d576cab76ce1be50a27c4e80e37078", size = 10839665, upload-time = "2026-06-25T17:19:44.702Z" },
{ url = "https://files.pythonhosted.org/packages/c6/d3/ac1798ba64f670698867fcfc591d50e7e421bef137db564858f619a30fcf/ruff-0.15.20-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:9ebd1fd9b9c95fc0bd7b2761aebec1f030013d2e193a2901b224af68fe47251b", size = 11208649, upload-time = "2026-06-25T17:19:48.787Z" },
{ url = "https://files.pythonhosted.org/packages/47/47/d3ac899991202095dfcf3d5176be4272642be3cf981a2f1a30f72a2afb95/ruff-0.15.20-py3-none-macosx_11_0_arm64.whl", hash = "sha256:c5b16cdd67ca108185cd36dce98c576350c03b1660a751de725fb049193a0632", size = 10622638, upload-time = "2026-06-25T17:19:51.354Z" },
{ url = "https://files.pythonhosted.org/packages/33/13/4e043fe30aa94d4ff5213a9881fc296d12960f5971b234a5263fdc225312/ruff-0.15.20-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3413bb3c3d2ca6a8208f1f4809cd2dca3c6de6d0b491c0e70847672bde6e6efd", size = 10984227, upload-time = "2026-06-25T17:19:54.044Z" },
{ url = "https://files.pythonhosted.org/packages/76/e6/92e7bf40388bc5800073b96564f56264f7e48bfd1a498f5ced6ae6d5a769/ruff-0.15.20-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bd7ec42b3bb3da066488db093308a69c4ac5ee6d2af333a86ba6e2eb2e7dd44b", size = 10622882, upload-time = "2026-06-25T17:19:57.037Z" },
{ url = "https://files.pythonhosted.org/packages/13/7a/43460be3f24495a3aa46d4b16873e2c4941b3b5f0b00cf88c03b7b94b339/ruff-0.15.20-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e1a36ad0eb77fba9aabfb69ede54de6f376d04ac18ebea022847046d340a8267", size = 11474808, upload-time = "2026-06-25T17:20:00.357Z" },
{ url = "https://files.pythonhosted.org/packages/27/a0/f37077884873221c6b33b4ab49eb18f9f88e54a16a25a5bca59bef46dd66/ruff-0.15.20-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b6df3b1e4610432f0386dba04d853b5f08cbbc903410c6fcc02f620f05aff53c", size = 12293094, upload-time = "2026-06-25T17:20:03.446Z" },
{ url = "https://files.pythonhosted.org/packages/a6/74/165545b60256a9704c21ac0ec4a0d07933b320812f9584836c9f4aca4292/ruff-0.15.20-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e89f198a1ea6ef0d727c1cf16088bc91a6cb0ab947dedc966715691647186eae", size = 11526176, upload-time = "2026-06-25T17:20:06.301Z" },
{ url = "https://files.pythonhosted.org/packages/86/b1/a976a136d40ade83ce743578399865f57001003a409acadc0ecbb3051082/ruff-0.15.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:309809086c2acb67624950a3c8133e80f32d0d3e27106c0cd60ff26657c9f24b", size = 11520767, upload-time = "2026-06-25T17:20:09.191Z" },
{ url = "https://files.pythonhosted.org/packages/19/0f/f032696cb01c9b54c0263fa393474d7758f1cdc021a01b04e3cbc2500999/ruff-0.15.20-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:2d2374caa2f2c2f9e2b7da0a50802cfb8b79f55a9b5e49379f564544fbf56487", size = 11500132, upload-time = "2026-06-25T17:20:13.602Z" },
{ url = "https://files.pythonhosted.org/packages/4b/f4/51b1a14bc69e8c224b15dab9cce8e99b425e0455d462caa2b3c9be2b6a8e/ruff-0.15.20-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:a1ed17b65293e0c2f22fc387bc13198a5de94bf4429589b0ff6946b0feaf21a3", size = 10943828, upload-time = "2026-06-25T17:20:16.635Z" },
{ url = "https://files.pythonhosted.org/packages/71/4b/fe267640783cd02bf6c5cc290b1df1051be2ec294c678b5c15fe19e52343/ruff-0.15.20-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:f701305e66b38ea6c91882490eb73459796808e4c6362a1b765255e0cdcd4053", size = 10645418, upload-time = "2026-06-25T17:20:19.4Z" },
{ url = "https://files.pythonhosted.org/packages/b0/c0/a65aa4ec2f5e87a1df32dc3ec1fede434fe3dfd5cbcf3b503cafc676ab54/ruff-0.15.20-py3-none-musllinux_1_2_i686.whl", hash = "sha256:5b9c0c367ad8e5d0d5b5b8537864c469a0a0e55417aadfbeca41fa61333be9f4", size = 11211770, upload-time = "2026-06-25T17:20:22.033Z" },
{ url = "https://files.pythonhosted.org/packages/5a/a4/0caa331d954ae2723d729d351c989cb4ca8b6077d5c6c2cb6de75e98c041/ruff-0.15.20-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:01cc00dd58f0df339d0e902219dd53990ea99996a0344e5d9cc8d45d5307e460", size = 11618698, upload-time = "2026-06-25T17:20:25.259Z" },
{ url = "https://files.pythonhosted.org/packages/10/9b/5f14927848d2fd4aa891fd88d883788c5a7baba561c7874732364045708c/ruff-0.15.20-py3-none-win32.whl", hash = "sha256:ed65ef510e43a137207e0f01cfcf998aeddb1aeeda5c9d35023e910284d7cf21", size = 10857322, upload-time = "2026-06-25T17:20:28.612Z" },
{ url = "https://files.pythonhosted.org/packages/fa/f0/fe47c501f9dea92a26d788ff98bb5d92ed4cb4c88792c5c88af6b697dc8e/ruff-0.15.20-py3-none-win_amd64.whl", hash = "sha256:a525c81c70fb0380344dd1d8745d8cc1c890b7fc94a58d5a07bd8eb9557b8415", size = 11993274, upload-time = "2026-06-25T17:20:31.871Z" },
{ url = "https://files.pythonhosted.org/packages/d7/2b/9555445e1201d92b3195f45cdb153a0b68f24e0a4273f6e3d5ab46e212bb/ruff-0.15.20-py3-none-win_arm64.whl", hash = "sha256:2f5b2a6d614e8700388806a14996c40fab2c47b819ef57d790a34878858ed9ca", size = 11343498, upload-time = "2026-06-25T17:20:35.03Z" },
]
[[package]]
name = "typing-extensions"
version = "4.15.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
]