docs: hxprobe plans, summaries, explanations, usage, and changelog

Plans (docs/plans/): - 2026-07-01-23-47-py-hxprobe-httpx.md — initial httpx probe design - 2026-07-02-09-32 through 14-05 — standalone project, toolchain, usage doc + Makefile, file input (-f), simplification pass, run-summary footer Summaries (docs/summaries/): one per completed feature, recording what was actually built, deviations from the plan, and verification steps Explanations (docs/explanations/): two deep-dives written during review — hxprobe concurrency model and worst-exit-code + render-loop analysis Usage (docs/usage/hxprobe.md): overview with pointer to hxprobe/USAGE.md for the full runnable reference Walkthrough (docs/py-latprobe-walkthrough.md): narrative tour of the latprobe Python package for interview / code-review context CHANGELOG.md: entries for all hxprobe features (toolchain, usage doc, file input, simplification, run-summary footer) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-02 13:22:59 +02:00
parent 13a562e966
commit f487a4b1bd
19 changed files with 2367 additions and 0 deletions
--- a/docs/plans/2026-07-02-09-32-hxprobe-standalone-project.md
+++ b/docs/plans/2026-07-02-09-32-hxprobe-standalone-project.md
@@ -0,0 +1,151 @@
+# Plan: extract `hxprobe` into a fully standalone top-level project
+
+## Context
+
+`hxprobe` currently lives at `python/hxprobe/` and imports shared code from
+`latprobe` (`latprobe.probe.{Options,Phase,Result,VerboseDetail,_parse_cert}`,
+`latprobe.cli.run`). That coupling was a deliberate reuse choice at the time,
+but the user now wants `hxprobe` to be a genuinely independent project — no
+`from latprobe import ...` anywhere, duplication accepted — and, per their
+follow-up answer, independent enough that it could be `cp -r`'d into its own
+repo tomorrow: own `pyproject.toml`, own venv, own Makefile section, own
+top-level directory (sibling to `go/` and `python/`), not nested under `python/`.
+
+Confirmed via `git status`/`git log`: everything hxprobe-related
+(`python/hxprobe/`, `python/pyproject.toml`, `python/tests/test_hx_*.py`,
+`docs/usage/py-hxprobe.md`, the two `docs/plans`/`docs/summaries` entries) is
+still uncommitted from this session, and `python/latprobe/cli.py` /
+`python/latprobe/probe.py` only diverge from the last commit (`24ea9c9`) by
+the reuse-oriented additions made to support hxprobe. So this is a clean,
+low-risk restructuring: move already-debugged code, revert latprobe with
+`git checkout --`, no git history surgery needed.
+
+## Target layout
+
+```
+hxprobe/                       (new, top-level, sibling of go/ and python/)
+├── pyproject.toml             (own manifest: httpx[http2] dependency)
+├── hxprobe/
+│   ├── __init__.py
+│   ├── probe.py                (own Options/Phase/Result/VerboseDetail/CertInfo
+│   │                            + own _parse_cert/_parse_cert_date + existing
+│   │                            _Trace/_TimingStream/_TimingBackend/
+│   │                            _TimingTransport/measure() — unchanged logic)
+│   ├── aggregate.py             (verbatim copy of latprobe/aggregate.py —
+│   │                            its `from .probe import Result` is already
+│   │                            package-relative, needs zero edits)
+│   ├── duration.py              (verbatim copy of latprobe/duration.py — no
+│   │                            imports at all)
+│   ├── cli.py                   (full standalone CLI — see below)
+│   └── __main__.py               (unchanged: `from .cli import run`)
+└── tests/
+    ├── __init__.py               (empty, matches python/tests/__init__.py)
+    ├── test_probe.py             (moved from python/tests/test_hx_probe.py)
+    ├── test_cli.py               (moved from python/tests/test_hx_cli.py)
+    └── test_integration.py       (moved from python/tests/test_integration_hx.py)
+```
+
+`python/` reverts to containing only `latprobe` — zero third-party deps, no
+`pyproject.toml`, no venv, exactly its pre-hxprobe state.
+
+## Step-by-step
+
+**1. Move already-debugged files (preserve the bug fixes already made):**
+`git mv`/`mv` (untracked, so plain `mv` is fine) `python/hxprobe/{probe.py,__init__.py,__main__.py}`
+to `hxprobe/hxprobe/`, and the three `python/tests/test_hx_*.py` /
+`test_integration_hx.py` files to `hxprobe/tests/` with the `hx_`/`_hx` name
+segments dropped (`test_probe.py`, `test_cli.py`, `test_integration.py`).
+Do **not** rewrite these from scratch — they already have the mark_dns /
+verbose-on-failure-path / Content-Length-on-keep-alive fixes found during
+the original implementation.
+
+**2. Inline the dataclasses into `hxprobe/hxprobe/probe.py`:**
+Replace `from latprobe.probe import Options, Phase, Result, VerboseDetail, _parse_cert`
+with local definitions copied verbatim from `python/latprobe/probe.py`:
+`CertInfo`, `VerboseDetail` (its `http_version`/`redirect_count` fields are
+now simply always-meaningful, no more "populated only by hxprobe" caveat
+comment needed), `Options` (with `follow_redirects`/`http2` as normal fields,
+no more "ignored by socket measure()" caveat), `Phase`, `Result`,
+`_parse_cert`, `_parse_cert_date`. Everything else in the file (`_Trace`,
+`_TimingStream`, `_TimingBackend`, `_TimingTransport`, `measure()`,
+`_classify`, `_unwrap`, `_fill_phases`, `_fill_verbose`) is untouched.
+
+**3. Create `hxprobe/hxprobe/aggregate.py` and `duration.py`:**
+Verbatim copies of `python/latprobe/aggregate.py` and `duration.py`.
+
+**4. Write `hxprobe/hxprobe/cli.py` as a full standalone CLI:**
+Start from the *current* `python/latprobe/cli.py` (it already has 100% of
+the needed logic, including the `--no-http2`/`--no-follow-redirects` flags,
+the verbose "Protocol" row, and the JSON `http_version`/`redirect_count`
+keys — all added earlier specifically for hxprobe). Strip the
+generalization scaffolding that only existed to let `latprobe` share this
+code:
+
+- Remove the `MeasureFn` type alias and the `measure_fn` parameter from
+  `_run_samples`/`run` — call `measure` directly (module-level import from
+  `.probe`).
+- Remove `run()`'s `prog`/`description`/`protocol_flags` parameters —
+  hardcode `prog="hxprobe"` and the httpx-specific description.
+- Make the `--no-http2`/`--no-follow-redirects` `add_argument` calls
+  unconditional (drop the `if protocol_flags:` guard).
+- Everything else (exit codes, `_ArgExit`/`_Parser`, phase-label/verbose
+  constants, all `_print_*`/`_build_json_entry` rendering) carries over
+  unchanged — it's already correct standalone logic.
+
+**5. Revert `python/latprobe/` to its pre-hxprobe state:**
+`git checkout -- python/latprobe/cli.py python/latprobe/probe.py` (safe:
+confirmed these are the only diffs since the last commit, and both diffs
+are exactly the reuse scaffolding being removed here).
+
+**6. Clean up the old shared-package artifacts:**
+Delete `python/hxprobe/`, `python/pyproject.toml`, `python/.venv/`, and the
+three `python/tests/test_hx_*`/`test_integration_hx.py` files (now moved).
+
+**7. Makefile:**
+- Remove `PY_VENV`/`PY_VENV_PYTHON` vars and the `py-deps` target; revert
+  `py-test`/`py-test-integration`/`py-check` to invoke `$(PYTHON)` directly
+  with no `py-deps` prerequisite (their pre-hxprobe form). Remove `hx-run`/
+  `hx-test-integration` from the Python section (moving out).
+- Add `HX_DIR := hxprobe`, `HX_VENV := $(HX_DIR)/.venv`,
+  `HX_VENV_PYTHON := $(HX_VENV)/bin/$(PYTHON)` and a new "── hxprobe
+  (standalone) ──" section: `hx-deps` (idempotent venv+pip install, mirrors
+  the removed `py-deps`), `hx-run`, `hx-test` (hermetic, `test_[!i]*.py`
+  glob), `hx-test-integration` (`test_integration.py` pattern), `hx-check`,
+  `hx-clean`.
+- Fold into the umbrella targets: `test: go-test py-test hx-test`,
+  `check: go-check py-check hx-check`, `clean: go-clean py-clean hx-clean`.
+
+**8. Docs:**
+- Rename `docs/usage/py-hxprobe.md` → `docs/usage/hxprobe.md` (drop the
+  `py-` prefix — it's no longer part of the Python port). Update: remove the
+  "reuses `latprobe.cli.run()` in full" claim (now false — say it's a fully
+  standalone implementation sharing only the *design*, not the code, with
+  `latprobe`); update setup/Makefile-target sections to `make hx-deps`/
+  `hx-run`/`hx-test`/`hx-test-integration`; update file paths from
+  `python/hxprobe/` to `hxprobe/hxprobe/`. Keep the TCP_NODELAY/Nagle finding
+  and the example transcripts (still accurate — behavior is unchanged, only
+  location/packaging changed).
+- `CHANGELOG.md`: new entry describing the extraction.
+- `docs/summaries/`: new dated summary per the CLAUDE.md convention
+  (leave the original `2026-07-02-00-29-py-hxprobe-httpx.md` as-is — it's a
+  historical record of that implementation; this is a follow-up).
+- `docs/plans/`: save this plan as
+  `docs/plans/<yyyy-mm-dd-hh-mm>-hxprobe-standalone-project.md` at
+  implementation start, per CLAUDE.md.
+
+## Verification
+
+1. `rm -rf hxprobe/.venv python/.venv` (fresh state) then `make hx-deps` —
+   creates `hxprobe/.venv`, installs `httpx[http2]` from `hxprobe/pyproject.toml`.
+2. `grep -rn "latprobe" hxprobe/` — must return nothing (proves the
+   decoupling).
+3. `make hx-run ARGS="-v http://github.com"` — same output as before (HTTP/2,
+   1 redirect followed, IP/TLS/cert shown).
+4. `make hx-test` — all hermetic hxprobe tests pass standalone.
+5. `make hx-test-integration` — live HTTP/2 + redirect tests still pass.
+6. `make py-test` — confirms `latprobe`'s own suite is back to its original,
+   dependency-free form and still green (no `py-deps` needed to run it).
+7. `make check` (top-level) — Go + latprobe + hxprobe all green in one gate.
+8. `diff <(python -m latprobe --help) <(git show 24ea9c9:python/latprobe/cli.py | ...)` —
+   or simpler: confirm `python -m latprobe --help` output is byte-identical
+   to before this whole feature existed (no leftover `--no-http2` etc.).