http-latency-prober/docs/plans/2026-07-01-00-49-integration-tests.md

Plan: Step 6 — Integration tests (Go)

Context

The Go latprobe tool is feature-complete (Steps 0–5): per-phase latency, multiple URLs, -n sampling, --json, and classified failure handling with distinct exit codes. There are currently no automated tests — every check so far has been manual.

We want integration tests that demonstrate the application's behaviour across a successful run and every failure mode: HTTP success (200), HTTP error status (404/500), DNS failure, connection refused, timeout, and TLS handshake failure — asserting on both the rendered output and the exit code.

The user chose both test layers: fast in-process tests for breadth, plus a few subprocess smoke tests that exercise the real compiled binary.

Prerequisite refactor (makes the CLI testable)

main() currently uses the global flag package, prints directly to os.Stdout/os.Stderr, and calls os.Exit — none of which is testable.

Extract the logic into a pure, injectable function in go/main.go:

func run(args []string, stdout, stderr io.Writer) int { ... }

func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) }

• Use a local flag.NewFlagSet("latprobe", flag.ContinueOnError) with fs.SetOutput(stderr); on parse error return exitUsage.
• Replace every fmt.Print* with fmt.Fprint*(stdout/stderr, …).
• Thread an io.Writer through printURL, printResult, printAggregate, printFailureSummary.
• Return the exit code instead of calling os.Exit. This is a mechanical change; no behaviour changes.

Small fix surfaced by the TLS test — go/internal/probe/probe.go

The current classifyErr detects TLS failure via "tlsStart set but tlsDone zero". But Go's httptrace calls TLSHandshakeDone with the error on a failed handshake, so tlsDone is set even on a cert error — meaning a TLS failure is currently misclassified as connect.

Fix (mirrors the existing dnsErr capture): record the handshake error in the TLSHandshakeDone hook into a local tlsErr, and in classifyErr return "tls" when tlsErr != nil. Classification order: dns → timeout → tls → connect (so a deadline during the handshake still classifies as timeout).

Test approach

Stdlib only (testing, net/http/httptest, os/exec, encoding/json) — no third-party deps, consistent with the project.

How each case is triggered (deterministically)

Case	Trigger
Success 200	httptest.NewServer returning 200
HTTP 404 / 500	httptest.NewServer returning the status
DNS failure	URL with reserved .invalid TLD (RFC 6761 — always NXDOMAIN)
Connection refused	net.Listen on 127.0.0.1:0, capture addr, Close(), use that addr
Timeout	server handler blocks on <-r.Context().Done(); client --timeout 200ms (handler returns as soon as the client disconnects, so Close() doesn't hang)
TLS failure	httptest.NewTLSServer (self-signed cert) → default client rejects → cert error

Layer 1 — in-process (go/run_test.go, package main)

Table-driven tests calling run(args, &stdoutBuf, &stderrBuf) and asserting on the returned exit code and output substrings. Cases:

• success 200 → exit 0; output contains Total, DNS lookup
• 500 without --fail → exit 0; output contains (500)
• 404 with --fail → exit 6; output contains 404 ✗
• DNS failure → exit 2; output contains ✗ dns:
• connection refused → exit 3; output contains ✗ connect:
• timeout (--timeout 200ms) → exit 4; output contains ✗ timeout:
• TLS failure → exit 5; output contains ✗ tls:
• multiple URLs, mixed (200 + .invalid) → exit = highest (2)
• -n 3 success → exit 0; output contains 3 samples
• no args → exit 1; stderr contains usage
• --json success → valid JSON, phases.total present, failed == 0
• --json DNS failure → valid JSON, errors[0].phase == "dns", succeeded == 0

JSON cases unmarshal stdout into the []jsonEntry shape (or a mirror struct) and assert on fields — verifying phase presence without brittle text matching.

Layer 2 — subprocess smoke tests (go/cli_test.go, package main)

TestMain builds the binary once with go build -o <tmp>/latprobe and stores the path; tests exec.Command it and read exit code via *exec.ExitError. A small representative set (real os.Exit path, real binary):

• success against a local httptest server → exit 0, stdout has Total
• DNS failure (https://*.invalid) → exit 2
• --json DNS failure → stdout parses as JSON with an errors entry

Files

• go/main.go — refactor to run(...) int (+ thread writer through printers)
• go/internal/probe/probe.go — capture tlsErr, fix classifyErr
• go/run_test.go — new, in-process integration matrix
• go/cli_test.go — new, TestMain + subprocess smoke tests
• docs/usage/step-6-integration-tests.md — how to run the tests
• CHANGELOG.md, README.md — bookkeeping
• docs/plans/2026-07-01-00-49-integration-tests.md — copy of this plan

Verification

cd go
go build ./...
go vet ./...
go test ./...            # all integration + subprocess tests pass
go test -v ./...         # human-readable per-case results showing behaviour

Confirm the matrix covers exit codes 0–6 and that a deliberately broken classification (e.g. revert the TLS fix) makes the TLS case fail — proving the tests actually assert behaviour.

Bookkeeping at execution time

1. Copy this plan to docs/plans/2026-07-01-00-49-integration-tests.md.
2. Add docs/usage/step-6-integration-tests.md.
3. Append a Step 6 entry to CHANGELOG.md; add a Step 6 row to README.md.