http-latency-prober/docs/plans/2026-07-01-00-49-integration-tests.md

# Plan: Step 6 — Integration tests (Go)

## Context

The Go `latprobe` tool is feature-complete (Steps 0–5): per-phase latency,
multiple URLs, `-n` sampling, `--json`, and classified failure handling with
distinct exit codes. There are currently **no automated tests** — every check so
far has been manual.

We want integration tests that demonstrate the application's behaviour across a
**successful run and every failure mode**: HTTP success (200), HTTP error status
(404/500), DNS failure, connection refused, timeout, and TLS handshake failure —
asserting on both the rendered output and the exit code.

The user chose **both test layers**: fast in-process tests for breadth, plus a
few subprocess smoke tests that exercise the real compiled binary.

## Prerequisite refactor (makes the CLI testable)

`main()` currently uses the global `flag` package, prints directly to
`os.Stdout`/`os.Stderr`, and calls `os.Exit` — none of which is testable.

Extract the logic into a pure, injectable function in `go/main.go`:

```go
func run(args []string, stdout, stderr io.Writer) int { ... }

func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) }
```

- Use a local `flag.NewFlagSet("latprobe", flag.ContinueOnError)` with
  `fs.SetOutput(stderr)`; on parse error return `exitUsage`.
- Replace every `fmt.Print*` with `fmt.Fprint*(stdout/stderr, …)`.
- Thread an `io.Writer` through `printURL`, `printResult`, `printAggregate`,
  `printFailureSummary`.
- Return the exit code instead of calling `os.Exit`.
This is a mechanical change; no behaviour changes.

## Small fix surfaced by the TLS test — `go/internal/probe/probe.go`

The current `classifyErr` detects TLS failure via "tlsStart set but tlsDone
zero". But Go's `httptrace` calls `TLSHandshakeDone` **with the error** on a
failed handshake, so `tlsDone` is set even on a cert error — meaning a TLS
failure is currently misclassified as `connect`.

Fix (mirrors the existing `dnsErr` capture): record the handshake error in the
`TLSHandshakeDone` hook into a local `tlsErr`, and in `classifyErr` return
`"tls"` when `tlsErr != nil`. Classification order: dns → timeout → tls →
connect (so a deadline during the handshake still classifies as `timeout`).

## Test approach

Stdlib only (`testing`, `net/http/httptest`, `os/exec`, `encoding/json`) — no
third-party deps, consistent with the project.

### How each case is triggered (deterministically)

| Case | Trigger |
|------|---------|
| Success 200 | `httptest.NewServer` returning 200 |
| HTTP 404 / 500 | `httptest.NewServer` returning the status |
| DNS failure | URL with reserved `.invalid` TLD (RFC 6761 — always NXDOMAIN) |
| Connection refused | `net.Listen` on `127.0.0.1:0`, capture addr, `Close()`, use that addr |
| Timeout | server handler blocks on `<-r.Context().Done()`; client `--timeout 200ms` (handler returns as soon as the client disconnects, so `Close()` doesn't hang) |
| TLS failure | `httptest.NewTLSServer` (self-signed cert) → default client rejects → cert error |

### Layer 1 — in-process (`go/run_test.go`, `package main`)

Table-driven tests calling `run(args, &stdoutBuf, &stderrBuf)` and asserting on
the returned exit code and output substrings. Cases:

- success 200 → exit 0; output contains `Total`, `DNS lookup`
- 500 without `--fail` → exit 0; output contains `(500)`
- 404 with `--fail` → exit 6; output contains `404 ✗`
- DNS failure → exit 2; output contains `✗ dns:`
- connection refused → exit 3; output contains `✗ connect:`
- timeout (`--timeout 200ms`) → exit 4; output contains `✗ timeout:`
- TLS failure → exit 5; output contains `✗ tls:`
- multiple URLs, mixed (200 + `.invalid`) → exit = highest (2)
- `-n 3` success → exit 0; output contains `3 samples`
- no args → exit 1; stderr contains usage
- `--json` success → valid JSON, `phases.total` present, `failed == 0`
- `--json` DNS failure → valid JSON, `errors[0].phase == "dns"`, `succeeded == 0`

JSON cases unmarshal `stdout` into the `[]jsonEntry` shape (or a mirror struct)
and assert on fields — verifying phase presence without brittle text matching.

### Layer 2 — subprocess smoke tests (`go/cli_test.go`, `package main`)

`TestMain` builds the binary once with `go build -o <tmp>/latprobe` and stores
the path; tests `exec.Command` it and read exit code via `*exec.ExitError`.
A small representative set (real `os.Exit` path, real binary):

- success against a local `httptest` server → exit 0, stdout has `Total`
- DNS failure (`https://*.invalid`) → exit 2
- `--json` DNS failure → stdout parses as JSON with an `errors` entry

## Files

- `go/main.go` — refactor to `run(...) int` (+ thread writer through printers)
- `go/internal/probe/probe.go` — capture `tlsErr`, fix `classifyErr`
- `go/run_test.go` — new, in-process integration matrix
- `go/cli_test.go` — new, `TestMain` + subprocess smoke tests
- `docs/usage/step-6-integration-tests.md` — how to run the tests
- `CHANGELOG.md`, `README.md` — bookkeeping
- `docs/plans/2026-07-01-00-49-integration-tests.md` — copy of this plan

## Verification

```sh
cd go
go build ./...
go vet ./...
go test ./...            # all integration + subprocess tests pass
go test -v ./...         # human-readable per-case results showing behaviour
```

Confirm the matrix covers exit codes 0–6 and that a deliberately broken
classification (e.g. revert the TLS fix) makes the TLS case fail — proving the
tests actually assert behaviour.

## Bookkeeping at execution time
1. Copy this plan to `docs/plans/2026-07-01-00-49-integration-tests.md`.
2. Add `docs/usage/step-6-integration-tests.md`.
3. Append a Step 6 entry to `CHANGELOG.md`; add a Step 6 row to `README.md`.