test(go): step 6 — integration tests covering all exit codes and output

Extract run(args, stdout, stderr) int from main() for in-process
testability. Fix TLS failure classification (tlsErr now captured from
TLSHandshakeDone hook). Add run_test.go with 14 table-driven in-process
tests and cli_test.go with TestMain + 3 subprocess smoke tests. All
servers use httptest; .invalid TLD for deterministic DNS failures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-07-01 01:00:20 +02:00
parent c323d879d0
commit a9534ec2c1
8 changed files with 727 additions and 95 deletions

View File

@@ -0,0 +1,125 @@
# Plan: Step 6 — Integration tests (Go)
## Context
The Go `latprobe` tool is feature-complete (Steps 05): per-phase latency,
multiple URLs, `-n` sampling, `--json`, and classified failure handling with
distinct exit codes. There are currently **no automated tests** — every check so
far has been manual.
We want integration tests that demonstrate the application's behaviour across a
**successful run and every failure mode**: HTTP success (200), HTTP error status
(404/500), DNS failure, connection refused, timeout, and TLS handshake failure —
asserting on both the rendered output and the exit code.
The user chose **both test layers**: fast in-process tests for breadth, plus a
few subprocess smoke tests that exercise the real compiled binary.
## Prerequisite refactor (makes the CLI testable)
`main()` currently uses the global `flag` package, prints directly to
`os.Stdout`/`os.Stderr`, and calls `os.Exit` — none of which is testable.
Extract the logic into a pure, injectable function in `go/main.go`:
```go
func run(args []string, stdout, stderr io.Writer) int { ... }
func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) }
```
- Use a local `flag.NewFlagSet("latprobe", flag.ContinueOnError)` with
`fs.SetOutput(stderr)`; on parse error return `exitUsage`.
- Replace every `fmt.Print*` with `fmt.Fprint*(stdout/stderr, …)`.
- Thread an `io.Writer` through `printURL`, `printResult`, `printAggregate`,
`printFailureSummary`.
- Return the exit code instead of calling `os.Exit`.
This is a mechanical change; no behaviour changes.
## Small fix surfaced by the TLS test — `go/internal/probe/probe.go`
The current `classifyErr` detects TLS failure via "tlsStart set but tlsDone
zero". But Go's `httptrace` calls `TLSHandshakeDone` **with the error** on a
failed handshake, so `tlsDone` is set even on a cert error — meaning a TLS
failure is currently misclassified as `connect`.
Fix (mirrors the existing `dnsErr` capture): record the handshake error in the
`TLSHandshakeDone` hook into a local `tlsErr`, and in `classifyErr` return
`"tls"` when `tlsErr != nil`. Classification order: dns → timeout → tls →
connect (so a deadline during the handshake still classifies as `timeout`).
## Test approach
Stdlib only (`testing`, `net/http/httptest`, `os/exec`, `encoding/json`) — no
third-party deps, consistent with the project.
### How each case is triggered (deterministically)
| Case | Trigger |
|------|---------|
| Success 200 | `httptest.NewServer` returning 200 |
| HTTP 404 / 500 | `httptest.NewServer` returning the status |
| DNS failure | URL with reserved `.invalid` TLD (RFC 6761 — always NXDOMAIN) |
| Connection refused | `net.Listen` on `127.0.0.1:0`, capture addr, `Close()`, use that addr |
| Timeout | server handler blocks on `<-r.Context().Done()`; client `--timeout 200ms` (handler returns as soon as the client disconnects, so `Close()` doesn't hang) |
| TLS failure | `httptest.NewTLSServer` (self-signed cert) → default client rejects → cert error |
### Layer 1 — in-process (`go/run_test.go`, `package main`)
Table-driven tests calling `run(args, &stdoutBuf, &stderrBuf)` and asserting on
the returned exit code and output substrings. Cases:
- success 200 → exit 0; output contains `Total`, `DNS lookup`
- 500 without `--fail` → exit 0; output contains `(500)`
- 404 with `--fail` → exit 6; output contains `404 ✗`
- DNS failure → exit 2; output contains `✗ dns:`
- connection refused → exit 3; output contains `✗ connect:`
- timeout (`--timeout 200ms`) → exit 4; output contains `✗ timeout:`
- TLS failure → exit 5; output contains `✗ tls:`
- multiple URLs, mixed (200 + `.invalid`) → exit = highest (2)
- `-n 3` success → exit 0; output contains `3 samples`
- no args → exit 1; stderr contains usage
- `--json` success → valid JSON, `phases.total` present, `failed == 0`
- `--json` DNS failure → valid JSON, `errors[0].phase == "dns"`, `succeeded == 0`
JSON cases unmarshal `stdout` into the `[]jsonEntry` shape (or a mirror struct)
and assert on fields — verifying phase presence without brittle text matching.
### Layer 2 — subprocess smoke tests (`go/cli_test.go`, `package main`)
`TestMain` builds the binary once with `go build -o <tmp>/latprobe` and stores
the path; tests `exec.Command` it and read exit code via `*exec.ExitError`.
A small representative set (real `os.Exit` path, real binary):
- success against a local `httptest` server → exit 0, stdout has `Total`
- DNS failure (`https://*.invalid`) → exit 2
- `--json` DNS failure → stdout parses as JSON with an `errors` entry
## Files
- `go/main.go` — refactor to `run(...) int` (+ thread writer through printers)
- `go/internal/probe/probe.go` — capture `tlsErr`, fix `classifyErr`
- `go/run_test.go` — new, in-process integration matrix
- `go/cli_test.go` — new, `TestMain` + subprocess smoke tests
- `docs/usage/step-6-integration-tests.md` — how to run the tests
- `CHANGELOG.md`, `README.md` — bookkeeping
- `docs/plans/2026-07-01-00-49-integration-tests.md` — copy of this plan
## Verification
```sh
cd go
go build ./...
go vet ./...
go test ./... # all integration + subprocess tests pass
go test -v ./... # human-readable per-case results showing behaviour
```
Confirm the matrix covers exit codes 06 and that a deliberately broken
classification (e.g. revert the TLS fix) makes the TLS case fail — proving the
tests actually assert behaviour.
## Bookkeeping at execution time
1. Copy this plan to `docs/plans/2026-07-01-00-49-integration-tests.md`.
2. Add `docs/usage/step-6-integration-tests.md`.
3. Append a Step 6 entry to `CHANGELOG.md`; add a Step 6 row to `README.md`.