# Execution log: OpenTelemetry tracing integrated with logr/zap logging

Plan: `docs/plans/2026-08-24-1025-otel-tracing.md`

- [x] Step 1 — Dependencies
- [x] Step 2 — New package `internal/tracing`
- [ ] Step 3 — `provider.WithTracing` decorator
- [ ] Step 4 — `cmd/main.go` wiring
- [ ] Step 5 — Reconciler spans
- [ ] Step 6 — Discovery server
- [ ] Step 7 — GC + health
- [ ] Step 8 — GCP wire-log enrichment
- [ ] Step 9 — Manifests + docs

## Step 1 — Dependencies

Aligned the pre-existing indirect skew (otel core v1.44.0 vs otlptrace exporters
v1.40.0) and added the new direct deps in one shot:

```bash
go get go.opentelemetry.io/otel@v1.45.0 \
  go.opentelemetry.io/otel/sdk@v1.45.0 \
  go.opentelemetry.io/otel/trace@v1.45.0 \
  go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc@v1.45.0 \
  go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp@v1.45.0 \
  go.opentelemetry.io/otel/exporters/stdout/stdouttrace@v1.45.0 \
  go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.70.0
go mod tidy
```

MVS side-effects worth recording: `logr v1.4.3→v1.4.4`, `httpsnoop
v1.0.4→v1.1.0`, `grpc-gateway/v2 v2.27.7→v2.29.0`, `proto/otlp v1.9.0→v1.11.0`,
plus a `genproto/googleapis/api` pseudo-version bump. Full `make test` passed
against the bumped graph (`google.golang.org/api v0.292.0` and k8s v0.36
tolerate otelhttp v0.70.0).

Worth noting: the first `go mod tidy` ran *before* any first-party code
imported `otlptracehttp`/`stdouttrace`, so it silently dropped those two
modules again; the Step 2 tidy re-added them. The plan's semconv question
resolved to `semconv/v1.43.0` — that's what `sdk@v1.45.0/resource/builtin.go`
imports, so first-party code uses the same version to avoid
`ErrSchemaURLConflict` in the common path.

## Step 2 — `internal/tracing` package

Landed as planned: `tracing.go` (env-gated `Setup`, hand-rolled exporter
selection), `logger.go` (`Start`/`StartSpan`/`ContextWithLogger` with the
base-logger ctx key that prevents duplicate `traceID` zap fields on nested
spans), `reconciler.go`, `transport.go`, `http.go`, `options.go`; tests for
all of it (first use of `sdk/trace/tracetest` in the repo).

Two deviations from the plan's letter:

- **k8s transport gating is a custom RoundTripper, not `otelhttp.WithFilter`**
  (`parentGatedTransport`): requests without a parent span bypass the otel
  transport entirely, so the no-root-spans-from-informers guarantee doesn't
  depend on otelhttp filter semantics for transports.
- **`HTTPMiddleware` must copy the route pattern back.** The logger-injecting
  inner handler wraps the request via `WithContext` (a shallow copy), so the
  mux records the matched pattern on the copy while otelhttp's post-routing
  span rename reads the original. Found by the middleware test (span named
  `"GET"` instead of `"GET /v1/things/{id}"`); fixed with `r.Pattern =
  r2.Pattern` after `next.ServeHTTP`.

Also: both the transport and the middleware pass explicit W3C propagators
instead of relying on the global, so behavior is deterministic under tests
and when tracing is disabled. `Setup` tests reset the global provider to a
fresh noop per case — restoring otel's own default delegate triggers a
"Setting tracer provider to its current value" warning from the SDK.