docs: rewrite CLAUDE.md to house conventions, fix flows.md and README

- CLAUDE.md in five sections with Go, testing, plan, changelog, branching
  and tagging rules; known sharp edges moved to Gotchas
- docs/flows.md: metrics are served on :8080, name collisions are rejected
  by the API server, last-cert deletion never cleans up
- README moved to the repo root, deploy command corrected

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-08 20:31:00 +02:00
co-authored by Claude Opus 5.5
parent 36030f4011
commit 7a559bacd0
4 changed files with 764 additions and 6 deletions
+493
View File
@@ -0,0 +1,493 @@
# Runtime Flows
Deep-dive reference for every control path in the gateway-cert-operator. Complements the user-facing overview in `docs/README.md`.
---
## 1. Overview
The operator sits between two Kubernetes APIs and keeps them in sync:
```
+--------------------+ +------------------------+
| cert-manager | | gateway-cert-operator |
| Certificate |----(watches)-------->| (this binary) |
| (source of | | |
| truth for TLS) | +----------+-------------+
+--------------------+ |
| patches Spec.Listeners
v
+--------------------+ +------------------------+
| Gateway API | | Gateway |
| controller |<----(watches)--------| (target object) |
| (routes traffic) | +------------------------+
+--------------------+
```
The operator makes no changes to cert-manager objects. Its only write action is a `PATCH` on `Gateway.Spec.Listeners`. All other interactions are reads.
---
## 2. Startup Flow
Executed once at process start (`main.go`):
```
process start
│
├─ init() [main.go:22-26]
│ ├─ clientgoscheme.AddToScheme(scheme) # core/v1: Secrets, Events, …
│ ├─ cmv1.AddToScheme(scheme) # cert-manager.io/v1 Certificate
│ └─ gwapiv1.Install(scheme) # gateway.networking.k8s.io/v1 Gateway
│
└─ main() [main.go:28]
│
├─ flag.StringVar --metrics-bind-address default ":8080" [main.go:32]
├─ flag.StringVar --health-probe-bind-address default ":8081" [main.go:33]
├─ flag.Parse() [main.go:34]
│
├─ ctrl.SetLogger(zap, UseDevMode=true, AddCaller) [main.go:36-41]
│ dev-mode console logger with file:line call site
│
├─ ctrl.GetConfigOrDie() # KUBECONFIG or in-cluster config
│ on err ──► os.Exit(1)
│
├─ ctrl.NewManager(cfg, Options{Scheme, HealthProbeBindAddress: ":8081"})
│ on err ──► log + os.Exit(1) [main.go:44-47]
│ NOTE: metricsAddr is parsed above but NOT passed here;
│ the manager falls back to its own default, which
│ is also ":8080" (see §10, item 2)
│
├─ controller.SetupCertificateReconciler(mgr) [main.go:53]
│ registers field index + watches (see §3)
│ on err ──► log + os.Exit(1)
│
├─ mgr.AddHealthzCheck("healthz", healthz.Ping) → GET /healthz:8081
├─ mgr.AddReadyzCheck("readyz", healthz.Ping) → GET /readyz:8081
│ on err ──► log + os.Exit(1)
│
└─ mgr.Start(ctrl.SetupSignalHandler()) [main.go:68]
SetupSignalHandler installs SIGINT / SIGTERM cancel
mgr.Start blocks; on shutdown → graceful stop
on err ──► log + os.Exit(1)
```
---
## 3. Watch Wiring and Event Sources
`SetupCertificateReconciler` registers two watches before starting the controller
(`certificate_controller.go:59-63`):
```
controller-runtime manager
│
├─ For(&cmv1.Certificate{}) primary watch
│ every Certificate create/update/delete
│ ──► enqueue Reconcile(cert.Namespace/cert.Name)
│
└─ Watches(&gwapiv1.Gateway{},
EnqueueRequestsFromMapFunc(r.gatewayToCertificates))
secondary watch
every Gateway create/update/delete
──► gatewayToCertificates maps Gateway → []Certificates
──► enqueue Reconcile for each matching Certificate
```
Full trigger table:
| ID | Event | Enqueued object | Code path |
|----|-------|-----------------|-----------|
| E1 | Certificate **create** | that Certificate | primary watch |
| E2 | Certificate **update** (any field — no predicate filter) | that Certificate | primary watch |
| E3 | Certificate **delete** | that Certificate | primary watch |
| E4 | Gateway **create / update / delete** | every Certificate targeting it (via field index) | secondary watch + `gatewayToCertificates` |
| E5 | Reconcile returns non-nil error | same Certificate | controller-runtime rate-limited backoff re-queue |
| E6 | Controller-runtime cache resync (~10 h default) | every known Certificate | implicit periodic replay |
---
## 4. Field Index
The field index is the only mechanism that lets the operator find all Certificates pointing at a given Gateway without scanning the entire namespace. It is registered during setup and queried in two places.
```
IndexFieldGatewayTarget = ".metadata.annotations.gatewayTarget"
(identifier used by controller-runtime; no annotation by this name exists)
Indexer function (certificate_controller.go:44-55):
┌─────────────────────────────────────────────────────────┐
│ for each Certificate obj: │
│ gwName = obj.Annotations["…/gateway-name"] │
│ if gwName == "" → return nil (not indexed) │
│ gwNs = obj.Annotations["…/gateway-namespace"] │
│ if gwNs == "" → gwNs = obj.Namespace (fallback) │
│ return [ "<gwNs>/<gwName>" ] │
└─────────────────────────────────────────────────────────┘
Key shape: "<targetGatewayNamespace>/<targetGatewayName>"
Two query sites:
┌──────────────────────────────────────────────────────┐
│ Reconcile [certificate_controller.go:95-99] │
│ List(Certs, MatchingFields{index: "<gwNs>/<gwName>"})│
│ returns all sibling Certs targeting the same GW │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ gatewayToCertificates [certificate_controller.go:138]│
│ List(Certs, MatchingFields{index: "<gw.Ns>/<gw.Name>"})│
│ used by the secondary watch to fan-out GW events │
│ to the relevant Certs │
└──────────────────────────────────────────────────────┘
```
The namespace-fallback rule (`gwNs = obj.Namespace` when the annotation is absent) is duplicated in two places: the indexer (`:51-53`) and `Reconcile` (`:89-91`). Both must apply the same rule or the index key used during write will differ from the key used during lookup.
---
## 5. Reconcile Decision Tree
Every enqueued item arrives here. Inputs: one `Reconcile.Request` containing the namespace/name of a Certificate. Side effects: 0 or 1 `PATCH` on a Gateway.
```
Reconcile(ctx, req)
│
├─(A) r.Get(ctx, req.NamespacedName, &cert) [controller:71]
│ │
│ ├─ NotFound ──► client.IgnoreNotFound → nil err
│ │ return Result{}, nil ►► STOP — NO ACTION
│ │
│ │ ┌─ IMPORTANT: the deleted cert's listener is NOT removed here.
│ │ │ The controller has no record of which Gateway was targeted.
│ │ │ Cleanup happens on the next E2/E4/E6 event for any sibling
│ │ │ cert that still references the same Gateway — at that point
│ │ │ BuildListenersFromCertificates omits the deleted cert and
│ │ │ MergeListeners replaces all auto-* listeners. Until that
│ │ │ event fires, the stale listener remains in the Gateway.
│ │ │ (See §10, item 3 for details.)
│ │ └───────────────────────────────────────────────────────────
│ │
│ └─ other error ──► return Result{}, err ►► REQUEUE (E5)
│
├─(B) gwName, ok := cert.Annotations["…/gateway-name"] [controller:82]
│ │
│ └─ !ok ──► log V(1) "missing gateway annotation"
│ return Result{}, nil ►► STOP — NO ACTION
│
├─(C) gwNamespace := cert.Annotations["…/gateway-namespace"] [controller:88]
│ └─ if "" ──► gwNamespace = cert.Namespace (DEFAULT FALLBACK)
│
├─(D) r.List(ctx, &certList, [controller:95]
│ MatchingFields{IndexFieldGatewayTarget: "<gwNs>/<gwName>"})
│ │
│ └─ err ──► return Result{}, err ►► REQUEUE
│
├─(E) desiredListeners :=
│ gw.BuildListenersFromCertificates(certList.Items, "auto-")
│ (see §6a) [controller:102]
│
├─(F) r.Get(ctx, {gwNs, gwName}, &gateway) [controller:106]
│ │
│ └─ err (including NotFound) ──► return Result{}, err ►► REQUEUE
│
│ NOTE: Gateway-not-found is NOT specially handled — the
│ reconciler will retry with backoff indefinitely. A typo'd
│ gateway-name annotation produces a perpetual retry loop.
│ (See §10, item 5.)
│
├─(G) updated := gw.MergeListeners( [controller:111]
│ gateway.Spec.Listeners,
│ desiredListeners, "auto-")
│ (see §6b)
│
├─(H) gw.ListenersEqual(gateway.Spec.Listeners, updated) [controller:113]
│ │
│ └─ true ──► log V(1) "already up to date"
│ return Result{}, nil ►► SHORT-CIRCUIT — NO PATCH
│
├─(I) patch := client.MergeFrom(gateway.DeepCopy()) [controller:118]
│ gateway.Spec.Listeners = updated
│ r.Patch(ctx, &gateway, patch) [controller:120]
│ │
│ └─ err ──► return Result{}, err ►► REQUEUE
│
└─(J) log Info "patched gateway listeners" [controller:124]
return Result{}, nil ►► DONE
Cluster I/O per reconcile:
Reads: 1 Certificate Get (step A)
1 Certificate List via index (step D)
1 Gateway Get (step F)
Writes: 0 or 1 Gateway Patch (step I, only when listeners changed)
```
---
## 6. Build / Merge / Equal Pipeline
The three pure functions in `internal/gateway/patch.go` do all computation. `Reconcile` only calls them; it holds no business logic of its own.
```
certList.Items
│
v
BuildListenersFromCertificates(certs, "auto-") [patch.go:22]
│
└──► desired []gwapiv1.Listener
│
│ gateway.Spec.Listeners (current state from API server)
│ │
v v
MergeListeners(existing, desired, "auto-") [patch.go:76]
│
└──► updated []gwapiv1.Listener
│
v
ListenersEqual(gateway.Spec.Listeners, updated) [patch.go:93]
│
├─ true ──► short-circuit, no patch
└─ false ──► Patch(gateway)
```
### 6a. BuildListenersFromCertificates
```
for cert in certs:
│
├─ if cert.DeletionTimestamp != nil ──► skip [patch.go:26]
│ (cert is being deleted; omitting it removes its listeners
│ from the desired set and MergeListeners will drop them)
│
├─ if cert.Spec.SecretName == "" ──► skip [patch.go:30]
│
└─ for dns in cert.Spec.DNSNames: [patch.go:34]
│
└─ emit Listener{
Name: "auto-<certName>-<sanitized-dns>"
Hostname: dns
Port: 443 (hardcoded, see §8)
Protocol: HTTPS
AllowedRoutes: { From: All }
TLS: {
Mode: Terminate
CertificateRefs: [{
Group: "" (core)
Kind: "Secret"
Name: cert.Spec.SecretName
Namespace: cert.Namespace (same as cert)
}]
}
}
sort by Name ascending [patch.go:67]
(deterministic order prevents spurious diffs on every reconcile)
```
### 6b. MergeListeners
```
existing listeners
│
├─ Name starts with "auto-" ──► DROP (managed, will be replaced)
└─ Name does NOT start with "auto-" ──► KEEP (user-managed)
merged = [kept user listeners] + [all desired auto-* listeners]
```
The result: user-created listeners appear first in their original order; operator-managed listeners follow, sorted by name.
**Edge case:** a user-created listener whose name happens to start with `auto-` will be treated as managed and will be deleted on the next reconcile.
### 6c. ListenersEqual
`ListenersEqual` is a **partial comparison** — only the fields the operator actively manages are checked:
| Compared | Ignored |
|----------|---------|
| listener `Name` | `Protocol` |
| `Hostname` | `AllowedRoutes` |
| `Port` | `TLS.Options` |
| first `CertificateRef` (`<ns>/<name>`) | cert refs beyond index 0 |
| | `Group`, `Kind` of ref |
Drift in an ignored field will not trigger a corrective patch. For example: if someone manually flips the Protocol of an auto-managed listener from HTTPS to HTTP, the operator will consider it "equal" and leave it alone.
The comparison is done via a name-keyed map (`listenerMap`, `patch.go:118`), so listener order does not affect the result.
---
## 7. Listener Naming and Sanitization
`listenerNameFor` (`patch.go:143`):
```
input: prefix="auto-" certName="my-app-cert" dnsName="app.example.com"
step 1: replace every "." with "-"
"app.example.com" ──► "app-example-com"
step 2: concatenate
"auto-" + "my-app-cert" + "-" + "app-example-com"
──► "auto-my-app-cert-app-example-com"
step 3: if len > 253 bytes: name = name[:253] (hard byte slice)
```
Example outputs for a cert with two DNS names:
| `dnsName` | listener `Name` |
|-----------|-----------------|
| `app.example.com` | `auto-my-app-cert-app-example-com` |
| `*.example.com` | `auto-my-app-cert-*-example-com` |
The second entry is a risk: `*` is not valid in a Gateway API `SectionName` (which must match RFC 1123). The API server will reject the patch and the reconciler will requeue forever (see §10, item 7).
The sanitization is also lossy: `a.b.com` and `a-b-com` produce the same result. If a certificate lists both, two listeners with the same name are appended; the API server rejects the patch (listener names must be unique), so no listener on that Gateway is updated.
---
## 8. Defaults Baked In
All of these are hardcoded in `internal/gateway/patch.go`. None are configurable via annotations (the `listener-port` annotation exists as a constant but is not read — see §10, item 1).
| Property | Value | Source |
|----------|-------|--------|
| Listener port | `443` | `defaultPort` `patch.go:13` |
| TLS mode | `TLSModeTerminate` | `tlsModeTerminate` `patch.go:14` |
| Allowed routes | `NamespacesFromAll` | `namespacesFromAll` `patch.go:15` |
| Protocol | `HTTPSProtocolType` | inline `patch.go:45` |
| CertificateRef Group | `""` (core API group) | `patch.go:55` |
| CertificateRef Kind | `"Secret"` | `patch.go:56` |
| Secret namespace | cert's own namespace | `patch.go:39,58` |
| Listener name prefix | `"auto-"` | `ListenerPrefix` `controller:28` |
| SectionName max length | 253 bytes | `patch.go:147` |
| Gateway namespace fallback | cert's own namespace | `controller:51-53` and `controller:89-91` |
---
## 9. End-to-End Scenarios
### Scenario 1 — First Certificate created, Gateway has no auto-listeners
```
1. User creates Certificate "my-cert" (namespace: default)
Annotations: gateway-name=my-gateway
Spec: dnsNames=[app.example.com], secretName=my-cert-tls
2. (E1) Cert create event fires → Reconcile(default/my-cert)
3. (A) Get cert → found
(B) gateway-name="my-gateway" → ok
(C) gateway-namespace missing → fallback to "default"
(D) List certs by index "default/my-gateway" → [my-cert]
(E) BuildListeners → [Listener{Name:"auto-my-cert-app-example-com", Port:443, …}]
(F) Get gateway "default/my-gateway" → found, Listeners=[]
(G) MergeListeners([], desired) → [auto-my-cert-app-example-com]
(H) ListenersEqual([], updated) → false (len differs)
(I) Patch gateway → Spec.Listeners=[auto-my-cert-app-example-com]
(J) Done. Log: "patched gateway listeners"
Cluster writes: 1 Gateway PATCH
```
### Scenario 2 — Second Certificate added for the same Gateway
```
1. User creates Certificate "other-cert" targeting the same Gateway
dnsNames=[api.example.com], secretName=other-cert-tls
2. (E1) Reconcile(default/other-cert)
3. (D) List certs by index "default/my-gateway" → [my-cert, other-cert]
(E) BuildListeners → [
auto-my-cert-app-example-com,
auto-other-cert-api-example-com
] (sorted alphabetically by name)
(F) Gateway still has [auto-my-cert-app-example-com]
(G) MergeListeners → [auto-my-cert-app-example-com,
auto-other-cert-api-example-com]
(H) ListenersEqual(old 1-item, new 2-item) → false
(I) Patch → Gateway now has both listeners
Cluster writes: 1 Gateway PATCH
```
### Scenario 3 — Certificate dnsNames updated
```
1. User edits "my-cert": dnsNames=[app.example.com] → [app.example.com, www.example.com]
2. (E2) Cert update event → Reconcile(default/my-cert)
3. (E) BuildListeners produces TWO listeners for my-cert now:
auto-my-cert-app-example-com
auto-my-cert-www-example-com
(plus any from other-cert if still present)
(H) ListenersEqual: len changed → false
(I) Patch gateway with new full set
Cluster writes: 1 Gateway PATCH
```
### Scenario 4 — Certificate deleted
```
1. User deletes "my-cert"
2. (E3) Cert delete event → Reconcile(default/my-cert)
3. (A) Get cert → NotFound
return Result{}, nil ──► STOP, NO ACTION
4. Gateway still contains "auto-my-cert-app-example-com" — stale.
5. Cleanup happens only when the next relevant event fires:
- If "other-cert" is updated or created → (E2) Reconcile(other-cert)
(D) List by index returns [other-cert] only (my-cert is gone)
(E) BuildListeners → only other-cert's listeners
(G) MergeListeners drops the stale auto-my-cert-* listener
(I) Patch removes it
- OR if the Gateway is manually edited → (E4) all targeting Certs
re-enqueued, same convergence path
- OR on the next cache resync (E6, ~10 h)
Every path above needs at least one surviving Certificate that targets
the Gateway. If my-cert was the last one, nothing is ever enqueued for
it and the stale listener stays until someone removes it by hand.
Cluster writes: 0 at deletion time; 1 Gateway PATCH at next sibling event
```
---
## 10. Known Limitations
Recorded as-found. None are fixed by this document.
**1. `AnnotationListenerPort` is dead code**
`certificate_controller.go:25` declares the constant; `BuildListenersFromCertificates` (`patch.go:44`) always uses `defaultPort = 443`. Setting `gateway-cert-operator.io/listener-port` on a Certificate has no effect.
**2. `--metrics-bind-address` flag is parsed but not wired**
`main.go:32` parses the flag; `main.go:44-47` never passes `MetricsBindAddress` to `ctrl.Options`. controller-runtime defaults an empty bind address to `:8080`, so metrics are served and the Service in `deploy/service_gateway-cert-operator-metrics.yaml` works; only changing the flag has no effect.
**3. No finalizer → cert deletion leaves stale listeners**
When a Certificate is deleted, Reconcile returns early at branch (A) with no cluster write. The stale `auto-*` listener persists in the Gateway until a sibling event triggers re-convergence (see Scenario 4 above), or permanently if no other Certificate targets that Gateway. Adding a finalizer to the Certificate would close this gap by allowing cleanup before the object is removed from the API server.
**4. `ListenersEqual` is a partial comparison**
Described in §6c. Drift in `Protocol`, `AllowedRoutes`, `TLS.Options`, or any `CertificateRef` beyond the first will not be detected or corrected.
**5. Gateway-not-found retries forever**
Branch (F) of Reconcile treats all Gateway `Get` errors the same (including `NotFound`) and returns them as re-queue-able errors. A Certificate with a typo'd `gateway-name` annotation will retry with rate-limited backoff indefinitely, generating log noise and API load. There is no terminal failure path.
**6. No leader election**
`ctrl.NewManager` at `main.go:44` does not set `LeaderElection: true`. The `deploy/deployment_gateway-cert-operator.yaml` runs `replicas: 1` which makes this safe today. Scaling to multiple replicas without enabling leader election would result in concurrent Gateway patches.
**7. `listenerNameFor` edge cases**
- **Wildcard hostnames**: `*.example.com` produces `SectionName` `auto-…-*-example-com`. The `*` character is not valid in an RFC 1123 label; the Gateway API server will reject the Patch, causing an error-retry loop.
- **Collisions after sanitization**: `a.b.com` and `a-b-com` produce identical sanitized names. The same happens across certificates (cert `a` with `b-c.com` and cert `a-b` with `c.com`) and across namespaces, since the name omits the cert namespace. Both listeners are appended and the API server rejects the patch for the whole Gateway.
- **Truncation at 253 bytes**: `patch.go:147` slices on bytes, not runes (safe for ASCII DNS names). Two DNS names sharing the first 253 characters after sanitization will map to the same listener name.
**Dead symbols in `patch.go`**
`gatewayGroup` and `gatewayKind` are declared at `patch.go:16-17` but never referenced. They can be removed without effect.