gatewayapi-certificates-operator/docs/flows.md

Runtime Flows

Deep-dive reference for every control path in the gateway-cert-operator. Complements the user-facing overview in docs/README.md.

---

1. Overview

The operator sits between two Kubernetes APIs and keeps them in sync:

   +--------------------+                      +------------------------+
   |   cert-manager     |                      |  gateway-cert-operator |
   |   Certificate      |----(watches)-------->|  (this binary)         |
   |   (source of       |                      |                        |
   |    truth for TLS)  |                      +----------+-------------+
   +--------------------+                                 |
                                                          | patches Spec.Listeners
                                                          v
   +--------------------+                      +------------------------+
   |   Gateway API      |                      |   Gateway              |
   |   controller       |<----(watches)--------|   (target object)      |
   |   (routes traffic) |                      +------------------------+
   +--------------------+

The operator makes no changes to cert-manager objects. Its only write action is a PATCH on Gateway.Spec.Listeners. All other interactions are reads.

---

2. Startup Flow

Executed once at process start (main.go):

process start
    │
    ├─ init()                                                [main.go:22-26]
    │    ├─ clientgoscheme.AddToScheme(scheme)   # core/v1: Secrets, Events, …
    │    ├─ cmv1.AddToScheme(scheme)             # cert-manager.io/v1 Certificate
    │    └─ gwapiv1.Install(scheme)              # gateway.networking.k8s.io/v1 Gateway
    │
    └─ main()                                               [main.go:28]
         │
         ├─ flag.StringVar  --metrics-bind-address   default ":8080"   [main.go:32]
         ├─ flag.StringVar  --health-probe-bind-address default ":8081" [main.go:33]
         ├─ flag.Parse()                                                [main.go:34]
         │
         ├─ ctrl.SetLogger(zap, UseDevMode=true, AddCaller)             [main.go:36-41]
         │      dev-mode console logger with file:line call site
         │
         ├─ ctrl.GetConfigOrDie()       # KUBECONFIG or in-cluster config
         │      on err ──► os.Exit(1)
         │
         ├─ ctrl.NewManager(cfg, Options{Scheme, Metrics, HealthProbeBindAddress})
         │      on err ──► log + os.Exit(1)                            [main.go:44-47]
         │
         ├─ controller.SetupCertificateReconciler(mgr)                  [main.go:53]
         │      registers field index + watches (see §3)
         │      on err ──► log + os.Exit(1)
         │
         ├─ mgr.AddHealthzCheck("healthz", healthz.Ping)  → GET /healthz:8081
         ├─ mgr.AddReadyzCheck("readyz",  healthz.Ping)   → GET /readyz:8081
         │      on err ──► log + os.Exit(1)
         │
         └─ mgr.Start(ctrl.SetupSignalHandler())                        [main.go:68]
                SetupSignalHandler installs SIGINT / SIGTERM cancel
                mgr.Start blocks; on shutdown → graceful stop
                on err ──► log + os.Exit(1)

---

3. Watch Wiring and Event Sources

SetupCertificateReconciler registers two watches before starting the controller (certificate_controller.go:59-63):

   controller-runtime manager
        │
        ├─ For(&cmv1.Certificate{})               primary watch
        │    every Certificate create/update/delete
        │    ──► enqueue Reconcile(cert.Namespace/cert.Name)
        │
        └─ Watches(&gwapiv1.Gateway{},
                   EnqueueRequestsFromMapFunc(r.gatewayToCertificates))
                                                  secondary watch
             every Gateway create/update/delete
             ──► gatewayToCertificates maps Gateway → []Certificates
             ──► enqueue Reconcile for each matching Certificate

Full trigger table:

ID	Event	Enqueued object	Code path
E1	Certificate create	that Certificate	primary watch
E2	Certificate update (any field — no predicate filter)	that Certificate	primary watch
E3	Certificate delete	that Certificate	primary watch
E4	Gateway create / update / delete	every Certificate targeting it (via field index)	secondary watch + gatewayToCertificates
E5	Reconcile returns non-nil error	same Certificate	controller-runtime rate-limited backoff re-queue
E6	Controller-runtime cache resync (~10 h default)	every known Certificate	implicit periodic replay

---

4. Field Index

The field index is the only mechanism that lets the operator find all Certificates pointing at a given Gateway without scanning the entire namespace. It is registered during setup and queried in two places.

   IndexFieldGatewayTarget = ".metadata.annotations.gatewayTarget"
   (identifier used by controller-runtime; no annotation by this name exists)

   Indexer function (certificate_controller.go:44-55):
   ┌─────────────────────────────────────────────────────────┐
   │  for each Certificate obj:                              │
   │    gwName = obj.Annotations["…/gateway-name"]           │
   │    if gwName == ""  →  return nil  (not indexed)        │
   │    gwNs   = obj.Annotations["…/gateway-namespace"]      │
   │    if gwNs == ""    →  gwNs = obj.Namespace  (fallback) │
   │    return [ "<gwNs>/<gwName>" ]                         │
   └─────────────────────────────────────────────────────────┘

   Key shape:  "<targetGatewayNamespace>/<targetGatewayName>"

   Two query sites:

        ┌──────────────────────────────────────────────────────┐
        │ Reconcile  [certificate_controller.go:95-99]         │
        │   List(Certs, MatchingFields{index: "<gwNs>/<gwName>"})│
        │   returns all sibling Certs targeting the same GW    │
        └──────────────────────────────────────────────────────┘

        ┌──────────────────────────────────────────────────────┐
        │ gatewayToCertificates  [certificate_controller.go:138]│
        │   List(Certs, MatchingFields{index: "<gw.Ns>/<gw.Name>"})│
        │   used by the secondary watch to fan-out GW events   │
        │   to the relevant Certs                              │
        └──────────────────────────────────────────────────────┘

The namespace-fallback rule (gwNs = obj.Namespace when the annotation is absent) is duplicated in two places: the indexer (:51-53) and Reconcile (:89-91). Both must apply the same rule or the index key used during write will differ from the key used during lookup.

---

5. Reconcile Decision Tree

Every enqueued item arrives here. Inputs: one Reconcile.Request containing the namespace/name of a Certificate. Side effects: 0 or 1 PATCH on a Gateway.

Reconcile(ctx, req)
    │
    ├─(A) r.Get(ctx, req.NamespacedName, &cert)              [controller:71]
    │       │
    │       ├─ NotFound ──► client.IgnoreNotFound → nil err
    │       │               return Result{}, nil  ►► STOP — NO ACTION
    │       │
    │       │   ┌─ IMPORTANT: the deleted cert's listener is NOT removed here.
    │       │   │  The controller has no record of which Gateway was targeted.
    │       │   │  Cleanup happens on the next E2/E4/E6 event for any sibling
    │       │   │  cert that still references the same Gateway — at that point
    │       │   │  BuildListenersFromCertificates omits the deleted cert and
    │       │   │  MergeListeners replaces all auto-* listeners. Until that
    │       │   │  event fires, the stale listener remains in the Gateway.
    │       │   │  (See §10, item 3 for details.)
    │       │   └───────────────────────────────────────────────────────────
    │       │
    │       └─ other error ──► return Result{}, err  ►► REQUEUE (E5)
    │
    ├─(B) gwName, ok := cert.Annotations["…/gateway-name"]  [controller:82]
    │       │
    │       └─ !ok ──► log V(1) "missing gateway annotation"
    │                   return Result{}, nil  ►► STOP — NO ACTION
    │
    ├─(C) gwNamespace := cert.Annotations["…/gateway-namespace"]  [controller:88]
    │       └─ if "" ──► gwNamespace = cert.Namespace   (DEFAULT FALLBACK)
    │
    ├─(D) r.List(ctx, &certList,                             [controller:95]
    │           MatchingFields{IndexFieldGatewayTarget: "<gwNs>/<gwName>"})
    │       │
    │       └─ err ──► return Result{}, err  ►► REQUEUE
    │
    ├─(E) desiredListeners :=
    │         gw.BuildListenersFromCertificates(certList.Items, "auto-")
    │       (see §6a)                                        [controller:102]
    │
    ├─(F) r.Get(ctx, {gwNs, gwName}, &gateway)               [controller:106]
    │       │
    │       └─ err (including NotFound) ──► return Result{}, err  ►► REQUEUE
    │
    │           NOTE: Gateway-not-found is NOT specially handled — the
    │           reconciler will retry with backoff indefinitely. A typo'd
    │           gateway-name annotation produces a perpetual retry loop.
    │           (See §10, item 5.)
    │
    ├─(G) updated := gw.MergeListeners(                      [controller:111]
    │                   gateway.Spec.Listeners,
    │                   desiredListeners, "auto-")
    │       (see §6b)
    │
    ├─(H) gw.ListenersEqual(gateway.Spec.Listeners, updated) [controller:113]
    │       │
    │       └─ true ──► log V(1) "already up to date"
    │                   return Result{}, nil  ►► SHORT-CIRCUIT — NO PATCH
    │
    ├─(I) patch := client.MergeFrom(gateway.DeepCopy())      [controller:118]
    │       gateway.Spec.Listeners = updated
    │       r.Patch(ctx, &gateway, patch)                    [controller:120]
    │       │
    │       └─ err ──► return Result{}, err  ►► REQUEUE
    │
    └─(J) log Info "patched gateway listeners"               [controller:124]
           return Result{}, nil  ►► DONE

Cluster I/O per reconcile:
  Reads:  1 Certificate Get  (step A)
          1 Certificate List via index  (step D)
          1 Gateway Get  (step F)
  Writes: 0 or 1 Gateway Patch  (step I, only when listeners changed)

---

6. Build / Merge / Equal Pipeline

The three pure functions in internal/gateway/patch.go do all computation. Reconcile only calls them; it holds no business logic of its own.

   certList.Items
        │
        v
   BuildListenersFromCertificates(certs, "auto-")          [patch.go:22]
        │
        └──► desired []gwapiv1.Listener
                │
                │    gateway.Spec.Listeners (current state from API server)
                │         │
                v         v
   MergeListeners(existing, desired, "auto-")              [patch.go:76]
        │
        └──► updated []gwapiv1.Listener
                │
                v
   ListenersEqual(gateway.Spec.Listeners, updated)         [patch.go:93]
        │
        ├─ true  ──► short-circuit, no patch
        └─ false ──► Patch(gateway)

6a. BuildListenersFromCertificates

for cert in certs:
    │
    ├─ if cert.DeletionTimestamp != nil  ──► skip          [patch.go:26]
    │     (cert is being deleted; omitting it removes its listeners
    │      from the desired set and MergeListeners will drop them)
    │
    ├─ if cert.Spec.SecretName == ""    ──► skip          [patch.go:30]
    │
    └─ for dns in cert.Spec.DNSNames:                     [patch.go:34]
            │
            └─ emit Listener{
                 Name:          "auto-<certName>-<sanitized-dns>"
                 Hostname:      dns
                 Port:          443           (hardcoded, see §8)
                 Protocol:      HTTPS
                 AllowedRoutes: { From: All }
                 TLS: {
                   Mode:  Terminate
                   CertificateRefs: [{
                     Group: ""  (core)
                     Kind:  "Secret"
                     Name:  cert.Spec.SecretName
                     Namespace: cert.Namespace  (same as cert)
                   }]
                 }
               }

sort by Name ascending                                    [patch.go:67]
(deterministic order prevents spurious diffs on every reconcile)

6b. MergeListeners

existing listeners
    │
    ├─ Name starts with "auto-"  ──► DROP   (managed, will be replaced)
    └─ Name does NOT start with "auto-"  ──► KEEP (user-managed)

merged = [kept user listeners] + [all desired auto-* listeners]

The result: user-created listeners appear first in their original order; operator-managed listeners follow, sorted by name.

Edge case: a user-created listener whose name happens to start with auto- will be treated as managed and will be deleted on the next reconcile.

6c. ListenersEqual

ListenersEqual is a partial comparison — only the fields the operator actively manages are checked:

Compared	Ignored
listener Name	Protocol
Hostname	AllowedRoutes
Port	TLS.Options
first CertificateRef (<ns>/<name>)	cert refs beyond index 0
	Group, Kind of ref

Drift in an ignored field will not trigger a corrective patch. For example: if someone manually flips the Protocol of an auto-managed listener from HTTPS to HTTP, the operator will consider it "equal" and leave it alone.

The comparison is done via a name-keyed map (listenerMap, patch.go:118), so listener order does not affect the result.

---

7. Listener Naming and Sanitization

listenerNameFor (patch.go:143):

input:  prefix="auto-"  certName="my-app-cert"  dnsName="app.example.com"

step 1: replace every "." with "-"
        "app.example.com"  ──►  "app-example-com"

step 2: concatenate
        "auto-" + "my-app-cert" + "-" + "app-example-com"
        ──►  "auto-my-app-cert-app-example-com"

step 3: if len > 253 bytes: name = name[:253]   (hard byte slice)

Example outputs for a cert with two DNS names:

dnsName	listener Name
app.example.com	auto-my-app-cert-app-example-com
*.example.com	auto-my-app-cert-*-example-com

The second entry is a risk: * is not valid in a Gateway API SectionName (which must match RFC 1123). The API server will reject the patch and the reconciler will requeue forever (see §10, item 7).

The sanitization is also lossy: a.b.com and a-b-com produce the same result. If a certificate lists both, two listeners with the same name are appended; the API server rejects the patch (listener names must be unique), so no listener on that Gateway is updated.

---

8. Defaults Baked In

All of these are hardcoded in internal/gateway/patch.go. None are configurable via annotations (the listener-port annotation exists as a constant but is not read — see §10, item 1).

Property	Value	Source
Listener port	443	defaultPort patch.go:13
TLS mode	TLSModeTerminate	tlsModeTerminate patch.go:14
Allowed routes	NamespacesFromAll	namespacesFromAll patch.go:15
Protocol	HTTPSProtocolType	inline patch.go:45
CertificateRef Group	"" (core API group)	patch.go:55
CertificateRef Kind	"Secret"	patch.go:56
Secret namespace	cert's own namespace	patch.go:39,58
Listener name prefix	"auto-"	ListenerPrefix controller:28
SectionName max length	253 bytes	patch.go:147
Gateway namespace fallback	cert's own namespace	controller:51-53 and controller:89-91

---

9. End-to-End Scenarios

Scenario 1 — First Certificate created, Gateway has no auto-listeners

1. User creates Certificate "my-cert" (namespace: default)
   Annotations: gateway-name=my-gateway
   Spec: dnsNames=[app.example.com], secretName=my-cert-tls

2. (E1) Cert create event fires → Reconcile(default/my-cert)

3. (A) Get cert → found
   (B) gateway-name="my-gateway" → ok
   (C) gateway-namespace missing → fallback to "default"
   (D) List certs by index "default/my-gateway" → [my-cert]
   (E) BuildListeners → [Listener{Name:"auto-my-cert-app-example-com", Port:443, …}]
   (F) Get gateway "default/my-gateway" → found, Listeners=[]
   (G) MergeListeners([], desired) → [auto-my-cert-app-example-com]
   (H) ListenersEqual([], updated) → false (len differs)
   (I) Patch gateway → Spec.Listeners=[auto-my-cert-app-example-com]
   (J) Done. Log: "patched gateway listeners"

Cluster writes: 1 Gateway PATCH

Scenario 2 — Second Certificate added for the same Gateway

1. User creates Certificate "other-cert" targeting the same Gateway
   dnsNames=[api.example.com], secretName=other-cert-tls

2. (E1) Reconcile(default/other-cert)

3. (D) List certs by index "default/my-gateway" → [my-cert, other-cert]
   (E) BuildListeners → [
         auto-my-cert-app-example-com,
         auto-other-cert-api-example-com
       ]  (sorted alphabetically by name)
   (F) Gateway still has [auto-my-cert-app-example-com]
   (G) MergeListeners → [auto-my-cert-app-example-com,
                          auto-other-cert-api-example-com]
   (H) ListenersEqual(old 1-item, new 2-item) → false
   (I) Patch → Gateway now has both listeners

Cluster writes: 1 Gateway PATCH

Scenario 3 — Certificate dnsNames updated

1. User edits "my-cert": dnsNames=[app.example.com] → [app.example.com, www.example.com]

2. (E2) Cert update event → Reconcile(default/my-cert)

3. (E) BuildListeners produces TWO listeners for my-cert now:
       auto-my-cert-app-example-com
       auto-my-cert-www-example-com
       (plus any from other-cert if still present)
   (H) ListenersEqual: len changed → false
   (I) Patch gateway with new full set

Cluster writes: 1 Gateway PATCH

Scenario 4 — Certificate deleted

1. User deletes "my-cert"

2. (E3) Cert delete event → Reconcile(default/my-cert)

3. (A) Get cert → NotFound
       return Result{}, nil  ──► STOP, NO ACTION

4. Gateway still contains "auto-my-cert-app-example-com" — stale.

5. Cleanup happens only when the next relevant event fires:
   - If "other-cert" is updated or created → (E2) Reconcile(other-cert)
       (D) List by index returns [other-cert] only (my-cert is gone)
       (E) BuildListeners → only other-cert's listeners
       (G) MergeListeners drops the stale auto-my-cert-* listener
       (I) Patch removes it
   - OR if the Gateway is manually edited → (E4) all targeting Certs
       re-enqueued, same convergence path
   - OR on the next cache resync (E6, ~10 h)

   Every path above needs at least one surviving Certificate that targets
   the Gateway. If my-cert was the last one, nothing is ever enqueued for
   it and the stale listener stays until someone removes it by hand.

Cluster writes: 0 at deletion time; 1 Gateway PATCH at next sibling event

---

10. Known Limitations

Recorded as-found. None are fixed by this document.

1. AnnotationListenerPort is dead code certificate_controller.go:25 declares the constant; BuildListenersFromCertificates (patch.go:44) always uses defaultPort = 443. Setting gateway-cert-operator.io/listener-port on a Certificate has no effect.

2. --metrics-bind-address flag was not wired (fixed) The flag used to be parsed and dropped; metrics were served anyway because controller-runtime defaults an empty bind address to :8080. main.go now passes it through metricsserver.Options.

3. No finalizer → cert deletion leaves stale listeners When a Certificate is deleted, Reconcile returns early at branch (A) with no cluster write. The stale auto-* listener persists in the Gateway until a sibling event triggers re-convergence (see Scenario 4 above), or permanently if no other Certificate targets that Gateway. Adding a finalizer to the Certificate would close this gap by allowing cleanup before the object is removed from the API server.

4. ListenersEqual is a partial comparison Described in §6c. Drift in Protocol, AllowedRoutes, TLS.Options, or any CertificateRef beyond the first will not be detected or corrected.

5. Gateway-not-found retries forever Branch (F) of Reconcile treats all Gateway Get errors the same (including NotFound) and returns them as re-queue-able errors. A Certificate with a typo'd gateway-name annotation will retry with rate-limited backoff indefinitely, generating log noise and API load. There is no terminal failure path.

6. No leader election ctrl.NewManager at main.go:44 does not set LeaderElection: true. The deploy/deployment_gateway-cert-operator.yaml runs replicas: 1 which makes this safe today. Scaling to multiple replicas without enabling leader election would result in concurrent Gateway patches.

7. listenerNameFor edge cases

• Wildcard hostnames: *.example.com produces SectionName auto-…-*-example-com. The * character is not valid in an RFC 1123 label; the Gateway API server will reject the Patch, causing an error-retry loop.
• Collisions after sanitization: a.b.com and a-b-com produce identical sanitized names. The same happens across certificates (cert a with b-c.com and cert a-b with c.com) and across namespaces, since the name omits the cert namespace. Both listeners are appended and the API server rejects the patch for the whole Gateway.
• Truncation at 253 bytes: patch.go:147 slices on bytes, not runes (safe for ASCII DNS names). Two DNS names sharing the first 253 characters after sanitization will map to the same listener name.

Dead symbols in patch.go gatewayGroup and gatewayKind are declared at patch.go:16-17 but never referenced. They can be removed without effect.