gatewayapi-certificates-operator/docs/flows.md

Runtime Flows

Deep-dive reference for every control path in the gateway-cert-operator. Complements the user-facing overview in docs/README.md.

---

1. Overview

The operator sits between two Kubernetes APIs and keeps them in sync:

   +--------------------+                      +------------------------+
   |   cert-manager     |                      |  gateway-cert-operator |
   |   Certificate      |----(watches)-------->|  (this binary)         |
   |   (source of       |                      |                        |
   |    truth for TLS)  |                      +----------+-------------+
   +--------------------+                                 |
                                                          | patches Spec.Listeners
                                                          v
   +--------------------+                      +------------------------+
   |   Gateway API      |                      |   Gateway              |
   |   controller       |<----(watches)--------|   (target object)      |
   |   (routes traffic) |                      +------------------------+
   +--------------------+

The operator makes no changes to cert-manager objects. Its only write action is a PATCH on Gateway.Spec.Listeners. All other interactions are reads.

---

2. Startup Flow

Executed once at process start (main.go):

process start
    │
    └─ main()                                               [main.go:24]
         │
         ├─ flag.StringVar  --metrics-bind-address   default ":8080"   [main.go:29]
         ├─ flag.StringVar  --health-probe-bind-address default ":8081" [main.go:30]
         ├─ flag.BoolVar    --version                                   [main.go:31]
         ├─ zap.Options{Development: true, AddCaller}.BindFlags         [main.go:32-33]
         │      adds --zap-devel, --zap-encoder, --zap-log-level, …
         ├─ flag.Parse()                                                [main.go:34]
         │
         ├─ --version ──► print version.Resolve(), return               [main.go:36-39]
         │      before any logger or cluster access
         │
         ├─ scheme := runtime.NewScheme()                               [main.go:41-44]
         │    ├─ clientgoscheme.AddToScheme(scheme)   # core/v1: Secrets, Events, …
         │    ├─ cmv1.AddToScheme(scheme)             # cert-manager.io/v1 Certificate
         │    └─ gwapiv1.Install(scheme)              # gateway.networking.k8s.io/v1 Gateway
         │
         ├─ ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts)))          [main.go:46]
         │      dev-mode console logger with file:line call site unless
         │      --zap-devel=false is passed
         ├─ log "starting gateway-cert-operator" commit=… goVersion=…   [main.go:48]
         │
         ├─ ctrl.GetConfigOrDie()       # KUBECONFIG or in-cluster config
         │      on err ──► os.Exit(1)
         │
         ├─ ctrl.NewManager(cfg, Options{Scheme, Metrics, HealthProbeBindAddress})
         │      on err ──► log + os.Exit(1)                            [main.go:50-54]
         │
         ├─ controller.SetupCertificateReconciler(mgr)                  [main.go:60]
         │      registers field index + watches (see §3)
         │      on err ──► log + os.Exit(1)
         │
         ├─ mgr.AddHealthzCheck("healthz", healthz.Ping)  → GET /healthz:8081
         ├─ mgr.AddReadyzCheck("readyz",  healthz.Ping)   → GET /readyz:8081
         │      on err ──► log + os.Exit(1)
         │
         └─ mgr.Start(ctrl.SetupSignalHandler())                        [main.go:75]
                SetupSignalHandler installs SIGINT / SIGTERM cancel
                mgr.Start blocks; on shutdown → graceful stop
                on err ──► log + os.Exit(1)

---

3. Watch Wiring and Event Sources

SetupCertificateReconciler registers two watches before starting the controller (certificate_controller.go:59-63):

   controller-runtime manager
        │
        ├─ For(&cmv1.Certificate{})               primary watch
        │    every Certificate create/update/delete
        │    ──► enqueue Reconcile(cert.Namespace/cert.Name)
        │
        └─ Watches(&gwapiv1.Gateway{},
                   EnqueueRequestsFromMapFunc(r.gatewayToCertificates))
                                                  secondary watch
             every Gateway create/update/delete
             ──► gatewayToCertificates maps Gateway → []Certificates
             ──► enqueue Reconcile for each matching Certificate

Full trigger table:

ID	Event	Enqueued object	Code path
E1	Certificate create	that Certificate	primary watch
E2	Certificate update (any field — no predicate filter)	that Certificate	primary watch
E3	Certificate delete	that Certificate	primary watch
E4	Gateway create / update / delete	every Certificate targeting it (via field index)	secondary watch + gatewayToCertificates
E5	Reconcile returns non-nil error	same Certificate	controller-runtime rate-limited backoff re-queue
E6	Controller-runtime cache resync (~10 h default)	every known Certificate	implicit periodic replay

---

4. Field Index

The field index is the only mechanism that lets the operator find all Certificates pointing at a given Gateway without scanning the entire namespace. It is registered during setup and queried in two places.

   IndexFieldGatewayTarget = ".metadata.annotations.gatewayTarget"
   (identifier used by controller-runtime; no annotation by this name exists)

   Indexer function (certificate_controller.go:44-55):
   ┌─────────────────────────────────────────────────────────┐
   │  for each Certificate obj:                              │
   │    gwName = obj.Annotations["…/gateway-name"]           │
   │    if gwName == ""  →  return nil  (not indexed)        │
   │    gwNs   = obj.Annotations["…/gateway-namespace"]      │
   │    if gwNs == ""    →  gwNs = obj.Namespace  (fallback) │
   │    return [ "<gwNs>/<gwName>" ]                         │
   └─────────────────────────────────────────────────────────┘

   Key shape:  "<targetGatewayNamespace>/<targetGatewayName>"

   Two query sites:

        ┌──────────────────────────────────────────────────────┐
        │ Reconcile  [certificate_controller.go:95-99]         │
        │   List(Certs, MatchingFields{index: "<gwNs>/<gwName>"})│
        │   returns all sibling Certs targeting the same GW    │
        └──────────────────────────────────────────────────────┘

        ┌──────────────────────────────────────────────────────┐
        │ gatewayToCertificates  [certificate_controller.go:138]│
        │   List(Certs, MatchingFields{index: "<gw.Ns>/<gw.Name>"})│
        │   used by the secondary watch to fan-out GW events   │
        │   to the relevant Certs                              │
        └──────────────────────────────────────────────────────┘

The namespace-fallback rule (gwNs = obj.Namespace when the annotation is absent) is duplicated in two places: the indexer (:51-53) and Reconcile (:89-91). Both must apply the same rule or the index key used during write will differ from the key used during lookup.

---

5. Reconcile Decision Tree

Every enqueued item arrives here. Inputs: one Reconcile.Request containing the namespace/name of a Certificate. Side effects: 0 or 1 PATCH on a Gateway.

Reconcile(ctx, req)
    │
    ├─(A) r.Get(ctx, req.NamespacedName, &cert)              [controller:71]
    │       │
    │       ├─ NotFound ──► client.IgnoreNotFound → nil err
    │       │               return Result{}, nil  ►► STOP — NO ACTION
    │       │
    │       │   ┌─ IMPORTANT: the deleted cert's listener is NOT removed here.
    │       │   │  The controller has no record of which Gateway was targeted.
    │       │   │  Cleanup happens on the next E2/E4/E6 event for any sibling
    │       │   │  cert that still references the same Gateway — at that point
    │       │   │  BuildListenersFromCertificates omits the deleted cert and
    │       │   │  MergeListeners replaces all auto-* listeners. Until that
    │       │   │  event fires, the stale listener remains in the Gateway.
    │       │   │  (See §10, item 3 for details.)
    │       │   └───────────────────────────────────────────────────────────
    │       │
    │       └─ other error ──► return Result{}, err  ►► REQUEUE (E5)
    │
    ├─(B) gwName, ok := cert.Annotations["…/gateway-name"]  [controller:82]
    │       │
    │       └─ !ok ──► log V(1) "missing gateway annotation"
    │                   return Result{}, nil  ►► STOP — NO ACTION
    │
    ├─(C) gwNamespace := cert.Annotations["…/gateway-namespace"]  [controller:88]
    │       └─ if "" ──► gwNamespace = cert.Namespace   (DEFAULT FALLBACK)
    │
    ├─(D) r.List(ctx, &certList,                             [controller:95]
    │           MatchingFields{IndexFieldGatewayTarget: "<gwNs>/<gwName>"})
    │       │
    │       └─ err ──► return Result{}, err  ►► REQUEUE
    │
    ├─(E) desiredListeners :=
    │         gw.BuildListenersFromCertificates(certList.Items, "auto-")
    │       (see §6a)                                        [controller:102]
    │
    ├─(F) r.Get(ctx, {gwNs, gwName}, &gateway)               [controller:106]
    │       │
    │       └─ err (including NotFound) ──► return Result{}, err  ►► REQUEUE
    │
    │           NOTE: Gateway-not-found is NOT specially handled — the
    │           reconciler will retry with backoff indefinitely. A typo'd
    │           gateway-name annotation produces a perpetual retry loop.
    │           (See §10, item 5.)
    │
    ├─(G) updated := gw.MergeListeners(                      [controller:111]
    │                   gateway.Spec.Listeners,
    │                   desiredListeners, "auto-")
    │       (see §6b)
    │
    ├─(H) gw.ListenersEqual(gateway.Spec.Listeners, updated) [controller:113]
    │       │
    │       └─ true ──► log V(1) "already up to date"
    │                   return Result{}, nil  ►► SHORT-CIRCUIT — NO PATCH
    │
    ├─(I) patch := client.MergeFrom(gateway.DeepCopy())      [controller:118]
    │       gateway.Spec.Listeners = updated
    │       r.Patch(ctx, &gateway, patch)                    [controller:120]
    │       │
    │       └─ err ──► return Result{}, err  ►► REQUEUE
    │
    └─(J) log Info "patched gateway listeners"               [controller:124]
           return Result{}, nil  ►► DONE

Cluster I/O per reconcile:
  Reads:  1 Certificate Get  (step A)
          1 Certificate List via index  (step D)
          1 Gateway Get  (step F)
  Writes: 0 or 1 Gateway Patch  (step I, only when listeners changed)

---

6. Build / Merge / Equal Pipeline

The three pure functions in internal/gateway/patch.go do all computation. Reconcile only calls them; it holds no business logic of its own.

   certList.Items
        │
        v
   BuildListenersFromCertificates(certs, "auto-")          [patch.go:22]
        │
        └──► desired []gwapiv1.Listener
                │
                │    gateway.Spec.Listeners (current state from API server)
                │         │
                v         v
   MergeListeners(existing, desired, "auto-")              [patch.go:76]
        │
        └──► updated []gwapiv1.Listener
                │
                v
   ListenersEqual(gateway.Spec.Listeners, updated)         [patch.go:93]
        │
        ├─ true  ──► short-circuit, no patch
        └─ false ──► Patch(gateway)

6a. BuildListenersFromCertificates

for cert in certs:
    │
    ├─ if cert.DeletionTimestamp != nil  ──► skip          [patch.go:26]
    │     (cert is being deleted; omitting it removes its listeners
    │      from the desired set and MergeListeners will drop them)
    │
    ├─ if cert.Spec.SecretName == ""    ──► skip          [patch.go:30]
    │
    └─ for dns in cert.Spec.DNSNames:                     [patch.go:34]
            │
            └─ emit Listener{
                 Name:          "auto-<certName>-<sanitized-dns>"
                 Hostname:      dns
                 Port:          443           (hardcoded, see §8)
                 Protocol:      HTTPS
                 AllowedRoutes: { From: All }
                 TLS: {
                   Mode:  Terminate
                   CertificateRefs: [{
                     Group: ""  (core)
                     Kind:  "Secret"
                     Name:  cert.Spec.SecretName
                     Namespace: cert.Namespace  (same as cert)
                   }]
                 }
               }

sort by Name ascending                                    [patch.go:67]
(deterministic order prevents spurious diffs on every reconcile)

6b. MergeListeners

existing listeners
    │
    ├─ Name starts with "auto-"  ──► DROP   (managed, will be replaced)
    └─ Name does NOT start with "auto-"  ──► KEEP (user-managed)

merged = [kept user listeners] + [all desired auto-* listeners]

The result: user-created listeners appear first in their original order; operator-managed listeners follow, sorted by name.

Edge case: a user-created listener whose name happens to start with auto- will be treated as managed and will be deleted on the next reconcile.

6c. ListenersEqual

ListenersEqual is a partial comparison — only the fields the operator actively manages are checked:

Compared	Ignored
listener Name	Protocol
Hostname	AllowedRoutes
Port	TLS.Options
first CertificateRef (<ns>/<name>)	cert refs beyond index 0
	Group, Kind of ref

Drift in an ignored field will not trigger a corrective patch. For example: if someone manually flips the Protocol of an auto-managed listener from HTTPS to HTTP, the operator will consider it "equal" and leave it alone.

The comparison is done via a name-keyed map (listenerMap, patch.go:118), so listener order does not affect the result.

---

7. Listener Naming and Sanitization

listenerNameFor and sanitizeDNSName (patch.go):

input:  prefix="auto-"  certName="my-app-cert"  dnsName="app.example.com"

step 1: sanitize the DNS name
        lowercase; "*" ──► "wildcard"; every other character outside
        [a-z0-9] ──► "-"; trim leading and trailing "-"
        "app.example.com"  ──►  "app-example-com"

step 2: concatenate
        "auto-" + "my-app-cert" + "-" + "app-example-com"
        ──►  "auto-my-app-cert-app-example-com"

step 3: if len > 253: cut to 244, trim trailing "-" and ".", append
        "-" + first 8 hex chars of sha256(full name)

Example outputs for a cert with two DNS names:

dnsName	listener Name
app.example.com	auto-my-app-cert-app-example-com
*.example.com	auto-my-app-cert-wildcard-example-com

The listener Hostname keeps the original DNS name, wildcard included.

The sanitization is also lossy: a.b.com and a-b-com produce the same result. If a certificate lists both, two listeners with the same name are appended; the API server rejects the patch (listener names must be unique), so no listener on that Gateway is updated.

---

8. Defaults Baked In

All of these are hardcoded in internal/gateway/patch.go. None are configurable via annotations (the listener-port annotation exists as a constant but is not read — see §10, item 1).

Property	Value	Source
Listener port	443	defaultPort patch.go:13
TLS mode	TLSModeTerminate	tlsModeTerminate patch.go:14
Allowed routes	NamespacesFromAll	namespacesFromAll patch.go:15
Protocol	HTTPSProtocolType	inline patch.go:45
CertificateRef Group	"" (core API group)	patch.go:55
CertificateRef Kind	"Secret"	patch.go:56
Secret namespace	cert's own namespace	patch.go:39,58
Listener name prefix	"auto-"	ListenerPrefix controller:28
SectionName max length	253 bytes	patch.go:147
Gateway namespace fallback	cert's own namespace	controller:51-53 and controller:89-91

---

9. End-to-End Scenarios

Scenario 1 — First Certificate created, Gateway has no auto-listeners

1. User creates Certificate "my-cert" (namespace: default)
   Annotations: gateway-name=my-gateway
   Spec: dnsNames=[app.example.com], secretName=my-cert-tls

2. (E1) Cert create event fires → Reconcile(default/my-cert)

3. (A) Get cert → found
   (B) gateway-name="my-gateway" → ok
   (C) gateway-namespace missing → fallback to "default"
   (D) List certs by index "default/my-gateway" → [my-cert]
   (E) BuildListeners → [Listener{Name:"auto-my-cert-app-example-com", Port:443, …}]
   (F) Get gateway "default/my-gateway" → found, Listeners=[]
   (G) MergeListeners([], desired) → [auto-my-cert-app-example-com]
   (H) ListenersEqual([], updated) → false (len differs)
   (I) Patch gateway → Spec.Listeners=[auto-my-cert-app-example-com]
   (J) Done. Log: "patched gateway listeners"

Cluster writes: 1 Gateway PATCH

Scenario 2 — Second Certificate added for the same Gateway

1. User creates Certificate "other-cert" targeting the same Gateway
   dnsNames=[api.example.com], secretName=other-cert-tls

2. (E1) Reconcile(default/other-cert)

3. (D) List certs by index "default/my-gateway" → [my-cert, other-cert]
   (E) BuildListeners → [
         auto-my-cert-app-example-com,
         auto-other-cert-api-example-com
       ]  (sorted alphabetically by name)
   (F) Gateway still has [auto-my-cert-app-example-com]
   (G) MergeListeners → [auto-my-cert-app-example-com,
                          auto-other-cert-api-example-com]
   (H) ListenersEqual(old 1-item, new 2-item) → false
   (I) Patch → Gateway now has both listeners

Cluster writes: 1 Gateway PATCH

Scenario 3 — Certificate dnsNames updated

1. User edits "my-cert": dnsNames=[app.example.com] → [app.example.com, www.example.com]

2. (E2) Cert update event → Reconcile(default/my-cert)

3. (E) BuildListeners produces TWO listeners for my-cert now:
       auto-my-cert-app-example-com
       auto-my-cert-www-example-com
       (plus any from other-cert if still present)
   (H) ListenersEqual: len changed → false
   (I) Patch gateway with new full set

Cluster writes: 1 Gateway PATCH

Scenario 4 — Certificate deleted

1. User deletes "my-cert"

2. (E3) Cert delete event → Reconcile(default/my-cert)

3. (A) Get cert → NotFound
       return Result{}, nil  ──► STOP, NO ACTION

4. Gateway still contains "auto-my-cert-app-example-com" — stale.

5. Cleanup happens only when the next relevant event fires:
   - If "other-cert" is updated or created → (E2) Reconcile(other-cert)
       (D) List by index returns [other-cert] only (my-cert is gone)
       (E) BuildListeners → only other-cert's listeners
       (G) MergeListeners drops the stale auto-my-cert-* listener
       (I) Patch removes it
   - OR if the Gateway is manually edited → (E4) all targeting Certs
       re-enqueued, same convergence path
   - OR on the next cache resync (E6, ~10 h)

   Every path above needs at least one surviving Certificate that targets
   the Gateway. If my-cert was the last one, nothing is ever enqueued for
   it and the stale listener stays until someone removes it by hand.

Cluster writes: 0 at deletion time; 1 Gateway PATCH at next sibling event

---

10. Known Limitations

Recorded as-found. None are fixed by this document.

1. AnnotationListenerPort is dead code certificate_controller.go:25 declares the constant; BuildListenersFromCertificates (patch.go:44) always uses defaultPort = 443. Setting gateway-cert-operator.io/listener-port on a Certificate has no effect.

2. --metrics-bind-address flag was not wired (fixed) The flag used to be parsed and dropped; metrics were served anyway because controller-runtime defaults an empty bind address to :8080. main.go now passes it through metricsserver.Options.

3. No finalizer → cert deletion leaves stale listeners When a Certificate is deleted, Reconcile returns early at branch (A) with no cluster write. The stale auto-* listener persists in the Gateway until a sibling event triggers re-convergence (see Scenario 4 above), or permanently if no other Certificate targets that Gateway. Adding a finalizer to the Certificate would close this gap by allowing cleanup before the object is removed from the API server.

4. ListenersEqual is a partial comparison Described in §6c. Drift in Protocol, AllowedRoutes, TLS.Options, or any CertificateRef beyond the first will not be detected or corrected.

5. Gateway-not-found retries forever Branch (F) of Reconcile treats all Gateway Get errors the same (including NotFound) and returns them as re-queue-able errors. A Certificate with a typo'd gateway-name annotation will retry with rate-limited backoff indefinitely, generating log noise and API load. There is no terminal failure path.

6. No leader election ctrl.NewManager at main.go:50 does not set LeaderElection: true. The deploy/deployment_gateway-cert-operator.yaml runs replicas: 1 which makes this safe today. Scaling to multiple replicas without enabling leader election would result in concurrent Gateway patches.

7. listenerNameFor edge cases

• Wildcard hostnames (fixed): * used to end up in the listener name and the API server rejected the patch. It is now spelled out as wildcard.
• Collisions after sanitization: a.b.com and a-b-com produce identical sanitized names, as do *.x.com and wildcard.x.com. The same happens across certificates (cert a with b-c.com and cert a-b with c.com) and across namespaces, since the name omits the cert namespace. Both listeners are appended and the API server rejects the patch for the whole Gateway.
• Truncation at 253 bytes (fixed): long names are cut and get a hash of the full name appended, so they stay valid and distinct.

Dead symbols in patch.go gatewayGroup and gatewayKind are declared at patch.go:16-17 but never referenced. They can be removed without effect.