Seed docs/architecture.md with the event-to-function reconcile flow diagrams
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
132
docs/architecture.md
Normal file
132
docs/architecture.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# Architecture
|
||||
|
||||
> **Status:** the operator is built through Step 4 (reconciler) of
|
||||
> [docs/plans/2026-08-07-1747-proxy-operator.md](plans/2026-08-07-1747-proxy-operator.md).
|
||||
> This document currently covers the event/reconcile flow; the components
|
||||
> table and the Decisions section arrive with Step 10, and the diagrams
|
||||
> below grow as the health engine, lease store, discovery API, and orphan
|
||||
> GC land.
|
||||
|
||||
## Event flow: cluster events → reconciler functions
|
||||
|
||||
Which functions run in response to which Kubernetes cluster events, as
|
||||
wired in `internal/controller/proxy_controller.go`.
|
||||
|
||||
### 1. How cluster events reach the reconciler
|
||||
|
||||
```text
|
||||
KUBERNETES CLUSTER EVENTS (wiring: SetupWithManager,
|
||||
───────────────────────── proxy_controller.go)
|
||||
|
||||
Proxy CR created / spec edited / Secret created / updated / deleted
|
||||
status patched / delete requested │
|
||||
│ │
|
||||
watch: For(&crawlv1alpha1.Proxy{}) watch: Watches(&corev1.Secret{}, ...)
|
||||
│ │
|
||||
│ r.proxiesForSecret(ctx, secret)
|
||||
│ │ r.List(Proxies in secret's namespace)
|
||||
│ │ keeps those whose
|
||||
│ │ spec.cloudInit.secretRef.name matches
|
||||
│ ▼
|
||||
│ [reconcile.Request per matching Proxy]
|
||||
▼ │
|
||||
┌─────────────────────────────────────────────────┴──┐
|
||||
│ controller-runtime workqueue │◄── RequeueAfter timers
|
||||
│ (dedup by namespace/name, rate-limited, │ (from prior reconciles)
|
||||
│ MaxConcurrentReconciles: 3) │◄── error backoff retries
|
||||
└──────────────────────────┬──────────────────────────┘
|
||||
▼
|
||||
ProxyReconciler.Reconcile(ctx, req)
|
||||
```
|
||||
|
||||
The Secret watch makes *rotating a cloud-init Secret* a first-class event:
|
||||
it re-enqueues every Proxy referencing that Secret, which is how secret
|
||||
rotation triggers VM replacement even though the Proxy spec is untouched.
|
||||
|
||||
### 2. Inside `Reconcile` — dispatch and the single status write
|
||||
|
||||
```text
|
||||
Reconcile(ctx, req)
|
||||
│ r.Get(ctx, req.NamespacedName, &p) ── fetch the Proxy (NotFound → done)
|
||||
│ base := p.DeepCopy() ── snapshot for the diff
|
||||
│ defer patchStatusIfChanged(ctx, base, &p) ──────────────────────────┐
|
||||
│ │
|
||||
├─ p.DeletionTimestamp set ──► reconcileDelete(ctx, &p) │
|
||||
├─ p.Spec.Mode == External ──► reconcileExternal(ctx, &p) │
|
||||
└─ otherwise (Managed) ──────► reconcileManaged(ctx, &p) │
|
||||
▼
|
||||
patchStatusIfChanged (status.go)
|
||||
│ p.Status.ObservedGeneration = p.Generation
|
||||
│ p.Status.Phase = computePhase(&p)
|
||||
│ equality.Semantic.DeepEqual(base, p)?
|
||||
└─ changed → r.Status().Patch(...) ◄── the ONLY
|
||||
unchanged → no API call status write
|
||||
```
|
||||
|
||||
### 3. `reconcileManaged` — the state machine
|
||||
|
||||
```text
|
||||
reconcileManaged(ctx, p)
|
||||
│
|
||||
├─ controllerutil.AddFinalizer? ──► r.Update ──► return {} (watch event re-triggers)
|
||||
├─ permanent-failure latch (FindStatusCondition == PermanentError
|
||||
│ at this generation) ──► return {} (silent until spec edit)
|
||||
├─ r.Providers[p.Spec.Provider] missing ──► setProvisioned(PermanentError) → Failed
|
||||
│
|
||||
├─ resolveCloudInit(ctx, p) ──► r.Get(Secret) if secretRef (error → CloudInitError + backoff)
|
||||
├─ hash := specHash(p, cloudInit) (spechash.go)
|
||||
│
|
||||
├─ status.providerID == "" ─────────► prov.Create(CreateRequest{Name: NameFromUID(p.UID), ...})
|
||||
│ │ setSpecHash → r.Update (annotation)
|
||||
│ └ stage providerID + Provisioned=False/Provisioning
|
||||
│ ──► RequeueAfter: ProvisioningPoll
|
||||
│
|
||||
├─ annotation != hash, annotation == "" ──► adopt: setSpecHash → r.Update
|
||||
│ ──► RequeueAfter: RequeueNow
|
||||
├─ annotation != hash, annotation != "" ──► replaceInstance:
|
||||
│ prov.Get ─ NotFound → setSpecHash, clear ID/IP
|
||||
│ │ ──► RequeueNow (next pass creates)
|
||||
│ └ exists → prov.Delete, Provisioned=False/Replacing
|
||||
│ ──► RequeueAfter: DeletionPoll
|
||||
│
|
||||
└─ annotation == hash ──► prov.Get(providerID)
|
||||
├─ ErrNotFound ──► clear ID/IP ──► RequeueNow (next pass creates)
|
||||
├─ Provisioning ──► Provisioned=False ──► ProvisioningPoll
|
||||
├─ Running ──► status.ip = inst.IP,
|
||||
│ Provisioned=True/Created ──► DriftPoll
|
||||
└─ Stopped/Termin. ──► prov.Delete (cattle) ──► DeletionPoll
|
||||
|
||||
any provider error ──► providerFailure(p, err) ── provider.Class(err):
|
||||
├─ ErrQuotaExceeded ──► condition QuotaExceeded ──► RequeueAfter: QuotaRetry (nil error)
|
||||
├─ ErrPermanent ──► condition PermanentError ──► phase Failed, no retry
|
||||
└─ ErrTransient ──► return err ──► workqueue exponential backoff
|
||||
```
|
||||
|
||||
### 4. `reconcileDelete` and `reconcileExternal`
|
||||
|
||||
```text
|
||||
reconcileDelete(ctx, p) reconcileExternal(ctx, p)
|
||||
├─ no finalizer ──► return {} │ status.ip = spec.endpoint.host
|
||||
├─ providerID == "" ──► RemoveFinalizer │ setProvisioned(True/ExternalEndpoint)
|
||||
│ → r.Update → object actually deleted └─ return {} (no finalizer, no
|
||||
├─ prov.Get → ErrNotFound ──► RemoveFinalizer provider calls ever;
|
||||
│ → r.Update → object actually deleted the health engine —
|
||||
└─ exists ──► prov.Delete Step 5 — drives the rest)
|
||||
→ Provisioned=False/Deleting
|
||||
──► RequeueAfter: DeletionPoll (poll until gone)
|
||||
```
|
||||
|
||||
### 5. What provider calls do back in the cluster (kubernetes pod provider)
|
||||
|
||||
```text
|
||||
prov.Create ──► buildPod (pure) ──► client.Create(corev1.Pod) ─┐ these cause Pod events,
|
||||
prov.Get ──► client.Get(Pod) → phase/IP → InstanceState │ but the operator does NOT
|
||||
prov.Delete ──► client.Delete(Pod, tolerate NotFound) │ watch Pods — it observes
|
||||
prov.ListByTag ─► client.List(Pods by labels, all namespaces) ─┘ them by polling prov.Get
|
||||
on each RequeueAfter tick
|
||||
```
|
||||
|
||||
The reconciler never watches provider-side resources (Pods now, GCP VMs
|
||||
later). All instance-state observation is poll-based through the
|
||||
`Provider` interface, so the same flow works identically for a cloud API
|
||||
that has no watch mechanism at all.
|
||||
Reference in New Issue
Block a user