kvm2: apply the milestone review
Shorter start wait with immediate verdicts for dead or paused domains, ListByTag without state or address RPCs, Timeout for an expired check round, no adoption of unmarked domains, reference-counted keyed locks, dead options removed, HEAD fallback on any non-2xx. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Claude Fable 5.1
+33
-25
@@ -30,12 +30,12 @@ and the symbol that shows it.
|
||||
| Axis | kubernetes | gcp | kvm | kvm2 |
|
||||
|---|---|---|---|---|
|
||||
| `readiness` | immediate: a Pod is scheduled within seconds (`stateFromPod`) | provisioning-window: minutes of boot (`mapState`) | provisioning-window: boot plus, on first use, an image download (`imageCache.ensure`) | provisioning-window: boot plus, on first use, an image download (`ensureImage`) |
|
||||
| `create-return` | submit: returns after `client.Create` | submit: `Insert` is never awaited, `Operation.Wait` is not called | **blocking**: waits through `DomainCreate` (`ensureRunning`), and the first image fetch runs inside (`imageCache.fetch`) | **blocking**: waits through `DomainCreate` and a 2-minute poll for running (`ensureRunning`, `waitRunning`), and the first image fetch runs inside (`fetchImage`) |
|
||||
| `create-return` | submit: returns after `client.Create` | submit: `Insert` is never awaited, `Operation.Wait` is not called | **blocking**: waits through `DomainCreate` (`ensureRunning`), and the first image fetch runs inside (`imageCache.fetch`) | **blocking**: waits through `DomainCreate` and a 15-second poll for running (`ensureRunning`, `waitRunning`), and the first image fetch runs inside (`fetchImage`) |
|
||||
| `delete-return` | returns-early: `client.Delete`, Pod terminates on its own | returns-early: `Delete` submits an operation each time | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) | waits: destroy, undefine and volume removal complete before return (`removeDomain`, `removeVolumes`) |
|
||||
| `ip-source` | assigned-at-create: `PodIP` (`instanceFromPod`) | assigned-at-create: the external NAT IP of the first NIC (`toInstance`) | discovered: ARP table first, then the guest agent, first routable IPv4 (`address`, `firstIPv4`) | discovered: the guest agent first, then the ARP table, first routable IPv4 (`addressSources`, `address`, `firstIPv4`) |
|
||||
| `gc-mark` | native-labels: Pod labels (`buildPod`) | native-labels: instance labels (`buildInsertRequest`) | written-metadata: a namespaced `<metadata>` element written in `DomainDefineXML` (`metadata.go`) | written-metadata: an element in kvm2's own namespace written in `DomainDefineXML` (`metadataURI`, `newInstanceMetadata`) |
|
||||
| `created-at` | backend-timestamp: `CreationTimestamp` | backend-timestamp: `CreationTimestamp`, parse error gives zero | written-at-create: `createdAt` in the metadata element (`newInstanceMetadata`) | written-at-create: `createdAt` in the metadata element, parse error gives zero (`newInstanceMetadata`, `createdAt`) |
|
||||
| `list-partial` | n/a: one call, all namespaces | omit-unreachable: `ReturnPartialSuccess` on `AggregatedList` | fail-whole per host, one unreadable domain skipped (`ListByTag`, `managedInstance`) | fail-whole per host, one unreadable domain skipped unless ctx ended (`ListByTag`) |
|
||||
| `list-partial` | n/a: one call, all namespaces | omit-unreachable: `ReturnPartialSuccess` on `AggregatedList` | fail-whole per host, one unreadable domain skipped (`ListByTag`, `managedInstance`) | fail-whole per host, one unreadable domain skipped unless ctx ended; `ID`, `UID` and `CreatedAt` from the name and the mark only, no state or address RPC (`ListByTag`) |
|
||||
| `quota` | absent: a quota 403 is Permanent (`classify`) | detected: 429, and 403 with reason `quotaExceeded` or `rateLimitExceeded` (`classify`, `hasReason`) | absent (`classify`) | detected: a pool-space precheck before any volume is created (`checkPoolSpace`, `neededBytes`) |
|
||||
| `not-found` | status-404: `apierrors.IsNotFound` | status-404: `isNotFound` | error-code: `ErrNoDomain`, `ErrNoStorageVol` (`hasCode`) | error-code: `ErrNoDomain`, `ErrNoStorageVol`, Transient inside Create (`classifyLibvirt`) |
|
||||
| `primary-key` | name-is-key: `namespace/name` | name-is-key: `zones/<zone>/instances/<name>` | name-is-key: `hosts/<host>/domains/<name>` | name-is-key: `hosts/<host>/domains/<name>`, with a UID guard on adoption (`create`) |
|
||||
@@ -56,7 +56,7 @@ against.
|
||||
|
||||
| Feature | kubernetes | gcp | kvm | kvm2 |
|
||||
|---|---|---|---|---|
|
||||
| Create verifies the adopted instance's UID | no | no | no | **yes**: a different non-empty UID in the metadata is Permanent, an empty one is adopted (`create`) |
|
||||
| Create verifies the adopted instance's UID | no | no | no | **yes**: a domain without the kvm2 mark or with a different non-empty UID is Permanent; a marked one with a matching or empty UID is adopted (`create`) |
|
||||
| Placement used | none | Zone, MachineType, Image | Zone (host), MachineType, Image | Zone (host), MachineType, Image |
|
||||
| CloudInit delivered | **no**: ignored | yes: `user-data` metadata | yes: NoCloud seed ISO built in-process (`buildSeedISO`) | yes: NoCloud seed ISO built in-process (`buildSeedISO`) |
|
||||
| Port used | yes: Squid config and containerPort (`squidConf`) | no: firewall is manual | no | no |
|
||||
@@ -372,9 +372,10 @@ same libvirt.
|
||||
logs the resolved parameters with `cloudInitBytes` at V(2). Then, under
|
||||
the per-name lock (`lockName`) and on the pooled connection
|
||||
(`hostTable.get`), `create`: `DomainLookupByName`; found means the
|
||||
adoption guard (a non-empty metadata UID that differs from `req.UID` is
|
||||
Permanent) and `ensureRunning`. Otherwise `StoragePoolLookupByName`,
|
||||
`imageSizeIfUncached` (marker lookup, then `HEAD` or a closed `GET`) and
|
||||
adoption guard (no kvm2 mark, or a non-empty metadata UID that differs
|
||||
from `req.UID`, is Permanent) and `ensureRunning`. Otherwise
|
||||
`StoragePoolLookupByName`, `imageSizeIfUncached` (marker lookup, then
|
||||
`HEAD`, or a closed `GET` when `HEAD` is not 2xx) and
|
||||
`checkPoolSpace` with `neededBytes` (image if uncached, plus
|
||||
`diskSizeGb`, plus 16 MiB), then `ensureImage` (`cachedImage` on the
|
||||
pooled connection, `fetchImage` streaming into `uploadImage` on a
|
||||
@@ -382,10 +383,11 @@ same libvirt.
|
||||
`ensureOverlay` (`ErrStorageVolExist` means reuse), `writeSeed` (delete,
|
||||
build, upload; always rebuilt), two `StorageVolGetPath`, `buildDomainXML`
|
||||
with file-type disks and the metadata element, `DomainDefineXML`,
|
||||
`ensureRunning` (`DomainGetState`; `DomainCreate` with
|
||||
`ErrOperationInvalid` as already running; `waitRunning` polls every
|
||||
500 ms for up to 2 minutes, Transient on expiry). `Create` returns only
|
||||
when the domain is running.
|
||||
`ensureRunning` (`DomainGetState`, where paused or pmsuspended is
|
||||
Permanent; `DomainCreate` with `ErrOperationInvalid` as already running;
|
||||
`waitRunning` polls every 500 ms for up to 15 seconds, Transient on
|
||||
expiry and at once for any inactive state other than `nostate`).
|
||||
`Create` returns only when the domain is running.
|
||||
- **Get and state.** `mapState(state, ip)`:
|
||||
|
||||
| libvirt state | IP | Provider state |
|
||||
@@ -398,8 +400,8 @@ same libvirt.
|
||||
`get`: `DomainLookupByName` (only `ErrNoDomain` reaches NotFound),
|
||||
`readMetadata` (`DomainGetMetadata` with `metadataURI`;
|
||||
`ErrNoDomainMetadata` means unmanaged with an empty UID; an element that
|
||||
does not parse is Permanent), `inspect` (`DomainGetState`, then for an
|
||||
active domain `address`). `addressSources` asks the guest agent first,
|
||||
does not parse is Permanent), then `DomainGetState` and, for an active
|
||||
domain, `address`. `addressSources` asks the guest agent first,
|
||||
then ARP; a source that fails is skipped at V(2), only ctx ending is an
|
||||
error, so a cancelled Get is Transient and not a false Provisioning.
|
||||
`firstIPv4` drops loopback, link-local and unspecified addresses and an
|
||||
@@ -413,9 +415,12 @@ same libvirt.
|
||||
when nil is returned.
|
||||
- **ListByTag.** `ConnectListAllDomains` (active and inactive) on every
|
||||
host in config order; `readMetadata` and `managed` keep kvm2's own
|
||||
domains; `inspect` fills state and address. A domain whose metadata or
|
||||
state cannot be read is skipped at V(1), unless ctx ended, which fails
|
||||
the call. A host that cannot be dialled or listed fails the whole call.
|
||||
domains, and the `Instance` is `ID`, `UID` and `CreatedAt` from the name
|
||||
and the element, with `State` and `IP` empty: GC reads nothing else
|
||||
(contract §2.4), so no state or address RPC is made. A domain whose
|
||||
metadata cannot be read is skipped at V(1), unless ctx ended, which
|
||||
fails the call. A host that cannot be dialled or listed fails the whole
|
||||
call.
|
||||
- **Errors.** `classify(op, err)`: an error already carrying a class keeps
|
||||
it; libvirt codes through `classifyLibvirt` (`ErrNoDomain` and
|
||||
`ErrNoStorageVol` NotFound, Transient when `op == "create"`;
|
||||
@@ -436,14 +441,16 @@ same libvirt.
|
||||
`ConnectGetHostname`, `StoragePoolLookupByName`, `StoragePoolGetInfo`;
|
||||
a pool that is not running is `poolNotActiveError`. Details are filled
|
||||
as facts arrive (`user` and `pool` always; `libvirtVersion`, `hostname`,
|
||||
`poolState`, `poolCapacityGiB`, `poolAvailableGiB`). `checkReason`: a
|
||||
`dialError` is judged by its stage (`credentials` gives
|
||||
`CredentialsUnavailable`; `ssh` with a `knownhosts.KeyError` or the auth
|
||||
text gives `Unauthorized`; any other stage, including a dial that ran
|
||||
out of time, gives `Unreachable`); then `context.DeadlineExceeded` or
|
||||
`Canceled` `Timeout`, `ErrAccessDenied` or `ErrAuthFailed` `Forbidden`,
|
||||
`ErrNoStoragePool` `PoolNotFound`, any other libvirt code `APIError`,
|
||||
anything else `Unreachable`; `PoolNotActive` from the pool state.
|
||||
`poolState`, `poolCapacityGiB`, `poolAvailableGiB`). `checkHost` answers
|
||||
`Timeout` whenever the round's ctx ended, whatever stage was running and
|
||||
whatever shape the error has; otherwise `checkReason`: a `dialError` is
|
||||
judged by its stage (`credentials` gives `CredentialsUnavailable`; `ssh`
|
||||
with a `knownhosts.KeyError` or the auth text gives `Unauthorized`; any
|
||||
other stage, including a dial that ran out of its own budget, gives
|
||||
`Unreachable`); then `context.DeadlineExceeded` or `Canceled` `Timeout`,
|
||||
`ErrAccessDenied` or `ErrAuthFailed` `Forbidden`, `ErrNoStoragePool`
|
||||
`PoolNotFound`, any other libvirt code `APIError`, anything else
|
||||
`Unreachable`; `PoolNotActive` from the pool state.
|
||||
- **Connections.** `hostTable` keeps one pooled connection per host,
|
||||
dialled lazily under a per-host one-slot semaphore so waiting ends with
|
||||
the caller's ctx; a dead backend (`liveness`) is closed and redialled on
|
||||
@@ -510,8 +517,9 @@ same libvirt.
|
||||
- Cancelling one operation cuts off the others on that host once: a
|
||||
cancelled `call` kills the pooled connection.
|
||||
- `Content-Length` is required for image downloads.
|
||||
- A paused or pmsuspended domain cannot be adopted by a repeat `Create`:
|
||||
`DomainCreate` refuses, `waitRunning` expires, Transient on every retry.
|
||||
- A paused or pmsuspended domain is not started by a repeat `Create`:
|
||||
`ensureRunning` returns Permanent, so the Proxy latches until its spec
|
||||
changes; the admin resumes the VM on the host.
|
||||
- Credential parse errors carry the parser's own text, which can quote a
|
||||
PEM header type or a malformed known_hosts token (never key bytes).
|
||||
- No reused-overlay check: an existing overlay is reused without checking
|
||||
|
||||
Reference in New Issue
Block a user