mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-11-12 10:35:06 +08:00
Benjamin Elder
327386942f
## What Make the micro-VM runtime choose how it restores guest RAM based on the cloud-hypervisor it is actually driving, so we can support v53. ## Why cloud-hypervisor 53.0.0 changed on-demand restore in two ways: the userfaultfd handler now background-prefaults every registered page (`#8150`), and a snapshot is refused while that runs (`#8556`). Together these make `memory_restore_mode=OnDemand` — what we use today — **unusable** on v53: the prefault storm starves the guest and its readiness probe never passes. The actor never comes up. cloud-hypervisor ships minor releases and rarely backports fixes to a patch branch, so pinning v52 means running an unmaintained hypervisor indefinitely. We need to be able to keep up. Restoring eagerly sidesteps both problems: it reads only the snapshot's populated extents and registers no userfaultfd, so nothing prefaults and nothing gates a later snapshot. But eager is the wrong choice on v52, where on-demand costs a tenth of the memory. So this is a per-VMM decision, not a configuration knob. ## How - `vmm.ping` already answers before every restore and its reply carries the version; we were discarding it. `Ping`/`WaitReady` now return it — no extra process, no extra round trip, nothing to cache. - Pick the mode from that version. The affected range is **bounded** (`prefaultingSince`/`prefaultingUntil`) rather than "53 and up forever", so when a release stops prefaulting unconditionally, on-demand's smaller footprint comes back by moving one constant. - An unreadable version restores eagerly and logs it. The two ways to be wrong are not equal: guessing on-demand on an affected version leaves the actor unable to start, while guessing eager only costs memory. - Eager makes the snapshot self-contained, so two things that follow from it: skip the merge (it would copy the whole resident set onto the restore source for nothing), and drop the staged memory image (nothing pages from it or merges against it afterwards). ## Measurements kind, arm64, 2 GiB guest, counter demo. Each config measured identically, no flags set. | | idle memfd | VMM RSS | pause (median) | per-actor disk | |---|---|---|---|---| | v52, before and after this PR | 16 MiB | 20–21 MiB | ~0.26 s | 158 MiB | | v53, before this PR | — | — | — | **actor never becomes ready** | | v53, with this PR | 158 MiB | 162 MiB | ~0.32 s | 158 MiB | On v52 the runtime detects `52.0.0`, selects on-demand, and reproduces the current shipped behaviour exactly — this is a no-op on what we run today. On v53 it detects `53.0.0`, selects eager, and the actor works. Within v53, the two follow-on changes are worth their own line: | | v53 pause (median) | v53 per-actor disk | |---|---|---| | mode selection only | ~0.69 s | 318 MiB | | + skip the merge | ~0.45 s | 318 MiB | | + drop the staged image | ~0.32 s | **158 MiB** | ## Testing - Unit tests for version parsing and selection, including the real `vmm.ping` payloads from both binaries we ship, and a test documenting how to retire the workaround. - e2e on kind against **both** binaries with no flags set: v52 → on-demand, v53 → eager, counter continuity across five pause/resume cycles each. - `go test -race ./cmd/ateom-microvm/...` green; builds for linux and darwin. ---- NOTE: This PR does not upgrade chv to v53+ by default, I'd like to close the remaining gaps first.