Files
substrate/hack/microvm-assets
Benjamin Elder cdc8a9b75d ateom-microvm: upgrade cloud-hypervisor to v53 and boot directly to agent (#972)
Follow-up to #847 

cloud-hypervisor v53 ships guest clock updates:

https://github.com/cloud-hypervisor/cloud-hypervisor/releases#:~:text=Guest%20Clock%20Updates%20Across%20Snapshot/Restore%20and%20Migration

In v52, we've been booting to systemd => {chrony, journald, kata-agent}
We don't really have much use for journald, as we get logs from
containers already.
chrony however was load-bearing to re-sync the clock on resume, and
therefore systemd was load-bearing (to run chronyd + kata-agent).

We don't need chrony anymore (tested with long suspend and probing the
reported time in the actor), so we can boot directly to the agent.

This buys us some startup time, and it drops the memory floor by ~40MB.
On golden for counter demo this is ~-26%.

Reducing memory snapshot size speeds up snapshot/restore.
2026-08-17 14:59:44 -04:00
..

Micro-VM runtime assets + counter demo (kind, fetch-not-bake)

The microvm runtime (cmd/ateom-microvm, kata + cloud-hypervisor) fetches its toolchain at runtime — nothing kata-specific is baked into the worker image. ateom drives the kata-agent directly (no kata shim, no containerd). Each actor container's rootfs is an overlay of a read-only lower (the OCI image, served into the guest over virtio-fs by virtiofsd) and a writable upper on a guest tmpfs, so virtiofsd is part of the asset set. The asset set is five files:

  • cloud-hypervisor — the VMM binary (fetched from its release)
  • virtiofsd — the virtio-fs daemon serving the RO lower (built from source; see assemble.sh)
  • vmlinux — the guest kernel (from kata-static)
  • rootfs.img — the guest rootfs image (from kata-static)
  • configuration-clh.toml — the base kata config (from kata-static)

These helpers assemble the asset set for your node arch, stage it into the cluster's rustfs S3 bucket, and the demo manifest's SandboxConfig points at it. When /dev/kvm is available, hack/create-kind-cluster.sh mounts it into the node and labels the node ate.dev/sandboxClass=microvm.

Tip

hack/run-microvm-demo.sh automates the full bring-up below (assets, control plane, demo apply) for kind OR GKE without editing committed files. The steps here are the manual equivalent.

Steps (run on a KVM-capable Linux host matching the node arch)

  1. Assemble assets for your arch:

    ARCH=arm64 hack/microvm-assets/assemble.sh
    

    Copy the printed sha256 sums into the SandboxConfig spec.assets in demos/counter/counter-microvm.yaml.tmpl (the committed values are arm64; other arches differ).

  2. Bring up the cluster + control plane:

    hack/create-kind-cluster.sh        # mounts /dev/kvm, labels node ate.dev/sandboxClass=microvm
    hack/install-ate-kind.sh           # control plane + rustfs (bucket: ate-snapshots)
    
  3. Stage assets into rustfs:

    OUT="$PWD/microvm-assets-arm64" hack/microvm-assets/stage-to-rustfs.sh
    
  4. Apply the demo + drive it:

    BUCKET_NAME=ate-snapshots envsubst < demos/counter/counter-microvm.yaml.tmpl | kubectl apply -f -
    

    Create an actor from counter-microvm, hit the in-RAM counter to increment it, suspend (checkpoint), resume on a different worker pod, and confirm the count continues — proving the guest-memory snapshot round-tripped across pods.

Notes

  • assets is single-arch (unlike runsc's amd64/arm64): stage assets matching the node arch.