OpenShell/crates/openshell-driver-vm

openshell-driver-vm

Status: Experimental. The VM compute driver is under active development and the interface still has VM-specific plumbing that will be generalized.

Standalone libkrun-backed ComputeDriver for OpenShell. The gateway spawns this binary as a subprocess, talks to it over a Unix domain socket with the openshell.compute.v1.ComputeDriver gRPC surface, and lets it manage per-sandbox microVMs. The runtime (libkrun + libkrunfw + gvproxy) and the sandbox supervisor are embedded directly in the binary; each sandbox guest rootfs is derived from a configured container image at create time.

How it fits together

flowchart LR
    subgraph host["Host process"]
        gateway["openshell-server<br/>(compute::vm::spawn)"]
        driver["openshell-driver-vm<br/>├── libkrun (VM)<br/>├── gvproxy (net)<br/>└── openshell-sandbox.zst"]
        gateway <-->|"gRPC over UDS<br/>compute-driver.sock"| driver
    end

    subgraph guest["Per-sandbox microVM"]
        init["/srv/openshell-vm-<br/>sandbox-init.sh"]
        supervisor["/opt/openshell/bin/<br/>openshell-sandbox<br/>(PID 1)"]
        init --> supervisor
    end

    driver -->|"CreateSandbox<br/>boots via libkrun"| guest
    supervisor -.->|"gRPC callback<br/>--grpc-endpoint"| gateway

    client["openshell-cli"] -->|"SSH proxy<br/>127.0.0.1:&lt;port&gt;"| supervisor
    client -->|"CreateSandbox / Watch"| gateway

Sandbox guests execute /opt/openshell/bin/openshell-sandbox as PID 1 inside the VM. gvproxy exposes a single inbound SSH port (host:<allocated> → guest:2222) and provides virtio-net egress.

Quick start (recommended)

mise run gateway:vm

First run takes a few minutes while mise run vm:setup stages libkrun/libkrunfw/gvproxy and mise run vm:supervisor builds the bundled guest supervisor. Subsequent runs are cached.

By default mise run gateway:vm:

• Listens on plaintext HTTP at 127.0.0.1:18081.
• Registers the CLI gateway vm-dev by writing ~/.config/openshell/gateways/vm-dev/metadata.json. It does not modify the workspace .env.
• Persists the gateway SQLite DB under .cache/gateway-vm/gateway.db.
• Places the VM driver state (per-sandbox rootfs plus run/compute-driver.sock) under /tmp/openshell-vm-driver-$USER-vm-dev/ so the AF_UNIX socket path stays under macOS SUN_LEN.
• Passes --driver-dir $PWD/target/debug so the freshly built openshell-driver-vm is used instead of an older installed copy from ~/.local/libexec/openshell, /usr/libexec/openshell, or /usr/local/libexec.

For GPU passthrough (VFIO), pass -- --gpu and run with root privileges:

sudo -E env "PATH=$PATH" mise run gateway:vm -- --gpu

GPU passthrough uses VFIO and requires host support for IOMMU, root privileges for bind/unbind operations, and a compatible sandbox image. The public GPU overview lives in the repository README.md.

Point the CLI at the gateway with one of:

openshell --gateway vm-dev status
openshell gateway select vm-dev    # then plain `openshell <command>`

Override defaults via environment:

# custom port (fails fast if in use)
OPENSHELL_SERVER_PORT=18091 mise run gateway:vm

# custom CLI gateway name + namespace
OPENSHELL_VM_GATEWAY_NAME=vm-feature-a \
OPENSHELL_SANDBOX_NAMESPACE=vm-feature-a \
mise run gateway:vm

# custom sandbox image
OPENSHELL_SANDBOX_IMAGE=ghcr.io/example/sandbox:latest mise run gateway:vm

Teardown:

rm -rf /tmp/openshell-vm-driver-$USER-vm-dev .cache/gateway-vm
rm -rf "${XDG_CONFIG_HOME:-$HOME/.config}/openshell/gateways/vm-dev"

Manual equivalent

If you want to drive the launch yourself instead of using mise run gateway:vm (i.e. tasks/scripts/gateway-vm.sh):

# 1. Stage runtime artifacts + supervisor bundle into target/vm-runtime-compressed/
mise run vm:setup
mise run vm:supervisor          # if openshell-sandbox.zst is not already present

# 2. Build both binaries with the staged artifacts embedded
OPENSHELL_VM_RUNTIME_COMPRESSED_DIR=$PWD/target/vm-runtime-compressed \
  cargo build -p openshell-server -p openshell-driver-vm

# 3. macOS only: codesign the driver for Hypervisor.framework
codesign \
  --entitlements crates/openshell-driver-vm/entitlements.plist \
  --force -s - target/debug/openshell-driver-vm

# 4. Start the gateway with the VM driver
mkdir -p /tmp/openshell-vm-driver-$USER-vm-dev .cache/gateway-vm
target/debug/openshell-gateway \
  --drivers vm \
  --disable-tls \
  --db-url "sqlite:.cache/gateway-vm/gateway.db?mode=rwc" \
  --driver-dir $PWD/target/debug \
  --sandbox-namespace vm-dev \
  --sandbox-image <compatible-image> \
  --grpc-endpoint http://host.containers.internal:18081 \
  --port 18081 \
  --vm-driver-state-dir /tmp/openshell-vm-driver-$USER-vm-dev

The gateway resolves openshell-driver-vm in this order: --driver-dir, conventional install locations (~/.local/libexec/openshell, /usr/libexec/openshell, /usr/local/libexec/openshell, /usr/local/libexec), then a sibling of the gateway binary.

Flags

Flag	Env var	Default	Purpose
--drivers vm	OPENSHELL_DRIVERS	kubernetes	Select the VM compute driver.
--grpc-endpoint URL	OPENSHELL_GRPC_ENDPOINT	—	Required. URL the sandbox guest dials to reach the gateway. Use http://host.containers.internal:<port> (or host.docker.internal / host.openshell.internal) so traffic flows through gvproxy's host-loopback NAT (HostIP 192.168.127.254 → host 127.0.0.1). Loopback URLs like http://127.0.0.1:<port> are rewritten automatically by the driver. The bare gateway IP (192.168.127.1) only carries gvproxy's own services and will not reach host-bound ports.
--vm-driver-state-dir DIR	OPENSHELL_VM_DRIVER_STATE_DIR	target/openshell-vm-driver	Per-sandbox rootfs, console logs, image cache, and private run/compute-driver.sock UDS.
--driver-dir DIR	OPENSHELL_DRIVER_DIR	unset	Override the directory searched for openshell-driver-vm.
--vm-driver-vcpus N	OPENSHELL_VM_DRIVER_VCPUS	2	vCPUs per sandbox.
--vm-driver-mem-mib N	OPENSHELL_VM_DRIVER_MEM_MIB	2048	Memory per sandbox, in MiB.
--vm-krun-log-level N	OPENSHELL_VM_KRUN_LOG_LEVEL	1	libkrun verbosity (0–5).
--vm-tls-ca PATH	OPENSHELL_VM_TLS_CA	—	CA cert for the guest's mTLS client bundle. Required when --grpc-endpoint uses https://.
--vm-tls-cert PATH	OPENSHELL_VM_TLS_CERT	—	Guest client certificate.
--vm-tls-key PATH	OPENSHELL_VM_TLS_KEY	—	Guest client private key.

See openshell-gateway --help for the full flag surface shared with the Kubernetes driver.

Verifying the gateway

The gateway is auto-registered by mise run gateway:vm. In another terminal:

./scripts/bin/openshell status
./scripts/bin/openshell sandbox create --name demo --from <compatible-image>
./scripts/bin/openshell sandbox connect demo

First sandbox takes 10–30 seconds to boot (image fetch/prepare/cache + libkrun + guest init). If --from is omitted, the VM driver uses the gateway's configured default sandbox image. Without either --from or --sandbox-image, VM sandbox creation fails. Subsequent creates reuse the prepared sandbox rootfs.

Logs and debugging

Raise log verbosity for both processes:

RUST_LOG=openshell_server=debug,openshell_driver_vm=debug \
  mise run gateway:vm

The VM guest's serial console is appended to <state-dir>/<sandbox-id>/console.log. Sandbox IDs must match [A-Za-z0-9._-]{1,128} before the driver uses them in host paths. The gateway-owned compute-driver socket lives at <state-dir>/run/compute-driver.sock; OpenShell creates run/ with owner-only permissions, removes same-owner stale sockets, and the gateway removes the socket on clean shutdown via ManagedDriverProcess::drop. UDS clients must match the driver UID and provide the expected gateway process PID by default. Standalone same-UID UDS mode requires the explicit --allow-same-uid-peer development flag. TCP mode is disabled by default because it is unauthenticated; use --allow-unauthenticated-tcp --bind-address 127.0.0.1:50061 only for local development.

Prerequisites

• macOS on Apple Silicon, or Linux on aarch64/x86_64 with KVM
• Rust toolchain
• Guest-supervisor cross-compile toolchain (needed on macOS, and on Linux when host arch ≠ guest arch):
  • Matching rustup target: rustup target add aarch64-unknown-linux-gnu (or x86_64-unknown-linux-gnu for an amd64 guest)
  • cargo install --locked cargo-zigbuild and brew install zig (or distro equivalent). vm:supervisor uses cargo zigbuild to cross-compile the in-VM openshell-sandbox supervisor binary.
• mise task runner
• Docker-compatible socket on the local CLI/gateway host when using openshell sandbox create --from ./Dockerfile or --from ./dir; the CLI builds the image and the VM driver exports it via the local Docker daemon
• gh CLI (used by mise run vm:setup to download pre-built runtime artifacts)

Releases

openshell-driver-vm is published as a normal OpenShell release artifact:

• development builds: the rolling dev release
• tagged builds: the corresponding v* release
• runtime tarballs: the rolling vm-runtime release, rebuilt on demand by release-vm-kernel.yml

On Debian-family Linux amd64 and arm64 systems, install-dev.sh installs the Debian package from the selected OPENSHELL_VERSION release tag. That package includes openshell-gateway and openshell-driver-vm, but leaves OPENSHELL_DRIVERS unset so the gateway uses its normal runtime auto-detection. Set OPENSHELL_DRIVERS=vm to force the VM driver.

On RPM-family Linux x86_64 and aarch64 systems, install-dev.sh installs the openshell and openshell-gateway RPM packages from the selected release tag. The RPM gateway package is configured for the Podman driver.

On Apple Silicon macOS, install-dev.sh stages the generated openshell.rb formula from the selected release in the nvidia/openshell Homebrew tap. Homebrew installs openshell, openshell-gateway, and openshell-driver-vm, ad-hoc signs the driver with the Hypervisor entitlement in post_install, and owns the brew services gateway lifecycle. The service also leaves OPENSHELL_DRIVERS unset so driver choice remains automatic unless the user explicitly overrides it.

TODOs

• The gateway still configures the driver via CLI args; this will move to a gRPC bootstrap call so the driver interface is uniform across backends. See the TODO(driver-abstraction) notes in crates/openshell-server/src/lib.rs and crates/openshell-server/src/compute/vm.rs.
• macOS local builds are codesigned by tasks/scripts/gateway-vm.sh; the generated Homebrew formula signs the release tarball driver for local installs.