Files
OpenShell/crates/openshell-driver-mxc
Philippe Martin 118b250f01 feat(sandbox): support rootfs tar as --from source for VM driver (#2863)
* feat(sandbox): support rootfs tar as --from source for VM driver

Accept flat rootfs tar archives (.tar, .tar.gz, .tgz) via the --from
flag for VM-backed gateways. The CLI detects the archive extension,
validates that the gateway uses the VM compute driver, and passes the
tar path through driver_config. The VM driver copies the tar into its
staging area and feeds it into the existing rootfs extraction and ext4
disk creation pipeline, skipping the container image pull/export steps.

Closes #2175

Signed-off-by: Philippe Martin <phmartin@redhat.com>

* fix(sandbox): validate rootfs tar path at the VM driver boundary

The rootfs_tar_path field in driver_config was passed from the API
caller directly to tokio::fs::copy without validation. An authenticated
user bypassing the CLI could supply arbitrary host paths (e.g.
/dev/zero for disk exhaustion, or readable host files for data
exfiltration).

Introduce a trusted staging directory that the VM driver creates on
startup and advertises via GetCapabilities. The CLI now copies the tar
into the staging directory before creating the sandbox, and the driver
validates that the received path is a regular file inside the staging
root and within a configurable size limit (default 10 GiB) before any
I/O.

New VmDriverConfig options:
- rootfs_tar_staging_dir: override the staging directory
  (default: <state_dir>/rootfs-tar-staging)
- rootfs_tar_max_bytes: override the size limit (default: 10 GiB)

Addresses GATOR-28b5152e-01.

Signed-off-by: Philippe Martin <phmartin@redhat.com>

* fix(sandbox): request-scoped staging, size pre-check, and cleanup for rootfs tar

Tighten the rootfs tar staging flow to address the remaining GATOR-01
obligations:

- Request-scoped staging: the CLI creates a unique per-request
  subdirectory (req-<pid>) under the staging root instead of placing
  files directly in the shared directory. The driver enforces that the
  tar path is at depth 2 (staging_root/<subdir>/<file>), preventing
  cross-request path selection.

- Size pre-check: the driver advertises rootfs_tar_max_bytes via
  GetCapabilities. The CLI reads this limit and rejects oversized files
  before copying, avoiding disk exhaustion in the staging directory.

- Cleanup: the driver removes the request staging subdirectory after
  consuming the tar (on cache hit, copy success, or copy failure),
  ensuring staged data does not persist beyond the request.

Signed-off-by: Philippe Martin <phmartin@redhat.com>

* fix(vm): restore rootfs-tar sandboxes from persisted image identity

On restore or restart, the one-shot staged tar archive has already been
cleaned up. Reading the persisted image identity from the sandbox state
directory and resolving the cached disk path directly avoids re-accessing
the deleted staging path.

Addresses GATOR-168b9210-01.

Signed-off-by: Philippe Martin <phmartin@redhat.com>

* fix(cli): use random staging dirs and enforce byte limit during rootfs tar copy

Replace PID-based request staging directories with tempfile-generated
random names to prevent collisions and make paths unpredictable.
Replace bare tokio::fs::copy with a streaming copy loop that enforces
the advertised max_bytes limit during transfer, closing the TOCTOU gap
between the pre-copy size check and the actual copy.

Signed-off-by: Philippe Martin <phmartin@nvidia.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>

* fix(sandbox): issue rootfs tar staging slots from the gateway

A caller could name any host path in `driver_config.vm.rootfs_tar_path`,
which the privileged VM driver then read. The CLI-side locality check did
not apply to direct API requests.

The gateway now owns staging. `BeginRootfsTarStaging` allocates a
request-scoped directory under the driver-advertised staging root and
returns an opaque single-use token; `CreateSandbox` carries the token, and
the gateway substitutes the path it allocated before dispatching to the
driver. `template.driver_config.<driver>.rootfs_tar_path` is rejected
outright in request validation, so a caller-supplied path never reaches
privileged I/O.

Tokens are bound to the issuing workspace and subject, consumed once, and
expire after 30 minutes. Outstanding slots are capped per caller and
overall, so one caller can neither exhaust the staging filesystem nor
starve others. An RAII guard reclaims the directory on every failure path
after consumption, and an age-gated sweep runs at startup and on each
reconcile pass for directories whose driver died before its own cleanup.

The token is stripped from the public sandbox before persistence: the
stored copy is returned verbatim by GetSandbox, ListSandboxes and
WatchSandbox to every member of the workspace.

Also fixes two defects this exposed:

- The CLI wrote `rootfs_tar_path` at the top level of `driver_config`, but
  the gateway forwards only `driver_config.<driver_name>`, silently
  dropping unmatched keys. The archive never reached the VM driver, so the
  documented `--from ./rootfs.tar` flow did not work at all. Config is now
  nested under `vm` and deep-merged, so a caller's existing VM settings
  survive instead of being clobbered by a shallow extend.

- Staging previously required `GetGatewayInfo`, which is restricted to
  `platform_admin`, making the feature unusable for ordinary users on any
  RBAC-enabled gateway. The new RPC matches CreateSandbox at
  `sandbox:write` / `workspace_role: user`.

`compute_driver.proto` is unchanged; the gateway reads the staging root
from the capabilities it already stores.

Refs #2175

Signed-off-by: Philippe Martin <phmartin@redhat.com>

* fix(vm): derive rootfs tar cache identity from archive contents

The prepared-disk cache key combined the archive's full path with an mtime
truncated to seconds, then mapped punctuation to `-`. Distinct paths such as
`/tmp/a/b.tar` and `/tmp/a-b.tar` collapsed onto the same key and reused each
other's disk, a rewrite within the same second kept stale contents, and a long
path could exceed filesystem component limits.

Identity is now a SHA-256 of the archive contents. This is also what makes the
cache work at all now that the gateway allocates a fresh staging directory per
request: a path-derived key would miss on every create.

The archive is hashed, the cache checked, and only on a miss copied — so a hit
skips writing a multi-gigabyte file. The copy is hashed as it is written and
rejected if the digest differs from the first pass, which closes the window
where the source changes during staging rather than approximating it with a
re-stat.

Refs #2175

Signed-off-by: Philippe Martin <phmartin@redhat.com>

* fix(vm): decompress gzip rootfs tar archives during staging

`--from` accepts `.tar.gz` and `.tgz`, but the driver staged whatever bytes
it was given and the guest image-prep VM extracts the staged file with a
plain `tar -xpf`. Compressed sources therefore depended on the guest tar
auto-detecting gzip, and the prepared disk was sized from the compressed
length, which is far too small for the expanded rootfs.

Staging now detects gzip from the archive's magic bytes -- the driver only
ever sees a gateway-issued staging path, never the caller's file name -- and
writes an uncompressed tar. The digest still covers the source bytes, so the
"archive changed while staging" check is unaffected, and expansion is bounded
by `rootfs_tar_max_bytes` so a compression bomb cannot fill the host disk.

`extract_rootfs_archive_to` sniffs gzip as well, so the host-side extraction
path matches.

Adds unit coverage for gzip staging, bounded expansion, and gzip extraction,
plus an e2e sandbox created from a gzip-compressed export.

Signed-off-by: Philippe Martin <phmartin@redhat.com>

---------

Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@nvidia.com>
2026-09-08 22:36:40 +00:00
..

openshell-driver-mxc

OpenShell compute driver backed by Microsoft MXC (wxc-exec) on Windows.

Design

This driver implements the gateway's ordinary in-process ComputeDriver contract and is linked into openshell-gateway. It sets driver_reports_runtime_readiness, so the gateway accepts driver-reported readiness without a supervisor session. The canonical create-time SandboxPolicy is carried by DriverSandboxSpec.policy. process_container launches a one-shot AppContainer and is the default. The opt-in isolation_session backend uses the state-aware provision → start → exec → stop → deprovision lifecycle. The driver launches and monitors the configured workload itself and self-reports readiness; there is no in-sandbox supervisor or ConnectSupervisor relay.

Capability Matrix

Capability MXC driver
Filesystem policy Read-only/read-write grants come only from SandboxPolicy. process_container enforces default-deny; isolation_session is an explicit grant-only compatibility mode.
Network policy Rejected synchronously during sandbox creation until an enforcing egress path is bound.
Process policy Unsupported; MXC supplies OS isolation only.
Interactive exec/connect/forward Unsupported; the configured workload runs in-driver.
Restart durability Unsupported; the in-memory registry cannot recover live sessions.

The filesystem enforcement proof has two paths:

  • A write to a path granted by the sandbox policy succeeds.
  • A process_container write outside the sandbox policy fails with Windows access denied, and the driver reports the failed workload.

Configuration ([openshell.drivers.mxc])

Gateway configuration contains only host runtime settings:

[openshell.drivers.mxc]
wxc_exec_path = "C:\\path\\to\\wxc-exec.exe"
# Default: process_container. isolation_session is grant-only and opt-in.
backend = "process_container"
default_configuration_id = "composable"
pc_least_privilege = false
pc_capabilities = []
debug = false

Supply workload settings for each sandbox. The public config is keyed by driver name; the gateway forwards only the inner mxc object to the driver:

$config = '{"mxc":{"command":["cmd","/c","echo hello > C:\\\\work\\\\demo\\\\hello.txt"],"cwd":"C:\\\\work\\\\demo"}}'
openshell sandbox create --name mxc-demo --policy demo.yaml `
  --driver-config-json $config --env MODE=demo --no-tty

The command array is required and preserves Windows argument boundaries. cwd is optional. Environment variables come from the standard sandbox and template environment maps; the driver never copies values from the gateway host environment.

Network policy support depends on the selected containment path. Policy replacement and merge updates use the gateway's standard sandbox configuration revision contract.

Prerequisites (live runs)

  • Windows 11 Insider build ≥ 26300.8553
  • IsoSessionApp.dll present and registered
  • wxc-exec.exe built with --features isolation_session

For off-box smoke tests against the in-process mock shim (no wxc-exec, no isolation session needed), set OPENSHELL_MXC_MOCK_WXC=1.

Policy mapping

The production driver maps the typed SandboxPolicy carried by the standard driver request to MXC configuration before it inserts a registry entry or invokes wxc-exec. Mapping failure therefore returns from CreateSandbox without leaving a partial sandbox. There is no in-process policy side channel or MXC-specific gateway composition variant.

EmbeddedPolicyMapper calls the embedded policy_map module directly and normalizes filesystem paths to Windows form. It does not add gateway-configured host paths. The policy supplied for the sandbox is the only source of filesystem grants.

The mapper retains an internal policy-splitting seam for future development, but the runtime exposes no governed-egress switch. Any network rule fails closed until an enforcing proxy is implemented and bound to the sandbox lifecycle.

Parity and matrix tests under tests/ cover the mapper on the Windows MSVC lane. The driver performs this mapping automatically; there is no separate policy-export command or example.

Packaging the demo for the demo box

Use examples/package-demo.ps1 to assemble the gateway EXE, CLI EXE, runtime DLLs (libz3.dll), demo.yaml, the gateway config, and the runbook into one folder, then copy that folder to the demo Windows host and follow mxc-demo-runbook.md inside it. The script prints a SHA256 manifest so the operator can sanity-check what landed before moving it.

Real-MXC test lane

Three tasks drive real wxc-exec.exe hardware; all are skip-safe — any test or scenario that requires an absent binary or backend prints a SKIP reason and exits 0 rather than failing.

Task What it runs When to use
windows:test:mxc-real:x64 tests/wxc_exec_real.rs — Tier-2 invoker tests with --ignored --test-threads=1 Pre-merge on any Windows host that has wxc-exec; dry-run tests always pass; enforcement tests probe-gate themselves
windows:e2e:mxc examples/run-mxc-e2e.ps1 — Tier-3 scenario runner, real binary, probe-gated Demo box / nightly; needs the gateway + CLI binaries in the script directory
windows:e2e:mxc:mock Same runner with -Mock — wiring-only, no real wxc-exec needed Any Windows host (CI, dev machine); validates wiring and the network-reject scenario

Probe script: examples/probe-mxc-host.ps1 is an operator/CI preflight that emits a JSON capability report (OS build, wxc-exec path/version, dry-run exit code, per-backend trial result, and a verdicts object). Run it before the real-MXC lane to understand what will PASS vs SKIP on a given host:

powershell -NoProfile -ExecutionPolicy Bypass `
  -File crates/openshell-driver-mxc/examples/probe-mxc-host.ps1

Skip semantics: tests in wxc_exec_real.rs are marked #[ignore = "requires real wxc-exec"] — the standard windows:test:x64 suite never runs them. OPENSHELL_WXC_EXEC_PATH overrides the default C:\mxc\wxc-exec.exe lookup. See docs4gtb/mxc-box-capabilities.md for the empirical capability snapshot of the development box (build 26200, processcontainer velocity keys not enabled, isolation_session absent).

Deferred work

  • Interactive exec/connect/forward → adapt-openshell-gateway-windows
  • Governed egress remains fail-closed until an enforcing proxy is implemented and bound to sandbox lifecycle.
  • Restart durability (deprovision orphaned sessions on startup) → follow-on
  • GPU passthrough → not pursued in host-side-governance design