mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-19 05:50:41 +08:00
* feat(sandbox): support rootfs tar as --from source for VM driver Accept flat rootfs tar archives (.tar, .tar.gz, .tgz) via the --from flag for VM-backed gateways. The CLI detects the archive extension, validates that the gateway uses the VM compute driver, and passes the tar path through driver_config. The VM driver copies the tar into its staging area and feeds it into the existing rootfs extraction and ext4 disk creation pipeline, skipping the container image pull/export steps. Closes #2175 Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox): validate rootfs tar path at the VM driver boundary The rootfs_tar_path field in driver_config was passed from the API caller directly to tokio::fs::copy without validation. An authenticated user bypassing the CLI could supply arbitrary host paths (e.g. /dev/zero for disk exhaustion, or readable host files for data exfiltration). Introduce a trusted staging directory that the VM driver creates on startup and advertises via GetCapabilities. The CLI now copies the tar into the staging directory before creating the sandbox, and the driver validates that the received path is a regular file inside the staging root and within a configurable size limit (default 10 GiB) before any I/O. New VmDriverConfig options: - rootfs_tar_staging_dir: override the staging directory (default: <state_dir>/rootfs-tar-staging) - rootfs_tar_max_bytes: override the size limit (default: 10 GiB) Addresses GATOR-28b5152e-01. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox): request-scoped staging, size pre-check, and cleanup for rootfs tar Tighten the rootfs tar staging flow to address the remaining GATOR-01 obligations: - Request-scoped staging: the CLI creates a unique per-request subdirectory (req-<pid>) under the staging root instead of placing files directly in the shared directory. The driver enforces that the tar path is at depth 2 (staging_root/<subdir>/<file>), preventing cross-request path selection. - Size pre-check: the driver advertises rootfs_tar_max_bytes via GetCapabilities. The CLI reads this limit and rejects oversized files before copying, avoiding disk exhaustion in the staging directory. - Cleanup: the driver removes the request staging subdirectory after consuming the tar (on cache hit, copy success, or copy failure), ensuring staged data does not persist beyond the request. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(vm): restore rootfs-tar sandboxes from persisted image identity On restore or restart, the one-shot staged tar archive has already been cleaned up. Reading the persisted image identity from the sandbox state directory and resolving the cached disk path directly avoids re-accessing the deleted staging path. Addresses GATOR-168b9210-01. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(cli): use random staging dirs and enforce byte limit during rootfs tar copy Replace PID-based request staging directories with tempfile-generated random names to prevent collisions and make paths unpredictable. Replace bare tokio::fs::copy with a streaming copy loop that enforces the advertised max_bytes limit during transfer, closing the TOCTOU gap between the pre-copy size check and the actual copy. Signed-off-by: Philippe Martin <phmartin@nvidia.com> Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox): issue rootfs tar staging slots from the gateway A caller could name any host path in `driver_config.vm.rootfs_tar_path`, which the privileged VM driver then read. The CLI-side locality check did not apply to direct API requests. The gateway now owns staging. `BeginRootfsTarStaging` allocates a request-scoped directory under the driver-advertised staging root and returns an opaque single-use token; `CreateSandbox` carries the token, and the gateway substitutes the path it allocated before dispatching to the driver. `template.driver_config.<driver>.rootfs_tar_path` is rejected outright in request validation, so a caller-supplied path never reaches privileged I/O. Tokens are bound to the issuing workspace and subject, consumed once, and expire after 30 minutes. Outstanding slots are capped per caller and overall, so one caller can neither exhaust the staging filesystem nor starve others. An RAII guard reclaims the directory on every failure path after consumption, and an age-gated sweep runs at startup and on each reconcile pass for directories whose driver died before its own cleanup. The token is stripped from the public sandbox before persistence: the stored copy is returned verbatim by GetSandbox, ListSandboxes and WatchSandbox to every member of the workspace. Also fixes two defects this exposed: - The CLI wrote `rootfs_tar_path` at the top level of `driver_config`, but the gateway forwards only `driver_config.<driver_name>`, silently dropping unmatched keys. The archive never reached the VM driver, so the documented `--from ./rootfs.tar` flow did not work at all. Config is now nested under `vm` and deep-merged, so a caller's existing VM settings survive instead of being clobbered by a shallow extend. - Staging previously required `GetGatewayInfo`, which is restricted to `platform_admin`, making the feature unusable for ordinary users on any RBAC-enabled gateway. The new RPC matches CreateSandbox at `sandbox:write` / `workspace_role: user`. `compute_driver.proto` is unchanged; the gateway reads the staging root from the capabilities it already stores. Refs #2175 Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(vm): derive rootfs tar cache identity from archive contents The prepared-disk cache key combined the archive's full path with an mtime truncated to seconds, then mapped punctuation to `-`. Distinct paths such as `/tmp/a/b.tar` and `/tmp/a-b.tar` collapsed onto the same key and reused each other's disk, a rewrite within the same second kept stale contents, and a long path could exceed filesystem component limits. Identity is now a SHA-256 of the archive contents. This is also what makes the cache work at all now that the gateway allocates a fresh staging directory per request: a path-derived key would miss on every create. The archive is hashed, the cache checked, and only on a miss copied — so a hit skips writing a multi-gigabyte file. The copy is hashed as it is written and rejected if the digest differs from the first pass, which closes the window where the source changes during staging rather than approximating it with a re-stat. Refs #2175 Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(vm): decompress gzip rootfs tar archives during staging `--from` accepts `.tar.gz` and `.tgz`, but the driver staged whatever bytes it was given and the guest image-prep VM extracts the staged file with a plain `tar -xpf`. Compressed sources therefore depended on the guest tar auto-detecting gzip, and the prepared disk was sized from the compressed length, which is far too small for the expanded rootfs. Staging now detects gzip from the archive's magic bytes -- the driver only ever sees a gateway-issued staging path, never the caller's file name -- and writes an uncompressed tar. The digest still covers the source bytes, so the "archive changed while staging" check is unaffected, and expansion is bounded by `rootfs_tar_max_bytes` so a compression bomb cannot fill the host disk. `extract_rootfs_archive_to` sniffs gzip as well, so the host-side extraction path matches. Adds unit coverage for gzip staging, bounded expansion, and gzip extraction, plus an e2e sandbox created from a gzip-compressed export. Signed-off-by: Philippe Martin <phmartin@redhat.com> --------- Signed-off-by: Philippe Martin <phmartin@redhat.com> Signed-off-by: Philippe Martin <phmartin@nvidia.com>
159 lines
5.4 KiB
Rust
159 lines
5.4 KiB
Rust
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
// SPDX-License-Identifier: Apache-2.0
|
|
|
|
#![cfg(feature = "e2e")]
|
|
|
|
//! E2E tests: create a sandbox from a flat rootfs tar archive, plain and
|
|
//! gzip-compressed.
|
|
//!
|
|
//! Prerequisites:
|
|
//! - A running VM-backed openshell gateway with a default sandbox image configured
|
|
//! - Docker daemon running (for image build + container export)
|
|
//! - The `openshell` binary (built automatically from the workspace)
|
|
|
|
use openshell_e2e::harness::container::ContainerEngine;
|
|
use openshell_e2e::harness::output::strip_ansi;
|
|
use openshell_e2e::harness::sandbox::SandboxGuard;
|
|
use std::path::{Path, PathBuf};
|
|
use std::process::Command;
|
|
|
|
const DOCKERFILE_CONTENT: &str = r#"FROM public.ecr.aws/docker/library/python:3.13-slim
|
|
|
|
# iproute2 is required for sandbox network namespace isolation.
|
|
RUN apt-get update && apt-get install -y --no-install-recommends iproute2 \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Create the sandbox user/group so the supervisor can switch to it.
|
|
RUN groupadd -g 1000660000 sandbox && \
|
|
useradd -m -u 1000660000 -g sandbox sandbox
|
|
|
|
RUN echo "rootfs-tar-e2e-marker" > /etc/marker.txt
|
|
|
|
CMD ["sleep", "infinity"]
|
|
"#;
|
|
|
|
const MARKER: &str = "rootfs-tar-e2e-marker";
|
|
|
|
/// Build a Docker image and export its filesystem as a flat rootfs tar.
|
|
///
|
|
/// `suffix` keeps the image tag and temporary container name unique so tests
|
|
/// exercising different archive formats can run concurrently.
|
|
fn export_rootfs_tar(engine: &ContainerEngine, tmpdir: &Path, suffix: &str) -> PathBuf {
|
|
let dockerfile_path = tmpdir.join("Dockerfile");
|
|
std::fs::write(&dockerfile_path, DOCKERFILE_CONTENT).expect("write Dockerfile");
|
|
|
|
let tag = format!(
|
|
"openshell/e2e-rootfs-tar-test-{suffix}:{}",
|
|
std::time::SystemTime::now()
|
|
.duration_since(std::time::UNIX_EPOCH)
|
|
.unwrap_or_default()
|
|
.as_secs()
|
|
);
|
|
|
|
let build_output = engine
|
|
.command()
|
|
.args(["build", "-t", &tag, "-f"])
|
|
.arg(&dockerfile_path)
|
|
.arg(tmpdir)
|
|
.output()
|
|
.expect("spawn docker build");
|
|
|
|
assert!(
|
|
build_output.status.success(),
|
|
"docker build failed:\n{}",
|
|
String::from_utf8_lossy(&build_output.stderr)
|
|
);
|
|
|
|
// Create a temporary container and export its filesystem as a flat rootfs
|
|
// tar (equivalent to `docker export`).
|
|
let container_name = format!(
|
|
"openshell-e2e-rootfs-export-{suffix}-{}",
|
|
std::process::id()
|
|
);
|
|
|
|
let create_output = engine
|
|
.command()
|
|
.args(["create", "--name", &container_name, &tag])
|
|
.output()
|
|
.expect("spawn docker create");
|
|
|
|
assert!(
|
|
create_output.status.success(),
|
|
"docker create failed:\n{}",
|
|
String::from_utf8_lossy(&create_output.stderr)
|
|
);
|
|
|
|
let rootfs_tar_path = tmpdir.join("rootfs.tar");
|
|
let export_output = engine
|
|
.command()
|
|
.args(["export", "-o"])
|
|
.arg(&rootfs_tar_path)
|
|
.arg(&container_name)
|
|
.output()
|
|
.expect("spawn docker export");
|
|
|
|
assert!(
|
|
export_output.status.success(),
|
|
"docker export failed:\n{}",
|
|
String::from_utf8_lossy(&export_output.stderr)
|
|
);
|
|
|
|
// Clean up the temporary container and image.
|
|
let _ = engine.command().args(["rm", &container_name]).output();
|
|
let _ = engine.command().args(["rmi", &tag]).output();
|
|
|
|
rootfs_tar_path
|
|
}
|
|
|
|
/// Create a sandbox from `archive` and assert the marker baked into the image
|
|
/// shows up in its output.
|
|
async fn assert_sandbox_from_archive(archive: &Path) {
|
|
let archive_str = archive.to_str().expect("archive path is UTF-8");
|
|
let mut guard = SandboxGuard::create(&["--from", archive_str, "--", "cat", "/etc/marker.txt"])
|
|
.await
|
|
.expect("sandbox create from rootfs tar");
|
|
|
|
let clean_output = strip_ansi(&guard.create_output);
|
|
assert!(
|
|
clean_output.contains(MARKER),
|
|
"expected marker '{MARKER}' in sandbox output for {}:\n{clean_output}",
|
|
archive.display()
|
|
);
|
|
|
|
guard.cleanup().await;
|
|
}
|
|
|
|
/// Build a Docker image, export its filesystem as a flat rootfs tar, then
|
|
/// create a sandbox from that tar and verify it contains the expected marker.
|
|
#[tokio::test]
|
|
async fn sandbox_from_rootfs_tar() {
|
|
let engine = ContainerEngine::from_env().expect("container engine available");
|
|
let tmpdir = tempfile::tempdir().expect("create tmpdir");
|
|
|
|
let rootfs_tar_path = export_rootfs_tar(&engine, tmpdir.path(), "plain");
|
|
|
|
assert_sandbox_from_archive(&rootfs_tar_path).await;
|
|
}
|
|
|
|
/// The CLI advertises `.tar.gz` and `.tgz` sources, so a gzip-compressed
|
|
/// export has to reach the sandbox the same way a plain tar does.
|
|
#[tokio::test]
|
|
async fn sandbox_from_gzipped_rootfs_tar() {
|
|
let engine = ContainerEngine::from_env().expect("container engine available");
|
|
let tmpdir = tempfile::tempdir().expect("create tmpdir");
|
|
|
|
let rootfs_tar_path = export_rootfs_tar(&engine, tmpdir.path(), "gzip");
|
|
let gzipped_path = tmpdir.path().join("rootfs.tar.gz");
|
|
let gzipped = std::fs::File::create(&gzipped_path).expect("create gzip archive");
|
|
let gzip_status = Command::new("gzip")
|
|
.arg("-c")
|
|
.arg(&rootfs_tar_path)
|
|
.stdout(gzipped)
|
|
.status()
|
|
.expect("spawn gzip");
|
|
assert!(gzip_status.success(), "gzip failed: {gzip_status}");
|
|
std::fs::remove_file(&rootfs_tar_path).expect("remove uncompressed archive");
|
|
|
|
assert_sandbox_from_archive(&gzipped_path).await;
|
|
}
|