mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-23 06:30:28 +08:00
* feat(podman): add Podman compute driver for rootless sandbox management Adds openshell-driver-podman, a new compute driver that manages OpenShell sandboxes as rootless Podman containers via the Podman REST API over a Unix socket. Enables local workstation sandboxes without Kubernetes. Driver features: - Bridge networking with ephemeral host-port mapping for rootless SSH reachability - Named volumes for workspace storage, Podman native health checks, GPU via CDI - Supervisor binary sideloaded via image volume mount (BYOC-compatible) - SSH handshake secret injected via Podman secrets API (not plaintext env) - Typed ContainerSpec structs, input validation, and path-traversal guards - Cgroups v2 required; fails fast on v1 hosts - Bounded event stream buffer; watch stream reconnection handled by server watch_loop - Graceful shutdown and standalone driver binary with gRPC bridge Rootless-specific fixes: - Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps - Add /run/netns tmpfs mount for ip netns in rootless containers - Use secret_env map (not secrets array) for env-var injection in libpod API - Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP Server/sandbox hardening: - Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback - Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting - Prevent OPENSHELL_* env var override by user-supplied spec environment maps - Disable SQLite pool idle_timeout/max_lifetime for in-memory databases - Emit deleted_event on 404-during-inspect instead of regressing sandbox phase - Key delete cleanup by stable sandbox_id to survive container label drift CLI fixes: - Restore --name as a named flag on sandbox create (not positional) - Fix exec command arg parsing to not consume sandboxed-command flags - Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL Build tooling: - Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes unified ce_* helpers; all build/cluster/VM scripts updated to use it - Add docker:build:supervisor mise task for standalone supervisor image - Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages - Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing Signed-off-by: Adam Miller <admiller@redhat.com> * fix(driver-podman): derive grpc endpoint from server bind port When a user starts the gateway on a non-default port (e.g. --port 8081), sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the default port 8080. The driver's auto-detection fallback read OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset, and fell back to DEFAULT_SERVER_PORT. Add gateway_port to PodmanComputeConfig and thread config.bind_address.port() from the server into the driver so the fallback uses the actual listening port. Remove the OPENSHELL_BIND_ADDRESS env var read and the extract_port_from_bind_address helper which are no longer needed. Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver binary for parity when the driver is run outside the embedded server path. Signed-off-by: Adam Miller <admiller@redhat.com> * fix(driver-podman): address PR feedback on env test safety and cluster DNS docs Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars and a static ENV_LOCK mutex, fixing a data race in parallel test execution. The prior safety comment incorrectly claimed Cargo runs tests single-threaded. Update debug-openshell-cluster skill to accurately document the DNS proxy strategy (setup_dns_proxy + public DNS fallback) and clarify the separation between cluster DNS and sandbox agent DNS enforcement. Signed-off-by: Adam Miller <admiller@redhat.com> * fix(e2e): resolve CI failures in auth timeout, test harness, and formatting - Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives - Add timeout to SandboxGuard::create() and create_with_upload() to prevent indefinite hangs (matches create_keep() which already had one) - Add missing '--' separator in no_proxy test before command args - Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs - Run cargo fmt on openshell-driver-podman - Refine cluster DNS docs for Podman in debug-openshell-cluster skill Signed-off-by: Adam Miller <admiller@redhat.com> * refactor(server): remove allows_loopback_endpoints from ComputeRuntime SSRF protection is now handled at the network and proxy layers (openshell-core net.rs, openshell-sandbox proxy.rs) rather than requiring per-driver flags on ComputeRuntime. Update architecture docs to reflect supervisor relay SSH transport and add rootless networking deep-dive. Signed-off-by: Adam Miller <admiller@redhat.com> --------- Signed-off-by: Adam Miller <admiller@redhat.com>
112 lines
3.5 KiB
Bash
Executable File
112 lines
3.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
|
|
# Build rootfs and compress to tarball for embedding in openshell-vm binary.
|
|
#
|
|
# This script:
|
|
# 1. Builds the rootfs using build-rootfs.sh
|
|
# 2. Compresses it to a zstd tarball for embedding
|
|
#
|
|
# Usage:
|
|
# ./build-rootfs-tarball.sh [--base]
|
|
#
|
|
# Options:
|
|
# --base Build a base rootfs (~200-300MB) without pre-loaded images.
|
|
# First boot will be slower but binary size is much smaller.
|
|
# Default: full rootfs with pre-loaded images (~2GB+).
|
|
#
|
|
# The resulting tarball is placed at target/vm-runtime-compressed/rootfs.tar.zst
|
|
# for inclusion in the embedded binary build.
|
|
|
|
set -euo pipefail
|
|
|
|
source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/container-engine.sh"
|
|
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
|
|
ROOTFS_BUILD_DIR="${ROOT}/target/rootfs-build"
|
|
OUTPUT_DIR="${ROOT}/target/vm-runtime-compressed"
|
|
OUTPUT="${OUTPUT_DIR}/rootfs.tar.zst"
|
|
|
|
# Parse arguments
|
|
BASE_ONLY=false
|
|
for arg in "$@"; do
|
|
case "$arg" in
|
|
--base)
|
|
BASE_ONLY=true
|
|
;;
|
|
--help|-h)
|
|
echo "Usage: $0 [--base]"
|
|
echo ""
|
|
echo "Options:"
|
|
echo " --base Build base rootfs (~200-300MB) without pre-loaded images"
|
|
echo " First boot will be slower but binary size is much smaller"
|
|
exit 0
|
|
;;
|
|
*)
|
|
echo "Unknown option: $arg"
|
|
echo "Use --help for usage information"
|
|
exit 1
|
|
;;
|
|
esac
|
|
done
|
|
|
|
# Check if container engine is running
|
|
if ! ce info &>/dev/null; then
|
|
echo "Error: container engine is not running" >&2
|
|
echo "Please start your container engine and try again" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [ "$BASE_ONLY" = true ]; then
|
|
echo "==> Building BASE rootfs for embedding"
|
|
echo " Build dir: ${ROOTFS_BUILD_DIR}"
|
|
echo " Output: ${OUTPUT}"
|
|
echo " Mode: base (no pre-loaded images, ~200-300MB)"
|
|
echo ""
|
|
|
|
# Build base rootfs
|
|
echo "==> Step 1/2: Building base rootfs..."
|
|
"${ROOT}/crates/openshell-vm/scripts/build-rootfs.sh" --base "${ROOTFS_BUILD_DIR}"
|
|
else
|
|
echo "==> Building FULL rootfs for embedding"
|
|
echo " Build dir: ${ROOTFS_BUILD_DIR}"
|
|
echo " Output: ${OUTPUT}"
|
|
echo " Mode: full (pre-loaded images, pre-initialized, ~2GB+)"
|
|
echo ""
|
|
|
|
# Build full rootfs
|
|
echo "==> Step 1/2: Building full rootfs (this may take 10-15 minutes)..."
|
|
"${ROOT}/crates/openshell-vm/scripts/build-rootfs.sh" "${ROOTFS_BUILD_DIR}"
|
|
fi
|
|
|
|
# Compress to tarball
|
|
echo ""
|
|
echo "==> Step 2/2: Compressing rootfs to tarball..."
|
|
mkdir -p "${OUTPUT_DIR}"
|
|
|
|
# Remove existing tarball if present
|
|
rm -f "${OUTPUT}"
|
|
|
|
# Get uncompressed size for display
|
|
echo " Uncompressed size: $(du -sh "${ROOTFS_BUILD_DIR}" | cut -f1)"
|
|
|
|
# Create tarball with zstd compression
|
|
# -19 = high compression (slower but smaller)
|
|
# -T0 = use all available threads
|
|
echo " Compressing with zstd (level 19, this may take a few minutes)..."
|
|
tar -C "${ROOTFS_BUILD_DIR}" -cf - . | zstd -19 -T0 -o "${OUTPUT}"
|
|
|
|
# Report results
|
|
echo ""
|
|
echo "==> Rootfs tarball created successfully!"
|
|
echo " Output: ${OUTPUT}"
|
|
echo " Compressed: $(du -sh "${OUTPUT}" | cut -f1)"
|
|
if [ "$BASE_ONLY" = true ]; then
|
|
echo " Type: base (first boot ~30-60s, images pulled on demand)"
|
|
else
|
|
echo " Type: full (first boot ~3-5s, images pre-loaded)"
|
|
fi
|
|
echo ""
|
|
echo "Next step: mise run vm:build"
|