mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-17 05:31:16 +08:00
* feat(podman): add Podman compute driver for rootless sandbox management Adds openshell-driver-podman, a new compute driver that manages OpenShell sandboxes as rootless Podman containers via the Podman REST API over a Unix socket. Enables local workstation sandboxes without Kubernetes. Driver features: - Bridge networking with ephemeral host-port mapping for rootless SSH reachability - Named volumes for workspace storage, Podman native health checks, GPU via CDI - Supervisor binary sideloaded via image volume mount (BYOC-compatible) - SSH handshake secret injected via Podman secrets API (not plaintext env) - Typed ContainerSpec structs, input validation, and path-traversal guards - Cgroups v2 required; fails fast on v1 hosts - Bounded event stream buffer; watch stream reconnection handled by server watch_loop - Graceful shutdown and standalone driver binary with gRPC bridge Rootless-specific fixes: - Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps - Add /run/netns tmpfs mount for ip netns in rootless containers - Use secret_env map (not secrets array) for env-var injection in libpod API - Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP Server/sandbox hardening: - Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback - Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting - Prevent OPENSHELL_* env var override by user-supplied spec environment maps - Disable SQLite pool idle_timeout/max_lifetime for in-memory databases - Emit deleted_event on 404-during-inspect instead of regressing sandbox phase - Key delete cleanup by stable sandbox_id to survive container label drift CLI fixes: - Restore --name as a named flag on sandbox create (not positional) - Fix exec command arg parsing to not consume sandboxed-command flags - Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL Build tooling: - Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes unified ce_* helpers; all build/cluster/VM scripts updated to use it - Add docker:build:supervisor mise task for standalone supervisor image - Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages - Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing Signed-off-by: Adam Miller <admiller@redhat.com> * fix(driver-podman): derive grpc endpoint from server bind port When a user starts the gateway on a non-default port (e.g. --port 8081), sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the default port 8080. The driver's auto-detection fallback read OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset, and fell back to DEFAULT_SERVER_PORT. Add gateway_port to PodmanComputeConfig and thread config.bind_address.port() from the server into the driver so the fallback uses the actual listening port. Remove the OPENSHELL_BIND_ADDRESS env var read and the extract_port_from_bind_address helper which are no longer needed. Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver binary for parity when the driver is run outside the embedded server path. Signed-off-by: Adam Miller <admiller@redhat.com> * fix(driver-podman): address PR feedback on env test safety and cluster DNS docs Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars and a static ENV_LOCK mutex, fixing a data race in parallel test execution. The prior safety comment incorrectly claimed Cargo runs tests single-threaded. Update debug-openshell-cluster skill to accurately document the DNS proxy strategy (setup_dns_proxy + public DNS fallback) and clarify the separation between cluster DNS and sandbox agent DNS enforcement. Signed-off-by: Adam Miller <admiller@redhat.com> * fix(e2e): resolve CI failures in auth timeout, test harness, and formatting - Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives - Add timeout to SandboxGuard::create() and create_with_upload() to prevent indefinite hangs (matches create_keep() which already had one) - Add missing '--' separator in no_proxy test before command args - Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs - Run cargo fmt on openshell-driver-podman - Refine cluster DNS docs for Podman in debug-openshell-cluster skill Signed-off-by: Adam Miller <admiller@redhat.com> * refactor(server): remove allows_loopback_endpoints from ComputeRuntime SSRF protection is now handled at the network and proxy layers (openshell-core net.rs, openshell-sandbox proxy.rs) rather than requiring per-driver flags on ComputeRuntime. Update architecture docs to reflect supervisor relay SSH transport and add rootless networking deep-dive. Signed-off-by: Adam Miller <admiller@redhat.com> --------- Signed-off-by: Adam Miller <admiller@redhat.com>
189 lines
7.7 KiB
Bash
Executable File
189 lines
7.7 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
|
|
# Sync mutable development artifacts into the existing VM rootfs.
|
|
# Runs on every `mise run vm` so that script changes, helm chart
|
|
# updates, manifest changes, and supervisor binary rebuilds are
|
|
# picked up without a full rootfs rebuild.
|
|
#
|
|
# This is fast (<1s) — it only copies files, no Docker or VM boot.
|
|
|
|
set -euo pipefail
|
|
|
|
source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/container-engine.sh"
|
|
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
|
|
SCRIPT_DIR="${ROOT}/crates/openshell-vm/scripts"
|
|
IMAGE_REPO_BASE="${IMAGE_REPO_BASE:-openshell}"
|
|
IMAGE_TAG="${IMAGE_TAG:-dev}"
|
|
SERVER_IMAGE="${IMAGE_REPO_BASE}/gateway:${IMAGE_TAG}"
|
|
NAME="default"
|
|
ROOTFS_ARGS=()
|
|
|
|
while [[ $# -gt 0 ]]; do
|
|
case "$1" in
|
|
--name)
|
|
NAME="$2"
|
|
shift 2
|
|
;;
|
|
--name=*)
|
|
NAME="${1#--name=}"
|
|
shift
|
|
;;
|
|
--rootfs)
|
|
ROOTFS_ARGS=("$1" "$2")
|
|
shift 2
|
|
;;
|
|
--rootfs=*)
|
|
ROOTFS_ARGS=("$1")
|
|
shift
|
|
;;
|
|
*)
|
|
echo "Unknown argument: $1" >&2
|
|
exit 1
|
|
;;
|
|
esac
|
|
done
|
|
|
|
ensure_args=(--name "${NAME}")
|
|
if [ "${#ROOTFS_ARGS[@]}" -gt 0 ]; then
|
|
ensure_args=("${ROOTFS_ARGS[@]}" "${ensure_args[@]}")
|
|
fi
|
|
|
|
if ! ROOTFS_DIR="$("${ROOT}/tasks/scripts/vm/ensure-vm-rootfs.sh" "${ensure_args[@]}" | tail -n 1 | sed 's/^using openshell-vm rootfs at //')"; then
|
|
echo "ERROR: ensure-vm-rootfs.sh failed — no rootfs available." >&2
|
|
exit 1
|
|
fi
|
|
|
|
patch_vm_helmchart() {
|
|
local helmchart="$1"
|
|
[ -f "${helmchart}" ] || return 0
|
|
|
|
sed_in_place() {
|
|
local expr="$1"
|
|
sed -i.bak -E "${expr}" "${helmchart}"
|
|
rm -f "${helmchart}.bak"
|
|
}
|
|
|
|
# Mirror the build-rootfs patching so the VM keeps using the locally
|
|
# imported openshell/gateway:dev image after incremental rootfs syncs.
|
|
sed_in_place 's|__IMAGE_PULL_POLICY__|IfNotPresent|g'
|
|
sed_in_place 's|__SANDBOX_IMAGE_PULL_POLICY__|"IfNotPresent"|g'
|
|
sed_in_place 's|__DB_URL__|"sqlite:/tmp/openshell.db"|g'
|
|
sed_in_place "s|repository:[[:space:]]*[^[:space:]]+|repository: ${SERVER_IMAGE%:*}|"
|
|
sed_in_place "s|tag:[[:space:]]*\"?[^\"[:space:]]+\"?|tag: \"${IMAGE_TAG}\"|"
|
|
sed_in_place 's|sshGatewayHost: __SSH_GATEWAY_HOST__|sshGatewayHost: ""|g'
|
|
sed_in_place 's|sshGatewayPort: __SSH_GATEWAY_PORT__|sshGatewayPort: 0|g'
|
|
sed_in_place 's|__DISABLE_GATEWAY_AUTH__|false|g'
|
|
sed_in_place 's|__DISABLE_TLS__|false|g'
|
|
sed_in_place 's|hostGatewayIP: __HOST_GATEWAY_IP__|hostGatewayIP: ""|g'
|
|
sed_in_place '/__CHART_CHECKSUM__/d'
|
|
}
|
|
|
|
if [ ! -d "${ROOTFS_DIR}/srv" ]; then
|
|
# Rootfs doesn't exist yet — nothing to sync. ensure-vm-rootfs.sh
|
|
# or build-rootfs.sh will create it.
|
|
exit 0
|
|
fi
|
|
|
|
echo "Syncing development artifacts into rootfs..."
|
|
|
|
# ── Init scripts and utilities ─────────────────────────────────────────
|
|
for script in openshell-vm-init.sh openshell-vm-exec-agent.py check-vm-capabilities.sh; do
|
|
src="${SCRIPT_DIR}/${script}"
|
|
dst="${ROOTFS_DIR}/srv/${script}"
|
|
if [ -f "$src" ]; then
|
|
if ! cmp -s "$src" "$dst" 2>/dev/null; then
|
|
cp "$src" "$dst"
|
|
chmod +x "$dst"
|
|
echo " updated: /srv/${script}"
|
|
fi
|
|
fi
|
|
done
|
|
|
|
# ── Helm chart ─────────────────────────────────────────────────────────
|
|
HELM_CHART_DIR="${ROOT}/deploy/helm/openshell"
|
|
CHART_STAGING="${ROOTFS_DIR}/opt/openshell/charts"
|
|
if [ -d "${HELM_CHART_DIR}" ]; then
|
|
if ! command -v helm >/dev/null 2>&1; then
|
|
echo " warning: helm not found — skipping chart sync (run: mise install)" >&2
|
|
else
|
|
mkdir -p "${CHART_STAGING}"
|
|
# Package into a temp dir and compare — only update if changed.
|
|
TMP_CHART=$(mktemp -d)
|
|
helm package "${HELM_CHART_DIR}" -d "${TMP_CHART}" >/dev/null 2>&1
|
|
for tgz in "${TMP_CHART}"/*.tgz; do
|
|
[ -f "$tgz" ] || continue
|
|
base=$(basename "$tgz")
|
|
if ! cmp -s "$tgz" "${CHART_STAGING}/${base}" 2>/dev/null; then
|
|
cp "$tgz" "${CHART_STAGING}/${base}"
|
|
echo " updated: /opt/openshell/charts/${base}"
|
|
fi
|
|
done
|
|
rm -rf "${TMP_CHART}"
|
|
fi
|
|
fi
|
|
|
|
# ── Kubernetes manifests ───────────────────────────────────────────────
|
|
MANIFEST_SRC="${ROOT}/deploy/kube/manifests"
|
|
MANIFEST_DST="${ROOTFS_DIR}/opt/openshell/manifests"
|
|
if [ -d "${MANIFEST_SRC}" ]; then
|
|
mkdir -p "${MANIFEST_DST}"
|
|
for manifest in "${MANIFEST_SRC}"/*.yaml; do
|
|
[ -f "$manifest" ] || continue
|
|
base=$(basename "$manifest")
|
|
if ! cmp -s "$manifest" "${MANIFEST_DST}/${base}" 2>/dev/null; then
|
|
cp "$manifest" "${MANIFEST_DST}/${base}"
|
|
echo " updated: /opt/openshell/manifests/${base}"
|
|
fi
|
|
done
|
|
fi
|
|
|
|
patch_vm_helmchart "${MANIFEST_DST}/openshell-helmchart.yaml"
|
|
patch_vm_helmchart "${ROOTFS_DIR}/var/lib/rancher/k3s/server/manifests/openshell-helmchart.yaml"
|
|
|
|
# ── Gateway image tarball ──────────────────────────────────────────────
|
|
# The VM rootfs airgap-imports openshell/gateway:dev from k3s/agent/images/.
|
|
# Keep that tarball in sync with the local Docker image so `mise run e2e:vm`
|
|
# validates the current openshell-server code, not whatever image happened to
|
|
# be baked into the rootfs last time it was rebuilt.
|
|
SERVER_IMAGE_TAR="${ROOTFS_DIR}/var/lib/rancher/k3s/agent/images/openshell-server.tar.zst"
|
|
SERVER_IMAGE_ID_FILE="${ROOTFS_DIR}/opt/openshell/.gateway-image-id"
|
|
if ce image inspect "${SERVER_IMAGE}" >/dev/null 2>&1; then
|
|
current_image_id=$(ce image inspect --format '{{.Id}}' "${SERVER_IMAGE}")
|
|
previous_image_id=""
|
|
if [ -f "${SERVER_IMAGE_ID_FILE}" ]; then
|
|
previous_image_id=$(cat "${SERVER_IMAGE_ID_FILE}")
|
|
fi
|
|
|
|
if [ "${current_image_id}" != "${previous_image_id}" ] || [ ! -f "${SERVER_IMAGE_TAR}" ]; then
|
|
mkdir -p "$(dirname "${SERVER_IMAGE_TAR}")" "$(dirname "${SERVER_IMAGE_ID_FILE}")"
|
|
tmp_tar=$(mktemp /tmp/openshell-server-image.XXXXXX)
|
|
ce save "${SERVER_IMAGE}" | zstd -f -T0 -3 -o "${tmp_tar}" >/dev/null
|
|
mv "${tmp_tar}" "${SERVER_IMAGE_TAR}"
|
|
printf '%s\n' "${current_image_id}" > "${SERVER_IMAGE_ID_FILE}"
|
|
echo " updated: /var/lib/rancher/k3s/agent/images/openshell-server.tar.zst"
|
|
fi
|
|
fi
|
|
|
|
# ── Supervisor binary ─────────────────────────────────────────────────
|
|
SUPERVISOR_TARGET="aarch64-unknown-linux-gnu"
|
|
SUPERVISOR_BIN="${ROOT}/target/${SUPERVISOR_TARGET}/release/openshell-sandbox"
|
|
SUPERVISOR_DST="${ROOTFS_DIR}/opt/openshell/bin/openshell-sandbox"
|
|
if [ -f "${SUPERVISOR_BIN}" ]; then
|
|
mkdir -p "$(dirname "${SUPERVISOR_DST}")"
|
|
if ! cmp -s "${SUPERVISOR_BIN}" "${SUPERVISOR_DST}" 2>/dev/null; then
|
|
cp "${SUPERVISOR_BIN}" "${SUPERVISOR_DST}"
|
|
chmod +x "${SUPERVISOR_DST}"
|
|
echo " updated: /opt/openshell/bin/openshell-sandbox"
|
|
fi
|
|
fi
|
|
|
|
# ── Fix execute permissions on k3s data binaries ──────────────────────
|
|
# docker export and macOS virtio-fs can strip execute bits.
|
|
chmod +x "${ROOTFS_DIR}"/var/lib/rancher/k3s/data/*/bin/* 2>/dev/null || true
|
|
chmod +x "${ROOTFS_DIR}"/var/lib/rancher/k3s/data/*/bin/aux/* 2>/dev/null || true
|
|
|
|
echo "Sync complete."
|