Files
OpenShell/tasks/scripts/vm/sync-vm-rootfs.sh
Adam Miller d44d8a1e27 feat: Openshell driver podman (#904)
* feat(podman): add Podman compute driver for rootless sandbox management

Adds openshell-driver-podman, a new compute driver that manages OpenShell
sandboxes as rootless Podman containers via the Podman REST API over a
Unix socket. Enables local workstation sandboxes without Kubernetes.

Driver features:
- Bridge networking with ephemeral host-port mapping for rootless SSH reachability
- Named volumes for workspace storage, Podman native health checks, GPU via CDI
- Supervisor binary sideloaded via image volume mount (BYOC-compatible)
- SSH handshake secret injected via Podman secrets API (not plaintext env)
- Typed ContainerSpec structs, input validation, and path-traversal guards
- Cgroups v2 required; fails fast on v1 hosts
- Bounded event stream buffer; watch stream reconnection handled by server watch_loop
- Graceful shutdown and standalone driver binary with gRPC bridge

Rootless-specific fixes:
- Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps
- Add /run/netns tmpfs mount for ip netns in rootless containers
- Use secret_env map (not secrets array) for env-var injection in libpod API
- Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP

Server/sandbox hardening:
- Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback
- Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting
- Prevent OPENSHELL_* env var override by user-supplied spec environment maps
- Disable SQLite pool idle_timeout/max_lifetime for in-memory databases
- Emit deleted_event on 404-during-inspect instead of regressing sandbox phase
- Key delete cleanup by stable sandbox_id to survive container label drift

CLI fixes:
- Restore --name as a named flag on sandbox create (not positional)
- Fix exec command arg parsing to not consume sandboxed-command flags
- Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL

Build tooling:
- Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes
  unified ce_* helpers; all build/cluster/VM scripts updated to use it
- Add docker:build:supervisor mise task for standalone supervisor image
- Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages
- Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): derive grpc endpoint from server bind port

When a user starts the gateway on a non-default port (e.g. --port 8081),
sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the
default port 8080. The driver's auto-detection fallback read
OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset,
and fell back to DEFAULT_SERVER_PORT.

Add gateway_port to PodmanComputeConfig and thread config.bind_address.port()
from the server into the driver so the fallback uses the actual listening
port. Remove the OPENSHELL_BIND_ADDRESS env var read and the
extract_port_from_bind_address helper which are no longer needed.

Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver
binary for parity when the driver is run outside the embedded server path.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): address PR feedback on env test safety and cluster DNS docs

Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars
and a static ENV_LOCK mutex, fixing a data race in parallel test execution.
The prior safety comment incorrectly claimed Cargo runs tests single-threaded.

Update debug-openshell-cluster skill to accurately document the DNS proxy
strategy (setup_dns_proxy + public DNS fallback) and clarify the separation
between cluster DNS and sandbox agent DNS enforcement.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(e2e): resolve CI failures in auth timeout, test harness, and formatting

- Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead
  of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives
- Add timeout to SandboxGuard::create() and create_with_upload() to
  prevent indefinite hangs (matches create_keep() which already had one)
- Add missing '--' separator in no_proxy test before command args
- Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs
- Run cargo fmt on openshell-driver-podman
- Refine cluster DNS docs for Podman in debug-openshell-cluster skill

Signed-off-by: Adam Miller <admiller@redhat.com>

* refactor(server): remove allows_loopback_endpoints from ComputeRuntime

SSRF protection is now handled at the network and proxy layers
(openshell-core net.rs, openshell-sandbox proxy.rs) rather than
requiring per-driver flags on ComputeRuntime. Update architecture
docs to reflect supervisor relay SSH transport and add rootless
networking deep-dive.

Signed-off-by: Adam Miller <admiller@redhat.com>

---------

Signed-off-by: Adam Miller <admiller@redhat.com>
2026-04-24 10:30:14 -07:00

189 lines
7.7 KiB
Bash
Executable File

#!/usr/bin/env bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Sync mutable development artifacts into the existing VM rootfs.
# Runs on every `mise run vm` so that script changes, helm chart
# updates, manifest changes, and supervisor binary rebuilds are
# picked up without a full rootfs rebuild.
#
# This is fast (<1s) — it only copies files, no Docker or VM boot.
set -euo pipefail
source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/container-engine.sh"
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
SCRIPT_DIR="${ROOT}/crates/openshell-vm/scripts"
IMAGE_REPO_BASE="${IMAGE_REPO_BASE:-openshell}"
IMAGE_TAG="${IMAGE_TAG:-dev}"
SERVER_IMAGE="${IMAGE_REPO_BASE}/gateway:${IMAGE_TAG}"
NAME="default"
ROOTFS_ARGS=()
while [[ $# -gt 0 ]]; do
case "$1" in
--name)
NAME="$2"
shift 2
;;
--name=*)
NAME="${1#--name=}"
shift
;;
--rootfs)
ROOTFS_ARGS=("$1" "$2")
shift 2
;;
--rootfs=*)
ROOTFS_ARGS=("$1")
shift
;;
*)
echo "Unknown argument: $1" >&2
exit 1
;;
esac
done
ensure_args=(--name "${NAME}")
if [ "${#ROOTFS_ARGS[@]}" -gt 0 ]; then
ensure_args=("${ROOTFS_ARGS[@]}" "${ensure_args[@]}")
fi
if ! ROOTFS_DIR="$("${ROOT}/tasks/scripts/vm/ensure-vm-rootfs.sh" "${ensure_args[@]}" | tail -n 1 | sed 's/^using openshell-vm rootfs at //')"; then
echo "ERROR: ensure-vm-rootfs.sh failed — no rootfs available." >&2
exit 1
fi
patch_vm_helmchart() {
local helmchart="$1"
[ -f "${helmchart}" ] || return 0
sed_in_place() {
local expr="$1"
sed -i.bak -E "${expr}" "${helmchart}"
rm -f "${helmchart}.bak"
}
# Mirror the build-rootfs patching so the VM keeps using the locally
# imported openshell/gateway:dev image after incremental rootfs syncs.
sed_in_place 's|__IMAGE_PULL_POLICY__|IfNotPresent|g'
sed_in_place 's|__SANDBOX_IMAGE_PULL_POLICY__|"IfNotPresent"|g'
sed_in_place 's|__DB_URL__|"sqlite:/tmp/openshell.db"|g'
sed_in_place "s|repository:[[:space:]]*[^[:space:]]+|repository: ${SERVER_IMAGE%:*}|"
sed_in_place "s|tag:[[:space:]]*\"?[^\"[:space:]]+\"?|tag: \"${IMAGE_TAG}\"|"
sed_in_place 's|sshGatewayHost: __SSH_GATEWAY_HOST__|sshGatewayHost: ""|g'
sed_in_place 's|sshGatewayPort: __SSH_GATEWAY_PORT__|sshGatewayPort: 0|g'
sed_in_place 's|__DISABLE_GATEWAY_AUTH__|false|g'
sed_in_place 's|__DISABLE_TLS__|false|g'
sed_in_place 's|hostGatewayIP: __HOST_GATEWAY_IP__|hostGatewayIP: ""|g'
sed_in_place '/__CHART_CHECKSUM__/d'
}
if [ ! -d "${ROOTFS_DIR}/srv" ]; then
# Rootfs doesn't exist yet — nothing to sync. ensure-vm-rootfs.sh
# or build-rootfs.sh will create it.
exit 0
fi
echo "Syncing development artifacts into rootfs..."
# ── Init scripts and utilities ─────────────────────────────────────────
for script in openshell-vm-init.sh openshell-vm-exec-agent.py check-vm-capabilities.sh; do
src="${SCRIPT_DIR}/${script}"
dst="${ROOTFS_DIR}/srv/${script}"
if [ -f "$src" ]; then
if ! cmp -s "$src" "$dst" 2>/dev/null; then
cp "$src" "$dst"
chmod +x "$dst"
echo " updated: /srv/${script}"
fi
fi
done
# ── Helm chart ─────────────────────────────────────────────────────────
HELM_CHART_DIR="${ROOT}/deploy/helm/openshell"
CHART_STAGING="${ROOTFS_DIR}/opt/openshell/charts"
if [ -d "${HELM_CHART_DIR}" ]; then
if ! command -v helm >/dev/null 2>&1; then
echo " warning: helm not found — skipping chart sync (run: mise install)" >&2
else
mkdir -p "${CHART_STAGING}"
# Package into a temp dir and compare — only update if changed.
TMP_CHART=$(mktemp -d)
helm package "${HELM_CHART_DIR}" -d "${TMP_CHART}" >/dev/null 2>&1
for tgz in "${TMP_CHART}"/*.tgz; do
[ -f "$tgz" ] || continue
base=$(basename "$tgz")
if ! cmp -s "$tgz" "${CHART_STAGING}/${base}" 2>/dev/null; then
cp "$tgz" "${CHART_STAGING}/${base}"
echo " updated: /opt/openshell/charts/${base}"
fi
done
rm -rf "${TMP_CHART}"
fi
fi
# ── Kubernetes manifests ───────────────────────────────────────────────
MANIFEST_SRC="${ROOT}/deploy/kube/manifests"
MANIFEST_DST="${ROOTFS_DIR}/opt/openshell/manifests"
if [ -d "${MANIFEST_SRC}" ]; then
mkdir -p "${MANIFEST_DST}"
for manifest in "${MANIFEST_SRC}"/*.yaml; do
[ -f "$manifest" ] || continue
base=$(basename "$manifest")
if ! cmp -s "$manifest" "${MANIFEST_DST}/${base}" 2>/dev/null; then
cp "$manifest" "${MANIFEST_DST}/${base}"
echo " updated: /opt/openshell/manifests/${base}"
fi
done
fi
patch_vm_helmchart "${MANIFEST_DST}/openshell-helmchart.yaml"
patch_vm_helmchart "${ROOTFS_DIR}/var/lib/rancher/k3s/server/manifests/openshell-helmchart.yaml"
# ── Gateway image tarball ──────────────────────────────────────────────
# The VM rootfs airgap-imports openshell/gateway:dev from k3s/agent/images/.
# Keep that tarball in sync with the local Docker image so `mise run e2e:vm`
# validates the current openshell-server code, not whatever image happened to
# be baked into the rootfs last time it was rebuilt.
SERVER_IMAGE_TAR="${ROOTFS_DIR}/var/lib/rancher/k3s/agent/images/openshell-server.tar.zst"
SERVER_IMAGE_ID_FILE="${ROOTFS_DIR}/opt/openshell/.gateway-image-id"
if ce image inspect "${SERVER_IMAGE}" >/dev/null 2>&1; then
current_image_id=$(ce image inspect --format '{{.Id}}' "${SERVER_IMAGE}")
previous_image_id=""
if [ -f "${SERVER_IMAGE_ID_FILE}" ]; then
previous_image_id=$(cat "${SERVER_IMAGE_ID_FILE}")
fi
if [ "${current_image_id}" != "${previous_image_id}" ] || [ ! -f "${SERVER_IMAGE_TAR}" ]; then
mkdir -p "$(dirname "${SERVER_IMAGE_TAR}")" "$(dirname "${SERVER_IMAGE_ID_FILE}")"
tmp_tar=$(mktemp /tmp/openshell-server-image.XXXXXX)
ce save "${SERVER_IMAGE}" | zstd -f -T0 -3 -o "${tmp_tar}" >/dev/null
mv "${tmp_tar}" "${SERVER_IMAGE_TAR}"
printf '%s\n' "${current_image_id}" > "${SERVER_IMAGE_ID_FILE}"
echo " updated: /var/lib/rancher/k3s/agent/images/openshell-server.tar.zst"
fi
fi
# ── Supervisor binary ─────────────────────────────────────────────────
SUPERVISOR_TARGET="aarch64-unknown-linux-gnu"
SUPERVISOR_BIN="${ROOT}/target/${SUPERVISOR_TARGET}/release/openshell-sandbox"
SUPERVISOR_DST="${ROOTFS_DIR}/opt/openshell/bin/openshell-sandbox"
if [ -f "${SUPERVISOR_BIN}" ]; then
mkdir -p "$(dirname "${SUPERVISOR_DST}")"
if ! cmp -s "${SUPERVISOR_BIN}" "${SUPERVISOR_DST}" 2>/dev/null; then
cp "${SUPERVISOR_BIN}" "${SUPERVISOR_DST}"
chmod +x "${SUPERVISOR_DST}"
echo " updated: /opt/openshell/bin/openshell-sandbox"
fi
fi
# ── Fix execute permissions on k3s data binaries ──────────────────────
# docker export and macOS virtio-fs can strip execute bits.
chmod +x "${ROOTFS_DIR}"/var/lib/rancher/k3s/data/*/bin/* 2>/dev/null || true
chmod +x "${ROOTFS_DIR}"/var/lib/rancher/k3s/data/*/bin/aux/* 2>/dev/null || true
echo "Sync complete."