Files
OpenShell/scripts/remote-deploy.sh
T
Adam Miller d44d8a1e27 feat: Openshell driver podman (#904)
* feat(podman): add Podman compute driver for rootless sandbox management

Adds openshell-driver-podman, a new compute driver that manages OpenShell
sandboxes as rootless Podman containers via the Podman REST API over a
Unix socket. Enables local workstation sandboxes without Kubernetes.

Driver features:
- Bridge networking with ephemeral host-port mapping for rootless SSH reachability
- Named volumes for workspace storage, Podman native health checks, GPU via CDI
- Supervisor binary sideloaded via image volume mount (BYOC-compatible)
- SSH handshake secret injected via Podman secrets API (not plaintext env)
- Typed ContainerSpec structs, input validation, and path-traversal guards
- Cgroups v2 required; fails fast on v1 hosts
- Bounded event stream buffer; watch stream reconnection handled by server watch_loop
- Graceful shutdown and standalone driver binary with gRPC bridge

Rootless-specific fixes:
- Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps
- Add /run/netns tmpfs mount for ip netns in rootless containers
- Use secret_env map (not secrets array) for env-var injection in libpod API
- Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP

Server/sandbox hardening:
- Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback
- Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting
- Prevent OPENSHELL_* env var override by user-supplied spec environment maps
- Disable SQLite pool idle_timeout/max_lifetime for in-memory databases
- Emit deleted_event on 404-during-inspect instead of regressing sandbox phase
- Key delete cleanup by stable sandbox_id to survive container label drift

CLI fixes:
- Restore --name as a named flag on sandbox create (not positional)
- Fix exec command arg parsing to not consume sandboxed-command flags
- Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL

Build tooling:
- Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes
  unified ce_* helpers; all build/cluster/VM scripts updated to use it
- Add docker:build:supervisor mise task for standalone supervisor image
- Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages
- Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): derive grpc endpoint from server bind port

When a user starts the gateway on a non-default port (e.g. --port 8081),
sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the
default port 8080. The driver's auto-detection fallback read
OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset,
and fell back to DEFAULT_SERVER_PORT.

Add gateway_port to PodmanComputeConfig and thread config.bind_address.port()
from the server into the driver so the fallback uses the actual listening
port. Remove the OPENSHELL_BIND_ADDRESS env var read and the
extract_port_from_bind_address helper which are no longer needed.

Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver
binary for parity when the driver is run outside the embedded server path.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): address PR feedback on env test safety and cluster DNS docs

Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars
and a static ENV_LOCK mutex, fixing a data race in parallel test execution.
The prior safety comment incorrectly claimed Cargo runs tests single-threaded.

Update debug-openshell-cluster skill to accurately document the DNS proxy
strategy (setup_dns_proxy + public DNS fallback) and clarify the separation
between cluster DNS and sandbox agent DNS enforcement.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(e2e): resolve CI failures in auth timeout, test harness, and formatting

- Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead
  of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives
- Add timeout to SandboxGuard::create() and create_with_upload() to
  prevent indefinite hangs (matches create_keep() which already had one)
- Add missing '--' separator in no_proxy test before command args
- Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs
- Run cargo fmt on openshell-driver-podman
- Refine cluster DNS docs for Podman in debug-openshell-cluster skill

Signed-off-by: Adam Miller <admiller@redhat.com>

* refactor(server): remove allows_loopback_endpoints from ComputeRuntime

SSRF protection is now handled at the network and proxy layers
(openshell-core net.rs, openshell-sandbox proxy.rs) rather than
requiring per-driver flags on ComputeRuntime. Update architecture
docs to reflect supervisor relay SSH transport and add rootless
networking deep-dive.

Signed-off-by: Adam Miller <admiller@redhat.com>

---------

Signed-off-by: Adam Miller <admiller@redhat.com>
2026-04-24 10:30:14 -07:00

287 lines
7.5 KiB
Bash
Executable File

#!/usr/bin/env bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Deploy the current checkout to a remote machine for gateway development/testing.
#
# The script syncs the local source tree to a remote host, bootstraps the toolchain
# there, builds the CLI and Docker images from the synced checkout, then starts or
# updates a gateway using `openshell gateway start`.
set -euo pipefail
usage() {
cat <<'EOF'
Usage:
./scripts/remote-deploy.sh <user@host> [options]
Options:
--remote-dir DIR Remote checkout directory (default: openshell)
--name NAME Cluster name (default: openshell)
--port PORT Gateway port (default: 8080)
--ssh-key PATH SSH private key for ssh/rsync
--skip-sync Skip rsync and use the existing remote checkout
--recreate Destroy and recreate the gateway from scratch
--plaintext Listen on plaintext HTTP instead of mTLS
--disable-gateway-auth Keep TLS but disable client certificate enforcement
--image-tag TAG Docker image tag to build/deploy (default: dev)
--cargo-version VERSION Override OPENSHELL_CARGO_VERSION for remote Docker builds
--help Show this help
Examples:
./scripts/remote-deploy.sh ubuntu@devbox
./scripts/remote-deploy.sh ubuntu@devbox --recreate --port 18080
./scripts/remote-deploy.sh ubuntu@devbox --plaintext --ssh-key ~/.ssh/devbox
./scripts/remote-deploy.sh my-sandbox -./scripts/remote-deploy.sh my-sandbox --remote-dir --name openshell --port 8080 --recreate --plaintext
EOF
}
info() { echo "==> $*"; }
err() { echo "ERROR: $*" >&2; }
require_value() {
local flag="$1"
local value="${2-}"
if [[ -z "${value}" ]]; then
err "${flag} requires a value"
exit 1
fi
}
REMOTE_HOST=""
REMOTE_DIR=${REMOTE_DIR:-openshell}
CLUSTER_NAME=${CLUSTER_NAME:-openshell}
GATEWAY_PORT=${GATEWAY_PORT:-8080}
SSH_KEY="${SSH_KEY:-}"
IMAGE_TAG=${IMAGE_TAG:-dev}
CARGO_VERSION=${OPENSHELL_CARGO_VERSION:-0.0.0-dev}
SKIP_SYNC=false
RECREATE=false
PLAINTEXT=false
DISABLE_GATEWAY_AUTH=false
while [[ $# -gt 0 ]]; do
case "$1" in
--remote-dir)
require_value "$1" "${2-}"
REMOTE_DIR="$2"
shift 2
;;
--name)
require_value "$1" "${2-}"
CLUSTER_NAME="$2"
shift 2
;;
--port)
require_value "$1" "${2-}"
GATEWAY_PORT="$2"
shift 2
;;
--ssh-key)
require_value "$1" "${2-}"
SSH_KEY="$2"
shift 2
;;
--skip-sync)
SKIP_SYNC=true
shift
;;
--recreate)
RECREATE=true
shift
;;
--plaintext)
PLAINTEXT=true
shift
;;
--disable-gateway-auth)
DISABLE_GATEWAY_AUTH=true
shift
;;
--image-tag)
require_value "$1" "${2-}"
IMAGE_TAG="$2"
shift 2
;;
--cargo-version)
require_value "$1" "${2-}"
CARGO_VERSION="$2"
shift 2
;;
--help|-h)
usage
exit 0
;;
--*)
err "Unknown argument: $1"
usage >&2
exit 1
;;
*)
if [[ -n "${REMOTE_HOST}" ]]; then
err "Multiple remote hosts provided: ${REMOTE_HOST} and $1"
usage >&2
exit 1
fi
REMOTE_HOST="$1"
shift
;;
esac
done
if [[ -z "${REMOTE_HOST}" ]]; then
err "Remote host is required"
usage >&2
exit 1
fi
if [[ "${PLAINTEXT}" == "true" && "${DISABLE_GATEWAY_AUTH}" == "true" ]]; then
err "--disable-gateway-auth is ignored when --plaintext is set; choose one mode"
exit 1
fi
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
SSH_ARGS=()
if [[ -n "${SSH_KEY}" ]]; then
SSH_ARGS=(-i "${SSH_KEY}")
fi
if [[ "${SKIP_SYNC}" != "true" ]]; then
info "Preparing ${REMOTE_HOST}:${REMOTE_DIR}"
ssh "${SSH_ARGS[@]}" "${REMOTE_HOST}" "mkdir -p '${REMOTE_DIR}'"
info "Syncing source to ${REMOTE_HOST}:${REMOTE_DIR}"
RSYNC_SSH=(ssh)
if [[ -n "${SSH_KEY}" ]]; then
RSYNC_SSH+=(-i "${SSH_KEY}")
fi
rsync -az --delete \
-e "${RSYNC_SSH[*]}" \
--exclude 'target/' \
--exclude '.git/' \
--exclude '.cache/' \
--exclude 'node_modules/' \
--exclude '*.pyc' \
--exclude '__pycache__/' \
--exclude '.venv/' \
--exclude 'e2e/' \
--exclude 'deploy/docker/.build/' \
"${REPO_ROOT}/" "${REMOTE_HOST}:${REMOTE_DIR}/"
info "Sync complete"
fi
SECURITY_MODE="mTLS enabled"
if [[ "${PLAINTEXT}" == "true" ]]; then
SECURITY_MODE="plaintext HTTP"
elif [[ "${DISABLE_GATEWAY_AUTH}" == "true" ]]; then
SECURITY_MODE="TLS enabled, client cert auth disabled"
fi
info "Deploying gateway on ${REMOTE_HOST} (port=${GATEWAY_PORT}, security=${SECURITY_MODE})"
ssh -t "${SSH_ARGS[@]}" "${REMOTE_HOST}" \
bash -s -- \
"${REMOTE_DIR}" \
"${CLUSTER_NAME}" \
"${GATEWAY_PORT}" \
"${IMAGE_TAG}" \
"${CARGO_VERSION}" \
"${RECREATE}" \
"${PLAINTEXT}" \
"${DISABLE_GATEWAY_AUTH}" <<'REMOTE_EOF'
set -euo pipefail
REMOTE_DIR="$1"
CLUSTER_NAME="$2"
GATEWAY_PORT="$3"
IMAGE_TAG="$4"
CARGO_VERSION="$5"
RECREATE="$6"
PLAINTEXT="$7"
DISABLE_GATEWAY_AUTH="$8"
cd "${REMOTE_DIR}"
if ! command -v mise >/dev/null 2>&1; then
echo "==> Installing mise..."
curl https://mise.run | sh
fi
export PATH="$HOME/.local/bin:$PATH"
echo "==> Installing tools via mise..."
mise trust --yes
mise install --yes
if ! command -v podman >/dev/null 2>&1 && ! command -v docker >/dev/null 2>&1; then
echo "ERROR: Neither podman nor docker is installed on the remote host." >&2
exit 1
fi
echo "==> Building openshell CLI..."
mise exec -- cargo build --release -p openshell-cli
mkdir -p "$HOME/.local/bin"
install -m 0755 target/release/openshell "$HOME/.local/bin/openshell"
# Ensure `mise exec -- openshell` uses the release binary rather than the local
# development shim, which expects git metadata that is not synced to the VM.
install -m 0755 target/release/openshell scripts/bin/openshell
# Prevent a stale repo-local .env from changing the deployment unexpectedly.
rm -f .env
echo "==> Building container images (tag=${IMAGE_TAG})..."
export OPENSHELL_CARGO_VERSION="${CARGO_VERSION}"
export IMAGE_TAG
mise exec -- tasks/scripts/docker-build-image.sh cluster
mise exec -- tasks/scripts/docker-build-image.sh gateway
export OPENSHELL_CLUSTER_IMAGE="openshell/cluster:${IMAGE_TAG}"
export OPENSHELL_PUSH_IMAGES="openshell/gateway:${IMAGE_TAG}"
start_args=(
gateway
start
--name "${CLUSTER_NAME}"
--port "${GATEWAY_PORT}"
)
if [[ "${RECREATE}" == "true" ]]; then
start_args+=(--recreate)
fi
if [[ "${PLAINTEXT}" == "true" ]]; then
start_args+=(--plaintext)
fi
if [[ "${DISABLE_GATEWAY_AUTH}" == "true" ]]; then
start_args+=(--disable-gateway-auth)
fi
echo "==> Starting gateway..."
mise exec -- openshell "${start_args[@]}"
echo ""
echo "============================================"
echo " Gateway deployed successfully"
echo " Cluster: ${CLUSTER_NAME}"
echo " Gateway port: ${GATEWAY_PORT}"
if [[ "${PLAINTEXT}" == "true" ]]; then
echo " Security: plaintext HTTP"
elif [[ "${DISABLE_GATEWAY_AUTH}" == "true" ]]; then
echo " Security: TLS enabled, client cert auth disabled"
else
echo " Security: mTLS enabled"
fi
echo "============================================"
REMOTE_EOF
PROTO="https"
if [[ "${PLAINTEXT}" == "true" ]]; then
PROTO="http"
fi
info "Done. Gateway is running on ${REMOTE_HOST}:${GATEWAY_PORT}"
info "Health check:"
info " curl ${PROTO}://${REMOTE_HOST}:${GATEWAY_PORT}/health"