* ci(docker): use prebuilt Rust binaries by default
Flip Docker image builds to consume staged native Rust artifacts, remove in-Docker Rust build stages, and publish per-arch images with a manifest merge.
Add local staging support for prebuilt gateway and sandbox binaries so development image builds continue to work without CI artifacts.
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
* ci(docker): address prebuilt build review feedback
* ci(rust): allow existing vfio complexity
* ci(rust): pin toolchain to 1.95
---------
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
* Adding qemu vm driver support with GPU pass-through
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
* Add GPU rootfs variant, harden VFIO binding, and fix networking and supervisor reliability issues discovered during GPU VM bring-up.
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
---------
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
* fix(net): catch IPv4-mapped blocked ranges in is_always_blocked_net
The IPv6 branch only checked whether the network address itself mapped
to a blocked IPv4 address. A broader prefix like ::ffff:168.0.0.0/103
has a public network address but spans ::ffff:169.254.0.0, so the old
code accepted it at policy load time while is_always_blocked_ip silently
rejected every connection at runtime.
Add three containment checks for the IPv4-mapped loopback, link-local,
and unspecified representatives. The existing network-address check is
kept because it handles single-host entries (/128) whose network address
is already in a blocked range.
Five new tests cover: single-host loopback and link-local mapped
addresses, broad prefixes that span each blocked range without starting
there, and a public single-host address that must not be blocked.
* fix(net): address clippy warnings in is_always_blocked_net
Use Ipv4Addr::LOCALHOST instead of Ipv4Addr::new(127, 0, 0, 1) and
collapse the nested if let / if into is_some_and.
Builds openshell-gateway and openshell-sandbox natively per-arch on the
nv-gha-runners shared CPU pool with a GHA-backed sccache, and uploads
the resulting binaries as artifacts shaped for Dockerfile.images'
BINARY_SOURCE=prebuilt path (added in #945).
Dispatch manually 4-5 times after merge to collect cold + warm numbers
and compare against the Rust portion of docker-build.yml's ARC baseline.
PR 4c will wire these artifacts into the real pipeline once numbers land.
Refs OS-128.
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
The default socket path assumed Linux conventions ($XDG_RUNTIME_DIR or
/run/user/{uid}), which do not exist on macOS. Use the podman machine
socket at $HOME/.local/share/containers/podman/machine/podman.sock
on macOS and show a targeted "podman machine start" hint when missing.
Signed-off-by: Florent Benoit <fbenoit@redhat.com>
Change sandbox user UID from 1000 to 1000660000 in custom image examples and E2E tests.
Using a high UID range (1000000000+) prevents conflicts with host users when running
without user namespace remapping, where container UIDs map directly to host UIDs.
This resolves fork failures caused by RLIMIT_NPROC enforcement when the host user
already has many threads running.
Signed-off-by: Derek Carr <decarr@redhat.com>
The docker-cleanup.sh script failed when no containers were running because
grep -v returned exit code 1 on empty input, causing the script to abort
due to set -euo pipefail.
Add || true to the volume detection pipeline so the script succeeds when
there are no running containers (in_use_volumes will be empty, which is
the correct behavior).
Signed-off-by: Derek Carr <decarr@redhat.com>
#966 hard-coded `buildkitd-config: /etc/buildkit/buildkitd.toml` inside
the `driver: local` branch of the setup-buildx composite action. The only
caller using that driver is shadow-docker-build.yml, which runs inside
the ghcr.io/nvidia/openshell/ci:latest container — so the host-side
buildkitd.toml was invisible to docker/setup-buildx-action and every
matrix job failed at "Set up buildx".
Revert the hard-coded path and expose it as an opt-in input on the
action (empty default, passed through to both the remote and local
branches). Wire shadow-docker-build.yml to bind-mount /etc/buildkit
into the ci container and pass the path explicitly, so the action can
read the file from inside the container. Remote-driver callers are
unaffected (empty input is a no-op).
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
* feat(docker): add BINARY_SOURCE selector for prebuilt Rust binaries
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
* fix(docker): preserve exec bit on prebuilt binary COPY
Adds --chmod=755 to the COPY instructions in the scratch-based
prebuilt binary stages. Without this, binaries produced by PR 4a and
shuttled through actions/upload-artifact + download-artifact lose
their executable bit during the roundtrip, and the resulting image's
ENTRYPOINT fails at runtime.
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
---------
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
* feat(podman): add Podman compute driver for rootless sandbox management
Adds openshell-driver-podman, a new compute driver that manages OpenShell
sandboxes as rootless Podman containers via the Podman REST API over a
Unix socket. Enables local workstation sandboxes without Kubernetes.
Driver features:
- Bridge networking with ephemeral host-port mapping for rootless SSH reachability
- Named volumes for workspace storage, Podman native health checks, GPU via CDI
- Supervisor binary sideloaded via image volume mount (BYOC-compatible)
- SSH handshake secret injected via Podman secrets API (not plaintext env)
- Typed ContainerSpec structs, input validation, and path-traversal guards
- Cgroups v2 required; fails fast on v1 hosts
- Bounded event stream buffer; watch stream reconnection handled by server watch_loop
- Graceful shutdown and standalone driver binary with gRPC bridge
Rootless-specific fixes:
- Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps
- Add /run/netns tmpfs mount for ip netns in rootless containers
- Use secret_env map (not secrets array) for env-var injection in libpod API
- Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP
Server/sandbox hardening:
- Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback
- Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting
- Prevent OPENSHELL_* env var override by user-supplied spec environment maps
- Disable SQLite pool idle_timeout/max_lifetime for in-memory databases
- Emit deleted_event on 404-during-inspect instead of regressing sandbox phase
- Key delete cleanup by stable sandbox_id to survive container label drift
CLI fixes:
- Restore --name as a named flag on sandbox create (not positional)
- Fix exec command arg parsing to not consume sandboxed-command flags
- Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL
Build tooling:
- Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes
unified ce_* helpers; all build/cluster/VM scripts updated to use it
- Add docker:build:supervisor mise task for standalone supervisor image
- Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages
- Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(driver-podman): derive grpc endpoint from server bind port
When a user starts the gateway on a non-default port (e.g. --port 8081),
sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the
default port 8080. The driver's auto-detection fallback read
OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset,
and fell back to DEFAULT_SERVER_PORT.
Add gateway_port to PodmanComputeConfig and thread config.bind_address.port()
from the server into the driver so the fallback uses the actual listening
port. Remove the OPENSHELL_BIND_ADDRESS env var read and the
extract_port_from_bind_address helper which are no longer needed.
Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver
binary for parity when the driver is run outside the embedded server path.
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(driver-podman): address PR feedback on env test safety and cluster DNS docs
Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars
and a static ENV_LOCK mutex, fixing a data race in parallel test execution.
The prior safety comment incorrectly claimed Cargo runs tests single-threaded.
Update debug-openshell-cluster skill to accurately document the DNS proxy
strategy (setup_dns_proxy + public DNS fallback) and clarify the separation
between cluster DNS and sandbox agent DNS enforcement.
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(e2e): resolve CI failures in auth timeout, test harness, and formatting
- Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead
of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives
- Add timeout to SandboxGuard::create() and create_with_upload() to
prevent indefinite hangs (matches create_keep() which already had one)
- Add missing '--' separator in no_proxy test before command args
- Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs
- Run cargo fmt on openshell-driver-podman
- Refine cluster DNS docs for Podman in debug-openshell-cluster skill
Signed-off-by: Adam Miller <admiller@redhat.com>
* refactor(server): remove allows_loopback_endpoints from ComputeRuntime
SSRF protection is now handled at the network and proxy layers
(openshell-core net.rs, openshell-sandbox proxy.rs) rather than
requiring per-driver flags on ComputeRuntime. Update architecture
docs to reflect supervisor relay SSH transport and add rootless
networking deep-dive.
Signed-off-by: Adam Miller <admiller@redhat.com>
---------
Signed-off-by: Adam Miller <admiller@redhat.com>
Without a per-arch SCCACHE_GHA_VERSION, both matrix jobs
(linux-amd64-cpu8 and linux-arm64-cpu8) target the same GHA cache
key. Run 1 on 2026-04-24 showed this concretely: arm64 started
first and landed 575/1064 cache writes; amd64 started ~1 min later
and every one of its 1062 writes failed, leaving the cache with no
amd64 entries at all.
Partitioning by matrix.runner ensures each arch gets its own
cache namespace, letting us see real warm-cache behavior on the
subsequent Run 2-4 dispatches.
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
Python runtime requires /dev/urandom access during initialization
to seed the hash randomizer. The _default_policy() in provider tests
was missing this path, causing exec_python tests to fail with:
'Fatal Python error: _Py_HashRandomization_Init: failed to get
random numbers to initialize Python'.
Add /dev/urandom to read_only paths to match the policy used in
test_sandbox_policy.py, allowing Python to initialize successfully.
Signed-off-by: Derek Carr <decarr@redhat.com>
Previously `openshell sandbox upload <local-dir> <remote-dir>` flattened
the local directory's contents into the remote destination, dropping the
source directory's basename. This diverged from the standard `scp -r` /
`cp -r` semantics that operators expect from any *nix-style transfer
tool, and could silently shadow files at the destination.
Now an upload of `foo` to `/sandbox/dest/` lands at `/sandbox/dest/foo/`,
matching `scp -r foo remote:/sandbox/dest/` and `cp -r foo /sandbox/dest/`.
Paths without a meaningful basename (`.`, `/`) preserve the legacy flat
extraction.
Closes#885
* feat(server): add Prometheus metrics infrastructure and gRPC/HTTP request metrics
Add metrics exposition via a dedicated server port (--metrics-port,
default disabled) following the same optional-listener pattern as the
health port. The metrics crate facade records counters and histograms
in the MultiplexedService hot path, and a PrometheusHandle renders them
at GET /metrics on the dedicated port.
Metrics added:
- openshell_server_grpc_requests_total (counter: method, code)
- openshell_server_grpc_request_duration_seconds (histogram: method, code)
- openshell_server_http_requests_total (counter: path, status)
- openshell_server_http_request_duration_seconds (histogram: path, status)
* feat(helm): add metrics port to openshell-server helm chart
Expose the Prometheus metrics endpoint in the helm chart by adding
service.metricsPort (default 9090) to values, the --metrics-port arg
and container port to the statefulset, and a metrics service port.
Set metricsPort to 0 to disable.
Alexander Watson