- Bump nvidia-container-toolkit from 1.18.2 to 1.19.0 to support the
-host-cuda-version flag used by newer CDI spec generation.
- Replace local filesystem check for host-files-for-container.d with
Docker API kernel version detection (contains "tegra"). This fixes
remote SSH deploys where the CLI machine may not have the directory.
- Only perform the Tegra check when GPU devices are requested.
* feat(sandbox): switch device plugin to CDI injection mode
Configure the NVIDIA device plugin to use deviceListStrategy=cdi-cri so
that GPU devices are injected via direct CDI device requests in the CRI.
Sandbox pods now only require the nvidia.com/gpu resource request —
runtimeClassName is no longer set on GPU pods.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Three issues prevent k3s from starting on kernels where the nf_tables
xt extension bridge (nft_compat) is unavailable:
1. kube-router's network policy controller uses the xt_comment iptables
extension and panics on startup with "Extension comment revision 0
not supported, missing kernel module?" Pass --disable-network-policy
to k3s so the controller never runs. The NSSH1 HMAC handshake remains
the primary sandbox SSH isolation boundary, so this does not weaken
the effective security posture.
2. flannel and kube-proxy also fail to insert rules via the nf_tables
iptables backend on the same kernels. Add an xt_comment probe at
cluster-entrypoint startup; if the probe fails, switch to
iptables-legacy via update-alternatives before any other netfilter
work so that flannel, kube-proxy, and the DNS proxy all use a
consistent backend.
3. The br_netfilter kernel module must be loaded on the host for
iptables rules to apply to pod bridge traffic. Without it, ClusterIP
DNAT (including kube-dns at 10.43.0.10) is never applied to pod
packets, causing silent DNS timeouts deep in the health-check loop.
Add an early check that fails fast with an actionable error message
if the module is not present, instructing the user to run
`sudo modprobe br_netfilter` on the host.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Docker COPY preserves build-host file permissions. On hosts with a
restrictive umask (e.g. 0027), the migrations directory is copied as
750 root:root, making it unreadable by the openshell user at runtime
and causing the server to crash with a permission denied error on
startup.
Using --chmod=755 ensures the directory is always world-readable
regardless of the build host umask.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
The OPENSHELL_CARGO_VERSION build arg contains a git commit hash that
changes on every build (e.g. 0.0.7-dev.11+g085b131ae). Declaring this
ARG near the top of each Dockerfile invalidated every layer below it --
including expensive dependency installs, toolchain setup, and the Rust
dependency pre-build step -- on every single commit.
Move the ARG declaration to just before the RUN that actually uses it so
upstream layers stay cached. This recovers ~5-6 minutes per build on the
gateway image (from ~9m back toward ~2-3m) and similarly improves cluster,
CLI, and Python wheel builds.
Also removes unused OPENSHELL_IMAGE_TAG ARG from cli-macos, python-wheels,
and python-wheels-macos Dockerfiles.
Kubernetes 1.35 (k3s v1.35.x) rejects cgroup v1 by default via the
failCgroupV1 kubelet config option. This breaks gateway startup on
distros that still use cgroup v1 (e.g. Rocky Linux 8, CentOS 7/8).
Detect cgroup v1 at container startup by checking for the absence of
/sys/fs/cgroup/cgroup.controllers and pass
--kubelet-arg=fail-cgroupv1=false so kubelet warns instead of exiting.
* feat(sandbox): inject host gateway hostAliases into sandbox pods
Sandbox pods running in the k3s cluster cannot resolve host.docker.internal
by default, preventing them from reaching services on the Docker host.
Detect the host gateway IP (default route) in the cluster entrypoint,
thread it through the Helm chart to the gateway server, and inject
hostAliases entries (host.docker.internal, host.openshell.internal)
into every sandbox pod spec. The injection is conditional -- when the
IP is empty (non-Docker deployments), no hostAliases are added.
---------
Co-authored-by: Piotr Mlocek <pmlocek@nvidia.com>
Compute version strings from git describe in openshell-core's build.rs
using the guess-next-dev scheme (e.g. 0.0.4-dev.6+g2bf9969). All binary
crates and the TUI splash screen now use the shared openshell_core::VERSION
constant instead of CARGO_PKG_VERSION.
In Docker/CI builds where .git is absent, falls back to CARGO_PKG_VERSION
which is already set correctly by the sed-patch pipeline. Also adds
OPENSHELL_CARGO_VERSION support to the cluster image and fast-deploy
supervisor builds for parity with the gateway.
cluster-deploy-fast.sh ran helm and kubectl directly on the host,
which requires a local kubeconfig pointing at the k3s API server.
Since the cluster container does not expose port 6443, these commands
always fail with 'connection refused'.
Install helm in the cluster container image and run all helm/kubectl
commands via docker exec with KUBECONFIG pre-configured. The local
chart directory is copied into the container before helm upgrade.
The k9s COPY instruction was added without a corresponding build stage,
causing Docker to look for a nonexistent docker.io/library/k9s image.
Add a multi-arch stage that downloads the k9s binary from GitHub releases.
Add --gpu flag to 'openshell gateway start' that enables NVIDIA GPU
access inside the cluster container. When enabled, the Docker container
receives GPU device requests via bollard, k3s auto-detects the NVIDIA
runtime, and the NVIDIA k8s-device-plugin is deployed via HelmChart CR
so Kubernetes workloads can request nvidia.com/gpu resources.
- Add GPU device request to Docker container creation (bollard DeviceRequest)
- Set GPU_ENABLED=true env var to gate GPU manifest deployment
- Multi-stage Dockerfile build to bundle NVIDIA Container Toolkit binaries
- Conditional GPU manifest copy in cluster entrypoint
- HelmChart CR for nvidia-device-plugin v0.18.2 with GFD and NFD enabled
- Thread gpu bool through DeployOptions, CLI flag, and run dispatch
---------
Co-authored-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(docker): rename server image to gateway
Rename Dockerfile.server to Dockerfile.gateway and update all image
references from openshell/server to openshell/gateway across Helm
charts, Kubernetes manifests, mise tasks, build/deploy scripts, CI
workflows, and documentation.
The underlying Rust binary (navigator-server) is unchanged -- this
rename only affects the Docker image name and Dockerfile.
* fix: catch remaining server->gateway references in docs and comments
Migrate all production container runtime stages from Debian/Ubuntu
upstream to NVIDIA's hardened Ubuntu base image for supply chain
consistency.
- Dockerfile.ci: ubuntu:24.04 -> nvidia base
- Dockerfile.server: debian:bookworm-slim -> nvidia base (runtime stage)
- Dockerfile.base (sandbox): python:3.12-slim-bookworm -> nvidia base
with system Python 3.12 from Ubuntu Noble repos
- Dockerfile.cluster: convert from single-stage rancher/k3s to
multistage build extracting k3s artifacts onto nvidia base
Closes#244
npm 11 does not support `npm dedupe -g` in global mode, causing the
sandbox container build to fail with error code EDEDUPEGLOBAL.
The dedupe was intended to remove nested copies of vulnerable
dependencies, but npm 11 already handles global package installation
differently. The force-upgrade of tar and @hono/node-server already
ensures the patched versions are installed.
Pass the computed cargo version through Docker and cluster packaging paths so deployed clusters report the built artifact version while keeping latest and explicit version tags aligned.
The entrypoint script uses 'openssl rand -hex 32' to generate the SSH
handshake secret (added in e9f1071), but the rancher/k3s base image
does not include openssl. This causes the container to exit with code
127 immediately on startup.
Fixes#136
* fix(security): harden sandbox SSH with mandatory HMAC secret, NetworkPolicy, and nonce replay detection
Closes#25
- Make NEMOCLAW_SSH_HANDSHAKE_SECRET mandatory: server and sandbox both
refuse to start if the secret is empty/unset. Cluster deployments
auto-generate it via openssl rand in the entrypoint script.
- Add Kubernetes NetworkPolicy restricting sandbox port 2222 ingress to
the gateway pod only, preventing lateral movement from other cluster
workloads.
- Add NSSH1 nonce replay detection with a TTL-bounded cache, rejecting
replayed handshakes within the timestamp validity window.
- Add unit tests for verify_preface (valid, replay, expired, bad HMAC,
malformed) and env injection.
* fix(deploy): pass sshHandshakeSecret in fast deploy helm upgrade
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Piotr Mlocek