* feat(policy): accept numeric UIDs in sandbox process identity validation
Allow run_as_user and run_as_group to be either the literal 'sandbox'
or a numeric UID/GID within [1000, 2_000_000_000]. This removes the
hard dependency on a baked-in 'sandbox' user in container images,
enabling compute drivers to inject resolved UIDs at sandbox creation.
Phase 1 of #1959.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(supervisor): accept numeric UIDs for process identity dropping
Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing
the hard dependency on a baked-in 'sandbox' user in container images.
Changes:
- validate_sandbox_user(): accepts numeric UIDs without passwd lookup
(logs OCSF event); keeps passwd check for "sandbox" name; rejects
non-numeric non-sandbox strings that fail passwd lookup
- prepare_filesystem(): passes numeric UIDs/GIDs directly to chown()
instead of requiring a passwd entry
- drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw
/ Gid::from_raw; skips initgroups when target uid matches current euid;
uses guard conditions before setgid/setuid calls
- session_user_and_home(): falls back to ("{uid}", "/sandbox") for
numeric UIDs, avoiding a passwd lookup that will fail
Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy
so callers have consistent range constants.
Phase 2 of #1959.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations
Phase 3 of the numeric-UID plan: allow operators to specify explicit
sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from
OpenShift SCC namespace annotations, and propagate resolved values to
supervisor container env vars and PVC init container securityContext.
Changes:
- Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig
- Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core
- Implement resolve_sandbox_identity() to fetch namespace annotations
and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range)
- Pass resolved UID/GID through SandboxPodParams to pod spec builder
- Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container
- Update PVC init container securityContext with resolved UID/GID
instead of hard-coded root
- Add comprehensive unit tests for resolution logic and annotation
parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC
annotation parsing)
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(driver-vm): add configurable sandbox UID/GID and update docs/examples
Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001)
in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields.
Changes:
- Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives
- Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root
to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow
- Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID
injection and the ability to skip baked-in sandbox user
- Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both
Kubernetes (with OpenShift SCC autodetection) and VM drivers
- Update sandbox-compute-drivers.mdx: add Sandbox User Identity section
explaining numeric UID support across all compute drivers
- Update rootfs tests to use non-default UIDs, verify config passthrough
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* code review changes
* fix(supervisor): harden tests for restricted CI container environments
Guard tests against CI-specific constraints: root without CAP_SETPCAP,
UIDs with no /etc/passwd entry, and restricted /proc access.
Signed-off-by: Seth Jennings <sjennings@nvidia.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
---------
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Seth Jennings <sjennings@nvidia.com>
The field was added during the Kubernetes driver extraction refactor
(#817) as a pass-through mechanism, but was never wired up to a CLI
flag, Python SDK helper, or any documentation. The only reachable
user path was raw gRPC construction.
The Kubernetes driver now always injects the default workspace PVC,
removing the branching logic that checked for a user-supplied VCT.
Field number 9 is reserved in the proto to prevent reuse.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* feat(kubernetes): support agent-sandbox v1beta1
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* ci(kubernetes): test agent-sandbox api versions
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): retry agent-sandbox raw 404s
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): harden agent-sandbox api setup
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): cache agent-sandbox api version
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* docs(kubernetes): document agent-sandbox upgrade behavior
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* refactor(kubernetes): collapse agent-sandbox api selection
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* feat(gpu)!: add resource requirements
BREAKING CHANGE: SandboxSpec.gpu and DriverSandboxSpec.gpu were replaced with resource_requirements.gpu, changing protobuf field 9 from a bool to a message for both public and driver APIs.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* refactor(gpu): pass requirements through sandbox create
Pass the coupled GPU requirement object through the CLI sandbox_create boundary instead of splitting presence and count into separate arguments.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* refactor(gpu): pass requirements to timeout message
Pass ResourceRequirements into the provisioning timeout message helper so GPU hints are derived from the same nested request object used to create the sandbox.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* refactor(gpu): pass driver requirements through helpers
Thread Option<GpuResourceRequirements> through driver validation and rendering helpers instead of splitting GPU presence and count into separate arguments.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* fix(gpu): validate exact device requests
Require exact driver GPU device lists to be tied to a GPU request, allow a single exact device to use the default countless request, and require explicit matching counts for multi-device lists.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
---------
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* feat(providers): support SPIFFE-backed token grants
Add provider profile token_grant metadata and expand endpoint-specific
dynamic credentials so sandbox supervisors can request SPIFFE JWT-SVIDs,
exchange them with an OAuth-style token endpoint, cache returned access
tokens, and inject bearer tokens into matching HTTP requests.
Wire Kubernetes and Helm deployments to mount the provider SPIFFE Workload
API socket into sandbox pods for token grant exchange.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(examples): add SPIFFE token grant demo
Add a reusable alpha/beta demo that deploys a SPIFFE-verifying token issuer
and protected services, imports a token-grant provider profile, creates a
sandbox, and verifies endpoint-specific bearer tokens.
The script leaves Kubernetes workloads in place, deletes sandboxes through
openshell unless KEEP_SANDBOX=1, and prints protected service logs as proof
of life.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden SPIFFE token grants
* fix(providers): harden dynamic token grants
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden token grant handling
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
Move duplicate in-container path constants from the Docker, Podman,
Kubernetes, and VM driver crates into openshell-core::driver_utils:
- TLS_CA_MOUNT_PATH, TLS_CERT_MOUNT_PATH, TLS_KEY_MOUNT_PATH,
SANDBOX_TOKEN_MOUNT_PATH — identical string literals that were
copy-pasted between openshell-driver-docker and
openshell-driver-podman.
- SUPERVISOR_CONTAINER_DIR (/opt/openshell/bin) and
SUPERVISOR_CONTAINER_BINARY (/opt/openshell/bin/openshell-sandbox) —
replaces three diverging local constants across docker, kubernetes,
podman, and vm-rootfs that referred to the same paths under different
names with different string values (the k8s constant was the directory;
the docker/vm constant was the full binary path).
Each driver crate now assigns its local constant from the canonical core
value, keeping existing names visible to tests without any call-site
churn. The openshell-core constants carry doc comments explaining when
to use the dir vs. the full binary path.
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
* feat(cli): add --env flag to sandbox create and exec
Add `--env KEY=VALUE` (repeatable) to `openshell sandbox create` and
`openshell sandbox exec`, wiring user-specified environment variables
into the existing `SandboxSpec.environment` and
`ExecSandboxRequest.environment` protobuf fields which were previously
only accessible via the SDK/gRPC API.
Fix a bug where user-specified environment variables set via
`SandboxSpec.environment` were silently dropped inside sandbox SSH
sessions. The SSH handler's `apply_child_env()` calls `env_clear()` to
strip supervisor-internal secrets before spawning user shells, but this
also wiped container-level env vars including user-specified ones. This
was introduced in 5fd4885a (2026-02-26, "feat(sandbox): VS Code
Remote-SSH support with platform detection fix") which added
`env_clear()` to fix a VS Code platform detection timeout caused by
leaked supervisor env vars.
The fix introduces an `OPENSHELL_USER_ENVIRONMENT` sidecar env var:
compute drivers (Docker, Podman, Kubernetes, VM) serialize the
user-specified environment as JSON into this variable at container
creation time, the sandbox supervisor deserializes it at startup, and
the SSH handler re-injects the values into child processes after
`env_clear()`. User-specified vars are applied at lowest priority so
proxy, TLS, and provider credentials still take precedence.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* fix(sandbox): address PR review feedback for env var injection
- VM driver: restructure build_guest_environment to apply user env
before required driver vars, preventing user-specified values from
overriding OPENSHELL_ENDPOINT, PATH, and other critical vars.
Matches the pattern already used by Docker/Podman/Kubernetes.
- All drivers: insert OPENSHELL_USER_ENVIRONMENT sidecar after
environment.extend(user_env) so a user-supplied key of the same
name cannot clobber the JSON payload the supervisor parses.
- Podman driver: fix template/spec env precedence — template env is
now applied first with spec overwriting, matching the other drivers
(spec is user-specified and should win over image defaults).
- CLI: add parse_env_pairs() that rejects OPENSHELL_* prefixed keys
with a clear error message. Both sandbox create and exec call sites
updated to use it.
- SSH handler: filter out OPENSHELL_* keys from user_environment
before injecting into child processes.
- Docs: add "Set Environment Variables" section to manage-sandboxes.mdx
documenting --env for both create and exec, per-command override
semantics, and the OPENSHELL_* reservation.
- Tests: add sandbox_create_env_rejects_reserved_prefix test.
- Fix rustfmt formatting in lib.rs, container.rs, driver.rs.
* fix(server): reject OPENSHELL_* env keys at API boundary
Add server-side validation to reject environment variable keys starting
with OPENSHELL_ in SandboxSpec.environment, SandboxTemplate.environment,
and ExecSandboxRequest.environment. This closes the gap where SDK/gRPC
callers (bypassing the CLI) could inject reserved keys that affect
supervisor-internal state.
The CLI already rejects these keys via parse_env_pairs, but the API
boundary is the authoritative enforcement point since all clients
(CLI, SDK, direct gRPC) pass through it.
* fix(server): validate env key names and control chars at API boundary
Add POSIX env key format validation (^[A-Za-z_][A-Za-z0-9_]*$) and
control character rejection for SandboxSpec.environment,
SandboxTemplate.environment, and ExecSandboxRequest.environment at the
gRPC validation layer. This catches invalid keys like "1BAD" or
"BAD-NAME" and values containing newlines at the API boundary rather
than letting them surface as driver-specific container/pod failures.
Also add matching CLI-side validation in parse_env_pairs so users get
immediate feedback. Update docs to say "OpenShell rejects" rather than
"the CLI rejects" since enforcement is now at both boundaries.
* fix(server): allow exec helper keys and add env size limit
Split exec env validation so OPENSHELL_PYFUNC_B64 (used by the Python
SDK's exec_python()) passes through while runtime-clobberable keys
like OPENSHELL_SANDBOX_ID are still rejected. Create-time validation
remains strict — all OPENSHELL_* keys are blocked.
Add a 256 KiB total serialized-size limit on create-time environment
maps. The drivers serialize the full map into a single
OPENSHELL_USER_ENVIRONMENT env var; capping the input at the API
boundary prevents driver/runtime-specific startup failures from
oversized env blocks.
Add template environment control-character test and exec helper key
acceptance test.
---------
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
Allow operators to configure a default Kubernetes runtimeClassName that
is applied to sandbox pods when the CreateSandbox request does not
specify one. This avoids requiring every API caller to explicitly set the
runtime class for clusters that always need a specific RuntimeClass
(e.g. kata-containers, nvidia).
The fallback is applied in the Kubernetes driver only — per-request
values still take priority, and an empty default (the built-in) preserves
existing behavior (field omitted, cluster default applies).
Move three duplicated definitions into openshell-core so every
consumer has a single canonical source:
- format_bytes: identical 14-line function existed in docker,
kubernetes, and vm drivers. Moved to openshell-core::progress
where all three already imported from.
- DEFAULT_SANDBOX_PIDS_LIMIT: i64 constant (2048) duplicated in
docker driver and podman config. Moved to openshell-core::config
alongside other shared defaults. Podman re-exports it for
internal call-site compatibility.
- current_time_ms: secrets.rs in openshell-sandbox reimplemented
the same logic as openshell-core::time::now_ms(). Remove the
local copy and call now_ms() directly via the existing dep.
Extract repeated patterns into shared helpers:
- Add impl_builder_setters! macro to openshell-ocsf/builders that
generates the identical severity(), status(), and message() setter
methods present on all 7 OCSF event builders
- Add SandboxContext::apply_common_fields() to consolidate the
four-line build() finalization (set_status, set_message, set_device,
set_container) repeated in every builder
- Add driver_utils::sandbox_token_path() to centralize the XDG state
path construction for sandbox JWT files used by both the Docker and
Podman drivers
- Add driver_utils::build_capabilities_response() to eliminate the
identical GetCapabilitiesResponse struct literal repeated across the
Docker, Podman, and Kubernetes compute drivers
Remove ~280 lines of duplicated code across 30 files in 5 areas:
- centered_rect: consolidate 5 identical TUI layout helpers into a
single pub fn in openshell-tui/src/ui/mod.rs
- server test helpers: replace ~100 inline Store::connect() calls
with local test_store() helpers; deduplicate test_server_state()
in grpc/service.rs to use the shared test_support version
- rogue PKI: extract 20-line rogue CA+client cert generation block
(duplicated in two integration tests) into generate_rogue_pki()
in tests/common/mod.rs
- provider tests: replace 8 identical 28-line test modules with a
single macro_rules! test_discovers_env_credential! invocation
- label constants: centralize openshell.ai/ container label keys
in openshell-core::driver_utils; update Docker and Kubernetes
drivers to import from there instead of redefining them locally
Move default_sandbox_image() and SUPERVISOR_IMAGE_BINARY_PATH into
openshell-core so all compute drivers share a single authoritative
definition. Eliminates four identical copies of default_sandbox_image()
and two identical copies of SUPERVISOR_IMAGE_BINARY_PATH.
Implement ProviderPlugin for ProviderDiscoverySpec to remove the
repeated three-method delegation boilerplate from eight simple
provider modules. Each now declares only a SPEC constant; the blanket
impl handles id(), discover_existing(), and credential_env_vars().
The Kubernetes driver hardcoded the workspace PVC size to 2Gi. Add a
workspace_default_storage_size field to KubernetesComputeConfig so
operators can tune it via TOML config, Helm values, or the
OPENSHELL_K8S_WORKSPACE_DEFAULT_STORAGE_SIZE environment variable.
* feat(gateway): add TOML configuration file (RFC 0003)
Introduces an opt-in --config / OPENSHELL_GATEWAY_CONFIG flag that loads a
TOML file with gateway-wide settings and per-driver tables. Source
precedence is CLI > env > file > built-in default, implemented via clap's
ValueSource so existing flags and env vars keep their priority.
Driver crates (kubernetes, docker, podman, vm) now derive Deserialize on
their config structs. SupervisorSideloadMethod gains Deserialize with
kebab-case rename. A per-driver inheritance allowlist on the loader side
overlays [openshell.gateway] shared defaults (default_image,
supervisor_image, image_pull_policy, guest_tls_*, ssh_handshake_skew_secs,
client_tls_secret_name, host_gateway_ip, enable_user_namespaces) onto
each [openshell.drivers.<name>] table before deserialization.
The Helm chart renders a new gateway-config ConfigMap and mounts it at
/etc/openshell/gateway.toml. The migrated OPENSHELL_* env entries are
dropped from the StatefulSet — only the Secret-backed
OPENSHELL_SSH_HANDSHAKE_SECRET remains. database_url stays on --db-url.
Adds examples/gateway/gateway.example.toml and updates architecture/gateway.md
with the source precedence and inheritance rules.
* docs(gateway): drop ssh_handshake_skew_secs and ssh_handshake_secret from examples
Both fields are scheduled for removal. Remove the example values and the
env-only note so the gateway.toml example and the architecture doc stop
recommending settings that will not exist much longer.
* docs(rfc): correct OPENSHELL_CONFIG to OPENSHELL_GATEWAY_CONFIG in RFC 0003
* docs(gateway): add per-driver TOML example configurations
Adds focused single-driver examples next to the comprehensive
gateway.example.toml: kubernetes, docker, podman, and microvm. Each one
demonstrates the realistic settings for that driver plus how shared
[openshell.gateway] defaults inherit into the driver table.
A new unit test (`checked_in_examples_parse`) loads every example through
the config_file loader so schema drift fails CI rather than silently
shipping a broken example.
* refactor(gateway): drop image_pull_policy from shared inheritance
Kubernetes and Podman use mutually-incompatible vocabularies for the same
TOML key:
- Kubernetes: `Always | IfNotPresent | Never` (free-form string passed
verbatim to the K8s API).
- Podman: `always | missing | never | newer` (strict lowercase enum
deserialised into `ImagePullPolicy`).
No value means the same thing in both drivers. Sharing the key at
`[openshell.gateway]` scope and inheriting it into every active driver's
table meant any value safe for one driver was either wrong or silently
dropped for the other (`IfNotPresent` → `ImagePullPolicy::Missing` after
`.unwrap_or_default()`). Operators run one driver per gateway, so the
"shared default" never pays for itself.
Make `image_pull_policy` driver-local:
- Remove the field from `GatewayFileSection` and from
`inheritable_keys()` for both Kubernetes and Podman.
- Drop the file→`RunArgs` merge for the gateway-scope key.
- Stop unconditionally clobbering the driver value with
`config.sandbox_image_pull_policy` in the runtime wiring — only apply
the CLI/env override when it was set (and, for Podman, only when it
parses into the lowercase enum).
- Move the key under `[openshell.drivers.kubernetes]` and
`[openshell.drivers.podman]` in every example, the RFC, the
architecture doc, and the Helm-rendered gateway ConfigMap.
The supervisor pull policy follows the same shape: it is K8s-only and
moves into `[openshell.drivers.kubernetes]` alongside `image_pull_policy`
in the Helm template.
* fix(gateway): address review feedback on TOML configuration
Resolves the P1 and P2 issues raised in PR #1317:
- Helm gateway ConfigMap moves `grpc_endpoint` under
`[openshell.drivers.kubernetes]` so the default install no longer fails
the gateway's `deny_unknown_fields` schema check.
- `kubernetes_config_from_file` and `podman_config_from_file` only let
the gateway-wide CLI/env `grpc_endpoint` overwrite the driver-table
value when it was actually supplied, preserving file-only configs.
- Kubernetes driver default `image_pull_policy` is now empty (was Podman
vocabulary "missing"), so default deployments let the Kubernetes API
apply its own policy instead of being rejected.
- New `disable_tls` gateway field plumbs `.Values.server.disableTls`
through the TOML ConfigMap instead of relying on env vars dropped from
the StatefulSet.
- StatefulSet pod template now carries a `checksum/gateway-config`
annotation so `helm upgrade` rolls pods when the ConfigMap changes.
- Auxiliary listener resolution preserves the full `SocketAddr` from
`health_bind_address` / `metrics_bind_address`, so a loopback-pinned
health port is not silently relocated onto the public bind address.
- `ssh_session_ttl_secs` from the file is now applied to `Config` (it
was previously accepted by the loader but never read).
New regression coverage: cli-level merge tests for the new fields plus
helm-unittest assertions for the ConfigMap shape, checksum annotation,
and `disable_tls` rendering.
* docs(gateway): consolidate gateway TOML examples into docs reference
Replaces the per-driver example files under examples/gateway/ with a
single published reference page at docs/reference/gateway-config.mdx
covering source precedence, layout, the full example, and the four
per-driver examples (Kubernetes, Docker, Podman, microVM). Drew flagged
during PR #1317 review that the examples belong with the user-facing
docs rather than in a sibling examples/ directory.
The cross-references in architecture/gateway.md and RFC 0003 are updated
to point at the new docs page; the round-trip test in config_file.rs is
removed (schema coverage stays on the inline parses_full_example test
and per-field merge tests — doc-snippet drift belongs in a separate
docs-lint, not in a cross-tree Rust unit test).
* refactor(core): move DEFAULT_K8S_NAMESPACE into K8s driver
The constant is Kubernetes-specific (used only by KubernetesComputeConfig's
Default impl) and does not belong in openshell-core. Relocate it to the
driver crate that owns the K8s vocabulary; openshell-core retains only
truly cross-cutting defaults.
* refactor(core): move Podman bridge default into Podman driver
DEFAULT_NETWORK_NAME is Podman vocabulary, consumed only by the Podman
driver. Also drops the unused DEFAULT_IMAGE_PULL_POLICY constant.
* docs(auth): scrub remaining SSH handshake secret references
Sweeps the trailing mentions left after the rebase: the gateway
config-file module doc, the Helm gateway-config ConfigMap header,
the gateway-config.mdx env-only note, and the RPM systemd unit
comment for init-gateway-env.sh.
* docs(gateway): clarify OPENSHELL_GRPC_ENDPOINT applies to all drivers
The previous comment implied the callback endpoint was Kubernetes-only,
but the value is propagated to every compute driver (Kubernetes, Docker,
Podman, VM) and must be reachable from wherever the sandbox runs.
* refactor(gateway): move driver options into config (#1394)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
* fix(e2e): regenerate gateway config via TOML for docker + podman harnesses
The gateway CLI flags moved into TOML config tables in 560550d2 (#1394),
which made every existing e2e/with-{docker,podman}-gateway.sh invocation
fail with "unexpected argument '--sandbox-namespace'" (and a long tail
of similar driver-specific options) before the gateway could even bind.
Replace the obsolete CLI flags with a synthesized
`[openshell.drivers.<driver>]` table written to `${STATE_DIR}/gateway.toml`
and passed via the new `--config` flag. Only the gateway-wide flags
that survived 560550d2 (bind-address, port, drivers, db-url, tls-*,
disable-tls, log-level, health-port) stay on the command line.
Both scripts get a small `toml_string` helper to properly TOML-quote
the values (the previous `%q` printf format produced bash-escape, not
TOML-escape). The Docker harness also corrects two field names that
diverged from the driver schema: `docker_network_name` →
`network_name`, and the supervisor binary/image plumbing now reads
through to `supervisor_bin` / `supervisor_image` in the same table.
The Podman harness drops `--ssh-gateway-port` (deleted in 560550d2 —
gRPC + SSH are multiplexed on the same port now) and substitutes
`network_name` + `gateway_port` for the obsolete `--sandbox-namespace`
(which the Podman driver never had as a typed field).
* fix(core): swap bind-only 0.0.0.0 SSH gateway host for cluster URL host
CLI's resolve_ssh_gateway treated 0.0.0.0 as a loopback "keep as-is"
when the cluster URL was also loopback, so the SSH proxy connected to
0.0.0.0:port. The unspecified address is never a valid connect target
and is not present in any TLS cert SAN, which produced BadCertificate
TLS handshake failures during `openshell sandbox create -- ...` in
docker/podman e2e (e.g. bypass_detection).
Resolution: when the server returns 0.0.0.0 or :: as the gateway host
and both endpoints are loopback, fall back to the cluster URL's host
(which the CLI is already using to reach the gateway, so it must
resolve and match the cert).
* fix(e2e): repair podman harness on macOS
Podman 5.x with the applehv/libkrun provider no longer creates the legacy
~/.local/share/containers/podman/machine/podman.sock symlink, and
`podman system service` is a Linux-only subcommand — the macOS client
delegates the API service to the VM. Both assumptions in the harness
were stale, so the script tried to start a temporary service that podman
rejected with "unknown flag: --time".
- Discover the macOS socket via `podman machine inspect` instead of the
hardcoded path.
- On Darwin, fail fast with a "start podman machine" message rather than
attempting the Linux-only `podman system service` fallback.
- Write socket_path into [openshell.drivers.podman] so the in-process
driver picks up the discovered socket; the driver reads TOML only
after the config refactor (560550d2), so OPENSHELL_PODMAN_SOCKET alone
was no longer enough.
* fix(server): use clone_from for TLS client CA assignment
clippy 1.95.0 rejects assigning the result of `Clone::clone()` to an
existing variable under `-D warnings` (`assigning_clones`). Switch to
`clone_from(&...)` to satisfy the lint and avoid the redundant
allocation.
---------
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
* refactor!(auth): drop SSH handshake secret in favor of mTLS
The OPENSHELL_SSH_HANDSHAKE_SECRET / x-sandbox-secret mechanism was
misnamed: it does not authenticate SSH (which flows over the
RelayStream gRPC RPC and is gated by mTLS plus supervisor Unix-socket
permissions). It only gated a small set of sandbox-to-gateway
control-plane RPCs, and production deployments already enforce mTLS
on that channel — so the shared secret was redundant.
Replace the secret check with an mTLS-presence marker. Sandbox-class
methods (ReportPolicyStatus, PushSandboxLogs,
GetSandboxProviderEnvironment, SubmitPolicyAnalysis, GetSandboxConfig,
GetInferenceBundle) accept callers without a Bearer token; the gRPC
mTLS handshake is the trust boundary. Dual-auth methods treat
Bearer-present as full-scope CLI access and Bearer-absent as
sandbox-restricted scope via validate_sandbox_caller_update.
Drops the secret from all drivers (K8s, Podman, VM), the sandbox gRPC
interceptor, the Helm chart (values + pre-install hook + StatefulSet
env), the RPM bootstrap script, the man pages, and the
debug-openshell-cluster skill. Also removes the never-read
ssh_handshake_skew_secs flag and config field.
BREAKING CHANGE: --ssh-handshake-secret / OPENSHELL_SSH_HANDSHAKE_SECRET
and --ssh-handshake-skew-secs / OPENSHELL_SSH_HANDSHAKE_SKEW_SECS are
removed from the gateway, sandbox, and all driver binaries. The
openshell-ssh-handshake K8s Secret is no longer managed by the chart;
operators may delete the orphan. Deployments using
--disable-gateway-auth must enforce caller authentication at the
fronting proxy, since the gateway no longer validates a per-request
secret on sandbox-class methods.
Refs OS-174.
* docs(auth): scrub residual SSH handshake secret references
Sweep across docs, e2e scripts, the Podman driver README/NETWORKING
notes, the RPM/Helm/setup guides, the gateway man page, and RFC 0003
to remove instructions and examples that still referenced
OPENSHELL_SSH_HANDSHAKE_SECRET / --ssh-handshake-secret /
ssh_handshake_skew_secs. The mechanism is gone; nothing should still
suggest setting it. Negative-assertion regression tests are kept so
the env var cannot silently be re-introduced.
* test(drivers): drop SSH handshake secret negative-assertion tests
The supporting code, env vars, CLI flags, and config plumbing are
gone — these tests asserted absence of strings that no longer have
any path to being set. Remove the guards from the Docker, Podman,
Kubernetes, and VM driver test modules.
Consolidate shared constants and helpers into openshell-core to avoid
duplicate definitions scattered across crates. Adds OPENSHELL_SANDBOX
to the sandbox_env module so all drivers and the sandbox supervisor
reference it via a typed constant rather than string literals, matching
the existing pattern for SANDBOX_ID, ENDPOINT, and the other env vars.
Extract node_selector and tolerations from the sandbox template's
platform_config and apply them to the pod spec. This allows callers
to schedule sandbox pods on specific node pools and tolerate taints
without requiring first-class proto fields.
The sandbox_to_k8s_spec function was inserting logLevel and environment
fields at the Sandbox CR spec level. Neither field exists in the
agents.x-k8s.io/v1alpha1 CRD schema, so Kubernetes silently dropped them
and the log_level value never reached the sandbox pod.
Remove the dead CR-level insertions. Build a merged environment map that
includes OPENSHELL_LOG_LEVEL when log_level is set, and pass it to
sandbox_template_to_k8s so the value flows through build_env_list into
the pod container spec.
Add a unit test verifying OPENSHELL_LOG_LEVEL appears in the container
env and that logLevel is absent from the CR spec.
Fixes#1265
* feat(k8s): support ImageVolumeSource for supervisor sideload
Add a config-driven switch between two supervisor binary delivery
methods: image-volume (default) mounts the supervisor OCI image
directly as a read-only volume (K8s >= v1.33), while init-container
preserves the existing emptyDir + copy-self pattern for older clusters.
Configurable via --supervisor-sideload-method CLI arg,
OPENSHELL_SUPERVISOR_SIDELOAD_METHOD env var, or Helm
supervisor.sideloadMethod value.
* feat(helm): auto-detect supervisor sideload method from cluster version
Use Helm .Capabilities.KubeVersion to choose the supervisor sideload
method automatically: image-volume on K8s >= v1.33, init-container on
older clusters. An explicit supervisor.sideloadMethod value overrides
auto-detection.
* fix(helm): raise auto-detect threshold to K8s v1.35
The ImageVolume feature gate is only enabled by default starting in
K8s v1.35, not v1.33 (where it is beta but off by default). Clusters
on v1.33-v1.34 can still opt in with an explicit sideloadMethod value.
Disables automountServiceAccountToken in sandbox pods for security
hardening. Sandbox pods should not have access to the Kubernetes API
by default.
Adds test case to verify the pod spec includes the disabled setting.
Signed-off-by: Derek Carr <decarr@redhat.com>
Add opt-in support for Kubernetes user namespace isolation on sandbox
pods. When enabled, container UID 0 maps to an unprivileged host UID
and capabilities become namespaced, providing defense-in-depth for the
supervisor process.
Configuration is two-layered: a cluster-wide default via
OPENSHELL_ENABLE_USER_NAMESPACES (default false) and a per-sandbox
override via the new `user_namespaces` field on SandboxTemplate.
When user namespaces are active, the pod security context is extended
with SETUID, SETGID, and DAC_READ_SEARCH capabilities to match the
bounding-set requirements inside a user namespace.
Introduces SandboxPodParams struct to replace long argument lists on
sandbox_to_k8s_spec and sandbox_template_to_k8s.
Validated end-to-end on OCP 4.22 (K8s 1.35.3, CRI-O 1.35, RHEL
CoreOS, kernel 5.14) with full SSH tunnel and non-identity UID mapping.
* fix(sandbox): add copy-self subcommand for scratch-image init container
The supervisor image is built FROM scratch and contains only the
openshell-sandbox binary. The Kubernetes side-load init container
previously invoked `sh -c "cp ..."`, which failed with
`executable file 'sh' not found in $PATH` because the scratch image
has no shell.
Add a `copy-self <DEST>` subcommand to the supervisor binary that
resolves its own executable, copies it to the destination path, and
sets mode 0755. Update the Kubernetes driver init container to invoke
this subcommand directly. Mirrors argoexec's emissary self-copy pattern.
* fix(docker): use ubuntu base for supervisor image
`FROM scratch` cannot exec the dynamically linked supervisor binary
(no `/lib/ld-linux-*.so.*`), so the Kubernetes side-load init container
hit `exec /openshell-sandbox: no such file or directory` even after
the `copy-self` subcommand replaced the broken `sh -c` command.
Switch the supervisor stage to `nvcr.io/nvidia/base/ubuntu:noble-20251013`
— the same base the gateway image already uses — so glibc and the
dynamic loader are present. Docker/Podman drivers continue to work
unchanged (they extract or volume-mount the binary into agent containers
that already have glibc); the Kubernetes init container now succeeds
end-to-end.
Verified on a k3d cluster: init container `openshell-supervisor-install`
exits 0 on first try, binary lands at `/opt/openshell/bin/openshell-sandbox`
(mode 0755), and the agent container reaches Ready.
* docs(docker): refresh supervisor target description
The file header still described the supervisor image as
`FROM scratch, binary only`. Update it to match the current
Ubuntu-base supervisor stage.
* Adding qemu vm driver support with GPU pass-through
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
* Add GPU rootfs variant, harden VFIO binding, and fix networking and supervisor reliability issues discovered during GPU VM bring-up.
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
---------
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
The 30s read_timeout on the shared kube client was killing the
long-lived watch streams during idle periods, causing a reconnect
cycle every 30 seconds. Use a separate client with no read_timeout
for watch_sandboxes so the streams stay open indefinitely.
Seth Jennings