Seth Jennings
6461677c32
* feat(policy): accept numeric UIDs in sandbox process identity validation Allow run_as_user and run_as_group to be either the literal 'sandbox' or a numeric UID/GID within [1000, 2_000_000_000]. This removes the hard dependency on a baked-in 'sandbox' user in container images, enabling compute drivers to inject resolved UIDs at sandbox creation. Phase 1 of #1959. Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(supervisor): accept numeric UIDs for process identity dropping Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing the hard dependency on a baked-in 'sandbox' user in container images. Changes: - validate_sandbox_user(): accepts numeric UIDs without passwd lookup (logs OCSF event); keeps passwd check for "sandbox" name; rejects non-numeric non-sandbox strings that fail passwd lookup - prepare_filesystem(): passes numeric UIDs/GIDs directly to chown() instead of requiring a passwd entry - drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw / Gid::from_raw; skips initgroups when target uid matches current euid; uses guard conditions before setgid/setuid calls - session_user_and_home(): falls back to ("{uid}", "/sandbox") for numeric UIDs, avoiding a passwd lookup that will fail Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy so callers have consistent range constants. Phase 2 of #1959. Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations Phase 3 of the numeric-UID plan: allow operators to specify explicit sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from OpenShift SCC namespace annotations, and propagate resolved values to supervisor container env vars and PVC init container securityContext. Changes: - Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig - Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core - Implement resolve_sandbox_identity() to fetch namespace annotations and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range) - Pass resolved UID/GID through SandboxPodParams to pod spec builder - Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container - Update PVC init container securityContext with resolved UID/GID instead of hard-coded root - Add comprehensive unit tests for resolution logic and annotation parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC annotation parsing) Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(driver-vm): add configurable sandbox UID/GID and update docs/examples Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001) in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields. Changes: - Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives - Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow - Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID injection and the ability to skip baked-in sandbox user - Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both Kubernetes (with OpenShift SCC autodetection) and VM drivers - Update sandbox-compute-drivers.mdx: add Sandbox User Identity section explaining numeric UID support across all compute drivers - Update rootfs tests to use non-default UIDs, verify config passthrough Signed-off-by: Seth Jennings <sjenning@redhat.com> * code review changes * fix(supervisor): harden tests for restricted CI container environments Guard tests against CI-specific constraints: root without CAP_SETPCAP, UIDs with no /etc/passwd entry, and restricted /proc access. Signed-off-by: Seth Jennings <sjennings@nvidia.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> --------- Signed-off-by: Seth Jennings <sjenning@redhat.com> Signed-off-by: Seth Jennings <sjennings@nvidia.com>
openshell-driver-kubernetes
Kubernetes-backed compute driver for OpenShell cluster deployments.
The driver uses the Kubernetes API to create, delete, fetch, and watch sandbox custom resources in the configured namespace. It runs in-process with the gateway server.
Runtime Model
The gateway stores platform state and delegates sandbox workload creation to
this driver. Kubernetes owns scheduling and pod lifecycle. The
openshell-sandbox supervisor inside each workload owns agent isolation,
credential injection, policy polling, logs, and the gateway relay.
Sandbox Resource
The driver works with the agents.x-k8s.io Sandbox custom resource. It
detects the served Sandbox API at runtime, caches the selected API version for
the gateway process, and uses v1beta1 when available before falling back to
v1alpha1. Restart the gateway after an in-place Agent Sandbox upgrade so the
driver can detect served API versions again. Driver events map Kubernetes object
state and platform events into the shared compute-driver protobuf surface used
by the gateway.
Kubernetes API calls use explicit timeouts so gRPC handlers do not block indefinitely when the API server is slow or unavailable.
Workspace Persistence
Sandbox pods use a PVC-backed /sandbox workspace. An init container seeds the
PVC from the image's original /sandbox contents on first start and writes a
sentinel so subsequent starts skip the copy.
This is a stopgap persistence model. It preserves user files across pod rescheduling but duplicates the base workspace and does not automatically apply image updates to existing PVCs. Future snapshotting should replace it.
Credentials, TLS, and Relay
The driver injects gateway callback configuration, sandbox identity, TLS client material, and the supervisor SSH socket path into the workload. Driver-owned values must override image-provided environment variables.
Sandbox pods run as service_account_name and keep
automountServiceAccountToken: false. The only Kubernetes token exposed to the
supervisor is an explicit, audience-bound projected token mounted at
/var/run/secrets/openshell/token for the one-shot IssueSandboxToken
bootstrap exchange.
The gateway uses the supervisor relay for connect, exec, and file sync. Sandbox pods do not need direct external ingress for SSH.
Container Security Context
The driver grants the sandbox agent container the Linux capabilities the
supervisor needs for namespace setup and policy enforcement. It can also request
a Kubernetes AppArmor profile through app_armor_profile.
Supported values are Unconfined, RuntimeDefault, and
Localhost/<profile-name>. An empty or unset value omits
securityContext.appArmorProfile. Helm deployments default sandbox agent
containers to Unconfined because runtime/default AppArmor profiles can block
the supervisor's network namespace mount setup on AppArmor-enabled nodes.
GPU Support
When a sandbox requests GPU support, the driver checks node allocatable capacity
for nvidia.com/gpu and requests the configured GPU count in the workload spec.
When no count is set, the driver requests one GPU resource. The sandbox image
must provide the user-space libraries needed by the agent workload.
Driver Config POC
The RFC 0005 POC accepts the selected SandboxTemplate.driver_config.kubernetes
block as DriverSandboxTemplate.driver_config. The Kubernetes driver owns the
nested schema and currently accepts:
pod.node_selectorpod.tolerationspod.runtime_class_namepod.priority_class_namecontainers.agent.resources.requestscontainers.agent.resources.limits
Nested keys inside the kubernetes block use snake_case. The top-level
driver_config envelope is keyed by driver names, so kubernetes is not part
of the nested schema.
Set this through the CLI with the public driver-keyed envelope. The gateway
forwards only the kubernetes object to this driver:
openshell sandbox create \
--driver-config-json '{"kubernetes":{"pod":{"runtime_class_name":"kata-containers","node_selector":{"pool":"gpu"}}}}' \
-- claude
Resource keys use native Kubernetes resource names and quantity strings. The
POC parser renders the keys listed above and rejects unknown fields.
pod.runtime_class_name maps to PodSpec runtimeClassName and overrides the
driver's configured default_runtime_class_name; the typed public
SandboxTemplate.runtime_class_name still takes precedence when set. Use the
public --gpu flag for the default GPU request, pass a count to --gpu for
counted GPU requests, and use driver_config only for additional driver-owned
resource details.