Matthew Grossman
1fff8b97b9
* fix(supervisor): report rejected OCI working directories as WorkspaceValidationFailed The RFC 0012 split dropped the supervisor exit status that drivers map to the WorkspaceValidationFailed condition. An image WORKDIR the sandbox identity cannot use then surfaced as ControlSupervisorStartFailed on Docker and as a signal kill on Podman. The supervisor now exits with the reserved status when the sandbox rejects the image working directory. Docker maps that supervisor exit during readiness and monitoring, and Podman maps the supervisor companion's exit instead of the workload's. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * feat(podman): honor OCI image working directories Podman sandboxes now use a custom OCI image WORKDIR as the workspace and as the working directory for agent commands, matching Docker. Empty, /, and /sandbox values keep the managed /sandbox workspace volume. A custom workspace stays in the image's container filesystem: no workspace volume, no archive upload, and no root setup step. Resolve the image ID, user, environment, and working directory from one pinned inspection, validate the workdir with the shared OCI rules, and reject Podman control-path overlaps and image volumes or driver mounts that cover it. The runtime starts from / and passes the resolved path to agent commands with --workdir. Add podman_oci_identity to the Podman CI test list. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(drivers): preserve custom workspace failures across supervisor cleanup Check the Podman custom workspace rather than the absent managed volume in the OCI identity E2E. Classify Bollard wait errors by exit code and record Docker workspace failures before removing the supervisor so readiness retains the specific failure reason. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(docker): use sandbox ID for recorded startup failures Keep the workload container ID for Docker inspection and use the sandbox ID to retrieve the monitor failure. Test with distinct identifiers so the lookup cannot accidentally pass with a container ID. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(podman): keep WORKDIR change focused on workspace behavior Remove the new cross-driver workspace failure classification path and its monitor/readiness workaround. Preserve Podman WORKDIR resolution, mount safety, and workspace rejection without requiring a distinct condition reason. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(podman): simplify inspected image metadata and test fixtures Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * code review Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
openshell-driver-podman
The Podman compute driver runs inside the gateway and uses the native libpod REST API over a Unix socket. Each sandbox has two independent containers:
openshell-sandboxowns the agent process in the workload container.openshell-supervisorevaluates policy, holds gateway credentials, and proxies approved egress in a separate companion container.
The driver provisions placement, identity, credentials, transport, and lifecycle. The shared isolation interface supplies exec, attach, signal, terminate, binary identity, DNS, TCP, and loopback-forwarding semantics.
Runtime posture
Caller driver config is disabled by default. Existing volumes require administrator-controlled approval labels; bind and supplemental image mounts are denied under enforcement. Private-volume names alone do not prove ownership. GPU devices are temporarily exempt. Admission runs before launch, restart, and periodically for running workloads. See resource admission configuration.
| Property | Workload | Supervisor |
|---|---|---|
| UID/GID | Pinned non-root workload identity | Same mapped identity |
| Capabilities | Drop all; add none | Drop all; add none |
| Seccomp | Runtime default plus sandbox-installed filters | Runtime default |
| Network | none; loopback only |
Podman host network |
| Gateway JWT and upstream credentials | Never mounted | Podman secrets |
| User volumes and CDI devices | Workload only | Never mounted |
| Channel | Private named volume, writable | Same volume, read-only |
Podman creates the namespaces and volume ownership before the workload runs.
Rootless operation uses the operator's Podman service and subordinate-ID
configuration; it does not require adding capabilities to either container.
The supervisor joins the workload's user namespace only to preserve UID/GID
mapping for shared-volume access. PID, mount, and network namespaces remain
separate. The channel volume uses shared SELinux relabeling (:z).
Before starting either container, the driver uploads volume-relative archives directly to the channel and workspace volume destinations. A rootfs upload on a stopped Podman container does not populate nested named volumes. Restart restores only the channel bootstrap into the existing channel volume, preserving the workspace. The workload starts before the supervisor so its user namespace exists when the supervisor joins it; a stopped supervisor resolves that namespace again on its next start. The driver creates the managed workspace volume owned by the workload's final UID and GID, so the workload never starts as root. Custom image workspaces have no workspace volume or upload.
The runtime must pass the sandbox's unprivileged enforcement probe, including nested seccomp notification and Landlock. Unsupported runtime defaults fail closed; do not switch to an unconfined profile or add capabilities.
Protected channel and network enforcement
agent -> openshell-sandbox === authenticated gRPC / private UDS === supervisor -> network
network=none TCP, DNS, control streams policy + gateway JWT
The workload has no external interface or published port. Seccomp socket
mediation carries TCP and DNS through one authenticated gRPC connection.
DNS remains supervisor-mediated; general UDP is unsupported. The driver sets
net.ipv4.ip_unprivileged_port_start=0 in the isolated workload network
namespace so the sandbox's loopback DNS relay can bind port 53 without a
capability. No nftables or nested network namespace setup runs in the sandbox.
The channel contains the sandbox bootstrap and sandbox-side TLS identity only.
Supervisor private keys and the runtime descriptor stay in the supervisor's private filesystem.
Landlock denies agent access to the top-level /.openshell control hierarchy.
The driver verifies Podman's reported network=none fence before launch and
restart. Host networking applies to the supervisor, not the agent.
Gateway sessions use the sandbox JWT and optional server-authenticated TLS. Only the gateway CA is delivered to the supervisor; user client certificates and private keys are not mounted into either container. The sandbox/supervisor channel always uses its separate, per-sandbox mutual TLS material. These are distinct authentication relationships.
Identity and trusted binaries
Both workload and supervisor images are pinned by immutable image ID. The
driver reads account files from a stopped workload-image container; it never
executes the image to resolve an account. Policy identity fields override OCI
USER independently. Named users/groups resolve against that image, including
supplementary groups. Root and unresolved identities fail before provisioning.
Images must not prepopulate the reserved /.openshell hierarchy; this prevents
image-controlled symlinks from aliasing private control state into user mounts.
sandbox_runtime_image supplies the statically linked musl
/openshell-sandbox binary. Podman's read-only image volume delivers it to the
workload; user-namespace modes that cannot use image volumes retain the trusted
binary extraction path. supervisor_image supplies the dynamically linked
glibc /openshell-supervisor binary outside the workload. Image and request
environment belong to agent children, never the supervisor process.
OCI working directory
OpenShell reads WORKDIR from the workload image. If it is unset, /, or
/sandbox, OpenShell uses its managed /sandbox workspace volume. A custom
path must be absolute and normalized, and cannot overlap /proc, /sys,
/dev, OpenShell-reserved paths, or the workload's private control and CA
mounts. Image volumes and driver mounts cannot cover it; mounts nested below it
remain valid.
A custom path stays in the image's container filesystem with its ownership and permissions. The workload starts as the final non-root user, which must be able to reach and write the directory. Agent commands use the path as their working directory.
Custom workspace contents survive stop/start of the same workload container, but are removed with that container. They do not live in a managed named volume and are not copied to a replacement container.
Lifecycle and readiness
Create builds both stopped containers and stages the private archives before starting either container. The sandbox does not execute the agent until the supervisor authenticates and confirms the common boundary contract. Failed creation removes only containers created by that attempt, then cleans up driver-owned volumes and secrets.
Stop retains both containers, workspace, channel, and secrets. Start restores the consumed sandbox bootstrap from a copy in the supervisor's private filesystem, verifies the fence, and starts the same pair. A failed supervisor start stops the workload. Delete removes the companion first, then the workload, channel, workspace, and driver-owned secrets. User-owned volumes are retained.
Only workload containers appear in sandbox list/watch results. Readiness uses the supervisor's private health socket; there is no shell, legacy marker, or TCP-listener shortcut. Watch reconciliation and supervisor exit/removal events stop a running workload whose companion is unavailable. The gateway also requires the authenticated supervisor session before publishing Ready.
Mounts, GPUs, and configuration
User bind, volume, tmpfs, and image mounts and CDI GPU selection remain
native Podman features and apply only to the workload. Bind mounts require the
operator's enable_bind_mounts opt-in and disabled label admission. Supplemental
image mounts also require disabled admission. Driver JSON requires
allow_driver_config = true. Reserved control paths and the workspace
root cannot be replaced. User-owned volumes are never created or deleted.
See gateway configuration for
operator settings and NETWORKING.md for supervisor networking.
The supervisor uses Podman's host network and owns the upstream proxy settings.
Omit health_check_interval_secs to disable Podman's periodic health command.
Explicit zero is invalid. OpenShell still gates readiness on the supervisor's
authenticated health signal.
Gateway OTLP configuration continues to export compute-driver spans under the
openshell-driver-podman service, preserving gateway trace context.