Matthew GrossmanandEvan Lezar
8c67c27e8b
* test(oci): share OCI image checks across Docker and Podman Add an oci-image feature testsuite that runs against installed artifacts. It covers custom WORKDIR placement, image content and ownership, workspace writes from the main process and exec, file transfer including directory-merge upload, OCI user identity, the managed /sandbox fallback, and rejection of an unwritable WORKDIR. CI runs the suite on the Docker rootful, Podman rootful, and Podman rootless tmachine guests. Replace the Docker-only custom_image e2e with the shared suite. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): require WorkspaceValidationFailed and cover image USER fallback The unwritable WORKDIR scenario now requires the WorkspaceValidationFailed reason, so unrelated provisioning failures cannot pass it. The numeric USER scenario supplies a policy without a process section, so the image USER is the only source of the sandbox identity. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(podman): leave shared OCI identity checks to the oci-image suite The oci-image feature suite covers the sandbox identity seen by the main process and sandbox exec on Docker and Podman. The Podman e2e keeps the checks only it can make: the pinned image ID and the isolated workload/supervisor container pair. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): accept existing workspace rejection diagnostics Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): fix cleanup ordering and restore coverage parity Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): remove migrated legacy workspace helpers and checks Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(drivers): report rejected OCI working directories as WorkspaceValidationFailed The supervisor exits with the reserved status when the sandbox rejects the image working directory. Docker maps that exit during readiness and monitoring and records it before removing the supervisor container, and Podman maps the supervisor companion's exit instead of the workload's. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): require WorkspaceValidationFailed for unwritable WORKDIR Unrelated provisioning failures can no longer pass the rejection scenario. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(oci): keep exit 78 reserved and report fixed workspace diagnostics The supervisor no longer passes on a workload's own exit status 78, so drivers cannot misreport it as a workspace rejection after a runtime restart. The rejection is recognized once at agent start and carried as a typed error. Docker reports the same fixed condition message as Podman and logs the supervisor output instead, and its monitor removes the supervisor container when a stop overlaps diagnostics collection. The OCI image suite registers images for cleanup only after they build. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(oci): simplify workspace failure handling and OCI image tests Report a recorded Docker workspace rejection once, where supervisor readiness fails, and match its fixed message exactly. Read the Podman supervisor exit status once per inspection. Choose the supervisor exit status in one function and derive the workspace error type. The OCI image suite shares step helpers, checks that a custom workspace does not also create /sandbox, and its playbook waits for a connected gateway and collects failure diagnostics through the shared role. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): replace podman_oci_identity with shared and driver suite checks The oci-image feature suite now checks, on Docker and Podman, that the image user has no capabilities, has no_new_privs set, and cannot read the sandbox control credentials, and that a custom WORKDIR is not a mount. The checks that need container inspection (pinned image ID, network=none, supervisor user and capabilities) move to a small test in the Podman driver suite. Remove the legacy podman_oci_identity e2e test. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): keep podman_oci_identity for Podman-only isolation checks Restore the Podman e2e test removed in the previous commit and drop only the checks the shared oci-image suite now runs on Docker and Podman: workload capabilities, no_new_privs, credential readability, and the workspace and /sandbox mount checks. The pinned image ID and the workload/supervisor isolation checks stay, as does the Podman driver suite unchanged. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(oci): simplify workspace failure reporting and OCI image suite Carry the Docker control supervisor's failure reason in a typed error instead of matching status text, build the fixed workspace message where the wait result is classified, and drop the fake Docker server test. Inline image cleanup ordering in the suite, read the container engine once, share sandbox naming, and stop granting a literal /sandbox in the image identity policy. Add e2e:oci-image mise tasks and use them from the ARM64 Docker workflow and TESTING.md. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): move shared container checks into the OCI image suite Inspect the containers the driver created through the gateway's engine in the shared suite: the workload runs the inspected image ID as the OCI identity without a network or supervisor credential mounts, and the supervisor is a separate non-root container. Keep only the Podman-specific supervisor identity and capability checks in podman_oci_identity. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): check the workspace rejection reason from sandbox JSON Read the WorkspaceValidationFailed condition reason from sandbox get --output json instead of matching CLI output, and restore the Docker driver's original workspace failure helpers. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): capture fixture image identity before sandbox creation Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> Co-authored-by: Evan Lezar <elezar@nvidia.com>
openshell-driver-podman
The Podman compute driver runs inside the gateway and uses the native libpod REST API over a Unix socket. Each sandbox has two independent containers:
openshell-sandboxowns the agent process in the workload container.openshell-supervisorevaluates policy, holds gateway credentials, and proxies approved egress in a separate companion container.
The driver provisions placement, identity, credentials, transport, and lifecycle. The shared isolation interface supplies exec, attach, signal, terminate, binary identity, DNS, TCP, and loopback-forwarding semantics.
Runtime posture
Caller driver config is disabled by default. Existing volumes require administrator-controlled approval labels; bind and supplemental image mounts are denied under enforcement. Private-volume names alone do not prove ownership. GPU devices are temporarily exempt. Admission runs before launch, restart, and periodically for running workloads. See resource admission configuration.
| Property | Workload | Supervisor |
|---|---|---|
| UID/GID | Pinned non-root workload identity | Same mapped identity |
| Capabilities | Drop all; add none | Drop all; add none |
| Seccomp | Runtime default plus sandbox-installed filters | Runtime default |
| Network | none; loopback only |
Podman host network |
| Gateway JWT and upstream credentials | Never mounted | Podman secrets |
| User volumes and CDI devices | Workload only | Never mounted |
| Channel | Private named volume, writable | Same volume, read-only |
Podman creates the namespaces and volume ownership before the workload runs.
Rootless operation uses the operator's Podman service and subordinate-ID
configuration; it does not require adding capabilities to either container.
The supervisor joins the workload's user namespace only to preserve UID/GID
mapping for shared-volume access. PID, mount, and network namespaces remain
separate. The channel volume uses shared SELinux relabeling (:z).
Before starting either container, the driver uploads volume-relative archives directly to the channel and workspace volume destinations. A rootfs upload on a stopped Podman container does not populate nested named volumes. Restart restores only the channel bootstrap into the existing channel volume, preserving the workspace. The workload starts before the supervisor so its user namespace exists when the supervisor joins it; a stopped supervisor resolves that namespace again on its next start. The driver creates the managed workspace volume owned by the workload's final UID and GID, so the workload never starts as root. Custom image workspaces have no workspace volume or upload.
The runtime must pass the sandbox's unprivileged enforcement probe, including nested seccomp notification and Landlock. Unsupported runtime defaults fail closed; do not switch to an unconfined profile or add capabilities.
Protected channel and network enforcement
agent -> openshell-sandbox === authenticated gRPC / private UDS === supervisor -> network
network=none TCP, DNS, control streams policy + gateway JWT
The workload has no external interface or published port. Seccomp socket
mediation carries TCP and DNS through one authenticated gRPC connection.
DNS remains supervisor-mediated; general UDP is unsupported. The driver sets
net.ipv4.ip_unprivileged_port_start=0 in the isolated workload network
namespace so the sandbox's loopback DNS relay can bind port 53 without a
capability. No nftables or nested network namespace setup runs in the sandbox.
The channel contains the sandbox bootstrap and sandbox-side TLS identity only.
Supervisor private keys and the runtime descriptor stay in the supervisor's private filesystem.
Landlock denies agent access to the top-level /.openshell control hierarchy.
The driver verifies Podman's reported network=none fence before launch and
restart. Host networking applies to the supervisor, not the agent.
Gateway sessions use the sandbox JWT and optional server-authenticated TLS. Only the gateway CA is delivered to the supervisor; user client certificates and private keys are not mounted into either container. The sandbox/supervisor channel always uses its separate, per-sandbox mutual TLS material. These are distinct authentication relationships.
Identity and trusted binaries
Both workload and supervisor images are pinned by immutable image ID. The
driver reads account files from a stopped workload-image container; it never
executes the image to resolve an account. Policy identity fields override OCI
USER independently. Named users/groups resolve against that image, including
supplementary groups. Root and unresolved identities fail before provisioning.
Images must not prepopulate the reserved /.openshell hierarchy; this prevents
image-controlled symlinks from aliasing private control state into user mounts.
sandbox_runtime_image supplies the statically linked musl
/openshell-sandbox binary. Podman's read-only image volume delivers it to the
workload; user-namespace modes that cannot use image volumes retain the trusted
binary extraction path. supervisor_image supplies the dynamically linked
glibc /openshell-supervisor binary outside the workload. Image and request
environment belong to agent children, never the supervisor process.
OCI working directory
OpenShell reads WORKDIR from the workload image. If it is unset, /, or
/sandbox, OpenShell uses its managed /sandbox workspace volume. A custom
path must be absolute and normalized, and cannot overlap /proc, /sys,
/dev, OpenShell-reserved paths, or the workload's private control and CA
mounts. Image volumes and driver mounts cannot cover it; mounts nested below it
remain valid.
A custom path stays in the image's container filesystem with its ownership and permissions. The workload starts as the final non-root user, which must be able to reach and write the directory. Agent commands use the path as their working directory.
Custom workspace contents survive stop/start of the same workload container, but are removed with that container. They do not live in a managed named volume and are not copied to a replacement container.
Lifecycle and readiness
Create builds both stopped containers and stages the private archives before starting either container. The sandbox does not execute the agent until the supervisor authenticates and confirms the common boundary contract. Failed creation removes only containers created by that attempt, then cleans up driver-owned volumes and secrets.
Stop retains both containers, workspace, channel, and secrets. Start restores the consumed sandbox bootstrap from a copy in the supervisor's private filesystem, verifies the fence, and starts the same pair. A failed supervisor start stops the workload. Delete removes the companion first, then the workload, channel, workspace, and driver-owned secrets. User-owned volumes are retained.
Only workload containers appear in sandbox list/watch results. Readiness uses the supervisor's private health socket; there is no shell, legacy marker, or TCP-listener shortcut. Watch reconciliation and supervisor exit/removal events stop a running workload whose companion is unavailable. The gateway also requires the authenticated supervisor session before publishing Ready.
Mounts, GPUs, and configuration
User bind, volume, tmpfs, and image mounts and CDI GPU selection remain
native Podman features and apply only to the workload. Bind mounts require the
operator's enable_bind_mounts opt-in and disabled label admission. Supplemental
image mounts also require disabled admission. Driver JSON requires
allow_driver_config = true. Reserved control paths and the workspace
root cannot be replaced. User-owned volumes are never created or deleted.
See gateway configuration for
operator settings and NETWORKING.md for supervisor networking.
The supervisor uses Podman's host network and owns the upstream proxy settings.
Omit health_check_interval_secs to disable Podman's periodic health command.
Explicit zero is invalid. OpenShell still gates readiness on the supervisor's
authenticated health signal.
Gateway OTLP configuration continues to export compute-driver spans under the
openshell-driver-podman service, preserving gateway trace context.