Files
OpenShell/crates/openshell-driver-docker
Matthew GrossmanandEvan Lezar 8c67c27e8b test(oci): add a shared Docker and Podman suite for image USER and WORKDIR (#3931)
* test(oci): share OCI image checks across Docker and Podman

Add an oci-image feature testsuite that runs against installed artifacts.
It covers custom WORKDIR placement, image content and ownership, workspace
writes from the main process and exec, file transfer including
directory-merge upload, OCI user identity, the managed /sandbox fallback,
and rejection of an unwritable WORKDIR.

CI runs the suite on the Docker rootful, Podman rootful, and Podman
rootless tmachine guests.

Replace the Docker-only custom_image e2e with the shared suite.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): require WorkspaceValidationFailed and cover image USER fallback

The unwritable WORKDIR scenario now requires the WorkspaceValidationFailed
reason, so unrelated provisioning failures cannot pass it. The numeric
USER scenario supplies a policy without a process section, so the image
USER is the only source of the sandbox identity.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(podman): leave shared OCI identity checks to the oci-image suite

The oci-image feature suite covers the sandbox identity seen by the main
process and sandbox exec on Docker and Podman. The Podman e2e keeps the
checks only it can make: the pinned image ID and the isolated
workload/supervisor container pair.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): accept existing workspace rejection diagnostics

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): fix cleanup ordering and restore coverage parity

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): remove migrated legacy workspace helpers and checks

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(drivers): report rejected OCI working directories as WorkspaceValidationFailed

The supervisor exits with the reserved status when the sandbox rejects the
image working directory. Docker maps that exit during readiness and
monitoring and records it before removing the supervisor container, and
Podman maps the supervisor companion's exit instead of the workload's.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): require WorkspaceValidationFailed for unwritable WORKDIR

Unrelated provisioning failures can no longer pass the rejection scenario.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(oci): keep exit 78 reserved and report fixed workspace diagnostics

The supervisor no longer passes on a workload's own exit status 78, so
drivers cannot misreport it as a workspace rejection after a runtime
restart. The rejection is recognized once at agent start and carried as a
typed error. Docker reports the same fixed condition message as Podman and
logs the supervisor output instead, and its monitor removes the supervisor
container when a stop overlaps diagnostics collection. The OCI image suite
registers images for cleanup only after they build.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* refactor(oci): simplify workspace failure handling and OCI image tests

Report a recorded Docker workspace rejection once, where supervisor
readiness fails, and match its fixed message exactly. Read the Podman
supervisor exit status once per inspection. Choose the supervisor exit
status in one function and derive the workspace error type.

The OCI image suite shares step helpers, checks that a custom workspace
does not also create /sandbox, and its playbook waits for a connected
gateway and collects failure diagnostics through the shared role.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): replace podman_oci_identity with shared and driver suite checks

The oci-image feature suite now checks, on Docker and Podman, that the
image user has no capabilities, has no_new_privs set, and cannot read
the sandbox control credentials, and that a custom WORKDIR is not a
mount. The checks that need container inspection (pinned image ID,
network=none, supervisor user and capabilities) move to a small test in
the Podman driver suite. Remove the legacy podman_oci_identity e2e test.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): keep podman_oci_identity for Podman-only isolation checks

Restore the Podman e2e test removed in the previous commit and drop
only the checks the shared oci-image suite now runs on Docker and
Podman: workload capabilities, no_new_privs, credential readability,
and the workspace and /sandbox mount checks. The pinned image ID and
the workload/supervisor isolation checks stay, as does the Podman
driver suite unchanged.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* refactor(oci): simplify workspace failure reporting and OCI image suite

Carry the Docker control supervisor's failure reason in a typed error
instead of matching status text, build the fixed workspace message where
the wait result is classified, and drop the fake Docker server test.

Inline image cleanup ordering in the suite, read the container engine
once, share sandbox naming, and stop granting a literal /sandbox in the
image identity policy. Add e2e:oci-image mise tasks and use them from the
ARM64 Docker workflow and TESTING.md.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): move shared container checks into the OCI image suite

Inspect the containers the driver created through the gateway's engine in
the shared suite: the workload runs the inspected image ID as the OCI
identity without a network or supervisor credential mounts, and the
supervisor is a separate non-root container. Keep only the Podman-specific
supervisor identity and capability checks in podman_oci_identity.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): check the workspace rejection reason from sandbox JSON

Read the WorkspaceValidationFailed condition reason from sandbox get
--output json instead of matching CLI output, and restore the Docker
driver's original workspace failure helpers.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): capture fixture image identity before sandbox creation

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
2026-10-13 13:00:59 +00:00
..

openshell-driver-docker

Docker-backed compute driver for local and remote OpenShell gateways.

The driver uses bollard to manage sandbox resources through the configured Docker API socket. When socket_path is unset, it selects the first standard local socket that responds to an API ping. An explicitly selected Docker driver falls back to /var/run/docker.sock when no candidate responds.

When the gateway configures [openshell.gateway.otlp], the in-process driver exports spans to the same OTLP/gRPC collector as openshell-driver-docker. The standalone driver accepts OPENSHELL_OTLP_ENDPOINT, continues W3C trace context from gateway RPC metadata, and flushes spans during graceful shutdown.

Runtime Model

Caller driver config is disabled by default. Existing volumes require administrator-controlled approval labels; raw bind mounts have no supported label resolver and are denied under enforcement. GPU devices are temporarily exempt. Inspect labels again before launch, restart, and during reconciliation. See resource admission configuration.

The driver creates two containers for each sandbox:

  • openshell-sandbox is PID 1 in the workload container. It owns the workload process tree, seccomp notification broker, mandatory Landlock baseline, binary identity, exec/signal/wait/PTY operations, and loopback forwarding.
  • openshell-supervisor runs in a separate companion container. It owns the gateway session, policy engine, credentials, interception CA, SSH relay, L7 inspection, DNS policy, and external upstream connections.

Both containers are non-root, request no capabilities, and set no-new-privileges. A shared named volume carries the authenticated Unix socket and sandbox bootstrap material. A second supervisor-only volume carries the supervisor JWT and gateway client credentials and is never mounted into the workload.

The workload uses network_mode=none. Its seccomp user-notification broker mediates every supported TCP and DNS operation, attributes it to the calling binary, and sends the request across the private channel. The supervisor authorizes the request before it opens an upstream connection. Docker's absent workload network is the mandatory outer fence if mediation fails or is bypassed. The trusted supervisor companion uses Docker host networking, where it reaches the gateway's primary loopback listener and originates approved egress.

The driver copies trusted runtime bytes from the configured supervisor image through the Docker archive API. No workload launch depends on a host bind mount or a tool supplied by the workload image, so the same path works with local, remote, and VM-backed Docker daemons.

Corporate Proxy Egress

[openshell.drivers.docker] accepts the operator-owned corporate proxy fields https_proxy, no_proxy, proxy_auth_file, proxy_auth_allow_insecure, proxy_connect_by_hostname, and proxy_ca_bundle. Sandbox environment, image contents, and per-sandbox driver configuration cannot select or override them.

The driver validates the proxy URL and cross-field relationships at gateway startup. It bounded-reads proxy credentials and the PEM CA bundle before use. Missing, unreadable, empty, oversized, malformed, or certificate-free CA files fail closed. A CA bundle requires https_proxy, although the proxy URL may be http:// when the proxy intercepts destination TLS.

For each supervisor launch, Docker copies the credential and CA contents into the existing supervisor-only named volume. It passes only the fixed container paths /.openshell/supervisor/upstream-proxy-auth and /.openshell/supervisor/upstream-proxy-ca-bundle.pem to the supervisor. It does not bind-mount the gateway-host files, which preserves remote-daemon support and keeps host paths out of workload container metadata.

The supervisor trusts the configured CA for its TLS connection to an HTTPS proxy and for destination certificates re-signed by a TLS-intercepting proxy. It also includes the corporate root in the generated combined trust bundle used by workload processes.

Identity and Workspace

Before creating the workload, the driver pins the image ID and reads its passwd/group databases through a stopped metadata container. It resolves the admitted policy identity, or the image Config.User fallback, into one exact non-root UID, primary GID, and supplementary-group set. Docker launches openshell-sandbox with that identity, and the sandbox uses the same identity for every canonical and exec process. UID or GID zero and unresolved symbolic identities are rejected.

An absolute OCI working directory becomes the workspace. An empty, root (/), or explicit /sandbox declaration uses /sandbox. Any other workdir must already exist without symlink components. The resolved identity must be able to traverse every parent and write and enter the workdir; OpenShell does not change its ownership or mode.

Image VOLUME declarations and user mounts must not cover the workdir, one of its parents, or the reserved /.openshell runtime/channel tree. OpenShell asks the kernel to validate access under the final identity, so POSIX ACL and host LSM decisions remain authoritative.

Container Contract

Setting Purpose
Exact non-root user and group_add Gives sandbox and workload the same immutable UID/GID/group identity required for capability-free observation.
cap_drop = ALL, no cap_add, no-new-privileges Prevents either container from acquiring Linux capabilities.
Docker default seccomp and AppArmor profiles Retains runtime hardening; startup confirmation fails closed if nested seccomp notification is unavailable.
network_mode = none on the workload Removes direct external routes.
network_mode = host on the supervisor Lets the trusted supervisor reach the gateway's primary loopback listener and originate approved upstream connections.
restart_policy = no Keeps canonical main-process exit terminal.
PidsLimit Applies the configured sandbox PID budget. Omit sandbox_pids_limit to use OpenShell's default. Explicit zero is invalid.
Private named volumes One carries the authenticated sandbox/supervisor channel. The other is mounted only into the supervisor and contains its JWT and private gateway credentials.
In-memory /run/openshell-supervisor-ca tmpfs Holds only the public supervisor CA certificate and trust bundle without making all of /run writable.
CDI GPU request Assigns the exact validated CDI devices requested by driver config or count-based selection.

Stop, Start, and Delete

Stop terminates the supervisor companion and stops the workload container without removing it. Docker retains the workload writable layer and attached volumes. Start stages a fresh sandbox bootstrap bundle, restarts that workload, and creates a new supervisor companion. A durably stopped sandbox stays stopped across gateway restarts.

Delete force-removes both containers, the driver-owned runtime volumes, and the host-private runtime descriptor. Missing or altered descriptor and channel resources fail closed; the driver does not run an older combined-supervisor layout.

Driver Config Mounts

The gateway forwards the docker block from --driver-config-json. Supported mount types are:

  • bind: an absolute daemon-host path, allowed only when [openshell.drivers.docker].enable_bind_mounts = true and label admission is explicitly disabled.
  • volume: an existing named volume. The driver never creates or removes a user-supplied volume. Bind-backed local volumes require enable_bind_mounts = true.
  • tmpfs: an in-memory filesystem with optional size and mode.

Host bind mounts are disabled by default because they expose daemon-host paths to sandbox requests. User bind and volume mounts are read-only by default. Targets must be absolute, normalized paths and cannot overlap the workspace root or OpenShell control paths.

Example:

docker volume create openshell-work

openshell sandbox create \
  --driver-config-json '{"docker":{"mounts":[{"type":"volume","source":"openshell-work","target":"/sandbox/work"}]}}' \
  -- claude

Runtime Image

sandbox_runtime_image contains the statically linked musl /openshell-sandbox binary. The driver extracts that binary as bytes and stages it into the stopped workload. supervisor_image contains the dynamically linked glibc /openshell-supervisor binary that runs in the host-networked supervisor container. Release and gateway image builds bake matching image tags into the binary.

Gateway session and TLS

OPENSHELL_ENDPOINT and gateway authentication material are injected only into the supervisor companion. The workload never receives the sandbox JWT, gateway client TLS key, policy authority, or interception CA private key.

When no endpoint is configured, the supervisor connects to 127.0.0.1:<gateway-port>. Set grpc_endpoint when the gateway is not on the Docker daemon host. A configured HTTPS server certificate must include the endpoint host in its subject alternative names.

The driver publishes host loopback as the backend address for host.openshell.internal. Policy DNS resolves that reserved name through the mediated path, so policies can reach host services without a Docker bridge, container DNS alias, or another gateway listener.

For HTTPS endpoints, the supervisor receives only the gateway CA and uses its sandbox-scoped bearer token to authenticate RPCs. User client certificates and private keys are not delivered to either container.

Docker Engine on Linux supports host networking directly. Docker Desktop requires host networking to be enabled in Settings and does not support it when Enhanced Container Isolation is enabled.

The supervisor owns these security-critical variables:

  • OPENSHELL_ENDPOINT
  • OPENSHELL_SANDBOX_ID
  • OPENSHELL_SANDBOX
  • OPENSHELL_SANDBOX_TOKEN_FILE
  • OPENSHELL_SSH_SOCKET_PATH
  • OPENSHELL_MAIN_PROCESS_SPEC
  • OPENSHELL_TLS_CA when HTTPS is enabled

Template and sandbox environment is encoded in the protected bootstrap and exposed only to workload children. Workload input cannot override security-critical supervisor variables.