Files
Jesse JaggarsandDrew Newberry 02b664bb0d refactor(config): normalize and enforce gateway schema v2 (#2814)
* refactor(config): normalize compute driver field names

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* refactor(config): introduce canonical gateway fields

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* refactor(config): enforce gateway schema version 2

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): preserve compute driver runtime guarantees

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): address schema v2 review regressions

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): complete schema v2 migration safeguards

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(config): expand schema v2 regression coverage

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(config): add schema v2 parity manifest

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): correct parity manifest inventory

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* docs(config): record schema v2 intentional changes

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* docs(config): disposition schema v2 parity gaps

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): add dual schema parity harness

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): establish compute lifecycle parity baseline

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): preserve gateway option compatibility

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): record gateway option parity

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* docs(config): close gateway-wide parity gaps

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(podman): apply configured pids limit

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): validate Podman option parity

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): add Kubernetes option parity harness

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): record Kubernetes option parity

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): disposition VM parity lanes

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): add external driver parity lane

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(e2e): preserve external driver pull policy

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): attest parity artifacts and launches

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): require clean parity build sources

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): bind parity runtime artifacts

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(e2e): use isolated supervisor tags

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(e2e): qualify parity image tags

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(e2e): serve parity supervisor locally

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): isolate parity podman services

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): harden parity evidence provenance

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): pin parity sandbox artifacts

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): attest parity runtime inputs

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): bind parity runtime evidence

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): record compute boundary parity

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(e2e): disposition cross-cutting parity lanes

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(packaging): preflight gateway config upgrades

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): preserve rebase integration guarantees

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(ci): isolate temporary git signing config

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): update remaining schema v2 consumers

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(ci): provide e2fs tools to VM tests

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): align preflight with gateway startup

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(vm): preserve rootfs tar configuration

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* chore(config): adopt duration unit constructors

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(packaging): preflight RPM gateway config

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(config): address driver review findings

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(e2e): require fresh semantic parity evidence

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(docker): update tests for renamed sandbox label

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* test(gateway): preserve selective driver coverage after rebase

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
2026-09-11 05:00:24 +00:00

12 KiB

openshell-driver-docker

Docker-backed compute driver for local OpenShell gateways.

When the gateway configures [openshell.gateway.otlp], Docker compute-driver spans export to the same OTLP/gRPC collector with the service name openshell-driver-docker. The in-process driver preserves the gateway trace context and emits the compute-driver RPC boundary that a standalone driver would expose.

mise run gateway:docker enables this export only when a local collector is listening on 127.0.0.1:4317. Otherwise, it omits the gateway OTLP configuration so the development gateway does not repeatedly report export failures.

The standalone openshell-driver-docker binary accepts OPENSHELL_OTLP_ENDPOINT. When set, it exports Docker driver spans to that collector, continues W3C trace context from gateway RPC metadata, and flushes spans during graceful shutdown.

The driver manages sandbox containers through the local Docker daemon with the bollard client. It is intended for developer environments where Docker is already available and running Kubernetes would be unnecessary.

The driver connects to [openshell.drivers.docker].socket_path when configured. Otherwise, it uses the first standard local Docker socket that responds to an API ping, which is the same selection mechanism used by gateway auto-detection. An explicitly selected Docker driver falls back to /var/run/docker.sock when no candidate responds.

Runtime Model

The gateway runs as a host process. The Docker driver creates one container per sandbox and starts the openshell-sandbox supervisor inside that container. The supervisor then creates the nested sandbox namespace for the agent process.

Stop and Start

Stop stops the managed container without removing it. Docker retains the container writable layer, attached volumes, labels, token material, and restart policy. Start starts that same container, so files in the resolved OCI workspace remain available. A durably stopped sandbox is excluded from gateway startup recovery and stays stopped across gateway restarts. Delete continues to force-remove the container and clean up driver-owned material. Graceful gateway shutdown sends StopSandbox for each sandbox whose persisted phase requires running compute without changing that persisted intent. On startup, the gateway sends an idempotent StartSandbox request for the same sandboxes, restarting their retained containers. Explicitly stopped sandboxes remain excluded.

Before creating the container, the driver inspects the final sandbox image and captures its immutable image ID, raw OCI Config.User, and OCI Config.WorkingDir. Container creation uses that image ID, preventing a mutable tag from changing between inspection and launch. The supervisor runs as root, resolves omitted policy identity fields from the image declaration, and drops only agent children to the resulting identity. Named OCI components remain names after validation; a missing group is filled with the user's numeric primary GID. Explicit process.run_as_user and process.run_as_group values take precedence independently.

An absolute OCI working directory becomes the agent workspace. An empty, root (/), or explicit /sandbox declaration uses /sandbox, which OpenShell creates when necessary and owns as a compatibility workspace. Any other image workdir must already exist without symlink components. The completed identity, including supplementary groups, must already be able to traverse every parent and write and enter the workdir. OpenShell does not change its ownership or mode.

OpenShell deliberately asks the Linux kernel to make this access decision under the completed sandbox identity instead of reproducing permission rules from ownership and mode bits. Mode-bit inspection alone can reject authority granted by a POSIX ACL or overlook a denial imposed by a Linux Security Module such as SELinux or AppArmor. OpenShell does not configure or otherwise manage ACLs or LSM policy here; the one-shot validator only observes the kernel's effective decision. This keeps the no-authority-expansion invariant aligned with the access the eventual workload will receive without adding a separate, incomplete permission model to OpenShell.

Image VOLUME declarations must not cover the workdir or one of its parents because Docker would mount the volume before the supervisor could validate the immutable image path. Workdirs under the standard OCI runtime namespaces /proc, /sys, and /dev are rejected, as are paths that overlap concrete OpenShell control resources. The workspace is the child cwd and HOME. The supervisor starts from /, then reports an invalid workdir as a readiness failure.

Docker containers join an OpenShell-managed bridge network. The driver injects host.openshell.internal and host.docker.internal so supervisors have stable names for reaching the gateway host. On Docker Desktop, Colima, Rancher Desktop, OrbStack, and macOS-hosted gateways, those names use Docker's host-gateway alias. The driver requests a separate IPv4 loopback callback listener when the primary listener does not already cover it. On native Linux Docker, the gateway also binds the bridge gateway IP so containers can call back to the host process.

Container Contract

The driver-controlled container settings are part of the sandbox security contract:

Setting Purpose
user = "0" The supervisor needs root inside the container to prepare namespaces, mounts, Landlock, and seccomp.
network_mode = openshell Places the supervisor on the managed Docker bridge network.
cap_add Grants supervisor-only capabilities required for namespace setup and process inspection.
apparmor=unconfined Avoids Docker's default profile blocking required mount operations.
restart_policy = no A canonical main-process exit remains terminal and is not silently restarted by Docker.
PidsLimit Enforces the sandbox PID budget at the Docker cgroup layer. [openshell.drivers.docker].sandbox_pids_limit defaults to 2048; explicit 0 is invalid.
CDI GPU request Uses opaque driver_config.cdi_devices values when set; otherwise selects the requested count of NVIDIA CDI GPUs in round-robin order when daemon CDI support is detected. Docker daemon /info can permit nvidia.com/gpu=all as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count.
policy-dns-transparent-tcp capability Declares that the combined Docker supervisor can own namespace-local DNS/TCP capture and coupled workload restart. The shared supervisor still owns DNS eligibility, mappings, authorization, pinned dialing, relaying, and OCSF decisions. The marker is stripped from the workload environment.

The agent child process does not retain these supervisor privileges.

Driver Config Mounts

The gateway forwards the docker block from --driver-config-json to this driver. The driver accepts user-supplied mounts entries with these Docker mount types:

  • bind: mounts an absolute host path when [openshell.drivers.docker] has enable_bind_mounts = true.
  • volume: mounts an existing Docker named volume. The driver validates that the volume exists before provisioning and never creates or removes it. Docker local-driver volumes created with bind options are treated as host bind mounts and require enable_bind_mounts = true.
  • tmpfs: mounts an in-memory filesystem with optional options, size_bytes, and mode.

Host bind mounts are disabled by default because they expose gateway host paths to sandbox requests. Image mounts are not part of the Docker driver-config schema. The driver still uses internal bind mounts for OpenShell-owned supervisor, token, and TLS material.

Docker bind mounts accept source, target, optional read_only, and an optional selinux_label of shared (applies :z) or private (applies :Z) for SELinux-enforcing hosts. Docker volume mounts may include subpath. User-supplied bind and volume mounts are read-only by default; set read_only: false to make them writable. Mount source, target, and subpath values must not contain surrounding whitespace. Mount targets must be absolute container paths and must not replace or contain the resolved workspace root. Nested workspace mounts remain valid. Mounts also must not overlap the configured SSH socket or the reserved /opt/openshell, /etc/openshell, /etc/openshell-tls, /run/openshell, /run/openshell-sidecar, and network namespace roots.

Example named-volume usage:

docker volume create openshell-work

openshell sandbox create \
  --driver-config-json '{"docker":{"mounts":[{"type":"volume","source":"openshell-work","target":"/sandbox/work"}]}}' \
  -- claude

Supervisor Binary Resolution

The Docker driver bind-mounts a host-side Linux openshell-sandbox binary into each sandbox container. Resolution order is:

  1. supervisor_bin in [openshell.drivers.docker].
  2. supervisor_image in [openshell.drivers.docker], extracting /openshell-sandbox from that image.
  3. A sibling openshell-sandbox next to the running openshell-gateway binary.
  4. A local Linux cargo target build for the Docker daemon architecture.
  5. The release-matched default supervisor image, extracting /openshell-sandbox.

Release and Docker-image gateway builds bake the matching supervisor image tag into the binary at compile time. The default Docker supervisor image is not :latest unless a custom build explicitly sets that tag.

Callback and TLS

OPENSHELL_ENDPOINT is injected from the gateway's configured gRPC endpoint. When no endpoint is configured, the driver uses host.openshell.internal:<gateway-port> with the appropriate HTTP or HTTPS scheme. Set host_gateway_ip only when the host has an explicit, locally assigned address that containers should use for callbacks; package-managed macOS gateways should leave it unset.

For HTTPS endpoints, the server certificate must include the endpoint host as a subject alternative name. Docker sandboxes also need the client TLS bundle mounted into the container and exposed with:

  • OPENSHELL_TLS_CA
  • OPENSHELL_TLS_CERT
  • OPENSHELL_TLS_KEY

HTTP endpoints reject TLS material because the supervisor would not use it.

Corporate proxy, SPIFFE, and AppArmor

https_proxy, no_proxy, and proxy_auth_file in [openshell.drivers.docker] are operator-owned supervisor settings. Docker passes the proxy URL and bypass list on the supervisor command line and mounts an optional user:pass auth file read-only at a root-only path. Credentials never appear in container environment or Docker labels. An auth file used with an http:// proxy requires proxy_auth_allow_insecure = true; an https:// proxy protects the Basic-auth header in its TLS session.

Set provider_spiffe_workload_api_socket to an absolute host UNIX socket to project its dedicated parent directory into the supervisor and set OPENSHELL_PROVIDER_SPIFFE_WORKLOAD_API_SOCKET to the guest path. TCP URIs are rejected for this projection. app_armor_profile uses the shared RuntimeDefault, Unconfined, or Localhost/<profile> vocabulary. Docker uses explicit Unconfined by default because the supervisor's namespace mount setup is incompatible with docker-default; requested confined profiles fail at startup if Docker does not report AppArmor support.

Environment Ownership

The driver merges template environment and sandbox spec environment first, then overwrites security-critical keys:

  • OPENSHELL_ENDPOINT
  • OPENSHELL_SANDBOX_ID
  • OPENSHELL_SANDBOX
  • OPENSHELL_SSH_SOCKET_PATH
  • OPENSHELL_MAIN_PROCESS_SPEC
  • TLS path variables when HTTPS is enabled

Do not allow sandbox images or templates to override these values.