Previously, Docker was auto-detected when the CLI was installed or a candidate
Unix socket existed. Neither check verified that the Docker API was responsive.
A similar check was done when auto-detecting Podman in the past, but was
replaced in 1f07bf04 with a probe of candidate Podman sockets instead.
This change applies the functional API probing approach introduced for Podman
in 1f07bf04 to Docker. It also makes Docker driver initialization use the same
socket-selection mechanism as Docker auto-detection instead of Bollard’s local
defaults. This means the previously auto-detectable Docker socket paths
$HOME/.docker/run/docker.sock and $XDG_RUNTIME_DIR/docker.sock will actually be
usable.
When no working compute driver can be auto-detected, the gateway exits early
with a message saying as much:
> configuration error: no compute driver configured and auto-detection found no
> suitable driver; set --drivers or OPENSHELL_DRIVERS to kubernetes, podman,
> docker, or vm
This makes for a better user experience when installing OpenShell without an
available supported compute driver.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* feat(kubernetes): add sidecar supervisor topology
Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid similar process id names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid similar process id names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(sandbox): avoid similar proxy id names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* docs(kubernetes): clarify sidecar topology limits
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): keep sidecar process leaf capless
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): refresh sidecar provider env snapshots
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* test(supervisor): align hot-swap identity regression
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): stage sidecar mtls files before proxy chown
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): simplify sidecar supervisor topology
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* chore(helm): reuse sidecar skaffold values
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid similar iptables helper names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(e2e): harden kube gateway wrapper setup
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid nft batch rollback on OCP
Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): preserve process identity in sidecar topology
Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table.
Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* refactor(kubernetes): replace sidecar snapshots with control socket
Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files.
Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* feat(kubernetes): support relaxed sidecar network identity
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(sandbox): satisfy sidecar clippy lint
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* refactor(kubernetes): standardize topology naming
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(sandbox): satisfy linux clippy timeout import
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): support kata sidecar on ipv4 pods
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): satisfy linux clippy for sidecar fallback
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* chore(kubernetes): remove stale supervisor topology references
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): enable sidecar binary policy inspection
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): harden sidecar control boundary
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): couple sidecar supervisor lifecycles
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Co-authored-by: Seth Jennings <sjenning@redhat.com>
* fix(core): pin supervisor image tag to gateway version for all drivers
The Podman and Kubernetes drivers defaulted the supervisor image to
`:latest` via DEFAULT_SUPERVISOR_IMAGE, while the Docker driver already
resolved a version-pinned tag. Extract the tag resolution logic into
openshell-core so all three drivers use the same
OPENSHELL_IMAGE_TAG > IMAGE_TAG > CARGO_PKG_VERSION priority chain.
Closes#2068
Signed-off-by: Florent Benoit <fbenoit@redhat.com>
* refactor(core): simplify supervisor image tag resolver to slice-based API
Remove the Docker driver's wrapper functions and call
openshell_core::config::default_supervisor_image() directly.
Simplify resolve_supervisor_image_tag to accept &[&str] instead
of three separate parameters.
Signed-off-by: Florent Benoit <fbenoit@nvidia.com>
Signed-off-by: Florent Benoit <fbenoit@redhat.com>
---------
Signed-off-by: Florent Benoit <fbenoit@redhat.com>
Signed-off-by: Florent Benoit <fbenoit@nvidia.com>
* feat(policy): accept numeric UIDs in sandbox process identity validation
Allow run_as_user and run_as_group to be either the literal 'sandbox'
or a numeric UID/GID within [1000, 2_000_000_000]. This removes the
hard dependency on a baked-in 'sandbox' user in container images,
enabling compute drivers to inject resolved UIDs at sandbox creation.
Phase 1 of #1959.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(supervisor): accept numeric UIDs for process identity dropping
Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing
the hard dependency on a baked-in 'sandbox' user in container images.
Changes:
- validate_sandbox_user(): accepts numeric UIDs without passwd lookup
(logs OCSF event); keeps passwd check for "sandbox" name; rejects
non-numeric non-sandbox strings that fail passwd lookup
- prepare_filesystem(): passes numeric UIDs/GIDs directly to chown()
instead of requiring a passwd entry
- drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw
/ Gid::from_raw; skips initgroups when target uid matches current euid;
uses guard conditions before setgid/setuid calls
- session_user_and_home(): falls back to ("{uid}", "/sandbox") for
numeric UIDs, avoiding a passwd lookup that will fail
Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy
so callers have consistent range constants.
Phase 2 of #1959.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations
Phase 3 of the numeric-UID plan: allow operators to specify explicit
sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from
OpenShift SCC namespace annotations, and propagate resolved values to
supervisor container env vars and PVC init container securityContext.
Changes:
- Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig
- Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core
- Implement resolve_sandbox_identity() to fetch namespace annotations
and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range)
- Pass resolved UID/GID through SandboxPodParams to pod spec builder
- Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container
- Update PVC init container securityContext with resolved UID/GID
instead of hard-coded root
- Add comprehensive unit tests for resolution logic and annotation
parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC
annotation parsing)
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(driver-vm): add configurable sandbox UID/GID and update docs/examples
Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001)
in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields.
Changes:
- Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives
- Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root
to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow
- Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID
injection and the ability to skip baked-in sandbox user
- Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both
Kubernetes (with OpenShift SCC autodetection) and VM drivers
- Update sandbox-compute-drivers.mdx: add Sandbox User Identity section
explaining numeric UID support across all compute drivers
- Update rootfs tests to use non-default UIDs, verify config passthrough
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* code review changes
* fix(supervisor): harden tests for restricted CI container environments
Guard tests against CI-specific constraints: root without CAP_SETPCAP,
UIDs with no /etc/passwd entry, and restricted /proc access.
Signed-off-by: Seth Jennings <sjennings@nvidia.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
---------
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Seth Jennings <sjennings@nvidia.com>
* feat(docker,podman): add SELinux label support for bind mounts
The Docker Engine structured Mount API does not support SELinux
relabelling (:z / :Z). Move user-supplied bind mounts from the
structured `mounts` field to the legacy string-format `binds` field,
which does support these options.
Add a shared `SelinuxLabel` enum (shared/private) to openshell-core so
both Docker and Podman drivers accept an optional `selinux_label` field
on bind mount configs. For Docker, labels are appended to the bind
string; for Podman, they are pushed to the mount options vec.
Signed-off-by: Florian Bergmann <fbergman@redhat.com>
* fix(docker): reject missing bind source paths on legacy binds
Moving user bind mounts from the structured Mount API to the legacy
Binds field changed Docker's behavior for missing source directories:
the legacy path silently creates them as empty root-owned dirs instead
of erroring. Add an explicit Path::exists() check to preserve the
fail-fast behavior operators expect.
Signed-off-by: Florian Bergmann <fbergman@redhat.com>
---------
Signed-off-by: Florian Bergmann <fbergman@redhat.com>
Add policy schema, proto, provider profile, OPA, and L7 proxy support for
`protocol: json-rpc` and `protocol: mcp`. Generic JSON-RPC endpoints match
exact method names only, with `method: "*"` as the all-method sentinel;
wildcard/glob methods and params matchers are rejected.
Parse JSON-RPC request bodies and batches in the forward proxy, deny
response-shaped client frames, limit receive-stream GET allowance to MCP
endpoints, and redact params in decision logs. Preserve L7 rule params on the
proto load path so MCP `tools/call` tool filters behave like YAML-loaded
policies.
Add MCP conformance coverage, JSON-RPC L7 e2e coverage, and docs for the new
protocols and current matcher limitations.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Co-authored-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
* feat(kubernetes): support agent-sandbox v1beta1
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* ci(kubernetes): test agent-sandbox api versions
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): retry agent-sandbox raw 404s
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): harden agent-sandbox api setup
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): cache agent-sandbox api version
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* docs(kubernetes): document agent-sandbox upgrade behavior
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* refactor(kubernetes): collapse agent-sandbox api selection
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* feat(server): support remote compute driver endpoints
Add named remote compute driver endpoint support to the gateway.
Remote drivers are selected by a non-reserved compute driver name and either a CLI/env socket endpoint or [openshell.drivers.<name>].socket_path. The VM driver now enters ComputeRuntime through the same acquired remote endpoint path, while Docker, Podman, and Kubernetes retain their in-process drivers.
Require --drivers/OPENSHELL_DRIVERS when pairing an ad-hoc socket endpoint so the socket does not imply a magic driver name, and keep reserved in-tree names unavailable for unmanaged socket endpoints.
Co-authored-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* test(server): cover remote compute driver UDS lifecycle
Signed-off-by: Evan Lezar <elezar@nvidia.com>
---------
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
* feat(gpu)!: add resource requirements
BREAKING CHANGE: SandboxSpec.gpu and DriverSandboxSpec.gpu were replaced with resource_requirements.gpu, changing protobuf field 9 from a bool to a message for both public and driver APIs.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* refactor(gpu): pass requirements through sandbox create
Pass the coupled GPU requirement object through the CLI sandbox_create boundary instead of splitting presence and count into separate arguments.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* refactor(gpu): pass requirements to timeout message
Pass ResourceRequirements into the provisioning timeout message helper so GPU hints are derived from the same nested request object used to create the sandbox.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* refactor(gpu): pass driver requirements through helpers
Thread Option<GpuResourceRequirements> through driver validation and rendering helpers instead of splitting GPU presence and count into separate arguments.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* fix(gpu): validate exact device requests
Require exact driver GPU device lists to be tied to a GPU request, allow a single exact device to use the default countless request, and require explicit matching counts for multi-device lists.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
---------
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* feat(bootstrap): add system gateway registry for installer defaults
Adds a read-only installer-seeded gateway registry that the CLI consults after per-user gateway config. The registry uses the same layout as per-user config with `active_gateway` at the root and `gateways/<name>/metadata.json` beneath it. By default the system config root is `/etc/openshell`, while `OPENSHELL_SYSTEM_GATEWAY_DIR` remains available as an override for packages that need a different location. User-managed gateways continue to shadow installer entries on name collision.
Originally-authored-by: Mark Shuttleworth <mark@ubuntu.com>
Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>
* feat(cli): show gateway config source in list and term
Expose whether a gateway registration comes from user or system config in `openshell gateway list`, the TUI gateway pane, and list JSON output. The CLI also refuses to remove system-managed registrations and the smoke tests cover the new list output.
Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>
* fix(bootstrap): preserve user shadowing on invalid metadata
Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>
* fix(cli): keep system fallback out of rollback state
* docs(gateway): describe system config fallback layout
* test(bootstrap): cover system gateway last_sandbox persistence
* test(gateway): cover system-only removal rejection
* fix(bootstrap): validate gateway names before path joins
---------
Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>
* feat(helm): support Deployment kind in HA gateway workloads
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(helm): handle null workload values
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* feat(podman): make container health check interval configurable
The Podman driver hardcoded a 3-second health check interval, which
spawns a conmon subprocess on every tick. On systems running multiple
sandboxes this creates sustained process churn and unnecessary CPU
overhead.
Add a `health_check_interval_secs` field to `PodmanComputeConfig`
(default: 10s) and wire it into the container health check spec.
Operators can tune it further via `[openshell.drivers.podman]` in
gateway.toml.
Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
* docs(podman): document health_check_interval_secs config field
Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
* docs(podman): document health_check_interval_secs zero-disables behavior
Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
---------
Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
* feat(providers): support SPIFFE-backed token grants
Add provider profile token_grant metadata and expand endpoint-specific
dynamic credentials so sandbox supervisors can request SPIFFE JWT-SVIDs,
exchange them with an OAuth-style token endpoint, cache returned access
tokens, and inject bearer tokens into matching HTTP requests.
Wire Kubernetes and Helm deployments to mount the provider SPIFFE Workload
API socket into sandbox pods for token grant exchange.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(examples): add SPIFFE token grant demo
Add a reusable alpha/beta demo that deploys a SPIFFE-verifying token issuer
and protected services, imports a token-grant provider profile, creates a
sandbox, and verifies endpoint-specific bearer tokens.
The script leaves Kubernetes workloads in place, deletes sandboxes through
openshell unless KEEP_SANDBOX=1, and prints protected service logs as proof
of life.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden SPIFFE token grants
* fix(providers): harden dynamic token grants
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden token grant handling
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
Allow operators to configure a default Kubernetes runtimeClassName that
is applied to sandbox pods when the CreateSandbox request does not
specify one. This avoids requiring every API caller to explicitly set the
runtime class for clusters that always need a specific RuntimeClass
(e.g. kata-containers, nvidia).
The fallback is applied in the Kubernetes driver only — per-request
values still take priority, and an empty default (the built-in) preserves
existing behavior (field omitted, cluster default applies).
Closes#1307
Default the Podman host gateway alias override to gvproxy's host-loopback IP on macOS while preserving host-gateway resolution on Linux. Wire the setting through Podman config, gateway TOML inheritance, and the standalone driver, and document the platform behavior.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Migrate all sandbox and VM driver network policy enforcement from
iptables to nftables. nftables provides atomic ruleset loading, a
cleaner rule syntax, and is the standard netfilter interface in modern
kernels.
Sandbox bypass enforcement (openshell-sandbox):
- Replace iptables chain of individual rule insertions with a single
atomic nftables ruleset load via nft -f
- New nft_ruleset module with pure functions for ruleset generation
and unit tests
- Combine log and reject rules in one inet family table (handles both
IPv4 and IPv6 in a single ruleset)
- Fall back to reject-only ruleset when kernel lacks nft_log support
- Enable net.netfilter.nf_log_all_netns so log rules work from
non-init network namespaces
- Use temp file for nft ruleset loading instead of stdin for
compatibility with minimal VM guest environments
VM TAP networking (openshell-driver-vm):
- Replace iptables NAT/forwarding rules with nftables equivalents
- New nft_ruleset module for TAP network rule generation with unit
tests
- Atomic table-per-TAP-device lifecycle (create/destroy)
- Host-side rules provide NAT infrastructure and defense-in-depth
isolation (input chain restricts VM to gateway port only, forward
chain blocks unsolicited inbound); primary security enforcement
happens inside the VM guest via the sandbox supervisor's own rules
VM init script:
- Load nft kernel modules at sandbox init
- Enable nf_log_all_netns sysctl for bypass detection logging
OCSF / docs:
- Update firewall rule engine references from iptables to nftables
- Document host firewall interaction model and two-layer enforcement
architecture in VM driver README and compute drivers reference
Closes#1335
Signed-off-by: Russell Bryant <rbryant@redhat.com>
* feat(gateway): add TOML configuration file (RFC 0003)
Introduces an opt-in --config / OPENSHELL_GATEWAY_CONFIG flag that loads a
TOML file with gateway-wide settings and per-driver tables. Source
precedence is CLI > env > file > built-in default, implemented via clap's
ValueSource so existing flags and env vars keep their priority.
Driver crates (kubernetes, docker, podman, vm) now derive Deserialize on
their config structs. SupervisorSideloadMethod gains Deserialize with
kebab-case rename. A per-driver inheritance allowlist on the loader side
overlays [openshell.gateway] shared defaults (default_image,
supervisor_image, image_pull_policy, guest_tls_*, ssh_handshake_skew_secs,
client_tls_secret_name, host_gateway_ip, enable_user_namespaces) onto
each [openshell.drivers.<name>] table before deserialization.
The Helm chart renders a new gateway-config ConfigMap and mounts it at
/etc/openshell/gateway.toml. The migrated OPENSHELL_* env entries are
dropped from the StatefulSet — only the Secret-backed
OPENSHELL_SSH_HANDSHAKE_SECRET remains. database_url stays on --db-url.
Adds examples/gateway/gateway.example.toml and updates architecture/gateway.md
with the source precedence and inheritance rules.
* docs(gateway): drop ssh_handshake_skew_secs and ssh_handshake_secret from examples
Both fields are scheduled for removal. Remove the example values and the
env-only note so the gateway.toml example and the architecture doc stop
recommending settings that will not exist much longer.
* docs(rfc): correct OPENSHELL_CONFIG to OPENSHELL_GATEWAY_CONFIG in RFC 0003
* docs(gateway): add per-driver TOML example configurations
Adds focused single-driver examples next to the comprehensive
gateway.example.toml: kubernetes, docker, podman, and microvm. Each one
demonstrates the realistic settings for that driver plus how shared
[openshell.gateway] defaults inherit into the driver table.
A new unit test (`checked_in_examples_parse`) loads every example through
the config_file loader so schema drift fails CI rather than silently
shipping a broken example.
* refactor(gateway): drop image_pull_policy from shared inheritance
Kubernetes and Podman use mutually-incompatible vocabularies for the same
TOML key:
- Kubernetes: `Always | IfNotPresent | Never` (free-form string passed
verbatim to the K8s API).
- Podman: `always | missing | never | newer` (strict lowercase enum
deserialised into `ImagePullPolicy`).
No value means the same thing in both drivers. Sharing the key at
`[openshell.gateway]` scope and inheriting it into every active driver's
table meant any value safe for one driver was either wrong or silently
dropped for the other (`IfNotPresent` → `ImagePullPolicy::Missing` after
`.unwrap_or_default()`). Operators run one driver per gateway, so the
"shared default" never pays for itself.
Make `image_pull_policy` driver-local:
- Remove the field from `GatewayFileSection` and from
`inheritable_keys()` for both Kubernetes and Podman.
- Drop the file→`RunArgs` merge for the gateway-scope key.
- Stop unconditionally clobbering the driver value with
`config.sandbox_image_pull_policy` in the runtime wiring — only apply
the CLI/env override when it was set (and, for Podman, only when it
parses into the lowercase enum).
- Move the key under `[openshell.drivers.kubernetes]` and
`[openshell.drivers.podman]` in every example, the RFC, the
architecture doc, and the Helm-rendered gateway ConfigMap.
The supervisor pull policy follows the same shape: it is K8s-only and
moves into `[openshell.drivers.kubernetes]` alongside `image_pull_policy`
in the Helm template.
* fix(gateway): address review feedback on TOML configuration
Resolves the P1 and P2 issues raised in PR #1317:
- Helm gateway ConfigMap moves `grpc_endpoint` under
`[openshell.drivers.kubernetes]` so the default install no longer fails
the gateway's `deny_unknown_fields` schema check.
- `kubernetes_config_from_file` and `podman_config_from_file` only let
the gateway-wide CLI/env `grpc_endpoint` overwrite the driver-table
value when it was actually supplied, preserving file-only configs.
- Kubernetes driver default `image_pull_policy` is now empty (was Podman
vocabulary "missing"), so default deployments let the Kubernetes API
apply its own policy instead of being rejected.
- New `disable_tls` gateway field plumbs `.Values.server.disableTls`
through the TOML ConfigMap instead of relying on env vars dropped from
the StatefulSet.
- StatefulSet pod template now carries a `checksum/gateway-config`
annotation so `helm upgrade` rolls pods when the ConfigMap changes.
- Auxiliary listener resolution preserves the full `SocketAddr` from
`health_bind_address` / `metrics_bind_address`, so a loopback-pinned
health port is not silently relocated onto the public bind address.
- `ssh_session_ttl_secs` from the file is now applied to `Config` (it
was previously accepted by the loader but never read).
New regression coverage: cli-level merge tests for the new fields plus
helm-unittest assertions for the ConfigMap shape, checksum annotation,
and `disable_tls` rendering.
* docs(gateway): consolidate gateway TOML examples into docs reference
Replaces the per-driver example files under examples/gateway/ with a
single published reference page at docs/reference/gateway-config.mdx
covering source precedence, layout, the full example, and the four
per-driver examples (Kubernetes, Docker, Podman, microVM). Drew flagged
during PR #1317 review that the examples belong with the user-facing
docs rather than in a sibling examples/ directory.
The cross-references in architecture/gateway.md and RFC 0003 are updated
to point at the new docs page; the round-trip test in config_file.rs is
removed (schema coverage stays on the inline parses_full_example test
and per-field merge tests — doc-snippet drift belongs in a separate
docs-lint, not in a cross-tree Rust unit test).
* refactor(core): move DEFAULT_K8S_NAMESPACE into K8s driver
The constant is Kubernetes-specific (used only by KubernetesComputeConfig's
Default impl) and does not belong in openshell-core. Relocate it to the
driver crate that owns the K8s vocabulary; openshell-core retains only
truly cross-cutting defaults.
* refactor(core): move Podman bridge default into Podman driver
DEFAULT_NETWORK_NAME is Podman vocabulary, consumed only by the Podman
driver. Also drops the unused DEFAULT_IMAGE_PULL_POLICY constant.
* docs(auth): scrub remaining SSH handshake secret references
Sweeps the trailing mentions left after the rebase: the gateway
config-file module doc, the Helm gateway-config ConfigMap header,
the gateway-config.mdx env-only note, and the RPM systemd unit
comment for init-gateway-env.sh.
* docs(gateway): clarify OPENSHELL_GRPC_ENDPOINT applies to all drivers
The previous comment implied the callback endpoint was Kubernetes-only,
but the value is propagated to every compute driver (Kubernetes, Docker,
Podman, VM) and must be reachable from wherever the sandbox runs.
* refactor(gateway): move driver options into config (#1394)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
* fix(e2e): regenerate gateway config via TOML for docker + podman harnesses
The gateway CLI flags moved into TOML config tables in 560550d2 (#1394),
which made every existing e2e/with-{docker,podman}-gateway.sh invocation
fail with "unexpected argument '--sandbox-namespace'" (and a long tail
of similar driver-specific options) before the gateway could even bind.
Replace the obsolete CLI flags with a synthesized
`[openshell.drivers.<driver>]` table written to `${STATE_DIR}/gateway.toml`
and passed via the new `--config` flag. Only the gateway-wide flags
that survived 560550d2 (bind-address, port, drivers, db-url, tls-*,
disable-tls, log-level, health-port) stay on the command line.
Both scripts get a small `toml_string` helper to properly TOML-quote
the values (the previous `%q` printf format produced bash-escape, not
TOML-escape). The Docker harness also corrects two field names that
diverged from the driver schema: `docker_network_name` →
`network_name`, and the supervisor binary/image plumbing now reads
through to `supervisor_bin` / `supervisor_image` in the same table.
The Podman harness drops `--ssh-gateway-port` (deleted in 560550d2 —
gRPC + SSH are multiplexed on the same port now) and substitutes
`network_name` + `gateway_port` for the obsolete `--sandbox-namespace`
(which the Podman driver never had as a typed field).
* fix(core): swap bind-only 0.0.0.0 SSH gateway host for cluster URL host
CLI's resolve_ssh_gateway treated 0.0.0.0 as a loopback "keep as-is"
when the cluster URL was also loopback, so the SSH proxy connected to
0.0.0.0:port. The unspecified address is never a valid connect target
and is not present in any TLS cert SAN, which produced BadCertificate
TLS handshake failures during `openshell sandbox create -- ...` in
docker/podman e2e (e.g. bypass_detection).
Resolution: when the server returns 0.0.0.0 or :: as the gateway host
and both endpoints are loopback, fall back to the cluster URL's host
(which the CLI is already using to reach the gateway, so it must
resolve and match the cert).
* fix(e2e): repair podman harness on macOS
Podman 5.x with the applehv/libkrun provider no longer creates the legacy
~/.local/share/containers/podman/machine/podman.sock symlink, and
`podman system service` is a Linux-only subcommand — the macOS client
delegates the API service to the VM. Both assumptions in the harness
were stale, so the script tried to start a temporary service that podman
rejected with "unknown flag: --time".
- Discover the macOS socket via `podman machine inspect` instead of the
hardcoded path.
- On Darwin, fail fast with a "start podman machine" message rather than
attempting the Linux-only `podman system service` fallback.
- Write socket_path into [openshell.drivers.podman] so the in-process
driver picks up the discovered socket; the driver reads TOML only
after the config refactor (560550d2), so OPENSHELL_PODMAN_SOCKET alone
was no longer enough.
* fix(server): use clone_from for TLS client CA assignment
clippy 1.95.0 rejects assigning the result of `Clone::clone()` to an
existing variable under `-D warnings` (`assigning_clones`). Switch to
`clone_from(&...)` to satisfy the lint and avoid the redundant
allocation.
---------
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
* test(e2e): add bypass detection test for sandbox REJECT rules
Validates that direct TCP connections bypassing the HTTP CONNECT proxy
are rejected immediately (ECONNREFUSED) rather than hanging until a
network timeout. This is implementation-agnostic — it tests observable
kernel behavior regardless of whether rules are installed via iptables
or nftables.
* feat(vm): fall back to Podman socket when Docker is unavailable
The VM driver resolves sandbox images from a local container engine
before falling back to registry pulls. Previously it only checked
Docker. Now it tries the Podman socket as a fallback, since Podman
exposes a Docker-compatible API via bollard.
On Linux, enable the Podman API socket with:
systemctl --user start podman.socket
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* chore(vm): fix formatting
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* refactor(vm): rename docker-specific identifiers to engine-neutral names
Rename resolve_local_docker_image → resolve_local_container_image,
local_docker_image_platform_mismatch → local_image_platform_mismatch,
and the local docker variable → engine since the function now supports
both Docker and Podman backends.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
---------
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* refactor!(auth): drop SSH handshake secret in favor of mTLS
The OPENSHELL_SSH_HANDSHAKE_SECRET / x-sandbox-secret mechanism was
misnamed: it does not authenticate SSH (which flows over the
RelayStream gRPC RPC and is gated by mTLS plus supervisor Unix-socket
permissions). It only gated a small set of sandbox-to-gateway
control-plane RPCs, and production deployments already enforce mTLS
on that channel — so the shared secret was redundant.
Replace the secret check with an mTLS-presence marker. Sandbox-class
methods (ReportPolicyStatus, PushSandboxLogs,
GetSandboxProviderEnvironment, SubmitPolicyAnalysis, GetSandboxConfig,
GetInferenceBundle) accept callers without a Bearer token; the gRPC
mTLS handshake is the trust boundary. Dual-auth methods treat
Bearer-present as full-scope CLI access and Bearer-absent as
sandbox-restricted scope via validate_sandbox_caller_update.
Drops the secret from all drivers (K8s, Podman, VM), the sandbox gRPC
interceptor, the Helm chart (values + pre-install hook + StatefulSet
env), the RPM bootstrap script, the man pages, and the
debug-openshell-cluster skill. Also removes the never-read
ssh_handshake_skew_secs flag and config field.
BREAKING CHANGE: --ssh-handshake-secret / OPENSHELL_SSH_HANDSHAKE_SECRET
and --ssh-handshake-skew-secs / OPENSHELL_SSH_HANDSHAKE_SKEW_SECS are
removed from the gateway, sandbox, and all driver binaries. The
openshell-ssh-handshake K8s Secret is no longer managed by the chart;
operators may delete the orphan. Deployments using
--disable-gateway-auth must enforce caller authentication at the
fronting proxy, since the gateway no longer validates a per-request
secret on sandbox-class methods.
Refs OS-174.
* docs(auth): scrub residual SSH handshake secret references
Sweep across docs, e2e scripts, the Podman driver README/NETWORKING
notes, the RPM/Helm/setup guides, the gateway man page, and RFC 0003
to remove instructions and examples that still referenced
OPENSHELL_SSH_HANDSHAKE_SECRET / --ssh-handshake-secret /
ssh_handshake_skew_secs. The mechanism is gone; nothing should still
suggest setting it. Negative-assertion regression tests are kept so
the env var cannot silently be re-introduced.
* test(drivers): drop SSH handshake secret negative-assertion tests
The supporting code, env vars, CLI flags, and config plumbing are
gone — these tests asserted absence of strings that no longer have
any path to being set. Remove the guards from the Docker, Podman,
Kubernetes, and VM driver test modules.
Piotr Mlocek