mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-16 05:20:22 +08:00
* feat(credentials): add provider credential storage drivers Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(credentials): harden credential update handling Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(credentials): harden credential driver security, correctness, and performance Address review findings from the credential storage drivers PR: - Route additional_credentials through the driver on refresh to prevent silent data loss for multi-credential providers (e.g. AWS STS) - Clean up stored credential handles on CAS failure during refresh to prevent orphaned secrets in external backends - Enforce namespace validation in the Kubernetes Secrets driver to prevent cross-namespace credential access when allow_reference_namespace is not enabled - Cache Vault Kubernetes auth tokens with 80% TTL to avoid re-authenticating on every credential operation - Parallelize resolve_credentials in all three drivers using try_join_all for faster sandbox startup - Add existingSecret support for the KEK Secret to fix helm template/GitOps workflows where lookup returns empty and regenerates the key - Document RBAC blast radius for the Kubernetes Secrets credential driver and recommend a dedicated namespace Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): add optimistic concurrency, fix thundering herd, parallelize operations Use resourceVersion optimistic concurrency with retry loop for K8s Secret ownership checks to prevent TOCTOU races. Switch Vault token cache from RwLock to Mutex with double-check pattern to prevent thundering herd on cache miss. Parallelize credential store and delete operations across independent keys using try_join_all. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): handle partial failures, add delete retry, consolidate cleanup Replace try_join_all with join_all in credential store/delete operations to handle partial failures — successfully-stored handles are cleaned up when another key fails. Add retry loop with conflict detection to db-credstore delete_credential, matching the K8s driver pattern. Consolidate 4 manual cleanup_pre_stored_provider_credentials call sites into a single error handler using an async block. Remove inconsistent .trim() from db-credstore validate_handle_owner. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): fix retry loop guard and remove unprotected validation Remove attempt-count guard from 409/Aborted match arms in retry loops so the post-loop Status::aborted error is reachable after exhausting retries. Previously, last-attempt conflicts fell through to the catch-all error arm, producing misleading Status::unavailable errors. Remove duplicate validation calls that ran after prepare_provider_credential_update but outside the cleanup-protected async block, which would leak pre-stored handles on failure. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): add workspace/provider UUID to credential backend paths Include workspace and provider ID in credential backend object paths to ensure cross-workspace uniqueness and prevent credential collision (GATOR-1806c9be-01). - Updated credential driver proto to include workspace and provider_id fields - Modified Vault driver to include workspace/provider_id in managed_secret_path - Modified Kubernetes Secrets driver to include workspace/provider_id in credential_owner_id and managed_secret_name - Updated all credential runtime calls to pass workspace/provider_id - Updated tests to use the new signatures This prevents two workspaces sharing the same external credential store from colliding on provider names, which was a critical security issue (CWE-639). * fix(credentials): preserve provider-level expiration for handle-backed credentials Compute effective expiration from both provider and driver values using the earliest non-zero timestamp and skip expired values before insertion (GATOR-1806c9be-02). - Modified resolve_provider_handles to check provider credential_expires_at_ms - Skip expired credentials during resolution instead of returning them - Use effective expiration (min of provider and driver) in resolution results - Fix inference.rs to preserve earliest expiration when merging This ensures handle-backed credentials respect the same expiration semantics as inline credentials. * fix(credentials): stage refresh changes under new handles before validation Stage credential replacements under new immutable handles instead of reusing existing handles to prevent overwriting committed values before validation/CAS (GATOR-1806c9be-03). - Stage credentials with empty existing_handles map to force new handle creation - Validate and CAS before the new values are committed to backend storage - Delete old handles only after successful CAS - On CAS failure, delete only the newly staged handles - This prevents CWE-362/CWE-367 race conditions where failed refreshes could still modify or delete the active credential The fix ensures that a rejected refresh cannot modify the backend object still referenced by the committed provider record. * fix(credentials): add timeouts to credential driver RPCs Apply configured timeouts to both startup capability negotiation and runtime RPCs to prevent indefinite hangs (GATOR-1806c9be-05). - Add DEFAULT_CREDENTIAL_DRIVER_RPC_TIMEOUT_SECS constant (30s) - Apply timeout to GetCapabilities during startup connection - Apply timeout to all runtime RPCs (store, delete, resolve) - Use tokio::time::timeout to bound the entire GetCapabilities operation during startup, not just the socket connection - Return contextual deadline errors on timeout This prevents a faulty or overloaded driver from hanging gateway operations indefinitely. * fix(credentials): fix test to use consistent workspace/provider identity The Kubernetes auth Vault resolve test was constructing a managed path with test-workspace/test-provider-id but sending default/prov-123 in the request, causing validation to reject the request (GATOR-18e32351-01). - Update test to use test-workspace and test-provider-id in the request to match the logical_path construction - This ensures the test exercises the intended code path and validates Kubernetes auth resolution properly The test now passes and correctly validates identity enforcement. * fix(credentials): use unique staging ID for refresh to avoid overwrites Stage refresh replacements under genuinely distinct immutable handles using a unique staging ID to prevent overwriting committed values (GATOR-1806c9be-03). - Generate a unique staging ID using UUID for each refresh operation - Use this staging ID when storing credentials instead of the real provider ID - Pass the same staging ID during cleanup on failure to delete only staged objects - This ensures deterministic paths (Vault) and object names (K8s) don't collide with the committed provider's credentials The fix prevents failed refreshes from silently replacing active credentials or breaking providers by deleting still-referenced backend objects. * fix(credentials): wrap credential driver RPCs in local timeouts Add local tokio::time::timeout wrappers around credential driver RPCs to bound non-compliant or stalled UDS peers (GATOR-1806c9be-05). - Wrap StoreCredential, DeleteCredential, and ResolveCredentials in local timeouts - Return contextual deadline_exceeded errors when timeouts occur - Keep existing gRPC timeout metadata for compliant implementations - GetCapabilities during startup was already wrapped in previous commit This ensures a faulty local driver cannot hang gateway operations indefinitely, even if it accepts the connection but never responds to the RPC. * fix(credentials): preserve ownership for staged refreshes Signed-off-by: Seth Jennings <sjenning@redhat.com> * fix(credentials): bound startup capability probe Signed-off-by: Seth Jennings <sjenning@redhat.com> * test(provider): authenticate credential handler requests Signed-off-by: Seth Jennings <sjenning@redhat.com> * fix(ci): grant actions read to credential driver e2e Signed-off-by: Seth Jennings <sjenning@redhat.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> Co-authored-by: Taylor Mutch <taylormutch@gmail.com> Co-authored-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
672 lines
39 KiB
Plaintext
672 lines
39 KiB
Plaintext
---
|
|
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
title: "Gateway Configuration File"
|
|
sidebar-title: "Gateway Config"
|
|
description: "Reference for the OpenShell gateway TOML configuration file (RFC 0003)."
|
|
keywords: "Generative AI, Cybersecurity, AI Agents, Sandboxing, Gateway, Configuration, TOML, Reference"
|
|
position: 5
|
|
---
|
|
|
|
The OpenShell gateway reads its configuration from a TOML file when `--config` or `OPENSHELL_GATEWAY_CONFIG` is set. When neither is set, the gateway reads `$XDG_CONFIG_HOME/openshell/gateway.toml` if that file exists. If no config file exists, the gateway starts from built-in defaults. Gateway process flags and gateway `OPENSHELL_*` environment variables override the file. Compute driver settings live in the driver TOML tables. See [RFC 0003](https://github.com/NVIDIA/OpenShell/blob/main/rfc/0003-gateway-configuration/README.md) for the full schema.
|
|
|
|
## Source Precedence
|
|
|
|
```text
|
|
Gateway CLI flag > gateway OPENSHELL_* env var > TOML file > built-in default
|
|
```
|
|
|
|
`database_url` is env-only. The loader rejects it when it appears in the file. When `OPENSHELL_DB_URL` is unset, the gateway stores its SQLite database under `$XDG_STATE_HOME/openshell/gateway/openshell.db`.
|
|
|
|
## Package-Managed Locations
|
|
|
|
Package-managed gateways do not require a TOML file. Create one at the package's optional config location when you need to override built-in defaults. Set `OPENSHELL_GATEWAY_CONFIG` in the launch environment to use a different file.
|
|
|
|
| Package | Optional Gateway TOML location |
|
|
|---|---|
|
|
| Homebrew | `$XDG_CONFIG_HOME/openshell/gateway.toml` when it exists, otherwise the Homebrew prefix config such as `/opt/homebrew/var/openshell/gateway.toml`. |
|
|
| Debian/Ubuntu | `$XDG_CONFIG_HOME/openshell/gateway.toml`, usually `~/.config/openshell/gateway.toml` for the systemd user service. |
|
|
| Fedora/RHEL RPM | `$XDG_CONFIG_HOME/openshell/gateway.toml`, usually `~/.config/openshell/gateway.toml` for the systemd user service. |
|
|
| Snap | `$SNAP_COMMON/gateway.toml`, usually `/var/snap/openshell/common/gateway.toml`. |
|
|
|
|
The Fedora/RHEL RPM template leaves `[openshell.gateway].bind_address` unset. The gateway therefore uses its built-in `127.0.0.1:17670` primary listener. The Podman driver negotiates separate, restricted listeners for sandbox callbacks, so the primary listener does not need a wildcard address. Set `bind_address` explicitly only when clients must reach the primary multiplexed API through another interface.
|
|
|
|
The Homebrew formula creates its prefix config once with `bind_address = "[::1]:17670"`. Keeping the primary API on IPv6 loopback leaves IPv4 loopback available for Podman Machine's sandbox callback listener. A user config takes precedence, and upgrades do not overwrite either config.
|
|
|
|
## Layout
|
|
|
|
The file is rooted at `[openshell]`. Gateway-wide settings live under `[openshell.gateway]`. Each compute driver owns its own `[openshell.drivers.<name>]` table. Credential drivers own `[openshell.credential_drivers.<name>]` tables. Shared compute-driver keys set at gateway scope are inherited into compute driver tables when not overridden.
|
|
|
|
```toml
|
|
[openshell]
|
|
version = 1
|
|
|
|
[openshell.gateway]
|
|
# ... gateway-wide settings ...
|
|
|
|
[openshell.gateway.tls]
|
|
# ... gateway listener TLS ...
|
|
|
|
[openshell.gateway.oidc]
|
|
# ... JWT bearer auth ...
|
|
|
|
[openshell.drivers.kubernetes]
|
|
# ... driver-specific settings ...
|
|
|
|
[openshell.credential_drivers.kubernetes-secrets]
|
|
# ... credential-driver-specific settings ...
|
|
```
|
|
|
|
## Full Example
|
|
|
|
A complete gateway configuration covering every section. Trim to the fields you need.
|
|
|
|
```toml
|
|
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
|
|
[openshell]
|
|
version = 1
|
|
|
|
[openshell.gateway]
|
|
bind_address = "0.0.0.0:8080"
|
|
health_bind_address = "0.0.0.0:8081"
|
|
metrics_bind_address = "0.0.0.0:9090"
|
|
|
|
log_level = "info"
|
|
|
|
# When empty, the gateway auto-detects Kubernetes, then Podman, then Docker.
|
|
# VM is never auto-detected and requires an explicit entry here.
|
|
compute_drivers = ["kubernetes"]
|
|
|
|
# Optional external provider credential storage backend. Omit this key to use
|
|
# the gateway's default encrypted database credential storage.
|
|
credential_drivers = ["kubernetes-secrets"]
|
|
|
|
sandbox_namespace = "openshell"
|
|
ssh_session_ttl_secs = 3600
|
|
|
|
# Reject invalid policy generations securely by default. Set
|
|
# "retain_last_valid" only when availability takes priority.
|
|
policy_validation_failure_mode = "fail_closed"
|
|
|
|
# Subject Alternative Names baked into the gateway server certificate.
|
|
# Wildcard DNS SANs (e.g. "*.dev.openshell.localhost") also enable sandbox
|
|
# service URLs under that domain.
|
|
server_sans = ["openshell", "*.dev.openshell.localhost"]
|
|
# Allow plaintext HTTP routing for loopback sandbox service URLs.
|
|
enable_loopback_service_http = true
|
|
|
|
# Set true only for local plaintext gateways or trusted TLS termination.
|
|
disable_tls = false
|
|
|
|
# Shared driver defaults. These inherit into [openshell.drivers.<name>] tables
|
|
# when the driver-specific table does not override them.
|
|
default_image = "ghcr.io/nvidia/openshell/sandbox:latest"
|
|
# Defaults to the gateway version; override to pin a specific build.
|
|
# supervisor_image = "ghcr.io/nvidia/openshell/supervisor:<version>"
|
|
client_tls_secret_name = "openshell-client-tls"
|
|
service_account_name = "openshell-sandbox"
|
|
host_gateway_ip = "10.0.0.1"
|
|
enable_user_namespaces = false
|
|
sa_token_ttl_secs = 3600
|
|
guest_tls_ca = "/etc/openshell/certs/ca.pem"
|
|
guest_tls_cert = "/etc/openshell/certs/client.pem"
|
|
guest_tls_key = "/etc/openshell/certs/client-key.pem"
|
|
|
|
# Optional gRPC rate limit. Both values must be positive to enable the limit.
|
|
# Set either value to 0, or omit both, to disable rate limiting.
|
|
grpc_rate_limit_requests = 120
|
|
grpc_rate_limit_window_seconds = 60
|
|
|
|
# Optional exact provider-profile source composition. When omitted, the
|
|
# gateway uses builtin + user.
|
|
provider_profile_sources = [
|
|
{ type = "builtin" },
|
|
{ type = "user" },
|
|
]
|
|
|
|
# Operator-run supervisor middleware. The gRPC endpoint must be reachable from
|
|
# both the gateway and sandbox supervisors.
|
|
[[openshell.supervisor.middleware]]
|
|
name = "local-content-guard"
|
|
grpc_endpoint = "http://host.openshell.internal:50051"
|
|
max_body_bytes = 262144
|
|
timeout = "500ms"
|
|
|
|
# Gateway listener TLS (distinct from the per-driver guest_tls_*).
|
|
[openshell.gateway.tls]
|
|
cert_path = "/etc/openshell/certs/gateway.pem"
|
|
key_path = "/etc/openshell/certs/gateway-key.pem"
|
|
client_ca_path = "/etc/openshell/certs/client-ca.pem"
|
|
require_client_auth = false
|
|
|
|
[openshell.gateway.gateway_jwt]
|
|
signing_key_path = "/etc/openshell/jwt/signing.pem"
|
|
public_key_path = "/etc/openshell/jwt/public.pem"
|
|
kid_path = "/etc/openshell/jwt/kid"
|
|
gateway_id = "openshell"
|
|
# Omit or set to 0 only for local single-player Docker, Podman, or VM gateways.
|
|
ttl_secs = 3600
|
|
|
|
[openshell.gateway.auth]
|
|
allow_unauthenticated_users = false
|
|
|
|
[openshell.gateway.mtls_auth]
|
|
enabled = false
|
|
|
|
# OTLP export. Omit this table entirely to disable it.
|
|
[openshell.gateway.otlp]
|
|
endpoint = "http://otel-collector.observability.svc:4317"
|
|
service_name = "openshell-gateway"
|
|
|
|
[openshell.gateway.oidc]
|
|
issuer = "https://idp.example.com/realms/openshell"
|
|
audience = "openshell-cli"
|
|
jwks_ttl_secs = 3600
|
|
roles_claim = "realm_access.roles"
|
|
admin_role = "openshell-admin"
|
|
user_role = "openshell-user"
|
|
scopes_claim = ""
|
|
|
|
[[openshell.gateway.interceptors]]
|
|
name = "quota"
|
|
grpc_endpoint = "unix:///run/openshell/interceptors/quota.sock"
|
|
order = 10
|
|
failure_policy = "fail_closed"
|
|
binding_policy = "allowlist"
|
|
timeout = "500ms"
|
|
max_response_bytes = 1048576
|
|
max_patches = 32
|
|
|
|
[[openshell.gateway.interceptors.bindings]]
|
|
rpc = "openshell.v1.OpenShell/CreateSandbox"
|
|
phases = ["modify_operation", "validate"]
|
|
failure_policy = "fail_closed"
|
|
|
|
[[openshell.gateway.interceptors.bindings]]
|
|
rpc = "openshell.v1.OpenShell/UpdateConfig"
|
|
phases = ["validate"]
|
|
|
|
[openshell.credential_drivers.kubernetes-secrets]
|
|
namespace = "openshell"
|
|
allow_reference_namespace = false
|
|
```
|
|
|
|
Local Docker, Podman, and VM gateways can also set `[openshell.gateway.mtls_auth] enabled = true` to authenticate CLI callers from verified client certificates. Kubernetes deployments must leave this unset and use OIDC or a trusted access proxy; the Helm chart does not render this table.
|
|
|
|
`[openshell.gateway] policy_validation_failure_mode` controls what sandbox supervisors do when a complete candidate policy fails runtime validation. The default, `fail_closed`, deactivates the previous network policy, closes relays pinned to it, and denies new egress until a valid generation loads. `retain_last_valid` leaves the previous valid generation active. Both modes reject the candidate atomically; startup always fails closed when no previous valid generation exists. Gateway mutation paths that can preflight a known effective scope reject invalid candidates before persistence and leave the active policy unchanged regardless of this setting. Changing the value requires restarting the gateway so it can reload `gateway.toml` and distribute the new posture to sandbox supervisors.
|
|
|
|
`[openshell.gateway.gateway_jwt] ttl_secs` controls gateway-minted sandbox JWT lifetime. When omitted, it defaults to `0`: the token `exp` claim and `expires_at_ms` response field become `0`, and the sandbox JWT does not expire. Use that default only for local single-player Docker, Podman, or VM gateways. Kubernetes and other shared deployments should set a positive TTL; Helm renders `3600` seconds by default, and the gateway logs a warning when a Kubernetes gateway uses `0`.
|
|
|
|
`[openshell.gateway.auth] allow_unauthenticated_users = true` is an unsafe local-development and trusted-proxy escape hatch. It accepts user-facing CLI/API calls without OIDC or mTLS credentials while sandbox supervisors still authenticate with gateway-minted sandbox JWTs. Leave it false for shared and production gateways.
|
|
|
|
## OTLP Export
|
|
|
|
`[openshell.gateway.otlp]` enables OpenTelemetry export over OTLP/gRPC. Omit the table to disable export; there is no separate `enabled` flag.
|
|
|
|
The gateway already uses the Rust `tracing` framework for structured logs sent to stdout and the sandbox log stream. Enabling this section adds an OpenTelemetry layer to the same tracing subscriber. It exports span trees to an OTLP collector without exporting, replacing, or redirecting the existing log events.
|
|
|
|
```toml
|
|
[openshell.gateway.otlp]
|
|
endpoint = "http://otel-collector.observability.svc:4317"
|
|
service_name = "openshell-gateway"
|
|
```
|
|
|
|
`endpoint` is required and must be a valid URI. If it is malformed, the gateway logs the configuration error and continues with export disabled. It does not connect at startup: an unreachable collector produces export failures, never a failure to serve.
|
|
|
|
The transport is **OTLP over gRPC only**. HTTP/protobuf and HTTP/JSON are not supported, and `OTEL_EXPORTER_OTLP_PROTOCOL` has no effect. Point `endpoint` at a collector's gRPC receiver, conventionally port `4317`, not the HTTP receiver on `4318`. A URI alone cannot distinguish the two, so an HTTP endpoint is accepted at startup and then fails on export.
|
|
|
|
The OpenTelemetry SDK logs export failures after startup. Spans in a failed batch are dropped rather than retried.
|
|
|
|
`service_name` sets the gateway's `service.name` resource attribute and defaults to `openshell-gateway`. The gateway also reports `service.version`.
|
|
|
|
Only OpenTelemetry traces are exported. Inbound gRPC and HTTP requests produce server spans named for the RPC or HTTP method. Store and compute-driver operations appear as child spans. Internal reconciliation, credential-refresh, and driver-watch loops create operation roots for their store work because no inbound request supplies a parent. The gateway continues valid W3C `traceparent` context and starts a new trace when none is supplied. Request spans carry `method`, `path`, and the `request_id` that also appears in gateway logs. Health endpoint spans use DEBUG level and are not exported by the default INFO filter.
|
|
|
|
The gateway forwards the OTLP configuration to managed external drivers. Each driver exports under its own service name.
|
|
|
|
### Tuning
|
|
|
|
This table decides whether and where to export. How the SDK exports is controlled by the standard OpenTelemetry environment variables, which the gateway reads through the SDK rather than mirroring as TOML keys:
|
|
|
|
| Variable | Effect |
|
|
|---|---|
|
|
| `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG` | Sampling strategy and ratio. Defaults to `parentbased_always_on`. |
|
|
| `OTEL_BSP_SCHEDULE_DELAY`, `OTEL_BSP_MAX_QUEUE_SIZE`, `OTEL_BSP_MAX_EXPORT_BATCH_SIZE`, `OTEL_BSP_EXPORT_TIMEOUT` | Batch span processor tuning. |
|
|
| `OTEL_RESOURCE_ATTRIBUTES` | Additional resource attributes, such as `deployment.environment=prod`. |
|
|
| `OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT`, `OTEL_SPAN_EVENT_COUNT_LIMIT`, `OTEL_SPAN_LINK_COUNT_LIMIT` | Per-span limits. |
|
|
| `OTEL_EXPORTER_OTLP_HEADERS`, `OTEL_EXPORTER_OTLP_COMPRESSION`, `OTEL_EXPORTER_OTLP_TIMEOUT` | Exporter transport tuning. |
|
|
| `OTEL_EXPORTER_OTLP_PROTOCOL` | No effect. The gateway is built with the gRPC exporter only. |
|
|
|
|
To sample 10% of traces:
|
|
|
|
```shell
|
|
OTEL_TRACES_SAMPLER=parentbased_traceidratio OTEL_TRACES_SAMPLER_ARG=0.1
|
|
```
|
|
|
|
`OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` are deliberately ignored. Enablement has one source, so an environment variable cannot silently turn export on or redirect it. `OTEL_RESOURCE_ATTRIBUTES` does apply and adds attributes, but a `service_name` set here wins over `OTEL_SERVICE_NAME`.
|
|
|
|
The gateway flushes buffered spans during shutdown, so spans from in-flight requests survive a `SIGTERM`.
|
|
|
|
## Supervisor Middleware Services
|
|
|
|
Register operator-run supervisor middleware services with one or more `[[openshell.supervisor.middleware]]` entries. Registration is static and operator-owned; changing it requires restarting the gateway.
|
|
|
|
```toml
|
|
[[openshell.supervisor.middleware]]
|
|
name = "local-content-guard"
|
|
grpc_endpoint = "http://host.openshell.internal:50051"
|
|
max_body_bytes = 262144
|
|
timeout = "500ms"
|
|
```
|
|
|
|
Each service implements the supervisor middleware gRPC contract and exposes bindings through `Describe`. Policies reference the operator-owned registration `name`, attaching the complete middleware and all of its bindings. Bindings are identified by operation and phase. A manifest may expose at most one binding for each operation and phase pair; V1 supports only `HttpRequest/pre_credentials`, so a service currently exposes one binding. Registration names must be unique, and operator-run registrations cannot claim the reserved `openshell/` namespace. The service-reported manifest name is diagnostic metadata and does not need to match the registration name.
|
|
|
|
The gateway connects to every registered service and validates `Describe` before it starts. The service must therefore be running before the gateway. Policy creation and full policy updates call `ValidateConfig`; an unavailable service or invalid middleware configuration rejects the policy before persistence.
|
|
|
|
`max_body_bytes` is the operator limit for every binding exposed by the service. It must be greater than zero, no larger than each binding's advertised limit, and no larger than the 4 MiB platform maximum. OpenShell rejects an oversized value instead of silently clamping it. Middleware gRPC servers should allow messages of at least 4 MiB plus 293 KiB so a maximum-size body and its protobuf envelope fit on the transport.
|
|
|
|
`timeout` is the operator-configured service-wide RPC timeout. It accepts the same compact duration syntax as gateway interceptors: an integer followed by `ms` or `s`, such as `500ms` or `2s`. Values must be between `10ms` and `30s`, inclusive. Omit the field to use the 500 ms platform default. A binding may advertise a shorter `timeout` in the `Describe` manifest, but it cannot extend the operator-configured deadline; OpenShell uses the smaller value. OpenShell validates both levels before accepting the service. The effective timeout covers `ValidateConfig` and `EvaluateHttpRequest`; `Describe` uses the service timeout because binding metadata is not available yet.
|
|
|
|
The service `grpc_endpoint` currently supports plaintext `http://` and TLS `https://` using the platform trust store. Custom trust roots, client authentication, health checks, and runtime registration are not currently supported. The endpoint must be reachable from both the gateway and sandbox supervisors; use `host.openshell.internal` or another shared address that can be resolved in both places.
|
|
|
|
See [Supervisor Middleware](/extensibility/supervisor-middleware) for selection, failure, body-limit, and operational guidance.
|
|
|
|
## Gateway Interceptors
|
|
|
|
`[[openshell.gateway.interceptors]]` configures gateway-side interceptor services. The gateway calls each service's `Describe` RPC at startup, validates its declared OpenShell RPC bindings against the compiled service descriptor, and applies matching phases from a central gRPC middleware path. Interceptors can target only methods in the gateway's built-in allowlist of unary mutation RPCs. New RPCs are non-interceptable until they are deliberately added to that allowlist; adding one does not require handler-specific interceptor code. Request bodies are exposed as protobuf JSON objects. Fields marked secret in the protobuf schema are recursively omitted from requests and post-commit responses. Interceptors cannot patch an omitted field or a containing object.
|
|
|
|
`binding_policy` controls how the manifest and operator binding configuration combine:
|
|
|
|
- `dynamic` enables valid manifest bindings and treats configured entries as optional narrowing overrides. This is the compatibility default. The gateway logs a startup warning because the interceptor controls its non-secret RPC authority.
|
|
- `allowlist` enables only configured RPC selectors and phases. The gateway ignores and logs extra manifest declarations, but fails startup when the manifest omits a configured RPC or phase.
|
|
- `exact` requires the configured and manifest RPC selectors and phases to match exactly.
|
|
|
|
Bindings under `allowlist` and `exact` require `rpc` or `service` plus `method`, and a nonempty `phases` list. They match by RPC rather than manifest binding ID. Duplicate selectors, `id`, and `disabled` are invalid in these modes; omit a binding to disable it. Binding failure policy comes from the binding, then the configured service, then defaults to `fail_closed`. Manifest failure policies do not override strict-mode operator configuration.
|
|
|
|
Bindings that include `post_commit` must resolve to `failure_policy = "fail_open"`; the gateway rejects fail-closed post-commit configuration at startup. Post-commit evaluation is observational. If response observation or interceptor evaluation fails after a handler succeeds, the gateway logs and counts the failure but returns the committed response unchanged.
|
|
|
|
`provider_profile_sources` selects the exact ordered provider-profile source set. Omit it to use the built-in and user-managed sources. Use `{ type = "builtin" }`, `{ type = "user" }`, and `{ type = "interceptor", name = "<configured-interceptor-name>" }` entries to compose a catalog. An interceptor entry must reference a configured service whose manifest advertises `provider_profiles = true`. Selecting only an interceptor makes that catalog authoritative; include local sources explicitly to compose them. Empty or duplicate source lists, unknown interceptor names, and duplicate normalized profile IDs fail closed. Source order controls collection and diagnostics, not override precedence.
|
|
|
|
For an authoritative interceptor catalog, select only that interceptor:
|
|
|
|
```toml
|
|
[openshell.gateway]
|
|
provider_profile_sources = [
|
|
{ type = "interceptor", name = "provider-governance" },
|
|
]
|
|
```
|
|
|
|
The gateway validates snapshot structure and provider-profile semantics. It treats a configured interceptor as the source trust boundary and does not verify signature, hash, or key annotations in profile payloads.
|
|
|
|
`failure_policy` accepts `fail_closed` or `fail_open`. `timeout` accepts `ms` and `s` suffixes. In `dynamic` mode, binding overrides may select a manifest binding by `id`, `rpc`, or `service` plus `method`; they can disable a binding, narrow its phases, or override its failure policy.
|
|
|
|
`image_pull_policy` is intentionally not a shared gateway key. Kubernetes and Docker use `Always`, `IfNotPresent`, or `Never`. Podman uses `always`, `missing`, `never`, or `newer`. Set it inside the relevant driver table.
|
|
|
|
## Credential Drivers
|
|
|
|
Set `credential_drivers` only when the gateway should store provider credentials in an external credential backend. OpenShell supports at most one enabled credential driver at a time. When `credential_drivers` is omitted, the gateway uses its default encrypted database credential storage. `credential_drivers = []` is invalid in the TOML file; omit the field for the default encrypted store, or select a backend such as `kubernetes-secrets` or `vault`.
|
|
|
|
Credential driver tables are backend-owned and live under `[openshell.credential_drivers.<name>]`. Built-in drivers default to in-tree transport, so they do not need a `transport` field. Use `transport = "uds"` with an absolute `socket_path` only for a remote gRPC driver over a Unix domain socket.
|
|
|
|
```toml
|
|
[openshell.gateway.credential_storage]
|
|
key_encryption_key_path = "/var/lib/openshell/credentials/key-encryption-key.bin"
|
|
```
|
|
|
|
For Kubernetes Secrets:
|
|
|
|
```toml
|
|
[openshell.gateway]
|
|
credential_drivers = ["kubernetes-secrets"]
|
|
|
|
[openshell.credential_drivers.kubernetes-secrets]
|
|
namespace = "openshell"
|
|
```
|
|
|
|
For Vault instead:
|
|
|
|
```toml
|
|
[openshell.gateway]
|
|
credential_drivers = ["vault"]
|
|
|
|
[openshell.credential_drivers.vault]
|
|
address = "http://vault.vault.svc.cluster.local:8200"
|
|
mount = "secret"
|
|
kv_version = "2"
|
|
auth_method = "kubernetes"
|
|
role = "openshell-gateway"
|
|
service_account_token_path = "/var/run/secrets/kubernetes.io/serviceaccount/token"
|
|
```
|
|
|
|
For the default encrypted database store, OpenShell stores provider credentials as JSON envelopes encrypted with AES-256-GCM in the gateway database. Each credential gets a random data-encryption key; the gateway wraps that key with a local key-encryption key. By default, the key-encryption key is created at `$XDG_STATE_HOME/openshell/gateway/credentials/key-encryption-key.bin` with owner-only permissions. Use `[openshell.gateway.credential_storage] key_encryption_key_env` instead of `key_encryption_key_path` to load a base64-encoded 32-byte key-encryption key from an environment variable. Back up the database and key-encryption key together; losing either makes stored credentials unrecoverable. In Kubernetes, the Helm chart creates a retained Secret containing the shared key-encryption key, injects it as `OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY`, and renders `key_encryption_key_env` for the gateway when no external credential driver is enabled. Multi-replica deployments need every replica to use the same database and key-encryption key; the chart default handles the key-encryption key side.
|
|
|
|
For GitOps and `helm template` workflows where `lookup` returns empty, the chart-generated KEK Secret gets a random value on every render, making credentials unrecoverable. Set `server.credentialStorage.existingSecret` to the name of a pre-provisioned Secret containing the key-encryption key under the `key-encryption-key` data key. When set, the chart skips KEK Secret generation and references the provided Secret directly.
|
|
|
|
```yaml
|
|
server:
|
|
credentialStorage:
|
|
existingSecret: my-preprovisioned-kek-secret
|
|
```
|
|
|
|
For `kubernetes-secrets`, `namespace` sets where OpenShell-managed provider Secret objects are stored. When omitted, the driver uses the in-cluster ServiceAccount namespace when available, otherwise `default`. The Helm chart creates a Role granting the gateway access to all Secrets in the credential namespace because OpenShell-managed Secret names are dynamic SHA-256 hashes that cannot be restricted with `resourceNames`. Deploy credential Secrets in a dedicated namespace (`server.credentialDrivers.kubernetesSecrets.namespace`) to limit the RBAC blast radius.
|
|
|
|
For `vault`, `address` points at the Vault service, `mount` and `kv_version` describe the KV engine where OpenShell-managed provider secrets are stored, and `auth_method = "kubernetes"` logs in with the gateway Pod's ServiceAccount token. For local or development validation, use `auth_method = "token_file"` with `token_path = "/path/to/token"`. Do not put literal Vault tokens in TOML.
|
|
|
|
Provider records that already contain inline database credentials remain readable for upgrade compatibility. New provider create/update requests still submit credential values through the normal API, but the gateway stores those values through the active credential storage path and persists only handles.
|
|
|
|
For remote credential drivers, set `transport = "uds"` with `socket_path`. Omit `command`, `args`, and `startup_timeout_secs` when another service manager prestarts the driver socket. Keep backend tokens out of TOML; point the driver at mounted token files or native identity mechanisms instead.
|
|
|
|
The built-in `kubernetes-secrets` and `vault` drivers can also run out of
|
|
process over UDS. Set `command` to the standalone driver binary and pass
|
|
driver-specific settings through `args`; the gateway appends `--bind-socket
|
|
<socket_path>` when it launches the process.
|
|
|
|
```toml
|
|
[openshell.gateway]
|
|
credential_drivers = ["kubernetes-secrets"]
|
|
|
|
[openshell.credential_drivers.kubernetes-secrets]
|
|
transport = "uds"
|
|
socket_path = "/run/openshell/credential-drivers/kubernetes-secrets.sock"
|
|
command = "/usr/libexec/openshell/openshell-driver-kubernetes-secrets"
|
|
args = ["--namespace", "openshell"]
|
|
```
|
|
|
|
```toml
|
|
[openshell.gateway]
|
|
credential_drivers = ["vault"]
|
|
|
|
[openshell.credential_drivers.vault]
|
|
transport = "uds"
|
|
socket_path = "/run/openshell/credential-drivers/vault.sock"
|
|
command = "/usr/libexec/openshell/openshell-driver-vault"
|
|
args = [
|
|
"--address", "http://vault.vault.svc.cluster.local:8200",
|
|
"--auth-method", "kubernetes",
|
|
"--role", "openshell-gateway",
|
|
]
|
|
```
|
|
|
|
## Driver References
|
|
|
|
Each example is a complete TOML file for one compute driver. The examples repeat `[openshell]` and `[openshell.gateway]` so they stay copyable, and the driver tables list the accepted driver-specific keys. Driver-specific values override inherited gateway defaults. The gateway rejects unknown driver fields after inheritance is merged.
|
|
|
|
### Kubernetes
|
|
|
|
The gateway runs as a Pod and creates sandbox Pods in another namespace. mTLS material for sandboxes is delivered through a Kubernetes Secret rather than host-side file paths.
|
|
|
|
```toml
|
|
[openshell]
|
|
version = 1
|
|
|
|
[openshell.gateway]
|
|
bind_address = "0.0.0.0:8080"
|
|
health_bind_address = "0.0.0.0:8081"
|
|
metrics_bind_address = "0.0.0.0:9090"
|
|
log_level = "info"
|
|
compute_drivers = ["kubernetes"]
|
|
|
|
[openshell.gateway.tls]
|
|
cert_path = "/etc/openshell-tls/server/tls.crt"
|
|
key_path = "/etc/openshell-tls/server/tls.key"
|
|
client_ca_path = "/etc/openshell-tls/client-ca/ca.crt"
|
|
|
|
[openshell.drivers.kubernetes]
|
|
namespace = "agents"
|
|
service_account_name = "openshell-sandbox"
|
|
default_image = "ghcr.io/nvidia/openshell/sandbox:latest"
|
|
image_pull_policy = "IfNotPresent"
|
|
image_pull_secrets = ["regcred"]
|
|
# Defaults to the gateway version; override to pin a specific build.
|
|
# supervisor_image = "ghcr.io/nvidia/openshell/supervisor:<version>"
|
|
supervisor_image_pull_policy = "IfNotPresent"
|
|
# Use the image volume on Kubernetes >= 1.35 (GA in 1.36); switch to "init-container"
|
|
# on older clusters or where the ImageVolume feature gate is off.
|
|
supervisor_sideload_method = "image-volume"
|
|
# "combined" runs the existing single supervisor container with full process,
|
|
# filesystem, and network enforcement in the agent container. "sidecar" moves
|
|
# pod-level network enforcement and gateway session handling into a network sidecar.
|
|
topology = "combined"
|
|
grpc_endpoint = "https://openshell-gateway.agents.svc:8080"
|
|
ssh_socket_path = "/run/openshell/ssh.sock"
|
|
client_tls_secret_name = "openshell-client-tls"
|
|
host_gateway_ip = "10.0.0.1"
|
|
enable_user_namespaces = false
|
|
app_armor_profile = "Unconfined"
|
|
workspace_default_storage_size = "10Gi"
|
|
# Kubernetes StorageClass for the workspace PVC. Empty (default) omits the
|
|
# field, using the cluster's default StorageClass. Set this on clusters with no
|
|
# default StorageClass, otherwise the workspace PVC stays Pending.
|
|
# workspace_storage_class = "fast-ssd"
|
|
# Kubernetes RuntimeClass applied to sandbox pods when the API request does
|
|
# not specify one. Empty (default) = omit the field, using the cluster default.
|
|
# default_runtime_class_name = "kata-containers"
|
|
# Kubelet clamps projected tokens below 600 seconds. The driver caps values at 86400.
|
|
sa_token_ttl_secs = 3600
|
|
# Optional SPIFFE Workload API socket mounted into sandbox pods for dynamic
|
|
# provider token grants. Use an absolute path under a dedicated directory;
|
|
# shared roots such as /run, /var, /tmp, and /etc are rejected.
|
|
# Supervisor-to-gateway auth still uses gateway JWTs.
|
|
provider_spiffe_workload_api_socket_path = "/spiffe-workload-api/spire-agent.sock"
|
|
# Explicit sandbox UID/GID for the supervisor container securityContext and
|
|
# PVC init container. When unset, the driver auto-detects from OpenShift SCC
|
|
# namespace annotations (openshift.io/sa.scc.uid-range) if present, falling
|
|
# back to 1000 on non-OpenShift clusters.
|
|
# sandbox_uid = 1500
|
|
# sandbox_gid = 1500
|
|
|
|
[openshell.drivers.kubernetes.sidecar]
|
|
# UID used by relaxed long-running network sidecars. Strict process/binary-aware
|
|
# sidecars run as UID 0 so Kubernetes grants the required /proc inspection
|
|
# capabilities into the effective set. In sidecar topology the network init
|
|
# container installs nftables rules that exempt the effective sidecar UID.
|
|
proxy_uid = 1337
|
|
# Keep process/binary-aware network policy enabled in sidecar topology. Set
|
|
# false to run the sidecar as proxy_uid, drop the sidecar's extra /proc
|
|
# inspection capabilities, and enforce endpoint/L7 policy without matching
|
|
# policy.binaries.
|
|
process_binary_aware_network_policy = true
|
|
```
|
|
|
|
### Docker
|
|
|
|
Sandboxes run as containers on a local bridge network. The supervisor binary is bind-mounted from the host (no in-cluster image pull required); guest mTLS material is supplied as host paths.
|
|
|
|
```toml
|
|
[openshell]
|
|
version = 1
|
|
|
|
[openshell.gateway]
|
|
bind_address = "127.0.0.1:17670"
|
|
log_level = "info"
|
|
compute_drivers = ["docker"]
|
|
|
|
[openshell.drivers.docker]
|
|
socket_path = "/var/run/docker.sock"
|
|
default_image = "ghcr.io/nvidia/openshell/sandbox:latest"
|
|
# Docker vocabulary: Always | IfNotPresent | Never. Empty behaves like IfNotPresent.
|
|
image_pull_policy = "IfNotPresent"
|
|
sandbox_namespace = "docker-dev"
|
|
# Empty auto-detects https://host.openshell.internal:<gateway-port> when guest TLS is set.
|
|
grpc_endpoint = "https://host.openshell.internal:17670"
|
|
# Skip the image-pull-and-extract step by pointing at a locally built binary.
|
|
supervisor_bin = "/usr/local/libexec/openshell/openshell-sandbox"
|
|
# When supervisor_bin is omitted, Docker extracts /openshell-sandbox from this image.
|
|
# Defaults to the gateway version; override to pin a specific build.
|
|
# supervisor_image = "ghcr.io/nvidia/openshell/supervisor:<version>"
|
|
guest_tls_ca = "/etc/openshell/certs/ca.pem"
|
|
guest_tls_cert = "/etc/openshell/certs/client.pem"
|
|
guest_tls_key = "/etc/openshell/certs/client-key.pem"
|
|
network_name = "openshell-docker"
|
|
host_gateway_ip = "172.17.0.1"
|
|
ssh_socket_path = "/run/openshell/ssh.sock"
|
|
# Unsafe operator override. Host bind mounts, including Docker local-driver
|
|
# bind-backed volumes, expose gateway-host paths inside sandboxes and can
|
|
# negate OpenShell isolation and filesystem controls.
|
|
enable_bind_mounts = false
|
|
# Set to 0 to leave Docker's runtime default unchanged.
|
|
sandbox_pids_limit = 2048
|
|
```
|
|
|
|
### Podman
|
|
|
|
Sandboxes run as Podman containers on a user-mode bridge network. The supervisor image is mounted read-only via Podman's `type=image` mount; guest mTLS material is supplied as host paths.
|
|
|
|
```toml
|
|
[openshell]
|
|
version = 1
|
|
|
|
[openshell.gateway]
|
|
bind_address = "127.0.0.1:17670"
|
|
log_level = "info"
|
|
compute_drivers = ["podman"]
|
|
|
|
[openshell.drivers.podman]
|
|
# Rootless socket path. For root Podman use /run/podman/podman.sock.
|
|
# Omit to auto-detect: the driver probes for a responsive Podman socket and
|
|
# fails to start if none respond.
|
|
socket_path = "/run/user/1000/podman/podman.sock"
|
|
default_image = "ghcr.io/nvidia/openshell/sandbox:latest"
|
|
image_pull_policy = "missing" # always | missing | never | newer
|
|
grpc_endpoint = "https://host.containers.internal:17670"
|
|
# The gateway overwrites gateway_port from bind_address at runtime.
|
|
gateway_port = 17670
|
|
network_name = "openshell"
|
|
# Omit for the platform default: empty on Linux, 192.168.127.254 on macOS Podman machine.
|
|
# Set "" to force Podman's host-gateway resolver.
|
|
# host_gateway_ip = "192.168.127.254"
|
|
sandbox_ssh_socket_path = "/run/openshell/ssh.sock"
|
|
stop_timeout_secs = 45
|
|
# Defaults to the gateway version; override to pin a specific build.
|
|
# supervisor_image = "ghcr.io/nvidia/openshell/supervisor:<version>"
|
|
guest_tls_ca = "/etc/openshell/certs/ca.pem"
|
|
guest_tls_cert = "/etc/openshell/certs/client.pem"
|
|
guest_tls_key = "/etc/openshell/certs/client-key.pem"
|
|
# Unsafe operator override. Host bind mounts, including Podman local-driver
|
|
# bind-backed volumes, expose gateway-host paths inside sandboxes and can
|
|
# negate OpenShell isolation and filesystem controls.
|
|
enable_bind_mounts = false
|
|
# Set to 0 to leave Podman's runtime default unchanged.
|
|
sandbox_pids_limit = 2048
|
|
# Health check interval in seconds. Lower values detect readiness faster
|
|
# but increase process churn (each check spawns a conmon subprocess).
|
|
# Set to 0 to disable health checks entirely. Default: 10.
|
|
health_check_interval_secs = 10
|
|
# Corporate forward proxy for sandbox egress. When set, the in-container
|
|
# supervisor chains policy-approved TLS tunnels through this proxy with HTTP
|
|
# CONNECT instead of dialing destinations directly. Plain-HTTP requests are
|
|
# not proxied and always dial the destination directly. Only http:// proxy
|
|
# URLs in explicit http://host:port form are supported: the scheme and port
|
|
# are both required, and a URL carrying a path, query, or fragment is
|
|
# rejected rather than silently truncated.
|
|
# NO_PROXY entries (hostnames, domain suffixes, IPs, CIDRs, each with an
|
|
# optional :port qualifier) are dialed directly. A port-qualified entry only
|
|
# bypasses that destination port. IP and CIDR entries also match hostnames
|
|
# through their validated DNS resolution; such a match dials directly only
|
|
# the resolved addresses inside the entry. This is an operator-owned egress
|
|
# boundary:
|
|
# sandbox and template environment cannot override it, and the conventional
|
|
# HTTPS_PROXY/HTTP_PROXY/NO_PROXY variables a sandbox sets do not affect it.
|
|
#
|
|
# The CONNECT request sent to the proxy targets a validated resolved IP,
|
|
# not the hostname, so the proxy performs no DNS resolution of its own and
|
|
# the tunnel stays bound to the address that passed the sandbox's SSRF and
|
|
# allowed_ips validation. The hostname still travels inside the tunnel (TLS
|
|
# SNI, application Host), so destination servers behave normally. In
|
|
# split-horizon networks, point the gateway host at the corporate resolver
|
|
# so internal names validate to their internal addresses. If the proxy's
|
|
# ACLs filter on hostnames and reject IP CONNECT targets, set
|
|
# proxy_connect_by_hostname = true as a last resort: the proxy then
|
|
# resolves the name itself, so a name that resolves differently at the
|
|
# proxy (split-horizon DNS, rebinding) can reach destinations the sandbox
|
|
# policy never approved, and the proxy's own ACLs become the effective
|
|
# egress control for proxied TLS.
|
|
#
|
|
# Configuration is fail-closed: an invalid proxy URL is rejected at gateway
|
|
# startup, setting no_proxy, proxy_auth_file, or proxy_connect_by_hostname
|
|
# without a proxy URL is rejected as well, and a set-but-invalid value
|
|
# reaching a sandbox (for example an unreadable or malformed auth file) is
|
|
# fatal to that sandbox's supervisor instead of silently falling back to
|
|
# direct or unauthenticated egress.
|
|
#
|
|
# Credentials must NOT be embedded in the URL (an inline user:pass@ is
|
|
# rejected at startup, since it would be stored here and exposed in container
|
|
# metadata). Instead point proxy_auth_file at a file containing "user:pass";
|
|
# the gateway delivers it to the supervisor through a root-only secret mount.
|
|
# The credential must use the user:pass form (non-empty user, no control
|
|
# characters); the same validation runs at sandbox-create time and in the
|
|
# supervisor, so a credential accepted here is never rejected in-container.
|
|
# Keep gateway.toml and the auth file owner-readable only (mode 0600).
|
|
#
|
|
# WARNING: the supervisor sends the credential as a Proxy-Authorization:
|
|
# Basic header over the plain-TCP connection to the http:// proxy. Basic
|
|
# auth is base64, not encryption: anyone on the network path between the
|
|
# sandbox host and the proxy can recover the credential. Because of that,
|
|
# proxy_auth_file requires the explicit acknowledgement
|
|
# proxy_auth_allow_insecure = true; without it the configuration is
|
|
# rejected at gateway startup. Only opt in when the path to the proxy is a
|
|
# trusted network segment.
|
|
# https_proxy = "http://proxy.corp.com:8080"
|
|
# no_proxy = "*.svc.cluster.local,10.0.0.0/8"
|
|
# proxy_auth_file = "/etc/openshell/secrets/proxy-auth"
|
|
# proxy_auth_allow_insecure = true
|
|
# Last resort for hostname-filtering proxy ACLs; see the warning above.
|
|
# proxy_connect_by_hostname = true
|
|
```
|
|
|
|
### MicroVM
|
|
|
|
Each sandbox runs inside its own libkrun microVM managed by the standalone `openshell-driver-vm` subprocess. Use this driver when you want stronger isolation than container namespaces alone.
|
|
|
|
```toml
|
|
[openshell]
|
|
version = 1
|
|
|
|
[openshell.gateway]
|
|
bind_address = "127.0.0.1:17670"
|
|
log_level = "info"
|
|
# VM is never auto-detected; an explicit entry here is required.
|
|
compute_drivers = ["vm"]
|
|
|
|
[openshell.drivers.vm]
|
|
state_dir = "/var/lib/openshell/vm"
|
|
# Where the gateway looks for the openshell-driver-vm subprocess binary.
|
|
driver_dir = "/usr/local/libexec/openshell"
|
|
default_image = "ghcr.io/nvidia/openshell/sandbox:latest"
|
|
grpc_endpoint = "https://host.containers.internal:17670"
|
|
# Empty falls back to default_image.
|
|
bootstrap_image = "ghcr.io/nvidia/openshell/sandbox:latest"
|
|
krun_log_level = 1
|
|
vcpus = 2
|
|
mem_mib = 2048
|
|
overlay_disk_mib = 4096
|
|
guest_tls_ca = "/var/lib/openshell/guest-tls/ca.pem"
|
|
guest_tls_cert = "/var/lib/openshell/guest-tls/client.pem"
|
|
guest_tls_key = "/var/lib/openshell/guest-tls/client-key.pem"
|
|
# Resolved sandbox UID/GID for the rootfs /etc/passwd entry.
|
|
# Defaults to 10001 when unset; matching GID is used if sandbox_gid is empty.
|
|
# sandbox_uid = 20001
|
|
```
|
|
|
|
### Extension Driver
|
|
|
|
Extension drivers run outside the gateway and expose the
|
|
`compute_driver.proto` gRPC service on a Unix socket. Use a non-reserved driver
|
|
name; built-in names such as `vm`, `docker`, `podman`, and `kubernetes` cannot
|
|
be selected through unmanaged socket endpoints. The selected driver name is the
|
|
key used for driver-owned sandbox config such as `template.driver_config.<name>`.
|
|
|
|
```toml
|
|
[openshell]
|
|
version = 1
|
|
|
|
[openshell.gateway]
|
|
bind_address = "127.0.0.1:17670"
|
|
log_level = "info"
|
|
compute_drivers = ["kyma"]
|
|
|
|
[openshell.drivers.kyma]
|
|
socket_path = "/run/openshell/kyma-compute-driver.sock"
|
|
```
|