* feat(network): support additional destination CAs Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): harden additional CA propagation Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): narrow additional CA review scope Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): harden additional CA rotation Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * feat(network): complete additional CA delivery Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(docker): remove stale runtime fixture field Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): adapt additional CA series to upstream main Resolve rebase fallout against current upstream main after the series was rebased 214 commits forward: - follow upstream #3110: supervisor→gateway TLS is CA-only, so configure_gateway_tls and its isolation tests drop the client identity - follow upstream tls_secret_names reduction to [String; 1] in the Podman driver while keeping the network trust bundle delivery - port [openshell.supervisor.network] rendering into the openshell.gatewayConfigToml serializer in _toml.tpl - adapt tests to upstream struct/function changes (DriverStartupContext, SandboxRunConfig, IsolationSpecInput, supervisor_pod, VM bootstrap image) - drop conformance-artifact-prefix plumbing removed upstream Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(network): adopt upstream additional CA bundle surface Unify the additional destination CA series on the mechanism merged to main in #3996: one operator pre-assembled PEM bundle, the --additional-ca-bundle supervisor flag, and the /etc/openshell/tls/additional/ca-bundle.pem staging path. - Replace the gateway-level additional_ca_cert_paths list with a single additional_ca_bundle file validated and staged at gateway startup. - Restore the upstream supervisor argument path and fold-into-roots behavior; drop the --network-additional-ca-bundle and --network-additional-ca-digest contracts. - Keep gateway normalization bounds, content-addressed staging, digest-based driver reconciliation, and stop/start rotation; the freshness check now runs at the gateway/driver boundary instead of the supervisor boundary. - Reject additional_ca_bundle in operator driver JSON config; standalone drivers self-source the bundle from --sandbox-additional-ca-bundle with the same validation. - Extend the Kubernetes ConfigMap name digest segment to keep the full 63-byte budget after the prefix rename. - Box the persisted-sandbox reconciliation futures so callers stay under the clippy::large_futures threshold. Updates Docker, Podman, Kubernetes, and VM delivery, Helm chart and tests, e2e wiring, gateway configuration docs, and the cluster debugging skill to the single mechanism. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(ci): make gateway startup tests wrap-tolerant and refresh example lockfile The network trust startup tests asserted on raw substrings of gateway diagnostics, but the CLI renderer wraps output at a fixed width with continuation markers and hard-breaks long paths. Compare assertions on whitespace- and continuation-squashed text so they hold regardless of wrap position. Format the e2e workspace (separate cargo workspace, not covered by the root cargo fmt run) and refresh examples/isolation-backend-v8/Cargo.lock for the libc dependency added to the network trust crate. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(podman): reuse TLS CA secret during stopped-pair trust replacement The additional CA e2e never ran in CI before the first maintainer test dispatch, and its local container inspection was broken in two ways: it located the supervisor container through e2e network membership (the supervisor is host-networked or runtime-bridged, so the filter matched the CI job container instead), and it required the gateway mTLS CA as a bind mount (Podman delivers it as a container secret when user namespaces remap UIDs). Select the supervisor by managed labels and accept the secret delivery form. The lifecycle leg then exposed a driver bug: stopped-pair trust replacement re-created the per-sandbox TLS CA secret, colliding with the secret still owned by the renamed stopped pair. Mirror the token and resolver secret handling: require and reuse the existing secret during replacement instead of creating it. Validated end-to-end with the Podman additional CA e2e lane. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(server): restore upstream recovery semantics for start-request failures Recovery no longer marks sandboxes Error/InvalidPersistedSandbox when driver_start_request fails during the startup or lifecycle sweeps. Gateway configuration problems (for example a missing launch signer) must not permanently fail unrelated sandbox records; the sweep logs a warning and skips them, matching the behavior before the additional CA work. Sandbox-record failures surface through the normal start path. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(podman): refresh stale supervisors without replacing workloads Replace the stopped-pair trust reconciliation with supervisor-only recreation. The destination-trust generation label moves from the workload to the supervisor container, and start_sandbox recreates only the supervisor when its generation is missing or stale. The workload container, its volumes, its secrets, and its workspace data are never replaced: the additional CA bundle is mounted only into the supervisor. Running sandboxes are never gated on trust labels, and recovery without fresh launch credentials keeps the existing supervisor. The intentional -removal fence machinery in the watcher existed only for workload replacement and is removed with it. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(docker): drop workload replacement for destination-trust changes Delete the stopped-workload trust reconciliation path and all of its machinery: the trust-generation label on workload containers, the generation match check in the start fence, the stopped-sandbox replacement, the bounded workspace archive copy with its 512 MiB cap and regular-file-only validation, and the boundary-state rollback helpers. No replacement machinery is needed: the trusted supervisor container is already re-created from the gateway's current configuration on every start via ensure_control_process_for_container, so stop/start picks up the current additional CA bundle while the workload container, its volumes, and its workspace data are never replaced. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs(configuration): drop reconciliation wording from additional CA lifecycle The stop/start lifecycle description already matches the supervisor-only refresh model; remove the two remaining references to restart reconciliation of workload containers. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): port additional CA test fixtures to the crypto interface Route the gateway TLS isolation test PKI, the destination CA source fixtures, and the gateway startup trust tests through openshell_crypto::pki now that rcgen access is centralized there: generate_keypair/self_signed/signed_by replace the direct rcgen calls and CertifiedKey field access follows the wrapper's shape. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(e2e): repair additional CA lane for Docker and non-container builds The first maintainer test dispatch revealed two latent issues in the additional CA e2e that no local run had exercised: the Podman gateway CA secret helper referenced ContainerEngine without the container feature gate, breaking Kubernetes and VM builds of the test, and the Docker branch asserted a client ca.crt bind mount that the Docker driver never creates. Gate the helper like its callers, and assert Docker's actual delivery path instead: the gateway mTLS CA staged as tls/ca.pem inside the read-only supervisor state volume, selected through the OPENSHELL_TLS_CA environment pointer. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(helm): restore alphabetical README ordering for supervisor keys The helm:docs check regenerates the chart README with helm-docs, which orders rows alphabetically; a conflict resolution had placed supervisor.network.additionalCaConfigMapName after the supervisor.podSecurityContext rows. Regenerate with mise run helm:docs. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * chore: refresh testkit lockfile for openshell-core libc dependency Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(driver-podman): stage operator CA bundle as driver-owned artifact Standalone Podman drivers accepting --sandbox-additional-ca-bundle previously kept the operator's source path as the trust artifact path. The immutable-artifact verification performed on every create/start requires exactly mode 0444, which an operator-owned source path cannot guarantee, so a readable-but-writable source failed reconciliation. Read the operator bundle with the same bounded, no-symlink validation, then stage the normalized material into a driver-owned artifact inside a private tempdir held for the driver process lifetime. The artifact is created 0600, synced, and only then restricted to 0444 so no observer ever sees a writable artifact. The operator source and its permissions are never referenced after startup. Addresses GATOR-291f43b2-02. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(driver-podman): pin retained workload image during supervisor refresh Supervisor-only trust reconciliation re-entered create_sandbox_with_existing_token, which re-pulled and re-resolved the workload image tag even though the workload container is retained. If the tag moved to a different image between stop and start, identity resolution and child-environment construction would reflect the new image while the retained workload still runs the original one, leaving mode-0600 bootstrap files the retained user cannot read. During a refresh with a retained workload, inspect the stopped container and pin to the immutable image ID it was created from (Config.Image); never pull or resolve the request tag. The requested reference is still used for child metadata so the retained workload's environment is byte-identical to its original launch. Addresses GATOR-291f43b2-01. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): verify host-only VM trust delivery and drop curl dependency The additional-CA lane's VM assertions read the destination CA out of the guest overlay and a guest-side supervisor-argv marker — artifacts from the guest-supervisor design that no longer exist now that the VM driver stages the CA host-only and runs the supervisor on the host. Inspect the driver-staged host artifact (digest-compared against the source), assert the guest overlay contains no CA material at the old mount paths, and keep the serial-console no-disclosure check. The workload validation scripts also ran curl, which exists only in the container-engine fixture image; the Kubernetes and VM lanes use the uv-based workload image and died instantly with "canonical main process already finished". Rewrite the main-process, exec, and lifecycle validation scripts in python3 (present in every lane's workload image). The network-policy binary identity is the resolved /proc/<pid>/exe target, so the versioned python3.12 binaries are allowlisted rather than the unversioned symlinks. The hostname mismatch check keeps its semantics: same reachable fixture, mismatched TLS SNI, and the failure must specifically be hostname verification. Addresses GATOR-291f43b2-04. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(kubernetes): allow admission-added labels on destination-trust ConfigMaps The destination-trust ConfigMap validation compared the whole label map for equality. A mutating admission webhook that attaches an organizational label to the created ConfigMap made the create response fail validation, and the retained ConfigMap then failed every subsequent create/start through the existing-object path, blocking sandbox launches despite the ownership labels, immutable flag, digest, and certificate bytes remaining correct. Validate the gateway-owned label pairs individually and allow unrelated labels; the exact trust-material checks are unchanged. Covered by a regression test with an admission-added label and a missing gateway-owned label. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(podman): close supervisor-refresh rollback gaps The supervisor-only trust refresh renamed the previous supervisor to a random rollback name and then performed the token-secret lookup. A transient lookup failure returned with the supervisor still renamed, where a normal delete could never find it, and the rollback rename and final rollback removal ignored their own failures, leaving orphans with attached control and channel resources. Complete every fallible preparation before the rename. On a failed replacement, restore the canonical name; if the restore itself fails, return an error naming the rollback container so the orphan stays identifiable — a later retry still succeeds because the canonical name is absent and the replacement path recreates a fresh supervisor. Treat not-found during the final rollback removal as success and name the orphan in both the log and the error on other removal failures; a retry observes the fresh canonical supervisor and resumes the normal start path. Covered by failure-injection tests for each transaction boundary: preparation failure before any rename, replacement failure restoring the canonical name, and restore failure naming the orphaned rollback container. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs: address destination-trust review corrections The gateway-configuration RFC documented an additional_ca_cert_paths list that the strict configuration parser never accepted; document the additional_ca_bundle single-path setting the parser actually enforces. The debug-openshell-cluster skill named the wrong generation annotation (openshell.ai/network-supervisor-trust-generation) and described digest argv and network-init canonicalization checks the implementation does not perform. Correct it to openshell.ai/additional-ca-generation and describe the actual delivery: the supervisor Pod mounts the immutable ConfigMap read-only and receives --additional-ca-bundle for the mount path, with the driver validating the digest annotation and exact bytes per creation. Classify the destination-trust generation hashes as deterministic identity derivation rather than authenticity proofs, following the openshell-crypto exclusions table: trust decisions come from the gateway's normalization, immutable staging, exact-byte validation, and TLS verification. Record the classification in the crypto README and at each hashing site. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): reconcile additional CA tests with suspend/resume and auth changes Upstream's suspend/resume lifecycle took StartSandboxRequest fields 7 and 8, so the durable-snapshot field moves from 7 to 9; update the wire-format test to assert the field-9 key (0x4a) and initialize the new resume controls. Repair the cli test module after the auth opt-in test merged alongside the environment-guard helpers, and initialize the gateway driver-startup context's network-trust fields in the shared supervision test. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(server): box start-recovery futures for clippy large-futures The durable-snapshot field grows StartSandboxRequest enough that the start-recovery match and the recovery-expiry test exceed the large-futures limit under -D warnings. Box::pin both futures, matching the pattern the recovery path used before the suspend/resume rework. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> --------- Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
OpenShell Helm Chart
Experimental - the Kubernetes deployment path is under active development. Expect rough edges and breaking changes.
This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at oci://ghcr.io/nvidia/openshell/helm-chart.
By default, this chart also creates the namespace-scoped resources needed by
sandboxes. For a shared-gateway deployment, install it with
workspaceResources.enabled=false, then install the
deploy/helm/openshell-workspace chart in every pre-provisioned workspace
namespace. The gateway and workspace releases can then be upgraded and removed
independently. Use Kubernetes operator workspace mode when one gateway serves
multiple pre-provisioned workspace namespaces.
Certificate hook metadata
Use pkiInitJob.podLabels and pkiInitJob.podAnnotations to add metadata to the
Pod templates of the <fullname>-certgen and <fullname>-certgen-backend-ca
Jobs. These settings apply to both hooks when enabled and do not affect gateway
or sandbox Pods. Chart-owned Pod labels take precedence when custom keys
conflict.
For example, disable Istio sidecar injection for the hook Pods in an injected namespace:
pkiInitJob:
podAnnotations:
sidecar.istio.io/inject: "false"
Cluster-scoped vs namespaced objects
Most objects in this chart are namespaced and land in the release namespace. Only two are cluster-scoped:
| Object | Default name |
|---|---|
ClusterRole |
<fullname>-node-reader-<release namespace> |
ClusterRoleBinding |
<fullname>-node-reader-<release namespace> |
By default the release creates both, so an install by a cluster-admin is unchanged. On clusters where cluster-scoped RBAC is owned by a different team, split the install in two.
A cluster-admin applies the cluster-scoped objects once per gateway ServiceAccount, rendered from the same values the release uses:
helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
--namespace openshell -f my-values.yaml \
--set rbac.create=true \
--set rbac.clusterScoped.create=true \
--set agentSandbox.preflight.enabled=false \
--show-only templates/clusterrole.yaml \
--show-only templates/clusterrolebinding.yaml | kubectl apply -f -
A namespace-admin then installs and upgrades the release with cluster-scoped
objects omitted, using ci/values-namespace-admin.yaml
or the equivalent --set:
helm upgrade --install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
--namespace openshell -f my-values.yaml \
--set rbac.clusterScoped.create=false
The gateway ServiceAccount name and namespace do not change, so the
pre-created ClusterRoleBinding keeps matching the release. This works with
serviceAccount.create=false too: the ClusterRoleBinding subject follows
serviceAccount.name, so render the admin step with the same values.
Which flag the installer needs
In the default shared workspace mode the release also creates a namespaced
sandbox Role granting Agent Sandbox (agents.x-k8s.io) permissions.
Kubernetes forbids granting permissions you do not hold, and the built-in
admin ClusterRole does not cover that CRD, so an installer holding only
admin cannot create it. rbac.clusterScoped.create=false alone is then not
enough and the install fails with attempting to grant RBAC permissions not currently held.
| Workspace mode | Installer holds | Use |
|---|---|---|
shared |
built-in admin only |
rbac.create=false, cluster-admin pre-creates all gateway RBAC |
shared |
admin plus the sandbox permissions in the namespace |
rbac.clusterScoped.create=false |
managed, operator |
built-in admin only |
rbac.clusterScoped.create=false |
managed and operator render no namespaced sandbox Role, so no extra grant
is needed there.
With rbac.create=false the cluster-admin applies the namespaced RBAC too,
adding it to the same render:
helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
--namespace openshell -f my-values.yaml \
--set rbac.create=true \
--set rbac.clusterScoped.create=true \
--set agentSandbox.preflight.enabled=false \
--show-only templates/clusterrole.yaml \
--show-only templates/clusterrolebinding.yaml \
--show-only templates/role.yaml \
--show-only templates/rolebinding.yaml \
--show-only templates/peer-role.yaml | kubectl apply -f -
To grant the installer the sandbox permissions instead, bind it to a Role
carrying the same rules as the chart's openshell-sandbox Role.
The certgen hook and credential driver RBAC keep their own flags
(pkiInitJob.enabled and
server.credentialDrivers.kubernetesSecrets.rbac.create).
Migrating an existing release
Helm deletes objects that leave a release manifest, so setting
rbac.clusterScoped.create=false on a release that already owns the
ClusterRole and ClusterRoleBinding deletes them. The gateway then loses
TokenReview until a cluster-admin re-applies them. Hand ownership over first, as
cluster-admin, so nothing is deleted:
kubectl annotate clusterrole "openshell-node-reader-<namespace>" \
helm.sh/resource-policy=keep --overwrite
kubectl annotate clusterrolebinding "openshell-node-reader-<namespace>" \
helm.sh/resource-policy=keep --overwrite
The objects then survive the upgrade that sets the flag, and the cluster-admin owns them from that point on. Fresh installs need no such step.
rbac.clusterScoped.create is independent of
server.drivers.kubernetes.workspaceMode. Managed and operator modes change
what the ClusterRole contains, but they never force the namespaced release to
apply it. Re-run the cluster-admin step after changing values that affect the
ClusterRole rules.
Prerequisites
Required: Your cluster CNI MUST enforce Kubernetes
NetworkPolicyfor ingress and egress in every sandbox namespace. OpenShell creates the policies, but Kubernetes accepts them even if no CNI enforces them. Without enforcement, sandbox workloads may connect directly and bypass supervisor network policy. Verify CNI support before installing OpenShell.
The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with:
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/sandbox.yaml
The chart does not install this cluster-scoped dependency. By default, it
fails before creating gateway resources when the cluster serves neither
supported Sandbox API (agents.x-k8s.io/v1beta1 or
agents.x-k8s.io/v1alpha1). Disable the check with
agentSandbox.preflight.enabled=false for offline helm template rendering,
where Helm cannot discover cluster APIs.
Install on Kubernetes
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version>
Install on OpenShift
See the full OpenShift install guide for details. Quick start:
# Precreate the openshell namespace
oc create ns openshell
# Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> -n openshell \
--set server.disableTls=true \
--set podSecurityContext.fsGroup=null \
--set securityContext.runAsUser=null
On OpenShift 4.22+, end-to-end TLS is supported via BackendTLSPolicy. See the
OpenShift install guide for details.
Available versions
| Tag | Source | Notes |
|---|---|---|
<semver> (e.g. 0.6.0) |
Tagged GitHub release | Tracks the matching gateway, sandbox, and supervisor image versions. Recommended for production. |
<semver>-pre.N (e.g. 0.1.0-pre.3) |
A specific prerelease candidate | Immutable candidate pin that tracks images with the same exact version. |
0.0.0-dev |
Latest commit on main |
Floating tag, overwritten on every push. appVersion is dev, so images resolve to the :dev tag. |
0.0.0-dev.<commit-sha> |
A specific commit on main |
Per-commit pin. Chart version and appVersion both use the full 40-character commit SHA, which matches the image tag pushed by CI. |
Prerelease and dev tags are intended for testing changes ahead of a release. Production deployments should pin to a stable tagged release.
Configuration
See values.yaml for source defaults. Selected overlays:
ci/values-gateway.yaml- gateway-only configurationci/values-cert-manager.yaml- cert-manager integrationci/values-keycloak.yaml- Keycloak OIDC integrationci/values-high-availability.yaml- CI overlay for multi-replica external PostgreSQL testingci/values-autoscaling.yaml- CI overlay for rendering the optional gateway HorizontalPodAutoscalerci/values-supervisor-autoscaling.yaml- CI overlay for rendering the optional shared supervisor memory HorizontalPodAutoscalerci/values-spire.yaml- SPIFFE/SPIRE provider token grantsci/values-spire-stack.yaml- SPIRE hardened chart values for local development
Additional destination CAs
Create an operator-owned ConfigMap with a ca.crt key when sandboxes must reach
TLS destinations signed by a private CA:
kubectl create configmap private-destination-ca -n openshell \
--from-file=ca.crt=/path/to/ca.crt
Then set the source ConfigMap name:
supervisor:
network:
additionalCaConfigMapName: private-destination-ca
The chart mounts the source into the gateway and renders
[openshell.supervisor.network].additional_ca_bundle. The gateway validates
and normalizes the one certificate-only PEM bundle at startup. The Kubernetes
driver creates a separate managed ConfigMap in each target sandbox namespace
and mounts it only in the dedicated supervisor Pod at
/etc/openshell/tls/additional/ca-bundle.pem. That supervisor receives the
path through --additional-ca-bundle and folds the bundle into system roots.
The configured roots augment default destination trust. They do not configure
the corporate proxy, OIDC, or gateway callback mTLS trust.
Changing only the source ConfigMap does not roll or reload the gateway. Restart
the gateway, then recreate or restart affected sandboxes to use the new roots.
Removing additionalCaConfigMapName removes the global TOML setting; newly
created or recreated sandbox pods then have no additional-CA volume or mount.
Existing running sandbox supervisors retain their startup trust until they are
restarted or recreated.
The driver creates one immutable, content-addressed ConfigMap generation in
each target namespace. Names begin with
openshell-additional-ca- and include DNS-safe hashes of the effective
gateway ID and normalized bundle generation. Existing objects are accepted only
when their ownership labels, generation annotation, immutability, and exact
ca.crt data match; the driver never patches or updates them.
Kubernetes cannot restrict create by resourceNames, and resourceNames
cannot express the digest-bearing name prefix needed for get. Enabling this
feature therefore gives the gateway/driver service account namespace-wide
ConfigMap get and create permissions in shared mode. In managed/operator
modes, the chart's ClusterRoleBinding makes those permissions cluster-wide for
the bound service account. The chart does not grant ConfigMap list, watch,
patch, update, or delete. Use a dedicated gateway service account and
namespace-scoped bindings when this boundary matters. If you render the TOML
setting outside this Helm value, grant get and create in every target
sandbox namespace.
Disabling the setting intentionally does not delete driver-managed generations: the gateway has no ConfigMap list/delete permissions, and an older object may still be referenced by a running or stopped sandbox. An operator with separate list/delete authority can inventory the gateway's generations by label:
kubectl -n <sandbox-namespace> get configmap \
-l 'openshell.ai/managed-by=openshell,openshell.ai/gateway-id=<effective-gateway-id>'
Before deleting a named generation, verify that no Sandbox resource or pod volume in that namespace references it. Delete only the confirmed unreferenced name; removing a referenced generation can make a later pod start or reschedule fail closed.
A configured CA can authenticate every policy-permitted TLS endpoint with a matching certificate chain and hostname. Scope private CAs accordingly.
Database backend
By default, OpenShell uses SQLite and runs the gateway as a StatefulSet so the database is backed by a per-pod PVC:
server:
dbUrl: "sqlite:/var/openshell/openshell.db"
Data PVC storage
Size the gateway data claim, openshell-data, and select its StorageClass with
server.dataStorageSize and server.dataStorageClass. Omitting both
requests 1Gi from the cluster's default StorageClass. Set
server.dataStorageClass on clusters with no default StorageClass,
otherwise the gateway PVC stays Pending. Both values are ignored under
workload.kind=deployment, which mounts no PVC.
server:
dataStorageSize: 20Gi
dataStorageClass: fast-ssd
volumeClaimTemplates is immutable, so set both at install time. To grow an
existing claim, expand it in place when its StorageClass allows expansion, then
recreate the StatefulSet so its template matches:
kubectl -n openshell patch pvc openshell-data-openshell-0 \
-p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'
kubectl -n openshell delete statefulset openshell --cascade=orphan
helm upgrade openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <installed-version> \
--reuse-values -n openshell --set server.dataStorageSize=20Gi
A recreated StatefulSet re-adopts existing claims unchanged, so changing StorageClass means copying the data to a new claim.
External PostgreSQL
Use external PostgreSQL when the gateway should connect to a database managed outside this chart. The OpenShell chart does not deploy a database; install PostgreSQL separately using the chart, operator, or managed service that fits your environment, then pass the connection URI through a Secret.
Create a Secret containing the PostgreSQL connection URI if one does not already exist:
kubectl create secret generic my-pg-credentials -n openshell \
--from-literal=uri="postgresql://user:pass@host:5432/dbname"
Then install the chart pointing at that Secret:
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
-n openshell \
--set workload.kind=deployment \
--set server.externalDbSecret=my-pg-credentials
Use workload.kind=deployment for external database-backed multi-replica
gateways. workload.kind=statefulset is still available for single-replica
SQLite installs and for operators who explicitly need StatefulSet identity or
storage semantics.
Credential storage
By default, the chart uses the gateway's encrypted database credential storage. The gateway writes encrypted provider credential envelopes to the OpenShell database. The chart creates a retained Kubernetes Secret with the shared key-encryption key and injects that key into every gateway pod, so the same default works for single-replica and external database-backed HA deployments.
Use gatewayConfig to select kubernetes-secrets or vault when credentials
should live in a cluster or external secret backend. Selecting an external
credential driver disables the default credential-storage key-encryption-key
Secret and environment injection. The map is rendered directly as gateway TOML,
so it uses the gateway's snake_case field names:
gatewayConfig:
openshell.gateway:
credential_drivers:
- vault
openshell.credential_drivers.vault:
address: https://vault.vault.svc.cluster.local:8200
mount: secret
kv_version: "2"
auth_method: kubernetes
role: openshell-gateway
gatewayConfigmust contain only non-secret values. Helm serializes unknown fields generically and cannot determine whether an arbitrary string, such asapi_token, is confidential. Do not put passwords, tokens, private keys, database URLs, or other secret material in this map. Use Secret-backed environment variables, files, volumes, or gateway credential drivers instead. The chart rejects known unsafe forms such asdatabase_url, inline URL credentials, and PEM private keys; it is not a general secret scanner.
For the Kubernetes Secret driver, use
openshell.credential_drivers.kubernetes-secrets.namespace in the same map.
The chart derives any required RBAC from the selected driver; use a dedicated
namespace to limit access to OpenShell-managed Secrets.
OpenShift
Append these flags to any of the PostgreSQL commands above for OpenShift:
--set server.disableTls=true \
--set podSecurityContext.fsGroup=null \
--set securityContext.runAsUser=null
High availability
Set replicaCount above 1 only with server.externalDbSecret; the default
SQLite database is per pod and cannot coordinate multiple gateway replicas.
The chart creates a headless peer Service for gateway-to-gateway relay traffic.
StatefulSet pods use stable pod DNS names through that headless Service.
Deployment pods advertise their pod IP with OPENSHELL_PEER_ENDPOINT, because
Kubernetes does not assign stable per-pod DNS names to Deployment replicas.
Gateway peer traffic uses Kubernetes ServiceAccount identity. Each gateway pod
mounts a projected, pod-bound ServiceAccount token with audience
openshell-gateway-peer; receiving replicas validate that token with the
Kubernetes TokenReview API, verify the live pod UID and Helm selector labels,
and authorize only the internal PeerRelay RPC. The chart does not create or
accept a shared gateway peer Secret.
With gateway TLS enabled, peer calls use the chart CA and client TLS Secret for
server verification and mTLS. The client verifies the stable gateway Service
DNS name while connecting directly to the owning pod. Custom TLS Secrets must
include that Service DNS name in the server certificate and provide the CA and
client credentials configured by server.tls.
Set autoscaling.enabled=true to render an autoscaling/v2
HorizontalPodAutoscaler for the gateway workload. The Deployment or
StatefulSet then omits spec.replicas, and the chart applies its
multi-replica checks to autoscaling.maxReplicas. CPU and memory targets
need a matching resources.requests entry, or a resources.limits entry,
which Kubernetes copies into the request. Add custom metrics from a metrics
adapter with autoscaling.metrics. See the
High Availability guide
for the metrics to scale and alert on.
Enabling autoscaling on an existing release removes spec.replicas from the
workload in that upgrade. Kubernetes can reset the workload to one replica
until the HPA scales it back to at least autoscaling.minReplicas, which
disconnects supervisor sessions from the other gateway pods; they reconnect to the remaining replica. Enable autoscaling
when you install the chart, or during a maintenance window. On an existing
release, upgrade with --reset-then-reuse-values, which needs Helm 3.14 or
later. With an older Helm, save the values with
helm get values <release> -o yaml > values.yaml and upgrade with
--reset-values -f values.yaml. --reuse-values keeps the
previous chart version's defaults, which lack the autoscaling values,
including the scale-down behavior, when that release predates them. For background, refer to
Migrating Deployments and StatefulSets to horizontal autoscaling.
Disabling autoscaling renders spec.replicas from replicaCount again,
which defaults to 1, so set replicaCount to the replica count you want in
that upgrade.
Set supervisor.autoscaling.enabled=true with the shared supervisor topology
to autoscale its Deployment independently of the gateway. It uses average
memory per Pod (supervisor.autoscaling.targetMemoryAverageValue, default
512Mi) through the resource metrics API, usually provided by Metrics Server.
Defaults keep two to four members and remove at most one member every two
minutes after a five-minute stabilization window. The HPA owns replicas, so
supervisor.replicas is ignored. Enabling it on an existing Deployment has
the same temporary reset-to-one risk described above; pick up new defaults
with --reset-then-reuse-values. When disabling it, set supervisor.replicas
explicitly. Existing sandboxes stay on their owners when scaling up, and
memory-based scaling does not guarantee spare capacity for handoff. See the
shared supervisor autoscaling guide.
Secret bootstrap
By default, a pre-install/pre-upgrade hook Job runs openshell-gateway generate-certs
to create the gateway's server/client mTLS Secrets and sandbox JWT signing Secret.
The Job uses the gateway image itself, so air-gapped environments only need to
mirror that one image (no separate openssl/alpine sidecar).
When certManager.enabled=true, cert-manager owns the TLS Secrets and the chart
runs the same hook in JWT-only mode because cert-manager does not create the
sandbox JWT signing Secret. This precedence applies even if
pkiInitJob.enabled remains true. Set pkiInitJob.enabled=false only when an
external non-cert-manager TLS source manages TLS and you pre-create the sandbox
JWT signing Secret.
SPIFFE/SPIRE provider token grants
Set gatewayConfig.openshell.drivers.kubernetes.provider_spiffe_workload_api_socket_path
to let sandbox supervisors use SPIFFE JWT-SVIDs for dynamic provider token
grants. The chart keeps supervisor-to-gateway authentication on gateway-minted
sandbox JWTs and passes the configured socket path to the Kubernetes driver.
gatewayConfig:
openshell.drivers.kubernetes:
provider_spiffe_workload_api_socket_path: /spiffe-workload-api/spire-agent.sock
For local development, uncomment the SPIRE Helm releases in skaffold.yaml and
add ci/values-spire.yaml to the OpenShell release values files.
The gateway verifies supervisor JWT-SVIDs with JWT bundles fetched from the SPIFFE Workload API, so this path does not require access to the SPIRE OIDC discovery endpoint or its TLS CA.
Values
| Key | Type | Default | Description |
|---|---|---|---|
| affinity | object | {} |
Affinity rules for the gateway pod. |
| agentSandbox.preflight.enabled | bool | true |
Check the live cluster for a supported Agent Sandbox API before rendering gateway resources. Disable only for offline rendering and linting. |
| autoscaling.behavior | object | {"scaleDown":{"policies":[{"periodSeconds":120,"type":"Pods","value":1}],"stabilizationWindowSeconds":300}} |
HPA scaling behavior. Scale-down hands the removed pod's supervisor sessions to the remaining replicas, so the default removes at most one replica every two minutes after a five-minute stabilization window. Helm merges maps: set autoscaling.behavior.scaleDown to null to drop the default. |
| autoscaling.enabled | bool | false |
Render a HorizontalPodAutoscaler and stop rendering spec.replicas. |
| autoscaling.maxReplicas | int | 4 |
Maximum gateway replicas. Each replica opens up to database_max_connections + database_lock_max_connections PostgreSQL connections (14 with the defaults; set them under gatewayConfig.openshell.gateway); size the database for rollouts at this count, as the High Availability guide describes. With a Deployment and the default pool sizes, the default of 4 needs max_connections of at least 126, more than the PostgreSQL default of 100. |
| autoscaling.metrics | list | [] |
Additional autoscaling/v2 MetricSpec entries appended verbatim, such as Pods metrics served by prometheus-adapter. |
| autoscaling.minReplicas | int | 2 |
Minimum gateway replicas. Use 2 or more to survive a pod failure. |
| autoscaling.targetCPUUtilizationPercentage | int | 80 |
Target average CPU utilization, as a percentage of resources.requests.cpu. Set to null to disable. Requires resources.requests.cpu, or resources.limits.cpu, which Kubernetes copies into the request. |
| autoscaling.targetMemoryUtilizationPercentage | int | nil |
Target average memory utilization, as a percentage of resources.requests.memory. Null disables it. Requires resources.requests.memory, or resources.limits.memory, which Kubernetes copies into the request. |
| certManager.caSecretName | string | "openshell-ca-tls" |
Secret created for the intermediate CA (Certificate with isCA: true). |
| certManager.certificateDuration | string | "8760h" |
Duration for cert-manager-issued certificates. |
| certManager.certificateRenewBefore | string | "720h" |
Renewal window for cert-manager-issued certificates. |
| certManager.clientCaFromServerTlsSecret | bool | true |
Mount gateway client CA from the internal server TLS secret's ca.crt. The internal server certificate is always signed by the chart CA — the same CA that signs the client (mTLS) certificate — so the default (true) is correct for all configurations, including when serverIssuerRef is set. Only set to false if you mount the client CA from a separate secret via server.tls.clientCaSecretName. |
| certManager.enabled | bool | false |
Create cert-manager Issuer and Certificate resources. When enabled, cert-manager owns TLS and the chart runs a JWT-only certgen hook to create the sandbox JWT signing Secret that cert-manager does not manage. |
| certManager.serverDnsNames | list | ["openshell","openshell.openshell.svc","openshell.openshell.svc.cluster.local","localhost","openshell.localhost","*.openshell.localhost","host.docker.internal"] |
DNS SANs on the cert-manager-issued server certificate. |
| certManager.serverIpAddresses | list | ["127.0.0.1"] |
IP SANs on the cert-manager-issued server certificate. |
| certManager.serverIssuerRef | object | {"group":"","kind":"","name":""} |
Override the issuerRef for the external server Certificate (e.g. a real LetsEncrypt/ACME ClusterIssuer for a publicly-trusted cert on an external hostname). When set, the chart creates a second server certificate from this issuer with only the hostnames in serverDnsNames; the internal server certificate is always signed by the chart's own CA. Leave name empty to use the chart CA for all server certificates (default). Requires certManager.enabled=true. |
| credentialDrivers.vault.caConfigMapName | string | "" |
ConfigMap containing the private Vault/OpenBao CA certificate under the ca.crt key. Helm mounts it only when the Vault driver is selected. |
| fullnameOverride | string | "" |
Override the full generated resource name. |
| gateway.image.digest | string | "" |
Gateway image digest. When set, this takes precedence over tag. |
| gateway.image.pullPolicy | string | nil |
Gateway image pull policy. Empty uses global.image.pullPolicy. |
| gateway.image.registry | string | "" |
Gateway image registry. Empty uses global.image.registry. |
| gateway.image.repository | string | "openshell/gateway" |
Gateway image repository. |
| gateway.image.tag | string | "" |
Gateway image tag. Defaults to the chart appVersion when empty. |
| gatewayConfig | object | {"openshell":{"version":2}} |
Non-secret gateway application configuration. Top-level keys name TOML tables and are rendered into the mounted gateway.toml file. Kubernetes resource inputs remain outside this map; template expressions derive the corresponding runtime values from their resource owner. |
| global.image.pullPolicy | string | "IfNotPresent" |
Shared OpenShell image pull policy. Individual image pull policies take precedence. |
| global.image.registry | string | "ghcr.io/nvidia" |
Shared OpenShell image registry. Individual image registries take precedence. |
| global.image.tag | string | "" |
Shared OpenShell image tag. Defaults to the chart appVersion when empty. |
| grpcRoute.backendTLSPolicy.caCertificateConfigMapName | string | "" |
Name of the ConfigMap containing the CA certificate (key: ca.crt) used to validate the gateway pod's TLS certificate. Defaults to <fullname>-backend-ca when empty. The certgen hook auto-creates this: with pkiInitJob (default), immediately on install/upgrade; with cert-manager, the hook polls for pkiInitJob.timeoutSeconds seconds waiting for cert-manager to issue the server certificate, then creates the ConfigMap. A single install usually succeeds; if cert-manager takes longer, increase pkiInitJob.timeoutSeconds. By default (pkiInitJob.failOnTimeout=true), the install fails if the timeout is reached; set failOnTimeout=false to allow the install to succeed and run helm upgrade after the certificate is issued. |
| grpcRoute.backendTLSPolicy.enabled | bool | false |
Create a BackendTLSPolicy resource for end-to-end TLS between the Gateway proxy and the OpenShell gateway pod. The traffic flow is: client → HTTPS → Gateway (terminate) → TLS (re-encrypt) → gateway pod. Requires server.disableTls=false and server.tls.enableMtls=false. The certgen hook auto-creates the backend CA ConfigMap. |
| grpcRoute.backendTLSPolicy.hostname | string | "" |
Hostname the Gateway proxy validates against the backend's TLS certificate SAN. Defaults to the service FQDN (<fullname>.<namespace>.svc.cluster.local) when empty, which matches the SAN included by both cert-manager and the pkiInitJob. |
| grpcRoute.enabled | bool | false |
Create a Gateway API GRPCRoute for the gateway service. |
| grpcRoute.gateway.className | string | "eg" |
GatewayClass for a chart-created Gateway. Set this to the name exposed by your Gateway API implementation. Envoy Gateway uses "eg" in the OpenShell examples. Ignored when gateway.create is false. |
| grpcRoute.gateway.create | bool | false |
When true, a Gateway resource is created in the release namespace. Set to false and provide name/namespace to attach to a pre-existing Gateway. Use sectionName to select a specific listener when needed. |
| grpcRoute.gateway.listener.allowedRoutes | string | "Same" |
"Same" restricts attached routes to the release namespace; "All" allows any namespace. |
| grpcRoute.gateway.listener.port | int | 80 |
Listener port for the generated Gateway resource. Use 443 with protocol HTTPS. |
| grpcRoute.gateway.listener.protocol | string | "HTTP" |
Listener protocol for the generated Gateway resource: HTTP or HTTPS. HTTPS terminates TLS at the Gateway listener; pair it with server.disableTls=true so the proxy forwards plaintext to the gateway pod, and use OIDC for client identity (the gateway never sees the client certificate). |
| grpcRoute.gateway.listener.tls.certificateRefs | list | [] |
certificateRefs for the HTTPS listener. Required when protocol is HTTPS. Each entry needs a name pointing at a kubernetes.io/tls Secret in the Gateway's namespace. May reference a cert-manager-issued Secret or the existing openshell-server-tls Secret (its SANs must include the external hostname). |
| grpcRoute.gateway.name | string | "" |
Name of the Gateway resource. Defaults to <fullname>. Set a distinct name when the selected controller creates a proxy Service with the Gateway name. When gateway.create is false, set this to the name of the existing Gateway. |
| grpcRoute.gateway.namespace | string | "" |
Namespace of the Gateway referenced by the GRPCRoute parentRef. Defaults to the release namespace. A chart-created Gateway always resides in the release namespace. |
| grpcRoute.gateway.sectionName | string | "" |
Listener name on an existing Gateway. Leave empty to let Gateway API listener matching select compatible listeners. |
| grpcRoute.hostnames | list | [] |
Hostnames the GRPCRoute matches on. Leave empty to match all hosts. |
| grpcRoute.replicaRouting.enabled | bool | false |
Route requests carrying an x-openshell-replica header straight to that gateway replica. The CLI sets the header on long-lived sandbox connections (SSH, port forwards, exec) using the owner the gateway reports, so they skip the relay through a peer replica. Renders one Service per replica. Requires workload.kind=statefulset, workload.allowMultiReplicaStatefulSet=true, and at most 15 replicas. |
| imagePullSecrets | list | [] |
Image pull secrets attached to gateway and helper pods. |
| nameOverride | string | "openshell" |
Override the chart name used in generated resource names. |
| networkPolicy.enabled | bool | true |
Restrict SSH ingress on sandbox pods to the gateway. In managed mode, the driver applies the equivalent policy to each workspace namespace. |
| nodeSelector | object | {} |
Node selector for the gateway pod. |
| openshiftRoute.annotations | object | {} |
Extra annotations on the Route (e.g. haproxy.router.openshift.io/*). |
| openshiftRoute.enabled | bool | false |
Create an OpenShift Route with TLS passthrough. |
| openshiftRoute.host | string | "" |
Hostname for the Route. Must match a SAN on the gateway's server cert. |
| pkiInitJob.enabled | bool | true |
Run a pre-install/pre-upgrade Job that creates gateway and client mTLS Secrets. When certManager.enabled=true, cert-manager owns TLS and this same hook runs in JWT-only mode even if pkiInitJob.enabled remains true. |
| pkiInitJob.failOnTimeout | bool | true |
Fail the helm install/upgrade if cert-manager does not issue the certificate within the polling timeout. When true (default), the install fails immediately if the timeout is reached, providing clear feedback that BackendTLSPolicy is non-functional. When false, the hook succeeds with a warning and you can run helm upgrade after cert-manager issues the certificate to create the backend CA ConfigMap. If you set this to false and see "TLS error: Secret is not supplied by SDS" when connecting to the gateway, check if the TLS secret exists and run helm upgrade to create the ConfigMap. |
| pkiInitJob.podAnnotations | object | {} |
Additional annotations on both certificate hook Pods. |
| pkiInitJob.podLabels | object | {} |
Additional labels on both certificate hook Pods. Chart-owned labels take precedence. |
| pkiInitJob.securityContext.allowPrivilegeEscalation | bool | false |
Whether the certgen hook container can gain additional privileges. |
| pkiInitJob.securityContext.capabilities.drop | list | ["ALL"] |
Linux capabilities dropped from the certgen hook container. |
| pkiInitJob.securityContext.runAsNonRoot | bool | true |
Require the certgen hook container to run as a non-root user. |
| pkiInitJob.securityContext.seccompProfile.type | string | "RuntimeDefault" |
Seccomp profile applied to the certgen hook container. |
| pkiInitJob.serverDnsNames | list | [] |
Extra DNS SANs to append to the server certificate. |
| pkiInitJob.serverIpAddresses | list | [] |
Extra IP SANs to append to the server certificate. |
| pkiInitJob.timeoutSeconds | int | 120 |
Maximum time in seconds for the certgen hook to poll for cert-manager certificates. When using cert-manager with BackendTLSPolicy, the hook polls for this many seconds waiting for the certificate to be issued, then creates the backend CA ConfigMap. The Job deadline is set to (timeoutSeconds + 30) to allow time for ConfigMap creation and cleanup. Increase this if cert-manager takes longer than 120 seconds to issue certificates. |
| podAnnotations | object | {} |
Extra annotations to add to the gateway pod. |
| podLabels | object | {} |
Extra labels to add to the gateway pod. |
| podLifecycle.terminationGracePeriodSeconds | int | 30 |
Maximum time, in seconds, Kubernetes waits for the gateway to exit before killing it. The gateway exits as soon as shutdown completes; the limit covers supervisor session cleanup and draining queued OCSF records. |
| podSecurityContext.fsGroup | int | 1000 |
fsGroup assigned to the gateway pod. |
| probes.liveness.failureThreshold | int | 3 |
Liveness probe failure threshold before the container is restarted. |
| probes.liveness.initialDelaySeconds | int | 2 |
Liveness probe initial delay, in seconds. |
| probes.liveness.periodSeconds | int | 5 |
Liveness probe period, in seconds. |
| probes.liveness.timeoutSeconds | int | 1 |
Liveness probe timeout, in seconds. |
| probes.readiness.failureThreshold | int | 3 |
Readiness probe failure threshold before the pod is marked not ready. |
| probes.readiness.initialDelaySeconds | int | 1 |
Readiness probe initial delay, in seconds. |
| probes.readiness.periodSeconds | int | 2 |
Readiness probe period, in seconds. |
| probes.readiness.timeoutSeconds | int | 1 |
Readiness probe timeout, in seconds. |
| probes.startup.failureThreshold | int | 30 |
Startup probe failure threshold before the container is killed. |
| probes.startup.periodSeconds | int | 2 |
Startup probe period, in seconds. |
| probes.startup.timeoutSeconds | int | 1 |
Startup probe timeout, in seconds. |
| rbac.clusterScoped.clusterRoleBindingName | string | "" |
Name for the ClusterRoleBinding. Empty uses the <fullname>-node-reader-<release namespace> default. |
| rbac.clusterScoped.clusterRoleName | string | "" |
Name for the ClusterRole. Empty uses the <fullname>-node-reader-<release namespace> default. |
| rbac.clusterScoped.create | bool | true |
Create the cluster-scoped ClusterRole and ClusterRoleBinding. Disable for a namespace-admin install where a cluster-admin applies them separately; the gateway ServiceAccount name and namespace are unchanged, so a pre-created ClusterRoleBinding still matches. |
| rbac.create | bool | true |
Create the RBAC objects that grant the gateway ServiceAccount access. Disable to supply the namespaced sandbox and peer Role/RoleBinding and the cluster-scoped ClusterRole/ClusterRoleBinding out of band. The certgen hook and credential driver RBAC keep their own flags. |
| replicaCount | int | 1 |
Number of OpenShell gateway replicas. Values greater than 1 require server.externalDbSecret because the default SQLite backend is per pod. |
| resources | object | {} |
Gateway pod resource requests and limits. |
| sandbox.image.digest | string | "" |
Sandbox image digest. When set, this takes precedence over tag. |
| sandbox.image.pullPolicy | string | nil |
Sandbox image pull policy. Leave unset to use the Kubernetes image default. |
| sandbox.image.repository | string | "nvcr.io/nvidia/base/ubuntu" |
Default standalone sandbox image repository. |
| sandbox.image.tag | string | "24.04" |
Sandbox image tag. Defaults to latest when empty. |
| sandboxRuntime.image.digest | string | "" |
Sandbox runtime image digest. When set, this takes precedence over tag. |
| sandboxRuntime.image.pullPolicy | string | nil |
Sandbox runtime image pull policy. Empty uses global.image.pullPolicy. |
| sandboxRuntime.image.registry | string | "" |
Sandbox runtime image registry. Empty uses global.image.registry. |
| sandboxRuntime.image.repository | string | "openshell/sandbox" |
Sandbox runtime image repository. |
| sandboxRuntime.image.tag | string | "" |
Sandbox runtime image tag. Defaults to the chart appVersion when empty. |
| sandboxServiceAccount.annotations | object | {} |
Annotations to add to the generated sandbox service account. |
| sandboxServiceAccount.create | bool | true |
Create a service account for sandbox pods. |
| sandboxServiceAccount.name | string | "" |
Existing service account name for sandbox pods when sandboxServiceAccount.create is false. |
| securityContext.allowPrivilegeEscalation | bool | false |
Whether the gateway container can gain additional privileges. |
| securityContext.capabilities.drop | list | ["ALL"] |
Linux capabilities dropped from the gateway container. |
| securityContext.runAsNonRoot | bool | true |
Require the gateway container to run as a non-root user. |
| securityContext.runAsUser | int | 1000 |
UID assigned to the gateway container. |
| securityContext.seccompProfile.type | string | "RuntimeDefault" |
Seccomp profile applied to the gateway container. |
| server.auth.allowUnauthenticatedUsers | bool | false |
UNSAFE: accept unauthenticated CLI/user requests as a local developer principal. Intended only for trusted local Skaffold/k3d development or a fully trusted fronting proxy. Leave false for shared or production clusters. |
| server.credentialDrivers.kubernetesSecrets.createNamespace | bool | false |
Create the credential namespace. Requires a namespace other than the release namespace. The Namespace is retained on uninstall so stored credentials survive; an existing Namespace not owned by this release is left untouched. |
| server.credentialDrivers.kubernetesSecrets.enabled | bool | false |
Enable the in-tree Kubernetes Secret credential driver. WARNING: The RBAC Role grants read/write access to ALL Secrets in the configured namespace. Use a dedicated namespace to limit blast radius. |
| server.credentialDrivers.kubernetesSecrets.namespace | string | "" |
Namespace where OpenShell-managed provider Secret objects are stored. Empty = Helm release namespace. A dedicated namespace is RECOMMENDED to isolate OpenShell-managed Secrets from other workloads. |
| server.credentialDrivers.kubernetesSecrets.rbac.create | bool | true |
Create a Role/RoleBinding granting the gateway ServiceAccount read/write access to managed provider Secrets. |
| server.credentialDrivers.vault.address | string | "" |
Vault service base URL. Non-loopback endpoints must use HTTPS, for example https://vault.vault.svc.cluster.local:8200. |
| server.credentialDrivers.vault.authMethod | string | "kubernetes" |
Authentication method. Use "kubernetes" in-cluster or "token_file" for local/dev validation. |
| server.credentialDrivers.vault.caConfigMapName | string | "" |
ConfigMap containing the private Vault CA certificate bundle in the ca.crt key. Leave empty to use platform trust roots. |
| server.credentialDrivers.vault.enabled | bool | false |
Enable the in-tree Vault credential driver. |
| server.credentialDrivers.vault.kubernetesAuthMount | string | "kubernetes" |
Vault Kubernetes auth mount. |
| server.credentialDrivers.vault.kvVersion | string | "2" |
Default KV engine version. Use "1" or "2". |
| server.credentialDrivers.vault.mount | string | "secret" |
Default KV mount name. |
| server.credentialDrivers.vault.role | string | "" |
Vault Kubernetes auth role when authMethod is kubernetes. |
| server.credentialDrivers.vault.serviceAccountTokenPath | string | "/var/run/secrets/kubernetes.io/serviceaccount/token" |
ServiceAccount token path used for Kubernetes auth. |
| server.credentialDrivers.vault.timeoutSecs | string | "" |
HTTP request timeout in seconds. Empty = driver default. |
| server.credentialDrivers.vault.tokenPath | string | "" |
Mounted token file path when authMethod is token_file. |
| server.credentialStorage.existingSecret | string | "" |
Name of a pre-existing Secret containing the key-encryption key. When set, the chart does NOT generate a new Secret; it references this one instead. The Secret must contain a key named "key-encryption-key" with a base64-encoded 32-byte value. Required for GitOps workflows that render manifests with helm template (where lookup is unavailable). |
| server.dataStorageClass | string | "" |
Kubernetes StorageClass for the gateway data PVC (StatefulSet workload only). Empty (default) = omit storageClassName, using the cluster's default StorageClass. Set this on clusters with no default StorageClass, otherwise the gateway PVC stays Pending. Use a block-backed class, not NFS or other network filesystems. Immutable after install. |
| server.dataStorageSize | string | "" |
Storage request for the gateway data PVC, openshell-data (StatefulSet workload only). Uses Kubernetes quantity syntax and requires an explicit unit ("20Gi", not "20"). Empty = built-in default (1Gi). Growing an existing release's claim is PVC expansion, not a values change. |
| server.dbUrl | string | "sqlite:/var/openshell/openshell.db" |
Gateway database URL (used for the default SQLite backend). SQLite runs in WAL mode and needs a local block-backed volume, not NFS or other network filesystems. |
| server.defaultRuntimeClassName | string | "" |
Default Kubernetes runtimeClassName for sandbox pods. Applied when a CreateSandbox request does not specify one. Empty (default) = omit the field, using the cluster's default RuntimeClass. Set to a RuntimeClass name (e.g. "kata-containers", "nvidia") to apply it to all sandboxes that don't explicitly override it. |
| server.disableTls | bool | false |
Disable TLS entirely - the server listens on plaintext HTTP. Set to true when a reverse proxy / tunnel terminates TLS at the edge. |
| server.drivers.kubernetes.allowDriverConfig | bool | false |
Allow caller driver JSON; external resources still require approval. |
| server.drivers.kubernetes.operatorNamespaceFile | string | "" |
Path to a JSON file containing an array of namespace names allowed in operator mode. Hot-reloaded on change. |
| server.drivers.kubernetes.operatorNamespaceLabel | string | "" |
K8s label selector for namespace discovery in operator mode. The driver watches namespaces matching this label. |
| server.drivers.kubernetes.resourceAdmission.enabled | bool | true |
Require operator approval labels on external sandbox attachments (GPU attachments exempt). |
| server.drivers.kubernetes.resourceAdmission.requiredLabels | string | nil |
Replacement label map; null uses the built-in admission labels. Empty map is invalid when enabled. |
| server.drivers.kubernetes.workspaceMode | string | "shared" |
How workspaces map to Kubernetes namespaces. "shared" (default): all sandboxes in a single namespace. "managed": auto-creates per-workspace namespaces. "operator": uses pre-provisioned namespaces. |
| server.enableLoopbackServiceHttp | bool | true |
Enable plaintext HTTP routing for loopback sandbox service URLs on TLS-enabled gateways. |
| server.enableUserNamespaces | bool | false |
Enable Kubernetes user namespace isolation (hostUsers: false) for sandbox pods. Requires Kubernetes 1.33+ with user namespace support available (beta through 1.35, GA in 1.36+), plus a supporting container runtime and Linux 5.12+. When enabled, container UID 0 maps to an unprivileged host UID and capabilities become namespaced. |
| server.enableWebsocketTunnel | bool | false |
Enable the WebSocket tunnel used by CLI/SDK clients behind an authenticated edge proxy. Leave disabled for direct gateway installs. |
| server.externalDbSecret | string | "" |
Name of a pre-existing Opaque Secret containing a PostgreSQL connection URI (key: uri). When set, the gateway reads OPENSHELL_DB_URL from this Secret instead of using dbUrl. The Secret must contain a uri key, e.g. postgresql://user:pass@host:5432/dbname. |
| server.extraEnv | list | [] |
Additional environment variables for the gateway container, such as OTEL_* exporter settings. Entries can read from a Secret with valueFrom. |
| server.extraVolumeMounts | list | [] |
Additional volume mounts for the gateway container. |
| server.extraVolumes | list | [] |
Additional volumes for the gateway pod. |
| server.grpcEndpoint | string | "" |
gRPC endpoint sandboxes call back into the gateway. Leave empty to derive it from the chart fullname, release namespace, service port, and disableTls flag, for example https://openshell.openshell.svc.cluster.local:8080. Override only when sandboxes must reach the gateway via a different hostname (e.g. an external ingress or a host alias). |
| server.grpcRateLimit.requests | int | 0 |
Maximum gRPC requests allowed per window. Must be positive (alongside windowSeconds) to enable rate limiting; 0 (default) disables it. |
| server.grpcRateLimit.windowSeconds | int | 0 |
gRPC rate-limit window length in seconds. Must be positive (alongside requests) to enable rate limiting; 0 (default) disables it. |
| server.hostGatewayIP | string | "" |
Host gateway IP for sandbox pod hostAliases. When set, sandbox pods get hostAliases entries mapping host.docker.internal and host.openshell.internal to this IP, allowing them to reach services running on the Docker host. Auto-detected by the cluster entrypoint script. |
| server.logLevel | string | "info" |
Gateway log level. |
| server.name | string | "" |
Operator-facing gateway name. Defaults to the chart fullname so all replicas in one installation share an identity. Set explicitly when one telemetry collector receives spans from multiple namespaces or clusters. |
| server.ocsfLog.enabled | bool | false |
Write gateway OCSF events as JSONL. |
| server.ocsfLog.maxFiles | int | 7 |
Rotated files retained when rotation is daily. |
| server.ocsfLog.path | string | "/tmp/gateway-ocsf.jsonl" |
OCSF JSONL path. The default is writable in the gateway container but does not persist across restarts. To keep records, mount a volume with server.extraVolumes and server.extraVolumeMounts and set a path on it. When replicas share the volume, set subPathExpr: $(OPENSHELL_POD_NAME) on the mount so each replica writes its own file. |
| server.ocsfLog.queueCapacity | int | 10000 |
Maximum records waiting for the file writer. |
| server.ocsfLog.queueMaxBytes | int | 16777216 |
Maximum encoded bytes waiting for the file writer. |
| server.ocsfLog.rotation | string | "daily" |
Rotate the active file daily in UTC, or never. |
| server.ocsfLog.schemaVersion | string | "" |
Optional OCSF downgrade target. Empty emits native OCSF 1.8.0. Supported values: "1.1", "1.3". |
| server.oidc.adminRole | string | "" |
Role name for admin access. Leave empty (with userRole also empty) for authentication-only mode. Both must be set or both empty. |
| server.oidc.audience | string | "openshell-cli" |
Expected audience claim for the API resource server. This should match the server's --oidc-audience, NOT the CLI client ID. |
| server.oidc.caConfigMapName | string | "" |
Name of a ConfigMap containing a CA certificate bundle (key: ca.crt) for verifying the OIDC issuer's TLS certificate. Required when the issuer uses a non-public CA (e.g. OpenShift ingress, private PKI). |
| server.oidc.dangerouslyAllowInsecureHttp | bool | false |
Development only: permit cleartext OIDC requests to numeric loopback addresses. This never permits HTTP to hostnames or non-loopback addresses. |
| server.oidc.issuer | string | "" |
OIDC issuer URL (e.g. https://keycloak.example.com/realms/openshell). |
| server.oidc.jwksAllowedOrigins | list | [] |
Additional trusted HTTPS origins allowed to serve JWKS. The issuer origin is always allowed. Entries must not include a path or query. |
| server.oidc.jwksTtl | int | 3600 |
JWKS key cache TTL in seconds. Must be greater than zero. |
| server.oidc.rolesClaim | string | "" |
Dot-separated path to the roles array in the JWT claims. Keycloak: "realm_access.roles", Entra ID: "roles", Okta: "groups". |
| server.oidc.scopesClaim | string | "" |
Dot-separated path to the scopes array in the JWT claims. |
| server.oidc.userRole | string | "" |
Role name for standard user access. |
| server.otlp.endpoint | string | "" |
OTLP/gRPC collector endpoint, conventionally using port 4317. |
| server.otlp.serviceName | string | "" |
Gateway OpenTelemetry service name. Empty uses openshell-gateway. |
| server.policyValidationFailureMode | string | "fail_closed" |
Posture when a candidate sandbox policy fails validation. fail_closed deactivates the previous policy; retain_last_valid keeps it active. |
| server.providerTokenGrants.spiffe.enabled | bool | false |
Mount the SPIFFE Workload API socket into gateway and sandbox pods for dynamic provider token grants. |
| server.providerTokenGrants.spiffe.workloadApiSocketPath | string | "/spiffe-workload-api/spire-agent.sock" |
Path to the SPIFFE Workload API socket mounted into gateway and sandbox pods. |
| server.sandboxGid | string | "" |
GID for sandbox pods (sandbox_gid). Empty (default) = same as sandboxUid. Must be an integer between 1 and 4294967294. |
| server.sandboxImagePullSecrets | list | [] |
Image pull secrets attached to sandbox pods. Referenced Secrets must exist in the sandbox namespace. |
| server.sandboxJwt.gatewayId | string | "" |
Stable gateway identity embedded in iss/aud of every minted token. Defaults to the release name so HA replicas share identity. |
| server.sandboxJwt.k8sSaTokenTtlSecs | int | 3600 |
Lifetime (seconds) of the projected ServiceAccount token kubelet writes into each sandbox pod for the IssueSandboxToken bootstrap exchange. Kubelet enforces a minimum of 600s; the driver clamps values outside [600, 86400]. Default 3600 — generous, since the supervisor consumes the token within seconds of pod start. |
| server.sandboxJwt.secretDefaultMode | string | "" |
File mode for the mounted JWT signing key Secret. Default 0400 (owner-read only). Override to 0440 or 0444 if the container UID does not match the volume file owner. |
| server.sandboxJwt.signingSecretName | string | "" |
Name of the Opaque Secret holding the signing key material. Empty falls back to the chart fullname with "-jwt-keys" appended. |
| server.sandboxJwt.ttlSecs | int | 3600 |
Token TTL in seconds. Defaults to 3600 (1h). |
| server.sandboxNamespace | string | "" |
Namespace where sandbox pods are created. Defaults to the Helm release namespace (.Release.Namespace) when left empty. |
| server.sandboxUid | string | "" |
UID for sandbox pods (sandbox_uid). Empty (default) = use the OpenShift SCC namespace annotation if present, otherwise the driver default. Must be an integer between 1 and 4294967294. |
| server.serviceHostnameFormat | string | "" |
Hostname format for exposed sandbox services. Empty or "readable" keeps readable <workspace>--<sandbox>[--<service>] hostnames; "hashed" mints opaque one-label hostnames discriminated by the gateway installation name (server.name, defaulting to the release name). The name must be unique among all gateways sharing the service base domain, and renaming the installation changes every minted hostname. |
| server.telemetryEnabled | bool | true |
Enable anonymous OpenShell telemetry from the gateway and the sandbox supervisors it launches. |
| server.tls.certSecretName | string | "openshell-server-tls" |
K8s secret (type kubernetes.io/tls) with tls.crt and tls.key for the server. |
| server.tls.clientCaSecretName | string | "openshell-server-client-ca" |
K8s secret with ca.crt for client certificate verification (mTLS). Only used when enableMtls is true. Set to "" to disable client certificate verification for HTTPS-only mode. |
| server.tls.clientTlsSecretName | string | "openshell-client-tls" |
K8s secret mounted into sandbox pods for mTLS to the server. |
| server.tls.enableMtls | bool | true |
Enable mTLS client certificate authentication. When false, the gateway runs HTTPS-only without requiring client certificates (use OIDC for auth instead). Must be false when using BackendTLSPolicy because ingress proxies cannot present client certificates to the backend. |
| server.workspaceDefaultStorageSize | string | "" |
Default storage size for the workspace PVC in sandbox pods. Uses Kubernetes quantity syntax (e.g. "2Gi", "10Gi", "500Mi"). Empty = built-in default (2Gi). |
| server.workspaceStorageClass | string | "" |
Kubernetes StorageClass for the workspace PVC in sandbox pods. Empty (default) = omit storageClassName, using the cluster's default StorageClass. Set this on clusters with no default StorageClass, otherwise the workspace PVC stays Pending and the sandbox never starts. |
| service.healthPort | int | 8081 |
Gateway health service port. |
| service.metricsPort | int | 9090 |
Gateway metrics service port. |
| service.port | int | 8080 |
Gateway gRPC/HTTP service port. |
| service.type | string | "ClusterIP" |
Kubernetes Service type for the gateway. |
| serviceAccount.annotations | object | {} |
Annotations to add to the generated service account. |
| serviceAccount.create | bool | true |
Create a service account for the gateway. |
| serviceAccount.name | string | "" |
Existing service account name to use when serviceAccount.create is false. |
| supervisor.affinity | object | {} |
Affinity rules for the shared supervisor pod, rendered only when gatewayConfig selects supervisor_topology = "gateway". Top-level affinity applies only to the gateway pod. |
| supervisor.autoscaling.behavior | object | {"scaleDown":{"policies":[{"periodSeconds":120,"type":"Pods","value":1}],"stabilizationWindowSeconds":300}} |
HPA scaling behavior. Remove at most one member every two minutes, after a five-minute stabilization window, to allow ownership handoff. |
| supervisor.autoscaling.enabled | bool | false |
Enable memory-based autoscaling of the shared supervisor Deployment. Requires supervisor_topology = "gateway" and the Kubernetes resource metrics API (usually Metrics Server). When enabled, the HPA owns the replica count and supervisor.replicas is ignored. |
| supervisor.autoscaling.maxReplicas | int | 4 |
Maximum supervisor replicas. |
| supervisor.autoscaling.minReplicas | int | 2 |
Minimum supervisor replicas. Keep at least 2 for failover capacity. |
| supervisor.autoscaling.targetMemoryAverageValue | string | "512Mi" |
Target average memory working set per supervisor Pod, as a positive Kubernetes quantity. This is an absolute value, not a percentage of requests. Scaling up adds capacity for new sandboxes; it does not move existing sandboxes away from a busy member. |
| supervisor.image.digest | string | "" |
Supervisor image digest. When set, this takes precedence over tag. |
| supervisor.image.pullPolicy | string | nil |
Supervisor image pull policy. Empty uses global.image.pullPolicy. Prefer always, if_not_present, or never; the chart also accepts legacy Kubernetes spellings Always, IfNotPresent, and Never. |
| supervisor.image.registry | string | "" |
Supervisor image registry. Empty uses global.image.registry. |
| supervisor.image.repository | string | "openshell/supervisor" |
Supervisor image repository. |
| supervisor.image.tag | string | "" |
Supervisor image tag. Defaults to the chart appVersion when empty. |
| supervisor.network.additionalCaConfigMapName | string | "" |
Name of an operator-owned ConfigMap with a ca.crt key containing additional destination CA certificates for sandbox supervisors. Empty disables additional destination trust. |
| supervisor.podDisruptionBudget.enabled | bool | true |
Render a PodDisruptionBudget with maxUnavailable 1 for the shared supervisor, used only when gatewayConfig selects supervisor_topology = "gateway". Disable it only when maintenance must evict more than one supervisor Pod at a time, or evict a member while another is unavailable. |
| supervisor.podSecurityContext.fsGroup | int | 65534 |
fsGroup assigned to the shared supervisor pod. |
| supervisor.podSecurityContext.runAsGroup | int | 65534 |
Primary GID assigned to the shared supervisor pod. |
| supervisor.podSecurityContext.runAsNonRoot | bool | true |
Require the shared supervisor to run as a non-root user. |
| supervisor.podSecurityContext.runAsUser | int | 65534 |
UID assigned to the shared supervisor pod. |
| supervisor.podSecurityContext.seccompProfile.type | string | "RuntimeDefault" |
Seccomp profile applied to the shared supervisor pod. |
| supervisor.recoveryTimeoutSeconds | int | 60 |
Seconds a sandbox workload stays frozen after losing its pool member's supervision before the sandbox runtime terminates it, rendered as [openshell.gateway.supervisor_pool] recovery_timeout_secs. Must be an integer from 10 to 300. Null leaves the gateway's pool default of 60 seconds. |
| supervisor.replicas | int | 1 |
Shared supervisor pool members, used only when gatewayConfig selects supervisor_topology = "gateway". The gateway always coordinates the shared supervisor as a high-availability pool: when a member fails or leaves, another member takes over its sandboxes without restarting their workloads. 1 has no failover; use 2 or more to survive losing a member. Only the supervisor Deployment's replica count depends on this value, so changing it neither restarts the gateway nor replaces running members. Ignored when supervisor.autoscaling.enabled is true. |
| supervisor.sandboxRuntime.boundaryPort | int | 5500 |
Workload boundary TLS listener port. |
| tolerations | list | [] |
Tolerations for the gateway pod. |
| upstreamProxy | object | {"authAllowInsecure":false,"authSecret":{"key":"","name":""},"caBundle":{"configMapName":"","key":"ca.crt"},"connectByHostname":false,"noProxy":"","url":""} |
Operator-owned corporate forward proxy for policy-approved TLS egress from Kubernetes sandboxes. The workload cannot select or override it. |
| upstreamProxy.authAllowInsecure | bool | false |
Required when authSecret is configured because Basic auth to an HTTP proxy is cleartext. |
| upstreamProxy.authSecret.key | string | "" |
Secret key containing the proxy credential. |
| upstreamProxy.authSecret.name | string | "" |
Existing Secret in the sandbox namespace containing a user:pass value. |
| upstreamProxy.caBundle.configMapName | string | "" |
ConfigMap in the release namespace holding the corporate proxy CA bundle. Required for an https:// proxy with a private CA, and for a TLS-intercepting proxy that re-signs upstream certificates. The gateway reads it and stages it into each sandbox's immutable supervisor bootstrap Secret, so the anchor stays in the gateway's trust domain rather than the workload namespace. Supply only the CA that signs your proxy's certificate, or that the proxy re-signs intercepted upstream certificates with. Public roots already come from the supervisor image and its TLS stack, so a full merged trust bundle (for example an OpenShift config.openshift.io/inject-trusted-cabundle ConfigMap) adds hundreds of kilobytes of duplicated roots and can exceed the sandbox boundary's control-frame budget. |
| upstreamProxy.caBundle.key | string | "ca.crt" |
Key inside that ConfigMap. Change this only to reuse an existing ConfigMap whose key is not ca.crt. |
| upstreamProxy.connectByHostname | bool | false |
Last-resort option for hostname-filtering proxy ACLs. It lets the proxy resolve CONNECT targets. |
| upstreamProxy.noProxy | string | "" |
Comma-separated destinations that bypass only the corporate proxy. |
| upstreamProxy.url | string | "" |
Proxy URL in http://host:port or https://host:port form. An https:// proxy whose certificate is not publicly trusted also needs caBundle below. |
| workload.allowMultiReplicaStatefulSet | bool | false |
Allow replicaCount > 1 while rendering a StatefulSet. Prefer workload.kind=deployment for external database-backed multi-replica gateways; this override exists for operators who explicitly require StatefulSet identity or storage semantics. |
| workload.kind | string | "statefulset" |
Gateway workload controller kind. Use statefulset for the default SQLite database, or deployment when server.externalDbSecret points at an external database. |
| workspaceResources.enabled | bool | true |
Create the sandbox ServiceAccount, Role, RoleBinding, and NetworkPolicy from this chart. Disable for a gateway-only release. |
Autogenerated from chart metadata using helm-docs v1.14.2