Files
OpenShell/deploy/helm/openshell/values.yaml
T
Brandon Squizzato cc780d4e17 feat(helm): add BackendTLSPolicy support (#2728)
* feat(helm): add optional BackendTLSPolicy for e2e TLS

Add grpcRoute.backendTLSPolicy values to optionally create a
BackendTLSPolicy resource that enables end-to-end TLS between the
Gateway proxy and the OpenShell gateway pod. The Gateway proxy
terminates client-facing TLS and re-encrypts when connecting to the
backend, validating the pod's certificate against a user-supplied CA
ConfigMap.

This removes the requirement to set server.disableTls=true when using
HTTPS at the Gateway listener. Supported on OpenShift 4.22+ and other
platforms with BackendTLSPolicy support in the Gateway API
implementation.

Update OpenShift and ingress documentation with e2e TLS instructions.

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* feat(helm,server): auto-create backend CA ConfigMap in certgen hook

Extend the generate-certs command with --backend-ca-configmap-name and
--backend-ca-source-secret flags. When BackendTLSPolicy is enabled, the
certgen pre-install hook creates the CA ConfigMap automatically:

- pkiInitJob mode (default): uses the CA from the generated PKI bundle.
  Fully automatic on first install.
- cert-manager mode: reads ca.crt from the server TLS Secret. On first
  install the Secret does not exist yet (cert-manager reconciles after
  templates are applied), so the ConfigMap is created on the first helm
  upgrade. Logs a warning on the initial skip.

The caCertificateConfigMapName value now defaults to <fullname>-backend-ca
when empty, so users only need to set backendTLSPolicy.enabled=true.

Update certgen RBAC to include configmaps get/create when the feature is
enabled. Add CLI arg parsing tests for the new flags.

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* refactor(helm): add server.tls.enableMtls flag for mTLS control

Replace automatic mTLS disabling based on BackendTLSPolicy with an
explicit server.tls.enableMtls flag that defaults to true. The user is
now responsible for setting this to false when using BackendTLSPolicy,
as ingress proxies cannot present client certificates to backends.

Updated:
- values.yaml: Added server.tls.enableMtls (default true)
- gateway-config.yaml: Check enableMtls instead of backendTLSPolicy
- _gateway-workload.tpl: Check enableMtls for client CA mount
- Tests: Updated to use enableMtls flag
- Docs: Added enableMtls=false to BackendTLSPolicy examples
- README: Document new flag and BackendTLSPolicy requirement

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* docs(helm): clarify cert-manager backend CA ConfigMap workflow

Update documentation to explain the two-step install process required
when using cert-manager with BackendTLSPolicy:

1. helm install - cert-manager issues the server certificate, but the
   certgen hook can't create the backend CA ConfigMap yet (cert-manager
   reconciles after templates are applied)
2. helm upgrade - certgen hook reads the CA from the cert-manager-issued
   certificate and creates the ConfigMap

Previously, the docs said "created on first upgrade" without explaining
why or that the feature won't work until then. The updated docs now:
- Explain the timing issue (cert-manager reconciles after chart install)
- Provide clear steps for the cert-manager workflow
- Note that pkiInitJob (default) creates it immediately on install
- Clarify that users must wait for the Certificate to be Ready before
  running the second upgrade

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* fix(docs): remove incorrect external hostname requirement for BackendTLSPolicy

BackendTLSPolicy validates the backend certificate against the service FQDN
(e.g., openshell.openshell.svc.cluster.local), not the external hostname.
The external hostname only needs to be on the Gateway listener certificate
for client-facing TLS.

The default certManager.serverDnsNames already includes all required service
FQDN variants, so no configuration is needed for BackendTLSPolicy to work.

Fixed incorrect documentation that claimed:
- "The server certificate SAN list must include the external hostname"
- Users need to "configure certManager.serverDnsNames with the external hostname"

Removed the unnecessary pkiInitJob.serverDnsNames override from the example
and clarified that:
- Gateway listener certificate needs the external hostname (for clients)
- Backend certificate needs the service FQDN (for Gateway proxy)
- The service FQDN is already in the defaults

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* docs: clarify ACME with LetsEncrypt reference

Change all references from "ACME issuer" to "LetsEncrypt/ACME issuer"
to help users understand that LetsEncrypt is the most common ACME
provider and what ACME means in practice.

Updated:
- docs/kubernetes/managing-certificates.mdx
- docs/kubernetes/openshift.mdx
- deploy/helm/openshell/values.yaml
- deploy/helm/openshell/README.md
- deploy/helm/openshell/ci/values-openshift-route-cert-manager.yaml

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* docs(openshift): restructure end-to-end TLS options and clarify Gateway hostname

Reorganize the OpenShift production deployment documentation:

1. Changed main section from "Production Deployments" to "Options for
   end-to-end TLS" for better clarity

2. Renamed subsections for consistency and clarity:
   - "End-to-end TLS using Gateway API and BackendTLSPolicy (OpenShift 4.22+)"
   - "End-to-end TLS using pass-through Route (all OpenShift versions)"

3. Clarified that the Gateway hostname is typically a wildcard:
   "typically a wildcard like *.openshell-ingress-gw.example.com"

4. Removed the recommendation to copy the cluster's wildcard certificate
   from openshift-ingress namespace, as this is not a recommended
   security best practice

These changes make it clearer that users have two end-to-end TLS options
and help them understand the typical naming pattern for Gateway hostnames.

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* feat(helm): eliminate two-stage install for BackendTLSPolicy with cert-manager

When using BackendTLSPolicy with cert-manager, the certgen hook now polls
for up to 90 seconds waiting for cert-manager to issue the TLS certificate
before creating the backend CA ConfigMap. This eliminates the need for a
second `helm upgrade` in most cases.

The hook polls every 2 seconds with progress logging every 10 seconds.
If cert-manager takes longer than 90 seconds, the hook times out gracefully
and logs a warning, preserving the fallback to manual ConfigMap creation
or a second upgrade.

The Job's activeDeadlineSeconds is 120s, so the 90s timeout leaves 30s
margin for ConfigMap creation and hook completion.

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* feat(helm): add configurable timeout for certgen hook

Add `pkiInitJob.timeoutSeconds` Helm value (default 120) to control how long
the certgen hook Job can run. When using cert-manager with BackendTLSPolicy,
the hook polls for (timeoutSeconds - 30) seconds to leave margin for ConfigMap
creation and cleanup.

This allows users to increase the timeout for environments where cert-manager
takes longer than 90 seconds to issue certificates, without requiring code
changes.

Example usage:
```yaml
pkiInitJob:
  timeoutSeconds: 180  # Hook polls for 150 seconds
```

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* docs(helm): document configurable certgen timeout

Update documentation to mention the pkiInitJob.timeoutSeconds value and
how it affects the cert-manager polling behavior when using BackendTLSPolicy.

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* feat(helm): add configurable failure behavior for certgen timeout

Add `pkiInitJob.failOnTimeout` Helm value (default false) to control whether
the certgen hook fails or succeeds when cert-manager does not issue a
certificate within the polling timeout.

When false (default), the hook succeeds with a warning and users can run
`helm upgrade` after cert-manager issues the certificate to create the
backend CA ConfigMap. This provides backwards-compatible behavior.

When true, the hook fails immediately if the timeout is reached, providing
clear feedback that BackendTLSPolicy is non-functional. This is useful for
strict validation requirements where incomplete installs should fail fast.

Example usage:
```yaml
pkiInitJob:
  timeoutSeconds: 180
  failOnTimeout: true  # Fail install if cert-manager takes >150s
```

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* feat(helm): change failOnTimeout default to true and add troubleshooting docs

Change `pkiInitJob.failOnTimeout` default from false to true to provide
immediate feedback when cert-manager does not issue certificates within
the polling timeout. This prevents silent failures where BackendTLSPolicy
is non-functional but the install appears to succeed.

Add comprehensive troubleshooting section to docs/kubernetes/ingress.mdx
documenting the specific error "TLS error: Secret is not supplied by SDS"
that occurs when the backend CA ConfigMap is missing, with step-by-step
resolution instructions.

Updated comments in values.yaml to clearly document the default behavior
and explain when administrators might see connectivity errors if they
override the default to failOnTimeout=false.

BREAKING CHANGE: pkiInitJob.failOnTimeout now defaults to true. Helm
installs will fail if cert-manager takes longer than (timeoutSeconds - 30)
seconds to issue certificates. To restore the old behavior of allowing
installs to succeed with a warning, set `pkiInitJob.failOnTimeout=false`.

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* fix(helm): make cert-manager resources pre-install hooks to fix ordering

Make Certificate and Issuer resources run as pre-install/pre-upgrade hooks
with weight -30, before the certgen hook (weight -20). This fixes the
chicken-and-egg problem where the certgen hook was waiting for Secrets
created by Certificates that hadn't been created yet.

**Hook ordering:**
1. Certificate and Issuer resources created (weight -30)
2. cert-manager issues certificates and creates Secrets
3. certgen hook runs (weight -20), finds Secrets, creates ConfigMap
4. Main resources (StatefulSet, Service, etc.) created

Previously, the certgen pre-install hook would run before any resources
were created, poll for a non-existent Secret, timeout, and fail. The
Certificate resources would never get created because Helm waits for
all pre-install hooks to succeed before creating main resources.

This fix allows single-stage installs to work reliably as long as
cert-manager can issue certificates within the polling timeout.

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* feat(helm): add validation to prevent enableMtls with BackendTLSPolicy

Add Helm chart validation that fails the install if both
server.tls.enableMtls=true and grpcRoute.backendTLSPolicy.enabled=true
are set, since this is an invalid configuration.

BackendTLSPolicy requires mTLS to be disabled because the Gateway proxy
cannot present client certificates to the backend. This validation provides
immediate, clear feedback at install time rather than allowing the
misconfiguration to be discovered through runtime errors.

Example error message:
```
Error: grpcRoute.backendTLSPolicy requires mTLS to be disabled because
the Gateway proxy cannot present client certificates to the backend;
set server.tls.enableMtls=false
```

Also updated documentation to mention this validation check.

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* docs(helm): clarify pkiInitJob.timeoutSeconds polling behavior

Improve documentation to clearly explain that pkiInitJob.timeoutSeconds
controls the Job deadline, but the actual polling timeout is
(timeoutSeconds - 30) to reserve 30 seconds for ConfigMap creation
and cleanup.

Added concrete example: "timeoutSeconds=180 allows 150 seconds of polling"
to make the relationship explicit and avoid confusion where users might
expect the hook to poll for the full timeout value.

Updated both values.yaml inline comments and ingress.mdx documentation
for consistency.

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* docs(openshift): remove outdated two-stage install instructions

Update OpenShift documentation to reflect that single-stage installs now
work with cert-manager and BackendTLSPolicy. The Certificate resources
run as pre-install hooks (weight -30) before certgen (weight -20),
allowing the hook to poll for and find the issued certificates.

Removed the outdated two-step process:
1. helm install (cert-manager issues cert, hook logs warning)
2. helm upgrade (hook creates ConfigMap)

Replaced with current single-stage behavior:
- Certificate resources created as pre-install hooks
- certgen hook polls for up to 90 seconds (configurable)
- Single helm install succeeds in most cases
- Fails fast by default if timeout reached

This brings openshift.mdx in line with the already-updated ingress.mdx
documentation.

Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>

* fix(helm): make pkiInitJob.timeoutSeconds the actual polling duration

The timeout value now represents the actual polling time that users
experience when waiting for cert-manager to issue certificates.
The Job activeDeadlineSeconds is set to (timeoutSeconds + 30) to
allow buffer time for ConfigMap creation and cleanup.

Previously, the hook polled for (timeoutSeconds - 30) seconds, which
was confusing when users set timeoutSeconds=180 and only got 150
seconds of actual polling.

Updated documentation in values.yaml, ingress.mdx, and openshift.mdx
to reflect the clearer behavior.

Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com>

* docs(helm): update values.yaml and README with correct polling duration

Updated the caCertificateConfigMapName description to reflect that the
hook polls for exactly pkiInitJob.timeoutSeconds seconds, not
(timeoutSeconds - 30) seconds.

Regenerated README.md with helm-docs.

Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com>

* docs(kubernetes): add OIDC configuration to helm install and CLI examples

Updated all helm install and openshell gateway add examples in ingress.mdx
and openshift.mdx to include OIDC issuer and audience configuration.

Examples now use concrete placeholder values:
- OIDC issuer: https://keycloak.example.com/realms/openshell
- OIDC audience: openshell-cli
- Hostname: gateway.example.com
- ClusterIssuer: letsencrypt-prod

This makes it clearer how to configure OIDC authentication, which is
required when using BackendTLSPolicy or HTTPS termination since the
Gateway proxy cannot present client certificates to the backend.

Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com>

* docs(kubernetes): explicitly list OIDC client ID in gateway add examples

Added --oidc-client-id openshell-cli to all openshell gateway add
commands in ingress.mdx and openshift.mdx, making the default client
ID explicit in the examples even though it's the CLI default.

This improves clarity and helps users understand the complete OIDC
configuration needed for gateway registration.

Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com>

* fix(helm): address PR review feedback for BackendTLSPolicy

- Read backend CA from the authoritative server Secret instead of the
  in-memory PKI bundle so enabling BackendTLSPolicy on an existing
  release uses the CA that actually signed the server certificate.
- Reconcile the backend CA ConfigMap on every hook run (compare and
  update) instead of skipping when it already exists, so CA rotations
  propagate automatically.
- Remove hook annotations from cert-manager Issuer/Certificate resources
  so they remain regular release objects managed by Helm lifecycle. Split
  the cert-manager backend CA ConfigMap creation into a separate
  post-install/post-upgrade hook Job that polls after cert-manager
  Certificate resources are applied.
- Update architecture/gateway.md, docs/reference/gateway-config.mdx,
  debug-openshell-cluster skill, and helm-dev-environment skill with
  BackendTLSPolicy, backend CA ConfigMap, enableMtls, and timeout
  documentation.

Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com>
Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* fix(docs): resolve markdown lint errors in helm README and kubernetes docs

Escape inline HTML angle brackets in README.md template placeholders,
remove trailing spaces, and add blank lines around fenced code blocks
in numbered lists.

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* Update docs

* fix(helm): escape inline HTML in values.yaml descriptions and sync mise lockfile

Wrap `<fullname>` and `<namespace>` template placeholders in backticks
so markdownlint does not flag them as inline HTML (MD033). Regenerate
mise.lock to match current mise.toml after rebase onto main.

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* Run 'mise lock'

* fix: align mise.lock with CI mise version output

The lockfile was regenerated locally with mise 2026.8.10 which resolves
uv Linux artifacts to gnu variants and adds provenance_verified fields,
but CI uses v2026.4.25 which produces musl variants without those fields.

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

* fix(docs): correct cert-manager hook ordering and clientCaSecretName comment

Update ingress.mdx and openshift.mdx to describe Certificate resources
as regular release objects with a post-install/post-upgrade Job, matching
the current implementation and architecture/gateway.md.

Fix values.yaml clientCaSecretName comment to state that "" disables
client certificate verification, matching the helper and access-control
docs.

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>

---------

Signed-off-by: Brandon Squizzato <bsquizza@redhat.com>
Signed-off-by: Brandon Squizzato <bsquizzato@nvidia.com>
Signed-off-by: Brandon Squizzato <bsquizza@nvidia.com>
2026-09-14 17:39:15 +00:00

615 lines
30 KiB
YAML

# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Default values for OpenShell
# -- Number of OpenShell gateway replicas. Values greater than 1 require
# server.externalDbSecret because the default SQLite backend is per pod.
replicaCount: 1
workload:
# -- Gateway workload controller kind. Use `statefulset` for the default
# SQLite database, or `deployment` when server.externalDbSecret points at an
# external database.
kind: statefulset
# -- Allow replicaCount > 1 while rendering a StatefulSet. Prefer
# workload.kind=deployment for external database-backed multi-replica
# gateways; this override exists for operators who explicitly require
# StatefulSet identity or storage semantics.
allowMultiReplicaStatefulSet: false
image:
# -- Gateway image repository.
repository: ghcr.io/nvidia/openshell/gateway
# -- Gateway image pull policy.
pullPolicy: IfNotPresent
# -- Gateway image tag. Defaults to the chart appVersion when empty.
tag: ""
# Supervisor image for the openshell-sandbox binary injected into sandbox pods.
# The default repository and empty tag use the version-pinned image built into
# the gateway. Changing the repository or setting a tag enables a Helm override.
supervisor:
image:
# -- Supervisor image repository. Changing it uses the effective gateway image tag unless tag is also set.
repository: ghcr.io/nvidia/openshell/supervisor
# -- Sandbox supervisor pull policy. Leave unset to use the Kubernetes
# image default. Prefer always, if_not_present, or never; the chart also
# accepts legacy Kubernetes spellings Always, IfNotPresent, and Never.
pullPolicy: null
# -- Supervisor image tag override. Empty uses the version pinned into the gateway unless repository is changed.
tag: ""
# -- How the supervisor binary is delivered into sandbox pods.
# Empty (default) = auto-detect from cluster version:
# K8s >= v1.35 -> "image-volume" (ImageVolume enabled by default; GA in v1.36)
# K8s < v1.35 -> "init-container" (copies via init container + emptyDir)
# On K8s v1.33-v1.34 with the ImageVolume feature gate manually enabled,
# set this to "image-volume" explicitly.
sideloadMethod: ""
# -- Supervisor pod topology for Kubernetes sandboxes.
# "combined" runs the current single supervisor container in the agent pod.
# "sidecar" runs network enforcement in a dedicated sidecar and the process
# supervisor as a low-capability wrapper in the agent container.
topology: "combined"
sidecar:
# -- UID for relaxed long-running network sidecars in sidecar topology.
# Strict process/binary-aware sidecars run as UID 0 so Kubernetes grants
# the required /proc inspection capabilities into the effective set. The
# network init container installs nftables rules that exempt the effective
# sidecar UID.
proxyUid: 1337
# -- Keep process/binary-aware network policy enabled in sidecar topology.
# When false, the network sidecar runs as proxyUid, drops the extra /proc
# inspection capabilities, and enforces endpoint/L7 policy without matching
# policy.binaries.
processBinaryAwareNetworkPolicy: true
# -- Operator-owned corporate forward proxy for policy-approved TLS egress
# from Kubernetes sandboxes. The workload cannot select or override it.
upstreamProxy:
# -- HTTP proxy URL in http://host:port form. HTTPS-to-proxy is not supported.
url: ""
# -- Comma-separated destinations that bypass only the corporate proxy.
noProxy: ""
authSecret:
# -- Existing Secret in the sandbox namespace containing a user:pass value.
name: ""
# -- Secret key containing the proxy credential.
key: ""
# -- Required when authSecret is configured because Basic auth to an HTTP proxy is cleartext.
authAllowInsecure: false
# -- Last-resort option for hostname-filtering proxy ACLs. It lets the proxy resolve CONNECT targets.
connectByHostname: false
# -- Image pull secrets attached to gateway and helper pods.
imagePullSecrets: []
# -- Override the chart name used in generated resource names.
nameOverride: "openshell"
# -- Override the full generated resource name.
fullnameOverride: ""
serviceAccount:
# -- Create a service account for the gateway.
create: true
# -- Annotations to add to the generated service account.
annotations: {}
# -- Existing service account name to use when serviceAccount.create is false.
name: ""
sandboxServiceAccount:
# -- Create a service account for sandbox pods.
create: true
# -- Annotations to add to the generated sandbox service account.
annotations: {}
# -- Existing service account name for sandbox pods when sandboxServiceAccount.create is false.
name: ""
# Namespace-scoped resources needed to run sandboxes. Disable this when the
# gateway and workspace prerequisites are managed as separate Helm releases
# using the openshell-workspace chart.
workspaceResources:
# -- Create the sandbox ServiceAccount, Role, RoleBinding, and NetworkPolicy
# from this chart. Disable for a gateway-only release.
enabled: true
# -- Extra annotations to add to the gateway pod.
podAnnotations: {}
# -- Extra labels to add to the gateway pod.
podLabels: {}
podSecurityContext:
# -- fsGroup assigned to the gateway pod.
fsGroup: 1000
securityContext:
# -- Require the gateway container to run as a non-root user.
runAsNonRoot: true
# -- UID assigned to the gateway container.
runAsUser: 1000
# -- Whether the gateway container can gain additional privileges.
allowPrivilegeEscalation: false
capabilities:
# -- Linux capabilities dropped from the gateway container.
drop:
- ALL
service:
# -- Kubernetes Service type for the gateway.
type: ClusterIP
# -- Gateway gRPC/HTTP service port.
port: 8080
# -- Gateway health service port.
healthPort: 8081
# -- Gateway metrics service port.
metricsPort: 9090
# Agent Sandbox is a cluster-scoped prerequisite for the Kubernetes compute
# driver. OpenShell deliberately does not install its CRDs or controller.
# Enable this check for live Helm installs to fail before creating gateway
# resources when neither supported Sandbox API is served. Disable it for
# offline `helm template` and lint workflows, which cannot discover APIs.
agentSandbox:
preflight:
# -- Check the live cluster for a supported Agent Sandbox API before rendering gateway resources. Disable only for offline rendering and linting.
enabled: true
# Pod restart behavior and health probe tuning.
podLifecycle:
# -- Grace period, in seconds, before Kubernetes terminates the gateway pod.
terminationGracePeriodSeconds: 5
probes:
startup:
# -- Startup probe period, in seconds.
periodSeconds: 2
# -- Startup probe timeout, in seconds.
timeoutSeconds: 1
# -- Startup probe failure threshold before the container is killed.
failureThreshold: 30
liveness:
# -- Liveness probe initial delay, in seconds.
initialDelaySeconds: 2
# -- Liveness probe period, in seconds.
periodSeconds: 5
# -- Liveness probe timeout, in seconds.
timeoutSeconds: 1
# -- Liveness probe failure threshold before the container is restarted.
failureThreshold: 3
readiness:
# -- Readiness probe initial delay, in seconds.
initialDelaySeconds: 1
# -- Readiness probe period, in seconds.
periodSeconds: 2
# -- Readiness probe timeout, in seconds.
timeoutSeconds: 1
# -- Readiness probe failure threshold before the pod is marked not ready.
failureThreshold: 3
# -- Gateway pod resource requests and limits.
resources: {}
# -- Node selector for the gateway pod.
nodeSelector: {}
# -- Tolerations for the gateway pod.
tolerations: []
# -- Affinity rules for the gateway pod.
affinity: {}
# Server configuration
server:
# -- Operator-facing gateway name. Defaults to the chart fullname so all
# replicas in one installation share an identity. Set explicitly when one
# telemetry collector receives spans from multiple namespaces or clusters.
name: ""
# -- Gateway log level.
logLevel: info
# OpenTelemetry trace export over OTLP/gRPC. Leave endpoint empty to disable.
otlp:
# -- OTLP/gRPC collector endpoint, conventionally using port 4317.
endpoint: ""
# -- Gateway OpenTelemetry service name. Empty uses openshell-gateway.
serviceName: ""
# -- Enable anonymous OpenShell telemetry from the gateway and the sandbox
# supervisors it launches.
telemetryEnabled: true
# -- Namespace where sandbox pods are created. Defaults to the Helm release
# namespace (.Release.Namespace) when left empty.
sandboxNamespace: ""
# -- Gateway database URL (used for the default SQLite backend).
dbUrl: "sqlite:/var/openshell/openshell.db"
# -- Name of a pre-existing Opaque Secret containing a PostgreSQL
# connection URI (key: uri). When set, the gateway reads OPENSHELL_DB_URL
# from this Secret instead of using dbUrl. The Secret must contain a
# `uri` key, e.g. postgresql://user:pass@host:5432/dbname.
externalDbSecret: ""
# -- Default sandbox image used when requests do not specify one.
sandboxImage: "ghcr.io/nvidia/openshell-community/sandboxes/base:latest"
# -- Pull policy for sandbox pods. Leave unset to use the Kubernetes image
# default (Always for :latest, IfNotPresent otherwise). Prefer always,
# if_not_present, or never; the chart also accepts legacy Kubernetes spellings
# Always, IfNotPresent, and Never.
sandboxImagePullPolicy: null
# -- Image pull secrets attached to sandbox pods. Referenced Secrets must exist
# in the sandbox namespace.
sandboxImagePullSecrets: []
# -- Default storage size for the workspace PVC in sandbox pods.
# Uses Kubernetes quantity syntax (e.g. "2Gi", "10Gi", "500Mi").
# Empty = built-in default (2Gi).
workspaceDefaultStorageSize: ""
# -- Kubernetes StorageClass for the workspace PVC in sandbox pods.
# Empty (default) = omit storageClassName, using the cluster's default
# StorageClass. Set this on clusters with no default StorageClass, otherwise
# the workspace PVC stays Pending and the sandbox never starts.
workspaceStorageClass: ""
# -- Default Kubernetes runtimeClassName for sandbox pods.
# Applied when a CreateSandbox request does not specify one.
# Empty (default) = omit the field, using the cluster's default RuntimeClass.
# Set to a RuntimeClass name (e.g. "kata-containers", "nvidia") to apply it
# to all sandboxes that don't explicitly override it.
defaultRuntimeClassName: ""
# -- gRPC endpoint sandboxes call back into the gateway. Leave empty to derive
# it from the chart fullname, release namespace, service port, and
# disableTls flag, for example https://openshell.openshell.svc.cluster.local:8080.
# Override only when sandboxes must reach the gateway via a different
# hostname (e.g. an external ingress or a host alias).
grpcEndpoint: ""
# The gateway terminates TLS directly. Its client CA authenticates sandbox
# callbacks; OIDC-enabled listeners permit bearer-only user clients while
# still validating any client certificate they present.
# -- Host gateway IP for sandbox pod hostAliases. When set, sandbox pods get
# hostAliases entries mapping host.docker.internal and host.openshell.internal
# to this IP, allowing them to reach services running on the Docker host.
# Auto-detected by the cluster entrypoint script.
hostGatewayIP: ""
# -- Enable Kubernetes user namespace isolation (hostUsers: false) for sandbox
# pods. Requires Kubernetes 1.33+ with user namespace support available
# (beta through 1.35, GA in 1.36+), plus a supporting container runtime and
# Linux 5.12+. When enabled, container UID 0 maps to an unprivileged host
# UID and capabilities become namespaced.
enableUserNamespaces: false
# -- Kubernetes AppArmor profile requested for sandbox agent containers.
# Default Unconfined avoids runtime/default AppArmor blocking the supervisor's
# network namespace mount setup on AppArmor-enabled nodes. Set to "" to omit
# the field, "RuntimeDefault" to force the runtime default profile, or
# "Localhost/profile-name" for an operator-managed localhost profile.
appArmorProfile: "Unconfined"
# Kubernetes compute driver settings.
drivers:
kubernetes:
# -- How workspaces map to Kubernetes namespaces.
# "shared" (default): all sandboxes in a single namespace.
# "managed": auto-creates per-workspace namespaces.
# "operator": uses pre-provisioned namespaces.
workspaceMode: "shared"
# -- K8s label selector for namespace discovery in operator mode.
# The driver watches namespaces matching this label.
operatorNamespaceLabel: ""
# -- Path to a JSON file containing an array of namespace names
# allowed in operator mode. Hot-reloaded on change.
operatorNamespaceFile: ""
# -- Disable TLS entirely - the server listens on plaintext HTTP.
# Set to true when a reverse proxy / tunnel terminates TLS at the edge.
disableTls: false
# -- Enable plaintext HTTP routing for loopback sandbox service URLs on
# TLS-enabled gateways.
enableLoopbackServiceHttp: true
# -- Posture when a candidate sandbox policy fails validation. `fail_closed`
# deactivates the previous policy; `retain_last_valid` keeps it active.
policyValidationFailureMode: fail_closed
# Optional gateway-wide gRPC request rate limit. Applies only to gRPC API
# traffic after protocol multiplexing; health, metrics, and loopback service
# HTTP routes are not rate limited. Both values must be positive to enable the
# limit, otherwise it is omitted from the rendered config and stays disabled.
grpcRateLimit:
# -- Maximum gRPC requests allowed per window. Must be positive (alongside
# windowSeconds) to enable rate limiting; 0 (default) disables it.
requests: 0
# -- gRPC rate-limit window length in seconds. Must be positive (alongside
# requests) to enable rate limiting; 0 (default) disables it.
windowSeconds: 0
# Default credential storage settings (used when no credential driver is
# enabled). The gateway encrypts provider credentials in the database using
# AES-256-GCM with a key-encryption key (KEK). By default, the Helm chart
# generates and retains a KEK Secret. For GitOps / helm-template workflows
# where `lookup` is unavailable, reference a pre-created Secret instead.
credentialStorage:
# -- Name of a pre-existing Secret containing the key-encryption key.
# When set, the chart does NOT generate a new Secret; it references this
# one instead. The Secret must contain a key named "key-encryption-key"
# with a base64-encoded 32-byte value. Required for GitOps workflows that
# render manifests with `helm template` (where `lookup` is unavailable).
existingSecret: ""
# Provider credential drivers store provider credential secret material in an
# external or native backend. When no driver is enabled, the gateway uses its
# default encrypted database credential storage with a retained Kubernetes
# Secret for the shared key-encryption key.
credentialDrivers:
kubernetesSecrets:
# -- Enable the in-tree Kubernetes Secret credential driver.
# WARNING: The RBAC Role grants read/write access to ALL Secrets in the
# configured namespace. Use a dedicated namespace to limit blast radius.
enabled: false
# -- Namespace where OpenShell-managed provider Secret objects are stored.
# Empty = Helm release namespace. A dedicated namespace is RECOMMENDED
# to isolate OpenShell-managed Secrets from other workloads.
namespace: ""
# -- Deprecated compatibility field. Credential storage no longer supports user-authored namespace references.
allowReferenceNamespace: false
rbac:
# -- Create a Role/RoleBinding granting the gateway ServiceAccount read/write access to managed provider Secrets.
create: true
vault:
# -- Enable the in-tree Vault credential driver.
enabled: false
# -- Vault service base URL, for example http://vault.vault.svc.cluster.local:8200.
address: ""
# -- Default KV mount name.
mount: secret
# -- Default KV engine version. Use "1" or "2".
kvVersion: "2"
# -- Authentication method. Use "kubernetes" in-cluster or "token_file" for local/dev validation.
authMethod: kubernetes
# -- Vault Kubernetes auth role when authMethod is kubernetes.
role: ""
# -- Vault Kubernetes auth mount.
kubernetesAuthMount: kubernetes
# -- ServiceAccount token path used for Kubernetes auth.
serviceAccountTokenPath: /var/run/secrets/kubernetes.io/serviceaccount/token
# -- Mounted token file path when authMethod is token_file.
tokenPath: ""
# -- HTTP request timeout in seconds. Empty = driver default.
timeoutSecs: ""
auth:
# -- UNSAFE: accept unauthenticated CLI/user requests as a local developer
# principal. Intended only for trusted local Skaffold/k3d development or a
# fully trusted fronting proxy. Leave false for shared or production clusters.
allowUnauthenticatedUsers: false
tls:
# -- K8s secret (type kubernetes.io/tls) with tls.crt and tls.key for the server.
certSecretName: openshell-server-tls
# -- Enable mTLS client certificate authentication. When false, the gateway
# runs HTTPS-only without requiring client certificates (use OIDC for auth
# instead). Must be false when using BackendTLSPolicy because ingress
# proxies cannot present client certificates to the backend.
enableMtls: true
# -- K8s secret with ca.crt for client certificate verification (mTLS).
# Only used when enableMtls is true. Set to "" to disable client certificate
# verification for HTTPS-only mode.
clientCaSecretName: openshell-server-client-ca
# -- K8s secret mounted into sandbox pods for mTLS to the server.
clientTlsSecretName: openshell-client-tls
# Gateway-minted sandbox JWT signing keys. The certgen hook generates an
# Ed25519 keypair and writes it to a secret containing signing.pem (PKCS#8),
# public.pem (SPKI), and kid (plain text). The hook runs in full PKI mode when
# pkiInitJob.enabled=true unless certManager.enabled=true, which takes
# precedence and runs the hook in JWT-only mode.
sandboxJwt:
# -- Name of the Opaque Secret holding the signing key material. Empty
# falls back to the chart fullname with "-jwt-keys" appended.
signingSecretName: ""
# -- Stable gateway identity embedded in iss/aud of every minted token.
# Defaults to the release name so HA replicas share identity.
gatewayId: ""
# -- Token TTL in seconds. Defaults to 3600 (1h).
ttlSecs: 3600
# -- Lifetime (seconds) of the projected ServiceAccount token kubelet
# writes into each sandbox pod for the IssueSandboxToken bootstrap
# exchange. Kubelet enforces a minimum of 600s; the driver clamps
# values outside [600, 86400]. Default 3600 — generous, since the
# supervisor consumes the token within seconds of pod start.
k8sSaTokenTtlSecs: 3600
# -- File mode for the mounted JWT signing key Secret. Default 0400
# (owner-read only). Override to 0440 or 0444 if the container UID
# does not match the volume file owner.
secretDefaultMode: ""
# Dynamic provider token grants. When SPIFFE is enabled here, both the
# gateway and sandbox supervisors mount the SPIFFE Workload API socket so
# token-exchange profiles can use gateway- and sandbox-scoped JWT-SVIDs.
# Supervisor-to-gateway authentication still uses gateway-minted sandbox JWTs.
providerTokenGrants:
spiffe:
# -- Mount the SPIFFE Workload API socket into gateway and sandbox pods for dynamic provider token grants.
enabled: false
# -- Path to the SPIFFE Workload API socket mounted into gateway and sandbox pods.
workloadApiSocketPath: /spiffe-workload-api/spire-agent.sock
# OIDC (OpenID Connect) configuration for JWT-based authentication.
# When issuer is set, the server validates Bearer tokens on gRPC requests.
oidc:
# -- OIDC issuer URL (e.g. https://keycloak.example.com/realms/openshell).
issuer: ""
# -- Expected audience claim for the API resource server.
# This should match the server's --oidc-audience, NOT the CLI client ID.
audience: "openshell-cli"
# -- JWKS key cache TTL in seconds. Must be greater than zero.
jwksTtl: 3600
# -- Dot-separated path to the roles array in the JWT claims.
# Keycloak: "realm_access.roles", Entra ID: "roles", Okta: "groups".
rolesClaim: ""
# -- Role name for admin access. Leave empty (with userRole also empty) for
# authentication-only mode. Both must be set or both empty.
adminRole: ""
# -- Role name for standard user access.
userRole: ""
# -- Dot-separated path to the scopes array in the JWT claims.
scopesClaim: ""
# -- Name of a ConfigMap containing a CA certificate bundle (key: ca.crt)
# for verifying the OIDC issuer's TLS certificate. Required when the
# issuer uses a non-public CA (e.g. OpenShift ingress, private PKI).
caConfigMapName: ""
# NetworkPolicy restricting SSH ingress on sandbox pods to the gateway only.
networkPolicy:
# -- Restrict SSH ingress on sandbox pods to the gateway. In managed mode,
# the driver applies the equivalent policy to each workspace namespace.
enabled: true
# Built-in TLS PKI bootstrap via a pre-install/pre-upgrade hook Job.
# Runs `openshell-gateway generate-certs` to create the server and client TLS
# Secrets in-cluster. Key material is written directly to K8s Secrets and
# never appears in Helm release history. Idempotent: existing secrets are
# left untouched on upgrade. Reuses the gateway image - no extra image to
# mirror in air-gapped environments.
#
# The server certificate already includes the built-in cluster SANs
# (`openshell`, `openshell.openshell.svc`, the cluster.local FQDN, `localhost`,
# `openshell.localhost`, `*.openshell.localhost`, `host.docker.internal`, and
# `127.0.0.1`) baked into the gateway binary. The lists below are additional
# SANs appended on top. Wildcard DNS SANs also enable sandbox service URLs under
# that domain, for example `*.apps.example.com` enables
# `<sandbox>--<service>.apps.example.com`.
pkiInitJob:
# -- Run a pre-install/pre-upgrade Job that creates gateway and client mTLS
# Secrets. When certManager.enabled=true, cert-manager owns TLS and this same
# hook runs in JWT-only mode even if pkiInitJob.enabled remains true.
enabled: true
# -- Extra DNS SANs to append to the server certificate.
serverDnsNames: []
# -- Extra IP SANs to append to the server certificate.
serverIpAddresses: []
# -- Maximum time in seconds for the certgen hook to poll for cert-manager
# certificates. When using cert-manager with BackendTLSPolicy, the hook
# polls for this many seconds waiting for the certificate to be issued,
# then creates the backend CA ConfigMap. The Job deadline is set to
# (timeoutSeconds + 30) to allow time for ConfigMap creation and cleanup.
# Increase this if cert-manager takes longer than 120 seconds to issue
# certificates.
timeoutSeconds: 120
# -- Fail the helm install/upgrade if cert-manager does not issue the
# certificate within the polling timeout. When true (default), the install
# fails immediately if the timeout is reached, providing clear feedback that
# BackendTLSPolicy is non-functional. When false, the hook succeeds with a
# warning and you can run `helm upgrade` after cert-manager issues the
# certificate to create the backend CA ConfigMap. If you set this to false and
# see "TLS error: Secret is not supplied by SDS" when connecting to the gateway,
# check if the TLS secret exists and run `helm upgrade` to create the ConfigMap.
failOnTimeout: true
# cert-manager Certificate/Issuer resources (requires cert-manager CRDs in-cluster).
# Does not install cert-manager itself.
certManager:
# -- Create cert-manager Issuer and Certificate resources. When enabled,
# cert-manager owns TLS and the chart runs a JWT-only certgen hook to create
# the sandbox JWT signing Secret that cert-manager does not manage.
enabled: false
# -- Secret created for the intermediate CA (Certificate with isCA: true).
caSecretName: openshell-ca-tls
# -- Override the issuerRef for the external server Certificate (e.g. a real
# LetsEncrypt/ACME ClusterIssuer for a publicly-trusted cert on an external hostname).
# When set, the chart creates a second server certificate from this issuer
# with only the hostnames in serverDnsNames; the internal server certificate
# is always signed by the chart's own CA. Leave name empty to use the chart
# CA for all server certificates (default). Requires certManager.enabled=true.
serverIssuerRef:
name: ""
kind: ""
group: ""
# -- Mount gateway client CA from the internal server TLS secret's ca.crt.
# The internal server certificate is always signed by the chart CA — the same
# CA that signs the client (mTLS) certificate — so the default (true) is
# correct for all configurations, including when serverIssuerRef is set.
# Only set to false if you mount the client CA from a separate secret via
# server.tls.clientCaSecretName.
clientCaFromServerTlsSecret: true
# -- Duration for cert-manager-issued certificates.
certificateDuration: 8760h
# -- Renewal window for cert-manager-issued certificates.
certificateRenewBefore: 720h
# -- DNS SANs on the cert-manager-issued server certificate.
serverDnsNames:
- openshell
- openshell.openshell.svc
- openshell.openshell.svc.cluster.local
- localhost
- openshell.localhost
- "*.openshell.localhost"
- host.docker.internal
# -- IP SANs on the cert-manager-issued server certificate.
serverIpAddresses:
- 127.0.0.1
# Kubernetes Gateway API - HTTPRoute and Gateway resources.
# Requires a Gateway API controller in the cluster. Install Envoy Gateway via
# the skaffold.yaml releases or independently:
# helm install eg oci://docker.io/envoyproxy/gateway-helm \
# --version v1.4.1 -n envoy-gateway-system --create-namespace
grpcRoute:
# -- Create a Gateway API GRPCRoute for the gateway service.
enabled: false
# -- Hostnames the GRPCRoute matches on. Leave empty to match all hosts.
hostnames: []
gateway:
# -- When true, a Gateway resource is created in the release namespace.
# Set to false and provide name/namespace to attach to a pre-existing Gateway.
create: false
# -- GatewayClass to reference. Envoy Gateway installs one named "eg".
className: "eg"
# -- Name of the Gateway resource. Defaults to the chart fullname.
name: ""
# -- Namespace of the Gateway referenced by the GRPCRoute parentRef.
# Defaults to the release namespace.
namespace: ""
# Listener settings (only used when gateway.create is true).
listener:
# -- Listener port for the generated Gateway resource. Use 443 with protocol HTTPS.
port: 80
# -- Listener protocol for the generated Gateway resource: HTTP or HTTPS.
# HTTPS terminates TLS at the Envoy Gateway listener; pair it with
# server.disableTls=true so Envoy forwards plaintext to the gateway pod,
# and use OIDC for client identity (the gateway never sees the client cert).
protocol: HTTP
# -- "Same" restricts attached routes to the release namespace; "All" allows any namespace.
allowedRoutes: Same
# TLS settings for the listener. Used only when protocol is HTTPS
# (mode is always Terminate).
tls:
# -- certificateRefs for the HTTPS listener. Required when protocol is
# HTTPS. Each entry needs a `name` pointing at a kubernetes.io/tls Secret
# in the Gateway's namespace. May reference a cert-manager-issued Secret
# or the existing openshell-server-tls Secret (its SANs must include the
# external hostname).
certificateRefs: []
# BackendTLSPolicy for end-to-end TLS between the Gateway proxy and the
# OpenShell gateway pod. When enabled, the Gateway proxy terminates
# client-facing TLS at the listener and re-encrypts when connecting to the
# backend service, validating the backend's certificate against the
# specified CA. Requires the gateway pod to serve TLS (server.disableTls
# must be false). Supported on OpenShift 4.22+ and other platforms with
# BackendTLSPolicy support in the Gateway API implementation.
backendTLSPolicy:
# -- Create a BackendTLSPolicy resource for end-to-end TLS between the
# Gateway proxy and the OpenShell gateway pod. The traffic flow is:
# client → HTTPS → Gateway (terminate) → TLS (re-encrypt) → gateway pod.
# Requires server.disableTls=false and server.tls.enableMtls=false. The
# certgen hook auto-creates the backend CA ConfigMap.
enabled: false
# -- Name of the ConfigMap containing the CA certificate (key: ca.crt) used
# to validate the gateway pod's TLS certificate. Defaults to
# `<fullname>-backend-ca` when empty. The certgen hook auto-creates this:
# with pkiInitJob (default), immediately on install/upgrade; with
# cert-manager, the hook polls for pkiInitJob.timeoutSeconds seconds
# waiting for cert-manager to issue the server certificate, then creates the
# ConfigMap. A single install usually succeeds; if cert-manager takes longer,
# increase pkiInitJob.timeoutSeconds. By default (pkiInitJob.failOnTimeout=true),
# the install fails if the timeout is reached; set failOnTimeout=false to allow
# the install to succeed and run `helm upgrade` after the certificate is issued.
caCertificateConfigMapName: ""
# -- Hostname the Gateway proxy validates against the backend's TLS
# certificate SAN. Defaults to the service FQDN
# (`<fullname>.<namespace>.svc.cluster.local`) when empty, which matches
# the SAN included by both cert-manager and the pkiInitJob.
hostname: ""
# OpenShift Route with TLS passthrough. The gateway terminates its own
# TLS/mTLS; the router only forwards based on SNI, so it never sees plaintext
# or the client certificate. Requires server.disableTls=false and a server
# cert whose SANs include the Route host (see certManager.serverIssuerRef).
openshiftRoute:
# -- Create an OpenShift Route with TLS passthrough.
enabled: false
# -- Hostname for the Route. Must match a SAN on the gateway's server cert.
host: ""
# -- Extra annotations on the Route (e.g. haproxy.router.openshift.io/*).
annotations: {}