OpenShell/deploy/helm/openshell/README.md.gotmpl

# OpenShell Helm Chart

<!--
This file is generated by helm-docs.
Edit README.md.gotmpl and values.yaml, then run `mise run helm:docs`.
-->

> **Experimental** - the Kubernetes deployment path is under active development. Expect rough edges and breaking changes.

This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at `oci://ghcr.io/nvidia/openshell/helm-chart`.

By default, this chart also creates the namespace-scoped resources needed by
sandboxes. For a shared-gateway deployment, install it with
`workspaceResources.enabled=false`, then install the
`deploy/helm/openshell-workspace` chart in every pre-provisioned workspace
namespace. The gateway and workspace releases can then be upgraded and removed
independently. Use Kubernetes `operator` workspace mode when one gateway serves
multiple pre-provisioned workspace namespaces.

## Prerequisites

> **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for
> ingress and egress in every sandbox namespace. OpenShell creates the policies,
> but Kubernetes accepts them even if no CNI enforces them. Without enforcement,
> sandbox workloads may connect directly and bypass supervisor network policy.
> Verify CNI support before installing OpenShell.

The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with:

```shell
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/sandbox.yaml
```

The chart does not install this cluster-scoped dependency. By default, it
fails before creating gateway resources when the cluster serves neither
supported Sandbox API (`agents.x-k8s.io/v1beta1` or
`agents.x-k8s.io/v1alpha1`). Disable the check with
`agentSandbox.preflight.enabled=false` for offline `helm template` rendering,
where Helm cannot discover cluster APIs.

## Install on Kubernetes

```shell
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version>
```

## Install on OpenShift

See the full [OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift) for details. Quick start:

```shell
# Precreate the openshell namespace
oc create ns openshell

# Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> -n openshell \
  --set server.disableTls=true \
  --set podSecurityContext.fsGroup=null \
  --set securityContext.runAsUser=null
```

On OpenShift 4.22+, end-to-end TLS is supported via `BackendTLSPolicy`. See the
[OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift#end-to-end-tls-openshift-422) for details.

## Available versions

| Tag | Source | Notes |
| --- | --- | --- |
| `<semver>` (e.g. `0.6.0`) | Tagged GitHub release | Tracks the matching gateway, sandbox, and supervisor image versions. Recommended for production. |
| `<semver>-pre.N` (e.g. `0.1.0-pre.3`) | A specific prerelease candidate | Immutable candidate pin that tracks images with the same exact version. |
| `0.0.0-dev` | Latest commit on `main` | Floating tag, overwritten on every push. `appVersion` is `dev`, so images resolve to the `:dev` tag. |
| `0.0.0-dev.<commit-sha>` | A specific commit on `main` | Per-commit pin. Chart version and `appVersion` both use the full 40-character commit SHA, which matches the image tag pushed by CI. |

Prerelease and `dev` tags are intended for testing changes ahead of a release. Production deployments should pin to a stable tagged release.

## Configuration

See [`values.yaml`](values.yaml) for source defaults. Selected overlays:

- [`ci/values-gateway.yaml`](ci/values-gateway.yaml) - gateway-only configuration
- [`ci/values-cert-manager.yaml`](ci/values-cert-manager.yaml) - cert-manager integration
- [`ci/values-keycloak.yaml`](ci/values-keycloak.yaml) - Keycloak OIDC integration
- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - CI overlay for multi-replica external PostgreSQL testing
- [`ci/values-spire.yaml`](ci/values-spire.yaml) - SPIFFE/SPIRE provider token grants
- [`ci/values-spire-stack.yaml`](ci/values-spire-stack.yaml) - SPIRE hardened chart values for local development

### Database backend

By default, OpenShell uses SQLite and runs the gateway as a StatefulSet so the
database is backed by a per-pod PVC:

```yaml
server:
  dbUrl: "sqlite:/var/openshell/openshell.db"
```

#### External PostgreSQL

Use external PostgreSQL when the gateway should connect to a database managed
outside this chart. The OpenShell chart does not deploy a database; install
PostgreSQL separately using the chart, operator, or managed service that fits
your environment, then pass the connection URI through a Secret.

Create a Secret containing the PostgreSQL connection URI if one does not
already exist:

```bash
kubectl create secret generic my-pg-credentials -n openshell \
  --from-literal=uri="postgresql://user:pass@host:5432/dbname"
```

Then install the chart pointing at that Secret:

```bash
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
  -n openshell \
  --set workload.kind=deployment \
  --set server.externalDbSecret=my-pg-credentials
```

Use `workload.kind=deployment` for external database-backed multi-replica
gateways. `workload.kind=statefulset` is still available for single-replica
SQLite installs and for operators who explicitly need StatefulSet identity or
storage semantics.

### Credential storage

By default, the chart uses the gateway's encrypted database credential storage.
The gateway writes encrypted provider credential envelopes to the OpenShell
database. The chart creates a retained Kubernetes Secret with the shared
key-encryption key and injects that key into every gateway pod, so the same
default works for single-replica and external database-backed HA deployments.

Use `kubernetes-secrets` or `vault` instead when credentials should live in a
cluster or external secret backend. Enabling one external credential driver
disables the default credential-storage key-encryption key Secret and env injection.

#### OpenShift

Append these flags to any of the PostgreSQL commands above for OpenShift:

```
--set server.disableTls=true \
--set podSecurityContext.fsGroup=null \
--set securityContext.runAsUser=null
```

### High availability

Set `replicaCount` above `1` only with `server.externalDbSecret`; the default
SQLite database is per pod and cannot coordinate multiple gateway replicas.
The chart creates a headless peer Service for gateway-to-gateway relay traffic.
StatefulSet pods use stable pod DNS names through that headless Service.
Deployment pods advertise their pod IP with `OPENSHELL_PEER_ENDPOINT`, because
Kubernetes does not assign stable per-pod DNS names to Deployment replicas.

Gateway peer traffic uses Kubernetes ServiceAccount identity. Each gateway pod
mounts a projected, pod-bound ServiceAccount token with audience
`openshell-gateway-peer`; receiving replicas validate that token with the
Kubernetes TokenReview API, verify the live pod UID and Helm selector labels,
and authorize only the internal `PeerRelay` RPC. The chart does not create or
accept a shared gateway peer Secret.

With gateway TLS enabled, peer calls use the chart CA and client TLS Secret for
server verification and mTLS. The client verifies the stable gateway Service
DNS name while connecting directly to the owning pod. Custom TLS Secrets must
include that Service DNS name in the server certificate and provide the CA and
client credentials configured by `server.tls`.

## Secret bootstrap

By default, a pre-install/pre-upgrade hook Job runs `openshell-gateway generate-certs`
to create the gateway's server/client mTLS Secrets and sandbox JWT signing Secret.
The Job uses the gateway image itself, so air-gapped environments only need to
mirror that one image (no separate openssl/alpine sidecar).

When `certManager.enabled=true`, cert-manager owns the TLS Secrets and the chart
runs the same hook in JWT-only mode because cert-manager does not create the
sandbox JWT signing Secret. This precedence applies even if
`pkiInitJob.enabled` remains true. Set `pkiInitJob.enabled=false` only when an
external non-cert-manager TLS source manages TLS and you pre-create the sandbox
JWT signing Secret.

## SPIFFE/SPIRE provider token grants

Set `server.providerTokenGrants.spiffe.enabled=true` to let the gateway and
sandbox supervisors use SPIFFE JWT-SVIDs for dynamic provider token grants. The
chart keeps supervisor-to-gateway authentication on gateway-minted sandbox JWTs,
mounts the SPIFFE CSI socket into the gateway pod, exports
`OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET`, and passes the socket path to
the Kubernetes driver so sandbox pods can mount the same socket.

For local development, uncomment the SPIRE Helm releases in `skaffold.yaml` and
add `ci/values-spire.yaml` to the OpenShell release values files.

The gateway verifies supervisor JWT-SVIDs with JWT bundles fetched from the
SPIFFE Workload API, so this path does not require access to the SPIRE OIDC
discovery endpoint or its TLS CA.

{{ template "chart.valuesSection" . }}
{{ template "helm-docs.versionFooter" . }}