mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-21 06:11:31 +08:00
* feat(helm): split gateway and workspace charts Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com> * fix(helm): preserve split chart upgrade compatibility Keep workspace manifests valid after value validation and default legacy reused values to the combined resource topology. * fix(ci): preserve VM runtime for E2E The Rust cache restores target/ after VM runtime artifacts are staged, overwriting target/vm-runtime-compressed before openshell-driver-vm is built. Stage the compressed runtime outside target and pass that location through OPENSHELL_VM_RUNTIME_COMPRESSED_DIR so build.rs can embed the supervisor. Also locate the Helm split-ownership test repository root from the script path rather than git rev-parse. The test runs in a container where the GitHub checkout can be owned by a different UID and rejected as dubious ownership. Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com> * fix(ci): install yq for Helm ownership test The split-chart ownership regression uses yq to inspect rendered YAML, but the Helm CI container installs only tools declared in mise. Declare and lock yq so mise install --locked provides the test dependency. Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com> --------- Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com>
173 lines
7.3 KiB
Go Template
173 lines
7.3 KiB
Go Template
# OpenShell Helm Chart
|
|
|
|
<!--
|
|
This file is generated by helm-docs.
|
|
Edit README.md.gotmpl and values.yaml, then run `mise run helm:docs`.
|
|
-->
|
|
|
|
> **Experimental** - the Kubernetes deployment path is under active development. Expect rough edges and breaking changes.
|
|
|
|
This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at `oci://ghcr.io/nvidia/openshell/helm-chart`.
|
|
|
|
By default, this chart also creates the namespace-scoped resources needed by
|
|
sandboxes. For a shared-gateway deployment, install it with
|
|
`workspaceResources.enabled=false`, then install the
|
|
`deploy/helm/openshell-workspace` chart in every pre-provisioned workspace
|
|
namespace. The gateway and workspace releases can then be upgraded and removed
|
|
independently. Use Kubernetes `operator` workspace mode when one gateway serves
|
|
multiple pre-provisioned workspace namespaces.
|
|
|
|
## Prerequisites
|
|
|
|
The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with:
|
|
|
|
```shell
|
|
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/manifest.yaml
|
|
```
|
|
|
|
The chart does not install this cluster-scoped dependency. By default, it
|
|
fails before creating gateway resources when the cluster serves neither
|
|
supported Sandbox API (`agents.x-k8s.io/v1beta1` or
|
|
`agents.x-k8s.io/v1alpha1`). Disable the check with
|
|
`agentSandbox.preflight.enabled=false` for offline `helm template` rendering,
|
|
where Helm cannot discover cluster APIs.
|
|
|
|
## Install on Kubernetes
|
|
|
|
```shell
|
|
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version>
|
|
```
|
|
|
|
## Install on OpenShift
|
|
|
|
See the full [OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift) for details. Quick start:
|
|
|
|
```shell
|
|
# Precreate the openshell namespace so we can create the SCC cluster role
|
|
oc create ns openshell
|
|
|
|
# Sandboxes are deployed into the openshell namespace and use the openshell-sandbox service account
|
|
oc adm policy add-scc-to-user privileged -z openshell-sandbox -n openshell
|
|
|
|
# Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway
|
|
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> -n openshell \
|
|
--set server.disableTls=true \
|
|
--set podSecurityContext.fsGroup=null \
|
|
--set securityContext.runAsUser=null
|
|
```
|
|
|
|
## Available versions
|
|
|
|
| Tag | Source | Notes |
|
|
| --- | --- | --- |
|
|
| `<semver>` (e.g. `0.6.0`) | Tagged GitHub release | Tracks the matching gateway and supervisor image versions. Recommended for production. |
|
|
| `0.0.0-dev` | Latest commit on `main` | Floating tag, overwritten on every push. `appVersion` is `dev`, so images resolve to the `:dev` tag. |
|
|
| `0.0.0-dev.<commit-sha>` | A specific commit on `main` | Per-commit pin. Chart version and `appVersion` both use the full 40-character commit SHA, which matches the image tag pushed by CI. |
|
|
|
|
The `dev` tags are intended for testing changes ahead of a release. Production deployments should pin to a tagged release.
|
|
|
|
## Configuration
|
|
|
|
See [`values.yaml`](values.yaml) for source defaults. Selected overlays:
|
|
|
|
- [`ci/values-gateway.yaml`](ci/values-gateway.yaml) - gateway-only configuration
|
|
- [`ci/values-cert-manager.yaml`](ci/values-cert-manager.yaml) - cert-manager integration
|
|
- [`ci/values-keycloak.yaml`](ci/values-keycloak.yaml) - Keycloak OIDC integration
|
|
- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - CI overlay for multi-replica external PostgreSQL testing
|
|
- [`ci/values-spire.yaml`](ci/values-spire.yaml) - SPIFFE/SPIRE provider token grants
|
|
- [`ci/values-spire-stack.yaml`](ci/values-spire-stack.yaml) - SPIRE hardened chart values for local development
|
|
|
|
### Database backend
|
|
|
|
By default, OpenShell uses SQLite and runs the gateway as a StatefulSet so the
|
|
database is backed by a per-pod PVC:
|
|
|
|
```yaml
|
|
server:
|
|
dbUrl: "sqlite:/var/openshell/openshell.db"
|
|
```
|
|
|
|
#### External PostgreSQL
|
|
|
|
Use external PostgreSQL when the gateway should connect to a database managed
|
|
outside this chart. The OpenShell chart does not deploy a database; install
|
|
PostgreSQL separately using the chart, operator, or managed service that fits
|
|
your environment, then pass the connection URI through a Secret.
|
|
|
|
Create a Secret containing the PostgreSQL connection URI if one does not
|
|
already exist:
|
|
|
|
```bash
|
|
kubectl create secret generic my-pg-credentials -n openshell \
|
|
--from-literal=uri="postgresql://user:pass@host:5432/dbname"
|
|
```
|
|
|
|
Then install the chart pointing at that Secret:
|
|
|
|
```bash
|
|
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
|
|
-n openshell \
|
|
--set workload.kind=deployment \
|
|
--set server.externalDbSecret=my-pg-credentials
|
|
```
|
|
|
|
Use `workload.kind=deployment` for external database-backed multi-replica
|
|
gateways. `workload.kind=statefulset` is still available for single-replica
|
|
SQLite installs and for operators who explicitly need StatefulSet identity or
|
|
storage semantics.
|
|
|
|
### Credential storage
|
|
|
|
By default, the chart uses the gateway's encrypted database credential storage.
|
|
The gateway writes encrypted provider credential envelopes to the OpenShell
|
|
database. The chart creates a retained Kubernetes Secret with the shared
|
|
key-encryption key and injects that key into every gateway pod, so the same
|
|
default works for single-replica and external database-backed HA deployments.
|
|
|
|
Use `kubernetes-secrets` or `vault` instead when credentials should live in a
|
|
cluster or external secret backend. Enabling one external credential driver
|
|
disables the default credential-storage key-encryption key Secret and env injection.
|
|
|
|
#### OpenShift
|
|
|
|
Append these flags to any of the PostgreSQL commands above for OpenShift:
|
|
|
|
```
|
|
--set server.disableTls=true \
|
|
--set podSecurityContext.fsGroup=null \
|
|
--set securityContext.runAsUser=null
|
|
```
|
|
|
|
## Secret bootstrap
|
|
|
|
By default, a pre-install/pre-upgrade hook Job runs `openshell-gateway generate-certs`
|
|
to create the gateway's server/client mTLS Secrets and sandbox JWT signing Secret.
|
|
The Job uses the gateway image itself, so air-gapped environments only need to
|
|
mirror that one image (no separate openssl/alpine sidecar).
|
|
|
|
When `certManager.enabled=true`, cert-manager owns the TLS Secrets and the chart
|
|
runs the same hook in JWT-only mode because cert-manager does not create the
|
|
sandbox JWT signing Secret. This precedence applies even if
|
|
`pkiInitJob.enabled` remains true. Set `pkiInitJob.enabled=false` only when an
|
|
external non-cert-manager TLS source manages TLS and you pre-create the sandbox
|
|
JWT signing Secret.
|
|
|
|
## SPIFFE/SPIRE provider token grants
|
|
|
|
Set `server.providerTokenGrants.spiffe.enabled=true` to let the gateway and
|
|
sandbox supervisors use SPIFFE JWT-SVIDs for dynamic provider token grants. The
|
|
chart keeps supervisor-to-gateway authentication on gateway-minted sandbox JWTs,
|
|
mounts the SPIFFE CSI socket into the gateway pod, exports
|
|
`OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET`, and passes the socket path to
|
|
the Kubernetes driver so sandbox pods can mount the same socket.
|
|
|
|
For local development, uncomment the SPIRE Helm releases in `skaffold.yaml` and
|
|
add `ci/values-spire.yaml` to the OpenShell release values files.
|
|
|
|
The gateway verifies supervisor JWT-SVIDs with JWT bundles fetched from the
|
|
SPIFFE Workload API, so this path does not require access to the SPIRE OIDC
|
|
discovery endpoint or its TLS CA.
|
|
|
|
{{ template "chart.valuesSection" . }}
|
|
{{ template "helm-docs.versionFooter" . }}
|