OpenShell/docs/kubernetes/ingress.mdx

---
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
title: "Ingress"
sidebar-title: "Ingress"
description: "Expose the OpenShell gateway externally using the Kubernetes Gateway API and a GRPCRoute."
keywords: "Generative AI, Cybersecurity, Kubernetes, Gateway API, Envoy Gateway, GRPCRoute, Ingress, External Access"
position: 5
---

By default, the OpenShell gateway is only reachable inside the cluster. To let CLI clients connect without a `kubectl port-forward`, expose the gateway through an ingress.

OpenShell uses the [Kubernetes Gateway API](https://gateway-api.sigs.k8s.io) for ingress. The chart creates a `GRPCRoute` that routes inbound gRPC traffic to the gateway pod. You need a Gateway API implementation installed on your cluster to fulfill the `GRPCRoute`. This page uses [Envoy Gateway](https://gateway.envoyproxy.io), which the chart is tested with.

## Install Envoy Gateway

Envoy Gateway installs the Gateway API CRDs and controller:

```shell
helm install eg \
  oci://docker.io/envoyproxy/gateway-helm \
  --version v1.8.1 \
  --namespace envoy-gateway-system \
  --create-namespace \
  --wait
```

## Create the GatewayClass

Create the `eg` GatewayClass that the OpenShell chart references:

```shell
kubectl apply -f - <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
EOF
```

Verify the GatewayClass is accepted:

```shell
kubectl get gatewayclass eg
```

The `ACCEPTED` column should show `True`.

## Install OpenShell with Gateway API enabled

Enable the GRPCRoute and let the chart create a Gateway resource in the `openshell` namespace:

```shell
helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set grpcRoute.enabled=true \
  --set grpcRoute.gateway.create=true \
  --set grpcRoute.gateway.className=eg
```

## Get the external address

After the Gateway is provisioned, Envoy Gateway creates a LoadBalancer service in the `openshell` namespace. Wait for it to get an external address:

```shell
kubectl -n openshell get svc -l gateway.envoyproxy.io/owning-gateway-name=openshell
```

After the `EXTERNAL-IP` is assigned, register the gateway with the CLI:

```shell
openshell gateway add http://<external-ip> --name production
openshell status
```

This setup is plaintext end-to-end and is intended for development. For external access, terminate TLS at the gateway as shown below.

## HTTPS (TLS termination)

Envoy Gateway can terminate TLS at the listener and forward plaintext to the OpenShell gateway pod:

```text
client → HTTPS → Envoy Gateway (terminate TLS) → plaintext → openshell gateway pod
```

Envoy Gateway only terminates TLS here — it does not perform OIDC. Do not enable an Envoy Gateway OIDC `SecurityPolicy` in front of the gateway: that flow relies on browser redirects and cookies and cannot work with the OpenShell CLI or headless agents. Instead, the OpenShell gateway validates an OIDC bearer token that the client sends in the gRPC `authorization` metadata, which Envoy forwards untouched.

Because Envoy terminates TLS, the OpenShell gateway never sees a client certificate, so client mTLS cannot provide identity on this path. Use OIDC bearer tokens for client identity instead.

For an interactive login on a headless machine, set `OPENSHELL_NO_BROWSER=1` before running `openshell gateway add`. The CLI uses the Device Authorization Grant with S256 PKCE and prompts the user to approve the login from another browser. For unattended agents and CI, set `OPENSHELL_OIDC_CLIENT_SECRET` to use the OAuth2 client-credentials grant instead. The client id comes from `--oidc-client-id` (default `openshell-cli`); pass it explicitly if your identity provider uses a different id. Interactive users with a local browser get the Authorization Code flow with PKCE by default.

### Provide a TLS certificate

Create a `kubernetes.io/tls` Secret in the `openshell` namespace with the certificate for your external hostname:

```shell
kubectl -n openshell create secret tls openshell-ingress-tls \
  --cert=tls.crt --key=tls.key
```

The Secret may also be issued by cert-manager, or you can reference the chart's existing `openshell-server-tls` Secret if its SANs include the external hostname.

### Install with HTTPS termination

Enable an HTTPS listener, point it at the Secret, disable gateway-pod TLS so Envoy forwards plaintext, and configure an OIDC issuer for client identity:

```shell
helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set grpcRoute.enabled=true \
  --set grpcRoute.gateway.create=true \
  --set grpcRoute.gateway.className=eg \
  --set grpcRoute.gateway.listener.protocol=HTTPS \
  --set grpcRoute.gateway.listener.port=443 \
  --set 'grpcRoute.gateway.listener.tls.certificateRefs[0].name=openshell-ingress-tls' \
  --set server.disableTls=true \
  --set server.oidc.issuer=https://keycloak.example.com/realms/openshell \
  --set server.oidc.audience=openshell-cli \
  --set 'grpcRoute.hostnames[0]=gateway.example.com'
```

Keep the certificate Secret in the release namespace. Referencing a Secret in another namespace requires a `ReferenceGrant`.

### Register over HTTPS

```shell
openshell gateway add https://gateway.example.com \
  --name production \
  --oidc-issuer https://keycloak.example.com/realms/openshell \
  --oidc-client-id openshell-cli
openshell status
```

See [Authentication](/kubernetes/setup) for OIDC issuer, audience, and roles configuration.

## End-to-end TLS (BackendTLSPolicy)

As an alternative to the plaintext backend path above, the chart can create a `BackendTLSPolicy` that tells the Gateway proxy to re-encrypt traffic when connecting to the OpenShell gateway pod:

```text
client → HTTPS → Gateway (terminate TLS) → TLS (re-encrypt) → openshell gateway pod
```

This keeps TLS on the gateway pod rather than disabling it with `server.disableTls=true`. The Gateway proxy validates the backend's certificate against a CA ConfigMap that the certgen hook auto-creates.

BackendTLSPolicy is a standard Gateway API resource. It is supported on OpenShift 4.22+ (via the OpenShift gateway controller) and on other platforms where the Gateway API implementation supports it (check your controller's documentation).

### Install with e2e TLS

The certgen hook automatically creates the backend CA ConfigMap when `backendTLSPolicy` is enabled:

```shell
helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set server.tls.enableMtls=false \
  --set grpcRoute.enabled=true \
  --set grpcRoute.gateway.create=true \
  --set grpcRoute.gateway.className=eg \
  --set grpcRoute.gateway.listener.protocol=HTTPS \
  --set grpcRoute.gateway.listener.port=443 \
  --set 'grpcRoute.gateway.listener.tls.certificateRefs[0].name=openshell-ingress-tls' \
  --set grpcRoute.backendTLSPolicy.enabled=true \
  --set server.oidc.issuer=https://keycloak.example.com/realms/openshell \
  --set server.oidc.audience=openshell-cli \
  --set 'grpcRoute.hostnames[0]=gateway.example.com'
```

Note that `server.disableTls` is **not** set — the gateway pod continues to serve TLS — but `server.tls.enableMtls=false` disables mTLS client certificate authentication because the Gateway proxy cannot present a client certificate to the backend. The chart will fail the install if you try to enable both `grpcRoute.backendTLSPolicy.enabled=true` and `server.tls.enableMtls=true` simultaneously. The BackendTLSPolicy hostname defaults to the service FQDN, which matches the SAN on the server certificate. Use OIDC for authentication (configured via `server.oidc.issuer`).

The example above uses the default `pkiInitJob` for TLS, which creates the backend CA ConfigMap immediately. If using cert-manager instead (`--set certManager.enabled=true`), the Certificate resources are regular release objects, and a separate post-install/post-upgrade Job (`<release>-certgen-backend-ca`) polls for up to 120 seconds waiting for cert-manager to issue the server certificate, then creates the backend CA ConfigMap. This means a single `helm install` is sufficient in most cases.

If cert-manager takes longer than 120 seconds to issue certificates, increase the polling timeout with `--set pkiInitJob.timeoutSeconds=<seconds>`. The hook polls for exactly this many seconds. For example, `timeoutSeconds=180` polls for 180 seconds. By default (`pkiInitJob.failOnTimeout=true`), the install fails if the timeout is reached, providing clear feedback that the BackendTLSPolicy is non-functional.

### Troubleshooting

If you see the error `remote connection failure, transport failure reason: TLS error: Secret is not supplied by SDS` when connecting through the Gateway:

1. Check if the backend CA ConfigMap exists:

   ```shell
   kubectl get configmap <release-name>-backend-ca -n <namespace>
   ```

2. If the ConfigMap is missing, verify the TLS secret exists:

   ```shell
   kubectl get secret <release-name>-server-tls -n <namespace>
   ```

3. If the secret exists but the ConfigMap doesn't, run `helm upgrade` to create it:

   ```shell
   helm upgrade <release-name> oci://ghcr.io/nvidia/openshell/helm-chart \
     --reuse-values --namespace <namespace>
   ```

This situation can occur if you set `pkiInitJob.failOnTimeout=false` and cert-manager issued the certificate after the hook timed out.

For OpenShift 4.22+, see [OpenShift](/kubernetes/openshift#end-to-end-tls-using-gateway-api-and-backendtlspolicy-openshift-422) for platform-specific instructions including Gateway and GatewayClass setup.

## Replica Routing

With more than one gateway replica, a sandbox's SSH, port-forward, and exec connections can land on a replica that does not hold the sandbox's supervisor session, and that replica relays them to the one that does. Enable replica routing to send those connections straight to the owning replica:

```shell
helm upgrade <release-name> oci://ghcr.io/nvidia/openshell/helm-chart \
  --reuse-values --namespace <namespace> \
  --set workload.kind=statefulset \
  --set workload.allowMultiReplicaStatefulSet=true \
  --set grpcRoute.replicaRouting.enabled=true
```

The chart renders one Service per replica and a GRPCRoute rule that matches the `x-openshell-replica` request header. The CLI sets that header from the owner the gateway reports and retries without it if the named replica is unavailable. Replica routing requires `workload.kind=statefulset`, `workload.allowMultiReplicaStatefulSet=true`, and at most 15 replicas.

If the release uses `workload.kind=deployment`, this upgrade replaces the Deployment with a StatefulSet, so every gateway pod restarts at once. The StatefulSet also requests a 1Gi PersistentVolumeClaim per replica, which stays `Pending` on clusters without a default StorageClass.

## SSH Relay

Sandbox SSH uses the gateway endpoint registered with the CLI. No separate Helm SSH host or port values are required.

## Next Steps

Return to [Setup](/kubernetes/setup) to complete the installation.