trending/OpenShell
1
0
Fork 0
mirror of https://github.com/NVIDIA/OpenShell.git synced 2026-10-17 05:31:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
codex/2183-cli-conformance-pre-implementation
OpenShell/docs/kubernetes/openshift.mdx
T
Jesse Jaggars c4b500a7de feat(helm): cert-manager external issuer + OpenShift passthrough Route (#2468) 
* fix(core): trust public root CAs alongside the sandbox mTLS CA

The supervisor gRPC client only trusted the CA configured via
OPENSHELL_TLS_CA, since tonic ClientTlsConfig starts with an empty root
store unless with_native_roots()/with_webpki_roots() is also enabled.
Deployments where the gateway server certificate is issued by a public CA
(e.g. cert-manager against an ACME issuer) caused every supervisor
connection to fail the TLS handshake with "UnknownCA", since the sandbox
mTLS CA and the server cert issuer were no longer the same.

Enable both native and webpki roots in addition to the configured CA.
tonic root store is a union of all configured sources, so this does not
weaken verification for existing self-signed deployments. webpki-roots
(compiled in) is enabled alongside native-roots since the supervisor
binary may run in minimal sandbox images without a populated system CA
bundle.

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* feat(helm): support external cert-manager issuers and OpenShift Route passthrough

Add certManager.serverIssuerRef/clientIssuerRef so the gateway and mTLS
client certificates can be issued by a real Issuer/ClusterIssuer (e.g.
ACME) instead of only the chart built-in self-signed CA.

Add openshiftRoute template for exposing the gateway via a TLS
passthrough Route so the gateway keeps terminating its own TLS/mTLS.

The server Certificate excludes internal-only SANs (cluster-local,
localhost, loopback) when an external issuer is configured, since ACME
issuers reject those per CA/Browser Forum baseline requirements. A
template-time fail guard catches the misconfiguration at helm install
time rather than asynchronously at cert-manager issuance time.

Includes Helm unittest coverage for both issuerRef overrides and Route
rendering, plus a CI values overlay for lint coverage.

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* docs: document cert-manager external issuer and OpenShift Route

Update managing-certificates.mdx with the serverIssuerRef workflow and
install-time validation behavior. Add a production section to the
OpenShift guide covering passthrough Route with a real certificate.
Regenerate Helm README for new certManager and openshiftRoute values.
Sync debug-openshell-cluster skill with new troubleshooting steps for
ACME issuance failures and supervisor UnknownCA from mismatched CAs.

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(helm,core): address PR review feedback on cert-manager external issuer

Addresses all five blocking review items from #2468:

1. Remove .with_native_roots() from supervisor gRPC client -- the
   supervisor runs inside the user-selected sandbox image, so the
   image CA bundle is not operator-controlled. Keep .with_webpki_roots()
   (compiled-in, not user-controlled) alongside the configured CA.

2. Fail at render time when serverIssuerRef.name is set but
   clientCaFromServerTlsSecret is still true. Add negative Helm test.

3. Remove clientIssuerRef -- changing only clientIssuerRef breaks both
   directions because trust bundles are not modeled separately. Change
   serverIssuerRef.kind default from ClusterIssuer to Issuer.

4. Add server.oidc.issuer and server.oidc.audience to the documented
   OpenShift production Helm command. Add Access Control prerequisite.

5. Fail at render time when openshiftRoute.enabled and disableTls are
   both true. Add negative Helm test.

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(drivers): strip GATEWAY_TLS_SERVER_NAME from Docker and Podman env

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* fix(helm,drivers): guard default clientCaSecretName and add env-strip tests

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

* feat(tls): SNI-based dual certificate for internal and external server TLS

Split the gateway server certificate into two: an internal cert issued by
the chart's own CA (for supervisor connections via cluster-local SANs) and
an external cert issued by an operator-configured Issuer such as ACME/Let's
Encrypt (for CLI and Route access via public SANs).

The gateway uses SNI-based certificate selection: connections whose SNI
hostname matches external_server_names receive the external cert; all
others (including those with no SNI) receive the internal cert.

Security improvement: remove .with_webpki_roots() from the supervisor
gRPC client so supervisors trust only the chart CA, closing a MITM vector
via publicly-trusted certificates in user-supplied container images.

Key changes:
- Add DualCertResolver with SNI-based cert selection and full test coverage
- Add external_cert_path, external_key_path, external_server_names to TlsConfig
- Validate partial external cert config (error on cert-without-key or vice versa)
- Validate empty external_server_names when external cert is configured
- Split cert-manager templates into internal + external Certificate resources
- Add Helm guards for misconfigured external issuer (empty serverDnsNames,
  internal-only SANs with external issuer, conflicting clientCaFromServerTlsSecret)
- Update gateway-config.mdx, managing-certificates.mdx, openshift.mdx docs
- Update debug-openshell-cluster skill for dual-cert troubleshooting

Signed-off-by: Pi Agent <agent@openshell.local>

* fix(drivers): strip GATEWAY_TLS_SERVER_NAME in VM driver and correct comments

Add the same GATEWAY_TLS_SERVER_NAME environment stripping to the VM
compute driver that Docker, Podman, and Kubernetes drivers already
perform. Without this, a sandbox user on the VM driver could override
the TLS server name the supervisor verifies.

Fix stale comments in Docker and Podman drivers that referenced
'with WebPKI roots trusted' — WebPKI roots are explicitly not trusted
after the tls-webpki-roots removal.

Use tls-ring instead of bare channel for tonic in openshell-core so the
TLS API (ClientTlsConfig, Endpoint::tls_config) is available without
pulling in any root certificate store.

Signed-off-by: Pi Agent <agent@openshell.local>

* fix(tls,helm): wildcard SNI matching and Route host validation

Add RFC 6125 single-level wildcard matching to DualCertResolver so
external_server_names entries like *.example.com correctly match SNI
hostnames like gw.example.com. Previously only exact matches worked,
silently falling back to the internal cert for wildcard configurations.

Add a Helm fail guard in route.yaml that rejects openshiftRoute.host
values not listed in certManager.serverDnsNames when an external issuer
is configured — catches cert/route hostname mismatches at install time
instead of at TLS connect time.

Quote the host field in route.yaml for robustness.

Signed-off-by: Pi Agent <agent@openshell.local>

* fix(helm): address blocking review items — client-CA guard, wildcard Route, serverIssuerRef gate

1. Remove the obsolete guard rejecting serverIssuerRef + clientCaFromServerTlsSecret=true.
   The internal server certificate is always signed by the chart CA (the same
   CA that signs the client cert), so clientCaFromServerTlsSecret=true is
   correct — its filtered ca.crt is exactly the right trust anchor.  The old
   workaround (mounting openshell-ca-tls directly) unnecessarily exposed the
   CA private key to the gateway container.  Remove the client-CA overrides
   from docs, CI overlay, and production examples.

2. Route host validation now supports wildcard certificates per RFC 6125:
   single-level wildcards like *.example.com match gateway.example.com but
   not deep.sub.example.com.  Require an explicit openshiftRoute.host when
   an external issuer is configured — without one, OpenShift generates a
   hostname absent from serverDnsNames.

3. Reject serverIssuerRef.name when certManager.enabled is false — the
   external certificate, its Secret mount, and the gateway TLS config all
   require cert-manager to be enabled.

Validated on ROSA (dev.dyee.p3) with branch-built images:
- Fresh install with letsencrypt-prod ClusterIssuer
- SNI dual-cert: external hostname served Let's Encrypt cert
- Supervisor mTLS via internal cert path: ConnectSupervisor accepted
- Client CA volume: filtered ca.crt from internal server secret (no key)
- CLI connected via Route + OIDC

Helm tests: 81 pass across 7 suites.

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>

---------

Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
Signed-off-by: Pi Agent <agent@openshell.local>
2026-08-14 00:12:11 +00:00

143 lines
6.0 KiB
Plaintext
Raw Permalink Blame History

---
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
title: "OpenShift"
sidebar-title: "OpenShift"
description: "Install the OpenShell Helm chart on OpenShift, including the SCC binding and chart overrides required by OpenShift's Security Context Constraints."
keywords: "Generative AI, Cybersecurity, Kubernetes, OpenShift, SCC, Security Context Constraints, Helm, Gateway, Installation"
position: 6
---
<Warning>
The OpenShift install path is experimental. It currently requires running sandbox pods under the `privileged` SCC and installing the gateway with TLS disabled. Use only for evaluation on a private network.
</Warning>
OpenShift's [Security Context Constraints](https://docs.openshift.com/container-platform/latest/authentication/managing-security-context-constraints.html) reject the chart's default pod security settings. Installing on OpenShift requires precreating the namespace, granting the `privileged` SCC to the sandbox service account, and overriding a few chart values so the cluster admission controller can assign UIDs and FS groups itself.
OpenShell installs sandbox nftables rules as individual commands. On OpenShift
nodes where optional conntrack or packet log expressions are unavailable, those
optional rules can fail without rolling back the required proxy bypass reject
rules.
## Prerequisites
- OpenShift 4.x cluster with `oc` configured
- Helm 3.x
- [Agent Sandbox](/kubernetes/setup#install-agent-sandbox) controller and CRDs installed
## Install
<Steps>
## Create the namespace
Pre-create the namespace so the SCC binding can be applied before the chart installs:
```shell
oc create ns openshell
```
## Grant the privileged SCC to sandbox pods
Sandbox pods run under the `openshell-sandbox` service account in the `openshell` namespace and require the `privileged` SCC:
```shell
oc adm policy add-scc-to-user privileged -z openshell-sandbox -n openshell
```
## Install the chart with OpenShift overrides
```shell
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
--version <version> \
--namespace openshell \
--set server.disableTls=true \
--set podSecurityContext.fsGroup=null \
--set securityContext.runAsUser=null
```
| Override | Reason |
|---|---|
| `server.disableTls=true` | Runs the gateway over plaintext HTTP for simpler evaluation. |
| `podSecurityContext.fsGroup=null` / `securityContext.runAsUser=null` | Clear the chart's hardcoded UID and fsGroup so OpenShift's SCC admission can assign them. |
## Wait for the gateway to be ready
```shell
oc -n openshell rollout status statefulset/openshell
```
If you set `workload.kind=deployment`, use
`oc -n openshell rollout status deployment/openshell` instead.
</Steps>
## Connect to the gateway
The gateway is now running over plaintext HTTP. Connect with `oc port-forward`:
```shell
oc -n openshell port-forward svc/openshell 8080:8080
```
Register the gateway with the CLI:
```shell
openshell gateway add http://127.0.0.1:8080 --local --name openshift
openshell status
```
## Production: expose externally with a real certificate
The steps above run the gateway over plaintext HTTP for quick evaluation. For
a real deployment, cert-manager can issue the gateway's server certificate
from a real Issuer or ClusterIssuer (for example, an ACME issuer), and an
OpenShift Route with TLS passthrough exposes it externally while the gateway
keeps terminating its own TLS and mTLS.
Install cert-manager and configure a working `ClusterIssuer` first — see
[Managing Certificates](/kubernetes/managing-certificates) for the
`certManager.serverIssuerRef` details. Configure an OIDC provider as described
in [Access Control](/kubernetes/access-control) — remote gateways authenticate
CLI users via OIDC, not mTLS, so the gateway must know the OIDC issuer URL.
Install the chart with:
```shell
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
--version <version> \
--namespace openshell \
--set podSecurityContext.fsGroup=null \
--set securityContext.runAsUser=null \
--set server.disableTls=false \
--set certManager.enabled=true \
--set certManager.serverIssuerRef.name=<cluster-issuer-name> \
--set certManager.serverIssuerRef.kind=ClusterIssuer \
--set certManager.serverDnsNames[0]=<external-hostname> \
--set openshiftRoute.enabled=true \
--set openshiftRoute.host=<external-hostname> \
--set server.oidc.issuer=<oidc-issuer-url> \
--set server.oidc.audience=<oidc-audience>
```
| Override | Reason |
|---|---|
| `certManager.serverIssuerRef` | Creates a second server certificate from your Issuer or ClusterIssuer for external clients. The gateway uses SNI to present this cert for the external hostname while continuing to present the internal (chart CA) cert to supervisors. The internal certificate's `ca.crt` is the chart CA that also signed the client cert, so the default `clientCaFromServerTlsSecret=true` is correct. |
| `openshiftRoute.enabled` / `openshiftRoute.host` | Creates an OpenShift Route with TLS passthrough — the router forwards the encrypted connection by SNI without decrypting, so the gateway uses the SNI hostname to select the external certificate. |
| `server.oidc.issuer` / `server.oidc.audience` | Configures server-side OIDC validation. Without these, the gateway expects mTLS client certificates and rejects OIDC-only CLI connections. See [Access Control](/kubernetes/access-control). |
Register the gateway with the CLI over OIDC. Remote gateways authenticate CLI
users via OIDC, not mTLS — see [Access Control](/kubernetes/access-control):
```shell
openshell gateway add https://<external-hostname> \
--name openshift \
--oidc-issuer <issuer-url>
openshell gateway login openshift
```
## Next Steps
- For more on certificate provisioning modes, refer to [Managing Certificates](/kubernetes/managing-certificates).
- To expose the gateway externally through the Kubernetes Gateway API instead of a Route, refer to [Ingress](/kubernetes/ingress).
- To configure OIDC authentication, refer to [Access Control](/kubernetes/access-control).
Reference in New Issue View Git Blame Copy Permalink