mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-20 06:00:06 +08:00
* feat(kubernetes): add sidecar supervisor topology Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid similar process id names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid similar process id names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(sandbox): avoid similar proxy id names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * docs(kubernetes): clarify sidecar topology limits Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): keep sidecar process leaf capless Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): refresh sidecar provider env snapshots Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * test(supervisor): align hot-swap identity regression Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): stage sidecar mtls files before proxy chown Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): simplify sidecar supervisor topology Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * chore(helm): reuse sidecar skaffold values Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid similar iptables helper names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(e2e): harden kube gateway wrapper setup Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid nft batch rollback on OCP Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules. Signed-off-by: Seth Jennings <sjenning@redhat.com> Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): preserve process identity in sidecar topology Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table. Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * refactor(kubernetes): replace sidecar snapshots with control socket Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files. Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * feat(kubernetes): support relaxed sidecar network identity Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(sandbox): satisfy sidecar clippy lint Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * refactor(kubernetes): standardize topology naming Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(sandbox): satisfy linux clippy timeout import Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): support kata sidecar on ipv4 pods Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): satisfy linux clippy for sidecar fallback Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * chore(kubernetes): remove stale supervisor topology references Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): enable sidecar binary policy inspection Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): harden sidecar control boundary Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): couple sidecar supervisor lifecycles Signed-off-by: Taylor Mutch <taylormutch@gmail.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> Co-authored-by: Seth Jennings <sjenning@redhat.com>
145 lines
5.5 KiB
Plaintext
145 lines
5.5 KiB
Plaintext
---
|
|
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
title: "Ingress"
|
|
sidebar-title: "Ingress"
|
|
description: "Expose the OpenShell gateway externally using the Kubernetes Gateway API and a GRPCRoute."
|
|
keywords: "Generative AI, Cybersecurity, Kubernetes, Gateway API, Envoy Gateway, GRPCRoute, Ingress, External Access"
|
|
position: 4
|
|
---
|
|
|
|
By default, the OpenShell gateway is only reachable inside the cluster. To let CLI clients connect without a `kubectl port-forward`, expose the gateway through an ingress.
|
|
|
|
OpenShell uses the [Kubernetes Gateway API](https://gateway-api.sigs.k8s.io) for ingress. The chart creates a `GRPCRoute` that routes inbound gRPC traffic to the gateway pod. You need a Gateway API implementation installed on your cluster to fulfill the `GRPCRoute`. This page uses [Envoy Gateway](https://gateway.envoyproxy.io), which the chart is tested with.
|
|
|
|
## Install Envoy Gateway
|
|
|
|
Envoy Gateway installs the Gateway API CRDs and controller:
|
|
|
|
```shell
|
|
helm install eg \
|
|
oci://docker.io/envoyproxy/gateway-helm \
|
|
--version v1.8.1 \
|
|
--namespace envoy-gateway-system \
|
|
--create-namespace \
|
|
--wait
|
|
```
|
|
|
|
## Create the GatewayClass
|
|
|
|
Create the `eg` GatewayClass that the OpenShell chart references:
|
|
|
|
```shell
|
|
kubectl apply -f - <<'EOF'
|
|
apiVersion: gateway.networking.k8s.io/v1
|
|
kind: GatewayClass
|
|
metadata:
|
|
name: eg
|
|
spec:
|
|
controllerName: gateway.envoyproxy.io/gatewayclass-controller
|
|
EOF
|
|
```
|
|
|
|
Verify the GatewayClass is accepted:
|
|
|
|
```shell
|
|
kubectl get gatewayclass eg
|
|
```
|
|
|
|
The `ACCEPTED` column should show `True`.
|
|
|
|
## Install OpenShell with Gateway API enabled
|
|
|
|
Enable the GRPCRoute and let the chart create a Gateway resource in the `openshell` namespace:
|
|
|
|
```shell
|
|
helm upgrade --install openshell \
|
|
oci://ghcr.io/nvidia/openshell/helm-chart \
|
|
--version <version> \
|
|
--namespace openshell \
|
|
--set grpcRoute.enabled=true \
|
|
--set grpcRoute.gateway.create=true \
|
|
--set grpcRoute.gateway.className=eg
|
|
```
|
|
|
|
## Get the external address
|
|
|
|
After the Gateway is provisioned, Envoy Gateway creates a LoadBalancer service in the `openshell` namespace. Wait for it to get an external address:
|
|
|
|
```shell
|
|
kubectl -n openshell get svc -l gateway.envoyproxy.io/owning-gateway-name=openshell
|
|
```
|
|
|
|
After the `EXTERNAL-IP` is assigned, register the gateway with the CLI:
|
|
|
|
```shell
|
|
openshell gateway add http://<external-ip> --name production
|
|
openshell status
|
|
```
|
|
|
|
This setup is plaintext end-to-end and is intended for development. For external access, terminate TLS at the gateway as shown below.
|
|
|
|
## HTTPS (TLS termination)
|
|
|
|
Envoy Gateway can terminate TLS at the listener and forward plaintext to the OpenShell gateway pod:
|
|
|
|
```text
|
|
client → HTTPS → Envoy Gateway (terminate TLS) → plaintext → openshell gateway pod
|
|
```
|
|
|
|
Envoy Gateway only terminates TLS here — it does not perform OIDC. Do not enable an Envoy Gateway OIDC `SecurityPolicy` in front of the gateway: that flow relies on browser redirects and cookies and cannot work with the OpenShell CLI or headless agents. Instead, the OpenShell gateway validates an OIDC bearer token that the client sends in the gRPC `authorization` metadata, which Envoy forwards untouched.
|
|
|
|
Because Envoy terminates TLS, the OpenShell gateway never sees a client certificate, so client mTLS cannot provide identity on this path. Use OIDC bearer tokens for client identity instead.
|
|
|
|
For headless agents and CI, the CLI obtains the token via the OAuth2 client-credentials grant (no browser): set `OPENSHELL_OIDC_CLIENT_SECRET` to the OAuth client secret before running `openshell gateway add`. The client id comes from `--oidc-client-id` (default `openshell-cli`); pass it explicitly if your IdP client uses a different id. Interactive human users get the browser-based Authorization Code + PKCE flow by default.
|
|
|
|
### Provide a TLS certificate
|
|
|
|
Create a `kubernetes.io/tls` Secret in the `openshell` namespace with the certificate for your external hostname:
|
|
|
|
```shell
|
|
kubectl -n openshell create secret tls openshell-ingress-tls \
|
|
--cert=tls.crt --key=tls.key
|
|
```
|
|
|
|
The Secret may also be issued by cert-manager, or you can reference the chart's existing `openshell-server-tls` Secret if its SANs include the external hostname.
|
|
|
|
### Install with HTTPS termination
|
|
|
|
Enable an HTTPS listener, point it at the Secret, disable gateway-pod TLS so Envoy forwards plaintext, and configure an OIDC issuer for client identity:
|
|
|
|
```shell
|
|
helm upgrade --install openshell \
|
|
oci://ghcr.io/nvidia/openshell/helm-chart \
|
|
--version <version> \
|
|
--namespace openshell \
|
|
--set grpcRoute.enabled=true \
|
|
--set grpcRoute.gateway.create=true \
|
|
--set grpcRoute.gateway.className=eg \
|
|
--set grpcRoute.gateway.listener.protocol=HTTPS \
|
|
--set grpcRoute.gateway.listener.port=443 \
|
|
--set 'grpcRoute.gateway.listener.tls.certificateRefs[0].name=openshell-ingress-tls' \
|
|
--set server.disableTls=true \
|
|
--set server.oidc.issuer=https://<issuer> \
|
|
--set 'grpcRoute.hostnames[0]=<external-hostname>'
|
|
```
|
|
|
|
Keep the certificate Secret in the release namespace. Referencing a Secret in another namespace requires a `ReferenceGrant`.
|
|
|
|
### Register over HTTPS
|
|
|
|
```shell
|
|
openshell gateway add https://<external-hostname> --name production --oidc-issuer https://<issuer>
|
|
openshell status
|
|
```
|
|
|
|
See [Authentication](/kubernetes/setup) for OIDC issuer, audience, and roles configuration.
|
|
|
|
## SSH Relay
|
|
|
|
Sandbox SSH uses the gateway endpoint registered with the CLI. No separate Helm SSH host or port values are required.
|
|
|
|
## Next Steps
|
|
|
|
Return to [Setup](/kubernetes/setup) to complete the installation.
|