Files
OpenShell/deploy/helm/openshell/skaffold.yaml
Taylor MutchandSeth Jennings 8eacb4779f feat(kubernetes): add sidecar supervisor topology (#2076)
* feat(kubernetes): add sidecar supervisor topology

Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar process id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar process id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): avoid similar proxy id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* docs(kubernetes): clarify sidecar topology limits

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): keep sidecar process leaf capless

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): refresh sidecar provider env snapshots

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* test(supervisor): align hot-swap identity regression

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): stage sidecar mtls files before proxy chown

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): simplify sidecar supervisor topology

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* chore(helm): reuse sidecar skaffold values

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar iptables helper names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(e2e): harden kube gateway wrapper setup

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid nft batch rollback on OCP

Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules.

Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): preserve process identity in sidecar topology

Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table.

Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* refactor(kubernetes): replace sidecar snapshots with control socket

Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files.

Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* feat(kubernetes): support relaxed sidecar network identity

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): satisfy sidecar clippy lint

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* refactor(kubernetes): standardize topology naming

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): satisfy linux clippy timeout import

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): support kata sidecar on ipv4 pods

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): satisfy linux clippy for sidecar fallback

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* chore(kubernetes): remove stale supervisor topology references

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): enable sidecar binary policy inspection

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): harden sidecar control boundary

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): couple sidecar supervisor lifecycles

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Co-authored-by: Seth Jennings <sjenning@redhat.com>
2026-07-10 13:01:39 -07:00

146 lines
5.5 KiB
YAML

# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Local dev: builds gateway + supervisor images via tasks/scripts/docker-build-image.sh,
# which first stages Rust binaries natively on the host (using cargo / cargo-zigbuild
# when cross-compiling) and then builds the image from the prebuilt binary. This
# mirrors CI and is faster than compiling inside Docker on every rebuild because
# the host's cargo target cache and sccache are reused across iterations.
#
# Run from repo root:
# mise run helm:skaffold:dev
# mise run helm:skaffold:run
#
# See https://skaffold.dev/docs/deployers/helm/ (setValueTemplates, IMAGE_* fields).
apiVersion: skaffold/v4beta14
kind: Config
metadata:
name: openshell
build:
local:
push: false
tagPolicy:
gitCommit: {}
artifacts:
- image: openshell/gateway
context: ../../..
custom:
buildCommand: |
CONTAINER_ENGINE_TARGET=local-k8s-cluster \
IMAGE_NAME="${IMAGE%:*}" \
IMAGE_TAG="${IMAGE##*:}" \
tasks/scripts/docker-build-image.sh gateway
dependencies:
paths:
- Cargo.toml
- Cargo.lock
- crates/**
- proto/**
- deploy/docker/Dockerfile.gateway
- tasks/scripts/docker-build-image.sh
- tasks/scripts/stage-prebuilt-binaries.sh
- image: openshell/supervisor
context: ../../..
custom:
buildCommand: |
CONTAINER_ENGINE_TARGET=local-k8s-cluster \
IMAGE_NAME="${IMAGE%:*}" \
IMAGE_TAG="${IMAGE##*:}" \
tasks/scripts/docker-build-image.sh supervisor
dependencies:
paths:
- Cargo.toml
- Cargo.lock
- crates/**
- proto/**
- deploy/docker/Dockerfile.supervisor
- tasks/scripts/docker-build-image.sh
- tasks/scripts/stage-prebuilt-binaries.sh
deploy:
helm:
releases:
# cert-manager — comment this in and add values-cert-manager.yaml below
# when you want cert-manager to manage TLS while certgen handles JWT.
# Requires cert-manager CRDs to be installed in the cluster first.
#- name: cert-manager
# remoteChart: oci://quay.io/jetstack/charts/cert-manager
# version: v1.20.2
# namespace: cert-manager
# createNamespace: true
# setValues:
# crds.enabled: true
# Envoy Gateway — Kubernetes Gateway API implementation.
# Installs the Gateway API CRDs and the "eg" GatewayClass.
# Required when grpcRoute.enabled is true in the openshell release.
#- name: envoy-gateway
# remoteChart: oci://docker.io/envoyproxy/gateway-helm
# version: v1.7.2
# namespace: envoy-gateway-system
# createNamespace: true
# # wait ensures Gateway API CRDs are registered before the openshell
# # release attempts to create Gateway and HTTPRoute resources.
# wait: true
# SPIRE — installs SPIRE Server, Agent, Controller Manager, CSI Driver,
# and OIDC Discovery Provider using the SPIFFE hardened charts.
# Uncomment both releases and ci/values-spire.yaml below to use
# SPIFFE JWT-SVIDs for dynamic provider token grants.
#- name: spire-crds
# repo: https://spiffe.github.io/helm-charts-hardened/
# remoteChart: spire-crds
# version: 0.5.0
# namespace: spire
# createNamespace: true
# wait: true
#- name: spire
# repo: https://spiffe.github.io/helm-charts-hardened/
# remoteChart: spire
# version: 0.29.0
# namespace: spire
# createNamespace: true
# valuesFiles:
# - ci/values-spire-stack.yaml
# wait: true
- name: openshell
chartPath: .
namespace: openshell
createNamespace: true
valuesFiles:
- values.yaml
- ci/values-skaffold.yaml
# Add ci/values-cert-manager.yaml here (and uncomment the cert-manager
# release above) to switch TLS generation to cert-manager.
#- ci/values-cert-manager.yaml
# To enable OIDC with a local Keycloak instance, run the one-time
# setup task first, then uncomment the line below:
# mise run keycloak:k8s:setup
#- ci/values-keycloak.yaml
# To enable the Gateway API HTTPRoute (requires Envoy Gateway above):
#- ci/values-gateway.yaml
# To enable SPIFFE/SPIRE provider token grants (requires the
# spire-crds and spire releases above):
#- ci/values-spire.yaml
# To exercise the Kubernetes supervisor sidecar topology:
#- ci/values-sidecar.yaml
# To test multi-replica external PostgreSQL behavior:
#- ci/values-high-availability.yaml
setValueTemplates:
image.repository: '{{.IMAGE_REPO_openshell_gateway}}'
image.tag: '{{.IMAGE_TAG_openshell_gateway}}'
supervisor.image.repository: '{{.IMAGE_REPO_openshell_supervisor}}'
supervisor.image.tag: '{{.IMAGE_TAG_openshell_supervisor}}'
profiles:
- name: sidecar
patches:
- op: add
path: /deploy/helm/releases/0/valuesFiles/-
value: ci/values-sidecar.yaml
- name: sidecar-mtls
patches:
- op: add
path: /deploy/helm/releases/0/valuesFiles/-
value: ci/values-sidecar.yaml
- op: add
path: /deploy/helm/releases/0/setValues
value:
server.disableTls: "false"