Files
OpenShell/deploy/helm/openshell/templates/gateway-config.yaml
Taylor MutchandSeth Jennings 8eacb4779f feat(kubernetes): add sidecar supervisor topology (#2076)
* feat(kubernetes): add sidecar supervisor topology

Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar process id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar process id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): avoid similar proxy id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* docs(kubernetes): clarify sidecar topology limits

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): keep sidecar process leaf capless

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): refresh sidecar provider env snapshots

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* test(supervisor): align hot-swap identity regression

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): stage sidecar mtls files before proxy chown

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): simplify sidecar supervisor topology

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* chore(helm): reuse sidecar skaffold values

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar iptables helper names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(e2e): harden kube gateway wrapper setup

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid nft batch rollback on OCP

Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules.

Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): preserve process identity in sidecar topology

Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table.

Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* refactor(kubernetes): replace sidecar snapshots with control socket

Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files.

Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* feat(kubernetes): support relaxed sidecar network identity

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): satisfy sidecar clippy lint

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* refactor(kubernetes): standardize topology naming

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): satisfy linux clippy timeout import

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): support kata sidecar on ipv4 pods

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): satisfy linux clippy for sidecar fallback

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* chore(kubernetes): remove stale supervisor topology references

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): enable sidecar binary policy inspection

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): harden sidecar control boundary

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): couple sidecar supervisor lifecycles

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Co-authored-by: Seth Jennings <sjenning@redhat.com>
2026-07-10 13:01:39 -07:00

151 lines
6.9 KiB
YAML

# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
{{/*
ConfigMap holding the gateway TOML config file (RFC 0003).
The gateway reads `/etc/openshell/gateway.toml` (mounted from this ConfigMap)
at startup. CLI flags and OPENSHELL_* env vars on the gateway workload container
still override anything in this file.
One value is intentionally NOT rendered here:
- server.dbUrl → passed via OPENSHELL_DB_URL env var (from Secret)
when server.externalDbSecret is set, otherwise
--db-url arg for SQLite
*/}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "openshell.fullname" . }}-config
labels:
{{- include "openshell.labels" . | nindent 4 }}
data:
gateway.toml: |
[openshell]
version = 1
[openshell.gateway]
bind_address = "0.0.0.0:{{ .Values.service.port }}"
{{- if .Values.service.healthPort }}
health_bind_address = "0.0.0.0:{{ .Values.service.healthPort }}"
{{- end }}
{{- if .Values.service.metricsPort }}
metrics_bind_address = "0.0.0.0:{{ .Values.service.metricsPort }}"
{{- end }}
log_level = {{ .Values.server.logLevel | quote }}
sandbox_namespace = {{ include "openshell.sandboxNamespace" . | quote }}
default_image = {{ .Values.server.sandboxImage | quote }}
{{- if include "openshell.supervisorImageOverrideEnabled" . }}
supervisor_image = {{ include "openshell.supervisorImage" . | quote }}
{{- end }}
{{- if .Values.server.hostGatewayIP }}
host_gateway_ip = {{ .Values.server.hostGatewayIP | quote }}
{{- end }}
{{- if .Values.server.enableUserNamespaces }}
enable_user_namespaces = true
{{- end }}
{{- if .Values.server.disableTls }}
disable_tls = true
{{- else }}
client_tls_secret_name = {{ .Values.server.tls.clientTlsSecretName | quote }}
{{- end }}
enable_loopback_service_http = {{ .Values.server.enableLoopbackServiceHttp }}
{{- $sans := list -}}
{{- if and .Values.certManager.enabled .Values.certManager.serverDnsNames }}
{{- $sans = .Values.certManager.serverDnsNames }}
{{- else if and .Values.pkiInitJob.enabled .Values.pkiInitJob.serverDnsNames }}
{{- $sans = .Values.pkiInitJob.serverDnsNames }}
{{- end }}
{{- if $sans }}
server_sans = [{{- range $i, $san := $sans }}{{ if $i }}, {{ end }}{{ $san | quote }}{{- end }}]
{{- end }}
{{- $rlRequests := int .Values.server.grpcRateLimit.requests }}
{{- $rlWindowSeconds := int .Values.server.grpcRateLimit.windowSeconds }}
{{- if or (lt $rlRequests 0) (lt $rlWindowSeconds 0) }}
{{- fail "server.grpcRateLimit.requests and server.grpcRateLimit.windowSeconds must not be negative; they map to unsigned gateway settings" }}
{{- end }}
{{- if and (gt $rlRequests 0) (gt $rlWindowSeconds 0) }}
grpc_rate_limit_requests = {{ $rlRequests }}
grpc_rate_limit_window_seconds = {{ $rlWindowSeconds }}
{{- else if or (gt $rlRequests 0) (gt $rlWindowSeconds 0) }}
{{- fail "server.grpcRateLimit requires both requests and windowSeconds to be positive to enable rate limiting, or both 0/unset to disable it" }}
{{- end }}
{{- if not .Values.server.disableTls }}
[openshell.gateway.tls]
cert_path = "/etc/openshell-tls/server/tls.crt"
key_path = "/etc/openshell-tls/server/tls.key"
client_ca_path = "/etc/openshell-tls/client-ca/ca.crt"
{{- end }}
{{- if .Values.server.auth.allowUnauthenticatedUsers }}
[openshell.gateway.auth]
allow_unauthenticated_users = true
{{- end }}
[openshell.gateway.gateway_jwt]
signing_key_path = "/etc/openshell-jwt/signing.pem"
public_key_path = "/etc/openshell-jwt/public.pem"
kid_path = "/etc/openshell-jwt/kid"
gateway_id = {{ .Values.server.sandboxJwt.gatewayId | default (include "openshell.fullname" .) | quote }}
ttl_secs = {{ .Values.server.sandboxJwt.ttlSecs | default 3600 }}
{{- if .Values.server.oidc.issuer }}
[openshell.gateway.oidc]
issuer = {{ .Values.server.oidc.issuer | quote }}
audience = {{ .Values.server.oidc.audience | quote }}
jwks_ttl_secs = {{ .Values.server.oidc.jwksTtl }}
{{- if .Values.server.oidc.rolesClaim }}
roles_claim = {{ .Values.server.oidc.rolesClaim | quote }}
{{- end }}
{{- if .Values.server.oidc.adminRole }}
admin_role = {{ .Values.server.oidc.adminRole | quote }}
{{- end }}
{{- if .Values.server.oidc.userRole }}
user_role = {{ .Values.server.oidc.userRole | quote }}
{{- end }}
{{- if .Values.server.oidc.scopesClaim }}
scopes_claim = {{ .Values.server.oidc.scopesClaim | quote }}
{{- end }}
{{- end }}
[openshell.drivers.kubernetes]
grpc_endpoint = {{ include "openshell.grpcEndpoint" . | quote }}
service_account_name = {{ include "openshell.sandboxServiceAccountName" . | quote }}
supervisor_sideload_method = {{ include "openshell.supervisorSideloadMethod" . | quote }}
topology = {{ .Values.supervisor.topology | default "combined" | quote }}
sa_token_ttl_secs = {{ .Values.server.sandboxJwt.k8sSaTokenTtlSecs | default 3600 }}
{{- if .Values.server.providerTokenGrants.spiffe.enabled }}
provider_spiffe_workload_api_socket_path = {{ .Values.server.providerTokenGrants.spiffe.workloadApiSocketPath | quote }}
{{- end }}
{{- if .Values.server.sandboxImagePullPolicy }}
image_pull_policy = {{ .Values.server.sandboxImagePullPolicy | quote }}
{{- end }}
{{- $sandboxImagePullSecretNames := list -}}
{{- range .Values.server.sandboxImagePullSecrets }}
{{- if .name }}
{{- $sandboxImagePullSecretNames = append $sandboxImagePullSecretNames .name }}
{{- end }}
{{- end }}
{{- if $sandboxImagePullSecretNames }}
image_pull_secrets = [{{- range $i, $name := $sandboxImagePullSecretNames }}{{ if $i }}, {{ end }}{{ $name | quote }}{{- end }}]
{{- end }}
{{- if .Values.server.workspaceDefaultStorageSize }}
workspace_default_storage_size = {{ .Values.server.workspaceDefaultStorageSize | quote }}
{{- end }}
{{- if .Values.server.defaultRuntimeClassName }}
default_runtime_class_name = {{ .Values.server.defaultRuntimeClassName | quote }}
{{- end }}
{{- if .Values.server.appArmorProfile }}
app_armor_profile = {{ .Values.server.appArmorProfile | quote }}
{{- end }}
{{- if .Values.supervisor.image.pullPolicy }}
supervisor_image_pull_policy = {{ .Values.supervisor.image.pullPolicy | quote }}
{{- end }}
[openshell.drivers.kubernetes.sidecar]
proxy_uid = {{ .Values.supervisor.sidecar.proxyUid | default 1337 }}
process_binary_aware_network_policy = {{ .Values.supervisor.sidecar.processBinaryAwareNetworkPolicy }}