Files
Emilien Macchianddivesh 8b3cc3fdc0 feat(server): add gateway capacity metrics and optional HPA (#3978)
* feat(server): add gateway capacity metrics and optional HPA

Expose per-replica supervisor sessions, pending relay capacity, relay
rejections and claim latency, and outbound peer request outcomes and
latency. Use bounded labels and Prometheus histograms for the new latency
metrics while preserving existing summary metrics.

Add an optional Helm HPA with external-database and resource validation,
conservative scale-down defaults, and support for custom metrics. Keep
certificate hook pods outside gateway workload selectors.

Document per-pod scraping, scaling limits, upgrade behavior, and
PostgreSQL connection sizing.

Part of #3528

Signed-off-by: Emilien Macchi <emacchi@redhat.com>

* feat(server): unify routing metrics and clarify replica capacity

Combine local relay setup and outbound peer requests in one counter, labeled by operation, route, target, outcome, and gRPC status. Preserve peer latency metrics.

Rename the rejection reason from global_capacity to replica_capacity to reflect the per-replica relay budget. Keep capacity limits unchanged.

Update tests and documentation.

Signed-off-by: divesh <dgude@nvidia.com>

* fix(server): align routed request metric labels and outcomes

Count a local relay as successful only when its supervisor claims it, the
same event that answers a peer relay on the owner, and rename the outcomes
to local_error and remote_error so they say where an attempt failed. Label
the peer latency histogram by operation, like the routed request counter,
and rename the target label to relay_kind so it does not read as the
Prometheus scrape target.

Rename RelayCapacity.global to per_replica, drop the per-sandbox relay
capacity gauge, which no per-sandbox series can pair with, describe the
latency histogram as peer-only, and restore the note that unavailable
spikes are expected during rollouts.

Part of #3528

Signed-off-by: Emilien Macchi <emacchi@redhat.com>

---------

Signed-off-by: Emilien Macchi <emacchi@redhat.com>
Signed-off-by: divesh <dgude@nvidia.com>
Co-authored-by: divesh <dgude@nvidia.com>
2026-10-05 15:38:40 +00:00

453 lines
18 KiB
Smarty

# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
{{/*
Expand the name of the chart.
*/}}
{{- define "openshell.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
*/}}
{{- define "openshell.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "openshell.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "openshell.labels" -}}
helm.sh/chart: {{ include "openshell.chart" . }}
{{ include "openshell.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "openshell.selectorLabels" -}}
app.kubernetes.io/name: {{ include "openshell.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Pod labels for the certgen hook Jobs. They keep the release instance label but
do not match openshell.selectorLabels, so gateway selectors (the workload,
Services, HorizontalPodAutoscaler, anti-affinity, and PodDisruptionBudgets)
never select hook pods.
*/}}
{{- define "openshell.certgenPodLabels" -}}
app.kubernetes.io/name: {{ printf "%s-certgen" (include "openshell.name" . | trunc 55 | trimSuffix "-") }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: certgen
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "openshell.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "openshell.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
{{/*
Create the name of the service account assigned to sandbox pods
*/}}
{{- define "openshell.sandboxServiceAccountName" -}}
{{- if .Values.sandboxServiceAccount.create }}
{{- default (printf "%s-sandbox" (include "openshell.fullname" .) | trunc 63 | trimSuffix "-") .Values.sandboxServiceAccount.name }}
{{- else }}
{{- default "default" .Values.sandboxServiceAccount.name }}
{{- end }}
{{- end }}
{{/*
Whether this chart owns workspace-scoped resources. Missing legacy values
default to enabled so upgrades with --reuse-values preserve the old topology.
*/}}
{{- define "openshell.workspaceResourcesEnabled" -}}
{{- $workspaceResources := .Values.workspaceResources | default dict -}}
{{- $enabled := true -}}
{{- if hasKey $workspaceResources "enabled" -}}
{{- $enabled = get $workspaceResources "enabled" -}}
{{- end -}}
{{- if $enabled -}}true{{- end -}}
{{- end }}
{{/*
Whether this chart owns gateway RBAC objects. Missing legacy values default to
enabled so upgrades with --reuse-values preserve the old topology.
*/}}
{{- define "openshell.rbacCreate" -}}
{{- $rbac := .Values.rbac | default dict -}}
{{- $create := true -}}
{{- if hasKey $rbac "create" -}}
{{- $create = get $rbac "create" -}}
{{- end -}}
{{- if $create -}}true{{- end -}}
{{- end }}
{{/*
The rbac.clusterScoped values map, tolerating missing legacy values.
*/}}
{{- define "openshell.clusterScopedRbacValues" -}}
{{- $rbac := .Values.rbac | default dict -}}
{{- $clusterScoped := dict -}}
{{- if hasKey $rbac "clusterScoped" -}}
{{- $clusterScoped = get $rbac "clusterScoped" | default dict -}}
{{- end -}}
{{- toYaml $clusterScoped -}}
{{- end }}
{{/*
Whether this chart owns the cluster-scoped ClusterRole and ClusterRoleBinding.
Disable for a namespace-admin install where a cluster-admin applies them
separately. Missing legacy values default to enabled.
*/}}
{{- define "openshell.clusterRbacCreate" -}}
{{- if include "openshell.rbacCreate" . -}}
{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}}
{{- $create := true -}}
{{- if hasKey $clusterScoped "create" -}}
{{- $create = get $clusterScoped "create" -}}
{{- end -}}
{{- if $create -}}true{{- end -}}
{{- end -}}
{{- end }}
{{/*
Name of the gateway ClusterRole. The release namespace is part of the default
name so multiple releases on one cluster do not collide.
*/}}
{{- define "openshell.clusterRoleName" -}}
{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}}
{{- $default := printf "%s-node-reader-%s" (include "openshell.fullname" .) .Release.Namespace -}}
{{- default $default (get $clusterScoped "clusterRoleName") -}}
{{- end }}
{{/*
Name of the gateway ClusterRoleBinding.
*/}}
{{- define "openshell.clusterRoleBindingName" -}}
{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}}
{{- $default := printf "%s-node-reader-%s" (include "openshell.fullname" .) .Release.Namespace -}}
{{- default $default (get $clusterScoped "clusterRoleBindingName") -}}
{{- end }}
{{/* Gateway image reference. A digest takes precedence over a tag. */}}
{{- define "openshell.image" -}}
{{- $image := .Values.gateway.image -}}
{{- $global := .Values.global.image -}}
{{- $registry := $image.registry | default $global.registry -}}
{{- $repository := ternary (printf "%s/%s" $registry $image.repository) $image.repository (ne $registry "") -}}
{{- if $image.digest -}}
{{- printf "%s@%s" $repository $image.digest -}}
{{- else -}}
{{- printf "%s:%s" $repository ($image.tag | default $global.tag | default .Chart.AppVersion) -}}
{{- end }}
{{- end }}
{{/* Sandbox image reference. A digest takes precedence over a tag. */}}
{{- define "openshell.sandboxImage" -}}
{{- $image := .Values.sandbox.image -}}
{{- if $image.digest -}}
{{- printf "%s@%s" $image.repository $image.digest -}}
{{- else -}}
{{- printf "%s:%s" $image.repository ($image.tag | default "latest") -}}
{{- end }}
{{- end }}
{{/* Official sandbox runtime repository used by the gateway's built-in default. */}}
{{- define "openshell.defaultSandboxRuntimeRepository" -}}
ghcr.io/nvidia/openshell/sandbox
{{- end }}
{{/* Whether Helm must propagate a sandbox runtime image override. */}}
{{- define "openshell.sandboxRuntimeImageOverrideEnabled" -}}
{{- $defaultRepository := include "openshell.defaultSandboxRuntimeRepository" . -}}
{{- $global := .Values.global.image -}}
{{- $registry := .Values.sandboxRuntime.image.registry | default $global.registry -}}
{{- $repository := ternary (printf "%s/%s" $registry .Values.sandboxRuntime.image.repository) .Values.sandboxRuntime.image.repository (ne $registry "") -}}
{{- if or (ne $repository $defaultRepository) .Values.sandboxRuntime.image.tag .Values.sandboxRuntime.image.digest .Values.global.image.tag -}}true{{- end -}}
{{- end }}
{{/* Sandbox runtime image override. */}}
{{- define "openshell.sandboxRuntimeImage" -}}
{{- $global := .Values.global.image -}}
{{- $registry := .Values.sandboxRuntime.image.registry | default $global.registry -}}
{{- $repository := ternary (printf "%s/%s" $registry .Values.sandboxRuntime.image.repository) .Values.sandboxRuntime.image.repository (ne $registry "") -}}
{{- if .Values.sandboxRuntime.image.digest -}}
{{- printf "%s@%s" $repository .Values.sandboxRuntime.image.digest -}}
{{- else -}}
{{- $tag := .Values.sandboxRuntime.image.tag | default $global.tag | default .Chart.AppVersion -}}
{{- printf "%s:%s" $repository $tag -}}
{{- end -}}
{{- end }}
{{/*
Whether the gateway listener should verify client certificates (mTLS).
An explicit empty server.tls.clientCaSecretName disables client-CA wiring in
both gateway.toml and the workload, overriding built-in PKI and cert-manager
defaults.
*/}}
{{- define "openshell.gatewayClientCaEnabled" -}}
{{- if .Values.server.disableTls -}}
{{- else if not .Values.server.tls.enableMtls -}}
{{- else if eq .Values.server.tls.clientCaSecretName "" -}}
{{- else if or .Values.server.tls.clientCaSecretName (and .Values.pkiInitJob.enabled (not .Values.certManager.enabled)) (and .Values.certManager.enabled .Values.certManager.clientCaFromServerTlsSecret) -}}
true
{{- end -}}
{{- end -}}
{{/* Supervisor image override. */}}
{{- define "openshell.supervisorImage" -}}
{{- $global := .Values.global.image -}}
{{- $registry := .Values.supervisor.image.registry | default $global.registry -}}
{{- $repository := ternary (printf "%s/%s" $registry .Values.supervisor.image.repository) .Values.supervisor.image.repository (ne $registry "") -}}
{{- if .Values.supervisor.image.digest -}}
{{- printf "%s@%s" $repository .Values.supervisor.image.digest -}}
{{- else -}}
{{- $tag := .Values.supervisor.image.tag | default $global.tag | default .Chart.AppVersion -}}
{{- printf "%s:%s" $repository $tag -}}
{{- end }}
{{- end }}
{{/*
Namespaced Issuer (selfSigned) for cert-manager CA bootstrap.
*/}}
{{- define "openshell.issuerSelfSigned" -}}
{{- printf "%s-selfsigned" (include "openshell.fullname" .) | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Namespace where sandbox pods are created. An explicit
.Values.server.sandboxNamespace is used verbatim. Otherwise it defaults to
.Release.Namespace so `helm install -n my-ns` works without extra overrides.
*/}}
{{- define "openshell.sandboxNamespace" -}}
{{- .Values.server.sandboxNamespace | default .Release.Namespace -}}
{{- end }}
{{/*
Secrets in the sandbox namespace whose contents the Kubernetes driver stages
into per-generation Secrets in workspace namespaces, as a JSON array. Empty in
shared workspace mode.
*/}}
{{- define "openshell.workspaceSecretSourceNames" -}}
{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}}
{{- $names := list -}}
{{- if and (ne $workspaceMode "shared") (not .Values.server.disableTls) -}}
{{- $names = append $names .Values.server.tls.clientTlsSecretName -}}
{{- end -}}
{{- if eq $workspaceMode "managed" -}}
{{- range .Values.server.sandboxImagePullSecrets -}}
{{- if .name -}}
{{- $names = append $names .name -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- uniq $names | toJson -}}
{{- end }}
{{/*
Namespace where Kubernetes Secret-backed provider credentials live.
*/}}
{{- define "openshell.credentialKubernetesSecretsNamespace" -}}
{{- .Values.server.credentialDrivers.kubernetesSecrets.namespace | default .Release.Namespace -}}
{{- end }}
{{/*
Name of the Secret holding the default credential storage key-encryption key.
When server.credentialStorage.existingSecret is set, returns that name instead
of the chart-generated name (for GitOps / helm-template workflows).
*/}}
{{- define "openshell.credentialStorageKeyEncryptionKeySecretName" -}}
{{- if .Values.server.credentialStorage.existingSecret -}}
{{- .Values.server.credentialStorage.existingSecret -}}
{{- else -}}
{{- printf "%s-credential-storage-key-encryption-key" (include "openshell.fullname" .) | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end }}
{{/*
Key inside the default credential storage key-encryption key Secret.
*/}}
{{- define "openshell.credentialStorageKeyEncryptionKeySecretKey" -}}
key-encryption-key
{{- end }}
{{/*
Gateway environment variable used to pass the default credential storage key-encryption key.
*/}}
{{- define "openshell.credentialStorageKeyEncryptionKeyEnvName" -}}
OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY
{{- end }}
{{/*
Name of the Secret holding gateway-minted sandbox JWT signing material.
*/}}
{{- define "openshell.sandboxJwtSecretName" -}}
{{- .Values.server.sandboxJwt.signingSecretName | default (printf "%s-jwt-keys" (include "openshell.fullname" .)) -}}
{{- end }}
{{- define "openshell.peerServiceName" -}}
{{- printf "%s-peer" (include "openshell.fullname" .) -}}
{{- end }}
{{/*
gRPC endpoint sandbox pods use to call back into the gateway. An explicit
.Values.server.grpcEndpoint is used verbatim. Otherwise it is derived from
the in-cluster Service DNS, release namespace, service port, and disableTls
flag — so the default value works for any release name or namespace without
override.
*/}}
{{- define "openshell.grpcEndpoint" -}}
{{- if .Values.server.grpcEndpoint -}}
{{- .Values.server.grpcEndpoint -}}
{{- else -}}
{{- $scheme := ternary "http" "https" (default false .Values.server.disableTls) -}}
{{- printf "%s://%s.%s.svc.cluster.local:%d" $scheme (include "openshell.fullname" .) .Release.Namespace (int .Values.service.port) -}}
{{- end -}}
{{- end }}
{{/*
Default server certificate DNS SANs derived from the release name and namespace.
Returns a YAML list. Append extra SANs from values with range loops.
*/}}
{{- define "openshell.defaultServerDnsNames" -}}
{{- $name := include "openshell.fullname" . -}}
{{- $ns := .Release.Namespace -}}
{{- list $name
(printf "%s.%s.svc" $name $ns)
(printf "%s.%s.svc.cluster.local" $name $ns)
"localhost"
(printf "%s.localhost" $name)
(printf "*.%s.localhost" $name)
"host.docker.internal"
"host.containers.internal"
| toYaml }}
{{- end }}
{{/*
Name of the ConfigMap holding the backend CA for BackendTLSPolicy validation.
*/}}
{{- define "openshell.backendCaConfigMapName" -}}
{{- .Values.grpcRoute.backendTLSPolicy.caCertificateConfigMapName | default (printf "%s-backend-ca" (include "openshell.fullname" .)) -}}
{{- end }}
{{/*
Gateway workload kind. StatefulSet is the default because the default SQLite
database requires persistent per-pod storage.
*/}}
{{- define "openshell.workloadKind" -}}
{{- $workload := .Values.workload | default dict -}}
{{- if not (kindIs "map" $workload) -}}
{{- fail "workload must be a map with kind and allowMultiReplicaStatefulSet fields." -}}
{{- end -}}
{{- default "statefulset" (get $workload "kind") | lower -}}
{{- end }}
{{/*
Translate chart image pull policy values to the canonical gateway vocabulary.
The Kubernetes spellings remain accepted so existing values files continue to
work across the schema-v2 chart upgrade.
*/}}
{{- define "openshell.canonicalImagePullPolicy" -}}
{{- $policy := printf "%v" . -}}
{{- if eq $policy "Always" -}}
always
{{- else if eq $policy "IfNotPresent" -}}
if_not_present
{{- else if eq $policy "Never" -}}
never
{{- else if has $policy (list "always" "if_not_present" "never") -}}
{{- $policy -}}
{{- else -}}
{{- fail (printf "image pull policy %q must be one of: always, if_not_present, never, Always, IfNotPresent, Never" $policy) -}}
{{- end -}}
{{- end }}
{{/*
Render a sandbox UID/GID chart value as an integer, or nothing when unset.
Takes a dict with `name` (the values key, for errors) and `value`. The bounds
match openshell_policy::MIN_SANDBOX_UID..=MAX_SANDBOX_UID. Helm parses YAML
numbers as float64, so the integer conversion also avoids `2e+09` rendering.
Booleans are rejected because they would otherwise convert to 1 or 0.
*/}}
{{- define "openshell.sandboxId" -}}
{{- if not (or (kindIs "invalid" .value) (eq (toString .value) "")) -}}
{{- $id := int64 .value -}}
{{- if or (kindIs "bool" .value) (ne (float64 .value) (float64 $id)) (lt $id 1) (gt $id 4294967294) -}}
{{- fail (printf "%s must be an integer between 1 and 4294967294" .name) -}}
{{- end -}}
{{- $id -}}
{{- end -}}
{{- end }}
{{/*
Validate chart values that Helm would otherwise accept silently.
*/}}
{{- define "openshell.validateValues" -}}
{{- $workloadKind := include "openshell.workloadKind" . -}}
{{- $workload := .Values.workload | default dict -}}
{{- $maxReplicas := int (include "openshell.maxReplicas" .) -}}
{{- $maxReplicasSource := include "openshell.maxReplicasSource" . -}}
{{- if and (hasKey .Values "postgres") (kindIs "map" .Values.postgres) (hasKey .Values.postgres "enabled") -}}
{{- fail "postgres.enabled was removed; the OpenShell chart no longer deploys PostgreSQL. Provision PostgreSQL separately and set server.externalDbSecret to a Secret containing a PostgreSQL URI." -}}
{{- end -}}
{{- if not (or (eq $workloadKind "statefulset") (eq $workloadKind "deployment")) -}}
{{- fail "workload.kind must be one of: statefulset, deployment." -}}
{{- end -}}
{{- if and (eq $workloadKind "deployment") (not .Values.server.externalDbSecret) -}}
{{- fail "workload.kind=deployment requires server.externalDbSecret; use workload.kind=statefulset for the default SQLite database." -}}
{{- end -}}
{{- include "openshell.validateAutoscaling" . -}}
{{- if and (gt $maxReplicas 1) (not .Values.server.externalDbSecret) -}}
{{- fail (printf "%s > 1 requires server.externalDbSecret; multiple gateway replicas cannot share the default per-pod SQLite database." $maxReplicasSource) -}}
{{- end -}}
{{- if and (eq $workloadKind "statefulset") (gt $maxReplicas 1) (not (get $workload "allowMultiReplicaStatefulSet" | default false)) -}}
{{- fail (printf "%s > 1 with workload.kind=statefulset requires workload.allowMultiReplicaStatefulSet=true; use workload.kind=deployment for external database-backed multi-replica gateways." $maxReplicasSource) -}}
{{- end -}}
{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}}
{{- if not (has $workspaceMode (list "shared" "managed" "operator")) -}}
{{- fail "server.drivers.kubernetes.workspaceMode must be one of: shared, managed, operator." -}}
{{- end -}}
{{- $credentialDrivers := list -}}
{{- if .Values.server.credentialDrivers.kubernetesSecrets.enabled -}}
{{- $credentialDrivers = append $credentialDrivers "kubernetes-secrets" -}}
{{- end -}}
{{- if .Values.server.credentialDrivers.vault.enabled -}}
{{- $credentialDrivers = append $credentialDrivers "vault" -}}
{{- end -}}
{{- if gt (len $credentialDrivers) 1 -}}
{{- fail "only one external server.credentialDrivers backend can be enabled at a time." -}}
{{- end -}}
{{- if kindIs "invalid" .Values.server.tls.clientCaSecretName -}}
{{- fail "server.tls.clientCaSecretName cannot be null; omit the key to use the chart default (openshell-server-client-ca), or set to \"\" to disable client certificate verification for HTTPS-only mode" -}}
{{- end -}}
{{- end }}