trending/OpenShell
1
0
Fork 0
mirror of https://github.com/NVIDIA/OpenShell.git synced 2026-10-20 06:00:06 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
pull-request/4145
OpenShell/deploy/helm/openshell/templates/_gateway-workload.tpl
T
Taylor MutchDrew NewberrydiveshDivesh Chowdary
 0a770d9173 feat(kubernetes): support HA gateway rebalancing (#1868) 
* feat(kubernetes): support HA gateway rebalancing

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* perf(server): cache peer connections, tokens, and owner lookups

Every forwarded relay rebuilt its setup from scratch: an owner lookup, a
blocking read of the peer token, a TLS connect to the owning replica, and
a TokenReview plus Pod GET on the receiving side. Sandbox service routing
does this per HTTP request, so the apiserver calls scaled with traffic.

Cache all of it on ServerState:

- peer channels pooled per endpoint, so relays multiplex over one
  connection instead of redialing
- peer tokens keyed by SHA-256, expiring at min(ttl, token exp) so a hit
  cannot accept an expired token
- owner records for 3s against a 45s ownership TTL, still freshness
  checked before use

Entries are evicted when a relay fails. Also raise HTTP/2
max_concurrent_streams to 1024, since pooling funnels every relay between
two replicas onto one connection and hyper's default of 200 sits below
the 256 pending-relay budget.

Signed-off-by: divesh <dgude@nvidia.com>

* perf(server): pool upstream connections for sandbox services

Each HTTP request to a sandbox service opened its own supervisor relay,
paying a new TCP connection and HTTP/1 handshake every time. Worse, it
counted against the 32 in-flight relay cap, so a service handling more
than 32 concurrent requests failed outright.

Pool idle upstreams per endpoint and port, up to 8 each for 15s. Reuse is
safe because the pool only returns a connection hyper reports as ready,
and HTTP/1 cannot start a request until the previous body has drained.
Upgrades are never pooled since they take the connection over, and a
failed send evicts that endpoint. Pruning is bounded per key, with the
full sweep limited to once per 30s.

Signed-off-by: divesh <dgude@nvidia.com>

* fix(server): address HA gateway review findings (#3449)

- Let a gateway own supervisor sessions without a peer endpoint. Requiring
  one whenever the store is PostgreSQL broke every single-instance
  PostgreSQL deployment, because no sandbox supervisor could connect.
  A cross-replica request to an owner that advertises no endpoint now fails
  immediately naming the cause, instead of retrying until the wait timeout.
- Close a supervisor session on heartbeat only when another replica owns it,
  or after renewals fail for the ownership TTL. A database error no longer
  drops every session heartbeating during an outage.
- Clamp owner record ages at zero so a skewed or corrupt stored timestamp
  cannot produce a negative age.
- Bound the cross-object advisory lock with a lock timeout, so a stuck holder
  fails instead of blocking every mutation in the fleet.
- Refuse to start when a peer endpoint is configured on a multi-replica
  backend but peer authentication is unavailable, and warn when a
  multi-replica backend has no peer endpoint at all.
- Reject a plaintext peer endpoint when the gateway serves TLS.
- Skip the sandbox watch poller on single-replica backends, where the local
  update bus already sees every write.
- Rate-limit the peer owner cache sweep so an insert no longer scans the
  whole map under the lock.
- Retry GET and HEAD on a pooled upstream the sandbox closed, instead of
  returning 502, and drop an emptied endpoint from the pool right away.
- Document the gateway peer environment variables and the post-rollout
  ownership skew operators should expect.

Signed-off-by: divesh <dgude@nvidia.com>

* fix(server): harden HA supervisor ownership

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: divesh <dgude@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: divesh <dgude@nvidia.com>
Co-authored-by: Divesh Chowdary <47188680+FrostGod@users.noreply.github.com>
2026-09-21 21:22:12 +00:00

292 lines
12 KiB
Smarty
Raw Permalink Blame History

# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
{{/*
Gateway pod template shared by the StatefulSet and Deployment workload shapes.
*/}}
{{- define "openshell.gatewayPodTemplate" -}}
metadata:
annotations:
# Roll the gateway workload when the rendered gateway TOML changes - the
# gateway only reads /etc/openshell/gateway.toml at startup, so without
# this annotation a `helm upgrade` that only mutates the ConfigMap would
# leave pods running with stale config.
checksum/gateway-config: {{ include (print $.Template.BasePath "/gateway-config.yaml") . | sha256sum }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "openshell.labels" . | nindent 4 }}
{{- with .Values.podLabels }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
terminationGracePeriodSeconds: {{ .Values.podLifecycle.terminationGracePeriodSeconds }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 4 }}
{{- end }}
serviceAccountName: {{ include "openshell.serviceAccountName" . }}
{{- if .Values.server.hostGatewayIP }}
hostAliases:
- ip: {{ .Values.server.hostGatewayIP | quote }}
hostnames:
- host.docker.internal
- host.openshell.internal
{{- end }}
{{- with .Values.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 4 }}
{{- end }}
containers:
- name: openshell-gateway
securityContext:
{{- toYaml .Values.securityContext | nindent 8 }}
image: {{ include "openshell.image" . | quote }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
args:
- --config
- /etc/openshell/gateway.toml
{{- if not .Values.server.externalDbSecret }}
- --db-url
- {{ .Values.server.dbUrl | quote }}
{{- end }}
env:
- name: OPENSHELL_REPLICA_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: OPENSHELL_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: OPENSHELL_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
{{- if eq (include "openshell.workloadKind" .) "deployment" }}
- name: OPENSHELL_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: OPENSHELL_PEER_ENDPOINT
value: {{ printf "%s://$(OPENSHELL_POD_IP):%d" (ternary "http" "https" (default false .Values.server.disableTls)) (int .Values.service.port) | quote }}
{{- end }}
- name: OPENSHELL_SERVICE_ACCOUNT_NAME
value: {{ include "openshell.serviceAccountName" . | quote }}
- name: OPENSHELL_PEER_SERVICE_NAME
value: {{ include "openshell.peerServiceName" . | quote }}
- name: OPENSHELL_PEER_TOKEN_AUDIENCE
value: "openshell-gateway-peer"
- name: OPENSHELL_PEER_SERVICE_ACCOUNT_TOKEN_FILE
value: /var/run/secrets/openshell-peer/token
- name: OPENSHELL_PEER_POD_LABELS
value: {{ printf "app.kubernetes.io/name=%s,app.kubernetes.io/instance=%s" (include "openshell.name" .) .Release.Name | quote }}
{{- if not .Values.server.disableTls }}
- name: OPENSHELL_PEER_TLS_SERVER_NAME
value: {{ printf "%s.%s.svc.cluster.local" (include "openshell.fullname" .) .Release.Namespace | quote }}
{{- if or .Values.pkiInitJob.enabled .Values.certManager.enabled }}
- name: OPENSHELL_PEER_TLS_CA_FILE
value: /etc/openshell-tls/server/ca.crt
{{- end }}
{{- if eq (include "openshell.gatewayClientCaEnabled" .) "true" }}
- name: OPENSHELL_PEER_TLS_CERT_FILE
value: /etc/openshell-tls/peer-client/tls.crt
- name: OPENSHELL_PEER_TLS_KEY_FILE
value: /etc/openshell-tls/peer-client/tls.key
{{- end }}
{{- end }}
{{- if not (or .Values.server.credentialDrivers.kubernetesSecrets.enabled .Values.server.credentialDrivers.vault.enabled) }}
- name: {{ include "openshell.credentialStorageKeyEncryptionKeyEnvName" . }}
valueFrom:
secretKeyRef:
name: {{ include "openshell.credentialStorageKeyEncryptionKeySecretName" . }}
key: {{ include "openshell.credentialStorageKeyEncryptionKeySecretKey" . }}
{{- end }}
{{- if .Values.server.externalDbSecret }}
- name: OPENSHELL_DB_URL
valueFrom:
secretKeyRef:
name: {{ .Values.server.externalDbSecret }}
key: uri
{{- end }}
# Most gateway settings live in the ConfigMap-backed TOML file
# mounted at /etc/openshell/gateway.toml. Secret-bearing settings use
# env vars that the TOML references by name. Some process-level
# settings consumed by libraries outside gateway code also remain here.
{{- if and .Values.server.oidc.issuer .Values.server.oidc.caConfigMapName }}
# OIDC issuer custom-CA: rustls/reqwest read SSL_CERT_FILE for
# outbound TLS verification. This is a process-level env var
# consumed by the TLS stack itself, not by gateway code, so it
# cannot be represented in the gateway TOML schema.
- name: SSL_CERT_FILE
value: /etc/openshell-tls/oidc-ca/ca.crt
{{- end }}
- name: OPENSHELL_TELEMETRY_ENABLED
value: {{ .Values.server.telemetryEnabled | quote }}
{{- if .Values.server.providerTokenGrants.spiffe.enabled }}
- name: OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET
value: {{ .Values.server.providerTokenGrants.spiffe.workloadApiSocketPath | quote }}
{{- end }}
volumeMounts:
{{- if eq (include "openshell.workloadKind" .) "statefulset" }}
- name: openshell-data
mountPath: /var/openshell
{{- end }}
# ConfigMap directory mounts expose keys through atomic-writer symlinks,
# while the gateway intentionally rejects symlinked configuration.
# The checksum annotation above rolls pods when this subPath changes.
- name: gateway-config
mountPath: /etc/openshell/gateway.toml
subPath: gateway.toml
readOnly: true
- name: sandbox-jwt
mountPath: /etc/openshell-jwt
readOnly: true
- name: gateway-peer-token
mountPath: /var/run/secrets/openshell-peer
readOnly: true
{{- if not .Values.server.disableTls }}
- name: tls-cert
mountPath: /etc/openshell-tls/server
readOnly: true
{{- if .Values.certManager.serverIssuerRef.name }}
- name: tls-external-cert
mountPath: /etc/openshell-tls/server-external
readOnly: true
{{- end }}
{{- if eq (include "openshell.gatewayClientCaEnabled" .) "true" }}
- name: peer-client-tls
mountPath: /etc/openshell-tls/peer-client
readOnly: true
- name: tls-client-ca
mountPath: /etc/openshell-tls/client-ca
readOnly: true
{{- end }}
{{- end }}
{{- if and .Values.server.oidc.issuer .Values.server.oidc.caConfigMapName }}
- name: oidc-ca
mountPath: /etc/openshell-tls/oidc-ca
readOnly: true
{{- end }}
{{- if and .Values.server.credentialDrivers.vault.enabled .Values.server.credentialDrivers.vault.caConfigMapName }}
- name: vault-ca
mountPath: /etc/openshell-tls/vault-ca
readOnly: true
{{- end }}
{{- if .Values.server.providerTokenGrants.spiffe.enabled }}
- name: spiffe-workload-api
mountPath: {{ dir .Values.server.providerTokenGrants.spiffe.workloadApiSocketPath | quote }}
readOnly: true
{{- end }}
ports:
- name: grpc
containerPort: {{ .Values.service.port }}
protocol: TCP
- name: health
containerPort: {{ .Values.service.healthPort }}
protocol: TCP
{{- if .Values.service.metricsPort }}
- name: metrics
containerPort: {{ .Values.service.metricsPort }}
protocol: TCP
{{- end }}
startupProbe:
httpGet:
path: /healthz
port: health
periodSeconds: {{ .Values.probes.startup.periodSeconds }}
timeoutSeconds: {{ .Values.probes.startup.timeoutSeconds }}
failureThreshold: {{ .Values.probes.startup.failureThreshold }}
livenessProbe:
httpGet:
path: /healthz
port: health
initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds }}
periodSeconds: {{ .Values.probes.liveness.periodSeconds }}
timeoutSeconds: {{ .Values.probes.liveness.timeoutSeconds }}
failureThreshold: {{ .Values.probes.liveness.failureThreshold }}
readinessProbe:
httpGet:
path: /readyz
port: health
initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds }}
periodSeconds: {{ .Values.probes.readiness.periodSeconds }}
timeoutSeconds: {{ .Values.probes.readiness.timeoutSeconds }}
failureThreshold: {{ .Values.probes.readiness.failureThreshold }}
resources:
{{- toYaml .Values.resources | nindent 8 }}
volumes:
- name: gateway-config
configMap:
name: {{ include "openshell.fullname" . }}-config
- name: sandbox-jwt
secret:
secretName: {{ include "openshell.sandboxJwtSecretName" . }}
defaultMode: {{ .Values.server.sandboxJwt.secretDefaultMode | default 0400 }}
- name: gateway-peer-token
projected:
defaultMode: 0400
sources:
- serviceAccountToken:
path: token
audience: openshell-gateway-peer
expirationSeconds: 3600
{{- if not .Values.server.disableTls }}
- name: tls-cert
secret:
secretName: {{ .Values.server.tls.certSecretName }}
{{- if .Values.certManager.serverIssuerRef.name }}
- name: tls-external-cert
secret:
secretName: {{ include "openshell.fullname" . }}-server-external-tls
{{- end }}
{{- if eq (include "openshell.gatewayClientCaEnabled" .) "true" }}
- name: peer-client-tls
secret:
secretName: {{ .Values.server.tls.clientTlsSecretName }}
- name: tls-client-ca
secret:
{{- if or (and .Values.pkiInitJob.enabled (not .Values.certManager.enabled)) (and .Values.certManager.enabled .Values.certManager.clientCaFromServerTlsSecret) }}
secretName: {{ .Values.server.tls.certSecretName }}
items:
- key: ca.crt
path: ca.crt
{{- else }}
secretName: {{ .Values.server.tls.clientCaSecretName }}
{{- end }}
{{- end }}
{{- end }}
{{- if and .Values.server.oidc.issuer .Values.server.oidc.caConfigMapName }}
- name: oidc-ca
configMap:
name: {{ .Values.server.oidc.caConfigMapName }}
{{- end }}
{{- if and .Values.server.credentialDrivers.vault.enabled .Values.server.credentialDrivers.vault.caConfigMapName }}
- name: vault-ca
configMap:
name: {{ .Values.server.credentialDrivers.vault.caConfigMapName }}
items:
- key: ca.crt
path: ca.crt
{{- end }}
{{- if .Values.server.providerTokenGrants.spiffe.enabled }}
- name: spiffe-workload-api
csi:
driver: csi.spiffe.io
readOnly: true
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
Reference in New Issue View Git Blame Copy Permalink