trending/OpenShell
1
0
Fork 0
mirror of https://github.com/NVIDIA/OpenShell.git synced 2026-10-20 06:00:06 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
pull-request/3634
OpenShell/deploy/helm/openshell/skaffold.yaml
T
Taylor MutchDrew NewberrydiveshDivesh Chowdary
 0a770d9173 feat(kubernetes): support HA gateway rebalancing (#1868) 
* feat(kubernetes): support HA gateway rebalancing

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* perf(server): cache peer connections, tokens, and owner lookups

Every forwarded relay rebuilt its setup from scratch: an owner lookup, a
blocking read of the peer token, a TLS connect to the owning replica, and
a TokenReview plus Pod GET on the receiving side. Sandbox service routing
does this per HTTP request, so the apiserver calls scaled with traffic.

Cache all of it on ServerState:

- peer channels pooled per endpoint, so relays multiplex over one
  connection instead of redialing
- peer tokens keyed by SHA-256, expiring at min(ttl, token exp) so a hit
  cannot accept an expired token
- owner records for 3s against a 45s ownership TTL, still freshness
  checked before use

Entries are evicted when a relay fails. Also raise HTTP/2
max_concurrent_streams to 1024, since pooling funnels every relay between
two replicas onto one connection and hyper's default of 200 sits below
the 256 pending-relay budget.

Signed-off-by: divesh <dgude@nvidia.com>

* perf(server): pool upstream connections for sandbox services

Each HTTP request to a sandbox service opened its own supervisor relay,
paying a new TCP connection and HTTP/1 handshake every time. Worse, it
counted against the 32 in-flight relay cap, so a service handling more
than 32 concurrent requests failed outright.

Pool idle upstreams per endpoint and port, up to 8 each for 15s. Reuse is
safe because the pool only returns a connection hyper reports as ready,
and HTTP/1 cannot start a request until the previous body has drained.
Upgrades are never pooled since they take the connection over, and a
failed send evicts that endpoint. Pruning is bounded per key, with the
full sweep limited to once per 30s.

Signed-off-by: divesh <dgude@nvidia.com>

* fix(server): address HA gateway review findings (#3449)

- Let a gateway own supervisor sessions without a peer endpoint. Requiring
  one whenever the store is PostgreSQL broke every single-instance
  PostgreSQL deployment, because no sandbox supervisor could connect.
  A cross-replica request to an owner that advertises no endpoint now fails
  immediately naming the cause, instead of retrying until the wait timeout.
- Close a supervisor session on heartbeat only when another replica owns it,
  or after renewals fail for the ownership TTL. A database error no longer
  drops every session heartbeating during an outage.
- Clamp owner record ages at zero so a skewed or corrupt stored timestamp
  cannot produce a negative age.
- Bound the cross-object advisory lock with a lock timeout, so a stuck holder
  fails instead of blocking every mutation in the fleet.
- Refuse to start when a peer endpoint is configured on a multi-replica
  backend but peer authentication is unavailable, and warn when a
  multi-replica backend has no peer endpoint at all.
- Reject a plaintext peer endpoint when the gateway serves TLS.
- Skip the sandbox watch poller on single-replica backends, where the local
  update bus already sees every write.
- Rate-limit the peer owner cache sweep so an insert no longer scans the
  whole map under the lock.
- Retry GET and HEAD on a pooled upstream the sandbox closed, instead of
  returning 502, and drop an emptied endpoint from the pool right away.
- Document the gateway peer environment variables and the post-rollout
  ownership skew operators should expect.

Signed-off-by: divesh <dgude@nvidia.com>

* fix(server): harden HA supervisor ownership

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: divesh <dgude@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: divesh <dgude@nvidia.com>
Co-authored-by: Divesh Chowdary <47188680+FrostGod@users.noreply.github.com>
2026-09-21 21:22:12 +00:00

178 lines
6.7 KiB
YAML
Raw Permalink Blame History

# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Local dev: builds gateway, sandbox, and supervisor images via tasks/scripts/docker-build-image.sh,
# which first stages Rust binaries natively on the host (using cargo / cargo-zigbuild
# when cross-compiling) and then builds the image from the prebuilt binary. This
# mirrors CI and is faster than compiling inside Docker on every rebuild because
# the host's cargo target cache and sccache are reused across iterations.
#
# Run from repo root:
# mise run helm:skaffold:dev
# mise run helm:skaffold:run
#
# See https://skaffold.dev/docs/deployers/helm/ (setValueTemplates, IMAGE_* fields).
apiVersion: skaffold/v4beta14
kind: Config
metadata:
name: openshell
build:
local:
push: false
tagPolicy:
gitCommit: {}
artifacts:
- image: openshell/gateway
context: ../../..
custom:
buildCommand: |
CONTAINER_ENGINE_TARGET=local-k8s-cluster \
IMAGE_NAME="${IMAGE%:*}" \
IMAGE_TAG="${IMAGE##*:}" \
tasks/scripts/docker-build-image.sh gateway
dependencies:
paths:
- Cargo.toml
- Cargo.lock
- crates/**
- proto/**
- deploy/docker/Dockerfile.gateway
- tasks/scripts/docker-build-image.sh
- tasks/scripts/stage-prebuilt-binaries.sh
- image: openshell/supervisor
context: ../../..
custom:
buildCommand: |
CONTAINER_ENGINE_TARGET=local-k8s-cluster \
IMAGE_NAME="${IMAGE%:*}" \
IMAGE_TAG="${IMAGE##*:}" \
tasks/scripts/docker-build-image.sh supervisor
dependencies:
paths:
- Cargo.toml
- Cargo.lock
- crates/**
- proto/**
- deploy/docker/Dockerfile.supervisor
- tasks/scripts/docker-build-image.sh
- tasks/scripts/stage-prebuilt-binaries.sh
- image: openshell/sandbox
context: ../../..
custom:
buildCommand: |
CONTAINER_ENGINE_TARGET=local-k8s-cluster \
IMAGE_NAME="${IMAGE%:*}" \
IMAGE_TAG="${IMAGE##*:}" \
tasks/scripts/docker-build-image.sh sandbox
dependencies:
paths:
- Cargo.toml
- Cargo.lock
- crates/**
- proto/**
- deploy/docker/Dockerfile.sandbox
- tasks/scripts/docker-build-image.sh
- tasks/scripts/stage-prebuilt-binaries.sh
deploy:
helm:
releases:
# cert-manager — comment this in and add values-cert-manager.yaml below
# when you want cert-manager to manage TLS while certgen handles JWT.
# Requires cert-manager CRDs to be installed in the cluster first.
#- name: cert-manager
# remoteChart: oci://quay.io/jetstack/charts/cert-manager
# version: v1.20.2
# namespace: cert-manager
# createNamespace: true
# setValues:
# crds.enabled: true
# Envoy Gateway — Kubernetes Gateway API implementation.
# Installs the Gateway API CRDs and the "eg" GatewayClass.
# Required when grpcRoute.enabled is true in the openshell release.
#- name: envoy-gateway
# remoteChart: oci://docker.io/envoyproxy/gateway-helm
# version: v1.7.2
# namespace: envoy-gateway-system
# createNamespace: true
# # wait ensures Gateway API CRDs are registered before the openshell
# # release attempts to create Gateway and HTTPRoute resources.
# wait: true
# SPIRE — installs SPIRE Server, Agent, Controller Manager, CSI Driver,
# and OIDC Discovery Provider using the SPIFFE hardened charts.
# Uncomment both releases and ci/values-spire.yaml below to use
# SPIFFE JWT-SVIDs for dynamic provider token grants.
#- name: spire-crds
# repo: https://spiffe.github.io/helm-charts-hardened/
# remoteChart: spire-crds
# version: 0.5.0
# namespace: spire
# createNamespace: true
# wait: true
#- name: spire
# repo: https://spiffe.github.io/helm-charts-hardened/
# remoteChart: spire
# version: 0.29.0
# namespace: spire
# createNamespace: true
# valuesFiles:
# - ci/values-spire-stack.yaml
# wait: true
- name: openshell
chartPath: .
namespace: openshell
createNamespace: true
valuesFiles:
- values.yaml
- ci/values-skaffold.yaml
# Add ci/values-cert-manager.yaml here (and uncomment the cert-manager
# release above) to switch TLS generation to cert-manager.
#- ci/values-cert-manager.yaml
# To enable OIDC with a local Keycloak instance, run the one-time
# setup task first, then uncomment the line below:
# mise run keycloak:k8s:setup
#- ci/values-keycloak.yaml
# To enable the Gateway API HTTPRoute (requires Envoy Gateway above):
#- ci/values-gateway.yaml
# To enable SPIFFE/SPIRE provider token grants (requires the
# spire-crds and spire releases above):
#- ci/values-spire.yaml
# To test multi-replica external PostgreSQL behavior:
#- ci/values-high-availability.yaml
setValueTemplates:
image.repository: '{{.IMAGE_REPO_openshell_gateway}}'
image.tag: '{{.IMAGE_TAG_openshell_gateway}}'
supervisor.image.repository: '{{.IMAGE_REPO_openshell_supervisor}}'
supervisor.image.tag: '{{.IMAGE_TAG_openshell_supervisor}}'
sandboxRuntime.image.repository: '{{.IMAGE_REPO_openshell_sandbox}}'
sandboxRuntime.image.tag: '{{.IMAGE_TAG_openshell_sandbox}}'
profiles:
# Full HA test path: installs Envoy Gateway and layers both HA replicas and
# Gateway API routing values onto the OpenShell release.
- name: high-availability
patches:
- op: add
path: /deploy/helm/releases/0
value:
name: envoy-gateway
remoteChart: oci://docker.io/envoyproxy/gateway-helm
version: v1.7.2
namespace: envoy-gateway-system
createNamespace: true
wait: true
- op: add
path: /deploy/helm/releases/1/valuesFiles/-
value: ci/values-high-availability.yaml
- op: add
path: /deploy/helm/releases/1/valuesFiles/-
value: ci/values-gateway.yaml
- name: credential-driver-kubernetes-secrets
patches:
- op: add
path: /deploy/helm/releases/0/valuesFiles/-
value: ci/values-credential-driver-kubernetes-secrets.yaml
- name: credential-driver-vault
patches:
- op: add
path: /deploy/helm/releases/0/valuesFiles/-
value: ci/values-credential-driver-vault.yaml
Reference in New Issue View Git Blame Copy Permalink