diff --git a/fern/.docs-snapshots.yml b/fern/.docs-snapshots.yml index 70a4a514c..d1ec012e6 100644 --- a/fern/.docs-snapshots.yml +++ b/fern/.docs-snapshots.yml @@ -1,8 +1,8 @@ snapshots: dev: - source-ref: 9a6148fc988a5e7a20a43805a52fdfaad766f8ae - source-sha: 9a6148fc988a5e7a20a43805a52fdfaad766f8ae - version: 0.1.3.dev112 + source-ref: 834b79a8c2351370e7c6c363423f5be18d440256 + source-sha: 834b79a8c2351370e7c6c363423f5be18d440256 + version: 0.1.3.dev113 latest: source-ref: 6648bd0c290efbc41ba131ee9831ee45cd431f94 source-sha: 6648bd0c290efbc41ba131ee9831ee45cd431f94 diff --git a/fern/pages-dev/kubernetes/high-availability.mdx b/fern/pages-dev/kubernetes/high-availability.mdx index 0153df050..505fe7c6f 100644 --- a/fern/pages-dev/kubernetes/high-availability.mdx +++ b/fern/pages-dev/kubernetes/high-availability.mdx @@ -178,8 +178,9 @@ When a non-owner gateway pod stops, Kubernetes removes it from ready Service endpoints and new requests go to another replica. Existing requests through the stopped pod disconnect and must be retried. -When an owner gateway pod stops, its connected supervisors reconnect through -the gateway Service and publish new ownership. Other replicas re-read ownership +When an owner gateway pod stops gracefully, it redirects its connected +supervisors to the replicas that now own them. After a crash, supervisors +reconnect through the gateway Service. Either way, they publish new ownership. Other replicas re-read ownership while waiting to relay a request. A request fails if no fresh, reachable owner appears before its normal deadline. @@ -287,16 +288,15 @@ With autoscaling enabled, the chart stops setting `spec.replicas` and ignores container request, so those targets need a matching `resources.requests` entry. A `resources.limits` entry also works, because Kubernetes copies a limit into an unset request. The chart rejects an `autoscaling.maxReplicas` -above `1` without `server.externalDbSecret`. Each scale-down disconnects the -removed replica's supervisor sessions, and they reconnect to the remaining -replicas, so the default behavior removes at most one pod every two minutes +above `1` without `server.externalDbSecret`. Each scale-down hands the removed +replica's supervisor sessions to the remaining replicas, so the default behavior removes at most one pod every two minutes after a five-minute stabilization window. Override it with `autoscaling.behavior`, or set `autoscaling.behavior.scaleDown` to `null` to drop the default. New replicas do not take existing sessions. -A scale-down disconnects every supervisor session on the removed pod at once, -and requests through that pod must be retried. To choose when that happens, +A scale-down moves every supervisor session on the removed pod at once, and +requests through that pod must be retried. To choose when that happens, set `autoscaling.behavior.scaleDown.selectPolicy` to `Disabled` so the autoscaler only adds replicas, and remove replicas yourself with `kubectl scale` at a quiet time. @@ -312,9 +312,8 @@ kubectl -n openshell describe hpa openshell Enabling autoscaling on an existing release removes `spec.replicas` from the workload in that upgrade. Kubernetes can reset the workload to one replica -until the HPA scales it back to at least `autoscaling.minReplicas`, which -disconnects supervisor sessions from the other gateway pods; they reconnect to -the remaining replica. Enable autoscaling +until the HPA scales it back to at least `autoscaling.minReplicas`, which moves +supervisor sessions from the other gateway pods to the remaining replica. Enable autoscaling when you install the chart, or during a maintenance window. On an existing release, upgrade with `--reset-then-reuse-values`, which needs Helm 3.14 or later. With an older Helm, save the values with diff --git a/fern/pages-dev/kubernetes/ingress.mdx b/fern/pages-dev/kubernetes/ingress.mdx index 6261ffb04..22a8eebb2 100644 --- a/fern/pages-dev/kubernetes/ingress.mdx +++ b/fern/pages-dev/kubernetes/ingress.mdx @@ -206,6 +206,22 @@ This situation can occur if you set `pkiInitJob.failOnTimeout=false` and cert-ma For OpenShift 4.22+, see [OpenShift](/kubernetes/openshift#end-to-end-tls-using-gateway-api-and-backendtlspolicy-openshift-422) for platform-specific instructions including Gateway and GatewayClass setup. +## Replica Routing + +With more than one gateway replica, a sandbox's SSH, port-forward, and exec connections can land on a replica that does not hold the sandbox's supervisor session, and that replica relays them to the one that does. Enable replica routing to send those connections straight to the owning replica: + +```shell +helm upgrade oci://ghcr.io/nvidia/openshell/helm-chart \ + --reuse-values --namespace \ + --set workload.kind=statefulset \ + --set workload.allowMultiReplicaStatefulSet=true \ + --set grpcRoute.replicaRouting.enabled=true +``` + +The chart renders one Service per replica and a GRPCRoute rule that matches the `x-openshell-replica` request header. The CLI sets that header from the owner the gateway reports and retries without it if the named replica is unavailable. Replica routing requires `workload.kind=statefulset`, `workload.allowMultiReplicaStatefulSet=true`, and at most 15 replicas. + +If the release uses `workload.kind=deployment`, this upgrade replaces the Deployment with a StatefulSet, so every gateway pod restarts at once. The StatefulSet also requests a 1Gi PersistentVolumeClaim per replica, which stays `Pending` on clusters without a default StorageClass. + ## SSH Relay Sandbox SSH uses the gateway endpoint registered with the CLI. No separate Helm SSH host or port values are required.