mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-28 07:13:04 +08:00
John Myers
29fffe2e8e
## Summary - Add HTTP/2 keep-alive settings (`http2_keep_alive_interval`, `keep_alive_while_idle`, `keep_alive_timeout`, `connect_timeout`) to the sandbox gRPC client channel, matching what the CLI already uses - Wrap the `PushSandboxLogs` client-streaming RPC in an outer reconnect loop with exponential backoff (1s → 30s) so that server restarts, network blips, or idle connection reaping no longer permanently kill sandbox log delivery - Buffer up to 200 log lines during reconnect attempts so they are delivered after re-establishing the connection - Detect stream death proactively via an `rpc_done` channel instead of waiting for the next batch send to fail ## Problem The `PushSandboxLogs` RPC is a long-lived client-streaming call that stays open for the sandbox's entire lifetime. Without HTTP/2 keep-alive, Kubernetes service proxies reap idle TCP connections, causing a silent `"transport error"`. The push loop would then exit permanently with no retry — all subsequent sandbox-source logs were lost. Unary RPCs like `GetSandboxPolicy` were unaffected because they complete in milliseconds and never sit idle. ## Test Plan - Deployed to local cluster, verified `[sandbox]` logs appear in `nav sandbox logs` output - Killed `navigator-0` while sandboxes were running; sandbox stderr showed `log push stream lost, reconnecting...` followed by successful reconnection - Both `[gateway]` and `[sandbox]` log sources confirmed present after server recovery