Kubernetes Isolation Tradeoffs

Current OpenShell keeps the supervisor and agent in one Kubernetes sandbox pod. The split-pod proposals move untrusted agent execution into a separate pod and use platform egress controls, while the Isolation Backend proposal turns the boundary machinery into an explicit contract.

current vs proposed states
OpenShell Kubernetes isolation tradeoffs Comparison of current in-pod isolation, proposed split-pod isolation, and an isolation backend abstraction for OpenShell Kubernetes sandboxes. A. Current Kubernetes State single sandbox pod, supervisor builds the inner boundary B. Split-Pod Proposal trusted supervisor pod, untrusted agent pod, CNI egress boundary C. Isolation Backend Interface common supervisor contract, backend-specific boundary realization Sandbox Pod openshell-sandbox supervisor root prelude, proxy, OPA, credentials SSH relay, logs, process spawn requires elevated pod capabilities Inner network namespace veth route forces egress to proxy nftables rejects direct TCP/UDP bypass bypass monitor records attempts Restricted agent process sandbox user, Landlock, seccomp binary identity via /proc spawn enter namespace Current tradeoffs + Strong in-pod egress forcing independent of CNI + Per-binary network policy input still available - Supervisor privilege sits in same pod as agent - Blocks restricted-v2 / hardened managed clusters - Agent and supervisor logs share the workload stream Supervisor Pod Proxy + OPA L7 policy gate Credentials never in agent pod Agent Pod Untrusted image non-root, no caps gVisor or Kata Sideload binary optional SSH/Landlock Kubernetes NetworkPolicy allow: agent pod to supervisor:3128 allow: DNS when configured deny: all other direct egress at CNI layer RuntimeClass gVisor userspace kernel or Kata VM boundary HTTP_PROXY non-cooperative traffic Split-pod tradeoffs + Agent pod can run non-root with zero capabilities + gVisor/Kata reduces host-kernel blast radius + Separate log streams for audit and agent output - Per-binary destination restrictions are lost cross-pod - Enforcement now depends on CNI NetworkPolicy support Supervisor stays policy authority proxy, OPA, credentials, audit, inference routing drives one runtime contract instead of inline topology code Isolation Backend contract realize boundary, return verifiable handle fail closed if boundary is not ready operator-selected, not workload-selected In-pod netns + nft status quo path Delegated split pod CNI boundary VM / Node microVM node agent Two invariants no unguarded workload egress before verification no workload execution before boundary ready Abstraction tradeoffs + Separates policy authority from boundary machinery + Lets split-pod, VM, or node-agent backends share a contract + Enables in-pod refactor first, delegated later - Requires verifiable handoff and fail-closed semantics - Scope is unsettled: network only vs fs/syscall/process too Decision Map The diagrams point to different places for the hard boundary. That changes who must be trusted and which controls are enforceable. Hard egress boundary current: netns + nftables inside pod split-pod: CNI NetworkPolicy outside agent pod backend: selected per environment, verified by contract Kernel isolation current: runc + seccomp + Landlock split-pod: gVisor userspace kernel or Kata VM backend: can represent in-pod, VM, or delegated runtime L7, credentials, inference current: local supervisor proxy split-pod: supervisor pod proxy backend: remains with supervisor policy authority Known loss / risk current: privileged setup near untrusted workload split-pod: per-binary policy and deny logs become weaker backend: verification and versioning become first-class Best fit local dev, k3d/k3s, environments allowing capabilities enterprise clusters, OpenShift/Gardener-style restrictions long-term architecture that lets both shapes coexist Sources: current code/docs; #899 restricted SCC; #981 split supervisor/agent pod; #1305 proxy/runtime decomposition; #1650 process/network split PR; #1703 external compute driver PR; #1737 Isolation Backend interface.

Current State

The Kubernetes driver creates one Sandbox CR whose pod runs the agent container with supervisor sideloading and elevated capabilities so the supervisor can create the inner network namespace.

Split Pod

The proposal moves the proxy and policy authority into a trusted supervisor pod. The agent pod becomes untrusted, non-root, no-capabilities, and optionally gVisor or Kata isolated.

NetworkPolicy

Today Helm renders an ingress-only sandbox SSH NetworkPolicy. The split-pod proposal adds egress NetworkPolicy so agent pods can reach only the supervisor proxy and DNS.

Backend Interface

The interface proposal keeps the supervisor as policy authority while letting in-pod, split-pod, VM, or node-agent backends realize the isolation boundary with explicit verification.