Kubernetes Node Enforcer Option

The node-enforcer branch adds a fourth Kubernetes isolation shape: keep the workload supervisor and proxy in the sandbox pod, but move privileged pod-network-namespace enforcement to an operator-controlled DaemonSet on each node.

branch: node-enforcer-poc/tmutch
OpenShell Kubernetes node-enforcer option Diagram of the node-enforcer topology and comparison to supervisor-netns, soft-proxy, split-pod, and isolation backend options. Option 4: Node-Enforcer Topology workload supervisor remains in the sandbox pod; privileged netns/nft enforcement moves to a node-local DaemonSet Gateway creates Sandbox CR injects role + enforcement mode Kubernetes Driver supervisor_role=workload network=external-enforcer Kubernetes node node-enforcer DaemonSet same supervisor image, role=enforcer hostNetwork + hostPID + privileged listens on host-network endpoint Namespace lookup find pod netns by pod IP in /proc Install nft table setns(CLONE_NEWNET) + nft -f Sandbox pod network namespace Workload supervisor policy loading, proxy, SSH, logs, agent lifecycle registers with node enforcer before spawning agent Local proxy OPA + L7 + creds Agent child sandbox user openshell_external_enforcer table External services allowed only through proxy Gateway session config, relay, logs create sandbox inject env HTTP registration pod IP + sandbox ID install pod-netns nft rules HTTP_PROXY UID 0 proxy egress non-root direct socket rejected gateway callback What moved out of the workload pod? + netns discovery and nftables installation are node-side infrastructure duties + workload supervisor keeps policy/proxy/credentials close to the agent Prototype hardening required: authenticate registrations, verify pod ownership, reconcile cleanup/stale state, avoid per-release DaemonSet port conflicts. Comparison Against Other Kubernetes Options The options differ mostly by where the hard network boundary lives and whether the workload pod needs elevated privileges. Option Hard egress boundary Privileged component What remains local to agent pod Main tradeoff Supervisor-netns inner netns + veth + nft sandbox workload supervisor all policy, proxy, fs, process, netns strong but requires elevated sandbox pod Soft-proxy none for direct sockets none for network setup proxy, policy, lifecycle, logs deployable but bypass is not kernel-blocked Split-pod CNI NetworkPolicy trusted supervisor pod / platform agent only; optional sideload SSH/Landlock loses trusted per-binary identity cross-pod Node-enforcer pod netns nft table installed by DaemonSet node enforcer only proxy, OPA, creds, SSH, agent lifecycle keeps proxy local; moves network privilege out Isolation Backend selected backend: netns, CNI, node, VM backend-specific supervisor as policy authority needs verifiable handoff and fail-closed contract How Node-Enforcer Fits the Isolation Backend Abstraction Supervisor contract policy authority, proxy, audit register before agent exec Backend handle pod IP + sandbox ID node-local enforcer endpoint Backend mechanism hostPID /proc scan setns + nft ruleset Required invariant no agent execution before boundary is installed/verified Open questions authn/authz cleanup + ownership checks Sources: node-enforcer-poc/tmutch branch files architecture/plans/node-enforcer-topology-findings.md, crates/openshell-sandbox/src/enforcer.rs, crates/openshell-sandbox/src/lib.rs, crates/openshell-driver-kubernetes/src/driver.rs, deploy/helm/openshell/templates/node-enforcer.yaml, docs/reference/sandbox-compute-drivers.mdx, plus proposal context from #981 and #1737.

What It Preserves

The workload supervisor still owns proxy, OPA, credential resolution, inference routing, SSH, logs, settings reloads, and agent lifecycle inside the sandbox pod.

What It Moves

The node enforcer owns pod network namespace discovery and coarse nftables egress enforcement, moving that privilege out of the sandbox workload pod.

What It Does Not Do

It does not split the agent into a separate pod and does not use Kubernetes NetworkPolicy as the hard boundary. Dynamic endpoint and L7 policy still live in the proxy.

Hardening Work

The POC needs authenticated registration, pod ownership verification, cleanup/reconciliation for stale namespaces, and shared-cluster deployment rules.