trending/OpenShell
1
0
Fork 0
mirror of https://github.com/NVIDIA/OpenShell.git synced 2026-10-30 07:33:59 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add diagrams

Browse Source
This commit is contained in:
Taylor Mutch
2026-06-08 22:22:53 -07:00
parent 80a9a76fb9
commit 8eef2e836e
4 changed files with 1449 additions and 102 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SPIKE.md → spike/README.md
+44 -102
View File
@@ -1,7 +1,18 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->
# Node Enforcer Topology Findings
Date: 2026-05-29
## Diagrams
- [OpenShell isolation architecture](openshell-isolation-diagram.html)
- [Kubernetes isolation tradeoffs](kubernetes-isolation-tradeoffs-diagram.html)
- [Kubernetes node-enforcer option](kubernetes-node-enforcer-options-diagram.html)
## Summary
This branch explored how OpenShell can remove `privileged: true` from sandbox
@@ -83,7 +94,8 @@ The current prototype enforcement flow is registration-driven:
3. The node enforcer DaemonSet runs the same supervisor binary with
`supervisor_role = "enforcer"`. In that role it does not start an agent
supervisor; it binds `0.0.0.0:17671` and waits for workload registrations.
supervisor; it binds the configured host-network listener and waits for
workload registrations.
4. The workload supervisor starts normally, loads policy, computes the effective
network enforcement mode, and registers with the node enforcer before
@@ -152,10 +164,9 @@ cluster because enforcement is node-local and pod-local, not gateway-local:
The important deployment constraint is that the node enforcer should be treated
as shared node infrastructure. A node can only have one process binding the
default host-network listener `0.0.0.0:17671`. If multiple gateway releases each
enable their own node-enforcer DaemonSet on the same nodes with the same listen
address, they will compete for the same port and duplicate the privileged
component.
same host-network listener. If multiple gateway releases each enable their own
node-enforcer DaemonSet on the same nodes with the same listen address, they
will compete for the same port and duplicate the privileged component.
For a shared cluster, the expected shape is one node-enforcer DaemonSet per
node pool or cluster security boundary, with all participating gateways pointing
@@ -172,58 +183,44 @@ registering workload is allowed to ask for enforcement on that pod.
## Validation
Validated against the development Kubernetes cluster using the normal OpenShell
gateway path. GKE was used as a real managed-cluster validation target, but it
should not be part of the implementation criteria.
Validated against the local Kubernetes development cluster using the normal
OpenShell gateway path and the node-enforcer Helm overlay.
The unchanged Kubernetes e2e test passed:
The Kubernetes smoke path passed with the node-enforcer overlay:
```shell
OPENSHELL_GATEWAY_ENDPOINT=http://34.171.165.42 \
cargo test --manifest-path e2e/rust/Cargo.toml \
--features e2e-kubernetes \
--test bypass_detection \
-- --nocapture
OPENSHELL_E2E_KUBE_CONTEXT=k3d-openshell-dev-tmutch \
OPENSHELL_E2E_KUBE_EXTRA_VALUES=deploy/helm/openshell/ci/values-node-enforcer.yaml \
OPENSHELL_E2E_KUBE_BUILD_IMAGES=1 \
OPENSHELL_E2E_KUBE_TEST=smoke \
e2e/rust/e2e-kubernetes.sh
```
Result:
```text
test bypass_attempt_is_rejected_fast ... ok
1 passed; 0 failed
```
The full existing Rust e2e suite also passed unchanged against the Kubernetes
gateway once the local e2e invocation used a named temporary gateway config:
The bypass-detection path also passed with the same overlay:
```shell
XDG_CONFIG_HOME=/private/tmp/openshell-gke/e2e-config \
OPENSHELL_GATEWAY=gke-e2e \
OPENSHELL_E2E_DRIVER=kubernetes \
cargo test --manifest-path e2e/rust/Cargo.toml \
--features e2e \
--no-fail-fast \
-- --nocapture
OPENSHELL_E2E_KUBE_CONTEXT=k3d-openshell-dev-tmutch \
OPENSHELL_E2E_KUBE_EXTRA_VALUES=deploy/helm/openshell/ci/values-node-enforcer.yaml \
OPENSHELL_E2E_KUBE_BUILD_IMAGES=1 \
OPENSHELL_E2E_KUBE_TEST=bypass_detection \
e2e/rust/e2e-kubernetes.sh
```
Result:
```text
all e2e test targets passed
```
Important test harness note: the e2e settings test expects the local CLI to be
built with `openshell-core/dev-settings`, which the normal e2e scripts already
do. Direct ad hoc cargo invocations should rebuild `openshell-cli` with that
feature before running the full suite.
Node-enforcer logs showed the expected action:
Manual validation also created a sandbox through the gateway and confirmed the
node enforcer acted on the sandbox network namespace:
```text
Observed sandbox workload registration
Reconciling sandbox network enforcement
Installing sandbox network egress enforcement for pod 10.40.1.35
Sandbox network egress enforcement installed for pod 10.40.1.35 in /proc/309096/ns/net
Installing sandbox network egress enforcement for registered pod
Sandbox network egress enforcement installed for registered pod
```
The manual sandbox check then attempted raw direct TCP from the sandbox user
path and observed a fast rejection instead of a hanging bypass attempt.
```text
direct-connect-exit-code 111
```
## Key Debug Finding
@@ -239,52 +236,13 @@ The fix was to require that the pod IP is present as a local address in the
target namespace:
```text
10.40.1.33
/32 host LOCAL
<pod-ip>
<host-local-route-marker>
```
That distinction selected the sandbox pod namespace instead of the host
namespace and made the unchanged e2e test pass.
## Envoy Gateway Timeout Finding
The managed-cluster validation exposed a second, unrelated deployment issue:
Envoy Gateway was terminating long-lived gRPC streams after about 15 seconds.
The symptom was:
```text
h2 protocol error: error reading a body from connection
```
This affected the existing `WatchSandbox`, `ForwardTcp`, and supervisor
`RelayStream` paths used by `sandbox create -- <cmd>`, upload/create, sync, and
TTY lifecycle tests. The sandbox pods themselves were running and the node
enforcer had installed rules correctly; the failure was at the external gateway
proxy layer.
The fix belongs to the deployment that installs Envoy Gateway, not to the
OpenShell product Helm chart. Envoy Gateway deployments can attach a
`BackendTrafficPolicy` to the OpenShell `GRPCRoute`:
```yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
name: openshell-grpc-streams
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: GRPCRoute
name: openshell
timeout:
http:
requestTimeout: 0s
maxStreamDuration: 0s
```
After that policy was applied, Envoy reported it as accepted and the previous
15-second failures passed without test changes.
## Product Finding
This topology shifts the privileged boundary from the sandbox workload to a
@@ -297,9 +255,6 @@ That is likely the right direction if the goal is:
- OpenShell keeps dynamic network proxy enforcement.
- The supervisor remains responsible for the agent lifecycle.
- Non-Kubernetes deployments can keep the existing combined supervisor mode.
- External ingress and Gateway API controllers are deployment concerns. If they
impose request or stream duration limits, the deployment must configure them
to preserve OpenShell's long-lived gRPC streams.
It is not a claim that the system has no privileged code. On Linux, dynamic
network namespace enforcement needs some trusted component with elevated
@@ -353,13 +308,9 @@ Known risks:
one enforcer DaemonSet per release, which is not safe to enable repeatedly on
the same nodes without coordination.
- The current rules are coarse: UID 0 is allowed, non-root TCP/UDP is rejected.
- IPv6 namespace lookup exists conceptually but has not been validated in the
managed-cluster path.
- IPv6 namespace lookup exists conceptually but has not been validated.
- Observability is useful but should become structured enough for operators to
prove what pod/netns/rules were acted on.
- Edge proxies can break otherwise healthy sandboxes if they impose short
request or stream duration timeouts on gRPC. This is deployment-specific and
should be validated wherever OpenShell is exposed through a proxy.
## Hardening Plan
@@ -414,8 +365,6 @@ The branch added configuration and code paths for:
- Helm `server.sandboxImagePullPolicy`
- Kubernetes driver injection of node and pod IP environment variables
- Supervisor image packaging with `nftables`
- Deployment-owned Envoy Gateway `BackendTrafficPolicy` for long-lived
OpenShell gRPC streams.
The development cluster currently uses:
@@ -423,8 +372,6 @@ The development cluster currently uses:
- `server.networkEnforcementMode: external-enforcer`
- `server.sandboxImagePullPolicy: IfNotPresent`
- `nodeEnforcer.enabled: true`
- An Envoy Gateway `BackendTrafficPolicy` with `requestTimeout = 0s` and
`maxStreamDuration = 0s`.
## Recommendation
@@ -436,8 +383,3 @@ Do not present it as removing privilege entirely. Present it as moving privilege
network enforcement into a narrow, operator-controlled component that can be
authenticated, audited, reconciled, and hardened independently from untrusted
agent workloads.
Keep Envoy Gateway and other ingress-controller tuning out of the product chart
unless OpenShell intentionally takes ownership of that controller. The product
requirement is that long-lived gRPC streams must be supported; the deployment
implementation should provide the controller-specific policy.
spike/kubernetes-isolation-tradeoffs-diagram.html
+434
View File
@@ -0,0 +1,434 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OpenShell Kubernetes Isolation Tradeoffs</title>
<style>
:root {
--bg: #05070b;
--panel: #0f172a;
--line: #334155;
--text: #e5e7eb;
--muted: #9ca3af;
--blue: #38bdf8;
--green: #34d399;
--amber: #fbbf24;
--red: #fb7185;
--violet: #a78bfa;
--orange: #fb923c;
}
* {
box-sizing: border-box;
}
body {
margin: 0;
min-height: 100vh;
color: var(--text);
background:
linear-gradient(140deg, rgba(56, 189, 248, 0.08), transparent 28rem),
linear-gradient(330deg, rgba(251, 191, 36, 0.07), transparent 30rem),
var(--bg);
font-family: "JetBrains Mono", ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
}
main {
width: min(1640px, calc(100vw - 32px));
margin: 0 auto;
padding: 28px 0 36px;
}
.header {
display: flex;
justify-content: space-between;
align-items: end;
gap: 24px;
margin-bottom: 18px;
}
h1 {
margin: 0;
font-size: clamp(24px, 4vw, 40px);
letter-spacing: 0;
font-weight: 780;
}
.subtitle {
margin: 9px 0 0;
color: var(--muted);
font-size: 13px;
line-height: 1.55;
max-width: 980px;
}
.tag {
display: inline-flex;
align-items: center;
gap: 8px;
padding: 8px 10px;
border: 1px solid rgba(148, 163, 184, 0.22);
border-radius: 8px;
color: #cbd5e1;
background: rgba(15, 23, 42, 0.72);
font-size: 11px;
white-space: nowrap;
}
.tag::before {
content: "";
width: 8px;
height: 8px;
border-radius: 999px;
background: var(--amber);
box-shadow: 0 0 16px rgba(251, 191, 36, 0.75);
}
.diagram {
overflow: auto;
border: 1px solid rgba(148, 163, 184, 0.24);
border-radius: 8px;
background: rgba(15, 23, 42, 0.72);
box-shadow: 0 20px 70px rgba(0, 0, 0, 0.34);
}
svg {
display: block;
width: 100%;
min-width: 1220px;
height: auto;
}
.notes {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 14px;
margin-top: 16px;
}
.note {
border: 1px solid rgba(148, 163, 184, 0.22);
border-radius: 8px;
background: rgba(15, 23, 42, 0.76);
padding: 14px;
}
.note h3 {
margin: 0 0 8px;
font-size: 12px;
letter-spacing: 0;
}
.note p {
margin: 0;
color: var(--muted);
font-size: 11px;
line-height: 1.55;
}
footer {
margin-top: 15px;
color: #64748b;
font-size: 10px;
line-height: 1.6;
}
footer a {
color: #93c5fd;
}
@media (max-width: 980px) {
main {
width: calc(100vw - 20px);
padding-top: 18px;
}
.header {
display: block;
}
.tag {
margin-top: 12px;
}
.notes {
grid-template-columns: 1fr;
}
}
</style>
</head>
<body>
<main>
<header class="header">
<div>
<h1>Kubernetes Isolation Tradeoffs</h1>
<p class="subtitle">
Current OpenShell keeps the supervisor and agent in one Kubernetes sandbox pod. The split-pod proposals move untrusted agent execution into a separate pod and use platform egress controls, while the Isolation Backend proposal turns the boundary machinery into an explicit contract.
</p>
</div>
<div class="tag">current vs proposed states</div>
</header>
<section class="diagram" aria-label="OpenShell Kubernetes isolation tradeoff diagram">
<svg viewBox="0 0 1600 1380" role="img" aria-labelledby="diagram-title diagram-desc">
<title id="diagram-title">OpenShell Kubernetes isolation tradeoffs</title>
<desc id="diagram-desc">Comparison of current in-pod isolation, proposed split-pod isolation, and an isolation backend abstraction for OpenShell Kubernetes sandboxes.</desc>
<defs>
<pattern id="grid" width="36" height="36" patternUnits="userSpaceOnUse">
<path d="M 36 0 L 0 0 0 36" fill="none" stroke="#1f2937" stroke-width="0.5"/>
</pattern>
<marker id="arrow-blue" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#38bdf8"/>
</marker>
<marker id="arrow-green" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#34d399"/>
</marker>
<marker id="arrow-amber" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#fbbf24"/>
</marker>
<marker id="arrow-red" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#fb7185"/>
</marker>
<marker id="arrow-violet" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#a78bfa"/>
</marker>
<style>
.title { fill: #f8fafc; font-size: 17px; font-weight: 780; letter-spacing: 0; }
.subtitle { fill: #94a3b8; font-size: 9px; letter-spacing: 0; }
.label { fill: #cbd5e1; font-size: 9px; letter-spacing: 0; }
.tiny { fill: #94a3b8; font-size: 7px; letter-spacing: 0; }
.box-title { fill: #f8fafc; font-size: 11px; font-weight: 750; letter-spacing: 0; }
.box-sub { fill: #cbd5e1; font-size: 8px; letter-spacing: 0; }
.warn { fill: #fbbf24; font-size: 8px; letter-spacing: 0; }
.loss { fill: #fb7185; font-size: 8px; letter-spacing: 0; }
.gain { fill: #34d399; font-size: 8px; letter-spacing: 0; }
.flow { fill: none; stroke-width: 1.6; stroke-linecap: round; stroke-linejoin: round; }
.dash { fill: none; stroke-width: 1.4; stroke-dasharray: 5 5; stroke-linecap: round; stroke-linejoin: round; }
.thin { fill: none; stroke-width: 1.1; stroke-linecap: round; stroke-linejoin: round; }
</style>
</defs>
<rect width="1600" height="1380" fill="#05070b"/>
<rect width="1600" height="1380" fill="url(#grid)" opacity="0.9"/>
<!-- Column panels -->
<rect x="40" y="42" width="480" height="865" rx="12" fill="rgba(30, 41, 59, 0.50)" stroke="#64748b" stroke-width="1.3"/>
<text x="66" y="78" class="title">A. Current Kubernetes State</text>
<text x="66" y="99" class="subtitle">single sandbox pod, supervisor builds the inner boundary</text>
<rect x="560" y="42" width="480" height="865" rx="12" fill="rgba(6, 78, 59, 0.18)" stroke="#34d399" stroke-width="1.3"/>
<text x="586" y="78" class="title">B. Split-Pod Proposal</text>
<text x="586" y="99" class="subtitle">trusted supervisor pod, untrusted agent pod, CNI egress boundary</text>
<rect x="1080" y="42" width="480" height="865" rx="12" fill="rgba(76, 29, 149, 0.18)" stroke="#a78bfa" stroke-width="1.3"/>
<text x="1106" y="78" class="title">C. Isolation Backend Interface</text>
<text x="1106" y="99" class="subtitle">common supervisor contract, backend-specific boundary realization</text>
<!-- Current column -->
<rect x="86" y="128" width="388" height="520" rx="12" fill="rgba(15, 23, 42, 0.86)" stroke="#94a3b8" stroke-width="1.2" stroke-dasharray="8 4"/>
<text x="108" y="156" class="label">Sandbox Pod</text>
<rect x="116" y="184" width="328" height="106" rx="7" fill="#0f172a" stroke="#34d399" stroke-width="1.4"/>
<text x="142" y="216" class="box-title">openshell-sandbox supervisor</text>
<text x="142" y="236" class="box-sub">root prelude, proxy, OPA, credentials</text>
<text x="142" y="253" class="box-sub">SSH relay, logs, process spawn</text>
<text x="142" y="271" class="warn">requires elevated pod capabilities</text>
<rect x="116" y="342" width="328" height="120" rx="7" fill="#0f172a" stroke="#38bdf8" stroke-width="1.4"/>
<text x="142" y="374" class="box-title">Inner network namespace</text>
<text x="142" y="394" class="box-sub">veth route forces egress to proxy</text>
<text x="142" y="411" class="box-sub">nftables rejects direct TCP/UDP bypass</text>
<text x="142" y="429" class="box-sub">bypass monitor records attempts</text>
<rect x="154" y="514" width="252" height="76" rx="7" fill="#0f172a" stroke="#fb7185" stroke-width="1.4"/>
<text x="180" y="544" class="box-title">Restricted agent process</text>
<text x="180" y="564" class="box-sub">sandbox user, Landlock, seccomp</text>
<text x="180" y="580" class="box-sub">binary identity via /proc</text>
<path class="flow" d="M280 290 L280 334" stroke="#34d399" marker-end="url(#arrow-green)"/>
<text x="292" y="318" class="label">spawn</text>
<path class="flow" d="M280 462 L280 506" stroke="#38bdf8" marker-end="url(#arrow-blue)"/>
<text x="292" y="489" class="label">enter namespace</text>
<rect x="88" y="694" width="386" height="170" rx="8" fill="rgba(2, 6, 23, 0.58)" stroke="#475569"/>
<text x="112" y="724" class="box-title">Current tradeoffs</text>
<text x="112" y="750" class="gain">+ Strong in-pod egress forcing independent of CNI</text>
<text x="112" y="772" class="gain">+ Per-binary network policy input still available</text>
<text x="112" y="794" class="loss">- Supervisor privilege sits in same pod as agent</text>
<text x="112" y="816" class="loss">- Blocks restricted-v2 / hardened managed clusters</text>
<text x="112" y="838" class="loss">- Agent and supervisor logs share the workload stream</text>
<!-- Split-pod column -->
<rect x="598" y="132" width="178" height="228" rx="12" fill="rgba(15, 23, 42, 0.86)" stroke="#34d399" stroke-width="1.2" stroke-dasharray="8 4"/>
<text x="624" y="160" class="label">Supervisor Pod</text>
<rect x="620" y="188" width="134" height="66" rx="7" fill="#0f172a" stroke="#34d399" stroke-width="1.4"/>
<text x="640" y="217" class="box-title">Proxy + OPA</text>
<text x="640" y="236" class="box-sub">L7 policy gate</text>
<rect x="620" y="276" width="134" height="54" rx="7" fill="#0f172a" stroke="#a78bfa" stroke-width="1.4"/>
<text x="640" y="307" class="box-title">Credentials</text>
<text x="640" y="323" class="box-sub">never in agent pod</text>
<rect x="820" y="132" width="178" height="228" rx="12" fill="rgba(15, 23, 42, 0.86)" stroke="#fb7185" stroke-width="1.2" stroke-dasharray="8 4"/>
<text x="844" y="160" class="label">Agent Pod</text>
<rect x="842" y="188" width="134" height="70" rx="7" fill="#0f172a" stroke="#fb7185" stroke-width="1.4"/>
<text x="862" y="217" class="box-title">Untrusted image</text>
<text x="862" y="236" class="box-sub">non-root, no caps</text>
<text x="862" y="251" class="box-sub">gVisor or Kata</text>
<rect x="842" y="278" width="134" height="52" rx="7" fill="#0f172a" stroke="#fbbf24" stroke-width="1.4"/>
<text x="858" y="308" class="box-title">Sideload binary</text>
<text x="858" y="323" class="box-sub">optional SSH/Landlock</text>
<rect x="635" y="420" width="320" height="95" rx="8" fill="#0f172a" stroke="#38bdf8" stroke-width="1.4"/>
<text x="662" y="452" class="box-title">Kubernetes NetworkPolicy</text>
<text x="662" y="472" class="box-sub">allow: agent pod to supervisor:3128</text>
<text x="662" y="489" class="box-sub">allow: DNS when configured</text>
<text x="662" y="506" class="box-sub">deny: all other direct egress at CNI layer</text>
<rect x="635" y="558" width="320" height="72" rx="8" fill="#0f172a" stroke="#fb923c" stroke-width="1.4"/>
<text x="662" y="588" class="box-title">RuntimeClass</text>
<text x="662" y="608" class="box-sub">gVisor userspace kernel or Kata VM boundary</text>
<path class="flow" d="M842 224 C800 224 802 224 762 224" stroke="#38bdf8" marker-end="url(#arrow-blue)"/>
<text x="774" y="211" class="label">HTTP_PROXY</text>
<path class="dash" d="M909 258 L909 412" stroke="#38bdf8" marker-end="url(#arrow-blue)"/>
<text x="922" y="382" class="label">non-cooperative traffic</text>
<path class="flow" d="M795 515 L795 550" stroke="#fb923c" marker-end="url(#arrow-amber)"/>
<rect x="598" y="694" width="402" height="170" rx="8" fill="rgba(2, 6, 23, 0.58)" stroke="#475569"/>
<text x="622" y="724" class="box-title">Split-pod tradeoffs</text>
<text x="622" y="750" class="gain">+ Agent pod can run non-root with zero capabilities</text>
<text x="622" y="772" class="gain">+ gVisor/Kata reduces host-kernel blast radius</text>
<text x="622" y="794" class="gain">+ Separate log streams for audit and agent output</text>
<text x="622" y="816" class="loss">- Per-binary destination restrictions are lost cross-pod</text>
<text x="622" y="838" class="loss">- Enforcement now depends on CNI NetworkPolicy support</text>
<!-- Isolation backend column -->
<rect x="1118" y="128" width="402" height="96" rx="8" fill="#0f172a" stroke="#34d399" stroke-width="1.4"/>
<text x="1146" y="160" class="box-title">Supervisor stays policy authority</text>
<text x="1146" y="181" class="box-sub">proxy, OPA, credentials, audit, inference routing</text>
<text x="1146" y="199" class="box-sub">drives one runtime contract instead of inline topology code</text>
<rect x="1118" y="272" width="402" height="102" rx="8" fill="#0f172a" stroke="#a78bfa" stroke-width="1.4"/>
<text x="1146" y="304" class="box-title">Isolation Backend contract</text>
<text x="1146" y="325" class="box-sub">realize boundary, return verifiable handle</text>
<text x="1146" y="342" class="box-sub">fail closed if boundary is not ready</text>
<text x="1146" y="359" class="box-sub">operator-selected, not workload-selected</text>
<rect x="1118" y="422" width="128" height="88" rx="8" fill="#0f172a" stroke="#38bdf8" stroke-width="1.4"/>
<text x="1142" y="456" class="box-title">In-pod</text>
<text x="1142" y="476" class="box-sub">netns + nft</text>
<text x="1142" y="492" class="box-sub">status quo path</text>
<rect x="1252" y="422" width="128" height="88" rx="8" fill="#0f172a" stroke="#34d399" stroke-width="1.4"/>
<text x="1274" y="456" class="box-title">Delegated</text>
<text x="1274" y="476" class="box-sub">split pod</text>
<text x="1274" y="492" class="box-sub">CNI boundary</text>
<rect x="1386" y="422" width="128" height="88" rx="8" fill="#0f172a" stroke="#fb923c" stroke-width="1.4"/>
<text x="1408" y="456" class="box-title">VM / Node</text>
<text x="1408" y="476" class="box-sub">microVM</text>
<text x="1408" y="492" class="box-sub">node agent</text>
<rect x="1118" y="562" width="402" height="72" rx="8" fill="#0f172a" stroke="#fbbf24" stroke-width="1.4"/>
<text x="1146" y="592" class="box-title">Two invariants</text>
<text x="1146" y="612" class="box-sub">no unguarded workload egress before verification</text>
<text x="1146" y="628" class="box-sub">no workload execution before boundary ready</text>
<path class="flow" d="M1319 224 L1319 264" stroke="#a78bfa" marker-end="url(#arrow-violet)"/>
<path class="flow" d="M1319 374 L1319 414" stroke="#a78bfa" marker-end="url(#arrow-violet)"/>
<path class="thin" d="M1319 510 L1319 554" stroke="#fbbf24" marker-end="url(#arrow-amber)"/>
<path class="thin" d="M1319 510 C1240 532 1184 536 1184 554" stroke="#38bdf8" marker-end="url(#arrow-blue)"/>
<path class="thin" d="M1319 510 C1418 532 1450 536 1450 554" stroke="#fb923c" marker-end="url(#arrow-amber)"/>
<rect x="1118" y="694" width="402" height="170" rx="8" fill="rgba(2, 6, 23, 0.58)" stroke="#475569"/>
<text x="1142" y="724" class="box-title">Abstraction tradeoffs</text>
<text x="1142" y="750" class="gain">+ Separates policy authority from boundary machinery</text>
<text x="1142" y="772" class="gain">+ Lets split-pod, VM, or node-agent backends share a contract</text>
<text x="1142" y="794" class="gain">+ Enables in-pod refactor first, delegated later</text>
<text x="1142" y="816" class="loss">- Requires verifiable handoff and fail-closed semantics</text>
<text x="1142" y="838" class="loss">- Scope is unsettled: network only vs fs/syscall/process too</text>
<!-- Cross-column comparison -->
<rect x="40" y="948" width="1520" height="300" rx="12" fill="rgba(15, 23, 42, 0.68)" stroke="#334155" stroke-width="1.2"/>
<text x="66" y="985" class="title">Decision Map</text>
<text x="66" y="1006" class="subtitle">The diagrams point to different places for the hard boundary. That changes who must be trusted and which controls are enforceable.</text>
<line x1="66" y1="1030" x2="1534" y2="1030" stroke="#334155"/>
<text x="80" y="1058" class="label">Hard egress boundary</text>
<text x="390" y="1058" class="box-sub">current: netns + nftables inside pod</text>
<text x="755" y="1058" class="box-sub">split-pod: CNI NetworkPolicy outside agent pod</text>
<text x="1185" y="1058" class="box-sub">backend: selected per environment, verified by contract</text>
<line x1="66" y1="1078" x2="1534" y2="1078" stroke="#1f2937"/>
<text x="80" y="1106" class="label">Kernel isolation</text>
<text x="390" y="1106" class="box-sub">current: runc + seccomp + Landlock</text>
<text x="755" y="1106" class="box-sub">split-pod: gVisor userspace kernel or Kata VM</text>
<text x="1185" y="1106" class="box-sub">backend: can represent in-pod, VM, or delegated runtime</text>
<line x1="66" y1="1126" x2="1534" y2="1126" stroke="#1f2937"/>
<text x="80" y="1154" class="label">L7, credentials, inference</text>
<text x="390" y="1154" class="box-sub">current: local supervisor proxy</text>
<text x="755" y="1154" class="box-sub">split-pod: supervisor pod proxy</text>
<text x="1185" y="1154" class="box-sub">backend: remains with supervisor policy authority</text>
<line x1="66" y1="1174" x2="1534" y2="1174" stroke="#1f2937"/>
<text x="80" y="1202" class="label">Known loss / risk</text>
<text x="390" y="1202" class="box-sub">current: privileged setup near untrusted workload</text>
<text x="755" y="1202" class="box-sub">split-pod: per-binary policy and deny logs become weaker</text>
<text x="1185" y="1202" class="box-sub">backend: verification and versioning become first-class</text>
<line x1="66" y1="1222" x2="1534" y2="1222" stroke="#1f2937"/>
<text x="80" y="1250" class="label">Best fit</text>
<text x="390" y="1250" class="box-sub">local dev, k3d/k3s, environments allowing capabilities</text>
<text x="755" y="1250" class="box-sub">enterprise clusters, OpenShift/Gardener-style restrictions</text>
<text x="1185" y="1250" class="box-sub">long-term architecture that lets both shapes coexist</text>
<!-- Source strip -->
<rect x="220" y="1288" width="1160" height="48" rx="8" fill="rgba(2, 6, 23, 0.7)" stroke="#334155"/>
<text x="244" y="1318" class="tiny">Sources: current code/docs; #899 restricted SCC; #981 split supervisor/agent pod; #1305 proxy/runtime decomposition; #1650 process/network split PR; #1703 external compute driver PR; #1737 Isolation Backend interface.</text>
</svg>
</section>
<section class="notes" aria-label="summary notes">
<article class="note">
<h3>Current State</h3>
<p>The Kubernetes driver creates one Sandbox CR whose pod runs the agent container with supervisor sideloading and elevated capabilities so the supervisor can create the inner network namespace.</p>
</article>
<article class="note">
<h3>Split Pod</h3>
<p>The proposal moves the proxy and policy authority into a trusted supervisor pod. The agent pod becomes untrusted, non-root, no-capabilities, and optionally gVisor or Kata isolated.</p>
</article>
<article class="note">
<h3>NetworkPolicy</h3>
<p>Today Helm renders an ingress-only sandbox SSH NetworkPolicy. The split-pod proposal adds egress NetworkPolicy so agent pods can reach only the supervisor proxy and DNS.</p>
</article>
<article class="note">
<h3>Backend Interface</h3>
<p>The interface proposal keeps the supervisor as policy authority while letting in-pod, split-pod, VM, or node-agent backends realize the isolation boundary with explicit verification.</p>
</article>
</section>
<footer>
Sources used: <a href="https://github.com/NVIDIA/OpenShell/issues/899">#899</a>,
<a href="https://github.com/NVIDIA/OpenShell/issues/981">#981</a>,
<a href="https://github.com/NVIDIA/OpenShell/issues/1305">#1305</a>,
<a href="https://github.com/NVIDIA/OpenShell/pull/1650">PR #1650</a>,
<a href="https://github.com/NVIDIA/OpenShell/pull/1703">PR #1703</a>,
<a href="https://github.com/NVIDIA/OpenShell/issues/1737">#1737</a>,
plus local files <code>architecture/compute-runtimes.md</code>,
<code>deploy/helm/openshell/templates/networkpolicy.yaml</code>, and
<code>crates/openshell-driver-kubernetes/src/driver.rs</code>.
</footer>
</main>
</body>
</html>
spike/kubernetes-node-enforcer-options-diagram.html
+412
View File
@@ -0,0 +1,412 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OpenShell Kubernetes Node Enforcer Option</title>
<style>
:root {
--bg: #06080d;
--panel: #101827;
--panel-2: #0b1120;
--text: #e5e7eb;
--muted: #9ca3af;
--line: #334155;
--blue: #38bdf8;
--green: #34d399;
--amber: #fbbf24;
--red: #fb7185;
--violet: #a78bfa;
--orange: #fb923c;
--teal: #2dd4bf;
}
* {
box-sizing: border-box;
}
body {
margin: 0;
min-height: 100vh;
color: var(--text);
background:
linear-gradient(145deg, rgba(45, 212, 191, 0.08), transparent 28rem),
linear-gradient(315deg, rgba(251, 146, 60, 0.08), transparent 28rem),
var(--bg);
font-family: "JetBrains Mono", ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
}
main {
width: min(1680px, calc(100vw - 32px));
margin: 0 auto;
padding: 28px 0 36px;
}
header {
display: flex;
justify-content: space-between;
align-items: end;
gap: 24px;
margin-bottom: 18px;
}
h1 {
margin: 0;
font-size: clamp(24px, 4vw, 40px);
font-weight: 780;
letter-spacing: 0;
}
.subtitle {
margin: 9px 0 0;
max-width: 1020px;
color: var(--muted);
font-size: 13px;
line-height: 1.55;
}
.branch {
padding: 8px 10px;
border-radius: 8px;
border: 1px solid rgba(148, 163, 184, 0.22);
background: rgba(15, 23, 42, 0.72);
color: #cbd5e1;
font-size: 11px;
white-space: nowrap;
}
.diagram {
overflow: auto;
border: 1px solid rgba(148, 163, 184, 0.24);
border-radius: 8px;
background: rgba(15, 23, 42, 0.72);
box-shadow: 0 20px 72px rgba(0, 0, 0, 0.36);
}
svg {
display: block;
width: 100%;
min-width: 1240px;
height: auto;
}
.notes {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 14px;
margin-top: 16px;
}
.note {
border: 1px solid rgba(148, 163, 184, 0.22);
border-radius: 8px;
background: rgba(15, 23, 42, 0.76);
padding: 14px;
}
.note h3 {
margin: 0 0 8px;
font-size: 12px;
letter-spacing: 0;
}
.note p {
margin: 0;
color: var(--muted);
font-size: 11px;
line-height: 1.55;
}
footer {
margin-top: 15px;
color: #64748b;
font-size: 10px;
line-height: 1.6;
}
footer a {
color: #93c5fd;
}
@media (max-width: 980px) {
main {
width: calc(100vw - 20px);
padding-top: 18px;
}
header {
display: block;
}
.branch {
display: inline-block;
margin-top: 12px;
}
.notes {
grid-template-columns: 1fr;
}
}
</style>
</head>
<body>
<main>
<header>
<div>
<h1>Kubernetes Node Enforcer Option</h1>
<p class="subtitle">
The node-enforcer branch adds a fourth Kubernetes isolation shape: keep the workload supervisor and proxy in the sandbox pod, but move privileged pod-network-namespace enforcement to an operator-controlled DaemonSet on each node.
</p>
</div>
<div class="branch">branch: node-enforcer-poc/tmutch</div>
</header>
<section class="diagram" aria-label="OpenShell Kubernetes node-enforcer option diagram">
<svg viewBox="0 0 1680 1460" role="img" aria-labelledby="diagram-title diagram-desc">
<title id="diagram-title">OpenShell Kubernetes node-enforcer option</title>
<desc id="diagram-desc">Diagram of the node-enforcer topology and comparison to supervisor-netns, soft-proxy, split-pod, and isolation backend options.</desc>
<defs>
<pattern id="grid" width="36" height="36" patternUnits="userSpaceOnUse">
<path d="M 36 0 L 0 0 0 36" fill="none" stroke="#1f2937" stroke-width="0.5"/>
</pattern>
<marker id="arrow-blue" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#38bdf8"/>
</marker>
<marker id="arrow-green" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#34d399"/>
</marker>
<marker id="arrow-amber" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#fbbf24"/>
</marker>
<marker id="arrow-red" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#fb7185"/>
</marker>
<marker id="arrow-violet" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#a78bfa"/>
</marker>
<style>
.title { fill: #f8fafc; font-size: 18px; font-weight: 780; letter-spacing: 0; }
.sub { fill: #94a3b8; font-size: 9px; letter-spacing: 0; }
.label { fill: #cbd5e1; font-size: 9px; letter-spacing: 0; }
.tiny { fill: #94a3b8; font-size: 7px; letter-spacing: 0; }
.box-title { fill: #f8fafc; font-size: 11px; font-weight: 750; letter-spacing: 0; }
.box-sub { fill: #cbd5e1; font-size: 8px; letter-spacing: 0; }
.gain { fill: #34d399; font-size: 8px; letter-spacing: 0; }
.loss { fill: #fb7185; font-size: 8px; letter-spacing: 0; }
.warn { fill: #fbbf24; font-size: 8px; letter-spacing: 0; }
.flow { fill: none; stroke-width: 1.6; stroke-linecap: round; stroke-linejoin: round; }
.dash { fill: none; stroke-width: 1.4; stroke-dasharray: 5 5; stroke-linecap: round; stroke-linejoin: round; }
.thin { fill: none; stroke-width: 1.1; stroke-linecap: round; stroke-linejoin: round; }
</style>
</defs>
<rect width="1680" height="1460" fill="#06080d"/>
<rect width="1680" height="1460" fill="url(#grid)" opacity="0.9"/>
<!-- Main topology panel -->
<rect x="40" y="42" width="1600" height="655" rx="12" fill="rgba(15, 23, 42, 0.68)" stroke="#334155" stroke-width="1.2"/>
<text x="68" y="78" class="title">Option 4: Node-Enforcer Topology</text>
<text x="68" y="99" class="sub">workload supervisor remains in the sandbox pod; privileged netns/nft enforcement moves to a node-local DaemonSet</text>
<!-- Gateway -->
<rect x="82" y="154" width="226" height="94" rx="8" fill="#0b1120" stroke="#34d399" stroke-width="1.4"/>
<text x="110" y="188" class="box-title">Gateway</text>
<text x="110" y="209" class="box-sub">creates Sandbox CR</text>
<text x="110" y="226" class="box-sub">injects role + enforcement mode</text>
<rect x="82" y="304" width="226" height="94" rx="8" fill="#0b1120" stroke="#a78bfa" stroke-width="1.4"/>
<text x="110" y="338" class="box-title">Kubernetes Driver</text>
<text x="110" y="359" class="box-sub">supervisor_role=workload</text>
<text x="110" y="376" class="box-sub">network=external-enforcer</text>
<!-- Cluster / node -->
<rect x="386" y="128" width="1184" height="530" rx="14" fill="rgba(2, 6, 23, 0.42)" stroke="#475569" stroke-width="1.3" stroke-dasharray="8 4"/>
<text x="414" y="156" class="label">Kubernetes node</text>
<rect x="430" y="202" width="324" height="290" rx="12" fill="rgba(15, 23, 42, 0.86)" stroke="#fb923c" stroke-width="1.4"/>
<text x="456" y="232" class="box-title">node-enforcer DaemonSet</text>
<text x="456" y="253" class="box-sub">same supervisor image, role=enforcer</text>
<text x="456" y="270" class="box-sub">hostNetwork + hostPID + privileged</text>
<text x="456" y="287" class="box-sub">listens on host-network endpoint</text>
<rect x="464" y="326" width="256" height="62" rx="7" fill="#0b1120" stroke="#fbbf24" stroke-width="1.3"/>
<text x="486" y="354" class="box-title">Namespace lookup</text>
<text x="486" y="372" class="box-sub">find pod netns by pod IP in /proc</text>
<rect x="464" y="412" width="256" height="58" rx="7" fill="#0b1120" stroke="#fb7185" stroke-width="1.3"/>
<text x="486" y="440" class="box-title">Install nft table</text>
<text x="486" y="458" class="box-sub">setns(CLONE_NEWNET) + nft -f</text>
<rect x="860" y="180" width="414" height="376" rx="12" fill="rgba(15, 23, 42, 0.86)" stroke="#38bdf8" stroke-width="1.4" stroke-dasharray="8 4"/>
<text x="888" y="210" class="label">Sandbox pod network namespace</text>
<rect x="894" y="242" width="332" height="90" rx="8" fill="#0b1120" stroke="#34d399" stroke-width="1.4"/>
<text x="924" y="274" class="box-title">Workload supervisor</text>
<text x="924" y="294" class="box-sub">policy loading, proxy, SSH, logs, agent lifecycle</text>
<text x="924" y="312" class="box-sub">registers with node enforcer before spawning agent</text>
<rect x="894" y="372" width="144" height="72" rx="8" fill="#0b1120" stroke="#34d399" stroke-width="1.4"/>
<text x="922" y="403" class="box-title">Local proxy</text>
<text x="922" y="423" class="box-sub">OPA + L7 + creds</text>
<rect x="1082" y="372" width="144" height="72" rx="8" fill="#0b1120" stroke="#fb7185" stroke-width="1.4"/>
<text x="1106" y="403" class="box-title">Agent child</text>
<text x="1106" y="423" class="box-sub">sandbox user</text>
<rect x="936" y="486" width="226" height="40" rx="7" fill="#0b1120" stroke="#fbbf24" stroke-width="1.2"/>
<text x="960" y="510" class="box-title">openshell_external_enforcer table</text>
<rect x="1322" y="230" width="202" height="84" rx="8" fill="#0b1120" stroke="#94a3b8" stroke-width="1.3"/>
<text x="1350" y="262" class="box-title">External services</text>
<text x="1350" y="282" class="box-sub">allowed only through proxy</text>
<rect x="1322" y="380" width="202" height="84" rx="8" fill="#0b1120" stroke="#a78bfa" stroke-width="1.3"/>
<text x="1350" y="412" class="box-title">Gateway session</text>
<text x="1350" y="432" class="box-sub">config, relay, logs</text>
<path class="flow" d="M308 201 C344 201 348 254 386 254" stroke="#34d399" marker-end="url(#arrow-green)"/>
<text x="322" y="187" class="label">create sandbox</text>
<path class="flow" d="M308 351 L386 351" stroke="#a78bfa" marker-end="url(#arrow-violet)"/>
<text x="322" y="339" class="label">inject env</text>
<path class="flow" d="M894 300 C820 300 812 248 762 248" stroke="#fbbf24" marker-end="url(#arrow-amber)"/>
<text x="770" y="288" class="label">HTTP registration</text>
<text x="772" y="304" class="tiny">pod IP + sandbox ID</text>
<path class="flow" d="M720 356 C780 356 810 506 928 506" stroke="#fb7185" marker-end="url(#arrow-red)"/>
<text x="744" y="430" class="label">install pod-netns nft rules</text>
<path class="flow" d="M1082 405 L1046 405" stroke="#38bdf8" marker-end="url(#arrow-blue)"/>
<text x="1046" y="391" class="label">HTTP_PROXY</text>
<path class="flow" d="M1038 405 L1294 272 L1314 272" stroke="#34d399" marker-end="url(#arrow-green)"/>
<text x="1170" y="328" class="label">UID 0 proxy egress</text>
<path class="dash" d="M1154 444 C1160 482 1202 512 1230 506" stroke="#fb7185" marker-end="url(#arrow-red)"/>
<text x="1182" y="488" class="label">non-root direct socket rejected</text>
<path class="dash" d="M1226 290 C1268 316 1282 406 1314 422" stroke="#a78bfa" marker-end="url(#arrow-violet)"/>
<text x="1240" y="374" class="label">gateway callback</text>
<rect x="82" y="534" width="674" height="92" rx="8" fill="rgba(2, 6, 23, 0.56)" stroke="#475569"/>
<text x="110" y="564" class="box-title">What moved out of the workload pod?</text>
<text x="110" y="588" class="gain">+ netns discovery and nftables installation are node-side infrastructure duties</text>
<text x="110" y="610" class="gain">+ workload supervisor keeps policy/proxy/credentials close to the agent</text>
<rect x="820" y="584" width="704" height="42" rx="8" fill="rgba(2, 6, 23, 0.56)" stroke="#475569"/>
<text x="848" y="610" class="warn">Prototype hardening required: authenticate registrations, verify pod ownership, reconcile cleanup/stale state, avoid per-release DaemonSet port conflicts.</text>
<!-- Comparison panel -->
<rect x="40" y="736" width="1600" height="372" rx="12" fill="rgba(15, 23, 42, 0.68)" stroke="#334155" stroke-width="1.2"/>
<text x="68" y="774" class="title">Comparison Against Other Kubernetes Options</text>
<text x="68" y="795" class="sub">The options differ mostly by where the hard network boundary lives and whether the workload pod needs elevated privileges.</text>
<line x1="68" y1="824" x2="1612" y2="824" stroke="#334155"/>
<text x="80" y="852" class="label">Option</text>
<text x="312" y="852" class="label">Hard egress boundary</text>
<text x="622" y="852" class="label">Privileged component</text>
<text x="910" y="852" class="label">What remains local to agent pod</text>
<text x="1260" y="852" class="label">Main tradeoff</text>
<line x1="68" y1="872" x2="1612" y2="872" stroke="#1f2937"/>
<text x="80" y="902" class="box-sub">Supervisor-netns</text>
<text x="312" y="902" class="box-sub">inner netns + veth + nft</text>
<text x="622" y="902" class="box-sub">sandbox workload supervisor</text>
<text x="910" y="902" class="box-sub">all policy, proxy, fs, process, netns</text>
<text x="1260" y="902" class="loss">strong but requires elevated sandbox pod</text>
<line x1="68" y1="924" x2="1612" y2="924" stroke="#1f2937"/>
<text x="80" y="954" class="box-sub">Soft-proxy</text>
<text x="312" y="954" class="box-sub">none for direct sockets</text>
<text x="622" y="954" class="box-sub">none for network setup</text>
<text x="910" y="954" class="box-sub">proxy, policy, lifecycle, logs</text>
<text x="1260" y="954" class="loss">deployable but bypass is not kernel-blocked</text>
<line x1="68" y1="976" x2="1612" y2="976" stroke="#1f2937"/>
<text x="80" y="1006" class="box-sub">Split-pod</text>
<text x="312" y="1006" class="box-sub">CNI NetworkPolicy</text>
<text x="622" y="1006" class="box-sub">trusted supervisor pod / platform</text>
<text x="910" y="1006" class="box-sub">agent only; optional sideload SSH/Landlock</text>
<text x="1260" y="1006" class="loss">loses trusted per-binary identity cross-pod</text>
<line x1="68" y1="1028" x2="1612" y2="1028" stroke="#1f2937"/>
<text x="80" y="1058" class="box-sub">Node-enforcer</text>
<text x="312" y="1058" class="box-sub">pod netns nft table installed by DaemonSet</text>
<text x="622" y="1058" class="box-sub">node enforcer only</text>
<text x="910" y="1058" class="box-sub">proxy, OPA, creds, SSH, agent lifecycle</text>
<text x="1260" y="1058" class="gain">keeps proxy local; moves network privilege out</text>
<line x1="68" y1="1080" x2="1612" y2="1080" stroke="#1f2937"/>
<text x="80" y="1102" class="box-sub">Isolation Backend</text>
<text x="312" y="1102" class="box-sub">selected backend: netns, CNI, node, VM</text>
<text x="622" y="1102" class="box-sub">backend-specific</text>
<text x="910" y="1102" class="box-sub">supervisor as policy authority</text>
<text x="1260" y="1102" class="warn">needs verifiable handoff and fail-closed contract</text>
<!-- Backend placement panel -->
<rect x="40" y="1148" width="1600" height="272" rx="12" fill="rgba(15, 23, 42, 0.68)" stroke="#334155" stroke-width="1.2"/>
<text x="68" y="1186" class="title">How Node-Enforcer Fits the Isolation Backend Abstraction</text>
<rect x="90" y="1226" width="248" height="88" rx="8" fill="#0b1120" stroke="#34d399" stroke-width="1.3"/>
<text x="118" y="1258" class="box-title">Supervisor contract</text>
<text x="118" y="1278" class="box-sub">policy authority, proxy, audit</text>
<text x="118" y="1295" class="box-sub">register before agent exec</text>
<rect x="442" y="1226" width="248" height="88" rx="8" fill="#0b1120" stroke="#a78bfa" stroke-width="1.3"/>
<text x="470" y="1258" class="box-title">Backend handle</text>
<text x="470" y="1278" class="box-sub">pod IP + sandbox ID</text>
<text x="470" y="1295" class="box-sub">node-local enforcer endpoint</text>
<rect x="794" y="1226" width="248" height="88" rx="8" fill="#0b1120" stroke="#fb923c" stroke-width="1.3"/>
<text x="822" y="1258" class="box-title">Backend mechanism</text>
<text x="822" y="1278" class="box-sub">hostPID /proc scan</text>
<text x="822" y="1295" class="box-sub">setns + nft ruleset</text>
<rect x="1146" y="1226" width="248" height="88" rx="8" fill="#0b1120" stroke="#fbbf24" stroke-width="1.3"/>
<text x="1174" y="1258" class="box-title">Required invariant</text>
<text x="1174" y="1278" class="box-sub">no agent execution before</text>
<text x="1174" y="1295" class="box-sub">boundary is installed/verified</text>
<path class="flow" d="M338 1270 L434 1270" stroke="#34d399" marker-end="url(#arrow-green)"/>
<path class="flow" d="M690 1270 L786 1270" stroke="#a78bfa" marker-end="url(#arrow-violet)"/>
<path class="flow" d="M1042 1270 L1138 1270" stroke="#fb923c" marker-end="url(#arrow-amber)"/>
<rect x="1428" y="1226" width="164" height="88" rx="8" fill="#0b1120" stroke="#fb7185" stroke-width="1.3"/>
<text x="1450" y="1258" class="box-title">Open questions</text>
<text x="1450" y="1278" class="box-sub">authn/authz</text>
<text x="1450" y="1295" class="box-sub">cleanup + ownership checks</text>
<text x="68" y="1370" class="tiny">Sources: node-enforcer-poc/tmutch branch files architecture/plans/node-enforcer-topology-findings.md, crates/openshell-sandbox/src/enforcer.rs, crates/openshell-sandbox/src/lib.rs, crates/openshell-driver-kubernetes/src/driver.rs, deploy/helm/openshell/templates/node-enforcer.yaml, docs/reference/sandbox-compute-drivers.mdx, plus proposal context from #981 and #1737.</text>
</svg>
</section>
<section class="notes" aria-label="node enforcer summary notes">
<article class="note">
<h3>What It Preserves</h3>
<p>The workload supervisor still owns proxy, OPA, credential resolution, inference routing, SSH, logs, settings reloads, and agent lifecycle inside the sandbox pod.</p>
</article>
<article class="note">
<h3>What It Moves</h3>
<p>The node enforcer owns pod network namespace discovery and coarse nftables egress enforcement, moving that privilege out of the sandbox workload pod.</p>
</article>
<article class="note">
<h3>What It Does Not Do</h3>
<p>It does not split the agent into a separate pod and does not use Kubernetes NetworkPolicy as the hard boundary. Dynamic endpoint and L7 policy still live in the proxy.</p>
</article>
<article class="note">
<h3>Hardening Work</h3>
<p>The POC needs authenticated registration, pod ownership verification, cleanup/reconciliation for stale namespaces, and shared-cluster deployment rules.</p>
</article>
</section>
<footer>
Related proposal context:
<a href="https://github.com/NVIDIA/OpenShell/issues/981">split-pod #981</a>,
<a href="https://github.com/NVIDIA/OpenShell/issues/1737">Isolation Backend #1737</a>.
Local branch sources are listed inside the diagram.
</footer>
</main>
</body>
</html>
spike/openshell-isolation-diagram.html
+559
View File
@@ -0,0 +1,559 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OpenShell Isolation Architecture</title>
<style>
:root {
--bg: #020617;
--panel: #0f172a;
--text: #e2e8f0;
--muted: #94a3b8;
--grid: #1e293b;
--cyan: #22d3ee;
--emerald: #34d399;
--violet: #a78bfa;
--amber: #fbbf24;
--rose: #fb7185;
--orange: #fb923c;
--slate: #94a3b8;
}
* {
box-sizing: border-box;
}
body {
margin: 0;
min-height: 100vh;
color: var(--text);
background:
radial-gradient(circle at 18% 12%, rgba(34, 211, 238, 0.08), transparent 24rem),
radial-gradient(circle at 78% 8%, rgba(251, 191, 36, 0.07), transparent 22rem),
var(--bg);
font-family: "JetBrains Mono", ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
}
main {
width: min(1560px, calc(100vw - 32px));
margin: 0 auto;
padding: 28px 0 36px;
}
.header {
display: flex;
justify-content: space-between;
gap: 24px;
align-items: end;
margin-bottom: 18px;
}
.title-row {
display: flex;
align-items: center;
gap: 12px;
}
.pulse {
width: 12px;
height: 12px;
border-radius: 999px;
background: var(--emerald);
box-shadow: 0 0 0 rgba(52, 211, 153, 0.65);
animation: pulse 2.1s infinite;
flex: 0 0 auto;
}
@keyframes pulse {
0% { box-shadow: 0 0 0 0 rgba(52, 211, 153, 0.55); }
70% { box-shadow: 0 0 0 12px rgba(52, 211, 153, 0); }
100% { box-shadow: 0 0 0 0 rgba(52, 211, 153, 0); }
}
h1 {
margin: 0;
font-size: clamp(24px, 4vw, 40px);
font-weight: 760;
letter-spacing: 0;
}
.subtitle {
margin: 8px 0 0 24px;
color: var(--muted);
font-size: 13px;
line-height: 1.55;
max-width: 900px;
}
.meta {
color: var(--muted);
font-size: 11px;
text-align: right;
line-height: 1.6;
white-space: nowrap;
}
.diagram-card {
border: 1px solid rgba(148, 163, 184, 0.24);
border-radius: 8px;
overflow: auto;
background: rgba(15, 23, 42, 0.72);
box-shadow: 0 22px 80px rgba(0, 0, 0, 0.32);
}
svg {
display: block;
width: 100%;
min-width: 1120px;
height: auto;
}
.cards {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 14px;
margin-top: 16px;
}
.card {
border: 1px solid rgba(148, 163, 184, 0.22);
border-radius: 8px;
background: rgba(15, 23, 42, 0.78);
padding: 14px 15px 16px;
}
.card-header {
display: flex;
align-items: center;
gap: 9px;
margin-bottom: 9px;
}
.card-dot {
width: 9px;
height: 9px;
border-radius: 999px;
flex: 0 0 auto;
}
.card-dot.cyan { background: var(--cyan); }
.card-dot.rose { background: var(--rose); }
.card-dot.amber { background: var(--amber); }
h3 {
margin: 0;
font-size: 13px;
font-weight: 720;
letter-spacing: 0;
}
ul {
margin: 0;
padding: 0;
list-style: none;
color: var(--muted);
font-size: 11px;
line-height: 1.65;
}
footer {
margin-top: 16px;
color: #64748b;
font-size: 10px;
line-height: 1.6;
}
@media (max-width: 900px) {
main {
width: calc(100vw - 20px);
padding-top: 18px;
}
.header {
display: block;
}
.subtitle {
margin-left: 0;
}
.meta {
margin-top: 12px;
text-align: left;
white-space: normal;
}
.cards {
grid-template-columns: 1fr;
}
}
</style>
</head>
<body>
<main>
<header class="header">
<div>
<div class="title-row">
<span class="pulse" aria-hidden="true"></span>
<h1>OpenShell Networking + Filesystem Isolation</h1>
</div>
<p class="subtitle">
Gateway policy delivery stays separate from sandbox-local enforcement. The supervisor prepares static controls, launches the agent as an unprivileged child, and forces observable egress through the policy proxy.
</p>
</div>
<div class="meta">
Standalone HTML/SVG<br>
Generated from repo docs + sandbox code
</div>
</header>
<section class="diagram-card" aria-label="OpenShell isolation architecture diagram">
<svg viewBox="0 0 1500 1360" role="img" aria-labelledby="diagram-title diagram-desc">
<title id="diagram-title">OpenShell isolation architecture</title>
<desc id="diagram-desc">A three-panel architecture diagram showing gateway and sandbox boundaries, network egress enforcement, and filesystem startup isolation.</desc>
<defs>
<pattern id="grid" width="40" height="40" patternUnits="userSpaceOnUse">
<path d="M 40 0 L 0 0 0 40" fill="none" stroke="#1e293b" stroke-width="0.5"/>
</pattern>
<marker id="arrow-cyan" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#22d3ee"/>
</marker>
<marker id="arrow-emerald" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#34d399"/>
</marker>
<marker id="arrow-amber" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#fbbf24"/>
</marker>
<marker id="arrow-rose" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#fb7185"/>
</marker>
<marker id="arrow-slate" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#94a3b8"/>
</marker>
<filter id="soft-shadow" x="-30%" y="-30%" width="160%" height="160%">
<feDropShadow dx="0" dy="8" stdDeviation="8" flood-color="#000000" flood-opacity="0.28"/>
</filter>
<style>
.panel-title { fill: #e2e8f0; font-size: 18px; font-weight: 760; letter-spacing: 0; }
.panel-sub { fill: #94a3b8; font-size: 10px; letter-spacing: 0; }
.box-title { fill: #f8fafc; font-size: 12px; font-weight: 740; letter-spacing: 0; }
.box-sub { fill: #cbd5e1; font-size: 9px; letter-spacing: 0; }
.box-note { fill: #94a3b8; font-size: 8px; letter-spacing: 0; }
.tiny { fill: #94a3b8; font-size: 7px; letter-spacing: 0; }
.label { fill: #cbd5e1; font-size: 9px; letter-spacing: 0; }
.legend { fill: #cbd5e1; font-size: 9px; letter-spacing: 0; }
.flow { fill: none; stroke-width: 1.7; stroke-linecap: round; stroke-linejoin: round; }
.flow-thin { fill: none; stroke-width: 1.2; stroke-linecap: round; stroke-linejoin: round; }
.flow-dash { fill: none; stroke-width: 1.5; stroke-dasharray: 5 5; stroke-linecap: round; stroke-linejoin: round; }
.boundary-label { fill: #fbbf24; font-size: 10px; font-weight: 700; letter-spacing: 0; }
</style>
</defs>
<rect width="1500" height="1360" fill="#020617"/>
<rect width="1500" height="1360" fill="url(#grid)" opacity="0.9"/>
<!-- Background panel boundaries -->
<rect x="38" y="36" width="1424" height="386" rx="12" fill="rgba(120, 53, 15, 0.08)" stroke="#fbbf24" stroke-width="1.3" stroke-dasharray="8 4"/>
<text x="58" y="64" class="panel-title">1. Runtime Boundary</text>
<text x="58" y="84" class="panel-sub">Gateway delivers desired state. Sandbox supervisor owns runtime enforcement.</text>
<rect x="38" y="452" width="1424" height="438" rx="12" fill="rgba(136, 19, 55, 0.055)" stroke="#fb7185" stroke-width="1.3" stroke-dasharray="8 4"/>
<text x="58" y="480" class="panel-title">2. Network Egress Enforcement</text>
<text x="58" y="500" class="panel-sub">Ordinary traffic is forced through the proxy, evaluated, and either forwarded, denied, logged, or routed to inference.local.</text>
<rect x="38" y="920" width="1424" height="278" rx="12" fill="rgba(6, 78, 59, 0.055)" stroke="#34d399" stroke-width="1.3" stroke-dasharray="8 4"/>
<text x="58" y="948" class="panel-title">3. Filesystem + Process Startup</text>
<text x="58" y="968" class="panel-sub">Static controls are prepared before exec and normally require sandbox recreation to change.</text>
<!-- Runtime boundary -->
<rect x="76" y="116" width="205" height="72" rx="6" fill="#0f172a"/>
<rect x="76" y="116" width="205" height="72" rx="6" fill="rgba(8, 51, 68, 0.4)" stroke="#22d3ee" stroke-width="1.5"/>
<text x="98" y="145" class="box-title">CLI / SDK / TUI</text>
<text x="98" y="164" class="box-sub">user-facing API clients</text>
<rect x="356" y="104" width="270" height="96" rx="6" fill="#0f172a"/>
<rect x="356" y="104" width="270" height="96" rx="6" fill="rgba(6, 78, 59, 0.4)" stroke="#34d399" stroke-width="1.5"/>
<text x="382" y="134" class="box-title">Gateway Control Plane</text>
<text x="382" y="154" class="box-sub">sandbox state, policy revisions</text>
<text x="382" y="171" class="box-sub">settings, credentials, inference routes</text>
<rect x="682" y="116" width="218" height="72" rx="6" fill="#0f172a"/>
<rect x="682" y="116" width="218" height="72" rx="6" fill="rgba(120, 53, 15, 0.3)" stroke="#fbbf24" stroke-width="1.5"/>
<text x="710" y="145" class="box-title">Compute Runtime</text>
<text x="710" y="164" class="box-sub">Docker, Podman, K8s, VM</text>
<rect x="950" y="92" width="404" height="310" rx="12" fill="rgba(15, 23, 42, 0.48)" stroke="#fbbf24" stroke-width="1.2" stroke-dasharray="8 4"/>
<text x="972" y="119" class="boundary-label">Sandbox workload</text>
<rect x="984" y="144" width="298" height="84" rx="6" fill="#0f172a"/>
<rect x="984" y="144" width="298" height="84" rx="6" fill="rgba(6, 78, 59, 0.4)" stroke="#34d399" stroke-width="1.5"/>
<text x="1012" y="174" class="box-title">Supervisor</text>
<text x="1012" y="193" class="box-sub">root prelude, policy load, config polling</text>
<text x="1012" y="210" class="box-sub">proxy, SSH relay, log push</text>
<rect x="984" y="268" width="298" height="72" rx="6" fill="#0f172a"/>
<rect x="984" y="268" width="298" height="72" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="1012" y="297" class="box-title">Restricted Agent Child</text>
<text x="1012" y="316" class="box-sub">sandbox user + group</text>
<text x="1012" y="331" class="box-note">Landlock, seccomp, netns active</text>
<rect x="1320" y="146" width="116" height="50" rx="6" fill="#0f172a"/>
<rect x="1320" y="146" width="116" height="50" rx="6" fill="rgba(76, 29, 149, 0.4)" stroke="#a78bfa" stroke-width="1.5"/>
<text x="1339" y="174" class="box-title">OPA</text>
<text x="1338" y="190" class="box-sub">policy engine</text>
<rect x="1320" y="214" width="116" height="50" rx="6" fill="#0f172a"/>
<rect x="1320" y="214" width="116" height="50" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="1334" y="242" class="box-title">Proxy</text>
<text x="1337" y="258" class="box-sub">egress gate</text>
<rect x="1320" y="282" width="116" height="50" rx="6" fill="#0f172a"/>
<rect x="1320" y="282" width="116" height="50" rx="6" fill="rgba(30, 41, 59, 0.5)" stroke="#94a3b8" stroke-width="1.5"/>
<text x="1334" y="310" class="box-title">Relay</text>
<text x="1332" y="326" class="box-sub">connect/exec/fs</text>
<path class="flow" d="M281 152 L347 152" stroke="#22d3ee" marker-end="url(#arrow-cyan)"/>
<text x="296" y="141" class="label">gRPC / HTTP</text>
<path class="flow" d="M626 152 L673 152" stroke="#34d399" marker-end="url(#arrow-emerald)"/>
<text x="634" y="141" class="label">launch request</text>
<path class="flow" d="M900 152 C926 152 926 186 976 186" stroke="#fbbf24" marker-end="url(#arrow-amber)"/>
<text x="909" y="137" class="label">provisions workload</text>
<path class="flow-dash" d="M984 170 C846 182 756 210 626 184" stroke="#34d399" marker-end="url(#arrow-emerald)"/>
<text x="718" y="203" class="label">outbound supervisor session</text>
<text x="720" y="219" class="tiny">config refresh, logs, relays</text>
<path class="flow" d="M1133 228 L1133 260" stroke="#34d399" marker-end="url(#arrow-emerald)"/>
<text x="1146" y="250" class="label">spawn + restrict</text>
<path class="flow-thin" d="M1282 188 L1312 171" stroke="#a78bfa" marker-end="url(#arrow-slate)"/>
<path class="flow-thin" d="M1282 198 L1312 239" stroke="#fb7185" marker-end="url(#arrow-rose)"/>
<path class="flow-thin" d="M1282 206 L1312 307" stroke="#94a3b8" marker-end="url(#arrow-slate)"/>
<!-- Network panel -->
<rect x="76" y="552" width="185" height="74" rx="6" fill="#0f172a"/>
<rect x="76" y="552" width="185" height="74" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="100" y="582" class="box-title">Agent Process</text>
<text x="100" y="601" class="box-sub">HTTP(S), git, model SDKs</text>
<text x="100" y="616" class="box-note">env: http_proxy / HTTPS_PROXY</text>
<rect x="76" y="676" width="185" height="82" rx="6" fill="#0f172a"/>
<rect x="76" y="676" width="185" height="82" rx="6" fill="rgba(30, 41, 59, 0.5)" stroke="#94a3b8" stroke-width="1.5"/>
<text x="98" y="705" class="box-title">Bypass Guards</text>
<text x="98" y="724" class="box-sub">netns + veth route</text>
<text x="98" y="741" class="box-sub">nft rejects non-proxy TCP/UDP</text>
<rect x="332" y="552" width="190" height="74" rx="6" fill="#0f172a"/>
<rect x="332" y="552" width="190" height="74" rx="6" fill="rgba(6, 78, 59, 0.4)" stroke="#34d399" stroke-width="1.5"/>
<text x="358" y="582" class="box-title">Local CONNECT Proxy</text>
<text x="358" y="601" class="box-sub">proxy endpoint inside netns</text>
<text x="358" y="616" class="box-note">terminates selected TLS/L7</text>
<rect x="584" y="540" width="190" height="62" rx="6" fill="#0f172a"/>
<rect x="584" y="540" width="190" height="62" rx="6" fill="rgba(76, 29, 149, 0.4)" stroke="#a78bfa" stroke-width="1.5"/>
<text x="610" y="568" class="box-title">Binary Identity</text>
<text x="610" y="586" class="box-sub">path, sha256, ancestors</text>
<rect x="584" y="626" width="190" height="62" rx="6" fill="#0f172a"/>
<rect x="584" y="626" width="190" height="62" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="610" y="654" class="box-title">SSRF Hard Blocks</text>
<text x="610" y="672" class="box-sub">internal IPs unless allowed</text>
<rect x="584" y="712" width="190" height="62" rx="6" fill="#0f172a"/>
<rect x="584" y="712" width="190" height="62" rx="6" fill="rgba(76, 29, 149, 0.4)" stroke="#a78bfa" stroke-width="1.5"/>
<text x="610" y="740" class="box-title">OPA L4 Decision</text>
<text x="610" y="758" class="box-sub">host, port, binary, policy</text>
<rect x="854" y="540" width="204" height="82" rx="6" fill="#0f172a"/>
<rect x="854" y="540" width="204" height="82" rx="6" fill="rgba(8, 51, 68, 0.4)" stroke="#22d3ee" stroke-width="1.5"/>
<text x="882" y="570" class="box-title">TLS / L7 Inspection</text>
<text x="882" y="589" class="box-sub">REST, GraphQL, WebSocket</text>
<text x="882" y="606" class="box-note">method, path, operation, fields</text>
<rect x="1136" y="540" width="210" height="82" rx="6" fill="#0f172a"/>
<rect x="1136" y="540" width="210" height="82" rx="6" fill="rgba(30, 41, 59, 0.5)" stroke="#94a3b8" stroke-width="1.5"/>
<text x="1166" y="570" class="box-title">External Services</text>
<text x="1166" y="589" class="box-sub">allowed upstream traffic</text>
<text x="1166" y="606" class="box-note">credentials resolved only in proxy</text>
<rect x="854" y="694" width="204" height="72" rx="6" fill="#0f172a"/>
<rect x="854" y="694" width="204" height="72" rx="6" fill="rgba(251, 146, 60, 0.3)" stroke="#fb923c" stroke-width="1.5"/>
<text x="884" y="723" class="box-title">inference.local</text>
<text x="884" y="742" class="box-sub">special route, not OPA network</text>
<rect x="1136" y="694" width="210" height="72" rx="6" fill="#0f172a"/>
<rect x="1136" y="694" width="210" height="72" rx="6" fill="rgba(251, 146, 60, 0.3)" stroke="#fb923c" stroke-width="1.5"/>
<text x="1164" y="723" class="box-title">openshell-router</text>
<text x="1164" y="742" class="box-sub">configured model backends</text>
<rect x="854" y="798" width="204" height="62" rx="6" fill="#0f172a"/>
<rect x="854" y="798" width="204" height="62" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="884" y="826" class="box-title">Deny / Audit / Log</text>
<text x="884" y="844" class="box-sub">OCSF + structured 403</text>
<rect x="1136" y="798" width="210" height="62" rx="6" fill="#0f172a"/>
<rect x="1136" y="798" width="210" height="62" rx="6" fill="rgba(120, 53, 15, 0.3)" stroke="#fbbf24" stroke-width="1.5"/>
<text x="1164" y="826" class="box-title">Policy Proposals</text>
<text x="1164" y="844" class="box-sub">L4 denial summaries to gateway</text>
<path class="flow" d="M261 589 L323 589" stroke="#34d399" marker-end="url(#arrow-emerald)"/>
<text x="276" y="577" class="label">ordinary egress</text>
<path class="flow-dash" d="M168 626 L168 668" stroke="#94a3b8" marker-end="url(#arrow-slate)"/>
<text x="185" y="651" class="label">bypass attempt</text>
<path class="flow" d="M522 589 C548 589 548 571 576 571" stroke="#a78bfa" marker-end="url(#arrow-slate)"/>
<path class="flow" d="M522 589 C548 589 548 657 576 657" stroke="#fb7185" marker-end="url(#arrow-rose)"/>
<path class="flow" d="M679 602 L679 618" stroke="#a78bfa" marker-end="url(#arrow-slate)"/>
<path class="flow" d="M679 688 L679 704" stroke="#a78bfa" marker-end="url(#arrow-slate)"/>
<path class="flow" d="M774 743 C810 743 810 581 846 581" stroke="#22d3ee" marker-end="url(#arrow-cyan)"/>
<text x="790" y="713" class="label">allowed + protocol inspect</text>
<path class="flow" d="M1058 581 L1127 581" stroke="#22d3ee" marker-end="url(#arrow-cyan)"/>
<text x="1073" y="570" class="label">forward</text>
<path class="flow" d="M774 743 L846 730" stroke="#fb923c" marker-end="url(#arrow-amber)"/>
<text x="788" y="759" class="label">special host</text>
<path class="flow" d="M1058 730 L1127 730" stroke="#fb923c" marker-end="url(#arrow-amber)"/>
<path class="flow-dash" d="M774 743 C816 764 812 829 846 829" stroke="#fb7185" marker-end="url(#arrow-rose)"/>
<text x="790" y="803" class="label">deny wins</text>
<path class="flow-dash" d="M1058 829 L1127 829" stroke="#fbbf24" marker-end="url(#arrow-amber)"/>
<text x="1070" y="818" class="label">L4 DenialEvent</text>
<!-- Filesystem panel -->
<rect x="76" y="1016" width="178" height="74" rx="6" fill="#0f172a"/>
<rect x="76" y="1016" width="178" height="74" rx="6" fill="rgba(76, 29, 149, 0.4)" stroke="#a78bfa" stroke-width="1.5"/>
<text x="102" y="1046" class="box-title">Policy YAML</text>
<text x="102" y="1065" class="box-sub">filesystem_policy</text>
<text x="102" y="1080" class="box-note">read_only + read_write</text>
<rect x="306" y="1006" width="188" height="94" rx="6" fill="#0f172a"/>
<rect x="306" y="1006" width="188" height="94" rx="6" fill="rgba(6, 78, 59, 0.4)" stroke="#34d399" stroke-width="1.5"/>
<text x="332" y="1036" class="box-title">Supervisor Prelude</text>
<text x="332" y="1055" class="box-sub">load + validate policy</text>
<text x="332" y="1072" class="box-sub">enrich baseline paths</text>
<text x="332" y="1087" class="box-note">workdir and runtime material</text>
<rect x="554" y="1006" width="196" height="94" rx="6" fill="#0f172a"/>
<rect x="554" y="1006" width="196" height="94" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="582" y="1036" class="box-title">Landlock Prepare</text>
<text x="582" y="1055" class="box-sub">open PathFds as root</text>
<text x="582" y="1072" class="box-sub">build ruleset</text>
<text x="582" y="1087" class="box-note">best_effort or hard_requirement</text>
<rect x="806" y="1006" width="190" height="94" rx="6" fill="#0f172a"/>
<rect x="806" y="1006" width="190" height="94" rx="6" fill="rgba(30, 41, 59, 0.5)" stroke="#94a3b8" stroke-width="1.5"/>
<text x="834" y="1036" class="box-title">pre_exec Child</text>
<text x="834" y="1055" class="box-sub">enter netns/mount ns</text>
<text x="834" y="1072" class="box-sub">drop to sandbox user</text>
<text x="834" y="1087" class="box-note">no supervisor identity socket</text>
<rect x="1050" y="1006" width="196" height="94" rx="6" fill="#0f172a"/>
<rect x="1050" y="1006" width="196" height="94" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="1078" y="1036" class="box-title">Enforce Controls</text>
<text x="1078" y="1055" class="box-sub">restrict_self()</text>
<text x="1078" y="1072" class="box-sub">seccomp filter</text>
<text x="1078" y="1087" class="box-note">raw socket and privilege paths blocked</text>
<rect x="1278" y="1006" width="154" height="94" rx="6" fill="#0f172a"/>
<rect x="1278" y="1006" width="154" height="94" rx="6" fill="rgba(8, 51, 68, 0.4)" stroke="#22d3ee" stroke-width="1.5"/>
<text x="1306" y="1036" class="box-title">Exec Agent</text>
<text x="1306" y="1055" class="box-sub">restricted runtime</text>
<text x="1306" y="1072" class="box-note">undeclared paths denied</text>
<rect x="242" y="1134" width="200" height="48" rx="6" fill="#0f172a"/>
<rect x="242" y="1134" width="200" height="48" rx="6" fill="rgba(8, 51, 68, 0.4)" stroke="#22d3ee" stroke-width="1.5"/>
<text x="266" y="1163" class="box-title">Read-only paths</text>
<text x="266" y="1178" class="box-note">/usr, /lib, /etc, /app...</text>
<rect x="476" y="1134" width="200" height="48" rx="6" fill="#0f172a"/>
<rect x="476" y="1134" width="200" height="48" rx="6" fill="rgba(6, 78, 59, 0.4)" stroke="#34d399" stroke-width="1.5"/>
<text x="500" y="1163" class="box-title">Read-write paths</text>
<text x="500" y="1178" class="box-note">/sandbox, /tmp, workdir...</text>
<rect x="710" y="1134" width="200" height="48" rx="6" fill="#0f172a"/>
<rect x="710" y="1134" width="200" height="48" rx="6" fill="rgba(120, 53, 15, 0.3)" stroke="#fbbf24" stroke-width="1.5"/>
<text x="734" y="1163" class="box-title">Runtime enrichment</text>
<text x="734" y="1178" class="box-note">proxy support, GPU devices</text>
<rect x="944" y="1134" width="200" height="48" rx="6" fill="#0f172a"/>
<rect x="944" y="1134" width="200" height="48" rx="6" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185" stroke-width="1.5"/>
<text x="968" y="1163" class="box-title">Blocked by default</text>
<text x="968" y="1178" class="box-note">undeclared filesystem paths</text>
<path class="flow" d="M254 1053 L297 1053" stroke="#a78bfa" marker-end="url(#arrow-slate)"/>
<path class="flow" d="M494 1053 L545 1053" stroke="#34d399" marker-end="url(#arrow-emerald)"/>
<path class="flow" d="M750 1053 L797 1053" stroke="#fb7185" marker-end="url(#arrow-rose)"/>
<path class="flow" d="M996 1053 L1041 1053" stroke="#94a3b8" marker-end="url(#arrow-slate)"/>
<path class="flow" d="M1246 1053 L1269 1053" stroke="#22d3ee" marker-end="url(#arrow-cyan)"/>
<path class="flow-thin" d="M400 1100 C400 1118 342 1118 342 1128" stroke="#22d3ee" marker-end="url(#arrow-cyan)"/>
<path class="flow-thin" d="M400 1100 C400 1120 576 1118 576 1128" stroke="#34d399" marker-end="url(#arrow-emerald)"/>
<path class="flow-thin" d="M400 1100 C400 1126 810 1118 810 1128" stroke="#fbbf24" marker-end="url(#arrow-amber)"/>
<path class="flow-thin" d="M1148 1100 C1148 1124 1044 1120 1044 1128" stroke="#fb7185" marker-end="url(#arrow-rose)"/>
<!-- Legend below all boundaries -->
<rect x="284" y="1238" width="930" height="70" rx="8" fill="rgba(15, 23, 42, 0.76)" stroke="rgba(148, 163, 184, 0.28)" stroke-width="1.2"/>
<text x="306" y="1264" class="legend">Legend</text>
<rect x="306" y="1280" width="16" height="10" rx="2" fill="rgba(8, 51, 68, 0.4)" stroke="#22d3ee"/>
<text x="332" y="1289" class="legend">client / protocol inspection</text>
<rect x="510" y="1280" width="16" height="10" rx="2" fill="rgba(6, 78, 59, 0.4)" stroke="#34d399"/>
<text x="536" y="1289" class="legend">gateway or supervisor control</text>
<rect x="744" y="1280" width="16" height="10" rx="2" fill="rgba(76, 29, 149, 0.4)" stroke="#a78bfa"/>
<text x="770" y="1289" class="legend">policy engine / policy data</text>
<rect x="956" y="1280" width="16" height="10" rx="2" fill="rgba(136, 19, 55, 0.4)" stroke="#fb7185"/>
<text x="982" y="1289" class="legend">security enforcement / deny path</text>
</svg>
</section>
<section class="cards" aria-label="diagram notes">
<article class="card">
<div class="card-header">
<span class="card-dot cyan"></span>
<h3>Network Isolation</h3>
</div>
<ul>
<li>Traffic is configured for proxy mode in sandbox policy conversion.</li>
<li>The child receives proxy environment variables before exec.</li>
<li>Network namespace and nftables reject direct TCP/UDP bypass attempts.</li>
</ul>
</article>
<article class="card">
<div class="card-header">
<span class="card-dot rose"></span>
<h3>Filesystem Isolation</h3>
</div>
<ul>
<li>Landlock PathFds are opened while the supervisor still has root privileges.</li>
<li>The child drops to the sandbox user before Landlock and seccomp enforcement.</li>
<li>Read-only, read-write, workdir, and enriched runtime paths define the visible filesystem.</li>
</ul>
</article>
<article class="card">
<div class="card-header">
<span class="card-dot amber"></span>
<h3>Dynamic Policy</h3>
</div>
<ul>
<li>Network policy can reload for new connections or safely parsed HTTP requests.</li>
<li>Filesystem and process controls are startup-time controls.</li>
<li>L4 denials can become pending gateway policy proposals.</li>
</ul>
</article>
</section>
<footer>
References inspected: docs/about/how-it-works.mdx, architecture/sandbox.md, architecture/security-policy.md, crates/openshell-policy/src/lib.rs, crates/openshell-sandbox/src/policy.rs, crates/openshell-sandbox/src/process.rs, crates/openshell-sandbox/src/sandbox/linux/landlock.rs, crates/openshell-sandbox/src/sandbox/linux/netns.rs, crates/openshell-sandbox/src/sandbox/linux/seccomp.rs, crates/openshell-sandbox/src/sandbox/linux/nft_ruleset.rs.
</footer>
</main>
</body>
</html>