mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-26 06:52:57 +08:00
docs(rfc): simplify isolation backend contract
This removes a lot of detail to keep the RFC succinct.
This commit is contained in:
@@ -12,42 +12,38 @@ links:
|
||||
- https://github.com/NVIDIA/OpenShell/issues/1680
|
||||
---
|
||||
|
||||
# RFC 0012 - Isolation Backend interface
|
||||
# RFC 0012 - Isolation Backend Interface
|
||||
|
||||
## Summary
|
||||
|
||||
OpenShell runs untrusted agent code inside an isolation boundary: the network, filesystem, syscall, and identity constraints that decide what the agent can reach and what can reach it. Building that boundary takes privilege, and today that privilege lives inside the agent's own container, beside the code it is meant to confine. This RFC makes the boundary a pluggable component, the **Isolation Backend**, separating the privileged work that builds it from the supervisor that operates it.
|
||||
OpenShell confines untrusted agent code with network, filesystem, syscall, and identity controls. Today the supervisor builds that boundary from the agent's container, placing isolation privilege beside the code it confines. This RFC makes the boundary a pluggable **Isolation Backend**.
|
||||
|
||||
The backend is neither a compute driver nor the supervisor. The driver still provisions the sandbox; the supervisor stays the policy authority (proxy, policy, audit) and the agent's only egress. The backend is the new piece between them: it establishes and enforces the boundary across all four dimensions and resolves who is behind each connection. The supervisor drives it through one runtime contract that is the same no matter where the boundary sits or which implementation provides it.
|
||||
The driver still provisions the sandbox. The supervisor remains the policy authority, audit source, and agent's only egress. The backend establishes and operates the boundary and supplies connection identity through one supervisor-facing contract.
|
||||
|
||||
Isolation Backends allow the privileged setup to move out of the agent's container: a sidecar, a separate pod, a microVM, a node component, and eventually outside the agent's kernel. Each placement is a new backend, not a new supervisor.
|
||||
That contract supports today's in-pod boundary and future sidecar, node, split-pod, and VM implementations. Each delegated placement is separate design and implementation work, not a supervisor fork.
|
||||
|
||||
## Motivation
|
||||
|
||||
An agent runs untrusted code, so OpenShell confines it behind an isolation boundary: the constraints that control what the agent can reach and what can reach it. Building that boundary takes privilege, and today the supervisor builds it from inside the agent's own container. So the privilege that builds the boundary sits inside the container the boundary is meant to confine.
|
||||
Boundary construction is embedded in the supervisor, so moving it anywhere else means changing the supervisor. One inline choice creates three problems:
|
||||
|
||||
That privilege is stuck there because building the boundary is written into the supervisor, so moving the boundary anywhere else means changing the supervisor. One inline choice creates three problems:
|
||||
|
||||
- A compromise reaches the privilege beside it. The boundary-building capabilities sit in the agent's own container, inside the blast radius of the untrusted code, so the machinery meant to contain the agent is reachable from the code it confines.
|
||||
- A compromise reaches the boundary-building privilege in the same container.
|
||||
- Restricted and multi-tenant clusters reject the deployment. Building the boundary inline needs added Linux capabilities in the agent container (seven today, including `NET_ADMIN` and `SYS_ADMIN`); the `restricted` Pod Security Standards profile those clusters require permits no added capability beyond `NET_BIND_SERVICE`, and five of the seven (`NET_ADMIN`, `SYS_ADMIN`, `SYS_PTRACE`, `SYSLOG`, `DAC_READ_SEARCH`) are off the `baseline` allowlist too, so the deployment fails `baseline` as well. (Tracked in #899.)
|
||||
- Each new placement forks the supervisor. The microVM driver and the outer gVisor sandbox each wire their own privileged setup into the supervisor, and it accretes a branch per placement.
|
||||
- Each new placement adds another branch to the supervisor.
|
||||
|
||||
All three trace to the same coupling: building the boundary and operating it are the same code. Separating them lets the privileged build move out of the agent's container, and lets a new placement be a new backend, not another branch in the supervisor. That calls for an interface, not a fixed set of placement modes. New placements keep arriving (sidecar, node component, VM, separate pod), and adding one should not require changing core OpenShell.
|
||||
All three come from coupling boundary construction to boundary operation. A common interface lets deployments move privilege without changing the supervisor.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Implementing any particular delegated backend.** This RFC defines the common contract. The sidecar, node, split-pod, gVisor, microVM, and other placements are separate implementation work, each settled in its own design.
|
||||
- **The mechanism for confirming a boundary the supervisor cannot observe.** When a backend builds the boundary where the supervisor cannot read it, how it attests the boundary is that backend's design, not this RFC's. The contract still requires that confirmation to be bound to the admitted backend and sandbox (see The boundary descriptor).
|
||||
- **The global authorization model.** The supervisor authenticates to the control plane today, but nothing scopes what an authenticated supervisor may do to its own sandbox. A separate proposal owns that scoping. This RFC defines the boundary and identity it would operate on, so it does not block on it; a delegated backend must still authenticate its caller and scope its operations to its own boundary.
|
||||
- **One-to-N supervisors.** This RFC keeps one supervisor per agent; sharing a supervisor across sandboxes is deferred (see Implementation plan).
|
||||
- **Implementing a delegated backend.** Each placement requires its own design and implementation.
|
||||
- **Defining remote confirmation.** This contract requires confirmation; each cross-kernel backend defines its attestation mechanism.
|
||||
- **Defining global authorization.** A separate proposal owns supervisor authorization. A delegated backend must still authenticate callers and scope them to one boundary.
|
||||
- **Sharing supervisors.** This RFC keeps one supervisor per agent.
|
||||
|
||||
## Proposal
|
||||
|
||||
Make the boundary a pluggable component the supervisor drives. We call it the **Isolation Backend**. The backend establishes and enforces the boundary and resolves who is behind each connection; the supervisor operates it as the policy authority (proxy, policy, audit). Moving the privileged establish-and-enforce work into the backend lets the agent container shed its isolation privilege.
|
||||
This RFC standardizes backend selection, lifecycle ordering, runtime interfaces, failure behavior, and invariants. How a backend constructs its boundary or transports these operations is implementation-specific.
|
||||
|
||||
This RFC standardizes backend selection, lifecycle ordering, the runtime-operated interfaces, failure behavior, and the backend invariants. The mechanisms behind those behaviors, and the full design of any particular delegated backend, are implementation-specific and settled in the RFC that introduces each one.
|
||||
|
||||
The work splits across two planes. At the control plane, the gateway and driver provision the sandbox when it is created. At the data plane, the supervisor operates the boundary while the agent runs, driving the backend through one runtime contract: the only interface between the supervisor and the backend, identical wherever the boundary sits.
|
||||
The gateway and driver provision the sandbox on the control plane. The supervisor drives the backend while the agent runs.
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
@@ -84,25 +80,21 @@ graph TB
|
||||
style Egress fill:#a5d6a7,stroke:#81c784,color:#00084D
|
||||
```
|
||||
|
||||
The supervisor straddles the boundary: the agent is contained inside, and the supervisor's proxy is its only egress.
|
||||
|
||||
This RFC defines the contract: its calls, their order, the failure categories a caller must handle (boundary unavailable, confirmation failed, attachment denied), and the agent lifecycle the warm-pool path depends on.
|
||||
The agent is contained behind the boundary; the supervisor's proxy is its only egress.
|
||||
|
||||
### Provisioning the boundary
|
||||
|
||||
Provisioning runs at sandbox creation, on the control plane, in a fixed order:
|
||||
|
||||
1. **Admission resolves the backend** from deployment config, not `SandboxPolicy`, so a workload cannot pick weaker isolation for itself. It fails closed if the backend cannot be realized and forbids silent downgrade. Different workload classes may resolve to different backends, an admission choice, not an interface change.
|
||||
2. **The driver provisions the compute resource:** whatever must exist before the supervisor runs, such as the agent container and the supervisor, a sidecar, a separate supervisor pod, a microVM, or a claim on a node daemon.
|
||||
3. **The named backend establishes the boundary before the agent runs.** Backends vary on one axis: where. The in-pod backend establishes in the supervisor's process, lazily, when the supervisor attaches to it. A backend that moves privilege out of the agent's container establishes elsewhere: a node daemon installs the netns routing and rules during the supervisor's `attach`, a VM driver brings up the guest at provisioning. Either way the supervisor only drives the contract; it never establishes the boundary itself.
|
||||
1. **Admission resolves the backend** from deployment config, not `SandboxPolicy`. Resolution fails closed; there is no downgrade.
|
||||
2. **The driver provisions the compute resource** and anything the selected backend needs before the supervisor starts.
|
||||
3. **The backend establishes the boundary before untrusted code runs.** This may happen during provisioning or `attach`, depending on the backend.
|
||||
|
||||
The driver and supervisor meet at two points: the driver provisions the resource and sets the descriptor; the supervisor reads the descriptor and drives the contract. Any driver and any backend plug in there without the supervisor changing. One establishment rule holds across placements: wherever a backend's preparation runs, during provisioning or during `attach`, it completes before the boundary is `Ready` and before any untrusted execution.
|
||||
|
||||
How a backend establishes its boundary is the part the contract does not cover, because that is where placements differ; each backend settles its own establishment in the RFC that introduces it. For a backend that establishes off-supervisor, that RFC also carries the security-critical work outside this interface: for a node daemon serving many sandboxes, admitting requests, isolating one sandbox's boundary from another, and reclaiming a boundary whose supervisor died.
|
||||
The driver provisions resources and sets the descriptor; the supervisor reads it and drives the contract. Each backend design owns its construction, authorization, tenant isolation, and orphan reclamation.
|
||||
|
||||
### The boundary descriptor
|
||||
|
||||
The supervisor learns which backend to instantiate, and how to attach, from a **boundary descriptor** the driver always sets in the `OPENSHELL_BOUNDARY_DESCRIPTOR` environment variable. It names the backend explicitly, in-pod included, and for a backend established off-supervisor it also carries how to attach. It is integrity-protected and bound to what provisioning admitted, so the supervisor cannot be pointed at a forged boundary. A missing, unreadable, or unverifiable descriptor fails the sandbox closed: there is no default, because defaulting would silently select a backend the deployment did not choose. The supervisor never passes it to the workload. The envelope is common and small, with a single canonical, size-bounded, versioned encoding (the wire format is settled in the implementing PR); only the payload is the backend's:
|
||||
The driver sets `OPENSHELL_BOUNDARY_DESCRIPTOR` for every sandbox, including in-pod. The common, size-bounded envelope names the backend and carries an opaque attachment payload. Common code retrieves trusted requirements from the authenticated admission record, verifies the descriptor against them, and constructs `VerifiedBoundaryDescriptor`; the factory then validates payload semantics and any endpoint authentication, freshness, or replay requirements. Missing or invalid input fails closed with no default backend. The supervisor does not pass the descriptor to the workload. All drivers and the supervisor use the same versioned boundary descriptor representation.
|
||||
|
||||
```rust
|
||||
struct BoundaryDescriptor {
|
||||
@@ -111,29 +103,28 @@ struct BoundaryDescriptor {
|
||||
payload: Vec<u8>, // backend-specific attachment data
|
||||
}
|
||||
|
||||
// The trusted expectations from admission, carried by the integrity-protected
|
||||
// provisioning envelope and retained by VerifiedBoundaryDescriptor.
|
||||
// Trusted expectations from the authenticated admission record, retained by
|
||||
// VerifiedBoundaryDescriptor after common verification.
|
||||
struct AdmittedBoundaryRequirements {
|
||||
backend_id: String,
|
||||
contract_version: u32,
|
||||
policy_digest: Hash,
|
||||
required_capabilities: RequiredCapabilities,
|
||||
required_confirmation: ConfirmationKind,
|
||||
minimum_identity: Assurance,
|
||||
}
|
||||
```
|
||||
|
||||
Admission supplies the trusted expectations as an authenticated `AdmittedBoundaryRequirements` block, carried by the integrity-protected provisioning envelope and retained by `VerifiedBoundaryDescriptor`: the backend id, the contract version, a policy digest, and the required capabilities. The supervisor checks the descriptor and the factory against them, and a backend's self-reported capabilities may only match or exceed them, never lower them. Common code authenticates the whole envelope, and only that verification mints a `VerifiedBoundaryDescriptor`, which has no public constructor, so an unverified descriptor cannot reach a factory; the factory then validates payload semantics, endpoint authentication, freshness, and replay protection. The envelope `version` is distinct from the contract version the backend speaks (a field of `BackendCapabilities`).
|
||||
The envelope version and backend contract version are distinct. Only common verification can construct `VerifiedBoundaryDescriptor`, which has no public constructor. An exact contract version implies support for every field of that `SandboxPolicy` version; confirmation kind and identity assurance are the only negotiated security capabilities.
|
||||
|
||||
The supervisor does not re-check the boundary's contents directly; a backend may build it where the supervisor cannot read. But every backend MUST confirm the boundary before returning `Ready`, and fail closed if it cannot. What varies is the evidence, not whether confirmation happens: a backend that shares the supervisor's kernel confirms effective enforcement directly, not the mere presence of rules; one that does not returns an attestation the supervisor verifies. `ConfirmationCapability` names that mechanism. A backend whose boundary can change after startup MUST keep confirming for the boundary's lifetime and fail the sandbox on drift.
|
||||
Every backend confirms effective enforcement before returning `Ready`. A local backend may observe it directly; a remote backend may return verified attestation. Remote evidence is fresh, bound to the verified descriptor, authenticated claim, and admitted policy digest, and verified against a trust root outside the workload's adversary domain; its protocol remains backend-specific. Mutable enforcement remains confirmed for the boundary's lifetime, and drift fails the sandbox.
|
||||
|
||||
The descriptor settles **boundary attachment** (how the supervisor attaches to the boundary) but not **identity binding**, which sandbox this is (the `sandbox_id`). They are different exchanges, and warm pooling pulls them apart. A warm-pooled pod is created before any sandbox exists (#1892): its boundary is established at boot against the image's baseline policy, identity-free, before any claim. The supervisor learns its `sandbox_id` later, when a claim binds, through a separate control-plane handoff: in the Kubernetes flow it presents its projected ServiceAccount token to the gateway, which re-anchors it through the pod's owning records and mints the sandbox JWT. That handoff is local while the supervisor shares the pod, and it is the exchange that must cross the boundary when the supervisor is delegated. Because the baseline policy is fixed at boot, a workload needing a custom policy cannot be served from the pool; it takes the cold path.
|
||||
|
||||
A pooled pod runs an OpenShell-controlled placeholder in this gap, not the workload, so it can hold a ready boundary before it holds a sandbox. The placeholder is trusted because its executable is OpenShell-controlled content, not because of its lifecycle position: a binary named `sleep` from the agent's own image would not qualify. The attribution property that keeps that safe is stated with the invariants below.
|
||||
Attachment and sandbox identity binding are separate. A warm-pooled boundary is established against its admitted image baseline before a sandbox exists; `claim` later binds the sandbox, policy, agent, and resources. An incompatible claim takes the cold path. Before claim, only digest-pinned OpenShell-controlled placeholder code may run.
|
||||
|
||||
### The runtime contract
|
||||
|
||||
The boundary spans four dimensions: network, filesystem, syscall, and identity. The in-pod backend realizes them as a network namespace and routing, Landlock, seccomp, and procfs; other backends realize them differently. The contract adds no enforcement of its own; it standardizes how the supervisor drives whichever backend a deployment admits.
|
||||
|
||||
A backend registers a factory under a `backend_id`. The supervisor resolves the factory, attaches to the admitted boundary, and drives it through a fixed sequence of states. Each transition consumes the prior state, and the state types have no public constructor, so no stage can be skipped or replayed. The Rust names below are illustrative; the states and their semantics are normative.
|
||||
A backend registers a factory under a `backend_id`. The supervisor resolves the factory, attaches to the admitted boundary, and drives it through a fixed sequence of states. Each transition consumes the prior state, so the supervisor cannot skip a stage or invoke a later transition through an earlier handle. The Rust names below are illustrative; the states and their semantics are normative.
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
@@ -144,8 +135,15 @@ trait IsolationBackendFactory: Send + Sync {
|
||||
-> Result<Box<dyn AttachedBoundary>, BackendError>;
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
trait BoundaryControl: Send + Sync { // retained by the supervisor after attach
|
||||
async fn wait_terminated(&self) -> Result<(), BackendError>;
|
||||
async fn shutdown(&self) -> Result<(), BackendError>; // idempotent
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
trait AttachedBoundary: Send { // the admitted boundary; no claimed or untrusted workload
|
||||
fn control(&self) -> Arc<dyn BoundaryControl>;
|
||||
async fn claim(self: Box<Self>, claim: ClaimContext) -> Result<Box<dyn ClaimedBoundary>, BackendError>;
|
||||
}
|
||||
#[async_trait]
|
||||
@@ -166,12 +164,8 @@ trait ReadyBoundary: Send { // enforcement and mediation confirmed
|
||||
#[async_trait]
|
||||
trait RunningBoundary: Send + Sync { // the agent is running behind the boundary
|
||||
fn agent(&self) -> Arc<dyn BoundaryProcess>;
|
||||
fn identity_source(&self) -> Arc<dyn IdentitySource>;
|
||||
fn exec(&self) -> Arc<dyn BoundaryExec>;
|
||||
fn port_forward(&self) -> Arc<dyn BoundaryPortForward>;
|
||||
fn events(&self) -> Arc<dyn EventSource>;
|
||||
async fn wait_terminated(&self) -> Result<(), BackendError>; // boundary-level termination, distinct from agent exit
|
||||
async fn shutdown(&self) -> Result<(), BackendError>; // idempotent release of delegated resources
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
@@ -182,19 +176,17 @@ trait BoundaryProcess: Send + Sync { // the agent, or a process started via exe
|
||||
}
|
||||
```
|
||||
|
||||
| State | Meaning | Permitted workload operations |
|
||||
| State | Meaning | Available to the supervisor |
|
||||
|---|---|---|
|
||||
| **Attached** | the admitted boundary is attached; no claimed or untrusted workload | none |
|
||||
| **Claimed** | bound to sandbox identity, policy, agent spec, and resources | none |
|
||||
| **Bound** | mediation interfaces are connected | identity and events only |
|
||||
| **Ready** | enforcement and mediation are confirmed | agent start only |
|
||||
| **Running** | the agent is started behind the boundary | `exec`, `connect`, `wait`, `signal` |
|
||||
| **Attached** | the admitted boundary is attached; no claimed or untrusted workload | boundary control |
|
||||
| **Claimed** | bound to sandbox identity, policy, agent spec, and resources | boundary control |
|
||||
| **Bound** | mediation interfaces are connected | control, mediation ingress, identity, events |
|
||||
| **Ready** | enforcement and mediation are confirmed | control, agent start |
|
||||
| **Running** | the agent is started behind the boundary | control, retained mediation handles, agent, `exec`, `connect` |
|
||||
|
||||
The ordering is the contract's security property, held by construction. `attach` resolves the admitted boundary without binding a sandbox; `claim` is the only transition that introduces sandbox identity, policy, agent spec, and resources, and a pooled boundary rejects a claim incompatible with its baseline policy. The cold path claims immediately; a warm-pooled boundary may sit `Attached` until a claim binds (the attachment-versus-identity split above). `start_agent` always returns a running process handle once the agent has started, and `wait` returns its exit status, identically for every backend; `exec` and `connect` do not exist before `Running`. A delegated backend answers these calls over its own transport, but the transport cannot change their meaning.
|
||||
The state order is structural: `attach` identifies the admitted boundary, `claim` introduces sandbox identity and workload requirements, only `ReadyBoundary` may start the agent, and only `RunningBoundary` exposes `exec` and `connect`. A pooled boundary may remain `Attached` until claimed. The supervisor retains `BoundaryControl` and owned runtime handles across consuming transitions.
|
||||
|
||||
The accessors return owned `Arc` handles on purpose: the supervisor keeps a runtime interface while the state object is consumed by the next transition. Those interfaces keep their roles: `IdentitySource` resolves who is behind a connection for the proxy (see Identity); `BoundaryExec` and `BoundaryPortForward` run `exec` and `connect` (loopback-scoped, since a non-loopback target opens an egress the proxy would otherwise mediate) for the SSH server and supervisor sessions; `EventSource` is one stream the orchestrator drains for denials, activity, bypass, and termination, and security events must not be dropped silently under backpressure. A denial event carries the connection's attribution at `Evidence` fidelity (the binary, its ancestor chain, and cmdline paths) plus the request's L7 method and path, not just host and port, because the orchestrator's policy-proposal path consumes it. The event payload and the identity model share one attribution shape, so a delegated backend forwarding events preserves that fidelity rather than collapsing a denial to an endpoint.
|
||||
|
||||
The three runtime-operated interfaces are small and owned:
|
||||
The runtime interfaces are small and owned:
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
@@ -234,16 +226,14 @@ trait BoundaryTerminal: Send + Sync {
|
||||
}
|
||||
```
|
||||
|
||||
The proxy obtains every workload connection through `MediationIngress::accept`, which yields a `MediatedConnection` (the connection's stream plus the `flow` token it resolves to an identity), identically for every backend: in-pod wraps its local accept inside the netns, a delegated backend pulls from its private transport. `ExecSpec` carries the command, args, environment, working directory, and a PTY request; `BoundaryTerminal` resizes a requested PTY; stdin, stdout, and stderr are owned async byte streams, and stdout and stderr stay distinct for non-PTY exec. `LoopbackTarget` is a validated loopback address, since a non-loopback target opens an egress the proxy would otherwise mediate. An `EventSource` is single-consumer: a second `subscribe` returns an error rather than a silently empty stream, and event loss and boundary termination are explicit, never silent. As in `BoundaryProcess`, exit status and signals are placement-neutral, so a local PID is never the process handle and a repeated `wait` returns one stable result.
|
||||
`MediationIngress` is the proxy's backend-neutral source of workload connections. In-pod wraps the proxy's local listener; delegated backends use their private transport. `ExecSpec` carries command, arguments, environment, working directory, and PTY settings. Streams are owned, non-PTY stdout and stderr remain separate, and PTYs support resize. Port forwarding accepts only validated loopback targets. Event loss, exit status, and signals are explicit and placement-neutral; a local PID is never the process handle. `BoundaryControl::wait_terminated` reports boundary loss separately from agent exit.
|
||||
|
||||
`MediationIngress`, the admission-requirements binding below, and the claim generation are contract additions the in-pod reference backend does not implement yet; the in-pod proxy still accepts directly. They are part of the contract so a delegated backend has them; adopting them in-pod is behavior-preserving refactor work.
|
||||
`MediationIngress`, `BoundaryControl`, and admission binding are contract additions not yet present in the in-pod reference implementation. Adopting them there is behavior-preserving refactor work.
|
||||
|
||||
The claim carries only the common content every backend needs:
|
||||
|
||||
```rust
|
||||
struct ClaimContext {
|
||||
claim_id: ClaimId, // single-use; verified against the admission record
|
||||
generation: u64, // a stale or replayed claim is rejected
|
||||
sandbox_id: SandboxId,
|
||||
policy: SandboxPolicy, // all four dimensions
|
||||
agent: AgentSpec,
|
||||
@@ -251,83 +241,79 @@ struct ClaimContext {
|
||||
}
|
||||
```
|
||||
|
||||
`claim` verifies every field against the same descriptor and admission record and the policy digest before it succeeds; the `claim_id` and `generation` make the claim single-use, so a stale or replayed claim is rejected. `ResourceBinding` is versioned, opaque data the compute driver issues and binds to this backend, boundary, sandbox, and allocation: the cgroup, runtime security context, and device set. `claim` validates it; the agent and every `exec` process run within it and cannot widen it; allocating those resources stays the compute driver's job. A backend's placement feeds admission and audit, never per-connection policy, so a policy decision is identical across placements.
|
||||
|
||||
A backend swap replaces the factory and the state implementations; the consumer code above them (the proxy, the SSH server, and the orchestrator) does not change, because each depends on a runtime interface, not on the backend.
|
||||
`claim` verifies these fields against the descriptor, admission record, and policy digest. A claim is authenticated, fresh, single-use, and bound to the boundary and sandbox; the backend design defines that mechanism. `ResourceBinding` is opaque, versioned driver data for the admitted cgroup, runtime security context, and devices. The agent and every `exec` descendant remain within it.
|
||||
|
||||
### Supervisor integration
|
||||
|
||||
The supervisor keeps a registry from `backend_id` to factory. Adding a backend adds an implementation and a registration; it does not add backend-specific branches to the supervisor's lifecycle, proxy, SSH, or session code. This is an in-tree registry, not dynamic out-of-tree loading, and the contract is not a universal delegated wire protocol: a delegated backend ships its own authenticated transport behind the same factory, consistent with RFC 0001, which makes pluggable drivers gRPC services rather than in-process traits.
|
||||
The supervisor keeps an in-tree registry from `backend_id` to factory. Adding a backend adds an implementation and registration, not branches in the lifecycle, proxy, SSH, or session code. Delegated transports remain private to their factories.
|
||||
|
||||
The supervisor runs the same sequence for every backend:
|
||||
|
||||
1. Read and validate the descriptor envelope.
|
||||
2. Match its `backend_id` to admission.
|
||||
3. Resolve the registered factory and check its capabilities.
|
||||
4. Attach to the boundary.
|
||||
4. Attach to the boundary and retain its `BoundaryControl`.
|
||||
5. Claim the sandbox identity, policy, agent spec, and resources.
|
||||
6. Bind identity and events into the proxy and orchestrator.
|
||||
6. Bind mediation ingress and identity into the proxy, and events into the orchestrator.
|
||||
7. Start mediation services.
|
||||
8. Confirm readiness.
|
||||
9. Start the agent.
|
||||
10. Enable `exec` and port forwarding.
|
||||
11. Wait for the agent or the boundary to terminate.
|
||||
12. Call the idempotent `shutdown` to release the boundary.
|
||||
11. Wait for the agent or `BoundaryControl` to terminate.
|
||||
12. Call `BoundaryControl::shutdown`.
|
||||
13. Surface the result for platform-specific cleanup.
|
||||
|
||||
A backend reports what it can do, so admission can reject a workload it cannot place:
|
||||
|
||||
```rust
|
||||
enum ConfirmationKind { Direct, Attested }
|
||||
|
||||
struct BackendCapabilities {
|
||||
backend_id: String,
|
||||
contract_version: u32,
|
||||
placement: BackendPlacement,
|
||||
confirmation: ConfirmationCapability,
|
||||
identity: Vec<IdentityEvidenceKind>,
|
||||
confirmation: Vec<ConfirmationKind>,
|
||||
maximum_identity: Assurance,
|
||||
}
|
||||
```
|
||||
|
||||
Capability comparison is exact: the contract version must match, the backend's capability set must include every capability the policy requires, and its confirmation and identity levels must meet the admitted minimums. Admission resolves requirements against a deployment capability catalog; the supervisor rechecks them against the runtime factory. The backend identity must agree across admission, the descriptor, and the factory; any shortfall fails closed.
|
||||
The backend ID and exact contract version must agree across admission, descriptor, and factory. The backend must support the required confirmation kind and minimum identity assurance. Any mismatch fails closed. All other contract behavior is mandatory conformance, not a negotiable capability.
|
||||
|
||||
### Failure semantics
|
||||
|
||||
A failure is any case where the supervisor cannot obtain or apply a valid result. The contract fixes the behavior; backend-specific retry limits and transport recovery are implementation details. Every failure carries a machine-readable kind:
|
||||
|
||||
```rust
|
||||
enum BackendErrorKind {
|
||||
DescriptorInvalid, VersionMismatch, CapabilityUnsupported, AttachmentDenied,
|
||||
Unavailable, ClaimFailed, BindFailed, ConfirmationFailed,
|
||||
StartFailed, OperationFailed, BoundaryTerminated,
|
||||
}
|
||||
enum BackendErrorKind { Invalid, Denied, Unavailable, Failed, Terminated }
|
||||
```
|
||||
|
||||
`Invalid` covers descriptor, version, backend, and capability mismatches; `Denied` covers authenticated attachment or claim rejection. The failing operation and backend diagnostics remain structured context.
|
||||
|
||||
| Failure | Required behavior |
|
||||
|---|---|
|
||||
| Missing, malformed, unsupported, or mismatched descriptor | Fail closed; never select another backend |
|
||||
| Attachment denied or verification failed | Terminal for this sandbox instance |
|
||||
| Boundary unavailable during attachment | Retry the same backend only; never downgrade |
|
||||
| Confirmation failed | Do not start workload code; surface the failure for cleanup |
|
||||
| Claim, bind, start, or runtime operation failed | Terminal; fail closed and clean up |
|
||||
| Boundary terminates after start | Stop new `exec` and `connect`, and fail the sandbox |
|
||||
|
||||
Only attachment is automatically retryable, and a retry reuses the same backend and boundary. Later mutating transitions are not blindly retried: a backend reconciles a lost response against stable operation and claim identities, so a lost `start_agent` reply returns the already-started process rather than launching a second agent, and an outcome it cannot reconcile fails closed. A partially completed transition leaves the boundary cleanable and any delegated resource reclaimable. `wait_terminated` reports boundary-level termination distinct from agent exit, and `shutdown` is an idempotent release. Boundary failure disables new `exec` and `connect`, drains the workload execution domain, and permits delegated-resource reclamation; terminating the agent terminates its whole descendant tree, and an `exec` session owns the subprocess tree it starts.
|
||||
Only `Unavailable` from `attach` is automatically retryable, always against the same backend and boundary. Mutating transitions reconcile a lost response or fail closed; they never create a second claim or agent. Failed `attach` cleans itself. After attachment, the retained `BoundaryControl` owns idempotent cleanup on every terminal path, while driver-provisioned resources remain driver-owned; backend orphan reclamation covers supervisor death. Boundary failure disables new `exec` and `connect`, drains or securely quarantines the workload, and fails the sandbox. Agent and `exec` termination cover their complete descendant trees.
|
||||
|
||||
### Backend invariants
|
||||
|
||||
These define what it means to be a backend; a backend that cannot hold all five is not valid.
|
||||
Every backend satisfies these invariants:
|
||||
|
||||
1. **No unguarded workload egress.** The property begins before untrusted execution and holds until the execution domain is drained: the workload reaches no egress but the proxy. A backend enforces this with an effective default-deny across every protocol the platform supports (IPv4 and IPv6, all L4, raw and packet sockets, and ingress), not a default-accept ruleset that names a few protocols, and it confirms the effective behavior rather than the mere presence of rules. It must not assume the platform provides the deny: a NetworkPolicy is inert without an enforcing CNI, and the in-pod case holds only while the host does not forward the sandbox subnet. The in-pod backend does not satisfy this today (a known gap, see Risks) and does not ship until it does.
|
||||
2. **No untrusted workload execution before Ready.** No workload process runs inside the boundary until it is `Ready`. Workload code is defined by provenance, not lifecycle position: it includes the agent image, the workspace, mounted volumes, and anything an init step runs from them. Only digest-pinned, OpenShell-controlled content may run before the claim binds and the boundary is `Ready`. `start_agent` is gated structurally by the `Ready` state, and `exec` and `connect` exist only on `RunningBoundary`, so the pre-`Running` ordering is the Rust typestate. Rejecting an operation after the boundary has terminated is dynamic, enforced at the call rather than the type, because a held `RunningBoundary` reference cannot be revoked by the type system.
|
||||
3. **No unattributed workload execution.** Every agent, `exec`, and `connect` requires a bound claim and the corresponding lifecycle state, so no untrusted process runs without a sandbox identity. This keeps warm pooling safe: a pooled boundary holds an OpenShell-controlled placeholder, never untrusted code, until a claim binds. Because the agent, `exec`, and `connect` are reachable only through the claimed chain, the contract makes attribution structural; supervisor adoption must route every workload launch through the contract and remove the legacy bypass paths that still exist today, such as the SSH server execing directly.
|
||||
4. **Preserve the compute driver's execution domain.** The agent and every `exec` process, with their complete descendant trees, run within the admitted cgroup, runtime security context, and device allocation. A privileged helper sees the workload's devices only when an admission-visible named capability authorizes that helper and device and audit records it; there is no implicit justification. The contract carries the domain as `ResourceBinding`; allocating it stays the compute driver's responsibility.
|
||||
5. **No silent weakening of policy.** A backend implements the admitted semantics of every `SandboxPolicy` field across all four dimensions, or it rejects the workload at admission or claim. It never silently drops or weakens a network, filesystem, syscall, identity, or resource requirement; admission, not the backend, is the source of truth for what must hold.
|
||||
1. **No unguarded workload egress.** Before untrusted execution and until the execution domain is drained, the workload reaches no egress but the proxy. Enforcement is effective default-deny across every protocol the platform supports, including IPv4, IPv6, raw and packet sockets, and ingress. The backend confirms behavior, not merely installed rules, and does not assume a CNI or host route provides the deny. The current in-pod backend does not yet satisfy this release gate (see Risks).
|
||||
2. **No untrusted workload execution before `Ready`.** Workload code includes the agent image, workspace, mounts, and image-provided init code. Only digest-pinned OpenShell content may run before claim and `Ready`. Typestate enforces ordering before `Running`; calls made after dynamic termination fail closed.
|
||||
3. **No unattributed workload execution.** Agent start, `exec`, and `connect` require a bound claim. A warm pool runs only its trusted placeholder before claim. Supervisor adoption removes direct launch paths that bypass the contract.
|
||||
4. **Preserve the driver's execution domain.** Every workload descendant remains within the admitted cgroup, runtime security context, and device allocation carried by `ResourceBinding`. A privileged helper sees workload devices only through an admission-visible, audited authorization.
|
||||
5. **No silent weakening.** A backend implements every admitted `SandboxPolicy` field across network, filesystem, syscall, and identity, or rejects the workload at admission or claim.
|
||||
|
||||
These invariants hold for the boundary's lifetime, not just up to `Ready`: confirmed enforcement stays in effect until the execution domain is drained. Landlock and seccomp are monotonic, applied at process entry and never relaxed; netns routing and nftables are mutable, so the contract does not treat them as irreversible. What it requires instead: the authority to mutate enforcement is fenced from the workload, any update preserves default-deny atomically, and loss of the enforcement or control lease fails closed. Monitoring for drift does not by itself satisfy enforcement. If enforcement, identity, the event stream, or the backend control channel is lost, the supervisor rejects new `exec` and `connect`, stops or quarantines the workload, and fails the sandbox.
|
||||
|
||||
These invariants hold against application-level escapes, not kernel-level ones: in a shared-kernel placement a kernel exploit reaches the enforcement itself (see Representative topologies on how far enforcement sits from the agent). They describe what a contained agent cannot do; they do not promise survival of a kernel compromise.
|
||||
The invariants hold until the execution domain is drained. Landlock and seccomp are monotonic; mutable routing and firewall state stays fenced from the workload and preserves default-deny atomically. Loss of enforcement, identity, events, or backend control rejects new operations, drains or quarantines the workload, and fails the sandbox. Shared-kernel backends do not claim to contain a kernel compromise.
|
||||
|
||||
### Identity
|
||||
|
||||
`IdentitySource::resolve(flow)` answers who is behind a connection, so policy can scope rules to a binary. It is an interface injected into the proxy at `bind`, not a method the orchestrator calls: the proxy is the only consumer, and it calls `resolve` on every connection. It is capability-gated: a backend that cannot provide identity returns `Unsupported`, and admission rejects any policy that needs a level the backend cannot meet. `Unsupported` is a declared capability, the backend provides no identity for this boundary at all, distinct from a per-connection failure, which is a `ResolveError` that fails closed.
|
||||
The proxy calls `IdentitySource::resolve(flow)` for each connection. Admission rejects a backend below the policy's required assurance. `Unsupported` means the backend provides no identity; a per-connection `ResolveError` fails identity-scoped policy closed.
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
@@ -351,52 +337,46 @@ struct Evidence {
|
||||
enum Assurance { None, Claimed, Observed, Attested }
|
||||
```
|
||||
|
||||
`Observed` is the backend reading and hashing the binary itself, which it can do via procfs while it shares the agent's kernel. `Attested` is narrower than a signature alone: it is fresh evidence, bound to this boundary, flow, sandbox id, and claim id and generation, cryptographically verified against an observer and trust root outside the agent's adversary domain. A signed statement produced inside a compromised guest does not meet that bar and is not `Attested`. `Observed` and `Attested` both require a binary digest; without one the resolver returns an error or a lower assurance rather than asserting a level it cannot support. The `Evidence` schema is stable, so the source can change (procfs today, attestation later) without touching the policy layer; the first cross-kernel backend defines the concrete attestation producer and its verification.
|
||||
`Observed` means the backend identifies the process behind the accepted flow and measures its live executable; a digest is required. `Attested` preserves that flow attribution and verified live-executable digest, then adds fresh cryptographic evidence bound to the boundary and authenticated sandbox claim from a trust root outside the agent's adversary domain. Evidence below those definitions reports a lower assurance. Binary-scoped rules require `Observed` or higher; missing, ambiguous, stale, or failed resolution denies them.
|
||||
|
||||
The backend provides the resolver, injected at `bind`; the proxy is its only consumer. When the backend shares the agent's kernel (in-pod, sidecar, node enforcer) the resolver reads procfs; when it does not, the resolver answers from across the guest boundary. The proxy calls `resolve` the same way in both, so where identity comes from never reaches the proxy. The call is on the per-connection hot path, so a slow or failed lookup fails closed (deny the binary-scoped rule), and a remote resolver bounds it with a timeout. Today the proxy reads procfs inline; routing it through the backend's injected `IdentitySource` is a behavior-preserving step (impl-plan step 1). The contract is fixed now so that step, and later backends, do not reshape it.
|
||||
|
||||
The `flow` argument is the opaque, versioned token from the `MediatedConnection`: the backend issues it per connection and resolves it to the owning process, and the supervisor never interprets it. The in-pod backend keys it on the workload-side TCP peer port; a backend that carries identity across a kernel boundary defines its own token, bound to the attachment, namespace, and connection, including who may forge a reference. New token shapes arrive under the contract version, not by widening a shared type.
|
||||
`Flow` is the backend-issued, opaque token from `MediatedConnection`. The supervisor never interprets it. Procfs lookup, token construction, remote correlation, timeouts, and attestation protocols belong to backend implementations; the interface and fail-closed semantics do not change.
|
||||
|
||||
### Representative topologies
|
||||
|
||||
No single placement is universal. Infrastructure and workload determine the choice, and a deployment selects the one that matches its threat model. The interface covers all of them. They fall into three classes by where the supervisor's kernel sits relative to the agent's, and containment strength follows from that distance: the further the supervisor sits from the agent's kernel, the more a compromised agent must break to reach the enforcement that confines it.
|
||||
The security-relevant distinction is whether the supervisor shares the agent's kernel:
|
||||
|
||||
| Class | A compromise reaches enforcement only by | Examples |
|
||||
| Class | Kernel-compromise ceiling | Examples |
|
||||
|---|---|---|
|
||||
| **Shared host kernel**: supervisor and agent on the host kernel | a host-kernel exploit | in-pod, sidecar, node enforcer, `runc` split-pod |
|
||||
| **Shared guest kernel**: supervisor and agent share one isolated kernel, a VM guest kernel or a userspace application kernel such as gVisor's; the host is isolated, the supervisor is not | a guest-kernel exploit | single-pod microVM, outer gVisor sandbox |
|
||||
| **Kernel-separated**: supervisor outside the agent's guest kernel | escaping the agent's guest kernel | split-pod with the agent under a VM or gVisor `runtimeClassName`, future node runtime |
|
||||
|
||||
The table is a kernel-compromise ceiling, the strongest adversary each class still contains, not an exhaustive list of compromise paths (the trusted computing base below adds others). The first two classes share a kernel with the agent, so that ceiling is the floor: privilege separation, not isolation from a kernel-level adversary. Shared-host placements share that ceiling but differ in application-level privilege exposure, by where the boundary-building privilege sits, in the agent's container (in-pod) or out of it (sidecar, node enforcer, split-pod), not in containment strength. Kernel separation is the only class that contains a kernel-level adversary; it is the class that raises the floor.
|
||||
The first two classes do not contain a kernel-level adversary; sidecar, node, and split-pod placements may still add privilege separation within that ceiling. A single-pod VM or gVisor RuntimeClass places the supervisor inside the agent's guest or application kernel; kernel separation requires the supervisor outside it. In every class, the trusted computing base includes the supervisor, backend, and their control channel.
|
||||
|
||||
Kernel separation depends on where the supervisor runs relative to the agent, not on the RuntimeClass alone: a single-pod VM or gVisor RuntimeClass isolates the pod from the host but runs the supervisor inside the agent's isolated kernel (a VM guest kernel, or gVisor's application kernel), so they share it, while the same RuntimeClass with the supervisor in a separate pod puts it outside (kernel-separated).
|
||||
|
||||
The trusted computing base is the supervisor, the backend that enforces the boundary, and the control channel between them: the supervisor is the proxy, decides policy on identity, and emits audit. The boundary therefore falls to a compromise of any of the three, not only to a kernel exploit. In a shared-kernel placement the supervisor sits in the agent's blast radius, and moving it to its own kernel removes it from that radius.
|
||||
|
||||
A [selection matrix](./topology-matrix.md) maps a deployment's starting point to a recommendation and works through the design logic, including the one supervisor/agent kernel split that stock Kubernetes cannot express.
|
||||
The non-normative [selection matrix](./topology-matrix.md) compares concrete placements.
|
||||
|
||||
## Implementation plan
|
||||
|
||||
Backends arrive incrementally behind one contract, behavior-preserving first. The supervisor's runtime calls do not change between steps; only the backend behind them does.
|
||||
Backends arrive incrementally behind one contract:
|
||||
|
||||
1. **Land the contract, registry, in-pod adapter, and supervisor selection together.** Define the contract and the `backend_id` registry, put today's in-pod path behind it as the reference backend, and route supervisor selection through the registry, with no change in behavior or privilege, validated by the existing test suite.
|
||||
2. **Repair the in-pod release gates, then enable it.** Close the egress-confirmation gap (invariant 1) and the trusted-init window (see Risks) so the in-pod backend confirms its own boundary; it ships only once it does.
|
||||
3. **Move the privilege off the agent container (separate design).** Design and implement the sidecar-assisted backend against the unchanged contract, dropping `NET_ADMIN` from the agent container. This is the first real privilege reduction.
|
||||
4. **Validate a different placement (separate design).** Design and implement a node or split-pod backend, exercising the same contract from a placement that establishes off-supervisor.
|
||||
5. **Add kernel-separated backends (separate designs).** microVM, outer gVisor, and a future node runtime that runs the supervisor outside the agent's kernel, each with its own attestation and cross-kernel identity, prioritized by demand.
|
||||
1. **Land the contract and in-pod adapter.** Add the registry and explicit descriptors behind a rollout gate; preserve current behavior.
|
||||
2. **Close the in-pod release gates.** Repair egress confirmation and the trusted-init window, then route the supervisor through the adapter and enable it.
|
||||
3. **Add a sidecar backend (separate design).** Move network-setup privilege out of the agent container; on Kubernetes, map `boundary_ready` to a native sidecar `startupProbe` so agent startup waits for the boundary and closes the `workspace-init` window where sidecar containers are available.
|
||||
4. **Add a node or split-pod backend (separate design).** Validate establishment outside the supervisor.
|
||||
5. **Add kernel-separated backends as demand warrants (separate designs).** Each defines its attestation and cross-kernel identity.
|
||||
|
||||
Drivers emit an explicit in-pod descriptor before any supervisor requires one, so the rollout never breaks an existing sandbox and there is never a silent default backend. The supervisor stays 1:1 with the agent (a node-level backend may still serve many sandboxes on its node; that is expected), and a backend is admission-validated config, so each step lands without a breaking change. RFC acceptance approves the contract and this sequence; it does not assert that every backend exists. The RFC becomes `implemented` only after the in-pod reference backend passes its release gates (invariant 1 and the init path, see Risks) and runs enabled through the contract; the other backends may remain follow-on work.
|
||||
Drivers emit explicit in-pod descriptors before the supervisor requires them; there is no implicit default. RFC acceptance approves the contract, not the follow-on backends. The RFC becomes `implemented` when the in-pod backend passes its release gates and runs through the contract.
|
||||
|
||||
Every backend passes one shared conformance check: backend and contract-version agreement, lifecycle ordering, no agent, `exec`, or `connect` before `Running`, uniform start and `wait` semantics, fail-closed on a missing or unconfirmed boundary, identity-resolution failure handling, execution-domain preservation, and boundary-termination propagation.
|
||||
Shared conformance tests cover version agreement, lifecycle ordering, process semantics, confirmation, identity failure, execution-domain preservation, cleanup, and boundary termination.
|
||||
|
||||
## Risks
|
||||
|
||||
- **The in-pod boundary is not self-confirming today.** Its egress confinement holds only while the host does not forward the sandbox subnet, and its nftables backstop is accept-by-default and covers only TCP and UDP, so reading the rules back proves nothing about other protocols or raw sockets; the backstop also disappears silently when `nft` is absent (`install_bypass_rules` returns `Ok`). Closing this, with a default-deny ceiling across all protocols (invariant 1) confirmed by effective behavior, is step 2 of the implementation plan; invariant 1 blocks shipping the in-pod backend until it does.
|
||||
- **Runtime control endpoints are a container escape independent of the boundary.** For any container-based backend, admission must apply a positive policy to what the agent container may reach: rejecting a couple of named sockets is not enough, because a container runtime socket (Docker, Podman, containerd, CRI-O, or the kubelet), a remote daemon endpoint, a host namespace, or a sensitive host path or device is a full escape regardless of how tight the network boundary is. A correct network boundary around a container that can still reach the runtime is not a contained sandbox.
|
||||
- **Init-path exposure.** A `workspace-init` step runs the agent's image as root after the pod netns exists but before egress confinement is installed, so it has an unmediated egress window, and its executable bytes come from the agent image (untrusted, by invariant 2). This is a deployment choice of the Kubernetes driver, not an interface requirement. The plain in-pod path cannot close the window, but a backend that establishes the boundary before the agent container starts can, including same-pod ones: a trusted native sidecar (an init container with `restartPolicy: Always`) placed first and gated by a startup probe brings the boundary up before the kubelet starts the agent container. For the plain in-pod case, the backend does not ship until the init path runs only OpenShell-controlled content (a trusted seed image, not the agent's) or the boundary already exists before any container starts. This is a release gate, not a recommendation.
|
||||
- **Delegated backends must earn their containment, not just satisfy the contract.** The interface provides the call sites and the structural-containment requirement, but a delegated backend's containment comes from where it places enforcement: cross-pod gate release, handle proxying, the runtime endpoint, and drift monitoring are all new construction it must deliver and prove. A backend that satisfies the contract but places enforcement where the agent can reach it is not contained. This RFC states those requirements; it does not meet them.
|
||||
- **Cross-kernel identity may not be cheap enough for binary-scoped policy.** When `resolve` is a remote call into a guest, an inbound proxy connection must resolve its flow on the hot path, and a slow or failed lookup fails closed to deny-all egress. Whether a cheap correlation path exists decides whether a kernel-separated backend can do binary-scoped policy at all. This is unproven; a future kernel-separated backend must establish it, and the in-pod and shared-kernel backends (where identity is a local procfs read) do not depend on it.
|
||||
- **The refactor spans several entry paths.** Moving namespace plumbing out of the supervisor's call sites (agent launch, SSH, supervisor sessions) and behind the runtime contract is real work, and the in-pod backend must come out behaviorally identical with the existing tests intact.
|
||||
- **In-pod confirmation is incomplete.** Its nftables ceiling is accept-by-default, covers only TCP and UDP, and disappears when `nft` is absent. Step 2 must establish and confirm effective default-deny before release.
|
||||
- **Runtime endpoints bypass containment.** Admission must positively constrain reachable runtime endpoints, host namespaces, paths, and devices.
|
||||
- **The init path runs too early.** `workspace-init` executes image-provided code before egress confinement. The in-pod backend cannot ship until this path uses trusted content or runs behind an established boundary.
|
||||
- **Contract conformance does not create containment.** Each delegated backend must prove that enforcement and its control channel are outside the workload's reach.
|
||||
- **Cross-kernel identity is unproven.** Remote correlation is on the connection hot path; timeout or lookup failure denies identity-scoped egress.
|
||||
- **Adoption spans several entry paths.** Agent launch, SSH, and supervisor sessions must all move behind the contract without changing behavior.
|
||||
|
||||
## Alternatives
|
||||
|
||||
@@ -404,15 +384,13 @@ Every backend passes one shared conformance check: backend and contract-version
|
||||
|
||||
OpenShell could wire one delegated topology (the split-pod proposal in #981, for example) straight into the supervisor and the driver.
|
||||
|
||||
This solves one placement but hardwires it. The next topology (a node runtime, a VM) forks the supervisor again, and the supervisor accretes placement-specific branches. The interface lets each topology land behind the same runtime contract, so the supervisor does not change as placements are added.
|
||||
|
||||
Defining the interface before a second backend exists is a deliberate trade. Its cost now is the factory, the state types, and the registry, paid by extracting today's in-pod path behind them with no change in behavior or privilege (impl-plan steps 1-2). The alternative is the status quo: a supervisor fork per placement. The pieces shaped for delegated use that the in-pod backend does not exercise (the `Flow` token format, the authenticated transport, attestation) are called out at their definitions as what the first delegated backend settles; new shapes arrive under the contract version rather than by widening a shared type.
|
||||
This solves one placement but hardwires it. The factory, states, and registry cost less than a supervisor fork for each later placement. Backend transports, `Flow` formats, and attestation remain follow-on design work.
|
||||
|
||||
### Fold boundary operation into the compute driver
|
||||
|
||||
Establishing the boundary is already a driver concern, so the runtime side could live there too.
|
||||
|
||||
Establishing and operating the boundary are different jobs. The driver provisions platform resources and stands up the boundary; operating it (proxying, policy, identity, the lifecycle the supervisor drives) is the supervisor's runtime concern, not the driver's. Keeping the runtime contract separate lets one supervisor-facing interface be satisfied across every driver and backend, rather than each driver reimplementing it.
|
||||
The driver provisions platform resources; the supervisor operates policy, identity, and runtime lifecycle. Putting runtime operations in compute drivers would duplicate the supervisor-facing contract across them.
|
||||
|
||||
### Select the backend through `SandboxPolicy`
|
||||
|
||||
@@ -435,31 +413,16 @@ The proxy egress work (#1511) owns the proxy above the boundary. This RFC define
|
||||
## Prior art
|
||||
|
||||
- **Driver-backed subsystems (CRI/CNI/CSI).** Kubernetes factors runtime, networking, and storage into pluggable driver contracts so the orchestrator drives one interface while implementations vary. RFC 0001 describes OpenShell's other subsystems the same way; this fills in the one it left as a box.
|
||||
- **istio's privilege placement.** istio reduces transparent-proxy deployment to a placement choice: `istio-init` holds `NET_ADMIN`/`NET_RAW` then exits, leaving an unprivileged Envoy sidecar (the analog of our sidecar placement); ambient mode runs a per-node `ztunnel` daemon with no per-pod sidecar (the analog of our node-enforcer placement). We borrow the placement options rather than the enforcement data path: OpenShell mediates through its own identity-aware proxy, not an iptables redirect. The proxy is OpenShell's own because it scopes egress rules to the connecting binary, resolved per connection from procfs or attestation (`IdentitySource`), which a mesh proxy's workload-level mTLS identity does not model.
|
||||
- **CRI exec/attach/port-forward.** `exec` and `connect` mirror CRI's `Exec` and `PortForward`; a future stdio attach would mirror CRI `Attach`. The gateway already speaks this shape (`ExecSandbox`, `CreateSshSession`, `ForwardTcp`). The contract borrows CRI's names where they fit and departs where they do not: CRI is pod/container-shaped and models neither the two-plane split nor the mediation attachment, so the lifecycle calls are its own.
|
||||
- **Istio privilege placement.** Init-sidecar and node-agent modes demonstrate that network setup can move without changing the policy data path. OpenShell keeps its identity-aware proxy.
|
||||
- **CRI exec/attach/port-forward.** `exec` and `connect` follow CRI's `Exec` and `PortForward` shape; lifecycle and mediation remain OpenShell-specific.
|
||||
|
||||
## Appendix: in-pod migration sketch
|
||||
|
||||
Behavior is identical to today; only the placement of boundary setup and process entry changes, now expressed as the contract's states. `ProcessHandle::spawn` already separates the spec from the boundary handle (it takes `netns: Option<&NetworkNamespace>`), so the in-pod `exec` is `BoundaryExec::exec` in embryo. The refactor: the in-pod factory's `attach` allocates a closed, policy-free boundary in the agent's container (still privileged there; no capabilities removed); `claim` binds the sandbox and applies the policy-dependent netns routing and rules; `bind` returns the mediation ingress, identity source, and event stream; `confirm` checks the boundary's effective enforcement (failing closed if `nft` is absent or the ceiling is not default-deny); `start_agent` forks the entrypoint into the netns and returns a `RunningBoundary`, so the supervisor's existing join becomes `running.agent().wait()`, with boundary-level termination surfaced by `wait_terminated`; `exec` and `connect` move from direct namespace plumbing to the runtime interfaces. Teardown is the idempotent `shutdown`.
|
||||
The in-pod adapter preserves behavior and privilege while moving existing code behind the states. `attach` creates or joins a closed, unclaimed boundary and returns its control handle; a pooled boundary retains its admitted baseline. `claim` binds the sandbox and policy; `bind` returns mediation, identity, and events; `confirm` checks effective enforcement; and `start_agent` returns `RunningBoundary`. Existing process launch, `exec`, and `connect` paths move behind their runtime interfaces. `BoundaryControl` owns termination and cleanup.
|
||||
|
||||
## Appendix: designing a backend
|
||||
|
||||
A backend author starts from the containment property they want, decides where enforcement sits to get it, and the contract calls follow:
|
||||
|
||||
1. **Pick the property.** What must the agent be unable to do, and what must stay protected from it. "Egress only through the proxy" is the network property; "the agent cannot reach the supervisor's kernel" is a stronger one.
|
||||
2. **Place the enforcement below the agent's reach.** That placement provides the containment. In-pod puts it in the agent's netns; a node enforcer puts it on the node; a VM puts it in the hypervisor. The further out, the stronger the property and the more the supervisor's reach-ins must travel through the contract.
|
||||
3. **Establish it, then operate it.** Establish that enforcement before the agent runs (inline for in-pod, via the driver for delegated); answer the runtime contract's lifecycle and in-boundary calls.
|
||||
|
||||
Worked example, a node enforcer that drops `NET_ADMIN` from the agent pod:
|
||||
|
||||
- **Property:** egress only through the proxy, with no isolation privilege in the agent pod.
|
||||
- **Placement:** a privileged per-node daemon installs the netns routing and rules from the node, so the pod needs no capability. The supervisor stays in the pod (it still shares the host kernel; this placement does not separate kernels).
|
||||
- **Establishment:** the supervisor's `attach` call reaches the daemon (the in-pod supervisor registering with the node daemon, expressed as the factory's `attach`, not a side protocol). The daemon enters the agent pod's netns, installs the routing and rules, confirms their effective enforcement, and returns the attached boundary. The supervisor holds no privilege in this exchange; the daemon does the privileged work and answers the call.
|
||||
- **Runtime:** `claim`, `bind`, `confirm`, and `start_agent` proceed as in-pod. Because the supervisor still shares the agent's kernel, `exec`, `connect`, and `resolve` use it directly, unchanged from in-pod.
|
||||
|
||||
A kernel-separated backend (a split-pod with the agent under a VM or gVisor RuntimeClass, or a node runtime that puts the supervisor outside the agent's kernel) follows the same loop with a stronger placement: the runtime interfaces now route over the backend's guest channel and the backend supplies identity evidence the supervisor cannot gather itself. The lifecycle states and the supervisor's mediation are unchanged.
|
||||
|
||||
A backend's separate design covers, at minimum: its backend ID and registered factory; its capabilities; its descriptor payload and verification; every lifecycle transition; the runtime interfaces and the process handle; how it confirms the common invariants; policy and resource-domain preservation; provisioning and cleanup integration; termination behavior; and the shared conformance tests.
|
||||
A backend-specific design covers its ID and factory, descriptor payload and verification, lifecycle transitions, runtime interfaces, confirmation, policy and resource preservation, provisioning, cleanup, termination, and conformance tests.
|
||||
|
||||
## Appendix: codebase grounding
|
||||
|
||||
|
||||
@@ -1,16 +1,10 @@
|
||||
# Codebase grounding (supporting material for RFC 0012)
|
||||
|
||||
This file backs the claims [RFC 0012](./README.md) makes about the current
|
||||
system, so a reviewer can verify them. It is supporting material, not part of the
|
||||
normative RFC.
|
||||
This non-normative file grounds RFC 0012's claims about the current system.
|
||||
|
||||
Verified against the RFC's parent commit `a5161d0` (`origin/main` has since
|
||||
advanced; these references are pinned to that parent). Form a permalink as
|
||||
`https://github.com/NVIDIA/OpenShell/blob/a5161d0/<path>#L<line>`. After #1650
|
||||
the supervisor split into `openshell-supervisor-network` (proxy, OPA, procfs
|
||||
identity) and `openshell-supervisor-process` (spawn, SSH, netns creation,
|
||||
seccomp/Landlock). References are line-pinned and will drift; the `rg` patterns
|
||||
let a reviewer re-locate them.
|
||||
References are pinned to the RFC's parent, `a5161d0`. Permalinks use
|
||||
`https://github.com/NVIDIA/OpenShell/blob/a5161d0/<path>#L<line>`; the `rg`
|
||||
patterns locate the same code on newer revisions.
|
||||
|
||||
| Claim | Reference |
|
||||
|---|---|
|
||||
@@ -26,4 +20,4 @@ let a reviewer re-locate them.
|
||||
| Agent command via CLI/`SANDBOX_COMMAND`; no admission-bound spec field today; the `sleep infinity` placeholder resolves `sleep` from the agent image's own filesystem | `crates/openshell-sandbox/src/main.rs:331`; K8s driver sets `sleep infinity` at `driver.rs:1886` (`rg -n "sleep infinity" crates/openshell-driver-kubernetes/src`) |
|
||||
| Init containers: `copy-self` (trusted, the OpenShell binary) and `workspace-init` (runs as root from the agent's own image, so its executables are image-provided); no native sidecars today | `driver.rs:191` (`WORKSPACE_INIT_CONTAINER_NAME`), `:993` (`copy-self` invocation), `:1185` (workspace-init container); `rg -n "restart_policy|workspace-init" crates/openshell-driver-kubernetes/src` |
|
||||
| Network policy is OPA per-CONNECT, not the boundary; identity via procfs | `crates/openshell-supervisor-network/src/opa.rs` (`NetworkInput`: `binary_path`/`binary_sha256`/`ancestors`/`cmdline_paths`), `procfs.rs`, glued in `proxy.rs:1611` (`NetworkInput` built in `evaluate_opa_tcp`) |
|
||||
| Static privilege ceiling on every spawned process; OPA never evaluates exec | `process.rs:440` (`ProcessHandle::spawn` applies the pre-exec ceiling: `drop_privileges` at `:932`, seccomp/Landlock `enforce` from `SandboxPolicy`; SSH reaches the same `enter_netns_and_sandbox`) |
|
||||
| Static privilege ceiling on every spawned process; OPA never evaluates exec | `process.rs:440` (`ProcessHandle::spawn`), `:603`/`:700` (`drop_privileges` call sites), `:613`/`:705` (sandbox enforcement); SSH reaches the same `enter_netns_and_sandbox` path |
|
||||
|
||||
@@ -1,37 +1,26 @@
|
||||
# Topology selection matrix
|
||||
|
||||
This is a **lite, draft** decision aid that maps a deployment's starting point to a
|
||||
recommended isolation topology from [RFC 0012](./README.md). It is supporting
|
||||
material, not part of the normative RFC. It is seeded from the design meeting that
|
||||
shaped RFC 0012 and is expected to be refined jointly (target-infrastructure
|
||||
dimensions, usage personas, and the kata-enablement work) and to eventually
|
||||
graduate into a docs page: "what are you trying to do, here is the recommended
|
||||
configuration."
|
||||
|
||||
[RFC 0012](./README.md) defines the interface and the design logic (the kernel-sharing
|
||||
axis, the forbidden cell, sequencing). This file holds the **placement catalog**
|
||||
(the per-placement detail) and the **selection matrix** (which to choose).
|
||||
This non-normative draft maps deployment needs to the placements in
|
||||
[RFC 0012](./README.md). The RFC defines the contract and kernel-sharing axis;
|
||||
this file compares placements and suggests when to use them.
|
||||
|
||||
## Placement catalog
|
||||
|
||||
The interface offers no single universal placement; different infrastructure and
|
||||
workloads warrant different ones (istio went the same way). Three near-term
|
||||
placements that share the agent's kernel, then the moves that separate it. Which
|
||||
share the agent's kernel, and their build status, is in [RFC 0012's Representative
|
||||
topologies](./README.md#representative-topologies).
|
||||
No placement is universal. The near-term options share the agent's kernel;
|
||||
stronger options move the supervisor outside it.
|
||||
|
||||
**Today-hardened (fallback).** The supervisor runs in the agent's pod, but non-root. Better hygiene than today, the simplest step, the fallback when no internal isolation exists. Privilege still sits in the pod, so it does not reach `restricted`. (Kata-enablement work can move a sandbox from the `privileged` to the `anyuid` SCC by treating the hypervisor as the boundary, but `anyuid` is a waypoint, not the destination: it still permits root and is not `restricted`.)
|
||||
**Today-hardened (fallback).** The supervisor remains in the agent pod. This is the simplest path, but isolation privilege remains in the pod and does not meet `restricted`.
|
||||
|
||||
**Sidecar-assisted (easiest UX).** A privileged sidecar in the **same pod** runs the Isolation Backend that programs interception; the supervisor stays with the agent and remains its proxy. This is a composite backend: it moves only the privileged network setup to the sidecar, while the in-pod supervisor still operates the proxy and applies the filesystem and syscall ceilings at process entry. Setting up interception still needs `NET_ADMIN`/`NET_RAW`, but only the sidecar carries it, not the agent container, making the pod spec defensible. Standard pod semantics; agent and sidecar share the pod's kernel. A native sidecar (an init container with `restartPolicy: Always`; beta and on by default since Kubernetes 1.29, GA in 1.33) realizes the `boundary_ready` gate: the kubelet starts the agent container only after the sidecar has started, and a startup probe on the sidecar holds it until the boundary is ready (a readiness probe only affects Pod readiness, not container start order), which this design requires. Moving the supervisor itself into the sidecar later is a different backend, one that needs an agent-side process shim to start and confine processes in the agent container; its own design defines that shim.
|
||||
**Sidecar-assisted.** A privileged same-pod sidecar programs network interception. The supervisor remains the proxy and applies filesystem and syscall ceilings. This moves only network-setup capabilities out of the agent container; all containers still share one kernel. A Kubernetes [native sidecar](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/) can map `boundary_ready` to a `startupProbe` that gates agent startup, closing the `workspace-init` window on clusters with sidecar containers available (beta since Kubernetes 1.29, GA in 1.33). Moving the supervisor requires a separate design and an agent-side process shim.
|
||||
|
||||
**Node enforcer / daemon-set (network-setup capabilities off the pod).** A privileged per-node daemon does the boundary setup, so the agent pod needs no network-setup capabilities. In the prototype, the unprivileged in-pod supervisor's `attach` call reaches the daemon, which enters the agent pod's network namespace and installs the egress rules (the registration is the contract call, not a separate protocol). NetworkPolicy does not replace this: it can provide coarse L3/L4 defense in depth, but requires an enforcing CNI, has a pod-start programming window, has protocol and selector limits, and cannot replace the identity-aware proxy. The boundary still funnels egress to the supervisor's proxy; the daemon does the privileged setup, it does not replace the proxy, and changing proxy policy updates the proxy, not the boundary enforcement. Like the sidecar, this is a composite backend: it relocates only the network setup, while the in-pod supervisor still operates the proxy and applies the filesystem and syscall ceilings. The supervisor stays in the agent pod, sharing the host kernel. Cost: a privileged node component and bespoke node setup. A node enforcer alone does not make the pod capability-free or `restricted`-compatible; it removes the network capabilities, and the rest of the seven-capability grant comes off only as process, filesystem, and identity placement move too (see "Capabilities, and how they come off").
|
||||
**Node enforcer / daemon set.** A privileged node daemon installs routing and rules during `attach`, so the agent pod needs no network-setup capabilities. The in-pod supervisor still runs the proxy and applies process-time ceilings. NetworkPolicy can add L3/L4 defense in depth but cannot replace identity-aware mediation. This placement adds a privileged node component and does not by itself make the pod capability-free or `restricted`-compatible.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph SCPod["sidecar: agent pod"]
|
||||
direction TB
|
||||
SCBackend["Isolation Backend (privileged sidecar)"]
|
||||
subgraph SCAgent["agent (unprivileged)"]
|
||||
subgraph SCAgent["agent + supervisor (network setup delegated)"]
|
||||
SCWork["workload"]
|
||||
SCSup["supervisor"]
|
||||
end
|
||||
@@ -41,7 +30,7 @@ graph LR
|
||||
SCSup -->|egress| GW1["gateway"]
|
||||
|
||||
NodeD["node daemon (privileged)"]
|
||||
subgraph NodePod["node enforcer: agent pod (unprivileged)"]
|
||||
subgraph NodePod["node enforcer: agent pod (network setup delegated)"]
|
||||
direction TB
|
||||
NodeWork["workload"]
|
||||
NodeSup["supervisor"]
|
||||
@@ -65,9 +54,9 @@ graph LR
|
||||
|
||||
Splitting the supervisor into a separate pod is one move; whether it separates kernels depends on the agent pod's RuntimeClass. The two variants land in different containment rows (see [RFC 0012's containment table](./README.md#representative-topologies)).
|
||||
|
||||
- **Split-pod, `runc` agent (privilege separation, shared host kernel).** The supervisor moves to a separate pod; the agent pod drops its privileged capabilities and is confined by NetworkPolicy. Both pods still share the host kernel, so this separates privilege, not kernels. Standard pod semantics, at the cost of coordinating two scheduled units per sandbox. NetworkPolicy here is a proof obligation, not a free win: it requires a policy-enforcing CNI, is default-allow and L3/L4 only, programs eventually-consistently (a pod can egress before the policy lands), and is label-scoped (so it is defense-in-depth unless pod-mutation RBAC is locked down). (A concrete proposal is tracked in #981.)
|
||||
- **Split-pod, VM or gVisor agent (kernel separation, expressible today).** The same separate-supervisor-pod move, but the agent pod runs under a Kata or gVisor `runtimeClassName`, so the agent gets its own kernel (a VM guest kernel under Kata, gVisor's application kernel under gVisor) and the supervisor sits outside it. Buildable on stock Kubernetes today. A stock one-pod Kata or gVisor RuntimeClass is not this: it puts the supervisor and agent under one such kernel (the outer-sandbox case, where they stay co-located).
|
||||
- **Future node runtime (kernel separation, off the standard path).** Keeping the supervisor outside the agent's kernel without a per-agent guest needs a node runtime built to place them on opposite sides of the boundary. Nobody has built one; it is the seam a kernel-separated backend would fill. Real backend work.
|
||||
- **Split-pod, `runc` agent.** Moving the supervisor to another pod separates privilege but not the host kernel. [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) requires an enforcing CNI; pods remain non-isolated for egress until selected by an effective egress policy, and enforcement remains L3/L4 and label-scoped. See #981.
|
||||
- **Split-pod, VM or gVisor agent.** Running only the agent pod under Kata or gVisor places the supervisor outside the agent's guest or application kernel. A one-pod RuntimeClass does not: it co-locates both processes inside that kernel.
|
||||
- **Future node runtime.** A node runtime could place supervisor and agent on opposite sides of a kernel boundary without a separate supervisor pod. That backend requires its own design.
|
||||
|
||||
### The design space, and the one cell stock Kubernetes forbids
|
||||
|
||||
@@ -78,35 +67,25 @@ Two axes set the space: whether the supervisor shares the agent's kernel, and th
|
||||
| One pod | Today-hardened, sidecar-assisted, and the single-pod outer sandbox (gVisor, Kata, or Firecracker). Per-container `securityContext` varies privilege; the proxy mediates within the pod netns. An outer sandbox isolates the host but runs the supervisor inside the agent's isolated kernel (a VM guest under Kata or Firecracker, gVisor's application kernel under gVisor), so it shares that kernel. | **Not expressible on stock Kubernetes.** The runtime that owns the kernel boundary is selected per pod (`runtimeClassName`), not per container, so two containers of one pod cannot have different kernels. |
|
||||
| Two pods / node runtime | Node enforcer, or a `runc` split-pod: both share the host kernel. | Split-pod with the agent pod under a VM or gVisor `runtimeClassName` (the supervisor stays in its own pod, outside the agent's guest), or a future node runtime that runs the supervisor outside the agent's kernel. |
|
||||
|
||||
The forbidden cell holds because of the CRI shape: the kubelet calls `RunPodSandbox` with the runtime handler once, per pod, then `CreateContainer` inside that one sandbox. The handler, and so the kernel, is fixed per sandbox; selecting it per container would need a new per-container CRI field plus a runtime that acts on it, not just a different backend behind the interface. The right column costs more: the supervisor can no longer read `/proc` or enter the netns directly, so those reach-ins route through the backend.
|
||||
The forbidden cell follows from Kubernetes [RuntimeClass](https://kubernetes.io/docs/concepts/containers/runtime-class/) and the CRI shape: the kubelet selects one runtime handler for `RunPodSandbox`, then creates containers inside that sandbox. Per-container kernels would need a CRI and runtime change, not another backend. In the right column, procfs and netns operations route through the backend.
|
||||
|
||||
### Capabilities, and how they come off
|
||||
|
||||
An unprivileged agent container means shedding the whole seven-capability grant, not just `NET_ADMIN`, and that happens in stages:
|
||||
Removing network setup is only the first step toward an unprivileged agent:
|
||||
|
||||
- A backend removes `NET_ADMIN` and the network use of `SYS_ADMIN` by moving network setup out of the agent's container.
|
||||
- Removing `SYS_ADMIN` fully needs process and filesystem placement to move too.
|
||||
- Removing `SYS_PTRACE`/`DAC_READ_SEARCH` needs identity attested by the backend instead of read from `/proc`.
|
||||
- Removing `SYS_PTRACE`/`DAC_READ_SEARCH` requires identity supplied by the backend instead of read in the agent container.
|
||||
|
||||
A same-pod sidecar moves `NET_ADMIN` out of the agent container but not the pod's security profile: the pod still needs an elevated security context, and on OpenShift an SCC exception, that `restricted` forbids (a service account carries no Linux capabilities itself; this is the friction adopters dislike). A node enforcer or split-pod removes the network-setup capabilities from the pod; reaching a fully unprivileged pod additionally requires moving process, filesystem, and identity off it, and even that is necessary, not sufficient, for `restricted`: SELinux/MCS labels, seccomp, arbitrary UIDs, procfs-based identity, and execution-domain preservation must all be handled too. (RFC 0012's Risks section carries the normative requirement that admission gate what the agent container may reach.)
|
||||
A sidecar moves network capabilities out of the agent container but not the pod security profile. A node enforcer moves them off the pod. Meeting `restricted` also requires compatible SELinux/MCS labels, seccomp, UID handling, identity, and execution-domain preservation.
|
||||
|
||||
## The dimensions
|
||||
|
||||
Two dimensions drive the recommendation, with a third that bumps the posture up a
|
||||
level rather than forming its own axis.
|
||||
Two dimensions choose a placement; data sensitivity raises the posture:
|
||||
|
||||
1. **Infrastructure available (the top-level question).** Do you have nested
|
||||
virtualization / kernel isolation (kata, gVisor) on your nodes? This is the
|
||||
first thing to decide. If yes, a kernel-grade boundary is on the table. If no,
|
||||
you are choosing among namespace-based topologies and trading off sidecar
|
||||
versus node placement.
|
||||
2. **What the agent does.** A privacy-router that only talks to an LLM has a small
|
||||
blast radius; an agent that runs arbitrary, never-reviewed code has a large
|
||||
one. The industry has no crisp definition of "agent," which is exactly why this
|
||||
dimension matters for the recommended posture.
|
||||
3. **Data sensitivity (the tie-breaker).** Whether PII or other sensitive data is
|
||||
in play. Rather than a third axis, treat it as a rule that moves you one cell
|
||||
toward stronger isolation within the table below.
|
||||
1. **Infrastructure.** Kernel isolation such as Kata or gVisor enables a kernel-separated placement; otherwise choose among namespace-based options.
|
||||
2. **Workload.** Arbitrary unreviewed code warrants stronger isolation than a narrow privacy-router workload.
|
||||
3. **Data sensitivity.** PII or other sensitive data moves the recommendation one step stronger.
|
||||
|
||||
## The matrix
|
||||
|
||||
@@ -115,7 +94,7 @@ recommended topology from RFC 0012.
|
||||
|
||||
| | Agent = privacy-router only (talks to an LLM, low blast radius) | Agent = arbitrary code execution (untrusted code, high blast radius) |
|
||||
|---|---|---|
|
||||
| **Has kernel isolation** (nested virt / bare metal; kata or gVisor available) | **Sidecar-assisted.** A privacy router's blast radius is small, so a kernel boundary's overhead rarely pays for itself against that threat model; keep it simple. (Move to kata if PII is in play.) | **Kernel-separated.** Put the agent in its own guest kernel, but keep the supervisor outside it: split-pod with the agent pod under Kata/gVisor, or a future node runtime. A stock one-pod Kata/gVisor RuntimeClass is not sufficient, it co-locates the supervisor in the same guest kernel. The recommended strong posture. |
|
||||
| **Has kernel isolation** (nested virt / bare metal; Kata or gVisor available) | **Sidecar-assisted.** A privacy router's blast radius is small, so a kernel boundary's overhead rarely pays for itself against that threat model. (Move to Kata if PII is in play.) | **Kernel-separated.** Put the agent in its own guest kernel, but keep the supervisor outside it: split-pod with the agent pod under Kata/gVisor, or a future node runtime. A one-pod Kata/gVisor RuntimeClass co-locates the supervisor in the same guest kernel. |
|
||||
| **No kernel isolation** (managed Kubernetes, no nested virt) | **Today-hardened.** Non-root supervisor in the agent pod; the simplest fallback. (Move to sidecar if PII is in play.) | **Sidecar-assisted, then node enforcer.** Sidecar to keep privilege off the agent container; node enforcer to move the network-setup privilege off the pod. The best achievable without a VM. |
|
||||
|
||||
**PII tie-breaker.** If sensitive data is in play, move one cell toward stronger
|
||||
@@ -124,24 +103,6 @@ neighbor would use, and an arbitrary-code deployment with kernel isolation
|
||||
available should prefer the kernel-separated path rather than a same-kernel
|
||||
sidecar.
|
||||
|
||||
## How to read a recommendation
|
||||
|
||||
1. Start with infrastructure (the row). No nested virt means the right-hand
|
||||
column's strongest option (kernel separation) is simply unavailable; do not
|
||||
recommend it.
|
||||
2. Within the row, pick the column by what the agent does.
|
||||
3. Apply the PII tie-breaker if relevant.
|
||||
4. The named topology links back to RFC 0012 for its mechanism, trade-offs, and
|
||||
the capabilities it removes.
|
||||
|
||||
## Status and ownership
|
||||
|
||||
- **Draft, lite.** Deliberately small to keep RFC 0012 reviewable. Expect the
|
||||
cells and dimensions to change.
|
||||
- **Open contributions.** Forthcoming work will propose roughly three target
|
||||
infrastructures and the trade-off dimensions, and a set of usage personas; the
|
||||
kata-enablement work (the `privileged` to `anyuid` SCC change) informs the
|
||||
"has kernel isolation" row. This file is the collection point until it graduates
|
||||
to docs.
|
||||
- **Not normative.** RFC 0012 defines the interface and the topologies. This file
|
||||
only recommends which topology to choose; it does not add requirements.
|
||||
This draft is intentionally non-normative. It may evolve into deployment documentation as concrete backends land.
|
||||
|
||||
Reference in New Issue
Block a user