feat(gateway): support selective compute driver builds (#3118)

* feat(gateway): support selective compute driver builds

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(gateway): support selective Windows MXC builds

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
This commit is contained in:
Drew Newberry
2026-09-11 00:36:09 +00:00
committed by GitHub
parent 1860010850
commit 38f2aef930
12 changed files with 240 additions and 50 deletions
@@ -252,6 +252,13 @@ in the gateway build graph, but their Unix-socket standalone binaries do not.
Windows must continue to reject unsupported compute drivers clearly.
The gateway's `compute-driver-mxc` feature independently links and registers
MXC on Windows. Each other `compute-driver-*` feature installs its own Windows
rejection stub without linking that driver crate. The default
`in-tree-compute-drivers` alias enables all five features. An MXC-only build
uses `--no-default-features --features compute-driver-mxc` (add `telemetry`
and `bundled-z3` as needed).
| Driver | Windows build behavior | Runtime behavior |
|---|---|---|
| Docker | Driver crate excluded; gateway registration stub retained. | Gateway construction returns unsupported. |
@@ -263,11 +270,16 @@ The focused contract tasks for either native architecture run:
```text
windows_builtin_compute_drivers_report_unsupported
default_registry_contains_exactly_the_enabled_compute_drivers
```
These tests are also included in the full x64 workspace test run. The focused
task is available for local diagnosis; GitHub Actions does not re-run it after
the full suite.
The same tasks also run gateway library tests for protocol-only, MXC-only,
Docker-stub-only, and MXC plus Docker-stub builds. Their logs use
`test-<target>-selective-<variant>.log`.
The default-feature tests are also included in the full workspace test run.
The focused task is available for local diagnosis and selective-build
validation; GitHub Actions does not re-run it after the full suite.
## Test Accounting Guidance
+10
View File
@@ -182,6 +182,16 @@ jobs:
cargo build -p openshell-sandbox --bin openshell-sandbox --no-default-features --features defaults-without-telemetry
tasks/scripts/verify-telemetry-compiled-out.sh absent target/debug/openshell-sandbox
- name: Verify selective gateway compute-driver builds
run: |
cargo test -p openshell-gateway --all-targets --no-default-features
cargo test -p openshell-gateway --all-targets --no-default-features --features compute-driver-docker
cargo test -p openshell-gateway --all-targets --no-default-features --features compute-driver-kubernetes
cargo test -p openshell-gateway --all-targets --no-default-features --features compute-driver-podman
cargo test -p openshell-gateway --all-targets --no-default-features --features compute-driver-vm
cargo test -p openshell-gateway --all-targets --no-default-features --features compute-driver-mxc
cargo test -p openshell-gateway --all-targets --no-default-features --features compute-driver-docker,compute-driver-vm
- name: Verify the defaults-without-telemetry feature alias tracks the default feature set
run: tasks/scripts/verify-defaults-without-telemetry.sh
Generated
+2
View File
@@ -5812,6 +5812,7 @@ dependencies = [
"log",
"percent-encoding",
"pin-project-lite",
"quinn",
"rustls",
"rustls-native-certs",
"rustls-pki-types",
@@ -6124,6 +6125,7 @@ dependencies = [
"aws-lc-rs",
"log",
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki",
"subtle",
+15
View File
@@ -274,6 +274,21 @@ cargo build --release -p openshell-driver-vm --no-default-features --features de
The resulting binaries contain no telemetry endpoint, no telemetry HTTP client, and no emission code. With telemetry compiled out, the gateway emits nothing and reports telemetry disabled to the sandboxes it launches. Cargo has no way to subtract a single default feature, so `defaults-without-telemetry` must be paired with `--no-default-features`; passing it on its own leaves the defaults in place and fails the build rather than producing a binary that still emits.
The gateway also exposes separate Cargo features for its built-in compute drivers: `compute-driver-kubernetes`, `compute-driver-docker`, `compute-driver-podman`, `compute-driver-vm`, and `compute-driver-mxc`. Disable the default feature set, then enable only the drivers and telemetry mode required by the target binary. For example:
```shell
# Docker only, with telemetry support.
cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-docker
# Docker and VM only, with telemetry compiled out.
cargo build --release -p openshell-gateway --no-default-features --features compute-driver-docker,compute-driver-vm
# Windows MXC only, with telemetry support and bundled Z3.
cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-mxc,bundled-z3
```
Regular builds retain their platform driver set through the default `in-tree-compute-drivers` compatibility feature. On Windows, `compute-driver-mxc` selects MXC; the other four features install unsupported-driver stubs. On other platforms, MXC is excluded.
Telemetry events are limited to anonymous operational categories and counts, such as sandbox lifecycle outcomes, provider profile buckets, policy decision counts, and aggregate network activity denial categories. OpenShell telemetry does not collect sandbox names or IDs, hostnames, file paths, binary paths, prompts, credentials, provider names, model names, or user content.
Opting out applies only to telemetry emitted by OpenShell. Third-party services, model providers, inference endpoints, agents, or tools that you configure and use with OpenShell may have their own terms and privacy practices.
+12 -7
View File
@@ -133,13 +133,18 @@ server constructs the common runtime adapter and snapshots `GetCapabilities`
for either result. A configured UDS endpoint still takes precedence over a
compiled registration with the same name.
The `openshell-gateway` composition crate groups first-party registrations
behind the `in-tree-compute-drivers` feature. `openshell-server` has no compute
driver dependencies or backend-name dispatch. Protocol-only gateway builds
disable the composition feature and link no compute-driver crates. E2E lanes
compose that gateway with Docker, Podman, Kubernetes, and VM driver executables
over the public UDS gRPC contract so an in-tree driver cannot silently depend
on a server-only API.
The `openshell-gateway` composition crate exposes one feature per first-party
registration: `compute-driver-kubernetes`, `compute-driver-docker`,
`compute-driver-podman`, `compute-driver-vm`, and `compute-driver-mxc`. Builds
can enable any subset. MXC links only on Windows; the other four features
install rejection stubs on Windows and link their drivers on other platforms.
The default `in-tree-compute-drivers` feature remains an alias for all five,
preserving each platform's default registrations.
`openshell-server` has no compute driver dependencies or backend-name dispatch.
Protocol-only gateway builds disable the default features and link no
compute-driver crates. E2E lanes compose that gateway with Docker, Podman,
Kubernetes, and VM driver executables over the public UDS gRPC contract so an
in-tree driver cannot silently depend on a server-only API.
## Stop and Start Lifecycle
+8
View File
@@ -31,6 +31,14 @@ Windows. These registrations preserve config-file selection and reject
unsupported drivers with a clear error without depending on their runtime
crates.
Each stub follows its corresponding `compute-driver-*` Cargo feature.
`compute-driver-mxc` independently links and registers MXC, so a gateway built
with only that feature has only the MXC registration. The default
`in-tree-compute-drivers` alias enables all five features and preserves the
existing MXC plus unsupported-driver registrations. The focused Windows
contract tasks cover default, protocol-only, MXC-only, Docker-stub-only, and
MXC plus Docker-stub compositions.
The Windows lane does not build, release, package, or smoke-test standalone
driver binaries for Docker, Kubernetes, Podman, or VM. Those binaries are Linux
or macOS deliverables only.
+1 -1
View File
@@ -14,7 +14,7 @@ repository.workspace = true
name = "openshell_driver_mxc"
[dependencies]
openshell-core = { path = "../openshell-core" }
openshell-core = { path = "../openshell-core", default-features = false }
tokio = { workspace = true }
tonic = { workspace = true }
futures = { workspace = true }
+11 -4
View File
@@ -40,9 +40,17 @@ openshell-driver-mxc = { path = "../openshell-driver-mxc", optional = true }
[features]
default = ["telemetry", "in-tree-compute-drivers"]
in-tree-compute-drivers = [
"dep:openshell-driver-docker",
"dep:openshell-driver-kubernetes",
"dep:openshell-driver-podman",
"compute-driver-docker",
"compute-driver-kubernetes",
"compute-driver-podman",
"compute-driver-vm",
"compute-driver-mxc",
]
compute-driver-mxc = ["dep:openshell-driver-mxc"]
compute-driver-docker = ["dep:openshell-driver-docker", "dep:openshell-otel"]
compute-driver-kubernetes = ["dep:openshell-driver-kubernetes", "dep:openshell-otel"]
compute-driver-podman = ["dep:openshell-driver-podman", "dep:openshell-otel"]
compute-driver-vm = [
"dep:openshell-otel",
"dep:hyper-util",
"dep:nix",
@@ -51,7 +59,6 @@ in-tree-compute-drivers = [
"dep:tonic",
"dep:tower",
"dep:tracing",
"dep:openshell-driver-mxc",
]
telemetry = ["openshell-core/telemetry", "openshell-server/telemetry"]
## Convenience alias: every default feature except `telemetry`. Build a
+113 -29
View File
@@ -16,12 +16,29 @@ compile_error!(
build a telemetry-free gateway with `--no-default-features --features defaults-without-telemetry`"
);
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-vm"))]
mod vm;
#[cfg(feature = "in-tree-compute-drivers")]
#[cfg(any(
all(target_os = "windows", feature = "compute-driver-mxc"),
all(
not(target_os = "windows"),
any(
feature = "compute-driver-docker",
feature = "compute-driver-kubernetes",
feature = "compute-driver-podman",
feature = "compute-driver-vm"
)
)
))]
use openshell_core::telemetry::TelemetryComputeDriver;
#[cfg(feature = "in-tree-compute-drivers")]
#[cfg(any(
target_os = "windows",
feature = "compute-driver-docker",
feature = "compute-driver-kubernetes",
feature = "compute-driver-podman",
feature = "compute-driver-vm"
))]
use openshell_server::ComputeDriverRegistration;
use openshell_server::ComputeDriverRegistry;
@@ -30,14 +47,24 @@ use openshell_server::ComputeDriverRegistry;
pub fn install_default_compute_drivers() -> ComputeDriverRegistry {
#[allow(unused_mut)]
let mut registry = ComputeDriverRegistry::new();
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(
not(target_os = "windows"),
any(
feature = "compute-driver-docker",
feature = "compute-driver-kubernetes",
feature = "compute-driver-podman",
feature = "compute-driver-vm"
)
))]
install_in_tree_compute_drivers(&mut registry);
#[cfg(all(target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(all(target_os = "windows", feature = "compute-driver-mxc"))]
install_mxc_compute_driver(&mut registry);
#[cfg(target_os = "windows")]
install_unsupported_windows_compute_drivers(&mut registry);
registry
}
#[cfg(all(target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(all(target_os = "windows", feature = "compute-driver-mxc"))]
fn install_mxc_compute_driver(registry: &mut ComputeDriverRegistry) {
let registration = ComputeDriverRegistration::new("mxc", u16::MAX, None, MxcFactory)
.expect("first-party driver name is valid")
@@ -46,8 +73,21 @@ fn install_mxc_compute_driver(registry: &mut ComputeDriverRegistry) {
registry
.install(registration)
.expect("first-party driver names are unique");
}
for name in ["docker", "kubernetes", "podman", "vm"] {
#[cfg(target_os = "windows")]
fn install_unsupported_windows_compute_drivers(registry: &mut ComputeDriverRegistry) {
let names: &[&str] = &[
#[cfg(feature = "compute-driver-docker")]
"docker",
#[cfg(feature = "compute-driver-kubernetes")]
"kubernetes",
#[cfg(feature = "compute-driver-podman")]
"podman",
#[cfg(feature = "compute-driver-vm")]
"vm",
];
for &name in names {
let registration = ComputeDriverRegistration::new(
name,
u16::MAX,
@@ -61,13 +101,13 @@ fn install_mxc_compute_driver(registry: &mut ComputeDriverRegistry) {
}
}
#[cfg(all(target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(target_os = "windows")]
#[derive(Clone, Copy)]
struct UnsupportedWindowsFactory {
name: &'static str,
}
#[cfg(all(target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(target_os = "windows")]
#[async_trait::async_trait]
impl openshell_server::ComputeDriverFactory for UnsupportedWindowsFactory {
async fn build(
@@ -78,16 +118,16 @@ impl openshell_server::ComputeDriverFactory for UnsupportedWindowsFactory {
}
}
#[cfg(all(target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(target_os = "windows")]
fn unsupported_windows_compute_driver(name: &str) -> openshell_core::Error {
openshell_core::Error::config(format!("compute driver '{name}' is unsupported on Windows"))
}
#[cfg(all(target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(all(target_os = "windows", feature = "compute-driver-mxc"))]
#[derive(Clone, Copy)]
struct MxcFactory;
#[cfg(all(target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(all(target_os = "windows", feature = "compute-driver-mxc"))]
#[async_trait::async_trait]
impl openshell_server::ComputeDriverFactory for MxcFactory {
async fn build(
@@ -103,9 +143,18 @@ impl openshell_server::ComputeDriverFactory for MxcFactory {
}
}
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(
not(target_os = "windows"),
any(
feature = "compute-driver-docker",
feature = "compute-driver-kubernetes",
feature = "compute-driver-podman",
feature = "compute-driver-vm"
)
))]
fn install_in_tree_compute_drivers(registry: &mut ComputeDriverRegistry) {
for registration in [
#[cfg(feature = "compute-driver-kubernetes")]
ComputeDriverRegistration::new(
"kubernetes",
100,
@@ -128,6 +177,7 @@ fn install_in_tree_compute_drivers(registry: &mut ComputeDriverRegistry) {
"sa_token_ttl_secs",
])
}),
#[cfg(feature = "compute-driver-podman")]
ComputeDriverRegistration::new(
"podman",
200,
@@ -148,6 +198,7 @@ fn install_in_tree_compute_drivers(registry: &mut ComputeDriverRegistry) {
"guest_tls_key",
])
}),
#[cfg(feature = "compute-driver-docker")]
ComputeDriverRegistration::new(
"docker",
300,
@@ -169,6 +220,7 @@ fn install_in_tree_compute_drivers(registry: &mut ComputeDriverRegistry) {
"guest_tls_key",
])
}),
#[cfg(feature = "compute-driver-vm")]
ComputeDriverRegistration::new("vm", u16::MAX, None, VmFactory).map(|registration| {
registration
.with_telemetry_category(TelemetryComputeDriver::anonymous_category("vm"))
@@ -187,11 +239,11 @@ fn install_in_tree_compute_drivers(registry: &mut ComputeDriverRegistry) {
}
}
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-kubernetes"))]
#[derive(Clone, Copy)]
struct KubernetesFactory;
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-kubernetes"))]
#[async_trait::async_trait]
impl openshell_server::ComputeDriverFactory for KubernetesFactory {
async fn build(
@@ -219,11 +271,11 @@ impl openshell_server::ComputeDriverFactory for KubernetesFactory {
}
}
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-docker"))]
#[derive(Clone, Copy)]
struct DockerFactory;
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-docker"))]
#[async_trait::async_trait]
impl openshell_server::ComputeDriverFactory for DockerFactory {
async fn build(
@@ -251,11 +303,11 @@ impl openshell_server::ComputeDriverFactory for DockerFactory {
}
}
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-podman"))]
#[derive(Clone, Copy)]
struct PodmanFactory;
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-podman"))]
#[async_trait::async_trait]
impl openshell_server::ComputeDriverFactory for PodmanFactory {
async fn build(
@@ -289,11 +341,11 @@ impl openshell_server::ComputeDriverFactory for PodmanFactory {
}
}
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-vm"))]
#[derive(Clone, Copy)]
struct VmFactory;
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(not(target_os = "windows"), feature = "compute-driver-vm"))]
#[async_trait::async_trait]
impl openshell_server::ComputeDriverFactory for VmFactory {
async fn build(
@@ -333,7 +385,14 @@ impl openshell_server::ComputeDriverFactory for VmFactory {
}
}
#[cfg(all(not(target_os = "windows"), feature = "in-tree-compute-drivers"))]
#[cfg(all(
not(target_os = "windows"),
any(
feature = "compute-driver-docker",
feature = "compute-driver-podman",
feature = "compute-driver-vm"
)
))]
fn apply_guest_tls(
ca: &mut Option<std::path::PathBuf>,
cert: &mut Option<std::path::PathBuf>,
@@ -351,19 +410,17 @@ fn apply_guest_tls(
}
}
#[cfg(all(test, target_os = "windows", feature = "in-tree-compute-drivers"))]
#[cfg(all(test, target_os = "windows"))]
mod windows_tests {
use super::*;
#[test]
fn windows_builtin_compute_drivers_report_unsupported() {
let registry = install_default_compute_drivers();
assert_eq!(
registry.installed_driver_names().collect::<Vec<_>>(),
["docker", "kubernetes", "mxc", "podman", "vm"]
);
for name in ["docker", "kubernetes", "podman", "vm"] {
for name in registry
.installed_driver_names()
.filter(|name| *name != "mxc")
{
let message = unsupported_windows_compute_driver(name).to_string();
assert!(
message.contains("unsupported on Windows"),
@@ -372,3 +429,30 @@ mod windows_tests {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn default_registry_contains_exactly_the_enabled_compute_drivers() {
let expected: Vec<&str> = vec![
#[cfg(feature = "compute-driver-docker")]
"docker",
#[cfg(feature = "compute-driver-kubernetes")]
"kubernetes",
#[cfg(all(target_os = "windows", feature = "compute-driver-mxc"))]
"mxc",
#[cfg(feature = "compute-driver-podman")]
"podman",
#[cfg(feature = "compute-driver-vm")]
"vm",
];
assert_eq!(
install_default_compute_drivers()
.installed_driver_names()
.collect::<Vec<_>>(),
expected
);
}
}
@@ -40,6 +40,33 @@ an exited canonical process remains a terminal sandbox result. Exit code zero
produces `Completed`; a nonzero or signal-normalized exit produces `Error`
with the exact exit code. Driver and supervisor failures remain `Error`.
## Build with Selected Compute Drivers
Source builds of `openshell-gateway` can include any subset of the Docker,
Podman, Kubernetes, VM, and MXC drivers. Enable the corresponding
`compute-driver-docker`, `compute-driver-podman`, `compute-driver-kubernetes`,
`compute-driver-vm`, or `compute-driver-mxc` Cargo features. For example, build
a Docker-only gateway with telemetry support:
```shell
cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-docker
```
On Windows, select only MXC with:
```shell
cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-mxc,bundled-z3
```
The default `in-tree-compute-drivers` feature retains the full platform driver
set, including MXC on Windows. MXC links only on Windows. The other four
features link drivers on non-Windows platforms; on Windows they install
registrations that report the driver as unsupported. A build with no default
features and no driver features connects to external drivers only.
Auto-detection probes only compiled registrations.
To select a driver omitted from a custom build, configure its external
`socket_path` as described below.
## Configure a Compute Driver
Configure the compute driver on the gateway. Current releases accept one driver per gateway. Set `compute_drivers` in the gateway TOML file:
+15 -5
View File
@@ -21,11 +21,21 @@ The target deployment flow is:
The `openshell-gateway` composition crate explicitly installs its compiled
Docker, Podman, Kubernetes, and VM registrations at startup; `openshell-server`
does not link compute-driver crates. With no configured driver, the
gateway probes only installed registrations in priority order (Kubernetes,
Podman, then Docker); VM has no probe and remains opt-in. A custom gateway
binary may install a different set, so confirm the binary's registered drivers
when auto-detection reports that no suitable driver is available.
does not link compute-driver crates. Custom gateway binaries may include a
subset of those registrations. With no configured driver, the gateway probes only
installed registrations in priority order (Kubernetes, Podman, then Docker);
VM has no probe and remains opt-in. Confirm the binary's registered drivers
when auto-detection reports that no suitable driver is available. If
configuration selects a driver that was not compiled in, the gateway treats
the name as an external driver and reports a missing `socket_path` unless an
endpoint is configured.
On Windows, custom binaries can include MXC independently. Registrations for
Docker, Podman, Kubernetes, and VM are rejection stubs when included; they do
not enable those runtimes on Windows.
See the [compute driver reference](https://docs.nvidia.com/openshell/latest/reference/sandbox-compute-drivers.md)
for selective-build options and external-driver configuration.
For local evaluation only, TLS may be disabled and the gateway can be reached through `http://127.0.0.1:<port>`.
+11 -1
View File
@@ -536,7 +536,8 @@ function Invoke-UnsupportedContractTests([string] $RustTarget) {
Assert-NativeTestTarget $RustTarget
$tests = @(
"windows_builtin_compute_drivers_report_unsupported"
"windows_builtin_compute_drivers_report_unsupported",
"default_registry_contains_exactly_the_enabled_compute_drivers"
)
foreach ($test in $tests) {
Invoke-VsCargo `
@@ -544,6 +545,15 @@ function Invoke-UnsupportedContractTests([string] $RustTarget) {
-CargoArgs "cargo test -p openshell-gateway --target $RustTarget $test $Z3ServerFeatures" `
-LogName "test-$RustTarget-unsupported-$test.log"
}
foreach ($features in @("", "compute-driver-mxc", "compute-driver-docker", "compute-driver-mxc,compute-driver-docker")) {
$featureArgs = if ($features) { "--features $features" } else { "" }
$variant = if ($features) { $features.Replace(",", "-") } else { "protocol-only" }
Invoke-VsCargo `
-RustTarget $RustTarget `
-CargoArgs "cargo test -p openshell-gateway --lib --target $RustTarget --no-default-features $featureArgs $Z3ServerFeatures" `
-LogName "test-$RustTarget-selective-$variant.log"
}
}
function Get-Sha256([string] $Path) {