Files
OpenShell/crates/openshell-driver-mxc/README.md
T
26f2f96393 feat(mxc): add Windows host proxy for MXC sandbox network egress (#3163)
* Implement Windows host proxy integration and update dependencies for OpenShell

* Update README and gateway config to clarify egress proxy address handling and allocation

* Refactor ProxyIdentityMode to return Result for static_binary and add tests for binary path and SHA256 hash

* Enhance platform_hosts_path for Windows to use SystemRoot and improve error handling for hosts file reading

* Refactor FileFingerprint to use Option for mtime and ctime, simplifying metadata handling

* Add conditional compilation for Windows host module

* add unit tests for OPA policy evaluation and identity handling

* remove openshell-supervisor-network from unsupported driver package test exclusion list

* feat(mxc): enable host proxy TLS state generation

Generate per-sandbox TLS state for the MXC host proxy so HTTPS L7 enforcement can use the same MITM path as Linux. Grant generated CA material to the MXC process and inject standard trust env vars, while matching Linux behavior by disabling TLS termination on CA setup failure and relying on proxy fail-closed handling.

* fix(docs): remove outdated notes on governed egress from docs

* fix(tests): update TLS environment variable paths to use temporary directory

* fix(examples): make run-mxc-e2e harness correct and orphan-free

The MXC e2e harness never actually exercised the fs scenarios: it started
the gateway once and patched agent_command per scenario AFTERWARDS, so the
running gateway kept launching the default demo agent (not shipped in the
kit) and every fs scenario failed with CreateProcessW error:2. It also
scored on the `sandbox create` exit code (non-zero due to the harmless
interactive attach), wrote sandbox records to the persistent gateway DB
(leaving orphans that collided on later runs), and its deny scenarios never
proved denial.

Changes:
- Start a FRESH gateway per scenario so each scenario's agent_command is
  actually loaded (root cause of CreateProcessW error:2).
- Score by on-disk artifact / expected outcome, not `sandbox create` exit.
- Real deny assertions: a control write to a granted path must succeed
  (proves the agent ran) while the denied write must be absent. fs-empty
  probes an ungranted out-of-share path (share_dir is mapped rw by design).
- Run the gateway on an ephemeral in-memory DB (sqlite::memory:) so the
  harness never writes to the persistent store and cannot leave orphan
  sandbox records; also use unique per-run sandbox names + pre-delete.
- Fix the process_container probe: use a real cwd + absolute cmd.exe
  (canonical wxc-exec does not expand %TEMP% -> 0x8007010B).
- Fix summary counts (@() so a single FAIL is counted and exit is non-zero).

Verified PASS=4 FAIL=0 on 7F203-MXC-003 (no BaseContainer velocity keys)
using a canonical wxc-exec build (AppContainer fallback).

Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(e2e): probe timeout is milliseconds (10ms->30000ms)

MXC process.timeout is wall-clock ms (wire.rs). The 10 value meant 10ms,
which the base-container tier (7F203-MXC-001/.181) enforced strictly and
timed the probe out. AppContainer path (.18/-003) happened to slip under
it. Bump to 30000ms so the process_container preflight probe is reliable
across both tiers.

Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(mxc): use native paths in real runtime probes

Signed-off-by: Shailendra Singh <shailendras@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(mxc): make processcontainer work with mxc-latest-released wxc-exec

Three fixes to support the release wxc-exec binary (BaseContainer dispatcher)
in addition to mxc-fixes-env-vars:

1. Seed process env from host (driver.rs)
   ProcessContainer starts with a completely blank environment -- no PATH,
   SystemRoot, or anything.  Seed the process env from the gateway host
   environment so the agent binary can locate DLLs and run.  Skip internal
   Windows drive-letter variables (keys starting with '=') which cause
   CreateProcessW to return ERROR_ENVVAR_NOT_FOUND.  User agent_env entries
   and TLS CA vars are applied as overrides on top of the host env.

2. Remove TLS readonly_paths grant (driver.rs)
   The release wxc-exec (BaseContainer dispatcher) requires write-DAC
   permission on every path in readonly_paths to set up AppContainer ACLs.
   Adding the proxy's temp TLS directory caused a DACL error and exit -1.
   The CA cert paths remain available to the agent via TLS env vars.

3. Remove allowedHosts from network JSON (mxc.rs)
   The release wxc-exec rejects network.allowedHosts / network.blockedHosts
   on Windows with "not yet supported".  Removed the loopback exemption
   attempt (127.0.0.1, ::1, localhost) from the network section.
   Intra-container loopback works natively in the release binary without
   it -- the spawner can connect to the server at 127.0.0.1:22000 directly.

Additional changes:
- mxc-ws-agent.rs: add relay-debug.txt error capture and relay-ready.txt
  marker for reliable timing of host client connections.
- mxc-ws-gateway.toml: debug = true for JSON config dump during diagnosis.
- run-ws-agent-test.ps1: default port changed to 17670 (gateway default);
  relay-ready.txt polling before ws-echo to avoid connecting before the
  spawner has established the proxy bridge.

Signed-off-by: Prashant Khodade <pkhodade@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(e2e): address CodeRabbit review on run-mxc-e2e.ps1 (MR !46)

Four robustness/correctness fixes from CodeRabbit:

1. Start-Gw: kill the spawned gateway before the "did not start within 30s"
   throw. If the process is alive but never binds the port, $gw is not yet
   assigned in the caller, so the finally block cannot reap it -> orphan
   gateway holding the port for the next run.

2. create-fail scoring: a non-zero `sandbox create` exit alone is not proof
   of a policy rejection (gateway-registration/transport/fixture errors also
   exit non-zero and would false-pass). PASS now requires a genuine
   rejection signal (network / invalid_argument / network_policies) AND that
   it is not an infrastructure failure; other non-zero exits go to FAIL with
   output captured.

3. deny scenarios (ControlTarget path): snapshot the deny target AFTER
   Wait-File lands the control artifact, so a late denied write (enforcement
   regression racing the control write) can no longer be recorded as PASS.

4. -KeepRunning: break out of the scenario loop after the first scenario so
   a later scenario does not start a second gateway on the same port
   (previously a reliable port collision instead of a usable debug mode).

Re-verified PASS=4 FAIL=0 on both boxes (7F203-MXC-001 base-container and
7F203-MXC-003 AppContainer fallback); network-policy-rejected correctly
scores as "policy rejection".

Signed-off-by: Akber Raza <akberr@nvidia.com>

* feat(mxc-e2e): collect run-mxc-e2e output into a results bundle

Mirror the sibling run-*.ps1 scripts by collecting every run's logs into a
timestamped results-e2e-<stamp>\ folder and zipping it. The bundle contains the
console transcript, per-scenario gateway stdout/stderr, the exact TOML rendered
for each scenario, the policy fixture used, and a summary.txt with the verdict
table.

Per-scenario gateway logs now land in gateway.<scenario>.log/.err.log inside the
bundle instead of a single fixed gateway.e2e.log in the script directory.

Wrap pre-flight, mode setup, scenario definitions, and the scenario loop in a
single try/catch/finally so the finally always writes the summary, stops the
transcript, and zips the bundle -- even on a pre-flight failure. The existing
per-scenario gateway-cleanup try/finally stays nested inside. All scenario
logic, scoring rules, and comments are preserved.

Signed-off-by: Prashant S Khodade <pkhodade@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(mxc-e2e): address CodeRabbit review on run-mxc-e2e.ps1

- Require -Scenario when -KeepRunning: the loop breaks after the first
  scenario, so a full-suite run would execute only one scenario yet still
  report the suite as PASS. Fail fast so a partial run can't be mislabeled
  complete.
- Start-Transcript now runs inside the guarded try block with a
  $transcriptStarted flag; Stop-Transcript is only called when it actually
  started, so a Start-Transcript failure still yields the results bundle.
- Wrap the -Scenario filter in @() so a single exact match stays an array
  (reliable .Count and a proper array for the scenario loop on PS 5.1).

Signed-off-by: Prashant S Khodade <pkhodade@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(examples): pass gateway config via OPENSHELL_GATEWAY_CONFIG for spaced paths

Start-Process -ArgumentList does not quote array elements, so launching the
gateway with a bare --config <path> token split on any space in the install
path (e.g. C:\Users\First Last\...), and clap rejected the fragment with
'unrecognized subcommand'. Every MXC example launcher that started the gateway
hit this when the kit was unzipped under a path containing a space.

Pass the config path through the OPENSHELL_GATEWAY_CONFIG env var (which the
gateway already reads via clap) and drop the --config token. Env vars carry
spaces safely.

Affected: run-ocsf-audit, run-mxc-e2e, run-demo, run-inference-test,
run-ollama-test. run-mtls-test was not affected (its launch passes no config
path). Root-caused and fix-verified on 7F203-MXC-003 from a spaced path.

Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(run-mxc-e2e): improve scoring logic and enhance command execution handling

* fix(mxc): reconcile proxy support after rebase

Restore the proxy-enabled OCSF audit example removed by 13185f6e now that the host CONNECT proxy is present. Adapt the proxy lifecycle test to the target branch's DriverSandboxSpec policy delivery contract.

Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(mxc): grant sandbox access to proxy CA

- Share the per-sandbox public CA bundle with the AppContainer
- Add real wxc-exec HTTPS proxy coverage and document trust isolation

Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(mxc): reject unsupported network middleware

- Reject middleware-bearing MXC policies before sandbox lifecycle begins
- Guard host proxy startup and document the unsupported registry path
- Add mapper, lifecycle, and host proxy regression coverage

Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(mxc): reconcile host proxy with main

- remove obsolete inference routing from the host proxy adapter
- use the workspace AWS-LC provider in host-proxy tests
- adapt the forward-proxy test to ProxyIdentityMode

Signed-off-by: Akber Raza <akberr@nvidia.com>

* fix(build): switch to bundled Z3 for Windows MSVC builds

* fix(mxc): reconcile host proxy after rebase

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Akber Raza <akberr@nvidia.com>
Signed-off-by: Shailendra Singh <shailendras@nvidia.com>
Signed-off-by: Prashant Khodade <pkhodade@nvidia.com>
Signed-off-by: Prashant S Khodade <pkhodade@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Jamie King <jamiek@nvidia.com>
Co-authored-by: Shailendra Singh <shailendras@nvidia.com>
Co-authored-by: Prashant Khodade <pkhodade@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
2026-09-14 22:58:49 +00:00

11 KiB

openshell-driver-mxc

OpenShell compute driver backed by Microsoft MXC (wxc-exec) on Windows.

Design

This driver implements the gateway's ordinary in-process ComputeDriver contract and is linked into openshell-gateway. It sets driver_reports_runtime_readiness, so the gateway accepts driver-reported readiness without a supervisor session. The canonical create-time SandboxPolicy is carried by DriverSandboxSpec.policy. process_container launches a one-shot AppContainer and is the default. The opt-in isolation_session backend uses the state-aware provision → start → exec → stop → deprovision lifecycle. The driver launches and monitors the configured workload itself and self-reports readiness; there is no in-sandbox supervisor or ConnectSupervisor relay.

Capability Matrix

Capability MXC driver Closing it requires
Filesystem policy (read-write / read-only grants) ✅ provision-time AppContainer shares —
Governed egress (CONNECT proxy + OPA + L7) Available behind egress_proxy on process_container; the driver starts a per-sandbox host CONNECT proxy, generates HTTPS MITM trust material, and injects the CA bundle into the sandbox process env Gateway event-bus wiring follow-on
Network policy Split into MXC network.proxy + trimmed OpenShell policy on process_container; isolation_session still rejects network config MXC feedback item M1 for persistent sessions
Network middleware ❌ rejected before launch because the MXC host proxy does not receive the gateway middleware registry Gateway middleware-registry injection
Process policy (seccomp, uid/gid) ❌ host-side governance design; OS isolation only not pursued
Interactive exec/connect/forward ❌ exec runs in-driver, no client attach gateway interactive-exec surgery (follow-on)
Bundled agent image ❌ no OCI image; relies on Windows host install —
Restart durability ❌ in-memory registry; restart orphans live sessions follow-on
Concurrent sandboxes ⚠️ isolation_session v1 is single-session MXC backend feature

The filesystem enforcement proof has two paths:

  • A write to a path granted by the sandbox policy succeeds.
  • A process_container write outside the sandbox policy fails with Windows access denied, and the driver reports the failed workload.

Configuration ([openshell.drivers.mxc])

Gateway configuration contains only host runtime settings:

[openshell.drivers.mxc]
wxc_exec_path = "C:\\path\\to\\wxc-exec.exe"
# Default: process_container. isolation_session is grant-only and opt-in.
backend = "process_container"
default_configuration_id = "composable"
pc_least_privilege = false
pc_capabilities = []
# Pattern-C governed egress. The address is a loopback seed; each sandbox
# receives a unique ephemeral proxy port.
egress_proxy = false
egress_proxy_addr = ""
debug = false
etw_audit = false

When egress_proxy is enabled, egress_proxy_addr must be a loopback IP:PORT seed. The driver preserves the configured IP and allocates a unique ephemeral port for each sandbox's network.proxy redirect.

Supply workload settings for each sandbox. The public config is keyed by driver name; the gateway forwards only the inner mxc object to the driver:

$config = '{"mxc":{"command":["cmd","/c","echo hello > C:\\\\work\\\\demo\\\\hello.txt"],"cwd":"C:\\\\work\\\\demo"}}'
openshell sandbox create --name mxc-demo --policy demo.yaml `
  --driver-config-json $config --env MODE=demo --no-tty

The command array is required and preserves Windows argument boundaries. cwd is optional. Environment variables come from the standard sandbox and template environment maps; the driver never copies values from the gateway host environment.

The host CONNECT proxy enforces network policy when governed egress is enabled. Live policy replacement or merge updates remain unsupported; delete and recreate the sandbox to apply a different policy.

When etw_audit is enabled, each gateway process owns a distinct real-time ETW session named from the stable OpenShell-MXC-ETW prefix, its process ID, and a per-start discriminator. Starting another gateway never stops an existing gateway's capture. Graceful shutdown stops the session by its owned handle. A force-killed gateway can leave a stale session; the audit example removes only matching sessions whose encoded owner process is no longer running.

The gateway-local OCSF JSONL sink is available only for the Windows/MXC path and is opt-in. Set OPENSHELL_OCSF_JSON=1 to enable it and optionally set OPENSHELL_OCSF_LOG_DIR to override its %PROGRAMDATA%\OpenShell\logs default. Other gateway deployments do not initialize this local file sink.

The ETW callback uses a non-blocking queue capped at 4,096 records and 16 MiB of copied event data. Records that exceed either limit are dropped instead of blocking the ETW pump or growing gateway memory. The gateway emits an immediate warning identifying the audit coverage gap and rate-limits follow-up warnings to once every 30 seconds while overload continues.

Audit attribution bootstraps only when the driver-owned wxc-exec PID and its kernel process start key both match the values attached to the ETW record; command text is never an ownership key. This generation key prevents a recycled PID from inheriting the previous process's attribution regardless of delivery delay. The process monitor retires the live PID at exit. Established identity, activity, and correlation-vector links remain available for five seconds so already in-flight ETW records can arrive, but retired PID evidence cannot resolve them. Records without matching generation evidence remain unattributed.

Prerequisites (live runs)

  • Windows 11 Insider build ≥ 26300.8553
  • IsoSessionApp.dll present and registered
  • wxc-exec.exe built with --features isolation_session
  • Any enforced App Control policy allows both openshell-gateway.exe and openshell.exe. Diagnose executable blocks with event 3077 in the Microsoft-Windows-CodeIntegrity/Operational log.

For off-box smoke tests against the in-process mock shim (no wxc-exec, no isolation session needed), set OPENSHELL_MXC_MOCK_WXC=1.

Policy mapping

The production driver maps the typed SandboxPolicy carried by the standard driver request to MXC configuration before it inserts a registry entry or invokes wxc-exec. Mapping failure therefore returns from CreateSandbox without leaving a partial sandbox. There is no in-process policy side channel or MXC-specific gateway composition variant.

When egress_proxy is enabled, EmbeddedPolicyMapper uses split_policy instead: MXC receives filesystem grants plus a loopback network.proxy redirect, and the driver starts a host CONNECT proxy from the trimmed network-only SandboxPolicy. Policies containing network_middlewares are rejected synchronously until this host-proxy path can receive the gateway's built-in and remote middleware registry. The proxy uses the configured agent command as the static sandbox process identity because MXC does not expose Linux-style procfs socket ownership. For HTTPS L7 inspection, the host proxy generates a per-sandbox CA and injects NODE_EXTRA_CA_CERTS, DENO_CERT, SSL_CERT_FILE, REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, and GIT_SSL_CAINFO into the agent process env. It does not add the generated CA directory to MXC read-only grants: released wxc-exec BaseContainer builds require WRITE_DAC on every such grant and reject the user-owned proxy temp directory. Instead, the driver adds the sandbox-unique directory as an internal read-write share so HTTPS clients can read the injected paths. The directory contains only public CA certificates; the ephemeral CA private key remains in the host proxy's memory. The driver seeds only SYSTEMROOT, WINDIR, PATH, COMSPEC, and LOCALAPPDATA from the gateway host before applying sandbox and TLS overrides, so required Windows bootstrap values remain available without exposing the gateway's full environment. The development export surface remains the policy-to-mxc example; there is no production openshell policy export-mxc subcommand yet.

If governed egress is disabled, any network rule fails closed rather than launching without an enforcement path.

Parity and matrix tests under tests/ cover the mapper on the Windows MSVC lane. The driver performs this mapping automatically; there is no separate policy-export command or example.

Packaging the demo for the demo box

Use examples/package-demo.ps1 to assemble the gateway EXE, CLI EXE, runtime DLLs (libz3.dll), demo.yaml, the gateway config, and the runbook into one folder, then copy that folder to the demo Windows host and follow mxc-demo-runbook.md inside it. The script prints a SHA256 manifest so the operator can sanity-check what landed before moving it.

Real-MXC test lane

Three tasks drive real wxc-exec.exe hardware; all are skip-safe — any test or scenario that requires an absent binary or backend prints a SKIP reason and exits 0 rather than failing.

Task What it runs When to use
windows:test:mxc-real:x64 tests/wxc_exec_real.rs — Tier-2 invoker tests with --ignored --test-threads=1, including an HTTPS request through the host proxy Pre-merge on any Windows host that has wxc-exec; dry-run tests always pass; enforcement tests probe-gate themselves
windows:e2e:mxc examples/run-mxc-e2e.ps1 — Tier-3 scenario runner, real binary, probe-gated Demo box / nightly; needs the gateway + CLI binaries in the script directory
windows:e2e:mxc:mock Same runner with -Mock — wiring-only, no real wxc-exec needed Any Windows host (CI, dev machine); validates wiring and the network-reject scenario

Probe script: examples/probe-mxc-host.ps1 is an operator/CI preflight that emits a JSON capability report (OS build, wxc-exec path/version, dry-run exit code, per-backend trial result, and a verdicts object). Run it before the real-MXC lane to understand what will PASS vs SKIP on a given host:

The probe uses a unique, user-owned Windows temp directory for every run. MXC treats config paths literally (it does not expand %TEMP%), and the per-run directory keeps AppContainer+DACL fallback mutations narrowly scoped.

powershell -NoProfile -ExecutionPolicy Bypass `
  -File crates/openshell-driver-mxc/examples/probe-mxc-host.ps1

Skip semantics: tests in wxc_exec_real.rs are marked #[ignore = "requires real wxc-exec"] — the standard windows:test:x64 suite never runs them. OPENSHELL_WXC_EXEC_PATH overrides the default C:\mxc\wxc-exec.exe lookup. See docs4gtb/mxc-box-capabilities.md for the empirical capability snapshot of the development box (build 26200, processcontainer velocity keys not enabled, isolation_session absent).

Deferred work

  • Interactive exec/connect/forward — gateway interactive-exec surgery (follow-on)
  • Restart durability (deprovision orphaned sessions on startup) → follow-on
  • GPU passthrough → not pursued in host-side-governance design