* fix(ocsf): attribute MXC proxy events to sandboxes
NVBug 6783086
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
* fix(network): scope Windows egress by socket owner
Resolve each accepted Windows proxy connection to its unique owning PID and executable before evaluating binary-scoped network policy. Fail closed when ownership or process identity cannot be established, and cover allowed and undeclared child processes with a real MXC regression.
* fix(network): preserve sandbox context with socket owners
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
(cherry picked from commit 32ea316deb6c83c157fdf2de4a2cdc3a7d2da1e3)
* fix(network): harden Windows process identity
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
---------
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
* fix(network): normalize Windows policy binary paths
Match Windows executable identities using a stable case-insensitive, separator-normalized representation across policy data, L4 input, and L7 relay evaluation. Preserve exact matching on other platforms and keep the original path for hashing and filesystem access.
NVBug 6782969
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
* fix(network): harden Windows binary matching
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
* fix(ci): scope Windows relay test imports
* fix(network): harden Windows binary path matching
---------
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Validate raw OPA object and array containers before normalization and
access-preset expansion can skip malformed values. Return one fixed
structural error without embedding authored policy data.
Preserve versionless and runtime-only OPA data and existing semantic
validation. Cover initial string/file loading, middleware callback order,
valid deny-rule enforcement, and rejected reload state and generation.
Document the loader contract and its engine-local rejection behavior.
Signed-off-by: Shiju <shiju@nvidia.com>
Previously, metadata events omitted both HTTP request and response objects,
early proxy rejections used HTTP Activity without request context, and
unsupported-scheme events did not expose enough safe HTTP context to satisfy
the OCSF 1.8 schema.
Now, metadata events include a method-only request and their actual HTTP
response codes without recording the metadata URL. Unsupported-scheme events
also include a method-only request plus the generated 400 response. Authority
mismatches and credential-resolution denials use HTTP Activity with their
generated 403 or 500 responses, and HTTP activity IDs are derived from the
request method.
Additionally, HttpActivityBuilder now enforces the OCSF request-or-response
constraint at compile time.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Show configured tool server addresses and their last observed connection
results together in sandbox status. Keep sandbox lifecycle readiness
separate so an external connection failure does not mark the sandbox unready.
Expose direct endpoint records through the CLI and SDKs, with plain-language
failure explanations and gateway acceptance times. Keep observation tracking,
runtime reporting, and gateway validation in dedicated endpoint status modules.
Preserve bounded reporting, request attribution, retry ordering, and
configuration and supervisor authority checks. Clear obsolete observations
while retaining the configured addresses, and document the distinction
between an observed HTTP response, current availability, and tool success.
Signed-off-by: Shiju <shiju@nvidia.com>
* Implement Windows host proxy integration and update dependencies for OpenShell
* Update README and gateway config to clarify egress proxy address handling and allocation
* Refactor ProxyIdentityMode to return Result for static_binary and add tests for binary path and SHA256 hash
* Enhance platform_hosts_path for Windows to use SystemRoot and improve error handling for hosts file reading
* Refactor FileFingerprint to use Option for mtime and ctime, simplifying metadata handling
* Add conditional compilation for Windows host module
* add unit tests for OPA policy evaluation and identity handling
* remove openshell-supervisor-network from unsupported driver package test exclusion list
* feat(mxc): enable host proxy TLS state generation
Generate per-sandbox TLS state for the MXC host proxy so HTTPS L7 enforcement can use the same MITM path as Linux. Grant generated CA material to the MXC process and inject standard trust env vars, while matching Linux behavior by disabling TLS termination on CA setup failure and relying on proxy fail-closed handling.
* fix(docs): remove outdated notes on governed egress from docs
* fix(tests): update TLS environment variable paths to use temporary directory
* fix(examples): make run-mxc-e2e harness correct and orphan-free
The MXC e2e harness never actually exercised the fs scenarios: it started
the gateway once and patched agent_command per scenario AFTERWARDS, so the
running gateway kept launching the default demo agent (not shipped in the
kit) and every fs scenario failed with CreateProcessW error:2. It also
scored on the `sandbox create` exit code (non-zero due to the harmless
interactive attach), wrote sandbox records to the persistent gateway DB
(leaving orphans that collided on later runs), and its deny scenarios never
proved denial.
Changes:
- Start a FRESH gateway per scenario so each scenario's agent_command is
actually loaded (root cause of CreateProcessW error:2).
- Score by on-disk artifact / expected outcome, not `sandbox create` exit.
- Real deny assertions: a control write to a granted path must succeed
(proves the agent ran) while the denied write must be absent. fs-empty
probes an ungranted out-of-share path (share_dir is mapped rw by design).
- Run the gateway on an ephemeral in-memory DB (sqlite::memory:) so the
harness never writes to the persistent store and cannot leave orphan
sandbox records; also use unique per-run sandbox names + pre-delete.
- Fix the process_container probe: use a real cwd + absolute cmd.exe
(canonical wxc-exec does not expand %TEMP% -> 0x8007010B).
- Fix summary counts (@() so a single FAIL is counted and exit is non-zero).
Verified PASS=4 FAIL=0 on 7F203-MXC-003 (no BaseContainer velocity keys)
using a canonical wxc-exec build (AppContainer fallback).
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(e2e): probe timeout is milliseconds (10ms->30000ms)
MXC process.timeout is wall-clock ms (wire.rs). The 10 value meant 10ms,
which the base-container tier (7F203-MXC-001/.181) enforced strictly and
timed the probe out. AppContainer path (.18/-003) happened to slip under
it. Bump to 30000ms so the process_container preflight probe is reliable
across both tiers.
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(mxc): use native paths in real runtime probes
Signed-off-by: Shailendra Singh <shailendras@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(mxc): make processcontainer work with mxc-latest-released wxc-exec
Three fixes to support the release wxc-exec binary (BaseContainer dispatcher)
in addition to mxc-fixes-env-vars:
1. Seed process env from host (driver.rs)
ProcessContainer starts with a completely blank environment -- no PATH,
SystemRoot, or anything. Seed the process env from the gateway host
environment so the agent binary can locate DLLs and run. Skip internal
Windows drive-letter variables (keys starting with '=') which cause
CreateProcessW to return ERROR_ENVVAR_NOT_FOUND. User agent_env entries
and TLS CA vars are applied as overrides on top of the host env.
2. Remove TLS readonly_paths grant (driver.rs)
The release wxc-exec (BaseContainer dispatcher) requires write-DAC
permission on every path in readonly_paths to set up AppContainer ACLs.
Adding the proxy's temp TLS directory caused a DACL error and exit -1.
The CA cert paths remain available to the agent via TLS env vars.
3. Remove allowedHosts from network JSON (mxc.rs)
The release wxc-exec rejects network.allowedHosts / network.blockedHosts
on Windows with "not yet supported". Removed the loopback exemption
attempt (127.0.0.1, ::1, localhost) from the network section.
Intra-container loopback works natively in the release binary without
it -- the spawner can connect to the server at 127.0.0.1:22000 directly.
Additional changes:
- mxc-ws-agent.rs: add relay-debug.txt error capture and relay-ready.txt
marker for reliable timing of host client connections.
- mxc-ws-gateway.toml: debug = true for JSON config dump during diagnosis.
- run-ws-agent-test.ps1: default port changed to 17670 (gateway default);
relay-ready.txt polling before ws-echo to avoid connecting before the
spawner has established the proxy bridge.
Signed-off-by: Prashant Khodade <pkhodade@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(e2e): address CodeRabbit review on run-mxc-e2e.ps1 (MR !46)
Four robustness/correctness fixes from CodeRabbit:
1. Start-Gw: kill the spawned gateway before the "did not start within 30s"
throw. If the process is alive but never binds the port, $gw is not yet
assigned in the caller, so the finally block cannot reap it -> orphan
gateway holding the port for the next run.
2. create-fail scoring: a non-zero `sandbox create` exit alone is not proof
of a policy rejection (gateway-registration/transport/fixture errors also
exit non-zero and would false-pass). PASS now requires a genuine
rejection signal (network / invalid_argument / network_policies) AND that
it is not an infrastructure failure; other non-zero exits go to FAIL with
output captured.
3. deny scenarios (ControlTarget path): snapshot the deny target AFTER
Wait-File lands the control artifact, so a late denied write (enforcement
regression racing the control write) can no longer be recorded as PASS.
4. -KeepRunning: break out of the scenario loop after the first scenario so
a later scenario does not start a second gateway on the same port
(previously a reliable port collision instead of a usable debug mode).
Re-verified PASS=4 FAIL=0 on both boxes (7F203-MXC-001 base-container and
7F203-MXC-003 AppContainer fallback); network-policy-rejected correctly
scores as "policy rejection".
Signed-off-by: Akber Raza <akberr@nvidia.com>
* feat(mxc-e2e): collect run-mxc-e2e output into a results bundle
Mirror the sibling run-*.ps1 scripts by collecting every run's logs into a
timestamped results-e2e-<stamp>\ folder and zipping it. The bundle contains the
console transcript, per-scenario gateway stdout/stderr, the exact TOML rendered
for each scenario, the policy fixture used, and a summary.txt with the verdict
table.
Per-scenario gateway logs now land in gateway.<scenario>.log/.err.log inside the
bundle instead of a single fixed gateway.e2e.log in the script directory.
Wrap pre-flight, mode setup, scenario definitions, and the scenario loop in a
single try/catch/finally so the finally always writes the summary, stops the
transcript, and zips the bundle -- even on a pre-flight failure. The existing
per-scenario gateway-cleanup try/finally stays nested inside. All scenario
logic, scoring rules, and comments are preserved.
Signed-off-by: Prashant S Khodade <pkhodade@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(mxc-e2e): address CodeRabbit review on run-mxc-e2e.ps1
- Require -Scenario when -KeepRunning: the loop breaks after the first
scenario, so a full-suite run would execute only one scenario yet still
report the suite as PASS. Fail fast so a partial run can't be mislabeled
complete.
- Start-Transcript now runs inside the guarded try block with a
$transcriptStarted flag; Stop-Transcript is only called when it actually
started, so a Start-Transcript failure still yields the results bundle.
- Wrap the -Scenario filter in @() so a single exact match stays an array
(reliable .Count and a proper array for the scenario loop on PS 5.1).
Signed-off-by: Prashant S Khodade <pkhodade@nvidia.com>
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(examples): pass gateway config via OPENSHELL_GATEWAY_CONFIG for spaced paths
Start-Process -ArgumentList does not quote array elements, so launching the
gateway with a bare --config <path> token split on any space in the install
path (e.g. C:\Users\First Last\...), and clap rejected the fragment with
'unrecognized subcommand'. Every MXC example launcher that started the gateway
hit this when the kit was unzipped under a path containing a space.
Pass the config path through the OPENSHELL_GATEWAY_CONFIG env var (which the
gateway already reads via clap) and drop the --config token. Env vars carry
spaces safely.
Affected: run-ocsf-audit, run-mxc-e2e, run-demo, run-inference-test,
run-ollama-test. run-mtls-test was not affected (its launch passes no config
path). Root-caused and fix-verified on 7F203-MXC-003 from a spaced path.
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(run-mxc-e2e): improve scoring logic and enhance command execution handling
* fix(mxc): reconcile proxy support after rebase
Restore the proxy-enabled OCSF audit example removed by 13185f6e now that the host CONNECT proxy is present. Adapt the proxy lifecycle test to the target branch's DriverSandboxSpec policy delivery contract.
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(mxc): grant sandbox access to proxy CA
- Share the per-sandbox public CA bundle with the AppContainer
- Add real wxc-exec HTTPS proxy coverage and document trust isolation
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(mxc): reject unsupported network middleware
- Reject middleware-bearing MXC policies before sandbox lifecycle begins
- Guard host proxy startup and document the unsupported registry path
- Add mapper, lifecycle, and host proxy regression coverage
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(mxc): reconcile host proxy with main
- remove obsolete inference routing from the host proxy adapter
- use the workspace AWS-LC provider in host-proxy tests
- adapt the forward-proxy test to ProxyIdentityMode
Signed-off-by: Akber Raza <akberr@nvidia.com>
* fix(build): switch to bundled Z3 for Windows MSVC builds
* fix(mxc): reconcile host proxy after rebase
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
---------
Signed-off-by: Akber Raza <akberr@nvidia.com>
Signed-off-by: Shailendra Singh <shailendras@nvidia.com>
Signed-off-by: Prashant Khodade <pkhodade@nvidia.com>
Signed-off-by: Prashant S Khodade <pkhodade@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Jamie King <jamiek@nvidia.com>
Co-authored-by: Shailendra Singh <shailendras@nvidia.com>
Co-authored-by: Prashant Khodade <pkhodade@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
forward_handler_preserves_ssrf_response_and_denial_stage's ssrf_denied
check was the only assertion in the test without a failure message,
unlike its sibling assertions. This test has failed intermittently in
CI with no way to see what response was actually returned, since the
message is where that gets surfaced.
Signed-off-by: politerealism <burdcat17@gmail.com>
* fix(supervisor): canonicalize L7 protocols before OPA
Normalize recognized protocol spellings before supervisor validation and Rego authorization so mixed-case MCP policies cannot take the REST rule path. Reject null MCP configuration instead of silently defaulting it, and cover both behaviors with regression tests.
Signed-off-by: Shiju <shiju@nvidia.com>
* test(policy): cover normalization for every supported protocol
Exercise TCP, REST, WebSocket, GraphQL, SQL, JSON-RPC and MCP through YAML and protobuf loading with lowercase, uppercase and mixed-case names.
Assert canonical names in raw OPA data and returned L7 configuration, preserve TCP L4 behavior, and verify that normalization leaves unrelated data unchanged and is idempotent.
Signed-off-by: Shiju <shiju@nvidia.com>
---------
Signed-off-by: Shiju <shiju@nvidia.com>
* refactor(inference): remove managed inference routes
Closes#3172
Remove the inference route control plane, inference.local data path, built-in router crate, and SDK surface. Move inference workloads to explicitly imported provider profiles and native endpoints, with migration cleanup and updated tests and documentation.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* fix(policy): preserve alternate upstream isolation
Restore the provider policy activation guard so legacy OpenAI and Anthropic providers configured for alternate base URLs do not grant egress to the built-in public vendor endpoints.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
---------
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* chore(deps): replace ring with AWS-LC
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(lint): address warnings after dependency upgrades
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(tls): limit provider initialization to reqwest clients
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
---------
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(supervisor): preserve MCP versions in runtime config
Carry canonical MCP version allowlists through both policy-loading paths into the supervisor's typed L7 endpoint configuration. Reject malformed, ambiguous, or cross-protocol version data during policy activation so invalid configuration cannot remove MCP inspection from a selected route.
Signed-off-by: Shiju <shiju@nvidia.com>
* fix(supervisor): accept mixed-case MCP version protocols
Validate materialized MCP version metadata through the parsed L7 protocol so supported mixed-case spellings remain valid. Cover YAML and protobuf ingress with pinned-default regression tests.
Signed-off-by: Shiju <shiju@nvidia.com>
---------
Signed-off-by: Shiju <shiju@nvidia.com>
Evaluate proposal coverage once per observed installed policy snapshot while preserving replacement and deadline behavior.
Signed-off-by: Shiju <shiju@nvidia.com>
Build protobuf-derived allow objects from selectors that are actually present. This keeps protobuf and YAML policy validation aligned for protocols such as GraphQL over WebSocket.
Signed-off-by: Shiju <shiju@nvidia.com>
* feat(middleware): define HTTP response pre-return interface
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): clarify HTTP response interface
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(middleware): align response result actions
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(middleware): expose response reason codes
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(middleware): share session end reasons
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* feat(middleware)!: finalize HTTP response pre-return contract
Replace the separate body_end event with HttpResponseBodyUnit.end_of_stream.
Every body-inspecting stage receives exactly one flagged unit, which may be
empty; a zero-byte body is one empty flagged unit and OpenShell never reads
ahead to set the flag.
Defer response trailers from V1 and reserve their field numbers. HTTP/1.0
clients and Content-Length bodies cannot carry trailers and that behavior was
undefined.
Add HttpResponsePreflight.permitted_body_modes, computed once from the
original upstream head so every stage sees the same list, and make an
unlisted selection a failure rather than a downgrade. Add the block_delivery
preflight action as a successful decision enforced regardless of on_error.
Expose Content-Length, Content-Encoding, and Content-Range read-only in
preflight. Cap STREAM_BYTES input units at half of max_payload_bytes and
permit deferring bytes across replacements only for fail_closed bindings,
surfaced as deferral_permitted.
Split PEER_DISCONNECT into DOWNSTREAM_DISCONNECT and UPSTREAM_DISCONNECT and
attribute WebSocket relay failures by direction instead of a generic peer
error. Compile the content-guard example in lint and branch checks so proto
renames cannot break it silently.
BREAKING CHANGE: WebSocketSessionEndReason and WebSocketSessionEnd are
replaced by the shared MiddlewareSessionEndReason and MiddlewareSessionEnd.
NORMAL_CLOSE is now NORMAL, UPSTREAM_REJECTED is now UPSTREAM_FAILURE, and
PEER_DISCONNECT is split into DOWNSTREAM_DISCONNECT and UPSTREAM_DISCONNECT.
Enum numbers are unchanged so binary wire compatibility is preserved;
generated symbols and JSON names change.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): describe skip as opting out of inspection
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* feat(middleware): add body-phase block_delivery and skip_remaining actions
Body results may now stop delivery or opt out of inspecting the rest of the
response after a prefix. One HttpResponseBlockDelivery message is shared by
preflight and body results and documents the difference between blocking
before and after head commitment. Drop the field reservations, since nothing
in this contract has shipped, and renumber session_end to close the gap.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(middleware): share HTTP body leaf messages across directions
HttpBodyUnit, HttpBodyPassThrough, HttpBodyTransform, HttpBodySkipRemaining,
and HttpBodyMode carry no response-specific semantics, so name them for reuse
by the streaming request hook. Envelopes, results, preflight, and
block_delivery stay response-specific because commitment semantics differ.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(middleware): keep HTTP body leaf messages response-specific
Reverts the shared HttpBody* naming. A direction-specific payload such as a
response-only semantic mode would otherwise add unreachable variants to the
other direction or force a source-breaking fork after 0.1.0. The streaming
request hook defines its own HttpRequestBody* messages and copies the shape;
SDKs present a direction-neutral body handler over both.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): simplify response proto comments
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* fix(middleware): reject undispatched response bindings
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): simplify phase field comment
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(middleware): rename HTTP response preflight result
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* feat(middleware): add HTTP response trailer results
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): define response block delivery
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): define stage-local response body modes
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): define final response body units
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): define streaming response deferral
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): defer whole-body accumulation timeout
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): align response result diagnostics
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* test(middleware): cover upstream WebSocket disconnect
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* fix(middleware): keep response streams unit-local
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(middleware): trim disconnect compatibility note
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
---------
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* feat(vm): support corporate HTTP forward proxy egress for microVM sandboxes
The corporate forward proxy machinery from #1792 is driver-agnostic and
already merged: openshell-supervisor-network implements CONNECT chaining,
NO_PROXY matching, credentials, https:// proxies and corporate CA trust, and
openshell-sandbox exposes it as six argv-only flags. Podman gained the driver
half in #2245/#2512 and Kubernetes in #2633; the VM driver had none of it, so
VM sandboxes on proxy-only networks could not reach any destination requiring
the proxy even when policy allowed it.
The blocking piece was not proxy logic but delivery: the VM guest init script
runs as PID 1 and execs a fixed supervisor command line, and libkrun's
krun_set_exec receives an empty argv, so there was no channel for driver-owned
supervisor arguments. The supervisor's proxy flags deliberately have no
environment fallback, and build_guest_environment merges user-supplied
environment, so the guest env is not a safe transport either.
Add a driver-authored argument file, mirroring the existing init.d manifest:
the driver writes /opt/openshell/supervisor-args into the overlay upperdir on
every launch and the guest reads it verbatim, one argument per line, appending
it to every supervisor exec. It is written even when empty, which is what makes
the channel unforgeable -- the upperdir always shadows the read-only image
layer, so an image can neither supply its own arguments nor disable the
operator's by omitting the file. Because both launch backends exec the same
init script, this covers libkrun and QEMU without touching either.
A microVM has no bind mounts or container secrets, so the credential and CA
bundle are staged into the per-sandbox overlay the way the gateway JWT already
is: credential root-only at 0600, CA at 0644, both rewritten every launch so a
removed setting clears prior material, and both deleted with the sandbox state
directory. This places the credential at rest in the overlay image on the
gateway host, which differs from the Podman secret model and is documented as
an explicit security consideration.
Validation is fail-closed and shared: a new
openshell_core::driver_utils::validate_upstream_proxy_settings holds the
pairing rules the Podman driver established, and both the gateway and the
driver call it so an invalid table names the offending key instead of
surfacing as an opaque driver-readiness timeout.
Guest egress leaves through gvproxy, so a proxy on the gateway host's loopback
is reachable only through host.openshell.internal; the guest to gateway
callback is unaffected.
Closes#3088
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): bound the proxy CA read and scope the host-loopback recipe
Two review findings on the corporate forward proxy support for microVM
sandboxes.
The driver read the operator's proxy_ca_bundle with an unbounded fs::read and
accepted it on a substring match for the PEM BEGIN CERTIFICATE marker. A
special file such as /dev/zero therefore grew driver memory without bound on
every authorized sandbox create, and a PEM block holding invalid DER passed
the host check but contributes no trust anchor in the guest, so every
supervisor would fail after boot with an error attributed to the sandbox
rather than to the setting.
Move the read into openshell-core as read_upstream_proxy_ca_bundle_file: it
reuses the credential reader's bounded-read path (non-regular files rejected
on fstat, size capped, read bounded even if the file grows), then requires at
least one anchor that RootCertStore::add_parsable_certificates accepts. The
supervisor's own reader now delegates to it, so host acceptance and guest
acceptance are the same function and cannot drift.
The published host-loopback recipe was written for libkrun only. gvproxy NATs
host.openshell.internal to the gateway host's 127.0.0.1, but GPU sandboxes run
on the QEMU/TAP backend where that name resolves to the TAP host address and
the driver's own nftables input chain accepts only the gateway port from the
guest — no proxy on the gateway host is reachable there at any bind address,
so an operator following the generic recipe lost all proxy-required egress
while configuration validation succeeded.
Scope the recipe to libkrun in every reference and reject a gateway-host proxy
URL when a launch plan resolves to QEMU, naming the reason, instead of booting
a sandbox whose policy-approved CONNECTs all time out.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): match the QEMU proxy preflight to the selected TAP host
The gateway-host proxy guard added for the QEMU/TAP backend classified the
wrong set of addresses in both directions.
It ran at the top of configure_qemu_launch_plan, before the subnet allocation
that settles plan.host_ip, so it could not compare against the address the
guest actually reaches the host on. An operator pointing https_proxy at the
sandbox's own TAP host address, such as 10.0.128.1, passed the check, and the
driver's nftables input chain — which accepts only the gateway port from the
guest — then dropped every policy-approved CONNECT, which is exactly the
silent timeout the guard exists to prevent.
In the other direction it rejected 192.168.127.254 unconditionally. That
address is special only to libkrun/gvproxy; on QEMU/TAP it is an ordinary
address that may be routable through the guest's masqueraded egress, so the
guard refused a working configuration.
Run the check after the launch plan's network allocation, on both the
freshly-allocated and already-complete paths, and compare IP literals with
that sandbox's selected TAP host. Loopback literals, localhost, and the
documented host aliases that write_host_gateway_aliases seeds to the TAP host
still classify as the gateway host, and the failure names the address. The
gvproxy host-loopback constant returns to being a documentation anchor.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
---------
Signed-off-by: Philippe Martin <phmartin@redhat.com>
This changes the DNS resolver test fixture to retry until TCP and UDP can share
an ephemeral port.
It fixes an issue seen in CI where a port that was successfully assigned for
one protocol failed for the other.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Requests denied by relay_with_route_selection because no L7 endpoint path
matched returned a 403 without emitting an OCSF event or recording policy
activity, leaving the deny invisible to operators and telemetry. Every
other early-return deny in that loop already logs.
Emit an HTTP Activity deny event and an l7_policy activity record before
writing the denial. Split event construction out of emit_l7_request_log
so the event shape is directly testable, and narrow both helpers to take
the request action rather than the full L7RequestInfo, which they only
used for that field.
Signed-off-by: William Trépanier <wtrep@icloud.com>
* fix(sandbox): terminate sandbox when proxy accept loop exits unexpectedly
Add a oneshot notification channel to ProxyHandle that fires when the
proxy task exits for any reason (panic, abort, or loop break). The
sandbox main loop now races this signal alongside the entrypoint process
and shutdown signals, terminating with a clear OCSF error if the proxy
dies. This prevents the sandbox from continuing to report Ready with a
dead proxy.
Follows the existing sidecar control channel pattern. All five wait
paths (process+sidecar, process-only, network+sidecar, network-only
on Linux, and non-Linux) now monitor proxy liveness.
Refs #2337
Signed-off-by: politerealism <burdcat17@gmail.com>
* test(proxy): add unit tests for exit detection drop-guard and take_exit_receiver
Verify the oneshot drop-guard pattern fires on normal task exit and
abort, and document the take_exit_receiver contract including the
double-call hazard.
Signed-off-by: politerealism <burdcat17@gmail.com>
* fix(proxy): classify terminal vs transient accept errors and exit on unrecoverable failures
Extract accept-loop error handling into classify_accept_error() with
AcceptAction enum so the decision logic is independently testable.
Terminal errors (EBADF/EINVAL/ENOTSOCK) now exit the loop immediately,
allowing the supervisor to detect proxy death. Unknown errors exit
after 10 consecutive failures. FD exhaustion retries with exponential
backoff capped at 5s (fix unreachable cap from min(6) to min(7)).
Adds 9 unit tests covering classification, counter interactions,
backoff progression, and bounded exit behavior.
Signed-off-by: politerealism <burdcat17@gmail.com>
* refactor(proxy): use libc constants and Option-based exit receiver
Replace raw errno numbers with libc:: constants behind #[cfg(unix)]
gates in classify_accept_error and its tests. Refactor
ProxyHandle::exited_rx to Option<Receiver<()>> so take_exit_receiver
uses .take() instead of allocating a dummy sender. Add a
ProxyHandle-level test that verifies the full drop → abort → receiver
fires path.
Signed-off-by: politerealism <burdcat17@gmail.com>
* fix(proxy): restore full transient errno classification from main
The refactored classifier only recognized EMFILE/ENFILE as retryable,
routing all other documented transient accept errors (ENOBUFS, ENOMEM,
ECONNABORTED, ECONNRESET, EINTR, ENETDOWN, etc.) through the unknown-
error budget which terminates the proxy after 10 consecutive failures.
Restore the complete transient errno table from main so these errors
retry indefinitely with appropriate backoff.
Signed-off-by: politerealism <burdcat17@gmail.com>
* fix(sandbox): resolve Clippy warnings in proxy-exit detection code
Replace redundant closure |p| p.take_exit_receiver() with the method
reference ProxyHandle::take_exit_receiver, and change all five
select! arms from _ = &mut proxy_exited to () = &mut proxy_exited
to satisfy clippy::ignored_unit_patterns. Also run cargo fmt on the
new proxy test in proxy.rs.
Signed-off-by: politerealism <burdcat17@gmail.com>
* ci: trigger required check re-run
Signed-off-by: politerealism <burdcat17@gmail.com>
* fix(sandbox): remove unnecessary std::pin qualification
Signed-off-by: politerealism <burdcat17@gmail.com>
---------
Signed-off-by: politerealism <burdcat17@gmail.com>
* feat(sandbox,podman): trust corporate CA for https:// proxies and intercepted TLS
The corporate proxy chaining only accepted plain http:// proxy URLs, so
operators whose forward proxy terminates TLS with a private corporate CA
had no way to reach it, and TLS-intercepting proxies (mitmproxy, squid
ssl-bump) that re-sign tunneled server certificates broke every upstream
handshake after CONNECT.
The supervisor now accepts https:// proxy URLs: it wraps the connection to
the proxy in TLS before the CONNECT handshake, verifying the proxy
certificate against the built-in Mozilla roots, the system CA bundle, and an
optional operator corporate CA bundle. The upstream dial returns a
Plain/Tls stream enum consumed generically by the relay paths.
The corporate CA is delivered as a driver-supplied command-line argument
(--upstream-proxy-ca-bundle), never an environment variable, matching the
hardened proxy-config model where a sandbox image cannot influence the
operator's egress boundary. It is folded into the sandbox combined trust
bundle (write_ca_files) and the L7 upstream verification store
(build_upstream_client_config) at startup, so intercepted upstream
handshakes succeed and sandbox workloads trust the re-signed certificates.
Configuration is fail-closed: a CA bundle set without a proxy, or an
unreadable or certificate-free file, is fatal rather than silently
weakening the trust boundary.
The shared parse_upstream_proxy_url validator accepts https:// (recording
the scheme so the driver and supervisor agree), keeping the explicit-port
requirement. The Podman driver gains a proxy_ca_bundle operator setting
(TOML, --sandbox-proxy-ca-bundle, OPENSHELL_SANDBOX_PROXY_CA_BUNDLE) that
bind-mounts the host PEM read-only into the sandbox (a CA certificate is not
secret) and points --upstream-proxy-ca-bundle at it, with a create-time
readability check.
The standalone dev gateway task passes OPENSHELL_SANDBOX_PROXY_CA_BUNDLE
through to the generated podman config, so a local gateway can be pointed at
a TLS-intercepting proxy without hand-editing the regenerated TOML.
Refs #1792
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): reject CA bundles with valid PEM framing but invalid X.509 DER
The proxy CA bundle validation counted PEM blocks that base64-decoded
successfully, but did not verify the decoded bytes were accepted as
trust anchors by RootCertStore. A bundle with syntactically valid PEM
framing but invalid DER would pass the startup check while contributing
zero usable anchors, causing opaque TLS failures at runtime instead of
a fail-closed startup error.
Validate decoded certificates through RootCertStore::add_parsable_certificates
and reject the bundle unless at least one is accepted.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): allow proxy auth without insecure acknowledgement for https:// proxies
For an https:// proxy the Proxy-Authorization credential travels inside
the verified TLS session, so the proxy_auth_allow_insecure
acknowledgement is unnecessary. Previously both http:// and https://
proxies required it, producing a misleading cleartext-risk diagnostic
for a path that is already encrypted.
Skip the requirement when the proxy URL uses https://; the
acknowledgement is still tolerated if set. Updated in both the
supervisor and Podman driver validation paths, with docs and tests.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix: format
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(kubernetes): use truly unsupported scheme in proxy validation test
https:// is now a supported proxy scheme after a13c4dce, so the
unsupported-scheme test must use a genuinely unsupported scheme.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(e2e): sign the https proxy fixture listener cert with a CA
The corporate-proxy E2E fixture served a single `openssl req -x509`
certificate as its TLS listener identity. OpenSSL marks that certificate
`basicConstraints: critical, CA:TRUE`, and rustls refuses a CA
certificate presented as an end-entity certificate (CaUsedAsEndEntity).
The supervisor's TLS handshake with the proxy therefore failed, the
upstream dial errored, and the workload's CONNECT was dropped without a
response, so podman_corporate_proxy_trusts_ca_bundle_for_https_proxy
failed on the approved destination while policy denial still worked.
Generate a corporate CA and a separate listener leaf signed by it, serve
the leaf chain, and publish only the CA as the bundle the supervisor
trusts. This is what an intercepting proxy actually presents, and it
exercises the corporate-CA trust path rather than pinning the listener
certificate itself.
Refs #1792
Signed-off-by: Philippe Martin <phmartin@redhat.com>
---------
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* feat(provider): add ability to request token exchange instead of client credentials as OAuth grant_type
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(proxy): add further tests for token exchange
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(provider): add runnable example for token exchange
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(e2e): cover Podman token exchange grants
Signed-off-by: Gordon Sim <gsim@redhat.com>
* refactor(oauth): extract duplicated functionality from server and supervisor
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(provider): evict nearest-to-expiry entry from intermediate token cache
Signed-off-by: Gordon Sim <gsim@redhat.com>
* doc(supervisor): add podman example for token exchange
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(provider): withhold token-exchange subject credentials
Signed-off-by: Gordon Sim <gsim@redhat.com>
---------
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(policy): bind reviews to applicable candidates
Build and validate the exact effective-policy candidate before approval, bind review to live policy/provider/credential inputs, and preserve inspected endpoint contracts during mechanistic expansion.
Closes#2821
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
* fix(policy): canonicalize advisor review inputs
Serialize nested protobuf maps in stable key order for proposal review tokens and effective-policy hashes. Narrow reused multi-port endpoint contracts to the denied port so advisor proposals cannot widen binary access. Add regressions for both cases.
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
* test(e2e): keep advisor sandbox running
Create the issue 2821 regression sandbox detached with a durable canonical main process so policy denial, approval, and hot-reload checks run before lifecycle exit.
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
* fix(policy): apply reviewed draft batches atomically
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
---------
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Apply the official OCSF ai_operation profile (introduced in v1.8.0) to
ApiActivity [6003] events when the inference proxy routes a model call
through inference.local. Attaches an ai_model object (name, ai_provider)
and puts token counts and latency in unmapped fields.
ApiActivity [6003] is the schema-correct class for the ai_operation
profile in v1.8.0 (HttpActivity only gets it in v1.9.0). In Splunk CIM,
ApiActivity maps to the "Change" data model, naturally separating
inference events from regular HTTP proxy traffic.
Changes:
- Add AiModel object and ai_model field on BaseEventData
- Add ApiActivityEvent struct and ApiActivityBuilder
- Add emit_ai_inference in proxy.rs using ApiActivity with ai_operation
- Vendor OCSF v1.8.0 schemas including api_activity class, ai_model
object, and ai_operation profile definitions
- Bump OCSF_VERSION to 1.8.0
- Update schema validation to skip profile-gated required fields
Shorthand: API:INFERENCE [INFO] claude-3-haiku via anthropic 701ms [POST /v1/messages]
Splunk/SIEM backward compatibility (v1.1/v1.3 CIM mapping) is tracked
separately in #2662 as a configurable serialization concern.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(security): authenticate extension services
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* feat(extension-core): verify gateway JWTs
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(extension-core): keep inbound verification external
This should become an extension SDK package.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* fix(security): harden the extension authentication contract
Follow-up hardening on the alpha extension authentication mechanism.
Claim contract:
- Extension tokens carry an explicit `typ` of `openshell-ext+jwt`. They
share a signing key with sandbox-to-gateway admission tokens and were
otherwise separated by audience alone, so a verifier that neglects to
check `aud` could accept a gateway credential. The header is a second,
independent discriminator.
- Publish OIDC-shaped discovery at `/.well-known/openid-configuration`
so a service configured with only the gateway URL can learn the exact
expected issuer and the JWKS location. It is shaped, not compliant:
`issuer` is the gateway identity, not the serving URL.
Audience agreement:
- `MiddlewareManifest` and `InterceptorManifest` gain `expected_audience`.
The audience is otherwise configured independently on each side of the
boundary, where a mismatch surfaces only as an opaque authentication
failure on every call. OpenShell now compares the two and fails at
startup. An empty field keeps the check off for existing services.
Compatibility:
- Add `allow_insecure_transport` per registration. Enabling gateway JWT
signing previously made any plaintext endpoint a hard startup failure,
including the endpoint form used in our own documentation. The opt-out
attaches no credential, is refused by the gateway if a supervisor asks
for one, and warns at every startup.
- Make the transport requirement kind-aware. A middleware endpoint must
be reachable from every sandbox supervisor, so only interceptors may
use a gateway-local Unix socket.
Credential lifecycle:
- Replace the process-global slot map with a supervisor-owned
`ExtensionCredentialStore` shared explicitly across the gateway
connections the supervisor opens, removing test-order coupling.
- Rotate only when a credential is missing or has passed four fifths of
its lifetime. Configuration polling ran every ten seconds against
fifteen-minute credentials, so each poll re-ran gateway effective-policy
resolution and re-minted the gateway token.
- Bound credential minting per sandbox, since each request resolves the
caller's effective policy.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs: record alpha extension authentication in RFC appendices
Restore the RFC 0009 and 0010 bodies to their accepted text and move
every extension-authentication update into appendices instead. An RFC
records a decision at a point in time; superseding detail belongs
alongside it rather than rewritten into it.
RFC 0009's appendix carries the shared contract: claims, authorization,
key distribution, the `allow_insecure_transport` replacement for the
body's `allow_insecure`, and residual risks. RFC 0010's records only
what differs for interceptors and links to it. The existing
protocol-extensions appendix, which parked the phase 2 transport
question, now points forward to what was built.
Also document the audience handshake, the discovery endpoint, the
`typ` requirement, and `jti` replay guidance in the extensibility and
gateway configuration pages, and correct the middleware transport
guidance: middleware endpoints must be reachable from sandbox
supervisors, so Unix sockets are not an option there.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* refactor(extension-core): abstract extension server trust
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* docs(core): update middleware manifest example
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* fix(extension-auth): preserve unsigned gateway compatibility
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* fix(extension-auth): reject cross-domain token replay
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
---------
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Prekshi Vyas