* feat(sandbox,gateway): route sandbox egress through corporate HTTP proxy
- Chain sandbox egress through a corporate HTTP proxy so outbound
traffic from within the sandbox respects the host proxy settings
- Forward sandbox proxy environment variables to the generated Podman
config so the proxy is applied consistently to Podman-managed workloads
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): make corporate proxy routing operator-owned
The operator-configured corporate egress proxy was injected under the
conventional HTTPS_PROXY/HTTP_PROXY/NO_PROXY names as defaults beneath
sandbox spec/template environment, so a sandbox creator could redirect
egress at an arbitrary proxy or disable proxying with NO_PROXY=*.
Route the boundary through reserved, supervisor-only variables
(OPENSHELL_UPSTREAM_HTTPS_PROXY/HTTP_PROXY/NO_PROXY) written in the
Podman driver's required-variable tier. Any sandbox-supplied value under
a reserved name is stripped before the operator value is applied, so the
supervisor never observes a reserved proxy variable the operator did not
set. The supervisor now reads only the reserved names and ignores the
conventional proxy variables the sandbox controls.
Add the reserved proxy variables to the supervisor-only child-environment
denylist so the corporate proxy URL and any embedded credentials are not
inherited by the sandbox workload, which reaches egress through the local
policy proxy and never needs them.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* feat(sandbox,podman): deliver corporate proxy credentials via secret file
Proxy credentials were embedded inline in the proxy URL, so they were
stored in gateway.toml and exposed in container metadata via
'podman inspect'.
Reject inline 'user:pass@' credentials in https_proxy/http_proxy at
startup (parsed with the url crate rather than hand-splitting), and add a
proxy_auth_file option pointing at a 'user:pass' file. The driver stages
that file as a per-sandbox root-only Podman secret, mounts it at a fixed
path, and exports only the path in the reserved
OPENSHELL_UPSTREAM_PROXY_AUTH_FILE variable, so the credential never
appears in config, environment, or container metadata. Reading the file
fails closed on a missing, empty, or control-character-bearing value.
The supervisor reads the credential from the mounted file and builds the
Proxy-Authorization: Basic header, rejecting control characters, and no
longer derives credentials from URL userinfo. The auth-file path is added
to the child-environment strip list, and the generated gateway.toml is
written owner-only (mode 0600).
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): fail closed on invalid upstream proxy configuration
The reserved OPENSHELL_UPSTREAM_* variables are an operator-owned egress
boundary, but the supervisor treated present-but-invalid values as unset:
an unsupported or malformed proxy URL was ignored with a warning, an
unreadable auth file proceeded without credentials, and a malformed
credential silently became unauthenticated. Any of these could quietly
downgrade the corporate proxy boundary to direct dialing or
unauthenticated proxy access.
Make every configured-but-invalid proxy or auth setting fatal to
supervisor proxy startup, emit an OCSF ConfigStateChange failure event
before refusing, and share URL validation semantics between the Podman
driver and the supervisor through a single validator in
openshell-core (parse_upstream_proxy_url), so a value accepted at
sandbox-create time can never be rejected in-container or vice versa.
Inline user:pass@ URL credentials are now fatal in the supervisor too
(previously warn-and-strip), matching the driver. Unset or empty
variables still mean no proxy; only present-but-invalid values fail.
Error paths never include credential content.
Addresses the fail-closed review item on #2245.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): finish the fail-closed upstream proxy credential contract
The upstream proxy URL already had a single shared validator, but the
credential did not: the Podman driver rejected only CR/LF/NUL while the
supervisor rejected every control character, so a credential accepted at
sandbox-create time (e.g. one containing a tab) could still be rejected
in-container. Present-but-whitespace reserved OPENSHELL_UPSTREAM_*
values were also silently treated as unset, quietly downgrading the
operator's egress boundary to direct dialing.
Add parse_upstream_proxy_credential to openshell-core as the single
source of truth for the documented user:pass credential form (non-empty
user, no control characters, trimmed) and use it in both the Podman
driver's secret staging and the supervisor's Proxy-Authorization header
construction. Error variants carry no payload so credential content can
never leak into messages.
Make a present-but-empty reserved variable fatal to supervisor proxy
startup instead of meaning "unset"; only fully unset variables disable
the proxy. The driver correspondingly rejects an empty no_proxy at
config time so it can never inject a value the supervisor refuses.
Addresses the remaining fail-closed credential/config review item on
Signed-off-by: Philippe Martin <phmartin@redhat.com>
#2245.
* fix(sandbox,podman): close remaining fail-open upstream proxy config paths
Two configuration paths could still silently run without the proxy
boundary the operator believed was in effect.
A no_proxy bypass list configured without any https_proxy/http_proxy
was accepted by both the driver and the supervisor and simply meant
"dial everything directly". Reject it on both sides, exactly like the
existing proxy_auth_file-without-proxy rule: an operator who wrote a
bypass list assumed proxying was active, so accepting it hides a
fail-open state.
The gateway.sh dev script guarded proxy settings with [[ -n "${VAR:-}"
]], which conflates unset with explicitly-empty and dropped the latter
before the gateway's validation could see it. Use ${VAR+x} instead so
a set-but-empty variable is written into gateway.toml and rejected at
startup by validate_proxy_config rather than silently discarded.
Addresses the remaining fail-open configuration review item on #2245.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): reject upstream proxy URLs with path, query, or fragment
parse_upstream_proxy_url accepted URLs like http://proxy.corp.com:8080/some/path
and silently discarded everything after host:port, a lenience inherited
from the original supervisor parser. A forward proxy is addressed by
host:port only, so extra components indicate a misconfiguration (for
example a pasted endpoint URL) and silently truncating them violates
the present-but-invalid-is-fatal contract enforced everywhere else in
this configuration surface.
Reject a path, query, or fragment in the shared validator with a new
UnexpectedComponent error. A bare trailing slash remains accepted
because the url crate normalizes an absent http path to "/", making the
two indistinguishable. Both the Podman driver (gateway startup) and the
supervisor (sandbox startup) inherit the rule through the shared
parser, keeping their semantics identical by construction.
Addresses the proxy URL component review item on #2245.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): remove plain-HTTP upstream proxy support
The http_proxy path tunneled plain-HTTP requests through the corporate
proxy with CONNECT to port 80 and then sent origin-form requests down
the tunnel. Conventional enterprise forward proxies expect plain HTTP
as absolute-form requests sent directly over the proxy connection, and
commonly refuse CONNECT to port 80, so the setting looked supported but
failed against typical deployments. Tunneling also blinds the proxy to
the one protocol it could inspect.
Narrow the feature to TLS (CONNECT) egress only, which is the
conventional and already-correct case: plain-HTTP requests now always
dial the destination directly, and only client CONNECT tunnels chain
through the corporate proxy. Remove the http_proxy config field, the
--sandbox-http-proxy / OPENSHELL_SANDBOX_HTTP_PROXY driver surface, the
reserved OPENSHELL_UPSTREAM_HTTP_PROXY variable, and the UpstreamScheme
plumbing. The feature never shipped, so this is a clean removal; a
stray http_proxy key in gateway.toml still fails loudly through the
config's deny_unknown_fields.
Removing the plain-HTTP proxy branch also removes its host-gateway
special case; the architecture doc now documents the real host-gateway
behavior (add driver-injected host aliases to the reserved NO_PROXY
list) instead of an invariant the HTTPS path never implemented.
Plain-HTTP forwarding through a corporate proxy can return later as
absolute-form forwarding behind its own design review.
Addresses the plain-HTTP forwarding review item on #2245.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): escape generated TOML and require explicit proxy URL form
Escape backslashes, quotes, and control characters when gateway.sh writes
proxy values into gateway.toml, so a hostile or unusual environment value
cannot corrupt the config or inject extra keys.
Restrict the upstream proxy URL grammar to the documented http://host:port
form: a scheme-less value is no longer normalized to http:// and a missing
port is no longer silently defaulted to 80. Docs, README, and CLI help now
state the explicit-form requirement consistently.
Also fix a test-only call of handle_tcp_connection that was missing the
upstream_proxy argument added in an earlier commit.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): preserve tunneled bytes read with the CONNECT response
The CONNECT handshake reads the corporate proxy's response in chunks, so
the read that completes the header block can also contain the first
tunneled payload bytes. Those bytes were discarded, silently corrupting
the start of the tunnel for server-speaks-first destinations or proxies
that coalesce writes.
connect_via now returns a PrefixedStream that replays any bytes received
past the response terminator before reading from the socket again; writes
pass through unchanged. Direct dials wrap the stream with an empty prefix
so downstream relay and TLS paths keep a single stream type, and
tls_connect_upstream is generalized to any AsyncRead + AsyncWrite stream.
Adds regression coverage for a combined response/payload read and for
prefix replay ordering.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): gate cleartext proxy Basic auth behind an explicit opt-in
Proxy-Authorization: Basic is base64 over the plain-TCP connection to the
http:// corporate proxy, so anyone on the network path between the sandbox
host and the proxy can recover the credential. Sending it is now an
explicit operator decision instead of an implicit side effect of
configuring proxy_auth_file.
Add a proxy_auth_allow_insecure driver setting (CLI
--sandbox-proxy-auth-allow-insecure, env
OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE), delivered to the supervisor
as the reserved OPENSHELL_UPSTREAM_PROXY_AUTH_ALLOW_INSECURE variable.
Fail-closed pairing on both sides: an auth file without the
acknowledgement is rejected at gateway startup and at supervisor startup,
as is the acknowledgement without an auth file or any value other than
'true'. gateway.sh writes the key only as a TOML boolean; a non-boolean
value is emitted as a quoted string so the gateway rejects it at startup
instead of risking injection.
Documents the exposure prominently in the gateway config reference, the
driver README, and the sandbox architecture doc.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): honor port qualifiers and resolved addresses in NO_PROXY
Two divergences from the documented NO_PROXY contract:
- A port-qualified entry (internal.corp:8443) was silently stripped to its
hostname and bypassed the proxy for every port, excluding traffic the
operator never listed. Entries now keep the optional :port qualifier
(also on IP and CIDR entries) and only apply to that destination port; a
trailing qualifier that is not a valid port stays part of the pattern
instead of widening the entry.
- IP and CIDR entries only matched IP-literal hosts, so a bypass like
10.0.0.0/8 never applied to hostnames resolving into that range. NO_PROXY
evaluation now sees the validated resolved addresses: an IP/CIDR entry
matching through resolution authorizes a direct dial of only the
addresses it contains, so a bypass scoped to an internal range cannot
widen into a direct dial of addresses outside it. Hostname-level matches
(loopback, wildcard, domain entries, IP-literal hosts) keep authorizing
all validated addresses.
proxy_for is replaced by decision(host, port, resolved) returning either
the proxy endpoint or the permitted direct-dial subset, and dial_upstream
restricts the direct connect to that subset.
Adds regression coverage for port-scoped bypasses on domain, IP, and CIDR
entries, invalid port qualifiers, resolved-address matching, and
split-resolution subset dialing.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): bind proxied CONNECT tunnels to validated addresses
The CONNECT request sent to the corporate proxy carried the destination
hostname, so the proxy resolved the name itself and the addresses that had
passed SSRF and allowed_ips validation were discarded. Split-horizon DNS
or rebinding at the proxy could then reach internal or otherwise
unapproved destinations through a tunnel the supervisor logged as
validated, and IP-range policy could not be enforced at all on proxied
dials.
CONNECT now targets a validated resolved address by default: the proxy
performs no DNS resolution and the tunnel stays bound to the answer the
supervisor checked. The hostname still travels inside the tunnel (TLS SNI,
application Host), so destination servers behave normally. In
split-horizon networks, operators point the gateway host at the corporate
resolver so internal names validate to their internal addresses.
For proxies whose ACLs filter on hostnames and reject IP CONNECT targets,
a new proxy_connect_by_hostname opt-in (CLI
--sandbox-proxy-connect-by-hostname, env
OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME, reserved
OPENSHELL_UPSTREAM_PROXY_CONNECT_BY_HOSTNAME) restores hostname CONNECT,
documented as re-opening proxy-side resolution and making the proxy's ACLs
the effective egress control. Fail-closed pairing on both sides: the
opt-in without a proxy, or any value other than 'true', is fatal.
Adds regression coverage for the IP CONNECT request line (including IPv6
bracketing and hostname non-leakage), the hostname opt-in, and the
config pairing rules in driver and supervisor.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): reject empty port after bracketed IPv6 proxy host
http://[fd00::1]: passed the explicit-port check because the bracketed
branch only tested for a colon after the bracket, then fell back to port
80 — violating the fail-closed http://host:port contract and potentially
sending configured Basic credentials to an unintended service. Require a
non-empty suffix after ]:, matching the unbracketed branch, and cover the
case in the shared parser tests.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): fall back across validated addresses in proxied CONNECT
The direct path hands TcpStream::connect the whole validated address list
and it falls back across them, but the validated-IP CONNECT path attempted
only the first address, so a dual-stack destination could fail through the
corporate proxy even when a later validated address was reachable.
connect_via_validated tries each validated address in order under one
aggregate CONNECT_HANDSHAKE_TIMEOUT budget, returning the first success;
when every attempt fails the error names the attempt count and carries the
last failure. An empty address list is rejected up front.
Adds regressions for first-fails/second-succeeds fallback (asserting both
CONNECT request lines), the aggregate all-addresses failure message, and
the empty-list rejection.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): strip new reserved proxy vars and complete docs/tests
Follow-ups from review:
- Add OPENSHELL_UPSTREAM_PROXY_AUTH_ALLOW_INSECURE and
OPENSHELL_UPSTREAM_PROXY_CONNECT_BY_HOSTNAME to the supervisor-only
strip list so workload child processes never inherit them, matching the
documented contract for the other reserved proxy variables, and cover
both in the supervisor-only variable test.
- Document all five OPENSHELL_SANDBOX_* proxy variables in the
mise run gateway help text, marked Podman-only and stating the
auth-file/acknowledgement pairing, and complete the gateway-key list in
the Podman README.
- Add NO_PROXY composition coverage for bracketed IPv6 entries with port
qualifiers, bare IPv6 entries, and IPv6 CIDR matching against an
IPv6-literal host and against a hostname's resolved addresses,
including the port-qualified CIDR form.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): cap each proxied CONNECT attempt within the shared budget
A proxy that accepted the first CONNECT request but never responded
consumed the entire aggregate handshake timeout, so later validated
addresses were never tried and the hang defeated the multi-address
fallback.
Each attempt is now time-boxed to its fair share of the time remaining
before the shared deadline (remaining / attempts_left): a hanging attempt
is cut off with enough budget left for every remaining address, while
time a fast failure does not use rolls over to later attempts and the
total never exceeds CONNECT_HANDSHAKE_TIMEOUT. A timed-out attempt is
recorded like any other failure, and the aggregate error distinguishes
all-attempted from budget-exhausted runs.
Adds a first-hangs/second-succeeds regression driven through a
test-visible budget parameter so it runs in about a second instead of a
real 30s window.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): deliver corporate proxy config on the supervisor argv
The proxy settings were injected as reserved OPENSHELL_UPSTREAM_* container
environment variables. The driver only wrote names the operator configured,
but container runtimes layer the spec environment over ENV values baked
into the sandbox image, so an image could supply NO_PROXY=*, enable
hostname CONNECT, or point an unconfigured deployment at an
attacker-controlled proxy whenever the operator left a field unset.
The settings now travel as supervisor command-line arguments
(--upstream-proxy, --upstream-no-proxy, --upstream-proxy-auth-file,
--upstream-proxy-auth-allow-insecure,
--upstream-proxy-connect-by-hostname) built by the driver from operator
config. The driver sets the container entrypoint and command explicitly,
so neither sandbox spec/template environment nor image ENV can influence
argv, and an omitted flag genuinely means unconfigured — in every
supervisor topology, since the supervisor no longer consults its
environment for these settings at all. Credentials stay on the root-only
secret mount; only the mount path appears on argv.
The reserved environment names, their strip-list entries, and the
env-based validation surface are removed. UpstreamProxyConfig::from_args
replaces from_env, reusing the same shared fail-closed validation and
pairing rules keyed by the CLI flag names.
Driver tests now assert the argv contract, including that sandbox-supplied
environment cannot add, remove, or redirect proxy flags; supervisor tests
cover from_args mapping and its pairing rules.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* docs(sandbox): align proxy comments with the argv transport
The argv migration left comments describing the configuration as reserved
environment variables ("reserved value", "present-but-empty variable",
"reserved upstream proxy variables"). Rephrase them as driver-supplied
arguments and operator settings so the documented trust boundary matches
the implementation. Comment-only change.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): parse bracketed IPv6 authorities in client CONNECT targets
parse_target split the CONNECT authority at the first colon, so an
IPv6-literal target like [2001:db8::1]:443 always failed port parsing and
IPv6-literal clients could never reach policy evaluation; a regression
test even locked in that failure. Parse the RFC 3986 bracketed form and
return the host bracket-free, matching what DNS resolution, SSRF
validation, NO_PROXY matching, and the upstream CONNECT builder expect.
Unclosed brackets, a missing or empty port after the bracket, and
non-numeric ports are rejected; unbracketed behavior is unchanged.
Replaces the failure-locking test with success coverage for bracketed
targets and adds malformed-bracket rejection cases.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* test(podman): cover proxy-auth secret cleanup across lifecycle failures
The per-sandbox proxy-auth credential secret is staged before the
container is created and removed on cleanup, but no test proved the
cleanup paths actually issue the secret removal. Add Podman-stub tests
that drive create_sandbox to a container-create failure and to a
start failure, and delete_sandbox for an out-of-band deletion, asserting
each path issues the DELETE for the per-sandbox proxy-auth secret so a
credential can never outlive the sandbox that owned it.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* docs: list corporate proxy keys in the Podman compute-driver overview
The Fern Podman driver section enumerated its gateway.toml keys but
omitted the corporate egress proxy settings. Add https_proxy, no_proxy,
proxy_auth_file, proxy_auth_allow_insecure, and proxy_connect_by_hostname
with a pointer to the gateway configuration reference for the full
contract. No navigation change: the reference folder already includes the
gateway configuration page.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* test(sandbox): cover the SSRF-to-TLS composition across the proxy tunnel
Existing tests exercised validated-IP CONNECT and the upstream-TLS helper
independently, but not the full boundary. Add an end-to-end regression
that stands up a fake corporate proxy tunneling to a fake TLS server and
drives the real path: connect_via_validated CONNECTs to the validated
address, the proxy splices the tunnel, and tls_connect_upstream verifies
the upstream certificate against the original hostname carried in SNI.
It asserts the CONNECT authority is the validated IP and never the
hostname, that verification succeeds for the matching hostname, and that a
mismatched hostname is rejected — proving a rebinding or split-horizon
substitution behind the proxy cannot pass certificate verification.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox,podman): bound proxy-auth reads, reject port 0, fix stale comment
Three review findings:
- CWE-400: the proxy-auth credential file was read with an unbounded
read_to_string on both the driver (sandbox-create) and supervisor
(startup) paths, so a huge file or a special file such as /dev/zero
could exhaust memory. Add a shared bounded reader in openshell-core that
rejects non-regular files, caps the size at 4 KiB, and reads at most that
many bytes; the driver runs it via spawn_blocking. Covers oversized,
special-file, and missing-path cases on both sides.
- Reject an upstream proxy URL with port 0: it passed the explicit-port
check and startup validation but is not a connectable TCP port, so every
proxied dial would fail later. Add a typed ZeroPort error with
shared-validator and Podman-config tests.
- Reword a driver-config comment that still described a 'reserved
variable' to match the argv transport.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): open proxy-auth file non-blocking to reject FIFOs promptly
read_upstream_proxy_credential_file opened the path with a blocking
File::open before the regular-file check, so a configured FIFO with no
writer would block open() indefinitely — hanging sandbox creation on the
driver and supervisor startup. Open with O_NONBLOCK on Unix so the open
returns immediately, then reject the non-regular file as before;
O_NONBLOCK has no effect on the later read of a regular file. Adds a
mkfifo regression asserting the reader returns promptly with a
non-regular-file error instead of hanging.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* test(podman): cover corporate proxy egress across driver and supervisor
The existing corporate-proxy tests construct config structs or call CONNECT
helpers directly, so none of them detect a break in the wiring between
layers: gateway TOML deserialization, the Podman argv and secret-mount
semantics, supervisor CLI parsing, or policy denial before proxy contact.
Add a Podman e2e that drives the whole chain against a fake authenticated
forward proxy and asserts that an approved TLS request traverses it with a
validated-IP CONNECT, a policy-denied destination is refused with 403
without ever reaching the proxy, credentials arrive through the mounted
per-sandbox secret, and deleting the sandbox removes that secret.
SupportContainer is a new harness fixture. Unlike ContainerHttpServer it
probes readiness with a TCP connect rather than an HTTP GET, so it can host
a forward proxy and TLS servers, and it exposes container logs and network
IP for assertions.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* test(podman): restart the gateway on the proxy-config panic path
The panic cleanup for the temporary corporate proxy configuration
restored the gateway TOML but left the gateway process running with the
temporary configuration still loaded, which could poison later test
binaries in the same run. Nothing restarted it: the only ManagedGateway
is the short-lived one inside restart_gateway, and its Drop only calls
start, which does not reload config for an already-running gateway.
Restore and synchronously stop/start the gateway in Drop, and set
restored only after the normal restore and restart both succeed so a
failed restart no longer suppresses the fallback.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(supervisor-network): derive the loopback proxy bypass from resolved IPs
The automatic bypass treated the host string "localhost" as proof that
the destination was loopback and returned every resolved address. A
sandbox controls its own /etc/hosts and resolve_socket_addrs consults it
before DNS, so a workload could map localhost to any policy-allowed
address and dial it directly, escaping the operator proxy and the
inspection and audit boundary it exists to provide.
Check the resolved addresses instead: the name bypasses only when the
resolution is non-empty and every address is loopback. A mixed answer is
not partially honored, and an IP literal is still authoritative for
itself. A spoofed localhost falls through to the entries below, so an
explicit operator NO_PROXY entry is still honored.
This matches the trust model detect_trusted_host_gateway already applies
to the same hosts file, which validates the mapped address rather than
the alias.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(podman): clean up proxy-auth secret when container already deleted
The delete_sandbox early-return path cleaned up the token secret but
skipped the proxy-auth secret, leaking it on disk. Also update tests
for recent API changes (Optional socket_path, workspace field,
list-based container lookup).
Signed-off-by: Philippe Martin <philippe@openshell.dev>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
---------
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <philippe@openshell.dev>
Closes#2112
The host `cargo zigbuild` for `*-unknown-linux-musl` opens ~333 `.rlib`
files at once during the static link, exceeding macOS's default soft
limit of 256 and failing with `ProcessFdQuotaExceeded`. This blocked the
docker/podman `mise run gateway` paths for macOS contributors; only the
VM driver path guarded against it.
Extract the VM path's `ensure_build_nofile_limit` guard into a shared
`tasks/scripts/build-env.sh` and call it from the host-staging chokepoint
(`stage-prebuilt-binaries.sh`), fixing docker, podman, and all
docker:*/multiarch host cross-compiles at once. De-duplicate the VM
script to source the shared helper. The guard is a no-op on Linux and
when cargo-zigbuild is absent, so CI and Linux dev are unaffected. The
limit is read from `OPENSHELL_BUILD_NOFILE_LIMIT` (default 8192),
honoring the legacy `OPENSHELL_VM_BUILD_NOFILE_LIMIT` for back-compat.
Also correct the stale comment in gateway-docker.sh (the cross-compile
runs on the host, not inside Linux containers) and document the guard in
architecture/build.md. Adds tasks/scripts/test-build-env.sh, wired into
`mise run test` via `test:build-env`.
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
Run cargo fmt for both the root workspace and the standalone e2e Rust workspace
from the rust format and format-check tasks. Apply rustfmt to the e2e sources
so the expanded formatting check passes.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* feat(kubernetes): add sidecar supervisor topology
Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid similar process id names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid similar process id names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(sandbox): avoid similar proxy id names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* docs(kubernetes): clarify sidecar topology limits
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): keep sidecar process leaf capless
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): refresh sidecar provider env snapshots
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* test(supervisor): align hot-swap identity regression
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): stage sidecar mtls files before proxy chown
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): simplify sidecar supervisor topology
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* chore(helm): reuse sidecar skaffold values
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid similar iptables helper names
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(e2e): harden kube gateway wrapper setup
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(supervisor): avoid nft batch rollback on OCP
Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): preserve process identity in sidecar topology
Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table.
Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* refactor(kubernetes): replace sidecar snapshots with control socket
Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files.
Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* feat(kubernetes): support relaxed sidecar network identity
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(sandbox): satisfy sidecar clippy lint
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* refactor(kubernetes): standardize topology naming
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(sandbox): satisfy linux clippy timeout import
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): support kata sidecar on ipv4 pods
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): satisfy linux clippy for sidecar fallback
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* chore(kubernetes): remove stale supervisor topology references
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): enable sidecar binary policy inspection
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): harden sidecar control boundary
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): couple sidecar supervisor lifecycles
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Co-authored-by: Seth Jennings <sjenning@redhat.com>
* docs: remove deprecated --keep flag from tutorials and examples
The --keep flag is deprecated, hidden, and a no-op since sandboxes
are kept by default. Remove references from tutorial docs and example
READMEs that explain it as a real feature.
- Remove --keep from sandbox create commands
- Remove --keep explanation text
- Clarify that sandboxes are kept by default
Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>
* chore: remove deprecated --keep usage from scripts and e2e tests
The --keep flag is a deprecated no-op since sandboxes are kept by
default. Stop passing it in internal scripts, e2e test scripts,
and example demo scripts.
Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>
---------
Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>
Add policy schema, proto, provider profile, OPA, and L7 proxy support for
`protocol: json-rpc` and `protocol: mcp`. Generic JSON-RPC endpoints match
exact method names only, with `method: "*"` as the all-method sentinel;
wildcard/glob methods and params matchers are rejected.
Parse JSON-RPC request bodies and batches in the forward proxy, deny
response-shaped client frames, limit receive-stream GET allowance to MCP
endpoints, and redact params in decision logs. Preserve L7 rule params on the
proto load path so MCP `tools/call` tool filters behave like YAML-loaded
policies.
Add MCP conformance coverage, JSON-RPC L7 e2e coverage, and docs for the new
protocols and current matcher limitations.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Co-authored-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
* feat(kubernetes): support agent-sandbox v1beta1
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* ci(kubernetes): test agent-sandbox api versions
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): retry agent-sandbox raw 404s
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): harden agent-sandbox api setup
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(kubernetes): cache agent-sandbox api version
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* docs(kubernetes): document agent-sandbox upgrade behavior
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* refactor(kubernetes): collapse agent-sandbox api selection
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Remove the production-crate dev-settings feature so e2e tests build the same gateway, CLI, and supervisor binaries that we ship. Move settings coverage from dummy test-only keys to the production ocsf_json_enabled setting.
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(sbom): release lock before sleeping in _rate_limit
time.sleep was called inside the _rate_lock block, blocking all threads
from checking their own domain rate limit while one thread slept.
With _MAX_WORKERS=12 querying crates.io, npm, and pypi concurrently,
this made the thread pool effectively serial.
Move the sleep outside the lock so threads for different domains
can proceed concurrently.
* fix(sbom): reserve next slot in _rate_limit to prevent same-domain races
Use time.monotonic() and store next_req = max(now, last + interval) so
concurrent same-domain callers each get a unique future slot rather than
all sleeping the same amount. Add regression tests for same-domain
spacing and different-domain non-blocking.
* fix(sbom): wire rate-limit tests into test suite via test:sbom task
Rename test_resolve_licenses.py to resolve_licenses_test.py to match
repo *_test.py convention. Add test:sbom task to tasks/test.toml and
include it in the top-level test depends so mise run test picks it up.
* fix(sbom): add SPDX header, fix cross-domain regression test to exercise lock contention
* fix(sbom): replace zip with itertools.pairwise to satisfy ruff lint
* fix(helm): build chart dependencies before lint
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* test(e2e): remove python gpu smoke test
Remove the Python GPU smoke test and its fixture. The e2e:k3s:gpu task only depended on e2e:python:gpu and did not have a separate k3s implementation, so remove that stale alias with the task it pointed at.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
(cherry picked from commit 221a10378e188656c710560740cbc9463c002db6)
---------
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* ci(docs): add docs website automation
* ci(docs): harden sync-docs inputs and add script tests
Route free-form workflow_dispatch inputs through env vars to avoid shell
injection, switch uv install to the approved setup-uv action, and add a
pytest suite for sync_docs_website.py wired into the test task.
* feat(telemetry): add build-time option to compile out telemetry
Gate anonymous telemetry emission behind a default-on `telemetry` Cargo
feature in openshell-core. The data model (enums, validation, emit_*/enabled*
signatures) stays always-compiled, while the endpoint, HTTP client, queue, and
emission code are feature-gated. With the feature off, enabled() returns false
and emit_* are no-ops, so dependent crates compile unchanged and no telemetry
endpoint, HTTP client, or emission code is included in the binary.
chrono and reqwest become optional dependencies of openshell-core, dropped from
its dependency graph when telemetry is disabled.
Thread the switch through the workspace: every crate depends on openshell-core
with default-features = false, and the default-on `telemetry` passthrough lives
on the binary crates that emit or collect telemetry (openshell-server,
openshell-sandbox, openshell-driver-vm). In-process drivers inherit it via
resolver v2 feature unification.
Build a telemetry-free binary with, e.g.:
cargo build --release -p openshell-server --no-default-features
The runtime OPENSHELL_TELEMETRY_ENABLED switch is unchanged for default builds.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* ci(telemetry): guard that telemetry can be compiled out
Add tasks/scripts/verify-telemetry-compiled-out.sh, which inspects a built
binary for telemetry markers (the telemetry endpoint host and client ID) that
exist only when emission code is compiled in. The rust:verify:telemetry-off
mise task builds the gateway with default features (positive control: markers
must be present, so the absent checks can never be silently vacuous) and with
--no-default-features (markers must be absent), and checks the
--no-default-features sandbox binary as well.
Wire the task into the Rust branch-checks job so a regression that reintroduces
telemetry code into a --no-default-features build fails CI.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
---------
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
Drop the vendored deploy/kube/manifests/agent-sandbox.yaml. CI and e2e
scripts now apply manifest.yaml directly from
github.com/kubernetes-sigs/agent-sandbox releases, pinned via
AGENT_SANDBOX_VERSION env (default v0.4.6, overridable) so internal runs
stay reproducible.
Public docs and the helm chart README continue to reference
/releases/latest/download/ — no OpenShell/agent-sandbox support matrix
exists yet (see #1649).
Adds an air-gap note in docs/kubernetes/setup.mdx enumerating the
manifest and image operators need to mirror to an internal registry.
Signed-off-by: Roshni Malani <rmalani@nvidia.com>
* fix(config): reject unknown fields in nested gateway config tables
The gateway TOML loader silently ignored keys placed under the wrong
table header. PR #1661 fixed one instance of this (supervisor_image
under [openshell.gateway.gateway_jwt]) but the root cause remained: the
nested gateway config tables did not deny unknown fields, so a misplaced
key was accepted and dropped instead of erroring.
Concretely, tasks/scripts/gateway.sh emitted `sandbox_namespace` right
after the [openshell.gateway.gateway_jwt] heredoc, so it landed inside
the gateway_jwt table rather than [openshell.gateway]. The k8s driver
already receives the namespace via [openshell.drivers.kubernetes], so
the stray line was dead config that parsed without complaint.
Changes:
- Add #[serde(deny_unknown_fields)] to the nested gateway config tables
that are part of the config-file parse tree: TlsConfig, OidcConfig,
MtlsAuthConfig, GatewayAuthConfig, GatewayJwtConfig.
- Remove the misplaced sandbox_namespace line from gateway.sh.
- Drop the unused Serialize/Deserialize derives from Config and
ServiceRoutingConfig (see below).
- Add a regression test asserting a key under the wrong nested table is
rejected.
The gateway.sh script appended supervisor_image after the
[openshell.gateway.gateway_jwt] table header, so TOML parsed it as a
gateway_jwt field. The Podman driver never saw the override and fell
back to the default ghcr.io/nvidia/openshell/supervisor:latest.
Move supervisor_image into [openshell.drivers.podman] where the driver
config deserializer expects it.
* fix(gateway): align package TLS bootstrap path
Closes#1593
Default package-managed gateway services to a stable local TLS directory and use that same value for certificate generation and runtime startup.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* test(packaging): validate package asset paths exist
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* ci(e2e): pin mise in kubernetes job
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Rootless Podman sandbox containers reach the host through pasta's local
connection bypass, which translates L2 frames to L4 host sockets. The
dev gateway script binds to 127.0.0.1 by default, which is not routable
through pasta. Auto-detect rootless mode and bind to 0.0.0.0 so sandbox
containers can connect to the gateway.
- Auto-detect rootless Podman in gateway.sh and export
OPENSHELL_BIND_ADDRESS=0.0.0.0 when not explicitly set
- Add e2e:podman:rootless mise task and CI matrix entry to validate
rootless Podman networking end-to-end
- CI creates a non-root user inside the privileged container to trigger
Podman's rootless code paths (pasta, user namespace isolation)
Signed-off-by: Naveen Malik <nmalik@redhat.com>
* feat(helm): add optional PostgreSQL backing store with Secret-based credentials
- Add postgres.enabled and postgres.deploy values to control database
backend (SQLite vs PostgreSQL) and subchart deployment independently.
- Introduce db-secret.yaml template for Opaque Secret with assembled
postgresql:// connection string injected via OPENSHELL_DB_URL env var.
- Add Bitnami PostgreSQL as optional subchart dependency keyed on
postgres.deploy to prevent subchart deployment in external mode.
- Externalize JWT signing key file mode via sandboxJwt.secretDefaultMode
with 0400 default matching upstream.
- Add validation guard for postgres.deploy=true without postgres.enabled.
- Add helm unit tests covering internal, external, URL-override, special
character encoding, and misconfiguration error paths.
- Update README with Kubernetes and OpenShift install examples for
bundled and external PostgreSQL configurations.
- Add helm dependency build to lint and unittest tasks.
* fix(helm): add database backend docs to README.md.gotmpl and regenerate
The helm-docs CI check failed because the Database backend section was
added directly to README.md instead of README.md.gotmpl. Move the
content to the template and regenerate so the check passes.
* fix(helm): use Secret-based DB credentials and support existingSecret
Replace the inline db-url stringData pattern with a proper Secret
containing individual fields plus a uri key. When postgres.deploy=true
the Bitnami service-binding secret is referenced directly; when
deploy=false users can supply postgres.external.existingSecret to
bring their own Secret, or let the chart generate one from the external
field values.
Also restructures the README database section for clarity, adds
helm-unittest coverage for the new secret resolution paths, and
fixes a markdown lint issue in the root README.
* refactor(helm): move OpenShift e2e script to e2e/rust/ and add mise task
Move test-openshift-scenarios.sh from deploy/helm/openshell/ci/ to
e2e/rust/e2e-openshift.sh, matching the existing e2e script naming
convention. Register it as `e2e:openshift` in tasks/test.toml — not
wired into the `test` or `e2e` aggregates so it only runs on explicit
invocation against a live OpenShift cluster.
* feat(e2e): add database backend scenarios to Kubernetes e2e
Extend with-kube-gateway.sh with an optional multi-scenario loop gated
by OPENSHELL_E2E_KUBE_DB_SCENARIOS=1. When enabled, the script installs
the Helm chart three times — SQLite (default), bundled PostgreSQL, and
external PostgreSQL with existingSecret — running the full test suite
against each backend. When unset, existing single-install behavior is
unchanged.
Also adds helm dependency build before helm install, fixing CI failures
caused by the missing PostgreSQL subchart dependency.
* refactor(helm): simplify PostgreSQL config to two orthogonal controls
Replace postgres.deploy and postgres.external.* with two simple controls:
- postgres.enabled: deploy the bundled Bitnami PostgreSQL subchart
- server.externalDbSecret: name of a pre-existing Secret with a uri key
Delete db-secret.yaml — the chart no longer generates Secrets from
individual credential fields. Users either get the Bitnami service-binding
secret (bundled) or bring their own via server.externalDbSecret.
Add validation that postgres.serviceBindings.enabled must stay true
when using bundled PostgreSQL, preventing a confusing runtime failure.
* feat(gateway): add readiness probe metrics and test-only store close
Emit Prometheus readiness metrics for database probes (healthy gauge and
outcome-labeled latency histogram) with coverage in health HTTP tests.
Restrict Store::close behind test support cfg to prevent accidental runtime
pool shutdown under live traffic.
Signed-off-by: Adrien Langou <alangou@nvidia.com>
* test(e2e): add simple e2e test with kubernetes to test /readyz
Signed-off-by: Adrien Langou <alangou@nvidia.com>
---------
Signed-off-by: Adrien Langou <alangou@nvidia.com>
* fix(sandbox): add mechanistic smoke test for L4 deny and document the L4/L7 split
The old smoke script exercised an L7 PUT which hung because the denial
aggregator is only wired to L4 CONNECT denies, not L7 enforcement.
Add mechanistic-smoke.sh which triggers an L4 deny, waits for the
aggregator to flush, and asserts a pending chunk appears under
openshell rule get --status pending.
Document the intentional L4-only scope of the mechanistic mapper in
architecture/sandbox.md.
Fixes#1333
Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
* refactor(smoke): remove redundant variable inits and merge double step call
Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
* fix(smoke): wire mechanistic smoke into mise and guard TMP_DIR
- Initialize TMP_DIR before trap to prevent unbound variable on early exit
- Add e2e:mechanistic-smoke mise task with gateway setup
- Document mechanistic smoke in policy-advisor README
* test(proxy): verify L4 deny enqueues a DenialEvent
Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
* fix(proxy): remove unnecessary path qualifications in L4 denial smoke test
---------
Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
This makes it so you can run the dev gateway and sandbox with:
```
mise run gateway
# in another shell
mise run sandbox
```
Signed-off-by: Kris Hicks <khicks@nvidia.com>
macOS ships bash 3.2 which lacks mapfile/readarray. Replace all three
occurrences in configure_ghcr_credentials, cluster_has_image, and
cluster_image_platform with a portable while-read loop, consistent
with the fix applied to docker-build-image.sh in #1334.
Refresh the CI image tool pins so Go-built tools are rebuilt with patched Go releases and move the sandbox Python runtime to 3.14.5.
Rebase the gateway runtime to a pinned distroless Debian 13 image with glibc 2.41-12+deb13u3 while preserving the existing UID/GID 1000 runtime identity for upgrade compatibility. Update rustls-webpki to 0.103.13 and clarify Linux k3d guidance now that k3d is not installed through mise on Linux.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
Drew Newberry