Commit Graph
72 Commits
Author SHA1 Message Date
Evan Lezar a226806026 test(e2e): run gpu workloads from manifest (#1709)
* test(e2e): add workload manifest build flow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): add gpu workload validation tests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(e2e): build gpu workloads before gpu e2e

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-30 09:54:12 +02:00
krishicksandddurst 7bce1223dc feat(policy): add JSON-RPC and MCP L7 policies (#1865)
Add policy schema, proto, provider profile, OPA, and L7 proxy support for
`protocol: json-rpc` and `protocol: mcp`. Generic JSON-RPC endpoints match
exact method names only, with `method: "*"` as the all-method sentinel;
wildcard/glob methods and params matchers are rejected.

Parse JSON-RPC request bodies and batches in the forward proxy, deny
response-shaped client frames, limit receive-stream GET allowance to MCP
endpoints, and redact params in decision logs. Preserve L7 rule params on the
proto load path so MCP `tools/call` tool filters behave like YAML-loaded
policies.

Add MCP conformance coverage, JSON-RPC L7 e2e coverage, and docs for the new
protocols and current matcher limitations.

Signed-off-by: Kris Hicks <khicks@nvidia.com>
Co-authored-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
2026-06-26 15:52:16 -07:00
Simon Scatton c7879a0a1d test(e2e): stop using custom e2e binary builds (#2000)
Remove the production-crate dev-settings feature so e2e tests build the same gateway, CLI, and supervisor binaries that we ship. Move settings coverage from dummy test-only keys to the production ocsf_json_enabled setting.

Signed-off-by: Simon Scatton <sscatton@nvidia.com>
2026-06-25 13:08:04 +00:00
Evan Lezar b6428cb9cd fix(build): align container engine selection (#1944)
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-22 10:50:16 -05:00
Drew Newberry f1245a33b2 test(e2e): retry transient forward proxy stale policy responses (#1929)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-06-16 17:11:58 -07:00
Evan Lezar 294c64eed1 fix(gpu): prefer single CDI devices for local runtimes (#1675)
Prefer a single CDI-qualified device when Docker or Podman resolves the default GPU request to one GPU.

Allow nvidia.com/gpu=all only as a WSL2 all-only compatibility fallback, using Docker daemon info and Podman's /dev/dxg probe to identify that case.

Update driver docs, architecture notes, and GPU e2e coverage for the default selection behavior.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-16 10:14:49 +02:00
Alex Lewontin fb83d1a3c8 feat(gateway): add system registry support and source indicators (#1625)
* feat(bootstrap): add system gateway registry for installer defaults

Adds a read-only installer-seeded gateway registry that the CLI consults after per-user gateway config. The registry uses the same layout as per-user config with `active_gateway` at the root and `gateways/<name>/metadata.json` beneath it. By default the system config root is `/etc/openshell`, while `OPENSHELL_SYSTEM_GATEWAY_DIR` remains available as an override for packages that need a different location. User-managed gateways continue to shadow installer entries on name collision.

Originally-authored-by: Mark Shuttleworth <mark@ubuntu.com>
Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>

* feat(cli): show gateway config source in list and term

Expose whether a gateway registration comes from user or system config in `openshell gateway list`, the TUI gateway pane, and list JSON output. The CLI also refuses to remove system-managed registrations and the smoke tests cover the new list output.

Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>

* fix(bootstrap): preserve user shadowing on invalid metadata

Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>

* fix(cli): keep system fallback out of rollback state

* docs(gateway): describe system config fallback layout

* test(bootstrap): cover system gateway last_sandbox persistence

* test(gateway): cover system-only removal rejection

* fix(bootstrap): validate gateway names before path joins

---------

Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>
2026-06-11 16:25:12 -07:00
Evan Lezar 1dc59853ff feat(gpu): move device selection to driver config (#1815)
* refactor(kubernetes): use shared proto struct helpers

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* refactor(docker): extract sandbox template validation

* feat(gpu)!: move device selection to driver config

BREAKING CHANGE: The openshell sandbox create --gpu-device flag and
corresponding API field were removed. Select specific GPUs through
driver-specific driver_config fields instead.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(kubernetes): reject unknown driver config fields

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-11 12:53:06 +02:00
Drew Newberry 530aaf1360 feat(drivers): support docker and podman config mounts (#1785)
* feat(drivers): support docker and podman config mounts

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(drivers): trim mount docs

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* test(e2e): cover local driver volume mounts

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(podman): satisfy linux clippy lint

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(drivers): gate bind mounts behind gateway config

* docs(sandbox): simplify mount examples

* cleanup

* test(e2e): stabilize branch checks

* fix(drivers): tighten local mount validation

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-06-10 11:46:20 -07:00
Russell Bryant 3a4463e36a fix(cli): fall back to regular upload when git filtering excludes all files (#1783)
* fix(cli): fall back to regular upload when git filtering excludes all files

When uploading a directory whose contents are entirely excluded by
.gitignore, git_sync_files returns an empty file list. The git-aware
upload path silently accepted this and printed "Upload complete" without
transferring any files. Now both call sites (direct CLI handler and
sandbox_upload_plan) check for an empty file list and fall through to the
regular, unfiltered upload path instead.

Closes #1778

Signed-off-by: Russell Bryant <rbryant@nvidia.com>

* test(e2e): add regression test for gitignored upload fallback (#1778)

Reproduces the reported workflow: download a file from a sandbox into a
local git repo where the target directory is gitignored, then re-upload
that directory without --no-git-ignore. Before the fix the upload
silently succeeded with zero files transferred.

Signed-off-by: Russell Bryant <rbryant@nvidia.com>

* fix(cli): warn on stderr when gitignore fallback triggers

Address PR review: print a notice when .gitignore filtering excludes
all files and the CLI falls back to unfiltered upload. Add a
GitFilteredEmpty variant to SandboxUploadPlan so the caller can
distinguish this case and emit a visible warning. Document the
default gitignore filtering and fallback behavior in manage-sandboxes.

* fix(cli): warn on stderr for direct upload gitignore fallback

The SandboxCommands::Upload path in main.rs had the same silent
fallback as sandbox_create. Restructure the git_sync_files check to
emit the same .gitignore warning before falling back to unfiltered
upload. Assert the warning in the E2E regression test.

* docs(sandboxes): fix misleading upload fallback guidance

Replace inaccurate "explicit file list" advice with actionable
alternatives: --no-git-ignore, uploading outside the work tree,
or git force-adding the files.

* fix(cli): collapse nested ifs to satisfy clippy::collapsible_if

* style(cli): apply rustfmt to main.rs

---------

Signed-off-by: Russell Bryant <rbryant@nvidia.com>
2026-06-08 22:16:50 -07:00
Philippe Martin eea9751c01 feat(cli): support multiple --upload flags on sandbox create (#1635) (#1645)
Closes #1635

Signed-off-by: Philippe Martin <phmartin@redhat.com>
2026-06-04 07:58:56 -07:00
Taylor Mutch e4bcfdfaaa fix(gateway): allow local sandbox jwt to not expire (#1721) 2026-06-03 13:50:56 -05:00
alangou 2bdc968ed9 fix(gateway): make readiness health checks dependency-aware (#1328)
* feat(gateway): add readiness probe metrics and test-only store close

Emit Prometheus readiness metrics for database probes (healthy gauge and
outcome-labeled latency histogram) with coverage in health HTTP tests.
Restrict Store::close behind test support cfg to prevent accidental runtime
pool shutdown under live traffic.

Signed-off-by: Adrien Langou <alangou@nvidia.com>

* test(e2e): add simple e2e test with kubernetes to test /readyz

Signed-off-by: Adrien Langou <alangou@nvidia.com>

---------

Signed-off-by: Adrien Langou <alangou@nvidia.com>
2026-05-27 11:00:06 -07:00
Evan Lezar cade0bb0ac test(e2e): default GPU probe image (#1450)
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-05-20 13:25:09 +02:00
John T. Myers d255cdd9c9 feat(providers): add credential refresh foundation (#1349)
* feat(providers): add credential refresh foundation

* feat(providers): mint oauth refresh token credentials

* fix(providers): allow delegated refresh bootstrap

* fix(providers): guard refresh credential modes

* fix(providers): refine refresh lifecycle UX

* fix(cli): restore provider test gateway response import

* fix(server): clean auth endpoint test qualifications

* fix(providers): tighten refresh authorization and collisions

* chore(providers): trim bundled v2 profiles

* fix(providers): resolve refresh rebase fallout

* fix(cli): accept rfc3339 credential expiry

* test(providers): update inferred claude provider type

* test(providers): avoid removed outlook default profile

* test(providers): isolate attach limit fixtures
2026-05-19 07:42:10 -07:00
Russell Bryant 65a3a7c2ec test(e2e): close Podman driver test coverage gaps (#1439)
* test(e2e): close Podman driver test coverage gaps

Add podman_gateway_resume test following the VM pattern (no
container-state assertions since Podman keeps containers running across
gateway restarts). Widen websocket_conformance feature gate from
e2e-docker to e2e-host-gateway so it runs on both Docker and Podman.
Fix e2e-podman.sh to default to the e2e-podman feature so Podman-
specific and host-gateway tests are actually included in Podman CI runs.

* refactor(e2e): extract shared CLI helpers from gateway_resume tests

Move run_cli, wait_for_healthy, sandbox_names, and
wait_for_sandbox_exec_contains into a new harness::cli module. All three
gateway_resume variants (Docker, Podman, VM) now import these shared
helpers instead of defining identical copies. Docker-specific container
introspection functions remain local to gateway_resume.rs.
2026-05-18 17:09:16 -07:00
Drew Newberry f819f7dcb5 fix(vm): restore sandboxes after gateway restart (#1407) 2026-05-15 15:08:59 -07:00
Drew Newberry 910d3f09f2 feat(vm): boot sandboxes from ext4 root disks (#1263)
* feat(vm): boot sandboxes from ext4 root disks

* feat(vm): boot sandboxes with writable overlay disk

* fix(sandbox): improve sandbox provisioning behavior

* fix(vm): repair sandbox image provisioning

* fix(vm): clean up rebase fallout

* perf(vm): cache empty overlay images

* perf(vm): skip repeated sandbox ownership repair

* fix(vm): address lifecycle review issues

* fix(vm): repair gateway vm startup

* wip
2026-05-15 13:23:10 -07:00
Russell Bryant bbfcac8a8e test(e2e): add bypass detection test for sandbox REJECT rules (#1368)
Validates that direct TCP connections bypassing the HTTP CONNECT proxy
are rejected immediately (ECONNREFUSED) rather than hanging until a
network timeout. This is implementation-agnostic — it tests observable
kernel behavior regardless of whether rules are installed via iptables
or nftables.
2026-05-13 17:41:10 -07:00
Tinson Lai c99849bcb7 fix(cli): cp-style sandbox download and workspace-boundary check (#1353)
* fix(cli): cp-style sandbox download and workspace-boundary check

`openshell sandbox download` produced a directory at the host destination
instead of writing the source bytes as a regular file, and did not refuse
sandbox-side paths that resolved outside `/sandbox` (only `creds.json` was
rejected, by filesystem policy rather than by an explicit check).

Mirrors the cp-style design from the upload-side fixes (#667 / #694 and
#885 / #952): destination resolution now respects trailing slashes and
existing-directory destinations, single-file sources land as regular files
via a staged extraction and atomic rename, and the function delegates the
SSH tar stream to a shared helper. Adds an explicit lexical
workspace-boundary check on the sandbox-side source — required on the
download side because filesystem policy alone is not a substitute for an
explicit check when crossing the trust boundary in the leak direction.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

* fix(cli): canonicalise sandbox download path remotely and quote tar member

The lexical check in `validate_sandbox_source_path` clears any path whose
components stay under `/sandbox` after `.` and `..` are folded out, but it
cannot see symlinks. A workspace-relative symlink such as
`/sandbox/etc-link -> /etc` lets `openshell sandbox download <name>
/sandbox/etc-link/passwd /tmp/passwd` slip through, then `tar -C
/sandbox/etc-link passwd` follows the link on the sandbox side and
streams `/etc/passwd` back to the host.

Add `canonicalize_sandbox_source_path` which runs `realpath -e --` over
SSH and rejects the result unless it still equals `/sandbox` or starts
with `/sandbox/`. The lexical guard stays as a cheap fail-fast for
obviously-bad inputs, and the rest of the download flow now uses the
canonical path everywhere a tar invocation or probe might dereference a
symlink.

While editing the single-file branch, also add the `--` separator before
the basename in `tar cf - -C <parent> -- <name>`. Without it, a sandbox
file whose basename starts with `-` (e.g. `--checkpoint-action=...`) is
parsed by GNU tar as an option rather than a member, which at best
fails the wrap and at worst triggers tar option behaviour. The directory
branch passes literal `.` and is unaffected.

Cover both fixes with pure-function tests: `is_under_sandbox_workspace`
gets accept/reject pairs (the same predicate the post-realpath
validator uses) and the new `build_single_file_tar_cmd` helper asserts
the `--` separator and shell escaping survive any future refactor.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

* chore(cli): drop redundant tempfile dev-dependency entry

`tempfile` was promoted from `[dev-dependencies]` to `[dependencies]`
once the cp-style single-file download path started using
`TempDir::new_in` from production code. The original `[dev-dependencies]`
line stayed behind by accident — entries under `[dependencies]` are
already available to test compilation, so the second declaration is
dead weight.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

* test(cli): add E2E coverage for sandbox download paths

Adds four end-to-end tests for `openshell sandbox download` so the
happy path and both Codex regressions have explicit coverage:

* `sandbox_download_file_only` — seeds a single file via the new
  `SandboxGuard::exec` harness helper and pulls it back, exercising
  `sandbox_sync_down_file` without an upload phase.
* `sandbox_download_directory_only` — seeds a small directory tree and
  pulls it recursively, exercising `sandbox_sync_down_directory`.
* `sandbox_download_rejects_symlinks_pointing_outside_workspace` —
  regression for the high-severity finding. Plants three escape shapes
  (`/sandbox/etc-link -> /etc`, `/sandbox/passwd-link -> /etc/passwd`,
  and a path with a symlinked component `/sandbox/etc-link/passwd`) and
  asserts all three are refused with a workspace-boundary error before
  any tar streams. Also checks the host destination stays empty so a
  silent leak would surface.
* `sandbox_download_handles_dash_leading_basename` — regression for the
  medium-severity finding. Seeds a file named `--checkpoint-action=evil`
  and verifies it downloads with byte-for-byte contents, proving the
  `--` separator stops tar parsing the basename as an option.

The harness gains a single new method, `SandboxGuard::exec`, that wraps
`openshell sandbox exec --name <name> --no-tty -- <argv>`. Existing
tests preferred the upload path to populate sandbox state, but the
regression cases need to plant filesystem state the upload flow cannot
produce (symlinks, files whose basenames clap would otherwise reject).

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

* docs(cli): note cp-style destination and workspace boundary on download

`openshell sandbox download` now refuses any source path that resolves
outside `/sandbox` (lexically or through a symlink), and the
single-file branch follows `cp`-style placement rules. Document both
so users hitting the workspace-boundary error or the
"land at exact path vs land inside dir" decision can self-serve.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

* test(cli): tolerate miette line-wrapping in symlink-rejection assertions

`sandbox_download_rejects_symlinks_pointing_outside_workspace` matched
the substring `"outside the sandbox workspace"` against the CLI's error
output. The check is correct in spirit but fragile in practice: miette
wraps long single-line errors across multiple lines at terminal width
and inserts a `│ ` continuation marker, so the substring breaks across
the wrap and the test fails even though the canonicalisation step
correctly refused the source.

Split the match into three short markers — `resolves to`, `outside the`,
`sandbox workspace` — each guaranteed to fit on one wrapped line. The
combined check is stricter than the original (it now requires evidence
that the remote `realpath -e` step actually ran, not just the boundary
refusal) without inheriting the wrap fragility.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

---------

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
2026-05-13 17:17:06 -07:00
Taylor MutchandDrew Newberry ba77967fc4 refactor(docker): split gateway/supervisor Dockerfiles and use native rust builds (#1316)
* wip

* refactor(docker): use native rust builds for split gateway/supervisor images

Drop the in-Docker BUILD_FROM_SOURCE path so both images consume only
prebuilt binaries staged natively via tasks/scripts/stage-prebuilt-binaries.sh.
This mirrors what CI does and reuses the host's cargo target cache and
sccache across rebuilds.

- Dockerfile.gateway: nvcr.io/nvidia/distroless/cc:v4.0.4 base (the 4.0.0
  tag does not exist on nvcr.io; the registry uses a v prefix). GNU-linked
  binary copied to /usr/local/bin.
- Dockerfile.supervisor: scratch base, static musl binary. Static linkage
  lets the image stay scratch while still being executable as a Kubernetes
  init container.
- skaffold.yaml: each artifact invokes tasks/scripts/docker-build-image.sh,
  which stages the binary natively (cargo / cargo-zigbuild) and then builds
  the image. Drops the cross-build.sh dependency from the supervisor build.
- seccomp.rs: add a local SYS_kexec_file_load constant for musl/aarch64.
  libc 0.2.185 omits the symbol from its musl/aarch64 bindings, so the
  supervisor's seccomp filter previously failed to compile for that target.
- architecture/build.md: describe the native-first pipeline and per-image
  runtime choices.

Local validation: gateway image 101MB (was 194MB), supervisor image 21.7MB.
helm:skaffold:run deploys cleanly; the static musl supervisor binary runs
correctly in a non-glibc agent container.

* refactor(docker): tighten binary perms via --chown + 0550

Replace `COPY --chmod=755` with `COPY --chown=<user> --chmod=0550` in
the gateway and supervisor Dockerfiles. The binary is no longer
world-readable or world-executable; ownership is pinned to the runtime
user.

- Gateway uses `--chown=nvs:nvs` + `USER nvs:nvs`, matching the only
  non-root user defined in `nvcr.io/nvidia/distroless/cc` (UID 1000) and
  the Helm chart's `securityContext.runAsUser: 1000`, which overrides
  the Dockerfile USER at runtime.
- Supervisor uses numeric `--chown=65534:65534` because the scratch base
  has no `/etc/passwd` for name resolution. The supervisor image is
  only consumed by the init-container copy-self path; the destination
  pod's runAsUser governs execute access.

Validated by deploying to a local k3d cluster via `helm:skaffold:run`
and confirming the gateway StatefulSet reaches 1/1 Running.

* ci(gpu): repoint GPU probe image lookup at Dockerfile.gateway

The previous awk parsed `FROM <image> AS gateway` from the now-deleted
`Dockerfile.images`. The new `Dockerfile.gateway` uses an ARG with a
default (`ARG GATEWAY_BASE_IMAGE=nvcr.io/nvidia/distroless/cc:v4.0.4`)
and `FROM ${GATEWAY_BASE_IMAGE} AS gateway`, so the old script returns
nothing.

Parse the ARG default value directly so the GPU prerequisites check
keeps using the gateway base image as a `nvidia-smi` probe target.

* ci(gpu): pin GPU probe to nvcr.io/nvidia/base/ubuntu:noble

The previous probe parsed the gateway base image out of the Dockerfile,
relying on the fact that the gateway ran on `nvcr.io/nvidia/base/ubuntu`
and that NVIDIA Container Toolkit CDI injection would populate
`nvidia-smi` and the supporting libs at runtime. The new gateway base
(`nvcr.io/nvidia/distroless/cc`) lacks `ldconfig`, a populated
`/usr/bin`, and the broader filesystem layout CDI injection assumes,
so it cannot serve as a GPU probe.

Pin the probe image explicitly to the NVIDIA-managed Ubuntu base. The
probe is independent of the gateway runtime and survives future base
swaps.

* fix(e2e-gpu): pass GPU probe image via env, drop Dockerfile.images parse

The Rust e2e test `gpu_request_for_each_discovered_device_matches_plain_container`
was still parsing the deleted `Dockerfile.images` to derive its GPU probe
image, panicking with `No such file or directory` after this PR's
Dockerfile split.

Move the probe image to a single source of truth in the workflow
(`OPENSHELL_E2E_GPU_PROBE_IMAGE` env at the job level) and require the
e2e test to read it from there. No silent codebase default — the test
panics with a pointer at the workflow if the env is missing, so it
fails loudly rather than drifting from CI.

The prereq probe step in the workflow now consumes the same env, so
the probe image is declared exactly once.

* fix(docker): keep supervisor binary root-owned for rootless Podman

The Podman driver mounts the supervisor image read-only into the sandbox
container at /opt/openshell/bin and runs that container as UID 0, but
deliberately drops DAC_OVERRIDE for hardening (container.rs:419). With
--chown=65534:65534 --chmod=0550 the binary was r-xr-x--- owned by UID
65534, so the container's UID 0 fell into "other" with no read or exec
access and the supervisor crashed on start (ContainerExited code 1).

Docker and Kubernetes both retain DAC_OVERRIDE, so root could still
exec the file — which is why this regression only surfaced in the
Podman e2e job.

Drop --chown from the supervisor COPY so the binary stays root-owned.
Keep --chmod=0550: the security win was dropping world-execute, not
changing the owner. The chown bought nothing here because the container
is always UID 0 regardless of driver, but it actively broke the only
driver that drops DAC_OVERRIDE.

---------

Co-authored-by: Drew Newberry <anewberry@nvidia.com>
2026-05-12 16:05:49 -07:00
Aaron Erickson 🦞andJohn Myers 9ea94b645d fix(sandbox): rewrite messaging credential placeholders (#1286)
* fix(sandbox): rewrite credential placeholders in websocket text frames

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): harden websocket credential rewrite

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* feat(sandbox): add websocket l7 inspection and compression

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): harden websocket upgrade validation

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* test(sandbox): cover route-selected websocket upgrades

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): harden websocket negotiation parsing

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* test(sandbox): add websocket conformance relay matrix

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* test(e2e): add websocket conformance lane

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(policy): support websocket incremental rules

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* feat(policy): enable websocket credential rewrite updates

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(cli): make websocket rewrite endpoint-local

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* feat(sandbox): support graphql websocket policy

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(policy): allow private IPs for websocket endpoints

* feat(sandbox): rewrite REST credential placeholders

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* refactor(sandbox): generalize credential aliases

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): rewrite encoded form credentials

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): close websocket policy and provider alias gaps

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(e2e): route websocket probe through host gateway

* fix(e2e): stabilize websocket probe handshake

* fix(e2e): exercise websocket probe through proxy

* ci: remove websocket conformance workflow

---------

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
2026-05-11 22:40:44 -07:00
Evan Lezar 5c98604f0f feat(gpu): honor device IDs in Docker and Podman (#1253)
* feat(gpu): honor device IDs in Docker and Podman

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(gpu): add Docker and Podman device selection e2e

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(gpu): run Docker GPU e2e workflow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-05-11 17:09:37 +02:00
John T. Myers 1d3b741ee3 feat(providers): support sandbox provider attach lifecycle (#1242)
* feat(providers): support sandbox provider attach lifecycle

Closes #1171

Adds sandbox provider list, attach, and detach API/CLI support while keeping provider policy and credential resolution derived from current sandbox attachments.

* fix(providers): refresh sandbox provider credentials

Adds provider environment revisions and generation-scoped sandbox credential snapshots so future SSH and exec launches pick up provider attach, detach, and credential updates without mutating already-running processes.

Also blocks provider deletion while attached to prevent stale sandbox provider references.

* fix(providers): serialize sandbox object mutations

* test(providers): cover sandbox provider attach lifecycle

* test(providers): accept versioned credential placeholders
2026-05-08 11:14:44 -07:00
Mrunal Patel 1f35abbefb feat(sandbox): add Kubernetes user namespace isolation (hostUsers: false) (#983)
Add opt-in support for Kubernetes user namespace isolation on sandbox
pods. When enabled, container UID 0 maps to an unprivileged host UID
and capabilities become namespaced, providing defense-in-depth for the
supervisor process.

Configuration is two-layered: a cluster-wide default via
OPENSHELL_ENABLE_USER_NAMESPACES (default false) and a per-sandbox
override via the new `user_namespaces` field on SandboxTemplate.

When user namespaces are active, the pod security context is extended
with SETUID, SETGID, and DAC_READ_SEARCH capabilities to match the
bounding-set requirements inside a user namespace.

Introduces SandboxPodParams struct to replace long argument lists on
sandbox_to_k8s_spec and sandbox_template_to_k8s.

Validated end-to-end on OCP 4.22 (K8s 1.35.3, CRI-O 1.35, RHEL
CoreOS, kernel 5.14) with full SSH tunnel and non-identity UID mapping.
2026-05-08 12:39:48 -04:00
Drew Newberry 70a0f6c547 refactor(cli): remove gateway lifecycle management (#1221) 2026-05-07 09:54:13 -07:00
Drew Newberry 23ad858a4f test(e2e): add podman rust suite (#1185)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-05-06 12:25:49 -07:00
Drew Newberry 4d388d2677 ci(vm): cleanup vm build infra (#1186) 2026-05-06 08:58:02 -07:00
Drew Newberry e4b4e923ae test(e2e): run suites against docker gateway (#1153) 2026-05-05 13:28:08 -07:00
Alexander Watson 25c4fdecd1 fix(examples): repair multi-agent notepad uploads (#1152) 2026-05-04 13:31:25 -07:00
John T. Myers 6b21804258 feat(policy): add GraphQL L7 inspection (#1083)
Support GraphQL L7 policies
2026-05-04 11:50:09 -07:00
Drew Newberry 08001ca616 fix(docker): harden supervisor startup and gateway routing (#1128) 2026-05-03 21:38:20 -07:00
Evan Lezar 0c0f3e32e1 feat(docker): enable CDI GPU sandboxes (#1036) 2026-04-30 13:23:57 -07:00
Drew Newberry 24724742a8 ci(rust): enforce -D warnings on clippy (#1008) 2026-04-29 12:12:53 -07:00
John T. Myers d414e69a20 refactor(server): unify policy persistence in objects table (#972)
* refactor(server): unify policy persistence in objects table

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* fix(server): clean sandbox-owned records on reconcile delete

* refactor(server): use protos for stored policy records

* refactor(server): move policy persistence into policy_store

* fix(server): restore compute runtime merge compatibility

* fix(server): validate draft chunk sandbox ownership

---------

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
2026-04-28 16:15:14 -07:00
Derek Carr de9dce0436 fix(e2e): use high UID range to avoid host user conflicts (#978)
Change sandbox user UID from 1000 to 1000660000 in custom image examples and E2E tests.
Using a high UID range (1000000000+) prevents conflicts with host users when running
without user namespace remapping, where container UIDs map directly to host UIDs.

This resolves fork failures caused by RLIMIT_NPROC enforcement when the host user
already has many threads running.

Signed-off-by: Derek Carr <decarr@redhat.com>
2026-04-27 09:46:39 -07:00
Derek Carr 5e28ea3a4b feat(server): add object meta convention to top-level objects (#919)
- adds filterable label selectors on resources

Closes #864

Signed-off-by: Derek Carr <decarr@redhat.com>
2026-04-27 07:16:07 -07:00
Adam Miller d44d8a1e27 feat: Openshell driver podman (#904)
* feat(podman): add Podman compute driver for rootless sandbox management

Adds openshell-driver-podman, a new compute driver that manages OpenShell
sandboxes as rootless Podman containers via the Podman REST API over a
Unix socket. Enables local workstation sandboxes without Kubernetes.

Driver features:
- Bridge networking with ephemeral host-port mapping for rootless SSH reachability
- Named volumes for workspace storage, Podman native health checks, GPU via CDI
- Supervisor binary sideloaded via image volume mount (BYOC-compatible)
- SSH handshake secret injected via Podman secrets API (not plaintext env)
- Typed ContainerSpec structs, input validation, and path-traversal guards
- Cgroups v2 required; fails fast on v1 hosts
- Bounded event stream buffer; watch stream reconnection handled by server watch_loop
- Graceful shutdown and standalone driver binary with gRPC bridge

Rootless-specific fixes:
- Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps
- Add /run/netns tmpfs mount for ip netns in rootless containers
- Use secret_env map (not secrets array) for env-var injection in libpod API
- Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP

Server/sandbox hardening:
- Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback
- Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting
- Prevent OPENSHELL_* env var override by user-supplied spec environment maps
- Disable SQLite pool idle_timeout/max_lifetime for in-memory databases
- Emit deleted_event on 404-during-inspect instead of regressing sandbox phase
- Key delete cleanup by stable sandbox_id to survive container label drift

CLI fixes:
- Restore --name as a named flag on sandbox create (not positional)
- Fix exec command arg parsing to not consume sandboxed-command flags
- Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL

Build tooling:
- Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes
  unified ce_* helpers; all build/cluster/VM scripts updated to use it
- Add docker:build:supervisor mise task for standalone supervisor image
- Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages
- Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): derive grpc endpoint from server bind port

When a user starts the gateway on a non-default port (e.g. --port 8081),
sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the
default port 8080. The driver's auto-detection fallback read
OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset,
and fell back to DEFAULT_SERVER_PORT.

Add gateway_port to PodmanComputeConfig and thread config.bind_address.port()
from the server into the driver so the fallback uses the actual listening
port. Remove the OPENSHELL_BIND_ADDRESS env var read and the
extract_port_from_bind_address helper which are no longer needed.

Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver
binary for parity when the driver is run outside the embedded server path.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): address PR feedback on env test safety and cluster DNS docs

Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars
and a static ENV_LOCK mutex, fixing a data race in parallel test execution.
The prior safety comment incorrectly claimed Cargo runs tests single-threaded.

Update debug-openshell-cluster skill to accurately document the DNS proxy
strategy (setup_dns_proxy + public DNS fallback) and clarify the separation
between cluster DNS and sandbox agent DNS enforcement.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(e2e): resolve CI failures in auth timeout, test harness, and formatting

- Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead
  of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives
- Add timeout to SandboxGuard::create() and create_with_upload() to
  prevent indefinite hangs (matches create_keep() which already had one)
- Add missing '--' separator in no_proxy test before command args
- Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs
- Run cargo fmt on openshell-driver-podman
- Refine cluster DNS docs for Podman in debug-openshell-cluster skill

Signed-off-by: Adam Miller <admiller@redhat.com>

* refactor(server): remove allows_loopback_endpoints from ComputeRuntime

SSRF protection is now handled at the network and proxy layers
(openshell-core net.rs, openshell-sandbox proxy.rs) rather than
requiring per-driver flags on ComputeRuntime. Update architecture
docs to reflect supervisor relay SSH transport and add rootless
networking deep-dive.

Signed-off-by: Adam Miller <admiller@redhat.com>

---------

Signed-off-by: Adam Miller <admiller@redhat.com>
2026-04-24 10:30:14 -07:00
John T. MyersandJohn Myers 8cf5ebdc89 test(e2e): fix gitignore upload assertion path (#962)
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
2026-04-24 09:26:50 -07:00
Drew Newberry a34b25a5ba test(e2e): fix rust upload path assertions (#960)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-04-24 08:28:40 -07:00
John T. Myers 28db08eeb8 fix(sandbox): disable child core dumps (#821)
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
2026-04-14 22:28:18 -07:00
Drew Newberry ddb85b1704 feat(vm): add openshell-vm crate with libkrun microVM gateway (#611) 2026-04-08 22:00:01 -07:00
John T. Myers 77e55ea989 test(e2e): replace flaky Python live policy update tests with Rust (#742)
Remove test_live_policy_update_and_logs and
test_live_policy_update_from_empty_network_policies from the Python e2e
suite. Both used a manual 90s poll loop against GetSandboxPolicyStatus
that flaked in CI with 'Policy v2 was not loaded within 90s'.

Add e2e/rust/tests/live_policy_update.rs with two replacement tests
that exercise the same policy lifecycle (version bumping, hash
idempotency, policy list history) through the CLI using the built-in
--wait flag for reliable synchronization.
2026-04-02 15:06:32 -07:00
Drew Newberry e837849009 feat(bootstrap): resume gateway from existing state and persist SSH handshake secret (#488) 2026-04-02 09:50:10 -07:00
Evan LezarandPiotr Mlocek 047de66b22 feat(bootstrap,cli): switch GPU injection to CDI where supported (#495)
* feat(bootstrap): switch GPU injection to CDI where supported

Use an explicit CDI device request (driver="cdi", device_ids=["nvidia.com/gpu=all"])
when the Docker daemon reports CDI spec directories via GET /info (SystemInfo.CDISpecDirs).
This makes device injection declarative and decouples spec generation from consumption.

When the daemon reports no CDI spec directories, fall back to the legacy NVIDIA device
request (driver="nvidia", count=-1) which relies on the NVIDIA Container Runtime hook.
Failure modes for both paths are equivalent: a missing or stale NVIDIA Container Toolkit
installation will cause container start to fail.

CDI spec generation is out of scope for this change; specs are expected to be
pre-generated out-of-band, for example by the NVIDIA Container Toolkit.

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Piotr Mlocek <pmlocek@nvidia.com>
2026-03-31 08:23:52 -07:00
John T. MyersandJohn Myers 758c62d18d fix(sandbox): handle per-path Landlock errors instead of abandoning entire ruleset (#677)
* fix(sandbox): handle per-path Landlock errors instead of abandoning entire ruleset

A single missing path (e.g., /app in containers without that directory)
caused PathFd::new() to propagate an error out of the entire Landlock
setup closure. Under BestEffort mode, this silently disabled all
filesystem restrictions for the sandbox.

Changes:
- Extract try_open_path() and classify_path_error() helpers that handle
  PathFd failures per-path instead of per-ruleset
- BestEffort mode: skip inaccessible paths with a warning, apply
  remaining rules
- HardRequirement mode: fail immediately on any inaccessible path
- Add zero-rule safety check to prevent applying an empty ruleset that
  would block all filesystem access
- Pre-filter system-injected baseline paths (e.g., /app) in enrichment
  functions so missing paths never reach Landlock
- Add unit tests for try_open_path, classify_path_error, and error
  classification for ENOENT, EACCES, ELOOP, ENAMETOOLONG, ENOTDIR
- Update user-facing docs and architecture docs with Landlock behavior
  tables, baseline path filtering, and compatibility mode semantics
- Fix stale ABI::V1 references in docs (code uses ABI::V2)

Closes #664

* fix(sandbox): use debug log for NotFound in Landlock best-effort mode

NotFound errors for stale baseline paths (e.g. /app persisted in the
server-stored policy but absent in this container) are expected in
best-effort mode. Downgrade from warn! to debug! so the message does
not leak into SSH exec stdout (the pre_exec hook inherits the tracing
subscriber whose writer targets fd 1).

Genuine errors (permission denied, symlink loops, etc.) remain at warn!
for operator visibility.

Also move custom_image e2e marker from /opt to /etc (a Landlock baseline
read-only path) since the security fix now properly enforces filesystem
restrictions.

---------

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
2026-03-30 13:19:19 -07:00
John T. MyersandJohn Myers 0832f11a66 fix(e2e): add uv-managed python binary glob to forward proxy L7 test (#686)
The base sandbox image installs Python via uv at
/sandbox/.uv/python/*/bin/python*, but the proxy resolves binary
identity via /proc/PID/exe (the real path, not the symlink). The test
policy only listed /usr/bin/python* and /usr/local/bin/python*, so OPA
denied the connection at L4 before L7 evaluation could run.

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
2026-03-30 11:58:45 -07:00
LateNightHackathonandlatenighthackathon 94fbb643b3 fix(proxy): add L7 inspection to forward proxy path (#666)
* refactor(l7): export evaluate_l7_request for cross-module use

Make evaluate_l7_request() public so the forward proxy path can
evaluate individual requests against L7 policy without going
through the full relay_with_inspection() loop.

Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com>

* fix(proxy): add L7 inspection to forward proxy path

The forward proxy previously rejected all requests to endpoints with
L7 rules (blanket 403), forcing clients through the CONNECT tunnel.
This meant policies like read-only (allow GET, block POST) had no
effect on plain http:// requests through the forward proxy.

Replace the blanket rejection with actual L7 evaluation:
- Query L7 config for the endpoint (same as before)
- Clone the OPA engine and evaluate the request method/path
- Allow if L7 policy permits, deny with 403 if enforcement is enforce
- Audit mode: log but allow (matching CONNECT path behavior)
- Fail-closed: deny on evaluation errors

The forward proxy uses Connection: close (one request per connection),
so a single evaluation suffices — no relay loop needed.

Update e2e tests to validate the new behavior:
- GET /allowed → 200 (L7 policy allows)
- POST /allowed → 403 (L7 policy denies, enforcement: enforce)

Update security-policy.md to reflect the new forward proxy L7 behavior.

Closes #643

Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com>

* style(proxy): apply cargo fmt formatting

Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com>

---------

Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
Co-authored-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
2026-03-30 09:24:38 -07:00
John T. Myers a831a8921b feat(settings): gateway-to-sandbox runtime settings channel (#474)
* feat(gateway/sandbox): add global and sandbox runtime settings flow
2026-03-20 14:08:57 -07:00
Drew Newberry 5565a8bba3 fix(cli): suppress browser popup during auth via OPENSHELL_NO_BROWSER env var (#419) 2026-03-18 09:38:51 -07:00