Commit Graph
18 Commits
Author SHA1 Message Date
Derek Carr 5952a5a23f feat(workspace): add workspace resource model with scoping, membershi… (#2243)
* feat(workspace): implement workspace model (Phase 1 of RFC 0011)

Implements workspace and membership model providing hard isolation
boundaries for multi-player OpenShell deployments.

Workspace CRUD with Kubernetes-style Terminating phase for graceful
deletion. All resources scoped by workspace via ObjectMeta. Membership
RPCs for workspace access control. Persistence migration shifts name
uniqueness to (object_type, workspace, name). Provider profiles support
platform and workspace scoping. Service routing uses workspace-prefixed
DNS labels. Inference routes renamed and workspace-scoped with
DeleteInferenceRoute RPC. Python SDK with WorkspaceClient, two-method
list pattern (workspace-scoped and for_all_workspaces), and workspace
parameter on all methods. CLI workspace flags, TUI workspace cycling.
K8s driver filters unmanaged CRs and uses delete preconditions. Podman
driver uses immutable container IDs. Label serialization fixed across
all put_if call sites.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(cli): delegate sandbox upload command to existing upload function

The standalone `sandbox upload` command reimplemented upload logic
inline with two bugs: it used `Path::exists()` which follows symlinks
(rejecting dangling symlinks), and it ran git-aware filtering on
symlink sources. The `run::sandbox_upload()` function already handles
both cases correctly via `sandbox_upload_plan()`. Replace the inline
logic with a call to the existing function.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(e2e): shorten sandbox names and fix test compatibility

Shorten the sandbox name in initial_sparse_policy_is_acknowledged_as_loaded
from 'e2e-2159-sparse-enrich' (22 chars) to 'e2e-sparse-enrich' (17 chars)
to comply with MAX_ROUTABLE_NAME_LEN (19 chars).

Also capture stderr in create_keep_with_args so future sandbox creation
failures include the actual CLI error instead of reporting empty output.

Signed-off-by: Derek Carr <decarr@redhat.com>

* test(workspace): add test coverage for workspace CRUD and persistence isolation

Add unit tests for workspace create happy path, get round-trip, get
not-found, get empty-name rejection, already-exists error, and
resolve_workspace not-found. Add persistence test proving cross-workspace
name uniqueness (same name in different workspaces produces separate
records). Add workspace name max-length boundary tests. Fix e2e harness
to include stderr in name-parse-failure error path. Align Python e2e
test_workspace_crud with try/finally pattern. Document provider profile
catalog workspace scoping gap in RFC 0011.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(examples): update examples for workspace model compatibility

Shorten sandbox names in demo scripts to fit the 19-character
MAX_ROUTABLE_NAME_LEN limit: policy-demo prefix to pd-, multi-agent
notepad derives a short SANDBOX_TAG from the run ID, governance
interceptor uses gs-PID-RANDOM. Update vscode-remote-sandbox.md SSH
host aliases from openshell-{name} to openshell-{name}.{workspace}
format.

Signed-off-by: Derek Carr <decarr@redhat.com>

* feat(sdk): add workspace-scoped client and workspace CRUD

Add WorkspaceScopedClient modeled after kube::Api::namespaced — captures
workspace once and injects it into every sandbox request. Add workspace
CRUD methods (create, get, list, delete) and list_sandboxes_all_workspaces
on OpenShellClient. Extend SandboxRef with workspace field and add
WorkspaceRef type. Include mock tests for all new operations.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(lint): resolve clippy warnings in workspace test assertions

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(docs): convert indented code blocks to fenced in RFC 0011

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(lint): resolve clippy warnings and apply cargo fmt across workspace

Auto-format with cargo fmt and fix clippy warnings exposed by the
reformat: unnecessary qualifications, map_unwrap_or, identical match
arms, unused variable prefix, dead code annotations, and let-unit-value
in e2e harness.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(workspace): address workspace scoping issues from review

- Add workspace field to settings JSON output (CLI)
- Skip Podman containers missing workspace label instead of defaulting
  to empty string, matching K8s driver behavior
- Add resource_version to list_by_scope SELECT in both SQLite and
  Postgres backends, with regression test
- Gate PolicyLocalContext proposal/lookup routes on workspace readiness,
  returning 503 when workspace is not yet discovered
- Block sandbox and provider creation in TUI all-workspaces mode
- Clear workspace vectors in TUI reset_sandbox_state

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(workspace): make provider profile catalog workspace-aware

Thread workspace through snapshot_catalog so the
EffectiveProviderProfileCatalog enforces workspace boundaries on both
read and write paths. UserProviderProfileSource now loads platform-scoped
profiles (workspace "") plus the target workspace's profiles, preventing
cross-workspace duplicate profile ID collisions that previously caused
global catalog failures.

Update RFC 0011 to reflect catalog scoping is implemented in Phase 1
rather than deferred to future work.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(persistence): include workspace column in atomic policy revision INSERT

put_policy_revision_atomic omitted the workspace column from the INSERT
into the objects table in both SQLite and Postgres backends, causing
atomically-written policy revisions to lose their workspace association.
Add workspace field to AtomicPolicyRevisionWrite and thread it through
both backend INSERT statements, matching the non-atomic put_policy_revision
path which already included it.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(proxy): skip ancestor walk when socket owner is the entrypoint

collect_ancestor_identities walked the entire process tree above the
entrypoint when the connecting process was the entrypoint itself,
SHA256-hashing every ancestor binary (IDE, shell, container runtime).
On dev machines with large binaries in the ancestor chain this exceeded
the 30-second test timeout. When start_pid == stop_pid there are no
intermediate ancestors to verify, so return an empty list immediately.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(workspace): make provider profile catalog scope-aware

Allow the same profile ID at platform and workspace scopes by
introducing layered catalog entries where workspace profiles shadow
platform profiles. Add source and scope fields to the ProviderProfile
proto and CLI output. Migrate List/Get handlers to the catalog,
fixing divergence with runtime profile resolution.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(e2e): align podman e2e labels with centralized driver constants

The podman driver moved its container labels to the centralized
openshell.ai/ prefix, but the e2e test harness and cleanup script
still referenced the old openshell.sandbox-* keys, causing the
local_driver_token_restart test to fail on container lookup.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(e2e): align python profile isolation test with scope-aware catalog

Platform profiles are now visible in workspace listings as fallbacks
per the layered catalog design. Update the assertion to match.

Signed-off-by: Derek Carr <decarr@redhat.com>

* fix(workspace): honor profile_workspace in runtime profile resolution

Runtime profile lookups now consult provider.profile_workspace via
get_type_profile_for_scope. Providers created with --global-profile
(profile_workspace="") resolve to the platform profile even when a
workspace profile shadows the same ID. All 6 runtime call sites
updated; type-only call sites remain scope-agnostic.

Signed-off-by: Derek Carr <decarr@redhat.com>

---------

Signed-off-by: Derek Carr <decarr@redhat.com>
2026-07-21 00:59:18 +00:00
Mesut Oezdil 8cf2673c0b docs: bump stated Rust MSRV from 1.88 to 1.90 (#2276)
* docs: bump stated Rust MSRV from 1.88 to 1.90

Cargo.toml sets rust-version = "1.90" (rust-toolchain.toml pins
1.95.0), so building with the previously documented 1.88 fails
Cargo's MSRV check.

* docs: bump e2e/rust MSRV to 1.90

* fix: align remaining Rust version fields to 1.90

examples/governance-interceptor/Cargo.toml still had rust-version
1.88. Also bump e2e/rust's prost dependency to 0.14 to match the
workspace, since it was on 0.13 in an otherwise standalone crate.
2026-07-17 12:28:23 -07:00
Drew Newberry 392ad63941 fix(driver-vm): run sandbox supervisor as guest pid 1 (#2299)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-07-15 12:24:51 -07:00
Evan Lezar a226806026 test(e2e): run gpu workloads from manifest (#1709)
* test(e2e): add workload manifest build flow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): add gpu workload validation tests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(e2e): build gpu workloads before gpu e2e

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-30 09:54:12 +02:00
krishicksandddurst 7bce1223dc feat(policy): add JSON-RPC and MCP L7 policies (#1865)
Add policy schema, proto, provider profile, OPA, and L7 proxy support for
`protocol: json-rpc` and `protocol: mcp`. Generic JSON-RPC endpoints match
exact method names only, with `method: "*"` as the all-method sentinel;
wildcard/glob methods and params matchers are rejected.

Parse JSON-RPC request bodies and batches in the forward proxy, deny
response-shaped client frames, limit receive-stream GET allowance to MCP
endpoints, and redact params in decision logs. Preserve L7 rule params on the
proto load path so MCP `tools/call` tool filters behave like YAML-loaded
policies.

Add MCP conformance coverage, JSON-RPC L7 e2e coverage, and docs for the new
protocols and current matcher limitations.

Signed-off-by: Kris Hicks <khicks@nvidia.com>
Co-authored-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
2026-06-26 15:52:16 -07:00
Drew Newberry 530aaf1360 feat(drivers): support docker and podman config mounts (#1785)
* feat(drivers): support docker and podman config mounts

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(drivers): trim mount docs

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* test(e2e): cover local driver volume mounts

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(podman): satisfy linux clippy lint

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(drivers): gate bind mounts behind gateway config

* docs(sandbox): simplify mount examples

* cleanup

* test(e2e): stabilize branch checks

* fix(drivers): tighten local mount validation

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-06-10 11:46:20 -07:00
Taylor Mutch e4bcfdfaaa fix(gateway): allow local sandbox jwt to not expire (#1721) 2026-06-03 13:50:56 -05:00
alangou 2bdc968ed9 fix(gateway): make readiness health checks dependency-aware (#1328)
* feat(gateway): add readiness probe metrics and test-only store close

Emit Prometheus readiness metrics for database probes (healthy gauge and
outcome-labeled latency histogram) with coverage in health HTTP tests.
Restrict Store::close behind test support cfg to prevent accidental runtime
pool shutdown under live traffic.

Signed-off-by: Adrien Langou <alangou@nvidia.com>

* test(e2e): add simple e2e test with kubernetes to test /readyz

Signed-off-by: Adrien Langou <alangou@nvidia.com>

---------

Signed-off-by: Adrien Langou <alangou@nvidia.com>
2026-05-27 11:00:06 -07:00
Russell Bryant 65a3a7c2ec test(e2e): close Podman driver test coverage gaps (#1439)
* test(e2e): close Podman driver test coverage gaps

Add podman_gateway_resume test following the VM pattern (no
container-state assertions since Podman keeps containers running across
gateway restarts). Widen websocket_conformance feature gate from
e2e-docker to e2e-host-gateway so it runs on both Docker and Podman.
Fix e2e-podman.sh to default to the e2e-podman feature so Podman-
specific and host-gateway tests are actually included in Podman CI runs.

* refactor(e2e): extract shared CLI helpers from gateway_resume tests

Move run_cli, wait_for_healthy, sandbox_names, and
wait_for_sandbox_exec_contains into a new harness::cli module. All three
gateway_resume variants (Docker, Podman, VM) now import these shared
helpers instead of defining identical copies. Docker-specific container
introspection functions remain local to gateway_resume.rs.
2026-05-18 17:09:16 -07:00
Drew Newberry f819f7dcb5 fix(vm): restore sandboxes after gateway restart (#1407) 2026-05-15 15:08:59 -07:00
Aaron Erickson 🦞andJohn Myers 9ea94b645d fix(sandbox): rewrite messaging credential placeholders (#1286)
* fix(sandbox): rewrite credential placeholders in websocket text frames

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): harden websocket credential rewrite

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* feat(sandbox): add websocket l7 inspection and compression

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): harden websocket upgrade validation

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* test(sandbox): cover route-selected websocket upgrades

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): harden websocket negotiation parsing

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* test(sandbox): add websocket conformance relay matrix

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* test(e2e): add websocket conformance lane

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(policy): support websocket incremental rules

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* feat(policy): enable websocket credential rewrite updates

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(cli): make websocket rewrite endpoint-local

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* feat(sandbox): support graphql websocket policy

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(policy): allow private IPs for websocket endpoints

* feat(sandbox): rewrite REST credential placeholders

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* refactor(sandbox): generalize credential aliases

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): rewrite encoded form credentials

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(sandbox): close websocket policy and provider alias gaps

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

* fix(e2e): route websocket probe through host gateway

* fix(e2e): stabilize websocket probe handshake

* fix(e2e): exercise websocket probe through proxy

* ci: remove websocket conformance workflow

---------

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
2026-05-11 22:40:44 -07:00
Taylor Mutch 59475aabf4 ci(kubernetes): add kube gateway e2e tests and gated CI workflow (#1251)
* test(helm): Add kube gateway e2e tests

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* ci(helm): add Branch Helm E2E workflow gated on test:e2e-helm

Adds a label-gated GitHub Actions workflow that exercises the Helm
chart end-to-end against the Rust e2e suite via `mise run e2e:helm`.

Pipeline:
- pr_metadata gates on the `test:e2e-helm` label via the pr-gate action.
- build-gateway / build-supervisor build and push Docker images using
  the reusable docker-build.yml workflow.
- helm-e2e (bare runner): apt-installs z3 build deps so cargo can
  compile the openshell-policy crate's z3-sys backend, creates a kind
  cluster via helm/kind-action, materializes the kind kubeconfig at the
  path mise's [env] block expects, side-loads the freshly built
  gateway/supervisor images, applies
  deploy/kube/manifests/agent-sandbox.yaml so the
  sandboxes.agents.x-k8s.io CRD and reconciling StatefulSet are in
  place, and finally runs `mise run e2e:helm`.

Also expands the `e2e:helm` task to run the full Rust e2e suite
(matching `e2e:podman`) instead of only the smoke test, with
OPENSHELL_E2E_KUBE_TEST as an opt-in single-test override for local
debugging.

Extends the e2e-label-help workflow so applying `test:e2e-helm` posts
the next-step hint pointing at this workflow.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* test(helm-e2e): run host-gateway tests on the kube driver

Six tests previously skipped on the kubernetes driver — three in
host_gateway_alias.rs plus the forward_proxy_l7 + graphql_l7 cases —
relied on `host.openshell.internal` reaching either a host process or a
sibling docker container. The Helm chart already supports this via
`server.hostGatewayIP`, but the e2e wrapper never set it.

- with-kube-gateway.sh: auto-detect the host-routable IP (CoreDNS
  `host.k3d.internal` first to handle Docker Desktop, docker network
  gateway as a fallback for kind on Linux CI) and pass it to helm
  install. Also import locally-built images into existing k3d clusters
  and wait for namespace deletion to complete before exit.
- e2e/rust/Cargo.toml + e2e-helm.sh: add an `e2e-host-gateway` feature
  that gates the three test files; docker and podman runs imply it,
  helm runs opt in by default (overridable via
  OPENSHELL_E2E_HELM_FEATURES for remote clusters where the test host
  is unreachable from pods).
- Drop the `skip_if_kube` helper and its callers — feature gating now
  decides whether the tests compile in.

Verified against a local k3d cluster: all six previously-skipped tests
pass, smoke regression intact.

* refactor(e2e): rename e2e:helm to e2e:kubernetes

The runner exercises the gateway on Kubernetes; helm is just the
deployment mechanism. Names now describe the target environment.

- mise task: `e2e:helm` -> `e2e:kubernetes`
- script: e2e/rust/e2e-helm.sh -> e2e/rust/e2e-kubernetes.sh
- env var: OPENSHELL_E2E_HELM_FEATURES -> OPENSHELL_E2E_KUBERNETES_FEATURES
- workflow: branch-helm-e2e.yml -> branch-kubernetes-e2e.yml
  (display name "Branch Kubernetes E2E", job "kubernetes-e2e")
- PR gate label: test:e2e-helm -> test:e2e-kubernetes (e2e-label-help
  hint workflow updated to match)

PR #1251 needs the new label applied; the old label can be removed.

* fix(e2e): prefer IPv4 host-gateway IP from dual-stack docker networks

kind dual-stacks its network; the wrapper's awk picked the first
non-empty gateway, which was the IPv6 entry (fc00:...). Sandbox pods
can't reach the test host's IPv4 listener through that, so the L7 and
host-gateway-alias tests failed in CI even though they passed locally
against k3d (where host.k3d.internal in CoreDNS short-circuits the
docker-network fallback).

Restrict the awk filter to IPv4 octets.

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
2026-05-11 12:57:23 -07:00
Evan Lezar 5c98604f0f feat(gpu): honor device IDs in Docker and Podman (#1253)
* feat(gpu): honor device IDs in Docker and Podman

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(gpu): add Docker and Podman device selection e2e

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(gpu): run Docker GPU e2e workflow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-05-11 17:09:37 +02:00
Drew Newberry b8e87431a5 fix(e2e): isolate kubernetes user namespace test (#1276) 2026-05-08 14:46:57 -07:00
Drew Newberry 23ad858a4f test(e2e): add podman rust suite (#1185)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-05-06 12:25:49 -07:00
Drew Newberry 984d1a6e5c chore: rename project from NemoClaw to OpenShell (#198) 2026-03-10 11:49:09 -07:00
Drew Newberry a2de1f24e5 feat: add Cloudflare tunnel auth support (#178) 2026-03-09 18:39:42 -07:00
Drew Newberry a8e9b43dcc refactor(e2e): replace bash e2e tests with Rust integration tests (#150) 2026-03-06 09:01:49 -08:00