Matthew Grossman fac82327b3 test(testkit): add the testkit library for end-to-end suites (#4724)
* test(testkit): add the testkit workspace for end-to-end suites

Add tests/testkit as its own Cargo workspace: testkit, testkit-macros,
and testkit-spec. testkit reads a target file, connects to the gateway
through openshell-sdk, creates and cleans up sandboxes, and runs the
installed CLI. #[testkit::test] requires a stable ID and a doc comment and
declares capability requirements. testkit-ready waits for a target's
gateway and CLI before tests run.

Nothing uses testkit yet. The lockfile is seeded from the root lockfile
so shared crates resolve to the same versions.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* ci(tests): format, lint and test testkit in Branch Checks

Nothing else builds testkit, so check it in Branch Checks and the
matching mise tasks to catch openshell-sdk changes that break it.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* docs(tests): describe how to write and run testkit tests

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(testkit): give each sandbox its own cleanup timeout

Cleanup shared one two-minute budget across all of a test's sandboxes, so
a slow delete left the rest with no time and could leak them on a shared
gateway. Delete them concurrently, each with the full timeout.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* docs(testkit): show how to exclude tests by capability

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* docs: list tests/testkit in the project structure tables

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* refactor(testkit): drop test IDs and capabilities for now

Nothing reads test IDs or capability requirements until a runner, waivers,
or result history exist, so remove the id and requires arguments and the
target file's capabilities section. Both can return as optional attribute
arguments without changing existing tests.

With them gone, testkit-spec only checked the function shape and read the
doc comment, so fold it into testkit-macros.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* docs(testkit): describe what testkit provides today

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* refactor(testkit): find the gateway the way the CLI does

Replace the target file (OPENSHELL_TEST_TARGET) with the CLI's own gateway
selection: OPENSHELL_GATEWAY_ENDPOINT, then OPENSHELL_GATEWAY, then the
active gateway. A registered gateway's endpoint, auth mode, and mTLS
material come from its CLI config directory. cx.cli runs OPENSHELL_BIN
against the same gateway and workspace as the SDK.

Suites now run unchanged under e2e/with-docker-gateway.sh and
e2e/with-podman-gateway.sh, and environments need no extra file.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(testkit): keep labels when creating a sandbox with a policy

The policy path creates the sandbox through gRPC and dropped the spec's
labels without an error. Pass them in the create request.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(testkit): use the registered gateway's credentials for an endpoint

When OPENSHELL_GATEWAY_ENDPOINT is set, the CLI takes credentials from the
gateway named by OPENSHELL_GATEWAY, or else the registered gateway with
that endpoint. testkit connected without a client certificate instead, so
an mTLS endpoint such as the one e2e/rust/e2e-vm.sh exports worked for
cx.cli but not for the SDK. Resolve it the way the CLI does, and pass
both the name and the endpoint to cx.cli.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* docs(tests): list testkit's gateway variables in precedence order

OPENSHELL_GATEWAY_ENDPOINT takes precedence over OPENSHELL_GATEWAY. Also
mention OPENSHELL_BIN.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(testkit): apply other attributes to the generated test

#[testkit::test] left attributes such as #[ignore] and #[cfg] on the
nested body function, where they have no effect on the test. Move them
to the generated test, and reject a second test attribute such as

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
#[test] or #[tokio::test].

* fix(testkit): show usage for testkit-ready --help

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(testkit): cover cleanup after pass, failure, and panic

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(testkit): reject #[should_panic] on testkit tests

A testkit test panics on any failure, including an unreachable gateway or
a failed cleanup, so #[should_panic] would let such a failure pass. Reject
it and point to checking the expected failure in the body.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(testkit): keep only the macro's misuse checks as unit tests

Suites built on testkit exercise gateway resolution, the CLI wrapper,
polling, errors, and cleanup against a real gateway. Keep one unit test
for the macro's compile errors, which a passing suite never reaches, and
drop the test-only seams the other tests needed.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* refactor(testkit): connect through the SDK's mTLS support

The SDK's ClientConfig now takes a client identity, so testkit no longer builds
its own channel. It still resolves the registered gateway and reads its
credential files, then hands the CA and client certificate to
OpenShellClient::connect. That keeps TLS behavior, connect timeouts and
keep-alive settings in the SDK, and drops testkit's direct tonic dependency.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(testkit): bound readiness and cleanup with timeouts

wait_ready checked elapsed time only after a health call returned, and cleanup
started its timeout only after the delete returned, so a gateway that accepts a
connection but never answers could hang either one. Wrap each whole operation in
tokio::time::timeout: the readiness wait, including every connect and health
call, and each sandbox's delete plus wait for deletion. A test covers a peer
that accepts connections and never replies.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

---------

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
2026-10-22 17:11:31 +00:00

OpenShell

License PyPI Security Policy Documentation

Important

New in OpenShell 0.1.x: a stable release cadence, new isolation primitives, an expanded extension surface, and new APIs. Read the 0.1.0 upgrade guide.

OpenShell is the safe, private runtime for fleets of autonomous AI agents. Agents are most useful when they can read files, install packages, call APIs, and use credentials. OpenShell gives them that capability without giving them unrestricted access to your data, secrets, or network. You declare what each agent can touch in a policy, and OpenShell enforces it.

How It Works

OpenShell governs what agents can do in two ways: it instruments the kernel to enforce policy on every file access, system call, and network connection at runtime, and it uses formal verification to check what a policy change would allow before it is applied.

  • Kernel-level enforcement. Each agent runs in an isolated sandbox. Kernel controls confine which files it can access and which system calls it can make, and every network connection passes through a policy check before it leaves the sandbox. Agents never see real credentials; OpenShell adds them only to requests bound for approved endpoints.
  • Formally verified policy changes. Before a policy change is approved, OpenShell uses formal verification to flag risky new access it would grant, such as reaching a new host with credentials or calling a new API method, so those changes wait for human review.

See Architecture for how the gateway, supervisor, and sandbox fit together.

Quickstart

You need Linux, macOS on Apple Silicon, or Windows with WSL 2 (experimental), plus Docker, Podman, or host virtualization. See the Support Matrix for details.

curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh
openshell sandbox create --name demo

The installer sets up the CLI and a local gateway. The default sandbox image is minimal Ubuntu with no agent installed. To run a real agent, follow Run Your First Agent: it runs OpenCode against a free OpenRouter model and shows how to approve new access as the agent needs it.

Explore Further

Agent Skills

Install the public OpenShell skills for your coding agent:

npx skills add NVIDIA/OpenShell

The skills teach your agent to drive the OpenShell CLI, write sandbox policies, and debug gateways and inference routing. They live in skills/ and work without an OpenShell source checkout.

SDKs

SDKs connect applications to an OpenShell gateway. They do not install the CLI. Use the same OpenShell release for the SDK and the gateway when possible.

Language Install Docs
Python uv add openshell README
TypeScript npm install @nvidia/openshell-sdk (GitHub Packages) README
Go go get github.com/NVIDIA/OpenShell/sdk/go@latest README
Rust cargo add openshell-sdk --git https://github.com/NVIDIA/OpenShell --tag <release-tag> Installation and usage

Community

OpenShell is built agent-first: it is developed with the same agent-driven workflows it enables. See CONTRIBUTING.md for development setup and the contribution workflow, and AGENTS.md for repository coding conventions.

Telemetry

OpenShell collects anonymous telemetry, limited to operational categories and counts, to help improve the project. It does not collect sandbox names, hostnames, file paths, prompts, credentials, provider or model names, or user content. To disable it, set OPENSHELL_TELEMETRY_ENABLED=false on the gateway, or server.telemetryEnabled=false for Helm installs. You can also compile telemetry out entirely. See Telemetry for details and the community telemetry reports for published usage trends.

Notice and Disclaimer

This software automatically retrieves, accesses or interacts with external materials. Those retrieved materials are not distributed with this software and are governed solely by separate terms, conditions and licenses. You are solely responsible for finding, reviewing and complying with all applicable terms, conditions, and licenses, and for verifying the security, integrity and suitability of any retrieved materials for your specific use case. This software is provided "AS IS", without warranty of any kind. The author makes no representations or warranties regarding any retrieved materials, and assumes no liability for any losses, damages, liabilities or legal consequences from your use or inability to use this software or any retrieved materials. Use this software and the retrieved materials at your own risk.

License

This project is licensed under the Apache License 2.0.

S
Description
GitHub Trending: NVIDIA/OpenShell
Readme Apache-2.0
249 MiB
Languages
Rust 87.6%
Go 4.5%
Shell 3.2%
Python 2.5%
TypeScript 0.9%
Other 1%