Matthew GrossmanandEvan Lezar
33138516ad
* feat(tmachine): add the Session API and a test-runner package
Add a library API that boots an environment, runs commands in it with timeouts,
copies files in and out, collects diagnostics and tears it down, so a caller can
drive a machine without writing an Ansible playbook per suite. SSH and exec
timeouts and QEMU cleanup on SIGTERM keep a hung guest or a cancelled CI job from
leaving QEMU running.
Add the `openshell-test-runner` flake package, which runs tests/runner with the
toolchains and VM tools it needs.
The Session API and test-runner package are adapted from NVIDIA/OpenShell#4629.
Co-authored-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* build(cargo): add a check that aligned workspaces lock shared crates like the root
`cargo --locked` only checks a lockfile against its own manifest. A workspace
outside the root that builds root crates through path dependencies, such as the
testkit suites building `openshell-sdk`, has its own Cargo.lock, so the two
lockfiles can resolve a shared crate to different versions without any check
failing. That is how aws-lc-sys 0.46 was picked up in #4629, which does not link
against the AWS-LC that Nix provides.
check_cargo_lockfile_alignment.py fails when an aligned workspace locks a
registry crate that the root lockfile also contains at a version the root
lockfile does not have. With --fix it re-locks the workspace starting from the
root lockfile. tests/testkit is the first aligned workspace.
The script and its tests come from #4629 (a37ebe917).
Co-authored-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* test(conformance): add a sandbox smoke suite built on testkit
Add the `sandbox` crate to the conformance workspace with one testkit test that
creates a sandbox, runs `echo` in it with exec, deletes it and checks it is gone.
It is deliberately small: it exists so the runner can run a real test in a
tmachine environment on CI before the existing suites move to testkit.
The workspace's other crate, the CLI conformance suite, keeps running as it does
today. The conformance workspace joins the lockfile alignment check, so its
lockfile is realigned with the root lockfile.
The test is adapted from the `sandbox-lifecycle` suite in #4629.
Co-authored-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* feat(runner): add a test runner that runs a test family on tmachine
`openshell-test-runner` builds a family's packages (a workspace under
tests/suites/) on the host as a nextest archive for the guest's musl target,
boots one tmachine environment by name with the named installer through the
Session API, runs nextest in it and brings back the JUnit report and log. No
per-suite Ansible playbook is involved:
nix run .#openshell-test-runner -- --environment ubuntu-docker-rootful \
--installer deb --family conformance --package sandbox
The runner points the tests at the system under test with environment
variables: OPENSHELL_GATEWAY (the gateway the installer registered, `--gateway`,
default `openshell`) and OPENSHELL_BIN (the installed CLI). It waits for the
gateway with `testkit-ready` before the tests start. Nextest runs with
`--test-threads 1` and a slow-timeout that terminates a hung test.
A run passes if nextest succeeds and its JUnit report shows at least one test
and no failures. `--filter` narrows the tests with a nextest filterset, and
`--keep` leaves the VM up for debugging. Results from an earlier run are
removed first, and the guest's diagnostics are collected after a failure.
Which tests run in which environments is left to the caller for now; there are
no plans, environment registry or per-test requirements, and a broken test is
`#[ignore]`d.
Most of this comes from #4629, adapted to testkit finding the gateway the way the
CLI does instead of reading a target file.
Co-authored-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* ci(tests): lint and test the runner, tmachine and conformance suite
Run fmt, clippy and nextest for tests/tmachine and tests/runner, and fmt and
clippy for the new `sandbox` suite crate, wherever tests/testkit already runs
them: the mise tasks and Branch Checks. The suite's tests need a gateway, so they
run through `mise run e2e:testkit` against a local Docker gateway and, on CI,
through the test runner rather than the unit-test job.
The lockfile task and the Cargo lockfiles job also run the alignment check, and
its own tests run with `mise run test`.
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* ci(tests): run testkit suites on tmachine from a reusable E2E Suites workflow
Add a reusable workflow with an explicit matrix: each row names a tmachine
environment and installer, a family and package, and the runner label. Each
row runs `nix run .#openshell-test-runner` once. The jobs use the same
integration inputs as the existing tmachine lanes, cache the tmachine disks
keyed on the Ansible tree, upload the JUnit report and logs whether or not the
run passes, and an aggregator job reports the overall result.
The only row runs the conformance sandbox suite on amd64 Ubuntu with rootful
Docker and the Debian package, which CI can provision today. Branch E2E calls
the workflow when the test:e2e label is set. The job is not in the Core E2E
result's needs yet, so it starts out advisory. The existing conformance,
feature and driver lanes are unchanged.
Modeled on the E2E Suites workflow in #4629.
Co-authored-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* docs(tests): document the runner and the conformance suite
Describe how the runner runs a family in a tmachine environment, locally and
on CI, in TESTING.md, add a README for the sandbox suite, point the testkit
README at the runner, and add rows for tests/suites, tests/tmachine and
tests/runner to the project-structure tables in AGENTS.md and CONTRIBUTING.md.
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* refactor(tests): simplify the test runner and tmachine
- Replace tmachine's global QEMU PID registry with until_signal, which
drops the work on SIGTERM/SIGINT/SIGHUP so kill_on_drop stops QEMU.
This also lets Ctrl-C end a --keep run.
- Decide a run's status from nextest's exit code (with --no-tests fail)
instead of parsing JUnit; print the end of the nextest log on failure.
- Move gateway diagnostics from tmachine's Session into the runner, drop
Session::run_playbooks, and make Session own its VM directly.
- Read TMACHINE_SSH_PORT once and reject invalid values; drop the racy
port pre-check.
- Build the runner with Nix so tmachine's firmware paths are compiled in,
and set ANSIBLE_CONFIG only in the wrapper.
- Copy the bundle directory in directly, fold lib.rs into main.rs, and
make the gateway name required.
- Drop the semver hint from the lockfile alignment check and trim
duplicated docs.
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* fix(tmachine): keep the SSH port check and name a run with no tests
Restore the check that the SSH forward port is free before QEMU starts. QEMU
reports a taken port only by exiting, so a layer build would otherwise run its
playbooks against the VM that already holds the port and cache the result.
Report nextest's no-tests exit code as "no test ran" instead of a bare exit
status.
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* fix(runner): keep logs from a timed-out run and gate on the E2E suites
nextest now writes its output to a file in the guest, and the runner copies
nextest.log and junit.xml out after every run, so a run that times out or
loses SSH still leaves a log. A failed JUnit copy no longer turns a test
failure into an error.
The runner exits with 1 when a test failed and 2 when the run could not
complete, and annotates the GitHub Actions run with which it was.
The E2E suites lane is now part of the Core E2E result. Its disk cache key
also covers tmachine's cache versions in setup.rs and install.rs.
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
* ci(e2e): push test runner builds to Cachix
E2E Suites set up Nix without a Cachix token, so every job rebuilt
openshell-test-runner from source. Accept an optional CACHIX_AUTH_TOKEN
secret, pass it to setup-nix, and pass it from Branch E2E, as Branch Checks
and the binary builds already do.
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
---------
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
Important
New in OpenShell 0.1.x: a stable release cadence, new isolation primitives, an expanded extension surface, and new APIs. Read the 0.1.0 upgrade guide.
OpenShell is the safe, private runtime for fleets of autonomous AI agents. Agents are most useful when they can read files, install packages, call APIs, and use credentials. OpenShell gives them that capability without giving them unrestricted access to your data, secrets, or network. You declare what each agent can touch in a policy, and OpenShell enforces it.
How It Works
OpenShell governs what agents can do in two ways: it instruments the kernel to enforce policy on every file access, system call, and network connection at runtime, and it uses formal verification to check what a policy change would allow before it is applied.
- Kernel-level enforcement. Each agent runs in an isolated sandbox. Kernel controls confine which files it can access and which system calls it can make, and every network connection passes through a policy check before it leaves the sandbox. Agents never see real credentials; OpenShell adds them only to requests bound for approved endpoints.
- Formally verified policy changes. Before a policy change is approved, OpenShell uses formal verification to flag risky new access it would grant, such as reaching a new host with credentials or calling a new API method, so those changes wait for human review.
See Architecture for how the gateway, supervisor, and sandbox fit together.
Quickstart
You need Linux, macOS on Apple Silicon, or Windows with WSL 2 (experimental), plus Docker, Podman, or host virtualization. See the Support Matrix for details.
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh
openshell sandbox create --name demo
The installer sets up the CLI and a local gateway. The default sandbox image is minimal Ubuntu with no agent installed. To run a real agent, follow Run Your First Agent: it runs OpenCode against a free OpenRouter model and shows how to approve new access as the agent needs it.
For native Windows support through Microsoft MXC, see the Windows build and runtime guide for setup and current limitations.
Explore Further
- Sandboxes: images, runtimes, GPUs, and lifecycle.
- Policies: filesystem, network, and process rules, with the advisor and prover for reviewing changes.
- Providers: credentials that work only at approved endpoints, including inference.
- Gateways: the control plane for sandboxes, policy, and access.
- Kubernetes: deploy the gateway with Helm. Your CNI must enforce
NetworkPolicy. - Extensibility: middleware, interceptors, and compute drivers.
- Tutorials: step-by-step policy and agent walkthroughs.
- Prerelease and development builds: try an upcoming release or the latest commit on
main.
Agent Skills
Install the public OpenShell skills for your coding agent:
npx skills add NVIDIA/OpenShell
The skills teach your agent to drive the OpenShell CLI, write sandbox policies, and debug gateways and inference routing. They live in skills/ and work without an OpenShell source checkout.
SDKs
SDKs connect applications to an OpenShell gateway. They do not install the CLI. Use the same OpenShell release for the SDK and the gateway when possible.
| Language | Install | Docs |
|---|---|---|
| Python | uv add openshell |
README |
| TypeScript | npm install @nvidia/openshell-sdk (GitHub Packages) |
README |
| Go | go get github.com/NVIDIA/OpenShell/sdk/go@latest |
README |
| Rust | cargo add openshell-sdk --git https://github.com/NVIDIA/OpenShell --tag <release-tag> |
Installation and usage |
Community
- Questions and discussion: GitHub Discussions
- Bug reports and feature requests: GitHub Issues, using the issue templates
- Security vulnerabilities: follow SECURITY.md. Do not open a GitHub issue.
- Roadmap: OpenShell Roadmap and the RFC board
- Try it in the cloud: Brev Launchable
OpenShell is built agent-first: it is developed with the same agent-driven workflows it enables. See CONTRIBUTING.md for development setup and the contribution workflow, and AGENTS.md for repository coding conventions.
Telemetry
OpenShell collects anonymous telemetry, limited to operational categories and counts, to help improve the project. It does not collect sandbox names, hostnames, file paths, prompts, credentials, provider or model names, or user content. To disable it, set OPENSHELL_TELEMETRY_ENABLED=false on the gateway, or server.telemetryEnabled=false for Helm installs. You can also compile telemetry out entirely. See Telemetry for details and the community telemetry reports for published usage trends.
Notice and Disclaimer
This software automatically retrieves, accesses or interacts with external materials. Those retrieved materials are not distributed with this software and are governed solely by separate terms, conditions and licenses. You are solely responsible for finding, reviewing and complying with all applicable terms, conditions, and licenses, and for verifying the security, integrity and suitability of any retrieved materials for your specific use case. This software is provided "AS IS", without warranty of any kind. The author makes no representations or warranties regarding any retrieved materials, and assumes no liability for any losses, damages, liabilities or legal consequences from your use or inability to use this software or any retrieved materials. Use this software and the retrieved materials at your own risk.
License
This project is licensed under the Apache License 2.0.