Shiju
25abc9e3c7
* feat(inference): allow local embeddings route Route OpenAI-compatible embeddings through the local inference proxy so sandboxed vector workloads reach a configured provider via the same route classification and auth path that chat, completion, and model discovery already use. - Add openai_embeddings to the OpenAI-compatible protocol set so providers (openai, nvidia) advertise embeddings routing. - Classify POST /v1/embeddings as the openai_embeddings protocol in the sandbox L7 patterns. - Serve embeddings buffered with an accurate Content-Length, since the response is a single JSON object rather than an SSE token stream. The streaming path appends an SSE error frame on a size-cap or idle-timeout truncation, which would corrupt a one-object body the client parses whole. protocol_returns_buffered_body() selects the path. - Probe an embeddings-only backend against /v1/embeddings during validation, after the chat and completion protocols so a multi-protocol route still prefers those. - Extract two shared helpers. http_status_text() backs both response formatters and adds 401/422/429/503 for embeddings passthrough and router error mapping; write_inference_router_error() backs the streaming and buffered routing paths. - Return an OpenAI-shaped embeddings body from the mock route. Tests cover profile lookup, L7 pattern detection, the mock body, and buffered Content-Length framing with no chunked transfer-encoding and no SSE error frame. Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): cap buffered inference response body The buffered proxy path read the whole upstream response into memory with no size bound. The route timeout bounds elapsed time but not memory, so a misbehaving or oversized upstream could force unbounded allocation in the sandbox proxy. The streaming path already caps each response at 32 MiB; the buffered path did not. Cap the buffered read at the same 32 MiB. An advertised over-cap body is rejected from its Content-Length before any bytes are read, and chunks accumulate under the same bound so a chunked or mislabeled body cannot slip past. An over-cap response fails as an upstream protocol error, surfaced as HTTP 502 at the proxy boundary, and is never partially returned. Tests - cargo test -p openshell-router \ proxy_to_backend_rejects_over_cap_response_body Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): validate embeddings models against all advertised protocols A managed route resolves to its provider profile's full protocol set, so an embeddings model such as text-embedding-3-small lists openai_chat_completions alongside openai_embeddings. Route verification probed only the first writable protocol and stopped on its failure. It sent a chat probe with the embedding model, the provider rejected it as wrong-shape, and the route failed validation before the embeddings probe ran. Embeddings-only configs could not be verified. Try the advertised protocols in preference order. A request-shape rejection (HTTP 400, 404, 405, 422) falls through to the next protocol, so an embeddings model validates against /v1/embeddings even when the chat probe rejects it. Credential, rate-limit, connectivity, and upstream-health failures stay terminal and stop validation at the first probe, so a bad key or a down backend is reported as itself rather than masked by a later probe. validation_probe becomes validation_probes, which returns the ordered list, and the per-probe fallback retry (max_completion_tokens versus max_tokens) moves into a shared helper. Tests - cargo test -p openshell-router \ verify_embeddings_model_falls_through_chat_probe - cargo test -p openshell-router verify_stops_on_credentials_failure Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): serve model discovery responses buffered GET /v1/models returns a single JSON model list, the same response shape as embeddings. The sandbox inference proxy was routing it through the SSE streaming path. A streaming size-cap or idle-timeout truncation appends an SSE error frame to the body, which corrupts a payload the client parses as one JSON object. Make response framing a property of the protocol. A new ResponseFraming field on InferenceApiPattern is set once per pattern in default_patterns. model_discovery and openai_embeddings are now Buffered, while chat completions, completions, responses, and Anthropic messages stay Streaming. The proxy dispatch gates on pattern.is_buffered(), which replaces the stringly-typed protocol_returns_buffered_body predicate so the streaming-versus-buffered decision lives in one place and cannot drift across the sites that read it. Model discovery now flows through the same buffered path as embeddings, framed with an accurate Content-Length and bounded by the buffered-read size cap that path already enforces. Tests - cargo test -p openshell-sandbox protocol_framing_classification - cargo test -p openshell-sandbox \ inference_model_discovery_served_buffered_with_content_length Signed-off-by: Shiju <shiju@nvidia.com> * docs(inference): document embeddings route in supported patterns Signed-off-by: Shiju <shiju@nvidia.com> --------- Signed-off-by: Shiju <shiju@nvidia.com>
NVIDIA OpenShell
OpenShell is the safe, private runtime for autonomous AI agents. It provides sandboxed execution environments that protect your data, credentials, and infrastructure — governed by declarative YAML policies that prevent unauthorized file access, data exfiltration, and uncontrolled network activity.
OpenShell is built agent-first. The project ships with agent skills for everything from gateway troubleshooting to policy generation, and we expect contributors to use them.
Alpha software — single-player mode. OpenShell is proof-of-life: one developer, one environment, one gateway. We are building toward multi-tenant enterprise deployments, but the starting point is getting your own environment up and running. Expect rough edges. Bring your agent.
Quickstart
Prerequisites
- A supported host — macOS, Windows with WSL 2, or Linux.
- A local runtime — Docker, Podman, or host virtualization enabled for MicroVM-backed sandboxes.
Install
Binary (recommended):
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh
From PyPI (requires uv):
uv tool install -U openshell
Both methods install the latest stable release by default. To install a specific version, set OPENSHELL_VERSION (binary) or pin the version with uv tool install openshell==<version>. A dev release is also available that tracks the latest commit on main.
Helm chart:
Experimental — the Kubernetes deployment path is under active development. Expect rough edges and breaking changes.
Deploy the OpenShell gateway into a Kubernetes cluster from the OCI chart published to GHCR:
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart
See deploy/helm/openshell/README.md for available versions, dev tag conventions, and configuration.
For deploying OpenShell on OpenShift, see deploy/helm/openshell/README.md#install-on-openshift.
Create a sandbox
openshell sandbox create -- claude # or opencode, codex, copilot
The sandbox container includes the following tools by default:
| Category | Tools |
|---|---|
| Agent | claude, opencode, codex, copilot |
| Language | python (3.14), node (22) |
| Developer | gh, git, vim, nano |
| Networking | ping, dig, nslookup, nc, traceroute, netstat |
For more details see https://github.com/NVIDIA/OpenShell-Community/tree/main/sandboxes/base.
See network policy in action
Every sandbox starts with minimal outbound access. You open additional access with a short YAML policy that the proxy enforces at the HTTP method and path level, without restarting anything.
# 1. Create a sandbox (starts with minimal outbound access)
openshell sandbox create
# 2. Inside the sandbox — blocked
sandbox$ curl -sS https://api.github.com/zen
curl: (56) Received HTTP code 403 from proxy after CONNECT
# 3. Back on the host — apply a read-only GitHub API policy
sandbox$ exit
openshell policy set demo --policy examples/sandbox-policy-quickstart/policy.yaml --wait
# 4. Reconnect — GET allowed, POST blocked by L7
openshell sandbox connect demo
sandbox$ curl -sS https://api.github.com/zen
Anything added dilutes everything else.
sandbox$ curl -sS -X POST https://api.github.com/repos/octocat/hello-world/issues -d '{"title":"oops"}'
{"error":"policy_denied","detail":"POST /repos/octocat/hello-world/issues not permitted by policy"}
See the full walkthrough or run the automated demo:
bash examples/sandbox-policy-quickstart/demo.sh
How It Works
OpenShell isolates each sandbox in its own container with policy-enforced egress routing. A lightweight gateway coordinates sandbox lifecycle, and every outbound connection is intercepted by the policy engine, which does one of three things:
- Allows — the destination and binary match a policy block.
- Routes for inference — strips caller credentials, injects backend credentials, and forwards to the managed model.
- Denies — blocks the request and logs it.
| Component | Role |
|---|---|
| Gateway | Control-plane API that coordinates sandbox lifecycle and acts as the auth boundary. |
| Sandbox | Isolated runtime with container supervision and policy-enforced egress routing. |
| Policy Engine | Enforces filesystem, network, and process constraints from application layer down to kernel. |
| Privacy Router | Privacy-aware LLM routing that keeps sensitive context on sandbox compute. |
OpenShell runs a gateway control plane that manages sandbox lifecycle through a configured compute driver. Supported compute platforms include Docker, Podman, MicroVM, and Kubernetes.
Protection Layers
OpenShell applies defense in depth across four policy domains:
| Layer | What it protects | When it applies |
|---|---|---|
| Filesystem | Prevents reads/writes outside allowed paths. | Locked at sandbox creation. |
| Network | Blocks unauthorized outbound connections. | Hot-reloadable at runtime. |
| Process | Blocks privilege escalation and dangerous syscalls. | Locked at sandbox creation. |
| Inference | Reroutes model API calls to controlled backends. | Hot-reloadable at runtime. |
Policies are declarative YAML files. Static sections (filesystem, process) are locked at creation; dynamic sections (network, inference) can be hot-reloaded on a running sandbox with openshell policy set.
Providers
Agents need credentials — API keys, tokens, service accounts. OpenShell manages these as providers: named credential bundles that are injected into sandboxes at creation. The CLI auto-discovers credentials for recognized agents (Claude, Codex, OpenCode, Copilot) from your shell environment, or you can create providers explicitly with openshell provider create. Credentials never leak into the sandbox filesystem; they are injected as environment variables at runtime.
GPU Support (Experimental)
Experimental — GPU passthrough works on supported hosts but is under active development. Expect rough edges and breaking changes.
OpenShell can pass host GPUs into sandboxes for local inference, fine-tuning, or any GPU workload. Add --gpu when creating a sandbox:
openshell sandbox create --gpu --from [gpu-enabled-sandbox] -- claude
Docker-backed GPU sandboxes auto-select CDI when available and otherwise fall back to Docker's NVIDIA GPU request path (--gpus all). GPU intent is also inferred automatically for community images with gpu in the name.
Requirements: NVIDIA drivers and the NVIDIA Container Toolkit must be installed on the host. The sandbox image itself must include the appropriate GPU drivers and libraries for your workload — the default base image does not. See the BYOC example for building a custom sandbox image with GPU support.
Supported Agents
| Agent | Source | Notes |
|---|---|---|
| Claude Code | base |
Works out of the box. Provider uses ANTHROPIC_API_KEY. |
| OpenCode | base |
Works out of the box. Provider uses OPENAI_API_KEY or OPENROUTER_API_KEY. |
| Codex | base |
Works out of the box. Provider uses OPENAI_API_KEY. |
| GitHub Copilot CLI | base |
Works out of the box. Provider uses GITHUB_TOKEN or COPILOT_GITHUB_TOKEN. |
| OpenClaw | NemoClaw | Run OpenClaw more securely inside NVIDIA OpenShell with managed inference using NemoClaw. |
| Hermes Agent | NemoClaw | Run Hermes Agent more securely inside NVIDIA OpenShell with managed inference using NemoClaw. |
| Ollama | Community | Launch with openshell sandbox create --from ollama. |
| Pi | Community | Launch with openshell sandbox create --from pi. |
Key Commands
| Command | Description |
|---|---|
openshell sandbox create -- <agent> |
Create a sandbox and launch an agent. |
openshell sandbox connect [name] |
SSH into a running sandbox. |
openshell sandbox list |
List all sandboxes. |
openshell provider create --type [type]] --from-existing |
Create a credential provider from env vars. |
openshell policy set <name> --policy file.yaml |
Apply or update a policy on a running sandbox. |
openshell policy get <name> |
Show the active policy. |
openshell inference set --provider <p> --model <m> |
Configure the inference.local endpoint. |
openshell logs [name] --tail |
Stream sandbox logs. |
openshell term |
Launch the real-time terminal UI for debugging. |
See the full documentation for command guides, tutorials, and reference material.
Terminal UI
OpenShell includes a real-time terminal dashboard for monitoring gateways, sandboxes, and providers — inspired by k9s.
openshell term
The TUI gives you a live, keyboard-driven view of your gateway and sandboxes. Navigate with Tab to switch panels, j/k to move through lists, Enter to select, and : for command mode. Gateway health and sandbox status auto-refresh every two seconds.
Community Sandboxes and BYOC
Use --from to create sandboxes from the OpenShell Community catalog, a local directory, or a container image:
openshell sandbox create --from gemini # community catalog
openshell sandbox create --from ./my-sandbox-dir # local Dockerfile
openshell sandbox create --from registry.io/img:v1 # container image
See the community sandboxes catalog and the BYOC example for details.
Explore with Your Agent
Clone the repo and point your coding agent at it. The project includes agent skills that can answer questions, walk you through workflows, and diagnose problems — no issue filing required.
git clone https://github.com/NVIDIA/OpenShell.git # or git@github.com:NVIDIA/OpenShell.git
cd OpenShell
# Point your agent here — it will discover the skills in .agents/skills/ automatically
Your agent can load skills for CLI usage (openshell-cli), gateway troubleshooting (debug-openshell-cluster), inference troubleshooting (debug-inference), policy generation (generate-sandbox-policy), and more. See CONTRIBUTING.md for the full skills table.
Built With Agents
OpenShell is developed using the same agent-driven workflows it enables. The .agents/skills/ directory contains workflow automation that powers the project's development cycle:
- Spike and build: Investigate a problem with
create-spike, then implement it withbuild-from-issueonce a human approves. - Triage and route: Community issues are assessed with
triage-issue, classified, and routed into the spike-build pipeline. - Security review:
review-security-issueproduces a severity assessment and remediation plan.fix-security-issueimplements it. - Policy authoring:
generate-sandbox-policycreates YAML policies from plain-language requirements or API documentation.
All implementation work is human-gated — agents propose plans, humans approve, agents build. See AGENTS.md for the full workflow chain documentation.
Getting Help
- Questions and discussion: GitHub Discussions
- Bug reports: GitHub Issues — use the bug report template
- Security vulnerabilities: See SECURITY.md — do not use GitHub Issues
- Agent-assisted help: Clone the repo and use the agent skills in
.agents/skills/for self-service diagnostics
Learn More
- Full Documentation — overview, architecture, tutorials, and reference
- Quickstart — detailed install and first sandbox walkthrough
- GitHub Sandbox Tutorial — end-to-end scoped GitHub repo access
- Architecture — detailed architecture docs and design decisions
- Roadmap — planned work and project priorities
- Support Matrix — platforms, versions, and kernel requirements
- Brev Launchable — try OpenShell on cloud compute without local setup
- Agent Instructions — system prompt and workflow documentation for agent contributors
Contributing
OpenShell is built agent-first — your agent is your first collaborator. Before opening issues or submitting code, point your agent at the repo and let it use the skills in .agents/skills/ to investigate, diagnose, and prototype. See CONTRIBUTING.md for the full agent skills table, contribution workflow, and development setup.
Telemetry
OpenShell collects anonymous telemetry to help improve the project for developers. This data is not used to track individual user behavior. It helps us understand aggregate usage of sandbox, provider, and policy workflows so we can prioritize product improvements and share usage trends with the community.
Disable telemetry by setting OPENSHELL_TELEMETRY_ENABLED=false on the gateway deployment. OpenShell propagates this deployment setting into sandbox supervisor environments so sandbox-side telemetry collection is disabled as well.
Telemetry events are limited to anonymous operational categories and counts, such as sandbox lifecycle outcomes, provider profile buckets, policy decision counts, and aggregate network activity denial categories. OpenShell telemetry does not collect sandbox names or IDs, hostnames, file paths, binary paths, prompts, credentials, provider names, model names, or user content.
Opting out applies only to telemetry emitted by OpenShell. Third-party services, model providers, inference endpoints, agents, or tools that you configure and use with OpenShell may have their own terms and privacy practices.
Notice and Disclaimer
This software automatically retrieves, accesses or interacts with external materials. Those retrieved materials are not distributed with this software and are governed solely by separate terms, conditions and licenses. You are solely responsible for finding, reviewing and complying with all applicable terms, conditions, and licenses, and for verifying the security, integrity and suitability of any retrieved materials for your specific use case. This software is provided "AS IS", without warranty of any kind. The author makes no representations or warranties regarding any retrieved materials, and assumes no liability for any losses, damages, liabilities or legal consequences from your use or inability to use this software or any retrieved materials. Use this software and the retrieved materials at your own risk.
License
This project is licensed under the Apache License 2.0.