Add OPENSHELL_SANDBOX_IMAGE_PULL_POLICY env/CLI flag that propagates
through Config -> SandboxClient -> pod spec generation. When set, the
policy is applied to both inline and pod-template sandbox containers.
Helm chart exposes server.sandboxImagePullPolicy (default empty = K8s
default behavior). cluster-deploy-fast sets it to Always so dev
clusters always pull fresh sandbox images.
* fix(cli): improve completion coverage and gateway selection
Closes#232
Add typed completion metadata for gateway, sandbox, and path arguments, make gateway selection interactive on TTYs, and expose root subcommands in completions while keeping non-interactive behavior stable.
Add --gpu flag to 'openshell gateway start' that enables NVIDIA GPU
access inside the cluster container. When enabled, the Docker container
receives GPU device requests via bollard, k3s auto-detects the NVIDIA
runtime, and the NVIDIA k8s-device-plugin is deployed via HelmChart CR
so Kubernetes workloads can request nvidia.com/gpu resources.
- Add GPU device request to Docker container creation (bollard DeviceRequest)
- Set GPU_ENABLED=true env var to gate GPU manifest deployment
- Multi-stage Dockerfile build to bundle NVIDIA Container Toolkit binaries
- Conditional GPU manifest copy in cluster entrypoint
- HelmChart CR for nvidia-device-plugin v0.18.2 with GFD and NFD enabled
- Thread gpu bool through DeployOptions, CLI flag, and run dispatch
---------
Co-authored-by: Piotr Mlocek <pmlocek@nvidia.com>
Replace the clear_process_identity + maybe_infer_sandbox_user workaround
with strict enforcement: run_as_user and run_as_group must always be
'sandbox'. If omitted from YAML, they are defaulted server-side via
ensure_sandbox_process_identity(). Any other value is rejected.
This fixes 'policy set' failing with 'process policy cannot be changed
on a live sandbox' for custom-image sandboxes, where the CLI was
clearing identity fields but the YAML had 'sandbox' explicitly set.
- Remove clear_process_identity() from CLI and TUI
- Add ensure_sandbox_process_identity() to default missing fields
- Rename RootProcessIdentity -> InvalidProcessIdentity
- Replace maybe_infer_sandbox_user with validate_sandbox_user (fail-fast)
- Update OPA test policies to use run_as_user: sandbox
* feat(inference): add sandbox-system inference route for platform-level inference
Add a separate system-level inference route that the sandbox supervisor
can use in-process for platform functions (e.g., embedded agent harness
for policy analysis), distinct from the user-facing inference.local
endpoint. The system route is accessed via an in-process API on the
supervisor, ensuring userland code in the sandbox netns cannot reach it.
- Extend proto with route_name fields on Set/Get inference messages
- Add ResolvedRoute.name field to the router for route segregation
- Server resolves both user and sandbox-system routes in bundles
- Sandbox partitions routes into user/system caches on refresh
- Expose InferenceContext::system_inference() in-process API
- CLI --sandbox flag targets the system route on set/get/update
- Integration tests using mock:// routes for the full in-process path
Closes#207
* refactor(cli): rename --sandbox flag to --system for inference commands
The --sandbox flag could be misread as targeting a user-level sandbox
operation. Rename to --system to clearly indicate it configures the
platform-level system inference route.
* style(cli): collapse short if-else per rustfmt 1.94
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The previous rename commit missed hardcoded namespace and workload
references in the bootstrap code. The Helm chart deploys to the
'openshell' namespace with 'openshell' workload names, but the
bootstrap code was still looking for 'navigator' namespace and
workloads, causing cluster startup to hang waiting for TLS secrets
that were never found.
Changes:
- lib.rs: Update secret fetch to use -n openshell
- runtime.rs: Rename functions and update all kubectl commands
to reference openshell namespace and workloads
UpdateProvider RPC was doing full replacement of credentials and config
maps, silently destroying data on partial updates. This changes the
semantics so empty maps preserve existing values, non-empty maps merge
(upsert), and empty-string values delete individual keys. Also makes
provider type immutable after creation and adds field validation on
update.
Closes#199
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(cli): reorganize help output like gh CLI
- Add custom help template with grouped command sections (SANDBOX, GATEWAY, ADDITIONAL)
- Add bold headers using ANSI escape codes
- Add USAGE section showing command/subcommand pattern
- Add consistent help template for subcommands with COMMANDS section
- Add examples for all subcommands (sandbox, gateway, forward, logs, policy, provider, inference)
- Show help when running a command without a subcommand
- Add ALIAS section to subcommand help for aliased commands
- Remove less useful aliases (prov, inf, st, t, comp), keep core ones (sb, gw, lg, fwd, pol)
- Add RequestingCompute provisioning step with elapsed time display
* fix(server): replay platform events for late WatchSandbox subscribers
The CLI was stuck on 'Requesting compute...' even when the pod was
already pulling the container image. This happened because Kubernetes
events (Scheduled, Pulling, etc.) that fired before the CLI subscribed
to WatchSandbox were lost.
Add a tail buffer to PlatformEventBus (mirroring TracingLogBus) so
platform events are buffered and replayed to late subscribers. This
ensures the CLI shows accurate provisioning status regardless of when
it connects.
Closes#196
Piggyback on the existing 2-second sandbox list refresh to detect policy
version changes. When the user is on the sandbox screen and the listed
current_policy_version differs from the displayed version, fetch the
updated policy via GetSandboxPolicy. Scroll position is preserved across
auto-refreshes so the user is not disrupted.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The ProxyCommand in handle_shell_connect and handle_exec_command was
using --gateway for the URL, but after the #175 rename --gateway means
cluster name and --gateway-endpoint is the URL flag. The ssh-proxy
subprocess could not match token mode (no --gateway-endpoint) or name
mode (no --name), so it exited immediately — causing a screen flash
and eventual terminal corruption from rapid suspend/resume cycles.
Use --gateway-endpoint for the URL and --gateway for the cluster name,
matching the format already used by start_port_forwards and the CLI.
Fixes#188
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
When --provider <name> is passed and no provider with that name exists on
the server, the CLI now checks if the name is a recognized provider type
(e.g. claude, nvidia, github). If it is, the provider is auto-created
from local credentials using the same discovery flow as inferred providers.
If the name is not a recognized type, the CLI returns a clear error
instead of deferring validation to the server.
This eliminates the FailedPrecondition error that occurred when passing
--provider nvidia on a fresh cluster where no providers had been created
yet.
* fix(security): add SSH session token expiry, connection limits, and lifecycle cleanup
Closes#22
SSH session tokens previously had no TTL and remained valid indefinitely.
This adds configurable token expiry (default 24h), per-token (10) and
per-sandbox (20) concurrent connection limits, session cleanup on sandbox
deletion, and a background reaper for expired/revoked sessions.
* fix(security): lower per-token concurrent connection limit from 10 to 3
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(proxy): support plain HTTP forward proxy for private IP endpoints
Add forward proxy mode to the sandbox proxy so that standard HTTP
libraries (httpx, requests, etc.) work with HTTP_PROXY for plain HTTP
calls to private IP endpoints. Previously, non-CONNECT methods were
unconditionally rejected with 403.
The forward proxy path requires all three conditions to be met:
- OPA policy explicitly allows the destination
- The matched endpoint has allowed_ips configured
- All resolved IPs are RFC 1918 private
This ensures plain HTTP never reaches the public internet while enabling
seamless access to internal services without custom CONNECT tunnel code.
Implementation:
- parse_proxy_uri(): parses absolute-form URIs into components
- rewrite_forward_request(): rewrites to origin-form, strips hop-by-hop
headers, adds Via and Connection: close
- handle_forward_proxy(): full handler with OPA eval, SSRF checks,
private-IP gate, upstream connect, and bidirectional relay
- Updated dispatch in handle_tcp_connection to route non-CONNECT methods
Includes 14 unit tests and 6 E2E tests (FWD-1 through FWD-6).
CONNECT path remains completely untouched.
Closes#155
* fix(proxy): remove InspectForInference match arm removed by #146
The inference routing simplification in #146 reduced NetworkAction to
Allow/Deny, removing InspectForInference. Drop the dead match arm from
handle_forward_proxy.
* fix(sandbox): restore BestEffort as default Landlock compatibility
The Landlock V2 upgrade in #151 changed the default from BestEffort to
HardRequirement. This causes all proxy-mode sandboxes to crash with
Permission denied when the policy omits the landlock field, because the
child process gets locked to only /etc/navigator-tls and /sandbox.
Restore BestEffort as the default so policies without an explicit
landlock field degrade gracefully.
Fixes#161
* fix(sandbox): inject baseline filesystem paths for proxy-mode sandboxes
Proxy-mode sandboxes need baseline filesystem paths (/usr, /lib, /etc,
/app, /var/log read-only; /sandbox, /tmp read-write) for the child
process to function under Landlock. Without these, the child can't exec
binaries, resolve DNS, or load shared libraries.
The supervisor now enriches the policy with these baseline paths at
startup, covering both standalone (file) and gateway (gRPC) modes. For
gateway mode, the enriched policy is synced back so users see the
effective policy via 'nemoclaw sandbox get'.
The gateway validation is relaxed to allow additive filesystem changes
(new paths can be added, existing paths cannot be removed) to support
the supervisor's enrichment sync-back.
Includes 2 E2E tests: BFS-1 (missing filesystem_policy) and BFS-2
(incomplete filesystem_policy).
Fixes#161
* fix(e2e): update assertion for relaxed filesystem validation message
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* fix(server): add field-level size limits to sandbox and provider creation
Closes#24
Add validate_sandbox_spec and provider field validation with named
constants. Configure explicit 1MB tonic max_decoding_message_size.
Inference routes excluded per #133 rearchitecture.
* chore: remove issue number references from code comments
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#26
All list RPCs (ListSandboxes, ListProviders, ListSandboxPolicies,
ListInferenceRoutes) passed the client-provided limit directly to SQL
queries with no upper bound. A client could send limit=u32::MAX and
cause the server to load all records into memory, risking OOM. This
introduces a MAX_PAGE_SIZE constant (1000) and a clamp_limit helper
that caps the limit in every list handler before it reaches the
persistence layer.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#27
Add remove() methods to TracingLogBus, PlatformEventBus, and
SandboxWatchBus to clean up entries when sandboxes are deleted.
Wire cleanup into both handle_deleted (K8s reconciler) and
delete_sandbox (gRPC handler). Reorder watch_sandbox to validate
sandbox existence before subscribing to buses, preventing entries
for non-existent IDs. Add one-time sandbox validation at stream
open in push_sandbox_logs.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(policy): add validation layer to reject unsafe sandbox policies
Add policy validation that checks for root process identity, path
traversal sequences, overly broad filesystem paths, and exceeding
filesystem rule limits. Validation runs at three entry points:
disk-loaded YAML policies (fallback to restrictive default on violation),
gRPC CreateSandbox, and gRPC UpdateSandboxPolicy (returns
INVALID_ARGUMENT). Filesystem paths are normalized before storage to
collapse traversal components.
Closes#33
* fix(e2e): correct policy update test to match immutable field behavior
The update policy test was asserting on validation errors for fields
(process, filesystem) that are immutable on live sandboxes. The server
rejects changes to these fields before validation runs. Updated the test
to verify the immutability guard instead.
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#130
Rename the TUI from "Gator" to use generic naming:
- CLI subcommand: `nemoclaw gator` → `nemoclaw term`
- TUI title bar: "Gator" → "NemoClaw"
- Docs/skills: "Gator" → "the TUI" or "NemoClaw TUI"
- File renames: gator.toml → term.toml, gator.md → tui.md
- Remove dead link to nonexistent plans/gator-tui.md
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#32
Add post-condition checks in drop_privileges() to verify that setgid()
and setuid() actually changed the effective IDs. Also verify that
setuid(0) fails after dropping privileges, confirming root cannot be
re-acquired. This is a defense-in-depth hardening measure per CWE-250
and CERT POS37-C. All added syscalls (geteuid, getegid, setuid) are
async-signal-safe, so they are safe in the pre_exec context.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The sandbox proxy had a control_plane_endpoints allowlist that let
sandboxed processes reach the gateway without OPA evaluation or SSRF
checks. This is a security issue: if the sandbox can talk to the
control plane, it could modify its own policy, exfiltrate data through
provider environment endpoints, or abuse other gateway APIs.
All connections now go through OPA policy evaluation and SSRF
protection with no exceptions. Removes AllowedEndpoint, parse_endpoint_url,
the is_control_plane bypass in handle_tcp_connection, and the engine field
from ConnectDecision.
* fix(security): harden sandbox SSH with mandatory HMAC secret, NetworkPolicy, and nonce replay detection
Closes#25
- Make NEMOCLAW_SSH_HANDSHAKE_SECRET mandatory: server and sandbox both
refuse to start if the secret is empty/unset. Cluster deployments
auto-generate it via openssl rand in the entrypoint script.
- Add Kubernetes NetworkPolicy restricting sandbox port 2222 ingress to
the gateway pod only, preventing lateral movement from other cluster
workloads.
- Add NSSH1 nonce replay detection with a TTL-bounded cache, rejecting
replayed handshakes within the timestamp validity window.
- Add unit tests for verify_preface (valid, replay, expired, bad HMAC,
malformed) and env injection.
* fix(deploy): pass sshHandshakeSecret in fast deploy helm upgrade
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* refactor(policy): consolidate duplicated YAML struct hierarchies into navigator-policy
Closes#96
Merge the Deserialize-only input structs and Serialize-only output structs
into a single set of types in navigator-policy that derive both Serialize
and Deserialize. This eliminates the duplicate PolicyYaml hierarchy in
navigator-cli and fixes three round-trip issues:
- filesystem_policy vs filesystem field name mismatch
- allowed_ips silently dropped on proto-to-YAML conversion
- network policy name field silently dropped on proto-to-YAML conversion
Also adds api_patterns support to the inference YAML schema and switches
network_policies from HashMap to BTreeMap for deterministic output ordering.
* fix(e2e): update non-CONNECT test assertion from 405 to 403
Align test_l4_non_connect_method_rejected with the proxy behavior
change in c06117e which intentionally returns 403 for non-CONNECT
requests.
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Drew Newberry