* feat(inference): add sandbox-system inference route for platform-level inference
Add a separate system-level inference route that the sandbox supervisor
can use in-process for platform functions (e.g., embedded agent harness
for policy analysis), distinct from the user-facing inference.local
endpoint. The system route is accessed via an in-process API on the
supervisor, ensuring userland code in the sandbox netns cannot reach it.
- Extend proto with route_name fields on Set/Get inference messages
- Add ResolvedRoute.name field to the router for route segregation
- Server resolves both user and sandbox-system routes in bundles
- Sandbox partitions routes into user/system caches on refresh
- Expose InferenceContext::system_inference() in-process API
- CLI --sandbox flag targets the system route on set/get/update
- Integration tests using mock:// routes for the full in-process path
Closes#207
* refactor(cli): rename --sandbox flag to --system for inference commands
The --sandbox flag could be misread as targeting a user-level sandbox
operation. Rename to --system to clearly indicate it configures the
platform-level system inference route.
* style(cli): collapse short if-else per rustfmt 1.94
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The previous rename commit missed hardcoded namespace and workload
references in the bootstrap code. The Helm chart deploys to the
'openshell' namespace with 'openshell' workload names, but the
bootstrap code was still looking for 'navigator' namespace and
workloads, causing cluster startup to hang waiting for TLS secrets
that were never found.
Changes:
- lib.rs: Update secret fetch to use -n openshell
- runtime.rs: Rename functions and update all kubectl commands
to reference openshell namespace and workloads
UpdateProvider RPC was doing full replacement of credentials and config
maps, silently destroying data on partial updates. This changes the
semantics so empty maps preserve existing values, non-empty maps merge
(upsert), and empty-string values delete individual keys. Also makes
provider type immutable after creation and adds field validation on
update.
Closes#199
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(cli): reorganize help output like gh CLI
- Add custom help template with grouped command sections (SANDBOX, GATEWAY, ADDITIONAL)
- Add bold headers using ANSI escape codes
- Add USAGE section showing command/subcommand pattern
- Add consistent help template for subcommands with COMMANDS section
- Add examples for all subcommands (sandbox, gateway, forward, logs, policy, provider, inference)
- Show help when running a command without a subcommand
- Add ALIAS section to subcommand help for aliased commands
- Remove less useful aliases (prov, inf, st, t, comp), keep core ones (sb, gw, lg, fwd, pol)
- Add RequestingCompute provisioning step with elapsed time display
* fix(server): replay platform events for late WatchSandbox subscribers
The CLI was stuck on 'Requesting compute...' even when the pod was
already pulling the container image. This happened because Kubernetes
events (Scheduled, Pulling, etc.) that fired before the CLI subscribed
to WatchSandbox were lost.
Add a tail buffer to PlatformEventBus (mirroring TracingLogBus) so
platform events are buffered and replayed to late subscribers. This
ensures the CLI shows accurate provisioning status regardless of when
it connects.
Closes#196
Piggyback on the existing 2-second sandbox list refresh to detect policy
version changes. When the user is on the sandbox screen and the listed
current_policy_version differs from the displayed version, fetch the
updated policy via GetSandboxPolicy. Scroll position is preserved across
auto-refreshes so the user is not disrupted.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The ProxyCommand in handle_shell_connect and handle_exec_command was
using --gateway for the URL, but after the #175 rename --gateway means
cluster name and --gateway-endpoint is the URL flag. The ssh-proxy
subprocess could not match token mode (no --gateway-endpoint) or name
mode (no --name), so it exited immediately — causing a screen flash
and eventual terminal corruption from rapid suspend/resume cycles.
Use --gateway-endpoint for the URL and --gateway for the cluster name,
matching the format already used by start_port_forwards and the CLI.
Fixes#188
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
When --provider <name> is passed and no provider with that name exists on
the server, the CLI now checks if the name is a recognized provider type
(e.g. claude, nvidia, github). If it is, the provider is auto-created
from local credentials using the same discovery flow as inferred providers.
If the name is not a recognized type, the CLI returns a clear error
instead of deferring validation to the server.
This eliminates the FailedPrecondition error that occurred when passing
--provider nvidia on a fresh cluster where no providers had been created
yet.
* fix(security): add SSH session token expiry, connection limits, and lifecycle cleanup
Closes#22
SSH session tokens previously had no TTL and remained valid indefinitely.
This adds configurable token expiry (default 24h), per-token (10) and
per-sandbox (20) concurrent connection limits, session cleanup on sandbox
deletion, and a background reaper for expired/revoked sessions.
* fix(security): lower per-token concurrent connection limit from 10 to 3
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(proxy): support plain HTTP forward proxy for private IP endpoints
Add forward proxy mode to the sandbox proxy so that standard HTTP
libraries (httpx, requests, etc.) work with HTTP_PROXY for plain HTTP
calls to private IP endpoints. Previously, non-CONNECT methods were
unconditionally rejected with 403.
The forward proxy path requires all three conditions to be met:
- OPA policy explicitly allows the destination
- The matched endpoint has allowed_ips configured
- All resolved IPs are RFC 1918 private
This ensures plain HTTP never reaches the public internet while enabling
seamless access to internal services without custom CONNECT tunnel code.
Implementation:
- parse_proxy_uri(): parses absolute-form URIs into components
- rewrite_forward_request(): rewrites to origin-form, strips hop-by-hop
headers, adds Via and Connection: close
- handle_forward_proxy(): full handler with OPA eval, SSRF checks,
private-IP gate, upstream connect, and bidirectional relay
- Updated dispatch in handle_tcp_connection to route non-CONNECT methods
Includes 14 unit tests and 6 E2E tests (FWD-1 through FWD-6).
CONNECT path remains completely untouched.
Closes#155
* fix(proxy): remove InspectForInference match arm removed by #146
The inference routing simplification in #146 reduced NetworkAction to
Allow/Deny, removing InspectForInference. Drop the dead match arm from
handle_forward_proxy.
* fix(sandbox): restore BestEffort as default Landlock compatibility
The Landlock V2 upgrade in #151 changed the default from BestEffort to
HardRequirement. This causes all proxy-mode sandboxes to crash with
Permission denied when the policy omits the landlock field, because the
child process gets locked to only /etc/navigator-tls and /sandbox.
Restore BestEffort as the default so policies without an explicit
landlock field degrade gracefully.
Fixes#161
* fix(sandbox): inject baseline filesystem paths for proxy-mode sandboxes
Proxy-mode sandboxes need baseline filesystem paths (/usr, /lib, /etc,
/app, /var/log read-only; /sandbox, /tmp read-write) for the child
process to function under Landlock. Without these, the child can't exec
binaries, resolve DNS, or load shared libraries.
The supervisor now enriches the policy with these baseline paths at
startup, covering both standalone (file) and gateway (gRPC) modes. For
gateway mode, the enriched policy is synced back so users see the
effective policy via 'nemoclaw sandbox get'.
The gateway validation is relaxed to allow additive filesystem changes
(new paths can be added, existing paths cannot be removed) to support
the supervisor's enrichment sync-back.
Includes 2 E2E tests: BFS-1 (missing filesystem_policy) and BFS-2
(incomplete filesystem_policy).
Fixes#161
* fix(e2e): update assertion for relaxed filesystem validation message
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* fix(server): add field-level size limits to sandbox and provider creation
Closes#24
Add validate_sandbox_spec and provider field validation with named
constants. Configure explicit 1MB tonic max_decoding_message_size.
Inference routes excluded per #133 rearchitecture.
* chore: remove issue number references from code comments
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#26
All list RPCs (ListSandboxes, ListProviders, ListSandboxPolicies,
ListInferenceRoutes) passed the client-provided limit directly to SQL
queries with no upper bound. A client could send limit=u32::MAX and
cause the server to load all records into memory, risking OOM. This
introduces a MAX_PAGE_SIZE constant (1000) and a clamp_limit helper
that caps the limit in every list handler before it reaches the
persistence layer.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#27
Add remove() methods to TracingLogBus, PlatformEventBus, and
SandboxWatchBus to clean up entries when sandboxes are deleted.
Wire cleanup into both handle_deleted (K8s reconciler) and
delete_sandbox (gRPC handler). Reorder watch_sandbox to validate
sandbox existence before subscribing to buses, preventing entries
for non-existent IDs. Add one-time sandbox validation at stream
open in push_sandbox_logs.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(policy): add validation layer to reject unsafe sandbox policies
Add policy validation that checks for root process identity, path
traversal sequences, overly broad filesystem paths, and exceeding
filesystem rule limits. Validation runs at three entry points:
disk-loaded YAML policies (fallback to restrictive default on violation),
gRPC CreateSandbox, and gRPC UpdateSandboxPolicy (returns
INVALID_ARGUMENT). Filesystem paths are normalized before storage to
collapse traversal components.
Closes#33
* fix(e2e): correct policy update test to match immutable field behavior
The update policy test was asserting on validation errors for fields
(process, filesystem) that are immutable on live sandboxes. The server
rejects changes to these fields before validation runs. Updated the test
to verify the immutability guard instead.
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#130
Rename the TUI from "Gator" to use generic naming:
- CLI subcommand: `nemoclaw gator` → `nemoclaw term`
- TUI title bar: "Gator" → "NemoClaw"
- Docs/skills: "Gator" → "the TUI" or "NemoClaw TUI"
- File renames: gator.toml → term.toml, gator.md → tui.md
- Remove dead link to nonexistent plans/gator-tui.md
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#32
Add post-condition checks in drop_privileges() to verify that setgid()
and setuid() actually changed the effective IDs. Also verify that
setuid(0) fails after dropping privileges, confirming root cannot be
re-acquired. This is a defense-in-depth hardening measure per CWE-250
and CERT POS37-C. All added syscalls (geteuid, getegid, setuid) are
async-signal-safe, so they are safe in the pre_exec context.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The sandbox proxy had a control_plane_endpoints allowlist that let
sandboxed processes reach the gateway without OPA evaluation or SSRF
checks. This is a security issue: if the sandbox can talk to the
control plane, it could modify its own policy, exfiltrate data through
provider environment endpoints, or abuse other gateway APIs.
All connections now go through OPA policy evaluation and SSRF
protection with no exceptions. Removes AllowedEndpoint, parse_endpoint_url,
the is_control_plane bypass in handle_tcp_connection, and the engine field
from ConnectDecision.
* fix(security): harden sandbox SSH with mandatory HMAC secret, NetworkPolicy, and nonce replay detection
Closes#25
- Make NEMOCLAW_SSH_HANDSHAKE_SECRET mandatory: server and sandbox both
refuse to start if the secret is empty/unset. Cluster deployments
auto-generate it via openssl rand in the entrypoint script.
- Add Kubernetes NetworkPolicy restricting sandbox port 2222 ingress to
the gateway pod only, preventing lateral movement from other cluster
workloads.
- Add NSSH1 nonce replay detection with a TTL-bounded cache, rejecting
replayed handshakes within the timestamp validity window.
- Add unit tests for verify_preface (valid, replay, expired, bad HMAC,
malformed) and env injection.
* fix(deploy): pass sshHandshakeSecret in fast deploy helm upgrade
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* refactor(policy): consolidate duplicated YAML struct hierarchies into navigator-policy
Closes#96
Merge the Deserialize-only input structs and Serialize-only output structs
into a single set of types in navigator-policy that derive both Serialize
and Deserialize. This eliminates the duplicate PolicyYaml hierarchy in
navigator-cli and fixes three round-trip issues:
- filesystem_policy vs filesystem field name mismatch
- allowed_ips silently dropped on proto-to-YAML conversion
- network policy name field silently dropped on proto-to-YAML conversion
Also adds api_patterns support to the inference YAML schema and switches
network_policies from HashMap to BTreeMap for deterministic output ordering.
* fix(e2e): update non-CONNECT test assertion from 405 to 403
Align test_l4_non_connect_method_rejected with the proxy behavior
change in c06117e which intentionally returns 403 for non-CONNECT
requests.
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(skills): create nemoclaw-cli agent skill and fix generate-sandbox-policy schema drift
Closes#83
Create a new nemoclaw-cli agent skill with tiered workflow guidance (basic
through advanced) and a supplementary CLI reference. The skill covers provider
management, sandbox lifecycle, policy iteration loops, BYOC pipelines, and
agent-assisted sandbox sessions. It cross-references generate-sandbox-policy
for policy content authoring and includes --help self-teaching as a fallback.
Fix critical schema drift in generate-sandbox-policy: rename
allowed_routing_hints to allowed_routes (matching the actual proto/serde
field name). Update default read_only paths to match dev-sandbox-policy.yaml.
Fix the same field name error in architecture/security-policy.md.
* policy updates
* fix(test): update OPA tests for renamed github_ssh_over_https policy key
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The sandbox SSH server used tokio::spawn fire-and-forget for each stdout/stderr
chunk, which provided no ordering guarantee under tokio's work-stealing scheduler.
Under load, chunks arrived out-of-order, corrupting tar streams during download.
Replace tokio::spawn with runtime.block_on() to ensure each chunk is fully sent
before reading the next. Remove AsyncWriteExt::shutdown() from the SSH proxy's
copy helper to prevent gateways from treating TCP half-close as full-close and
truncating in-flight downloads.
Implement channel_eof handler on the sandbox SSH server to close the child
process stdin when the client signals EOF. Without this, commands like
'cat | tar xf -' used by sync --up would hang waiting for stdin EOF
indefinitely. Add ValueHint::AnyPath to the --up argument for shell completion.
* feat(tui): add port forwarding support to Gator (#80)
Extract forward PID management, resolve_ssh_gateway, and shell_escape from
navigator-cli into navigator-core::forward as a shared module. Add a Ports
field to the TUI create sandbox modal, a NOTES column to the sandbox table,
and a Forwards row to the sandbox detail view. On creation with ports, Gator
polls for Ready state then spawns background SSH tunnels. Forward cleanup
runs automatically on sandbox delete.
Closes#80
* fix(tui): fix Ports field invisible due to modal height overflow
The modal height calculation used content_height + 3 but block chrome
(borders + padding) is 4 rows, not 3. Combined with the 3-row Ports
field layout, the input line was clipped to zero height on standard
24-row terminals. Switch Ports to a compact single-line inline layout
and fix the chrome arithmetic.
* wip: BYOC example with port forwarding and TUI command field fix
- Move examples/bring-your-own-container.md into its own directory
- Add example Dockerfile and app.py (Python REST API with /hello endpoint)
- Rewrite README with full CLI and TUI port-forward workflows
- Change TUI Command field default from /bin/bash to empty so custom
image entrypoints run without manual clearing
* feat(tui): wire up Command field to SSH exec after sandbox creation
When a command is specified in the create sandbox modal, the TUI now
waits for the sandbox to reach Ready (with the pacman animation),
starts any port forwards, then suspends the TUI and executes the
command via SSH — matching the CLI's post-creation flow.
- Refactor spawn_create_sandbox to poll for Ready inline when ports
or command are set, keeping the animation visible throughout
- Add start_port_forwards() helper called within the create task
- Add handle_exec_command() to suspend TUI and run SSH exec
- Remove unused ForwardResult event variant (forwards are now started
within the create task, not as a separate event)
* fix(tui): fix command exec, forward timeouts, and create modal UX
- Fix handle_exec_command to suspend TUI and run SSH attached (matching
CLI behavior) so the process stays alive for the session duration
- Fix shell_escape double-quoting: escape each word individually
- Add ConnectTimeout and 20s spawn timeout to forward SSH so a stalled
auth doesn't freeze the create flow forever
- Remove dead ForwardResult event variant
- Add cluster_name to start_port_forwards for ProxyCommand
- Add spacer between Providers and Ports in create modal
- Fix Command placeholder: 'runs /bin/bash if empty' (not image entrypoint)
- Update BYOC README: document that CMD is replaced by supervisor,
command must be passed explicitly, remove TUI-specific sections
* fix(sandbox): demote Landlock fallback log from warn to debug
The Landlock filesystem sandbox emits a noisy warning when a policy
path does not exist (e.g. /app in custom images). Since BestEffort
mode intentionally continues without Landlock, demote to debug.
* docs: fix BYOC Dockerfile comments for supervisor CMD override
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#42
- Change non-CONNECT proxy response from 405 to 403 to align with
how CONNECT denials are surfaced
- Add structured deny logging for non-CONNECT requests with hostname
extraction from absolute-form URIs
- Revise 7 user-facing error messages across proxy.rs and
dev-sandbox-policy.rego to follow consistent principle: generic
policy-deny messages for non-inference requests, descriptive messages
for recognized inference endpoints
- Update E2E test assertion to match new error message
- Update architecture docs to reflect new behavior
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Drew Newberry