Commit Graph
41 Commits
Author SHA1 Message Date
Alexander Watson 142a3a3d31 fix(examples): harden multi-agent notepad 409 retry and improve docs (#1166)
* fix(examples): harden multi-agent notepad 409 retry and improve docs

Add decorrelated jitter to the GitHub Contents API retry loop so racing
workers don't retry in lockstep, and widen the retry set to 5xx for
transient server errors. Without jitter, N concurrent workers trip the
same backoff schedule and re-collide on each beat.

Also:
- Extract the inline 220-line sandbox runner from demo.sh into a
  separate runner.sh. demo.sh stays as host orchestration; runner.sh
  is the per-sandbox program. demo.sh shrinks 460 to 240 lines.
- Rewrite README to lead with "Why GitHub as a shared notepad?" and
  add a "Memory architecture variants" section covering pile,
  append-journal, and indexed-memory patterns.
- Switch Quick Start to use \`gh auth token\` as the primary path.
- Pass the worker slice arg explicitly instead of reading \$9 from the
  outer dispatcher.
- Add .gitignore for personal run_demo.sh helpers.

* docs updates
2026-05-05 08:36:12 -07:00
Drew Newberry f56c09c7df docs: update gateway deployment architecture (#1108) 2026-05-04 22:52:29 -07:00
Alexander Watson 25c4fdecd1 fix(examples): repair multi-agent notepad uploads (#1152) 2026-05-04 13:31:25 -07:00
Drew Newberry 2e0afeabe1 feat(vm): derive guest rootfs from sandbox images (#957) 2026-05-03 23:23:30 -07:00
Alexander Watson ebbd9dee5a docs(examples): add multi-agent notepad demo (#991)
Signed-off-by: Alexander Watson <zredlined@gmail.com>
2026-04-30 09:25:19 -07:00
Derek Carr de9dce0436 fix(e2e): use high UID range to avoid host user conflicts (#978)
Change sandbox user UID from 1000 to 1000660000 in custom image examples and E2E tests.
Using a high UID range (1000000000+) prevents conflicts with host users when running
without user namespace remapping, where container UIDs map directly to host UIDs.

This resolves fork failures caused by RLIMIT_NPROC enforcement when the host user
already has many threads running.

Signed-off-by: Derek Carr <decarr@redhat.com>
2026-04-27 09:46:39 -07:00
Piotr Mlocek df38d1f66f feat(ci): add Markdown and Mermaid linting (#933) 2026-04-24 11:27:02 -07:00
Vinicius Corrêa 8c4b17221c Missed input parameter (#645)
Missed "--from openclaw" input parameter.

Without this input sandbox container didn't install "openclaw":
'Stop with: openshell forward stop 18789 eager-dragonfly
'/bin/bash: line 1: openclaw-start: command not found
2026-03-30 13:50:25 -07:00
John T. Myers f37b69b5e5 feat(sandbox): auto-detect TLS and terminate unconditionally for credential injection (#544)
* feat(sandbox): auto-detect TLS and terminate unconditionally for credential injection

Closes #533

The proxy now auto-detects TLS by peeking the first bytes of each
connection. When TLS is detected, it terminates unconditionally —
enabling credential injection and optional L7 inspection without
requiring explicit 'tls: terminate' in the policy.
2026-03-23 18:45:18 -07:00
Alexander Watson a0aea6904b Added pauses and syntax highlighting to demo for clarity (#358) 2026-03-16 09:05:34 -07:00
John T. Myers d3e1b31db9 feat(sandbox): log connection attempts that bypass proxy path (#326)
* feat(sandbox): log connection attempts that bypass proxy path

Add iptables LOG + REJECT rules inside the sandbox network namespace to
detect and diagnose direct connection attempts that bypass the HTTP
CONNECT proxy. This provides two improvements:

1. Fast-fail UX: applications get immediate ECONNREFUSED instead of a
   30-second timeout when they bypass the proxy
2. Diagnostics: a /dev/kmsg monitor emits structured BYPASS_DETECT
   tracing events with destination, protocol, process identity, and
   actionable hints

Both TCP and UDP bypass attempts are covered (UDP catches DNS bypass).
The feature degrades gracefully if iptables or /dev/kmsg are unavailable.

Closes #268

* chore: track .python-version to pin Python 3.13.12 for uv

The sandbox base image runs Python 3.13. A stale venv on 3.12 causes
all exec_python E2E tests to fail because cloudpickle bytecode is not
compatible across minor versions.

* fix(cluster): preserve hostGatewayIP across fast deploys

The fast deploy's helm upgrade was missing the hostGatewayIP value that
the bootstrap entrypoint injects into the HelmChart CR. This caused
host.openshell.internal hostAliases to be lost from the gateway pod
and sandbox pods after any fast deploy, breaking host gateway routing.

Read the IP from the HelmChart CR and pass it through to helm upgrade.

* wip: fix iptables path resolution, use dmesg for kmsg, add CAP_SYSLOG

* fix(sandbox): restore NetworkNamespace Drop impl, remove dead kmsg code

The Drop impl for NetworkNamespace was accidentally deleted during the
bypass detection refactor, which would cause network namespaces and
veth interfaces to leak on every sandbox shutdown.

Also removes dead kmsg volume/mount code (bypass monitor uses dmesg
instead of direct /dev/kmsg access) and removes an accidentally
committed session transcript file.
2026-03-16 00:32:13 -07:00
Drew Newberry 0a7ffe1669 ci(release): gate python wheels on e2e for tagged releases (#319) 2026-03-15 00:39:00 -07:00
Alexander Watson 158c925342 docs(examples): add sandbox policy quickstart walkthrough (#266)
* docs(examples): add sandbox policy quickstart walkthrough

Add an interactive getting-started example that demonstrates OpenShell's
network policy system end-to-end: default-deny, L7 read-only access,
and audit logging — all with a single YAML policy file.

- examples/sandbox-policy-quickstart/policy.yaml: policy with default
  static fields (filesystem, landlock, process) so it works out of the
  box with `openshell policy set`
- examples/sandbox-policy-quickstart/demo.sh: automated demo script
  using printf (portable) and openshell ssh-proxy for sandbox exec
- examples/sandbox-policy-quickstart/README.md: step-by-step manual
  walkthrough
- README.md: add "See network policy in action" section linking to the
  quickstart

Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor
Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor

* docs: soften default-deny wording to minimal outbound access

Sandbox defaults vary by type and community configs, so
"all outbound traffic is blocked" is too absolute.

Made-with: Cursor

* docs: use curl -sS so L4 deny errors are visible

curl -s suppresses stderr, hiding the 403 from the CONNECT
proxy. Adding -S ensures the error message is always shown.

Made-with: Cursor

---------

Signed-off-by: Alexander Watson <zredlined@gmail.com>
2026-03-13 11:32:32 -07:00
Drew Newberry fbd93a4632 refactor: rename navigator- crate prefix to openshell- (#277) 2026-03-13 02:02:18 -07:00
Drew Newberry 7b0a243304 ci: remove sandbox docker build from publish and e2e workflows (#275) 2026-03-13 00:56:07 -07:00
Drew Newberry 89d21d7852 refactor(sandbox): sandboxes are managed as separate community images (#267) 2026-03-12 22:06:52 -07:00
Piotr Mlocek 14e296d318 fix(cli): add --no-keep for ephemeral sandbox create cleanup (#258) 2026-03-12 17:18:39 -07:00
John T. Myers 2e4c2fcc84 fix(proxy): stream inference responses instead of buffering entire body (#261)
* fix(proxy): stream inference responses instead of buffering entire body

The inference.local proxy path called response.bytes().await which
buffered the entire upstream response before sending anything to the
client. For streaming SSE responses this inflated TTFB from sub-second
to the full generation time, causing clients with TTFB timeouts to abort.

Add a streaming proxy variant that returns response headers immediately
and forwards body chunks incrementally using HTTP chunked transfer
encoding. Non-streaming responses and mock routes continue to work
through the existing buffered path.

Closes #260

* docs: update architecture docs and example for inference streaming

* test(example): add direct NVIDIA endpoint tests via L7 TLS intercept

Expand inference example to 4 test cases: inference.local and direct
endpoint, each streaming and non-streaming. The direct path exercises
the L7 REST relay (relay_chunked) to verify it already streams
correctly. NVIDIA_API_KEY is picked up from the sandbox env when
started with --provider nvidia.
2026-03-12 10:36:41 -07:00
John T. Myers 454327d890 feat(policy): add policy recommendation plumbing (#204) (#222)
* feat(policy): add policy recommendation plumbing — denial aggregation, transport, approval pipeline, and mechanistic recommendations

Implement the infrastructure layer for automated policy recommendations (#204):

- Proto: 9 new RPCs and messages for draft policy lifecycle (submit, get, approve, reject, approve-all, edit, undo, clear, history)
- Persistence: SQLite/Postgres migrations and store methods for draft_policy_chunks and denial_summaries tables
- Server: Full gRPC handler implementations with mechanistic mapper that auto-generates NetworkPolicyRule proposals from denial summaries
- Sandbox: DenialAggregator with MPSC channel, deduplication, periodic flush to gateway via SubmitPolicyAnalysis
- CLI: 'openshell draft' subcommand with get/approve/reject/approve-all/undo/clear/history operations
- TUI: Draft recommendations panel accessible from sandbox policy view
- Docs: Architecture documentation in architecture/policy-advisor.md

* feat(policy): add L7-aware mechanistic mapper and policy advisor CTF example

Add L7 rule generation to mechanistic mapper (build_l7_rules,
generalise_path, looks_like_id) with 3 new unit tests. Add
examples/policy-advisor/ with a 7-gate CTF script, restrictive
sandbox policy, and walkthrough README.

* fix(policy): use sandbox name for denial flush and add TUI draft badges

Fix denial aggregator passing sandbox UUID instead of name to
SubmitPolicyAnalysis, which caused 'sandbox not found' errors on
flush. Add notification badges to the TUI sandbox list and detail
header showing pending draft recommendation counts.

* fix(policy): deduplicate draft chunks and tolerate overlapping OPA rules

Skip draft chunk creation when a pending/approved chunk already covers
the same host:port endpoint, preventing duplicate rules across denial
aggregator flush cycles.

Rewrite three OPA complete rules (network_policy_for_request,
matched_network_policy, matched_endpoint_config) to tolerate multiple
matching policies without triggering a "complete rule conflict" error.
network_policy_for_request becomes a boolean, matched_network_policy
uses a set comprehension with min(), and matched_endpoint_config uses
an array comprehension with index-0 selection.

* feat(tui): interactive draft actions, highlight bar, and detail popup

Rework the draft recommendations panel to match the logs UX:
- Highlight bar (green accent + background) instead of arrow marker
- Viewport-aware j/k scrolling with g/G for top/bottom
- Enter opens a full-screen detail popup showing endpoints, binaries,
  rationale, security notes, and action hints

Add approve/reject/approve-all draft actions:
- [a] approve selected chunk, [x] reject, [A] approve all pending
- Actions work from both the list view and the detail popup
- gRPC calls run async; result updates status bar and refreshes data
- Nav bar shows all available keybindings

Fix draft count refresh: sandbox_draft_counts now refreshes on every
tick (not just Dashboard), so the detail header badge updates in
real time.

Improve badge labels: show 'N pending' instead of a bare number in
both the dashboard sandbox list and sandbox detail header.

* refactor(policy): DB-level draft chunk dedup with hit counter and timestamps

Replace the in-memory HashSet dedup in SubmitPolicyAnalysis with a
database-level upsert. New denormalized columns on draft_policy_chunks:
- host, port: extracted from proposed_rule at insert time
- hit_count: incremented on conflict (same sandbox + host + port)
- first_seen_ms, last_seen_ms: track when the endpoint was first and
  most recently proposed

A partial unique index (WHERE status IN ('pending','approved')) ensures
only one active chunk per endpoint per sandbox; rejected/superseded
chunks don't block new proposals.

Surface hit_count and first/last_seen in:
- CLI: 'openshell draft get' shows 'Hits: N (first ..., last ...)'
- TUI: detail popup shows hits row; list view shows 'Nx' suffix

* fix(policy): optimistic retry on policy version conflicts + structured logging

merge_chunk_into_policy and remove_chunk_from_policy now retry up to 5
times on UNIQUE constraint violations (version conflicts from concurrent
approvals). Each attempt re-reads the latest policy, re-merges the rule,
and increments the version. This eliminates the race condition where
rapid successive approvals would fail with a DB error.

Add structured tracing to all draft action handlers:
- ApproveDraftChunk: logs rule_name, host, port, hit_count before merge
  and version + policy_hash after success
- RejectDraftChunk: logs rule_name, host, port, reason
- ApproveAllDraftChunks: logs pending_count at start, per-chunk merge
  progress, and final summary with chunks_approved/skipped
- UndoDraftChunk: logs before/after with rule_name and version
- Retry attempts log as warnings with attempt number and conflicting
  version

* wip: forward proxy fix, mapper allowed_ips, TUI polish, CTF rewrite

* fix(tui): use correct --gateway flag for ssh-proxy ProxyCommand

* chore: add Docker cleanup script for stale images, volumes, and build cache

* feat(tui): approve-all confirmation modal and CTF cleanup

Add [A] confirmation popup that snapshots pending chunks, shows a
scrollable list, and approves each chunk individually on confirm.
This prevents approving chunks that arrived after the modal opened.

Remove transient issue #205 reference from CTF victory banner.

* fix(tui): correct import ordering for rustfmt

* wip: stateful toggle model, rename to network rules

Draft chunks now follow a toggle state machine:
  pending -> approved | rejected (initial decision)
  approved <-> rejected (toggle)

One row per (sandbox_id, host, port) via expanded unique index.
Rejecting an approved rule removes it from the active policy.
Re-approving a rejected rule merges it back.

Rename CLI from 'draft' to 'rule', TUI from 'Draft Recommendations'
to 'Network Rules'. State-aware keybindings: approved shows [x] Revoke,
rejected shows [a] Approve. Fix sandbox detail hiding delete confirmation
behind pending message.

* refactor(policy): move mapper sandbox-side, slim schema, per-binary granularity

Move mechanistic mapper from gateway to sandbox so all analysis runs
sandbox-side (N sandboxes = N independent pipelines). Gateway is now a
thin validate + persist + approval layer.

Architectural changes:
- Move mechanistic_mapper.rs from navigator-server to navigator-sandbox
- Sandbox flush flow: aggregator drains -> mapper runs -> proposals sent
- Gateway SubmitPolicyAnalysis: validate + persist only, no mapper
- Drop denial_summaries table (write-only, zero readers)
- Consolidate migrations 003+004+005 into single 003

Schema slimming:
- Drop 5 unused columns from draft_policy_chunks (stage, denial_refs,
  supersedes_chunk_id, analysis_mode, decided_by)
- Add per-binary granularity: binary column, widen unique index to
  (sandbox_id, host, port, binary)
- Mapper groups by (host, port, binary), one proposal per triple
- Merge appends binary to existing rule; revoke removes just that binary

CTF & UX:
- 7-gate CTF: add Gate 3 (curl -> ifconfig.me:80) for per-binary demo
- TUI shows binary short name in list, full path in detail popup
- CLI output shows binary field
- Idempotent rule names, hit_count accumulates real denial counts
- Rationale text no longer bakes in stale denial count
2026-03-12 08:28:26 -07:00
Drew Newberry 95d7ae077e refactor(cli): remove kubeconfig port, add doctor llm-help, update debug docs (#252) 2026-03-11 21:25:25 -07:00
Drew Newberry bcc6dad1d0 feat(cli): launch sandbox editors via managed ssh include (#226) 2026-03-11 20:06:38 -07:00
Drew Newberry d6c6e97679 chore: remove navigator references from codebase (#208) 2026-03-10 14:46:11 -07:00
Drew Newberry 984d1a6e5c chore: rename project from NemoClaw to OpenShell (#198) 2026-03-10 11:49:09 -07:00
Piotr MlocekandMiyoung Choi 107c85d1d7 docs(inference): clarify local inference routing (#190)
* docs(inference): clarify local inference routing

* docs(inference): update provider and model examples

* fix doc build

---------

Co-authored-by: Miyoung Choi <miyoungc@nvidia.com>
2026-03-09 21:34:14 -07:00
Drew Newberry a2de1f24e5 feat: add Cloudflare tunnel auth support (#178) 2026-03-09 18:39:42 -07:00
Drew Newberry ed53c35d2b feat(cli): improve sandbox provisioning status messages and UX (#175) 2026-03-08 16:36:50 -07:00
Drew Newberry 07b9d5d00e feat(cli): restructure CLI commands for simpler UX (#156) 2026-03-06 16:26:39 -08:00
Piotr Mlocek 31d7ca53ff refactor(inference): simplify routing — introduce inference.local, remove implicit catch-all (#146)
Remove multi-route CRUD system and replace with single managed cluster
route (inference.local). Key changes:

- Remove inference route CRUD RPCs and CLI commands
- Remove InspectForInference OPA action; policy is binary allow/deny
- Introduce AuthHeader enum and InferenceProviderProfile in navigator-core
- Router is now provider-agnostic: auth style carried on ResolvedRoute
- Replace InferenceRouteSpec with ClusterInferenceConfig (2 fields vs 8)
- Rename proto: routing_hint->name, SandboxResolvedRoute->ResolvedRoute,
  GetSandboxInferenceBundle->GetInferenceBundle, drop sandbox_id param
- Rename RouteConfig.route -> RouteConfig.name; use inference.local
- Add 'nemoclaw cluster inference update' for partial config changes
- Delete stale navigator.inference.v1.rs checked-in proto file
- Update architecture docs, agent skills, and CLI reference

Closes #133
2026-03-06 13:54:23 -08:00
Drew Newberry 3da64744f0 feat(cli): add --from flag to sandbox create for unified image sources (#89) 2026-03-04 21:15:13 -08:00
Drew Newberry 99bba8028a feat(sandbox): support policy discovery and restrictive defaults on sandbox containers (#84) 2026-03-04 17:17:22 -08:00
Drew Newberry 90da02a7ed chore: simplify contributing workflow and documentation (#92) 2026-03-04 13:06:32 -08:00
John T. MyersandJohn Myers 66df9f7614 feat(tui): add port forwarding support to Gator (#81)
* feat(tui): add port forwarding support to Gator (#80)

Extract forward PID management, resolve_ssh_gateway, and shell_escape from
navigator-cli into navigator-core::forward as a shared module. Add a Ports
field to the TUI create sandbox modal, a NOTES column to the sandbox table,
and a Forwards row to the sandbox detail view. On creation with ports, Gator
polls for Ready state then spawns background SSH tunnels. Forward cleanup
runs automatically on sandbox delete.

Closes #80

* fix(tui): fix Ports field invisible due to modal height overflow

The modal height calculation used content_height + 3 but block chrome
(borders + padding) is 4 rows, not 3. Combined with the 3-row Ports
field layout, the input line was clipped to zero height on standard
24-row terminals. Switch Ports to a compact single-line inline layout
and fix the chrome arithmetic.

* wip: BYOC example with port forwarding and TUI command field fix

- Move examples/bring-your-own-container.md into its own directory
- Add example Dockerfile and app.py (Python REST API with /hello endpoint)
- Rewrite README with full CLI and TUI port-forward workflows
- Change TUI Command field default from /bin/bash to empty so custom
  image entrypoints run without manual clearing

* feat(tui): wire up Command field to SSH exec after sandbox creation

When a command is specified in the create sandbox modal, the TUI now
waits for the sandbox to reach Ready (with the pacman animation),
starts any port forwards, then suspends the TUI and executes the
command via SSH — matching the CLI's post-creation flow.

- Refactor spawn_create_sandbox to poll for Ready inline when ports
  or command are set, keeping the animation visible throughout
- Add start_port_forwards() helper called within the create task
- Add handle_exec_command() to suspend TUI and run SSH exec
- Remove unused ForwardResult event variant (forwards are now started
  within the create task, not as a separate event)

* fix(tui): fix command exec, forward timeouts, and create modal UX

- Fix handle_exec_command to suspend TUI and run SSH attached (matching
  CLI behavior) so the process stays alive for the session duration
- Fix shell_escape double-quoting: escape each word individually
- Add ConnectTimeout and 20s spawn timeout to forward SSH so a stalled
  auth doesn't freeze the create flow forever
- Remove dead ForwardResult event variant
- Add cluster_name to start_port_forwards for ProxyCommand
- Add spacer between Providers and Ports in create modal
- Fix Command placeholder: 'runs /bin/bash if empty' (not image entrypoint)
- Update BYOC README: document that CMD is replaced by supervisor,
  command must be passed explicitly, remove TUI-specific sections

* fix(sandbox): demote Landlock fallback log from warn to debug

The Landlock filesystem sandbox emits a noisy warning when a policy
path does not exist (e.g. /app in custom images). Since BestEffort
mode intentionally continues without Landlock, demote to debug.

* docs: fix BYOC Dockerfile comments for supervisor CMD override

---------

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
2026-03-03 17:31:44 -08:00
Drew Newberry 9099bc3972 chore: rename Navigator to NemoClaw across user facing contracts (#73) 2026-03-03 11:41:19 -08:00
Alexander Watson 1d7909cb38 chore: add open-source compliance files and SPDX headers (#71)
Add Apache 2.0 licensing, SPDX copyright headers on all source files,
DCO enforcement, third-party notices, and CI enforcement.

- LICENSE: Apache License 2.0 full text
- DCO: Developer Certificate of Origin 1.1
- SPDX headers on all 176 source files (.rs, .py, .proto, .rego, .sh,
  .toml, .yaml, Dockerfiles)
- scripts/update_license_headers.py: header management with --check mode
- scripts/generate_third_party_notices.py: dependency license aggregation
- THIRD-PARTY-NOTICES: generated listing of all Rust and Python deps
- build/license.toml: mise tasks for license:check and license:update
- CI: license-headers job in checks.yml, DCO check workflow
- CONTRIBUTING.md: DCO sign-off requirement and license header docs
- Cargo.toml: license changed to Apache-2.0, repository URL updated
- pyproject.toml: license field added

Closes #58
2026-03-03 09:30:56 -08:00
John T. MyersandJohn Myers ff52643237 feat(sandbox): allow egress to private IP space via allowed_ips policy field (#60)
* feat(sandbox): allow egress to private IP space via allowed_ips policy field

Add an allowed_ips field to NetworkEndpoint that accepts CIDR notation and
exact IPs, enabling sandboxes to reach private IP space under policy control
while maintaining SSRF protections for loopback and link-local addresses.

Three modes are supported:
- Default (no allowed_ips): all private IPs blocked (existing behavior)
- Host + allowlist: domain must resolve to an IP in allowed_ips
- Hostless allowlist (no host): any domain allowed if resolved IP matches



Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
2026-03-02 19:29:34 -08:00
Drew Newberry 5fd4885a7c feat(sandbox): VS Code Remote-SSH support with platform detection fix and network policy (!42)
Closes #34

## Summary

- Add `nav ssh-proxy` name mode (`--cluster`/`--name`) and `nav sandbox ssh-config` for VS Code Remote-SSH integration
- Fix VS Code Remote-SSH platform misdetection (was defaulting to Windows/PowerShell)
- Fix sandbox network policy for VS Code server connectivity
- Fix sandbox user shell configuration

## Problem

VS Code Remote-SSH was failing to connect to Navigator sandboxes due to three issues:

1. **Platform misdetection**: VS Code sends `uname -rsv` after a `ready:` marker and expects a response within ~1ms. Forced PTY allocation in `shell_request`, interactive bash (`-i`), and leaked supervisor env vars added enough latency that VS Code timed out and defaulted to "Platform: windows", then tried `powershell`.
2. **Broken network policy**: The `vscode` entry in `dev-sandbox-policy.yaml` was at the YAML top-level instead of nested under `network_policies:` (indentation bug). The VS Code server binary also needed a glob entry.
3. **No login shell**: The sandbox user had `/usr/sbin/nologin` as its shell.

## Changes

| File | Change |
|------|--------|
| `crates/navigator-sandbox/src/ssh.rs` | Remove forced PTY in `shell_request`, remove `-i` flag from pipe exec, add `env_clear()` + explicit `SHELL`/`PATH` to both shell spawn paths |
| `deploy/docker/sandbox/Dockerfile.base` | Change sandbox user shell to `/bin/bash`, set home to `/sandbox`, add `.bashrc`/`.profile` |
| `dev-sandbox-policy.yaml` | Fix `vscode:` indentation under `network_policies:`, add `/sandbox/.vscode-server/*` binary glob |
| `crates/navigator-cli/src/main.rs` | Add `--cluster`/`--name` to `SshProxy`, add `SshConfig`/`Policy`/`Logs` subcommands (rebase merge) |
| `crates/navigator-cli/src/ssh.rs` | Update `print_ssh_config` to use `--cluster` flag |
| `examples/vscode-remote-sandbox.md` | User-facing example and docs |

## How to use

```bash
# Create a persistent sandbox
nav sandbox create --keep my-sandbox

# Generate SSH config
nav sandbox ssh-config my-sandbox >> ~/.ssh/config

# Open VS Code
code --remote ssh-remote+nav-my-sandbox /sandbox
```

## Testing

- All 307 tests pass
- Clippy clean on modified crates
2026-02-26 09:06:37 -08:00
Piotr Mlocek f869182d8a feat(sandbox): move inference execution to sandbox-local routing (!79) (!52)
> **🏗️ build-from-issue-agent**

Closes #79

## Summary

Replaces the gateway-proxied inference model (`ProxyInference` RPC) with sandbox-local execution. The sandbox now resolves routes from either a standalone YAML route file or a cluster bundle fetched via the new `GetSandboxInferenceBundle` RPC, then forwards requests directly to inference backends using `navigator-router`.

### Benefits of removing the gRPC hop

Moving inference execution from the gateway to the sandbox eliminates the gRPC round-trip for every inference request:

- **No payload size limits**: Direct HTTP forwarding replaces protobuf-over-gRPC serialization.
- **Streaming-ready**: Preserves connection semantics for future SSE support (e.g., `stream: true`).
- **Lower latency**: Direct sandbox → backend path instead of sandbox → gateway → backend round-trip.
- **Reduced gateway load**: Gateway only serves the control-plane bundle delivery RPC.
- **Simpler error model**: HTTP status codes flow through directly without gRPC wrapping.

## Changes Made

- **Proto**: Removed `ProxyInference` RPC. Added `GetSandboxInferenceBundle` RPC for route bundle delivery.
- **Gateway**: Replaced `proxy_inference` with `get_sandbox_inference_bundle`. Removed `Router` and `navigator-router` dependency — gateway is now control-plane only for inference.
- **Router**: Switched config file format from TOML to YAML. Added `Debug` impl that redacts API keys.
- **Sandbox**: New `InferenceContext` with local `Router` + route cache. Routes loaded from `--inference-routes` file (standalone) or cluster bundle via gRPC, with 30s background refresh. New `--inference-routes` / `NAVIGATOR_INFERENCE_ROUTES` CLI arg.
- **Dev sandbox**: Added `inference-routes.yaml` at repo root with default NVIDIA NIM route. `mise run sandbox` now mounts it automatically and supports `-e` flag to forward env vars (e.g., `mise run sandbox`).
- **Example**: Added standalone example (`examples/inference/routes.yaml`) and rewrote README to cover both standalone and cluster workflows.
- **E2E tests**: Updated existing test names/docs. Added tests for Anthropic messages protocol and multi-route policy filtering.

## Tests Added

- **Unit (21 new):** `router_error_to_http` (6 variants), `load_from_file` YAML round-trip (5), `resolve_sandbox_inference_bundle` gRPC handler (6), `build_inference_context` route loading (4)
- **E2E (2 new):** Anthropic messages protocol routing, route filtering by `allowed_routes`
- **Total:** 188 unit tests passing across 3 crates

## Documentation Updated

- `architecture/gateway.md`: Removed router, updated Inference Service
- `architecture/sandbox.md`: Updated orchestration flow, InferenceContext, route loading
- `architecture/inference-routing.md`: Complete rewrite for sandbox-local architecture

## Verification

- [x] All unit tests passing (188 across 3 crates)
- [x] Pre-commit Rust checks passing (fmt, clippy, tests)
- [x] Architecture documentation updated
- [x] E2E tests updated with new test cases
- [x] Standalone example and documentation added
- [x] Default inference-routes.yaml with env passthrough for mise run sandbox
2026-02-25 09:57:42 -08:00
Piotr Mlocek 34fd3cbf64 feat(inference): inference interception and routing (!38)
Closes #67

## Summary

Implements transparent inference interception and routing for sandboxed AI agents. The sandbox proxy intercepts outbound AI SDK calls (OpenAI, Anthropic) and reroutes them through the gateway to policy-controlled backends — enabling organizations to redirect inference traffic to local or self-hosted models without modifying agent code.

**Decision model** — a tri-state OPA evaluation for every CONNECT request:
1. Binary + endpoint explicitly allowed in `network_policies` → **allow** (pass through)
2. Not explicitly allowed + `inference.allowed_routes` configured → **inspect for inference** (TLS intercept, detect API patterns, route through gateway)
3. Otherwise → **deny**

No endpoint declarations or binary lists needed for inference routing. Just configure `inference.allowed_routes`.

## Key Changes

### Sandbox (interception)
- **OPA policy**: New `network_action` Rego rule with three outcomes (`allow`, `inspect_for_inference`, `deny`). New `NetworkAction` enum replaces `PolicyDecision.allowed` bool for the proxy's main decision path.
- **Proxy**: New `InspectForInference` path — TLS-terminates client, parses HTTP, detects inference API patterns (`POST /v1/chat/completions`, `/v1/completions`, `/v1/messages`), strips auth headers, forwards via gRPC.
- **New module**: `l7/inference.rs` — `InferenceApiPattern`, `detect_inference_pattern()`, HTTP request/response parsing.
- **gRPC client**: New `proxy_inference()` for sandbox→gateway forwarding.
- **Sandbox init**: Creates OPA engine when inference is configured, even without `network_policies`.

### Gateway (dispatch)
- **InferenceService**: `ProxyInference` RPC loads sandbox policy, resolves allowed routes, dispatches to router. Full CRUD for inference routes.
- **Proto**: `InferenceRoute`, `InferenceRouteSpec`, `ProxyInferenceRequest/Response`, Inference gRPC service.

### Router (backend proxying)
- **New crate**: `navigator-router` with `Router`, `proxy_with_candidates()`, protocol-based route selection, backend HTTP proxying with auth header rewriting.
- **Mock support** for testing (`mock://` scheme).

### CLI
- `nav inference create/update/delete/list` commands for route management.

### Python SDK
- Updated protobuf bindings. Removed old `inference.py` client (replaced by transparent interception).

### Documentation
- New `architecture/inference-routing.md` — end-to-end system documentation.
- Updated `architecture/sandbox.md` — proxy, OPA, and source index sections.
- Updated `architecture/README.md` — new subsystem overview and diagram.

## Addendum: Chunked Transfer Compatibility

This branch now also fixes intercepted SDK requests that send chunked request bodies:

- `inspect_for_inference` now accepts `Transfer-Encoding: chunked` and decodes chunked request bodies before forwarding to the gateway
- Removed the prior `411 Length Required` response for chunked intercepted requests
- Added request/response header sanitization for framing and hop-by-hop headers (`content-length`, `transfer-encoding`, `connection`, etc.) to keep forwarded requests and returned responses valid
- Added unit tests for chunked parsing and header sanitization

Note: this improves compatibility for streaming-style SDK request patterns; true token-by-token passthrough response streaming is still a separate follow-up.

## Minimal Policy for Inference Routing

```yaml
inference:
  allowed_routes:
    - local
```

Any outgoing connection from a binary not explicitly allowed in `network_policies` will be intercepted and checked for inference API patterns.

## Test Plan

- [x] `cargo test --workspace` — all tests pass
- [x] `mise run pre-commit` — all checks pass
- [x] E2E: OpenAI chat completions routed through gateway
- [x] E2E: Anthropic messages routed through gateway
- [x] E2E: Python OpenAI SDK from sandbox (`examples/inference/inference.py`)
- [x] E2E test: `e2e/python/test_inference_routing.py`
2026-02-23 11:48:48 -08:00
Drew Newberry 2f808eae7b feat(cli): replace rsync with tar-over-SSH for sandbox file sync (!41)
Closes #20, #36, #46

## Summary
- Replace the external `rsync` dependency with built-in tar-over-SSH for file synchronization, using the `tar` crate to stream archives through the existing SSH proxy tunnel
- Add a new `nav sandbox sync` subcommand supporting bidirectional file transfer (`--up` to push local files, `--down` to pull sandbox files)
- Add sync workflow example in `examples/sync-files.md`

## Changes

| File | Change |
|---|---|
| `crates/navigator-cli/Cargo.toml` | Add `tar = "0.4"` dependency |
| `crates/navigator-cli/src/ssh.rs` | Remove `sandbox_rsync()`, add `sandbox_sync_up()`, `sandbox_sync_down()`, `sandbox_sync_up_files()` |
| `crates/navigator-cli/src/run.rs` | Add `sandbox_sync_command()` dispatcher, update re-exports, update `--sync` call site |
| `crates/navigator-cli/src/main.rs` | Add `Sync` variant to `SandboxCommands` with `--up`/`--down` flags |
| `architecture/sandbox-connect.md` | Rewrite File Sync section for tar-over-SSH and new command |
| `examples/sync-files.md` | New example walkthrough for sync workflows |

## New UX

```bash
# Push local files up to sandbox
nav sandbox sync my-sandbox --up ./src /sandbox/src

# Pull sandbox files down to local
nav sandbox sync my-sandbox --down /sandbox/output ./output

# Existing --sync flag on create still works (now uses tar internally)
nav sandbox create --sync -- python main.py
```

## Design

- **No new host dependencies**: `tar` crate handles archive creation/extraction in Rust; `ssh` is already required. The sandbox side uses the `tar` binary from the base image.
- **No new server-side infrastructure**: reuses the existing SSH tunnel, no new gRPC RPCs or gateway changes.
- **Bidirectional**: the same tar-pipe pattern works for both push and pull by reversing which side produces and which consumes the archive.
- **No compression** for v1 — the SSH tunnel is local-network; can add gzip via `flate2` later.

## Test Plan
- All existing tests pass (`mise run pre-commit` green: fmt, clippy, 206 Rust tests, 6 Python tests)
- Manual e2e testing required for actual sandbox sync operations
2026-02-23 08:22:52 -08:00
Drew Newberry 6dc97171ed feat(sandbox): add image build/push and fix cluster deploy (!34)
Closes #44

## Summary

- Add `nav sandbox image push --dockerfile <path>` command to build and push custom container images into the cluster
- Fix sandbox Docker build failure caused by OpenClaw installer TTY access
- Fix sandbox proxy crash when network namespace is unavailable (fall back to loopback)
- Add e2e test for the full custom image build and sandbox creation flow
- Rename and rewrite BYOC architecture doc to reflect current state

## Changes

### `navigator-bootstrap`
- New `build` module (`src/build.rs`) with `build_and_push_image()` public API
- Builds via bollard `Docker::build_image()`, streams output to caller
- `.dockerignore` support with glob matching, 8 unit tests

### `navigator-cli`
- Added `SandboxImageCommands::Push` subcommand with `--dockerfile`, `--tag`, `--context`, `--build-arg` flags
- Added `test:e2e:custom-image` mise task

### `navigator-sandbox`
- Proxy now falls back to `127.0.0.1:3128` when no network namespace or explicit bind address is available, instead of erroring out

### Docker / Infra
- Sandbox Dockerfile: replaced OpenClaw native installer (`curl | bash`) with `npm install -g openclaw` to avoid `/dev/tty` failures in Docker build

### Documentation
- Renamed `architecture/sandbox-byoc.md` to `sandbox-custom-containers.md`
- Rewrote doc to describe current behavior (CLI usage, push pipeline, supervisor behavior, design decisions) instead of listing implementation changes

### Tests
- Added `e2e/bash/test_sandbox_custom_image.sh` covering image push + sandbox create with custom image
2026-02-22 20:32:18 -08:00
Drew Newberry 1c939a255a feat(sandbox): enable port forwarding and setup openclaw (!33) 2026-02-19 16:03:23 -08:00