* feat(sandbox): integrate OCSF structured logging for all sandbox events
WIP: Replace ad-hoc tracing calls with OCSF event builders across all
sandbox subsystems (network, SSH, process, filesystem, config, lifecycle).
- Register ocsf_logging_enabled setting (defaults false)
- Replace stdout/file fmt layers with OcsfShorthandLayer
- Add conditional OcsfJsonlLayer for /var/log/openshell-ocsf.log
- Update LogPushLayer to extract OCSF shorthand for gRPC push
- Migrate ~106 log sites to OCSF builders (NetworkActivity, HttpActivity,
SshActivity, ProcessActivity, DetectionFinding, ConfigStateChange,
AppLifecycle)
- Add openshell-ocsf to all Docker build contexts
* fix(scripts): attach provider to all smoke test phases to avoid rate limits
GitHub's unauthenticated API rate limit (60/hour) causes flaky 403s for
Phases 1, 2, and 4. Fix by attaching the provider to all sandboxes and
upgrading the Phase 1 policy to L7 so credential injection works.
Phase 4 (tls:skip) cannot inject credentials by design, so relax the
assertion to accept either 200 or 403 from upstream -- both prove the
proxy forwarded the request.
* fix(ocsf): remove timestamp from shorthand format to avoid double-timestamp
The display layer (gateway logs, TUI, sandbox logs CLI) already prepends
a timestamp. Having one in the shorthand output too produces redundant
double-timestamps like:
15:49:11 sandbox INFO 15:49:11.649 I NET:OPEN ALLOWED ...
Now the shorthand is just the severity + structured content:
15:49:11 sandbox INFO I NET:OPEN ALLOWED ...
* refactor(ocsf): replace single-char severity with bracketed labels
Replace cryptic single-character severity codes (I/L/M/H/C/F) with
readable bracketed labels: [LOW], [MED], [HIGH], [CRIT], [FATAL].
Informational severity (the happy-path default) is omitted entirely to
keep normal log output clean and avoid redundancy with the tracing-level
INFO that the display layer already provides.
Before: sandbox INFO I NET:OPEN ALLOWED ...
After: sandbox INFO NET:OPEN ALLOWED ...
Before: sandbox INFO M NET:OPEN DENIED ...
After: sandbox INFO [MED] NET:OPEN DENIED ...
* feat(sandbox): use OCSF level label for structured events in log push
Set the level field to 'OCSF' instead of 'INFO' for OCSF events in the
gRPC log push. This visually distinguishes structured OCSF events from
plain tracing output in the TUI and CLI sandbox logs:
sandbox OCSF NET:OPEN [INFO] ALLOWED python3(42) -> api.example.com:443
sandbox OCSF NET:OPEN [MED] DENIED python3(42) -> blocked.com:443
sandbox INFO Fetching sandbox policy via gRPC
* fix(sandbox): convert new Landlock path-skip warning to OCSF
PR #677 added a warn!() for inaccessible Landlock paths in best-effort
mode. Convert to ConfigStateChangeBuilder with degraded state so it
flows through the OCSF shorthand format consistently.
* fix(sandbox): use rolling appender for OCSF JSONL file
Match the main openshell.log rotation mechanics (daily, 3 files max)
instead of a single unbounded append-only file. Prevents disk exhaustion
when ocsf_logging_enabled is left on in long-running sandboxes.
* fix(sandbox): address reviewer warnings for OCSF integration
W1: Remove redundant 'OCSF' prefix from shorthand file layer — the
class name (NET:OPEN, HTTP:GET) already identifies structured events
and the LogPushLayer separately sets the level field.
W2: Log a debug message when OCSF_CTX.set() is called a second time
instead of silently discarding via let _.
W3: Document the boundary between OCSF-migrated events and intentionally
plain tracing calls (DEBUG/TRACE, transient, internal plumbing).
W4: Migrate remaining iptables LOG rule failure warnings in netns.rs
(IPv4 TCP/UDP, IPv6 TCP/UDP) to ConfigStateChangeBuilder for
consistency with the IPv4 bypass rule failure already migrated.
W5: Migrate malformed inference request warn to NetworkActivity with
ActivityId::Refuse and SeverityId::Medium.
W6: Use Medium severity for L7 deny decisions (both CONNECT tunnel and
FORWARD proxy paths) to match the CONNECT deny severity pattern.
Allows and audits remain Informational.
* refactor(sandbox): rename ocsf_logging_enabled to ocsf_json_enabled
The shorthand logs are already OCSF-structured events. The setting
specifically controls the JSONL file export, so the name should reflect
that: ocsf_json_enabled.
* fix(ocsf): add timestamps to shorthand file layer output
The OcsfShorthandLayer writes directly to the log file with no outer
display layer to supply timestamps. Add a UTC timestamp prefix to every
line so the file output matches what tracing::fmt used to provide.
Before: CONFIG:VALIDATED [INFO] Validated 'sandbox' user exists in image
After: 2026-04-01T15:49:11.649Z CONFIG:VALIDATED [INFO] Validated ...
* fix(docker): touch openshell-ocsf source to invalidate cargo cache
The supervisor-workspace stage touches sandbox and core sources to force
recompilation over the rust-deps dummy stubs, but openshell-ocsf was
missing. This caused the Docker cargo cache to use stale ocsf objects
from the deps stage, preventing changes to the ocsf crate (like the
timestamp fix) from appearing in the final binary.
Also adds a shorthand layer test verifying timestamp output, and drafts
the observability docs section.
* fix(ocsf): add OCSF level prefix to file layer shorthand output
Without a level prefix, OCSF events in the log file have no visual
anchor at the position where standard tracing lines show INFO/WARN.
This makes scanning the file harder since the eye has nothing consistent
to lock onto after the timestamp.
Before: 2026-04-01T04:04:13.065Z CONFIG:DISCOVERY [INFO] ...
After: 2026-04-01T04:04:13.065Z OCSF CONFIG:DISCOVERY [INFO] ...
* fix(ocsf): clean up shorthand formatting for listen and SSH events
- Fix double space in NET:LISTEN, SSH:LISTEN, and other events where
action is empty (e.g., 'NET:LISTEN [INFO] 10.200.0.1' -> 'NET:LISTEN [INFO] 10.200.0.1')
- Add listen address to SSH:LISTEN event (was empty)
- Downgrade SSH handshake intermediate steps (reading preface, verifying)
from OCSF events to debug!() traces. Only the final verdict
(accepted/denied) is an OCSF event now, reducing noise from 3 events
to 1 per SSH connection.
- Apply same spacing fix to HTTP shorthand for consistency.
* docs(observability): update examples with OCSF prefix and formatting fixes
Align doc examples with the deployed output:
- Add OCSF level prefix to all shorthand examples in the log file
- Show mixed OCSF + standard tracing in the file format section
- Update listen events (no double space, SSH includes address)
- Show one SSH:OPEN per connection instead of three
- Update grep patterns to use 'OCSF NET:' etc.
* docs(agents): add OCSF logging guidance to AGENTS.md
Add a Sandbox Logging (OCSF) section to AGENTS.md so agents have
in-context guidance for deciding whether new log emissions should use
OCSF structured logging or plain tracing. Covers event class selection,
severity guidelines, builder API usage, dual-emit pattern for security
findings, and the no-secrets rule.
Also adds openshell-ocsf to the Architecture Overview table.
* fix: remove workflow files accidentally included during rebase
These files were already merged to main in separate PRs. They got
pulled into our branch during rebase conflict resolution for the
deleted docs-preview-pr.yml file.
* docs(observability): use sandbox connect instead of raw SSH
Users access sandboxes via 'openshell sandbox connect', not direct SSH.
* fix(docs): correct settings CLI syntax in OCSF JSON export page
The settings CLI requires --key and --value named flags, not positional
arguments. Also fix the per-sandbox form: the sandbox name is a
positional argument, not a --sandbox flag.
* fix(e2e): update log assertions for OCSF shorthand format
The E2E tests asserted on the old tracing::fmt key=value format
(action=allow, l7_decision=audit, FORWARD, L7_REQUEST, always-blocked).
Update to match the new OCSF shorthand (ALLOWED/DENIED, HTTP:, NET:,
engine:ssrf, policy:).
* feat(sandbox): convert WebSocket upgrade log calls to OCSF
PR #718 added two log calls for WebSocket upgrade handling:
- 101 Switching Protocols info → NetworkActivity with Upgrade activity.
This is a significant state change (L7 enforcement drops to raw relay).
- Unsolicited 101 without client Upgrade header → DetectionFinding with
High severity. A non-compliant upstream sending 101 without a client
Upgrade request could be attempting to bypass L7 inspection.
* fix(install): restrict tar extraction to expected binary member
Prevents CWE-22 path traversal by extracting only the expected APP_NAME
member instead of the full archive contents. Adds --no-same-owner and
--no-same-permissions for defense-in-depth.
OS-20
* fix(deploy): quote registry credentials in YAML heredocs
Wraps username/password values with a yaml_quote helper to prevent YAML
injection from special characters in registry credentials (CWE-94).
Applied to all three heredoc blocks that emit registries.yaml auth.
OS-23
* fix(server): redact session token in SSH tunnel rate-limit log
Logs only the last 4 characters of bearer tokens to prevent credential
exposure in log aggregation systems (CWE-532).
OS-18
* fix(server): escape gateway_display in auth connect page
Applies html_escape() to the Host/X-Forwarded-Host header value before
rendering it into the HTML template, preventing HTML injection (CWE-79).
OS-17
* fix(server): prevent XSS via code param with validation and proper JS escaping
Adds server-side validation rejecting confirmation codes that do not
match the CLI-generated format, replaces manual JS string escaping with
serde_json serialization (handling U+2028/U+2029 line terminators), and
adds a Content-Security-Policy header with nonce-based script-src.
OS-16
* fix(sandbox): add byte cap and idle timeout to streaming inference relay
Prevents resource exhaustion from upstream inference endpoints that stream
indefinitely or hold connections open. Adds a 32 MiB total body limit
and 30-second per-chunk idle timeout (CWE-400).
OS-21
* fix(policy): narrow port field from u32 to u16 to reject invalid values
Prevents meaningless port values >65535 from being accepted in policy
YAML definitions. The proto field remains uint32 (protobuf has no u16)
with validation at the conversion boundary.
OS-22
* fix(deps): migrate from archived serde_yaml to serde_yml
Replaces serde_yaml 0.9 (archived, RUSTSEC-2024-0320) with serde_yml
0.0.12, a maintained API-compatible fork. All import sites updated
across openshell-policy, openshell-sandbox, and openshell-router.
OS-19
* fix(server): re-validate sandbox-submitted security_notes and cap hit_count
The gateway now re-runs security heuristics on proposed policy chunks
instead of trusting sandbox-provided security_notes, validates host
wildcards, caps hit_count at 100, and clamps confidence to [0,1]. The
TUI approve-all path is updated to use ApproveAllDraftChunks RPC which
respects the security_notes filtering gate (CWE-284, confused deputy).
OS-15
* chore: apply cargo fmt and update Cargo.lock for serde_yml
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Detect 101 Switching Protocols in relay_response() and switch to raw
bidirectional TCP relay instead of re-entering the HTTP parsing loop.
Previously, is_bodiless_response() treated 101 as a generic 1xx
informational response, forwarding only the headers and returning to
the HTTP parsing loop. After a 101, subsequent bytes are upgraded
protocol frames (e.g. WebSocket), not HTTP — causing the relay to
block or silently drop all post-upgrade traffic.
Changes:
- Add RelayOutcome enum (Reusable/Consumed/Upgraded) replacing bool
return type across L7Provider::relay trait and all relay functions
- Detect 101 before generic 1xx handler in relay_response(), capture
overflow bytes, return RelayOutcome::Upgraded
- Validate client sent Upgrade + Connection: Upgrade headers before
accepting 101 (rejects unsolicited upgrades from non-compliant
upstream servers)
- Extract shared handle_upgrade() helper used by both relay_rest()
and relay_passthrough_with_credentials()
- Add l7_decision=allow_upgrade audit log annotation for upgrades
- Add unit tests for 101 overflow capture, unsolicited 101 rejection,
and client_requested_upgrade header validation
- Add integration test: WebSocket echo through L7Provider::relay
Fixes: #652
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The image push pipeline buffered the entire Docker image tar 3x in
memory (export, tar wrap, Bytes copy), causing OOM kills for images
over ~1-2 GB. Replace the in-memory pipeline with a temp-file +
streaming upload: export to a NamedTempFile, then stream the outer
tar (header, 8 MiB file chunks, footer) directly into
upload_to_container via body_try_stream. Peak memory drops from ~3x
image size to ~8 MiB constant.
Also adds incremental export progress reporting every 100 MiB.
Closes#273
Verify inference endpoints synchronously on the server during set/update, expose a --no-verify escape hatch in the CLI and Python helper, and return actionable failures when validation does not pass.
* feat(tui): add log copy and visual selection mode
Add clipboard support for the TUI log viewer using OSC 52 escape
sequences. Users can copy individual log lines (y), the visible
viewport (Y), or enter visual selection mode (v) to select a range
of lines with j/k and yank with y.
- Add clipboard module with OSC 52 base64 encoding
- Add format_log_line_plain() for plain-text log rendering
- Add visual selection mode with highlight styling and status bar
- Context-switch nav bar hints between normal and visual modes
- Add 7 unit tests for plain-text log formatting
Closes#274
* fix(tui): write OSC 52 to /dev/tty instead of stdout
ratatui owns stdout via the alternate screen buffer, so OSC 52
escape sequences written to stdout are swallowed by the backend.
Write directly to /dev/tty to bypass the buffer and reach the
terminal emulator.
* feat(tui): support light terminal backgrounds with adaptive theme
Replace the hardcoded dark-only color constants in theme.rs with a
runtime Theme struct that has dark() and light() factory constructors.
The active theme is stored on App and threaded through all draw
functions, enabling the TUI to render correctly on both dark and light
terminal backgrounds.
- Add Theme struct with 16 semantic style fields and ThemeMode enum
- Detect terminal background via COLORFGBG env var at startup
- Add --theme dark|light|auto CLI flag and OPENSHELL_THEME env var
- Migrate all 499 styles:: references across 11 UI files to app.theme
- Clean up 9 inline Style constructions that bypassed the theme system
- Add unit tests verifying dark/light palettes and legacy regression
Closes#264
* fix(tui): use OSC 11 query for reliable light/dark terminal detection
Replace COLORFGBG env var heuristic with terminal-colorsaurus crate,
which sends an OSC 11 query to read the actual background color. This
fixes auto-detection in iTerm2 and other terminals that don't set
COLORFGBG.
* feat(policy): add validation layer to reject unsafe sandbox policies
Add policy validation that checks for root process identity, path
traversal sequences, overly broad filesystem paths, and exceeding
filesystem rule limits. Validation runs at three entry points:
disk-loaded YAML policies (fallback to restrictive default on violation),
gRPC CreateSandbox, and gRPC UpdateSandboxPolicy (returns
INVALID_ARGUMENT). Filesystem paths are normalized before storage to
collapse traversal components.
Closes#33
* fix(e2e): correct policy update test to match immutable field behavior
The update policy test was asserting on validation errors for fields
(process, filesystem) that are immutable on live sandboxes. The server
rejects changes to these fields before validation runs. Updated the test
to verify the immutability guard instead.
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(tui): add port forwarding support to Gator (#80)
Extract forward PID management, resolve_ssh_gateway, and shell_escape from
navigator-cli into navigator-core::forward as a shared module. Add a Ports
field to the TUI create sandbox modal, a NOTES column to the sandbox table,
and a Forwards row to the sandbox detail view. On creation with ports, Gator
polls for Ready state then spawns background SSH tunnels. Forward cleanup
runs automatically on sandbox delete.
Closes#80
* fix(tui): fix Ports field invisible due to modal height overflow
The modal height calculation used content_height + 3 but block chrome
(borders + padding) is 4 rows, not 3. Combined with the 3-row Ports
field layout, the input line was clipped to zero height on standard
24-row terminals. Switch Ports to a compact single-line inline layout
and fix the chrome arithmetic.
* wip: BYOC example with port forwarding and TUI command field fix
- Move examples/bring-your-own-container.md into its own directory
- Add example Dockerfile and app.py (Python REST API with /hello endpoint)
- Rewrite README with full CLI and TUI port-forward workflows
- Change TUI Command field default from /bin/bash to empty so custom
image entrypoints run without manual clearing
* feat(tui): wire up Command field to SSH exec after sandbox creation
When a command is specified in the create sandbox modal, the TUI now
waits for the sandbox to reach Ready (with the pacman animation),
starts any port forwards, then suspends the TUI and executes the
command via SSH — matching the CLI's post-creation flow.
- Refactor spawn_create_sandbox to poll for Ready inline when ports
or command are set, keeping the animation visible throughout
- Add start_port_forwards() helper called within the create task
- Add handle_exec_command() to suspend TUI and run SSH exec
- Remove unused ForwardResult event variant (forwards are now started
within the create task, not as a separate event)
* fix(tui): fix command exec, forward timeouts, and create modal UX
- Fix handle_exec_command to suspend TUI and run SSH attached (matching
CLI behavior) so the process stays alive for the session duration
- Fix shell_escape double-quoting: escape each word individually
- Add ConnectTimeout and 20s spawn timeout to forward SSH so a stalled
auth doesn't freeze the create flow forever
- Remove dead ForwardResult event variant
- Add cluster_name to start_port_forwards for ProxyCommand
- Add spacer between Providers and Ports in create modal
- Fix Command placeholder: 'runs /bin/bash if empty' (not image entrypoint)
- Update BYOC README: document that CMD is replaced by supervisor,
command must be passed explicitly, remove TUI-specific sections
* fix(sandbox): demote Landlock fallback log from warn to debug
The Landlock filesystem sandbox emits a noisy warning when a policy
path does not exist (e.g. /app in custom images). Since BestEffort
mode intentionally continues without Landlock, demote to debug.
* docs: fix BYOC Dockerfile comments for supervisor CMD override
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(sandbox): allow egress to private IP space via allowed_ips policy field
Add an allowed_ips field to NetworkEndpoint that accepts CIDR notation and
exact IPs, enabling sandboxes to reach private IP space under policy control
while maintaining SSRF protections for loopback and link-local addresses.
Three modes are supported:
- Default (no allowed_ips): all private IPs blocked (existing behavior)
- Host + allowlist: domain must resolve to an IP in allowed_ips
- Hostless allowlist (no host): any domain allowed if resolved IP matches
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
> **🏗️ build-from-issue-agent**
Closes#71
## Summary
Add dynamic shell completion support using `clap_complete`'s `CompleteEnv`, plus a `navigator completions <shell>` subcommand (rustup-style) that outputs completion scripts to stdout with detailed per-shell setup instructions in `--help`.
Completions are dynamic — the shell calls back into the `navigator` binary on each tab-press, so they are always in sync with the installed version. This also lays the groundwork for future runtime completers (e.g., listing sandbox names on tab, tracked in #83).
## Changes Made
- `Cargo.toml`: Added `clap_complete = { version = "4.5", features = ["unstable-dynamic"] }` to workspace dependencies
- `crates/navigator-cli/Cargo.toml`: Added `clap_complete` dependency
- `crates/navigator-cli/src/main.rs`:
- Added `CompleteEnv::with_factory(Cli::command).complete()` at the top of `main()` (before TLS/argument parsing)
- Added `Completions { shell }` subcommand that re-invokes the binary with `COMPLETE=<shell>` and pipes the output
- Added `CompletionShell` enum (bash, fish, zsh, powershell)
- Added `COMPLETIONS_HELP` with per-shell setup instructions (similar to `rustup completions --help`)
- Added 3 unit tests
- `CONTRIBUTING.md`: Added shell completions section documenting `nav` wrapper setup
## Usage
```bash
# Generate completions
navigator completions fish > ~/.config/fish/completions/navigator.fish
navigator completions bash > ~/.local/share/bash-completion/completions/navigator
navigator completions zsh > ~/.zfunc/_navigator
navigator completions powershell >> $PROFILE
# See full per-shell instructions
navigator completions --help
# For the nav wrapper (rewrite registration target)
navigator completions fish | sed 's/--command navigator/--command nav/' > ~/.config/fish/completions/nav.fish
```
## Design Decision: Why stdout-only (no auto-install)
We researched how 12 popular CLI tools handle completion installation:
| Tool | Command | Writes files? | Modifies RC? |
|------|---------|---------------|-------------|
| gh, glab, kubectl, docker, rustup, mise, starship | `completion <shell>` | stdout only | No |
| atuin | `gen-completions --shell <shell>` | stdout (`--out-dir` option) | No |
| gcloud | `install.sh` | Yes | **Yes** |
| Amazon Q | `q integrations install` | Yes | **Yes** |
**10 of 12 tools** follow the same pattern: output to stdout, never modify RC files, explicit shell selection, clear `--help` instructions. Only gcloud and Amazon Q modify RC files (and both have been criticized for it). Our approach matches the industry consensus.
## Deviations from Plan
- The original issue proposed a static `completions <shell>` subcommand using `clap_complete::generate()`. After discussion, we switched to dynamic `CompleteEnv` which is always up-to-date and enables future runtime completers.
- Added a `navigator completions <shell>` subcommand (not in original plan) for discoverability — it wraps the `COMPLETE` env var mechanism with a familiar CLI UX.
## Tests Added
- **Unit:** `cli_debug_assert` — validates CLI structure; `completions_engine_returns_candidates` — smoke test confirming the completion engine returns subcommand candidates; `completions_subcommand_appears_in_candidates` — verifies `completions` appears in tab-completion results
- **Integration:** N/A
- **E2E:** N/A
## Verification
- [x] All tests passing
- [x] Pre-commit checks passing (clippy warnings are pre-existing, not from this change)
- [x] CONTRIBUTING.md updated with nav wrapper completion setup
Closes#78
## What You Can Do
### Live policy updates
Update a running sandbox's network policy without recreating it:
```bash
nav sandbox policy set <sandbox> --policy new-policy.yaml --wait --timeout 60
```
The sandbox hot-reloads the policy within 30s (configurable). On failure, the previous policy stays active (last-known-good).
Idempotent — submitting the same policy twice returns the existing version:
```
✓ Policy version 3 submitted (hash: a1b2c3d4e5f6)
$ nav sandbox policy set test --policy same.yaml
· Policy unchanged (version 3, hash: a1b2c3d4e5f6)
```
### Policy history & inspection
```bash
nav sandbox policy list <sandbox> # version history with status
nav sandbox policy get <sandbox> # current policy metadata
nav sandbox policy get <sandbox> --full # print full policy as YAML
nav sandbox policy get <sandbox> --rev 2 --full # specific revision as YAML
```
### Sandbox logs
Stream logs from both the gateway and sandbox supervisor in one view:
```bash
nav sandbox logs <sandbox> # one-shot, last 2000 lines
nav sandbox logs <sandbox> --tail # live streaming
nav sandbox logs <sandbox> --source sandbox # supervisor logs only
nav sandbox logs <sandbox> --source gateway # gateway logs only
nav sandbox logs <sandbox> --level warn # warnings and errors only
nav sandbox logs <sandbox> --since 5m # last 5 minutes
nav sandbox logs <sandbox> --tail --source sandbox --level info
```
Each log line is tagged with its source and includes structured fields:
```
[1772055394.673] [sandbox] [INFO ] [navigator_sandbox::proxy] CONNECT action=allow dst_host=api.anthropic.com dst_port=443 policy=claude_code
[1772055061.005] [gateway] [INFO ] [navigator_server::grpc] GetSandboxPolicy served from policy history
```
---
## Implementation
### Proto changes
- 7 new RPCs: `UpdateSandboxPolicy`, `GetSandboxPolicyStatus`, `ListSandboxPolicies`, `ReportPolicyStatus`, `GetSandboxLogs`, `PushSandboxLogs` (client-streaming)
- `SandboxLogLine`: added `source` (gateway/sandbox), `fields` (structured key-value map)
- `WatchSandboxRequest`: added `log_since_ms`, `log_sources`, `log_min_level`
- `GetSandboxLogsRequest`: added `sources`, `min_level`
- `PolicyStatus` enum, `SandboxPolicyRevision` message
- `Sandbox.current_policy_version` field
### Server
- **Policy persistence**: New `sandbox_policies` table (SQLite + Postgres) with per-sandbox monotonic versions, status tracking, and policy hash
- **UpdateSandboxPolicy**: Validates static field immutability (filesystem/landlock/process), network mode consistency (Block↔Proxy), deterministic hash comparison for idempotent updates
- **Lazy backfill**: First `GetSandboxPolicy` call creates version 1 from `spec.policy` for existing sandboxes
- **Log broker**: `TracingLogBus::publish_external()` injects sandbox-pushed logs into the same broadcast channel + tail buffer (2000 lines). Server forces `source="sandbox"` and `sandbox_id` on all pushed logs
- **Source/level filtering**: Applied server-side in both `GetSandboxLogs` and `WatchSandbox` streams
- **Version supersession**: When a new version is loaded, all older pending+loaded versions are marked superseded
### Sandbox
- **`OpaEngine::reload_from_proto()`**: Full `from_proto()` pipeline (L7 validation, access preset expansion) with atomic engine swap. On failure, previous engine untouched (LKG)
- **Policy poll loop**: Background task polls every 30s (configurable via `NAVIGATOR_POLICY_POLL_INTERVAL_SECS`), reports status via `ReportPolicyStatus` RPC
- **`LogPushLayer`**: Tracing layer captures events at INFO+ (configurable via `NAVIGATOR_LOG_PUSH_LEVEL`), sends structured fields via `PushSandboxLogs` client-streaming RPC. Background task batches 50 lines / flushes every 500ms. Best-effort (drops on full channel, never blocks)
- **`CachedNavigatorClient`**: Persistent mTLS channel for both policy polling and log push
### Database migration
- `002_create_sandbox_policies.sql` (SQLite + Postgres)
## Tests
- **Unit**: 8 policy persistence tests (put/get/list/status/supersede/isolation)
- **Integration**: 4 test files updated with new RPC stubs
- **E2E**: `test_live_policy_update_and_logs` — full lifecycle: create → set same (unchanged) → push new → wait for load → verify connectivity → push same (unchanged) → verify history → fetch logs
## Documentation
- `architecture/sandbox.md`: Log streaming architecture, LogPushLayer, push task, server broker, source tagging, structured fields, CLI filtering, failure modes
- `architecture/security-policy.md`: Live update semantics, deterministic hashing, CLI filter flags, policy inspection
- `architecture/plans/issue-78-sandbox-log-streaming.md`: Design plan for log streaming
## Security
- Trust boundary documented: shared mTLS cert model (per-sandbox auth tracked in #80)
- Server forces `source="sandbox"` and `sandbox_id` on pushed logs (can't impersonate gateway or other sandboxes)
- Per-batch line cap (100) prevents flooding
> **🏗️ build-from-issue-agent**
Closes#79
## Summary
Replaces the gateway-proxied inference model (`ProxyInference` RPC) with sandbox-local execution. The sandbox now resolves routes from either a standalone YAML route file or a cluster bundle fetched via the new `GetSandboxInferenceBundle` RPC, then forwards requests directly to inference backends using `navigator-router`.
### Benefits of removing the gRPC hop
Moving inference execution from the gateway to the sandbox eliminates the gRPC round-trip for every inference request:
- **No payload size limits**: Direct HTTP forwarding replaces protobuf-over-gRPC serialization.
- **Streaming-ready**: Preserves connection semantics for future SSE support (e.g., `stream: true`).
- **Lower latency**: Direct sandbox → backend path instead of sandbox → gateway → backend round-trip.
- **Reduced gateway load**: Gateway only serves the control-plane bundle delivery RPC.
- **Simpler error model**: HTTP status codes flow through directly without gRPC wrapping.
## Changes Made
- **Proto**: Removed `ProxyInference` RPC. Added `GetSandboxInferenceBundle` RPC for route bundle delivery.
- **Gateway**: Replaced `proxy_inference` with `get_sandbox_inference_bundle`. Removed `Router` and `navigator-router` dependency — gateway is now control-plane only for inference.
- **Router**: Switched config file format from TOML to YAML. Added `Debug` impl that redacts API keys.
- **Sandbox**: New `InferenceContext` with local `Router` + route cache. Routes loaded from `--inference-routes` file (standalone) or cluster bundle via gRPC, with 30s background refresh. New `--inference-routes` / `NAVIGATOR_INFERENCE_ROUTES` CLI arg.
- **Dev sandbox**: Added `inference-routes.yaml` at repo root with default NVIDIA NIM route. `mise run sandbox` now mounts it automatically and supports `-e` flag to forward env vars (e.g., `mise run sandbox`).
- **Example**: Added standalone example (`examples/inference/routes.yaml`) and rewrote README to cover both standalone and cluster workflows.
- **E2E tests**: Updated existing test names/docs. Added tests for Anthropic messages protocol and multi-route policy filtering.
## Tests Added
- **Unit (21 new):** `router_error_to_http` (6 variants), `load_from_file` YAML round-trip (5), `resolve_sandbox_inference_bundle` gRPC handler (6), `build_inference_context` route loading (4)
- **E2E (2 new):** Anthropic messages protocol routing, route filtering by `allowed_routes`
- **Total:** 188 unit tests passing across 3 crates
## Documentation Updated
- `architecture/gateway.md`: Removed router, updated Inference Service
- `architecture/sandbox.md`: Updated orchestration flow, InferenceContext, route loading
- `architecture/inference-routing.md`: Complete rewrite for sandbox-local architecture
## Verification
- [x] All unit tests passing (188 across 3 crates)
- [x] Pre-commit Rust checks passing (fmt, clippy, tests)
- [x] Architecture documentation updated
- [x] E2E tests updated with new test cases
- [x] Standalone example and documentation added
- [x] Default inference-routes.yaml with env passthrough for mise run sandbox
Closes#48, #52
## Summary
- Replace the envoy-gateway-based TLS setup with inline PKI generation during cluster bootstrap, generating CA, server, and client certificates directly in the `navigator-bootstrap` crate
- Remove all envoy gateway Helm templates (`gateway.yaml`, `gatewayclass.yaml`, `grpcroute.yaml`, PKI job, traffic policies) and the `Dockerfile.pki-job`
- Add native mTLS support to the navigator server with `tokio-rustls`, mounting client TLS certs as volumes into sandbox pods
- Update cluster entrypoint, healthcheck, and deploy scripts to work with the new direct-TLS architecture
- Add TLS security e2e test and fix formatting/clippy warnings
## Test Plan
- All unit tests pass (`cargo test --workspace`)
- Clippy clean (`cargo clippy --workspace --all-targets`)
- Format clean (`cargo fmt --all -- --check`)
- Python tests pass (`uv run pytest python/`)
- Full `mise run pre-commit` passes
Closes#67
## Summary
Implements transparent inference interception and routing for sandboxed AI agents. The sandbox proxy intercepts outbound AI SDK calls (OpenAI, Anthropic) and reroutes them through the gateway to policy-controlled backends — enabling organizations to redirect inference traffic to local or self-hosted models without modifying agent code.
**Decision model** — a tri-state OPA evaluation for every CONNECT request:
1. Binary + endpoint explicitly allowed in `network_policies` → **allow** (pass through)
2. Not explicitly allowed + `inference.allowed_routes` configured → **inspect for inference** (TLS intercept, detect API patterns, route through gateway)
3. Otherwise → **deny**
No endpoint declarations or binary lists needed for inference routing. Just configure `inference.allowed_routes`.
## Key Changes
### Sandbox (interception)
- **OPA policy**: New `network_action` Rego rule with three outcomes (`allow`, `inspect_for_inference`, `deny`). New `NetworkAction` enum replaces `PolicyDecision.allowed` bool for the proxy's main decision path.
- **Proxy**: New `InspectForInference` path — TLS-terminates client, parses HTTP, detects inference API patterns (`POST /v1/chat/completions`, `/v1/completions`, `/v1/messages`), strips auth headers, forwards via gRPC.
- **New module**: `l7/inference.rs` — `InferenceApiPattern`, `detect_inference_pattern()`, HTTP request/response parsing.
- **gRPC client**: New `proxy_inference()` for sandbox→gateway forwarding.
- **Sandbox init**: Creates OPA engine when inference is configured, even without `network_policies`.
### Gateway (dispatch)
- **InferenceService**: `ProxyInference` RPC loads sandbox policy, resolves allowed routes, dispatches to router. Full CRUD for inference routes.
- **Proto**: `InferenceRoute`, `InferenceRouteSpec`, `ProxyInferenceRequest/Response`, Inference gRPC service.
### Router (backend proxying)
- **New crate**: `navigator-router` with `Router`, `proxy_with_candidates()`, protocol-based route selection, backend HTTP proxying with auth header rewriting.
- **Mock support** for testing (`mock://` scheme).
### CLI
- `nav inference create/update/delete/list` commands for route management.
### Python SDK
- Updated protobuf bindings. Removed old `inference.py` client (replaced by transparent interception).
### Documentation
- New `architecture/inference-routing.md` — end-to-end system documentation.
- Updated `architecture/sandbox.md` — proxy, OPA, and source index sections.
- Updated `architecture/README.md` — new subsystem overview and diagram.
## Addendum: Chunked Transfer Compatibility
This branch now also fixes intercepted SDK requests that send chunked request bodies:
- `inspect_for_inference` now accepts `Transfer-Encoding: chunked` and decodes chunked request bodies before forwarding to the gateway
- Removed the prior `411 Length Required` response for chunked intercepted requests
- Added request/response header sanitization for framing and hop-by-hop headers (`content-length`, `transfer-encoding`, `connection`, etc.) to keep forwarded requests and returned responses valid
- Added unit tests for chunked parsing and header sanitization
Note: this improves compatibility for streaming-style SDK request patterns; true token-by-token passthrough response streaming is still a separate follow-up.
## Minimal Policy for Inference Routing
```yaml
inference:
allowed_routes:
- local
```
Any outgoing connection from a binary not explicitly allowed in `network_policies` will be intercepted and checked for inference API patterns.
## Test Plan
- [x] `cargo test --workspace` — all tests pass
- [x] `mise run pre-commit` — all checks pass
- [x] E2E: OpenAI chat completions routed through gateway
- [x] E2E: Anthropic messages routed through gateway
- [x] E2E: Python OpenAI SDK from sandbox (`examples/inference/inference.py`)
- [x] E2E test: `e2e/python/test_inference_routing.py`
Closes#20, #36, #46
## Summary
- Replace the external `rsync` dependency with built-in tar-over-SSH for file synchronization, using the `tar` crate to stream archives through the existing SSH proxy tunnel
- Add a new `nav sandbox sync` subcommand supporting bidirectional file transfer (`--up` to push local files, `--down` to pull sandbox files)
- Add sync workflow example in `examples/sync-files.md`
## Changes
| File | Change |
|---|---|
| `crates/navigator-cli/Cargo.toml` | Add `tar = "0.4"` dependency |
| `crates/navigator-cli/src/ssh.rs` | Remove `sandbox_rsync()`, add `sandbox_sync_up()`, `sandbox_sync_down()`, `sandbox_sync_up_files()` |
| `crates/navigator-cli/src/run.rs` | Add `sandbox_sync_command()` dispatcher, update re-exports, update `--sync` call site |
| `crates/navigator-cli/src/main.rs` | Add `Sync` variant to `SandboxCommands` with `--up`/`--down` flags |
| `architecture/sandbox-connect.md` | Rewrite File Sync section for tar-over-SSH and new command |
| `examples/sync-files.md` | New example walkthrough for sync workflows |
## New UX
```bash
# Push local files up to sandbox
nav sandbox sync my-sandbox --up ./src /sandbox/src
# Pull sandbox files down to local
nav sandbox sync my-sandbox --down /sandbox/output ./output
# Existing --sync flag on create still works (now uses tar internally)
nav sandbox create --sync -- python main.py
```
## Design
- **No new host dependencies**: `tar` crate handles archive creation/extraction in Rust; `ssh` is already required. The sandbox side uses the `tar` binary from the base image.
- **No new server-side infrastructure**: reuses the existing SSH tunnel, no new gRPC RPCs or gateway changes.
- **Bidirectional**: the same tar-pipe pattern works for both push and pull by reversing which side produces and which consumes the archive.
- **No compression** for v1 — the SSH tunnel is local-network; can add gzip via `flate2` later.
## Test Plan
- All existing tests pass (`mise run pre-commit` green: fmt, clippy, 206 Rust tests, 6 Python tests)
- Manual e2e testing required for actual sandbox sync operations
Closes#44
## Summary
- Add `nav sandbox image push --dockerfile <path>` command to build and push custom container images into the cluster
- Fix sandbox Docker build failure caused by OpenClaw installer TTY access
- Fix sandbox proxy crash when network namespace is unavailable (fall back to loopback)
- Add e2e test for the full custom image build and sandbox creation flow
- Rename and rewrite BYOC architecture doc to reflect current state
## Changes
### `navigator-bootstrap`
- New `build` module (`src/build.rs`) with `build_and_push_image()` public API
- Builds via bollard `Docker::build_image()`, streams output to caller
- `.dockerignore` support with glob matching, 8 unit tests
### `navigator-cli`
- Added `SandboxImageCommands::Push` subcommand with `--dockerfile`, `--tag`, `--context`, `--build-arg` flags
- Added `test:e2e:custom-image` mise task
### `navigator-sandbox`
- Proxy now falls back to `127.0.0.1:3128` when no network namespace or explicit bind address is available, instead of erroring out
### Docker / Infra
- Sandbox Dockerfile: replaced OpenClaw native installer (`curl | bash`) with `npm install -g openclaw` to avoid `/dev/tty` failures in Docker build
### Documentation
- Renamed `architecture/sandbox-byoc.md` to `sandbox-custom-containers.md`
- Rewrote doc to describe current behavior (CLI usage, push pipeline, supervisor behavior, design decisions) instead of listing implementation changes
### Tests
- Added `e2e/bash/test_sandbox_custom_image.sh` covering image push + sandbox create with custom image
## Summary
- Add `Provider` entity for managing 3p deps from a sandbox
- Add provider CRUD API/server persistence and new CLI workflows (`nav provider create/get/list/update/delete`), including `--from-existing` laptop discovery.
- Integrate providers into sandbox create flow: infer from command (`-- claude`), support repeatable `--provider <type>`, prompt before auto-create, and allow manual in-sandbox setup.
- Add a dedicated `navigator-providers` crate with per-provider modules and mockable discovery test helpers.
## Key UX Changes
- `nav sandbox create --provider gitlab -- claude`
- Missing provider prompt now asks before creating from local state.
- `nav provider list --names` for scripting/cleanup.
## Test Plan
- `mise run cluster:deploy`
- `mise run test:e2e:sandbox`
- `mise run pre-commit`
Closes#19Closes#22Closes#11
Closes#13
## Summary
- add Python sandbox execution APIs for command and callable workflows
- consolidate sandbox policy fixtures and expand e2e test coverage for policy and Python exec paths
- update CI/build config and images for sandbox e2e execution dependencies
## Test Plan
- mise run pre-commit
Closes#26
## Summary
Consolidates sandbox policy data into `dev-sandbox-policy.yaml` and deletes `dev-sandbox-policy-data.rego`. The YAML file becomes the single source of truth for sandbox policy data across both paths:
- **`nav create sandbox`** (CLI reads YAML → proto → gRPC → server stores → sandbox fetches)
- **`mise run sandbox`** (sandbox reads YAML directly via regorus)
## Why YAML
The policy data file is user-facing — it's the interface users interact with to configure sandbox security policy (allowed endpoints, binary identities, filesystem access, etc.).
Keeping this as YAML rather than rego:
- **Decouples data from enforcement engine** — if we change from OPA/rego to a different policy engine, the user's data file stays the same
- **Still works with regorus** — regorus can natively load YAML data via its `yaml` feature, so there's no conversion step in the local dev path
- **Enables future indirection** — we can add a transformation layer between the YAML file and what the policy engine receives, without changing the user-facing format
## Changes
### Policy data (`dev-sandbox-policy.yaml`)
- Flat top-level structure: `filesystem_policy`, `landlock`, `process`, `inference`, `network_policies`
- Renamed `filesystem` → `filesystem_policy` to match rego/proto convention
- Deleted `dev-sandbox-policy-data.rego` (was the rego equivalent of this data)
### Rego rules (`dev-sandbox-policy.rego`)
- All data references updated: `data.sandbox.*` → `data.*` (dropped `sandbox.` prefix)
- `package navigator.sandbox` unchanged (that's the rego rule namespace, not the data path)
### Sandbox entrypoint (`navigator-sandbox`)
- `--rego-policy`/`--rego-data` → `--policy-rules`/`--policy-data`
- `NAVIGATOR_REGO_POLICY`/`NAVIGATOR_REGO_DATA` → `NAVIGATOR_POLICY_RULES`/`NAVIGATOR_POLICY_DATA`
- `from_files()`: loads YAML natively via `regorus::Value::from_yaml_file()`
- `from_strings()` and `reload()`: accept YAML data via `Value::from_yaml_str()`
- `proto_to_opa_data_json()`: emits data at root level (no `"sandbox"` wrapper)
- Enabled regorus `yaml` feature
### CLI (`navigator-cli`)
- `DevSandboxPolicyFile.filesystem` → `filesystem_policy`
### Tests
- All inline test data converted from rego/JSON to YAML
- `test_engine()` uses `from_strings(rules, yaml)` with the dev YAML file
### Documentation
- Updated `architecture/plans/sandbox-policy-refactor/plan.md`, `architecture/plans/opa/plan.md`, `architecture/sandbox.md`, `architecture/containers.md`
In proxy mode, processes could bypass HTTP_PROXY env vars and connect
directly to any IP since seccomp cannot filter connect() by destination.
This adds network namespace isolation:
- Create isolated netns with veth pair (10.200.0.1 <-> 10.200.0.2)
- Proxy binds to host side of veth
- Sandboxed process enters netns via setns() in pre_exec
- Only network path out is through the proxy
Also adds CAP_SYS_ADMIN and CAP_NET_ADMIN to container configs and
installs iproute2 for namespace management.
Drew Newberry