* feat(policy): add policy recommendation plumbing — denial aggregation, transport, approval pipeline, and mechanistic recommendations
Implement the infrastructure layer for automated policy recommendations (#204):
- Proto: 9 new RPCs and messages for draft policy lifecycle (submit, get, approve, reject, approve-all, edit, undo, clear, history)
- Persistence: SQLite/Postgres migrations and store methods for draft_policy_chunks and denial_summaries tables
- Server: Full gRPC handler implementations with mechanistic mapper that auto-generates NetworkPolicyRule proposals from denial summaries
- Sandbox: DenialAggregator with MPSC channel, deduplication, periodic flush to gateway via SubmitPolicyAnalysis
- CLI: 'openshell draft' subcommand with get/approve/reject/approve-all/undo/clear/history operations
- TUI: Draft recommendations panel accessible from sandbox policy view
- Docs: Architecture documentation in architecture/policy-advisor.md
* feat(policy): add L7-aware mechanistic mapper and policy advisor CTF example
Add L7 rule generation to mechanistic mapper (build_l7_rules,
generalise_path, looks_like_id) with 3 new unit tests. Add
examples/policy-advisor/ with a 7-gate CTF script, restrictive
sandbox policy, and walkthrough README.
* fix(policy): use sandbox name for denial flush and add TUI draft badges
Fix denial aggregator passing sandbox UUID instead of name to
SubmitPolicyAnalysis, which caused 'sandbox not found' errors on
flush. Add notification badges to the TUI sandbox list and detail
header showing pending draft recommendation counts.
* fix(policy): deduplicate draft chunks and tolerate overlapping OPA rules
Skip draft chunk creation when a pending/approved chunk already covers
the same host:port endpoint, preventing duplicate rules across denial
aggregator flush cycles.
Rewrite three OPA complete rules (network_policy_for_request,
matched_network_policy, matched_endpoint_config) to tolerate multiple
matching policies without triggering a "complete rule conflict" error.
network_policy_for_request becomes a boolean, matched_network_policy
uses a set comprehension with min(), and matched_endpoint_config uses
an array comprehension with index-0 selection.
* feat(tui): interactive draft actions, highlight bar, and detail popup
Rework the draft recommendations panel to match the logs UX:
- Highlight bar (green accent + background) instead of arrow marker
- Viewport-aware j/k scrolling with g/G for top/bottom
- Enter opens a full-screen detail popup showing endpoints, binaries,
rationale, security notes, and action hints
Add approve/reject/approve-all draft actions:
- [a] approve selected chunk, [x] reject, [A] approve all pending
- Actions work from both the list view and the detail popup
- gRPC calls run async; result updates status bar and refreshes data
- Nav bar shows all available keybindings
Fix draft count refresh: sandbox_draft_counts now refreshes on every
tick (not just Dashboard), so the detail header badge updates in
real time.
Improve badge labels: show 'N pending' instead of a bare number in
both the dashboard sandbox list and sandbox detail header.
* refactor(policy): DB-level draft chunk dedup with hit counter and timestamps
Replace the in-memory HashSet dedup in SubmitPolicyAnalysis with a
database-level upsert. New denormalized columns on draft_policy_chunks:
- host, port: extracted from proposed_rule at insert time
- hit_count: incremented on conflict (same sandbox + host + port)
- first_seen_ms, last_seen_ms: track when the endpoint was first and
most recently proposed
A partial unique index (WHERE status IN ('pending','approved')) ensures
only one active chunk per endpoint per sandbox; rejected/superseded
chunks don't block new proposals.
Surface hit_count and first/last_seen in:
- CLI: 'openshell draft get' shows 'Hits: N (first ..., last ...)'
- TUI: detail popup shows hits row; list view shows 'Nx' suffix
* fix(policy): optimistic retry on policy version conflicts + structured logging
merge_chunk_into_policy and remove_chunk_from_policy now retry up to 5
times on UNIQUE constraint violations (version conflicts from concurrent
approvals). Each attempt re-reads the latest policy, re-merges the rule,
and increments the version. This eliminates the race condition where
rapid successive approvals would fail with a DB error.
Add structured tracing to all draft action handlers:
- ApproveDraftChunk: logs rule_name, host, port, hit_count before merge
and version + policy_hash after success
- RejectDraftChunk: logs rule_name, host, port, reason
- ApproveAllDraftChunks: logs pending_count at start, per-chunk merge
progress, and final summary with chunks_approved/skipped
- UndoDraftChunk: logs before/after with rule_name and version
- Retry attempts log as warnings with attempt number and conflicting
version
* wip: forward proxy fix, mapper allowed_ips, TUI polish, CTF rewrite
* fix(tui): use correct --gateway flag for ssh-proxy ProxyCommand
* chore: add Docker cleanup script for stale images, volumes, and build cache
* feat(tui): approve-all confirmation modal and CTF cleanup
Add [A] confirmation popup that snapshots pending chunks, shows a
scrollable list, and approves each chunk individually on confirm.
This prevents approving chunks that arrived after the modal opened.
Remove transient issue #205 reference from CTF victory banner.
* fix(tui): correct import ordering for rustfmt
* wip: stateful toggle model, rename to network rules
Draft chunks now follow a toggle state machine:
pending -> approved | rejected (initial decision)
approved <-> rejected (toggle)
One row per (sandbox_id, host, port) via expanded unique index.
Rejecting an approved rule removes it from the active policy.
Re-approving a rejected rule merges it back.
Rename CLI from 'draft' to 'rule', TUI from 'Draft Recommendations'
to 'Network Rules'. State-aware keybindings: approved shows [x] Revoke,
rejected shows [a] Approve. Fix sandbox detail hiding delete confirmation
behind pending message.
* refactor(policy): move mapper sandbox-side, slim schema, per-binary granularity
Move mechanistic mapper from gateway to sandbox so all analysis runs
sandbox-side (N sandboxes = N independent pipelines). Gateway is now a
thin validate + persist + approval layer.
Architectural changes:
- Move mechanistic_mapper.rs from navigator-server to navigator-sandbox
- Sandbox flush flow: aggregator drains -> mapper runs -> proposals sent
- Gateway SubmitPolicyAnalysis: validate + persist only, no mapper
- Drop denial_summaries table (write-only, zero readers)
- Consolidate migrations 003+004+005 into single 003
Schema slimming:
- Drop 5 unused columns from draft_policy_chunks (stage, denial_refs,
supersedes_chunk_id, analysis_mode, decided_by)
- Add per-binary granularity: binary column, widen unique index to
(sandbox_id, host, port, binary)
- Mapper groups by (host, port, binary), one proposal per triple
- Merge appends binary to existing rule; revoke removes just that binary
CTF & UX:
- 7-gate CTF: add Gate 3 (curl -> ifconfig.me:80) for per-binary demo
- TUI shows binary short name in list, full path in detail popup
- CLI output shows binary field
- Idempotent rule names, hit_count accumulates real denial counts
- Rationale text no longer bakes in stale denial count
* feat(proxy): support plain HTTP forward proxy for private IP endpoints
Add forward proxy mode to the sandbox proxy so that standard HTTP
libraries (httpx, requests, etc.) work with HTTP_PROXY for plain HTTP
calls to private IP endpoints. Previously, non-CONNECT methods were
unconditionally rejected with 403.
The forward proxy path requires all three conditions to be met:
- OPA policy explicitly allows the destination
- The matched endpoint has allowed_ips configured
- All resolved IPs are RFC 1918 private
This ensures plain HTTP never reaches the public internet while enabling
seamless access to internal services without custom CONNECT tunnel code.
Implementation:
- parse_proxy_uri(): parses absolute-form URIs into components
- rewrite_forward_request(): rewrites to origin-form, strips hop-by-hop
headers, adds Via and Connection: close
- handle_forward_proxy(): full handler with OPA eval, SSRF checks,
private-IP gate, upstream connect, and bidirectional relay
- Updated dispatch in handle_tcp_connection to route non-CONNECT methods
Includes 14 unit tests and 6 E2E tests (FWD-1 through FWD-6).
CONNECT path remains completely untouched.
Closes#155
* fix(proxy): remove InspectForInference match arm removed by #146
The inference routing simplification in #146 reduced NetworkAction to
Allow/Deny, removing InspectForInference. Drop the dead match arm from
handle_forward_proxy.
* fix(sandbox): restore BestEffort as default Landlock compatibility
The Landlock V2 upgrade in #151 changed the default from BestEffort to
HardRequirement. This causes all proxy-mode sandboxes to crash with
Permission denied when the policy omits the landlock field, because the
child process gets locked to only /etc/navigator-tls and /sandbox.
Restore BestEffort as the default so policies without an explicit
landlock field degrade gracefully.
Fixes#161
* fix(sandbox): inject baseline filesystem paths for proxy-mode sandboxes
Proxy-mode sandboxes need baseline filesystem paths (/usr, /lib, /etc,
/app, /var/log read-only; /sandbox, /tmp read-write) for the child
process to function under Landlock. Without these, the child can't exec
binaries, resolve DNS, or load shared libraries.
The supervisor now enriches the policy with these baseline paths at
startup, covering both standalone (file) and gateway (gRPC) modes. For
gateway mode, the enriched policy is synced back so users see the
effective policy via 'nemoclaw sandbox get'.
The gateway validation is relaxed to allow additive filesystem changes
(new paths can be added, existing paths cannot be removed) to support
the supervisor's enrichment sync-back.
Includes 2 E2E tests: BFS-1 (missing filesystem_policy) and BFS-2
(incomplete filesystem_policy).
Fixes#161
* fix(e2e): update assertion for relaxed filesystem validation message
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#32
Add post-condition checks in drop_privileges() to verify that setgid()
and setuid() actually changed the effective IDs. Also verify that
setuid(0) fails after dropping privileges, confirming root cannot be
re-acquired. This is a defense-in-depth hardening measure per CWE-250
and CERT POS37-C. All added syscalls (geteuid, getegid, setuid) are
async-signal-safe, so they are safe in the pre_exec context.
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The sandbox proxy had a control_plane_endpoints allowlist that let
sandboxed processes reach the gateway without OPA evaluation or SSRF
checks. This is a security issue: if the sandbox can talk to the
control plane, it could modify its own policy, exfiltrate data through
provider environment endpoints, or abuse other gateway APIs.
All connections now go through OPA policy evaluation and SSRF
protection with no exceptions. Removes AllowedEndpoint, parse_endpoint_url,
the is_control_plane bypass in handle_tcp_connection, and the engine field
from ConnectDecision.
* feat(skills): create nemoclaw-cli agent skill and fix generate-sandbox-policy schema drift
Closes#83
Create a new nemoclaw-cli agent skill with tiered workflow guidance (basic
through advanced) and a supplementary CLI reference. The skill covers provider
management, sandbox lifecycle, policy iteration loops, BYOC pipelines, and
agent-assisted sandbox sessions. It cross-references generate-sandbox-policy
for policy content authoring and includes --help self-teaching as a fallback.
Fix critical schema drift in generate-sandbox-policy: rename
allowed_routing_hints to allowed_routes (matching the actual proto/serde
field name). Update default read_only paths to match dev-sandbox-policy.yaml.
Fix the same field name error in architecture/security-policy.md.
* policy updates
* fix(test): update OPA tests for renamed github_ssh_over_https policy key
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(sandbox): allow egress to private IP space via allowed_ips policy field
Add an allowed_ips field to NetworkEndpoint that accepts CIDR notation and
exact IPs, enabling sandboxes to reach private IP space under policy control
while maintaining SSRF protections for loopback and link-local addresses.
Three modes are supported:
- Default (no allowed_ips): all private IPs blocked (existing behavior)
- Host + allowlist: domain must resolve to an IP in allowed_ips
- Hostless allowlist (no host): any domain allowed if resolved IP matches
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#78
## What You Can Do
### Live policy updates
Update a running sandbox's network policy without recreating it:
```bash
nav sandbox policy set <sandbox> --policy new-policy.yaml --wait --timeout 60
```
The sandbox hot-reloads the policy within 30s (configurable). On failure, the previous policy stays active (last-known-good).
Idempotent — submitting the same policy twice returns the existing version:
```
✓ Policy version 3 submitted (hash: a1b2c3d4e5f6)
$ nav sandbox policy set test --policy same.yaml
· Policy unchanged (version 3, hash: a1b2c3d4e5f6)
```
### Policy history & inspection
```bash
nav sandbox policy list <sandbox> # version history with status
nav sandbox policy get <sandbox> # current policy metadata
nav sandbox policy get <sandbox> --full # print full policy as YAML
nav sandbox policy get <sandbox> --rev 2 --full # specific revision as YAML
```
### Sandbox logs
Stream logs from both the gateway and sandbox supervisor in one view:
```bash
nav sandbox logs <sandbox> # one-shot, last 2000 lines
nav sandbox logs <sandbox> --tail # live streaming
nav sandbox logs <sandbox> --source sandbox # supervisor logs only
nav sandbox logs <sandbox> --source gateway # gateway logs only
nav sandbox logs <sandbox> --level warn # warnings and errors only
nav sandbox logs <sandbox> --since 5m # last 5 minutes
nav sandbox logs <sandbox> --tail --source sandbox --level info
```
Each log line is tagged with its source and includes structured fields:
```
[1772055394.673] [sandbox] [INFO ] [navigator_sandbox::proxy] CONNECT action=allow dst_host=api.anthropic.com dst_port=443 policy=claude_code
[1772055061.005] [gateway] [INFO ] [navigator_server::grpc] GetSandboxPolicy served from policy history
```
---
## Implementation
### Proto changes
- 7 new RPCs: `UpdateSandboxPolicy`, `GetSandboxPolicyStatus`, `ListSandboxPolicies`, `ReportPolicyStatus`, `GetSandboxLogs`, `PushSandboxLogs` (client-streaming)
- `SandboxLogLine`: added `source` (gateway/sandbox), `fields` (structured key-value map)
- `WatchSandboxRequest`: added `log_since_ms`, `log_sources`, `log_min_level`
- `GetSandboxLogsRequest`: added `sources`, `min_level`
- `PolicyStatus` enum, `SandboxPolicyRevision` message
- `Sandbox.current_policy_version` field
### Server
- **Policy persistence**: New `sandbox_policies` table (SQLite + Postgres) with per-sandbox monotonic versions, status tracking, and policy hash
- **UpdateSandboxPolicy**: Validates static field immutability (filesystem/landlock/process), network mode consistency (Block↔Proxy), deterministic hash comparison for idempotent updates
- **Lazy backfill**: First `GetSandboxPolicy` call creates version 1 from `spec.policy` for existing sandboxes
- **Log broker**: `TracingLogBus::publish_external()` injects sandbox-pushed logs into the same broadcast channel + tail buffer (2000 lines). Server forces `source="sandbox"` and `sandbox_id` on all pushed logs
- **Source/level filtering**: Applied server-side in both `GetSandboxLogs` and `WatchSandbox` streams
- **Version supersession**: When a new version is loaded, all older pending+loaded versions are marked superseded
### Sandbox
- **`OpaEngine::reload_from_proto()`**: Full `from_proto()` pipeline (L7 validation, access preset expansion) with atomic engine swap. On failure, previous engine untouched (LKG)
- **Policy poll loop**: Background task polls every 30s (configurable via `NAVIGATOR_POLICY_POLL_INTERVAL_SECS`), reports status via `ReportPolicyStatus` RPC
- **`LogPushLayer`**: Tracing layer captures events at INFO+ (configurable via `NAVIGATOR_LOG_PUSH_LEVEL`), sends structured fields via `PushSandboxLogs` client-streaming RPC. Background task batches 50 lines / flushes every 500ms. Best-effort (drops on full channel, never blocks)
- **`CachedNavigatorClient`**: Persistent mTLS channel for both policy polling and log push
### Database migration
- `002_create_sandbox_policies.sql` (SQLite + Postgres)
## Tests
- **Unit**: 8 policy persistence tests (put/get/list/status/supersede/isolation)
- **Integration**: 4 test files updated with new RPC stubs
- **E2E**: `test_live_policy_update_and_logs` — full lifecycle: create → set same (unchanged) → push new → wait for load → verify connectivity → push same (unchanged) → verify history → fetch logs
## Documentation
- `architecture/sandbox.md`: Log streaming architecture, LogPushLayer, push task, server broker, source tagging, structured fields, CLI filtering, failure modes
- `architecture/security-policy.md`: Live update semantics, deterministic hashing, CLI filter flags, policy inspection
- `architecture/plans/issue-78-sandbox-log-streaming.md`: Design plan for log streaming
## Security
- Trust boundary documented: shared mTLS cert model (per-sandbox auth tracked in #80)
- Server forces `source="sandbox"` and `sandbox_id` on pushed logs (can't impersonate gateway or other sandboxes)
- Per-batch line cap (100) prevents flooding
Closes#48, #52
## Summary
- Replace the envoy-gateway-based TLS setup with inline PKI generation during cluster bootstrap, generating CA, server, and client certificates directly in the `navigator-bootstrap` crate
- Remove all envoy gateway Helm templates (`gateway.yaml`, `gatewayclass.yaml`, `grpcroute.yaml`, PKI job, traffic policies) and the `Dockerfile.pki-job`
- Add native mTLS support to the navigator server with `tokio-rustls`, mounting client TLS certs as volumes into sandbox pods
- Update cluster entrypoint, healthcheck, and deploy scripts to work with the new direct-TLS architecture
- Add TLS security e2e test and fix formatting/clippy warnings
## Test Plan
- All unit tests pass (`cargo test --workspace`)
- Clippy clean (`cargo clippy --workspace --all-targets`)
- Format clean (`cargo fmt --all -- --check`)
- Python tests pass (`uv run pytest python/`)
- Full `mise run pre-commit` passes
> **🔧 security-fix-agent**
Closes#62
## Security Fix
### Summary
The CONNECT proxy accepted hostnames from clients and connected to whatever IP they resolved to, with no validation against internal address ranges. While the OPA policy is default-deny, a misconfigured or overly permissive policy could allow SSRF to cloud metadata (169.254.169.254), localhost, or RFC1918 services. This fix adds DNS resolution before connecting and rejects any host that resolves to an internal IP.
### Severity Assessment
- **Impact:** Medium — if exploited, could reach cloud metadata (IAM creds), cluster-internal services, or host-local services
- **Exploitability:** Very low — requires OPA policy misconfiguration or DNS rebinding attack
- **Affected components:** `crates/navigator-sandbox/src/proxy.rs` — `handle_tcp_connection`
### Changes Made
- `crates/navigator-sandbox/src/proxy.rs`: Added `is_internal_ip()` helper that checks IPv4 loopback/private/link-local, IPv6 loopback/link-local, and IPv4-mapped IPv6. Added `resolve_and_reject_internal()` that resolves DNS and rejects internal IPs. Inserted check between OPA allow and `TcpStream::connect`, with control plane endpoints exempt.
- `architecture/security-policy.md`: Added SSRF Protection section with blocked ranges table and flow diagram
- `architecture/sandbox.md`: Updated proxy connection flow diagram and added SSRF protection subsection
- `architecture/README.md`: Added internal IP rejection step to proxy description
### Tests Added
- **Unit:** 17 tests in `proxy::tests` — covers IPv4 loopback/private/link-local, IPv6 loopback/link-local, IPv4-mapped IPv6, public IPs, DNS resolution of localhost/127.0.0.1/169.254.169.254, and DNS failure handling
- **Integration/E2E:** N/A — the proxy runs inside a Linux network namespace; unit tests for IP checking and DNS resolution cover the security boundary
### Documentation Updated
- `architecture/security-policy.md`: New SSRF Protection section with blocked IP ranges and Mermaid flowchart
- `architecture/sandbox.md`: Updated proxy flow diagram and added SSRF protection subsection
- `architecture/README.md`: Added step 4 to proxy description
### Verification
All 85 sandbox tests pass including 17 new proxy SSRF tests. Pre-commit (fmt, clippy, full test suite) passes clean with zero warnings.
Add a --policy CLI flag to sandbox create that overrides the built-in
default and NAVIGATOR_SANDBOX_POLICY env var for specifying a custom
sandbox policy YAML file. Also add /dev/null to the default filesystem
read_write list and update security-policy docs with formatting fixes.
Add two sub-agents that have been useful thus far. The documentation agent has memory that is tracked.
Used the docs sub-agent to create detailed docs for all sub-systems + and overview doc.
Drew Newberry