* feat(policy): accept numeric UIDs in sandbox process identity validation
Allow run_as_user and run_as_group to be either the literal 'sandbox'
or a numeric UID/GID within [1000, 2_000_000_000]. This removes the
hard dependency on a baked-in 'sandbox' user in container images,
enabling compute drivers to inject resolved UIDs at sandbox creation.
Phase 1 of #1959.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(supervisor): accept numeric UIDs for process identity dropping
Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing
the hard dependency on a baked-in 'sandbox' user in container images.
Changes:
- validate_sandbox_user(): accepts numeric UIDs without passwd lookup
(logs OCSF event); keeps passwd check for "sandbox" name; rejects
non-numeric non-sandbox strings that fail passwd lookup
- prepare_filesystem(): passes numeric UIDs/GIDs directly to chown()
instead of requiring a passwd entry
- drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw
/ Gid::from_raw; skips initgroups when target uid matches current euid;
uses guard conditions before setgid/setuid calls
- session_user_and_home(): falls back to ("{uid}", "/sandbox") for
numeric UIDs, avoiding a passwd lookup that will fail
Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy
so callers have consistent range constants.
Phase 2 of #1959.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations
Phase 3 of the numeric-UID plan: allow operators to specify explicit
sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from
OpenShift SCC namespace annotations, and propagate resolved values to
supervisor container env vars and PVC init container securityContext.
Changes:
- Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig
- Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core
- Implement resolve_sandbox_identity() to fetch namespace annotations
and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range)
- Pass resolved UID/GID through SandboxPodParams to pod spec builder
- Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container
- Update PVC init container securityContext with resolved UID/GID
instead of hard-coded root
- Add comprehensive unit tests for resolution logic and annotation
parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC
annotation parsing)
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* feat(driver-vm): add configurable sandbox UID/GID and update docs/examples
Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001)
in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields.
Changes:
- Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives
- Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root
to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow
- Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID
injection and the ability to skip baked-in sandbox user
- Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both
Kubernetes (with OpenShift SCC autodetection) and VM drivers
- Update sandbox-compute-drivers.mdx: add Sandbox User Identity section
explaining numeric UID support across all compute drivers
- Update rootfs tests to use non-default UIDs, verify config passthrough
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* code review changes
* fix(supervisor): harden tests for restricted CI container environments
Guard tests against CI-specific constraints: root without CAP_SETPCAP,
UIDs with no /etc/passwd entry, and restricted /proc access.
Signed-off-by: Seth Jennings <sjennings@nvidia.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
---------
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Seth Jennings <sjennings@nvidia.com>
* feat(providers): support SPIFFE-backed token grants
Add provider profile token_grant metadata and expand endpoint-specific
dynamic credentials so sandbox supervisors can request SPIFFE JWT-SVIDs,
exchange them with an OAuth-style token endpoint, cache returned access
tokens, and inject bearer tokens into matching HTTP requests.
Wire Kubernetes and Helm deployments to mount the provider SPIFFE Workload
API socket into sandbox pods for token grant exchange.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(examples): add SPIFFE token grant demo
Add a reusable alpha/beta demo that deploys a SPIFFE-verifying token issuer
and protected services, imports a token-grant provider profile, creates a
sandbox, and verifies endpoint-specific bearer tokens.
The script leaves Kubernetes workloads in place, deletes sandboxes through
openshell unless KEEP_SANDBOX=1, and prints protected service logs as proof
of life.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden SPIFFE token grants
* fix(providers): harden dynamic token grants
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden token grant handling
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
Migrate all sandbox and VM driver network policy enforcement from
iptables to nftables. nftables provides atomic ruleset loading, a
cleaner rule syntax, and is the standard netfilter interface in modern
kernels.
Sandbox bypass enforcement (openshell-sandbox):
- Replace iptables chain of individual rule insertions with a single
atomic nftables ruleset load via nft -f
- New nft_ruleset module with pure functions for ruleset generation
and unit tests
- Combine log and reject rules in one inet family table (handles both
IPv4 and IPv6 in a single ruleset)
- Fall back to reject-only ruleset when kernel lacks nft_log support
- Enable net.netfilter.nf_log_all_netns so log rules work from
non-init network namespaces
- Use temp file for nft ruleset loading instead of stdin for
compatibility with minimal VM guest environments
VM TAP networking (openshell-driver-vm):
- Replace iptables NAT/forwarding rules with nftables equivalents
- New nft_ruleset module for TAP network rule generation with unit
tests
- Atomic table-per-TAP-device lifecycle (create/destroy)
- Host-side rules provide NAT infrastructure and defense-in-depth
isolation (input chain restricts VM to gateway port only, forward
chain blocks unsolicited inbound); primary security enforcement
happens inside the VM guest via the sandbox supervisor's own rules
VM init script:
- Load nft kernel modules at sandbox init
- Enable nf_log_all_netns sysctl for bypass detection logging
OCSF / docs:
- Update firewall rule engine references from iptables to nftables
- Document host firewall interaction model and two-layer enforcement
architecture in VM driver README and compute drivers reference
Closes#1335
Signed-off-by: Russell Bryant <rbryant@redhat.com>
The --sync flag was never implemented. The correct flag is --upload
with a path argument. Updated examples and workflow snippet to reflect
the actual CLI interface.
Fixes#1339
Signed-off-by: Mesut Oezdil <versusfinem@gmail.com>
* feat(policy): plumb chunk_ids and rejection_reason through proposal pipeline
Prereq plumbing for the agent revise-and-resubmit loop. Two narrow
additive proto changes unblock the upcoming /wait endpoint (#1092),
prover validation badge (#1097), and reject --guidance surfaces (#1098).
- SubmitPolicyAnalysisResponse: add accepted_chunk_ids so the in-sandbox
agent gets handles to watch its proposals. Surfaced through the typed
grpc_client wrapper and policy.local's POST /v1/proposals 202 body.
Closes#1094.
- PolicyChunk + StoredDraftChunk + DraftChunkPayload: add
validation_result (gateway prover verdict, populated by #1097) and
rejection_reason (operator free-form text). Both plain strings; no
enums, no parsing on the read path. Closes#1096.
- RejectDraftChunk now persists the existing reason field into the
chunk's rejection_reason so it round-trips back to the agent via
GetDraftPolicy. UndoDraftChunk clears it on the way back to pending
so consumers cannot read a stale guidance string from a prior reject
-> re-approve -> undo cycle.
Whole surface stays gated behind agent_policy_proposals_enabled. Two
focused tests cover the round-trip and the undo-clears guarantee.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* feat(sandbox): add /v1/proposals/{id} and /wait long-poll to policy.local
The agent feedback channel back from policy.local. Two new routes let
the in-sandbox agent learn its proposal's outcome on a single blocking
HTTP call — zero LLM tokens during the wait.
- GET /v1/proposals/{chunk_id} returns the chunk's current state in one
gateway call.
- GET /v1/proposals/{chunk_id}/wait?timeout=<s> blocks until the chunk
transitions out of pending. Default 60s, clamped [1, 300]. Agent
re-issues on timeout to extend.
Response carries the chunk's status plus the two feedback fields shipped
in the prereq commit: rejection_reason (free-form reviewer text) and
validation_result (gateway prover verdict, empty until #1097). On
timeout: same shape with timed_out: true so the agent can disambiguate
without parsing.
Wait handler short-polls GetDraftPolicy every 1s inside the request with
a tokio::time::Instant deadline. One gateway connection is opened per
request and reused across all polls, so a 60s wait does one TLS
handshake instead of sixty. A future commit can swap the loop body for
a tokio::sync::broadcast driven by a watcher task — the agent-visible
contract (URL, query, response shape) is independent of the polling
implementation.
All routes stay behind agent_policy_proposals_enabled. Closes#1092.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* docs(sandbox): teach policy_advisor skill the wait + redraft loop
The agent-facing instructions for the feedback loop. The endpoints
exist; this is the doc that makes them usable.
policy_advisor.md gains:
- API entries for GET /v1/proposals/{chunk_id} and /wait?timeout=<s>,
including the field semantics (status, rejection_reason,
validation_result, timed_out).
- A note on the submit response's accepted_chunk_ids /
rejection_reasons split so the agent handles partial acceptance.
- Step 6 saves the chunk_ids and addresses any submit-time rejections
before waiting.
- Step 7 walks the four wait outcomes: approved (retry, with the
honest "may still fail" caveat), rejected (read rejection_reason
AND validation_result; address whichever has content), still-pending
with timed_out (re-call), non-2xx (surface, do not retry).
skills.rs gains two assertions on the skill content so a future edit
cannot drop the wait endpoint or the rejection_reason directive
silently.
Closes#1095.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* test(policy-advisor): add end-to-end smoke for the agent feedback loop
A focused smoke that exercises the new policy.local /wait endpoint
on a live gateway + sandbox, separate from the existing no-LLM
regression harness (which still drives the OLD retry-with-bash-loop
recovery pattern).
Two flows:
- Flow A — approve-and-retry: agent submits, /wait blocks, host runs
`openshell rule approve`, /wait returns status=approved. Confirms
the happy path round-trip latency.
- Flow B — reject-with-guidance: agent submits, /wait blocks, host
runs `openshell rule reject --reason "..."`, /wait returns
status=rejected with the exact reviewer text in rejection_reason.
Confirms the free-form guidance contract round-trips through the
agent feedback channel.
No GitHub credentials needed — proposals are synthetic and never
trigger outbound traffic. Both flows expect agent_policy_proposals_enabled=true
and a running gateway.
Adds three cases to sandbox-runner.sh: submit-test-proposal (no GH
deps), proposal-status, proposal-wait. The existing put-file and
submit-proposal cases used by test.sh are untouched.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(policy-advisor): surface real CLI errors from wait-smoke preflight
The preflight piped openshell's stderr to /dev/null and relied on jq to
default the missing setting key to "<unset>", but under `set -euo
pipefail` a non-zero exit from openshell makes the whole pipeline fail
and the command substitution exits the script silently before the
intended fail() message can print.
Capture stderr explicitly, check the CLI exit code, and surface the
real error plus the expected fix (port-forward + gateway add + select)
when the CLI cannot reach the gateway.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(policy-advisor): pass --json to settings get in wait-smoke preflight
`openshell settings get --global` defaults to a human-readable table;
jq cannot parse it and the preflight died with a numeric-literal error.
Pass --json so jq gets actual JSON. Also touched up the suggested
recovery commands in the preflight error to match the real CLI shape
(`gateway add <endpoint> --name <name>` and the env-var override
warning).
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(policy): dedup draft chunks only in mechanistic mode; return effective id
The smoke harness for the agent feedback loop caught a real bug in the
gateway: SubmitPolicyAnalysis's response carried a chunk_id that was
never persisted whenever the SQL ON CONFLICT path fired. Two failure
modes, both load-bearing:
- Agent-authored proposals targeting the same host/port/binary
(e.g. the redraft-after-rejection loop) silently folded into one row
and any RejectDraftChunk by the new chunk_id failed with "chunk not
found." Latent since #1151, surfaced by #1094 returning chunk_ids.
- Mechanistic mode had the same class of bug — the dedup fold-in is
the intended behavior there, but the response still advertised the
newly-generated UUID instead of the existing row's id. Less visible
because no current caller reads mechanistic chunk_ids back, but the
proto contract was violated either way.
Fix in three parts:
- put_draft_chunk now takes Option<&str> dedup_key explicitly and
returns the effective row id (via RETURNING). None binds NULL to the
dedup_key column, which bypasses the partial-index ON CONFLICT path
entirely. Caller-decides semantics replace store-side magic.
- handle_submit_policy_analysis picks dedup_key per chunk using an
allowlist (only "mechanistic" dedups) and pushes the returned
effective_id to accepted_chunk_ids. New modes default to no-dedup so
a misconfigured caller cannot silently lose proposals.
- The two-copy draft_chunk_dedup_key helper consolidated to one
observation_dedup_key in policy_store.rs with a doc comment.
Tests:
- agent_authored_submits_for_same_endpoint_do_not_dedup pins the
redraft-loop contract: two intentional submissions with the same
host/port/binary get distinct chunk_ids, both findable via
GetDraftPolicy, both rejectable by id.
- mechanistic_submits_for_same_endpoint_dedup_into_one_chunk locks in
the observation-mode dedup AND asserts both submits return the same
effective_id — would have caught the deeper bug.
Proto: SubmitPolicyAnalysisRequest.analysis_mode doc updated to
describe the actual semantics (mechanistic dedups, agent_authored and
unknown modes do not).
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* docs(examples): retarget policy-management demo at the /wait endpoint
The narrated demo (examples/agent-driven-policy-management) has been the
public face of this feature since #1151. Its agent prompt told Codex to
retry the original PUT every few seconds for up to 120 seconds — a
polling workaround for the missing /wait endpoint that this branch
shipped. Update the demo to exercise /wait so the canonical reading of
the feature reflects the actual UX win.
- agent-task.md: step 4 is now "call /wait, branch on status" with the
three outcomes spelled out (approved → retry once; rejected → read
rejection_reason and revise or stop; pending+timed_out → re-issue
/wait once, do NOT busy-loop or shorten the timeout). Also makes
explicit that the demo submits one rule per proposal so
accepted_chunk_ids[0] is the safe single id to wait on.
- demo.sh: header docstring rewritten as a six-step loop that mirrors
the README. narrate_sandbox_workflow drops its parallel numbering and
uses bullets (the runtime narration is the agent's sub-actions, not a
separate decomposition of the loop). Approve step header and success
message now reference /wait waking the agent, not "policy hot-reload
retry."
- README.md: top-of-file flow expanded from 5 to 6 steps to include the
/wait call and chunk_id capture; "Going further" section now describes
both regression scripts and the boundary between them (real-GitHub
retry vs. synthetic /wait wire test). Slow-path qualifier corrected
from "image pull on first run" to "sandbox cold-start (SSH bring-up
plus Codex install)".
- wait-smoke.sh header rewritten to make it unambiguous this is a
regression, NOT a tutorial, with explicit prereq commands instead of
prereq descriptions, and a pointer at demo.sh for the narrated story.
No code paths change; this is the readability pass.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(examples): pass --yes on demo.sh's global setting writes
Global setting updates require explicit confirmation in non-interactive
mode; demo.sh's enable_agent_proposals and the cleanup restore path
were missing --yes and hard-failed the preflight. Pre-existing issue
that surfaced now that more of the demo runs through this path.
No other behavior change.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* feat(sandbox): emit OCSF audit events for policy proposal lifecycle
The demo's policy decision trace previously showed only the proxy
enforcement story (HTTP:PUT DENIED, CONFIG:LOADED, HTTP:PUT ALLOWED).
It was silent about who proposed what or who decided what — the
audit-trail receipts for the agent feedback loop were missing.
policy.local now emits sandbox-side OCSF events at the observation
moments, into the same stream as the existing CONFIG:LOADED:
- CONFIG:PROPOSED on submit_proposal acceptance. Per accepted chunk:
the message names the chunk_id, target endpoint, L7 method/path, and
binary so the trace correlates against the inbox card via chunk_id.
- CONFIG:APPROVED on /wait observation of approved status.
- CONFIG:REJECTED on /wait observation of rejected status. Carries the
reviewer's free-form rejection_reason in the message AND as an
unmapped field, both sanitized (control chars stripped, capped at
200 chars with an ellipsis marker). The agent still reads the raw
text via GET /v1/proposals/{id}; sanitization is audit-side only,
per AGENTS.md's no-secrets-in-OCSF rule.
The submit path defends the audit_summaries / accepted_chunk_ids
index pairing against a future gateway change that compresses past
rejected chunks (the proto doesn't promise 1:1 ordering with the
request). Today client-side validation makes the lengths always
match; if they don't, the pairing falls back to a generic per-id
event rather than mis-attribute.
The wait handler's emit site fires once per terminal-status
observation. Multiple concurrent waiters on the same chunk would
each emit one event; acceptable for single-waiter-per-chunk demos
and the right place to dedup is the SIEM.
demo.sh's trace filter now surfaces the four CONFIG: events
alongside HTTP:PUT, so the trace at the end of every run tells the
full story from deny to allow via propose -> approve.
wait-smoke.sh's prereq notes recommend redirecting kubectl
port-forward output so its "Handling connection for 8090" lines
don't bleed into demo narration.
Three new unit tests on the sandbox-side helpers — summary builder
happy path, fallback, and the rejection_reason sanitizer.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* feat(policy): /wait awaits local policy reload; demo auto-approves redrafts
Three things in one commit, all surfaced by running the demo end-to-end
against a real gateway and finding the agent had to draft a broader
second proposal.
1. /wait race fix. Previously /wait returned `approved` the moment it
observed the gateway's chunk status flip, but the local supervisor
reloads policy on its own poll cycle (~10s in practice). The agent's
retry would race the reload and hit the still-old policy, getting
denied. Codex then drafted a broader rule and re-submitted — sound
agent behavior, but not what /wait should provoke. Now /wait captures
the local policy version at start, and after observed-approved waits
for the supervisor to load a strictly-newer version before returning.
Bounded by the caller's deadline; best-effort return if the deadline
elapses without the version bumping. Two new unit tests pin the
happy path and the deadline-clamped fallback.
2. demo.sh auto-approve loop. Replaces approve_when_pending +
wait_for_agent with one approve_pending_until_agent_exits function
that keeps watching for pending chunks and approving them until the
agent process exits (or the configured timeout). Defense in depth
against future redraft scenarios for any reason; today (post-fix #1)
the agent should only submit one proposal per task, but we don't
want to hang silently if it does submit more.
3. UX. Step headers now carry "[t+1.2s]" relative timestamps so reading
the run output makes latency visible (the demo's whole point is the
wait is cheap — surface that). A spin_wait helper renders an ASCII
spinner during the watch loop so the demo never looks frozen on a
TTY. Falls back to plain sleep on non-TTY contexts.
Closes the race condition diagnosed from the trace timing where the
gateway approved at t+0, sandbox observed at t+0.3s, but the supervisor
didn't load v2 until t+9.4s — well after the agent had already retried
and been denied.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(sandbox): /wait detects policy reload by content, not the schema version
The previous attempt at the /wait-after-approve race fix compared
`SandboxPolicy.version` between /wait start and the policy reload —
but that field is the *schema* version (constant 1), not a revision
counter. Every comparison was `current(1) > baseline(1) == false`, so
the wait blocked until the agent's 300s timeout regardless of whether
the supervisor had actually reloaded. The demo SSH connection then
timed out around the 240s mark.
Diagnosed from a live run's OCSF trace: supervisor pulled v2 at
+8.5s after approval (CONFIG:LOADED), but the sandbox-side
CONFIG:APPROVED that my /wait emits didn't fire until +304s — exactly
at the 300s deadline.
Fix: compare the whole policy via prost's derived PartialEq. Any
field change (network_policies map being the only one that actually
mutates today) flips equality. A clone-per-200ms-tick on a few-KB
proto is cheap inside the bounded wait window.
Tests rewritten to match the new contract: the supervisor-reload
fixture now keeps `version: 1` constant and changes `network_policies`
contents, mirroring the exact failure mode from the live run.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(examples): redact tokens with python literal-string replace, not sed
The sed-based redact_log in demo.sh broke when one of the auth tokens
contained a character that conflicted with sed's pattern parser
("unterminated substitute pattern" on the Codex JWT). The whole log
tail then blanks on failure, hiding the very failure context we're
trying to surface.
Switch to a python subprocess that takes the tokens via argv and does
literal str.replace. No regex, no delimiter games, no truncation.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(sandbox): scope /wait reload check to the approved rule
Reviewer (John Myers) flagged two failure modes in the prior whole-policy
fingerprint approach used by policy.local /wait:
- False sleep: when the supervisor reloads between two /wait calls
(the skill tells the agent to re-issue on timed_out), the new call
snapshots the already-updated policy as baseline and burns the full
timeout waiting for a change that never comes.
- False wakeup: any unrelated reload (other agent's approval, settings
change) flips the diff, but the chunk's actual rule may not be loaded
yet — the agent retries and hits policy_denied for no real signal.
Replace the diff with rule-coverage. New public helper
openshell_policy::policy_covers_rule reuses endpoints_overlap (so it
matches add_rule's merge semantics, including the fold-into-existing-key
case) plus an L7 allow check on method/path (so an existing endpoint
that doesn't yet contain the proposed method doesn't signal coverage).
Add policy_reloaded: true|false to the /wait response on approve, with
a 500ms floor on the reload-wait phase so approvals arriving near the
deadline still get a fair shot at reloaded=true. Update the
policy_advisor skill to branch on it: reloaded=true → retry;
reloaded=false → re-issue /wait once with timeout=30, then surface to
user. Don't loop tightly.
Tests:
- 9 new unit tests in openshell-policy pinning coverage semantics
(L4-only, L7 method gap, fold-into-existing-key, empty binaries).
- 4 new tokio tests in policy_local mirroring John's exact scenarios.
- wait-smoke.sh asserts policy_reloaded=true on Flow A.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(server): make GetDraftPolicy dual-auth so /wait works under OIDC
policy.local calls GetDraftPolicy from inside the sandbox supervisor
via the sandbox gRPC client, which authenticates with the shared
x-sandbox-secret. GetDraftPolicy was listed only in the Bearer-auth
scope table (config:read) and was not in SANDBOX_SECRET_METHODS or
DUAL_AUTH_METHODS, so OIDC-enabled gateways rejected those calls and
the /wait long-poll surfaced gateway_lookup_failed. Local/no-OIDC
setups happened to work because the auth check is short-circuited.
Add GetDraftPolicy to DUAL_AUTH_METHODS, matching the existing
GetSandboxConfig pattern (called by both CLI reviewer surfaces with
Bearer and the sandbox supervisor with x-sandbox-secret). Dual-auth
short-circuits the scope check for sandbox-secret callers, so the
config:read entry in authz.rs continues to gate Bearer-only flows.
Mirror the openshell_get_sandbox_config_is_dual_auth assertion for
GetDraftPolicy.
Note: ssh_handshake_secret is server-wide, not per-sandbox, so a
sandbox-secret caller can today name any sandbox in a SubmitPolicyAnalysis
request — and now in a GetDraftPolicy request. The exposure is
symmetric with the existing SANDBOX_SECRET_METHODS pattern. Filed as a
follow-up: per-sandbox secret binding, tracked separately.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(ci): address rebased check failures
Signed-off-by: Alexander Watson <zredlined@gmail.com>
---------
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* docs(rfc): add agent-driven policy management
* docs(rfc): switch policy MVP to local API
* docs(rfc): clarify policy advisor skill and local logs
* feat(sandbox): add agent-driven policy proposal loop
* test(examples): add codex policy dogfood loop
* refactor(examples): make policy demo agent-agnostic
* refactor(examples): colocate policy validation harness
* docs(examples): add policy demo env sample
* docs(examples): use placeholder env example
* feat(sandbox): wire policy.local denials to OCSF JSONL log
Wires GET /v1/denials?last=N on the sandbox-local policy advisor API to read
recent OCSF JSONL events from /var/log/openshell-ocsf.YYYY-MM-DD.log, filter
to network/L7 denials (action_id=2, class_uid 4001/4002), and return a
compact summary newest-first. Default limit is 10, capped at 100. Ran inside
spawn_blocking so file I/O does not block the policy.local handler.
Other cleanup:
- POST /v1/proposals now uses the typed grpc_client wrapper instead of
raw_client, so accepted/rejected counts surface to the agent uniformly.
Wrapper return type extended to the response struct.
- Drop the 'add_rule' snake_case alias in the proposal JSON; canonical form
is camelCase 'addRule', matching the PolicyMergeOperation convention used
elsewhere.
- skills/policy_advisor.md updated to match: documents the now-real
/v1/denials?last=10 endpoint and uses 'addRule' consistently.
- skills.rs test asserts on the canonical 'addRule' phrase rather than the
removed 'PolicyMergeOperation' substring.
* feat(cli): show L7 protocol/method/path in rule get output
format_endpoint() previously rendered only host:port, dropping protocol,
access, and the L7 rules array. That made openshell rule get text output
unable to distinguish a broad L4 grant from a method/path-scoped L7 REST
rule -- exactly the distinction a developer needs at approval time.
New rendering tags each endpoint with its enforcement layer and surfaces
allow/deny rules:
bare L4: api.example:443 [L4]
L7 read-only: api.example:443 [L7 rest, access=read-only]
L7 method/path: api.example:443 [L7 rest, allow PUT /v1/foo/bar]
Pure display change: no proto, gateway, or behavior changes. Unit test
covers all three rendering cases with synthetic fixtures.
* refactor(examples): rewrite policy demo as Codex-default loop
Re-shape examples/agent-driven-policy-management/ to be a single, clean
end-to-end demonstration of the agent-driven policy loop. A Codex agent
inside an OpenShell sandbox attempts a GitHub Contents API write, hits a
structured 403 from the L7 proxy, reads the policy_advisor skill, drafts a
narrow addRule proposal via http://policy.local/v1/proposals, the host
auto-approves, the sandbox hot-reloads policy, and the agent's retry
succeeds. Whole loop runs in roughly two minutes.
Demo cleanup:
- Drop .env file ceremony. Defaults resolve from gh: owner via
'gh api user --jq .login', repo defaults to 'openshell-policy-demo',
token from gh auth token / GITHUB_TOKEN / GH_TOKEN. With gh auth login
and codex login already done, 'bash demo.sh' Just Works.
- Codex-specific. Bootstraps ~/.codex/auth.json from credentials injected
by the OpenShell provider, runs codex exec --sandbox danger-full-access
(OpenShell is the actual security boundary; bwrap nesting cannot create
user namespaces inside the sandbox container).
- Tighter narrative output: a single 'Preflight' step, a run summary banner
before launch, an inline narration of what's happening inside the sandbox
while we poll for the proposal (including the literal structured 403
body the agent acts on), and an OCSF trace at the end filtered to the
three events that tell the story (DENY, RELOAD, ALLOW).
- Replace Python heredoc templating with sed; uploads use the single-flag
pattern (--upload "${PAYLOAD_DIR}:/sandbox") with files referenced at
the basename-prefixed path that #952 / #1028 established.
- README documents the trust model honestly: structured rule is the
contract, agent rationale is a hint, prover validation badge in
progress per RFC 0001.
Move the deterministic no-LLM regression harness out of examples/ into
e2e/policy-advisor/ -- it was a parallel demo, not an example. Same loop
without the LLM, useful for iterating on the proxy and policy.local API.
* style(sandbox,cli): apply rustfmt
Whitespace-only fixups caught by mise run pre-commit. No functional change.
* perf(examples): cap Codex reasoning at 'low' in policy demo
The demo task is mechanical (one HTTP request, parse a structured 403,
post a JSON proposal, retry). Codex's default high-effort reasoning
roughly doubles the demo's wall time without improving outcomes; running
at 'low' lands the same minimal L7 grant in roughly half the time.
Override with DEMO_CODEX_REASONING=medium (or higher) to compare runs.
* fix(sandbox): harden policy.local denials endpoint
Three changes addressing review feedback before merging the agent-driven
policy management MVP:
- Distinguish "OCSF JSONL enabled, no denials" from "OCSF JSONL disabled,
nothing to read." The endpoint now returns a `log_available` flag and an
explanatory `note` when the log file is missing, so the in-sandbox agent
can give the developer an accurate hint instead of a misleading empty
list.
- Stop echoing the OCSF `message` field in the per-denial summary. The
proxy's denial messages can include the request path with query string
(e.g., `?access_token=...`); the structured `host`/`port`/`method`/
`path`/`binary` fields carry everything the agent needs to draft a
proposal, and `path` is sourced from `http_request.url.path` which
already excludes the query string.
- Cap `read_request_body` at a 15s timeout. Bounds slowloris-style stalls
from a misbehaving in-sandbox process. The proxy listener only accepts
loopback connections so practical impact is small, but this is cheap
defense-in-depth.
New tests cover the missing-log signal and the message-redaction guarantee.
* fix(examples): redact tokens in agent log tail and validate DEMO_FILE_DIR
Two small hardening passes on the policy management demo:
- `fail()` now pipes the agent log tail through a redactor that masks the
GitHub token and Codex credential triple before printing. Codex itself is
well-behaved about not echoing the token, but a misbehaving tool call
could leak it; this is a final safety net before the log hits the
developer's terminal (and any clipboard or chat history that follows).
- `validate_env` now regex-checks DEMO_FILE_DIR with the same allow-list
the other path-shaped variables use. The value is interpolated through
sed with `|` as the delimiter when rendering the agent task; rejecting
unsupported characters keeps the templating predictable and stops a
user-supplied value from breaking out into a shell context.
* refactor(sandbox): centralize policy.local routes and skill path
Addresses review feedback that the deny body's `next_steps` array and the
route table could drift apart. The route paths and skill location now live
as `pub const`s in `policy_local.rs` and feed both:
- the dispatcher in `route_request` that matches against them
- a new `agent_next_steps()` helper that builds the JSON the L7 deny body
embeds
`l7/rest.rs::deny_response_body` calls `policy_local::agent_next_steps()`
instead of inlining the array, so adding or renaming a route is a one-line
change in `policy_local.rs` and the agent contract follows automatically.
* feat(sandbox): switch /v1/denials to shorthand log pass-through
Previously /v1/denials parsed `/var/log/openshell-ocsf.*.log` (OCSF JSONL)
and returned structured per-event objects. JSONL is opt-in via
`ocsf_json_enabled`, so the endpoint returned an empty list with a "log
not enabled" hint by default — agents had to navigate a setup step before
the inspect-recent-denials guidance was useful.
Switch to reading the shorthand log at `/var/log/openshell.*.log`, which
is always-on and the same human-readable format `openshell logs` displays.
The endpoint now returns raw shorthand lines (newest first) — the agent
reads them directly, no field parsing.
Tradeoffs:
- Removes the JSONL-on-by-default debate: shorthand is already on, no
defaults change.
- Updating shorthand is a single-file change in this repo; no schema rev
needed when we want to add fields.
Implementation:
- `read_recent_denial_lines` walks shorthand log files newest-first,
filters lines with ` OCSF ` AND ` DENIED ` (the OCSF action label,
uppercase, space-bounded).
- `collect_shorthand_log_files` matches `openshell.<date>.log`; the
trailing dot in `SHORTHAND_LOG_PREFIX = "openshell."` excludes
`openshell-ocsf.<date>.log` so JSONL-on doesn't bleed into responses.
- 4096-byte cap per surfaced line as defense against pathological inputs.
- Skill doc updated to reflect that `/v1/denials` returns raw shorthand
lines, not structured fields.
Defense-in-depth on query-string secrets:
- `redact_query_strings` strips `?<query>` to `?[redacted]` from each
surfaced line. The L7 relay path emits OCSF events using
`redacted_target` (secret-placeholder redaction), but the FORWARD deny
path in `proxy.rs` populates `OcsfUrl::new("http", host, path, port)`
and `.message(...)` with the raw request path — query string included.
Stripping queries at the consumer guards `/v1/denials` regardless of
whether the upstream emit sites are tightened. The on-disk log is not
rewritten by this change; that is a separate hardening task tracked
for the FORWARD path emit sites in proxy.rs.
- `truncate_at_char_boundary` is UTF-8 safe; redaction runs before
truncation so a cut cannot slice mid-secret.
Tests:
- `recent_denials_returns_newest_first_from_shorthand_lines` covers the
happy path with mixed allowed/denied/non-OCSF lines.
- `recent_denials_skips_jsonl_log_files` confirms JSONL files don't
surface even if present.
- `recent_denials_truncates_pathological_lines` covers the cap.
- `is_ocsf_denial_line_filters_correctly` covers the line-level filter.
- `redact_query_strings_removes_query_from_url_token` and
`redact_query_strings_removes_query_in_reason_tag` cover the redaction
in both URL token and `[reason:...]` contexts.
- `truncate_at_char_boundary_does_not_panic_on_multibyte` covers the
UTF-8 safety.
* chore(sandbox): align proto inits with main's L7 GraphQL additions
Post-rebase fixups after #1083 (GraphQL L7 inspection) landed on main and
introduced new fields on the proto types this branch constructs:
- `crates/openshell-sandbox/src/l7/relay.rs`: two `deny_with_redacted_target`
call sites (REST and GraphQL relay deny paths) now pass the
`DenyResponseContext` argument that `rest::send_deny_response` expects.
Both sites pass `host`, `port`, and `binary` from the existing
`L7EvalContext`, matching the pattern used at the primary deny site.
- `crates/openshell-sandbox/src/policy_local.rs`: `L7Allow`, `L7DenyRule`,
and `NetworkEndpoint` proto initializers now populate the new GraphQL
and path-scoping fields with empty defaults. Agent-authored proposals
via `policy.local` target REST/SQL/L4 today; GraphQL operation matching
is set on the gateway side or via direct YAML, so empty defaults are
correct here.
No behavior change. `cargo test -p openshell-sandbox --lib` (650 tests) and
`cargo clippy -p openshell-sandbox --lib --tests -- -D warnings` clean.
* feat(sandbox): gate agent policy proposals behind opt-in feature flag
The agent-driven policy proposal surface delivered by this PR (skill
install, `policy.local` API, `next_steps` array on L7 deny bodies) is
now opt-in via the new `agent_policy_proposals_enabled` setting. Default
false. Same shape as `providers_v2_enabled`: registered in
`openshell-core::settings`, sandbox-level, hot-toggleable via the
existing settings poll loop.
Why: the surface is a novel agent-controlled mutation point in every
sandbox. The per-proposal developer approval gate is a correctness
control, but it doesn't address "should this sandbox have an
agent-authoring API at all" — compliance teams may want that question
closed. The flag is the second gate.
Implementation:
- New registry entry + `AGENT_POLICY_PROPOSALS_ENABLED_KEY` constant in
`openshell-core::settings`.
- `lib.rs`: process-wide `OnceLock<Arc<AtomicBool>>` mirroring the
`OCSF_CTX` pattern. `agent_proposals_enabled()` is the single read
point.
- Initial settings fetch added to `run_sandbox` so skill install honors
the flag at startup (not just on the poll loop's first tick).
- Skill install in `run_sandbox` is gated on the flag.
- `policy_local::route_request` returns `404 feature_disabled` for all
routes when the flag is off — including the otherwise-public
`current_policy` and `denials` routes. When the surface is off it's
off entirely.
- `policy_local::agent_next_steps` returns an empty array when the flag
is off so deny bodies don't advertise routes that 404.
- Poll loop updates the atomic on each tick, lazily installs the skill
on a false→true transition (no claw-back on true→false; stale skill
on disk is harmless because route + next_steps gate on the live atom).
Tests:
- Shared `test_helpers::ProposalsFlagGuard` mutex+atomic guard for the
process-wide flag, used across `policy_local::tests` and
`l7::rest::tests`.
- New: `agent_next_steps_returns_empty_when_flag_off`,
`agent_next_steps_returns_full_array_when_flag_on`,
`route_request_returns_feature_disabled_when_flag_off`.
- Updated existing tests that exercise the deny body or the route
dispatcher to set the flag on first.
- Full sandbox lib test suite: 653 pass, clippy clean.
Demo and e2e:
- `examples/agent-driven-policy-management/demo.sh` and
`e2e/policy-advisor/test.sh` now snapshot the prior global value of
the setting, set it to true before sandbox creation (so the
supervisor's initial poll picks it up), and restore on exit (delete
if previously unset, otherwise write the prior value back).
Docs:
- RFC 0001 MVP-implementation note documents the flag, default, and
intended soft-launch posture.
* test(policy-advisor): require proposal opt-in for e2e
* refactor(sandbox): group policy poll loop state
* test(e2e): isolate Kubernetes user namespace test
---------
Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* fix(helm): derive grpcEndpoint from chart context
The chart hardcoded server.grpcEndpoint to
https://openshell.openshell.svc.cluster.local:8080, which only matched the
in-cluster Service DNS for the standard release name and namespace. A new
helper now builds <scheme>://<fullname>.<namespace>.svc.cluster.local:<port>
from chart context, picking the scheme from server.disableTls. An explicit
server.grpcEndpoint override is passed through verbatim.
* chore(scripts): validate k3d cluster name length early
helm-k3s-local.sh derives the cluster name from the current branch suffix.
Long branch names produced names exceeding k3d's 32-char cap and failed
deep inside k3d cluster create with a confusing validation error. cmd_create
now bails out before invoking docker/k3d with a copy-pasteable
HELM_K3S_CLUSTER_NAME override hint. Status, start, stop, delete, and help
remain unaffected so an over-long derived name does not block diagnostics.
* fix(examples): harden multi-agent notepad 409 retry and improve docs
Add decorrelated jitter to the GitHub Contents API retry loop so racing
workers don't retry in lockstep, and widen the retry set to 5xx for
transient server errors. Without jitter, N concurrent workers trip the
same backoff schedule and re-collide on each beat.
Also:
- Extract the inline 220-line sandbox runner from demo.sh into a
separate runner.sh. demo.sh stays as host orchestration; runner.sh
is the per-sandbox program. demo.sh shrinks 460 to 240 lines.
- Rewrite README to lead with "Why GitHub as a shared notepad?" and
add a "Memory architecture variants" section covering pile,
append-journal, and indexed-memory patterns.
- Switch Quick Start to use \`gh auth token\` as the primary path.
- Pass the worker slice arg explicitly instead of reading \$9 from the
outer dispatcher.
- Add .gitignore for personal run_demo.sh helpers.
* docs updates
Change sandbox user UID from 1000 to 1000660000 in custom image examples and E2E tests.
Using a high UID range (1000000000+) prevents conflicts with host users when running
without user namespace remapping, where container UIDs map directly to host UIDs.
This resolves fork failures caused by RLIMIT_NPROC enforcement when the host user
already has many threads running.
Signed-off-by: Derek Carr <decarr@redhat.com>
* feat(sandbox): auto-detect TLS and terminate unconditionally for credential injection
Closes#533
The proxy now auto-detects TLS by peeking the first bytes of each
connection. When TLS is detected, it terminates unconditionally —
enabling credential injection and optional L7 inspection without
requiring explicit 'tls: terminate' in the policy.
* feat(sandbox): log connection attempts that bypass proxy path
Add iptables LOG + REJECT rules inside the sandbox network namespace to
detect and diagnose direct connection attempts that bypass the HTTP
CONNECT proxy. This provides two improvements:
1. Fast-fail UX: applications get immediate ECONNREFUSED instead of a
30-second timeout when they bypass the proxy
2. Diagnostics: a /dev/kmsg monitor emits structured BYPASS_DETECT
tracing events with destination, protocol, process identity, and
actionable hints
Both TCP and UDP bypass attempts are covered (UDP catches DNS bypass).
The feature degrades gracefully if iptables or /dev/kmsg are unavailable.
Closes#268
* chore: track .python-version to pin Python 3.13.12 for uv
The sandbox base image runs Python 3.13. A stale venv on 3.12 causes
all exec_python E2E tests to fail because cloudpickle bytecode is not
compatible across minor versions.
* fix(cluster): preserve hostGatewayIP across fast deploys
The fast deploy's helm upgrade was missing the hostGatewayIP value that
the bootstrap entrypoint injects into the HelmChart CR. This caused
host.openshell.internal hostAliases to be lost from the gateway pod
and sandbox pods after any fast deploy, breaking host gateway routing.
Read the IP from the HelmChart CR and pass it through to helm upgrade.
* wip: fix iptables path resolution, use dmesg for kmsg, add CAP_SYSLOG
* fix(sandbox): restore NetworkNamespace Drop impl, remove dead kmsg code
The Drop impl for NetworkNamespace was accidentally deleted during the
bypass detection refactor, which would cause network namespaces and
veth interfaces to leak on every sandbox shutdown.
Also removes dead kmsg volume/mount code (bypass monitor uses dmesg
instead of direct /dev/kmsg access) and removes an accidentally
committed session transcript file.
* docs(examples): add sandbox policy quickstart walkthrough
Add an interactive getting-started example that demonstrates OpenShell's
network policy system end-to-end: default-deny, L7 read-only access,
and audit logging — all with a single YAML policy file.
- examples/sandbox-policy-quickstart/policy.yaml: policy with default
static fields (filesystem, landlock, process) so it works out of the
box with `openshell policy set`
- examples/sandbox-policy-quickstart/demo.sh: automated demo script
using printf (portable) and openshell ssh-proxy for sandbox exec
- examples/sandbox-policy-quickstart/README.md: step-by-step manual
walkthrough
- README.md: add "See network policy in action" section linking to the
quickstart
Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor
Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor
* docs: soften default-deny wording to minimal outbound access
Sandbox defaults vary by type and community configs, so
"all outbound traffic is blocked" is too absolute.
Made-with: Cursor
* docs: use curl -sS so L4 deny errors are visible
curl -s suppresses stderr, hiding the 403 from the CONNECT
proxy. Adding -S ensures the error message is always shown.
Made-with: Cursor
---------
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(proxy): stream inference responses instead of buffering entire body
The inference.local proxy path called response.bytes().await which
buffered the entire upstream response before sending anything to the
client. For streaming SSE responses this inflated TTFB from sub-second
to the full generation time, causing clients with TTFB timeouts to abort.
Add a streaming proxy variant that returns response headers immediately
and forwards body chunks incrementally using HTTP chunked transfer
encoding. Non-streaming responses and mock routes continue to work
through the existing buffered path.
Closes#260
* docs: update architecture docs and example for inference streaming
* test(example): add direct NVIDIA endpoint tests via L7 TLS intercept
Expand inference example to 4 test cases: inference.local and direct
endpoint, each streaming and non-streaming. The direct path exercises
the L7 REST relay (relay_chunked) to verify it already streams
correctly. NVIDIA_API_KEY is picked up from the sandbox env when
started with --provider nvidia.
* feat(policy): add policy recommendation plumbing — denial aggregation, transport, approval pipeline, and mechanistic recommendations
Implement the infrastructure layer for automated policy recommendations (#204):
- Proto: 9 new RPCs and messages for draft policy lifecycle (submit, get, approve, reject, approve-all, edit, undo, clear, history)
- Persistence: SQLite/Postgres migrations and store methods for draft_policy_chunks and denial_summaries tables
- Server: Full gRPC handler implementations with mechanistic mapper that auto-generates NetworkPolicyRule proposals from denial summaries
- Sandbox: DenialAggregator with MPSC channel, deduplication, periodic flush to gateway via SubmitPolicyAnalysis
- CLI: 'openshell draft' subcommand with get/approve/reject/approve-all/undo/clear/history operations
- TUI: Draft recommendations panel accessible from sandbox policy view
- Docs: Architecture documentation in architecture/policy-advisor.md
* feat(policy): add L7-aware mechanistic mapper and policy advisor CTF example
Add L7 rule generation to mechanistic mapper (build_l7_rules,
generalise_path, looks_like_id) with 3 new unit tests. Add
examples/policy-advisor/ with a 7-gate CTF script, restrictive
sandbox policy, and walkthrough README.
* fix(policy): use sandbox name for denial flush and add TUI draft badges
Fix denial aggregator passing sandbox UUID instead of name to
SubmitPolicyAnalysis, which caused 'sandbox not found' errors on
flush. Add notification badges to the TUI sandbox list and detail
header showing pending draft recommendation counts.
* fix(policy): deduplicate draft chunks and tolerate overlapping OPA rules
Skip draft chunk creation when a pending/approved chunk already covers
the same host:port endpoint, preventing duplicate rules across denial
aggregator flush cycles.
Rewrite three OPA complete rules (network_policy_for_request,
matched_network_policy, matched_endpoint_config) to tolerate multiple
matching policies without triggering a "complete rule conflict" error.
network_policy_for_request becomes a boolean, matched_network_policy
uses a set comprehension with min(), and matched_endpoint_config uses
an array comprehension with index-0 selection.
* feat(tui): interactive draft actions, highlight bar, and detail popup
Rework the draft recommendations panel to match the logs UX:
- Highlight bar (green accent + background) instead of arrow marker
- Viewport-aware j/k scrolling with g/G for top/bottom
- Enter opens a full-screen detail popup showing endpoints, binaries,
rationale, security notes, and action hints
Add approve/reject/approve-all draft actions:
- [a] approve selected chunk, [x] reject, [A] approve all pending
- Actions work from both the list view and the detail popup
- gRPC calls run async; result updates status bar and refreshes data
- Nav bar shows all available keybindings
Fix draft count refresh: sandbox_draft_counts now refreshes on every
tick (not just Dashboard), so the detail header badge updates in
real time.
Improve badge labels: show 'N pending' instead of a bare number in
both the dashboard sandbox list and sandbox detail header.
* refactor(policy): DB-level draft chunk dedup with hit counter and timestamps
Replace the in-memory HashSet dedup in SubmitPolicyAnalysis with a
database-level upsert. New denormalized columns on draft_policy_chunks:
- host, port: extracted from proposed_rule at insert time
- hit_count: incremented on conflict (same sandbox + host + port)
- first_seen_ms, last_seen_ms: track when the endpoint was first and
most recently proposed
A partial unique index (WHERE status IN ('pending','approved')) ensures
only one active chunk per endpoint per sandbox; rejected/superseded
chunks don't block new proposals.
Surface hit_count and first/last_seen in:
- CLI: 'openshell draft get' shows 'Hits: N (first ..., last ...)'
- TUI: detail popup shows hits row; list view shows 'Nx' suffix
* fix(policy): optimistic retry on policy version conflicts + structured logging
merge_chunk_into_policy and remove_chunk_from_policy now retry up to 5
times on UNIQUE constraint violations (version conflicts from concurrent
approvals). Each attempt re-reads the latest policy, re-merges the rule,
and increments the version. This eliminates the race condition where
rapid successive approvals would fail with a DB error.
Add structured tracing to all draft action handlers:
- ApproveDraftChunk: logs rule_name, host, port, hit_count before merge
and version + policy_hash after success
- RejectDraftChunk: logs rule_name, host, port, reason
- ApproveAllDraftChunks: logs pending_count at start, per-chunk merge
progress, and final summary with chunks_approved/skipped
- UndoDraftChunk: logs before/after with rule_name and version
- Retry attempts log as warnings with attempt number and conflicting
version
* wip: forward proxy fix, mapper allowed_ips, TUI polish, CTF rewrite
* fix(tui): use correct --gateway flag for ssh-proxy ProxyCommand
* chore: add Docker cleanup script for stale images, volumes, and build cache
* feat(tui): approve-all confirmation modal and CTF cleanup
Add [A] confirmation popup that snapshots pending chunks, shows a
scrollable list, and approves each chunk individually on confirm.
This prevents approving chunks that arrived after the modal opened.
Remove transient issue #205 reference from CTF victory banner.
* fix(tui): correct import ordering for rustfmt
* wip: stateful toggle model, rename to network rules
Draft chunks now follow a toggle state machine:
pending -> approved | rejected (initial decision)
approved <-> rejected (toggle)
One row per (sandbox_id, host, port) via expanded unique index.
Rejecting an approved rule removes it from the active policy.
Re-approving a rejected rule merges it back.
Rename CLI from 'draft' to 'rule', TUI from 'Draft Recommendations'
to 'Network Rules'. State-aware keybindings: approved shows [x] Revoke,
rejected shows [a] Approve. Fix sandbox detail hiding delete confirmation
behind pending message.
* refactor(policy): move mapper sandbox-side, slim schema, per-binary granularity
Move mechanistic mapper from gateway to sandbox so all analysis runs
sandbox-side (N sandboxes = N independent pipelines). Gateway is now a
thin validate + persist + approval layer.
Architectural changes:
- Move mechanistic_mapper.rs from navigator-server to navigator-sandbox
- Sandbox flush flow: aggregator drains -> mapper runs -> proposals sent
- Gateway SubmitPolicyAnalysis: validate + persist only, no mapper
- Drop denial_summaries table (write-only, zero readers)
- Consolidate migrations 003+004+005 into single 003
Schema slimming:
- Drop 5 unused columns from draft_policy_chunks (stage, denial_refs,
supersedes_chunk_id, analysis_mode, decided_by)
- Add per-binary granularity: binary column, widen unique index to
(sandbox_id, host, port, binary)
- Mapper groups by (host, port, binary), one proposal per triple
- Merge appends binary to existing rule; revoke removes just that binary
CTF & UX:
- 7-gate CTF: add Gate 3 (curl -> ifconfig.me:80) for per-binary demo
- TUI shows binary short name in list, full path in detail popup
- CLI output shows binary field
- Idempotent rule names, hit_count accumulates real denial counts
- Rationale text no longer bakes in stale denial count
* feat(tui): add port forwarding support to Gator (#80)
Extract forward PID management, resolve_ssh_gateway, and shell_escape from
navigator-cli into navigator-core::forward as a shared module. Add a Ports
field to the TUI create sandbox modal, a NOTES column to the sandbox table,
and a Forwards row to the sandbox detail view. On creation with ports, Gator
polls for Ready state then spawns background SSH tunnels. Forward cleanup
runs automatically on sandbox delete.
Closes#80
* fix(tui): fix Ports field invisible due to modal height overflow
The modal height calculation used content_height + 3 but block chrome
(borders + padding) is 4 rows, not 3. Combined with the 3-row Ports
field layout, the input line was clipped to zero height on standard
24-row terminals. Switch Ports to a compact single-line inline layout
and fix the chrome arithmetic.
* wip: BYOC example with port forwarding and TUI command field fix
- Move examples/bring-your-own-container.md into its own directory
- Add example Dockerfile and app.py (Python REST API with /hello endpoint)
- Rewrite README with full CLI and TUI port-forward workflows
- Change TUI Command field default from /bin/bash to empty so custom
image entrypoints run without manual clearing
* feat(tui): wire up Command field to SSH exec after sandbox creation
When a command is specified in the create sandbox modal, the TUI now
waits for the sandbox to reach Ready (with the pacman animation),
starts any port forwards, then suspends the TUI and executes the
command via SSH — matching the CLI's post-creation flow.
- Refactor spawn_create_sandbox to poll for Ready inline when ports
or command are set, keeping the animation visible throughout
- Add start_port_forwards() helper called within the create task
- Add handle_exec_command() to suspend TUI and run SSH exec
- Remove unused ForwardResult event variant (forwards are now started
within the create task, not as a separate event)
* fix(tui): fix command exec, forward timeouts, and create modal UX
- Fix handle_exec_command to suspend TUI and run SSH attached (matching
CLI behavior) so the process stays alive for the session duration
- Fix shell_escape double-quoting: escape each word individually
- Add ConnectTimeout and 20s spawn timeout to forward SSH so a stalled
auth doesn't freeze the create flow forever
- Remove dead ForwardResult event variant
- Add cluster_name to start_port_forwards for ProxyCommand
- Add spacer between Providers and Ports in create modal
- Fix Command placeholder: 'runs /bin/bash if empty' (not image entrypoint)
- Update BYOC README: document that CMD is replaced by supervisor,
command must be passed explicitly, remove TUI-specific sections
* fix(sandbox): demote Landlock fallback log from warn to debug
The Landlock filesystem sandbox emits a noisy warning when a policy
path does not exist (e.g. /app in custom images). Since BestEffort
mode intentionally continues without Landlock, demote to debug.
* docs: fix BYOC Dockerfile comments for supervisor CMD override
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(sandbox): allow egress to private IP space via allowed_ips policy field
Add an allowed_ips field to NetworkEndpoint that accepts CIDR notation and
exact IPs, enabling sandboxes to reach private IP space under policy control
while maintaining SSRF protections for loopback and link-local addresses.
Three modes are supported:
- Default (no allowed_ips): all private IPs blocked (existing behavior)
- Host + allowlist: domain must resolve to an IP in allowed_ips
- Hostless allowlist (no host): any domain allowed if resolved IP matches
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Closes#34
## Summary
- Add `nav ssh-proxy` name mode (`--cluster`/`--name`) and `nav sandbox ssh-config` for VS Code Remote-SSH integration
- Fix VS Code Remote-SSH platform misdetection (was defaulting to Windows/PowerShell)
- Fix sandbox network policy for VS Code server connectivity
- Fix sandbox user shell configuration
## Problem
VS Code Remote-SSH was failing to connect to Navigator sandboxes due to three issues:
1. **Platform misdetection**: VS Code sends `uname -rsv` after a `ready:` marker and expects a response within ~1ms. Forced PTY allocation in `shell_request`, interactive bash (`-i`), and leaked supervisor env vars added enough latency that VS Code timed out and defaulted to "Platform: windows", then tried `powershell`.
2. **Broken network policy**: The `vscode` entry in `dev-sandbox-policy.yaml` was at the YAML top-level instead of nested under `network_policies:` (indentation bug). The VS Code server binary also needed a glob entry.
3. **No login shell**: The sandbox user had `/usr/sbin/nologin` as its shell.
## Changes
| File | Change |
|------|--------|
| `crates/navigator-sandbox/src/ssh.rs` | Remove forced PTY in `shell_request`, remove `-i` flag from pipe exec, add `env_clear()` + explicit `SHELL`/`PATH` to both shell spawn paths |
| `deploy/docker/sandbox/Dockerfile.base` | Change sandbox user shell to `/bin/bash`, set home to `/sandbox`, add `.bashrc`/`.profile` |
| `dev-sandbox-policy.yaml` | Fix `vscode:` indentation under `network_policies:`, add `/sandbox/.vscode-server/*` binary glob |
| `crates/navigator-cli/src/main.rs` | Add `--cluster`/`--name` to `SshProxy`, add `SshConfig`/`Policy`/`Logs` subcommands (rebase merge) |
| `crates/navigator-cli/src/ssh.rs` | Update `print_ssh_config` to use `--cluster` flag |
| `examples/vscode-remote-sandbox.md` | User-facing example and docs |
## How to use
```bash
# Create a persistent sandbox
nav sandbox create --keep my-sandbox
# Generate SSH config
nav sandbox ssh-config my-sandbox >> ~/.ssh/config
# Open VS Code
code --remote ssh-remote+nav-my-sandbox /sandbox
```
## Testing
- All 307 tests pass
- Clippy clean on modified crates
> **🏗️ build-from-issue-agent**
Closes#79
## Summary
Replaces the gateway-proxied inference model (`ProxyInference` RPC) with sandbox-local execution. The sandbox now resolves routes from either a standalone YAML route file or a cluster bundle fetched via the new `GetSandboxInferenceBundle` RPC, then forwards requests directly to inference backends using `navigator-router`.
### Benefits of removing the gRPC hop
Moving inference execution from the gateway to the sandbox eliminates the gRPC round-trip for every inference request:
- **No payload size limits**: Direct HTTP forwarding replaces protobuf-over-gRPC serialization.
- **Streaming-ready**: Preserves connection semantics for future SSE support (e.g., `stream: true`).
- **Lower latency**: Direct sandbox → backend path instead of sandbox → gateway → backend round-trip.
- **Reduced gateway load**: Gateway only serves the control-plane bundle delivery RPC.
- **Simpler error model**: HTTP status codes flow through directly without gRPC wrapping.
## Changes Made
- **Proto**: Removed `ProxyInference` RPC. Added `GetSandboxInferenceBundle` RPC for route bundle delivery.
- **Gateway**: Replaced `proxy_inference` with `get_sandbox_inference_bundle`. Removed `Router` and `navigator-router` dependency — gateway is now control-plane only for inference.
- **Router**: Switched config file format from TOML to YAML. Added `Debug` impl that redacts API keys.
- **Sandbox**: New `InferenceContext` with local `Router` + route cache. Routes loaded from `--inference-routes` file (standalone) or cluster bundle via gRPC, with 30s background refresh. New `--inference-routes` / `NAVIGATOR_INFERENCE_ROUTES` CLI arg.
- **Dev sandbox**: Added `inference-routes.yaml` at repo root with default NVIDIA NIM route. `mise run sandbox` now mounts it automatically and supports `-e` flag to forward env vars (e.g., `mise run sandbox`).
- **Example**: Added standalone example (`examples/inference/routes.yaml`) and rewrote README to cover both standalone and cluster workflows.
- **E2E tests**: Updated existing test names/docs. Added tests for Anthropic messages protocol and multi-route policy filtering.
## Tests Added
- **Unit (21 new):** `router_error_to_http` (6 variants), `load_from_file` YAML round-trip (5), `resolve_sandbox_inference_bundle` gRPC handler (6), `build_inference_context` route loading (4)
- **E2E (2 new):** Anthropic messages protocol routing, route filtering by `allowed_routes`
- **Total:** 188 unit tests passing across 3 crates
## Documentation Updated
- `architecture/gateway.md`: Removed router, updated Inference Service
- `architecture/sandbox.md`: Updated orchestration flow, InferenceContext, route loading
- `architecture/inference-routing.md`: Complete rewrite for sandbox-local architecture
## Verification
- [x] All unit tests passing (188 across 3 crates)
- [x] Pre-commit Rust checks passing (fmt, clippy, tests)
- [x] Architecture documentation updated
- [x] E2E tests updated with new test cases
- [x] Standalone example and documentation added
- [x] Default inference-routes.yaml with env passthrough for mise run sandbox
Closes#67
## Summary
Implements transparent inference interception and routing for sandboxed AI agents. The sandbox proxy intercepts outbound AI SDK calls (OpenAI, Anthropic) and reroutes them through the gateway to policy-controlled backends — enabling organizations to redirect inference traffic to local or self-hosted models without modifying agent code.
**Decision model** — a tri-state OPA evaluation for every CONNECT request:
1. Binary + endpoint explicitly allowed in `network_policies` → **allow** (pass through)
2. Not explicitly allowed + `inference.allowed_routes` configured → **inspect for inference** (TLS intercept, detect API patterns, route through gateway)
3. Otherwise → **deny**
No endpoint declarations or binary lists needed for inference routing. Just configure `inference.allowed_routes`.
## Key Changes
### Sandbox (interception)
- **OPA policy**: New `network_action` Rego rule with three outcomes (`allow`, `inspect_for_inference`, `deny`). New `NetworkAction` enum replaces `PolicyDecision.allowed` bool for the proxy's main decision path.
- **Proxy**: New `InspectForInference` path — TLS-terminates client, parses HTTP, detects inference API patterns (`POST /v1/chat/completions`, `/v1/completions`, `/v1/messages`), strips auth headers, forwards via gRPC.
- **New module**: `l7/inference.rs` — `InferenceApiPattern`, `detect_inference_pattern()`, HTTP request/response parsing.
- **gRPC client**: New `proxy_inference()` for sandbox→gateway forwarding.
- **Sandbox init**: Creates OPA engine when inference is configured, even without `network_policies`.
### Gateway (dispatch)
- **InferenceService**: `ProxyInference` RPC loads sandbox policy, resolves allowed routes, dispatches to router. Full CRUD for inference routes.
- **Proto**: `InferenceRoute`, `InferenceRouteSpec`, `ProxyInferenceRequest/Response`, Inference gRPC service.
### Router (backend proxying)
- **New crate**: `navigator-router` with `Router`, `proxy_with_candidates()`, protocol-based route selection, backend HTTP proxying with auth header rewriting.
- **Mock support** for testing (`mock://` scheme).
### CLI
- `nav inference create/update/delete/list` commands for route management.
### Python SDK
- Updated protobuf bindings. Removed old `inference.py` client (replaced by transparent interception).
### Documentation
- New `architecture/inference-routing.md` — end-to-end system documentation.
- Updated `architecture/sandbox.md` — proxy, OPA, and source index sections.
- Updated `architecture/README.md` — new subsystem overview and diagram.
## Addendum: Chunked Transfer Compatibility
This branch now also fixes intercepted SDK requests that send chunked request bodies:
- `inspect_for_inference` now accepts `Transfer-Encoding: chunked` and decodes chunked request bodies before forwarding to the gateway
- Removed the prior `411 Length Required` response for chunked intercepted requests
- Added request/response header sanitization for framing and hop-by-hop headers (`content-length`, `transfer-encoding`, `connection`, etc.) to keep forwarded requests and returned responses valid
- Added unit tests for chunked parsing and header sanitization
Note: this improves compatibility for streaming-style SDK request patterns; true token-by-token passthrough response streaming is still a separate follow-up.
## Minimal Policy for Inference Routing
```yaml
inference:
allowed_routes:
- local
```
Any outgoing connection from a binary not explicitly allowed in `network_policies` will be intercepted and checked for inference API patterns.
## Test Plan
- [x] `cargo test --workspace` — all tests pass
- [x] `mise run pre-commit` — all checks pass
- [x] E2E: OpenAI chat completions routed through gateway
- [x] E2E: Anthropic messages routed through gateway
- [x] E2E: Python OpenAI SDK from sandbox (`examples/inference/inference.py`)
- [x] E2E test: `e2e/python/test_inference_routing.py`
Seth Jennings