When the VM driver extracts OCI image layers on a non-root host (e.g.
macOS UID 501), fs::copy and tar::unpack create files owned by the host
user. The sandbox user inside the VM then cannot write to its own home
directory.
Add a conditional chown in the VM init script that detects the mismatch
and fixes ownership before the supervisor starts. The check is skipped
when ownership is already correct (e.g. Linux-as-root extraction).
Signed-off-by: Florent Benoit <fbenoit@redhat.com>
The sandbox_to_k8s_spec function was inserting logLevel and environment
fields at the Sandbox CR spec level. Neither field exists in the
agents.x-k8s.io/v1alpha1 CRD schema, so Kubernetes silently dropped them
and the log_level value never reached the sandbox pod.
Remove the dead CR-level insertions. Build a merged environment map that
includes OPENSHELL_LOG_LEVEL when log_level is set, and pass it to
sandbox_template_to_k8s so the value flows through build_env_list into
the pod container spec.
Add a unit test verifying OPENSHELL_LOG_LEVEL appears in the container
env and that logLevel is absent from the CR spec.
Fixes#1265
* feat(k8s): support ImageVolumeSource for supervisor sideload
Add a config-driven switch between two supervisor binary delivery
methods: image-volume (default) mounts the supervisor OCI image
directly as a read-only volume (K8s >= v1.33), while init-container
preserves the existing emptyDir + copy-self pattern for older clusters.
Configurable via --supervisor-sideload-method CLI arg,
OPENSHELL_SUPERVISOR_SIDELOAD_METHOD env var, or Helm
supervisor.sideloadMethod value.
* feat(helm): auto-detect supervisor sideload method from cluster version
Use Helm .Capabilities.KubeVersion to choose the supervisor sideload
method automatically: image-volume on K8s >= v1.33, init-container on
older clusters. An explicit supervisor.sideloadMethod value overrides
auto-detection.
* fix(helm): raise auto-detect threshold to K8s v1.35
The ImageVolume feature gate is only enabled by default starting in
K8s v1.35, not v1.33 (where it is beta but off by default). Clusters
on v1.33-v1.34 can still opt in with an explicit sideloadMethod value.
Colima, Lima, Rancher Desktop, and OrbStack all run dockerd inside a
host VM. Their bridge gateway IP is reachable from inside containers
but not from the OpenShell server process running on the host, the
same constraint Docker Desktop has — yet the existing
is_docker_desktop check rejects them, leaving callbacks routed at a
bridge IP nothing on the host can listen on.
Detect these runtimes by daemon Name (Lima sets the VM hostname to
colima*, lima-*, rancher-desktop, orbstack) and supplemental labels
(dev.rancherdesktop.*, dev.orbstack.*), and route them through
host-gateway like Docker Desktop.
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Disables automountServiceAccountToken in sandbox pods for security
hardening. Sandbox pods should not have access to the Kubernetes API
by default.
Adds test case to verify the pod spec includes the disabled setting.
Signed-off-by: Derek Carr <decarr@redhat.com>
* docs(rfc): add agent-driven policy management
* docs(rfc): switch policy MVP to local API
* docs(rfc): clarify policy advisor skill and local logs
* feat(sandbox): add agent-driven policy proposal loop
* test(examples): add codex policy dogfood loop
* refactor(examples): make policy demo agent-agnostic
* refactor(examples): colocate policy validation harness
* docs(examples): add policy demo env sample
* docs(examples): use placeholder env example
* feat(sandbox): wire policy.local denials to OCSF JSONL log
Wires GET /v1/denials?last=N on the sandbox-local policy advisor API to read
recent OCSF JSONL events from /var/log/openshell-ocsf.YYYY-MM-DD.log, filter
to network/L7 denials (action_id=2, class_uid 4001/4002), and return a
compact summary newest-first. Default limit is 10, capped at 100. Ran inside
spawn_blocking so file I/O does not block the policy.local handler.
Other cleanup:
- POST /v1/proposals now uses the typed grpc_client wrapper instead of
raw_client, so accepted/rejected counts surface to the agent uniformly.
Wrapper return type extended to the response struct.
- Drop the 'add_rule' snake_case alias in the proposal JSON; canonical form
is camelCase 'addRule', matching the PolicyMergeOperation convention used
elsewhere.
- skills/policy_advisor.md updated to match: documents the now-real
/v1/denials?last=10 endpoint and uses 'addRule' consistently.
- skills.rs test asserts on the canonical 'addRule' phrase rather than the
removed 'PolicyMergeOperation' substring.
* feat(cli): show L7 protocol/method/path in rule get output
format_endpoint() previously rendered only host:port, dropping protocol,
access, and the L7 rules array. That made openshell rule get text output
unable to distinguish a broad L4 grant from a method/path-scoped L7 REST
rule -- exactly the distinction a developer needs at approval time.
New rendering tags each endpoint with its enforcement layer and surfaces
allow/deny rules:
bare L4: api.example:443 [L4]
L7 read-only: api.example:443 [L7 rest, access=read-only]
L7 method/path: api.example:443 [L7 rest, allow PUT /v1/foo/bar]
Pure display change: no proto, gateway, or behavior changes. Unit test
covers all three rendering cases with synthetic fixtures.
* refactor(examples): rewrite policy demo as Codex-default loop
Re-shape examples/agent-driven-policy-management/ to be a single, clean
end-to-end demonstration of the agent-driven policy loop. A Codex agent
inside an OpenShell sandbox attempts a GitHub Contents API write, hits a
structured 403 from the L7 proxy, reads the policy_advisor skill, drafts a
narrow addRule proposal via http://policy.local/v1/proposals, the host
auto-approves, the sandbox hot-reloads policy, and the agent's retry
succeeds. Whole loop runs in roughly two minutes.
Demo cleanup:
- Drop .env file ceremony. Defaults resolve from gh: owner via
'gh api user --jq .login', repo defaults to 'openshell-policy-demo',
token from gh auth token / GITHUB_TOKEN / GH_TOKEN. With gh auth login
and codex login already done, 'bash demo.sh' Just Works.
- Codex-specific. Bootstraps ~/.codex/auth.json from credentials injected
by the OpenShell provider, runs codex exec --sandbox danger-full-access
(OpenShell is the actual security boundary; bwrap nesting cannot create
user namespaces inside the sandbox container).
- Tighter narrative output: a single 'Preflight' step, a run summary banner
before launch, an inline narration of what's happening inside the sandbox
while we poll for the proposal (including the literal structured 403
body the agent acts on), and an OCSF trace at the end filtered to the
three events that tell the story (DENY, RELOAD, ALLOW).
- Replace Python heredoc templating with sed; uploads use the single-flag
pattern (--upload "${PAYLOAD_DIR}:/sandbox") with files referenced at
the basename-prefixed path that #952 / #1028 established.
- README documents the trust model honestly: structured rule is the
contract, agent rationale is a hint, prover validation badge in
progress per RFC 0001.
Move the deterministic no-LLM regression harness out of examples/ into
e2e/policy-advisor/ -- it was a parallel demo, not an example. Same loop
without the LLM, useful for iterating on the proxy and policy.local API.
* style(sandbox,cli): apply rustfmt
Whitespace-only fixups caught by mise run pre-commit. No functional change.
* perf(examples): cap Codex reasoning at 'low' in policy demo
The demo task is mechanical (one HTTP request, parse a structured 403,
post a JSON proposal, retry). Codex's default high-effort reasoning
roughly doubles the demo's wall time without improving outcomes; running
at 'low' lands the same minimal L7 grant in roughly half the time.
Override with DEMO_CODEX_REASONING=medium (or higher) to compare runs.
* fix(sandbox): harden policy.local denials endpoint
Three changes addressing review feedback before merging the agent-driven
policy management MVP:
- Distinguish "OCSF JSONL enabled, no denials" from "OCSF JSONL disabled,
nothing to read." The endpoint now returns a `log_available` flag and an
explanatory `note` when the log file is missing, so the in-sandbox agent
can give the developer an accurate hint instead of a misleading empty
list.
- Stop echoing the OCSF `message` field in the per-denial summary. The
proxy's denial messages can include the request path with query string
(e.g., `?access_token=...`); the structured `host`/`port`/`method`/
`path`/`binary` fields carry everything the agent needs to draft a
proposal, and `path` is sourced from `http_request.url.path` which
already excludes the query string.
- Cap `read_request_body` at a 15s timeout. Bounds slowloris-style stalls
from a misbehaving in-sandbox process. The proxy listener only accepts
loopback connections so practical impact is small, but this is cheap
defense-in-depth.
New tests cover the missing-log signal and the message-redaction guarantee.
* fix(examples): redact tokens in agent log tail and validate DEMO_FILE_DIR
Two small hardening passes on the policy management demo:
- `fail()` now pipes the agent log tail through a redactor that masks the
GitHub token and Codex credential triple before printing. Codex itself is
well-behaved about not echoing the token, but a misbehaving tool call
could leak it; this is a final safety net before the log hits the
developer's terminal (and any clipboard or chat history that follows).
- `validate_env` now regex-checks DEMO_FILE_DIR with the same allow-list
the other path-shaped variables use. The value is interpolated through
sed with `|` as the delimiter when rendering the agent task; rejecting
unsupported characters keeps the templating predictable and stops a
user-supplied value from breaking out into a shell context.
* refactor(sandbox): centralize policy.local routes and skill path
Addresses review feedback that the deny body's `next_steps` array and the
route table could drift apart. The route paths and skill location now live
as `pub const`s in `policy_local.rs` and feed both:
- the dispatcher in `route_request` that matches against them
- a new `agent_next_steps()` helper that builds the JSON the L7 deny body
embeds
`l7/rest.rs::deny_response_body` calls `policy_local::agent_next_steps()`
instead of inlining the array, so adding or renaming a route is a one-line
change in `policy_local.rs` and the agent contract follows automatically.
* feat(sandbox): switch /v1/denials to shorthand log pass-through
Previously /v1/denials parsed `/var/log/openshell-ocsf.*.log` (OCSF JSONL)
and returned structured per-event objects. JSONL is opt-in via
`ocsf_json_enabled`, so the endpoint returned an empty list with a "log
not enabled" hint by default — agents had to navigate a setup step before
the inspect-recent-denials guidance was useful.
Switch to reading the shorthand log at `/var/log/openshell.*.log`, which
is always-on and the same human-readable format `openshell logs` displays.
The endpoint now returns raw shorthand lines (newest first) — the agent
reads them directly, no field parsing.
Tradeoffs:
- Removes the JSONL-on-by-default debate: shorthand is already on, no
defaults change.
- Updating shorthand is a single-file change in this repo; no schema rev
needed when we want to add fields.
Implementation:
- `read_recent_denial_lines` walks shorthand log files newest-first,
filters lines with ` OCSF ` AND ` DENIED ` (the OCSF action label,
uppercase, space-bounded).
- `collect_shorthand_log_files` matches `openshell.<date>.log`; the
trailing dot in `SHORTHAND_LOG_PREFIX = "openshell."` excludes
`openshell-ocsf.<date>.log` so JSONL-on doesn't bleed into responses.
- 4096-byte cap per surfaced line as defense against pathological inputs.
- Skill doc updated to reflect that `/v1/denials` returns raw shorthand
lines, not structured fields.
Defense-in-depth on query-string secrets:
- `redact_query_strings` strips `?<query>` to `?[redacted]` from each
surfaced line. The L7 relay path emits OCSF events using
`redacted_target` (secret-placeholder redaction), but the FORWARD deny
path in `proxy.rs` populates `OcsfUrl::new("http", host, path, port)`
and `.message(...)` with the raw request path — query string included.
Stripping queries at the consumer guards `/v1/denials` regardless of
whether the upstream emit sites are tightened. The on-disk log is not
rewritten by this change; that is a separate hardening task tracked
for the FORWARD path emit sites in proxy.rs.
- `truncate_at_char_boundary` is UTF-8 safe; redaction runs before
truncation so a cut cannot slice mid-secret.
Tests:
- `recent_denials_returns_newest_first_from_shorthand_lines` covers the
happy path with mixed allowed/denied/non-OCSF lines.
- `recent_denials_skips_jsonl_log_files` confirms JSONL files don't
surface even if present.
- `recent_denials_truncates_pathological_lines` covers the cap.
- `is_ocsf_denial_line_filters_correctly` covers the line-level filter.
- `redact_query_strings_removes_query_from_url_token` and
`redact_query_strings_removes_query_in_reason_tag` cover the redaction
in both URL token and `[reason:...]` contexts.
- `truncate_at_char_boundary_does_not_panic_on_multibyte` covers the
UTF-8 safety.
* chore(sandbox): align proto inits with main's L7 GraphQL additions
Post-rebase fixups after #1083 (GraphQL L7 inspection) landed on main and
introduced new fields on the proto types this branch constructs:
- `crates/openshell-sandbox/src/l7/relay.rs`: two `deny_with_redacted_target`
call sites (REST and GraphQL relay deny paths) now pass the
`DenyResponseContext` argument that `rest::send_deny_response` expects.
Both sites pass `host`, `port`, and `binary` from the existing
`L7EvalContext`, matching the pattern used at the primary deny site.
- `crates/openshell-sandbox/src/policy_local.rs`: `L7Allow`, `L7DenyRule`,
and `NetworkEndpoint` proto initializers now populate the new GraphQL
and path-scoping fields with empty defaults. Agent-authored proposals
via `policy.local` target REST/SQL/L4 today; GraphQL operation matching
is set on the gateway side or via direct YAML, so empty defaults are
correct here.
No behavior change. `cargo test -p openshell-sandbox --lib` (650 tests) and
`cargo clippy -p openshell-sandbox --lib --tests -- -D warnings` clean.
* feat(sandbox): gate agent policy proposals behind opt-in feature flag
The agent-driven policy proposal surface delivered by this PR (skill
install, `policy.local` API, `next_steps` array on L7 deny bodies) is
now opt-in via the new `agent_policy_proposals_enabled` setting. Default
false. Same shape as `providers_v2_enabled`: registered in
`openshell-core::settings`, sandbox-level, hot-toggleable via the
existing settings poll loop.
Why: the surface is a novel agent-controlled mutation point in every
sandbox. The per-proposal developer approval gate is a correctness
control, but it doesn't address "should this sandbox have an
agent-authoring API at all" — compliance teams may want that question
closed. The flag is the second gate.
Implementation:
- New registry entry + `AGENT_POLICY_PROPOSALS_ENABLED_KEY` constant in
`openshell-core::settings`.
- `lib.rs`: process-wide `OnceLock<Arc<AtomicBool>>` mirroring the
`OCSF_CTX` pattern. `agent_proposals_enabled()` is the single read
point.
- Initial settings fetch added to `run_sandbox` so skill install honors
the flag at startup (not just on the poll loop's first tick).
- Skill install in `run_sandbox` is gated on the flag.
- `policy_local::route_request` returns `404 feature_disabled` for all
routes when the flag is off — including the otherwise-public
`current_policy` and `denials` routes. When the surface is off it's
off entirely.
- `policy_local::agent_next_steps` returns an empty array when the flag
is off so deny bodies don't advertise routes that 404.
- Poll loop updates the atomic on each tick, lazily installs the skill
on a false→true transition (no claw-back on true→false; stale skill
on disk is harmless because route + next_steps gate on the live atom).
Tests:
- Shared `test_helpers::ProposalsFlagGuard` mutex+atomic guard for the
process-wide flag, used across `policy_local::tests` and
`l7::rest::tests`.
- New: `agent_next_steps_returns_empty_when_flag_off`,
`agent_next_steps_returns_full_array_when_flag_on`,
`route_request_returns_feature_disabled_when_flag_off`.
- Updated existing tests that exercise the deny body or the route
dispatcher to set the flag on first.
- Full sandbox lib test suite: 653 pass, clippy clean.
Demo and e2e:
- `examples/agent-driven-policy-management/demo.sh` and
`e2e/policy-advisor/test.sh` now snapshot the prior global value of
the setting, set it to true before sandbox creation (so the
supervisor's initial poll picks it up), and restore on exit (delete
if previously unset, otherwise write the prior value back).
Docs:
- RFC 0001 MVP-implementation note documents the flag, default, and
intended soft-launch posture.
* test(policy-advisor): require proposal opt-in for e2e
* refactor(sandbox): group policy poll loop state
* test(e2e): isolate Kubernetes user namespace test
---------
Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
Add opt-in support for Kubernetes user namespace isolation on sandbox
pods. When enabled, container UID 0 maps to an unprivileged host UID
and capabilities become namespaced, providing defense-in-depth for the
supervisor process.
Configuration is two-layered: a cluster-wide default via
OPENSHELL_ENABLE_USER_NAMESPACES (default false) and a per-sandbox
override via the new `user_namespaces` field on SandboxTemplate.
When user namespaces are active, the pod security context is extended
with SETUID, SETGID, and DAC_READ_SEARCH capabilities to match the
bounding-set requirements inside a user namespace.
Introduces SandboxPodParams struct to replace long argument lists on
sandbox_to_k8s_spec and sandbox_template_to_k8s.
Validated end-to-end on OCP 4.22 (K8s 1.35.3, CRI-O 1.35, RHEL
CoreOS, kernel 5.14) with full SSH tunnel and non-identity UID mapping.
Closes#1206
Warn when gateway select saves a new active gateway while OPENSHELL_GATEWAY is set to a different value, and document the override precedence.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
Allow CLI to connect to gateways with self-signed or untrusted
certificates by skipping TLS verification when --gateway-insecure
is set via flag or OPENSHELL_GATEWAY_INSECURE env var.
* fix(sandbox): add copy-self subcommand for scratch-image init container
The supervisor image is built FROM scratch and contains only the
openshell-sandbox binary. The Kubernetes side-load init container
previously invoked `sh -c "cp ..."`, which failed with
`executable file 'sh' not found in $PATH` because the scratch image
has no shell.
Add a `copy-self <DEST>` subcommand to the supervisor binary that
resolves its own executable, copies it to the destination path, and
sets mode 0755. Update the Kubernetes driver init container to invoke
this subcommand directly. Mirrors argoexec's emissary self-copy pattern.
* fix(docker): use ubuntu base for supervisor image
`FROM scratch` cannot exec the dynamically linked supervisor binary
(no `/lib/ld-linux-*.so.*`), so the Kubernetes side-load init container
hit `exec /openshell-sandbox: no such file or directory` even after
the `copy-self` subcommand replaced the broken `sh -c` command.
Switch the supervisor stage to `nvcr.io/nvidia/base/ubuntu:noble-20251013`
— the same base the gateway image already uses — so glibc and the
dynamic loader are present. Docker/Podman drivers continue to work
unchanged (they extract or volume-mount the binary into agent containers
that already have glibc); the Kubernetes init container now succeeds
end-to-end.
Verified on a k3d cluster: init container `openshell-supervisor-install`
exits 0 on first try, binary lands at `/opt/openshell/bin/openshell-sandbox`
(mode 0755), and the agent container reaches Ready.
* docs(docker): refresh supervisor target description
The file header still described the supervisor image as
`FROM scratch, binary only`. Update it to match the current
Ubuntu-base supervisor stage.
Wires up a first-class `gateway list` command that calls the existing
`run::gateway_list()` helper. Resolves the dead reference in
`crates/openshell-bootstrap/src/errors.rs:248`, where the port-conflict
recovery hint already pointed users at this command.
Updates the manage-gateways doc to teach the new verb instead of the
`gateway select`-with-no-arg fallback.
Closes#1173
Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>
* fix(driver-podman): use explicit cap_drop instead of cap_drop:ALL
cap_drop:ALL removes every capability in Podman's default set — including
CAP_DAC_OVERRIDE, CAP_CHOWN, CAP_FOWNER, CAP_SETUID, CAP_SETGID — and the
original cap_add list failed to re-add them which we need.
We also don't need to add CAP_SETUID and CAP_SETGID - those are defaults.
Signed-off-by: Colin Walters <walters@verbum.org>
* fix(e2e): fix gateway health check and SSH port in e2e-podman.sh
Two bugs in the Podman e2e gateway startup:
1. Health check: curl probed /healthz on the main gRPC port, which always
returns 404 because the health router is only served on --health-port
(a separate listener, disabled by default). Allocate a second free port
and pass --health-port to the gateway; health check now probes that port.
2. SSH port: --ssh-gateway-port was not set, so the gateway always told
clients to connect SSH to port 8080 regardless of the actual port.
Explicitly pass --ssh-gateway-port ${PORT} to fix sandbox exec/SSH.
Signed-off-by: Colin Walters <walters@verbum.org>
---------
Signed-off-by: Colin Walters <walters@verbum.org>
Add a UUID-based request-ID middleware using tower-http's request-id
feature. Each inbound request receives a unique x-request-id header
(or preserves a client-supplied one), which is recorded in the tracing
span and propagated to the response.
This enables operators to correlate log lines across the middleware
stack for a single request under concurrent load, and lets clients
reference specific requests in bug reports.
Signed-off-by: sauagarwa <sauagarw@redhat.com>
* fix(helm): grant node read access for GPU capacity checks
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
* ci: add WSL GPU failure diagnostics
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
* fix(gpu): bump device plugin for WSL CDI
* fix(sandbox): allow WSL GPU paths in Landlock
Allow /dev/dxg and /usr/lib/wsl as GPU baseline paths so WSL CDI GPU sandboxes can initialize NVML. Native Linux skips these entries when the paths do not exist.
* ci: remove temporary WSL GPU diagnostics
---------
Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
relay_response_to_client had no callers; all call sites use relay_response
directly. The allow(dead_code) comment on parse_target_query was also
misleading since proxy.rs calls it from production code, so that annotation
is removed along with the incorrect comment.
User::from_name / Group::from_name may return Ok(None), producing our explicit "Sandbox … not found" errors, or Err when NSS fails first with ENOENT ("ENOENT: No such file or directory") before a definitive unknown-name result.
Accept either wording so nonexistent user/group tests stay meaningful across environments.
Signed-off-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
Walk ../../proto recursively instead of maintaining a hard-coded list. Sort paths for deterministic codegen and keep cargo:rerun-if-changed scoped to the proto tree.
Signed-off-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
When no drivers are explicitly configured, the server now automatically
detects the appropriate compute driver by checking the runtime environment:
- Kubernetes: detected via KUBERNETES_SERVICE_HOST env var (set inside pods)
- Podman: detected by checking if podman binary is available on PATH
- Docker: detected by checking if docker binary is available on PATH
Priority order: Kubernetes → Podman → Docker. VM is never auto-detected
and must be selected explicitly via --drivers vm.
The Auto variant is internal-only and does not serialize to config files.
The default --drivers value is now empty, triggering auto-detection.
Florent BENOIT