trending/OpenShell
1
0
Fork 0
mirror of https://github.com/NVIDIA/OpenShell.git synced 2026-10-21 06:11:31 +08:00
Code Issues Packages Projects Releases Wiki Activity
639 Commits 504 Branches 156 Tags
md/python-sdk-demo-notebook
Commit Graph
79 Commits
Author SHA1 Message Date
Alexander WatsonandJohn Myers 1c79b2131f feat: agent-driven policy management MVP (#1151) 
* docs(rfc): add agent-driven policy management

* docs(rfc): switch policy MVP to local API

* docs(rfc): clarify policy advisor skill and local logs

* feat(sandbox): add agent-driven policy proposal loop

* test(examples): add codex policy dogfood loop

* refactor(examples): make policy demo agent-agnostic

* refactor(examples): colocate policy validation harness

* docs(examples): add policy demo env sample

* docs(examples): use placeholder env example

* feat(sandbox): wire policy.local denials to OCSF JSONL log

Wires GET /v1/denials?last=N on the sandbox-local policy advisor API to read
recent OCSF JSONL events from /var/log/openshell-ocsf.YYYY-MM-DD.log, filter
to network/L7 denials (action_id=2, class_uid 4001/4002), and return a
compact summary newest-first. Default limit is 10, capped at 100. Ran inside
spawn_blocking so file I/O does not block the policy.local handler.

Other cleanup:

- POST /v1/proposals now uses the typed grpc_client wrapper instead of
  raw_client, so accepted/rejected counts surface to the agent uniformly.
  Wrapper return type extended to the response struct.
- Drop the 'add_rule' snake_case alias in the proposal JSON; canonical form
  is camelCase 'addRule', matching the PolicyMergeOperation convention used
  elsewhere.
- skills/policy_advisor.md updated to match: documents the now-real
  /v1/denials?last=10 endpoint and uses 'addRule' consistently.
- skills.rs test asserts on the canonical 'addRule' phrase rather than the
  removed 'PolicyMergeOperation' substring.

* feat(cli): show L7 protocol/method/path in rule get output

format_endpoint() previously rendered only host:port, dropping protocol,
access, and the L7 rules array. That made openshell rule get text output
unable to distinguish a broad L4 grant from a method/path-scoped L7 REST
rule -- exactly the distinction a developer needs at approval time.

New rendering tags each endpoint with its enforcement layer and surfaces
allow/deny rules:

  bare L4:           api.example:443 [L4]
  L7 read-only:      api.example:443 [L7 rest, access=read-only]
  L7 method/path:    api.example:443 [L7 rest, allow PUT /v1/foo/bar]

Pure display change: no proto, gateway, or behavior changes. Unit test
covers all three rendering cases with synthetic fixtures.

* refactor(examples): rewrite policy demo as Codex-default loop

Re-shape examples/agent-driven-policy-management/ to be a single, clean
end-to-end demonstration of the agent-driven policy loop. A Codex agent
inside an OpenShell sandbox attempts a GitHub Contents API write, hits a
structured 403 from the L7 proxy, reads the policy_advisor skill, drafts a
narrow addRule proposal via http://policy.local/v1/proposals, the host
auto-approves, the sandbox hot-reloads policy, and the agent's retry
succeeds. Whole loop runs in roughly two minutes.

Demo cleanup:

- Drop .env file ceremony. Defaults resolve from gh: owner via
  'gh api user --jq .login', repo defaults to 'openshell-policy-demo',
  token from gh auth token / GITHUB_TOKEN / GH_TOKEN. With gh auth login
  and codex login already done, 'bash demo.sh' Just Works.
- Codex-specific. Bootstraps ~/.codex/auth.json from credentials injected
  by the OpenShell provider, runs codex exec --sandbox danger-full-access
  (OpenShell is the actual security boundary; bwrap nesting cannot create
  user namespaces inside the sandbox container).
- Tighter narrative output: a single 'Preflight' step, a run summary banner
  before launch, an inline narration of what's happening inside the sandbox
  while we poll for the proposal (including the literal structured 403
  body the agent acts on), and an OCSF trace at the end filtered to the
  three events that tell the story (DENY, RELOAD, ALLOW).
- Replace Python heredoc templating with sed; uploads use the single-flag
  pattern (--upload "${PAYLOAD_DIR}:/sandbox") with files referenced at
  the basename-prefixed path that #952 / #1028 established.
- README documents the trust model honestly: structured rule is the
  contract, agent rationale is a hint, prover validation badge in
  progress per RFC 0001.

Move the deterministic no-LLM regression harness out of examples/ into
e2e/policy-advisor/ -- it was a parallel demo, not an example. Same loop
without the LLM, useful for iterating on the proxy and policy.local API.

* style(sandbox,cli): apply rustfmt

Whitespace-only fixups caught by mise run pre-commit. No functional change.

* perf(examples): cap Codex reasoning at 'low' in policy demo

The demo task is mechanical (one HTTP request, parse a structured 403,
post a JSON proposal, retry). Codex's default high-effort reasoning
roughly doubles the demo's wall time without improving outcomes; running
at 'low' lands the same minimal L7 grant in roughly half the time.

Override with DEMO_CODEX_REASONING=medium (or higher) to compare runs.

* fix(sandbox): harden policy.local denials endpoint

Three changes addressing review feedback before merging the agent-driven
policy management MVP:

- Distinguish "OCSF JSONL enabled, no denials" from "OCSF JSONL disabled,
  nothing to read." The endpoint now returns a `log_available` flag and an
  explanatory `note` when the log file is missing, so the in-sandbox agent
  can give the developer an accurate hint instead of a misleading empty
  list.
- Stop echoing the OCSF `message` field in the per-denial summary. The
  proxy's denial messages can include the request path with query string
  (e.g., `?access_token=...`); the structured `host`/`port`/`method`/
  `path`/`binary` fields carry everything the agent needs to draft a
  proposal, and `path` is sourced from `http_request.url.path` which
  already excludes the query string.
- Cap `read_request_body` at a 15s timeout. Bounds slowloris-style stalls
  from a misbehaving in-sandbox process. The proxy listener only accepts
  loopback connections so practical impact is small, but this is cheap
  defense-in-depth.

New tests cover the missing-log signal and the message-redaction guarantee.

* fix(examples): redact tokens in agent log tail and validate DEMO_FILE_DIR

Two small hardening passes on the policy management demo:

- `fail()` now pipes the agent log tail through a redactor that masks the
  GitHub token and Codex credential triple before printing. Codex itself is
  well-behaved about not echoing the token, but a misbehaving tool call
  could leak it; this is a final safety net before the log hits the
  developer's terminal (and any clipboard or chat history that follows).
- `validate_env` now regex-checks DEMO_FILE_DIR with the same allow-list
  the other path-shaped variables use. The value is interpolated through
  sed with `|` as the delimiter when rendering the agent task; rejecting
  unsupported characters keeps the templating predictable and stops a
  user-supplied value from breaking out into a shell context.

* refactor(sandbox): centralize policy.local routes and skill path

Addresses review feedback that the deny body's `next_steps` array and the
route table could drift apart. The route paths and skill location now live
as `pub const`s in `policy_local.rs` and feed both:

- the dispatcher in `route_request` that matches against them
- a new `agent_next_steps()` helper that builds the JSON the L7 deny body
  embeds

`l7/rest.rs::deny_response_body` calls `policy_local::agent_next_steps()`
instead of inlining the array, so adding or renaming a route is a one-line
change in `policy_local.rs` and the agent contract follows automatically.

* feat(sandbox): switch /v1/denials to shorthand log pass-through

Previously /v1/denials parsed `/var/log/openshell-ocsf.*.log` (OCSF JSONL)
and returned structured per-event objects. JSONL is opt-in via
`ocsf_json_enabled`, so the endpoint returned an empty list with a "log
not enabled" hint by default — agents had to navigate a setup step before
the inspect-recent-denials guidance was useful.

Switch to reading the shorthand log at `/var/log/openshell.*.log`, which
is always-on and the same human-readable format `openshell logs` displays.
The endpoint now returns raw shorthand lines (newest first) — the agent
reads them directly, no field parsing.

Tradeoffs:
- Removes the JSONL-on-by-default debate: shorthand is already on, no
  defaults change.
- Updating shorthand is a single-file change in this repo; no schema rev
  needed when we want to add fields.

Implementation:
- `read_recent_denial_lines` walks shorthand log files newest-first,
  filters lines with ` OCSF ` AND ` DENIED ` (the OCSF action label,
  uppercase, space-bounded).
- `collect_shorthand_log_files` matches `openshell.<date>.log`; the
  trailing dot in `SHORTHAND_LOG_PREFIX = "openshell."` excludes
  `openshell-ocsf.<date>.log` so JSONL-on doesn't bleed into responses.
- 4096-byte cap per surfaced line as defense against pathological inputs.
- Skill doc updated to reflect that `/v1/denials` returns raw shorthand
  lines, not structured fields.

Defense-in-depth on query-string secrets:
- `redact_query_strings` strips `?<query>` to `?[redacted]` from each
  surfaced line. The L7 relay path emits OCSF events using
  `redacted_target` (secret-placeholder redaction), but the FORWARD deny
  path in `proxy.rs` populates `OcsfUrl::new("http", host, path, port)`
  and `.message(...)` with the raw request path — query string included.
  Stripping queries at the consumer guards `/v1/denials` regardless of
  whether the upstream emit sites are tightened. The on-disk log is not
  rewritten by this change; that is a separate hardening task tracked
  for the FORWARD path emit sites in proxy.rs.
- `truncate_at_char_boundary` is UTF-8 safe; redaction runs before
  truncation so a cut cannot slice mid-secret.

Tests:
- `recent_denials_returns_newest_first_from_shorthand_lines` covers the
  happy path with mixed allowed/denied/non-OCSF lines.
- `recent_denials_skips_jsonl_log_files` confirms JSONL files don't
  surface even if present.
- `recent_denials_truncates_pathological_lines` covers the cap.
- `is_ocsf_denial_line_filters_correctly` covers the line-level filter.
- `redact_query_strings_removes_query_from_url_token` and
  `redact_query_strings_removes_query_in_reason_tag` cover the redaction
  in both URL token and `[reason:...]` contexts.
- `truncate_at_char_boundary_does_not_panic_on_multibyte` covers the
  UTF-8 safety.

* chore(sandbox): align proto inits with main's L7 GraphQL additions

Post-rebase fixups after #1083 (GraphQL L7 inspection) landed on main and
introduced new fields on the proto types this branch constructs:

- `crates/openshell-sandbox/src/l7/relay.rs`: two `deny_with_redacted_target`
  call sites (REST and GraphQL relay deny paths) now pass the
  `DenyResponseContext` argument that `rest::send_deny_response` expects.
  Both sites pass `host`, `port`, and `binary` from the existing
  `L7EvalContext`, matching the pattern used at the primary deny site.
- `crates/openshell-sandbox/src/policy_local.rs`: `L7Allow`, `L7DenyRule`,
  and `NetworkEndpoint` proto initializers now populate the new GraphQL
  and path-scoping fields with empty defaults. Agent-authored proposals
  via `policy.local` target REST/SQL/L4 today; GraphQL operation matching
  is set on the gateway side or via direct YAML, so empty defaults are
  correct here.

No behavior change. `cargo test -p openshell-sandbox --lib` (650 tests) and
`cargo clippy -p openshell-sandbox --lib --tests -- -D warnings` clean.

* feat(sandbox): gate agent policy proposals behind opt-in feature flag

The agent-driven policy proposal surface delivered by this PR (skill
install, `policy.local` API, `next_steps` array on L7 deny bodies) is
now opt-in via the new `agent_policy_proposals_enabled` setting. Default
false. Same shape as `providers_v2_enabled`: registered in
`openshell-core::settings`, sandbox-level, hot-toggleable via the
existing settings poll loop.

Why: the surface is a novel agent-controlled mutation point in every
sandbox. The per-proposal developer approval gate is a correctness
control, but it doesn't address "should this sandbox have an
agent-authoring API at all" — compliance teams may want that question
closed. The flag is the second gate.

Implementation:
- New registry entry + `AGENT_POLICY_PROPOSALS_ENABLED_KEY` constant in
  `openshell-core::settings`.
- `lib.rs`: process-wide `OnceLock<Arc<AtomicBool>>` mirroring the
  `OCSF_CTX` pattern. `agent_proposals_enabled()` is the single read
  point.
- Initial settings fetch added to `run_sandbox` so skill install honors
  the flag at startup (not just on the poll loop's first tick).
- Skill install in `run_sandbox` is gated on the flag.
- `policy_local::route_request` returns `404 feature_disabled` for all
  routes when the flag is off — including the otherwise-public
  `current_policy` and `denials` routes. When the surface is off it's
  off entirely.
- `policy_local::agent_next_steps` returns an empty array when the flag
  is off so deny bodies don't advertise routes that 404.
- Poll loop updates the atomic on each tick, lazily installs the skill
  on a false→true transition (no claw-back on true→false; stale skill
  on disk is harmless because route + next_steps gate on the live atom).

Tests:
- Shared `test_helpers::ProposalsFlagGuard` mutex+atomic guard for the
  process-wide flag, used across `policy_local::tests` and
  `l7::rest::tests`.
- New: `agent_next_steps_returns_empty_when_flag_off`,
  `agent_next_steps_returns_full_array_when_flag_on`,
  `route_request_returns_feature_disabled_when_flag_off`.
- Updated existing tests that exercise the deny body or the route
  dispatcher to set the flag on first.
- Full sandbox lib test suite: 653 pass, clippy clean.

Demo and e2e:
- `examples/agent-driven-policy-management/demo.sh` and
  `e2e/policy-advisor/test.sh` now snapshot the prior global value of
  the setting, set it to true before sandbox creation (so the
  supervisor's initial poll picks it up), and restore on exit (delete
  if previously unset, otherwise write the prior value back).

Docs:
- RFC 0001 MVP-implementation note documents the flag, default, and
  intended soft-launch posture.

* test(policy-advisor): require proposal opt-in for e2e

* refactor(sandbox): group policy poll loop state

* test(e2e): isolate Kubernetes user namespace test

---------

Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-05-08 17:14:27 -07:00
John T. Myers 1d3b741ee3 feat(providers): support sandbox provider attach lifecycle (#1242) 
* feat(providers): support sandbox provider attach lifecycle

Closes #1171

Adds sandbox provider list, attach, and detach API/CLI support while keeping provider policy and credential resolution derived from current sandbox attachments.

* fix(providers): refresh sandbox provider credentials

Adds provider environment revisions and generation-scoped sandbox credential snapshots so future SSH and exec launches pick up provider attach, detach, and credential updates without mutating already-running processes.

Also blocks provider deletion while attached to prevent stale sandbox provider references.

* fix(providers): serialize sandbox object mutations

* test(providers): cover sandbox provider attach lifecycle

* test(providers): accept versioned credential placeholders
 2026-05-08 11:14:44 -07:00
Drew Newberry 62619eefc7 fix(docker): use supervisor image entrypoint path (#1259) 
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
 2026-05-07 17:39:44 -07:00
Drew Newberry 70a0f6c547 refactor(cli): remove gateway lifecycle management (#1221) 2026-05-07 09:54:13 -07:00
Taylor Mutch b1cba79620 fix(sandbox): add copy-self subcommand for scratch-image init container (#1208) 
* fix(sandbox): add copy-self subcommand for scratch-image init container

The supervisor image is built FROM scratch and contains only the
openshell-sandbox binary. The Kubernetes side-load init container
previously invoked `sh -c "cp ..."`, which failed with
`executable file 'sh' not found in $PATH` because the scratch image
has no shell.

Add a `copy-self <DEST>` subcommand to the supervisor binary that
resolves its own executable, copies it to the destination path, and
sets mode 0755. Update the Kubernetes driver init container to invoke
this subcommand directly. Mirrors argoexec's emissary self-copy pattern.

* fix(docker): use ubuntu base for supervisor image

`FROM scratch` cannot exec the dynamically linked supervisor binary
(no `/lib/ld-linux-*.so.*`), so the Kubernetes side-load init container
hit `exec /openshell-sandbox: no such file or directory` even after
the `copy-self` subcommand replaced the broken `sh -c` command.

Switch the supervisor stage to `nvcr.io/nvidia/base/ubuntu:noble-20251013`
— the same base the gateway image already uses — so glibc and the
dynamic loader are present. Docker/Podman drivers continue to work
unchanged (they extract or volume-mount the binary into agent containers
that already have glibc); the Kubernetes init container now succeeds
end-to-end.

Verified on a k3d cluster: init container `openshell-supervisor-install`
exits 0 on first try, binary lands at `/opt/openshell/bin/openshell-sandbox`
(mode 0755), and the agent container reaches Ready.

* docs(docker): refresh supervisor target description

The file header still described the supervisor image as
`FROM scratch, binary only`. Update it to match the current
Ubuntu-base supervisor stage.
 2026-05-06 13:02:26 -07:00
John T. Myers 6b21804258 feat(policy): add GraphQL L7 inspection (#1083) 
Support GraphQL L7 policies
 2026-05-04 11:50:09 -07:00
Drew Newberry 08001ca616 fix(docker): harden supervisor startup and gateway routing (#1128) 2026-05-03 21:38:20 -07:00
jtoelke2 32857eb650 fix(helm): grant node read access for GPU capacity checks (#1106) 
* fix(helm): grant node read access for GPU capacity checks

Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>

* ci: add WSL GPU failure diagnostics

Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>

* fix(gpu): bump device plugin for WSL CDI

* fix(sandbox): allow WSL GPU paths in Landlock

Allow /dev/dxg and /usr/lib/wsl as GPU baseline paths so WSL CDI GPU sandboxes can initialize NVML. Native Linux skips these entries when the paths do not exist.

* ci: remove temporary WSL GPU diagnostics

---------

Signed-off-by: Jonas Toelke <jtoelke@nvidia.com>
 2026-05-01 17:49:49 -05:00
Mesut Oezdil 76093e78a3 refactor(sandbox): remove dead relay_response_to_client wrapper (#1125) 
relay_response_to_client had no callers; all call sites use relay_response
directly. The allow(dead_code) comment on parse_target_query was also
misleading since proxy.rs calls it from production code, so that annotation
is removed along with the incorrect comment.
 2026-05-01 15:15:41 -07:00
John T. Myers 5d0a44cbac fix(sandbox): invalidate stale l7 tunnels after reload (#1118) 
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-05-01 13:01:30 -07:00
ddurst-nvidia 4c2564892b fix(sandbox): accept ENOENT in drop_privileges identity lookup tests (#1123) 
User::from_name / Group::from_name may return Ok(None), producing our explicit "Sandbox … not found" errors, or Err when NSS fails first with ENOENT ("ENOENT: No such file or directory") before a definitive unknown-name result.

Accept either wording so nonexistent user/group tests stay meaningful across environments.

Signed-off-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
 2026-05-01 13:01:10 -07:00
Mrunal Patel 084505425b feat(auth): add OIDC/Keycloak authentication with RBAC and scope-based permissions (#935) 
* feat(auth): add OIDC/Keycloak authentication with RBAC

Add OAuth2/OIDC authentication to the gateway server with role-based
access control, CLI login flows, and full deployment plumbing.

Server: JWT validation against configurable OIDC issuer (oidc.rs),
JWKS key caching with TTL and rotation handling, method classification
(unauthenticated/sandbox-secret/dual-auth/bearer), identity extraction
with provider-agnostic Identity type, and RBAC enforcement via
AuthzPolicy with configurable admin/user roles and auth-only mode.

CLI: browser-based Authorization Code + PKCE flow, Client Credentials
flow for CI/automation, token storage with refresh, gateway add/login/
logout commands, OIDC bearer token injection over mTLS transport,
discovery endpoint for auto-configuration.

Security: sandbox-secret scope restriction on UpdateConfig (policy
sync only), anti-spoofing header stripping, dual-auth fallthrough
from sandbox-secret to Bearer token.

Deployment: OIDC config wired through DeployOptions, Docker env vars,
Helm values/templates, HelmChart manifest, cluster-entrypoint.sh, and
bootstrap scripts. Keycloak dev server script with pre-configured
realm (test users, roles, PKCE client, CI client).

Tested with Keycloak. The roles claim path and role names are
configurable to support other OIDC providers.

* feat(auth): add OAuth2 scope-based fine-grained permissions

Add opt-in scope enforcement on top of existing OIDC role-based access
control. When --oidc-scopes-claim is set, the server extracts scopes
from the JWT and checks them per-method against an exhaustive scope map.

Scopes: sandbox:read, sandbox:write, provider:read, provider:write,
config:read, config:write, inference:read, inference:write, and
openshell:all (wildcard). Methods not in the scope map require
openshell:all. Scopes layer on top of roles and cannot escalate
privilege. Auth-only mode (empty role names) still enforces scopes
when enabled.

Server: scopes_claim in OidcConfig, scope extraction from JWT
(space-delimited and JSON array formats), standard OIDC scope
filtering, scope check in AuthzPolicy after role check.

CLI: --oidc-scopes on gateway add/start stored in metadata and
consumed by gateway login, --oidc-scopes-claim on gateway start
forwarded to server, scopes parameter in browser and client
credentials OAuth2 flows with openid deduplication.

Deployment: oidc_scopes_claim wired through DeployOptions, docker.rs,
Helm, bootstrap scripts, and cluster entrypoint.

Keycloak: realm config updated with built-in OIDC scopes and 9
OpenShell client scopes as optional on openshell-cli and openshell:all
as default on openshell-ci.

* fix(auth): address branch review findings

Add GetInferenceBundle to sandbox-secret methods so sandbox inference
route refresh works under OIDC. Make GetSandboxConfig dual-auth so CLI
users can read sandbox settings with Bearer tokens.

Preserve OIDC gateway metadata on restart — a bare gateway start
without --oidc-* flags no longer erases the stored OIDC registration.

Document CI client ID requirement (openshell-ci vs openshell-cli) in
the testing guide. Add security note about auth-only mode blast radius
for GitHub Actions.

* fix(auth): complete review findings for OIDC auth boundary

Move OpenShell/GetSandboxConfig from sandbox-secret-only to dual-auth
so CLI users can read sandbox settings with Bearer tokens while sandbox
supervisors continue using the shared secret.

Add sandbox secret interceptor to the inference bundle fetch path so
GetInferenceBundle works under OIDC-enabled gateways. Extract shared
interceptor constructor to avoid duplication.

Add GetSandboxConfig to the config:read scope map so scope enforcement
applies consistently when scopes are enabled.

Refactor OIDC metadata preservation into apply_oidc_gateway_metadata()
with explicit resume semantics — only preserve existing OIDC metadata
on real resume paths, not on fresh deployments.

Update architecture docs and testing guide to reflect the corrected
method classifications and add new test coverage for interceptor
injection, scope requirements, metadata preservation, and dual-auth
classification.

* refactor(auth): use oauth2 crate for CLI OIDC flows

Replace hand-written PKCE generation, authorization URL construction,
token exchange, client credentials, and token refresh with the oauth2
crate's typed API.

Eliminates sha2, hex, and getrandom dependencies from the CLI. The
custom urlencoded() helper and manual form POST logic are replaced by
BasicClient methods with proper type-state safety.

Discovery and the callback server remain custom since the oauth2 crate
does not provide OIDC discovery or a localhost redirect listener.

* refactor(auth): move server auth modules into auth/ directory

Group oidc.rs, authz.rs, identity.rs, and the auth HTTP endpoints
under src/auth/ module directory. No behavioral changes.

  auth/mod.rs      — module root, re-exports HTTP router
  auth/oidc.rs     — JWT validation, JWKS caching, method classification
  auth/authz.rs    — role and scope authorization policy
  auth/identity.rs — provider-agnostic Identity type
  auth/http.rs     — /auth/connect and /auth/oidc-config endpoints

* fix(auth): use RequestBody auth type for client credentials flow

The oauth2 crate defaults to BasicAuth (HTTP Basic header) but Keycloak
and most OIDC providers expect client_secret_post (credentials in the
request body). Set AuthType::RequestBody explicitly to match the
pre-refactor behavior.

Also re-export Identity, IdentityProvider, and JwksCache from the auth
module so ServerState's public API remains nameable by external consumers.

* fix(auth): forward OPENSHELL_OIDC_SCOPES through cluster bootstrap

Pass --oidc-scopes to gateway start so the metadata includes requested
scopes after cluster bootstrap. Without this, users had to manually
edit metadata.json to set scopes for gateway login.

Usage: OPENSHELL_OIDC_SCOPES="openshell:all" mise run cluster

* test(auth): add OIDC e2e tests for RBAC, scopes, and client credentials

Add 10 end-to-end tests covering OIDC authentication against a live
K3s cluster with Keycloak:

RBAC (5 tests): admin can create providers, user cannot, user can list
sandboxes, unauthenticated requests rejected, health probe works
without auth.

Scopes (4 tests): sandbox-scoped token can list sandboxes but not
providers, openshell:all grants full access, no-scopes token denied.

Client credentials (1 test): CI token via client_credentials grant.

Tests are opt-in via OPENSHELL_E2E_OIDC=1 and OPENSHELL_E2E_OIDC_SCOPES=1
env vars. They derive the Keycloak URL from gateway metadata to match
the server's configured issuer.

Run with:

  OPENSHELL_E2E_OIDC=1 OPENSHELL_E2E_OIDC_SCOPES=1 \
  PYTHONPATH=python uv run pytest e2e/python/oidc/ -v

* fix(docs): fix markdown lint errors in OIDC architecture docs

Add blank lines before lists and fenced code blocks to satisfy
markdownlint MD031 and MD032 rules.
 2026-04-30 10:37:23 -07:00
Piotr Mlocek ee2de81bc9 fix(sandbox): preserve encoded slash policy from proto (#1073) 2026-04-29 16:36:49 -07:00
John T. Myers 0914f3f4f7 fix(sandbox): log L7 parse denials (#1072) 
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-04-29 16:15:37 -07:00
Drew Newberry 24724742a8 ci(rust): enforce -D warnings on clippy (#1008) 2026-04-29 12:12:53 -07:00
vince-brisebois 2adddaa6b2 feat: Adding qemu vm driver support with GPU pass-through (#992) 
* Adding qemu vm driver support with GPU pass-through

Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>

* Add GPU rootfs variant, harden VFIO binding, and fix networking and supervisor reliability issues discovered during GPU VM bring-up.

Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>

---------

Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
 2026-04-29 09:18:27 -07:00
John T. MyersandJohn Myers 2646b8c6c8 fix(sandbox): deny ambiguous socket ownership (#958) 
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-27 15:12:45 -07:00
Adam Miller d44d8a1e27 feat: Openshell driver podman (#904) 
* feat(podman): add Podman compute driver for rootless sandbox management

Adds openshell-driver-podman, a new compute driver that manages OpenShell
sandboxes as rootless Podman containers via the Podman REST API over a
Unix socket. Enables local workstation sandboxes without Kubernetes.

Driver features:
- Bridge networking with ephemeral host-port mapping for rootless SSH reachability
- Named volumes for workspace storage, Podman native health checks, GPU via CDI
- Supervisor binary sideloaded via image volume mount (BYOC-compatible)
- SSH handshake secret injected via Podman secrets API (not plaintext env)
- Typed ContainerSpec structs, input validation, and path-traversal guards
- Cgroups v2 required; fails fast on v1 hosts
- Bounded event stream buffer; watch stream reconnection handled by server watch_loop
- Graceful shutdown and standalone driver binary with gRPC bridge

Rootless-specific fixes:
- Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps
- Add /run/netns tmpfs mount for ip netns in rootless containers
- Use secret_env map (not secrets array) for env-var injection in libpod API
- Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP

Server/sandbox hardening:
- Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback
- Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting
- Prevent OPENSHELL_* env var override by user-supplied spec environment maps
- Disable SQLite pool idle_timeout/max_lifetime for in-memory databases
- Emit deleted_event on 404-during-inspect instead of regressing sandbox phase
- Key delete cleanup by stable sandbox_id to survive container label drift

CLI fixes:
- Restore --name as a named flag on sandbox create (not positional)
- Fix exec command arg parsing to not consume sandboxed-command flags
- Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL

Build tooling:
- Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes
  unified ce_* helpers; all build/cluster/VM scripts updated to use it
- Add docker:build:supervisor mise task for standalone supervisor image
- Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages
- Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): derive grpc endpoint from server bind port

When a user starts the gateway on a non-default port (e.g. --port 8081),
sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the
default port 8080. The driver's auto-detection fallback read
OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset,
and fell back to DEFAULT_SERVER_PORT.

Add gateway_port to PodmanComputeConfig and thread config.bind_address.port()
from the server into the driver so the fallback uses the actual listening
port. Remove the OPENSHELL_BIND_ADDRESS env var read and the
extract_port_from_bind_address helper which are no longer needed.

Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver
binary for parity when the driver is run outside the embedded server path.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(driver-podman): address PR feedback on env test safety and cluster DNS docs

Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars
and a static ENV_LOCK mutex, fixing a data race in parallel test execution.
The prior safety comment incorrectly claimed Cargo runs tests single-threaded.

Update debug-openshell-cluster skill to accurately document the DNS proxy
strategy (setup_dns_proxy + public DNS fallback) and clarify the separation
between cluster DNS and sandbox agent DNS enforcement.

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix(e2e): resolve CI failures in auth timeout, test harness, and formatting

- Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead
  of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives
- Add timeout to SandboxGuard::create() and create_with_upload() to
  prevent indefinite hangs (matches create_keep() which already had one)
- Add missing '--' separator in no_proxy test before command args
- Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs
- Run cargo fmt on openshell-driver-podman
- Refine cluster DNS docs for Podman in debug-openshell-cluster skill

Signed-off-by: Adam Miller <admiller@redhat.com>

* refactor(server): remove allows_loopback_endpoints from ComputeRuntime

SSRF protection is now handled at the network and proxy layers
(openshell-core net.rs, openshell-sandbox proxy.rs) rather than
requiring per-driver flags on ComputeRuntime. Update architecture
docs to reflect supervisor relay SSH transport and add rootless
networking deep-dive.

Signed-off-by: Adam Miller <admiller@redhat.com>

---------

Signed-off-by: Adam Miller <admiller@redhat.com>
 2026-04-24 10:30:14 -07:00
John T. MyersandJohn Myers 7f8e2109e3 fix(sandbox): route console logs to stderr (#949) 
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-24 07:49:49 -07:00
Tinson Lai 2f8e8ac311 fix(sandbox): inject GIT_SSL_CAINFO so git clone trusts the sandbox CA (#918) 
Ubuntu Noble's git links against libcurl-gnutls, which does not read
SSL_CERT_FILE. Without GIT_SSL_CAINFO, `git clone` over HTTPS fails
inside every community sandbox image with "server certificate
verification failed".

Add GIT_SSL_CAINFO to tls_env_vars pointing at the same combined CA
bundle already used by CURL_CA_BUNDLE / REQUESTS_CA_BUNDLE.

Fixes #790
Fixes NVIDIA/NemoClaw#2270
Fixes NVIDIA/NemoClaw#1828

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
 2026-04-22 13:35:21 -07:00
John T. Myers f954e5927a fix(sandbox): resolve sandbox host aliases in SSRF checks (#912) 
Closes #879

Resolve policy hostnames against the sandbox's /etc/hosts before DNS so Kubernetes hostAliases are visible to the proxy while keeping the existing allowed_ips SSRF enforcement semantics intact.

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-04-22 12:37:32 -07:00
John T. Myers e28ca07863 fix(sandbox): preserve explicit read-only baseline paths (#910) 
Closes #905

Skip proxy baseline read-write enrichment when a path is already explicitly configured as read-only, add regression coverage for the proto and local policy code paths, and document the precedence rule in the sandbox architecture docs.

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-04-22 12:18:24 -07:00
Piotr Mlocek a6d45528c1 feat(server,sandbox): supervisor-initiated SSH connect and exec over gRPC-multiplexed relay (#867) 2026-04-21 08:38:18 -07:00
John T. Myers c960d480fd fix(sandbox): canonicalize HTTP request-targets before L7 policy evaluation (#878) 
* fix(sandbox): canonicalize HTTP request-targets before L7 policy evaluation

The L7 REST proxy evaluated OPA path rules against the raw request-target
and forwarded the raw header block byte-for-byte to the upstream. The
upstream normalized `..`, `%2e%2e`, and `//` before dispatching, so a
compromised agent inside the sandbox could bypass any path-based allow
rule by prefixing the blocked path with an allowed one (e.g.
`GET /public/../secret` matches `/public/**` at the proxy but the
upstream serves `/secret`). The inference-routing detection had the same
normalization gap via `normalize_inference_path`, which only stripped
scheme+authority and preserved dot-segments verbatim.

- Add `crates/openshell-sandbox/src/l7/path.rs` with
  `canonicalize_request_target`. Percent-decodes (with uppercase hex
  re-emission), resolves `.` / `..` segments per RFC 3986 5.2.4,
  collapses doubled slashes, strips trailing `;params`, rejects
  fragments / raw non-ASCII / control bytes / encoded slashes (unless
  explicitly enabled per-endpoint), and returns a `rewritten` flag so
  callers know when to rewrite the outbound request line.
- Wire the canonicalizer into `rest.rs::parse_http_request`. The
  canonical path is the `target` on `L7Request` that OPA sees, and
  when the canonical form differs from the raw input the request line
  is rebuilt in `raw_header` so the upstream dispatches on the exact
  bytes the policy evaluated.
- Canonicalize the forward (plain HTTP proxy) path at
  `proxy.rs`'s second L7 evaluation site. A non-canonical request-target
  is rejected with 400 Bad Request and an OCSF `NetworkActivity` Fail
  event rather than silently passed through.
- Replace the old `normalize_inference_path` body with a call to the
  canonicalizer. On canonicalization error the raw path is returned
  (so inference detection simply misses and the normal forward path
  handles and rejects).
- Document the invariant in `sandbox-policy.rego`: `input.request.path`
  is pre-canonicalized so rules must not attempt defensive matching
  against `..` or `%2e%2e`.
- Architecture doc (`architecture/sandbox.md`) lists the new module.
- 24 new unit tests cover dot segments, percent-encoded dot segments,
  double-slash collapse, encoded-slash reject/opt-in, null/control byte
  rejection, legitimate percent-encoded bytes round-tripping, mixed-case
  percent normalization, fragment rejection, absolute-form stripping,
  empty / length-bounded / non-ASCII handling, and regression payloads
  for the specific bypasses above.

Tracks OS-99.

* fix(sandbox): close forward-proxy parser-differential and wire allow_encoded_slash

Addresses two review findings on the L7 path canonicalization change.

1. `handle_forward_proxy` previously evaluated OPA on the canonical path
   but wrote the raw path to the upstream via `rewrite_forward_request`,
   leaving the parser-differential open for plain-HTTP forward-proxy
   traffic even though it was closed for the L7 tunnel path. `path` is
   now reassigned to the canonical form inside the L7 block before the
   outbound write, so the bytes dispatched to the upstream match what
   OPA evaluated. Covered by a new regression test against
   `rewrite_forward_request`.

2. `allow_encoded_slash` is now wired through the YAML policy schema,
   the proto `NetworkEndpoint` message, the Rego endpoint config, and
   the `L7EndpointConfig` → `RestProvider` canonicalization options.
   Operators can opt in per-endpoint for upstreams like GitLab that
   embed `%2F` in namespaced resource paths; the default remains
   strict. The `RestProvider` gains a constructor
   `RestProvider::with_options` so `relay_rest` constructs a provider
   with per-endpoint options while `relay_passthrough_with_credentials`
   retains strict defaults.

Integration tests:
- `parse_http_request_canonicalizes_target_and_rewrites_raw_header`
  drives a non-canonical request through the parser and asserts both
  `L7Request.target` (what OPA evaluates) and `raw_header` (what the
  upstream receives) are canonical.
- `parse_http_request_canonicalization_preserves_query_string`,
  `parse_http_request_leaves_canonical_input_byte_for_byte`,
  `parse_http_request_rejects_traversal_above_root`,
  `parse_http_request_preserves_http_10_version_on_rewrite`.
- `parse_http_request_accepts_encoded_slash_when_endpoint_opts_in` /
  `_rejects_encoded_slash_by_default` verify the
  `allow_encoded_slash` gate.
- `test_rewrite_forward_request_uses_canonical_path_on_the_wire`
  asserts the fix to the forward-proxy flow.
- `parse_l7_config_allow_encoded_slash_{defaults_false,opt_in}` verify
  the YAML/Rego wiring.
 2026-04-21 07:08:07 -07:00
John T. Myers 8a813aba49 fix(sandbox): apply supervisor seccomp prelude (#891) 
* fix(sandbox): apply supervisor seccomp prelude

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* fix(sandbox): delay supervisor prelude until bootstrap

---------

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-04-20 13:09:23 -07:00
John T. Myers 40e9bf6feb feat(policy): add incremental sandbox policy updates (#860) 
* feat(policy): add incremental sandbox policy updates

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* docs(policies): expand incremental update guidance

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* feat(policy): audit incremental updates in gateway logs

* docs(policy): quote glob specs in shell examples

---------

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-04-20 08:00:02 -07:00
Piotr Mlocek e39bb38040 test(sandbox): fix flaky arm64 procfs binary_path tests (#881) 2026-04-17 17:47:37 -07:00
mjamiv ae7e901000 fix(sandbox): strip " (deleted)" suffix from unlinked /proc/<pid>/exe paths (#844) 
* fix(sandbox): strip " (deleted)" suffix from unlinked /proc/<pid>/exe paths

When a running binary is unlinked from its filesystem path — the common
case is a `docker cp` hot-swap of `/opt/openshell/bin/openshell-sandbox`
during the `cluster-deploy-fast` dev upgrade workflow — the Linux kernel
appends the literal string ` (deleted)` to the `/proc/<pid>/exe` readlink
target. The tainted `PathBuf` then flows into `collect_ancestor_binaries`
and on into `BinaryIdentityCache::verify_or_cache`, which tries to
`std::fs::metadata` the path. `stat()` fails with `ENOENT` because the
literal suffix isn't a real filesystem path, and the CONNECT proxy denies
every outbound request with:

    ancestor integrity check failed for \
      /opt/openshell/bin/openshell-sandbox (deleted): \
      Failed to stat ...: No such file or directory (os error 2)

Reproduced in production 2026-04-15: a cluster-deploy-fast-style hot-swap
of the supervisor binary caused every pod whose PID 1 held the now-deleted
inode to deny ALL outbound CONNECTs (slack.com, registry.npmjs.org,
169.254.169.254, etc.), breaking Slack REST delivery, npm installs, and
IMDS probes simultaneously. Existing pre-hot-swap TCP tunnels (e.g. Slack
Socket Mode WSS) kept working because they never re-evaluate the proxy.

Strip the suffix in `binary_path()` so downstream callers see the clean,
grep-friendly path. This aligns the cache key and log messages with the
original on-disk location.

Note: stripping the suffix does NOT by itself make the identity cache
tolerant of a legitimate binary replacement — `verify_or_cache` will now
`stat` and hash whatever currently lives at the stripped path, which is
the NEW binary, and surface a clearer `Binary integrity violation` error.
Fully unblocking the cluster-deploy-fast hot-swap workflow needs a
follow-up that either (a) reads running-binary content from
`/proc/<pid>/exe` directly via `File::open` (procfs resolves this to the
live in-memory executable even when the original inode has been unlinked),
or (b) keys the identity cache by exec dev+inode instead of path. Happy to
send that as a separate PR once the approach is decided — filing this
narrow fix first because it stands on its own: it fixes a concrete
misleading error and unblocks the obvious next step.

Added `binary_path_strips_deleted_suffix` test that copies `/bin/sleep`
to a temp path, spawns a child from it, unlinks the temp binary, verifies
the raw readlink contains the ` (deleted)` suffix, then asserts the public
API returns the stripped path.

Signed-off-by: mjamiv <michael.commack@gmail.com>

* fix(sandbox): narrow the " (deleted)" suffix strip and exercise it end-to-end

Address review feedback from @johntmyers on the initial version of this PR:

procfs::binary_path
- Only strip the kernel's " (deleted)" suffix when stat() on the raw
  readlink target reports NotFound. A live executable whose basename
  literally ends with " (deleted)" is now returned unchanged instead of
  being silently truncated, which matters because identity.rs hashes
  whatever this function returns as a trust anchor.
- Operate on raw bytes via OsStrExt, so filenames that are not valid
  UTF-8 still get exactly one suffix stripped. The previous
  strip_suffix-on-&str path skipped non-UTF-8 entirely and fell through
  to returning the tainted path.
- Expand the doc comment to describe both guardrails.

procfs tests
- binary_path_preserves_live_deleted_basename: copy /bin/sleep to a
  live file literally named "sleepy (deleted)", spawn it, and assert
  that the returned path still ends with " (deleted)".
- binary_path_strips_suffix_for_non_utf8_filename: exec a binary whose
  basename contains a 0xFF byte, unlink it, and assert that
  binary_path returns the stripped non-UTF-8 path. Writes the bytes
  with OpenOptions + sync_all + explicit drop so the write fd is fully
  released before exec() to avoid ETXTBSY under concurrent tests.

proxy: extract resolve_process_identity helper
- Pull the peer-resolution + TOFU verify + ancestor walk + cmdline
  collection block out of evaluate_opa_tcp into resolve_process_identity.
- Introduce IdentityError which carries the deny reason along with
  whatever partial identity data was resolved before the failure so
  evaluate_opa_tcp can thread that into ConnectDecision unchanged.
- evaluate_opa_tcp now calls the helper and proceeds straight to the
  OPA evaluate step; the surface visible to OPA and OCSF is unchanged.

proxy: end-to-end regression test for the hot-swap contract
- resolve_process_identity_surfaces_binary_integrity_violation_on_hot_swap
  stands up a real TcpListener, copies /bin/bash to a temp path,
  primes BinaryIdentityCache with that binary, spawns bash with a
  /dev/tcp one-liner that opens a real connection to the listener,
  and captures the peer's ephemeral port from accept().
- Simulates `docker cp` correctly: unlink the running binary (which
  persists via the child's exec mapping) and create a fresh file
  with different bytes at the same path. Writing in place is
  rejected by the kernel with ETXTBSY, so the old single-inode
  approach did not actually model the production failure mode.
- Asserts the error returned by resolve_process_identity contains
  "Binary integrity violation" (from BinaryIdentityCache) and does
  NOT contain "Failed to stat" or "(deleted)" — the pre-PR-#844
  failure mode. The binary field on the error is populated and is
  free of the tainted suffix.
- Skips cleanly if /bin/bash is not installed. Child process is
  always reaped before the assertion block so a failure does not
  leak a sleeping process.

---------

Signed-off-by: mjamiv <michael.commack@gmail.com>
 2026-04-17 14:25:59 -07:00
John T. MyersandJohn Myers 4e8dbcfe5c fix(sandbox): harden seccomp, inference routing, and process limits (#869) 
* fix(sandbox): block AF_NETLINK in seccomp unconditionally

Move AF_NETLINK to the unconditional socket-domain block list alongside
AF_PACKET, AF_BLUETOOTH, and AF_VSOCK. Previously it was only blocked in
NetworkMode::Block, leaving it accessible in Proxy mode where network
namespace isolation already scopes netlink to the sandbox's own veth —
making this a defense-in-depth hardening rather than a live exposure.

Closes OS-94

* fix(sandbox): scope inference.local interception to port 443

The pre-OPA interception for inference.local matched on hostname alone,
allowing any port to bypass OPA policy evaluation — including under
deny-all (network_policies: {}). Add a port check so only port 443
takes the interception path; all other ports on inference.local now
fall through to OPA and are subject to normal policy evaluation.

Closes OS-95

* fix(sandbox): enforce RLIMIT_NPROC to prevent fork bomb DoS

Set a hard limit of 512 processes per UID in harden_child_process(),
applied before privilege drop so the sandbox user cannot raise it.
Prevents unrestricted fork() from exhausting the process table — most
relevant for local dev mode where K8s pod cgroup pids.max is absent.

Closes OS-96

* chore(sandbox): fix rustfmt formatting for seccomp blocked_domains

---------

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-17 08:26:11 -07:00
Matthias Oßwald 3b21df190d feat(sandbox): load system CA certificates for upstream TLS connections (#862) 
The proxy's upstream TLS client only trusted Mozilla root CAs
(webpki-roots), which prevented TLS termination from working with
internal/corporate hosts using private CA certificates.

Load system CA certificates from the container's trust store
(e.g. /etc/ssl/certs/ca-certificates.crt) in addition to
webpki-roots. This allows custom sandbox images to include
corporate CAs via update-ca-certificates.

Signed-off-by: Matthias Osswald <mat.osswald@sap.com>
 2026-04-16 13:40:25 -07:00
John T. Myers 25d2530b3c fix(inference): allowlist routed request headers (#826) 
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-04-15 15:19:18 -07:00
mjamiv 1a57519f2d fix(sandbox): escape control characters in format_sse_error (#842) 
format_sse_error only escaped `\` and `"`, leaving two problems:

1. Control characters (`\n`, `\r`, `\t`, and all `\u0000-\u001F`) in
   `reason` produce output that fails `serde_json::from_str` — defeating
   #834's goal of giving clients a parseable SSE truncation signal.

2. An unescaped `\n\n` inside `reason` splits the single error event
   into two SSE frames, letting a misbehaving upstream inject a forged
   frame (e.g. a fake tool_calls delta) into the client's stream. Latent
   today since all in-tree callers pass static strings, but a footgun
   for any future caller passing upstream error text, and the function's
   docstring already invites dynamic reasons.

Replace the manual escape with `serde_json::to_writer` (already a
workspace dep of `openshell-sandbox`). Add unit tests for control
character escaping and SSE event-boundary injection.

Closes #840

Signed-off-by: mjamiv <michael.commack@gmail.com>
 2026-04-15 07:28:08 -07:00
John T. Myers 28db08eeb8 fix(sandbox): disable child core dumps (#821) 
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
 2026-04-14 22:28:18 -07:00
John T. Myers 28e1ff7b40 feat(policy): add deny rules to network policy schema (#822) 
* feat(policy): add deny rules to network policy schema

Closes #565

Add L7 deny rules that block specific requests even when allowed by
access presets or explicit allow rules. Deny rules mirror the full
capability set of allow rules (method, path, query params, SQL command)
and take precedence -- if a request matches any deny rule, it is blocked
regardless of allow rules.

This enables the "allow everything except these specific operations"
pattern without enumerating every allowed endpoint. For example, granting
read-write access to GitHub while blocking PR approvals, branch
protection changes, and ruleset modifications.

* fix(policy): deny query matching fails closed, mirror allow-side validation

Addresses PR review findings P1 and P3:

P1: Deny-side query matching now uses fail-closed semantics. If ANY
value for a query key matches the deny matcher, the deny fires. The
previous implementation reused allow-side "all values must match"
logic which allowed ?force=true&force=false to bypass a deny on
force=true.

P3: Deny-side query validation now mirrors the full allow-side checks:
empty any lists, non-string matcher values, glob+any mutual exclusion,
glob type checks, and glob syntax warnings are all validated.
 2026-04-14 21:52:14 -07:00
John T. MyersandJohn Myers e0db01e9ff fix(sandbox): preserve ownership for existing read_write paths (#827) 
Closes #783

Only chown read_write paths that the supervisor creates at startup, and leave pre-existing image paths with their original ownership. Add sandbox tests for creation, symlink rejection, and existing-path ownership preservation, and update architecture docs to match the new behavior.

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-14 21:51:47 -07:00
John T. Myers 355d845d3e fix(inference): prevent silent truncation of large streaming responses (#834) 
* fix(inference): prevent silent truncation of large streaming responses

The L7 inference proxy silently dropped tool_calls from large streaming
responses due to an aggressive 30s per-chunk idle timeout and a reqwest
total-request timeout that capped the entire body stream. Reasoning models
that pause during "thinking" phases triggered these timeouts, producing
valid-looking but truncated HTTP responses with no client-visible error.

- Extract prepare_backend_request() helper and create a streaming variant
  that omits the total request timeout; body stream liveness is now
  enforced solely by the per-chunk idle timeout
- Add 30s connect_timeout to the reqwest Client builder
- Increase CHUNK_IDLE_TIMEOUT from 30s to 120s for reasoning models
- Inject SSE error events (proxy_stream_error) before the HTTP chunked
  terminator on all truncation paths so clients can detect data loss
- Wrap the streaming relay in BufWriter to reduce per-chunk TLS flush
  overhead
- Bump OCSF severity for streaming truncation from Low to Medium

Closes #829

* fix(inference): drop BufWriter to preserve SSE incremental delivery

The BufWriter introduced in the previous commit buffered SSE frames until
the 16KB capacity filled or the stream ended, defeating incremental token
delivery and potentially reintroducing client-visible timeouts on healthy
streams. Revert to per-chunk write_all+flush but keep the single
format_chunk() call that coalesces framing into one write.

Also fix the streaming integration test: add a 3s mock delay that exceeds
the 1s route timeout so the test actually validates that the streaming
path omits the total request timeout. Previously the mock responded
immediately, passing regardless of timeout behavior.
 2026-04-14 13:34:39 -07:00
John T. Myers 1cabd2563c fix(sandbox): harden seccomp denylist, SSRF protection, and inference policy enforcement (#819) 
* fix(sandbox): harden seccomp denylist, SSRF protection, and inference policy enforcement

- Remove seccomp skip in NetworkMode::Allow so baseline syscall
  restrictions apply regardless of network mode
- Block cross-process manipulation syscalls (process_vm_writev,
  pidfd_open, pidfd_getfd, pidfd_send_signal) symmetric with existing
  ptrace and process_vm_readv blocks
- Block clone/clone3 with CLONE_NEWUSER flag, new mount API syscalls
  (fsopen, fsconfig, fsmount, fspick, move_mount, open_tree), and
  namespace manipulation (setns, umount2, pivot_root)
- Block userfaultfd and perf_event_open consistent with Docker default
  seccomp profile
- Deny and close keep-alive inference connections after a non-inference
  request instead of silently continuing the loop
- Add CGNAT (100.64.0.0/10), benchmarking (198.18.0.0/15), and other
  special-use IP ranges to SSRF protection in both proxy and
  mechanistic mapper

* fix(sandbox): install clone3 seccomp filter before main filter

* test(ocsf): fix shorthand firewall engine expectation
 2026-04-13 12:16:10 -07:00
John T. Myers 09af1b6bad fix(sandbox): add JSON bodies to proxy 403/502 responses and include port in HTTP log URLs (#809) 
L4 CONNECT and forward-proxy denial responses (403) and upstream
failure responses (502) were returned with empty bodies, making them
indistinguishable from each other when using curl -s. Add a
build_json_error_response() helper that produces consistent JSON
bodies with error and detail fields, matching the format already
used by the L7 deny path.

Also fix Url::to_display_string() in openshell-ocsf to include
non-default ports in the shorthand log output. Requests to e.g.
http://host:9876/path were logged as http://host/path, dropping the
port number.

Closes #807, closes #808
 2026-04-13 10:56:03 -07:00
John T. Myers 29a3b1cacd fix(sandbox): two-phase Landlock to fix privilege ordering and add enforcement tests (#810) 
* fix(sandbox): add parent-side Landlock availability probe logging

Landlock status was only logged from inside the pre_exec child process
where the tracing/OCSF pipeline is non-functional after fork. This made
Landlock failures completely invisible in sandbox logs.

Add a probe_availability() function that issues the raw
landlock_create_ruleset syscall to check kernel support, and call it
from the parent process before fork in all three spawn paths (entrypoint,
SSH PTY, SSH pipe). Uses std::sync::Once to emit exactly once per
sandbox lifetime.

WIP - addresses logging gap from #803.

* test(e2e): add Landlock filesystem enforcement tests

Verify Landlock availability logging and enforcement in e2e:
- OCSF probe event appears in sandbox logs
- Read-only paths block writes, allow reads
- Read-write paths allow both
- Paths outside policy are denied entirely
- User-owned paths outside policy are still blocked (proves
  Landlock enforces independently of Unix DAC permissions)

Requires Linux host with Landlock support (GitHub Actions runners,
Docker Desktop linuxkit). Related to #803.

* test(e2e): add xfail tests proving #803 privilege ordering bug

Two strict xfail tests that demonstrate the root cause of #803:
- PathFd::new() runs as uid 998 after drop_privileges, so root-only
  paths (mode 700) silently fail and Landlock degrades
- When ALL paths fail, best_effort silently drops Landlock entirely

These tests will pass after the two-phase Landlock fix (open PathFds
as root before drop_privileges, restrict_self after).

* fix(test): fix Landlock e2e tests based on test run results

- Remove log-reading tests: the Landlock probe logs to the
  supervisor's container stdout, not the in-sandbox file appender
  at /var/log/openshell*.log*
- Replace broken xfail test (checked for child-process log messages
  that never reach the file appender) with a stat()-based test that
  verifies /root is actually in the Landlock allowlist
- Keep enforcement tests (all passing) and the root-only policy
  xfail test (correctly proves #803 bug)

* fix(test): remove mixed-policy xfail test

Landlock doesn't restrict stat(), so the test passed unexpectedly.
In the mixed policy case where /root is silently skipped, Landlock
still applies (using other paths) and blocks /root even harder
(not in allowlist = denied). The observable security degradation
only occurs when ALL paths fail, which the existing xfail test
already covers.

* fix(sandbox): two-phase Landlock to fix privilege ordering (#803)

Split Landlock apply into prepare() and enforce():
- prepare() runs as root before drop_privileges: opens PathFds,
  creates ruleset, adds rules. Root-only paths (mode 700) now
  succeed instead of silently failing as uid 998.
- enforce() runs after drop_privileges: calls restrict_self()
  which does not require root.

This fixes the root cause of #803 where drop_privileges() ran
before sandbox::apply(), causing PathFd::new() to fail on
root-only paths. In best_effort mode this silently dropped all
Landlock restrictions.

The fix applies to all three spawn paths: entrypoint (process.rs),
SSH PTY (ssh.rs), and SSH pipe exec (ssh.rs).

Removes xfail marker from e2e test that now passes.

* fix(sandbox): address PR review feedback

- enforce() now respects best_effort: if restrict_self() fails and
  policy is best_effort, log and degrade instead of aborting startup
- log_sandbox_readiness distinguishes best_effort (degraded) from
  hard_requirement (will fail) in OCSF messages
 2026-04-13 10:55:34 -07:00
John T. MyersandJohn Myers 9e721df9dd fix(sandbox): resolve symlinked binary paths in network policy matching (#774) 
* fix(sandbox): resolve symlinked binary paths in network policy matching

Policy binary paths specified as symlinks (e.g., /usr/bin/python3) were
silently denied because the kernel reports the canonical path via
/proc/<pid>/exe (e.g., /usr/bin/python3.11). The strict string equality
in Rego never matched.

Expand policy binary paths by resolving symlinks through the container
filesystem (/proc/<pid>/root/) after the entrypoint starts. The OPA data
now contains both the original and resolved paths, so Rego's existing
strict equality check naturally matches either.

- Add resolve_binary_in_container() helper for Linux symlink resolution
- Add from_proto_with_pid() and reload_from_proto_with_pid() to OpaEngine
- Trigger one-shot OPA rebuild after entrypoint_pid is stored
- Thread entrypoint_pid through run_policy_poll_loop for hot-reloads
- Improve deny reason with symlink debugging hint
- Add 18 new tests including hot-reload and Linux symlink e2e tests

Closes #770

* fix(sandbox): skip procfs-dependent tests when /proc/<pid>/root/ is inaccessible

The Linux-specific symlink resolution tests depend on /proc/<pid>/root/
being readable, which requires CAP_SYS_PTRACE or permissive ptrace
scope. This is unavailable in CI containers, rootless containers, and
hardened hosts. Add a procfs_root_accessible() guard that skips these
tests gracefully instead of failing.

* fix(sandbox): add explicit logging when symlink resolution fails and improve deny messages

When /proc/<pid>/root/ is inaccessible (restricted ptrace, rootless
containers, hardened hosts), resolve_binary_in_container now logs a
per-binary warning with the specific error, the path it tried, and
actionable guidance (use canonical path or grant CAP_SYS_PTRACE).
Previously this was completely silent.

The Rego deny reason for binary mismatches now leads with 'SYMLINK HINT'
and includes a concrete fix command ('readlink -f' inside the sandbox)
plus what to look for in logs if automatic resolution isn't working.

* fix(sandbox): use read_link instead of canonicalize for symlink resolution

std::fs::canonicalize resolves /proc/<pid>/root itself (a kernel
pseudo-symlink to /) which strips the prefix needed for path extraction.
This caused resolution to silently fail in all environments, not just CI.

Replace with an iterative read_link loop that walks the symlink chain
within the container namespace without resolving the /proc mount point.
Add normalize_path helper for relative symlink targets containing ..
components. Update procfs_root_accessible test guard to actually probe
the full resolution path instead of just checking path existence.

* fix(sandbox): defer symlink resolution until container filesystem is ready

The one-shot resolve ran immediately after ProcessHandle::spawn, before
the child's mount namespace and /proc/<pid>/root/ were populated. This
caused symlink_metadata to fail with ENOENT on every binary, and the
poll loop never retried because it only reloads when the policy hash
changes on the server.

Replace the synchronous resolve with an async task that probes
/proc/<pid>/root/ with retries (10 attempts, 500ms apart, 5s total).
The child's mount namespace is typically ready within a few hundred ms.

Also inline error values into warning message strings so they appear in
default log output (not just as structured tracing fields that may be
elided), and add debug-level logs before each symlink_metadata call to
aid diagnosis.

---------

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-13 10:10:46 -07:00
John T. Myers 2ca553a4a0 fix(sandbox): validate always-blocked IPs at load time, enrich denial logs, and filter un-fixable proposals (#814) (#815) 
Policies with allowed_ips entries targeting loopback, link-local, or
unspecified ranges now fail at connection time instead of being silently
blocked at runtime. The shorthand log format for DENIED events includes
a [reason:...] suffix so operators can distinguish 'allowlist miss' from
'structurally un-allowable'. The mechanistic mapper skips proposals for
always-blocked destinations, preventing the infinite TUI notification
loop. The gateway validates proposed rules on approval as defense-in-depth.

- Extract shared IP helpers (is_always_blocked_ip, is_always_blocked_net,
  is_internal_ip) to openshell_core::net
- Reject always-blocked entries in parse_allowed_ips with hard error
- Skip implicit allowed_ips synthesis for always-blocked literal IP hosts
- Add status_detail to HttpActivityBuilder for denial reason propagation
- Enrich NET and HTTP shorthand with [reason:...] for DENIED events
- Add engine: tag to HTTP shorthand (consistency with NET shorthand)
- Filter always-blocked proposals in mechanistic mapper generate_proposals
- Add validate_rule_not_always_blocked server-side defense-in-depth
- Update architecture docs, published docs, and E2E test assertions
 2026-04-13 09:47:50 -07:00
Evan Lezar 03486f810d fix(sandbox): split drop_privileges test to unblock non-root CI (#623) 
The drop_privileges_succeeds_for_current_user test failed on non-root
systems because initgroups(3) requires CAP_SETGID even when switching
to the current user. Add a new drop_privileges_succeeds_for_current_group
test that omits run_as_user, bypassing initgroups while still exercising
setgid() and the GID post-condition verification. Mark the original
full-path test with #[ignore] so it can be run explicitly as root.

Fixes #622
 2026-04-13 14:53:55 +02:00
John T. MyersandJohn Myers 13051df321 fix(policy): reject TLD wildcard patterns in network policy endpoints (#791) 
TLD wildcards like *.com were accepted by policy validation but silently
failed at the proxy layer — 0 bytes returned, no denial logged. Now
rejected at submission time with a clear error message.

Closes #787

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-09 13:11:35 -07:00
John T. Myers b7779bdefa feat(sandbox): integrate OCSF structured logging for sandbox events (#720) 
* feat(sandbox): integrate OCSF structured logging for all sandbox events

WIP: Replace ad-hoc tracing calls with OCSF event builders across all
sandbox subsystems (network, SSH, process, filesystem, config, lifecycle).

- Register ocsf_logging_enabled setting (defaults false)
- Replace stdout/file fmt layers with OcsfShorthandLayer
- Add conditional OcsfJsonlLayer for /var/log/openshell-ocsf.log
- Update LogPushLayer to extract OCSF shorthand for gRPC push
- Migrate ~106 log sites to OCSF builders (NetworkActivity, HttpActivity,
  SshActivity, ProcessActivity, DetectionFinding, ConfigStateChange,
  AppLifecycle)
- Add openshell-ocsf to all Docker build contexts

* fix(scripts): attach provider to all smoke test phases to avoid rate limits

GitHub's unauthenticated API rate limit (60/hour) causes flaky 403s for
Phases 1, 2, and 4. Fix by attaching the provider to all sandboxes and
upgrading the Phase 1 policy to L7 so credential injection works.

Phase 4 (tls:skip) cannot inject credentials by design, so relax the
assertion to accept either 200 or 403 from upstream -- both prove the
proxy forwarded the request.

* fix(ocsf): remove timestamp from shorthand format to avoid double-timestamp

The display layer (gateway logs, TUI, sandbox logs CLI) already prepends
a timestamp. Having one in the shorthand output too produces redundant
double-timestamps like:

  15:49:11 sandbox INFO  15:49:11.649 I NET:OPEN ALLOWED ...

Now the shorthand is just the severity + structured content:

  15:49:11 sandbox INFO  I NET:OPEN ALLOWED ...

* refactor(ocsf): replace single-char severity with bracketed labels

Replace cryptic single-character severity codes (I/L/M/H/C/F) with
readable bracketed labels: [LOW], [MED], [HIGH], [CRIT], [FATAL].

Informational severity (the happy-path default) is omitted entirely to
keep normal log output clean and avoid redundancy with the tracing-level
INFO that the display layer already provides.

Before: sandbox INFO  I NET:OPEN ALLOWED ...
After:  sandbox INFO  NET:OPEN ALLOWED ...

Before: sandbox INFO  M NET:OPEN DENIED ...
After:  sandbox INFO  [MED] NET:OPEN DENIED ...

* feat(sandbox): use OCSF level label for structured events in log push

Set the level field to 'OCSF' instead of 'INFO' for OCSF events in the
gRPC log push. This visually distinguishes structured OCSF events from
plain tracing output in the TUI and CLI sandbox logs:

  sandbox OCSF  NET:OPEN [INFO] ALLOWED python3(42) -> api.example.com:443
  sandbox OCSF  NET:OPEN [MED] DENIED python3(42) -> blocked.com:443
  sandbox INFO  Fetching sandbox policy via gRPC

* fix(sandbox): convert new Landlock path-skip warning to OCSF

PR #677 added a warn!() for inaccessible Landlock paths in best-effort
mode. Convert to ConfigStateChangeBuilder with degraded state so it
flows through the OCSF shorthand format consistently.

* fix(sandbox): use rolling appender for OCSF JSONL file

Match the main openshell.log rotation mechanics (daily, 3 files max)
instead of a single unbounded append-only file. Prevents disk exhaustion
when ocsf_logging_enabled is left on in long-running sandboxes.

* fix(sandbox): address reviewer warnings for OCSF integration

W1: Remove redundant 'OCSF' prefix from shorthand file layer — the
    class name (NET:OPEN, HTTP:GET) already identifies structured events
    and the LogPushLayer separately sets the level field.

W2: Log a debug message when OCSF_CTX.set() is called a second time
    instead of silently discarding via let _.

W3: Document the boundary between OCSF-migrated events and intentionally
    plain tracing calls (DEBUG/TRACE, transient, internal plumbing).

W4: Migrate remaining iptables LOG rule failure warnings in netns.rs
    (IPv4 TCP/UDP, IPv6 TCP/UDP) to ConfigStateChangeBuilder for
    consistency with the IPv4 bypass rule failure already migrated.

W5: Migrate malformed inference request warn to NetworkActivity with
    ActivityId::Refuse and SeverityId::Medium.

W6: Use Medium severity for L7 deny decisions (both CONNECT tunnel and
    FORWARD proxy paths) to match the CONNECT deny severity pattern.
    Allows and audits remain Informational.

* refactor(sandbox): rename ocsf_logging_enabled to ocsf_json_enabled

The shorthand logs are already OCSF-structured events. The setting
specifically controls the JSONL file export, so the name should reflect
that: ocsf_json_enabled.

* fix(ocsf): add timestamps to shorthand file layer output

The OcsfShorthandLayer writes directly to the log file with no outer
display layer to supply timestamps. Add a UTC timestamp prefix to every
line so the file output matches what tracing::fmt used to provide.

Before: CONFIG:VALIDATED [INFO] Validated 'sandbox' user exists in image
After:  2026-04-01T15:49:11.649Z CONFIG:VALIDATED [INFO] Validated ...

* fix(docker): touch openshell-ocsf source to invalidate cargo cache

The supervisor-workspace stage touches sandbox and core sources to force
recompilation over the rust-deps dummy stubs, but openshell-ocsf was
missing. This caused the Docker cargo cache to use stale ocsf objects
from the deps stage, preventing changes to the ocsf crate (like the
timestamp fix) from appearing in the final binary.

Also adds a shorthand layer test verifying timestamp output, and drafts
the observability docs section.

* fix(ocsf): add OCSF level prefix to file layer shorthand output

Without a level prefix, OCSF events in the log file have no visual
anchor at the position where standard tracing lines show INFO/WARN.
This makes scanning the file harder since the eye has nothing consistent
to lock onto after the timestamp.

Before: 2026-04-01T04:04:13.065Z CONFIG:DISCOVERY [INFO] ...
After:  2026-04-01T04:04:13.065Z OCSF CONFIG:DISCOVERY [INFO] ...

* fix(ocsf): clean up shorthand formatting for listen and SSH events

- Fix double space in NET:LISTEN, SSH:LISTEN, and other events where
  action is empty (e.g., 'NET:LISTEN [INFO]  10.200.0.1' -> 'NET:LISTEN [INFO] 10.200.0.1')
- Add listen address to SSH:LISTEN event (was empty)
- Downgrade SSH handshake intermediate steps (reading preface, verifying)
  from OCSF events to debug!() traces. Only the final verdict
  (accepted/denied) is an OCSF event now, reducing noise from 3 events
  to 1 per SSH connection.
- Apply same spacing fix to HTTP shorthand for consistency.

* docs(observability): update examples with OCSF prefix and formatting fixes

Align doc examples with the deployed output:
- Add OCSF level prefix to all shorthand examples in the log file
- Show mixed OCSF + standard tracing in the file format section
- Update listen events (no double space, SSH includes address)
- Show one SSH:OPEN per connection instead of three
- Update grep patterns to use 'OCSF NET:' etc.

* docs(agents): add OCSF logging guidance to AGENTS.md

Add a Sandbox Logging (OCSF) section to AGENTS.md so agents have
in-context guidance for deciding whether new log emissions should use
OCSF structured logging or plain tracing. Covers event class selection,
severity guidelines, builder API usage, dual-emit pattern for security
findings, and the no-secrets rule.

Also adds openshell-ocsf to the Architecture Overview table.

* fix: remove workflow files accidentally included during rebase

These files were already merged to main in separate PRs. They got
pulled into our branch during rebase conflict resolution for the
deleted docs-preview-pr.yml file.

* docs(observability): use sandbox connect instead of raw SSH

Users access sandboxes via 'openshell sandbox connect', not direct SSH.

* fix(docs): correct settings CLI syntax in OCSF JSON export page

The settings CLI requires --key and --value named flags, not positional
arguments. Also fix the per-sandbox form: the sandbox name is a
positional argument, not a --sandbox flag.

* fix(e2e): update log assertions for OCSF shorthand format

The E2E tests asserted on the old tracing::fmt key=value format
(action=allow, l7_decision=audit, FORWARD, L7_REQUEST, always-blocked).
Update to match the new OCSF shorthand (ALLOWED/DENIED, HTTP:, NET:,
engine:ssrf, policy:).

* feat(sandbox): convert WebSocket upgrade log calls to OCSF

PR #718 added two log calls for WebSocket upgrade handling:

- 101 Switching Protocols info → NetworkActivity with Upgrade activity.
  This is a significant state change (L7 enforcement drops to raw relay).

- Unsolicited 101 without client Upgrade header → DetectionFinding with
  High severity. A non-compliant upstream sending 101 without a client
  Upgrade request could be attempting to bypass L7 inspection.
 2026-04-07 13:01:13 -07:00
John T. MyersandJohn Myers eea495e6b9 fix: remediate 9 security findings from external audit (OS-15 through OS-23) (#744) 
* fix(install): restrict tar extraction to expected binary member

Prevents CWE-22 path traversal by extracting only the expected APP_NAME
member instead of the full archive contents. Adds --no-same-owner and
--no-same-permissions for defense-in-depth.

OS-20

* fix(deploy): quote registry credentials in YAML heredocs

Wraps username/password values with a yaml_quote helper to prevent YAML
injection from special characters in registry credentials (CWE-94).
Applied to all three heredoc blocks that emit registries.yaml auth.

OS-23

* fix(server): redact session token in SSH tunnel rate-limit log

Logs only the last 4 characters of bearer tokens to prevent credential
exposure in log aggregation systems (CWE-532).

OS-18

* fix(server): escape gateway_display in auth connect page

Applies html_escape() to the Host/X-Forwarded-Host header value before
rendering it into the HTML template, preventing HTML injection (CWE-79).

OS-17

* fix(server): prevent XSS via code param with validation and proper JS escaping

Adds server-side validation rejecting confirmation codes that do not
match the CLI-generated format, replaces manual JS string escaping with
serde_json serialization (handling U+2028/U+2029 line terminators), and
adds a Content-Security-Policy header with nonce-based script-src.

OS-16

* fix(sandbox): add byte cap and idle timeout to streaming inference relay

Prevents resource exhaustion from upstream inference endpoints that stream
indefinitely or hold connections open. Adds a 32 MiB total body limit
and 30-second per-chunk idle timeout (CWE-400).

OS-21

* fix(policy): narrow port field from u32 to u16 to reject invalid values

Prevents meaningless port values >65535 from being accepted in policy
YAML definitions. The proto field remains uint32 (protobuf has no u16)
with validation at the conversion boundary.

OS-22

* fix(deps): migrate from archived serde_yaml to serde_yml

Replaces serde_yaml 0.9 (archived, RUSTSEC-2024-0320) with serde_yml
0.0.12, a maintained API-compatible fork. All import sites updated
across openshell-policy, openshell-sandbox, and openshell-router.

OS-19

* fix(server): re-validate sandbox-submitted security_notes and cap hit_count

The gateway now re-runs security heuristics on proposed policy chunks
instead of trusting sandbox-provided security_notes, validates host
wildcards, caps hit_count at 100, and clamps confidence to [0,1]. The
TUI approve-all path is updated to use ApproveAllDraftChunks RPC which
respects the security_notes filtering gate (CWE-284, confused deputy).

OS-15

* chore: apply cargo fmt and update Cargo.lock for serde_yml

---------

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-02 20:32:59 -07:00
John T. Myers 8887d7c66a fix(sandbox): harden seccomp filter to block dangerous syscalls (#740) 2026-04-02 14:30:51 -07:00
Drew Newberry e837849009 feat(bootstrap): resume gateway from existing state and persist SSH handshake secret (#488) 2026-04-02 09:50:10 -07:00
John T. MyersandJohn Myers c6f3087888 fix(sandbox): relay WebSocket frames after HTTP 101 Switching Protocols (#718) 
Detect 101 Switching Protocols in relay_response() and switch to raw
bidirectional TCP relay instead of re-entering the HTTP parsing loop.

Previously, is_bodiless_response() treated 101 as a generic 1xx
informational response, forwarding only the headers and returning to
the HTTP parsing loop. After a 101, subsequent bytes are upgraded
protocol frames (e.g. WebSocket), not HTTP — causing the relay to
block or silently drop all post-upgrade traffic.

Changes:
- Add RelayOutcome enum (Reusable/Consumed/Upgraded) replacing bool
  return type across L7Provider::relay trait and all relay functions
- Detect 101 before generic 1xx handler in relay_response(), capture
  overflow bytes, return RelayOutcome::Upgraded
- Validate client sent Upgrade + Connection: Upgrade headers before
  accepting 101 (rejects unsolicited upgrades from non-compliant
  upstream servers)
- Extract shared handle_upgrade() helper used by both relay_rest()
  and relay_passthrough_with_credentials()
- Add l7_decision=allow_upgrade audit log annotation for upgrades
- Add unit tests for 101 overflow capture, unsolicited 101 rejection,
  and client_requested_upgrade header validation
- Add integration test: WebSocket echo through L7Provider::relay

Fixes: #652

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-04-01 21:39:41 -07:00
Piotr Mlocek 1c659c1c12 fix(sandbox/bootstrap): GPU Landlock baseline paths and CDI spec missing diagnosis (#710) 
* fix(sandbox): add GPU device nodes and nvidia-persistenced to landlock baseline

Landlock READ_FILE/WRITE_FILE restricts open(2) on character device files
even when DAC permissions would otherwise allow it. GPU sandboxes need
/dev/nvidiactl, /dev/nvidia-uvm, /dev/nvidia-uvm-tools, /dev/nvidia-modeset,
and per-GPU /dev/nvidiaX nodes in the policy to allow NVML initialization.

Additionally, CDI bind-mounts /run/nvidia-persistenced/socket into the
container. NVML tries to connect to this socket at init time; if the
directory is not in the landlock policy, it receives EACCES (not
ECONNREFUSED), which causes NVML to abort with NVML_ERROR_INSUFFICIENT_PERMISSIONS
even though nvidia-persistenced is optional.

Both classes of paths are auto-added to the baseline when /dev/nvidiactl is
present. Per-GPU device nodes are enumerated at runtime to handle multi-GPU
configurations.
 2026-04-01 18:04:25 -07:00
John T. MyersandJohn Myers 9c8d6c714d fix(sandbox): eliminate Box::leak memory leak in rewrite_forward_request (#715) 
Remove Box::leak usage that permanently leaked one String allocation per
forward proxy request. Write the rewritten request line directly to the
output buffer during iteration instead of mutating a Vec<&str> element.

Closes #709

Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
 2026-03-31 19:53:46 -07:00