Allow CLI to connect to gateways with self-signed or untrusted
certificates by skipping TLS verification when --gateway-insecure
is set via flag or OPENSHELL_GATEWAY_INSECURE env var.
Add a UUID-based request-ID middleware using tower-http's request-id
feature. Each inbound request receives a unique x-request-id header
(or preserves a client-supplied one), which is recorded in the tracing
span and propagated to the response.
This enables operators to correlate log lines across the middleware
stack for a single request under concurrent load, and lets clients
reference specific requests in bug reports.
Signed-off-by: sauagarwa <sauagarw@redhat.com>
* feat(auth): add OIDC/Keycloak authentication with RBAC
Add OAuth2/OIDC authentication to the gateway server with role-based
access control, CLI login flows, and full deployment plumbing.
Server: JWT validation against configurable OIDC issuer (oidc.rs),
JWKS key caching with TTL and rotation handling, method classification
(unauthenticated/sandbox-secret/dual-auth/bearer), identity extraction
with provider-agnostic Identity type, and RBAC enforcement via
AuthzPolicy with configurable admin/user roles and auth-only mode.
CLI: browser-based Authorization Code + PKCE flow, Client Credentials
flow for CI/automation, token storage with refresh, gateway add/login/
logout commands, OIDC bearer token injection over mTLS transport,
discovery endpoint for auto-configuration.
Security: sandbox-secret scope restriction on UpdateConfig (policy
sync only), anti-spoofing header stripping, dual-auth fallthrough
from sandbox-secret to Bearer token.
Deployment: OIDC config wired through DeployOptions, Docker env vars,
Helm values/templates, HelmChart manifest, cluster-entrypoint.sh, and
bootstrap scripts. Keycloak dev server script with pre-configured
realm (test users, roles, PKCE client, CI client).
Tested with Keycloak. The roles claim path and role names are
configurable to support other OIDC providers.
* feat(auth): add OAuth2 scope-based fine-grained permissions
Add opt-in scope enforcement on top of existing OIDC role-based access
control. When --oidc-scopes-claim is set, the server extracts scopes
from the JWT and checks them per-method against an exhaustive scope map.
Scopes: sandbox:read, sandbox:write, provider:read, provider:write,
config:read, config:write, inference:read, inference:write, and
openshell:all (wildcard). Methods not in the scope map require
openshell:all. Scopes layer on top of roles and cannot escalate
privilege. Auth-only mode (empty role names) still enforces scopes
when enabled.
Server: scopes_claim in OidcConfig, scope extraction from JWT
(space-delimited and JSON array formats), standard OIDC scope
filtering, scope check in AuthzPolicy after role check.
CLI: --oidc-scopes on gateway add/start stored in metadata and
consumed by gateway login, --oidc-scopes-claim on gateway start
forwarded to server, scopes parameter in browser and client
credentials OAuth2 flows with openid deduplication.
Deployment: oidc_scopes_claim wired through DeployOptions, docker.rs,
Helm, bootstrap scripts, and cluster entrypoint.
Keycloak: realm config updated with built-in OIDC scopes and 9
OpenShell client scopes as optional on openshell-cli and openshell:all
as default on openshell-ci.
* fix(auth): address branch review findings
Add GetInferenceBundle to sandbox-secret methods so sandbox inference
route refresh works under OIDC. Make GetSandboxConfig dual-auth so CLI
users can read sandbox settings with Bearer tokens.
Preserve OIDC gateway metadata on restart — a bare gateway start
without --oidc-* flags no longer erases the stored OIDC registration.
Document CI client ID requirement (openshell-ci vs openshell-cli) in
the testing guide. Add security note about auth-only mode blast radius
for GitHub Actions.
* fix(auth): complete review findings for OIDC auth boundary
Move OpenShell/GetSandboxConfig from sandbox-secret-only to dual-auth
so CLI users can read sandbox settings with Bearer tokens while sandbox
supervisors continue using the shared secret.
Add sandbox secret interceptor to the inference bundle fetch path so
GetInferenceBundle works under OIDC-enabled gateways. Extract shared
interceptor constructor to avoid duplication.
Add GetSandboxConfig to the config:read scope map so scope enforcement
applies consistently when scopes are enabled.
Refactor OIDC metadata preservation into apply_oidc_gateway_metadata()
with explicit resume semantics — only preserve existing OIDC metadata
on real resume paths, not on fresh deployments.
Update architecture docs and testing guide to reflect the corrected
method classifications and add new test coverage for interceptor
injection, scope requirements, metadata preservation, and dual-auth
classification.
* refactor(auth): use oauth2 crate for CLI OIDC flows
Replace hand-written PKCE generation, authorization URL construction,
token exchange, client credentials, and token refresh with the oauth2
crate's typed API.
Eliminates sha2, hex, and getrandom dependencies from the CLI. The
custom urlencoded() helper and manual form POST logic are replaced by
BasicClient methods with proper type-state safety.
Discovery and the callback server remain custom since the oauth2 crate
does not provide OIDC discovery or a localhost redirect listener.
* refactor(auth): move server auth modules into auth/ directory
Group oidc.rs, authz.rs, identity.rs, and the auth HTTP endpoints
under src/auth/ module directory. No behavioral changes.
auth/mod.rs — module root, re-exports HTTP router
auth/oidc.rs — JWT validation, JWKS caching, method classification
auth/authz.rs — role and scope authorization policy
auth/identity.rs — provider-agnostic Identity type
auth/http.rs — /auth/connect and /auth/oidc-config endpoints
* fix(auth): use RequestBody auth type for client credentials flow
The oauth2 crate defaults to BasicAuth (HTTP Basic header) but Keycloak
and most OIDC providers expect client_secret_post (credentials in the
request body). Set AuthType::RequestBody explicitly to match the
pre-refactor behavior.
Also re-export Identity, IdentityProvider, and JwksCache from the auth
module so ServerState's public API remains nameable by external consumers.
* fix(auth): forward OPENSHELL_OIDC_SCOPES through cluster bootstrap
Pass --oidc-scopes to gateway start so the metadata includes requested
scopes after cluster bootstrap. Without this, users had to manually
edit metadata.json to set scopes for gateway login.
Usage: OPENSHELL_OIDC_SCOPES="openshell:all" mise run cluster
* test(auth): add OIDC e2e tests for RBAC, scopes, and client credentials
Add 10 end-to-end tests covering OIDC authentication against a live
K3s cluster with Keycloak:
RBAC (5 tests): admin can create providers, user cannot, user can list
sandboxes, unauthenticated requests rejected, health probe works
without auth.
Scopes (4 tests): sandbox-scoped token can list sandboxes but not
providers, openshell:all grants full access, no-scopes token denied.
Client credentials (1 test): CI token via client_credentials grant.
Tests are opt-in via OPENSHELL_E2E_OIDC=1 and OPENSHELL_E2E_OIDC_SCOPES=1
env vars. They derive the Keycloak URL from gateway metadata to match
the server's configured issuer.
Run with:
OPENSHELL_E2E_OIDC=1 OPENSHELL_E2E_OIDC_SCOPES=1 \
PYTHONPATH=python uv run pytest e2e/python/oidc/ -v
* fix(docs): fix markdown lint errors in OIDC architecture docs
Add blank lines before lists and fenced code blocks to satisfy
markdownlint MD031 and MD032 rules.
* Adding qemu vm driver support with GPU pass-through
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
* Add GPU rootfs variant, harden VFIO binding, and fix networking and supervisor reliability issues discovered during GPU VM bring-up.
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
---------
Signed-off-by: Vincent Caux-Brisebois <vcauxbrisebo@nvidia.com>
* feat(podman): add Podman compute driver for rootless sandbox management
Adds openshell-driver-podman, a new compute driver that manages OpenShell
sandboxes as rootless Podman containers via the Podman REST API over a
Unix socket. Enables local workstation sandboxes without Kubernetes.
Driver features:
- Bridge networking with ephemeral host-port mapping for rootless SSH reachability
- Named volumes for workspace storage, Podman native health checks, GPU via CDI
- Supervisor binary sideloaded via image volume mount (BYOC-compatible)
- SSH handshake secret injected via Podman secrets API (not plaintext env)
- Typed ContainerSpec structs, input validation, and path-traversal guards
- Cgroups v2 required; fails fast on v1 hosts
- Bounded event stream buffer; watch stream reconnection handled by server watch_loop
- Graceful shutdown and standalone driver binary with gRPC bridge
Rootless-specific fixes:
- Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps
- Add /run/netns tmpfs mount for ip netns in rootless containers
- Use secret_env map (not secrets array) for env-var injection in libpod API
- Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP
Server/sandbox hardening:
- Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback
- Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting
- Prevent OPENSHELL_* env var override by user-supplied spec environment maps
- Disable SQLite pool idle_timeout/max_lifetime for in-memory databases
- Emit deleted_event on 404-during-inspect instead of regressing sandbox phase
- Key delete cleanup by stable sandbox_id to survive container label drift
CLI fixes:
- Restore --name as a named flag on sandbox create (not positional)
- Fix exec command arg parsing to not consume sandboxed-command flags
- Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL
Build tooling:
- Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes
unified ce_* helpers; all build/cluster/VM scripts updated to use it
- Add docker:build:supervisor mise task for standalone supervisor image
- Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages
- Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(driver-podman): derive grpc endpoint from server bind port
When a user starts the gateway on a non-default port (e.g. --port 8081),
sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the
default port 8080. The driver's auto-detection fallback read
OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset,
and fell back to DEFAULT_SERVER_PORT.
Add gateway_port to PodmanComputeConfig and thread config.bind_address.port()
from the server into the driver so the fallback uses the actual listening
port. Remove the OPENSHELL_BIND_ADDRESS env var read and the
extract_port_from_bind_address helper which are no longer needed.
Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver
binary for parity when the driver is run outside the embedded server path.
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(driver-podman): address PR feedback on env test safety and cluster DNS docs
Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars
and a static ENV_LOCK mutex, fixing a data race in parallel test execution.
The prior safety comment incorrectly claimed Cargo runs tests single-threaded.
Update debug-openshell-cluster skill to accurately document the DNS proxy
strategy (setup_dns_proxy + public DNS fallback) and clarify the separation
between cluster DNS and sandbox agent DNS enforcement.
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(e2e): resolve CI failures in auth timeout, test harness, and formatting
- Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead
of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives
- Add timeout to SandboxGuard::create() and create_with_upload() to
prevent indefinite hangs (matches create_keep() which already had one)
- Add missing '--' separator in no_proxy test before command args
- Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs
- Run cargo fmt on openshell-driver-podman
- Refine cluster DNS docs for Podman in debug-openshell-cluster skill
Signed-off-by: Adam Miller <admiller@redhat.com>
* refactor(server): remove allows_loopback_endpoints from ComputeRuntime
SSRF protection is now handled at the network and proxy layers
(openshell-core net.rs, openshell-sandbox proxy.rs) rather than
requiring per-driver flags on ComputeRuntime. Update architecture
docs to reflect supervisor relay SSH transport and add rootless
networking deep-dive.
Signed-off-by: Adam Miller <admiller@redhat.com>
---------
Signed-off-by: Adam Miller <admiller@redhat.com>
* feat(server): add Prometheus metrics infrastructure and gRPC/HTTP request metrics
Add metrics exposition via a dedicated server port (--metrics-port,
default disabled) following the same optional-listener pattern as the
health port. The metrics crate facade records counters and histograms
in the MultiplexedService hot path, and a PrometheusHandle renders them
at GET /metrics on the dedicated port.
Metrics added:
- openshell_server_grpc_requests_total (counter: method, code)
- openshell_server_grpc_request_duration_seconds (histogram: method, code)
- openshell_server_http_requests_total (counter: path, status)
- openshell_server_http_request_duration_seconds (histogram: path, status)
* feat(helm): add metrics port to openshell-server helm chart
Expose the Prometheus metrics endpoint in the helm chart by adding
service.metricsPort (default 9090) to values, the --metrics-port arg
and container port to the statefulset, and a metrics service port.
Set metricsPort to 0 to disable.
Policies with allowed_ips entries targeting loopback, link-local, or
unspecified ranges now fail at connection time instead of being silently
blocked at runtime. The shorthand log format for DENIED events includes
a [reason:...] suffix so operators can distinguish 'allowlist miss' from
'structurally un-allowable'. The mechanistic mapper skips proposals for
always-blocked destinations, preventing the infinite TUI notification
loop. The gateway validates proposed rules on approval as defense-in-depth.
- Extract shared IP helpers (is_always_blocked_ip, is_always_blocked_net,
is_internal_ip) to openshell_core::net
- Reject always-blocked entries in parse_allowed_ips with hard error
- Skip implicit allowed_ips synthesis for always-blocked literal IP hosts
- Add status_detail to HttpActivityBuilder for denial reason propagation
- Enrich NET and HTTP shorthand with [reason:...] for DENIED events
- Add engine: tag to HTTP shorthand (consistency with NET shorthand)
- Filter always-blocked proposals in mechanistic mapper generate_proposals
- Add validate_rule_not_always_blocked server-side defense-in-depth
- Update architecture docs, published docs, and E2E test assertions
* feat(prover): add native Rust policy prover with Z3 solver
Add openshell-prover crate implementing formal policy verification
using Z3 SMT solving. Answers two questions about any sandbox policy:
"Can data leave?" and "Can the agent write despite read-only intent?"
Native Rust — no Python subprocess, no PYTHONPATH, no uv dependency.
Z3 bundled via z3-sys for self-contained builds.
Replaces the Python prototype from #703.
Closes#699
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* fix(prover): skip L7-write in exfil, write bypass only on read-only intent
Port two fixes from the Python branch:
- Exfil query skips endpoints where L7 is enforced and working
- Write bypass only fires on explicit read-only intent, not L4-only
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* chore(prover): add missing SPDX license headers to registry and testdata YAML files
* fix(prover): revert serde_yaml to serde_yml to match workspace dependency on main
* fix(prover): apply cargo fmt formatting to prover and cli source files
* ci: add clang and libclang-dev to CI image for z3-sys bindgen
z3-sys requires libclang at build time for bindgen to generate FFI
bindings. Without it, the Rust CI jobs fail on the prover crate.
* fix(prover): use system libz3 instead of compiling from source
Add libz3-dev to the CI image and drop the z3 `bundled` feature from
the workspace dependency. This eliminates the ~30 min z3 C++ build
that ran on every CI cache miss.
sccache only wraps rustc — it does not intercept the cmake C++ build
that z3-sys runs when `bundled` is enabled. The cargo target cache
helped on warm runs but evicts on toolchain bumps and fork PRs.
With the system library pre-installed in the CI image, z3 link time
is always zero.
A `bundled-z3` opt-in feature is added to the prover crate for local
development without system z3:
cargo build -p openshell-prover --features bundled-z3
Regular local dev: brew install z3 (macOS) or apt install libz3-dev
(Linux), then cargo build just works.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor
* fix(prover): use u16 for ports, align include_workdir default with runtime
Port fields changed from u32 to u16 across all prover types (policy,
model, finding, queries). Prevents the prover from silently accepting
port values >65535 that the runtime rejects, which would produce
misleading PASS results on invalid policies.
Change include_workdir serde default from true to false to match the
runtime (openshell-policy uses #[serde(default)] which gives false).
The previous mismatch caused the prover to model a /sandbox path that
does not exist at runtime, producing false analysis.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor
* refactor(prover): embed registry at compile time, replace hand-rolled glob
Embed binary and API capability registry YAML files into the binary at
compile time using include_dir!. The previous approach used
env!("CARGO_MANIFEST_DIR") which bakes in the build machine's source
path — works in tests but breaks for installed binaries. The CWD
fallback was equally fragile.
The --registry CLI flag still works as a filesystem override for custom
registries. Credentials remain filesystem-loaded (user-supplied data).
Replace the hand-rolled glob matching (~60 lines) with the glob crate's
Pattern::matches(), which is already a transitive dependency.
Remove unused dependencies: openshell-policy, openshell-core, thiserror,
tracing. The prover parses policy YAML directly into its own types and
does not use the policy or core crate APIs.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor
* docs: add Z3 system library to prerequisites
The prover crate now links against system libz3 instead of compiling
from source. Document the install steps for macOS, Ubuntu, and Fedora,
and note the bundled-z3 feature flag as a fallback.
Signed-off-by: Alexander Watson <zredlined@gmail.com>
Made-with: Cursor
* fix(prover): apply cargo fmt formatting
Made-with: Cursor
---------
Signed-off-by: Alexander Watson <zredlined@gmail.com>
* feat(sandbox): integrate OCSF structured logging for all sandbox events
WIP: Replace ad-hoc tracing calls with OCSF event builders across all
sandbox subsystems (network, SSH, process, filesystem, config, lifecycle).
- Register ocsf_logging_enabled setting (defaults false)
- Replace stdout/file fmt layers with OcsfShorthandLayer
- Add conditional OcsfJsonlLayer for /var/log/openshell-ocsf.log
- Update LogPushLayer to extract OCSF shorthand for gRPC push
- Migrate ~106 log sites to OCSF builders (NetworkActivity, HttpActivity,
SshActivity, ProcessActivity, DetectionFinding, ConfigStateChange,
AppLifecycle)
- Add openshell-ocsf to all Docker build contexts
* fix(scripts): attach provider to all smoke test phases to avoid rate limits
GitHub's unauthenticated API rate limit (60/hour) causes flaky 403s for
Phases 1, 2, and 4. Fix by attaching the provider to all sandboxes and
upgrading the Phase 1 policy to L7 so credential injection works.
Phase 4 (tls:skip) cannot inject credentials by design, so relax the
assertion to accept either 200 or 403 from upstream -- both prove the
proxy forwarded the request.
* fix(ocsf): remove timestamp from shorthand format to avoid double-timestamp
The display layer (gateway logs, TUI, sandbox logs CLI) already prepends
a timestamp. Having one in the shorthand output too produces redundant
double-timestamps like:
15:49:11 sandbox INFO 15:49:11.649 I NET:OPEN ALLOWED ...
Now the shorthand is just the severity + structured content:
15:49:11 sandbox INFO I NET:OPEN ALLOWED ...
* refactor(ocsf): replace single-char severity with bracketed labels
Replace cryptic single-character severity codes (I/L/M/H/C/F) with
readable bracketed labels: [LOW], [MED], [HIGH], [CRIT], [FATAL].
Informational severity (the happy-path default) is omitted entirely to
keep normal log output clean and avoid redundancy with the tracing-level
INFO that the display layer already provides.
Before: sandbox INFO I NET:OPEN ALLOWED ...
After: sandbox INFO NET:OPEN ALLOWED ...
Before: sandbox INFO M NET:OPEN DENIED ...
After: sandbox INFO [MED] NET:OPEN DENIED ...
* feat(sandbox): use OCSF level label for structured events in log push
Set the level field to 'OCSF' instead of 'INFO' for OCSF events in the
gRPC log push. This visually distinguishes structured OCSF events from
plain tracing output in the TUI and CLI sandbox logs:
sandbox OCSF NET:OPEN [INFO] ALLOWED python3(42) -> api.example.com:443
sandbox OCSF NET:OPEN [MED] DENIED python3(42) -> blocked.com:443
sandbox INFO Fetching sandbox policy via gRPC
* fix(sandbox): convert new Landlock path-skip warning to OCSF
PR #677 added a warn!() for inaccessible Landlock paths in best-effort
mode. Convert to ConfigStateChangeBuilder with degraded state so it
flows through the OCSF shorthand format consistently.
* fix(sandbox): use rolling appender for OCSF JSONL file
Match the main openshell.log rotation mechanics (daily, 3 files max)
instead of a single unbounded append-only file. Prevents disk exhaustion
when ocsf_logging_enabled is left on in long-running sandboxes.
* fix(sandbox): address reviewer warnings for OCSF integration
W1: Remove redundant 'OCSF' prefix from shorthand file layer — the
class name (NET:OPEN, HTTP:GET) already identifies structured events
and the LogPushLayer separately sets the level field.
W2: Log a debug message when OCSF_CTX.set() is called a second time
instead of silently discarding via let _.
W3: Document the boundary between OCSF-migrated events and intentionally
plain tracing calls (DEBUG/TRACE, transient, internal plumbing).
W4: Migrate remaining iptables LOG rule failure warnings in netns.rs
(IPv4 TCP/UDP, IPv6 TCP/UDP) to ConfigStateChangeBuilder for
consistency with the IPv4 bypass rule failure already migrated.
W5: Migrate malformed inference request warn to NetworkActivity with
ActivityId::Refuse and SeverityId::Medium.
W6: Use Medium severity for L7 deny decisions (both CONNECT tunnel and
FORWARD proxy paths) to match the CONNECT deny severity pattern.
Allows and audits remain Informational.
* refactor(sandbox): rename ocsf_logging_enabled to ocsf_json_enabled
The shorthand logs are already OCSF-structured events. The setting
specifically controls the JSONL file export, so the name should reflect
that: ocsf_json_enabled.
* fix(ocsf): add timestamps to shorthand file layer output
The OcsfShorthandLayer writes directly to the log file with no outer
display layer to supply timestamps. Add a UTC timestamp prefix to every
line so the file output matches what tracing::fmt used to provide.
Before: CONFIG:VALIDATED [INFO] Validated 'sandbox' user exists in image
After: 2026-04-01T15:49:11.649Z CONFIG:VALIDATED [INFO] Validated ...
* fix(docker): touch openshell-ocsf source to invalidate cargo cache
The supervisor-workspace stage touches sandbox and core sources to force
recompilation over the rust-deps dummy stubs, but openshell-ocsf was
missing. This caused the Docker cargo cache to use stale ocsf objects
from the deps stage, preventing changes to the ocsf crate (like the
timestamp fix) from appearing in the final binary.
Also adds a shorthand layer test verifying timestamp output, and drafts
the observability docs section.
* fix(ocsf): add OCSF level prefix to file layer shorthand output
Without a level prefix, OCSF events in the log file have no visual
anchor at the position where standard tracing lines show INFO/WARN.
This makes scanning the file harder since the eye has nothing consistent
to lock onto after the timestamp.
Before: 2026-04-01T04:04:13.065Z CONFIG:DISCOVERY [INFO] ...
After: 2026-04-01T04:04:13.065Z OCSF CONFIG:DISCOVERY [INFO] ...
* fix(ocsf): clean up shorthand formatting for listen and SSH events
- Fix double space in NET:LISTEN, SSH:LISTEN, and other events where
action is empty (e.g., 'NET:LISTEN [INFO] 10.200.0.1' -> 'NET:LISTEN [INFO] 10.200.0.1')
- Add listen address to SSH:LISTEN event (was empty)
- Downgrade SSH handshake intermediate steps (reading preface, verifying)
from OCSF events to debug!() traces. Only the final verdict
(accepted/denied) is an OCSF event now, reducing noise from 3 events
to 1 per SSH connection.
- Apply same spacing fix to HTTP shorthand for consistency.
* docs(observability): update examples with OCSF prefix and formatting fixes
Align doc examples with the deployed output:
- Add OCSF level prefix to all shorthand examples in the log file
- Show mixed OCSF + standard tracing in the file format section
- Update listen events (no double space, SSH includes address)
- Show one SSH:OPEN per connection instead of three
- Update grep patterns to use 'OCSF NET:' etc.
* docs(agents): add OCSF logging guidance to AGENTS.md
Add a Sandbox Logging (OCSF) section to AGENTS.md so agents have
in-context guidance for deciding whether new log emissions should use
OCSF structured logging or plain tracing. Covers event class selection,
severity guidelines, builder API usage, dual-emit pattern for security
findings, and the no-secrets rule.
Also adds openshell-ocsf to the Architecture Overview table.
* fix: remove workflow files accidentally included during rebase
These files were already merged to main in separate PRs. They got
pulled into our branch during rebase conflict resolution for the
deleted docs-preview-pr.yml file.
* docs(observability): use sandbox connect instead of raw SSH
Users access sandboxes via 'openshell sandbox connect', not direct SSH.
* fix(docs): correct settings CLI syntax in OCSF JSON export page
The settings CLI requires --key and --value named flags, not positional
arguments. Also fix the per-sandbox form: the sandbox name is a
positional argument, not a --sandbox flag.
* fix(e2e): update log assertions for OCSF shorthand format
The E2E tests asserted on the old tracing::fmt key=value format
(action=allow, l7_decision=audit, FORWARD, L7_REQUEST, always-blocked).
Update to match the new OCSF shorthand (ALLOWED/DENIED, HTTP:, NET:,
engine:ssrf, policy:).
* feat(sandbox): convert WebSocket upgrade log calls to OCSF
PR #718 added two log calls for WebSocket upgrade handling:
- 101 Switching Protocols info → NetworkActivity with Upgrade activity.
This is a significant state change (L7 enforcement drops to raw relay).
- Unsolicited 101 without client Upgrade header → DetectionFinding with
High severity. A non-compliant upstream sending 101 without a client
Upgrade request could be attempting to bypass L7 inspection.
* fix(install): restrict tar extraction to expected binary member
Prevents CWE-22 path traversal by extracting only the expected APP_NAME
member instead of the full archive contents. Adds --no-same-owner and
--no-same-permissions for defense-in-depth.
OS-20
* fix(deploy): quote registry credentials in YAML heredocs
Wraps username/password values with a yaml_quote helper to prevent YAML
injection from special characters in registry credentials (CWE-94).
Applied to all three heredoc blocks that emit registries.yaml auth.
OS-23
* fix(server): redact session token in SSH tunnel rate-limit log
Logs only the last 4 characters of bearer tokens to prevent credential
exposure in log aggregation systems (CWE-532).
OS-18
* fix(server): escape gateway_display in auth connect page
Applies html_escape() to the Host/X-Forwarded-Host header value before
rendering it into the HTML template, preventing HTML injection (CWE-79).
OS-17
* fix(server): prevent XSS via code param with validation and proper JS escaping
Adds server-side validation rejecting confirmation codes that do not
match the CLI-generated format, replaces manual JS string escaping with
serde_json serialization (handling U+2028/U+2029 line terminators), and
adds a Content-Security-Policy header with nonce-based script-src.
OS-16
* fix(sandbox): add byte cap and idle timeout to streaming inference relay
Prevents resource exhaustion from upstream inference endpoints that stream
indefinitely or hold connections open. Adds a 32 MiB total body limit
and 30-second per-chunk idle timeout (CWE-400).
OS-21
* fix(policy): narrow port field from u32 to u16 to reject invalid values
Prevents meaningless port values >65535 from being accepted in policy
YAML definitions. The proto field remains uint32 (protobuf has no u16)
with validation at the conversion boundary.
OS-22
* fix(deps): migrate from archived serde_yaml to serde_yml
Replaces serde_yaml 0.9 (archived, RUSTSEC-2024-0320) with serde_yml
0.0.12, a maintained API-compatible fork. All import sites updated
across openshell-policy, openshell-sandbox, and openshell-router.
OS-19
* fix(server): re-validate sandbox-submitted security_notes and cap hit_count
The gateway now re-runs security heuristics on proposed policy chunks
instead of trusting sandbox-provided security_notes, validates host
wildcards, caps hit_count at 100, and clamps confidence to [0,1]. The
TUI approve-all path is updated to use ApproveAllDraftChunks RPC which
respects the security_notes filtering gate (CWE-284, confused deputy).
OS-15
* chore: apply cargo fmt and update Cargo.lock for serde_yml
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Detect 101 Switching Protocols in relay_response() and switch to raw
bidirectional TCP relay instead of re-entering the HTTP parsing loop.
Previously, is_bodiless_response() treated 101 as a generic 1xx
informational response, forwarding only the headers and returning to
the HTTP parsing loop. After a 101, subsequent bytes are upgraded
protocol frames (e.g. WebSocket), not HTTP — causing the relay to
block or silently drop all post-upgrade traffic.
Changes:
- Add RelayOutcome enum (Reusable/Consumed/Upgraded) replacing bool
return type across L7Provider::relay trait and all relay functions
- Detect 101 before generic 1xx handler in relay_response(), capture
overflow bytes, return RelayOutcome::Upgraded
- Validate client sent Upgrade + Connection: Upgrade headers before
accepting 101 (rejects unsolicited upgrades from non-compliant
upstream servers)
- Extract shared handle_upgrade() helper used by both relay_rest()
and relay_passthrough_with_credentials()
- Add l7_decision=allow_upgrade audit log annotation for upgrades
- Add unit tests for 101 overflow capture, unsolicited 101 rejection,
and client_requested_upgrade header validation
- Add integration test: WebSocket echo through L7Provider::relay
Fixes: #652
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
The image push pipeline buffered the entire Docker image tar 3x in
memory (export, tar wrap, Bytes copy), causing OOM kills for images
over ~1-2 GB. Replace the in-memory pipeline with a temp-file +
streaming upload: export to a NamedTempFile, then stream the outer
tar (header, 8 MiB file chunks, footer) directly into
upload_to_container via body_try_stream. Peak memory drops from ~3x
image size to ~8 MiB constant.
Also adds incremental export progress reporting every 100 MiB.
Closes#273
Verify inference endpoints synchronously on the server during set/update, expose a --no-verify escape hatch in the CLI and Python helper, and return actionable failures when validation does not pass.
* feat(tui): add log copy and visual selection mode
Add clipboard support for the TUI log viewer using OSC 52 escape
sequences. Users can copy individual log lines (y), the visible
viewport (Y), or enter visual selection mode (v) to select a range
of lines with j/k and yank with y.
- Add clipboard module with OSC 52 base64 encoding
- Add format_log_line_plain() for plain-text log rendering
- Add visual selection mode with highlight styling and status bar
- Context-switch nav bar hints between normal and visual modes
- Add 7 unit tests for plain-text log formatting
Closes#274
* fix(tui): write OSC 52 to /dev/tty instead of stdout
ratatui owns stdout via the alternate screen buffer, so OSC 52
escape sequences written to stdout are swallowed by the backend.
Write directly to /dev/tty to bypass the buffer and reach the
terminal emulator.
* feat(tui): support light terminal backgrounds with adaptive theme
Replace the hardcoded dark-only color constants in theme.rs with a
runtime Theme struct that has dark() and light() factory constructors.
The active theme is stored on App and threaded through all draw
functions, enabling the TUI to render correctly on both dark and light
terminal backgrounds.
- Add Theme struct with 16 semantic style fields and ThemeMode enum
- Detect terminal background via COLORFGBG env var at startup
- Add --theme dark|light|auto CLI flag and OPENSHELL_THEME env var
- Migrate all 499 styles:: references across 11 UI files to app.theme
- Clean up 9 inline Style constructions that bypassed the theme system
- Add unit tests verifying dark/light palettes and legacy regression
Closes#264
* fix(tui): use OSC 11 query for reliable light/dark terminal detection
Replace COLORFGBG env var heuristic with terminal-colorsaurus crate,
which sends an OSC 11 query to read the actual background color. This
fixes auto-detection in iTerm2 and other terminals that don't set
COLORFGBG.
* feat(policy): add validation layer to reject unsafe sandbox policies
Add policy validation that checks for root process identity, path
traversal sequences, overly broad filesystem paths, and exceeding
filesystem rule limits. Validation runs at three entry points:
disk-loaded YAML policies (fallback to restrictive default on violation),
gRPC CreateSandbox, and gRPC UpdateSandboxPolicy (returns
INVALID_ARGUMENT). Filesystem paths are normalized before storage to
collapse traversal components.
Closes#33
* fix(e2e): correct policy update test to match immutable field behavior
The update policy test was asserting on validation errors for fields
(process, filesystem) that are immutable on live sandboxes. The server
rejects changes to these fields before validation runs. Updated the test
to verify the immutability guard instead.
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(tui): add port forwarding support to Gator (#80)
Extract forward PID management, resolve_ssh_gateway, and shell_escape from
navigator-cli into navigator-core::forward as a shared module. Add a Ports
field to the TUI create sandbox modal, a NOTES column to the sandbox table,
and a Forwards row to the sandbox detail view. On creation with ports, Gator
polls for Ready state then spawns background SSH tunnels. Forward cleanup
runs automatically on sandbox delete.
Closes#80
* fix(tui): fix Ports field invisible due to modal height overflow
The modal height calculation used content_height + 3 but block chrome
(borders + padding) is 4 rows, not 3. Combined with the 3-row Ports
field layout, the input line was clipped to zero height on standard
24-row terminals. Switch Ports to a compact single-line inline layout
and fix the chrome arithmetic.
* wip: BYOC example with port forwarding and TUI command field fix
- Move examples/bring-your-own-container.md into its own directory
- Add example Dockerfile and app.py (Python REST API with /hello endpoint)
- Rewrite README with full CLI and TUI port-forward workflows
- Change TUI Command field default from /bin/bash to empty so custom
image entrypoints run without manual clearing
* feat(tui): wire up Command field to SSH exec after sandbox creation
When a command is specified in the create sandbox modal, the TUI now
waits for the sandbox to reach Ready (with the pacman animation),
starts any port forwards, then suspends the TUI and executes the
command via SSH — matching the CLI's post-creation flow.
- Refactor spawn_create_sandbox to poll for Ready inline when ports
or command are set, keeping the animation visible throughout
- Add start_port_forwards() helper called within the create task
- Add handle_exec_command() to suspend TUI and run SSH exec
- Remove unused ForwardResult event variant (forwards are now started
within the create task, not as a separate event)
* fix(tui): fix command exec, forward timeouts, and create modal UX
- Fix handle_exec_command to suspend TUI and run SSH attached (matching
CLI behavior) so the process stays alive for the session duration
- Fix shell_escape double-quoting: escape each word individually
- Add ConnectTimeout and 20s spawn timeout to forward SSH so a stalled
auth doesn't freeze the create flow forever
- Remove dead ForwardResult event variant
- Add cluster_name to start_port_forwards for ProxyCommand
- Add spacer between Providers and Ports in create modal
- Fix Command placeholder: 'runs /bin/bash if empty' (not image entrypoint)
- Update BYOC README: document that CMD is replaced by supervisor,
command must be passed explicitly, remove TUI-specific sections
* fix(sandbox): demote Landlock fallback log from warn to debug
The Landlock filesystem sandbox emits a noisy warning when a policy
path does not exist (e.g. /app in custom images). Since BestEffort
mode intentionally continues without Landlock, demote to debug.
* docs: fix BYOC Dockerfile comments for supervisor CMD override
---------
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(sandbox): allow egress to private IP space via allowed_ips policy field
Add an allowed_ips field to NetworkEndpoint that accepts CIDR notation and
exact IPs, enabling sandboxes to reach private IP space under policy control
while maintaining SSRF protections for loopback and link-local addresses.
Three modes are supported:
- Default (no allowed_ips): all private IPs blocked (existing behavior)
- Host + allowlist: domain must resolve to an IP in allowed_ips
- Hostless allowlist (no host): any domain allowed if resolved IP matches
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Taylor Mutch