* feat(podman): add Podman compute driver for rootless sandbox management
Adds openshell-driver-podman, a new compute driver that manages OpenShell
sandboxes as rootless Podman containers via the Podman REST API over a
Unix socket. Enables local workstation sandboxes without Kubernetes.
Driver features:
- Bridge networking with ephemeral host-port mapping for rootless SSH reachability
- Named volumes for workspace storage, Podman native health checks, GPU via CDI
- Supervisor binary sideloaded via image volume mount (BYOC-compatible)
- SSH handshake secret injected via Podman secrets API (not plaintext env)
- Typed ContainerSpec structs, input validation, and path-traversal guards
- Cgroups v2 required; fails fast on v1 hosts
- Bounded event stream buffer; watch stream reconnection handled by server watch_loop
- Graceful shutdown and standalone driver binary with gRPC bridge
Rootless-specific fixes:
- Skip drop_privileges when user namespace lacks SETUID/SETGID/DAC_READ_SEARCH caps
- Add /run/netns tmpfs mount for ip netns in rootless containers
- Use secret_env map (not secrets array) for env-var injection in libpod API
- Resolve SSH endpoint to 127.0.0.1:<host_port> instead of unreachable bridge IP
Server/sandbox hardening:
- Split loopback and link-local SSRF gates; Podman/VM drivers allow loopback
- Close SSRF bypass in SSH tunnel Host path by resolving DNS before connecting
- Prevent OPENSHELL_* env var override by user-supplied spec environment maps
- Disable SQLite pool idle_timeout/max_lifetime for in-memory databases
- Emit deleted_event on 404-during-inspect instead of regressing sandbox phase
- Key delete cleanup by stable sandbox_id to survive container label drift
CLI fixes:
- Restore --name as a named flag on sandbox create (not positional)
- Fix exec command arg parsing to not consume sandboxed-command flags
- Propagate SSH verbosity via OPENSHELL_SSH_LOG_LEVEL
Build tooling:
- Add tasks/scripts/container-engine.sh: auto-detects Podman or Docker, exposes
unified ce_* helpers; all build/cluster/VM scripts updated to use it
- Add docker:build:supervisor mise task for standalone supervisor image
- Add openshell-driver-podman to Dockerfile.images pre-fetch/build stages
- Add e2e/rust/e2e-podman.sh and e2e:podman mise task for full lifecycle testing
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(driver-podman): derive grpc endpoint from server bind port
When a user starts the gateway on a non-default port (e.g. --port 8081),
sandbox containers were receiving OPENSHELL_ENDPOINT pointing at the
default port 8080. The driver's auto-detection fallback read
OPENSHELL_BIND_ADDRESS from the environment, which was stale or unset,
and fell back to DEFAULT_SERVER_PORT.
Add gateway_port to PodmanComputeConfig and thread config.bind_address.port()
from the server into the driver so the fallback uses the actual listening
port. Remove the OPENSHELL_BIND_ADDRESS env var read and the
extract_port_from_bind_address helper which are no longer needed.
Add --gateway-port / OPENSHELL_GATEWAY_PORT to the standalone driver
binary for parity when the driver is run outside the embedded server path.
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(driver-podman): address PR feedback on env test safety and cluster DNS docs
Replace hand-rolled unsafe TempEnvVar RAII guard with temp_env::with_vars
and a static ENV_LOCK mutex, fixing a data race in parallel test execution.
The prior safety comment incorrectly claimed Cargo runs tests single-threaded.
Update debug-openshell-cluster skill to accurately document the DNS proxy
strategy (setup_dns_proxy + public DNS fallback) and clarify the separation
between cluster DNS and sandbox agent DNS enforcement.
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix(e2e): resolve CI failures in auth timeout, test harness, and formatting
- Short-circuit browser_auth_flow when OPENSHELL_NO_BROWSER=1 instead
of waiting the full 120s AUTH_TIMEOUT for a callback that never arrives
- Add timeout to SandboxGuard::create() and create_with_upload() to
prevent indefinite hangs (matches create_keep() which already had one)
- Add missing '--' separator in no_proxy test before command args
- Add #![cfg(feature = "e2e")] gate to sandbox_lifecycle.rs
- Run cargo fmt on openshell-driver-podman
- Refine cluster DNS docs for Podman in debug-openshell-cluster skill
Signed-off-by: Adam Miller <admiller@redhat.com>
* refactor(server): remove allows_loopback_endpoints from ComputeRuntime
SSRF protection is now handled at the network and proxy layers
(openshell-core net.rs, openshell-sandbox proxy.rs) rather than
requiring per-driver flags on ComputeRuntime. Update architecture
docs to reflect supervisor relay SSH transport and add rootless
networking deep-dive.
Signed-off-by: Adam Miller <admiller@redhat.com>
---------
Signed-off-by: Adam Miller <admiller@redhat.com>
* feat(server): add Prometheus metrics infrastructure and gRPC/HTTP request metrics
Add metrics exposition via a dedicated server port (--metrics-port,
default disabled) following the same optional-listener pattern as the
health port. The metrics crate facade records counters and histograms
in the MultiplexedService hot path, and a PrometheusHandle renders them
at GET /metrics on the dedicated port.
Metrics added:
- openshell_server_grpc_requests_total (counter: method, code)
- openshell_server_grpc_request_duration_seconds (histogram: method, code)
- openshell_server_http_requests_total (counter: path, status)
- openshell_server_http_request_duration_seconds (histogram: path, status)
* feat(helm): add metrics port to openshell-server helm chart
Expose the Prometheus metrics endpoint in the helm chart by adding
service.metricsPort (default 9090) to values, the --metrics-port arg
and container port to the statefulset, and a metrics service port.
Set metricsPort to 0 to disable.
Move /health, /healthz, and /readyz to a dedicated plaintext HTTP port
(default 8081) so Kubernetes probes work without mTLS client certificates.
- Add health_bind_address to Config with --health-port CLI arg
- Spawn standalone axum::serve for health_router on the health port
- Remove health routes from the main multiplexed HTTP router
- Update Helm statefulset probes from tcpSocket to httpGet on health port
- Fix cluster-healthcheck.sh to open/close TCP without sending data,
avoiding InvalidContentType TLS errors in the gateway log
The CLI and TUI build an SSH ProxyCommand string by interpolating fields
from CreateSshSessionResponse into a shell command. Three fields -
sandbox_id, token, and the assembled gateway_url (composed from
gateway_scheme, gateway_host, gateway_port, connect_path) - were not
passed through shell_escape, even though exe_command and gateway_name
were. The resulting string is handed to `ssh -o ProxyCommand=...` which
OpenSSH routes through `/bin/sh -c`, so a hostile or compromised gateway
could inject shell metacharacters and execute arbitrary commands under
the developer's workstation account.
- Add build_proxy_command in openshell-core::forward that wraps every
interpolated value in shell_escape. Use it at all four ProxyCommand
sites: openshell-cli/src/ssh.rs (ssh_session_config) and the three
TUI sites in openshell-tui/src/lib.rs (shell connect, sandbox exec,
port-forward reconciliation).
- Add validate_ssh_session_response in openshell-core::forward, called
at each site immediately after response.into_inner(). Enforces a
conservative charset per field (sandbox_id, token, gateway_host,
gateway_scheme, gateway_port, connect_path, host_key_fingerprint) so
malformed responses fail loudly at the gRPC trust boundary before
shell escaping is attempted. Belt-and-suspenders with the escape.
- Harden render_ssh_config so `gateway` and `name` are shell-escaped
even though validate_gateway_name currently gates `gateway`.
- Document the charset contract on CreateSshSessionResponse in
proto/openshell.proto: servers must uphold these rules and clients
reject responses that violate them.
- Add regression unit tests covering adversarial values in every field
and a build_proxy_command test asserting no shell metacharacter
remains outside single-quoted regions.
Tracks OS-100.
`git describe --tags --long --match "v*"` matches the `vm-dev` tag
alongside release tags. When `vm-dev` sits on or past the latest
release tag in the commit graph (currently the case on main), git
picks it, and the resulting `vm-dev-N-gSHA` string strips the leading
`v` down to `m-dev-N-gSHA`. With `commits == 0` that's returned
verbatim, producing a binary that reports itself as `m-dev`:
$ openshell --version
openshell m-dev
Confirmed on v0.0.28 (#832) and reproduced on v0.0.29 as well.
Restrict the glob to numeric release tags (`v[0-9]*`). All release
tags follow `v\d+\.\d+\.\d+`, so this loses no valid version — it
only filters out `vm-dev`, `vm-prod`, and any similar non-release
tags that share the `v` prefix.
Verified locally on this repo's current HEAD:
# before
$ git describe --tags --long --match 'v*'
vm-dev-0-g355d845
# after
$ git describe --tags --long --match 'v[0-9]*'
v0.0.29-2-g355d845
Closes#832
Signed-off-by: mjamiv <michael.commack@gmail.com>
* fix(sandbox): harden seccomp denylist, SSRF protection, and inference policy enforcement
- Remove seccomp skip in NetworkMode::Allow so baseline syscall
restrictions apply regardless of network mode
- Block cross-process manipulation syscalls (process_vm_writev,
pidfd_open, pidfd_getfd, pidfd_send_signal) symmetric with existing
ptrace and process_vm_readv blocks
- Block clone/clone3 with CLONE_NEWUSER flag, new mount API syscalls
(fsopen, fsconfig, fsmount, fspick, move_mount, open_tree), and
namespace manipulation (setns, umount2, pivot_root)
- Block userfaultfd and perf_event_open consistent with Docker default
seccomp profile
- Deny and close keep-alive inference connections after a non-inference
request instead of silently continuing the loop
- Add CGNAT (100.64.0.0/10), benchmarking (198.18.0.0/15), and other
special-use IP ranges to SSRF protection in both proxy and
mechanistic mapper
* fix(sandbox): install clone3 seccomp filter before main filter
* test(ocsf): fix shorthand firewall engine expectation
Policies with allowed_ips entries targeting loopback, link-local, or
unspecified ranges now fail at connection time instead of being silently
blocked at runtime. The shorthand log format for DENIED events includes
a [reason:...] suffix so operators can distinguish 'allowlist miss' from
'structurally un-allowable'. The mechanistic mapper skips proposals for
always-blocked destinations, preventing the infinite TUI notification
loop. The gateway validates proposed rules on approval as defense-in-depth.
- Extract shared IP helpers (is_always_blocked_ip, is_always_blocked_net,
is_internal_ip) to openshell_core::net
- Reject always-blocked entries in parse_allowed_ips with hard error
- Skip implicit allowed_ips synthesis for always-blocked literal IP hosts
- Add status_detail to HttpActivityBuilder for denial reason propagation
- Enrich NET and HTTP shorthand with [reason:...] for DENIED events
- Add engine: tag to HTTP shorthand (consistency with NET shorthand)
- Filter always-blocked proposals in mechanistic mapper generate_proposals
- Add validate_rule_not_always_blocked server-side defense-in-depth
- Update architecture docs, published docs, and E2E test assertions
Extract shared resolve_community_image() into openshell-core so both
CLI and TUI expand bare sandbox names (e.g. "base") to full registry
references. Previously the TUI passed the bare name directly, causing
ImagePullBackOff in Kubernetes.
Closes#786
* feat(sandbox): integrate OCSF structured logging for all sandbox events
WIP: Replace ad-hoc tracing calls with OCSF event builders across all
sandbox subsystems (network, SSH, process, filesystem, config, lifecycle).
- Register ocsf_logging_enabled setting (defaults false)
- Replace stdout/file fmt layers with OcsfShorthandLayer
- Add conditional OcsfJsonlLayer for /var/log/openshell-ocsf.log
- Update LogPushLayer to extract OCSF shorthand for gRPC push
- Migrate ~106 log sites to OCSF builders (NetworkActivity, HttpActivity,
SshActivity, ProcessActivity, DetectionFinding, ConfigStateChange,
AppLifecycle)
- Add openshell-ocsf to all Docker build contexts
* fix(scripts): attach provider to all smoke test phases to avoid rate limits
GitHub's unauthenticated API rate limit (60/hour) causes flaky 403s for
Phases 1, 2, and 4. Fix by attaching the provider to all sandboxes and
upgrading the Phase 1 policy to L7 so credential injection works.
Phase 4 (tls:skip) cannot inject credentials by design, so relax the
assertion to accept either 200 or 403 from upstream -- both prove the
proxy forwarded the request.
* fix(ocsf): remove timestamp from shorthand format to avoid double-timestamp
The display layer (gateway logs, TUI, sandbox logs CLI) already prepends
a timestamp. Having one in the shorthand output too produces redundant
double-timestamps like:
15:49:11 sandbox INFO 15:49:11.649 I NET:OPEN ALLOWED ...
Now the shorthand is just the severity + structured content:
15:49:11 sandbox INFO I NET:OPEN ALLOWED ...
* refactor(ocsf): replace single-char severity with bracketed labels
Replace cryptic single-character severity codes (I/L/M/H/C/F) with
readable bracketed labels: [LOW], [MED], [HIGH], [CRIT], [FATAL].
Informational severity (the happy-path default) is omitted entirely to
keep normal log output clean and avoid redundancy with the tracing-level
INFO that the display layer already provides.
Before: sandbox INFO I NET:OPEN ALLOWED ...
After: sandbox INFO NET:OPEN ALLOWED ...
Before: sandbox INFO M NET:OPEN DENIED ...
After: sandbox INFO [MED] NET:OPEN DENIED ...
* feat(sandbox): use OCSF level label for structured events in log push
Set the level field to 'OCSF' instead of 'INFO' for OCSF events in the
gRPC log push. This visually distinguishes structured OCSF events from
plain tracing output in the TUI and CLI sandbox logs:
sandbox OCSF NET:OPEN [INFO] ALLOWED python3(42) -> api.example.com:443
sandbox OCSF NET:OPEN [MED] DENIED python3(42) -> blocked.com:443
sandbox INFO Fetching sandbox policy via gRPC
* fix(sandbox): convert new Landlock path-skip warning to OCSF
PR #677 added a warn!() for inaccessible Landlock paths in best-effort
mode. Convert to ConfigStateChangeBuilder with degraded state so it
flows through the OCSF shorthand format consistently.
* fix(sandbox): use rolling appender for OCSF JSONL file
Match the main openshell.log rotation mechanics (daily, 3 files max)
instead of a single unbounded append-only file. Prevents disk exhaustion
when ocsf_logging_enabled is left on in long-running sandboxes.
* fix(sandbox): address reviewer warnings for OCSF integration
W1: Remove redundant 'OCSF' prefix from shorthand file layer — the
class name (NET:OPEN, HTTP:GET) already identifies structured events
and the LogPushLayer separately sets the level field.
W2: Log a debug message when OCSF_CTX.set() is called a second time
instead of silently discarding via let _.
W3: Document the boundary between OCSF-migrated events and intentionally
plain tracing calls (DEBUG/TRACE, transient, internal plumbing).
W4: Migrate remaining iptables LOG rule failure warnings in netns.rs
(IPv4 TCP/UDP, IPv6 TCP/UDP) to ConfigStateChangeBuilder for
consistency with the IPv4 bypass rule failure already migrated.
W5: Migrate malformed inference request warn to NetworkActivity with
ActivityId::Refuse and SeverityId::Medium.
W6: Use Medium severity for L7 deny decisions (both CONNECT tunnel and
FORWARD proxy paths) to match the CONNECT deny severity pattern.
Allows and audits remain Informational.
* refactor(sandbox): rename ocsf_logging_enabled to ocsf_json_enabled
The shorthand logs are already OCSF-structured events. The setting
specifically controls the JSONL file export, so the name should reflect
that: ocsf_json_enabled.
* fix(ocsf): add timestamps to shorthand file layer output
The OcsfShorthandLayer writes directly to the log file with no outer
display layer to supply timestamps. Add a UTC timestamp prefix to every
line so the file output matches what tracing::fmt used to provide.
Before: CONFIG:VALIDATED [INFO] Validated 'sandbox' user exists in image
After: 2026-04-01T15:49:11.649Z CONFIG:VALIDATED [INFO] Validated ...
* fix(docker): touch openshell-ocsf source to invalidate cargo cache
The supervisor-workspace stage touches sandbox and core sources to force
recompilation over the rust-deps dummy stubs, but openshell-ocsf was
missing. This caused the Docker cargo cache to use stale ocsf objects
from the deps stage, preventing changes to the ocsf crate (like the
timestamp fix) from appearing in the final binary.
Also adds a shorthand layer test verifying timestamp output, and drafts
the observability docs section.
* fix(ocsf): add OCSF level prefix to file layer shorthand output
Without a level prefix, OCSF events in the log file have no visual
anchor at the position where standard tracing lines show INFO/WARN.
This makes scanning the file harder since the eye has nothing consistent
to lock onto after the timestamp.
Before: 2026-04-01T04:04:13.065Z CONFIG:DISCOVERY [INFO] ...
After: 2026-04-01T04:04:13.065Z OCSF CONFIG:DISCOVERY [INFO] ...
* fix(ocsf): clean up shorthand formatting for listen and SSH events
- Fix double space in NET:LISTEN, SSH:LISTEN, and other events where
action is empty (e.g., 'NET:LISTEN [INFO] 10.200.0.1' -> 'NET:LISTEN [INFO] 10.200.0.1')
- Add listen address to SSH:LISTEN event (was empty)
- Downgrade SSH handshake intermediate steps (reading preface, verifying)
from OCSF events to debug!() traces. Only the final verdict
(accepted/denied) is an OCSF event now, reducing noise from 3 events
to 1 per SSH connection.
- Apply same spacing fix to HTTP shorthand for consistency.
* docs(observability): update examples with OCSF prefix and formatting fixes
Align doc examples with the deployed output:
- Add OCSF level prefix to all shorthand examples in the log file
- Show mixed OCSF + standard tracing in the file format section
- Update listen events (no double space, SSH includes address)
- Show one SSH:OPEN per connection instead of three
- Update grep patterns to use 'OCSF NET:' etc.
* docs(agents): add OCSF logging guidance to AGENTS.md
Add a Sandbox Logging (OCSF) section to AGENTS.md so agents have
in-context guidance for deciding whether new log emissions should use
OCSF structured logging or plain tracing. Covers event class selection,
severity guidelines, builder API usage, dual-emit pattern for security
findings, and the no-secrets rule.
Also adds openshell-ocsf to the Architecture Overview table.
* fix: remove workflow files accidentally included during rebase
These files were already merged to main in separate PRs. They got
pulled into our branch during rebase conflict resolution for the
deleted docs-preview-pr.yml file.
* docs(observability): use sandbox connect instead of raw SSH
Users access sandboxes via 'openshell sandbox connect', not direct SSH.
* fix(docs): correct settings CLI syntax in OCSF JSON export page
The settings CLI requires --key and --value named flags, not positional
arguments. Also fix the per-sandbox form: the sandbox name is a
positional argument, not a --sandbox flag.
* fix(e2e): update log assertions for OCSF shorthand format
The E2E tests asserted on the old tracing::fmt key=value format
(action=allow, l7_decision=audit, FORWARD, L7_REQUEST, always-blocked).
Update to match the new OCSF shorthand (ALLOWED/DENIED, HTTP:, NET:,
engine:ssrf, policy:).
* feat(sandbox): convert WebSocket upgrade log calls to OCSF
PR #718 added two log calls for WebSocket upgrade handling:
- 101 Switching Protocols info → NetworkActivity with Upgrade activity.
This is a significant state change (L7 enforcement drops to raw relay).
- Unsolicited 101 without client Upgrade header → DetectionFinding with
High severity. A non-compliant upstream sending 101 without a client
Upgrade request could be attempting to bypass L7 inspection.
* feat(sandbox): inject host gateway hostAliases into sandbox pods
Sandbox pods running in the k3s cluster cannot resolve host.docker.internal
by default, preventing them from reaching services on the Docker host.
Detect the host gateway IP (default route) in the cluster entrypoint,
thread it through the Helm chart to the gateway server, and inject
hostAliases entries (host.docker.internal, host.openshell.internal)
into every sandbox pod spec. The injection is conditional -- when the
IP is empty (non-Docker deployments), no hostAliases are added.
---------
Co-authored-by: Piotr Mlocek <pmlocek@nvidia.com>
Compute version strings from git describe in openshell-core's build.rs
using the guess-next-dev scheme (e.g. 0.0.4-dev.6+g2bf9969). All binary
crates and the TUI splash screen now use the shared openshell_core::VERSION
constant instead of CARGO_PKG_VERSION.
In Docker/CI builds where .git is absent, falls back to CARGO_PKG_VERSION
which is already set correctly by the sed-patch pipeline. Also adds
OPENSHELL_CARGO_VERSION support to the cluster image and fast-deploy
supervisor builds for parity with the gateway.
Drew Newberry