* feat(providers): support SPIFFE-backed token grants
Add provider profile token_grant metadata and expand endpoint-specific
dynamic credentials so sandbox supervisors can request SPIFFE JWT-SVIDs,
exchange them with an OAuth-style token endpoint, cache returned access
tokens, and inject bearer tokens into matching HTTP requests.
Wire Kubernetes and Helm deployments to mount the provider SPIFFE Workload
API socket into sandbox pods for token grant exchange.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(examples): add SPIFFE token grant demo
Add a reusable alpha/beta demo that deploys a SPIFFE-verifying token issuer
and protected services, imports a token-grant provider profile, creates a
sandbox, and verifies endpoint-specific bearer tokens.
The script leaves Kubernetes workloads in place, deletes sandboxes through
openshell unless KEEP_SANDBOX=1, and prints protected service logs as proof
of life.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden SPIFFE token grants
* fix(providers): harden dynamic token grants
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(providers): harden token grant handling
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Gordon Sim <gsim@redhat.com>
generate_security_notes() flagged a destination as internal/private using naive
string prefixes. host.starts_with("172.") matched all of 172.0.0.0/8 even
though only 172.16.0.0/12 is private (RFC 1918), so the advisory flagged public
addresses (172.15/172.32) and missed CGNAT (100.64.0.0/10), IPv6 ULA, and the
other reserved ranges; it also flagged non-IP hostnames that merely start with
10./172.. The two copies of the check (openshell-sandbox mechanistic_mapper.rs
and openshell-server grpc/policy.rs) now parse the host as an IP and defer to
openshell-core net::is_internal_ip, the same RFC-accurate, unit-tested classifier
the runtime SSRF enforcement path uses, so the proposed-rule advisory matches
actual enforcement.
Per review, is_internal_v4 now uses the stable Ipv4Addr predicates as far as
they go: is_documentation() (RFC 5737) replaces the two manual TEST-NET-2/3
checks and also closes a gap, since 192.0.2.0/24 (TEST-NET-1) was previously
unflagged, and is_broadcast() (RFC 919) is added. Manual checks remain only for
the ranges without a stable predicate: CGNAT 100.64.0.0/10, IETF protocol
assignments 192.0.0.0/24, and benchmarking 198.18.0.0/15.
Adds regression coverage: both note suites now assert hostname-prefix false
positives (10.example.com, 172.example.com) are not flagged and IPv6 ULA
(fd00::1) is flagged; net.rs covers the newly classified TEST-NET-1 and
broadcast addresses.
Closes#1777
Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
Co-authored-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
Move duplicate in-container path constants from the Docker, Podman,
Kubernetes, and VM driver crates into openshell-core::driver_utils:
- TLS_CA_MOUNT_PATH, TLS_CERT_MOUNT_PATH, TLS_KEY_MOUNT_PATH,
SANDBOX_TOKEN_MOUNT_PATH — identical string literals that were
copy-pasted between openshell-driver-docker and
openshell-driver-podman.
- SUPERVISOR_CONTAINER_DIR (/opt/openshell/bin) and
SUPERVISOR_CONTAINER_BINARY (/opt/openshell/bin/openshell-sandbox) —
replaces three diverging local constants across docker, kubernetes,
podman, and vm-rootfs that referred to the same paths under different
names with different string values (the k8s constant was the directory;
the docker/vm constant was the full binary path).
Each driver crate now assigns its local constant from the canonical core
value, keeping existing names visible to tests without any call-site
churn. The openshell-core constants carry doc comments explaining when
to use the dir vs. the full binary path.
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
* feat(telemetry): add build-time option to compile out telemetry
Gate anonymous telemetry emission behind a default-on `telemetry` Cargo
feature in openshell-core. The data model (enums, validation, emit_*/enabled*
signatures) stays always-compiled, while the endpoint, HTTP client, queue, and
emission code are feature-gated. With the feature off, enabled() returns false
and emit_* are no-ops, so dependent crates compile unchanged and no telemetry
endpoint, HTTP client, or emission code is included in the binary.
chrono and reqwest become optional dependencies of openshell-core, dropped from
its dependency graph when telemetry is disabled.
Thread the switch through the workspace: every crate depends on openshell-core
with default-features = false, and the default-on `telemetry` passthrough lives
on the binary crates that emit or collect telemetry (openshell-server,
openshell-sandbox, openshell-driver-vm). In-process drivers inherit it via
resolver v2 feature unification.
Build a telemetry-free binary with, e.g.:
cargo build --release -p openshell-server --no-default-features
The runtime OPENSHELL_TELEMETRY_ENABLED switch is unchanged for default builds.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* ci(telemetry): guard that telemetry can be compiled out
Add tasks/scripts/verify-telemetry-compiled-out.sh, which inspects a built
binary for telemetry markers (the telemetry endpoint host and client ID) that
exist only when emission code is compiled in. The rust:verify:telemetry-off
mise task builds the gateway with default features (positive control: markers
must be present, so the absent checks can never be silently vacuous) and with
--no-default-features (markers must be absent), and checks the
--no-default-features sandbox binary as well.
Wire the task into the Rust branch-checks job so a regression that reintroduces
telemetry code into a --no-default-features build fails CI.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
---------
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* feat(cli): add --env flag to sandbox create and exec
Add `--env KEY=VALUE` (repeatable) to `openshell sandbox create` and
`openshell sandbox exec`, wiring user-specified environment variables
into the existing `SandboxSpec.environment` and
`ExecSandboxRequest.environment` protobuf fields which were previously
only accessible via the SDK/gRPC API.
Fix a bug where user-specified environment variables set via
`SandboxSpec.environment` were silently dropped inside sandbox SSH
sessions. The SSH handler's `apply_child_env()` calls `env_clear()` to
strip supervisor-internal secrets before spawning user shells, but this
also wiped container-level env vars including user-specified ones. This
was introduced in 5fd4885a (2026-02-26, "feat(sandbox): VS Code
Remote-SSH support with platform detection fix") which added
`env_clear()` to fix a VS Code platform detection timeout caused by
leaked supervisor env vars.
The fix introduces an `OPENSHELL_USER_ENVIRONMENT` sidecar env var:
compute drivers (Docker, Podman, Kubernetes, VM) serialize the
user-specified environment as JSON into this variable at container
creation time, the sandbox supervisor deserializes it at startup, and
the SSH handler re-injects the values into child processes after
`env_clear()`. User-specified vars are applied at lowest priority so
proxy, TLS, and provider credentials still take precedence.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* fix(sandbox): address PR review feedback for env var injection
- VM driver: restructure build_guest_environment to apply user env
before required driver vars, preventing user-specified values from
overriding OPENSHELL_ENDPOINT, PATH, and other critical vars.
Matches the pattern already used by Docker/Podman/Kubernetes.
- All drivers: insert OPENSHELL_USER_ENVIRONMENT sidecar after
environment.extend(user_env) so a user-supplied key of the same
name cannot clobber the JSON payload the supervisor parses.
- Podman driver: fix template/spec env precedence — template env is
now applied first with spec overwriting, matching the other drivers
(spec is user-specified and should win over image defaults).
- CLI: add parse_env_pairs() that rejects OPENSHELL_* prefixed keys
with a clear error message. Both sandbox create and exec call sites
updated to use it.
- SSH handler: filter out OPENSHELL_* keys from user_environment
before injecting into child processes.
- Docs: add "Set Environment Variables" section to manage-sandboxes.mdx
documenting --env for both create and exec, per-command override
semantics, and the OPENSHELL_* reservation.
- Tests: add sandbox_create_env_rejects_reserved_prefix test.
- Fix rustfmt formatting in lib.rs, container.rs, driver.rs.
* fix(server): reject OPENSHELL_* env keys at API boundary
Add server-side validation to reject environment variable keys starting
with OPENSHELL_ in SandboxSpec.environment, SandboxTemplate.environment,
and ExecSandboxRequest.environment. This closes the gap where SDK/gRPC
callers (bypassing the CLI) could inject reserved keys that affect
supervisor-internal state.
The CLI already rejects these keys via parse_env_pairs, but the API
boundary is the authoritative enforcement point since all clients
(CLI, SDK, direct gRPC) pass through it.
* fix(server): validate env key names and control chars at API boundary
Add POSIX env key format validation (^[A-Za-z_][A-Za-z0-9_]*$) and
control character rejection for SandboxSpec.environment,
SandboxTemplate.environment, and ExecSandboxRequest.environment at the
gRPC validation layer. This catches invalid keys like "1BAD" or
"BAD-NAME" and values containing newlines at the API boundary rather
than letting them surface as driver-specific container/pod failures.
Also add matching CLI-side validation in parse_env_pairs so users get
immediate feedback. Update docs to say "OpenShell rejects" rather than
"the CLI rejects" since enforcement is now at both boundaries.
* fix(server): allow exec helper keys and add env size limit
Split exec env validation so OPENSHELL_PYFUNC_B64 (used by the Python
SDK's exec_python()) passes through while runtime-clobberable keys
like OPENSHELL_SANDBOX_ID are still rejected. Create-time validation
remains strict — all OPENSHELL_* keys are blocked.
Add a 256 KiB total serialized-size limit on create-time environment
maps. The drivers serialize the full map into a single
OPENSHELL_USER_ENVIRONMENT env var; capping the input at the API
boundary prevents driver/runtime-specific startup failures from
oversized env blocks.
Add template environment control-character test and exec helper key
acceptance test.
---------
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* feat(inference): allow local embeddings route
Route OpenAI-compatible embeddings through the local inference proxy so
sandboxed vector workloads reach a configured provider via the same
route classification and auth path that chat, completion, and model
discovery already use.
- Add openai_embeddings to the OpenAI-compatible protocol set so
providers (openai, nvidia) advertise embeddings routing.
- Classify POST /v1/embeddings as the openai_embeddings protocol in the
sandbox L7 patterns.
- Serve embeddings buffered with an accurate Content-Length, since the
response is a single JSON object rather than an SSE token stream. The
streaming path appends an SSE error frame on a size-cap or idle-timeout
truncation, which would corrupt a one-object body the client parses
whole. protocol_returns_buffered_body() selects the path.
- Probe an embeddings-only backend against /v1/embeddings during
validation, after the chat and completion protocols so a multi-protocol
route still prefers those.
- Extract two shared helpers. http_status_text() backs both response
formatters and adds 401/422/429/503 for embeddings passthrough and
router error mapping; write_inference_router_error() backs the streaming
and buffered routing paths.
- Return an OpenAI-shaped embeddings body from the mock route.
Tests cover profile lookup, L7 pattern detection, the mock body, and
buffered Content-Length framing with no chunked transfer-encoding and no
SSE error frame.
Signed-off-by: Shiju <shiju@nvidia.com>
* fix(inference): cap buffered inference response body
The buffered proxy path read the whole upstream response into memory with
no size bound. The route timeout bounds elapsed time but not memory, so a
misbehaving or oversized upstream could force unbounded allocation in the
sandbox proxy. The streaming path already caps each response at 32 MiB;
the buffered path did not.
Cap the buffered read at the same 32 MiB. An advertised over-cap body is
rejected from its Content-Length before any bytes are read, and chunks
accumulate under the same bound so a chunked or mislabeled body cannot
slip past. An over-cap response fails as an upstream protocol error,
surfaced as HTTP 502 at the proxy boundary, and is never partially
returned.
Tests
- cargo test -p openshell-router \
proxy_to_backend_rejects_over_cap_response_body
Signed-off-by: Shiju <shiju@nvidia.com>
* fix(inference): validate embeddings models against all advertised protocols
A managed route resolves to its provider profile's full protocol set, so
an embeddings model such as text-embedding-3-small lists
openai_chat_completions alongside openai_embeddings. Route verification
probed only the first writable protocol and stopped on its failure. It
sent a chat probe with the embedding model, the provider rejected it as
wrong-shape, and the route failed validation before the embeddings probe
ran. Embeddings-only configs could not be verified.
Try the advertised protocols in preference order. A request-shape
rejection (HTTP 400, 404, 405, 422) falls through to the next protocol,
so an embeddings model validates against /v1/embeddings even when the
chat probe rejects it. Credential, rate-limit, connectivity, and
upstream-health failures stay terminal and stop validation at the first
probe, so a bad key or a down backend is reported as itself rather than
masked by a later probe.
validation_probe becomes validation_probes, which returns the ordered
list, and the per-probe fallback retry (max_completion_tokens versus
max_tokens) moves into a shared helper.
Tests
- cargo test -p openshell-router \
verify_embeddings_model_falls_through_chat_probe
- cargo test -p openshell-router verify_stops_on_credentials_failure
Signed-off-by: Shiju <shiju@nvidia.com>
* fix(inference): serve model discovery responses buffered
GET /v1/models returns a single JSON model list, the same response shape
as embeddings. The sandbox inference proxy was routing it through the SSE
streaming path. A streaming size-cap or idle-timeout truncation appends
an SSE error frame to the body, which corrupts a payload the client
parses as one JSON object.
Make response framing a property of the protocol. A new ResponseFraming
field on InferenceApiPattern is set once per pattern in default_patterns.
model_discovery and openai_embeddings are now Buffered, while chat
completions, completions, responses, and Anthropic messages stay
Streaming. The proxy dispatch gates on pattern.is_buffered(), which
replaces the stringly-typed protocol_returns_buffered_body predicate so
the streaming-versus-buffered decision lives in one place and cannot
drift across the sites that read it.
Model discovery now flows through the same buffered path as embeddings,
framed with an accurate Content-Length and bounded by the buffered-read
size cap that path already enforces.
Tests
- cargo test -p openshell-sandbox protocol_framing_classification
- cargo test -p openshell-sandbox \
inference_model_discovery_served_buffered_with_content_length
Signed-off-by: Shiju <shiju@nvidia.com>
* docs(inference): document embeddings route in supported patterns
Signed-off-by: Shiju <shiju@nvidia.com>
---------
Signed-off-by: Shiju <shiju@nvidia.com>
Auto-detection previously treated Podman as available only when the podman CLI
was visible on PATH. However, package manager services can run with a
restricted PATH, which lets Docker be selected even when a Podman API socket is
reachable. Additionally, podman may symlink /var/run/docker.sock to podman's
machine unix socket, which would be incorrectly detected as Docker. Worse
still: the podman machine may not even be running.
This replaces the Podman binary check with a functional HTTP probe against the
standard Podman socket paths. The probe requires /_ping to answer with a
Libpod-Api-Version header before treating the socket as Podman, which lets the
gateway select the embedded Podman driver only when the API is usable.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* feat(providers): add Google Vertex AI provider
Adds Vertex AI provider profiles, routing, credential refresh plumbing, CLI support, docs, and regression coverage. Keeps the related NETLINK_ROUTE seccomp allowance needed by Vertex client tooling that calls getifaddrs.
* docs: add Vertex AI sandbox usage for Claude Code and OpenCode
Cover the full end-to-end setup for running Claude Code and OpenCode
inside an OpenShell sandbox via inference.local with a Vertex AI backend:
- google-vertex-ai.mdx: add 'Use from a Sandbox' section with tabbed
examples for Claude Code (--bare flag, no /v1 suffix) and OpenCode
(/v1 suffix required). Add providers_v2_enabled prerequisite and
--no-verify note for global region. Document policy proposals table
covering metadata.google.internal (always blocked), downloads.claude.ai,
and storage.googleapis.com.
- inference-routing.mdx: expand 'Use the Local Endpoint' section with
tabbed examples for Claude Code, OpenCode, Python OpenAI SDK, and
Python Anthropic SDK. Add notes explaining the /v1 path suffix
difference between clients.
- supported-agents.mdx: update Claude Code and OpenCode rows to mention
inference.local support and correct base URL requirements.
* fix: address vertex review findings
* test(sandbox): retry on spurious Ok in fork-exec ambiguity test
On arm64 under heavy CI load, the /proc fd scan in
find_socket_inode_owners can transiently miss the parent process's
socket fd entry, returning only the child as an owner. This causes
resolve_process_identity to return Ok (single owner, no ambiguity
check fires) instead of the expected ambiguous-ownership Err.
Extend the retry loop to also handle unexpected Ok results, mirroring
the existing retry for transient Err results. 10 retries at 50ms gives
a 500ms settling window, which is sufficient for procfs to stabilize
on loaded arm64 runners.
* fix: address vertex review regressions
* docs(router): clarify stream_response semantics for Vertex rawPredict routing
Document the three call sites of prepare_backend_request and their
stream_response values in a caller table:
- send_backend_request: false → :rawPredict (unary endpoint)
- send_backend_request_streaming: true → :streamRawPredict
- verify_backend_endpoint: explicitly false to probe the unary endpoint
Cross-reference the table from build_provider_url and
is_vertex_anthropic_rawpredict_route so the stream_response=true guard
in the suffix upgrade branch is understood in full context.
Also note that is_vertex_anthropic_rawpredict_route is a structural
predicate (model_in_path + anthropic_messages + :rawPredict suffix),
not a named-provider check, so any future provider with the same route
shape inherits the transforms automatically.
* fix(config): reject unknown fields in nested gateway config tables
The gateway TOML loader silently ignored keys placed under the wrong
table header. PR #1661 fixed one instance of this (supervisor_image
under [openshell.gateway.gateway_jwt]) but the root cause remained: the
nested gateway config tables did not deny unknown fields, so a misplaced
key was accepted and dropped instead of erroring.
Concretely, tasks/scripts/gateway.sh emitted `sandbox_namespace` right
after the [openshell.gateway.gateway_jwt] heredoc, so it landed inside
the gateway_jwt table rather than [openshell.gateway]. The k8s driver
already receives the namespace via [openshell.drivers.kubernetes], so
the stray line was dead config that parsed without complaint.
Changes:
- Add #[serde(deny_unknown_fields)] to the nested gateway config tables
that are part of the config-file parse tree: TlsConfig, OidcConfig,
MtlsAuthConfig, GatewayAuthConfig, GatewayJwtConfig.
- Remove the misplaced sandbox_namespace line from gateway.sh.
- Drop the unused Serialize/Deserialize derives from Config and
ServiceRoutingConfig (see below).
- Add a regression test asserting a key under the wrong nested table is
rejected.
Move three duplicated definitions into openshell-core so every
consumer has a single canonical source:
- format_bytes: identical 14-line function existed in docker,
kubernetes, and vm drivers. Moved to openshell-core::progress
where all three already imported from.
- DEFAULT_SANDBOX_PIDS_LIMIT: i64 constant (2048) duplicated in
docker driver and podman config. Moved to openshell-core::config
alongside other shared defaults. Podman re-exports it for
internal call-site compatibility.
- current_time_ms: secrets.rs in openshell-sandbox reimplemented
the same logic as openshell-core::time::now_ms(). Remove the
local copy and call now_ms() directly via the existing dep.
* refactor(proto): move phase and current_policy_version into SandboxStatus
Move phase and current_policy_version from SandboxSpec into
SandboxStatus to correctly model mutable runtime state. Update all
callers in the gateway server, TUI, and Python SDK to read and write
these fields through SandboxStatus accessors.
Signed-off-by: Derek Carr <decarr@redhat.com>
* fix(server): preserve sandbox status on statusless driver updates
When a driver update arrives without a status payload (e.g. before
Kubernetes populates the status subresource), preserve the stored
phase, conditions, and current policy version instead of resetting
them. Adds a regression test covering the edge case.
Signed-off-by: Derek Carr <decarr@redhat.com>
---------
Signed-off-by: Derek Carr <decarr@redhat.com>
* feat(server): per-handler gRPC auth annotations
Move scope, role, and auth-mode metadata to the handler definition site
via #[rpc_authz] + #[rpc_auth] proc macros. The previously hand-maintained
SCOPED_METHODS, ADMIN_METHODS, UNAUTHENTICATED_METHODS, and
ALLOWED_SANDBOX_METHODS tables are now generated from per-method
annotations on the tonic service impls, with canonical gRPC paths
derived from the service name and method name.
Adds a new openshell-server-macros proc-macro crate, an aggregator in
auth/method_authz.rs, and an exhaustiveness test that decodes the
protobuf FileDescriptorSet (now emitted by openshell-core/build.rs) and
verifies every RPC has an annotation.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* refactor(server): rename `sandbox-secret` auth mode to `sandbox`
PR #1404 replaced the shared sandbox secret with per-sandbox
gateway-minted JWTs. A handler marked `sandbox` now authenticates as a
specific `Principal::Sandbox`, not as a holder of a shared credential.
Rename `auth = "sandbox-secret"` to `auth = "sandbox"` and
`AuthMode::SandboxSecret` to `AuthMode::Sandbox` so the name matches
the post-#1404 identity model.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* fix(server): enforce per-handler AuthMode at the router
Addresses review feedback on the per-handler auth-annotation work.
- Router-level enforcement of #[rpc_auth] auth mode (HIGH). The previous
router only checked is_sandbox_callable() for Principal::Sandbox; user
principals still flowed into AuthzPolicy::check() and bypassed the
per-handler declaration. A user with `openshell:all` could therefore
reach `sandbox`-only handlers like GetSandboxProviderEnvironment,
ReportPolicyStatus, PushSandboxLogs, and SubmitPolicyAnalysis even
though their annotations said sandbox-only. Adds an
is_user_callable() predicate and rejects User principals at the
router for `sandbox` / `unauthenticated` methods.
- Proc macro now errors on duplicate keys in #[rpc_auth(...)] (LOW). A
second `auth`, `scope`, or `role` previously silently overwrote the
first value; now it fails to compile.
- Regression tests: a unit test for is_user_callable() and a router
test that proves a user with admin role + openshell:all cannot reach
the nine sandbox-only handlers.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* docs(server): finish renaming sandbox-secret to sandbox in method_authz doc comments
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* refactor(server-macros): drop standalone `rpc_auth` stub
The stub was a safety net that fired only when a method had
`#[rpc_auth(...)]` without an enclosing `#[rpc_authz]`. Triggering it
required `rpc_auth` to be imported, which is why both call sites carried
`#[allow(unused_imports)] use openshell_server_macros::{rpc_auth, rpc_authz};`.
Drop the stub and the unused-import workaround. A missing `#[rpc_authz]`
now surfaces as rustc's standard "cannot find attribute `rpc_auth` in
this scope" — clear enough, and one fewer import + lint exception.
Addresses review comment on PR #1596.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* refactor(server-macros): emit fixed `AUTH_METADATA` const per service
The previous trait-derived const name turned `OpenShell` into
`OPEN_SHELL_AUTH_METADATA`, splitting the project name across an
underscore. Each impl already lives in its own module
(`crate::grpc::`, `crate::inference::`), so the module path is enough
to disambiguate between services — a fixed `AUTH_METADATA` name reads
more naturally.
Aggregator in `auth/method_authz.rs` now references
`crate::grpc::AUTH_METADATA` and `crate::inference::AUTH_METADATA`
directly.
Addresses review comment on PR #1596.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* docs(server-macros): fix typo in AUTH_METADATA_CONST doc comment
OpenShell is one word; reference name in the doc should be
OPENSHELL_AUTH_METADATA, not OPEN_SHELL_AUTH_METADATA.
Addresses review nit on PR #1596.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
---------
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Extract repeated patterns into shared helpers:
- Add impl_builder_setters! macro to openshell-ocsf/builders that
generates the identical severity(), status(), and message() setter
methods present on all 7 OCSF event builders
- Add SandboxContext::apply_common_fields() to consolidate the
four-line build() finalization (set_status, set_message, set_device,
set_container) repeated in every builder
- Add driver_utils::sandbox_token_path() to centralize the XDG state
path construction for sandbox JWT files used by both the Docker and
Podman drivers
- Add driver_utils::build_capabilities_response() to eliminate the
identical GetCapabilitiesResponse struct literal repeated across the
Docker, Podman, and Kubernetes compute drivers
Remove ~280 lines of duplicated code across 30 files in 5 areas:
- centered_rect: consolidate 5 identical TUI layout helpers into a
single pub fn in openshell-tui/src/ui/mod.rs
- server test helpers: replace ~100 inline Store::connect() calls
with local test_store() helpers; deduplicate test_server_state()
in grpc/service.rs to use the shared test_support version
- rogue PKI: extract 20-line rogue CA+client cert generation block
(duplicated in two integration tests) into generate_rogue_pki()
in tests/common/mod.rs
- provider tests: replace 8 identical 28-line test modules with a
single macro_rules! test_discovers_env_credential! invocation
- label constants: centralize openshell.ai/ container label keys
in openshell-core::driver_utils; update Docker and Kubernetes
drivers to import from there instead of redefining them locally
Move default_sandbox_image() and SUPERVISOR_IMAGE_BINARY_PATH into
openshell-core so all compute drivers share a single authoritative
definition. Eliminates four identical copies of default_sandbox_image()
and two identical copies of SUPERVISOR_IMAGE_BINARY_PATH.
Implement ProviderPlugin for ProviderDiscoverySpec to remove the
repeated three-method delegation boilerplate from eight simple
provider modules. Each now declares only a SPEC constant; the blanket
impl handles id(), discover_existing(), and credential_env_vars().
Add resource_version-based optimistic concurrency control to the
persistence layer. Every write now requires an explicit WriteCondition
(MustCreate or MatchResourceVersion), enforced at compile time by
gating unconditional put/put_message behind #[cfg(test)].
- Add WriteCondition enum and put_if for conditional writes
- Add update_message_cas for atomic read-modify-write operations
- Add list_messages/list_messages_with_selector helpers that hydrate
resource_version from authoritative DB rows
- Convert all production write paths to CAS-aware methods
- Gate put/put_message behind #[cfg(test)] to prevent non-CAS writes
- Use structured PersistenceError::UniqueViolation matching instead of
string matching for duplicate detection
- Hydrate resource_version from WriteResult directly on creates,
eliminating unnecessary read-after-write round trips
Signed-off-by: Derek Carr <decarr@redhat.com>
* feat(gateway): add TOML configuration file (RFC 0003)
Introduces an opt-in --config / OPENSHELL_GATEWAY_CONFIG flag that loads a
TOML file with gateway-wide settings and per-driver tables. Source
precedence is CLI > env > file > built-in default, implemented via clap's
ValueSource so existing flags and env vars keep their priority.
Driver crates (kubernetes, docker, podman, vm) now derive Deserialize on
their config structs. SupervisorSideloadMethod gains Deserialize with
kebab-case rename. A per-driver inheritance allowlist on the loader side
overlays [openshell.gateway] shared defaults (default_image,
supervisor_image, image_pull_policy, guest_tls_*, ssh_handshake_skew_secs,
client_tls_secret_name, host_gateway_ip, enable_user_namespaces) onto
each [openshell.drivers.<name>] table before deserialization.
The Helm chart renders a new gateway-config ConfigMap and mounts it at
/etc/openshell/gateway.toml. The migrated OPENSHELL_* env entries are
dropped from the StatefulSet — only the Secret-backed
OPENSHELL_SSH_HANDSHAKE_SECRET remains. database_url stays on --db-url.
Adds examples/gateway/gateway.example.toml and updates architecture/gateway.md
with the source precedence and inheritance rules.
* docs(gateway): drop ssh_handshake_skew_secs and ssh_handshake_secret from examples
Both fields are scheduled for removal. Remove the example values and the
env-only note so the gateway.toml example and the architecture doc stop
recommending settings that will not exist much longer.
* docs(rfc): correct OPENSHELL_CONFIG to OPENSHELL_GATEWAY_CONFIG in RFC 0003
* docs(gateway): add per-driver TOML example configurations
Adds focused single-driver examples next to the comprehensive
gateway.example.toml: kubernetes, docker, podman, and microvm. Each one
demonstrates the realistic settings for that driver plus how shared
[openshell.gateway] defaults inherit into the driver table.
A new unit test (`checked_in_examples_parse`) loads every example through
the config_file loader so schema drift fails CI rather than silently
shipping a broken example.
* refactor(gateway): drop image_pull_policy from shared inheritance
Kubernetes and Podman use mutually-incompatible vocabularies for the same
TOML key:
- Kubernetes: `Always | IfNotPresent | Never` (free-form string passed
verbatim to the K8s API).
- Podman: `always | missing | never | newer` (strict lowercase enum
deserialised into `ImagePullPolicy`).
No value means the same thing in both drivers. Sharing the key at
`[openshell.gateway]` scope and inheriting it into every active driver's
table meant any value safe for one driver was either wrong or silently
dropped for the other (`IfNotPresent` → `ImagePullPolicy::Missing` after
`.unwrap_or_default()`). Operators run one driver per gateway, so the
"shared default" never pays for itself.
Make `image_pull_policy` driver-local:
- Remove the field from `GatewayFileSection` and from
`inheritable_keys()` for both Kubernetes and Podman.
- Drop the file→`RunArgs` merge for the gateway-scope key.
- Stop unconditionally clobbering the driver value with
`config.sandbox_image_pull_policy` in the runtime wiring — only apply
the CLI/env override when it was set (and, for Podman, only when it
parses into the lowercase enum).
- Move the key under `[openshell.drivers.kubernetes]` and
`[openshell.drivers.podman]` in every example, the RFC, the
architecture doc, and the Helm-rendered gateway ConfigMap.
The supervisor pull policy follows the same shape: it is K8s-only and
moves into `[openshell.drivers.kubernetes]` alongside `image_pull_policy`
in the Helm template.
* fix(gateway): address review feedback on TOML configuration
Resolves the P1 and P2 issues raised in PR #1317:
- Helm gateway ConfigMap moves `grpc_endpoint` under
`[openshell.drivers.kubernetes]` so the default install no longer fails
the gateway's `deny_unknown_fields` schema check.
- `kubernetes_config_from_file` and `podman_config_from_file` only let
the gateway-wide CLI/env `grpc_endpoint` overwrite the driver-table
value when it was actually supplied, preserving file-only configs.
- Kubernetes driver default `image_pull_policy` is now empty (was Podman
vocabulary "missing"), so default deployments let the Kubernetes API
apply its own policy instead of being rejected.
- New `disable_tls` gateway field plumbs `.Values.server.disableTls`
through the TOML ConfigMap instead of relying on env vars dropped from
the StatefulSet.
- StatefulSet pod template now carries a `checksum/gateway-config`
annotation so `helm upgrade` rolls pods when the ConfigMap changes.
- Auxiliary listener resolution preserves the full `SocketAddr` from
`health_bind_address` / `metrics_bind_address`, so a loopback-pinned
health port is not silently relocated onto the public bind address.
- `ssh_session_ttl_secs` from the file is now applied to `Config` (it
was previously accepted by the loader but never read).
New regression coverage: cli-level merge tests for the new fields plus
helm-unittest assertions for the ConfigMap shape, checksum annotation,
and `disable_tls` rendering.
* docs(gateway): consolidate gateway TOML examples into docs reference
Replaces the per-driver example files under examples/gateway/ with a
single published reference page at docs/reference/gateway-config.mdx
covering source precedence, layout, the full example, and the four
per-driver examples (Kubernetes, Docker, Podman, microVM). Drew flagged
during PR #1317 review that the examples belong with the user-facing
docs rather than in a sibling examples/ directory.
The cross-references in architecture/gateway.md and RFC 0003 are updated
to point at the new docs page; the round-trip test in config_file.rs is
removed (schema coverage stays on the inline parses_full_example test
and per-field merge tests — doc-snippet drift belongs in a separate
docs-lint, not in a cross-tree Rust unit test).
* refactor(core): move DEFAULT_K8S_NAMESPACE into K8s driver
The constant is Kubernetes-specific (used only by KubernetesComputeConfig's
Default impl) and does not belong in openshell-core. Relocate it to the
driver crate that owns the K8s vocabulary; openshell-core retains only
truly cross-cutting defaults.
* refactor(core): move Podman bridge default into Podman driver
DEFAULT_NETWORK_NAME is Podman vocabulary, consumed only by the Podman
driver. Also drops the unused DEFAULT_IMAGE_PULL_POLICY constant.
* docs(auth): scrub remaining SSH handshake secret references
Sweeps the trailing mentions left after the rebase: the gateway
config-file module doc, the Helm gateway-config ConfigMap header,
the gateway-config.mdx env-only note, and the RPM systemd unit
comment for init-gateway-env.sh.
* docs(gateway): clarify OPENSHELL_GRPC_ENDPOINT applies to all drivers
The previous comment implied the callback endpoint was Kubernetes-only,
but the value is propagated to every compute driver (Kubernetes, Docker,
Podman, VM) and must be reachable from wherever the sandbox runs.
* refactor(gateway): move driver options into config (#1394)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
* fix(e2e): regenerate gateway config via TOML for docker + podman harnesses
The gateway CLI flags moved into TOML config tables in 560550d2 (#1394),
which made every existing e2e/with-{docker,podman}-gateway.sh invocation
fail with "unexpected argument '--sandbox-namespace'" (and a long tail
of similar driver-specific options) before the gateway could even bind.
Replace the obsolete CLI flags with a synthesized
`[openshell.drivers.<driver>]` table written to `${STATE_DIR}/gateway.toml`
and passed via the new `--config` flag. Only the gateway-wide flags
that survived 560550d2 (bind-address, port, drivers, db-url, tls-*,
disable-tls, log-level, health-port) stay on the command line.
Both scripts get a small `toml_string` helper to properly TOML-quote
the values (the previous `%q` printf format produced bash-escape, not
TOML-escape). The Docker harness also corrects two field names that
diverged from the driver schema: `docker_network_name` →
`network_name`, and the supervisor binary/image plumbing now reads
through to `supervisor_bin` / `supervisor_image` in the same table.
The Podman harness drops `--ssh-gateway-port` (deleted in 560550d2 —
gRPC + SSH are multiplexed on the same port now) and substitutes
`network_name` + `gateway_port` for the obsolete `--sandbox-namespace`
(which the Podman driver never had as a typed field).
* fix(core): swap bind-only 0.0.0.0 SSH gateway host for cluster URL host
CLI's resolve_ssh_gateway treated 0.0.0.0 as a loopback "keep as-is"
when the cluster URL was also loopback, so the SSH proxy connected to
0.0.0.0:port. The unspecified address is never a valid connect target
and is not present in any TLS cert SAN, which produced BadCertificate
TLS handshake failures during `openshell sandbox create -- ...` in
docker/podman e2e (e.g. bypass_detection).
Resolution: when the server returns 0.0.0.0 or :: as the gateway host
and both endpoints are loopback, fall back to the cluster URL's host
(which the CLI is already using to reach the gateway, so it must
resolve and match the cert).
* fix(e2e): repair podman harness on macOS
Podman 5.x with the applehv/libkrun provider no longer creates the legacy
~/.local/share/containers/podman/machine/podman.sock symlink, and
`podman system service` is a Linux-only subcommand — the macOS client
delegates the API service to the VM. Both assumptions in the harness
were stale, so the script tried to start a temporary service that podman
rejected with "unknown flag: --time".
- Discover the macOS socket via `podman machine inspect` instead of the
hardcoded path.
- On Darwin, fail fast with a "start podman machine" message rather than
attempting the Linux-only `podman system service` fallback.
- Write socket_path into [openshell.drivers.podman] so the in-process
driver picks up the discovered socket; the driver reads TOML only
after the config refactor (560550d2), so OPENSHELL_PODMAN_SOCKET alone
was no longer enough.
* fix(server): use clone_from for TLS client CA assignment
clippy 1.95.0 rejects assigning the result of `Clone::clone()` to an
existing variable under `-D warnings` (`assigning_clones`). Switch to
`clone_from(&...)` to satisfy the lint and avoid the redundant
allocation.
---------
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
* fix(sandbox): exempt host gateway from SSRF block for rootless Podman
pasta assigns a link-local IP (169.254.x.x) as the host gateway in
rootless Podman containers, which was always blocked by SSRF guards.
Read the trusted gateway IP from /etc/hosts at proxy startup (injected
by the driver via host.openshell.internal / host.containers.internal
aliases) and exempt only that specific IP from the link-local block,
while keeping 169.254.169.254 (cloud metadata) hard-blocked regardless.
* fix(net): add is_link_local_ip() helper and harden trusted-gateway SSRF
Extract a public is_link_local_ip(IpAddr) helper in openshell-core::net
covering IPv4 169.254.0.0/16, IPv6 fe80::/10, and IPv4-mapped variants.
Refactor the inline IPv6 fe80::/10 bitmask in is_always_blocked_ip() and
is_internal_ip() to use the new helper, eliminating two copies of the
same pattern.
Harden the trusted-gateway SSRF exemption in proxy.rs: both
detect_trusted_host_gateway() and resolve_and_check_trusted_gateway()
now reject always-blocked non-link-local IPs (loopback, unspecified)
with a warning. The exemption was intended only for link-local addresses
used by rootless Podman with pasta; a /etc/hosts entry mapping
host.openshell.internal to 127.0.0.1 or 0.0.0.0 previously bypassed
the loopback/unspecified SSRF invariant.
Add 169.254. prefix to the generate_security_notes() host checks in
mechanistic_mapper and policy server, closing a gap where link-local
literal hosts were not flagged as internal/private in advisory notes.
* fix(sandbox): warn on ambiguous /etc/hosts gateway and add real function tests
When detect_trusted_host_gateway() finds more than one distinct IP for
host.openshell.internal, emit a structured warning so operators can
diagnose unexpected /etc/hosts state. The first entry still wins (and
any runtime mismatch is caught by resolve_and_check_trusted_gateway),
but the ambiguity is now observable.
Replace inline guard-logic tests with real calls to
resolve_and_check_trusted_gateway() using IP literals, which bypass DNS
and give deterministic results. This exercises the actual function
wiring rather than mirroring the predicate logic manually.
Add parse_hosts_file_for_host tests covering: single entry, same-IP
deduplication across lines, multiple distinct IPs (documents first-wins
ordering), multi-alias lines, missing alias, and comment handling.
* fix(sandbox): narrow trusted-gateway exemption to link-local IPs only
Two security fixes addressing PR review feedback:
1. Replace the compound is_always_blocked_ip && !is_link_local_ip guard
in detect_trusted_host_gateway() with a plain !is_link_local_ip check.
Private RFC 1918 addresses (e.g. Docker bridge 172.17.0.1, K8s node
192.168.x.x) are not always-blocked, so they slipped through the old
guard and received the SSRF exemption, bypassing resolve_and_reject_internal().
The exemption is only valid for the rootless Podman/pasta link-local case.
2. Normalize IPv4-mapped IPv6 in is_cloud_metadata_ip() so that
::ffff:169.254.169.254 is correctly identified as cloud metadata.
is_link_local_ip() already recognises this form as link-local; without
the normalization it would pass the metadata guard and receive the
trusted-gateway exemption, violating the 'metadata is never exempted'
invariant.
Apply the same !is_link_local_ip tightening to resolve_and_check_trusted_gateway()
as defense-in-depth, and update/add tests to cover both guards including
IPv4-mapped metadata, private IP rejection, and the Docker bridge scenario.
Move EdgeAuthInterceptor from openshell-cli to openshell-core so the TUI
can use it without a circular dependency. The TUI now checks gateway
metadata for OIDC auth mode, loads the stored bearer token, and injects
it into gRPC requests via the shared interceptor.
When switching gateways, the TUI dispatches on auth_mode to build either
an mTLS or OIDC-authenticated channel. Expired tokens surface a clear
error directing users to re-authenticate with `openshell gateway login`.
Make --tls-client-ca optional and make client certificates always
optional when a CA is configured. This decouples HTTPS encryption
from mTLS authentication, allowing mTLS and OIDC bearer tokens to
coexist as parallel authentication mechanisms.
When --tls-client-ca is provided, client certificates are validated
against the CA when presented but never required. Clients may connect
with or without a certificate — authentication is handled at the
application layer (e.g. OIDC).
Two TLS modes are now supported:
- HTTPS with optional mTLS (--tls-client-ca provided)
- HTTPS-only (--tls-client-ca omitted)
The --disable-gateway-auth flag is preserved for backward
compatibility but is now a no-op. The allow_unauthenticated field
has been removed from TlsConfig. The Helm chart conditionally
includes the client-ca volume and env var based on whether
clientCaSecretName is configured.
* refactor!(auth): drop SSH handshake secret in favor of mTLS
The OPENSHELL_SSH_HANDSHAKE_SECRET / x-sandbox-secret mechanism was
misnamed: it does not authenticate SSH (which flows over the
RelayStream gRPC RPC and is gated by mTLS plus supervisor Unix-socket
permissions). It only gated a small set of sandbox-to-gateway
control-plane RPCs, and production deployments already enforce mTLS
on that channel — so the shared secret was redundant.
Replace the secret check with an mTLS-presence marker. Sandbox-class
methods (ReportPolicyStatus, PushSandboxLogs,
GetSandboxProviderEnvironment, SubmitPolicyAnalysis, GetSandboxConfig,
GetInferenceBundle) accept callers without a Bearer token; the gRPC
mTLS handshake is the trust boundary. Dual-auth methods treat
Bearer-present as full-scope CLI access and Bearer-absent as
sandbox-restricted scope via validate_sandbox_caller_update.
Drops the secret from all drivers (K8s, Podman, VM), the sandbox gRPC
interceptor, the Helm chart (values + pre-install hook + StatefulSet
env), the RPM bootstrap script, the man pages, and the
debug-openshell-cluster skill. Also removes the never-read
ssh_handshake_skew_secs flag and config field.
BREAKING CHANGE: --ssh-handshake-secret / OPENSHELL_SSH_HANDSHAKE_SECRET
and --ssh-handshake-skew-secs / OPENSHELL_SSH_HANDSHAKE_SKEW_SECS are
removed from the gateway, sandbox, and all driver binaries. The
openshell-ssh-handshake K8s Secret is no longer managed by the chart;
operators may delete the orphan. Deployments using
--disable-gateway-auth must enforce caller authentication at the
fronting proxy, since the gateway no longer validates a per-request
secret on sandbox-class methods.
Refs OS-174.
* docs(auth): scrub residual SSH handshake secret references
Sweep across docs, e2e scripts, the Podman driver README/NETWORKING
notes, the RPM/Helm/setup guides, the gateway man page, and RFC 0003
to remove instructions and examples that still referenced
OPENSHELL_SSH_HANDSHAKE_SECRET / --ssh-handshake-secret /
ssh_handshake_skew_secs. The mechanism is gone; nothing should still
suggest setting it. Negative-assertion regression tests are kept so
the env var cannot silently be re-introduced.
* test(drivers): drop SSH handshake secret negative-assertion tests
The supporting code, env vars, CLI flags, and config plumbing are
gone — these tests asserted absence of strings that no longer have
any path to being set. Remove the guards from the Docker, Podman,
Kubernetes, and VM driver test modules.
Consolidate shared constants and helpers into openshell-core to avoid
duplicate definitions scattered across crates. Adds OPENSHELL_SANDBOX
to the sandbox_env module so all drivers and the sandbox supervisor
reference it via a typed constant rather than string literals, matching
the existing pattern for SANDBOX_ID, ENDPOINT, and the other env vars.
* docs(rfc): add agent-driven policy management
* docs(rfc): switch policy MVP to local API
* docs(rfc): clarify policy advisor skill and local logs
* feat(sandbox): add agent-driven policy proposal loop
* test(examples): add codex policy dogfood loop
* refactor(examples): make policy demo agent-agnostic
* refactor(examples): colocate policy validation harness
* docs(examples): add policy demo env sample
* docs(examples): use placeholder env example
* feat(sandbox): wire policy.local denials to OCSF JSONL log
Wires GET /v1/denials?last=N on the sandbox-local policy advisor API to read
recent OCSF JSONL events from /var/log/openshell-ocsf.YYYY-MM-DD.log, filter
to network/L7 denials (action_id=2, class_uid 4001/4002), and return a
compact summary newest-first. Default limit is 10, capped at 100. Ran inside
spawn_blocking so file I/O does not block the policy.local handler.
Other cleanup:
- POST /v1/proposals now uses the typed grpc_client wrapper instead of
raw_client, so accepted/rejected counts surface to the agent uniformly.
Wrapper return type extended to the response struct.
- Drop the 'add_rule' snake_case alias in the proposal JSON; canonical form
is camelCase 'addRule', matching the PolicyMergeOperation convention used
elsewhere.
- skills/policy_advisor.md updated to match: documents the now-real
/v1/denials?last=10 endpoint and uses 'addRule' consistently.
- skills.rs test asserts on the canonical 'addRule' phrase rather than the
removed 'PolicyMergeOperation' substring.
* feat(cli): show L7 protocol/method/path in rule get output
format_endpoint() previously rendered only host:port, dropping protocol,
access, and the L7 rules array. That made openshell rule get text output
unable to distinguish a broad L4 grant from a method/path-scoped L7 REST
rule -- exactly the distinction a developer needs at approval time.
New rendering tags each endpoint with its enforcement layer and surfaces
allow/deny rules:
bare L4: api.example:443 [L4]
L7 read-only: api.example:443 [L7 rest, access=read-only]
L7 method/path: api.example:443 [L7 rest, allow PUT /v1/foo/bar]
Pure display change: no proto, gateway, or behavior changes. Unit test
covers all three rendering cases with synthetic fixtures.
* refactor(examples): rewrite policy demo as Codex-default loop
Re-shape examples/agent-driven-policy-management/ to be a single, clean
end-to-end demonstration of the agent-driven policy loop. A Codex agent
inside an OpenShell sandbox attempts a GitHub Contents API write, hits a
structured 403 from the L7 proxy, reads the policy_advisor skill, drafts a
narrow addRule proposal via http://policy.local/v1/proposals, the host
auto-approves, the sandbox hot-reloads policy, and the agent's retry
succeeds. Whole loop runs in roughly two minutes.
Demo cleanup:
- Drop .env file ceremony. Defaults resolve from gh: owner via
'gh api user --jq .login', repo defaults to 'openshell-policy-demo',
token from gh auth token / GITHUB_TOKEN / GH_TOKEN. With gh auth login
and codex login already done, 'bash demo.sh' Just Works.
- Codex-specific. Bootstraps ~/.codex/auth.json from credentials injected
by the OpenShell provider, runs codex exec --sandbox danger-full-access
(OpenShell is the actual security boundary; bwrap nesting cannot create
user namespaces inside the sandbox container).
- Tighter narrative output: a single 'Preflight' step, a run summary banner
before launch, an inline narration of what's happening inside the sandbox
while we poll for the proposal (including the literal structured 403
body the agent acts on), and an OCSF trace at the end filtered to the
three events that tell the story (DENY, RELOAD, ALLOW).
- Replace Python heredoc templating with sed; uploads use the single-flag
pattern (--upload "${PAYLOAD_DIR}:/sandbox") with files referenced at
the basename-prefixed path that #952 / #1028 established.
- README documents the trust model honestly: structured rule is the
contract, agent rationale is a hint, prover validation badge in
progress per RFC 0001.
Move the deterministic no-LLM regression harness out of examples/ into
e2e/policy-advisor/ -- it was a parallel demo, not an example. Same loop
without the LLM, useful for iterating on the proxy and policy.local API.
* style(sandbox,cli): apply rustfmt
Whitespace-only fixups caught by mise run pre-commit. No functional change.
* perf(examples): cap Codex reasoning at 'low' in policy demo
The demo task is mechanical (one HTTP request, parse a structured 403,
post a JSON proposal, retry). Codex's default high-effort reasoning
roughly doubles the demo's wall time without improving outcomes; running
at 'low' lands the same minimal L7 grant in roughly half the time.
Override with DEMO_CODEX_REASONING=medium (or higher) to compare runs.
* fix(sandbox): harden policy.local denials endpoint
Three changes addressing review feedback before merging the agent-driven
policy management MVP:
- Distinguish "OCSF JSONL enabled, no denials" from "OCSF JSONL disabled,
nothing to read." The endpoint now returns a `log_available` flag and an
explanatory `note` when the log file is missing, so the in-sandbox agent
can give the developer an accurate hint instead of a misleading empty
list.
- Stop echoing the OCSF `message` field in the per-denial summary. The
proxy's denial messages can include the request path with query string
(e.g., `?access_token=...`); the structured `host`/`port`/`method`/
`path`/`binary` fields carry everything the agent needs to draft a
proposal, and `path` is sourced from `http_request.url.path` which
already excludes the query string.
- Cap `read_request_body` at a 15s timeout. Bounds slowloris-style stalls
from a misbehaving in-sandbox process. The proxy listener only accepts
loopback connections so practical impact is small, but this is cheap
defense-in-depth.
New tests cover the missing-log signal and the message-redaction guarantee.
* fix(examples): redact tokens in agent log tail and validate DEMO_FILE_DIR
Two small hardening passes on the policy management demo:
- `fail()` now pipes the agent log tail through a redactor that masks the
GitHub token and Codex credential triple before printing. Codex itself is
well-behaved about not echoing the token, but a misbehaving tool call
could leak it; this is a final safety net before the log hits the
developer's terminal (and any clipboard or chat history that follows).
- `validate_env` now regex-checks DEMO_FILE_DIR with the same allow-list
the other path-shaped variables use. The value is interpolated through
sed with `|` as the delimiter when rendering the agent task; rejecting
unsupported characters keeps the templating predictable and stops a
user-supplied value from breaking out into a shell context.
* refactor(sandbox): centralize policy.local routes and skill path
Addresses review feedback that the deny body's `next_steps` array and the
route table could drift apart. The route paths and skill location now live
as `pub const`s in `policy_local.rs` and feed both:
- the dispatcher in `route_request` that matches against them
- a new `agent_next_steps()` helper that builds the JSON the L7 deny body
embeds
`l7/rest.rs::deny_response_body` calls `policy_local::agent_next_steps()`
instead of inlining the array, so adding or renaming a route is a one-line
change in `policy_local.rs` and the agent contract follows automatically.
* feat(sandbox): switch /v1/denials to shorthand log pass-through
Previously /v1/denials parsed `/var/log/openshell-ocsf.*.log` (OCSF JSONL)
and returned structured per-event objects. JSONL is opt-in via
`ocsf_json_enabled`, so the endpoint returned an empty list with a "log
not enabled" hint by default — agents had to navigate a setup step before
the inspect-recent-denials guidance was useful.
Switch to reading the shorthand log at `/var/log/openshell.*.log`, which
is always-on and the same human-readable format `openshell logs` displays.
The endpoint now returns raw shorthand lines (newest first) — the agent
reads them directly, no field parsing.
Tradeoffs:
- Removes the JSONL-on-by-default debate: shorthand is already on, no
defaults change.
- Updating shorthand is a single-file change in this repo; no schema rev
needed when we want to add fields.
Implementation:
- `read_recent_denial_lines` walks shorthand log files newest-first,
filters lines with ` OCSF ` AND ` DENIED ` (the OCSF action label,
uppercase, space-bounded).
- `collect_shorthand_log_files` matches `openshell.<date>.log`; the
trailing dot in `SHORTHAND_LOG_PREFIX = "openshell."` excludes
`openshell-ocsf.<date>.log` so JSONL-on doesn't bleed into responses.
- 4096-byte cap per surfaced line as defense against pathological inputs.
- Skill doc updated to reflect that `/v1/denials` returns raw shorthand
lines, not structured fields.
Defense-in-depth on query-string secrets:
- `redact_query_strings` strips `?<query>` to `?[redacted]` from each
surfaced line. The L7 relay path emits OCSF events using
`redacted_target` (secret-placeholder redaction), but the FORWARD deny
path in `proxy.rs` populates `OcsfUrl::new("http", host, path, port)`
and `.message(...)` with the raw request path — query string included.
Stripping queries at the consumer guards `/v1/denials` regardless of
whether the upstream emit sites are tightened. The on-disk log is not
rewritten by this change; that is a separate hardening task tracked
for the FORWARD path emit sites in proxy.rs.
- `truncate_at_char_boundary` is UTF-8 safe; redaction runs before
truncation so a cut cannot slice mid-secret.
Tests:
- `recent_denials_returns_newest_first_from_shorthand_lines` covers the
happy path with mixed allowed/denied/non-OCSF lines.
- `recent_denials_skips_jsonl_log_files` confirms JSONL files don't
surface even if present.
- `recent_denials_truncates_pathological_lines` covers the cap.
- `is_ocsf_denial_line_filters_correctly` covers the line-level filter.
- `redact_query_strings_removes_query_from_url_token` and
`redact_query_strings_removes_query_in_reason_tag` cover the redaction
in both URL token and `[reason:...]` contexts.
- `truncate_at_char_boundary_does_not_panic_on_multibyte` covers the
UTF-8 safety.
* chore(sandbox): align proto inits with main's L7 GraphQL additions
Post-rebase fixups after #1083 (GraphQL L7 inspection) landed on main and
introduced new fields on the proto types this branch constructs:
- `crates/openshell-sandbox/src/l7/relay.rs`: two `deny_with_redacted_target`
call sites (REST and GraphQL relay deny paths) now pass the
`DenyResponseContext` argument that `rest::send_deny_response` expects.
Both sites pass `host`, `port`, and `binary` from the existing
`L7EvalContext`, matching the pattern used at the primary deny site.
- `crates/openshell-sandbox/src/policy_local.rs`: `L7Allow`, `L7DenyRule`,
and `NetworkEndpoint` proto initializers now populate the new GraphQL
and path-scoping fields with empty defaults. Agent-authored proposals
via `policy.local` target REST/SQL/L4 today; GraphQL operation matching
is set on the gateway side or via direct YAML, so empty defaults are
correct here.
No behavior change. `cargo test -p openshell-sandbox --lib` (650 tests) and
`cargo clippy -p openshell-sandbox --lib --tests -- -D warnings` clean.
* feat(sandbox): gate agent policy proposals behind opt-in feature flag
The agent-driven policy proposal surface delivered by this PR (skill
install, `policy.local` API, `next_steps` array on L7 deny bodies) is
now opt-in via the new `agent_policy_proposals_enabled` setting. Default
false. Same shape as `providers_v2_enabled`: registered in
`openshell-core::settings`, sandbox-level, hot-toggleable via the
existing settings poll loop.
Why: the surface is a novel agent-controlled mutation point in every
sandbox. The per-proposal developer approval gate is a correctness
control, but it doesn't address "should this sandbox have an
agent-authoring API at all" — compliance teams may want that question
closed. The flag is the second gate.
Implementation:
- New registry entry + `AGENT_POLICY_PROPOSALS_ENABLED_KEY` constant in
`openshell-core::settings`.
- `lib.rs`: process-wide `OnceLock<Arc<AtomicBool>>` mirroring the
`OCSF_CTX` pattern. `agent_proposals_enabled()` is the single read
point.
- Initial settings fetch added to `run_sandbox` so skill install honors
the flag at startup (not just on the poll loop's first tick).
- Skill install in `run_sandbox` is gated on the flag.
- `policy_local::route_request` returns `404 feature_disabled` for all
routes when the flag is off — including the otherwise-public
`current_policy` and `denials` routes. When the surface is off it's
off entirely.
- `policy_local::agent_next_steps` returns an empty array when the flag
is off so deny bodies don't advertise routes that 404.
- Poll loop updates the atomic on each tick, lazily installs the skill
on a false→true transition (no claw-back on true→false; stale skill
on disk is harmless because route + next_steps gate on the live atom).
Tests:
- Shared `test_helpers::ProposalsFlagGuard` mutex+atomic guard for the
process-wide flag, used across `policy_local::tests` and
`l7::rest::tests`.
- New: `agent_next_steps_returns_empty_when_flag_off`,
`agent_next_steps_returns_full_array_when_flag_on`,
`route_request_returns_feature_disabled_when_flag_off`.
- Updated existing tests that exercise the deny body or the route
dispatcher to set the flag on first.
- Full sandbox lib test suite: 653 pass, clippy clean.
Demo and e2e:
- `examples/agent-driven-policy-management/demo.sh` and
`e2e/policy-advisor/test.sh` now snapshot the prior global value of
the setting, set it to true before sandbox creation (so the
supervisor's initial poll picks it up), and restore on exit (delete
if previously unset, otherwise write the prior value back).
Docs:
- RFC 0001 MVP-implementation note documents the flag, default, and
intended soft-launch posture.
* test(policy-advisor): require proposal opt-in for e2e
* refactor(sandbox): group policy poll loop state
* test(e2e): isolate Kubernetes user namespace test
---------
Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
Add opt-in support for Kubernetes user namespace isolation on sandbox
pods. When enabled, container UID 0 maps to an unprivileged host UID
and capabilities become namespaced, providing defense-in-depth for the
supervisor process.
Configuration is two-layered: a cluster-wide default via
OPENSHELL_ENABLE_USER_NAMESPACES (default false) and a per-sandbox
override via the new `user_namespaces` field on SandboxTemplate.
When user namespaces are active, the pod security context is extended
with SETUID, SETGID, and DAC_READ_SEARCH capabilities to match the
bounding-set requirements inside a user namespace.
Introduces SandboxPodParams struct to replace long argument lists on
sandbox_to_k8s_spec and sandbox_template_to_k8s.
Validated end-to-end on OCP 4.22 (K8s 1.35.3, CRI-O 1.35, RHEL
CoreOS, kernel 5.14) with full SSH tunnel and non-identity UID mapping.
Walk ../../proto recursively instead of maintaining a hard-coded list. Sort paths for deterministic codegen and keep cargo:rerun-if-changed scoped to the proto tree.
Signed-off-by: ddurst <267424412+ddurst-nvidia@users.noreply.github.com>
Drew Newberry