mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-15 05:09:44 +08:00
gateway-interceptor-impl
873
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew Newberry
|
efd8bef835 |
feat(gateway): add operation interceptors
Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
dependabot[bot]
|
f4a50059e3 |
chore(deps): bump astral-sh/setup-uv from 8.0.0 to 8.2.0 (#1926)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.0.0 to 8.2.0. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](https://github.com/astral-sh/setup-uv/compare/cec208311dfd045dd5311c1add060b2062131d57...fac544c07dec837d0ccb6301d7b5580bf5edae39) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 8.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Taylor Mutch
|
ec71b1ac73 | fix(sandbox): apply initial OCSF JSON setting (#1921) | ||
|
Jeff MAURY
|
ed65bfd86d |
feat(cli): add JSON/YAML output format to provider list command (#1830)
* feat(cli): add JSON/YAML output format to provider list command Closes #1745 Add -o/--output flag support (json, yaml, table) to openshell provider list by migrating it to use the existing generic output formatter from #1750. This follows the established pattern used by sandbox list, gateway list, and provider list-profiles commands. Changes: - crates/openshell-cli/src/run.rs: Add provider_to_json() helper that maps Provider proto fields to JSON, including only credential keys (not values) for security. Add output parameter to provider_list() and insert early-return for structured formats. - crates/openshell-cli/src/main.rs: Add output field to ProviderCommands::List with conflicts_with constraints between names and output flags. Update invocation to pass output parameter. - crates/openshell-cli/tests/provider_commands_integration.rs: Update provider_list() call to include output parameter. Security: The provider_to_json() helper only exposes credential keys, never credential values, preventing CWE-532 (insertion of sensitive information into output). * refactor(cli): expose only config keys in provider list JSON/YAML output Update provider_to_json() to only include config keys (not values) in structured output, matching the security model used for credentials. This prevents potential exposure of sensitive configuration values. Changed field name from "config" to "config_keys" to clarify that only keys are included, consistent with "credential_keys" field naming. * test(cli): add comprehensive tests for provider list JSON/YAML output Add three layers of test coverage for provider list structured output: 1. Unit tests in run.rs for provider_to_json() (8 tests): - Core field validation (id, name, type) - Security: Credential keys exposed, values hidden - Security: Config keys exposed, values hidden - Metadata field inclusion/omission logic - Empty field omission behavior - Credential expiration times 2. CLI parsing tests in main.rs (4 tests): - JSON/YAML output format acceptance - Conflicts between --names and -o flags 3. Integration tests in provider_commands_integration.rs (3 tests): - JSON output execution with populated provider - YAML output execution with populated provider - Empty provider list edge case Security focus: Multiple tests verify that credential and config VALUES are never exposed in JSON/YAML output, only keys. This prevents CWE-532 (insertion of sensitive information into output). All 778 existing unit tests continue to pass, plus 15 new tests added. * feat(cli): format created_at as human-readable datetime in provider list JSON/YAML Update provider_to_json() to format created_at_ms as a human-readable datetime string using format_epoch_ms(), matching the pattern used by sandbox_to_json(). Changes: - Field name: created_at_ms → created_at - Value format: Raw milliseconds → "YYYY-MM-DD HH:MM:SS" - Implementation: Use format_epoch_ms(meta.created_at_ms) This change: - Aligns provider list output with sandbox list output for consistency - Makes JSON/YAML output more human-readable for interactive use - Follows the established pattern across all *_to_json() functions Breaking change: Field renamed and type changed (number → string). However, this feature was just added in PR #1830 and hasn't been released yet, so no users are affected. Example output: Before: {"created_at_ms": 1234567890000} After: {"created_at": "2009-02-13 23:31:30"} Tests updated: - provider_to_json_includes_metadata_fields_when_present: Assert formatted string - provider_to_json_omits_zero_metadata_fields: Update field name - provider_to_json_formats_created_at_as_human_readable: New test validating format * docs(cli): document JSON/YAML output format for provider list Add documentation for the -o/--output flag on provider list command, following the established pattern from sandbox list documentation. Changes: - Add examples showing -o json and -o yaml usage - Explain output structure (metadata, credential keys, config keys) - Emphasize security design (keys-only exposure, never values) - Cross-reference existing Credential Injection section The documentation matches the feature added in |
||
|
rhubenovandEvan Lezar
|
1ca23bc5c8 |
refactor(openshell-sandbox): Split sandbox into process and network subcrates. (#1650)
* refactor(sandbox): extract run_networking from run_sandbox
Lifts TLS state generation, network namespace setup, proxy startup,
bypass monitor spawn, and SSH-side proxy URL / netns FD computation
out of run_sandbox into a sibling async fn `run_networking` that
returns a Networking struct. The identity cache moves with it (only
consumed by the proxy). Entrypoint PID allocation moves just above
the call site so it can be passed in.
No behavior changes — same OCSF emits, same async order, same RAII
lifetimes for the proxy and bypass-monitor handles, now held by the
returned Networking value in run_sandbox's frame.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(sandbox): extract run_process and lift netns to run_sandbox
Lifts the post-networking tail of `run_sandbox` (zombie reaper, SSH
server, supervisor session, process spawn, OPA probe, policy poll loop,
denial aggregator, wait/exit) into a sibling async fn `run_process`.
Also moves network namespace creation out of `run_networking` into a new
`create_netns_for_proxy` helper invoked from `run_sandbox`, so
`run_networking` is purely the proxy component (OPA evaluation, TLS
interception, credential injection, inference routing, gRPC control
API). The netns is then borrowed into both `run_networking` and
`run_process`.
No behavior change.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* chore(workspace): scaffold openshell-supervisor-networking and openshell-supervisor-process crates
Add empty placeholder crates that subsequent commits will populate as the
sandbox decomposition proceeds. Both crates compile clean as part of the
workspace and are picked up automatically by the existing
`members = ["crates/*"]` glob.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift DenialEvent to openshell-core
The DenialEvent struct is emitted by both the proxy/L7 layer (networking-side)
and the bypass monitor (process-side), and crosses the run_networking ->
run_process API boundary. Move it to openshell-core so the eventual
supervisor-networking and supervisor-process crates can both reference it
without depending on each other. DenialAggregator and the channel/flush
helpers stay in openshell-sandbox for now.
A thin `pub use openshell_core::DenialEvent;` re-export from
denial_aggregator.rs keeps every existing `crate::denial_aggregator::DenialEvent`
call site resolving without further edits.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift normalize_path to openshell-core
Move the lexical path-normalization helper from openshell-policy to
openshell-core::paths so it can be reached from crates that sit below
openshell-policy in the dependency graph. openshell-policy keeps its
existing public API via a `pub use` re-export, so all current call sites
(e.g. openshell-sandbox/src/policy.rs) continue to resolve unchanged.
This is a prerequisite for lifting openshell-sandbox/src/policy.rs into
openshell-core: that file's `From<ProtoFilesystemPolicy>` impl calls
normalize_path, and lifting it as-is would cycle through openshell-policy.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift SandboxPolicy and friends to openshell-core
Move openshell-sandbox/src/policy.rs (SandboxPolicy, NetworkPolicy,
ProxyPolicy, FilesystemPolicy, LandlockPolicy, ProcessPolicy, NetworkMode,
LandlockCompatibility, plus their Proto* TryFrom/From impls) to
openshell-core/src/policy.rs.
Both prospective supervisor leaves (networking and process) dispatch on
SandboxPolicy. Hosting it in openshell-core lets either leaf reach for it
without depending on the other (or on the future orchestrator).
The From<ProtoFilesystemPolicy> impl now calls the in-crate
openshell_core::paths::normalize_path lifted in the previous commit, which
is what made this move cycle-free.
Update all crate::policy::* call sites in openshell-sandbox to
openshell_core::policy::*.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move child_env from openshell-sandbox
child_env (proxy_env_vars, tls_env_vars) is process-side only — consumed
by process.rs and ssh.rs. With the orchestrator staying in
openshell-sandbox (Shape A), openshell-sandbox depends on the new leaf
crates, so process-only modules can land in
openshell-supervisor-process directly.
Add openshell-supervisor-process as a path dependency of
openshell-sandbox. Update process.rs and ssh.rs to import from
openshell_supervisor_process::child_env.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move skills from openshell-sandbox
Move the static skills installer (and its embedded resource directory)
out of openshell-sandbox into openshell-supervisor-process. The module
is process-side only — invoked once during sandbox start to drop
agent skill files into the workspace — and has no cross-leaf consumers.
Adds miette as a dependency and tempfile as a dev-dependency on
openshell-supervisor-process.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move mechanistic_mapper from openshell-sandbox
Move the mechanistic mapper (HTTP method/path → operation classifier
that derives policy proposals from connection summaries) out of
openshell-sandbox into openshell-supervisor-networking. Single internal
caller (run_policy_poll_loop in lib.rs) and only depends on
openshell-core + tracing — no cross-leaf entanglement.
First population of the openshell-supervisor-networking crate; adds
openshell-core and tracing as dependencies.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift procfs to openshell-core
Move procfs (PID lookups, ancestor walking, /proc/net/tcp socket-owner
resolution, file SHA256 hashing) from openshell-sandbox into
openshell-core. The module is consumed cross-leaf — by bypass_monitor
on the process side and by identity / proxy on the networking side —
so it has to sit below both leaves.
Adds tracing, sha2, and hex as dependencies on openshell-core.
Updates the three call sites in openshell-sandbox to import from
openshell_core::procfs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move identity from openshell-sandbox
Move BinaryIdentityCache (path → SHA256 cache used to identify the
process behind an outbound connection) from openshell-sandbox into
openshell-supervisor-networking. The cache is consumed only by the
networking-side proxy and the orchestrator; with procfs already in
openshell-core there are no remaining cross-leaf dependencies.
Adds miette as a dependency and tempfile as a dev-dependency on
openshell-supervisor-networking. Adds a Default impl for
BinaryIdentityCache to satisfy clippy::new_without_default now that
the type is publicly exposed across crates.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move agent-proposals flag from openshell-sandbox
Move AGENT_PROPOSALS_ENABLED, agent_proposals_enabled(), and the
test-only ProposalsFlagGuard out of openshell-sandbox into
openshell-supervisor-process::proposals. The flag is read only by the
process-side policy_local route handler and the orchestrator; lifting
it to openshell-core would have made core carry sandbox-owned runtime
state without buying anything.
The test-only ProposalsFlagGuard is still consumed from networking-side
l7/rest tests today (until the wider Q2 OCSF/gRPC injection work lands).
Expose it via a new optional `test-helpers` feature on
openshell-supervisor-process so test crates opt in explicitly without
pulling tokio sync primitives into production builds.
openshell-sandbox keeps its existing crate-private path
(`crate::AGENT_PROPOSALS_ENABLED`, `crate::test_helpers`) via re-exports
so call sites and tests are unchanged.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift secrets to openshell-core
Move crates/openshell-sandbox/src/secrets.rs to crates/openshell-core/src/secrets.rs so both supervisor leaves can reach SecretResolver and the placeholder helpers without depending on openshell-sandbox.
Add base64 to openshell-core deps (only stdlib + base64 are used). Promote previously pub(crate) constructors and methods on SecretResolver to pub since cross-crate callers (provider_credentials, proxy/L7 tests) now name them across the crate boundary. Update import paths in proxy.rs, l7/{rest,relay,websocket}.rs, and provider_credentials.rs from crate::secrets to openshell_core::secrets.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift provider_credentials to openshell-core
Move crates/openshell-sandbox/src/provider_credentials.rs to crates/openshell-core/src/provider_credentials.rs. Both supervisor leaves now name ProviderCredentialState in their function signatures (run_networking takes &ProviderCredentialState, run_process takes ProviderCredentialState by value), and under Shape A leaves can't depend on openshell-sandbox, so the type must live in openshell-core.
The orchestrator (run_sandbox in openshell-sandbox) remains the only writer: it constructs ProviderCredentialState::from_environment and the policy poll loop calls install_environment on credential rotation. Both leaves stay pure readers via snapshot()/resolver().
Update import paths in proxy.rs, ssh.rs, and lib.rs from crate::provider_credentials to openshell_core::provider_credentials.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* style: rustfmt import ordering
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(ocsf): move SandboxContext singleton from openshell-sandbox
Move the process-wide OCSF SandboxContext OnceLock + LazyLock fallback + getter from openshell-sandbox/src/lib.rs into a new openshell-ocsf::ctx module. The type already lives in openshell-ocsf, so its singleton lives next to it.
Add openshell_ocsf::ctx::set_ctx() and openshell_ocsf::ctx::ctx(). The orchestrator (run_sandbox) now calls set_ctx during startup. Sandbox keeps a pub(crate) use openshell_ocsf::ctx::ctx as ocsf_ctx; re-export so the 138 existing crate::ocsf_ctx() call sites resolve unchanged.
When the sandbox modules themselves migrate into the leaf crates, they'll import openshell_ocsf::ctx directly and the re-export goes away.
Under Shape A neither leaf can depend on openshell-sandbox; both already depend on openshell-ocsf to construct events, so this adds no new dep edge.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift grpc_client to openshell-core
Both prospective leaves (supervisor-networking and supervisor-process)
need CachedOpenShellClient, AuthedChannel, and the connect/fetch
helpers. Under Shape A the leaves cannot depend on openshell-sandbox,
so the type has to live below them. openshell-core already pulls in
tonic and miette; this enables tonic's channel/tls features and adds
tokio as a direct dep.
Updates all crate::grpc_client::* call sites in openshell-sandbox to
openshell_core::grpc_client::*. No re-export shim — the call-site
count was small enough to update directly.
See architecture/plans/sandbox-split-design-choices.md for the full
rationale and trade-offs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move denial_aggregator from openshell-sandbox
DenialAggregator and FlushableDenialSummary belong with the proxy and L7
layer that emit denials. Moves the file into openshell-supervisor-networking;
adds tokio as a regular dep there since DenialAggregator uses
tokio::sync::mpsc.
Drops the pub use openshell_core::DenialEvent re-export inside the moved
file (no longer needed cross-crate). Updates bypass_monitor.rs, proxy.rs,
and lib.rs to import openshell_core::DenialEvent directly.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move log_push from openshell-sandbox
LogPushLayer is a process-side tracing layer that streams sandbox logs
to the gateway via gRPC. Moves into openshell-supervisor-process; adds
openshell-core, openshell-ocsf, tokio-stream, tracing, and
tracing-subscriber as direct deps there.
Updates the only external call site (openshell-sandbox/src/main.rs) to
import from openshell_supervisor_process::log_push.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move bypass_monitor from openshell-sandbox
bypass_monitor reads /dev/kmsg for nftables drop log lines and emits
denial events. Pure process-side concern, called only from
run_networking which spawns it on the netns. Moves into
openshell-supervisor-process; all deps (openshell-core, openshell-ocsf,
tokio, tracing) were already declared there.
Replaces crate::ocsf_ctx() shim calls inside the moved file with
openshell_ocsf::ctx::ctx() — first leaf-side caller to import the OCSF
context singleton directly instead of going through openshell-sandbox's
re-export.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move debug_rpc from openshell-sandbox
debug_rpc is the CLI subcommand handler that exercises authenticated
gRPC calls (issue-token, refresh-token, get-config, etc.). Pure
process-side concern, called only from openshell-sandbox/main.rs.
Adds base64, hex, serde_json, sha2, and tonic (with channel/tls
features) as direct deps on openshell-supervisor-process. Updates the
single call site in main.rs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move supervisor_session from openshell-sandbox
supervisor_session opens a bidirectional gRPC stream that lets the
gateway initiate shells inside the sandbox. Pure process-side concern,
called only from run_process. Adds uuid as a direct dep on
openshell-supervisor-process.
Replaces crate::ocsf_ctx() shim calls inside the moved file with
openshell_ocsf::ctx::ctx() — same pattern as bypass_monitor.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): lift managed_children tracker from openshell-sandbox
The MANAGED_CHILDREN set tracks PIDs of supervisor-spawned children
(entrypoint + SSH sessions) so the orchestrator's SIGCHLD reaper can
distinguish them from incidental zombies. Pure process-side concern,
moves to openshell_supervisor_process::managed_children with three
public fns: register, unregister, is_managed.
Updates lib.rs reaper, process.rs, and ssh.rs to call through the new
module path. Drops the now-unused HashSet import from lib.rs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move sandbox hardening from openshell-sandbox
Lift the process-only hardening pieces (landlock, seccomp, PreparedSandbox,
prepare/enforce, log_sandbox_readiness, top-level apply, and
apply_supervisor_startup_hardening) from crates/openshell-sandbox/src/sandbox/
to crates/openshell-supervisor-process/src/sandbox/.
Leave netns.rs and nft_ruleset.rs in openshell-sandbox for now, since both
eventual leaf crates (supervisor-networking and supervisor-process) read from
NetworkNamespace and its final home is decided when run_networking and
run_process are extracted.
Replace crate::ocsf_ctx() shims in landlock.rs and the new linux/mod.rs with
direct openshell_ocsf::ctx::ctx() calls. Update call sites in lib.rs,
process.rs, and ssh.rs to import sandbox from openshell_supervisor_process
while keeping the netns import unchanged.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift proposals flag from openshell-supervisor-process
Move proposals.rs (AGENT_PROPOSALS_ENABLED OnceLock + agent_proposals_enabled
reader + test_helpers::ProposalsFlagGuard) from openshell-supervisor-process
to openshell-core so both eventual leaf crates can read it without depending
on each other.
The flag is process-wide singleton state initialised once during sandbox
startup and read by both the policy.local route (networking-side) and the
skills installer (process-side) — same shape as openshell_ocsf::ctx.
Move the test-helpers Cargo feature alongside it: openshell-core gains the
feature, openshell-supervisor-process loses it, and openshell-sandbox's
dev-dependency now enables openshell-core/test-helpers. Update the sandbox
re-export shim to point at openshell_core::proposals.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift netns + nft_ruleset from openshell-sandbox
Move NetworkNamespace and the nft_ruleset bypass-rule generator from
crates/openshell-sandbox/src/sandbox/linux/ to crates/openshell-core/src/netns/.
Both eventual leaf crates (supervisor-networking and supervisor-process) read
from NetworkNamespace, so it must live somewhere both can depend on without
violating the Shape A no-leaf-to-leaf rule.
Replace crate::ocsf_ctx() shims in netns with direct openshell_ocsf::ctx::ctx()
calls, matching the pattern used in already-migrated process modules. Update
super::nft_ruleset references inside netns to nft_ruleset since the module
is now a sibling sub-module of netns/mod.rs.
Add openshell-ocsf and uuid as linux-only dependencies of openshell-core, and
gate pub mod netns on target_os = "linux" since the implementation uses
netlink, ip(8), and namespace fds. Delete the now-empty sandbox/{mod.rs,
linux/mod.rs} stubs and update NetworkNamespace import paths in lib.rs and
process.rs to point at openshell_core::netns.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move process.rs and ssh.rs from openshell-sandbox
Lift the entrypoint process spawn module and the embedded SSH server
module into openshell-supervisor-process. openshell-sandbox now
re-exports ProcessHandle/ProcessStatus and calls
openshell_supervisor_process::ssh::run_ssh_server directly.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move proxy, l7, opa, policy_local from openshell-sandbox
Lift the egress proxy, L7 enforcement modules, OPA engine, and policy.local
advisor API into openshell-supervisor-networking. Move accompanying data
files (sandbox-policy.rego), test fixtures (testdata/), and integration
tests (system_inference, websocket_upgrade). Sandbox lib.rs now references
these via openshell_supervisor_networking::* and ProxyHandle::start_with_bind_addr
is exposed as pub for the orchestrator call site.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(sandbox): hoist policy poll loop and denial aggregator into orchestrator
Move the symlink-resolver, policy poll loop, and denial-aggregator flush
spawns out of run_process and into run_sandbox so run_process no longer
needs OpaEngine, retained_proto, the local policy context, the sandbox
name, the gateway endpoint for telemetry, the OCSF flag, or the denial
receiver. These long-running orchestrator-owned tasks now live alongside
the other sandbox-startup wiring, matching the design log decision in
architecture/plans/sandbox-split-design-choices.md (Q5).
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move run_process from openshell-sandbox
Lift the workload supervision entry point (zombie reaper, SSH server
spawn, supervisor session, entrypoint child spawn, exit-with-timeout)
into its own module in openshell-supervisor-process. The orchestrator
in openshell-sandbox now calls openshell_supervisor_process::run::run_process
directly. With this move run_process names only types from openshell-core,
openshell-ocsf, openshell-supervisor-process itself, std, and tokio —
no openshell-supervisor-networking dependency.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move bypass_monitor from supervisor-process
Bypass detection is network-policy enforcement: it parses nftables LOG
entries from /dev/kmsg and emits OCSF NetworkActivity / DetectionFinding
events plus DenialEvents into the same channel the proxy feeds. Its
lifetime is tied to the network namespace, not to the workload child.
Moving it to openshell-supervisor-networking puts it next to the proxy
and the denial aggregator that consume its output, and unblocks moving
run_networking out of openshell-sandbox without a leaf-to-leaf dep.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move inference route helpers from openshell-sandbox
Move build_inference_context, partition_routes, bundle_to_resolved_routes,
spawn_route_refresh, the InferenceRouteSource enum, and the route refresh
interval helpers into a new openshell-supervisor-networking::inference_routes
module along with their unit tests. The orchestrator now calls into the
networking leaf for inference context construction; the leaf owns its own
route bundle resolution end-to-end.
The new module is named inference_routes to avoid colliding with the
existing l7::inference module, which handles request-time HTTP parsing
and pattern matching rather than route bundle setup.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move run_networking from openshell-sandbox
Move the Networking handle struct, run_networking, and the Linux-only
create_netns_for_proxy helper into a new openshell-supervisor-networking::run
module. The orchestrator in openshell-sandbox now invokes
openshell_supervisor_networking::run::{create_netns_for_proxy, run_networking}
and reads the Networking fields directly; the leaf owns the entire
networking-stack startup path (CA generation, proxy task, bypass monitor,
inference context, denial channel) end-to-end.
The Networking RAII handle fields (proxy, bypass_monitor) are now public
without leading underscores so the public API satisfies clippy's
pub_underscore_fields lint while still serving as drop guards held by the
orchestrator's frame.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* fix(workspace): align Cargo deps and call sites for split crates
The recent module lifts left two Linux-only gaps that the macOS host
workspace check skipped:
- openshell-core's netns module needs libc, tempfile, and nix on Linux,
but only openshell-ocsf and uuid were carried over.
- openshell-supervisor-process's seccomp/landlock modules need landlock
and seccompiler, which still lived on openshell-sandbox.
- openshell-sandbox's runtime_pid_limit branch referenced an unqualified
process:: that pointed at the old in-crate module.
Move landlock/seccompiler to supervisor-process, add the missing core
deps, qualify the call sites, and drop sandbox deps that no longer have
runtime users (landlock, seccompiler, target-gated tempfile/uuid, the
unix libc/rustix block).
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): rename openshell-supervisor-networking to openshell-supervisor-network
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): own denial-aggregator flush end-to-end
Move the denial-aggregator spawn and flush_proposals_to_gateway out of
run_sandbox and into run_networking. The networking leaf already owns
every other input (proxy + bypass_monitor as producers, denial channel,
mechanistic_mapper, denial_aggregator) and already opens its own gRPC
connections (inference_routes, policy_local) — the orchestrator was the
only piece left straddling the boundary.
Networking now drives the full path: producers -> channel -> aggregator
-> flush -> gateway. Drops denial_rx from Networking; adds sandbox_name
to run_networking so SubmitPolicyAnalysis can resolve by sandbox name
(falls back to ID when unset). Same shape as log_push in the process leaf.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): own symlink-resolution task
Move the OPA binary-symlink resolver out of run_sandbox and into
run_networking. The task probes /proc/<entrypoint_pid>/root/ until the
workload's mount namespace is accessible, then rebuilds the OPA engine
with resolved binary paths so policy rules match canonical names instead
of symlinks.
Both inputs (Arc<OpaEngine>, retained_proto) are networking-leaf concerns
and were already plumbed into run_networking; the entrypoint_pid Arc is
read lazily after the process leaf populates it. Adds retained_proto as
a parameter and spawns the resolver early in run_networking so the probe
loop starts before the proxy comes up.
Same shape as the denial-flush move: networking owns its own background
task end-to-end; the orchestrator stops hosting work that doesn't
conceptually belong to it.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move seccomp install into run_process
The supervisor seccomp prelude is part of "set up the workload-side
process tree", not part of orchestration. Move the call site from
run_sandbox into the top of run_process and drop the now-unused
re-export from openshell-sandbox::lib.
Timing is preserved: by the time the orchestrator calls run_process,
run_networking has already returned, so netns + nftables setup is
complete.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move check_runtime_pid_limit into run_process
The PID-limit precondition is process-side: it gates whether the workload
child can be spawned at all. Move the call from run_sandbox into the top
of run_process, alongside the seccomp prelude. Same shape as the seccomp
move — function already lives process-side, only the call site relocates.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move validate_sandbox_user to process crate
The sandbox-user check is a precondition for privilege-dropping the
workload child; it has no relevance to networking. Move the function
next to drop_privileges in openshell-supervisor-process::process and
call it from the top of run_process.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move prepare_filesystem to process crate
Creating and chowning read_write directories is workload-side
preparation, not orchestration. Move prepare_filesystem and its
prepare_read_write_path helper (plus tests) into
openshell-supervisor-process::process and call from run_process,
alongside validate_sandbox_user.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move startup skill install into run_process
The eager initial-settings fetch + agent skill install is process-side:
the install materializes files the workload's filesystem sees. The
orchestrator still owns the AGENT_PROPOSALS_ENABLED OnceLock init
because the policy poll loop also reads it; only the early fetch and
install hop into run_process.
Behavior unchanged. Best-effort: any RPC or install failure is logged
but does not fail startup.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): own PolicyLocalContext construction
Move the PolicyLocalContext construction from run_sandbox into
run_networking. The orchestrator was building it solely to thread it into
the networking leaf and to share it with the policy poll loop; now
run_networking builds it from inputs it already takes (retained_proto,
openshell_endpoint, sandbox_name|sandbox_id) and exposes it on the
returned Networking struct.
The orchestrator's poll loop now grabs the Arc clone from
networking.policy_local_ctx, so the orchestrator no longer imports
openshell_supervisor_network::policy_local.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* feat(supervisor): add --mode flag to gate network/process leaves
Add a --mode flag (default "network,process") that selects which
supervisor leaves run in the current process. Two new shapes are
unlocked without splitting the binary:
--mode=network # network-only sidecar
--mode=process # process-only supervisor
--mode=network,process # combined (default; current behavior)
In network-only mode the orchestrator skips run_process and waits on
SIGINT/SIGTERM before tearing down the proxy. The entrypoint PID stays
at 0 for the lifetime of the process, which silently degrades the
proxy's binary-identity TOFU and the bypass monitor's PID enrichment;
this is correct in a split-pod topology where the workload's /proc
lives in another pod.
In process-only mode run_networking is skipped entirely. SSH sessions
get no proxy URL, no netns FD, and no CA paths, matching what a
split-pod consumer would expect when network enforcement is delegated
to a sidecar.
The policy poll loop continues to run unconditionally; its OPA-reload
and policy.local hooks already gate on the resources only present when
network is enabled, and the env-refresh / proposals-toggle hooks
remain active in process mode.
Closes a step toward the RFC-0001 supervisor topology proposed in
issue #1305 by drew.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* style(supervisor-process): rustfmt long debug! line
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): pull DenialEvent down from core
DenialEvent is only emitted and consumed inside openshell-supervisor-network
(proxy, bypass monitor, denial aggregator). It never crossed the leaf
boundary, so the earlier lift to openshell-core was speculative. Move it
back into the network crate where its only callers live.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): pull procfs down from core
procfs was lifted to openshell-core under the assumption it would be
shared cross-leaf, but on the current branch all three callers
(bypass_monitor, identity, proxy) live in openshell-supervisor-network.
No file in openshell-supervisor-process imports it. Move the module to
the network crate and drop sha2/hex from openshell-core, which were
pulled in only for procfs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* style(supervisor-network): run cargo fmt
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* fix(supervisor-network): add libc dev-dependency for procfs tests
The procfs/bypass_monitor/proxy test modules use libc::{fork, exec,
fcntl, kill, waitpid} but the dep wasn't declared in this crate's
Cargo.toml. It was previously satisfied transitively when these
modules lived in openshell-core; the move left the test target
unable to resolve libc.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(sandbox): move denial aggregator to orchestrator
The denial aggregator and mechanistic mapper consume denial events
produced by the proxy and (subsequently) the bypass monitor. With both
supervisor leaves becoming pure producers of `DenialEvent`, the
consumer-side aggregation belongs in the orchestrator, not in either
leaf.
Move `denial_aggregator.rs` and `mechanistic_mapper.rs` from
`openshell-supervisor-network` to `openshell-sandbox` (the
orchestrator). The orchestrator now owns the unbounded denial channel:
it constructs `(tx, rx)`, hands `tx` to `run_networking` for the proxy
to clone, drains `rx` via the aggregator task, and runs the gateway
flush helper itself.
`run_networking`'s signature gains a `denial_tx` parameter and loses
its internal channel construction, aggregator spawn, and
`flush_proposals_to_gateway` helper. `DenialEvent` stays in
`openshell-supervisor-network` for now; a follow-up commit will lift
it to `openshell-core` alongside the bypass monitor relocation.
* refactor(supervisor-process): pull bypass monitor down from network
`bypass_monitor` is process-isolation machinery: it tails the kernel
log via `dmesg --follow`, parses nftables LOG lines emitted from the
workload's network namespace, resolves PIDs via `/proc`, and emits
OCSF events plus optional `DenialEvent`s. None of this touches the
proxy, OPA, TLS, or any other supervisor-network state — it only
shared the denial channel because both feed the same aggregator.
Move `bypass_monitor.rs` from `openshell-supervisor-network` to
`openshell-supervisor-process` (as `bypass_monitor/mod.rs`). Spawn it
in `run_process` where the netns name and entrypoint PID are already
in scope. The orchestrator hands an extra `bypass_denial_tx` clone of
the denial channel sender to `run_process` for this purpose.
Lift `DenialEvent` from `openshell-supervisor-network` to
`openshell-core`. Both supervisor leaves now produce it, so it needs
a shared location that neither leaf depends on. This reverses an
earlier commit that pulled the type into the network leaf when it was
the only producer.
Copy the minimal subset of `/proc` parsers used by `bypass_monitor`
into a private `bypass_monitor::procfs` submodule. The alternative —
extracting a shared procfs crate — is a much larger refactor that
this commit does not need; supervisor-network's `procfs.rs` continues
to serve the proxy and identity cache.
* refactor(supervisor-process): derive ssh netns fd inside run_process
The ssh_netns_fd was computed in run_networking purely to forward it
through the Networking struct and back into run_process. supervisor-network
never read it. Move the derivation to run_process where the
NetworkNamespace handle is already in scope.
* refactor(supervisor-process): derive ssh proxy url inside run_process
The ssh_proxy_url was computed in run_networking purely to forward it
through the Networking struct and back into run_process. supervisor-network
never read it. Move the derivation to run_process where the
NetworkNamespace handle and SandboxPolicy are already in scope.
After this commit the Networking struct no longer carries any SSH-shaped
fields, and supervisor-network reads only host_ip from the netns (for the
proxy bind address).
* refactor(supervisor-network): take proxy bind ip directly instead of netns
run_networking only ever read host_ip from the netns it was passed (the
SSH plumbing reads moved to run_process in earlier commits). Replace the
NetworkNamespace parameter with a plain Option<IpAddr> the orchestrator
extracts. supervisor-network's run module no longer references the netns
type for any consumer, only for create_netns_for_proxy (which still lives
in this crate; relocates next).
* refactor(supervisor-process): move netns ownership out of core
Relocates the NetworkNamespace handle, nft ruleset builder, and
create_netns_for_proxy constructor into openshell-supervisor-process.
The orchestrator (openshell-sandbox) phantom-owns the RAII handle for
the duration of run_sandbox; supervisor-network no longer references
the type at all.
Drops uuid, libc, nix, openshell-ocsf, and tempfile from core's Linux
target deps (all were exclusive to netns). tempfile becomes a Linux
runtime dep on supervisor-process for nft ruleset application.
* chore(sandbox): prune leaf-only deps from orchestrator manifest
cargo-machete flagged 26 direct dependencies that were carried over
from the pre-split monolith and are no longer used by the orchestrator
itself: regorus, russh, rcgen, tokio-rustls, ipnet, apollo-parser,
openshell-router, anyhow, base64, bytes, flate2, glob, hex, hmac, nix,
rand_core, rustls-pemfile, serde, serde_yml, sha1, sha2, thiserror,
tokio-stream, uuid, webpki-roots.
These now live (transitively) in openshell-supervisor-network and
openshell-supervisor-process where they are actually consumed.
* chore(deps): prune unused deps from supervisor crates
- Drop unused `url` from openshell-supervisor-network.
- Mark `prost` and `prost-types` as cargo-machete-ignored in
openshell-core: they have no source-level `use`, but the tonic-
generated proto code references them via `::prost::Message` etc.
- openshell-supervisor-process is already clean.
* fix(supervisor-network): wait for entrypoint PID before symlink probe
The OPA symlink-resolution task reads entrypoint_pid once at the top
of the spawned closure. Because the spawn happens before run_process
publishes the workload PID, the load returns 0, the probe path bakes
in as /proc/0/root/, and the loop exhausts its retries against a path
that does not exist on Linux. The reload never fires, so policies that
whitelist symlinked binaries (e.g. /usr/bin/python3 → python3.11) get
silent denials when the workload exec's the realpath.
Split the wait into two phases: 5s polling entrypoint_pid for a
non-zero value, then the existing 5s window probing /proc/<pid>/root/.
Distinct warn messages on each timeout so future debugging can tell
"PID never published" apart from "container fs never appeared".
* fix(sandbox): restore GPU procfs baseline (#1522)
Signed-off-by: Evan Lezar <elezar@nvidia.com>
(cherry picked from commit
|
||
|
krishicks
|
ac3bb631ac |
docs(rfc): improve template and add creation skill (#1889)
Add a create-rfc skill that directs agents through the OpenShell RFC process and template. Expand the RFC template with clearer section guidance and suggested lengths. Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
Taylor Mutch
|
62aa5e3247 |
ci(branch-checks): align Python checks with pre-commit (#1908)
Signed-off-by: Taylor Mutch <taylormutch@gmail.com> |
||
|
Evan Lezar
|
8c0153438c |
test(e2e): add GPU workload image artifacts (#1484)
* test(e2e): add GPU workload image artifacts Closes #1476 Define local GPU workload image sources for smoke-pass, smoke-fail, and cuda-basic validation, plus a mise build task that tags images with the source revision and records the latest local image refs. Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(e2e): add workload manifest build flow Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(e2e): fix gpu workload README lint Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(e2e): address gpu workload review feedback Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Piotr Mlocek
|
6c8cf38b63 |
ci(docs): add docs website automation (#1788)
* ci(docs): add docs website automation * ci(docs): harden sync-docs inputs and add script tests Route free-form workflow_dispatch inputs through env vars to avoid shell injection, switch uv install to the approved setup-uv action, and add a pytest suite for sync_docs_website.py wired into the test task. |
||
|
Mesut Oezdil
|
ec197a43ef | fix(e2e): correct return type of _stub_with_token (#1897) v0.0.63 | ||
|
Taylor Mutch
|
21ff5db950 |
ci(stale): add stale issue and PR workflow (#1890)
* ci(stale): add stale issue and PR workflow Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix: remove draft-pr from exemption --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> |
||
|
Alex Lewontin
|
fb83d1a3c8 |
feat(gateway): add system registry support and source indicators (#1625)
* feat(bootstrap): add system gateway registry for installer defaults Adds a read-only installer-seeded gateway registry that the CLI consults after per-user gateway config. The registry uses the same layout as per-user config with `active_gateway` at the root and `gateways/<name>/metadata.json` beneath it. By default the system config root is `/etc/openshell`, while `OPENSHELL_SYSTEM_GATEWAY_DIR` remains available as an override for packages that need a different location. User-managed gateways continue to shadow installer entries on name collision. Originally-authored-by: Mark Shuttleworth <mark@ubuntu.com> Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com> * feat(cli): show gateway config source in list and term Expose whether a gateway registration comes from user or system config in `openshell gateway list`, the TUI gateway pane, and list JSON output. The CLI also refuses to remove system-managed registrations and the smoke tests cover the new list output. Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com> * fix(bootstrap): preserve user shadowing on invalid metadata Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com> * fix(cli): keep system fallback out of rollback state * docs(gateway): describe system config fallback layout * test(bootstrap): cover system gateway last_sandbox persistence * test(gateway): cover system-only removal rejection * fix(bootstrap): validate gateway names before path joins --------- Signed-off-by: Alex Lewontin <alex.lewontin@canonical.com>v0.0.62 |
||
|
Derek Carr
|
e73745f10e |
feat(gateway): add reconciler lease for HA multi-replica deployments (#1577)
Introduce a database-backed lease that ensures only one gateway replica runs the watch and reconcile loops. Includes lease primitives with CAS safety, cooperative cancellation via watch channels, SQLite bypass for single-replica deployments, and integration tests covering failover, contention, and CAS chain integrity. Signed-off-by: Derek Carr <decarr@redhat.com> |
||
|
alangou
|
f33fd02f4d |
fix(server): use public tonic body type in gRPC rate limiter (#1872)
Signed-off-by: Adrien Langou <alangou@nvidia.com> |
||
|
Evan Lezar
|
58a3777d07 |
fix(drivers): filter bind-backed named volumes (#1861)
* fix(podman): filter bind-backed named volumes Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(drivers): detect recursive bind-backed volumes Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com>v0.0.61 |
||
|
alangou
|
b6c87a76ab |
feat(server): add grpc rate limiting gateway-wide (#1566)
Signed-off-by: Adrien Langou <alangou@nvidia.com> |
||
|
Evan Lezar
|
1dc59853ff |
feat(gpu): move device selection to driver config (#1815)
* refactor(kubernetes): use shared proto struct helpers Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(docker): extract sandbox template validation * feat(gpu)!: move device selection to driver config BREAKING CHANGE: The openshell sandbox create --gpu-device flag and corresponding API field were removed. Select specific GPUs through driver-specific driver_config fields instead. Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(kubernetes): reject unknown driver config fields Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Taylor Mutch
|
7dab612feb |
feat(helm): support Deployment kind in HA gateway workloads (#1867)
* feat(helm): support Deployment kind in HA gateway workloads Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(helm): handle null workload values Signed-off-by: Taylor Mutch <taylormutch@gmail.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> |
||
|
Taylor Mutch
|
4b44d629cf |
fix(helm): use stable gateway container name (#1864)
Signed-off-by: Taylor Mutch <taylormutch@gmail.com> |
||
|
Sergey
|
42e7b80943 |
feat(podman): make container health check interval configurable (#1833)
* feat(podman): make container health check interval configurable The Podman driver hardcoded a 3-second health check interval, which spawns a conmon subprocess on every tick. On systems running multiple sandboxes this creates sustained process churn and unnecessary CPU overhead. Add a `health_check_interval_secs` field to `PodmanComputeConfig` (default: 10s) and wire it into the container health check spec. Operators can tune it further via `[openshell.drivers.podman]` in gateway.toml. Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com> * docs(podman): document health_check_interval_secs config field Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com> * docs(podman): document health_check_interval_secs zero-disables behavior Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com> --------- Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com> |
||
|
Russell BryantandEvan Lezar
|
c5ce3ed623 |
AGENTS.md: Add more detailed signoff guidance (#1852)
* AGENTS.md: Add more detailed signoff guidance My coding agent started hallucinating my email address to be `rbryant@nvidia.com`. Add more guidance in `AGENTS.md` about how the `Signed-off-by` header should be added so that it uses the email address from git configuration instead of one it makes up. Signed-off-by: Russell Bryant <rbryant@redhat.com> * Update AGENTS.md Co-authored-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Russell Bryant <rbryant@redhat.com> Co-authored-by: Evan Lezar <elezar@nvidia.com> |
||
|
Evan Lezar
|
4a7f8e7c59 |
fix(ci): use existing snap gateway wrapper (#1859)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Drew Newberry
|
d8e0ef5b2c |
fix(ci): pin snap artifact downloads to valid action (#1855)
Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Drew Newberry
|
530aaf1360 |
feat(drivers): support docker and podman config mounts (#1785)
* feat(drivers): support docker and podman config mounts Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(drivers): trim mount docs Signed-off-by: Drew Newberry <anewberry@nvidia.com> * test(e2e): cover local driver volume mounts Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(podman): satisfy linux clippy lint Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(drivers): gate bind mounts behind gateway config * docs(sandbox): simplify mount examples * cleanup * test(e2e): stabilize branch checks * fix(drivers): tighten local mount validation Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Evan Lezar
|
9e805dc35d |
fix(build): use zigbuild for musl supervisor staging (#1850)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Taylor Mutch
|
702cbc4f63 |
feat(providers): support SPIFFE-backed token grants (#1784)
* feat(providers): support SPIFFE-backed token grants Add provider profile token_grant metadata and expand endpoint-specific dynamic credentials so sandbox supervisors can request SPIFFE JWT-SVIDs, exchange them with an OAuth-style token endpoint, cache returned access tokens, and inject bearer tokens into matching HTTP requests. Wire Kubernetes and Helm deployments to mount the provider SPIFFE Workload API socket into sandbox pods for token grant exchange. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Gordon Sim <gsim@redhat.com> * test(examples): add SPIFFE token grant demo Add a reusable alpha/beta demo that deploys a SPIFFE-verifying token issuer and protected services, imports a token-grant provider profile, creates a sandbox, and verifies endpoint-specific bearer tokens. The script leaves Kubernetes workloads in place, deletes sandboxes through openshell unless KEEP_SANDBOX=1, and prints protected service logs as proof of life. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(providers): harden SPIFFE token grants * fix(providers): harden dynamic token grants Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(providers): harden token grant handling --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Gordon Sim <gsim@redhat.com> |
||
|
LateNightHackathonandlatenighthackathon
|
c1d3b43ddb |
fix(policy): classify advisory private-IP notes with the canonical is_internal_ip (#1777) (#1824)
generate_security_notes() flagged a destination as internal/private using naive
string prefixes. host.starts_with("172.") matched all of 172.0.0.0/8 even
though only 172.16.0.0/12 is private (RFC 1918), so the advisory flagged public
addresses (172.15/172.32) and missed CGNAT (100.64.0.0/10), IPv6 ULA, and the
other reserved ranges; it also flagged non-IP hostnames that merely start with
10./172.. The two copies of the check (openshell-sandbox mechanistic_mapper.rs
and openshell-server grpc/policy.rs) now parse the host as an IP and defer to
openshell-core net::is_internal_ip, the same RFC-accurate, unit-tested classifier
the runtime SSRF enforcement path uses, so the proposed-rule advisory matches
actual enforcement.
Per review, is_internal_v4 now uses the stable Ipv4Addr predicates as far as
they go: is_documentation() (RFC 5737) replaces the two manual TEST-NET-2/3
checks and also closes a gap, since 192.0.2.0/24 (TEST-NET-1) was previously
unflagged, and is_broadcast() (RFC 919) is added. Manual checks remain only for
the ranges without a stable predicate: CGNAT 100.64.0.0/10, IETF protocol
assignments 192.0.0.0/24, and benchmarking 198.18.0.0/15.
Adds regression coverage: both note suites now assert hostname-prefix false
positives (10.example.com, 172.example.com) are not flagged and IPv6 ULA
(fd00::1) is flagged; net.rs covers the newly classified TEST-NET-1 and
broadcast addresses.
Closes #1777
Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
Co-authored-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
|
||
|
LateNightHackathonandlatenighthackathon
|
84c24a0e6d |
fix(ocsf): widen the shorthand [reason:] budget so denial endpoints stay readable (NVIDIA/NemoClaw#4760) (#1799)
A denied egress logs an OCSF NET:OPEN DENIED line whose [reason:] field carries the full destination endpoint and the rejecting policy name, but the shorthand formatter capped the reason at 80 bytes and appended a literal "...". A typical reason is longer than that, so the shorthand denial logs showed the endpoint cut mid-string with the port and policy name lost: NET:OPEN [MED] DENIED ... [reason:endpoint host.example:44...] Raise MAX_REASON_LEN to 256 so a full endpoint plus policy name fits, and route both reason_tag and message_tag through a shared char-boundary truncation helper. The previous `&text[..MAX]` byte slice would also panic if the cut fell inside a multibyte UTF-8 character. Per review: reason_tag now also normalizes embedded newlines to spaces before truncating (matching message_tag), so endpoint- or error-derived text cannot inject CR/LF into the single-line shorthand (CWE-117); the multibyte-boundary test now slices inside a multibyte character to actually exercise the step-back. The issue was reported on the NemoClaw tracker (NVIDIA/NemoClaw#4760), but the OCSF shorthand emitter is OpenShell code, so the fix lands here. Cross-repo references do not auto-close, so no closing keyword is used. Ref: NVIDIA/NemoClaw#4760 (https://github.com/NVIDIA/NemoClaw/issues/4760) Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com> Co-authored-by: latenighthackathon <latenighthackathon@users.noreply.github.com> |
||
|
Evan Lezar
|
27fd31c98b |
fix(cli)!: require explicit gpu sandbox flag (#1835)
BREAKING CHANGE: GPU sandbox mode is no longer inferred from sandbox image names. Users must pass --gpu to request GPU resources. Signed-off-by: Evan Lezar <elezar@nvidia.com>v0.0.60 |
||
|
Zygmunt Krynicki
|
713d46c54e |
feat(snap): expand snap description with setup instructions (#1695)
Add a description covering the snap elements, step-by-step setup for Docker and interface connections, gateway verification commands, and links to documentation and source code. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com> |
||
|
Zygmunt Krynicki
|
d2a522dd34 |
feat(snap): switch to prebuilt binaries shared with other packages (#1651)
Switch the snapcraft build from 'source: .' (which copies the full 20 GB repo including 18 GB of target/ and build artifacts into LXD) to 'source: ./snap/prebuilt' (~130 MB of only the binaries and metadata that snapcraft needs). The snap-package workflow now downloads the same cli, gateway, and supervisor binaries that other packaging jobs (deb, rpm, tarballs) use. A new 'Prepare snap build directory' step assembles snap/prebuilt/ with the 3 binaries, gateway wrapper script, LICENSE, README.md, and a precomputed version string — everything snapcraft needs without any source tree access. This addresses CI/CD Improvements (issue #1361) by reducing data transferred into the LXD sandbox by ~150x and eliminating the need for snapcraft to traverse the entire repository tree. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com> |
||
|
Taylor Mutch
|
c4ca283c1a | refactor(helm): require external postgres for ha (#1844) | ||
|
Eric Curtin
|
70acbaf407 |
refactor(driver-utils): centralize container mount path constants (#1841)
Move duplicate in-container path constants from the Docker, Podman, Kubernetes, and VM driver crates into openshell-core::driver_utils: - TLS_CA_MOUNT_PATH, TLS_CERT_MOUNT_PATH, TLS_KEY_MOUNT_PATH, SANDBOX_TOKEN_MOUNT_PATH — identical string literals that were copy-pasted between openshell-driver-docker and openshell-driver-podman. - SUPERVISOR_CONTAINER_DIR (/opt/openshell/bin) and SUPERVISOR_CONTAINER_BINARY (/opt/openshell/bin/openshell-sandbox) — replaces three diverging local constants across docker, kubernetes, podman, and vm-rootfs that referred to the same paths under different names with different string values (the k8s constant was the directory; the docker/vm constant was the full binary path). Each driver crate now assigns its local constant from the canonical core value, keeping existing names visible to tests without any call-site churn. The openshell-core constants carry doc comments explaining when to use the dir vs. the full binary path. Signed-off-by: Eric Curtin <eric.curtin@docker.com> |
||
|
Russell Bryant
|
3aba30c36f |
feat(telemetry): add build-time option to compile out telemetry (#1845)
* feat(telemetry): add build-time option to compile out telemetry Gate anonymous telemetry emission behind a default-on `telemetry` Cargo feature in openshell-core. The data model (enums, validation, emit_*/enabled* signatures) stays always-compiled, while the endpoint, HTTP client, queue, and emission code are feature-gated. With the feature off, enabled() returns false and emit_* are no-ops, so dependent crates compile unchanged and no telemetry endpoint, HTTP client, or emission code is included in the binary. chrono and reqwest become optional dependencies of openshell-core, dropped from its dependency graph when telemetry is disabled. Thread the switch through the workspace: every crate depends on openshell-core with default-features = false, and the default-on `telemetry` passthrough lives on the binary crates that emit or collect telemetry (openshell-server, openshell-sandbox, openshell-driver-vm). In-process drivers inherit it via resolver v2 feature unification. Build a telemetry-free binary with, e.g.: cargo build --release -p openshell-server --no-default-features The runtime OPENSHELL_TELEMETRY_ENABLED switch is unchanged for default builds. Signed-off-by: Russell Bryant <russell.bryant@gmail.com> * ci(telemetry): guard that telemetry can be compiled out Add tasks/scripts/verify-telemetry-compiled-out.sh, which inspects a built binary for telemetry markers (the telemetry endpoint host and client ID) that exist only when emission code is compiled in. The rust:verify:telemetry-off mise task builds the gateway with default features (positive control: markers must be present, so the absent checks can never be silently vacuous) and with --no-default-features (markers must be absent), and checks the --no-default-features sandbox binary as well. Wire the task into the Rust branch-checks job so a regression that reintroduces telemetry code into a --no-default-features build fails CI. Signed-off-by: Russell Bryant <russell.bryant@gmail.com> --------- Signed-off-by: Russell Bryant <russell.bryant@gmail.com> |
||
|
Russell Bryant
|
3a4463e36a |
fix(cli): fall back to regular upload when git filtering excludes all files (#1783)
* fix(cli): fall back to regular upload when git filtering excludes all files When uploading a directory whose contents are entirely excluded by .gitignore, git_sync_files returns an empty file list. The git-aware upload path silently accepted this and printed "Upload complete" without transferring any files. Now both call sites (direct CLI handler and sandbox_upload_plan) check for an empty file list and fall through to the regular, unfiltered upload path instead. Closes #1778 Signed-off-by: Russell Bryant <rbryant@nvidia.com> * test(e2e): add regression test for gitignored upload fallback (#1778) Reproduces the reported workflow: download a file from a sandbox into a local git repo where the target directory is gitignored, then re-upload that directory without --no-git-ignore. Before the fix the upload silently succeeded with zero files transferred. Signed-off-by: Russell Bryant <rbryant@nvidia.com> * fix(cli): warn on stderr when gitignore fallback triggers Address PR review: print a notice when .gitignore filtering excludes all files and the CLI falls back to unfiltered upload. Add a GitFilteredEmpty variant to SandboxUploadPlan so the caller can distinguish this case and emit a visible warning. Document the default gitignore filtering and fallback behavior in manage-sandboxes. * fix(cli): warn on stderr for direct upload gitignore fallback The SandboxCommands::Upload path in main.rs had the same silent fallback as sandbox_create. Restructure the git_sync_files check to emit the same .gitignore warning before falling back to unfiltered upload. Assert the warning in the E2E regression test. * docs(sandboxes): fix misleading upload fallback guidance Replace inaccurate "explicit file list" advice with actionable alternatives: --no-git-ignore, uploading outside the work tree, or git force-adding the files. * fix(cli): collapse nested ifs to satisfy clippy::collapsible_if * style(cli): apply rustfmt to main.rs --------- Signed-off-by: Russell Bryant <rbryant@nvidia.com>v0.0.59 |
||
|
Zygmunt Krynicki
|
4025894a94 | chore(snap): remove early snap packaging (#1648) | ||
|
Russell Bryant
|
7274a6bea3 |
feat(cli): add --env flag to sandbox create/exec and fix env var passthrough (#1730)
* feat(cli): add --env flag to sandbox create and exec
Add `--env KEY=VALUE` (repeatable) to `openshell sandbox create` and
`openshell sandbox exec`, wiring user-specified environment variables
into the existing `SandboxSpec.environment` and
`ExecSandboxRequest.environment` protobuf fields which were previously
only accessible via the SDK/gRPC API.
Fix a bug where user-specified environment variables set via
`SandboxSpec.environment` were silently dropped inside sandbox SSH
sessions. The SSH handler's `apply_child_env()` calls `env_clear()` to
strip supervisor-internal secrets before spawning user shells, but this
also wiped container-level env vars including user-specified ones. This
was introduced in
|
||
|
Jeff MAURYandClaude Sonnet 4.5
|
4da07f6d2e |
feat(cli): add generic output formatter to eliminate --output flag duplication (#1753)
* feat(cli): add generic output formatter to eliminate --output flag duplication Closes #1750 Creates a new output.rs module with generic helper functions for formatting CLI command output in JSON, YAML, or table formats. This eliminates ~38 lines of duplicated code across 4 commands, provides a single source of truth for output formatting, and makes it easier to add new output formats in the future. Changes: - crates/openshell-cli/src/output.rs: New module with 6 helper functions and 14 tests - crates/openshell-cli/src/lib.rs: Register output module - crates/openshell-cli/src/run.rs: Migrate 4 commands to use helpers - gateway_list(): 23 lines → 5 lines - sandbox_list(): 17 lines → 5 lines - provider_list_profiles(): 12 lines → 7 lines - provider_profile_export(): 11 lines → 8 lines - architecture/cli.md: New CLI architecture documentation - architecture/README.md: Add reference to CLI architecture doc Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: Jeff MAURY <jmaury@redhat.com> * fix: apply recommandations from gator-agent Signed-off-by: Jeff MAURY <jmaury@redhat.com> * fix: fix wrong CLI output test Signed-off-by: Jeff MAURY <jmaury@redhat.com> --------- Signed-off-by: Jeff MAURY <jmaury@redhat.com> Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
Patrick Riel
|
1f5e1234e9 | feat(vm): add vm life cycle extensions (#1583) | ||
|
Eric Curtin
|
1399f371e0 |
refactor: deduplicate OCSF builder setters and persistence helpers (#1800)
Eliminate three categories of repeated code:
1. OCSF builder setter methods
Add six macros to builders/mod.rs (impl_activity_setter!,
impl_action_disposition_setters!, impl_actor_process_setter!,
impl_dst_endpoint_setter!, impl_src_endpoint_addr_setter!,
impl_firewall_rule_setter!) that generate the identical one-liner
setters duplicated across network, http, ssh, process, finding,
and lifecycle builders. Removes ~55 hand-written method bodies.
2. Persistence protobuf decode-and-hydrate block
Extract a private decode_record<T>() helper that decodes an
ObjectRecord into a protobuf message and hydrates resource_version.
The identical three-line block was repeated in get_message,
get_message_by_name, list_messages, and list_messages_with_selector.
Each method now delegates to decode_record via .map().transpose()
or .into_iter().map().collect().
3. Store enum dispatch arms
Add a store_dispatch! macro that expands
match self { Postgres(s) => s.method(...).await, Sqlite(s) => ... }.
Applied to all 13 dispatch-only methods on Store, removing ~80
lines of mechanical match arms with zero logic.
|
||
|
Evan Lezar
|
f236279669 |
docs: document DCO commit sign-off requirement (#1811)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Evan Lezar
|
88b5f3dcec |
test(cli): avoid browser launch in auth rollback test (#1808)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Shiju
|
25abc9e3c7 |
feat(inference): allow local embeddings route (#1774)
* feat(inference): allow local embeddings route Route OpenAI-compatible embeddings through the local inference proxy so sandboxed vector workloads reach a configured provider via the same route classification and auth path that chat, completion, and model discovery already use. - Add openai_embeddings to the OpenAI-compatible protocol set so providers (openai, nvidia) advertise embeddings routing. - Classify POST /v1/embeddings as the openai_embeddings protocol in the sandbox L7 patterns. - Serve embeddings buffered with an accurate Content-Length, since the response is a single JSON object rather than an SSE token stream. The streaming path appends an SSE error frame on a size-cap or idle-timeout truncation, which would corrupt a one-object body the client parses whole. protocol_returns_buffered_body() selects the path. - Probe an embeddings-only backend against /v1/embeddings during validation, after the chat and completion protocols so a multi-protocol route still prefers those. - Extract two shared helpers. http_status_text() backs both response formatters and adds 401/422/429/503 for embeddings passthrough and router error mapping; write_inference_router_error() backs the streaming and buffered routing paths. - Return an OpenAI-shaped embeddings body from the mock route. Tests cover profile lookup, L7 pattern detection, the mock body, and buffered Content-Length framing with no chunked transfer-encoding and no SSE error frame. Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): cap buffered inference response body The buffered proxy path read the whole upstream response into memory with no size bound. The route timeout bounds elapsed time but not memory, so a misbehaving or oversized upstream could force unbounded allocation in the sandbox proxy. The streaming path already caps each response at 32 MiB; the buffered path did not. Cap the buffered read at the same 32 MiB. An advertised over-cap body is rejected from its Content-Length before any bytes are read, and chunks accumulate under the same bound so a chunked or mislabeled body cannot slip past. An over-cap response fails as an upstream protocol error, surfaced as HTTP 502 at the proxy boundary, and is never partially returned. Tests - cargo test -p openshell-router \ proxy_to_backend_rejects_over_cap_response_body Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): validate embeddings models against all advertised protocols A managed route resolves to its provider profile's full protocol set, so an embeddings model such as text-embedding-3-small lists openai_chat_completions alongside openai_embeddings. Route verification probed only the first writable protocol and stopped on its failure. It sent a chat probe with the embedding model, the provider rejected it as wrong-shape, and the route failed validation before the embeddings probe ran. Embeddings-only configs could not be verified. Try the advertised protocols in preference order. A request-shape rejection (HTTP 400, 404, 405, 422) falls through to the next protocol, so an embeddings model validates against /v1/embeddings even when the chat probe rejects it. Credential, rate-limit, connectivity, and upstream-health failures stay terminal and stop validation at the first probe, so a bad key or a down backend is reported as itself rather than masked by a later probe. validation_probe becomes validation_probes, which returns the ordered list, and the per-probe fallback retry (max_completion_tokens versus max_tokens) moves into a shared helper. Tests - cargo test -p openshell-router \ verify_embeddings_model_falls_through_chat_probe - cargo test -p openshell-router verify_stops_on_credentials_failure Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): serve model discovery responses buffered GET /v1/models returns a single JSON model list, the same response shape as embeddings. The sandbox inference proxy was routing it through the SSE streaming path. A streaming size-cap or idle-timeout truncation appends an SSE error frame to the body, which corrupts a payload the client parses as one JSON object. Make response framing a property of the protocol. A new ResponseFraming field on InferenceApiPattern is set once per pattern in default_patterns. model_discovery and openai_embeddings are now Buffered, while chat completions, completions, responses, and Anthropic messages stay Streaming. The proxy dispatch gates on pattern.is_buffered(), which replaces the stringly-typed protocol_returns_buffered_body predicate so the streaming-versus-buffered decision lives in one place and cannot drift across the sites that read it. Model discovery now flows through the same buffered path as embeddings, framed with an accurate Content-Length and bounded by the buffered-read size cap that path already enforces. Tests - cargo test -p openshell-sandbox protocol_framing_classification - cargo test -p openshell-sandbox \ inference_model_discovery_served_buffered_with_content_length Signed-off-by: Shiju <shiju@nvidia.com> * docs(inference): document embeddings route in supported patterns Signed-off-by: Shiju <shiju@nvidia.com> --------- Signed-off-by: Shiju <shiju@nvidia.com>v0.0.58 |
||
|
Eric Curtin
|
3558888091 |
refactor(tui): extract shared draw_text_field and draw_confirm_popup helpers (#1790)
The openshell-tui UI module had two private draw_text_field functions with near-identical logic in create_sandbox.rs and create_provider.rs (differing only in cursor glyph and whether a gap row is shown), and four confirm-dialog rendering blocks duplicated across global_settings.rs and sandbox_settings.rs. Extract both patterns into module-level helpers in ui/mod.rs: - draw_text_field: accepts cursor (&str) and show_gap (bool) to cover both callers; create_sandbox uses '█'/true, create_provider uses '_'/false. - draw_confirm_popup: takes the already-built lines vec, a border style, a width, and the enclosing area, and owns the Clear + Block + Paragraph rendering that was duplicated in all four confirm functions. Remove the now-redundant imports (Clear, Paragraph) from the settings files and drop the local use super::centered_popup as centered_rect aliases that are no longer needed. Signed-off-by: Eric Curtin <eric.curtin@docker.com> |
||
|
Calum Murray
|
b392b2eefe |
feat(providersv2): add path auth_style (#1622)
Signed-off-by: Calum Murray <cmurray@redhat.com> |
||
|
John T. MyersandJohn Myers
|
13e8318a0d |
fix(sandbox): stop log push after auth failure (#1787)
Co-authored-by: John Myers <johntmyers@users.noreply.github.com> |
||
|
Evan Lezar
|
c3964a6513 |
feat(kubernetes): support driver config passthrough (#1744)
* test(docker): default sandbox template fixtures Signed-off-by: Evan Lezar <elezar@nvidia.com> * feat(cli): add driver config json plumbing Signed-off-by: Evan Lezar <elezar@nvidia.com> * feat(kubernetes): tolerate invalid driver config Signed-off-by: Evan Lezar <elezar@nvidia.com> * feat(kubernetes): prototype driver config specifics Signed-off-by: Evan Lezar <elezar@nvidia.com> * docs(kubernetes): note driver-local config schema boundary Signed-off-by: Evan Lezar <elezar@nvidia.com> * feat(kubernetes): support runtime class driver config Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Eric Curtin
|
35afcf8aa1 |
refactor(tui): extract shared setting edit overlay (#1776)
The draw_edit_overlay function in sandbox_settings and global_settings was nearly identical — the only difference was which settings slice the entry was resolved from (app.sandbox_settings vs app.global_settings). Extract the common rendering logic into draw_setting_edit_overlay in ui/mod.rs, taking key/kind/edit/area/theme as parameters. Each pane now resolves its own entry and delegates to the single implementation. Net: -75 lines, +61 lines across the three files. |
||
|
Shiju
|
97986d9059 |
fix(server): resume unspecified sandbox phase (#1765)
Legacy or never-initialized sandbox rows decode to the proto-default Unspecified phase. The startup resume sweep skipped them, leaving the backend unreconciled: never resumed, never errored. Unspecified was the only non-terminal phase skipped, since unrecognized values already fall through to Unknown and get resumed. Treat Unspecified as running so the sweep reconciles it from live driver state: resume when the backend resource exists, mark Error when it is gone. Tests - cargo test -p openshell-server \ resume_persisted_sandboxes_resumes_running_phases Signed-off-by: Shiju <shiju@nvidia.com>v0.0.57 |
||
|
Taylor Mutch
|
e26a1b1ffc |
fix(kubernetes): configure sandbox apparmor profile (#1767)
Signed-off-by: Taylor Mutch <taylormutch@gmail.com> |