Show configured tool server addresses and their last observed connection
results together in sandbox status. Keep sandbox lifecycle readiness
separate so an external connection failure does not mark the sandbox unready.
Expose direct endpoint records through the CLI and SDKs, with plain-language
failure explanations and gateway acceptance times. Keep observation tracking,
runtime reporting, and gateway validation in dedicated endpoint status modules.
Preserve bounded reporting, request attribution, retry ordering, and
configuration and supervisor authority checks. Clear obsolete observations
while retaining the configured addresses, and document the distinction
between an observed HTTP response, current availability, and tool success.
Signed-off-by: Shiju <shiju@nvidia.com>
Keep watching briefly when a compute driver reports ContainerExited without a canonical main-process result. This prevents --no-keep cleanup from deleting the sandbox before the supervisor publishes the authoritative exit status.
Signed-off-by: Shiju <shiju@nvidia.com>
* refactor(inference): remove managed inference routes
Closes#3172
Remove the inference route control plane, inference.local data path, built-in router crate, and SDK surface. Move inference workloads to explicitly imported provider profiles and native endpoints, with migration cleanup and updated tests and documentation.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* fix(policy): preserve alternate upstream isolation
Restore the provider policy activation guard so legacy OpenAI and Anthropic providers configured for alternate base URLs do not grant egress to the built-in public vendor endpoints.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
---------
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* chore(deps): replace ring with AWS-LC
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(lint): address warnings after dependency upgrades
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(tls): limit provider initialization to reqwest clients
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
---------
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* feat(sandbox): support rootfs tar as --from source for VM driver
Accept flat rootfs tar archives (.tar, .tar.gz, .tgz) via the --from
flag for VM-backed gateways. The CLI detects the archive extension,
validates that the gateway uses the VM compute driver, and passes the
tar path through driver_config. The VM driver copies the tar into its
staging area and feeds it into the existing rootfs extraction and ext4
disk creation pipeline, skipping the container image pull/export steps.
Closes#2175
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): validate rootfs tar path at the VM driver boundary
The rootfs_tar_path field in driver_config was passed from the API
caller directly to tokio::fs::copy without validation. An authenticated
user bypassing the CLI could supply arbitrary host paths (e.g.
/dev/zero for disk exhaustion, or readable host files for data
exfiltration).
Introduce a trusted staging directory that the VM driver creates on
startup and advertises via GetCapabilities. The CLI now copies the tar
into the staging directory before creating the sandbox, and the driver
validates that the received path is a regular file inside the staging
root and within a configurable size limit (default 10 GiB) before any
I/O.
New VmDriverConfig options:
- rootfs_tar_staging_dir: override the staging directory
(default: <state_dir>/rootfs-tar-staging)
- rootfs_tar_max_bytes: override the size limit (default: 10 GiB)
Addresses GATOR-28b5152e-01.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): request-scoped staging, size pre-check, and cleanup for rootfs tar
Tighten the rootfs tar staging flow to address the remaining GATOR-01
obligations:
- Request-scoped staging: the CLI creates a unique per-request
subdirectory (req-<pid>) under the staging root instead of placing
files directly in the shared directory. The driver enforces that the
tar path is at depth 2 (staging_root/<subdir>/<file>), preventing
cross-request path selection.
- Size pre-check: the driver advertises rootfs_tar_max_bytes via
GetCapabilities. The CLI reads this limit and rejects oversized files
before copying, avoiding disk exhaustion in the staging directory.
- Cleanup: the driver removes the request staging subdirectory after
consuming the tar (on cache hit, copy success, or copy failure),
ensuring staged data does not persist beyond the request.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): restore rootfs-tar sandboxes from persisted image identity
On restore or restart, the one-shot staged tar archive has already been
cleaned up. Reading the persisted image identity from the sandbox state
directory and resolving the cached disk path directly avoids re-accessing
the deleted staging path.
Addresses GATOR-168b9210-01.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(cli): use random staging dirs and enforce byte limit during rootfs tar copy
Replace PID-based request staging directories with tempfile-generated
random names to prevent collisions and make paths unpredictable.
Replace bare tokio::fs::copy with a streaming copy loop that enforces
the advertised max_bytes limit during transfer, closing the TOCTOU gap
between the pre-copy size check and the actual copy.
Signed-off-by: Philippe Martin <phmartin@nvidia.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): issue rootfs tar staging slots from the gateway
A caller could name any host path in `driver_config.vm.rootfs_tar_path`,
which the privileged VM driver then read. The CLI-side locality check did
not apply to direct API requests.
The gateway now owns staging. `BeginRootfsTarStaging` allocates a
request-scoped directory under the driver-advertised staging root and
returns an opaque single-use token; `CreateSandbox` carries the token, and
the gateway substitutes the path it allocated before dispatching to the
driver. `template.driver_config.<driver>.rootfs_tar_path` is rejected
outright in request validation, so a caller-supplied path never reaches
privileged I/O.
Tokens are bound to the issuing workspace and subject, consumed once, and
expire after 30 minutes. Outstanding slots are capped per caller and
overall, so one caller can neither exhaust the staging filesystem nor
starve others. An RAII guard reclaims the directory on every failure path
after consumption, and an age-gated sweep runs at startup and on each
reconcile pass for directories whose driver died before its own cleanup.
The token is stripped from the public sandbox before persistence: the
stored copy is returned verbatim by GetSandbox, ListSandboxes and
WatchSandbox to every member of the workspace.
Also fixes two defects this exposed:
- The CLI wrote `rootfs_tar_path` at the top level of `driver_config`, but
the gateway forwards only `driver_config.<driver_name>`, silently
dropping unmatched keys. The archive never reached the VM driver, so the
documented `--from ./rootfs.tar` flow did not work at all. Config is now
nested under `vm` and deep-merged, so a caller's existing VM settings
survive instead of being clobbered by a shallow extend.
- Staging previously required `GetGatewayInfo`, which is restricted to
`platform_admin`, making the feature unusable for ordinary users on any
RBAC-enabled gateway. The new RPC matches CreateSandbox at
`sandbox:write` / `workspace_role: user`.
`compute_driver.proto` is unchanged; the gateway reads the staging root
from the capabilities it already stores.
Refs #2175
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): derive rootfs tar cache identity from archive contents
The prepared-disk cache key combined the archive's full path with an mtime
truncated to seconds, then mapped punctuation to `-`. Distinct paths such as
`/tmp/a/b.tar` and `/tmp/a-b.tar` collapsed onto the same key and reused each
other's disk, a rewrite within the same second kept stale contents, and a long
path could exceed filesystem component limits.
Identity is now a SHA-256 of the archive contents. This is also what makes the
cache work at all now that the gateway allocates a fresh staging directory per
request: a path-derived key would miss on every create.
The archive is hashed, the cache checked, and only on a miss copied — so a hit
skips writing a multi-gigabyte file. The copy is hashed as it is written and
rejected if the digest differs from the first pass, which closes the window
where the source changes during staging rather than approximating it with a
re-stat.
Refs #2175
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): decompress gzip rootfs tar archives during staging
`--from` accepts `.tar.gz` and `.tgz`, but the driver staged whatever bytes
it was given and the guest image-prep VM extracts the staged file with a
plain `tar -xpf`. Compressed sources therefore depended on the guest tar
auto-detecting gzip, and the prepared disk was sized from the compressed
length, which is far too small for the expanded rootfs.
Staging now detects gzip from the archive's magic bytes -- the driver only
ever sees a gateway-issued staging path, never the caller's file name -- and
writes an uncompressed tar. The digest still covers the source bytes, so the
"archive changed while staging" check is unaffected, and expansion is bounded
by `rootfs_tar_max_bytes` so a compression bomb cannot fill the host disk.
`extract_rootfs_archive_to` sniffs gzip as well, so the host-side extraction
path matches.
Adds unit coverage for gzip staging, bounded expansion, and gzip extraction,
plus an e2e sandbox created from a gzip-compressed export.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
---------
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@nvidia.com>
* fix(sandbox): detect an available login shell instead of hardcoding /bin/bash
The built-in default sandbox command and the interactive SSH session
hardcoded /bin/bash. Minimal images such as Alpine ship only /bin/sh
(BusyBox ash), so sandbox startup failed with an opaque "No such file or
directory (os error 2)" that never named the missing binary.
Add openshell-core::shell with shell-path constants and a runtime
detect_login_shell() that resolves a shell present in the sandbox image
($SHELL if executable, then bash, then /bin/sh). Use it for:
- the built-in default command (only the default is remapped; explicit
user commands are never rewritten), resolved in the supervisor so it
inspects the sandbox filesystem rather than the gateway's
- the SSH interactive shell
- the SHELL environment variable
Also name the program in the spawn error so a missing shell/binary is
diagnosable instead of a bare ENOENT.
Refs #3146
Signed-off-by: Akram <akram.benaissi@gmail.com>
* fix(sandbox): drop $SHELL preference in shell detection
$SHELL is image/user-controlled and the detected shell is later invoked
with `-lc`, so an executable that is not a compatible shell (e.g.
SHELL=/bin/false) would pass the executable check and then break command
execution even when /bin/sh is available. Resolve only from known shell
paths instead.
Also add a USR_BASH constant for /usr/bin/bash rather than a string
literal in SHELL_CANDIDATES.
Refs #3146
Signed-off-by: Akram <akram.benaissi@gmail.com>
* fix(sandbox): resolve the default login shell in the supervisor (empty command = default)
Addresses review: interactive PTY SSH now uses the detected shell, the shell
tests are portable across the Windows lane, and default-shell provenance is
carried without a new spec field.
An omitted command is left empty end to end and resolved in the supervisor,
which is the only place that sees the sandbox image:
- The CLI forwards the command as-is; the gateway persists an omitted command
as empty (no baked /bin/bash -l) and requests a TTY.
- MainProcessConfig carries the command empty (the transport now allows it);
the supervisor resolves a login shell that exists in the sandbox image (bash
when present, otherwise /bin/sh on minimal images like Alpine) and logs the
resolved shell.
- Interactive PTY SSH (spawn_pty_shell) uses the detected shell; a shared
build_ssh_shell_command helper covers the PTY and non-PTY paths, with a
deterministic sh-only regression test.
- Unix-only shell tests are gated with cfg(unix).
An explicit command is always run verbatim.
Refs #3146
Signed-off-by: Akram <akram.benaissi@gmail.com>
---------
Signed-off-by: Akram <akram.benaissi@gmail.com>
Move all provider-related functions, helpers, constants, and tests from
the monolithic run.rs (~2,700 lines) into a dedicated
commands/provider.rs module. This is PR3 of the CLI refactor series
(issue #2304).
The extraction follows the same pattern established in PR2 (gateway):
- Self-contained module with own imports
- pub use re-exports in run.rs so callers (main.rs) are unchanged
- Inline #[cfg(test)] mod tests
Signed-off-by: Varsha Prasad Narsing <vnarsing@nvidia.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(cli): preserve provider type on credential updates
provider_update built every UpdateProviderRequest with an empty type and
profile workspace. Policy interceptors evaluate the request before the
gateway merges it with stored state, so they cannot tell the target is a
profile-managed provider and deny credential-only updates. Creating a
provider from an interceptor-vended profile works; rotating its
credentials does not.
Look up the stored provider and forward its type and profile workspace
verbatim. The gateway treats profile_workspace as immutable, so the
stored value is passed through rather than recomputed from the request
workspace.
The read is best-effort: a caller holding provider:write without
provider:read must still be able to rotate credentials, so a denied read
falls back to the previous empty metadata rather than failing the
update. --from-existing and --from-oidc-token need the stored type, so
they surface the error instead.
This also removes a duplicate GetProvider call, since the oidc-token and
from-existing paths each fetched the same provider separately.
Signed-off-by: Nathan DeMoss <ndemoss28@gmail.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: Nathan DeMoss <ndemoss28@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(cli): require ANSI-capable terminal before colorizing
Follow-up to #3026, raised in review.
`auto` treated any terminal as styleable, so `TERM=dumb openshell ...`
still emitted escapes into a terminal that renders them literally. An
unset TERM had the same problem.
This is partly a regression that #3026 introduced. `console`, which
drives indicatif and dialoguer, already refused to colorize when TERM is
`dumb` or unset, and miette applies the same check through
supports-color. #3026 overrides both with its own switch, so it replaced
two working checks rather than only failing to add one. tracing and the
owo-colors wrapper never had detection, so those two are a gap rather
than a regression.
Add the capability check to the `auto` branch only, matching console's
unix rule: `dumb` is not capable, and an unset TERM is not capable
because nothing identifies a capable terminal. Empty is treated as unset,
which diverges from console — it reads `TERM=""` as capable since the
value is not `dumb` — because an empty value names no terminal type and
every other variable here already treats empty as unset.
Because the check sits after the explicit branches, `--color always` and
FORCE_COLOR still force styling on a dumb terminal, and `--color never`
and NO_COLOR still suppress it on a capable one. TERM is a unix signal;
Windows consoles enable virtual terminal processing and do not set it, so
the check does not apply there.
The existing pty test now pins TERM. It previously inherited the ambient
value, which would make its outcome depend on the environment now that
capability is consulted — CI runners frequently leave TERM unset.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* refactor(cli): combine stream and terminal capability checks
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* docs(cli): clarify table color behavior
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* test(cli): cover redirected status table colors
Signed-off-by: Evan Lezar <elezar@nvidia.com>
---------
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
* feat(server): add sandbox workload templates
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(go-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(rust-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(python-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(typescript-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* docs(agents): document sandbox workload templates
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): support default GPU requests in sandbox templates
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(server): cap sandbox templates per workspace
Signed-off-by: Gordon Sim <gsim@redhat.com>
* docs(architecture): document sandbox workload template boundaries
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(cli+sdk): expose sandbox workload template provenance
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): include sandbox template annotations in output
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(sandbox): add label selectors to template listing
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(e2e): cover sandbox template failure paths
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(server): preserve command and ttl when creating sandbox from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(server): add field coverage test for template merge
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): add pagination support to fake client
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): warn on env vars that looks like secrets
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(sdk-ts): propagate sandbox workspace through lifecycle calls
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(docs): update workspace management docs
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(ts-sdk): support command and tty when creating from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): support command and tty when creating from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(python-sdk): verify command and tty handling when creating from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): update docs and ClientInterface
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(server): validate sandbox create specs before I/O
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): guard empty DNS-1123 label validation
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(python-sdk): allow empty template builder mappings
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): align template GPU JSON default output
Signed-off-by: Gordon Sim <gsim@redhat.com>
---------
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): suppress ANSI color when stdout is not a terminal
The CLI colorized output unconditionally. owo-colors is built without
its `supports-colors` feature, so `.green()` and friends emitted escape
sequences regardless of destination, and nothing in the CLI read
NO_COLOR. Piping any command through grep or awk matched against bytes
the caller could not see; `forward list` was the case that surfaced it,
where an escape sits immediately before the STATUS word and defeats a
pattern anchored on whitespace.
Add a `color` module holding a process-wide switch resolved once in
run_async, before any output. Command modules import its `Colorize`
trait in place of `OwoColorize`; the method names match, so the ~450
call sites are unchanged, but each consults the switch when it renders
and delegates to owo-colors so the escape bytes stay identical. The two
traits collide by design: importing both in one module is an ambiguity
error, which keeps unconditional coloring from returning.
owo-colors is not the only styled path, and the rest each carry their
own default, so the switch governs them too:
- tracing_subscriber formats with ANSI on, does no terminal detection,
and writes to stdout, so `openshell -v ... | ...` leaked escapes the
same way the tables did. It now takes the setting via with_ansi.
- indicatif and dialoguer both style through console, which has its
own detection but cannot learn about --color. Overriding console's
global switch covers every progress bar and prompt rather than the
specific ones constructed today. Both the stdout and stderr switches
are set, since prompts and progress bars draw to stderr.
- miette renders errors through its own handler, likewise unaware of
--color, so init installs one built from the setting.
Resolution order: `--color always|never`, then NO_COLOR, then
CLICOLOR_FORCE, then whether stdout is a terminal. The decision is made
against stdout even for stderr text, since stdout is what gets parsed;
`--color always` restores styling when redirecting.
Padding is unaffected — the format spec is forwarded to the inner
Display, so widths measure text rather than text plus escapes.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* fix(cli): resolve color per output stream
Review feedback on #3026.
Resolving one answer from stdout and handing it to every library meant a
redirected stream inherited the other stream's terminal check. Running
`openshell ... 2> build.log` from a terminal wrote escapes into the log,
because console's stderr switch and miette's handler were both given
stdout's answer. That is worse than the behavior before this branch,
where both libraries did their own per-stream detection.
Resolve `auto` separately for stdout and stderr and hand each library
the answer for the stream it writes to: tracing and console's stdout
switch get stdout, miette and console's stderr switch get stderr. The
owo-colors wrapper is the exception, since its call sites are split
across println! and eprintln! and a Painted value cannot tell which
macro will consume it; it styles only when both streams accept escapes,
erring toward plain text rather than risking a redirected stream.
Existing tests could not catch this: Command::output gives both streams
pipes, so a per-stream decision and a single stdout-derived one look
identical. Add a test that puts stdout on a pty and stderr on a pipe,
which fails when stderr is handed stdout's answer.
Replace CLICOLOR_FORCE with FORCE_COLOR. The clicolors spec does not say
how to treat `0`, and implementations that special-case it disagree with
force-color.org, which keys on presence and non-emptiness only. Using
FORCE_COLOR gives it the same rule as NO_COLOR: set and non-empty means
yes, whatever the value. Nothing depended on CLICOLOR_FORCE, which was
introduced earlier on this branch and never released.
Carry the whole style in an owo_colors::Style rather than dispatching a
local enum through a six-arm match, and merge styles when chaining so
`x.green().bold()` emits one `\x1b[32;1m...\x1b[0m` instead of nesting
two wrappers. No call site styles already-styled text, so merging is
safe; the emitted bytes are shorter and there is a single reset.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
---------
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
`sandbox exec` seeded its exit code to 0 and only overwrote it on an
`Exit` event, so a stream that ended early, was cancelled, or was truncated
reported a successful run. The gateway already treats the same condition as
a relay failure (`Status::unavailable`); mirror that on the CLI side so exit
0 always means an observed exit status of 0.
Fixes#2732
Signed-off-by: rootkiller6788 <rootkiller6788@users.noreply.github.com>
Co-authored-by: rootkiller6788 <rootkiller6788@users.noreply.github.com>
Characterization tests for behavior that later work changes. No behavior
change.
- Pin the `openshell logs` output format, so a change to how that text is
produced shows up as a diff rather than passing silently.
- Cover the supervisor log push layer.
- Assert OcsfEvent survives a JSON round trip. Serialization is written
by hand and deserialization dispatches on class_uid into independent
per-variant paths, so a field can serialize correctly and still be
dropped or rejected on the way back in.
Refs #1055
Signed-off-by: Kris Hicks <khicks@nvidia.com>
parse_duration_to_ms multiplied the parsed number by its unit
multiplier without a range check. A large --since value on
`openshell logs` overflows i64: a debug build panics with "attempt to
multiply with overflow", and a release build wraps to a nonsense log
cutoff.
Use checked_mul and return a diagnostic naming the offending input.
Every in-range result is unchanged, and the largest accepted duration
still leaves the caller's `now_ms - dur_ms` well inside i64.
Signed-off-by: Nathan DeMoss <ndemoss28@gmail.com>
* feat(provider): add ability to request token exchange instead of client credentials as OAuth grant_type
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(proxy): add further tests for token exchange
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(provider): add runnable example for token exchange
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(e2e): cover Podman token exchange grants
Signed-off-by: Gordon Sim <gsim@redhat.com>
* refactor(oauth): extract duplicated functionality from server and supervisor
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(provider): evict nearest-to-expiry entry from intermediate token cache
Signed-off-by: Gordon Sim <gsim@redhat.com>
* doc(supervisor): add podman example for token exchange
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(provider): withhold token-exchange subject credentials
Signed-off-by: Gordon Sim <gsim@redhat.com>
---------
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(policy): bind reviews to applicable candidates
Build and validate the exact effective-policy candidate before approval, bind review to live policy/provider/credential inputs, and preserve inspected endpoint contracts during mechanistic expansion.
Closes#2821
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
* fix(policy): canonicalize advisor review inputs
Serialize nested protobuf maps in stable key order for proposal review tokens and effective-policy hashes. Narrow reused multi-port endpoint contracts to the denied port so advisor proposals cannot widen binary access. Add regressions for both cases.
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
* test(e2e): keep advisor sandbox running
Create the issue 2821 regression sandbox detached with a durable canonical main process so policy denial, approval, and hot-reload checks run before lifecycle exit.
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
* fix(policy): apply reviewed draft batches atomically
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
---------
Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
* feat(cli): support OIDC device authorization grant for headless login
Closes#2793
Add OAuth 2.0 Device Authorization Grant (RFC 8628) support to the OpenShell CLI's OIDC login flow. When running in a headless environment (OPENSHELL_NO_BROWSER=1) without a client secret configured, the CLI now uses the device code flow instead of the browser-based PKCE flow.
The device code flow:
- Requests a device code and user code from the IdP's device authorization endpoint
- Displays a verification URL and user code to the user
- Polls the token endpoint until the user completes authorization or the code expires
- Supports slow_down responses per RFC 8628 by increasing the polling interval
This implementation:
- Extends OidcDiscovery to optionally capture device_authorization_endpoint
- Adds oidc_device_code_flow function with proper error handling for all RFC 8628 error codes
- Updates gateway add and gateway login to dispatch to device flow when browser is suppressed
- Adds comprehensive unit tests for device flow structs and response parsing
- Updates gateway authentication documentation to describe the device code fallback
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
* fix(cli): validate OIDC device token responses
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
* fix(cli): add PKCE to OIDC device flow
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
* docs(cli): document PKCE device flow
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
---------
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
* feat(credentials): add provider credential storage drivers
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(credentials): harden credential update handling
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix(credentials): harden credential driver security, correctness, and performance
Address review findings from the credential storage drivers PR:
- Route additional_credentials through the driver on refresh to prevent
silent data loss for multi-credential providers (e.g. AWS STS)
- Clean up stored credential handles on CAS failure during refresh to
prevent orphaned secrets in external backends
- Enforce namespace validation in the Kubernetes Secrets driver to
prevent cross-namespace credential access when allow_reference_namespace
is not enabled
- Cache Vault Kubernetes auth tokens with 80% TTL to avoid re-authenticating
on every credential operation
- Parallelize resolve_credentials in all three drivers using try_join_all
for faster sandbox startup
- Add existingSecret support for the KEK Secret to fix helm template/GitOps
workflows where lookup returns empty and regenerates the key
- Document RBAC blast radius for the Kubernetes Secrets credential driver
and recommend a dedicated namespace
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): add optimistic concurrency, fix thundering herd, parallelize operations
Use resourceVersion optimistic concurrency with retry loop for K8s
Secret ownership checks to prevent TOCTOU races. Switch Vault token
cache from RwLock to Mutex with double-check pattern to prevent
thundering herd on cache miss. Parallelize credential store and delete
operations across independent keys using try_join_all.
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): handle partial failures, add delete retry, consolidate cleanup
Replace try_join_all with join_all in credential store/delete operations
to handle partial failures — successfully-stored handles are cleaned up
when another key fails. Add retry loop with conflict detection to
db-credstore delete_credential, matching the K8s driver pattern.
Consolidate 4 manual cleanup_pre_stored_provider_credentials call sites
into a single error handler using an async block. Remove inconsistent
.trim() from db-credstore validate_handle_owner.
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): fix retry loop guard and remove unprotected validation
Remove attempt-count guard from 409/Aborted match arms in retry loops
so the post-loop Status::aborted error is reachable after exhausting
retries. Previously, last-attempt conflicts fell through to the
catch-all error arm, producing misleading Status::unavailable errors.
Remove duplicate validation calls that ran after
prepare_provider_credential_update but outside the cleanup-protected
async block, which would leak pre-stored handles on failure.
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* fix(credentials): add workspace/provider UUID to credential backend paths
Include workspace and provider ID in credential backend object paths to ensure
cross-workspace uniqueness and prevent credential collision (GATOR-1806c9be-01).
- Updated credential driver proto to include workspace and provider_id fields
- Modified Vault driver to include workspace/provider_id in managed_secret_path
- Modified Kubernetes Secrets driver to include workspace/provider_id in
credential_owner_id and managed_secret_name
- Updated all credential runtime calls to pass workspace/provider_id
- Updated tests to use the new signatures
This prevents two workspaces sharing the same external credential store from
colliding on provider names, which was a critical security issue (CWE-639).
* fix(credentials): preserve provider-level expiration for handle-backed credentials
Compute effective expiration from both provider and driver values using the
earliest non-zero timestamp and skip expired values before insertion
(GATOR-1806c9be-02).
- Modified resolve_provider_handles to check provider credential_expires_at_ms
- Skip expired credentials during resolution instead of returning them
- Use effective expiration (min of provider and driver) in resolution results
- Fix inference.rs to preserve earliest expiration when merging
This ensures handle-backed credentials respect the same expiration semantics
as inline credentials.
* fix(credentials): stage refresh changes under new handles before validation
Stage credential replacements under new immutable handles instead of reusing
existing handles to prevent overwriting committed values before validation/CAS
(GATOR-1806c9be-03).
- Stage credentials with empty existing_handles map to force new handle creation
- Validate and CAS before the new values are committed to backend storage
- Delete old handles only after successful CAS
- On CAS failure, delete only the newly staged handles
- This prevents CWE-362/CWE-367 race conditions where failed refreshes could
still modify or delete the active credential
The fix ensures that a rejected refresh cannot modify the backend object still
referenced by the committed provider record.
* fix(credentials): add timeouts to credential driver RPCs
Apply configured timeouts to both startup capability negotiation and runtime
RPCs to prevent indefinite hangs (GATOR-1806c9be-05).
- Add DEFAULT_CREDENTIAL_DRIVER_RPC_TIMEOUT_SECS constant (30s)
- Apply timeout to GetCapabilities during startup connection
- Apply timeout to all runtime RPCs (store, delete, resolve)
- Use tokio::time::timeout to bound the entire GetCapabilities operation
during startup, not just the socket connection
- Return contextual deadline errors on timeout
This prevents a faulty or overloaded driver from hanging gateway operations
indefinitely.
* fix(credentials): fix test to use consistent workspace/provider identity
The Kubernetes auth Vault resolve test was constructing a managed path with
test-workspace/test-provider-id but sending default/prov-123 in the request,
causing validation to reject the request (GATOR-18e32351-01).
- Update test to use test-workspace and test-provider-id in the request to
match the logical_path construction
- This ensures the test exercises the intended code path and validates
Kubernetes auth resolution properly
The test now passes and correctly validates identity enforcement.
* fix(credentials): use unique staging ID for refresh to avoid overwrites
Stage refresh replacements under genuinely distinct immutable handles using
a unique staging ID to prevent overwriting committed values (GATOR-1806c9be-03).
- Generate a unique staging ID using UUID for each refresh operation
- Use this staging ID when storing credentials instead of the real provider ID
- Pass the same staging ID during cleanup on failure to delete only staged objects
- This ensures deterministic paths (Vault) and object names (K8s) don't collide
with the committed provider's credentials
The fix prevents failed refreshes from silently replacing active credentials
or breaking providers by deleting still-referenced backend objects.
* fix(credentials): wrap credential driver RPCs in local timeouts
Add local tokio::time::timeout wrappers around credential driver RPCs to
bound non-compliant or stalled UDS peers (GATOR-1806c9be-05).
- Wrap StoreCredential, DeleteCredential, and ResolveCredentials in local timeouts
- Return contextual deadline_exceeded errors when timeouts occur
- Keep existing gRPC timeout metadata for compliant implementations
- GetCapabilities during startup was already wrapped in previous commit
This ensures a faulty local driver cannot hang gateway operations indefinitely,
even if it accepts the connection but never responds to the RPC.
* fix(credentials): preserve ownership for staged refreshes
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* fix(credentials): bound startup capability probe
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* test(provider): authenticate credential handler requests
Signed-off-by: Seth Jennings <sjenning@redhat.com>
* fix(ci): grant actions read to credential driver e2e
Signed-off-by: Seth Jennings <sjenning@redhat.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Co-authored-by: Taylor Mutch <taylormutch@gmail.com>
Co-authored-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
* perf(net): set TCP_NODELAY on all tunnel and proxy TCP hops
The sandbox tunnel added ~44 ms of latency to every small
request/response because no socket in the path disabled Nagle's
algorithm, so sub-MSS writes waited on delayed ACKs at each hop.
Set TCP_NODELAY on every latency-sensitive TCP socket:
- gateway: accepted connections on the public listener (gRPC relay
frames and WS tunnel writes)
- CLI: edge tunnel local accept + underlying WebSocket TCP stream,
insecure TLS connector (tonic's default connector already does
this), and service-forward accepted sockets
- supervisor: direct-tcpip connect into the sandbox netns, TCP relay
target dials, egress proxy accepted connections, and all upstream
CONNECT/HTTP dials (via a new connect_upstream helper)
Setting TCP_NODELAY on connect is best-effort: a failure only costs
latency, so we log and continue rather than fail the connection.
The sandbox SSH transport rides a unix domain socket and gRPC client
channels use tonic defaults (nodelay on), so no change is needed there.
Fixes#2219
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* refactor(net): house TCP_NODELAY helper in a shared net module
Address review feedback on the TCP_NODELAY change:
- Move the shared set-nodelay helper out of supervisor_session into a
new crate-private `net` module in openshell-supervisor-process, so
ssh and supervisor_session no longer reach across modules through a
pub(crate) item.
- Make the best-effort comments at each call site terse and consistent.
No behavior change; the benchmark ladder reproduces the same numbers.
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* refactor(net): consolidate TCP_NODELAY helpers into openshell_core::net
Move the best-effort TCP_NODELAY helpers into the shared openshell_core::net
module so every crate dials and configures sockets the same way:
- Add set_tcp_nodelay_best_effort (accepted/existing streams) and
connect_tcp_nodelay_best_effort (dial + set) with unit tests.
- Migrate all call sites in openshell-cli, openshell-server, and the
supervisor crates to the shared helpers.
- Remove the crate-private net module from openshell-supervisor-process.
- Document socket guidance in AGENTS.md (Network Sockets).
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* perf(net): set TCP_NODELAY on exec bridge and metadata server
The gateway-side single-use SSH-over-relay loopback bridge and the
sandbox IMDS metadata server were missed latency-sensitive TCP hops.
Set TCP_NODELAY on the accepted client connection and both russh client
dials of the exec bridge — interactive keystrokes and line-buffered PTY
output are the most tinygram-heavy traffic in the system — and on the
metadata server's accepted connections.
Also log unrecognized MaybeTlsStream variants in the edge tunnel so a
future TLS-backend change surfaces a silent TCP_NODELAY miss instead of
skipping it quietly.
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
* perf(net): set TCP_NODELAY on openshell-sdk socket paths
The openshell-sdk crate landed on main with its own copies of the CLI's
hand-rolled sockets, which the CLI and TUI are meant to consume. Give
them the same treatment as the CLI equivalents:
- edge_tunnel: the accepted local tunnel connection and the WebSocket's
underlying TCP socket (plain and rustls variants).
- transport: the dial in InsecureTlsConnector, tonic's custom-connector
path.
Only these hand-rolled sockets need it. Tonic's own connector defaults
tcp_nodelay to true and applies it itself, so plain Endpoint::connect
callers were already covered.
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
---------
Signed-off-by: Jim Meyer <jim@meyer4hire.com>
Update the docs for `sandbox create` and `exec` to dissuade use of `--env` for
secrets, and enhance the docs for `--provider` to explain what it's for.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Evan Lezar