Previously, every event from a sandbox reused the sandbox ID as its event
ID. Consumers deduplicating security records could mistake separate events
for the same record, and missing device types or empty image objects could
prevent schema validation.
Give each event its own ID, retain the sandbox association separately, and
classify the environment as Other/Sandbox while keeping the OS separate.
Omit unknown container details instead of emitting empty objects.
Security tooling can now distinguish events from the same sandbox and read
their identity consistently after serialization.
Refs #1055
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* refactor(inference): remove managed inference routes
Closes#3172
Remove the inference route control plane, inference.local data path, built-in router crate, and SDK surface. Move inference workloads to explicitly imported provider profiles and native endpoints, with migration cleanup and updated tests and documentation.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* fix(policy): preserve alternate upstream isolation
Restore the provider policy activation guard so legacy OpenAI and Anthropic providers configured for alternate base URLs do not grant egress to the built-in public vendor endpoints.
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
---------
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* feat(sandbox): support rootfs tar as --from source for VM driver
Accept flat rootfs tar archives (.tar, .tar.gz, .tgz) via the --from
flag for VM-backed gateways. The CLI detects the archive extension,
validates that the gateway uses the VM compute driver, and passes the
tar path through driver_config. The VM driver copies the tar into its
staging area and feeds it into the existing rootfs extraction and ext4
disk creation pipeline, skipping the container image pull/export steps.
Closes#2175
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): validate rootfs tar path at the VM driver boundary
The rootfs_tar_path field in driver_config was passed from the API
caller directly to tokio::fs::copy without validation. An authenticated
user bypassing the CLI could supply arbitrary host paths (e.g.
/dev/zero for disk exhaustion, or readable host files for data
exfiltration).
Introduce a trusted staging directory that the VM driver creates on
startup and advertises via GetCapabilities. The CLI now copies the tar
into the staging directory before creating the sandbox, and the driver
validates that the received path is a regular file inside the staging
root and within a configurable size limit (default 10 GiB) before any
I/O.
New VmDriverConfig options:
- rootfs_tar_staging_dir: override the staging directory
(default: <state_dir>/rootfs-tar-staging)
- rootfs_tar_max_bytes: override the size limit (default: 10 GiB)
Addresses GATOR-28b5152e-01.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): request-scoped staging, size pre-check, and cleanup for rootfs tar
Tighten the rootfs tar staging flow to address the remaining GATOR-01
obligations:
- Request-scoped staging: the CLI creates a unique per-request
subdirectory (req-<pid>) under the staging root instead of placing
files directly in the shared directory. The driver enforces that the
tar path is at depth 2 (staging_root/<subdir>/<file>), preventing
cross-request path selection.
- Size pre-check: the driver advertises rootfs_tar_max_bytes via
GetCapabilities. The CLI reads this limit and rejects oversized files
before copying, avoiding disk exhaustion in the staging directory.
- Cleanup: the driver removes the request staging subdirectory after
consuming the tar (on cache hit, copy success, or copy failure),
ensuring staged data does not persist beyond the request.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): restore rootfs-tar sandboxes from persisted image identity
On restore or restart, the one-shot staged tar archive has already been
cleaned up. Reading the persisted image identity from the sandbox state
directory and resolving the cached disk path directly avoids re-accessing
the deleted staging path.
Addresses GATOR-168b9210-01.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(cli): use random staging dirs and enforce byte limit during rootfs tar copy
Replace PID-based request staging directories with tempfile-generated
random names to prevent collisions and make paths unpredictable.
Replace bare tokio::fs::copy with a streaming copy loop that enforces
the advertised max_bytes limit during transfer, closing the TOCTOU gap
between the pre-copy size check and the actual copy.
Signed-off-by: Philippe Martin <phmartin@nvidia.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(sandbox): issue rootfs tar staging slots from the gateway
A caller could name any host path in `driver_config.vm.rootfs_tar_path`,
which the privileged VM driver then read. The CLI-side locality check did
not apply to direct API requests.
The gateway now owns staging. `BeginRootfsTarStaging` allocates a
request-scoped directory under the driver-advertised staging root and
returns an opaque single-use token; `CreateSandbox` carries the token, and
the gateway substitutes the path it allocated before dispatching to the
driver. `template.driver_config.<driver>.rootfs_tar_path` is rejected
outright in request validation, so a caller-supplied path never reaches
privileged I/O.
Tokens are bound to the issuing workspace and subject, consumed once, and
expire after 30 minutes. Outstanding slots are capped per caller and
overall, so one caller can neither exhaust the staging filesystem nor
starve others. An RAII guard reclaims the directory on every failure path
after consumption, and an age-gated sweep runs at startup and on each
reconcile pass for directories whose driver died before its own cleanup.
The token is stripped from the public sandbox before persistence: the
stored copy is returned verbatim by GetSandbox, ListSandboxes and
WatchSandbox to every member of the workspace.
Also fixes two defects this exposed:
- The CLI wrote `rootfs_tar_path` at the top level of `driver_config`, but
the gateway forwards only `driver_config.<driver_name>`, silently
dropping unmatched keys. The archive never reached the VM driver, so the
documented `--from ./rootfs.tar` flow did not work at all. Config is now
nested under `vm` and deep-merged, so a caller's existing VM settings
survive instead of being clobbered by a shallow extend.
- Staging previously required `GetGatewayInfo`, which is restricted to
`platform_admin`, making the feature unusable for ordinary users on any
RBAC-enabled gateway. The new RPC matches CreateSandbox at
`sandbox:write` / `workspace_role: user`.
`compute_driver.proto` is unchanged; the gateway reads the staging root
from the capabilities it already stores.
Refs #2175
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): derive rootfs tar cache identity from archive contents
The prepared-disk cache key combined the archive's full path with an mtime
truncated to seconds, then mapped punctuation to `-`. Distinct paths such as
`/tmp/a/b.tar` and `/tmp/a-b.tar` collapsed onto the same key and reused each
other's disk, a rewrite within the same second kept stale contents, and a long
path could exceed filesystem component limits.
Identity is now a SHA-256 of the archive contents. This is also what makes the
cache work at all now that the gateway allocates a fresh staging directory per
request: a path-derived key would miss on every create.
The archive is hashed, the cache checked, and only on a miss copied — so a hit
skips writing a multi-gigabyte file. The copy is hashed as it is written and
rejected if the digest differs from the first pass, which closes the window
where the source changes during staging rather than approximating it with a
re-stat.
Refs #2175
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): decompress gzip rootfs tar archives during staging
`--from` accepts `.tar.gz` and `.tgz`, but the driver staged whatever bytes
it was given and the guest image-prep VM extracts the staged file with a
plain `tar -xpf`. Compressed sources therefore depended on the guest tar
auto-detecting gzip, and the prepared disk was sized from the compressed
length, which is far too small for the expanded rootfs.
Staging now detects gzip from the archive's magic bytes -- the driver only
ever sees a gateway-issued staging path, never the caller's file name -- and
writes an uncompressed tar. The digest still covers the source bytes, so the
"archive changed while staging" check is unaffected, and expansion is bounded
by `rootfs_tar_max_bytes` so a compression bomb cannot fill the host disk.
`extract_rootfs_archive_to` sniffs gzip as well, so the host-side extraction
path matches.
Adds unit coverage for gzip staging, bounded expansion, and gzip extraction,
plus an e2e sandbox created from a gzip-compressed export.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
---------
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@nvidia.com>
* feat(ocsf): configurable schema version for SIEM backward compatibility
Add a gateway-configurable OCSF schema version target that downgrades
JSONL output for SIEMs that only support older schema versions. AWS
Security Lake requires v1.1.0, Splunk CIM Add-On targets v1.1-v1.3.
The downgrade filter strips profile-gated fields (ai_model, container,
observation_point_id), removes unknown profiles from metadata.profiles,
rewrites metadata.version, and adds an unmapped.downgraded_from
breadcrumb so auditors can distinguish "no model involved" from "model
attribution stripped."
Supported target versions (1.1, 1.3) are enforced by an allow-list in
the settings registry. Invalid values are rejected with a clear error.
The setting flows to sandboxes via the settings bundle and takes effect
on the next poll cycle. The shorthand log output is unaffected.
Closes#2662
Signed-off-by: Adel Zaalouk <zanetworker@gmail.com>
Signed-off-by: Adel Zaalouk <azaalouk@redhat.com>
* fix(ocsf): align downgrade with schema version
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
---------
Signed-off-by: Adel Zaalouk <zanetworker@gmail.com>
Signed-off-by: Adel Zaalouk <azaalouk@redhat.com>
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
* docs(gateway-config): fix stale community sandbox image path
Signed-off-by: Yuedong Wu <dwcn22@outlook.com>
* docs(sandbox-image): purge remaining stale image references
Rebasing onto main surfaced four more instances of the same dead
ghcr.io/nvidia/openshell/sandbox path, introduced by commits merged
after this branch was opened: three test fixtures (driver-docker,
openshell-ocsf, compute::mod) and one user-facing default in the
SPIFFE token-exchange Podman demo README. Correct all four to
ghcr.io/nvidia/openshell-community/sandboxes/base, consistent with
the rest of this fix.
Signed-off-by: Yuedong Wu <dwcn22@outlook.com>
---------
Signed-off-by: Yuedong Wu <dwcn22@outlook.com>
* fix(sandbox): detect an available login shell instead of hardcoding /bin/bash
The built-in default sandbox command and the interactive SSH session
hardcoded /bin/bash. Minimal images such as Alpine ship only /bin/sh
(BusyBox ash), so sandbox startup failed with an opaque "No such file or
directory (os error 2)" that never named the missing binary.
Add openshell-core::shell with shell-path constants and a runtime
detect_login_shell() that resolves a shell present in the sandbox image
($SHELL if executable, then bash, then /bin/sh). Use it for:
- the built-in default command (only the default is remapped; explicit
user commands are never rewritten), resolved in the supervisor so it
inspects the sandbox filesystem rather than the gateway's
- the SSH interactive shell
- the SHELL environment variable
Also name the program in the spawn error so a missing shell/binary is
diagnosable instead of a bare ENOENT.
Refs #3146
Signed-off-by: Akram <akram.benaissi@gmail.com>
* fix(sandbox): drop $SHELL preference in shell detection
$SHELL is image/user-controlled and the detected shell is later invoked
with `-lc`, so an executable that is not a compatible shell (e.g.
SHELL=/bin/false) would pass the executable check and then break command
execution even when /bin/sh is available. Resolve only from known shell
paths instead.
Also add a USR_BASH constant for /usr/bin/bash rather than a string
literal in SHELL_CANDIDATES.
Refs #3146
Signed-off-by: Akram <akram.benaissi@gmail.com>
* fix(sandbox): resolve the default login shell in the supervisor (empty command = default)
Addresses review: interactive PTY SSH now uses the detected shell, the shell
tests are portable across the Windows lane, and default-shell provenance is
carried without a new spec field.
An omitted command is left empty end to end and resolved in the supervisor,
which is the only place that sees the sandbox image:
- The CLI forwards the command as-is; the gateway persists an omitted command
as empty (no baked /bin/bash -l) and requests a TTY.
- MainProcessConfig carries the command empty (the transport now allows it);
the supervisor resolves a login shell that exists in the sandbox image (bash
when present, otherwise /bin/sh on minimal images like Alpine) and logs the
resolved shell.
- Interactive PTY SSH (spawn_pty_shell) uses the detected shell; a shared
build_ssh_shell_command helper covers the PTY and non-PTY paths, with a
deterministic sh-only regression test.
- Unix-only shell tests are gated with cfg(unix).
An explicit command is always run verbatim.
Refs #3146
Signed-off-by: Akram <akram.benaissi@gmail.com>
---------
Signed-off-by: Akram <akram.benaissi@gmail.com>
Centralize compute-driver RPC descriptors, stream instrumentation, provider
routing, and standalone installation in openshell-otel. Use typed RPC
constants so gateway and in-process driver paths cannot panic on unknown
operation strings or repeat runtime method parsing.
Emit semantic-convention rpc.service and rpc.method attributes, preserve
trace context and resource identity across deployment modes, and route both
RPC boundary and backend crate spans to each selected driver provider. Leave
consumer-dropped watch spans unset while recording observed terminal status,
and avoid reboxing untraced external-driver streams.
Derive each driver tracing identity from Cargo package and crate metadata and
attach its descriptor to the compute-driver registration, keeping provider
selection and target routing tied to the registered implementation. Share
tracing setup and round-trip test support across Docker, Podman, Kubernetes,
and VM, and update the gateway tracing documentation.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* feat(middleware): broaden HTTP header mutation authority
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* fix(middleware): protect response credential headers from mutation
Response middleware could write or remove Set-Cookie, WWW-Authenticate,
Authentication-Info, and Proxy-Authentication-Info, letting a stage plant
or strip credentials the sandbox client acts on. Protect them in both
directions, matching the request profile's treatment of Authorization and
Cookie, and reserve the x-openshell-credential prefix for responses too.
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
* fix(middleware): reject credential placeholder writes
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
---------
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
The tutorial told users to exit the sandbox and reconnect later, but
exiting the interactive shell stops the sandbox's main process and it is
not reconnectable under the default restart policy. Switch to the
two-terminal flow already used by the github-sandbox tutorial so the
sandbox stays running, matching what examples/sandbox-policy-quickstart/
demo.sh actually does.
Related: #2998, #2798
Signed-off-by: Russell Bryant <rbryant@redhat.com>
* feat(skills): separate public and contributor workflows
Closes#2736
Publish the four user-facing OpenShell skills from the top-level skills directory, mark contributor workflows internal, and update portability guidance, validation, and documentation.
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
* docs(skills): clarify public skill audit scope
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
* docs(skills): use markdown documentation links
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
* docs(skills): align public and contributor guidance
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
---------
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
* fix(cli): require ANSI-capable terminal before colorizing
Follow-up to #3026, raised in review.
`auto` treated any terminal as styleable, so `TERM=dumb openshell ...`
still emitted escapes into a terminal that renders them literally. An
unset TERM had the same problem.
This is partly a regression that #3026 introduced. `console`, which
drives indicatif and dialoguer, already refused to colorize when TERM is
`dumb` or unset, and miette applies the same check through
supports-color. #3026 overrides both with its own switch, so it replaced
two working checks rather than only failing to add one. tracing and the
owo-colors wrapper never had detection, so those two are a gap rather
than a regression.
Add the capability check to the `auto` branch only, matching console's
unix rule: `dumb` is not capable, and an unset TERM is not capable
because nothing identifies a capable terminal. Empty is treated as unset,
which diverges from console — it reads `TERM=""` as capable since the
value is not `dumb` — because an empty value names no terminal type and
every other variable here already treats empty as unset.
Because the check sits after the explicit branches, `--color always` and
FORCE_COLOR still force styling on a dumb terminal, and `--color never`
and NO_COLOR still suppress it on a capable one. TERM is a unix signal;
Windows consoles enable virtual terminal processing and do not set it, so
the check does not apply there.
The existing pty test now pins TERM. It previously inherited the ambient
value, which would make its outcome depend on the environment now that
capability is consulted — CI runners frequently leave TERM unset.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* refactor(cli): combine stream and terminal capability checks
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* docs(cli): clarify table color behavior
Signed-off-by: Evan Lezar <elezar@nvidia.com>
* test(cli): cover redirected status table colors
Signed-off-by: Evan Lezar <elezar@nvidia.com>
---------
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
* feat(vm): support corporate HTTP forward proxy egress for microVM sandboxes
The corporate forward proxy machinery from #1792 is driver-agnostic and
already merged: openshell-supervisor-network implements CONNECT chaining,
NO_PROXY matching, credentials, https:// proxies and corporate CA trust, and
openshell-sandbox exposes it as six argv-only flags. Podman gained the driver
half in #2245/#2512 and Kubernetes in #2633; the VM driver had none of it, so
VM sandboxes on proxy-only networks could not reach any destination requiring
the proxy even when policy allowed it.
The blocking piece was not proxy logic but delivery: the VM guest init script
runs as PID 1 and execs a fixed supervisor command line, and libkrun's
krun_set_exec receives an empty argv, so there was no channel for driver-owned
supervisor arguments. The supervisor's proxy flags deliberately have no
environment fallback, and build_guest_environment merges user-supplied
environment, so the guest env is not a safe transport either.
Add a driver-authored argument file, mirroring the existing init.d manifest:
the driver writes /opt/openshell/supervisor-args into the overlay upperdir on
every launch and the guest reads it verbatim, one argument per line, appending
it to every supervisor exec. It is written even when empty, which is what makes
the channel unforgeable -- the upperdir always shadows the read-only image
layer, so an image can neither supply its own arguments nor disable the
operator's by omitting the file. Because both launch backends exec the same
init script, this covers libkrun and QEMU without touching either.
A microVM has no bind mounts or container secrets, so the credential and CA
bundle are staged into the per-sandbox overlay the way the gateway JWT already
is: credential root-only at 0600, CA at 0644, both rewritten every launch so a
removed setting clears prior material, and both deleted with the sandbox state
directory. This places the credential at rest in the overlay image on the
gateway host, which differs from the Podman secret model and is documented as
an explicit security consideration.
Validation is fail-closed and shared: a new
openshell_core::driver_utils::validate_upstream_proxy_settings holds the
pairing rules the Podman driver established, and both the gateway and the
driver call it so an invalid table names the offending key instead of
surfacing as an opaque driver-readiness timeout.
Guest egress leaves through gvproxy, so a proxy on the gateway host's loopback
is reachable only through host.openshell.internal; the guest to gateway
callback is unaffected.
Closes#3088
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): bound the proxy CA read and scope the host-loopback recipe
Two review findings on the corporate forward proxy support for microVM
sandboxes.
The driver read the operator's proxy_ca_bundle with an unbounded fs::read and
accepted it on a substring match for the PEM BEGIN CERTIFICATE marker. A
special file such as /dev/zero therefore grew driver memory without bound on
every authorized sandbox create, and a PEM block holding invalid DER passed
the host check but contributes no trust anchor in the guest, so every
supervisor would fail after boot with an error attributed to the sandbox
rather than to the setting.
Move the read into openshell-core as read_upstream_proxy_ca_bundle_file: it
reuses the credential reader's bounded-read path (non-regular files rejected
on fstat, size capped, read bounded even if the file grows), then requires at
least one anchor that RootCertStore::add_parsable_certificates accepts. The
supervisor's own reader now delegates to it, so host acceptance and guest
acceptance are the same function and cannot drift.
The published host-loopback recipe was written for libkrun only. gvproxy NATs
host.openshell.internal to the gateway host's 127.0.0.1, but GPU sandboxes run
on the QEMU/TAP backend where that name resolves to the TAP host address and
the driver's own nftables input chain accepts only the gateway port from the
guest — no proxy on the gateway host is reachable there at any bind address,
so an operator following the generic recipe lost all proxy-required egress
while configuration validation succeeded.
Scope the recipe to libkrun in every reference and reject a gateway-host proxy
URL when a launch plan resolves to QEMU, naming the reason, instead of booting
a sandbox whose policy-approved CONNECTs all time out.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* fix(vm): match the QEMU proxy preflight to the selected TAP host
The gateway-host proxy guard added for the QEMU/TAP backend classified the
wrong set of addresses in both directions.
It ran at the top of configure_qemu_launch_plan, before the subnet allocation
that settles plan.host_ip, so it could not compare against the address the
guest actually reaches the host on. An operator pointing https_proxy at the
sandbox's own TAP host address, such as 10.0.128.1, passed the check, and the
driver's nftables input chain — which accepts only the gateway port from the
guest — then dropped every policy-approved CONNECT, which is exactly the
silent timeout the guard exists to prevent.
In the other direction it rejected 192.168.127.254 unconditionally. That
address is special only to libkrun/gvproxy; on QEMU/TAP it is an ordinary
address that may be routable through the guest's masqueraded egress, so the
guard refused a working configuration.
Run the check after the launch plan's network allocation, on both the
freshly-allocated and already-complete paths, and compare IP literals with
that sandbox's selected TAP host. Loopback literals, localhost, and the
documented host aliases that write_host_gateway_aliases seeds to the TAP host
still classify as the gateway host, and the failure names the address. The
gvproxy host-loopback constant returns to being a documentation anchor.
Signed-off-by: Philippe Martin <phmartin@redhat.com>
---------
Signed-off-by: Philippe Martin <phmartin@redhat.com>
* feat(server): add sandbox workload templates
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(go-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(rust-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(python-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(typescript-sdk): add sandbox workload template support
Signed-off-by: Gordon Sim <gsim@redhat.com>
* docs(agents): document sandbox workload templates
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): support default GPU requests in sandbox templates
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(server): cap sandbox templates per workspace
Signed-off-by: Gordon Sim <gsim@redhat.com>
* docs(architecture): document sandbox workload template boundaries
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(cli+sdk): expose sandbox workload template provenance
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): include sandbox template annotations in output
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(sandbox): add label selectors to template listing
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(e2e): cover sandbox template failure paths
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(server): preserve command and ttl when creating sandbox from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(server): add field coverage test for template merge
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): add pagination support to fake client
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): warn on env vars that looks like secrets
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(sdk-ts): propagate sandbox workspace through lifecycle calls
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(docs): update workspace management docs
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(ts-sdk): support command and tty when creating from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): support command and tty when creating from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* test(python-sdk): verify command and tty handling when creating from template
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): update docs and ClientInterface
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(server): validate sandbox create specs before I/O
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(go-sdk): guard empty DNS-1123 label validation
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(python-sdk): allow empty template builder mappings
Signed-off-by: Gordon Sim <gsim@redhat.com>
* fix(cli): align template GPU JSON default output
Signed-off-by: Gordon Sim <gsim@redhat.com>
---------
Signed-off-by: Gordon Sim <gsim@redhat.com>
* feat(helm): split gateway and workspace charts
Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com>
* fix(helm): preserve split chart upgrade compatibility
Keep workspace manifests valid after value validation and default legacy reused values to the combined resource topology.
* fix(ci): preserve VM runtime for E2E
The Rust cache restores target/ after VM runtime artifacts are staged,
overwriting target/vm-runtime-compressed before openshell-driver-vm is built.
Stage the compressed runtime outside target and pass that location through
OPENSHELL_VM_RUNTIME_COMPRESSED_DIR so build.rs can embed the supervisor.
Also locate the Helm split-ownership test repository root from the script
path rather than git rev-parse. The test runs in a container where the
GitHub checkout can be owned by a different UID and rejected as dubious
ownership.
Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com>
* fix(ci): install yq for Helm ownership test
The split-chart ownership regression uses yq to inspect rendered YAML,
but the Helm CI container installs only tools declared in mise.
Declare and lock yq so mise install --locked provides the test dependency.
Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com>
---------
Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com>
* fix(cli): suppress ANSI color when stdout is not a terminal
The CLI colorized output unconditionally. owo-colors is built without
its `supports-colors` feature, so `.green()` and friends emitted escape
sequences regardless of destination, and nothing in the CLI read
NO_COLOR. Piping any command through grep or awk matched against bytes
the caller could not see; `forward list` was the case that surfaced it,
where an escape sits immediately before the STATUS word and defeats a
pattern anchored on whitespace.
Add a `color` module holding a process-wide switch resolved once in
run_async, before any output. Command modules import its `Colorize`
trait in place of `OwoColorize`; the method names match, so the ~450
call sites are unchanged, but each consults the switch when it renders
and delegates to owo-colors so the escape bytes stay identical. The two
traits collide by design: importing both in one module is an ambiguity
error, which keeps unconditional coloring from returning.
owo-colors is not the only styled path, and the rest each carry their
own default, so the switch governs them too:
- tracing_subscriber formats with ANSI on, does no terminal detection,
and writes to stdout, so `openshell -v ... | ...` leaked escapes the
same way the tables did. It now takes the setting via with_ansi.
- indicatif and dialoguer both style through console, which has its
own detection but cannot learn about --color. Overriding console's
global switch covers every progress bar and prompt rather than the
specific ones constructed today. Both the stdout and stderr switches
are set, since prompts and progress bars draw to stderr.
- miette renders errors through its own handler, likewise unaware of
--color, so init installs one built from the setting.
Resolution order: `--color always|never`, then NO_COLOR, then
CLICOLOR_FORCE, then whether stdout is a terminal. The decision is made
against stdout even for stderr text, since stdout is what gets parsed;
`--color always` restores styling when redirecting.
Padding is unaffected — the format spec is forwarded to the inner
Display, so widths measure text rather than text plus escapes.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* fix(cli): resolve color per output stream
Review feedback on #3026.
Resolving one answer from stdout and handing it to every library meant a
redirected stream inherited the other stream's terminal check. Running
`openshell ... 2> build.log` from a terminal wrote escapes into the log,
because console's stderr switch and miette's handler were both given
stdout's answer. That is worse than the behavior before this branch,
where both libraries did their own per-stream detection.
Resolve `auto` separately for stdout and stderr and hand each library
the answer for the stream it writes to: tracing and console's stdout
switch get stdout, miette and console's stderr switch get stderr. The
owo-colors wrapper is the exception, since its call sites are split
across println! and eprintln! and a Painted value cannot tell which
macro will consume it; it styles only when both streams accept escapes,
erring toward plain text rather than risking a redirected stream.
Existing tests could not catch this: Command::output gives both streams
pipes, so a per-stream decision and a single stdout-derived one look
identical. Add a test that puts stdout on a pty and stderr on a pipe,
which fails when stderr is handed stdout's answer.
Replace CLICOLOR_FORCE with FORCE_COLOR. The clicolors spec does not say
how to treat `0`, and implementations that special-case it disagree with
force-color.org, which keys on presence and non-emptiness only. Using
FORCE_COLOR gives it the same rule as NO_COLOR: set and non-empty means
yes, whatever the value. Nothing depended on CLICOLOR_FORCE, which was
introduced earlier on this branch and never released.
Carry the whole style in an owo_colors::Style rather than dispatching a
local enum through a six-arm match, and merge styles when chaining so
`x.green().bold()` emits one `\x1b[32;1m...\x1b[0m` instead of nesting
two wrappers. No call site styles already-styled text, so merging is
safe; the emitted bytes are shorter and there is a single reset.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
---------
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
* feat(gateway): add installation name configuration
Add a first-class operator-assigned gateway name with TOML, CLI, environment,
and Helm configuration surfaces. Local gateways default to openshell, while
Helm defaults to the chart fullname; operators sharing a collector across
namespaces or clusters can set a globally distinct name.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* feat(gateway): identify gateways in exported traces
Attach the configured gateway installation name and compute driver to the
gateway OpenTelemetry resource so operators can filter traces from multiple
installations that share a collector. Forward the gateway name and OTLP
endpoint to managed external drivers so their distinct service resources carry
the same installation identity.
Keep service.name stable per process type, omit blank resource values, and
leave per-span operation names and request attributes unchanged.
Refs #2507
Signed-off-by: Kris Hicks <khicks@nvidia.com>
---------
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Mirror the VM, Podman, and Docker driver tracing setup for Kubernetes.
Export standalone driver spans through OTLP/gRPC as the distinct
openshell-driver-kubernetes service, preserve gateway trace context, record
lifecycle operations and gRPC failures, and flush spans on shutdown.
Kubernetes currently runs in-process when selected as a built-in gateway
driver. Use the temporary server-boundary shim shared with Podman and Docker
so traces retain the shape they will have when Kubernetes moves to a
separate process. Move the common ComputeDriver RPC tracing layer into
openshell-otel to keep all drivers aligned.
Propagate the active W3C context through the controller-reserved Sandbox
annotation and enable Agent Sandbox OTLP export in the local k3s workflow.
This connects asynchronous controller reconciliation spans to the originating
OpenShell create trace.
Expose gateway OTLP configuration through Helm and add an Aspire collector
to the local k3s workflow. Extend helm:k3s:forward with OTLP ingest and trace
UI forwarding for Kubernetes and local container gateway development.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* feat(tui): show persisted guidance on rejected policy chunks
The reviewer's rejection note is stored in PolicyChunk.rejection_reason and
already reaches the TUI in GetDraftPolicyResponse.chunks, but openshell-tui
never read the field. The note was dropped at the last step, so a reviewer
had no way to recall why a chunk had been rejected.
Render it in two places, following the truncate-in-list / full-in-popup
convention in the TUI development guide: a truncated, dimmed suffix on the
list row, and a "Guidance:" line in the detail popup.
Gate the accessor on status == "rejected" rather than on the field alone.
Approving a chunk passes None for the reason and the gateway writes the
field only when Some, so a chunk that was rejected and later approved still
carries the old note. Reading the field unconditionally would surface a
stale rejection on an approved rule.
Part of #1098, Definition of Done item "Rejected chunks show persisted
guidance". The other items on that issue are untouched.
Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
* fix(tui): scroll the draft detail popup so long guidance stays reachable
A rejection reason has no server-side length cap, so a paragraph-length one
overflowed the fixed 22-row detail popup: the tail was clipped and the later
fields and both action hints were pushed off screen. The same overflow already
affected the unbounded rationale and security_notes fields, so fix the popup
rather than special-case the guidance line.
Split the popup's inner area into a scrolling body and a pinned hint row,
following the pattern in ui/create_provider.rs, and drive it with j/k, the
arrow keys, PageUp, PageDown, g and G. The approve and close controls now stay
on screen at every scroll position, and the bottom border carries the scroll
position.
Wrap free-form values explicitly instead of relying on Paragraph's Wrap, so the
rendered row count is exactly lines.len() and the scroll clamp cannot under-run
the content. Paragraph::line_count would answer the same question but sits
behind ratatui's unstable-rendered-line-info feature.
Add deterministic TestBackend coverage at 80x24 with a 2,000-character reason,
covering the head and tail, the pinned hints, over-scroll clamping, and the
pre-existing long-rationale case. Document the reviewer-facing guidance in the
policy advisor page.
Part of #1098.
Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
* fix(tui): wrap the denial row and hint footer on a narrow popup
Making the scroll clamp exact meant dropping Paragraph's Wrap, which also
removed wrapping from the lines that do not go through push_wrapped. At 70
columns the denial row clipped mid-value and lost the last-seen timestamp, and
at 60 a pending chunk with scrollable content pushed [Esc] Close past the right
edge of the single hint row. The key still worked, but that is the same
controls-not-visible failure one axis over.
Keep the denial row on one line while it fits and wrap it onto the label indent
when it does not, so the common 80-column layout is unchanged. Pack the footer
hints into as many rows as they need without splitting a hint, and derive the
footer height from that: adding a hint row shrinks the body and can itself
change whether the content scrolls, so the two settle together.
The earlier tests were all 80x24, which is why they missed this. Cover the
denial timestamps and the approve, reject and close hints at 60, 70 and 80
columns, plus the packing helper directly.
Part of #1098.
Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
* fix(tui): measure popup wrapping in display columns, not chars
wrap_value packed rows by chars().count(), and a CJK glyph is one char but two
terminal columns. A double-width rejection reason therefore produced rows about
twice the popup's width: the right half of every row was clipped, and unlike
the vertical case there is no horizontal scroll to recover it.
Measure width with Span::width(), the same measurement ratatui applies when it
lays cells out, so the wrap agrees with the renderer by construction and no new
dependency is needed. This covers word packing, the hard-break path for a word
wider than the line, the label indent, the footer hint packing, and the denial
row's fits-on-one-line check.
The list row's shortened copy had the same defect through truncate_str, which
counts chars. Leave truncate_str alone for its two existing callers and add
truncate_display for the guidance row.
Cover it with an 80x24 render regression that interleaves markers through the
CJK text: a trailing marker lands on its own short row in both the broken and
fixed layouts, so it would not detect this. The two unit tests measure with an
independent column oracle rather than the helper under test, which would make
them tautological.
Part of #1098.
Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
---------
Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
The Maturin-based wheel packaging was a historical remnant from when the local gateway launch path and OpenShell CLI were coupled in one binary. The gateway and CLI now ship as standalone artifacts, so the Python distribution should contain only the SDK.
Build a single platform-independent setuptools wheel, verify that it cannot contain native code or an openshell entry point, and simplify the release jobs and documentation for SDK-only PyPI installs.
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* fix(core): fall back to podman CLI when no API socket responds
Auto-detection only checked well-known Podman socket paths, so a Podman
machine exposing its API socket at a non-standard location went
undetected. The symlink at a well-known path is not always present; it
varies by Podman version, machine provider, and platform.
Extend detect_podman_socket() to fall back to podman CLI discovery when
no well-known candidate responds: podman info --format json determines
whether the service is local or remote, and podman machine inspect
resolves the host-side forwarded socket for VM-backed machines. All
existing callers (driver auto-detection, the Podman driver, and the VM
driver's container-engine fallback) pick this up without change.
Select the machine backing the active Podman connection instead of the
first entry from podman machine inspect, honoring podman's connection
precedence: CONTAINER_CONNECTION, then CONTAINER_HOST (mapped to a
connection by URI), then the containers.conf default. An explicit
endpoint that maps to no known machine is left unresolved rather than
guessing an unrelated machine. When CONTAINER_HOST is an explicit unix://
socket, use that path directly since podman info connects through it.
Update the gateway config reference and the Podman driver README, which
described probe-only detection.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
* fix(core): inspect podman machine by name and bound discovery probes
Address two startup defects in Podman socket auto-detection:
- Remote discovery ran `podman machine inspect` with no arguments, which
inspects only `podman-machine-default`. On a host whose default
connection is a different machine, selection fell back to the first
(wrong) entry, so the gateway could operate against the wrong backend.
Resolve the active machine first and inspect it by name, trying the
`-root`-stripped machine for rootful connections and returning None
when no machine can be mapped instead of substituting another.
- `podman info`, `podman machine inspect`, and `podman system connection
list` used unbounded `Command::output()`, so a stalled machine, SSH
connection, or helper could hang gateway startup indefinitely. Route
all three through a bounded runner that kills and reaps the child on a
documented deadline and returns None so detection can continue.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
* fix(core): bound podman probe stdout drainage and kill descendants
The previous timeout only bounded waiting for the direct child to exit.
Draining stdout still called read_to_end, which waits for pipe EOF — a
Podman probe can leave a daemonized descendant (SSH multiplexer, gvproxy)
that inherited the stdout pipe, so drainage could wait indefinitely even
after the direct child exited, defeating the deadline.
Run each probe as its own process-group leader, cap stdout drainage by
the same deadline via a channel, and on expiry kill the whole process
group so descendants holding the pipe are terminated and EOF is reached.
Add a regression test where the direct child exits but a backgrounded
descendant keeps stdout open.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
* fix(core): make podman probe deadline absolute against escaped descendants
The prior drainage timeout still ended in an untimed rx.recv() after
killing the process group. A descendant that escaped the probe's process
group (e.g. via setsid) while holding the stdout pipe would not be killed,
so read_to_end never saw EOF, the reader never sent, and that recv() could
block gateway startup indefinitely.
On drainage timeout, best-effort kill the process group and return None
immediately, abandoning the reader thread instead of waiting on it again.
The deadline now bounds the whole call regardless of what descendants do.
Add a regression test whose stdout-holding descendant escapes the process
group (`set -m`) and assert the call still returns promptly.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
---------
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Piotr Mlocek