Commit Graph
96 Commits
Author SHA1 Message Date
Ignas Baranauskas a72711697d chore: remove deprecated --keep flag from docs, scripts, and e2e tests (#2126)
* docs: remove deprecated --keep flag from tutorials and examples

The --keep flag is deprecated, hidden, and a no-op since sandboxes
are kept by default. Remove references from tutorial docs and example
READMEs that explain it as a real feature.

- Remove --keep from sandbox create commands
- Remove --keep explanation text
- Clarify that sandboxes are kept by default

Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>

* chore: remove deprecated --keep usage from scripts and e2e tests

The --keep flag is a deprecated no-op since sandboxes are kept by
default. Stop passing it in internal scripts, e2e test scripts,
and example demo scripts.

Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>

---------

Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>
2026-07-07 14:36:25 +02:00
Evan Lezar a226806026 test(e2e): run gpu workloads from manifest (#1709)
* test(e2e): add workload manifest build flow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): add gpu workload validation tests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(e2e): build gpu workloads before gpu e2e

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-30 09:54:12 +02:00
Taylor Mutch 7e0cce405d fix(build): use zig archive tools for cross builds (#2014)
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
2026-06-26 15:35:54 -05:00
Taylor Mutch ba21bb32a2 feat(kubernetes): support agent-sandbox v1beta1 (#2009)
* feat(kubernetes): support agent-sandbox v1beta1

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* ci(kubernetes): test agent-sandbox api versions

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): retry agent-sandbox raw 404s

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): harden agent-sandbox api setup

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): cache agent-sandbox api version

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* docs(kubernetes): document agent-sandbox upgrade behavior

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* refactor(kubernetes): collapse agent-sandbox api selection

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
2026-06-26 11:44:52 -07:00
Simon Scatton c7879a0a1d test(e2e): stop using custom e2e binary builds (#2000)
Remove the production-crate dev-settings feature so e2e tests build the same gateway, CLI, and supervisor binaries that we ship. Move settings coverage from dummy test-only keys to the production ocsf_json_enabled setting.

Signed-off-by: Simon Scatton <sscatton@nvidia.com>
2026-06-25 13:08:04 +00:00
Piotr Mlocek 82d03f1c98 fix(linux): lower host glibc floor to 2.28 to support RHEL/Rocky 8 (#1934) 2026-06-22 14:09:26 -07:00
Evan Lezar b6428cb9cd fix(build): align container engine selection (#1944)
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-22 10:50:16 -05:00
Taylor Mutch 62aa5e3247 ci(branch-checks): align Python checks with pre-commit (#1908)
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
2026-06-15 18:58:01 +00:00
Evan Lezar 8c0153438c test(e2e): add GPU workload image artifacts (#1484)
* test(e2e): add GPU workload image artifacts

Closes #1476

Define local GPU workload image sources for smoke-pass, smoke-fail, and cuda-basic validation, plus a mise build task that tags images with the source revision and records the latest local image refs.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): add workload manifest build flow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): fix gpu workload README lint

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): address gpu workload review feedback

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-15 11:26:06 -07:00
Piotr Mlocek 6c8cf38b63 ci(docs): add docs website automation (#1788)
* ci(docs): add docs website automation

* ci(docs): harden sync-docs inputs and add script tests

Route free-form workflow_dispatch inputs through env vars to avoid shell
injection, switch uv install to the approved setup-uv action, and add a
pytest suite for sync_docs_website.py wired into the test task.
2026-06-15 16:08:28 +00:00
Evan Lezar 9e805dc35d fix(build): use zigbuild for musl supervisor staging (#1850)
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-06-10 20:09:39 +02:00
Russell Bryant 3aba30c36f feat(telemetry): add build-time option to compile out telemetry (#1845)
* feat(telemetry): add build-time option to compile out telemetry

Gate anonymous telemetry emission behind a default-on `telemetry` Cargo
feature in openshell-core. The data model (enums, validation, emit_*/enabled*
signatures) stays always-compiled, while the endpoint, HTTP client, queue, and
emission code are feature-gated. With the feature off, enabled() returns false
and emit_* are no-ops, so dependent crates compile unchanged and no telemetry
endpoint, HTTP client, or emission code is included in the binary.

chrono and reqwest become optional dependencies of openshell-core, dropped from
its dependency graph when telemetry is disabled.

Thread the switch through the workspace: every crate depends on openshell-core
with default-features = false, and the default-on `telemetry` passthrough lives
on the binary crates that emit or collect telemetry (openshell-server,
openshell-sandbox, openshell-driver-vm). In-process drivers inherit it via
resolver v2 feature unification.

Build a telemetry-free binary with, e.g.:
  cargo build --release -p openshell-server --no-default-features

The runtime OPENSHELL_TELEMETRY_ENABLED switch is unchanged for default builds.

Signed-off-by: Russell Bryant <russell.bryant@gmail.com>

* ci(telemetry): guard that telemetry can be compiled out

Add tasks/scripts/verify-telemetry-compiled-out.sh, which inspects a built
binary for telemetry markers (the telemetry endpoint host and client ID) that
exist only when emission code is compiled in. The rust:verify:telemetry-off
mise task builds the gateway with default features (positive control: markers
must be present, so the absent checks can never be silently vacuous) and with
--no-default-features (markers must be absent), and checks the
--no-default-features sandbox binary as well.

Wire the task into the Rust branch-checks job so a regression that reintroduces
telemetry code into a --no-default-features build fails CI.

Signed-off-by: Russell Bryant <russell.bryant@gmail.com>

---------

Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
2026-06-09 14:19:26 -07:00
Zygmunt Krynicki 4025894a94 chore(snap): remove early snap packaging (#1648) 2026-06-08 22:09:09 -07:00
Roshni Malani 586c385bdf chore(k8s): use upstream agent-sandbox manifest in CI/e2e (#1657)
Drop the vendored deploy/kube/manifests/agent-sandbox.yaml. CI and e2e
scripts now apply manifest.yaml directly from
github.com/kubernetes-sigs/agent-sandbox releases, pinned via
AGENT_SANDBOX_VERSION env (default v0.4.6, overridable) so internal runs
stay reproducible.

Public docs and the helm chart README continue to reference
/releases/latest/download/ — no OpenShell/agent-sandbox support matrix
exists yet (see #1649).

Adds an air-gap note in docs/kubernetes/setup.mdx enumerating the
manifest and image operators need to mirror to an internal registry.

Signed-off-by: Roshni Malani <rmalani@nvidia.com>
2026-06-04 11:35:06 -07:00
Piotr Mlocek 3d441e73ac fix(config): reject unknown fields in nested gateway config tables (#1666)
* fix(config): reject unknown fields in nested gateway config tables

The gateway TOML loader silently ignored keys placed under the wrong
table header. PR #1661 fixed one instance of this (supervisor_image
under [openshell.gateway.gateway_jwt]) but the root cause remained: the
nested gateway config tables did not deny unknown fields, so a misplaced
key was accepted and dropped instead of erroring.

Concretely, tasks/scripts/gateway.sh emitted `sandbox_namespace` right
after the [openshell.gateway.gateway_jwt] heredoc, so it landed inside
the gateway_jwt table rather than [openshell.gateway]. The k8s driver
already receives the namespace via [openshell.drivers.kubernetes], so
the stray line was dead config that parsed without complaint.

Changes:
- Add #[serde(deny_unknown_fields)] to the nested gateway config tables
  that are part of the config-file parse tree: TlsConfig, OidcConfig,
  MtlsAuthConfig, GatewayAuthConfig, GatewayJwtConfig.
- Remove the misplaced sandbox_namespace line from gateway.sh.
- Drop the unused Serialize/Deserialize derives from Config and
  ServiceRoutingConfig (see below).
- Add a regression test asserting a key under the wrong nested table is
  rejected.
2026-06-01 17:12:42 -07:00
Jesse Jaggars 019a986e6d fix(gateway): place supervisor_image under podman driver TOML table (#1661)
The gateway.sh script appended supervisor_image after the
[openshell.gateway.gateway_jwt] table header, so TOML parsed it as a
gateway_jwt field. The Podman driver never saw the override and fell
back to the default ghcr.io/nvidia/openshell/supervisor:latest.
Move supervisor_image into [openshell.drivers.podman] where the driver
config deserializer expects it.
2026-06-01 15:25:22 -07:00
Taylor Mutch 7cea9d9b2b fix(gateway): align package TLS bootstrap path (#1601)
* fix(gateway): align package TLS bootstrap path

Closes #1593

Default package-managed gateway services to a stable local TLS directory and use that same value for certificate generation and runtime startup.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* test(packaging): validate package asset paths exist

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* ci(e2e): pin mise in kubernetes job

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
2026-06-01 14:56:10 -05:00
Alexander Watson e98ea3ee93 feat(policy): add agentic approval loop (#1528) 2026-05-29 21:11:31 -07:00
Naveen Malik f1ed347a40 fix(driver-podman): bind gateway to 0.0.0.0 in rootless mode (#1623)
Rootless Podman sandbox containers reach the host through pasta's local
connection bypass, which translates L2 frames to L4 host sockets. The
dev gateway script binds to 127.0.0.1 by default, which is not routable
through pasta. Auto-detect rootless mode and bind to 0.0.0.0 so sandbox
containers can connect to the gateway.

- Auto-detect rootless Podman in gateway.sh and export
  OPENSHELL_BIND_ADDRESS=0.0.0.0 when not explicitly set
- Add e2e:podman:rootless mise task and CI matrix entry to validate
  rootless Podman networking end-to-end
- CI creates a non-root user inside the privileged container to trigger
  Podman's rootless code paths (pasta, user namespace isolation)

Signed-off-by: Naveen Malik <nmalik@redhat.com>
2026-05-29 13:03:26 -05:00
krishicks fa84e437ab fix(gateway): configure local dev auth (#1575)
This makes it so you can run the dev gateway and sandbox with:

```
mise run gateway
# in another shell
mise run sandbox
```

Signed-off-by: Kris Hicks <khicks@nvidia.com>
2026-05-26 11:52:36 -07:00
Mesut Oezdil 88508a01be fix(scripts): replace mapfile with bash 3.2-compatible read loop in helm-k3s-local (#1539)
macOS ships bash 3.2 which lacks mapfile/readarray. Replace all three
occurrences in configure_ghcr_credentials, cluster_has_image, and
cluster_image_platform with a portable while-read loop, consistent
with the fix applied to docker-build-image.sh in #1334.
2026-05-26 09:06:46 -07:00
Taylor Mutch 7d38aa8b8e fix(homebrew): repair local driver bootstrap state (#1527)
* fix(homebrew): repair local driver bootstrap state

* fix(bootstrap): satisfy default SAN doc lint
2026-05-22 12:12:39 -07:00
Piotr Mlocek 0dc08a185e fix(release): build host Linux binaries with glibc floor (#1490) 2026-05-22 12:12:03 -07:00
Drew Newberry 603b3e27fa docs: update NemoClaw/OpenClaw references (#1529) 2026-05-22 10:31:59 -07:00
Taylor Mutch a3b16c18ab feat(auth): per-sandbox authentication to gateway (#1404) 2026-05-21 17:58:55 -07:00
Florent BENOIT b93a3d8661 fix(scripts): use portable lowercase in normalize_bool for Bash 3.2 (#1493) 2026-05-21 07:15:07 -07:00
John T. Myers dbba580e84 fix(security): refresh CI and gateway image dependencies (#1432)
Refresh the CI image tool pins so Go-built tools are rebuilt with patched Go releases and move the sandbox Python runtime to 3.14.5.

Rebase the gateway runtime to a pinned distroless Debian 13 image with glibc 2.41-12+deb13u3 while preserving the existing UID/GID 1000 runtime identity for upgrade compatibility. Update rustls-webpki to 0.103.13 and clarify Linux k3d guidance now that k3d is not installed through mise on Linux.

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
2026-05-18 14:25:08 -07:00
Drew Newberry f257ed0193 refactor(packaging): rely on gateway runtime defaults (#1415)
* fix(packaging): use gateway TOML config in packages

* refactor(packaging): rely on gateway runtime defaults
2026-05-18 14:13:19 -07:00
Evan Lezar 555680cefc fix(docker): fall back to host arch for local builds (#1420)
(cherry picked from commit 392ab51f58fc18e03e721cdfa21f666091c67b80)

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-05-18 14:24:36 -05:00
Drew Newberry f819f7dcb5 fix(vm): restore sandboxes after gateway restart (#1407) 2026-05-15 15:08:59 -07:00
Drew Newberry 910d3f09f2 feat(vm): boot sandboxes from ext4 root disks (#1263)
* feat(vm): boot sandboxes from ext4 root disks

* feat(vm): boot sandboxes with writable overlay disk

* fix(sandbox): improve sandbox provisioning behavior

* fix(vm): repair sandbox image provisioning

* fix(vm): clean up rebase fallout

* perf(vm): cache empty overlay images

* perf(vm): skip repeated sandbox ownership repair

* fix(vm): address lifecycle review issues

* fix(vm): repair gateway vm startup

* wip
2026-05-15 13:23:10 -07:00
Taylor MutchandDrew Newberry b61a98dbad feat(gateway): add TOML configuration file (RFC 0003) (#1317)
* feat(gateway): add TOML configuration file (RFC 0003)

Introduces an opt-in --config / OPENSHELL_GATEWAY_CONFIG flag that loads a
TOML file with gateway-wide settings and per-driver tables. Source
precedence is CLI > env > file > built-in default, implemented via clap's
ValueSource so existing flags and env vars keep their priority.

Driver crates (kubernetes, docker, podman, vm) now derive Deserialize on
their config structs. SupervisorSideloadMethod gains Deserialize with
kebab-case rename. A per-driver inheritance allowlist on the loader side
overlays [openshell.gateway] shared defaults (default_image,
supervisor_image, image_pull_policy, guest_tls_*, ssh_handshake_skew_secs,
client_tls_secret_name, host_gateway_ip, enable_user_namespaces) onto
each [openshell.drivers.<name>] table before deserialization.

The Helm chart renders a new gateway-config ConfigMap and mounts it at
/etc/openshell/gateway.toml. The migrated OPENSHELL_* env entries are
dropped from the StatefulSet — only the Secret-backed
OPENSHELL_SSH_HANDSHAKE_SECRET remains. database_url stays on --db-url.

Adds examples/gateway/gateway.example.toml and updates architecture/gateway.md
with the source precedence and inheritance rules.

* docs(gateway): drop ssh_handshake_skew_secs and ssh_handshake_secret from examples

Both fields are scheduled for removal. Remove the example values and the
env-only note so the gateway.toml example and the architecture doc stop
recommending settings that will not exist much longer.

* docs(rfc): correct OPENSHELL_CONFIG to OPENSHELL_GATEWAY_CONFIG in RFC 0003

* docs(gateway): add per-driver TOML example configurations

Adds focused single-driver examples next to the comprehensive
gateway.example.toml: kubernetes, docker, podman, and microvm. Each one
demonstrates the realistic settings for that driver plus how shared
[openshell.gateway] defaults inherit into the driver table.

A new unit test (`checked_in_examples_parse`) loads every example through
the config_file loader so schema drift fails CI rather than silently
shipping a broken example.

* refactor(gateway): drop image_pull_policy from shared inheritance

Kubernetes and Podman use mutually-incompatible vocabularies for the same
TOML key:

  - Kubernetes: `Always | IfNotPresent | Never` (free-form string passed
    verbatim to the K8s API).
  - Podman: `always | missing | never | newer` (strict lowercase enum
    deserialised into `ImagePullPolicy`).

No value means the same thing in both drivers. Sharing the key at
`[openshell.gateway]` scope and inheriting it into every active driver's
table meant any value safe for one driver was either wrong or silently
dropped for the other (`IfNotPresent` → `ImagePullPolicy::Missing` after
`.unwrap_or_default()`). Operators run one driver per gateway, so the
"shared default" never pays for itself.

Make `image_pull_policy` driver-local:

  - Remove the field from `GatewayFileSection` and from
    `inheritable_keys()` for both Kubernetes and Podman.
  - Drop the file→`RunArgs` merge for the gateway-scope key.
  - Stop unconditionally clobbering the driver value with
    `config.sandbox_image_pull_policy` in the runtime wiring — only apply
    the CLI/env override when it was set (and, for Podman, only when it
    parses into the lowercase enum).
  - Move the key under `[openshell.drivers.kubernetes]` and
    `[openshell.drivers.podman]` in every example, the RFC, the
    architecture doc, and the Helm-rendered gateway ConfigMap.

The supervisor pull policy follows the same shape: it is K8s-only and
moves into `[openshell.drivers.kubernetes]` alongside `image_pull_policy`
in the Helm template.

* fix(gateway): address review feedback on TOML configuration

Resolves the P1 and P2 issues raised in PR #1317:

- Helm gateway ConfigMap moves `grpc_endpoint` under
  `[openshell.drivers.kubernetes]` so the default install no longer fails
  the gateway's `deny_unknown_fields` schema check.
- `kubernetes_config_from_file` and `podman_config_from_file` only let
  the gateway-wide CLI/env `grpc_endpoint` overwrite the driver-table
  value when it was actually supplied, preserving file-only configs.
- Kubernetes driver default `image_pull_policy` is now empty (was Podman
  vocabulary "missing"), so default deployments let the Kubernetes API
  apply its own policy instead of being rejected.
- New `disable_tls` gateway field plumbs `.Values.server.disableTls`
  through the TOML ConfigMap instead of relying on env vars dropped from
  the StatefulSet.
- StatefulSet pod template now carries a `checksum/gateway-config`
  annotation so `helm upgrade` rolls pods when the ConfigMap changes.
- Auxiliary listener resolution preserves the full `SocketAddr` from
  `health_bind_address` / `metrics_bind_address`, so a loopback-pinned
  health port is not silently relocated onto the public bind address.
- `ssh_session_ttl_secs` from the file is now applied to `Config` (it
  was previously accepted by the loader but never read).

New regression coverage: cli-level merge tests for the new fields plus
helm-unittest assertions for the ConfigMap shape, checksum annotation,
and `disable_tls` rendering.

* docs(gateway): consolidate gateway TOML examples into docs reference

Replaces the per-driver example files under examples/gateway/ with a
single published reference page at docs/reference/gateway-config.mdx
covering source precedence, layout, the full example, and the four
per-driver examples (Kubernetes, Docker, Podman, microVM). Drew flagged
during PR #1317 review that the examples belong with the user-facing
docs rather than in a sibling examples/ directory.

The cross-references in architecture/gateway.md and RFC 0003 are updated
to point at the new docs page; the round-trip test in config_file.rs is
removed (schema coverage stays on the inline parses_full_example test
and per-field merge tests — doc-snippet drift belongs in a separate
docs-lint, not in a cross-tree Rust unit test).

* refactor(core): move DEFAULT_K8S_NAMESPACE into K8s driver

The constant is Kubernetes-specific (used only by KubernetesComputeConfig's
Default impl) and does not belong in openshell-core. Relocate it to the
driver crate that owns the K8s vocabulary; openshell-core retains only
truly cross-cutting defaults.

* refactor(core): move Podman bridge default into Podman driver

DEFAULT_NETWORK_NAME is Podman vocabulary, consumed only by the Podman
driver. Also drops the unused DEFAULT_IMAGE_PULL_POLICY constant.

* docs(auth): scrub remaining SSH handshake secret references

Sweeps the trailing mentions left after the rebase: the gateway
config-file module doc, the Helm gateway-config ConfigMap header,
the gateway-config.mdx env-only note, and the RPM systemd unit
comment for init-gateway-env.sh.

* docs(gateway): clarify OPENSHELL_GRPC_ENDPOINT applies to all drivers

The previous comment implied the callback endpoint was Kubernetes-only,
but the value is propagated to every compute driver (Kubernetes, Docker,
Podman, VM) and must be reachable from wherever the sandbox runs.

* refactor(gateway): move driver options into config (#1394)

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(e2e): regenerate gateway config via TOML for docker + podman harnesses

The gateway CLI flags moved into TOML config tables in 560550d2 (#1394),
which made every existing e2e/with-{docker,podman}-gateway.sh invocation
fail with "unexpected argument '--sandbox-namespace'" (and a long tail
of similar driver-specific options) before the gateway could even bind.

Replace the obsolete CLI flags with a synthesized
`[openshell.drivers.<driver>]` table written to `${STATE_DIR}/gateway.toml`
and passed via the new `--config` flag. Only the gateway-wide flags
that survived 560550d2 (bind-address, port, drivers, db-url, tls-*,
disable-tls, log-level, health-port) stay on the command line.

Both scripts get a small `toml_string` helper to properly TOML-quote
the values (the previous `%q` printf format produced bash-escape, not
TOML-escape). The Docker harness also corrects two field names that
diverged from the driver schema: `docker_network_name` →
`network_name`, and the supervisor binary/image plumbing now reads
through to `supervisor_bin` / `supervisor_image` in the same table.

The Podman harness drops `--ssh-gateway-port` (deleted in 560550d2 —
gRPC + SSH are multiplexed on the same port now) and substitutes
`network_name` + `gateway_port` for the obsolete `--sandbox-namespace`
(which the Podman driver never had as a typed field).

* fix(core): swap bind-only 0.0.0.0 SSH gateway host for cluster URL host

CLI's resolve_ssh_gateway treated 0.0.0.0 as a loopback "keep as-is"
when the cluster URL was also loopback, so the SSH proxy connected to
0.0.0.0:port. The unspecified address is never a valid connect target
and is not present in any TLS cert SAN, which produced BadCertificate
TLS handshake failures during `openshell sandbox create -- ...` in
docker/podman e2e (e.g. bypass_detection).

Resolution: when the server returns 0.0.0.0 or :: as the gateway host
and both endpoints are loopback, fall back to the cluster URL's host
(which the CLI is already using to reach the gateway, so it must
resolve and match the cert).

* fix(e2e): repair podman harness on macOS

Podman 5.x with the applehv/libkrun provider no longer creates the legacy
~/.local/share/containers/podman/machine/podman.sock symlink, and
`podman system service` is a Linux-only subcommand — the macOS client
delegates the API service to the VM. Both assumptions in the harness
were stale, so the script tried to start a temporary service that podman
rejected with "unknown flag: --time".

- Discover the macOS socket via `podman machine inspect` instead of the
  hardcoded path.
- On Darwin, fail fast with a "start podman machine" message rather than
  attempting the Linux-only `podman system service` fallback.
- Write socket_path into [openshell.drivers.podman] so the in-process
  driver picks up the discovered socket; the driver reads TOML only
  after the config refactor (560550d2), so OPENSHELL_PODMAN_SOCKET alone
  was no longer enough.

* fix(server): use clone_from for TLS client CA assignment

clippy 1.95.0 rejects assigning the result of `Clone::clone()` to an
existing variable under `-D warnings` (`assigning_clones`). Switch to
`clone_from(&...)` to satisfy the lint and avoid the redundant
allocation.

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
2026-05-15 12:43:48 -07:00
Florent BENOIT a1fb9bd954 fix(scripts): replace mapfile with bash 3.2-compatible read loop (#1334)
macOS ships bash 3.2 which lacks mapfile (introduced in bash 4.0),
causing docker-build-image.sh and stage-prebuilt-binaries.sh to fail
on stock macOS developer machines.
2026-05-15 13:38:25 -05:00
Taylor Mutch 7a0c444445 refactor!(auth): drop SSH handshake secret (#1274)
* refactor!(auth): drop SSH handshake secret in favor of mTLS

The OPENSHELL_SSH_HANDSHAKE_SECRET / x-sandbox-secret mechanism was
misnamed: it does not authenticate SSH (which flows over the
RelayStream gRPC RPC and is gated by mTLS plus supervisor Unix-socket
permissions). It only gated a small set of sandbox-to-gateway
control-plane RPCs, and production deployments already enforce mTLS
on that channel — so the shared secret was redundant.

Replace the secret check with an mTLS-presence marker. Sandbox-class
methods (ReportPolicyStatus, PushSandboxLogs,
GetSandboxProviderEnvironment, SubmitPolicyAnalysis, GetSandboxConfig,
GetInferenceBundle) accept callers without a Bearer token; the gRPC
mTLS handshake is the trust boundary. Dual-auth methods treat
Bearer-present as full-scope CLI access and Bearer-absent as
sandbox-restricted scope via validate_sandbox_caller_update.

Drops the secret from all drivers (K8s, Podman, VM), the sandbox gRPC
interceptor, the Helm chart (values + pre-install hook + StatefulSet
env), the RPM bootstrap script, the man pages, and the
debug-openshell-cluster skill. Also removes the never-read
ssh_handshake_skew_secs flag and config field.

BREAKING CHANGE: --ssh-handshake-secret / OPENSHELL_SSH_HANDSHAKE_SECRET
and --ssh-handshake-skew-secs / OPENSHELL_SSH_HANDSHAKE_SKEW_SECS are
removed from the gateway, sandbox, and all driver binaries. The
openshell-ssh-handshake K8s Secret is no longer managed by the chart;
operators may delete the orphan. Deployments using
--disable-gateway-auth must enforce caller authentication at the
fronting proxy, since the gateway no longer validates a per-request
secret on sandbox-class methods.

Refs OS-174.

* docs(auth): scrub residual SSH handshake secret references

Sweep across docs, e2e scripts, the Podman driver README/NETWORKING
notes, the RPM/Helm/setup guides, the gateway man page, and RFC 0003
to remove instructions and examples that still referenced
OPENSHELL_SSH_HANDSHAKE_SECRET / --ssh-handshake-secret /
ssh_handshake_skew_secs. The mechanism is gone; nothing should still
suggest setting it. Negative-assertion regression tests are kept so
the env var cannot silently be re-introduced.

* test(drivers): drop SSH handshake secret negative-assertion tests

The supporting code, env vars, CLI flags, and config plumbing are
gone — these tests asserted absence of strings that no longer have
any path to being set. Remove the guards from the Docker, Podman,
Kubernetes, and VM driver test modules.
2026-05-14 13:14:30 -07:00
Drew Newberry 0471c6d2ab fix(gateway): keep vm driver opt-in (#1375) 2026-05-13 22:32:58 -07:00
Taylor MutchandDrew Newberry ba77967fc4 refactor(docker): split gateway/supervisor Dockerfiles and use native rust builds (#1316)
* wip

* refactor(docker): use native rust builds for split gateway/supervisor images

Drop the in-Docker BUILD_FROM_SOURCE path so both images consume only
prebuilt binaries staged natively via tasks/scripts/stage-prebuilt-binaries.sh.
This mirrors what CI does and reuses the host's cargo target cache and
sccache across rebuilds.

- Dockerfile.gateway: nvcr.io/nvidia/distroless/cc:v4.0.4 base (the 4.0.0
  tag does not exist on nvcr.io; the registry uses a v prefix). GNU-linked
  binary copied to /usr/local/bin.
- Dockerfile.supervisor: scratch base, static musl binary. Static linkage
  lets the image stay scratch while still being executable as a Kubernetes
  init container.
- skaffold.yaml: each artifact invokes tasks/scripts/docker-build-image.sh,
  which stages the binary natively (cargo / cargo-zigbuild) and then builds
  the image. Drops the cross-build.sh dependency from the supervisor build.
- seccomp.rs: add a local SYS_kexec_file_load constant for musl/aarch64.
  libc 0.2.185 omits the symbol from its musl/aarch64 bindings, so the
  supervisor's seccomp filter previously failed to compile for that target.
- architecture/build.md: describe the native-first pipeline and per-image
  runtime choices.

Local validation: gateway image 101MB (was 194MB), supervisor image 21.7MB.
helm:skaffold:run deploys cleanly; the static musl supervisor binary runs
correctly in a non-glibc agent container.

* refactor(docker): tighten binary perms via --chown + 0550

Replace `COPY --chmod=755` with `COPY --chown=<user> --chmod=0550` in
the gateway and supervisor Dockerfiles. The binary is no longer
world-readable or world-executable; ownership is pinned to the runtime
user.

- Gateway uses `--chown=nvs:nvs` + `USER nvs:nvs`, matching the only
  non-root user defined in `nvcr.io/nvidia/distroless/cc` (UID 1000) and
  the Helm chart's `securityContext.runAsUser: 1000`, which overrides
  the Dockerfile USER at runtime.
- Supervisor uses numeric `--chown=65534:65534` because the scratch base
  has no `/etc/passwd` for name resolution. The supervisor image is
  only consumed by the init-container copy-self path; the destination
  pod's runAsUser governs execute access.

Validated by deploying to a local k3d cluster via `helm:skaffold:run`
and confirming the gateway StatefulSet reaches 1/1 Running.

* ci(gpu): repoint GPU probe image lookup at Dockerfile.gateway

The previous awk parsed `FROM <image> AS gateway` from the now-deleted
`Dockerfile.images`. The new `Dockerfile.gateway` uses an ARG with a
default (`ARG GATEWAY_BASE_IMAGE=nvcr.io/nvidia/distroless/cc:v4.0.4`)
and `FROM ${GATEWAY_BASE_IMAGE} AS gateway`, so the old script returns
nothing.

Parse the ARG default value directly so the GPU prerequisites check
keeps using the gateway base image as a `nvidia-smi` probe target.

* ci(gpu): pin GPU probe to nvcr.io/nvidia/base/ubuntu:noble

The previous probe parsed the gateway base image out of the Dockerfile,
relying on the fact that the gateway ran on `nvcr.io/nvidia/base/ubuntu`
and that NVIDIA Container Toolkit CDI injection would populate
`nvidia-smi` and the supporting libs at runtime. The new gateway base
(`nvcr.io/nvidia/distroless/cc`) lacks `ldconfig`, a populated
`/usr/bin`, and the broader filesystem layout CDI injection assumes,
so it cannot serve as a GPU probe.

Pin the probe image explicitly to the NVIDIA-managed Ubuntu base. The
probe is independent of the gateway runtime and survives future base
swaps.

* fix(e2e-gpu): pass GPU probe image via env, drop Dockerfile.images parse

The Rust e2e test `gpu_request_for_each_discovered_device_matches_plain_container`
was still parsing the deleted `Dockerfile.images` to derive its GPU probe
image, panicking with `No such file or directory` after this PR's
Dockerfile split.

Move the probe image to a single source of truth in the workflow
(`OPENSHELL_E2E_GPU_PROBE_IMAGE` env at the job level) and require the
e2e test to read it from there. No silent codebase default — the test
panics with a pointer at the workflow if the env is missing, so it
fails loudly rather than drifting from CI.

The prereq probe step in the workflow now consumes the same env, so
the probe image is declared exactly once.

* fix(docker): keep supervisor binary root-owned for rootless Podman

The Podman driver mounts the supervisor image read-only into the sandbox
container at /opt/openshell/bin and runs that container as UID 0, but
deliberately drops DAC_OVERRIDE for hardening (container.rs:419). With
--chown=65534:65534 --chmod=0550 the binary was r-xr-x--- owned by UID
65534, so the container's UID 0 fell into "other" with no read or exec
access and the supervisor crashed on start (ContainerExited code 1).

Docker and Kubernetes both retain DAC_OVERRIDE, so root could still
exec the file — which is why this regression only surfaced in the
Podman e2e job.

Drop --chown from the supervisor COPY so the binary stays root-owned.
Keep --chmod=0550: the security win was dropping world-execute, not
changing the owner. The chown bought nothing here because the container
is always UID 0 regardless of driver, but it actively broke the only
driver that drops DAC_OVERRIDE.

---------

Co-authored-by: Drew Newberry <anewberry@nvidia.com>
2026-05-12 16:05:49 -07:00
Florent BENOIT 764d930680 fix(vm): use bash 3.2-safe empty array expansion in supervisor build script (#1311)
macOS ships bash 3.2, which treats empty array expansion ("${arr[@]}")
as an unbound variable error under set -u. Unlike set -e, this error
is not caught by if-constructs — it kills the shell immediately with
no output, causing mise run vm:supervisor to fail silently.

Use the ${arr[@]+"${arr[@]}"} idiom so the script works on both
bash 3.2 (macOS default) and bash 4+.

Signed-off-by: Florent Benoit <fbenoit@redhat.com>
2026-05-11 15:50:30 -05:00
Zygmunt Krynicki dfd47683e7 (feat) early snap support (#1238) 2026-05-10 21:12:30 -07:00
Russell Bryant 57a80ed2ad fix(gateway): update Podman supervisor build task name (#1288)
The Podman path in the dev gateway script references the
build:docker:supervisor-sideload mise task, which was removed in
d8b84773 ("feat(rpm): add RPM packaging with Packit/COPR and GHA
release publishing (#1126)"). That commit consolidated the sideload
and standalone supervisor image builds into a single
build:docker:supervisor task.

Update the reference so `mise run gateway` works with the Podman
compute driver.

Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
2026-05-09 00:17:12 -05:00
Taylor Mutch 316c788eac fix(helm): derive grpcEndpoint from chart context (#1241)
* fix(helm): derive grpcEndpoint from chart context

The chart hardcoded server.grpcEndpoint to
https://openshell.openshell.svc.cluster.local:8080, which only matched the
in-cluster Service DNS for the standard release name and namespace. A new
helper now builds <scheme>://<fullname>.<namespace>.svc.cluster.local:<port>
from chart context, picking the scheme from server.disableTls. An explicit
server.grpcEndpoint override is passed through verbatim.

* chore(scripts): validate k3d cluster name length early

helm-k3s-local.sh derives the cluster name from the current branch suffix.
Long branch names produced names exceeding k3d's 32-char cap and failed
deep inside k3d cluster create with a confusing validation error. cmd_create
now bails out before invoking docker/k3d with a copy-pasteable
HELM_K3S_CLUSTER_NAME override hint. Status, start, stop, delete, and help
remain unaffected so an over-long derived name does not block diagnostics.
2026-05-08 14:10:19 -07:00
Drew Newberry daa2a362d5 fix(packaging): enable mTLS for local packages (#1271) 2026-05-08 13:02:12 -07:00
Drew Newberry 028763d4db refactor(vm): remove legacy openshell-vm crate (#1239)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-05-07 10:50:20 -07:00
Drew Newberry 70a0f6c547 refactor(cli): remove gateway lifecycle management (#1221) 2026-05-07 09:54:13 -07:00
Drew Newberry 938005584a fix(packaging): let gateway auto-detect package driver (#1236)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
2026-05-07 09:13:43 -07:00
Drew Newberry fe41d679a4 fix(installer): install release formula from Homebrew tap (#1222) 2026-05-06 18:42:46 -07:00
Drew Newberry 5949b2fa34 feat(installer): support macOS dev installs (#1183) 2026-05-06 15:50:07 -07:00
Drew Newberry da26ed3212 fix(release): stabilize dev build packaging (#1213) 2026-05-06 13:00:44 -07:00
Drew Newberry 4d388d2677 ci(vm): cleanup vm build infra (#1186) 2026-05-06 08:58:02 -07:00
Adam Miller d8b84773ca feat(rpm): add RPM packaging with Packit/COPR and GHA release publishing (#1126) 2026-05-05 15:42:00 -07:00
Taylor Mutch 5116cc27b7 feat(helm): add kubernetes local-dev environment (#1158) 2026-05-05 13:42:21 -07:00