Remove the production-crate dev-settings feature so e2e tests build the same gateway, CLI, and supervisor binaries that we ship. Move settings coverage from dummy test-only keys to the production ocsf_json_enabled setting.
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
* ci(docs): add docs website automation
* ci(docs): harden sync-docs inputs and add script tests
Route free-form workflow_dispatch inputs through env vars to avoid shell
injection, switch uv install to the approved setup-uv action, and add a
pytest suite for sync_docs_website.py wired into the test task.
* ci(stale): add stale issue and PR workflow
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* fix: remove draft-pr from exemption
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Switch the snapcraft build from 'source: .' (which copies the full
20 GB repo including 18 GB of target/ and build artifacts into LXD)
to 'source: ./snap/prebuilt' (~130 MB of only the binaries and
metadata that snapcraft needs).
The snap-package workflow now downloads the same cli, gateway, and
supervisor binaries that other packaging jobs (deb, rpm, tarballs)
use. A new 'Prepare snap build directory' step assembles snap/prebuilt/
with the 3 binaries, gateway wrapper script, LICENSE, README.md, and
a precomputed version string — everything snapcraft needs without
any source tree access.
This addresses CI/CD Improvements (issue #1361) by reducing data
transferred into the LXD sandbox by ~150x and eliminating the need
for snapcraft to traverse the entire repository tree.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
* feat(telemetry): add build-time option to compile out telemetry
Gate anonymous telemetry emission behind a default-on `telemetry` Cargo
feature in openshell-core. The data model (enums, validation, emit_*/enabled*
signatures) stays always-compiled, while the endpoint, HTTP client, queue, and
emission code are feature-gated. With the feature off, enabled() returns false
and emit_* are no-ops, so dependent crates compile unchanged and no telemetry
endpoint, HTTP client, or emission code is included in the binary.
chrono and reqwest become optional dependencies of openshell-core, dropped from
its dependency graph when telemetry is disabled.
Thread the switch through the workspace: every crate depends on openshell-core
with default-features = false, and the default-on `telemetry` passthrough lives
on the binary crates that emit or collect telemetry (openshell-server,
openshell-sandbox, openshell-driver-vm). In-process drivers inherit it via
resolver v2 feature unification.
Build a telemetry-free binary with, e.g.:
cargo build --release -p openshell-server --no-default-features
The runtime OPENSHELL_TELEMETRY_ENABLED switch is unchanged for default builds.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
* ci(telemetry): guard that telemetry can be compiled out
Add tasks/scripts/verify-telemetry-compiled-out.sh, which inspects a built
binary for telemetry markers (the telemetry endpoint host and client ID) that
exist only when emission code is compiled in. The rust:verify:telemetry-off
mise task builds the gateway with default features (positive control: markers
must be present, so the absent checks can never be silently vacuous) and with
--no-default-features (markers must be absent), and checks the
--no-default-features sandbox binary as well.
Wire the task into the Rust branch-checks job so a regression that reintroduces
telemetry code into a --no-default-features build fails CI.
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
---------
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
The Ubuntu Snap canary downloads its artifact from a different workflow run
(the triggering Release Dev run) via run-id. Cross-run downloads require
authentication, so pass github.token to actions/download-artifact.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
Install the Snap built by the triggering Release Dev workflow by setting
merge-multiple: true on the artifact download. actions/download-artifact
otherwise extracts each artifact into its own subdirectory, leaving the
package at release/snap-linux-amd64/*.snap, so the install glob
./release/*.snap matched nothing. Merging flattens the artifact's contents
directly into release/ where the dangerous local snap install expects it.
Harden the Snap canary setup by enabling snapd.socket, waiting for snap
seeding (snap wait system seed.loaded), and running every step with strict
shell options (set -euo pipefail) so failures surface immediately.
Register the snapped gateway with the CLI as the documented local plaintext
snap-docker gateway, and print version and snap services, before running
openshell status so the canary verifies a configured and reachable gateway
instead of only the install.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
The RPM canary needs to exercise the install.sh user-service path, but a GitHub
Actions job container does not boot with systemd as PID 1. The Fedora RPM
canary needs to exercise the install.sh user-service path, but a GitHub Actions
job container does not boot with systemd as PID 1. This means the Fedora RPM
canary was incomplete as compared to the others.
With this change, we run Fedora as a nested privileged systemd container
instead, wait for systemd to become reachable, then start the root user manager
so systemctl --user works for the RPM gateway unit, achieving parity with the
other canary tests.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* ci(kubernetes): pin mise in e2e workflow
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* ci(kubernetes): mirror postgres image for ha e2e
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* ci(kubernetes): reuse e2e workflow for ha
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
release-helm and tag-ghcr-release now depend on the release job.
This is to prevent a GHCR image or helm chart from being published when some
other aspect of the release fails.
Signed-off-by: Kris Hicks <khicks@nvidia.com>
* fix(gateway): align package TLS bootstrap path
Closes#1593
Default package-managed gateway services to a stable local TLS directory and use that same value for certificate generation and runtime startup.
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* test(packaging): validate package asset paths exist
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
* ci(e2e): pin mise in kubernetes job
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
---------
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Rootless Podman sandbox containers reach the host through pasta's local
connection bypass, which translates L2 frames to L4 host sockets. The
dev gateway script binds to 127.0.0.1 by default, which is not routable
through pasta. Auto-detect rootless mode and bind to 0.0.0.0 so sandbox
containers can connect to the gateway.
- Auto-detect rootless Podman in gateway.sh and export
OPENSHELL_BIND_ADDRESS=0.0.0.0 when not explicitly set
- Add e2e:podman:rootless mise task and CI matrix entry to validate
rootless Podman networking end-to-end
- CI creates a non-root user inside the privileged container to trigger
Podman's rootless code paths (pasta, user namespace isolation)
Signed-off-by: Naveen Malik <nmalik@redhat.com>
* ci: pin azure/setup-helm and helm/kind-action to commit SHAs
* chore(python): add py.typed marker for PEP 561 compliance
* ci: use full semver in pinned action version comments
Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
---------
Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
- Add publish-manifest input to docker-build.yml (default true); single-arch
branch callers set it false so the merge job is skipped and the shared
bare :SHA tag in GHCR is never written by branch workflows
- branch-kubernetes-e2e: retag :SHA-amd64 to :SHA before kind load so
Helm's image.tag matches what is loaded in kind containerd
- branch-e2e: pass image-tag as :SHA-arm64 to e2e-test so the arch-specific
GHCR tag is used directly without depending on the bare tag
- bare :SHA in GHCR is now written only by test-gpu.yml (multi-arch build),
eliminating the last-writer-wins race across concurrent workflows
The VM driver artifacts were already attested. Expand the attest step
to cover CLI, gateway, supervisor, deb, rpm, and wheel artifacts so all
release binaries can be verified with gh attestation verify.
Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
Adds a helm:test mise task that installs the helm-unittest plugin if
not present and runs chart unit tests under deploy/helm/openshell.
Installs the plugin into Dockerfile.ci so CI runs do not need to
download it each time.
Closes#1281
Signed-off-by: Mesut Oezdil <versusfinem@gmail.com>
* wip
* refactor(docker): use native rust builds for split gateway/supervisor images
Drop the in-Docker BUILD_FROM_SOURCE path so both images consume only
prebuilt binaries staged natively via tasks/scripts/stage-prebuilt-binaries.sh.
This mirrors what CI does and reuses the host's cargo target cache and
sccache across rebuilds.
- Dockerfile.gateway: nvcr.io/nvidia/distroless/cc:v4.0.4 base (the 4.0.0
tag does not exist on nvcr.io; the registry uses a v prefix). GNU-linked
binary copied to /usr/local/bin.
- Dockerfile.supervisor: scratch base, static musl binary. Static linkage
lets the image stay scratch while still being executable as a Kubernetes
init container.
- skaffold.yaml: each artifact invokes tasks/scripts/docker-build-image.sh,
which stages the binary natively (cargo / cargo-zigbuild) and then builds
the image. Drops the cross-build.sh dependency from the supervisor build.
- seccomp.rs: add a local SYS_kexec_file_load constant for musl/aarch64.
libc 0.2.185 omits the symbol from its musl/aarch64 bindings, so the
supervisor's seccomp filter previously failed to compile for that target.
- architecture/build.md: describe the native-first pipeline and per-image
runtime choices.
Local validation: gateway image 101MB (was 194MB), supervisor image 21.7MB.
helm:skaffold:run deploys cleanly; the static musl supervisor binary runs
correctly in a non-glibc agent container.
* refactor(docker): tighten binary perms via --chown + 0550
Replace `COPY --chmod=755` with `COPY --chown=<user> --chmod=0550` in
the gateway and supervisor Dockerfiles. The binary is no longer
world-readable or world-executable; ownership is pinned to the runtime
user.
- Gateway uses `--chown=nvs:nvs` + `USER nvs:nvs`, matching the only
non-root user defined in `nvcr.io/nvidia/distroless/cc` (UID 1000) and
the Helm chart's `securityContext.runAsUser: 1000`, which overrides
the Dockerfile USER at runtime.
- Supervisor uses numeric `--chown=65534:65534` because the scratch base
has no `/etc/passwd` for name resolution. The supervisor image is
only consumed by the init-container copy-self path; the destination
pod's runAsUser governs execute access.
Validated by deploying to a local k3d cluster via `helm:skaffold:run`
and confirming the gateway StatefulSet reaches 1/1 Running.
* ci(gpu): repoint GPU probe image lookup at Dockerfile.gateway
The previous awk parsed `FROM <image> AS gateway` from the now-deleted
`Dockerfile.images`. The new `Dockerfile.gateway` uses an ARG with a
default (`ARG GATEWAY_BASE_IMAGE=nvcr.io/nvidia/distroless/cc:v4.0.4`)
and `FROM ${GATEWAY_BASE_IMAGE} AS gateway`, so the old script returns
nothing.
Parse the ARG default value directly so the GPU prerequisites check
keeps using the gateway base image as a `nvidia-smi` probe target.
* ci(gpu): pin GPU probe to nvcr.io/nvidia/base/ubuntu:noble
The previous probe parsed the gateway base image out of the Dockerfile,
relying on the fact that the gateway ran on `nvcr.io/nvidia/base/ubuntu`
and that NVIDIA Container Toolkit CDI injection would populate
`nvidia-smi` and the supporting libs at runtime. The new gateway base
(`nvcr.io/nvidia/distroless/cc`) lacks `ldconfig`, a populated
`/usr/bin`, and the broader filesystem layout CDI injection assumes,
so it cannot serve as a GPU probe.
Pin the probe image explicitly to the NVIDIA-managed Ubuntu base. The
probe is independent of the gateway runtime and survives future base
swaps.
* fix(e2e-gpu): pass GPU probe image via env, drop Dockerfile.images parse
The Rust e2e test `gpu_request_for_each_discovered_device_matches_plain_container`
was still parsing the deleted `Dockerfile.images` to derive its GPU probe
image, panicking with `No such file or directory` after this PR's
Dockerfile split.
Move the probe image to a single source of truth in the workflow
(`OPENSHELL_E2E_GPU_PROBE_IMAGE` env at the job level) and require the
e2e test to read it from there. No silent codebase default — the test
panics with a pointer at the workflow if the env is missing, so it
fails loudly rather than drifting from CI.
The prereq probe step in the workflow now consumes the same env, so
the probe image is declared exactly once.
* fix(docker): keep supervisor binary root-owned for rootless Podman
The Podman driver mounts the supervisor image read-only into the sandbox
container at /opt/openshell/bin and runs that container as UID 0, but
deliberately drops DAC_OVERRIDE for hardening (container.rs:419). With
--chown=65534:65534 --chmod=0550 the binary was r-xr-x--- owned by UID
65534, so the container's UID 0 fell into "other" with no read or exec
access and the supervisor crashed on start (ContainerExited code 1).
Docker and Kubernetes both retain DAC_OVERRIDE, so root could still
exec the file — which is why this regression only surfaced in the
Podman e2e job.
Drop --chown from the supervisor COPY so the binary stays root-owned.
Keep --chmod=0550: the security win was dropping world-execute, not
changing the owner. The chown bought nothing here because the container
is always UID 0 regardless of driver, but it actively broke the only
driver that drops DAC_OVERRIDE.
---------
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
Simon Scatton