mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-29 07:23:25 +08:00
fa2d381eaa9e917fd074b4bc49efea8a2ad3e43e
417
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jesse Jaggars
|
2c31ac7e60 |
feat(network): support additional destination CAs (#3292)
* feat(network): support additional destination CAs Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): harden additional CA propagation Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): narrow additional CA review scope Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): harden additional CA rotation Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * feat(network): complete additional CA delivery Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(docker): remove stale runtime fixture field Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): adapt additional CA series to upstream main Resolve rebase fallout against current upstream main after the series was rebased 214 commits forward: - follow upstream #3110: supervisor→gateway TLS is CA-only, so configure_gateway_tls and its isolation tests drop the client identity - follow upstream tls_secret_names reduction to [String; 1] in the Podman driver while keeping the network trust bundle delivery - port [openshell.supervisor.network] rendering into the openshell.gatewayConfigToml serializer in _toml.tpl - adapt tests to upstream struct/function changes (DriverStartupContext, SandboxRunConfig, IsolationSpecInput, supervisor_pod, VM bootstrap image) - drop conformance-artifact-prefix plumbing removed upstream Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(network): adopt upstream additional CA bundle surface Unify the additional destination CA series on the mechanism merged to main in #3996: one operator pre-assembled PEM bundle, the --additional-ca-bundle supervisor flag, and the /etc/openshell/tls/additional/ca-bundle.pem staging path. - Replace the gateway-level additional_ca_cert_paths list with a single additional_ca_bundle file validated and staged at gateway startup. - Restore the upstream supervisor argument path and fold-into-roots behavior; drop the --network-additional-ca-bundle and --network-additional-ca-digest contracts. - Keep gateway normalization bounds, content-addressed staging, digest-based driver reconciliation, and stop/start rotation; the freshness check now runs at the gateway/driver boundary instead of the supervisor boundary. - Reject additional_ca_bundle in operator driver JSON config; standalone drivers self-source the bundle from --sandbox-additional-ca-bundle with the same validation. - Extend the Kubernetes ConfigMap name digest segment to keep the full 63-byte budget after the prefix rename. - Box the persisted-sandbox reconciliation futures so callers stay under the clippy::large_futures threshold. Updates Docker, Podman, Kubernetes, and VM delivery, Helm chart and tests, e2e wiring, gateway configuration docs, and the cluster debugging skill to the single mechanism. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(ci): make gateway startup tests wrap-tolerant and refresh example lockfile The network trust startup tests asserted on raw substrings of gateway diagnostics, but the CLI renderer wraps output at a fixed width with continuation markers and hard-breaks long paths. Compare assertions on whitespace- and continuation-squashed text so they hold regardless of wrap position. Format the e2e workspace (separate cargo workspace, not covered by the root cargo fmt run) and refresh examples/isolation-backend-v8/Cargo.lock for the libc dependency added to the network trust crate. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(podman): reuse TLS CA secret during stopped-pair trust replacement The additional CA e2e never ran in CI before the first maintainer test dispatch, and its local container inspection was broken in two ways: it located the supervisor container through e2e network membership (the supervisor is host-networked or runtime-bridged, so the filter matched the CI job container instead), and it required the gateway mTLS CA as a bind mount (Podman delivers it as a container secret when user namespaces remap UIDs). Select the supervisor by managed labels and accept the secret delivery form. The lifecycle leg then exposed a driver bug: stopped-pair trust replacement re-created the per-sandbox TLS CA secret, colliding with the secret still owned by the renamed stopped pair. Mirror the token and resolver secret handling: require and reuse the existing secret during replacement instead of creating it. Validated end-to-end with the Podman additional CA e2e lane. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(server): restore upstream recovery semantics for start-request failures Recovery no longer marks sandboxes Error/InvalidPersistedSandbox when driver_start_request fails during the startup or lifecycle sweeps. Gateway configuration problems (for example a missing launch signer) must not permanently fail unrelated sandbox records; the sweep logs a warning and skips them, matching the behavior before the additional CA work. Sandbox-record failures surface through the normal start path. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(podman): refresh stale supervisors without replacing workloads Replace the stopped-pair trust reconciliation with supervisor-only recreation. The destination-trust generation label moves from the workload to the supervisor container, and start_sandbox recreates only the supervisor when its generation is missing or stale. The workload container, its volumes, its secrets, and its workspace data are never replaced: the additional CA bundle is mounted only into the supervisor. Running sandboxes are never gated on trust labels, and recovery without fresh launch credentials keeps the existing supervisor. The intentional -removal fence machinery in the watcher existed only for workload replacement and is removed with it. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(docker): drop workload replacement for destination-trust changes Delete the stopped-workload trust reconciliation path and all of its machinery: the trust-generation label on workload containers, the generation match check in the start fence, the stopped-sandbox replacement, the bounded workspace archive copy with its 512 MiB cap and regular-file-only validation, and the boundary-state rollback helpers. No replacement machinery is needed: the trusted supervisor container is already re-created from the gateway's current configuration on every start via ensure_control_process_for_container, so stop/start picks up the current additional CA bundle while the workload container, its volumes, and its workspace data are never replaced. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs(configuration): drop reconciliation wording from additional CA lifecycle The stop/start lifecycle description already matches the supervisor-only refresh model; remove the two remaining references to restart reconciliation of workload containers. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): port additional CA test fixtures to the crypto interface Route the gateway TLS isolation test PKI, the destination CA source fixtures, and the gateway startup trust tests through openshell_crypto::pki now that rcgen access is centralized there: generate_keypair/self_signed/signed_by replace the direct rcgen calls and CertifiedKey field access follows the wrapper's shape. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(e2e): repair additional CA lane for Docker and non-container builds The first maintainer test dispatch revealed two latent issues in the additional CA e2e that no local run had exercised: the Podman gateway CA secret helper referenced ContainerEngine without the container feature gate, breaking Kubernetes and VM builds of the test, and the Docker branch asserted a client ca.crt bind mount that the Docker driver never creates. Gate the helper like its callers, and assert Docker's actual delivery path instead: the gateway mTLS CA staged as tls/ca.pem inside the read-only supervisor state volume, selected through the OPENSHELL_TLS_CA environment pointer. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(helm): restore alphabetical README ordering for supervisor keys The helm:docs check regenerates the chart README with helm-docs, which orders rows alphabetically; a conflict resolution had placed supervisor.network.additionalCaConfigMapName after the supervisor.podSecurityContext rows. Regenerate with mise run helm:docs. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * chore: refresh testkit lockfile for openshell-core libc dependency Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(driver-podman): stage operator CA bundle as driver-owned artifact Standalone Podman drivers accepting --sandbox-additional-ca-bundle previously kept the operator's source path as the trust artifact path. The immutable-artifact verification performed on every create/start requires exactly mode 0444, which an operator-owned source path cannot guarantee, so a readable-but-writable source failed reconciliation. Read the operator bundle with the same bounded, no-symlink validation, then stage the normalized material into a driver-owned artifact inside a private tempdir held for the driver process lifetime. The artifact is created 0600, synced, and only then restricted to 0444 so no observer ever sees a writable artifact. The operator source and its permissions are never referenced after startup. Addresses GATOR-291f43b2-02. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(driver-podman): pin retained workload image during supervisor refresh Supervisor-only trust reconciliation re-entered create_sandbox_with_existing_token, which re-pulled and re-resolved the workload image tag even though the workload container is retained. If the tag moved to a different image between stop and start, identity resolution and child-environment construction would reflect the new image while the retained workload still runs the original one, leaving mode-0600 bootstrap files the retained user cannot read. During a refresh with a retained workload, inspect the stopped container and pin to the immutable image ID it was created from (Config.Image); never pull or resolve the request tag. The requested reference is still used for child metadata so the retained workload's environment is byte-identical to its original launch. Addresses GATOR-291f43b2-01. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): verify host-only VM trust delivery and drop curl dependency The additional-CA lane's VM assertions read the destination CA out of the guest overlay and a guest-side supervisor-argv marker — artifacts from the guest-supervisor design that no longer exist now that the VM driver stages the CA host-only and runs the supervisor on the host. Inspect the driver-staged host artifact (digest-compared against the source), assert the guest overlay contains no CA material at the old mount paths, and keep the serial-console no-disclosure check. The workload validation scripts also ran curl, which exists only in the container-engine fixture image; the Kubernetes and VM lanes use the uv-based workload image and died instantly with "canonical main process already finished". Rewrite the main-process, exec, and lifecycle validation scripts in python3 (present in every lane's workload image). The network-policy binary identity is the resolved /proc/<pid>/exe target, so the versioned python3.12 binaries are allowlisted rather than the unversioned symlinks. The hostname mismatch check keeps its semantics: same reachable fixture, mismatched TLS SNI, and the failure must specifically be hostname verification. Addresses GATOR-291f43b2-04. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(kubernetes): allow admission-added labels on destination-trust ConfigMaps The destination-trust ConfigMap validation compared the whole label map for equality. A mutating admission webhook that attaches an organizational label to the created ConfigMap made the create response fail validation, and the retained ConfigMap then failed every subsequent create/start through the existing-object path, blocking sandbox launches despite the ownership labels, immutable flag, digest, and certificate bytes remaining correct. Validate the gateway-owned label pairs individually and allow unrelated labels; the exact trust-material checks are unchanged. Covered by a regression test with an admission-added label and a missing gateway-owned label. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(podman): close supervisor-refresh rollback gaps The supervisor-only trust refresh renamed the previous supervisor to a random rollback name and then performed the token-secret lookup. A transient lookup failure returned with the supervisor still renamed, where a normal delete could never find it, and the rollback rename and final rollback removal ignored their own failures, leaving orphans with attached control and channel resources. Complete every fallible preparation before the rename. On a failed replacement, restore the canonical name; if the restore itself fails, return an error naming the rollback container so the orphan stays identifiable — a later retry still succeeds because the canonical name is absent and the replacement path recreates a fresh supervisor. Treat not-found during the final rollback removal as success and name the orphan in both the log and the error on other removal failures; a retry observes the fresh canonical supervisor and resumes the normal start path. Covered by failure-injection tests for each transaction boundary: preparation failure before any rename, replacement failure restoring the canonical name, and restore failure naming the orphaned rollback container. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs: address destination-trust review corrections The gateway-configuration RFC documented an additional_ca_cert_paths list that the strict configuration parser never accepted; document the additional_ca_bundle single-path setting the parser actually enforces. The debug-openshell-cluster skill named the wrong generation annotation (openshell.ai/network-supervisor-trust-generation) and described digest argv and network-init canonicalization checks the implementation does not perform. Correct it to openshell.ai/additional-ca-generation and describe the actual delivery: the supervisor Pod mounts the immutable ConfigMap read-only and receives --additional-ca-bundle for the mount path, with the driver validating the digest annotation and exact bytes per creation. Classify the destination-trust generation hashes as deterministic identity derivation rather than authenticity proofs, following the openshell-crypto exclusions table: trust decisions come from the gateway's normalization, immutable staging, exact-byte validation, and TLS verification. Record the classification in the crypto README and at each hashing site. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(network): reconcile additional CA tests with suspend/resume and auth changes Upstream's suspend/resume lifecycle took StartSandboxRequest fields 7 and 8, so the durable-snapshot field moves from 7 to 9; update the wire-format test to assert the field-9 key (0x4a) and initialize the new resume controls. Repair the cli test module after the auth opt-in test merged alongside the environment-guard helpers, and initialize the gateway driver-startup context's network-trust fields in the shared supervision test. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(server): box start-recovery futures for clippy large-futures The durable-snapshot field grows StartSandboxRequest enough that the start-recovery match and the recovery-expiry test exceed the large-futures limit under -D warnings. Box::pin both futures, matching the pattern the recovery path used before the suspend/resume rework. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> --------- Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> |
||
|
Taylor Mutch
|
ac7f615cd1 |
fix(ci): allow accepted protobuf breaks for v0.1.6 prereleases (#4976)
* fix(ci): allow accepted protobuf breaks for one prerelease Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(ci): cover all v0.1.6 prerelease candidates Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * docs(ci): document full rollback of temporary protobuf exception Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(ci): complete protobuf test copyright header Signed-off-by: Taylor Mutch <taylormutch@gmail.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> |
||
|
Taylor Mutch
|
2e1e2c50a6 |
chore(ci): extend issue stale window and exempt roadmap issues (#4962)
* chore(ci): extend issue stale window to 28 days Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * chore(ci): exempt roadmap issues from stale handling Signed-off-by: Taylor Mutch <taylormutch@gmail.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> |
||
|
Simon Scatton
|
37b7302796 |
chore(ci): update rust-cache to include upstream fixes (#4748)
Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Simon Scatton
|
72bc3d3668 |
ci(release): infer optional workflow tags (#4707)
* ci(release): infer optional workflow tags Signed-off-by: Simon Scatton <sscatton@nvidia.com> * fix(release): use resolved tag in success notifications Signed-off-by: Simon Scatton <sscatton@nvidia.com> * fix(release): pin prerelease notes to resolved candidate Signed-off-by: Simon Scatton <sscatton@nvidia.com> --------- Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
krishicks
|
e0ba078920 |
ci: keep rust caches warm on main and stop saving binary caches on branches (#4881)
Branch Checks never runs on main, so PR mirrors had no main rust-checks cache to restore and each one saved its own. Per-binary release build caches were also saved from every branch. Together they filled the 50 GB cache quota within hours and evicted main's caches. Save per-binary caches only on main, and add a Rust Cache workflow that builds the Branch Checks test targets on each push to main so PRs and merge groups can restore a matching rust-checks cache. Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
Oliver Calder
|
72e4eaa89c |
feat(snap): publish prerelease builds to latest/beta channel (#4870)
* feat(snap): publish prerelease builds to latest/beta channel Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fixup! feat(snap): publish prerelease builds to latest/beta channel Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): use concurrency group when checking version to publish snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> --------- Signed-off-by: Oliver Calder <oliver.calder@canonical.com> |
||
|
krishicks
|
5bfe2fbc88 |
docs(contributing): exempt build and flaky-test fixes from the issue requirement (#4875)
Prior to this change, CONTRIBUTING.md, the PR template, and the create-github-pr skill required every PR except automated dependency updates to close an issue. A maintainer fixing a broken build or flaky tests had to file an issue that only restated the failure the PR already describes. Now, authors with WRITE, MAINTAIN, or ADMIN access may open a PR that only fixes a broken build or flaky tests without a separate issue, and describe the failure in the PR instead. Authors without write access still need an accepted issue; the PR Issue Acceptance workflow already passes write-access authors without checking for linked issues, so it needs no change. The branch naming rule now covers PRs without an issue, and the sync-agent-infra consistency check states the new rule. Signed-off-by: Kris Hicks <khicks@nvidia.com> |
||
|
Oliver Calder
|
cb42f5d9e8 |
fix(snap): use openshell-prover instead of openshell.prover (#4835)
The alias `openshell-prover` has been granted for OpenShell: https://forum.snapcraft.io/t/alias-request-for-openshell/53658 Thus, we can remove guidance differentiating `openshell.prover` for the snap from `openshell-prover` for other packaging formats. It is still worth noting that installing a locally-built snap directly, rather than from the store, means there is no alias. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> |
||
|
Matthew Grossman
|
fac82327b3 |
test(testkit): add the testkit library for end-to-end suites (#4724)
* test(testkit): add the testkit workspace for end-to-end suites Add tests/testkit as its own Cargo workspace: testkit, testkit-macros, and testkit-spec. testkit reads a target file, connects to the gateway through openshell-sdk, creates and cleans up sandboxes, and runs the installed CLI. #[testkit::test] requires a stable ID and a doc comment and declares capability requirements. testkit-ready waits for a target's gateway and CLI before tests run. Nothing uses testkit yet. The lockfile is seeded from the root lockfile so shared crates resolve to the same versions. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(tests): format, lint and test testkit in Branch Checks Nothing else builds testkit, so check it in Branch Checks and the matching mise tasks to catch openshell-sdk changes that break it. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(tests): describe how to write and run testkit tests Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(testkit): give each sandbox its own cleanup timeout Cleanup shared one two-minute budget across all of a test's sandboxes, so a slow delete left the rest with no time and could leak them on a shared gateway. Delete them concurrently, each with the full timeout. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(testkit): show how to exclude tests by capability Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs: list tests/testkit in the project structure tables Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): drop test IDs and capabilities for now Nothing reads test IDs or capability requirements until a runner, waivers, or result history exist, so remove the id and requires arguments and the target file's capabilities section. Both can return as optional attribute arguments without changing existing tests. With them gone, testkit-spec only checked the function shape and read the doc comment, so fold it into testkit-macros. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(testkit): describe what testkit provides today Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): find the gateway the way the CLI does Replace the target file (OPENSHELL_TEST_TARGET) with the CLI's own gateway selection: OPENSHELL_GATEWAY_ENDPOINT, then OPENSHELL_GATEWAY, then the active gateway. A registered gateway's endpoint, auth mode, and mTLS material come from its CLI config directory. cx.cli runs OPENSHELL_BIN against the same gateway and workspace as the SDK. Suites now run unchanged under e2e/with-docker-gateway.sh and e2e/with-podman-gateway.sh, and environments need no extra file. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(testkit): keep labels when creating a sandbox with a policy The policy path creates the sandbox through gRPC and dropped the spec's labels without an error. Pass them in the create request. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(testkit): use the registered gateway's credentials for an endpoint When OPENSHELL_GATEWAY_ENDPOINT is set, the CLI takes credentials from the gateway named by OPENSHELL_GATEWAY, or else the registered gateway with that endpoint. testkit connected without a client certificate instead, so an mTLS endpoint such as the one e2e/rust/e2e-vm.sh exports worked for cx.cli but not for the SDK. Resolve it the way the CLI does, and pass both the name and the endpoint to cx.cli. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(tests): list testkit's gateway variables in precedence order OPENSHELL_GATEWAY_ENDPOINT takes precedence over OPENSHELL_GATEWAY. Also mention OPENSHELL_BIN. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(testkit): apply other attributes to the generated test #[testkit::test] left attributes such as #[ignore] and #[cfg] on the nested body function, where they have no effect on the test. Move them to the generated test, and reject a second test attribute such as Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> #[test] or #[tokio::test]. * fix(testkit): show usage for testkit-ready --help Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(testkit): cover cleanup after pass, failure, and panic Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(testkit): reject #[should_panic] on testkit tests A testkit test panics on any failure, including an unreachable gateway or a failed cleanup, so #[should_panic] would let such a failure pass. Reject it and point to checking the expected failure in the body. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(testkit): keep only the macro's misuse checks as unit tests Suites built on testkit exercise gateway resolution, the CLI wrapper, polling, errors, and cleanup against a real gateway. Keep one unit test for the macro's compile errors, which a passing suite never reaches, and drop the test-only seams the other tests needed. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): connect through the SDK's mTLS support The SDK's ClientConfig now takes a client identity, so testkit no longer builds its own channel. It still resolves the registered gateway and reads its credential files, then hands the CA and client certificate to OpenShellClient::connect. That keeps TLS behavior, connect timeouts and keep-alive settings in the SDK, and drops testkit's direct tonic dependency. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(testkit): bound readiness and cleanup with timeouts wait_ready checked elapsed time only after a health call returned, and cleanup started its timeout only after the delete returned, so a gateway that accepts a connection but never answers could hang either one. Wrap each whole operation in tokio::time::timeout: the readiness wait, including every connect and health call, and each sandbox's delete plus wait for deletion. A test covers a peer that accepts connections and never replies. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Drew Newberry
|
c7dd586b5d |
ci(compatibility): review driver-facing executable flags (#4644)
* ci(compatibility): review driver-facing executable flags Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(compatibility): name CLI review surfaces Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Jim Meyer
|
f7e37af1b7 |
ci(release): tag Go SDK module on stable releases (#4262)
Go resolves the sdk/go submodule version from sdk/go/vX.Y.Z tags, so without one pkg.go.dev only shows v0.0.0 pseudo-versions from main. Add a Release Tag job that creates the prefixed tag at the release commit and asks the Go module proxy to fetch it. Signed-off-by: Jim Meyer <jimeyer@nvidia.com> |
||
|
Matthew Grossman
|
0f5b4f0fbb |
ci(release): reuse the promoted prerelease's release notes preview (#4754)
* ci(release): reuse the promoted prerelease's release notes preview Stable releases now publish the selected prerelease's release-notes artifact with the prerelease tag replaced by the stable tag. Release falls back to generating notes when the preview is missing, expired, or malformed, or when dispatched with regenerate-notes=true. The selection script exposes the qualified Pre-release run ID for the download. Closes #4753 Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(release): reuse release notes from the promotion job Download and retag the preview in Select Qualified Prerelease, which already knows the qualified Pre-release run, instead of a separate job. Drop the regenerate-notes input and preview format checks. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(release): name the publish notes input after the caller's result Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Simon Scatton
|
92f3a41b7c |
ci: update setup-buildx action to v4.4.1 (#4741)
Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Simon Scatton
|
bdb956f2cd |
fix(ci): grant prerelease publish pull request read permission (#4747)
Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Oliver Calder
|
d4ad1bbd03 |
fix(snap): only enable gateway user service for one user (#4615)
* fix(snap): only enable gateway user service for one user Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * docs(snap): correct snap docs for the gateway user service Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): add documentation of global to per-user enablement migration Signed-off-by: Oliver Calder <oliver.calder@canonical.com> --------- Signed-off-by: Oliver Calder <oliver.calder@canonical.com> |
||
|
Drew Newberry
|
50b7fcd02b |
ci(release): generate PR-based stable release notes (#4654)
* ci(release): generate PR-based stable release notes Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(release): clarify PR release note length Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci(release): route note generation through inference hub Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci(release): configure inference endpoint and model with actions variables Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci(release): add a notes-only workflow preview Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci(release): retain prerelease notes as advisory artifacts Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(ci): shorten release notes workflow overview Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(release): resolve note comparisons within the generator Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(ci): simplify release notes workflow setup Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci: refine release notes guidance and upgrade section Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci: prioritize compatibility requirements in release notes Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci: simplify public compatibility release guidance Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci: clarify release-facing upgrade guidance Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Varsha
|
1cb9ef8ab4 |
refactor(crypto): introduce a generic interface for crypto libraries and refactor AWS-LC (#3351)
* refactor(crypto): centralize AWS-LC behind backend interface Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * refactor(crypto): route new host proxy tests through facade Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): make backend boundary check portable Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): route Google JWT signing through facade and verify context dispatch Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * refactor(crypto): propagate digest backend failures Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * refactor(crypto): route Vault TLS fixtures through facade Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * refactor(crypto): adapt isolation runtime callers after main rebase Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): declare sandbox dependency and remove boundary lint Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * refactor(crypto): support independent backends and backend-free CA import Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * chore(crypto): refresh feature suite lockfile after rebase Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): route runtime integrity and entropy through backend Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * chore(crypto): refresh lockfile after upstream rebase Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): route VM socket entropy through backend after rebase Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * docs(crypto): document architecture and contributor boundaries Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * test(crypto): adapt upstream fixtures to backend facade Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * test(crypto): route relay TLS fixture through facade Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * test(crypto): adapt service bearer TLS fixture after rebase Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): route provider profile TLS initialization through facade Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): adapt upstream VM and supervisor paths to facade Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * ci(crypto): enforce backend independence in branch checks Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * test(crypto): adapt upstream token grant TLS fixture Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): avoid format collect in executable hashing Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): address provider selection and decryption review Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): adapt upstream operator TLS paths after rebase Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(crypto): adapt upstream TLS fixtures and protocol tests Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> --------- Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> |
||
|
Matthew Grossman
|
834b14f720 |
ci(helm): retry chart pushes when GHCR briefly misses a new manifest (#4578)
helm push uploads the chart manifest by digest, then fetches it back to apply the version tag. GHCR sometimes returns 404 for a manifest it just accepted, failing the push with 'failed to perform "Tag" on destination: <digest>: not found'. This failed Release Dev on Oct 5 and the v0.1.5-pre.2 Release Tag run after qualification passed. Retry each chart push up to four times with a short backoff. A retry finds the blobs and manifest already present and only applies the tag. Closes #4577 Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Matthew Grossman
|
c38e2799f1 |
ci(release): list failed jobs in pre-release failure notices (#4719)
* ci(release): name the failed stage and failing jobs in pre-release failure notices The pre-release failure notification said only that the run failed. Name the failed stage (build, publication, or release qualification), link the tag, and list each failing job of the run attempt with a link. Skip the aggregate Release Qualification job unless it is the only failure, and still send the message if the jobs lookup fails. Closes #4718 Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(release): simplify the failure notice to a list of failed jobs Drop the stage headline and the aggregate-job filtering. Keep the existing message and append a link to each failed job. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Simon Scatton
|
b1cfc91720 |
ci(release): promote qualified prereleases to stable (#4611)
* ci(release): promote qualified prereleases to stable Signed-off-by: Simon Scatton <sscatton@nvidia.com> * fix(release): preserve qualified images during stable publication Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci(release): run shared release jobs with Nix Signed-off-by: Simon Scatton <sscatton@nvidia.com> * fix(release): recheck latest before publishing shared channels Signed-off-by: Simon Scatton <sscatton@nvidia.com> --------- Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Piotr Mlocek
|
744b8890af |
fix(ci): paginate required gate job lookups (#4663)
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> |
||
|
Piotr Mlocekanddivesh
|
604b9f66f6 |
feat(config): deliver gateway configuration over the supervisor session (#4320)
Supervisors poll the gateway for configuration every 10 seconds. This PR adds an opt-in push mode (config_delivery_mode = "push"): supervisors still start by polling, and once their ConnectSupervisor session is accepted, the gateway delivers every later change over it and supervisors apply it and report the result. poll stays the default, and older supervisors and gateways keep polling. --------- Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> Signed-off-by: divesh <dgude@nvidia.com> Co-authored-by: divesh <dgude@nvidia.com> |
||
|
Drew Newberry
|
77929d9c14 |
feat(sandbox): publish and activate sandbox backend protocol (#3898)
* feat(isolation): publish backend protocol contracts independently of runtime Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(isolation): expose trusted launch and registry data contracts Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(isolation): name the required capability sandbox Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(sandbox): align protocol and compute capability names Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(sandbox): finalize public protocol contract names Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(isolation)!: activate backend contracts in sandbox runtime and driver selection Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(sandbox): simplify runtime and fix backend example Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(examples): organize WASI backend in Rust Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(sandbox): defer protocol conformance fixtures Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(examples): add supervised V8 backend demo Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(examples): stream V8 guest output with default policy Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(examples): handle V8 rejections and roll back failed launches Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(examples): align V8 crypto dependencies with CI AWS-LC Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(sandbox): represent Sandbox Protocol operations in protobuf Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(isolation): replace backend traits with RPC clients Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(proto): remove legacy isolation protocol reservations Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(proto): clarify Sandbox Protocol names Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(isolation): clarify authentication bootstrap and supervisor APIs Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(isolation): simplify policy enforcement confirmation Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(sandbox): rotate launch generations and stabilize retry sessions Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(sandbox): remove redundant backend adapters and TCP flags Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(sandbox): repair Docker restarts and macOS lint Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(isolation): simplify RPC contracts and backend registration Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(sandbox): deliver TCP denials before closing the RPC Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Evan LezarandMatthew Grossman
|
a2982bf53d |
fix(ci): restore default e2e coverage with a Kubernetes recovery waiver (#4424)
* fix(ci): make conformance-only e2e selection explicit Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(ci): use mapped NodePort for kind gateway sessions Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(ci): waive SSH transport recovery only on Kubernetes Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(e2e): log a notice when runners skip Rust e2e tests Emit a GitHub Actions notice from the Docker, Podman, Kubernetes, and VM runners when e2e-conformance-only skips the Rust tests, so a narrowed lane is visible in the job summary instead of silently reporting success. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(testing): move e2e conformance-only selector to shared section The selector applies to all four driver runners, so document it with the Rust CLI e2e suites instead of under the Kubernetes task variants. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Co-authored-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Simon Scatton
|
253884414a |
ci(release): decouple prerelease publication from qualification (#4420)
* ci(release): publish prereleases before qualification Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci(release): decouple prerelease publication from qualification Signed-off-by: Simon Scatton <sscatton@nvidia.com> --------- Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Matthew Grossman
|
c3ca45f1dd |
ci(release): notify Slack when a prerelease succeeds (#4605)
* ci(release): notify Slack when a prerelease succeeds Post a top-level message to the triage webhook when a prerelease completes, so recovery after failures is visible. The message omits the triage mention. Closes #4604 Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(release): link the prerelease tag in the success notification Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Simon Scatton
|
c06ba109c6 |
ci: disable Determinate Nix job summaries (#4601)
Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Simon Scatton
|
05050a6d64 |
refactor(ci): extract release qualification and serialize Codex reviews (#4419)
* refactor(ci): extract release qualification and serialize Codex reviews Signed-off-by: Simon Scatton <sscatton@nvidia.com> * fix(ci): queue Codex reviews without security ordering Signed-off-by: Simon Scatton <sscatton@nvidia.com> * fix(ci): retain qualification evidence after build failures Signed-off-by: Simon Scatton <sscatton@nvidia.com> * docs(ci): shorten compatibility review guidance Signed-off-by: Simon Scatton <sscatton@nvidia.com> --------- Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Simon Scatton
|
2c8f30f3b1 |
ci: install Determinate Nix in shared setup action (#4593)
Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Simon Scatton
|
10c6319757 |
ci(compatibility): require explicit risk acceptance (#4595)
* ci(compatibility): require explicit risk acceptance Signed-off-by: Simon Scatton <sscatton@nvidia.com> * test(compatibility): remove report regression tests Signed-off-by: Simon Scatton <sscatton@nvidia.com> --------- Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
dependabot[bot]andJohn Myers
|
68b10822ce |
chore(deps): bump github/codeql-action/init from 4.37.7 to 4.37.9 (#3002)
* chore(deps): bump github/codeql-action/init from 4.37.7 to 4.37.9 Bumps [github/codeql-action/init](https://github.com/github/codeql-action) from 4.37.7 to 4.37.9. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd...cdf488f595d80d6e07e03d4674febd5ab45fa938) --- updated-dependencies: - dependency-name: github/codeql-action/init dependency-version: 4.37.9 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * fix(ci): align CodeQL action versions for initialization and analysis Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com> |
||
|
dependabot[bot]
|
2f415e40b2 |
chore(deps): bump actions/attest from 4.2.1 to 4.2.2 (#2625)
Bumps [actions/attest](https://github.com/actions/attest) from 4.2.1 to 4.2.2. - [Release notes](https://github.com/actions/attest/releases) - [Changelog](https://github.com/actions/attest/blob/main/RELEASE.md) - [Commits](https://github.com/actions/attest/compare/508db95dd578ae2727ebd6217d5ba78e4fbda05d...1e69f48acb82d1966a394da916b4c1698aa569d6) --- updated-dependencies: - dependency-name: actions/attest dependency-version: 4.2.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
4d9a1d2a54 |
chore(deps): bump github/codeql-action/upload-sarif (#4552)
Bumps [github/codeql-action/upload-sarif](https://github.com/github/codeql-action) from 4.37.7 to 4.38.3. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd...24c54180a607b1449ed407dd24f251e4e9147c8d) --- updated-dependencies: - dependency-name: github/codeql-action/upload-sarif dependency-version: 4.38.3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Evan Lezar
|
fc0e9db8f3 |
ci: replace stale issue exemptions with explicit frozen labels (#4534)
* ci: use lifecycle:frozen as the sole stale issue exemption Signed-off-by: Evan Lezar <elezar@nvidia.com> * ci: support state:frozen as a stale issue exemption Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
|
e30c938bb1 |
feat(server): scope mutation locks and bound lock waits (#4147)
* test(server): add a local PostgreSQL test runner
The openshell-server tests that need a real PostgreSQL server are
ignored by default and had no shared way to run. The mutation-replay
test reads its own OPENSHELL_REPLAY_TEST_DATABASE_URL, so every
contributor had to provision a database by hand, and the advisory-lock
tests that follow in this series need the same setup.
Add mise run test:rust:postgres. It runs every ignored postgres_* test
in openshell-server against OPENSHELL_TEST_POSTGRES_URL, or starts a
disposable PostgreSQL container with Docker or Podman (CONTAINER_ENGINE
selects one) on the image pinned by the Kubernetes e2e fixture and
removes it on exit. Tests run one at a time because advisory locks are
database-wide. The runner always points the legacy replay variable at
the selected database, so an inherited URL cannot send that test to a
different server. test:postgres-runner checks that selection with a fake
cargo and needs no database or container engine. CI does not run the
PostgreSQL tests; TESTING.md documents the task.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* fix(server): hold the mutation guard while deleting a provider
DeleteProvider checked that no sandbox referenced the provider and then
deleted the record without holding the sandbox mutation guard. Sandbox
create and provider attach take that guard while they write provider
references, so one of them could add a reference after the attached
sandbox check passed, and the delete then removed a provider that a
sandbox spec still named.
Take the guard before the attached sandbox check, as provider create and
update already do, so the check and the delete run against a stable
set of sandbox references, and reject an empty name after authorization
but before the guard. A new test holds the guard, attaches the provider
while the delete waits, and asserts that the delete fails with
FailedPrecondition and leaves the provider in place.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* fix(server): release staged credentials when a refresh fails early
A provider credential refresh stages the minted values under new
credential handles before it commits the provider update, and deletes
those handles when validation or persistence fails. Two earlier returns
skipped that cleanup. The minted expiry was converted to a protobuf
timestamp only after staging, so an expiry outside the timestamp range
failed the refresh and left the staged values in the credential driver.
A failure to acquire the sandbox mutation guard returned the same way.
Convert the expiry before staging anything, so a bad value fails before
any handle exists, and delete the staged handles when the guard cannot
be acquired. A new test refreshes a stored credential with an expiry of
i64::MAX and asserts that the call fails, the provider keeps its
original handles, and the credential driver holds the same number of
values as before. The guard failure path has no test here, because the
SQLite guard used by unit tests cannot fail.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* feat(server): acquire mutation locks on a dedicated PostgreSQL lock pool
The cross-object mutation guard took its advisory-lock session from the
10-connection data pool and closed it on every release, so each guarded
mutation opened a new database connection, and lock holders competed
with their own critical sections for data connections. A lock wait that
hit the 10 second lock_timeout surfaced as INTERNAL.
Add a lazy, dedicated 4-connection lock pool and a new
persistence::mutation_lock module for the lock key, mode and set types.
Store::acquire_distributed_mutation_guard now takes a lock set and a
deadline, and each lock statement sets lock_timeout to the time left
before that deadline, so a cancelled acquisition never waits past it on
the server. Returned connections are scrubbed with
pg_advisory_unlock_all() and reused, and each return is bounded by 5
seconds so a stalled session cannot keep a pool permit.
A lock statement that PostgreSQL ends with SQLSTATE 55P03 fails with a
new LockTimeout error, and so does a wait for a lock connection during
which every one was checked out at some point, or that started with less
than a second left, too little to open one. The API returns it as
UNAVAILABLE with reason MUTATION_LOCK_TIMEOUT and a 1 second retry
delay, because the mutation lock was not acquired, so the request's
guarded writes did not run. Only the guard acquisition classifies 55P03;
any other 55P03 stays a database error. A lock connection that
PostgreSQL does not open by the deadline despite at least a second to do
so (refused, out of connection slots, starting up, or not answering, all
of which SQLx retries silently) is a database error, not a lock timeout.
Mutations still take the legacy global key exclusively, so old and new
replicas keep excluding each other during a rolling upgrade. Each
PostgreSQL-backed gateway pod can now open up to 14 database
connections (10 data plus 4 lock) instead of 10, and the chart's
autoscaling.maxReplicas comment says what its default of 4 needs. The
SSH host identity lock stays on a data connection, because sandbox
creation takes it while holding its mutation guard and a second
lock-pool connection there could exhaust the pool. The provider
credential refresh lock stays there too, so a refresh still progresses
while every lock connection is in use.
Part of #3528
Co-authored-by: divesh <dgude@nvidia.com>
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* perf(server): partition mutation locks by workspace and sandbox
Every cross-object mutation, and every lifecycle and supervisor update,
took one process-wide mutex and, on PostgreSQL, one fleet-wide advisory
lock. Unrelated sandboxes and providers therefore serialized across all
replicas, and one slow critical section delayed every other mutation in
the fleet.
Replace that lock with hierarchical intention locks on a global key, one
key per workspace and one key per sandbox, each held shared or
exclusive. Global settings and policy writers and platform-scope profile
writers hold the global key exclusively. Provider and workspace-scoped
profile writers, including DeleteProvider and credential refresh, hold
their workspace key exclusively. Sandbox writers and supervisor reports
hold their sandbox key exclusively under shared global and workspace
keys. Lifecycle, driver-watch and reconcile paths take only
process-local keys and keep relying on compare-and-swap across replicas.
Workspace and sandbox keys are domain-separated SHA-256 prefixes. Each
guard takes its local keys and then its advisory locks in ascending
order under one 10 second deadline, and no path nests guards. The
settings mutex is removed, since the global and sandbox keys cover it.
Sandbox settings are keyed by sandbox name, which a sandbox key no
longer covers once another sandbox reuses the name, so a settings
update returns NOT_FOUND unless its sandbox still holds the name after
the read, and ABORTED if the settings row was recreated before the
write.
The global key keeps its legacy value, so older replicas, which hold it
exclusively for every mutation, still exclude new ones during a rolling
upgrade. Guards now also give up after 10 seconds on SQLite, and
provider refresh reports a guard timeout as UNAVAILABLE instead of
INTERNAL. A supervisor report for a missing sandbox returns NOT_FOUND.
Startup endpoint-status reconciliation still holds the global key for
its whole scan.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* fix(server): reconcile endpoint status on startup one sandbox at a time
Startup endpoint-status reconciliation held the global mutation key
exclusively for its whole scan. On PostgreSQL that blocked every guarded
mutation on every replica until the starting gateway had walked all
sandboxes, and the scan first waited behind every in-flight sandbox
mutation. It also paged with an offset, so sandboxes deleted mid-scan
shifted rows between pages, and a write that raced the scan (a Conflict
from a lifecycle update on another replica, or a sandbox deleted after
it was listed) failed gateway startup.
Walk the sandboxes with keyset paging instead, and reset each candidate
under its own sandbox-scoped guard, up to four at a time to match the
lock pool. A candidate with a fresh shared owner is skipped before any
guard is taken. Under the guard the reconciliation re-checks the owner,
re-reads the sandbox, skips it when it is gone or no longer carries
endpoint status, and writes against the version it just read. A Conflict
or a database error at write time, which is how a row deleted after the
re-read surfaces, is retried with a fresh read, at most five attempts
per sandbox. A lock timeout on the guard is retried the same way, so a
holder of the sandbox, its workspace or the global key that releases
within the retry window cannot stop a replica from starting. No
lock-timeout retry starts once the scan has run for three lock timeouts,
so a scan stuck behind a long holder fails within about 40 seconds,
inside the chart's 60-second startup probe, and logs the sandbox it gave
up on. Store::list_all_messages loses its only caller and is removed.
Reconciliation stays fail-closed: a holder that outlasts the retry
window, an owner lookup failure, or a sandbox that keeps changing still
fails startup, so stale endpoint success is never served. User-visible
change: a restarting gateway no longer stalls mutations of unrelated
sandboxes and providers on its peers while it reconciles.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* perf(server): skip the endpoint-status lock when a session was replaced
When a supervisor session ends, the gateway resets that sandbox's tool
server endpoint evidence unless a replacement session already owns
endpoint observation. The replacement check ran only after the reset
had read the sandbox and taken its sandbox mutation guard, so a reset
whose supervisor had already reconnected still queued behind other
mutations of that sandbox, and on PostgreSQL it used a lock-pool
connection, only to find it had nothing to do.
Check for a replacement first: a live local session, or a fresh owner
record on a peer. When one exists, return without reading the sandbox
or taking the guard. Otherwise take the guard and repeat the same check
under it, because a replacement's pre-acknowledgement reset runs under
the same sandbox key and resetting after it would wipe the
replacement's fresh evidence. Both checks share one helper, so they
cannot drift apart. When no replacement exists, the reset costs one
extra owner-row read.
This helps most when a replica notices a dead stream late, after the
supervisor has already reconnected elsewhere (a network partition or a
keepalive expiry).
Deferring to a replacement leaves the old session's evidence to the
replacement's pre-acknowledgement reset, which can time out on its
bounded mutation lock wait. That reset's failure path removed the
replacement and released its owner record but scheduled no
invalidation, so the old session's success evidence could outlive both
sessions. An old session that ended after its replacement registered
already deferred this way, and the unguarded check now makes every
disconnect that finds the replacement defer too. When the reset fails
while the replacement is still current, also schedule the retried
invalidation that follows a session end. That failure path keeps the
lifecycle demotion from abandon_session_setup, but runs it in the
spawned task before the retry and only while this session is still
current, so a timed-out reset returns promptly.
Part of #3528
Co-authored-by: divesh <dgude@nvidia.com>
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* test(server): cover staged credential cleanup on a lock timeout
A provider credential refresh deletes its staged handles when it cannot
acquire the provider mutation guard, but that path had no test: the
SQLite guard used by unit tests could not fail. Mutation locks now
acquire under a deadline that tests can shorten, so the timeout path
can be exercised directly.
The new test shortens the lock deadline to 50 ms, holds the workspace
mutation guard, and refreshes a provider that already stores an AWS
access key. The refresh returns UNAVAILABLE with the
MUTATION_LOCK_TIMEOUT reason, the provider keeps its original handle,
which still resolves to the old key, and the credential driver holds
the same number of values as before the refresh.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* feat(server): export mutation lock wait and timeout metrics
A handler mutation guard can now wait up to 10 seconds for its keys and
then fail with UNAVAILABLE, but nothing showed operators how long guards
waited or how often they gave up, short of reading warning logs on each
replica.
Export openshell_server_mutation_lock_wait_seconds, a histogram of the
time one guard took to acquire all of its keys (local registry plus
PostgreSQL advisory locks), recorded on success, and
openshell_server_mutation_lock_timeouts_total, a counter of guard
acquisitions that timed out. Both carry a bounded scope label: global,
workspace or sandbox. The platform scope, whose workspace is empty,
counts as global. The histogram uses the same 1 ms to 15 s buckets as
the relay and peer histograms, and the timeout counter starts at 0 for
every scope, so rate() works on an idle replica. The timeout warning
now also logs the scope. A guard that fails for another reason, such as
a lock connection PostgreSQL did not open, is not counted as a timeout
and logs a "mutation lock acquisition failed" warning instead.
Lifecycle, driver-watch and reconcile paths take only process-local
locks and never time out, so they are not recorded.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* test(server): add PostgreSQL mutation lock integration tests
The scoped mutation locks were only unit-tested on SQLite, where the
database layer is a no-op. Add ignored postgres_* tests that run the
advisory-lock layer against a real PostgreSQL through mise run
test:rust:postgres, each in its own disposable schema (TestSchema).
persistence/mutation_lock_pg_tests.rs uses two stores on one database
as two replicas. It covers disjoint sandbox scopes holding at once, a
workspace key blocking only sandboxes in that workspace, the global key
blocking every scope, and exclusion in both directions against a raw
holder of the legacy global key, whose 55P03 stays a database error
outside a guard acquisition. It checks that timed-out and cancelled
acquisitions release the keys they already took, that a released lock
connection returns to the pool without residual locks, that a release
stalled on the network closes its session and the pool recovers, and
that the lock pool never grows past its size. It also checks that a
full lock pool times out as lock contention, while a lock connection
that PostgreSQL never answers fails as a database error.
compute/mutation_guard.rs runs the random scope mix across two
PostgreSQL runtimes and checks that a by-id guard waits for a workspace
writer on the other replica. The provider refresh lock test moves onto
the runner too, and checks that a refresh proceeds while every lock
connection is held.
A bench_postgres_* capacity check measures a paced burst of 500
reconnects 12 ms apart into one replica and fails when the p99 lock
wait reaches a fifth of the lock timeout. Machine load moves those
waits, so mise run test:rust:postgres skips it and mise run
test:rust:postgres:bench runs it alone, through a new
OPENSHELL_TEST_POSTGRES_PREFIX setting in the runner.
The tests add test-only helpers: the lock pool's size and idle count
and the backend pid of a held lock session. Nothing changes outside
tests and the test runner.
Part of #3528
Co-authored-by: divesh <dgude@nvidia.com>
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* ci: run the PostgreSQL-backed server tests
Branch Checks now runs the postgres_* tests through the same runner
developers use, on an x86_64 Linux runner with Docker. They cover lock
scope exclusion, interop with replicas still on the old global key,
cancellation, lock-pool bounds and connection failures, which no other
CI lane checks directly. The load-sensitive bench_postgres_* capacity
checks stay local.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* docs: document scoped mutation locks and lock-pool sizing
In the High Availability guide, size PostgreSQL for 14 connections per
gateway pod, 10 for data and 4 for the new lock pool, so three or more
replicas now exceed what the PostgreSQL default leaves, and keep
headroom for lock sessions that cancelled requests leave behind.
Explain which changes lock a sandbox, a workspace, or the whole fleet,
and that a lock wait over 10 seconds returns a retryable UNAVAILABLE
with reason MUTATION_LOCK_TIMEOUT. A new upgrade subsection says to
raise max_connections to the 14-per-pod size before upgrading, because
the rollout already runs new pods at that count, and warns that older
replicas keep serializing every mutation until the rollout finishes.
Monitor Capacity gains the lock metrics and their queries, and names
slow driver, credential, middleware, or profile source calls as another
cause of lock waits.
List the lock wait histogram and timeout counter, with their scope
label, on the Gateway Metrics page. The counter also counts background
work and leaves out lock connections that PostgreSQL does not open
although at least a second of the wait remained.
In the cluster debugging skill, replace the single advisory-lock
paragraph with the lock levels and pool, what each timeout detail
means, how a lock connection that PostgreSQL does not open shows up,
when terminating a lock backend is safe, a pg_locks query that shows
holders and waiters, and how to spot the global key. The per-pod greps
also read the timeout counter, and new rows cover lock timeouts and
lock connections that PostgreSQL does not open.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* feat(server): check mutation lock ordering in debug builds
The mutation lock ordering rules were prose only, so a nested guard on
a different key passed every test and could only deadlock under
contention. Debug builds now register each held guard with the Tokio
task (or the thread, outside a task) that acquired it, and panic before
waiting when a local key, a blocking lifecycle gate or the SSH identity
lock would break rules 1, 4 or 5. Startup endpoint-status
reconciliation and the tests that hold a guard while driving guarded
code run under lock_order::branch(), which marks deliberate concurrency
within one task. Release builds compile the check out.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* fix(server): do not hold gateway shutdown for the endpoint-status retry
When a replacement supervisor session failed its endpoint-status reset,
the spawned cleanup task kept the session lifetime guard through
retry_endpoint_status_after_supervisor_disconnect. That retry only ends
once the reset is durable, so a gateway shutdown during a database or
lock outage waited the full 10 s and reported ownership cleanup as
incomplete. The guard now covers only the lifecycle demotion, as in
finish_supervisor_session, and a SQLite test checks that shutdown
completes while the retry is still blocked.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* refactor(server): make the platform profile lock scope explicit
Platform-scope provider profile writers (import, update, delete) pass an
empty workspace on purpose and need X(global), because they validate
sandboxes in every workspace. MutationScope::profiles() now names that
escalation and picks Global for the platform scope, instead of relying
on MutationScope::Workspace("") to mean it. An empty workspace name now
trips a debug assertion, and release builds keep X(global) as a
fail-safe that over-locks rather than under-locks.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* perf(server): amortize local mutation lock table sweeps
LocalMutationLocks swept its whole table on every key lookup, so a
burst of N concurrent acquisitions on one replica cost O(N^2) under a
blocking mutex. The table is now swept only when a new key grows it
past the larger of 64 and twice the live entries left by the last
sweep, which makes acquisition amortized O(1). Released entries,
including those of cancelled waiters, stay bounded at about twice the
peak live set. The lifecycle gate registry keeps its per-call sweep.
Part of #3528
Signed-off-by: Emilien Macchi <emacchi@redhat.com>
* feat(server): make the database pool ceiling configurable
Both stores hardcoded their pool ceiling: 10 connections for Postgres, 5
for on-disk SQLite. One pool is shared by every database-backed RPC, so
that number is also the gateway's ceiling on concurrent database work.
Once every connection is checked out, callers queue on acquire and sqlx
logs "time to acquire exceeded slow threshold"; sandbox creates then time
out and are retried, which adds load rather than shedding it.
The right ceiling is deployment-specific — it depends on how many gateway
replicas share the database and what max_connections the server itself
allows — so it cannot be a single number baked into the binary.
Expose it on the same config surface as the rest of the gateway's
settings: --db-max-connections, OPENSHELL_DB_MAX_CONNECTIONS, and the
TOML key database_max_connections, resolved in that precedence order.
Helm deployments set the key through gatewayConfig. Omitting it keeps
each backend's previous value, so existing deployments do not move.
Values below 1 are rejected rather than silently replaced by the default:
a zero pool would block every acquire, and quietly ignoring a typo would
reproduce the ceiling the operator is trying to lift. An in-memory SQLite
database stays pinned to one connection, since the database lives inside
that connection.
Refs: #2561
Signed-off-by: Bryce Wilkinson <22760097+bjw123@users.noreply.github.com>
(cherry picked from commit
|
||
|
Jim Meyer
|
5a7728c304 |
fix(ci): skip maintainer approval in merge queue (#4496)
* fix(ci): skip maintainer approval in merge queue Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * refactor(ci): remove maintainer queue-state lookup Signed-off-by: Jim Meyer <jimeyer@nvidia.com> --------- Signed-off-by: Jim Meyer <jimeyer@nvidia.com> |
||
|
Simon Scatton
|
d3e6a35e47 |
ci(compatibility): review PR breaks and acknowledge release risks (#4488)
* ci(compatibility): review PR breaks and acknowledge release risks Signed-off-by: Simon Scatton <sscatton@nvidia.com> * fix(ci): use matching compatibility review tooling Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci(compatibility): scope Rust SDK review to public exports Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci(compatibility): account for Go struct field additions Signed-off-by: Simon Scatton <sscatton@nvidia.com> * ci(compatibility): restrict egress and fix report retries Signed-off-by: Simon Scatton <sscatton@nvidia.com> --------- Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
alangou
|
dfb0f4f917 |
ci(security): temporarily remove Codex from release scans (#4469)
* ci(security): temporarily make release Codex scans advisory Signed-off-by: Adrien Langou <alangou@nvidia.com> * ci(security): remove Codex from release scans Signed-off-by: Adrien Langou <alangou@nvidia.com> --------- Signed-off-by: Adrien Langou <alangou@nvidia.com> |
||
|
Matthew GrossmanandEvan Lezar
|
8c67c27e8b |
test(oci): add a shared Docker and Podman suite for image USER and WORKDIR (#3931)
* test(oci): share OCI image checks across Docker and Podman Add an oci-image feature testsuite that runs against installed artifacts. It covers custom WORKDIR placement, image content and ownership, workspace writes from the main process and exec, file transfer including directory-merge upload, OCI user identity, the managed /sandbox fallback, and rejection of an unwritable WORKDIR. CI runs the suite on the Docker rootful, Podman rootful, and Podman rootless tmachine guests. Replace the Docker-only custom_image e2e with the shared suite. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): require WorkspaceValidationFailed and cover image USER fallback The unwritable WORKDIR scenario now requires the WorkspaceValidationFailed reason, so unrelated provisioning failures cannot pass it. The numeric USER scenario supplies a policy without a process section, so the image USER is the only source of the sandbox identity. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(podman): leave shared OCI identity checks to the oci-image suite The oci-image feature suite covers the sandbox identity seen by the main process and sandbox exec on Docker and Podman. The Podman e2e keeps the checks only it can make: the pinned image ID and the isolated workload/supervisor container pair. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): accept existing workspace rejection diagnostics Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): fix cleanup ordering and restore coverage parity Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(oci): remove migrated legacy workspace helpers and checks Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(drivers): report rejected OCI working directories as WorkspaceValidationFailed The supervisor exits with the reserved status when the sandbox rejects the image working directory. Docker maps that exit during readiness and monitoring and records it before removing the supervisor container, and Podman maps the supervisor companion's exit instead of the workload's. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): require WorkspaceValidationFailed for unwritable WORKDIR Unrelated provisioning failures can no longer pass the rejection scenario. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(oci): keep exit 78 reserved and report fixed workspace diagnostics The supervisor no longer passes on a workload's own exit status 78, so drivers cannot misreport it as a workspace rejection after a runtime restart. The rejection is recognized once at agent start and carried as a typed error. Docker reports the same fixed condition message as Podman and logs the supervisor output instead, and its monitor removes the supervisor container when a stop overlaps diagnostics collection. The OCI image suite registers images for cleanup only after they build. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(oci): simplify workspace failure handling and OCI image tests Report a recorded Docker workspace rejection once, where supervisor readiness fails, and match its fixed message exactly. Read the Podman supervisor exit status once per inspection. Choose the supervisor exit status in one function and derive the workspace error type. The OCI image suite shares step helpers, checks that a custom workspace does not also create /sandbox, and its playbook waits for a connected gateway and collects failure diagnostics through the shared role. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): replace podman_oci_identity with shared and driver suite checks The oci-image feature suite now checks, on Docker and Podman, that the image user has no capabilities, has no_new_privs set, and cannot read the sandbox control credentials, and that a custom WORKDIR is not a mount. The checks that need container inspection (pinned image ID, network=none, supervisor user and capabilities) move to a small test in the Podman driver suite. Remove the legacy podman_oci_identity e2e test. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): keep podman_oci_identity for Podman-only isolation checks Restore the Podman e2e test removed in the previous commit and drop only the checks the shared oci-image suite now runs on Docker and Podman: workload capabilities, no_new_privs, credential readability, and the workspace and /sandbox mount checks. The pinned image ID and the workload/supervisor isolation checks stay, as does the Podman driver suite unchanged. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(oci): simplify workspace failure reporting and OCI image suite Carry the Docker control supervisor's failure reason in a typed error instead of matching status text, build the fixed workspace message where the wait result is classified, and drop the fake Docker server test. Inline image cleanup ordering in the suite, read the container engine once, share sandbox naming, and stop granting a literal /sandbox in the image identity policy. Add e2e:oci-image mise tasks and use them from the ARM64 Docker workflow and TESTING.md. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): move shared container checks into the OCI image suite Inspect the containers the driver created through the gateway's engine in the shared suite: the workload runs the inspected image ID as the OCI identity without a network or supervisor credential mounts, and the supervisor is a separate non-root container. Keep only the Podman-specific supervisor identity and capability checks in podman_oci_identity. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): check the workspace rejection reason from sandbox JSON Read the WorkspaceValidationFailed condition reason from sandbox get --output json instead of matching CLI output, and restore the Docker driver's original workspace failure helpers. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(oci): capture fixture image identity before sandbox creation Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> Co-authored-by: Evan Lezar <elezar@nvidia.com> |
||
|
Simon Scatton
|
a9a635ab41 |
fix(release): qualify prerelease tag pushes (#4335)
Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Evan Lezar
|
2ffc3d2b4d |
ci: require accepted issues for community pull requests (#4340)
Evaluate issue acceptance using trusted default-branch code and the PR author's repository permissions. Report the native workflow job result, protect enforcement files, and align PR creation and review guidance. Closes #4339 Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
dependabot[bot]
|
01c249f225 |
chore(deps): bump astral-sh/setup-uv from 10.0.0 to 10.0.1 (#2767)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 10.0.0 to 10.0.1. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](https://github.com/astral-sh/setup-uv/compare/ae62891fec2bb8e7d6c99fc78c9fec3a63790f8d...20cfd1bf945f4377ade1205e4dbc17946fc9a30d) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 10.0.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Jim Meyer
|
85c2c64798 |
chore(codeowners): remove wildcard ownership rule (#3168)
Remove blanket review routing while preserving the explicit owners of the vouch list, workflows, actions, approval helpers, and SBOM check already on main. Signed-off-by: Jim Meyer <jimeyer@nvidia.com> |
||
|
Jim Meyer
|
dc967d8a33 |
chore(ci): close stale issues after 14 additional days (#3985)
Closes #2401 Signed-off-by: Jim Meyer <jimeyer@nvidia.com> |
||
|
Matthew Grossman
|
fc8d0d7ada |
test(tmachine): add a K3s HA conformance lane with PostgreSQL and mTLS (#4081)
* test(tmachine): add K3s PostgreSQL and mTLS conformance installer Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(tmachine): reach the HA TLS gateway through its ClusterIP Drop the port-forward restart removed by the ClusterIP change, resolve the Service DNS name covered by the server certificate to the ClusterIP, bound each authenticated API wait attempt, and collect diagnostics from every gateway Pod. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(tmachine): run K3s PostgreSQL and mTLS conformance Add the k3s-ha-tls installer to every core conformance matrix so Branch E2E, Release Dev, Release Tag, and manual Integration Tests cover three gateway replicas sharing PostgreSQL behind mTLS. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(tmachine): rename k3s-ha-tls installer to k3s-ha TLS is part of the supported HA configuration once gateway replicas share PostgreSQL, so the installer name only needs to distinguish HA from the single-replica k3s lane. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(tmachine): install K3s HA in a single Helm rollout Add an optional Helm values overlay to the K3s installer so the k3s-ha installer stages PostgreSQL first and installs the three-replica TLS gateway directly, instead of installing a plaintext gateway and upgrading. Drop the redundant unauthenticated-users override and the install-time API wait, which the conformance playbook already performs. Share K3s installer inputs in the tmachine configuration. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(tmachine): make K3s HA staging rerunnable and wait for PostgreSQL Apply the OpenShell namespace instead of creating it so the staging play tolerates an existing namespace, and wait for the PostgreSQL rollout so gateway Pods do not start crash-looping against a missing database. Reflow and pluralize the K3s lane notes in CI.md. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Oliver Calder
|
16c8ca5c43 |
fix(docs): use lowercase "snap" as it is not a proper noun (#4387)
Signed-off-by: Oliver Calder <oliver.calder@canonical.com> |
||
|
Evan Lezar
|
c4dc30dc1b |
test(conformance): run driver suites with cargo (#3866)
* test(conformance): run driver suites with cargo Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(e2e): resolve conformance CLI path before cargo tests Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(conformance): remove unused CLI scenario APIs Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Matthew Grossman
|
d789ec613b |
test(tmachine): use K3s ClusterIP, wait for gateway, add failure diagnostics (#4258)
* test(k3s): connect tmachine clients directly to ClusterIP Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(conformance): wait for a connected gateway on every installer Replace the K3s-specific boot readiness hook with one shared gateway wait in the conformance playbook. Skip conformance and collect the existing diagnostics when the gateway never connects. Discover the gateway Service port with its ClusterIP, and capture both iptables backends in K3s diagnostics. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(tmachine): centralize gateway diagnostics and trim K3s changes Move failure diagnostics into one openshell_diagnostics role that collects the systemd gateway journal and, when K3s is installed, the K3s diagnostics. Drop install-time diagnostics so the K3s installer change stays minimal, revert unrelated skill edits, and shorten the CI.md note. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Jim Meyer
|
61d42102c1 |
feat(ci): gate merges on maintainer approval via a silent status check (#3176)
Adds the OpenShell / Maintainer Approval status check. It passes only when someone listed in MAINTAINERS.md has approved the pull request, and it notifies no one: a required status check is the only native primitive that enforces who must approve without requesting a review from them. The job runs on pull_request_review and merge_group with read-only permissions and posts nothing anywhere; its exit code is the check result. Both MAINTAINERS.md and the decision helper are read from main via a sparse checkout, never from the pull request ref, so a contributor cannot add themselves to the list or rewrite the decision logic and self-approve. Because pull_request_review runs the pull request's own copy of the workflow file, CODEOWNERS now covers the whole enforcement surface -- workflows, actions, CODEOWNERS itself, the zizmor config, MAINTAINERS.md, and the enforcement scripts -- naming the outside collaborators explicitly alongside the team, since org policy keeps them off GitHub teams. The ruleset's native code owner requirement is what blocks a merge that disables the gate, regardless of what the tampered check reports. Also adds a change-alert workflow that comments the approver-set delta on pull requests touching MAINTAINERS.md, and documents both in CONTRIBUTING. Signed-off-by: Jim Meyer <jimeyer@nvidia.com> |