10 Commits
Author SHA1 Message Date
Polite_realism db94c17682 test(podman): run the e2e binaries that pass in the tmachine guest (#4422)
* test(podman): run the e2e binaries that pass in the tmachine guest

Remove ten binaries from podmanE2eFollowUpBinaries so the rootless tmachine e2e-podman archive runs them: credential_gating, forward_proxy_graphql_l7, forward_proxy_jsonrpc_l7, forward_proxy_l7_bypass, host_gateway_alias, landlock, no_proxy, podman_oci_identity, provider_refresh_handles, and websocket_conformance. All ten passed in two consecutive probe runs. Update the exclusion comments and TESTING.md to match what the remaining entries still need. Contributes to #3712.

Signed-off-by: politerealism <burdcat17@gmail.com>

* test(podman): explain why each remaining e2e binary is excluded

Give every remaining podmanE2eFollowUpBinaries entry its own reason, and mark the five that the wrapper-based Podman lane still runs (driver_config_volume, local_driver_token_restart, podman_corporate_proxy, podman_gateway_start, provider_token_exchange) so it is clear they are covered there until tmachine provides what they need. Contributes to #3712.

Signed-off-by: politerealism <burdcat17@gmail.com>

---------

Signed-off-by: politerealism <burdcat17@gmail.com>
2026-10-13 13:25:12 +00:00
Matthew GrossmanandEvan Lezar 8c67c27e8b test(oci): add a shared Docker and Podman suite for image USER and WORKDIR (#3931)
* test(oci): share OCI image checks across Docker and Podman

Add an oci-image feature testsuite that runs against installed artifacts.
It covers custom WORKDIR placement, image content and ownership, workspace
writes from the main process and exec, file transfer including
directory-merge upload, OCI user identity, the managed /sandbox fallback,
and rejection of an unwritable WORKDIR.

CI runs the suite on the Docker rootful, Podman rootful, and Podman
rootless tmachine guests.

Replace the Docker-only custom_image e2e with the shared suite.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): require WorkspaceValidationFailed and cover image USER fallback

The unwritable WORKDIR scenario now requires the WorkspaceValidationFailed
reason, so unrelated provisioning failures cannot pass it. The numeric
USER scenario supplies a policy without a process section, so the image
USER is the only source of the sandbox identity.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(podman): leave shared OCI identity checks to the oci-image suite

The oci-image feature suite covers the sandbox identity seen by the main
process and sandbox exec on Docker and Podman. The Podman e2e keeps the
checks only it can make: the pinned image ID and the isolated
workload/supervisor container pair.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): accept existing workspace rejection diagnostics

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): fix cleanup ordering and restore coverage parity

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(oci): remove migrated legacy workspace helpers and checks

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(drivers): report rejected OCI working directories as WorkspaceValidationFailed

The supervisor exits with the reserved status when the sandbox rejects the
image working directory. Docker maps that exit during readiness and
monitoring and records it before removing the supervisor container, and
Podman maps the supervisor companion's exit instead of the workload's.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): require WorkspaceValidationFailed for unwritable WORKDIR

Unrelated provisioning failures can no longer pass the rejection scenario.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix(oci): keep exit 78 reserved and report fixed workspace diagnostics

The supervisor no longer passes on a workload's own exit status 78, so
drivers cannot misreport it as a workspace rejection after a runtime
restart. The rejection is recognized once at agent start and carried as a
typed error. Docker reports the same fixed condition message as Podman and
logs the supervisor output instead, and its monitor removes the supervisor
container when a stop overlaps diagnostics collection. The OCI image suite
registers images for cleanup only after they build.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* refactor(oci): simplify workspace failure handling and OCI image tests

Report a recorded Docker workspace rejection once, where supervisor
readiness fails, and match its fixed message exactly. Read the Podman
supervisor exit status once per inspection. Choose the supervisor exit
status in one function and derive the workspace error type.

The OCI image suite shares step helpers, checks that a custom workspace
does not also create /sandbox, and its playbook waits for a connected
gateway and collects failure diagnostics through the shared role.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): replace podman_oci_identity with shared and driver suite checks

The oci-image feature suite now checks, on Docker and Podman, that the
image user has no capabilities, has no_new_privs set, and cannot read
the sandbox control credentials, and that a custom WORKDIR is not a
mount. The checks that need container inspection (pinned image ID,
network=none, supervisor user and capabilities) move to a small test in
the Podman driver suite. Remove the legacy podman_oci_identity e2e test.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): keep podman_oci_identity for Podman-only isolation checks

Restore the Podman e2e test removed in the previous commit and drop
only the checks the shared oci-image suite now runs on Docker and
Podman: workload capabilities, no_new_privs, credential readability,
and the workspace and /sandbox mount checks. The pinned image ID and
the workload/supervisor isolation checks stay, as does the Podman
driver suite unchanged.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* refactor(oci): simplify workspace failure reporting and OCI image suite

Carry the Docker control supervisor's failure reason in a typed error
instead of matching status text, build the fixed workspace message where
the wait result is classified, and drop the fake Docker server test.

Inline image cleanup ordering in the suite, read the container engine
once, share sandbox naming, and stop granting a literal /sandbox in the
image identity policy. Add e2e:oci-image mise tasks and use them from the
ARM64 Docker workflow and TESTING.md.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): move shared container checks into the OCI image suite

Inspect the containers the driver created through the gateway's engine in
the shared suite: the workload runs the inspected image ID as the OCI
identity without a network or supervisor credential mounts, and the
supervisor is a separate non-root container. Keep only the Podman-specific
supervisor identity and capability checks in podman_oci_identity.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): check the workspace rejection reason from sandbox JSON

Read the WorkspaceValidationFailed condition reason from sandbox get
--output json instead of matching CLI output, and restore the Docker
driver's original workspace failure helpers.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* test(oci): capture fixture image identity before sandbox creation

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
2026-10-13 13:00:59 +00:00
Evan Lezar c4dc30dc1b test(conformance): run driver suites with cargo (#3866)
* test(conformance): run driver suites with cargo

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(e2e): resolve conformance CLI path before cargo tests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* refactor(conformance): remove unused CLI scenario APIs

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-10-12 15:35:03 +00:00
Polite_realismandEvan Lezar b8ffe5244c test(podman): move podman_preflight into driver-podman integration tests (#3783)
* test(podman): move podman_preflight into driver-podman integration tests

podman_preflight verifies that openshell-driver-podman fails fast when
its Podman socket is unreachable. It only needs the standalone driver
binary, not a gateway, so it never fit the gateway-backed e2e-podman
harness it lived under and never ran anywhere in CI.

Move it into crates/openshell-driver-podman/tests/ as a plain Cargo
integration test. It now runs via the existing required workspace test
job with no special mise task, workflow step, or coverage exception.

Signed-off-by: politerealism <burdcat17@gmail.com>

* test(podman): make preflight diagnostics portable

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: politerealism <burdcat17@gmail.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
2026-09-30 06:05:18 +00:00
Evan Lezar eef8bec0c9 test(e2e): run podman suite with tmachine (#3637)
* test(e2e): remove superseded podman userns coverage

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(tmachine): run podman e2e archive

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(tmachine): generate podman e2e archive inventory

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-09-28 11:15:33 +00:00
Evan Lezar df88bedb31 fix(podman): support rootless user namespace configurations (#3527)
* test(podman): cover user namespace configurations

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(podman): support keep-id runtime groups

Signed-off-by: Evan Lezar <elezar@nvidia.com>

refactor(podman): generalize keep-id group handling

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(podman): run driver integration tests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-09-23 00:30:29 +00:00
Evan Lezar 2263685cf3 test(tmachine): migrate Keycloak provider refresh coverage (#3404)
* test(tmachine): add Keycloak provider refresh suite

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* refactor(tmachine): share container runtime detection

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(tmachine): run feature suites in GitHub Actions

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(tmachine): run conformance with Podman tests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(tmachine): cover provider refresh with Podman

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(integration): split input preparation from runners

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-09-18 13:56:07 +02:00
Simon Scatton 0a0a563dd3 ci(conformance): run tmachine suites in release dev (#3382)
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
2026-09-17 17:13:56 +02:00
Evan Lezar 292559c41c test(tmachine): run smoke tests from nextest archives (#3372)
* test(conformance): package CLI smoke test archive

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(tmachine): execute nextest archives in guests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2026-09-16 12:49:55 +00:00
Simon ScattonandEvan Lezar 9b52b43b39 test(tmachine): add portable VM-based container runtime testing (#3371)
* test(tmachine): add Docker VM scenario

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* test(tmachine): add portable container scenarios

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* chore(tmachine): isolate downloaded Ansible roles

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* test(tmachine): increase VM resources

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* test(tmachine): improve artifact builds and diagnostics

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* fix(nix): pin tmachine runtime on macOS

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* feat(tests): enable tty and fix supervisor image path

* fix(nix): isolate testing tools from default shell

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* fix(tmachine): initialize test runner working directory

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* fix(tmachine): include Ansible sources in layer cache keys

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* fix(tmachine): build and load separate runtime images

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* fix(tmachine): use local runtime images for Docker

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

---------

Signed-off-by: Simon Scatton <sscatton@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>
2026-09-16 11:54:28 +00:00