mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-11-03 08:17:44 +08:00
Evan Lezar
898ca460ed
Add the openshell-test runner, which runs a test plan end to end: nix run .#test-runner -- test --plan tests/plans/cpu-tests.yaml - Plans (tests/plans/*.yaml) select families, suites, and environments. Unknown families, suites, environments, and fields fail before anything is built or booted. - The catalog is read from source: each tests/*.rs file of a family workspace containing #[testkit::test] functions is a suite. Parsing moves into testkit-spec, shared by the macro and the runner, so they cannot disagree about a test's ID, description, or requirements. - Jobs are grouped by environment so each one is provisioned once. Tests whose requirements the environment lacks are excluded by exact name and reported as unsupported, as are environments the host cannot boot. - Each family is cross-built into a nextest archive bundle for the guest, checked against the catalog, run in a tmachine session against the target formed from the installer's gateway section and the environment's capabilities, and its JUnit report, log, and (on failure) gateway diagnostics are copied to artifacts/test-results/<plan>/<env>/<family>/. - --dry-run prints resolved jobs and filtersets; --env and --suite narrow a plan; --keep leaves each VM up after its tests; `catalog` prints every suite and test as JSON. - The test-runner Nix package provides the cross toolchains, QEMU, Ansible, firmware, and tmachine configuration. Document test plans and conformance suites in TESTING.md. Signed-off-by: Evan Lezar <elezar@nvidia.com>