Evan Lezar 898ca460ed test(runner): run test plans in tmachine environments ...

Add the openshell-test runner, which runs a test plan end to end:

  nix run .#test-runner -- test --plan tests/plans/cpu-tests.yaml

- Plans (tests/plans/*.yaml) select families, suites, and environments.
  Unknown families, suites, environments, and fields fail before anything
  is built or booted.
- The catalog is read from source: each tests/*.rs file of a family
  workspace containing #[testkit::test] functions is a suite. Parsing moves
  into testkit-spec, shared by the macro and the runner, so they cannot
  disagree about a test's ID, description, or requirements.
- Jobs are grouped by environment so each one is provisioned once. Tests
  whose requirements the environment lacks are excluded by exact name and
  reported as unsupported, as are environments the host cannot boot.
- Each family is cross-built into a nextest archive bundle for the guest,
  checked against the catalog, run in a tmachine session against the target
  formed from the installer's gateway section and the environment's
  capabilities, and its JUnit report, log, and (on failure) gateway
  diagnostics are copied to artifacts/test-results/<plan>/<env>/<family>/.
- --dry-run prints resolved jobs and filtersets; --env and --suite narrow a
  plan; --keep leaves each VM up after its tests; `catalog` prints every
  suite and test as JSON.
- The test-runner Nix package provides the cross toolchains, QEMU, Ansible,
  firmware, and tmachine configuration.

Document test plans and conformance suites in TESTING.md.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

2026-10-15 23:34:23 +02:00

..

src

test(runner): run test plans in tmachine environments

2026-10-15 23:34:23 +02:00

Cargo.lock

test(runner): run test plans in tmachine environments

2026-10-15 23:34:23 +02:00

Cargo.toml

test(runner): run test plans in tmachine environments

2026-10-15 23:34:23 +02:00