mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-10-02 03:24:42 +08:00
Steven Shriver
898f6e6495
Today CI depends on process exit codes. This leads to a few issues with
reporting results. For example, a green build cannot be distinguished
from one where no tests ran at all; a `-run` filter that matches
nothing; an e2e suite short-circuiting without `--e2e`; or a
testcontainer skip: all currently exit 0.
This change adds gotestsum as a pinned tool module allowing opt-in for
JUnit reporting in both test runners by using `E2E_JUNIT_FILE` and
`ROOT_JUNIT_FILE` respectively. When left unset, the runners exec `go
test` directly and gotestsum is never built leaving local runs unchanged
for now.
Additionally, this change pins two execution bounds that `go test` would
otherwise infer. Both remain overridable via environment variables as
listed below.
- `-p`: system defaults didn't align with cluster capacity so pinning
the value avoids over allocation. (override with `E2E_PARALLELISM`).
- `-timeout`: Suite defaults conflicted with some E2E test timings
causing some tests that are expected to timeout at the same time to
abort rather than fail as expected. Setting an 30m default avoids the
conlficting timeouts producing clean failure signals (override with
`E2E_TIMEOUT`).
Fixes #1685