CI: Fail on missing preconditions and exceeded boundaries (#1754)

This change addresses six ways CI pipeline could report false success or
hang on broken infrastructure. Importantly, this change does two things
to reduce load on infrastructure:

1. It prevents jobs from running for [the default action limit of
360m](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idtimeout-minutes)
by pinning the timeout to `45m`.
2. It limits pull_request jobs from continuing execution once superseded
by new changes.

- **Silent container test skipping**: Tests now explicitly fail if `CI`
or `REQUIRE_DOCKER` is set (while still skipping on local machines
lacking Docker), with the check implemented in `dockerenv` to avoid an
import cycle between `storetest` and `atepg`.
- **Unbounded trust bundle wait**: Enforced a shared 120-second timeout
across both bundles (overridable via `ATE_INSTALL_TRUST_BUNDLE_TIMEOUT`)
and added diagnostic dumping of bundles, controller pods, and logs
before returning a non-zero exit code.
- **Missing sandbox preflight validation**: Added early preflight checks
for `/dev/kvm` and `SandboxConfig/microvm` that fail fast and print
actionable remediation instructions.
- **Skipped migration checks on main**: Configured the migration
immutability check to run on pushes to main to catch modified migrations
at the point of merge.
- **Missing job timeouts and concurrency limits**: Defined explicit
timeout-minutes (45m and 120m bounds) and added concurrency groups that
automatically cancel superseded pull request runs without canceling runs
on main.

Fixes #1747


- [x] Tests pass
- Silent container skipping, unbounded trust bundles, and missing
sandbox were all forced locally and confirmed to exist with changes here
resolving each.
- The latter half of the scenarios exist in CI only due to being GH
Action trigger issues.
This commit is contained in:
Steven Shriver
2026-09-23 17:45:42 +00:00
committed by GitHub
parent 898f6e6495
commit 52c6a03c69
7 changed files with 82 additions and 1 deletions
+13 -1
View File
@@ -27,9 +27,17 @@ on:
# and run it in a throwaway cluster.
permissions:
contents: read
# Superseded pushes to a pull request are cancelled; every commit that lands on
# main still gets its own full run.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
run-tests:
runs-on: ubuntu-latest
# A bound well clear of the observed runtime. Without one the platform
# default applies, and a wedged step burns a free-tier slot for six hours.
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
@@ -39,8 +47,9 @@ jobs:
uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
with:
go-version-file: 'go.mod'
# Unconditional: a released migration becoming permanent is exactly what
# this guards, and that happens on the push to main, not on the PR.
- name: Verify immutable PostgreSQL migrations
if: github.event_name == 'pull_request'
run: hack/verify/postgresql-migrations.sh
- run: go test -race -v ./...
# tools/apitool has its own module so we need to run it explicitly.
@@ -60,6 +69,9 @@ jobs:
e2e-test-matrix:
name: E2E (${{ matrix.dataplane }})
runs-on: ubuntu-latest
# Cluster bring-up, image builds and six sequential lanes; generous, but far
# short of the six-hour platform default a hung lane would otherwise consume.
timeout-minutes: 120
continue-on-error: ${{ matrix.experimental }} # TODO: Make AgentGateway required once tests show stability
strategy:
fail-fast: false