Files
e2e/docs/ci/github-actions.mdx
Kacper KapuściakandOskar Kwaśniewski d593863b41 feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init (#1647)
* feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init

e2e init asks for a CI pipeline in a project with the web engine (--ci github-actions|gitlab|none). It writes one job for the project's package manager and Node version that installs Chromium, runs e2e run, and keeps the report, and names the secret or CI/CD variable for the model key. It never overwrites a pipeline file; beside an existing .gitlab-ci.yml it writes a file to include.

* fix(cli): handle cubic's findings on the init CI step

Exact GitLab stage names, a distinct job name beside an existing e2e job, loopback baseURL in an existing config, no writes through a symlinked .github or .gitlab, --ci none in the skill, gitlab in the telemetry comment, and the GitLab Node image's major version in the docs.

* refactor(cli): keep the init CI pipelines as YAML templates

The GitHub Actions workflow and the GitLab job live in packages/e2e/templates/ci/ as complete pipelines for npm at the repository root, linted as they are. init loads them with the yaml package's Document API and edits a copy for the project's package manager, folders, Node version, secrets, and target, keeping the comments. The pinned actions now live only in the templates.

* fix(cli): point setup-bun at the package.json of a project in a subfolder

* docs(ci): link the GitLab page from init and the GitLab template

* feat(cli): write the iOS and Android CI jobs in e2e init

init writes one job per platform the config's targets run on: web targets in the web job, iOS and Android targets in their own jobs, which build the app, boot a simulator or an emulator, and install the app. Targets come from the new config, or from loading an existing one after the confirmation and the install. Expo and bare React Native apps get their build; any other app gets a step to replace. GitHub's mobile jobs follow e2e-inc/e2e-react-native's workflow; GitLab's run on self-managed runners.

* fix(cli): reinstall an emulator system image that holds no image in the Android CI job

* fix(cli): handle cubic's findings on the mobile CI jobs

Upload the emulator log from the GitLab Android job, count peer and optional engine dependencies, ask nothing for a new config with no engine, key caches on Bun's bun.lockb when that is the lockfile, and run the symlink test with a Windows junction.

* fix(cli): put current Android command-line tools in place before the emulator image

The runner image's cmdline-tools can be older than the system image's package format, and android-emulator-runner uses them to create the AVD, which then fails with 'contains no system images'. The Android job downloads command-line tools 23.0, checks its SHA-256, and installs the image with it.

* fix(cli): import moved core modules from @e2e-dev/core in init

* fix(cli): count a pipeline as set up only when a command runs e2e run

init matched the text 'e2e run' anywhere in a workflow, so a comment or a step name such as 'our old e2e runner' made --ci write nothing. It now reads the commands (run, script, before_script, after_script) from the parsed YAML and matches e2e run as words of their own.

Also: --ci help and the InitOptions JSDoc say tests on every platform, and yaml is pinned exactly like the other dependencies.

* chore(ci): bump setup-node and upload-artifact pins in the init templates, docs, and skill

docs/ci/vercel-previews.mdx on main pins setup-node v7.1.0 and upload-artifact v7.0.2, the pins this repository's own workflows run. The templates, three docs pages, and the skill still had v7.0.0 and v7.0.1, so the pin test failed once main was merged in.

---------

Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-16 17:21:49 +00:00

267 lines
9.8 KiB
Plaintext

---
title: GitHub Actions
description: Run web, iOS, and Android tests on GitHub Actions, upload the reports, and post the results on the pull request.
"og:image": "https://e2e.com/docs/images/og/ci/github-actions.png?v=1a59aebfc8"
"twitter:image": "https://e2e.com/docs/images/og/ci/github-actions.png?v=1a59aebfc8"
---
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
export const setupPrompt = `Add a GitHub Actions workflow that runs this project's e2e tests. Docs: \`https://e2e.com/docs/ci/github-actions.md\` and \`https://e2e.com/docs/ci.md\`
1. Read both pages. Use the web workflow for web targets and the iOS and Android workflow for mobile targets.
2. Adapt the workflow to this project: package manager, Node version, how the app starts or builds, and the app paths.
3. For a mobile job, replace the iOS runtime with one the runner image has, and keep it pinned.
4. List the repository secrets the workflow reads, such as the model key, and ask me to add them.
5. After I push, read the job log and fix errors until the job passes.`;
These two workflows run web targets, and iOS and Android targets. For CI
defaults, model keys, and exit codes, see [Continuous integration](/ci).
`e2e init --ci github-actions` writes these jobs for your targets and
package manager, with the build for an Expo or bare React Native app; see
[CI pipeline](/reference/cli#ci-pipeline).
<SetupPrompt>
<Prompt description={setupPrompt} children={setupPrompt} actions={["copy", "cursor"]} />
</SetupPrompt>
## Web
This workflow runs deterministic and agent tests in one job. It assumes a
config [command](/web#starting-your-app) that starts the app, and AI
Gateway for the model calls. The model key and user credentials come from
repository secrets:
```yaml title=".github/workflows/e2e.yml" expandable
name: e2e
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: pnpm/action-setup@9fd676a19091d4595eefd76e4bd31c97133911f1 # v4.2.0
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 26
cache: pnpm
- run: pnpm install --frozen-lockfile
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('pnpm-lock.yaml') }}
- name: Install browsers
run: pnpm exec e2e-web install chromium --with-deps
- name: Run e2e
run: npx e2e run --reporter list,junit
env:
APP_URL: http://127.0.0.1:3000
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
E2E_USER_ADMIN_USERNAME: ${{ secrets.E2E_USER_ADMIN_USERNAME }}
E2E_USER_ADMIN_PASSWORD: ${{ secrets.E2E_USER_ADMIN_PASSWORD }}
- name: Upload report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-report
path: |
.e2e/report.json
.e2e/junit.xml
if-no-files-found: warn
- name: Upload artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: e2e-artifacts
path: .e2e/results
if-no-files-found: warn
retention-days: 7
```
`npx e2e` runs the version the lockfile installs. The
[install command](/reference/web#installing-browsers) downloads the browsers
for the engine's pinned Playwright version, with their system libraries; the
cache step skips the download until the lockfile changes. A
test that passed on retry keeps its failed attempt's evidence in the upload.
## Cache the timings history
Restore and save [`.e2e/timings.json`](/ci#cache-the-timings-history) with
`actions/cache`, so a fresh runner starts the longest files first. Add the
step before the run:
```yaml
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .e2e/timings.json
key: e2e-timings-${{ github.ref_name }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
e2e-timings-${{ github.ref_name }}-
e2e-timings-main-
```
Each run, and each rerun, saves a new entry and restores the newest one for its branch. A pull
request can read the base branch's cache but not write to it, so a first run
on a new branch starts from the history of `main`.
The step saves after the job passes. To save after a failed run too, use
`actions/cache/restore` before the run and `actions/cache/save` after it with
`if: ${{ !cancelled() }}`. The history only helps with
[`workers`](/reference/config#workers) of 2 or more, and the workflow above
runs one.
## iOS and Android
This workflow runs one job for each platform and follows the rules in
[Mobile jobs](/ci#mobile-jobs). Any build command that writes a simulator
`.app` without starting the app can replace `xcodebuild`. For Expo,
`npx expo run:ios --configuration Release --device generic --no-bundler --output build/ios`
builds without a simulator and writes the `.app` under `build/ios`.
```yaml title=".github/workflows/e2e-mobile.yml" expandable
name: e2e mobile
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
jobs:
ios:
runs-on: macos-26
timeout-minutes: 60
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 26
cache: npm
- run: npm ci
- run: pod install
working-directory: ios
- name: Build for the simulator
run: |
xcodebuild -workspace ios/MyApp.xcworkspace -scheme MyApp \
-configuration Release -sdk iphonesimulator \
-destination 'generic/platform=iOS Simulator' \
-derivedDataPath build
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.agent-device/apple-runner/derived
key: agent-device-runner-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Boot a simulator and install the app
run: |
UDID=$(xcrun simctl create e2e "iPhone 17" com.apple.CoreSimulator.SimRuntime.iOS-26-5)
xcrun simctl bootstatus "$UDID" -b
xcrun simctl install "$UDID" build/Build/Products/Release-iphonesimulator/MyApp.app
- run: npx e2e run --target ios --reporter list,junit
env:
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: e2e-ios
path: .e2e
if-no-files-found: warn
android:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 26
cache: npm
- uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6.1.0
with:
distribution: temurin
java-version: 17
- run: npm ci
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
# React Native and Expo templates sign release builds with the debug keystore.
- name: Build the APK
run: cd android && ./gradlew :app:assembleRelease
- name: Run on an emulator
uses: reactivecircus/android-emulator-runner@a421e43855164a8197daf9d8d40fe71c6996bb0d # v2.38.0
with:
api-level: 36
arch: x86_64
target: google_apis
profile: pixel_7
emulator-options: -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim
disable-animations: true
script: |
adb install -r android/app/build/outputs/apk/release/app-release.apk
npx e2e run --target android --reporter list,junit
env:
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: e2e-android
path: .e2e
if-no-files-found: warn
```
The emulator runner action runs each `script` line in a new shell and stops
the emulator when the step ends. Install and run in the same step.
This repository's own
[`mobile.yml`](https://github.com/e2e-inc/e2e/blob/main/.github/workflows/mobile.yml)
adds a cached Expo build per native fingerprint, two simulators for two
workers, and a `--last-failed` rerun.
## Pull request comments
Add the [`@e2e-dev/github` reporter](/github) to post each job's results as
one pull request comment, updated on a rerun and copied to the job summary.
The job needs `pull-requests: write` and `GITHUB_TOKEN` in the test step's
env; the workflows above are read-only. See [Setup](/github#setup). A fork's
`pull_request` run gets a read-only token, so nothing is posted
([why](/github#when-nothing-is-posted)).
## Next
<CardGroup cols={2}>
<Card title="Pull request comments" icon="code-pull-request" href="/github">
One comment per run, edited in place on a rerun.
</Card>
<Card title="Continuous integration" icon="rotate" href="/ci">
CI defaults, sharding, reruns, and exit codes.
</Card>
</CardGroup>