mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-21 03:05:26 +08:00
* feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init e2e init asks for a CI pipeline in a project with the web engine (--ci github-actions|gitlab|none). It writes one job for the project's package manager and Node version that installs Chromium, runs e2e run, and keeps the report, and names the secret or CI/CD variable for the model key. It never overwrites a pipeline file; beside an existing .gitlab-ci.yml it writes a file to include. * fix(cli): handle cubic's findings on the init CI step Exact GitLab stage names, a distinct job name beside an existing e2e job, loopback baseURL in an existing config, no writes through a symlinked .github or .gitlab, --ci none in the skill, gitlab in the telemetry comment, and the GitLab Node image's major version in the docs. * refactor(cli): keep the init CI pipelines as YAML templates The GitHub Actions workflow and the GitLab job live in packages/e2e/templates/ci/ as complete pipelines for npm at the repository root, linted as they are. init loads them with the yaml package's Document API and edits a copy for the project's package manager, folders, Node version, secrets, and target, keeping the comments. The pinned actions now live only in the templates. * fix(cli): point setup-bun at the package.json of a project in a subfolder * docs(ci): link the GitLab page from init and the GitLab template * feat(cli): write the iOS and Android CI jobs in e2e init init writes one job per platform the config's targets run on: web targets in the web job, iOS and Android targets in their own jobs, which build the app, boot a simulator or an emulator, and install the app. Targets come from the new config, or from loading an existing one after the confirmation and the install. Expo and bare React Native apps get their build; any other app gets a step to replace. GitHub's mobile jobs follow e2e-inc/e2e-react-native's workflow; GitLab's run on self-managed runners. * fix(cli): reinstall an emulator system image that holds no image in the Android CI job * fix(cli): handle cubic's findings on the mobile CI jobs Upload the emulator log from the GitLab Android job, count peer and optional engine dependencies, ask nothing for a new config with no engine, key caches on Bun's bun.lockb when that is the lockfile, and run the symlink test with a Windows junction. * fix(cli): put current Android command-line tools in place before the emulator image The runner image's cmdline-tools can be older than the system image's package format, and android-emulator-runner uses them to create the AVD, which then fails with 'contains no system images'. The Android job downloads command-line tools 23.0, checks its SHA-256, and installs the image with it. * fix(cli): import moved core modules from @e2e-dev/core in init * fix(cli): count a pipeline as set up only when a command runs e2e run init matched the text 'e2e run' anywhere in a workflow, so a comment or a step name such as 'our old e2e runner' made --ci write nothing. It now reads the commands (run, script, before_script, after_script) from the parsed YAML and matches e2e run as words of their own. Also: --ci help and the InitOptions JSDoc say tests on every platform, and yaml is pinned exactly like the other dependencies. * chore(ci): bump setup-node and upload-artifact pins in the init templates, docs, and skill docs/ci/vercel-previews.mdx on main pins setup-node v7.1.0 and upload-artifact v7.0.2, the pins this repository's own workflows run. The templates, three docs pages, and the skill still had v7.0.0 and v7.0.1, so the pin test failed once main was merged in. --------- Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
267 lines
9.8 KiB
Plaintext
267 lines
9.8 KiB
Plaintext
---
|
|
title: GitHub Actions
|
|
description: Run web, iOS, and Android tests on GitHub Actions, upload the reports, and post the results on the pull request.
|
|
"og:image": "https://e2e.com/docs/images/og/ci/github-actions.png?v=1a59aebfc8"
|
|
"twitter:image": "https://e2e.com/docs/images/og/ci/github-actions.png?v=1a59aebfc8"
|
|
---
|
|
|
|
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
|
|
export const setupPrompt = `Add a GitHub Actions workflow that runs this project's e2e tests. Docs: \`https://e2e.com/docs/ci/github-actions.md\` and \`https://e2e.com/docs/ci.md\`
|
|
|
|
1. Read both pages. Use the web workflow for web targets and the iOS and Android workflow for mobile targets.
|
|
2. Adapt the workflow to this project: package manager, Node version, how the app starts or builds, and the app paths.
|
|
3. For a mobile job, replace the iOS runtime with one the runner image has, and keep it pinned.
|
|
4. List the repository secrets the workflow reads, such as the model key, and ask me to add them.
|
|
5. After I push, read the job log and fix errors until the job passes.`;
|
|
|
|
These two workflows run web targets, and iOS and Android targets. For CI
|
|
defaults, model keys, and exit codes, see [Continuous integration](/ci).
|
|
`e2e init --ci github-actions` writes these jobs for your targets and
|
|
package manager, with the build for an Expo or bare React Native app; see
|
|
[CI pipeline](/reference/cli#ci-pipeline).
|
|
|
|
<SetupPrompt>
|
|
|
|
<Prompt description={setupPrompt} children={setupPrompt} actions={["copy", "cursor"]} />
|
|
|
|
</SetupPrompt>
|
|
|
|
## Web
|
|
|
|
This workflow runs deterministic and agent tests in one job. It assumes a
|
|
config [command](/web#starting-your-app) that starts the app, and AI
|
|
Gateway for the model calls. The model key and user credentials come from
|
|
repository secrets:
|
|
|
|
```yaml title=".github/workflows/e2e.yml" expandable
|
|
name: e2e
|
|
|
|
on:
|
|
pull_request:
|
|
push:
|
|
branches: [main]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
|
|
|
jobs:
|
|
e2e:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
- uses: pnpm/action-setup@9fd676a19091d4595eefd76e4bd31c97133911f1 # v4.2.0
|
|
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
|
|
with:
|
|
node-version: 26
|
|
cache: pnpm
|
|
- run: pnpm install --frozen-lockfile
|
|
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('pnpm-lock.yaml') }}
|
|
|
|
- name: Install browsers
|
|
run: pnpm exec e2e-web install chromium --with-deps
|
|
|
|
- name: Run e2e
|
|
run: npx e2e run --reporter list,junit
|
|
env:
|
|
APP_URL: http://127.0.0.1:3000
|
|
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
|
|
E2E_USER_ADMIN_USERNAME: ${{ secrets.E2E_USER_ADMIN_USERNAME }}
|
|
E2E_USER_ADMIN_PASSWORD: ${{ secrets.E2E_USER_ADMIN_PASSWORD }}
|
|
|
|
- name: Upload report
|
|
if: ${{ !cancelled() }}
|
|
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
|
with:
|
|
name: e2e-report
|
|
path: |
|
|
.e2e/report.json
|
|
.e2e/junit.xml
|
|
if-no-files-found: warn
|
|
|
|
- name: Upload artifacts
|
|
if: ${{ !cancelled() }}
|
|
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
|
with:
|
|
name: e2e-artifacts
|
|
path: .e2e/results
|
|
if-no-files-found: warn
|
|
retention-days: 7
|
|
```
|
|
|
|
`npx e2e` runs the version the lockfile installs. The
|
|
[install command](/reference/web#installing-browsers) downloads the browsers
|
|
for the engine's pinned Playwright version, with their system libraries; the
|
|
cache step skips the download until the lockfile changes. A
|
|
test that passed on retry keeps its failed attempt's evidence in the upload.
|
|
|
|
## Cache the timings history
|
|
|
|
Restore and save [`.e2e/timings.json`](/ci#cache-the-timings-history) with
|
|
`actions/cache`, so a fresh runner starts the longest files first. Add the
|
|
step before the run:
|
|
|
|
```yaml
|
|
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: .e2e/timings.json
|
|
key: e2e-timings-${{ github.ref_name }}-${{ github.run_id }}-${{ github.run_attempt }}
|
|
restore-keys: |
|
|
e2e-timings-${{ github.ref_name }}-
|
|
e2e-timings-main-
|
|
```
|
|
|
|
Each run, and each rerun, saves a new entry and restores the newest one for its branch. A pull
|
|
request can read the base branch's cache but not write to it, so a first run
|
|
on a new branch starts from the history of `main`.
|
|
|
|
The step saves after the job passes. To save after a failed run too, use
|
|
`actions/cache/restore` before the run and `actions/cache/save` after it with
|
|
`if: ${{ !cancelled() }}`. The history only helps with
|
|
[`workers`](/reference/config#workers) of 2 or more, and the workflow above
|
|
runs one.
|
|
|
|
## iOS and Android
|
|
|
|
This workflow runs one job for each platform and follows the rules in
|
|
[Mobile jobs](/ci#mobile-jobs). Any build command that writes a simulator
|
|
`.app` without starting the app can replace `xcodebuild`. For Expo,
|
|
`npx expo run:ios --configuration Release --device generic --no-bundler --output build/ios`
|
|
builds without a simulator and writes the `.app` under `build/ios`.
|
|
|
|
```yaml title=".github/workflows/e2e-mobile.yml" expandable
|
|
name: e2e mobile
|
|
|
|
on:
|
|
pull_request:
|
|
push:
|
|
branches: [main]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
ios:
|
|
runs-on: macos-26
|
|
timeout-minutes: 60
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
|
|
with:
|
|
node-version: 26
|
|
cache: npm
|
|
- run: npm ci
|
|
- run: pod install
|
|
working-directory: ios
|
|
|
|
- name: Build for the simulator
|
|
run: |
|
|
xcodebuild -workspace ios/MyApp.xcworkspace -scheme MyApp \
|
|
-configuration Release -sdk iphonesimulator \
|
|
-destination 'generic/platform=iOS Simulator' \
|
|
-derivedDataPath build
|
|
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ~/.agent-device/apple-runner/derived
|
|
key: agent-device-runner-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
|
|
|
|
- name: Boot a simulator and install the app
|
|
run: |
|
|
UDID=$(xcrun simctl create e2e "iPhone 17" com.apple.CoreSimulator.SimRuntime.iOS-26-5)
|
|
xcrun simctl bootstatus "$UDID" -b
|
|
xcrun simctl install "$UDID" build/Build/Products/Release-iphonesimulator/MyApp.app
|
|
|
|
- run: npx e2e run --target ios --reporter list,junit
|
|
env:
|
|
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
|
|
|
|
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
|
if: ${{ !cancelled() }}
|
|
with:
|
|
name: e2e-ios
|
|
path: .e2e
|
|
if-no-files-found: warn
|
|
|
|
android:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 60
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
|
|
with:
|
|
node-version: 26
|
|
cache: npm
|
|
- uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6.1.0
|
|
with:
|
|
distribution: temurin
|
|
java-version: 17
|
|
- run: npm ci
|
|
|
|
- name: Enable KVM
|
|
run: |
|
|
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
|
|
sudo udevadm control --reload-rules
|
|
sudo udevadm trigger --name-match=kvm
|
|
# React Native and Expo templates sign release builds with the debug keystore.
|
|
- name: Build the APK
|
|
run: cd android && ./gradlew :app:assembleRelease
|
|
|
|
- name: Run on an emulator
|
|
uses: reactivecircus/android-emulator-runner@a421e43855164a8197daf9d8d40fe71c6996bb0d # v2.38.0
|
|
with:
|
|
api-level: 36
|
|
arch: x86_64
|
|
target: google_apis
|
|
profile: pixel_7
|
|
emulator-options: -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim
|
|
disable-animations: true
|
|
script: |
|
|
adb install -r android/app/build/outputs/apk/release/app-release.apk
|
|
npx e2e run --target android --reporter list,junit
|
|
env:
|
|
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
|
|
|
|
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
|
if: ${{ !cancelled() }}
|
|
with:
|
|
name: e2e-android
|
|
path: .e2e
|
|
if-no-files-found: warn
|
|
```
|
|
|
|
The emulator runner action runs each `script` line in a new shell and stops
|
|
the emulator when the step ends. Install and run in the same step.
|
|
|
|
This repository's own
|
|
[`mobile.yml`](https://github.com/e2e-inc/e2e/blob/main/.github/workflows/mobile.yml)
|
|
adds a cached Expo build per native fingerprint, two simulators for two
|
|
workers, and a `--last-failed` rerun.
|
|
|
|
## Pull request comments
|
|
|
|
Add the [`@e2e-dev/github` reporter](/github) to post each job's results as
|
|
one pull request comment, updated on a rerun and copied to the job summary.
|
|
|
|
The job needs `pull-requests: write` and `GITHUB_TOKEN` in the test step's
|
|
env; the workflows above are read-only. See [Setup](/github#setup). A fork's
|
|
`pull_request` run gets a read-only token, so nothing is posted
|
|
([why](/github#when-nothing-is-posted)).
|
|
|
|
## Next
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Pull request comments" icon="code-pull-request" href="/github">
|
|
One comment per run, edited in place on a rerun.
|
|
</Card>
|
|
<Card title="Continuous integration" icon="rotate" href="/ci">
|
|
CI defaults, sharding, reruns, and exit codes.
|
|
</Card>
|
|
</CardGroup>
|