--- title: Pull request comments description: The @e2e-dev/github reporter posts one comment per run from GitHub Actions and keeps it current. --- `@e2e-dev/github` posts test results as a pull request comment, including failures, source links, and links to workflow artifacts. Reruns update the same comment, and a `--last-failed` rerun updates it as one run. The reporter also writes the results to the job summary. ## Setup ```bash npm npm install --save-dev @e2e-dev/github ``` ```bash pnpm pnpm add -D @e2e-dev/github ``` ```bash bun bun add -d @e2e-dev/github ``` ```ts title="e2e.config.ts" import type { E2EConfig } from 'e2e'; import { web } from '@e2e-dev/web'; import { github } from '@e2e-dev/github'; export default { targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }], reporters: ['list', github()], } satisfies E2EConfig; ``` Give the workflow permission to write pull request comments and pass `GITHUB_TOKEN` to the test step: ```yaml title=".github/workflows/e2e.yml" permissions: contents: read pull-requests: write jobs: e2e: steps: # ...checkout, install, browsers - name: Run e2e run: npx e2e run --reporter list,junit env: GITHUB_TOKEN: ${{ github.token }} - name: Upload artifacts if: ${{ !cancelled() }} uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: e2e-artifacts path: .e2e/artifacts ``` Upload artifacts even when a retry passes. Each failure's evidence links open the run's Artifacts section; inside the download, every test has a directory named after it and the agent that ran it (`web/tests_billing.e2e.ts__upgrades/default/attempt-0/`), so keep `path: .e2e/artifacts` and the download lines up with the report. A `--last-failed` rerun keeps the first run's directories and puts its own under `rerun-1/`, so a flaky test's failed attempt still links to its own files. A project below the checkout root, such as `packages/e2e-tests`, gets source links under that path. The checkout root is the nearest directory holding `.git` inside `GITHUB_WORKSPACE`, so `actions/checkout` with a `path` input works too. ## The comment ```markdown ### 🔴 e2e: 1 failed, 1 flaky, 3 passed, 1 skipped 9 agent steps · 4 replayed from cache · 31 model calls · 128k tokens (62% cached) · $0.31 **🔴 billing › upgrades to Pro** [tests/billing.e2e.ts:12](https://github.com/acme/shop/blob/1a2b3c/tests/billing.e2e.ts#L12) **ASSERTION_FAILED** at step 4 of 6: `agent.act` "Pick the Pro plan and pay", after 38.0s and 12 model calls > expected heading "Your cart" to be visible > The Pay button opened a page that still shows the Free plan. - Failed the same way on both attempts: **ASSERTION_FAILED** at step 4. - Turn 9: `tap({"target":"n41"})` → Screen changed: /checkout/plan. New: #n50 heading "Free plan" - Screen: `/checkout/plan` Evidence: [screenshot](https://github.com/acme/shop/actions/runs/123#artifacts), [video](https://github.com/acme/shop/actions/runs/123#artifacts), [trace](https://github.com/acme/shop/actions/runs/123#artifacts)
⚠️ 1 flaky test passed on a retry **⚠️ invite a member by email** [tests/team.e2e.ts:21](https://github.com/acme/shop/blob/1a2b3c/tests/team.e2e.ts#L21) **STEP_TIMEOUT** at step 3 of 3: `agent.waitFor` "the invitation email", after 30.0s > the invitation email took too long Evidence: [screenshot](https://github.com/acme/shop/actions/runs/123#artifacts), [trace](https://github.com/acme/shop/actions/runs/123#artifacts)
All 6 tests in 5 files | | Test | Agent | Time | | --- | --- | --- | --- | | 🔴 | **tests/billing.e2e.ts** · 1 failed | 3 steps · 19 calls | 41.2s | | 🔴 | billing › upgrades to Pro | 3 steps · 19 calls | 41.2s | | ⚠️ | **tests/team.e2e.ts** · 1 flaky | 1 step | 6.4s | | ⚠️ | invite a member by email (1 failed attempt first) | 1 step | 6.4s | ...
e2e 0.13.0 · 2m 15s · web · [run artifacts](https://github.com/acme/shop/actions/runs/123#artifacts) ``` The comment shows test counts and agent usage, followed by each failure's error, source line, and evidence links. Flaky tests and the full test list are collapsed; the list is one table with a row per file above its tests, and its agent column appears only when the run used the agent. Run-level errors appear above the test failures. Large reports are shortened to fit GitHub's comment limit. Text from the app under test that GitHub would turn into a mention, an issue link, or a URL is shown as code, so a screen reading `@someone` or `https://...` notifies no one and links nowhere. The built-in `markdown` reporter writes the same summary to `.e2e/summary.md`. ## Rerunning the failed tests A step that gives failed tests a second chance keeps one comment for both passes: ```yaml - name: Run e2e run: npx e2e run || npx e2e run --last-failed env: GITHUB_TOKEN: ${{ github.token }} ``` The comment covers the whole suite, not only the tests the rerun ran again. A test that failed and then passed shows as flaky, a test that failed twice stays failed, one that was interrupted and then passed shows as passed, and the rest keep their first-pass results. The comment stays red while the rerun left out something it owed: a failed test, one `--max-failures` stopped before it ran, or a failed `beforeAll` or `afterAll` whose tests another filter kept out of the rerun. It shows ⏹️ when the rerun left out a test the first pass never finished. The rerun's own exit code counts only what it ran. A failed attempt's evidence links to the first run's files, which the rerun keeps. ## Matrix jobs The reporter identifies a comment by project, workflow, and job. Matrix jobs share those identifiers, so give each matrix entry a distinct `key`: ```ts reporters: ['list', github({ key: process.env.MATRIX_BROWSER })], ``` Set `MATRIX_BROWSER: ${{ matrix.browser }}` in the step's `env`. The key also appears in the headline, such as `e2e chromium: 77 passed`. Keep it stable across runs so reruns update the existing comment. ## When nothing is posted The reporter never changes the run's status or exit code. When it cannot post, it says why in one row under the summary: ``` Report .e2e/report.json GitHub not posted: set GITHUB_TOKEN in the step's env (GITHUB_TOKEN: ${{ github.token }}); written to the job summary ``` | Situation | What happens | | --- | --- | | Not on GitHub Actions | Nothing is written; the row says so. | | A `push`, `schedule`, or `workflow_dispatch` run | The job summary is written; there is no pull request to comment on. | | No `GITHUB_TOKEN` in the step's `env` | The job summary is written; the row shows the line to add. `GH_TOKEN` is read too. | | A `pull_request` event from a fork | Its token is read-only, so posting fails and the reporter explains why. | | An `issue_comment` event | On a pull request's thread, the comment is posted without source links, since the checked-out commit is the default branch. | | Missing `pull-requests: write` | The same failure, with the permission named. | The token is read when the run finishes and is excluded from the report. GitHub Enterprise Server uses `GITHUB_SERVER_URL` and `GITHUB_API_URL`.