mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-22 07:55:36 +08:00
* feat(mcp)!: e2e mcp runs headless by default and takes --headed, like run Sessions were headed outside CI and took --headless, the opposite of e2e run and e2e explore. --headless stays accepted and hidden: it now asks for the default, and a refused flag would reach an MCP host as nothing but a failed connection. * docs(skill): --headed shows the UI when the engine supports it * feat(mcp): open_session takes headed, so the agent picks per session The server's --headed sets the default for sessions that do not say, and headed: true or false overrides it, so a user who asks to watch needs no change to the MCP client config.
839 lines
45 KiB
Plaintext
839 lines
45 KiB
Plaintext
---
|
||
title: CLI
|
||
description: Commands, flags, output formats, and exit codes.
|
||
---
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e init [directory]
|
||
npx e2e login [provider]
|
||
npx e2e logout [provider]
|
||
npx e2e models [provider]
|
||
npx e2e guide [topic]
|
||
npx e2e mcp [options]
|
||
npx e2e run [files...] [options]
|
||
npx e2e explore [goal] [options]
|
||
npx e2e list [files...] [options]
|
||
npx e2e cache ls|clear|stats [--config <path>]
|
||
npx e2e feedback -m <text> [options]
|
||
npx e2e telemetry [status|enable|disable]
|
||
npx e2e --help
|
||
npx e2e --version
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm dlx e2e init [directory]
|
||
pnpm exec e2e login [provider]
|
||
pnpm exec e2e logout [provider]
|
||
pnpm exec e2e models [provider]
|
||
pnpm exec e2e guide [topic]
|
||
pnpm exec e2e mcp [options]
|
||
pnpm exec e2e run [files...] [options]
|
||
pnpm exec e2e explore [goal] [options]
|
||
pnpm exec e2e list [files...] [options]
|
||
pnpm exec e2e cache ls|clear|stats [--config <path>]
|
||
pnpm exec e2e feedback -m <text> [options]
|
||
pnpm exec e2e telemetry [status|enable|disable]
|
||
pnpm exec e2e --help
|
||
pnpm exec e2e --version
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e init [directory]
|
||
bunx e2e login [provider]
|
||
bunx e2e logout [provider]
|
||
bunx e2e models [provider]
|
||
bunx e2e guide [topic]
|
||
bunx e2e mcp [options]
|
||
bunx e2e run [files...] [options]
|
||
bunx e2e explore [goal] [options]
|
||
bunx e2e list [files...] [options]
|
||
bunx e2e cache ls|clear|stats [--config <path>]
|
||
bunx e2e feedback -m <text> [options]
|
||
bunx e2e telemetry [status|enable|disable]
|
||
bunx e2e --help
|
||
bunx e2e --version
|
||
```
|
||
</CodeGroup>
|
||
|
||
The CLI needs Node.js 24.8 or newer, or 22.22.3 or newer on Node.js 22. Any package manager installs it (npm, pnpm, Yarn with or without Plug'n'Play, Bun); the CLI itself runs on Node.js, not on the Bun or Deno runtimes (`bunx e2e` runs it on Node.js). Under Yarn Plug'n'Play on Node.js 24.8 through 24.11.0, a `.cts` file fails to load (Node.js rejects the source Yarn's loader hands on); 24.11.1 and later load it.
|
||
|
||
## Help and version
|
||
|
||
| Flag | Behavior |
|
||
| --- | --- |
|
||
| `-h`, `--help` | Prints the help of the command it follows and exits 0. `e2e help run` is the same as `e2e run --help`. |
|
||
| `-v`, `--version` | Prints the installed `e2e` version and exits 0. |
|
||
|
||
```console
|
||
$ npx e2e --version
|
||
0.14.0
|
||
```
|
||
|
||
Output is colored on a terminal or in CI, plain when piped or when `NO_COLOR`
|
||
is set. `FORCE_COLOR` keeps the colors. `e2e` without a command prints the
|
||
help on stderr and exits 2.
|
||
|
||
On a terminal at least 55 columns wide, `e2e --help` opens with the e2e
|
||
wordmark, the site's lettering drawn in block characters; piped output and
|
||
narrower terminals get the text alone.
|
||
|
||
## e2e init
|
||
|
||
Scaffolds a project and adds `e2e` to `devDependencies`. Existing
|
||
config and test files are skipped. With a `directory` argument the project is
|
||
scaffolded there.
|
||
|
||
On a terminal, the wizard opens with the e2e wordmark written in over about
|
||
a second, letter by letter, the way a pen writes it. With `--yes`, in CI, on
|
||
a dumb terminal, and on a terminal shorter than 11 rows the wordmark prints at
|
||
once, and piped output never shows it. Ctrl-C or SIGTERM while it is being
|
||
written exits 130.
|
||
|
||
For a new project, the wizard asks you to choose:
|
||
|
||
- The engine. **Web** (Playwright, the default) adds `@e2e-dev/web`.
|
||
**Mobile (iOS/Android)** adds `@e2e-dev/mobile`, defaults to iOS on
|
||
macOS and Android elsewhere, and opens Settings. **None** adds nothing.
|
||
- A model provider or subscription for agent steps. Choose Vercel AI Gateway,
|
||
OpenRouter, an OpenAI-compatible endpoint, ChatGPT, GitHub Copilot,
|
||
OpenCode Console, or SuperGrok. Choose None for tests that only use
|
||
locators. The wizard adds the required packages and model config. Subscription users then sign in
|
||
with `e2e login`. See [Models](/models) and [Subscriptions](/subscriptions).
|
||
- Where to install the agent skill (see [e2e guide](#e2e-guide)):
|
||
`.agents/skills/` for Codex, Cursor, Copilot, Gemini CLI, OpenCode, and
|
||
Zed, and `.claude/skills/` for Claude Code. Both are preselected; with
|
||
both, `.claude/skills/e2e` is a symlink to `.agents/skills/e2e`, the
|
||
layout `npx skills add` produces.
|
||
- Where to register the [MCP server](/reference/mcp): `.mcp.json` for
|
||
Claude Code and `.cursor/mcp.json` for Cursor. Both are preselected.
|
||
|
||
After confirming the file changes, choose whether to install the selected
|
||
dependencies. The installer follows the project's `packageManager` field or
|
||
lockfile, then the invoking package manager, then npm.
|
||
|
||
| Argument or flag | Type | Default | Behavior |
|
||
| --- | --- | --- | --- |
|
||
| `[directory]` | path | the working directory | Where to scaffold; created when missing |
|
||
| `-y`, `--yes` | boolean | `false` | Uses the Web engine and Vercel AI Gateway, installs the skill and MCP config in both locations, and skips dependency installation |
|
||
|
||
| Creates | Content |
|
||
| --- | --- |
|
||
| `package.json` | Created as a private ESM package when missing; otherwise only missing dependencies and scripts are added. Sibling packages (`@e2e-dev/web`, `@e2e-dev/mobile`) get the caret range of the version released with the CLI, or that exact version when the CLI is a prerelease. A `test:e2e` script runs `e2e run` unless one exists |
|
||
| `e2e.config.ts` | One target for the selected engine, with the app in its `app`. Playwright reads `APP_URL` (default `http://localhost:3000`) into `app.url` and shows `app.command` in a comment. Mobile sets `app: { bundleId: 'Settings' }` on iOS and `com.android.settings` on Android. With a gateway, `agents.default` is `{ model, system }` |
|
||
| `tests/example.e2e.ts` | An app-open test that passes against any page, plus a commented agent test. Device setup gets a Settings check instead |
|
||
| `.gitignore` | Appends `node_modules/`, `.e2e/artifacts/`, `.e2e/cache/`, `.e2e/sessions/`, `.e2e/report.json`, `.e2e/ai-trace.json`, `.e2e/junit.xml`, `.e2e/summary.md`, `.e2e/failures/`, `.e2e/logs/`, and `.e2e/videos/`. Committing the cache is opt-in; see [commit the replay cache](/cache#commit-the-replay-cache) |
|
||
| `.agents/skills/e2e/`, `.claude/skills/e2e/` | `SKILL.md` and one `references/<topic>.md` per `e2e guide` topic. With both chosen, the Claude Code directory is a relative symlink (`../../.agents/skills/e2e`) to the copy |
|
||
| `.mcp.json`, `.cursor/mcp.json` | Registers `e2e mcp` without changing other servers |
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e init --yes
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm dlx e2e init --yes
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e init --yes
|
||
```
|
||
</CodeGroup>
|
||
|
||
Re-running `init` keeps an existing config and adds no packages for it. Once
|
||
a skill directory exists, later runs refresh copies that differ from the
|
||
installed version and never add a directory the project did not choose. A
|
||
`.claude/skills/e2e/` copy from an earlier version becomes the symlink when
|
||
it holds nothing but the shipped files; one with other files in it stays a
|
||
copy and is refreshed. A skill directory that is a symlink elsewhere, or sits
|
||
under one, is never written through: `--yes` skips it with a warning naming
|
||
the link and its target, and an interactive run offers to replace a linked
|
||
skill directory with a copy, or with the link where the link goes. A
|
||
copy with a directory where a file goes, or a file where a directory goes, is
|
||
skipped with a warning naming the entry; `init` removes nothing to repair it.
|
||
|
||
Cancellation leaves files untouched and exits 0. An invalid `package.json`
|
||
exits 2 before any write. Without a terminal on stdin and stdout, `init`
|
||
without `--yes` exits 2 and names the flag.
|
||
|
||
## e2e login
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e login openai # ChatGPT, browser sign-in
|
||
npx e2e login openai --device # sign in with a device code
|
||
npx e2e login github-copilot --from-gh # use your GitHub CLI login
|
||
npx e2e login github-copilot --client-id <id> [--enterprise-url <host>]
|
||
npx e2e login opencode-console # OpenCode Console, device code
|
||
npx e2e login spacexai # SuperGrok, device code
|
||
npx e2e login # choose a provider
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e login openai # ChatGPT, browser sign-in
|
||
pnpm exec e2e login openai --device # sign in with a device code
|
||
pnpm exec e2e login github-copilot --from-gh # use your GitHub CLI login
|
||
pnpm exec e2e login github-copilot --client-id <id> [--enterprise-url <host>]
|
||
pnpm exec e2e login opencode-console # OpenCode Console, device code
|
||
pnpm exec e2e login spacexai # SuperGrok, device code
|
||
pnpm exec e2e login # choose a provider
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e login openai # ChatGPT, browser sign-in
|
||
bunx e2e login openai --device # sign in with a device code
|
||
bunx e2e login github-copilot --from-gh # use your GitHub CLI login
|
||
bunx e2e login github-copilot --client-id <id> [--enterprise-url <host>]
|
||
bunx e2e login opencode-console # OpenCode Console, device code
|
||
bunx e2e login spacexai # SuperGrok, device code
|
||
bunx e2e login # choose a provider
|
||
```
|
||
</CodeGroup>
|
||
|
||
Signs in to a subscription and stores credentials in
|
||
`$XDG_CONFIG_HOME/e2e/oauth.json`, by default `~/.config/e2e/oauth.json`. A failed sign-in exits 1, a cancelled one 130; an
|
||
unknown provider exits 2. See [Subscriptions](/subscriptions) for model
|
||
configuration.
|
||
|
||
| Argument or flag | Type | Default | Behavior |
|
||
| --- | --- | --- | --- |
|
||
| `[provider]` | `openai`, `github-copilot`, `opencode-console`, or `spacexai` | a picker | The subscription to sign in to |
|
||
| `--device` | boolean | `false` | ChatGPT: show a code to enter on another device instead of opening a browser |
|
||
| `--from-gh` | boolean | `false` | GitHub Copilot: reuse the token of the signed-in GitHub CLI |
|
||
| `--client-id <id>` | string | none | GitHub Copilot: the client id of your GitHub OAuth App with the device flow enabled |
|
||
| `--enterprise-url <host>` | string | none | GitHub Copilot: the GitHub Enterprise host |
|
||
|
||
## e2e logout
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e logout # pick one of the stored logins
|
||
npx e2e logout openai # forget one
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e logout # pick one of the stored logins
|
||
pnpm exec e2e logout openai # forget one
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e logout # pick one of the stored logins
|
||
bunx e2e logout openai # forget one
|
||
```
|
||
</CodeGroup>
|
||
|
||
## e2e models
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e models # every stored login
|
||
npx e2e models openai # one provider
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e models # every stored login
|
||
pnpm exec e2e models openai # one provider
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e models # every stored login
|
||
bunx e2e models openai # one provider
|
||
```
|
||
</CodeGroup>
|
||
|
||
Asks the vendor which models the stored login serves and prints one line per
|
||
model: the id `chatgpt()`, `copilot()`, `opencodeConsole()`, or `grok()`
|
||
takes, the vendor's name for it, and its notes (reasoning levels, vision,
|
||
preview, hidden, for Copilot `not enabled` or `no chat or responses` when
|
||
`copilot()` cannot call it, and for OpenCode Console the plan, Zen or Go,
|
||
that serves it). Without a
|
||
stored login, or when the named provider has none, exits 1 with the sign-in
|
||
command; when one of several logins fails to list, the others still print and
|
||
the command exits 1. An unknown provider exits 2. See
|
||
[Pick a model](/subscriptions#pick-a-model).
|
||
|
||
## e2e guide
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e guide # the overview
|
||
npx e2e guide setup # one topic
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e guide # the overview
|
||
pnpm exec e2e guide setup # one topic
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e guide # the overview
|
||
bunx e2e guide setup # one topic
|
||
```
|
||
</CodeGroup>
|
||
|
||
Prints the skill that `init` installs. `guide` alone prints the overview;
|
||
`guide <topic>` prints one of `setup`, `writing-tests`, `agent`, `running`,
|
||
`explore`, `debugging`, `mcp`, or `bug-bash`. An unknown topic exits 2 and names the
|
||
valid ones. The same skill installs with `npx skills add tester-army/e2e`.
|
||
See [Coding agents](/coding-agents).
|
||
|
||
## e2e mcp
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
|
||
```
|
||
</CodeGroup>
|
||
|
||
An MCP server on stdio that gives a coding agent a live session on the app.
|
||
Flags, tools, and session rules are on the [MCP server](/reference/mcp) page.
|
||
|
||
## e2e run
|
||
|
||
### Positional arguments
|
||
|
||
```bash
|
||
e2e run [files...]
|
||
```
|
||
|
||
Test files, directories, or globs relative to the project root:
|
||
|
||
- a file path (`tests/signup.e2e.ts`) selects that file;
|
||
- a directory (`tests/agent`, `.`) selects every discovered test file beneath
|
||
it;
|
||
- a bare name selects every discovered file whose path ends with it at a
|
||
segment boundary. `signup.e2e.ts` and `agent/signup.e2e.ts` both select
|
||
`tests/agent/signup.e2e.ts`. Without a `/` or `.` it matches the file name
|
||
with extensions dropped, so `signup` selects it too;
|
||
- any of these with `:line` appended (`tests/signup.e2e.ts:12`, `signup:12`)
|
||
selects only the test declared at that line of each file it names: the line
|
||
the `test(` call opens on, as an editor or a report's `source` shows it. A
|
||
serial group runs whole when a line names one member. Several lines of one
|
||
file add up; naming the file without a line selects it whole. When no test
|
||
is declared at a named line, `NO_TESTS` names the positional and the lines
|
||
the file declares tests at. A positional that exists on disk as written is
|
||
a path, even when its name ends in `:12`;
|
||
- a glob (`'tests/**/*.smoke.e2e.ts'`, quoted) uses the grammar of the config
|
||
`tests` globs: `*`, `?`, and a whole `**` segment. Braces, character
|
||
classes, and extglobs are `INVALID_GLOB` with a hint rather than a pattern
|
||
that matches nothing: there is no expansion, so name one glob per
|
||
alternative. `./`, `..`, a trailing `/`, and an absolute path inside the
|
||
project resolve as they do for any positional path, and an existing file or
|
||
directory is selected by its path whatever its name is spelled with.
|
||
|
||
Positionals narrow the config globs and the tag, platform, and capability
|
||
filters; they never bypass them. Names and globs are case-sensitive on every
|
||
OS. A path outside the project root is `COLLECTION_ERROR`.
|
||
|
||
The runner imports every discovered test file, even when you select only a
|
||
few. This lets a selected test find its session setup in another file.
|
||
Other ordinary tests remain unselected. A collection error in a selected
|
||
file, or in any file of a run with no positionals, fails the run. A file the
|
||
positionals left out that fails to import or register its tests is skipped
|
||
with a notice, so a half-written file elsewhere in the suite does not stop
|
||
`npx e2e run tests/login.e2e.ts`; if a selected test's session then has no
|
||
setup test, the error names that file as where the setup may be.
|
||
|
||
A positional starting with `-` that names no file is a usage error, exit 2.
|
||
This is what `pnpm test:e2e -- --headed` produces: the `--` reaches e2e and
|
||
`--headed` would otherwise be taken for a file. The message names the direct
|
||
form, `pnpm exec e2e run --headed`.
|
||
For npm, the direct form is `npm exec -- e2e run --headed`; npm needs the
|
||
separator before the binary to pass its flags through.
|
||
|
||
When nothing is left to run, `NO_TESTS` names the cause: no file matched the
|
||
globs (with look-alike files such as `tests/login.test.ts`), a positional
|
||
matched nothing (with the nearest path), a matched file registered no tests,
|
||
or every test was filtered out.
|
||
|
||
### Flags
|
||
|
||
| Flag | Type | Default | Behavior |
|
||
| --- | --- | --- | --- |
|
||
| `--config <path>` | string | discovered | Explicit config path. A missing file is `CONFIG_NOT_FOUND`. |
|
||
| `--target <ids>` | comma-separated or repeated | every configured target | Selects targets by name. Only the app commands of selected targets start. An unknown ID is `UNKNOWN_TARGET`. |
|
||
| `--tag <tags>` | comma-separated or repeated | none | Only tests carrying the tags: any of them, or every one with `--tag-mode all`. When the filter leaves nothing to run, `NO_TESTS` counts the tests it filtered, marks each tag no test declares, and names the nearest declared one. |
|
||
| `--tag-mode <mode>` | `any` or `all` | `any` | How several tags combine: any of them, or every one. |
|
||
| `--exclude-tag <tags>` | comma-separated or repeated | none | Leaves out every test carrying any of the tags, whatever `--tag` or a positional selected. A serial group still runs whole when any member is selected; tag the serial `describe` to exclude the flow. |
|
||
| `--grep <pattern>` | regular expression, repeatable | none | Only tests whose title matches one of the patterns. The text matched is the describe titles and the test title joined by one space (`checkout pays`); the file and the tags are not part of it. A bare pattern is case-sensitive; `'/pattern/i'` adds flags. A pattern that does not compile is a usage error, exit 2. |
|
||
| `--grep-invert <pattern>` | regular expression, repeatable | none | Leaves out every test whose title matches one of the patterns. Same text and pattern forms as `--grep`; both together keep the tests `--grep` matches minus the ones `--grep-invert` matches. |
|
||
| `--last-failed` | boolean | `false` | Only the tests the previous run did not pass, read from the `report.json` this run overwrites, `<output>/report.json` (`--output`, else the config's [`output`](/reference/config#output)): failed, timed out, interrupted, or skipped because a setup, a serial predecessor, a hook, or the worker failed or `--max-failures` stopped the run first, and every test it selected in the describe or file, on that target, of a `beforeAll` or `afterAll` that failed (for a setup, the tests that consume its session). A rerun that runs the hook again reports it again if it still fails, and a rerun that runs any test of its scope without it failing resolves it. A test another filter (`--grep`, `--tag`, `--target`, `--agent`, a positional) leaves out of the rerun stays owed: the rerun's report carries its last result, and the hook failure whose scope it is in, under `run.carried`, the next `--last-failed` selects it again, and the [`@e2e-dev/github`](/github) page stays red until it passes (⏹️ for a test the run before interrupted). The rerun's own exit code and summary count only what it ran. It keeps the artifacts the previous report names and writes its own under `artifacts/rerun-<n>/`, so no attempt overwrites an earlier one's files. A test is named per target and agent. No report is `NO_LAST_RUN`; a previous run with nothing failed selects nothing, `NO_TESTS` unless `--pass-with-no-tests`. |
|
||
| `--shard <index/total>` | `2/3` | none | Runs one contiguous slice of the selected tests, cut once every other filter applied. Slices differ in size by at most one test, a serial group counts as one and is never split, and each shard brings only the setup tests its own tests need. Every shard runs the same command with a different index; the order is the report order, so the same suite shards the same way on every machine. A shard that gets no test is `NO_TESTS` unless `--pass-with-no-tests`. See [Sharding](/ci#sharding). |
|
||
| `--headed` | boolean | `false` | Visible UI when the engine supports it. |
|
||
| `--agent <names>` | comma-separated or repeated | `default` | Runs unpinned tests once per selected agent. For a test with `{ agent }`, selects the shared names when there is overlap; otherwise keeps the test's selection. An unknown name is `INVALID_CONFIG`. |
|
||
| `--retries <n>` | integer 0-10 | config | Replaces `config.retries`. |
|
||
| `--workers <n>` | integer 1-1024 | config | Replaces `config.workers`. |
|
||
| `--repeat-each <n>` | integer, at least 1 | `1` | Runs every selected test this many times. Each run is a result of its own, with `repeat` 0 through `n - 1` and its own `id`; the list reporter titles the later runs `(repeat #1)`, `(repeat #2)`, and their artifacts go under a `repeat-<n>` directory. Setup tests run once. With the replay cache on, the first run records and the later ones replay, so pair it with `--no-cache` to exercise the model each time. The `list` summary adds a `Repeats` row (`1 of 2 tests passed all 5 runs`) and one line per test that did not pass every run, naming each run that failed; the `markdown` page carries the same tally. |
|
||
| `--max-failures <n>` | integer, at least 1 | none | Stops the run once this many tests have failed or timed out. Nothing more is dispatched, the tests running on other workers end as `interrupted`, and the tests not started, the rest of the file whose test reached the limit included, are skipped with cause `failure-limit` and the reason `run stopped after 3 failures (--max-failures 3)`. The exit code is the failures' own, 1, not 130. |
|
||
| `--reporter <ids>` | comma-separated list | config | Replaces the built-in IDs in `config.reporters`: `list`, `json`, `junit`, `markdown`. `json` with `list` is `INVALID_CONFIG`. [Reporter objects](/reference/reporters) in the config keep running. |
|
||
| `--output <dir>` | string | config, else `.e2e` | The results directory for this run, over the config's [`output`](/reference/config#output): the report, the reporter files, the AI trace, `artifacts/` (cleared once the run's tests start, down to what the previous report names on a `--last-failed` rerun), and `sessions/`. Resolved from the project root; one the run cannot own is `INVALID_CONFIG`. The removed `--artifacts <dir>` fails with the `--output` it maps to: `--artifacts out/artifacts` is `--output out`, and a directory whose parent is the project root or outside it maps to `--output .e2e`. |
|
||
| `--no-cache` | boolean | `false` | Replay cache off for this run, whatever `config.cache` says. |
|
||
| `--strict-cache` | boolean | `false` | Turns on `cache.strict` for this run: a step whose recording exists but no longer replays fails with `REPLAY_STALE` instead of handing off to the agent. See [Fail on a stale recording](/cache#fail-on-a-stale-recording). |
|
||
| `--pass-with-no-tests` | boolean | `false` | Exit 0 instead of `NO_TESTS` when nothing is selected. |
|
||
| `--debug` | boolean | `false` | Phase timings and an agent step table on stderr after the run. |
|
||
| `--ai-trace` | boolean | `false` | Records every model call to `<output>/ai-trace.json`, an AI SDK devtools database. Open it with [unbox-ai](https://github.com/tester-army/unbox-ai). |
|
||
| `--trace [mode]` | `off`, `on`, `retain-on-failure`, `on-first-retry`, or `on-all-retries` | config | Which attempts record a trace, over the config's and every target's `trace`; a test's own `trace` still wins. A bare `--trace` is `on`, `--trace off` skips the cost for one run. It applies to the targets whose engine can trace; the run names the others in one notice. The value is optional and greedy, so `--trace tests/a.e2e.ts` reads the file as a mode and fails: write `--trace=<mode>`, or put test files before `--trace`. See [trace](/reference/config#trace). |
|
||
| `--video [mode]` | a recording mode, as `--trace` | config | Which attempts record a video (WebM on a browser engine, MP4 on a device engine), over the config's and every target's `video`; a test's own `video` still wins. A bare `--video` is `on`. It applies to the targets whose engine can record; the run names the others in one notice. Greedy like `--trace`: write `--video=<mode>`, or put test files before `--video`. See [video](/reference/config#video). |
|
||
|
||
Out-of-range `--retries` and `--workers` fail with `INVALID_CONFIG` before
|
||
anything starts, the same as the config value would. `--target`, `--tag`,
|
||
`--exclude-tag`, and `--agent` take each name once; an empty value (`--tag ''`,
|
||
or `--tag "$TAGS"` with the variable unset) is a usage error, exit 2, never a
|
||
wider selection.
|
||
|
||
Filters only narrow, and each applies after the one before: positionals, then
|
||
`.only`, `--tag`, `--exclude-tag`, `--grep`, `--grep-invert`, `--last-failed`,
|
||
then the platform and capability rules, then `--shard`. When they leave
|
||
nothing to run, `NO_TESTS` counts the tests each filter removed, names the tags
|
||
and patterns, and marks a tag no test declares.
|
||
|
||
`--debug` appends two tables to stderr. Phase timings are summed across the
|
||
runner and every worker. The agent step table lists every agent step in
|
||
execution order with its per-phase split, model calls, tokens, and cost when
|
||
the provider reported one.
|
||
|
||
```console
|
||
$ npx e2e run --debug
|
||
...
|
||
[e2e debug] phase timings (wall 14.2s)
|
||
phase count total avg max
|
||
scheduler 1 11.8s 11.8s 11.8s
|
||
test.body 6 10.1s 1.7s 3.9s
|
||
session.launch 6 1.1s 183ms 410ms
|
||
engine.prepare 1 612ms 612ms 612ms
|
||
collect 1 420ms 420ms 420ms
|
||
config.load 1 118ms 118ms 118ms
|
||
realm.import 1 96ms 96ms 96ms
|
||
session.close 6 88ms 15ms 31ms
|
||
engine.init 1 74ms 74ms 74ms
|
||
|
||
[e2e debug] agent steps (execution order, model anthropic/claude-sonnet-4.5, total $0.0312)
|
||
step total model observe action calls tokens in/out cached cost
|
||
act "add a todo named groceries" 3.9s 2.8s 410ms 620ms 3 9412/388 61% (5740) $0.0198
|
||
assert "the list shows groceries" 1.2s 1.1s 88ms 0ms 1 3120/41 0% (0) $0.0114
|
||
```
|
||
|
||
`cached` is the share of input tokens the provider served from its prompt
|
||
cache, `-` when it reports no split. The report records the same per step
|
||
(`model.cacheReadTokens`, `model.cacheWriteTokens`) and per run
|
||
(`usage.modelCachedTokens`).
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e run
|
||
npx e2e run tests/signup.e2e.ts --tag smoke
|
||
npx e2e run tests/signup.e2e.ts:12
|
||
npx e2e run tests/agent
|
||
npx e2e run 'tests/**/*.smoke.e2e.ts'
|
||
npx e2e run --target web,webkit --tag smoke,billing --tag-mode all
|
||
npx e2e run --tag smoke --exclude-tag slow
|
||
npx e2e run --grep checkout --grep-invert '/refund/i'
|
||
npx e2e run --last-failed
|
||
npx e2e run --shard 2/3
|
||
npx e2e run --reporter json --workers 4 --retries 2
|
||
npx e2e run --max-failures 3
|
||
npx e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts
|
||
npx e2e run --reporter list,junit
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e run
|
||
pnpm exec e2e run tests/signup.e2e.ts --tag smoke
|
||
pnpm exec e2e run tests/signup.e2e.ts:12
|
||
pnpm exec e2e run tests/agent
|
||
pnpm exec e2e run 'tests/**/*.smoke.e2e.ts'
|
||
pnpm exec e2e run --target web,webkit --tag smoke,billing --tag-mode all
|
||
pnpm exec e2e run --tag smoke --exclude-tag slow
|
||
pnpm exec e2e run --grep checkout --grep-invert '/refund/i'
|
||
pnpm exec e2e run --last-failed
|
||
pnpm exec e2e run --shard 2/3
|
||
pnpm exec e2e run --reporter json --workers 4 --retries 2
|
||
pnpm exec e2e run --max-failures 3
|
||
pnpm exec e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts
|
||
pnpm exec e2e run --reporter list,junit
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e run
|
||
bunx e2e run tests/signup.e2e.ts --tag smoke
|
||
bunx e2e run tests/signup.e2e.ts:12
|
||
bunx e2e run tests/agent
|
||
bunx e2e run 'tests/**/*.smoke.e2e.ts'
|
||
bunx e2e run --target web,webkit --tag smoke,billing --tag-mode all
|
||
bunx e2e run --tag smoke --exclude-tag slow
|
||
bunx e2e run --grep checkout --grep-invert '/refund/i'
|
||
bunx e2e run --last-failed
|
||
bunx e2e run --shard 2/3
|
||
bunx e2e run --reporter json --workers 4 --retries 2
|
||
bunx e2e run --max-failures 3
|
||
bunx e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts
|
||
bunx e2e run --reporter list,junit
|
||
```
|
||
</CodeGroup>
|
||
|
||
### Output
|
||
|
||
| Reporter | Writes |
|
||
| --- | --- |
|
||
| `list` | Terminal progress, failures with code frames, and a run summary. On a TTY, live progress includes the current agent step, model turns, tool calls, and reasoning excerpts. Untrusted fields are capped at 8 KiB and stripped of control characters. |
|
||
| `json` | The report document to stdout and nothing else there, so `e2e run --reporter json \| jq` works. Same document as `<output>/report.json`, described by `schema/report-v1.schema.json` in the package. The shape changes only with the `schemaVersion` field (`report-1` today). Cannot be combined with `list`. |
|
||
| `junit` | `junit.xml` beside the report. One `<testsuite>` per file, one `<testcase>` per test-target pair named `title > path [target]`. A test-class error is a `<failure>`, any other an `<error>`, a skip a `<skipped>` with its reason. An interrupted test reached no verdict, so it is a `<skipped>` whose message starts `interrupted:`. Run-level errors form a `run` suite. |
|
||
| `markdown` | `summary.md` beside the report with counts, cost, errors, and failure details. Failed and flaky tests also get a page under `failures/` there, with steps, model turns, and the screen at failure. Interrupted tests are counted on their own and get no page. Exploration runs show findings. [`@e2e-dev/github`](/github) uses this summary for pull request comments. |
|
||
|
||
Every run that gets to its tests writes `report.json` under the [output directory](/reference/config#output)
|
||
(`.e2e` by default) whatever the reporters are, and the reporters' files
|
||
beside it. A run that stops before (a config or collection error,
|
||
`NO_TESTS`, `NO_LAST_RUN`, `UNSUPPORTED_ARTIFACT`, an app command that
|
||
fails to start, an interrupt) writes none of them and leaves the previous
|
||
run's files in place; its errors are on the terminal and in the exit code. A `junit.xml` or `summary.md` that could not be written is a
|
||
line on stderr, never a run error.
|
||
|
||
`run.summary` in the report counts the results. `discovered` is every
|
||
test-target pair the runner collected and `selected` the ones this run
|
||
chose; the selection's left-out pairs are in `results` as `selected: false`
|
||
but in no other count. `passed`, `failed` (timed out included),
|
||
`interrupted`, `flaky`, and `skipped` count the selected results and add up
|
||
to `selected`. `interrupted` is a test the run stopped before its verdict
|
||
(a signal, `--max-failures`, or a run-level error such as
|
||
`MODEL_UNAVAILABLE`), never a failure. `executed` is the selected
|
||
results that ran at least one attempt.
|
||
|
||
With `--ai-trace`, the run also writes `<output>/ai-trace.json`: one entry per
|
||
model round trip with the exact prompt, tool definitions, response, usage,
|
||
and provider metadata. Observations in it are already redacted. Inline image
|
||
and file bytes are replaced with byte counts. Open it with
|
||
`npx unbox-ai .e2e/ai-trace.json` for the default output; see [Debugging a run](/debugging#agent-steps).
|
||
|
||
### Signals
|
||
|
||
Signals escalate.
|
||
|
||
1. The first `SIGINT` or `SIGTERM` interrupts. The running test ends with
|
||
status `interrupted`, its `afterEach` hooks and session close run within
|
||
the cleanup budget, and every worker disposes its engine. The tests not
|
||
started are skipped with cause `infrastructure-unavailable` and the reason
|
||
`run interrupted before execution`, so the report lists every planned
|
||
test and `--last-failed` runs the interrupted and the unstarted ones
|
||
again. The runner writes the report and exits 130. An interrupt before
|
||
any test starts (during collection, an engine's `prepare`, or app
|
||
startup) writes no report and leaves the previous one in place.
|
||
2. A second signal forces. Workers dispose their engines at once and are
|
||
killed when the cleanup budget is spent. The report is still written,
|
||
and so are `junit.xml` and `summary.md` when the `junit` and
|
||
`markdown` reporters are on; a custom reporter's
|
||
`onRunFinished` is abandoned.
|
||
3. A third signal exits on the spot, after killing the process groups of every
|
||
app command the run started. A process attached through
|
||
`reuseExisting` is left alone.
|
||
|
||
A worker whose runner disappears disposes its engine and exits within the
|
||
cleanup budget.
|
||
|
||
## e2e explore
|
||
|
||
```bash
|
||
e2e explore [goal]
|
||
```
|
||
|
||
A run with no test file. The agent gets a goal, plans one exploration step
|
||
at a time, reports defects through its `report_finding` tool, and ends with an
|
||
assessment. The default goal is `Explore the app and find bugs`. See
|
||
[Exploring without a test](/explore).
|
||
|
||
| Flag | Type | Default | Behavior |
|
||
| --- | --- | --- | --- |
|
||
| `--config <path>` | string | discovered | Explicit config path, with the same errors as `run`. |
|
||
| `--target <id>` | string | the first configured target | The one target to explore. Several targets and no flag explores the first and says so on stderr. An unknown name is `UNKNOWN_TARGET`. |
|
||
| `--agent <name>` | string | `default` | The configured agent the exploration runs as. Its model, tools, and context are reused; a hand-rolled `StepExecutor` is replaced by the built-in agent, with a notice. An unknown name is `INVALID_CONFIG`. |
|
||
| `--session <name>` | string | none | Starts signed in. The setup test in the config's test files that declares this session runs first, and the exploration restores what it saved, as a test with `{ session }` does. No other test runs. A name no setup test declares is `COLLECTION_ERROR`, naming the declared ones, before any app process starts. See [Start signed in](/explore#start-signed-in). |
|
||
| `--max-steps <n>` | integer 1-12 | `8` | Exploration steps at most. Out of range exits 2. |
|
||
| `--timeout <ms>` | integer 180000-900000 | `600000` | The run's wall clock. The last minute is kept for the closing assessment. Out of range exits 2. |
|
||
| `--headed` | boolean | `false` | Requests visible UI when the engine supports it. |
|
||
| `--reporter <ids>` | comma-separated list | resolved config value | As for `run`. The exploration summary rows print under the `list` summary; `json` carries the record in the report. |
|
||
| `--output <dir>` | string | config, else `.e2e` | As for `run`; a finding's evidence screenshot is one of the attempt's artifacts under `<output>/artifacts`. |
|
||
| `--debug` | boolean | `false` | As for `run`: phase timings and the agent step table. |
|
||
| `--ai-trace` | boolean | `false` | As for `run`. |
|
||
| `--trace [mode]` | a recording mode | config | As for `run`: one trace of the whole exploration. A bare `--trace` is `on`; `retain-on-failure` keeps it only when the exploration did not pass. The exploration is one attempt, so `on-first-retry` and `on-all-retries` record nothing, and neither does the CI default: `CI=1 e2e explore` needs `--trace on` for a trace, and its plan-time notice says so. Put the goal before `--trace`, or write `--trace=<mode>`. |
|
||
| `--video [mode]` | a recording mode | config | As for `run`: one video of the whole exploration (WebM on a browser engine, MP4 on a device engine). A bare `--video` is `on`; `retain-on-failure` keeps it only when the exploration did not pass. The exploration is one attempt, so a retry mode records nothing. Put the goal before `--video`, or write `--video=<mode>`. |
|
||
|
||
The run has one result, under the virtual file `explore`, titled by the goal.
|
||
With `--session`, the report also holds the setup test's result and lists the
|
||
project's other tests as skipped with cause `filtered`, as `run` lists the
|
||
tests of a file no positional named.
|
||
The exploration runs with the replay cache off and zero retries whatever the
|
||
config says. A `--session` setup test runs as it would for `run`, with the
|
||
config's agents, cache, and retries.
|
||
`<output>/report.json` gains `run.explore`: the goal, the budgets, why the run
|
||
ended (`finished`, `step-limit`, `time`, `stuck`, or `aborted`), the
|
||
assessment, the steps, and the findings.
|
||
|
||
Exit codes: `0` when steps ran and no finding of kind `issue` was reported,
|
||
`1` when one was or when no step ran at all, `2` and `3` as for `run`, `130`
|
||
on Ctrl-C.
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e explore
|
||
npx e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'
|
||
npx e2e explore --target web --max-steps 4 --headed
|
||
npx e2e explore 'Hunt for broken forms and dead links' --agent thorough --video
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e explore
|
||
pnpm exec e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'
|
||
pnpm exec e2e explore --target web --max-steps 4 --headed
|
||
pnpm exec e2e explore 'Hunt for broken forms and dead links' --agent thorough --video
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e explore
|
||
bunx e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'
|
||
bunx e2e explore --target web --max-steps 4 --headed
|
||
bunx e2e explore 'Hunt for broken forms and dead links' --agent thorough --video
|
||
```
|
||
</CodeGroup>
|
||
|
||
## e2e list
|
||
|
||
```bash
|
||
e2e list [files...]
|
||
```
|
||
|
||
A dry run. `list` selects tests exactly as `run` does, prints one line per
|
||
test-target pair, and exits. No app process, engine, or worker starts, and
|
||
nothing is written.
|
||
|
||
Positionals and the selection flags (`--config`, `--target`, `--tag`,
|
||
`--tag-mode`, `--exclude-tag`, `--grep`, `--grep-invert`, `--last-failed`,
|
||
`--shard`, `--pass-with-no-tests`) are those of `run`; `--last-failed` reads
|
||
`report.json` under the config's [`output`](/reference/config#output). `--reporter` accepts `list` (the default) or `json`.
|
||
|
||
| Reporter | Writes |
|
||
| --- | --- |
|
||
| `list` | One line per pair, `file › title [target]`, in selection order, with the describe path joined by ` › ` (`tests/billing.e2e.ts › checkout › pays [web]`), then each tag the test declares as `#tag` (`tests/billing.e2e.ts › checkout › pays [web] #smoke #billing`). A skipped test keeps its line with `(skipped: <reason>)` appended. Setup tests are listed like any other. |
|
||
| `json` | `{ "pairs": [...] }` to stdout. Each pair carries `file`, `title`, `titlePath`, `kind` (`test` or `setup`), `tags` (the tags the test declares, `[]` when none), `target`, `disposition` (`run` or `skip`), and `skipReason` when skipped. |
|
||
|
||
Pairs a filter removed are not listed.
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e list
|
||
npx e2e list tests/signup.e2e.ts
|
||
npx e2e list --tag smoke --target web
|
||
npx e2e list --grep checkout --exclude-tag slow
|
||
npx e2e list --reporter json
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e list
|
||
pnpm exec e2e list tests/signup.e2e.ts
|
||
pnpm exec e2e list --tag smoke --target web
|
||
pnpm exec e2e list --grep checkout --exclude-tag slow
|
||
pnpm exec e2e list --reporter json
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e list
|
||
bunx e2e list tests/signup.e2e.ts
|
||
bunx e2e list --tag smoke --target web
|
||
bunx e2e list --grep checkout --exclude-tag slow
|
||
bunx e2e list --reporter json
|
||
```
|
||
</CodeGroup>
|
||
|
||
## e2e cache
|
||
|
||
Inspect or clear the replay cache. These commands load the discovered config
|
||
or `--config <path>`, honor `cache.dir`, and exit 0 on an empty store. The
|
||
default directory is `.e2e/cache/`.
|
||
|
||
| Command | Prints |
|
||
| --- | --- |
|
||
| `e2e cache ls` | One row per entry: the test and target it was recorded for, the first 12 characters of its instruction digest, its age, and how many actions a replay would run (`(truncated)` when recording hit the action cap). Rows are grouped by test, then target, oldest first. |
|
||
| `e2e cache stats` | The store directory, the number of readable entries, their total size, and the number of unreadable files when there are any. |
|
||
| `e2e cache clear` | Deletes every cache file, then the directory when nothing else is left in it, and reports how many files went. |
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e cache ls
|
||
npx e2e cache stats
|
||
npx e2e cache clear
|
||
npx e2e cache ls --config packages/web/e2e.config.ts
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e cache ls
|
||
pnpm exec e2e cache stats
|
||
pnpm exec e2e cache clear
|
||
pnpm exec e2e cache ls --config packages/web/e2e.config.ts
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e cache ls
|
||
bunx e2e cache stats
|
||
bunx e2e cache clear
|
||
bunx e2e cache ls --config packages/web/e2e.config.ts
|
||
```
|
||
</CodeGroup>
|
||
|
||
```text
|
||
TEST TARGET INSTRUCTION AGE ACTIONS
|
||
tests/billing.e2e.ts::upgrades web 4f1c0a9d3b62 2h 4
|
||
tests/signup.e2e.ts::signs%20up web 9ab7715ce004 3d 7 (truncated)
|
||
```
|
||
|
||
An entry recorded before entries carried provenance prints `-` in those
|
||
columns and still replays. A file the store cannot read (truncated, over the
|
||
1 MiB ceiling, or from a newer schema) is counted separately on stderr, treated
|
||
as a miss by a run, and removed by `clear`. Files the runner never wrote are
|
||
left alone.
|
||
|
||
<Note>
|
||
These commands read the file store only. With a custom `cache.store` they
|
||
exit 2 and say so.
|
||
</Note>
|
||
|
||
## e2e feedback
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e feedback -m <text> [--type bug|docs|feature|other] [options]
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e feedback -m <text> [--type bug|docs|feature|other] [options]
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e feedback -m <text> [--type bug|docs|feature|other] [options]
|
||
```
|
||
</CodeGroup>
|
||
|
||
Sends one report about e2e itself to the e2e team: something broke, the docs
|
||
or the skill misled, or a capability is missing. Coding agents are welcome to
|
||
send one when they hit a problem with e2e, not with the app under test.
|
||
|
||
| Flag | Meaning |
|
||
| --- | --- |
|
||
| `-m, --message <text>` | Required. One or two sentences on the problem, at most 2000 characters |
|
||
| `--type <type>` | `bug`, `docs`, `feature`, or `other` (default) |
|
||
| `--task <text>` | What you were trying to do |
|
||
| `--expected <text>` | What you expected to happen |
|
||
| `--actual <text>` | What happened instead, with the error code and message |
|
||
| `--approach <text>` | What you tried, workarounds included |
|
||
| `--command <text>` | The e2e command or API involved, at most 200 characters |
|
||
| `--agent <text>` | The coding agent and model sending it, at most 200 characters |
|
||
| `--dry-run` | Print the event that would be sent and send nothing |
|
||
|
||
`--task`, `--expected`, `--actual`, and `--approach` take up to 4000
|
||
characters each. A blank or longer value exits 2.
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout"
|
||
npx e2e feedback --type docs -m "The mcp topic never says how to close a session"
|
||
npx e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout"
|
||
pnpm exec e2e feedback --type docs -m "The mcp topic never says how to close a session"
|
||
pnpm exec e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout"
|
||
bunx e2e feedback --type docs -m "The mcp topic never says how to close a session"
|
||
bunx e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run
|
||
```
|
||
</CodeGroup>
|
||
|
||
The report goes to the same PostHog project as [telemetry](/telemetry#feedback),
|
||
as one `e2e_feedback` event, together with the anonymous machine facts every
|
||
telemetry event carries. Before sending, the value of every environment
|
||
variable whose name holds `KEY`, `TOKEN`, `SECRET`, `PASS`, `_PWD`,
|
||
`CREDENTIAL`, `AUTH`, `COOKIE`, or `PRIVATE` becomes `<secret:NAME>` when it
|
||
is at least six characters long, the minimum for a config secret, since a
|
||
shorter one would rewrite ordinary words. Bearer tokens,
|
||
URL credentials, private keys, JWTs, and common API key shapes become
|
||
`<redacted>`. Do not paste app data, page content, or test files.
|
||
|
||
Feedback is sent only when you run the command, so a saved
|
||
`e2e telemetry disable` does not stop it. `E2E_TELEMETRY_DISABLED=1` and
|
||
`DO_NOT_TRACK=1` do: with either set the command exits 2 and sends nothing,
|
||
so a CI job or a fleet that opted out sends nothing even when an agent runs
|
||
it. `E2E_TELEMETRY_DEBUG=1` prints it like `--dry-run`. It prints a reference
|
||
id and exits 0 once PostHog accepts the event, or exits 3 when PostHog did
|
||
not confirm it; the report may still have arrived, so the message names its
|
||
reference id.
|
||
|
||
## e2e telemetry
|
||
|
||
Shows whether anonymous usage telemetry is on, and switches it.
|
||
[Telemetry](/telemetry) lists every property sent.
|
||
|
||
| Command | Effect |
|
||
| --- | --- |
|
||
| `e2e telemetry`, `e2e telemetry status` | Prints `Status: enabled`, or `Status: disabled` with the reason: the variable that is set, the saved choice, or a preferences directory that cannot be written. |
|
||
| `e2e telemetry disable` | Saves the opt-out to `~/.config/e2e/telemetry.json` (`$XDG_CONFIG_HOME/e2e/` when set, `%APPDATA%\e2e\` on Windows) and prints the status. |
|
||
| `e2e telemetry enable` | Saves the opt-in and prints the status. A variable set in the environment still wins, and the status says so. |
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e telemetry
|
||
npx e2e telemetry disable
|
||
E2E_TELEMETRY_DEBUG=1 npx e2e run
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e telemetry
|
||
pnpm exec e2e telemetry disable
|
||
E2E_TELEMETRY_DEBUG=1 pnpm exec e2e run
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e telemetry
|
||
bunx e2e telemetry disable
|
||
E2E_TELEMETRY_DEBUG=1 bunx e2e run
|
||
```
|
||
</CodeGroup>
|
||
|
||
`E2E_TELEMETRY_DISABLED=1` and `DO_NOT_TRACK=1` turn telemetry off without
|
||
touching the file. `E2E_TELEMETRY_DEBUG=1` prints every event to stderr and
|
||
sends nothing. The first command on a machine other than `e2e init` and
|
||
`e2e telemetry` prints a notice once; in CI nothing is printed and no file is written.
|
||
|
||
## Exit codes
|
||
|
||
| Code | Meaning |
|
||
| ---: | --- |
|
||
| 0 | All selected tests passed, were flaky, or were explicitly skipped |
|
||
| 1 | Final test or setup failure, or test timeout |
|
||
| 2 | CLI, config, collection, dependency, credential, model-config, or policy error |
|
||
| 3 | Engine, app process, model provider, artifact, or cleanup infrastructure failure; `e2e feedback` that could not be delivered |
|
||
| 4 | Internal runner invariant or unhandled runner error |
|
||
| 130 | Interrupted by a user or CI signal |
|
||
|
||
Precedence for a mixed run is `130 > 4 > 3 > 2 > 1 > 0`. The report keeps every
|
||
individual result; precedence affects only the process exit code.
|
||
|
||
Argument parsing failures exit 2 with a one-line error on stderr.
|
||
|
||
## Diagnostics
|
||
|
||
A message that rejects a name suggests the nearest valid one. Keys from other
|
||
runners' configs (`testDir`, `baseURL`, `webServer`, `projects`) say where
|
||
that fact lives here. A failing `import` in the config or a test file ends
|
||
with the install or add command for the project's package manager, or the
|
||
subpaths the package exports. A model credential the provider rejects names
|
||
the variable the provider reads.
|