mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-22 18:19:48 +08:00
436 lines
23 KiB
Plaintext
436 lines
23 KiB
Plaintext
---
|
||
title: CLI
|
||
description: e2e init, guide, mcp, run, explore, list, cache, and telemetry, flag by flag.
|
||
---
|
||
|
||
```bash
|
||
npx e2e init [directory]
|
||
npx e2e guide [topic]
|
||
npx e2e mcp [options]
|
||
npx e2e run [files...] [options]
|
||
npx e2e explore [goal] [options]
|
||
npx e2e list [files...] [options]
|
||
npx e2e cache ls|clear|stats [--config <path>]
|
||
npx e2e telemetry [status|enable|disable]
|
||
npx e2e --help
|
||
npx e2e --version
|
||
```
|
||
|
||
The CLI needs Node.js 22.12 or newer.
|
||
|
||
## Help and version
|
||
|
||
| Flag | Behavior |
|
||
| --- | --- |
|
||
| `-h`, `--help` | Prints the help of the command it follows and exits 0. `e2e help run` is the same as `e2e run --help`. |
|
||
| `-v`, `--version` | Prints the installed `e2e` version and exits 0. |
|
||
|
||
```console
|
||
$ npx e2e --version
|
||
0.11.0
|
||
```
|
||
|
||
Output is colored on a terminal or in CI, plain when piped or when `NO_COLOR`
|
||
is set. `FORCE_COLOR` keeps the colors. `e2e` without a command prints the
|
||
help on stderr and exits 2.
|
||
|
||
## e2e init
|
||
|
||
Scaffolds a project and adds `e2e` to `devDependencies`. Existing
|
||
config and test files are skipped. With a `directory` argument the project is
|
||
scaffolded there.
|
||
|
||
The wizard asks three things when it creates a config:
|
||
|
||
- The engine. **Web** (Playwright, the default) adds `@e2edev/playwright`.
|
||
**Mobile (iOS/Android)** adds `@e2edev/agent-device`, defaults to iOS on
|
||
macOS and Android elsewhere, and opens Settings. **None** adds nothing.
|
||
- The model gateway agent steps use. **Vercel AI Gateway** (the default),
|
||
**OpenRouter**, an **OpenAI-compatible endpoint** (prompts for an HTTPS or
|
||
loopback base URL), a **ChatGPT Plus/Pro**, **GitHub Copilot**, or
|
||
**SuperGrok subscription** (see [Subscriptions](/subscriptions)), or
|
||
**None**. A gateway adds `ai@^7.0.0`, the gateway's provider package when
|
||
it has one, and an `agents` block that constructs the model. A
|
||
subscription adds `@e2edev/oauth` and names `e2e login <provider>` as the
|
||
step before the first run.
|
||
- Where to install the agent skill (see [e2e guide](#e2e-guide)):
|
||
`.agents/skills/` for Codex, Cursor, Copilot, Gemini CLI, OpenCode, and
|
||
Zed, and `.claude/skills/` for Claude Code. Both are preselected.
|
||
|
||
After confirming the file changes, choose whether to install the selected
|
||
dependencies. The installer follows the project's `packageManager` field or
|
||
lockfile, then the invoking package manager, then npm.
|
||
|
||
| Argument or flag | Type | Default | Behavior |
|
||
| --- | --- | --- | --- |
|
||
| `[directory]` | path | the working directory | Where to scaffold; created when missing |
|
||
| `-y`, `--yes` | boolean | `false` | Skips prompts: Web engine, Vercel AI Gateway, skill in both directories, nothing installed |
|
||
|
||
| Creates | Content |
|
||
| --- | --- |
|
||
| `package.json` | Created as a private ESM package when missing; otherwise only missing dependencies and scripts are added. Sibling packages (`@e2edev/playwright`, `@e2edev/agent-device`, `@e2edev/oauth`) get the caret range of the version released with the CLI, or that exact version when the CLI is a prerelease. A `test:e2e` script runs `e2e run` unless one exists |
|
||
| `e2e.config.ts` | One target for the selected engine. Playwright reads `APP_URL` (default `http://localhost:3000`) and shows the `command` option in a comment. With a gateway, `agents.default` is `createAgent({ model, system })` |
|
||
| `tests/example.e2e.ts` | An app-open test that passes against any page, plus a commented agent test. Device setup gets a Settings check instead |
|
||
| `.gitignore` | Appends `node_modules/`, `.e2e/artifacts/`, `.e2e/cache/`, `.e2e/sessions/`, `.e2e/report.json`, `.e2e/ai-trace.json`, `.e2e/junit.xml`, `.e2e/summary.md`, `.e2e/failures/`, and `.e2e/logs/`. Committing the cache is opt-in; see [commit your traces](/reference/config#commit-your-traces) |
|
||
| `.agents/skills/e2e/`, `.claude/skills/e2e/` | `SKILL.md` and one `references/<topic>.md` per `e2e guide` topic |
|
||
|
||
```bash
|
||
npx e2e init --yes
|
||
```
|
||
|
||
Re-running `init` keeps an existing config and adds no packages for it. Once
|
||
a skill directory exists, later runs refresh copies that differ from the
|
||
installed version and never add a directory the project did not choose.
|
||
|
||
Cancellation leaves files untouched and exits 0. An invalid `package.json`
|
||
exits 2 before any write. Without a terminal on stdin and stdout, `init`
|
||
without `--yes` exits 2 and names the flag.
|
||
|
||
## e2e login
|
||
|
||
```bash
|
||
npx e2e login openai # ChatGPT Plus/Pro, opens the browser
|
||
npx e2e login openai --device # a code to enter on another device
|
||
npx e2e login github-copilot --from-gh # reuse the GitHub CLI's login
|
||
npx e2e login github-copilot --client-id <id> [--enterprise-url <host>]
|
||
npx e2e login spacexai # SuperGrok / X Premium+, device code
|
||
npx e2e login # pick one; signed-in ones are marked
|
||
```
|
||
|
||
Signs in to a personal subscription and stores the login in
|
||
`~/.config/e2e/oauth.json` for the models `@e2edev/oauth` constructs
|
||
(`chatgpt()`, `copilot()`, `grok()`). An unknown provider is a usage error,
|
||
exit 2. Needs `@e2edev/oauth` in the project; without it the command exits 1
|
||
with the install line. See [Subscriptions](/subscriptions).
|
||
|
||
## e2e logout
|
||
|
||
```bash
|
||
npx e2e logout # pick one of the stored logins
|
||
npx e2e logout openai # forget one
|
||
```
|
||
|
||
## e2e guide
|
||
|
||
```bash
|
||
npx e2e guide # the overview
|
||
npx e2e guide setup # one topic
|
||
```
|
||
|
||
Prints the skill that `init` installs. `guide` alone prints the overview;
|
||
`guide <topic>` prints one of `setup`, `writing-tests`, `agent`, `running`,
|
||
`explore`, `debugging`, or `mcp`. An unknown topic exits 2 and names the
|
||
valid ones. The same skill installs with `npx skills add tester-army/e2e`.
|
||
See [Coding agents](/coding-agents).
|
||
|
||
## e2e mcp
|
||
|
||
```bash
|
||
npx e2e mcp [--config <path>] [--target <name>] [--headless]
|
||
```
|
||
|
||
An MCP server on stdio that gives a coding agent a live session on the app.
|
||
Flags, tools, and session rules are on the [MCP server](/reference/mcp) page.
|
||
|
||
## e2e run
|
||
|
||
### Positional arguments
|
||
|
||
```bash
|
||
e2e run [files...]
|
||
```
|
||
|
||
Test files, directories, or globs relative to the project root:
|
||
|
||
- a file path (`tests/signup.e2e.ts`) selects that file;
|
||
- a directory (`tests/agent`, `.`) selects every discovered test file beneath
|
||
it;
|
||
- a bare name selects every discovered file whose path ends with it at a
|
||
segment boundary. `signup.e2e.ts` and `agent/signup.e2e.ts` both select
|
||
`tests/agent/signup.e2e.ts`. Without a `/` or `.` it matches the file name
|
||
with extensions dropped, so `signup` selects it too;
|
||
- a glob (`'tests/**/*.smoke.e2e.ts'`, quoted) uses the grammar of the config
|
||
`tests` globs: `*`, `?`, and a whole `**` segment. Braces, character
|
||
classes, and extglobs are `INVALID_GLOB` with a hint rather than a pattern
|
||
that matches nothing: there is no expansion, so name one glob per
|
||
alternative. `./`, `..`, a trailing `/`, and an absolute path inside the
|
||
project resolve as they do for any positional path, and an existing file or
|
||
directory is selected by its path whatever its name is spelled with.
|
||
|
||
Positionals narrow the config globs and the tag, platform, and capability
|
||
filters; they never bypass them. Names and globs are case-sensitive on every
|
||
OS. A path outside the project root is `COLLECTION_ERROR`.
|
||
|
||
Every discovered file is still collected when positionals name a few: a
|
||
selected test that consumes a session pulls in the setup test that produces
|
||
it, wherever that setup lives, so `e2e run login.e2e.ts` needs no second
|
||
positional for `auth.setup.e2e.ts`. Ordinary tests in the other files are
|
||
reported as unselected. The flip side is that a collection error in any file
|
||
fails the run, whichever files were named.
|
||
|
||
A positional starting with `-` that names no file is a usage error, exit 2.
|
||
This is what `pnpm test:e2e -- --headed` produces: the `--` reaches e2e and
|
||
`--headed` would otherwise be taken for a file. The message names the direct
|
||
form, `pnpm exec e2e run --headed`.
|
||
|
||
When nothing is left to run, `NO_TESTS` names the cause: no file matched the
|
||
globs (with look-alike files such as `tests/login.test.ts`), a positional
|
||
matched nothing (with the nearest path), a matched file registered no tests,
|
||
or every test was filtered out.
|
||
|
||
### Flags
|
||
|
||
| Flag | Type | Default | Behavior |
|
||
| --- | --- | --- | --- |
|
||
| `--config <path>` | string | discovered | Explicit config path. A missing file is `CONFIG_NOT_FOUND`. |
|
||
| `--target <ids>` | comma-separated or repeated | every configured target | Selects targets by name. Only selected targets start app commands and services. An unknown ID is `UNKNOWN_TARGET`. |
|
||
| `--tag <tags>` | comma-separated or repeated | none | Only tests carrying the tags: any of them, or every one with `--tag-mode all`. When the filter leaves nothing to run, `NO_TESTS` counts the tests it filtered, marks each tag no test declares, and names the nearest declared one. |
|
||
| `--tag-mode <mode>` | `any` or `all` | `any` | How several tags combine: any of them, or every one. |
|
||
| `--headed` | boolean | `false` | Visible UI when the engine supports it. |
|
||
| `--agent <names>` | comma-separated or repeated | `default` | The configured agents unpinned tests run as, one result per name. A test pinned with `{ agent }` runs as the pinned names the flag also gives, or as its whole pin when the flag names none of them. An unknown name is `INVALID_CONFIG`. |
|
||
| `--retries <n>` | integer 0-10 | config | Replaces `config.retries`. |
|
||
| `--workers <n>` | integer 1-1024 | config | Replaces `config.workers`. |
|
||
| `--reporter <ids>` | comma-separated list | config | Replaces the built-in IDs in `config.reporters`: `list`, `json`, `junit`, `markdown`. `json` with `list` is `INVALID_CONFIG`. [Reporter objects](/reference/reporters) in the config keep running. |
|
||
| `--artifacts <dir>` | string | `.e2e/artifacts` | Relocates the report and artifact tree. |
|
||
| `--no-cache` | boolean | `false` | Trace cache off for this run, whatever `config.cache` says. |
|
||
| `--pass-with-no-tests` | boolean | `false` | Exit 0 instead of `NO_TESTS` when nothing is selected. |
|
||
| `--debug` | boolean | `false` | Phase timings and an agent step table on stderr after the run. |
|
||
| `--ai-trace` | boolean | `false` | Records every model call to `.e2e/ai-trace.json`, an AI SDK devtools database. Open it with [unbox-ai](https://github.com/tester-army/unbox-ai). |
|
||
| `--video` | boolean | `false` | Records every attempt (WebM on a browser engine, MP4 on a device engine). Adds the `video` kind to `config.artifacts` for this run; an engine that cannot record fails with `UNSUPPORTED_ARTIFACT`. See [video](/reference/config#video). |
|
||
|
||
Out-of-range `--retries` and `--workers` fail with `INVALID_CONFIG` before
|
||
anything starts, the same as the config value would. `--target`, `--tag`, and
|
||
`--agent` take each name once; an empty value (`--tag ''`, or `--tag "$TAGS"`
|
||
with the variable unset) is a usage error, exit 2, never a wider selection.
|
||
|
||
`--debug` appends two tables to stderr. Phase timings are summed across the
|
||
runner and every worker. The agent step table lists every agent step in
|
||
execution order with its per-phase split, model calls, tokens, and cost when
|
||
the provider reported one.
|
||
|
||
```console
|
||
$ npx e2e run --debug
|
||
...
|
||
[e2e debug] phase timings (wall 14.2s)
|
||
phase count total avg max
|
||
scheduler 1 11.8s 11.8s 11.8s
|
||
test.body 6 10.1s 1.7s 3.9s
|
||
session.launch 6 1.1s 183ms 410ms
|
||
engine.prepare 1 612ms 612ms 612ms
|
||
collect 1 420ms 420ms 420ms
|
||
config.load 1 118ms 118ms 118ms
|
||
realm.import 1 96ms 96ms 96ms
|
||
session.close 6 88ms 15ms 31ms
|
||
engine.init 1 74ms 74ms 74ms
|
||
|
||
[e2e debug] agent steps (execution order, model anthropic/claude-sonnet-4.5, total $0.0312)
|
||
step total model observe action calls tokens in/out cached cost
|
||
act "add a todo named groceries" 3.9s 2.8s 410ms 620ms 3 9412/388 61% (5740) $0.0198
|
||
assert "the list shows groceries" 1.2s 1.1s 88ms 0ms 1 3120/41 0% (0) $0.0114
|
||
```
|
||
|
||
`cached` is the share of input tokens the provider served from its prompt
|
||
cache, `-` when it reports no split. The report records the same per step
|
||
(`model.cacheReadTokens`, `model.cacheWriteTokens`) and per run
|
||
(`usage.modelCachedTokens`).
|
||
|
||
```bash
|
||
npx e2e run
|
||
npx e2e run tests/signup.e2e.ts --tag smoke
|
||
npx e2e run tests/agent
|
||
npx e2e run 'tests/**/*.smoke.e2e.ts'
|
||
npx e2e run --target web,webkit --tag smoke,billing --tag-mode all
|
||
npx e2e run --reporter json --workers 4 --retries 2
|
||
npx e2e run --reporter list,junit
|
||
```
|
||
|
||
### Output
|
||
|
||
| Reporter | Writes |
|
||
| --- | --- |
|
||
| `list` | Progress to the terminal: a `RUN` banner, one line per setup step, one line per test file and target with failed and agent-driven tests listed beneath, a `Failed Tests` section with each error and a code frame, and a summary (`Test Files`, `Tests`, `AI`, `Cache`, `Start at`, `Duration`, `Report`). On a TTY a live window shows the running tests and the current agent step with its model turns, tool calls, and an excerpt of the model's reasoning. Untrusted fields are capped at 8 KiB with control characters stripped. |
|
||
| `json` | The report document to stdout and nothing else there, so `e2e run --reporter json \| jq` works. Same document as `.e2e/report.json`, described by `schema/report-v1.schema.json` in the package. The shape changes only with the `schemaVersion` field (`report-1` today). Cannot be combined with `list`. |
|
||
| `junit` | `.e2e/junit.xml` beside the report. One `<testsuite>` per file, one `<testcase>` per test-target pair named `title > path [target]`. A test-class error is a `<failure>`, any other an `<error>`, a skip a `<skipped>` with its reason. Run-level errors form a `run` suite. |
|
||
| `markdown` | `.e2e/summary.md` beside the report: a headline with counts and spend, run-level errors, one block per failed test with its error and its facts (expected and observed, what a locator asked for), the step it went wrong at, whether every attempt failed alike, the last model turns, the screen's location and closest nodes, the line to look at, evidence paths, and a link to the test's own page; the flaky tests folded with the same block each; and every test folded away by file, each file with its counts and time. Each failed or flaky test also gets a page under `.e2e/failures/` with every step, every kept model turn, and the screen at failure inline. An `e2e explore` run renders its findings instead. This is the page [`@e2edev/github`](/github) posts as the pull request comment. |
|
||
|
||
Every run writes `.e2e/report.json` under the artifact parent whatever the
|
||
reporters are. A `junit.xml` or `summary.md` that could not be written is a
|
||
line on stderr, never a run error.
|
||
|
||
With `--ai-trace`, the run also writes `.e2e/ai-trace.json`: one entry per
|
||
model round trip with the exact prompt, tool definitions, response, usage,
|
||
and provider metadata. Observations in it are already redacted. Inline image
|
||
and file bytes are replaced with byte counts. Open it with
|
||
`npx unbox-ai .e2e/ai-trace.json`; see [Debugging a run](/debugging#agent-steps).
|
||
|
||
### Signals
|
||
|
||
Signals escalate.
|
||
|
||
1. The first `SIGINT` or `SIGTERM` interrupts. The running test ends with
|
||
status `interrupted`, its `afterEach` hooks and session close run within
|
||
the cleanup budget, every worker disposes its engine, and the runner writes
|
||
a partial report and exits 130.
|
||
2. A second signal forces. Workers dispose their engines at once and are
|
||
killed when the cleanup budget is spent. The report is still written.
|
||
3. A third signal exits on the spot, after killing the process groups of every
|
||
app command and service the run started. A process attached through
|
||
`reuseExisting` is left alone.
|
||
|
||
A worker whose runner disappears disposes its engine and exits within the
|
||
cleanup budget.
|
||
|
||
## e2e explore
|
||
|
||
```bash
|
||
e2e explore [goal]
|
||
```
|
||
|
||
A run with no test file. The agent gets a goal, plans one exploration step
|
||
at a time, reports defects through its `report_finding` tool, and ends with an
|
||
assessment. The default goal is `Explore the app and find bugs`. See
|
||
[Exploring without a test](/explore).
|
||
|
||
| Flag | Type | Default | Behavior |
|
||
| --- | --- | --- | --- |
|
||
| `--config <path>` | string | discovered | Explicit config path, with the same errors as `run`. |
|
||
| `--target <id>` | string | the first configured target | The one target to explore. Several targets and no flag explores the first and says so on stderr. An unknown name is `UNKNOWN_TARGET`. |
|
||
| `--agent <name>` | string | `default` | The configured agent the exploration runs as. Its model, tools, and context are reused; a hand-rolled `StepExecutor` is replaced by the built-in agent, with a notice. An unknown name is `INVALID_CONFIG`. |
|
||
| `--max-steps <n>` | integer 1-12 | `8` | Exploration steps at most. Out of range exits 2. |
|
||
| `--timeout <ms>` | integer 180000-900000 | `600000` | The run's wall clock. The last minute is kept for the closing assessment. Out of range exits 2. |
|
||
| `--headed` | boolean | `false` | Requests visible UI when the engine supports it. |
|
||
| `--reporter <ids>` | comma-separated list | resolved config value | As for `run`. The exploration summary rows print under the `list` summary; `json` carries the record in the report. |
|
||
| `--artifacts <dir>` | string | `.e2e/artifacts` | Relocates the report and the artifact tree; a finding's evidence screenshot is one of the attempt's artifacts there. |
|
||
| `--debug` | boolean | `false` | As for `run`: phase timings and the agent step table. |
|
||
| `--ai-trace` | boolean | `false` | As for `run`. |
|
||
| `--video` | boolean | `false` | As for `run`: one video of the whole exploration (WebM on a browser engine, MP4 on a device engine) when the engine can record; otherwise the run fails with `UNSUPPORTED_ARTIFACT` before exploration starts. |
|
||
|
||
The run has one result, under the virtual file `explore`, titled by the goal.
|
||
The trace cache is off and retries are zero whatever the config says.
|
||
`.e2e/report.json` gains `run.explore`: the goal, the budgets, why the run
|
||
ended (`finished`, `step-limit`, `time`, `stuck`, or `aborted`), the
|
||
assessment, the steps, and the findings.
|
||
|
||
Exit codes: `0` when steps ran and no finding of kind `issue` was reported,
|
||
`1` when one was or when no step ran at all, `2` and `3` as for `run`, `130`
|
||
on Ctrl-C.
|
||
|
||
```bash
|
||
npx e2e explore
|
||
npx e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'
|
||
npx e2e explore --target web --max-steps 4 --headed
|
||
npx e2e explore 'Hunt for broken forms and dead links' --agent thorough --video
|
||
```
|
||
|
||
## e2e list
|
||
|
||
```bash
|
||
e2e list [files...]
|
||
```
|
||
|
||
A dry run. `list` selects tests exactly as `run` does, prints one line per
|
||
test-target pair, and exits. No app process, engine, or worker starts, and
|
||
nothing is written under `.e2e/`.
|
||
|
||
Positionals and the selection flags (`--config`, `--target`, `--tag`,
|
||
`--tag-mode`, `--pass-with-no-tests`) are those of `run`. `--reporter`
|
||
accepts `list` (the default) or `json`.
|
||
|
||
| Reporter | Writes |
|
||
| --- | --- |
|
||
| `list` | One line per pair, `file › title [target]`, in selection order, with the describe path joined by ` › ` (`tests/billing.e2e.ts › checkout › pays [web]`), then each tag the test declares as `#tag` (`tests/billing.e2e.ts › checkout › pays [web] #smoke #billing`). A skipped test keeps its line with `(skipped: <reason>)` appended. Setup tests are listed like any other. |
|
||
| `json` | `{ "pairs": [...] }` to stdout. Each pair carries `file`, `title`, `titlePath`, `kind` (`test` or `setup`), `tags` (the tags the test declares, `[]` when none), `target`, `disposition` (`run` or `skip`), and `skipReason` when skipped. |
|
||
|
||
Pairs a filter removed are not listed.
|
||
|
||
```bash
|
||
npx e2e list
|
||
npx e2e list tests/signup.e2e.ts
|
||
npx e2e list --tag smoke --target web
|
||
npx e2e list --reporter json
|
||
```
|
||
|
||
## e2e cache
|
||
|
||
Reads and empties the trace cache under `.e2e/cache/`. Entries are files
|
||
named after the digest of their key, so these commands are the only way to
|
||
see what is in there. All three resolve the store through the discovered
|
||
config or `--config <path>`, honor `cache.dir`, and exit 0 on an empty store.
|
||
|
||
| Command | Prints |
|
||
| --- | --- |
|
||
| `e2e cache ls` | One row per entry: the test and target it was recorded for, the first 12 characters of its instruction digest, its age, and how many actions a replay would run (`(truncated)` when recording hit the action cap). Rows are grouped by test, then target, oldest first. |
|
||
| `e2e cache stats` | The store directory, the number of readable entries, their total size, and the number of unreadable files when there are any. |
|
||
| `e2e cache clear` | Deletes every cache file, then the directory when nothing else is left in it, and reports how many files went. |
|
||
|
||
```bash
|
||
npx e2e cache ls
|
||
npx e2e cache stats
|
||
npx e2e cache clear
|
||
npx e2e cache ls --config packages/web/e2e.config.ts
|
||
```
|
||
|
||
```text
|
||
TEST TARGET INSTRUCTION AGE ACTIONS
|
||
tests/billing.e2e.ts::upgrades web 4f1c0a9d3b62 2h 4
|
||
tests/signup.e2e.ts::signs%20up web 9ab7715ce004 3d 7 (truncated)
|
||
```
|
||
|
||
An entry recorded before entries carried provenance prints `-` in those
|
||
columns and still replays. A file the store cannot read (truncated, over the
|
||
1 MiB ceiling, or from a newer schema) is counted separately on stderr, treated
|
||
as a miss by a run, and removed by `clear`. Files the runner never wrote are
|
||
left alone.
|
||
|
||
<Note>
|
||
These commands read the file store only. With a custom `cache.store` they
|
||
exit 2 and say so.
|
||
</Note>
|
||
|
||
## e2e telemetry
|
||
|
||
Shows whether anonymous usage telemetry is on, and switches it.
|
||
[Telemetry](/telemetry) lists every property sent.
|
||
|
||
| Command | Effect |
|
||
| --- | --- |
|
||
| `e2e telemetry`, `e2e telemetry status` | Prints `Status: enabled`, or `Status: disabled` with the reason: the variable that is set, the saved choice, or a preferences directory that cannot be written. |
|
||
| `e2e telemetry disable` | Saves the opt-out to `~/.config/e2e/telemetry.json` (`$XDG_CONFIG_HOME/e2e/` when set, `%APPDATA%\e2e\` on Windows) and prints the status. |
|
||
| `e2e telemetry enable` | Saves the opt-in and prints the status. A variable set in the environment still wins, and the status says so. |
|
||
|
||
```bash
|
||
npx e2e telemetry
|
||
npx e2e telemetry disable
|
||
E2E_TELEMETRY_DEBUG=1 npx e2e run
|
||
```
|
||
|
||
`E2E_TELEMETRY_DISABLED=1` and `DO_NOT_TRACK=1` turn telemetry off without
|
||
touching the file. `E2E_TELEMETRY_DEBUG=1` prints every event to stderr and
|
||
sends nothing. The first command on a machine prints a notice once; in CI
|
||
nothing is printed and no file is written.
|
||
|
||
## Exit codes
|
||
|
||
| Code | Meaning |
|
||
| ---: | --- |
|
||
| 0 | All selected tests passed, were flaky, or were explicitly skipped |
|
||
| 1 | Final test or setup failure, or test timeout |
|
||
| 2 | CLI, config, collection, dependency, credential, model-config, or policy error |
|
||
| 3 | Engine, app process, model provider, artifact, or cleanup infrastructure failure |
|
||
| 4 | Internal runner invariant or unhandled runner error |
|
||
| 130 | Interrupted by a user or CI signal |
|
||
|
||
Precedence for a mixed run is `130 > 4 > 3 > 2 > 1 > 0`. The report keeps every
|
||
individual result; precedence affects only the process exit code.
|
||
|
||
Argument parsing failures exit 2 with a one-line error on stderr.
|
||
|
||
## Diagnostics
|
||
|
||
A message that rejects a name suggests the nearest valid one. Keys from other
|
||
runners' configs (`testDir`, `baseURL`, `webServer`, `projects`) say where
|
||
that fact lives here. A failing `import` in the config or a test file ends
|
||
with the install or add command for the project's package manager, or the
|
||
subpaths the package exports. A model credential the provider rejects names
|
||
the variable the provider reads.
|