--- title: Debugging a run description: Read the trace page, open the evidence, watch the browser, and see what the model was sent. "og:image": "https://e2e.com/docs/images/og/debugging.png?v=38a51086e1" "twitter:image": "https://e2e.com/docs/images/og/debugging.png?v=38a51086e1" --- A failed test prints its error and the path to its [trace page](#traces). For a summary of the whole run, add the `markdown` reporter. It writes `.e2e/summary.md`. ## Read the failure The `list` reporter's `Failed Tests` section shows the error, the failing line, nearby matches for a failed locator, what the app logged, and the trace page. It quotes the app's first 3 errors (console errors, uncaught exceptions, failed requests, 5xx responses), redacted like the trace page. `.e2e/summary.md` quotes them too: ```text FAIL |web| tests/todos.e2e.ts > shows the remaining count LOCATOR_NOT_FOUND: locator did not become visible: getByRole("heading", name: "Todos") ❯ tests/todos.e2e.ts:31:3 ❯ at http://127.0.0.1:4271/nope ❯ app 2 errors, 2 warnings logged × console error: Error occurred prerendering page "/todos" × network error: GET /api/todos 500 Internal Server Error ❯ trace .e2e/results/todos-shows-the-remaining-count-796fac5f12ab34cd/trace.md ``` The exit code tells the type of failure: | Exit | Meaning | Retry the job? | | ---: | --- | --- | | 1 | A test failed | Only if the test is flaky | | 2 | Configuration, collection, or policy | Fix the reported problem first | | 3 | Engine, app process, model provider, artifact, or cleanup infrastructure failure | If the cause is temporary | | 4 | An internal runner error | Report it | | 130 | The run was interrupted | | ## Traces By default, each failed, timed out, interrupted, or flaky test gets a trace page, `trace.md`, in its [results directory](#artifacts). Open it in an editor's markdown preview so its links work. After the browser or device the test ran on, it shows: - **Steps.** The node an action landed on, the values an assertion read while it waited, the replay cache's decision, what the app logged, where the page went, and how the screen changed since the step before. - **App log.** Console output, uncaught exceptions, failed requests, and 4xx and 5xx responses, each with its step: up to 50 lines, errors and warnings first. An attempt keeps at most 200 errors and warnings and 100 other lines. - **The agent's last turns.** - **Screen at failure.** The screenshot and the semantic tree, captured in at most 5 s. An app that doesn't answer in time gets no screen section. ````markdown # ✗ todos › shows the remaining count `tests/todos.e2e.ts` · failed · 2.2s **ASSERTION_FAILED** ```text expect.toHaveText failed locator: getByRole("status", name: "Remaining") expected: text "1 remaining" observed: text "0 remaining" (match count 1) ``` Ran on: browser `chromium 141.0.7390.37` · user agent `Mozilla/5.0 (Macintosh; …) HeadlessChrome/141.0.7390.37 Safari/537.36` Look at: `tests/todos.e2e.ts:15` ## Steps 1. ✓ `app.open` `/todos` (37ms in beforeEach) — `tests/todos.e2e.ts:4` 2. ✓ `agent.act` "Add a todo named Milk." (429ms, 2 model calls) — `tests/todos.e2e.ts:12` - cache: no replay: nothing recorded for this step yet; recording saved (`.e2e/cache/1227d5a6….json`) - type "Milk" into textbox "New todo" (17ms) - press "Enter" on the focused field (5ms) - ✗ network error: `POST /api/todos 500 Internal Server Error` - screen: 6 changes since step 1 at `/todos`, 38 nodes - `added listitem "Milk Delete Milk" testid="todo"` - `added checkbox "Milk"` - `changed status "Remaining" text="1 remaining" (was: status "Remaining" text="0 remaining")` - 3 more 3. ✓ `locator.check` `getByLabel("Milk")` (13ms) — `tests/todos.e2e.ts:14` - check checkbox "Milk" (13ms) - screen: 3 changes since step 2 at `/todos`, 38 nodes - `added checkbox "Milk" [checked]` - `changed status "Remaining" text="0 remaining" (was: status "Remaining" text="1 remaining")` - `removed checkbox "Milk"` 4. ✗ `expect.toHaveText` `getByRole("status", name: "Remaining")` (1.6s) — **ASSERTION_FAILED** — `tests/todos.e2e.ts:15` - expect gave up after 16 reads in 1.6s: text "0 remaining" (1 match) x16 ## App log What the app logged, oldest first, with the step it happened in. - step 2 · ✗ network error: `POST /api/todos 500 Internal Server Error` ## Screen at failure `http://127.0.0.1:4271/todos` · viewport 1280x720 · 38 nodes ![screenshot at failure](attempt-1/screenshots/001-failure.png) ```text #n29 textbox "New todo" #n30 button "Add" #n35 listitem "Milk Delete Milk" testid="todo" #n36 checkbox "Milk" [checked] #n40 status "Remaining" text="0 remaining" ``` ```` An assertion that read one value the whole time has a wrong expectation, or the app never reached that state: look in the app log for a failed request. An assertion whose readings changed (`"2 remaining" x3 -> at 410ms "1 remaining"`) raced the app. `screen: unchanged` after an action means the action had no visible effect. On web, a step's screen is the one an action left. On a device, it's the screen the step acted on. The [`trace`](/reference/config#trace) mode sets which tests keep a page. `--trace` keeps one for every test of a run, passing ones included. `--trace off` writes none. A test's own `trace` option wins over both. ## Artifacts Each test gets one directory under `.e2e/results/` (or [``](/reference/config#output)`/results/`), with its trace page and one `attempt-` per attempt. A serial group's tests share one directory: ```text .e2e/results/todos-adds-a-todo-1a2b3c4d5e6f7a8b/trace.md .e2e/results/todos-adds-a-todo-1a2b3c4d5e6f7a8b/attempt-1/screen-at-failure.txt .e2e/results/todos-adds-a-todo-1a2b3c4d5e6f7a8b/attempt-1/screenshots/001-failure.png ``` Each run that reaches its tests empties `results/` first. A `--last-failed` rerun keeps the files that the report it reruns names, and writes its own attempts under each test's `rerun-/`. | Artifact | When | | --- | --- | | Screenshots | A failed attempt captures one when the engine can. The built-in `agent.assert` captures one after either verdict, unless `screenshot: false`. `app.screenshot('label')` captures one on request. They mask secure fields only. See [screenshots after a secret fill](/reference/security#screenshots-after-a-secret-fill). | | Downloads | Files saved through `browser.waitForDownload`, kind `download`, as the app served them. [Downloads](/reference/security#downloads) says when the runner masks secrets in them. | | Videos | When you [record them](#watch-it-happen). Videos mask nothing. See [where redaction stops](/reference/security#where-redaction-stops). | | Agent transcripts | With [`--debug`](#agent-steps). | ## For scripts: report.json Every run that reaches its tests also writes `report.json` to the [output directory](/reference/config#output), `.e2e` by default. It holds every result of the run in a versioned schema. `--reporter json` prints it to stdout: ```bash jq '.run | {status, exitCode, errors}' .e2e/report.json jq '.run.results[] | select(.status != "passed") | .attempts[-1] | {status, error, failure, steps: [.steps[] | select(.status != "passed") | {api, label, source, status, error}], artifacts}' .e2e/report.json ``` `run.errors` holds run-level failures. The last attempt of each failed result holds: | Field | Holds | | --- | --- | | `error` | The code, message, source line, and assertion details | | `failure` | The screen and screenshot artifacts, and any locator candidates | | `steps` | The calls that led to the failure. Each step's `events` hold what it did (`engine`), what it polled (`poll`), and where the app went (`navigation`). Agent steps add `cache` and recent model turns. `observationTruncated: true` means at least one semantic observation was incomplete. | | `appLog` | What the app logged. Each line's `step` is the `index` of its step, absent before the first step. | | `artifacts` | The captured files | A serial group's tests have no `attempts` of their own. Their results are in `run.serialGroups`, one member for each test in each attempt. A retry the run interrupted or that never reached its members has `cutRetry: true` and does not hold the verdicts, so skip it: ```bash jq '.run.serialGroups[] | (.attempts | map(select(.cutRetry != true)) | last).members[]? | select(.status != "passed") | {testId, status, error, failure}' .e2e/report.json ``` ## Watch it happen ```bash npm npx e2e run --headed # a visible browser npx e2e run --video # record every attempt npx e2e run --video=retain-on-failure # record every attempt, keep the failures ``` ```bash pnpm pnpm exec e2e run --headed # a visible browser pnpm exec e2e run --video # record every attempt pnpm exec e2e run --video=retain-on-failure # record every attempt, keep the failures ``` ```bash bun bunx e2e run --headed # a visible browser bunx e2e run --video # record every attempt bunx e2e run --video=retain-on-failure # record every attempt, keep the failures ``` `--video` takes an optional mode. Put test files before it, or write `--video=`. Each recording goes into its attempt's directory: WebM per page on web, MP4 on a device. The failure recap prints the path, or the URL when a hosted browser keeps the recording. Subtract the recording's `startedAt` from a step's `startedAt` to find that step in the video. In CI, set `video: 'on-first-retry'` in the config. A test that passes the first time records nothing, and a flaky test leaves a recording of its retry. A target's `video` wins over the config's, and `--video` wins over both. A test's own option, such as `test('checkout', { video: 'on' }, ...)`, wins over all of them. The config's `video` and `--video` apply only to targets whose engine can record, and one notice names the others. When a target or a test asks for a recording its engine can't make, the run fails with `UNSUPPORTED_ARTIFACT` before any test starts. [Recording](/reference/config#recording) has every mode and file. ## Step through a test `--step` runs the first step, then holds each later step until you press a key. Add `--headed` to watch the browser between steps. ```bash npm npx e2e run tests/checkout.e2e.ts --step --headed ``` ```bash pnpm pnpm exec e2e run tests/checkout.e2e.ts --step --headed ``` ```bash bun bunx e2e run tests/checkout.e2e.ts --step --headed ``` The held step shows under its test, below the steps that already ran: ```text ❯ |web| tests/checkout.e2e.ts 0/1 └── pays with a card (k3f9) ✓ app.open "/checkout" 412ms ▶ step 2 locator.fill "getByLabel("Email")" tests/checkout.e2e.ts:9 press enter to run this step, press h to show help, press q to quit ``` The keys follow [`--watch`](/reference/cli#watch-mode). These work at a prompt: | Key | Does | | --- | --- | | Enter | runs the held step and holds the next one | | `n` | runs the rest of this test, retries included, and holds at the next test | | `a` | runs the rest of the run without holding steps | | `h` | shows every key under the prompt, or hides them | | `q`, Esc | stops the run as Ctrl-C does: the test ends `interrupted`, the rest are skipped, exit 130 | - A key pressed within 150 ms of a prompt is ignored, so a key meant for the last step never runs this one. Ctrl-C works at any time, also while a step runs. - The time a step is held counts toward nothing: not its duration, not the test's `timeout`, not `maxDuration`. A test that hangs after a step still times out on the time it had left. The test's own duration in the report includes the holds. - Setup tests, retries, `afterEach` hooks, fixture teardowns, and steps inside an `expect.poll` read run without stopping. An `agent.act` is one step. - Steps started together, as in `Promise.all`, share one prompt and run on one Enter, so a `waitForResponse` beside the click that triggers it never waits for a key. A step already running when another is held keeps running on its own timeout. - A hosted browser or device can end a session that sits idle too long while a step is held. `--step` runs one test at a time. It is refused in CI, with `--watch`, with `--workers` above 1, and with the json reporter. ## Step through a test without a terminal When stdin or stdout is not a terminal, as in a coding agent's shell, `--step` runs the test in the background. The command returns at the first held step, with the screen that step waits on. Run each later command from the same directory. A shell that gives commands a terminal gets the key prompt instead; redirect stdin from the null device (`< /dev/null`, or `< NUL` in Windows `cmd`). ```bash npm npx e2e run tests/checkout.e2e.ts --step npx e2e run --step next ``` ```bash pnpm pnpm exec e2e run tests/checkout.e2e.ts --step pnpm exec e2e run --step next ``` ```bash bun bunx e2e run tests/checkout.e2e.ts --step bunx e2e run --step next ``` ```text ✓ app.open "/checkout" 124ms Paused (k3f9) before step 2 of pays with a card [web]: locator.fill "getByLabel("Email")" (tests/checkout.e2e.ts:9) Screen: http://localhost:3000/checkout heading "Checkout" textbox "Email" button "Pay" Next: e2e run --step next to run it, e2e run --step rest to run the rest, or e2e run --step stop to end the run. ``` | Command | Does | | --- | --- | | `e2e run --step next` | runs the held step and returns at the next one | | `e2e run --step rest` | runs the rest of the run without holding steps | | `e2e run --step stop` | stops the run as `q` does, exit 130, even while another command waits | - Each answer starts with what the run printed since the last command: a line per step that ran, as the first line above, with the error under a failed one. The last answer ends with the run summary and `The stepped run has ended.`, and the command exits with the run's exit code. - The screen lists the app's nodes as a trace page does, with secrets redacted. - One directory has one stepped run. A second `--step` there exits 2 until the first ends. - `next` and `rest` only act on a step a command has printed. When a command is killed while it waits, the next one prints what it missed: the step, without running it, or the end of the run. A `next` or `rest` sent while a step runs waits for the next held step without acting. Either way, the answer opens with a line saying the action was not applied. - A step held for 30 minutes with no command stops the run. - With `--step`, a file or directory named `next`, `rest`, or `stop` needs a path, such as `./next`. ## Pause a test With [`screen.pause()`](/reference/screen#screen-pause), you can pause the test at a line, with the app open, until you resume it. You can use it in any `e2e run`, with or without [`--step`](#step-through-a-test). The test is paused each time the line runs, such as on every pass of a loop. ```ts title="tests/checkout.e2e.ts" import { test } from '@e2e-dev/web'; test('pays with a card', async ({ app, screen }) => { await app.open('/checkout'); await screen.pause(); }); ``` Add `--headed` to look at the browser while the test is paused: ```bash npm npx e2e run tests/checkout.e2e.ts --headed ``` ```bash pnpm pnpm exec e2e run tests/checkout.e2e.ts --headed ``` ```bash bun bunx e2e run tests/checkout.e2e.ts --headed ``` In a terminal, the pause is shown under its test: ```text ❯ |web| tests/checkout.e2e.ts 0/1 └── pays with a card (k3f9) ▶ paused screen.pause() tests/checkout.e2e.ts:5 press enter to resume, press h to show help, press q to quit ``` Press Enter to resume the test, `h` to list the keys, or `q` to stop the run with exit 130, as at a [`--step` prompt](#step-through-a-test). Without a terminal, as in a coding agent's shell, the pause is printed with the screen it waits on: ```text Paused (k3f9) in pays with a card [web]: screen.pause() (tests/checkout.e2e.ts:5) Screen: http://localhost:3000/checkout heading "Checkout" Next: e2e run --step next to resume the test, or e2e run --step stop to end the run. ``` To resume the test, run `e2e run --step next` from another shell in the same directory. To end the run with exit 130, run `e2e run --step stop`. You can use these commands with or without a terminal. At this pause, you can send `rest` in place of `next`. To [attach over MCP](/reference/mcp#attaching-to-a-held-test) from the same directory, pass the ID from the pause header, as in `open_session {attach: "k3f9"}`. You get an answer once the test is paused again or the run ends. At the end, you get `The run has ended.` and the run's exit code. See [Step through a test without a terminal](#step-through-a-test-without-a-terminal) for what each answer holds. - Paused time doesn't count toward any timeout. But without a terminal, the run is stopped after 30 minutes at a pause without a command. - The test is paused on every attempt, retries and setup tests included. - With several workers, pauses are shown one at a time, oldest first, while the other workers keep running. - Call `screen.pause()` with `await` in a [`beforeEach`](/reference/test#beforeeach) hook, [fixture](/fixtures) setup, or the test body. See [`screen.pause()`](/reference/screen#screen-pause) for what happens anywhere else. - You can control one run per directory at a time with `e2e run --step` commands. A `--step` run started without a terminal is controlled from its start. A run without `--step` is controlled from its first pause. If another run in the directory is already controlled this way: - Without a terminal, the test fails at the pause with `RUNNER_BUSY`. End the other run with `e2e run --step stop`, or run this one in a terminal. - In a terminal, you can still use the keys. A line is printed to say that the commands go to the other run. - With `CI` set, the test fails at the pause with `PAUSE_UNAVAILABLE`. Remove the call before you commit. ## Console output `console.log` and `console.error` in a test print while it runs, under a heading that names the stream, target, file, and test: ```text stdout | |web| tests/todos.e2e.ts > adds a todo hello from the test ``` Output from outside a test, such as a file's top level or a reporter, prints under `stdout | runner` or `stderr | runner`. Both streams print where `list` does: stdout, or stderr beside `json`. Without the `list` reporter, or another reporter that prints to the terminal with [`onOutput`](/reference/reporters#onoutput), output goes to its own stream as written, except with the `json` reporter (`--reporter json` or in `reporters`), which keeps stdout for the report and prints output on stderr. Output from the config load prints before the run header, or a warning replaces it if the load fails. The runner [redacts](/reference/security#redaction) secret values. The report doesn't include console output, so use `expect` or an artifact for anything a later reader needs. ## Agent steps These flags show what the agent did: ```bash npm npx e2e run --debug # timings and a step table on stderr npx e2e run --ai-trace # every model call, for a trace viewer npx e2e run --no-cache # run the agent live instead of replaying ``` ```bash pnpm pnpm exec e2e run --debug # timings and a step table on stderr pnpm exec e2e run --ai-trace # every model call, for a trace viewer pnpm exec e2e run --no-cache # run the agent live instead of replaying ``` ```bash bun bunx e2e run --debug # timings and a step table on stderr bunx e2e run --ai-trace # every model call, for a trace viewer bunx e2e run --no-cache # run the agent live instead of replaying ``` `--debug` adds the run's phase timings and a table with one row per agent step: duration, model calls, tokens, prompt-cache share, and cost when the provider reports one. Each planned step also saves its model transcript as an artifact. ```console [e2e debug] agent steps (execution order, model openai/gpt-6-luna-fast, total $0.0312) step total model observe action calls tokens in/out cached cost act "add a todo named groceries" 3.9s 2.8s 410ms 620ms 3 9412/388 61% (5740) $0.0198 assert "the list shows groceries" 1.2s 1.1s 88ms 0ms 1 3120/41 0% (0) $0.0114 ``` `--ai-trace` writes `.e2e/ai-trace.json`: one run per agent step, one entry per model call, with the redacted input and image bytes replaced by counts. Read it with [unbox-ai](https://github.com/e2e-inc/unbox-ai) to see prompts, tool calls, and token usage: ```bash npm npx unbox-ai runs .e2e/ai-trace.json npx unbox-ai summary .e2e/ai-trace.json --run 0 ``` ```bash pnpm pnpm dlx unbox-ai runs .e2e/ai-trace.json pnpm dlx unbox-ai summary .e2e/ai-trace.json --run 0 ``` ```bash bun bunx unbox-ai runs .e2e/ai-trace.json bunx unbox-ai summary .e2e/ai-trace.json --run 0 ``` A replayed step makes no model call and leaves no AI trace. Add `--no-cache` to trace the whole flow or rule out a stale [recording](/cache). ## Common errors | Code | Usual cause | Fix | | --- | --- | --- | | `APP_UNREACHABLE` | Nothing answers at `app.url`, or `app.command` never became ready. On iOS, the message may name the automation runner instead | Set `app.command.log` and read it; check the port; raise `app.command.startupTimeout`. For the iOS runner, follow the message (see [Mobile](/mobile#troubleshooting)) | | `APP_ALREADY_RUNNING` | Something already serves `app.url` when the runner wanted to start `app.command` | Stop it, or set `reuseExisting: true` for local runs | | `LOCATOR_NOT_FOUND` | Wrong role or name, text not exact, element inside an iframe, page not open | Read the markup for the accessible name; try `exact: false`; `browser.frameLocator` for iframes; `--headed` to look | | `LOCATOR_AMBIGUOUS` | Two matches: a hidden duplicate, a repeated label | Add `{ name }`, scope under a container, `filter`, `first()`, or `{ visible: true }` | | `ASSERTION_FAILED` | The expectation is wrong, or the state settles after 5 s; for `agent.assert`, the judgment was false | Compare with the screenshot; `{ timeout }` on the matcher; rewrite the question | | `ASSERTION_INCONCLUSIVE` | `agent.assert` or `agent.extract` asked about something the screen doesn't show: on another page, still loading, or only in pixels | Open or wait for the right screen first; add `vision: true` when the answer is in pixels (the message says so) | | `ACTION_FAILED` | Element covered, disabled, or detached, or an operation timed out | Wait on the right condition with `expect` first; close overlays | | `STEP_BUDGET_EXHAUSTED`, `STEP_TIMEOUT` | The agent ran out of actions, model calls, or time | Scope the goal to one step. Raise the agent entry's `maxSteps` or `maxModelCalls` for the budget, the config `timeout` for an `act` step, or the agent's `judgmentTimeout` for `assert`, `waitFor`, and `extract` | | `MODEL_UNAVAILABLE` | No model constructed in the config, or GitHub Copilot doesn't serve the configured model to your account | See [Models](/models); for Copilot, [Pick a model](/subscriptions#pick-a-model) | | `MODEL_PROVIDER_FAILED` | Bad key, rate limit, no credits, or every try of one request went 120 s without an answer | Check the key the provider reads and the quota; for a slow local model, a smaller screen (`agents..maxObservationBytes`) or a faster model | | `MODEL_REFUSED` | The model refused the prompt (finish reason `content-filter`). Exit 1 | Rephrase the instruction, or switch to another model | The [errors reference](/reference/errors) lists every code, its class, and its exit code. ## Next Every flag of `e2e run`. Every error code.