Files
e2e/docs/reference/errors.mdx
T

137 lines
13 KiB
Plaintext

---
title: Errors
description: Every code you can see, its class, and its exit code.
---
Every error carries a class. The class determines the exit code; the code
identifies the cause. A model never selects either.
| Class | Exit code |
| --- | ---: |
| test | 1 |
| configuration | 2 |
| infrastructure | 3 |
| internal | 4 |
| interrupted | 130 |
## AgentError
```ts
/** Runner-classified agent failure. */
export class AgentError extends Error {
/** Closed union, assigned by the runner, never taken from model text. */
readonly code: AgentErrorCode;
/** Sanitized, bounded prose; the same value as `message`. */
readonly explanation: string;
/** Artifact path, present only when evidence was captured and permitted. */
readonly screenshot?: string;
/** True when this failure reports a blocked step, not a product failure. */
readonly blocked: boolean;
}
```
Test files load in their own module realm, so `instanceof AgentError` can be
false. Use `isAgentError` from `e2e` and branch on `code`:
```ts
import { isAgentError } from 'e2e';
try {
await agent.act('upgrade the workspace to Pro');
} catch (error) {
if (isAgentError(error) && error.code === 'STEP_TIMEOUT') {
// the step ran out of time; the message carries the last judgment
}
throw error;
}
```
| Code | When it happens | Class | Exit |
| --- | --- | --- | ---: |
| `AUTH_CREDENTIAL_UNAVAILABLE` | A `Secret` or `credentials.user()` names a credential absent from `config.credentials`, or the call happened outside an active runner | configuration | 2 |
| `AUTH_CREDENTIAL_INVALID` | A configured credential was rejected by the application (blocked: credentials) | configuration | 2 |
| `SECRET_UNAVAILABLE` | A `Secret` or `secrets.get()` names a secret absent from `config.secrets`, or the call happened outside an active runner | configuration | 2 |
| `ENVIRONMENT_UNAVAILABLE` | The environment under test is down or unreachable beyond the app itself (blocked: environment) | infrastructure | 3 |
| `SEED_DATA_MISSING` | Data the step depends on does not exist in the environment (blocked: seed_data) | configuration | 2 |
| `TEST_SETUP_FAILED` | Test prerequisites outside the product failed to arrange (blocked: test_setup). Also a `test.extend()` fixture that broke its contract: returned without calling `use()`, called it twice, or redefined a fixture the target's engine contributes | configuration | 2 |
| `AUTOMATION_UNSUPPORTED` | The step needs an interaction the available toolset cannot perform, e.g. drag-and-drop (blocked: automation) | test | 1 |
| `MODEL_UNAVAILABLE` | No model configured: neither `createAgent({ model })` nor `agents.<name>.model` holds an AI SDK model instance. Checked once per run when the `agent` fixture is first acquired; recorded as one run-level error that stops the run. | configuration | 2 |
| `POLICY_DENIED` | Origin, sink, or purpose denied before dispatch. Also reading a value from a secure field. | configuration | 2 |
| `APP_UNREACHABLE` | The app process or URL did not respond, including a navigation the network refused (connection refused, name not resolved, timed out) because nothing listens at the engine's `url`. After a `command` or service is spawned, an early exit or a spent `startupTimeout` ends the message with the last lines appended to its `log` (`command.env` values replaced by `<secret:NAME>` markers), reports `no output` when nothing was appended, or says to set a `log` when none is configured | infrastructure | 3 |
| `APP_ALREADY_RUNNING` | `readyUrl` already answered before the engine's `command` (or a `readyUrl` service) was spawned and `reuseExisting` was not in effect; stop that process or set `reuseExisting: true` locally | infrastructure | 3 |
| `MODEL_PROVIDER_FAILED` | Provider transport failure (network error, 5xx, rate limit) after the adapter's transport retries (up to 5 per model call, honoring `retry-after`, bounded by the remaining step timeout) | infrastructure | 3 |
| `CANCELLED` | The runner cancelled the operation, without a process signal | infrastructure | 3 |
| `MODEL_OUTPUT_INVALID` | Structured output still failed validation after the repair budget was spent | test | 1 |
| `APP_NOT_OPEN` | An operation needed an open app; `app.open()` had not run, or the engine reported invalid state | test | 1 |
| `LOCATOR_NOT_FOUND` | Zero matches within the deadline, or the model declined to name a node | test | 1 |
| `LOCATOR_AMBIGUOUS` | More than one match for a single-node operation. Fails immediately, no polling. | test | 1 |
| `ACTION_FAILED` | Actionability failure, operation timeout, or an action that may already have committed | test | 1 |
| `STEP_BUDGET_EXHAUSTED` | The model-call or action-step budget ran out before a conclusion | test | 1 |
| `STEP_TIMEOUT` | The invocation deadline expired. Carries the last judgment where one exists. | test | 1 |
| `CONTEXT_OVERFLOW` | A model request exceeded the model's context window. `agent.act` shrinks the step's history (superseded screens elided, long texts cut) and retries once when that changes the request; judgment calls (`assert`, `waitFor`, `extract`) report it directly. A smaller `agent.maxObservationBytes` or a shorter step is the fix (blocked: automation) | test | 1 |
| `ASSERTION_FAILED` | `agent.assert` judged false, or a locator, `web`, or value matcher failed | test | 1 |
| `ASSERTION_INCONCLUSIVE` | `agent.assert` could not judge: the screen did not show enough to decide either way (the relevant part is not on screen, still loading, or unreadable). A failure, never a pass; the explanation says what was missing | test | 1 |
| `AUTHENTICATION_FAILED` | The executor concluded that signing in failed inside the app: a product verdict. `AUTH_CREDENTIAL_INVALID` is the blocked counterpart for a configured credential the app rejects | test | 1 |
| `STEP_NO_CONCLUSION` | The executor used every turn without calling `complete_step`; the step fails with the agent's last summary | test | 1 |
## Runner codes
These appear in terminal output and in the report as plain string codes.
| Code | When it happens | Class | Exit |
| --- | --- | --- | ---: |
| `CONFIG_NOT_FOUND` | `--config <path>` names a file that does not exist, or no `e2e.config.ts` or `e2e.config.mts` exists from the working directory up to the repository root | configuration | 2 |
| `CONFIG_AMBIGUOUS` | Both `e2e.config.ts` and `e2e.config.mts` in one directory | configuration | 2 |
| `CONFIG_LOAD_FAILED` | The config module failed to import: a package that is declared but not installed, an undeclared package, a wrong subpath, or a removed export. The message names the fix | configuration | 2 |
| `INVALID_CONFIG` | Unknown key, out-of-range value, invalid enum, missing `targets`, a target key outside `{ name, platform, engine }`, an `engine` that is not a `defineEngine` handle, `json` reporter combined with `list`, duplicate or malformed target name, oversized `agent.context`, an `expect.poll` `timeout` or `interval` that is not a finite number of milliseconds | configuration | 2 |
| `CONFIG_NOT_DETERMINISTIC` | A worker re-resolved the config module and got a different digest than the runner | configuration | 2 |
| `APP_URL_REQUIRED` | A navigation or relative URL on a target whose engine declares no `url`, or a declared `command` with no `url` or `readyUrl` to poll | configuration | 2 |
| `INVALID_APP_URL` | Base URL with userinfo, a query, a fragment, a non-HTTP scheme, plain HTTP on a non-loopback host, port 0 on a host other than a literal loopback address (`127.0.0.1`, `[::1]`), a loopback address the run could not bind a free port on, or unparseable | configuration | 2 |
| `INVALID_GLOB` | Empty pattern, a leading `!` exclusion, or unsupported glob syntax | configuration | 2 |
| `UNKNOWN_TARGET` | `--target` named an ID that is not configured | configuration | 2 |
| `TARGET_REQUIRED` | `e2e mcp`: the config declares several targets and neither `--target` nor `open_session` named one | configuration | 2 |
| `NO_SESSION` | `e2e mcp`: `call`, `tools`, or `close_session` before `open_session` or after the session ended (the message names why), or naming a `session` id that is not the open one | configuration | 2 |
| `SESSION_OPEN` | `e2e mcp`: `open_session` while a session is already open; use it or `close_session` first | configuration | 2 |
| `UNKNOWN_TOPIC` | `e2e mcp`: an `e2e://guide/{topic}` resource named a topic the skill does not have; the message lists them | configuration | 2 |
| `UNKNOWN_TOOL` | `e2e mcp`: `call` or `tools` named a tool the session's catalog does not have; the message lists the catalog | configuration | 2 |
| `UNSUPPORTED_ARTIFACT` | A configured artifact kind (`screenshot`, `trace`, `video`) the target's engine cannot produce | configuration | 2 |
| `UNSUPPORTED_CAPABILITY` | A fixture, action, or tier the target's engine does not declare: a contributed fixture it does not contribute, `screen` without `locate`, state restore without `state`, screenshots without `artifacts` | configuration | 2 |
| `NO_TESTS` | Zero runnable ordinary test-target pairs. The message names the cause: no file matched the `tests` globs (with look-alike files), a positional matched nothing, a file registered no tests, or filters excluded everything. Suppress with `--pass-with-no-tests`. | configuration | 2 |
| `ONLY_IN_CI` | A focused test reached a CI run | configuration | 2 |
| `COLLECTION_ERROR` | Registration outside collection, an invalid title, a non-function body, an async `describe` body, nested serial groups, an option forbidden inside a serial group, a setup test inside `describe`, an invalid session name, or a positional path outside the project root | configuration | 2 |
| `POLICY_DENIED` | Origin, cookie, scheme, secret-sink, or route policy denial; `app.screenshot()` after a secret has tainted the attempt | configuration | 2 |
| `SESSION_MISMATCH` | A stored session does not match the current run, target, engine, platform, or app identity | configuration | 2 |
| `SESSION_EXPIRED` | A stored session is past its expiry at restore time (configuration, exit 2), or a setup test saved state whose expiry is not after its creation (test, exit 1) | configuration or test | 2 or 1 |
| `SESSION_INVALID` | A stored session failed authentication. Tampered or corrupt. | configuration | 2 |
| `SESSION_UNAVAILABLE` | A test requires a session no selected setup test produced | configuration | 2 |
| `SESSION_CONTRACT` | A setup test saved a session twice, saved an undeclared name, or finished without saving a declared one | test | 1 |
| `TEST_TIMEOUT` | The attempt exceeded the resolved test `timeout` | test | 1 |
| `HOOK_FAILED` | `beforeAll` or `afterAll` threw. Skips the suite's tests. | test | 1 |
| `LAUNCH_TIMEOUT` | Engine init or attempt start exceeded `config.launchTimeout` | infrastructure | 3 |
| `CLEANUP_TIMEOUT` | Engine attempt end exceeded `config.cleanupTimeout` | infrastructure | 3 |
| `TRACE_WITHHELD` | A trace from an attempt that filled a secret could not be rewritten, so it was deleted. Recorded in the attempt's `secondaryErrors`; never fails the attempt. | infrastructure | 3 |
| `BROWSER_INSTALL_FAILED` | Browser provisioning failed | infrastructure | 3 |
| `WORKER_INIT_FAILED` | A worker process never became ready | infrastructure | 3 |
| `WORKER_EXIT` | A worker exited unexpectedly between tests | infrastructure | 3 |
| `WORKER_CRASH` | A worker exited during a specific test | infrastructure | 3 |
| `WORKER_ENTRY_MISSING` | The worker entry module could not be resolved. Packaging fault. | infrastructure | 3 |
| `ENGINE_FAILURE` | Any engine fault with no more specific mapping | infrastructure | 3 |
| `WORKER_PROTOCOL` | A worker reported a unit other than the one it was running, or started an attempt while another was still running | infrastructure | 3 |
| `SESSION_KEY_INVALID` | A worker received a session key that is not 32 bytes. Packaging or IPC fault. | infrastructure | 3 |
| `REPORT_WRITE_FAILED` | `report.json` or `ai-trace.json` could not be written. Recorded as a run error; the in-memory outcome is still complete. A `junit.xml` that could not be written is the `junit` reporter's stderr line instead | infrastructure | 3 |
| `EVALUATE_FAILED` | `web.evaluate` threw inside the page (`@e2edev/playwright`) | test | 1 |
| `INTERRUPTED` | A user or CI signal aborted the run | interrupted | 130 |
Errors raised by argument validation in test code use `INVALID_ARGUMENT` and
`INVALID_LOCATOR`, both test failures with exit code 1. An unclassified thrown
value from test code is reported as `ERROR`. The `internal` class and exit
code 4 are reserved for runner invariant violations; no first-party code raises
a code in that class today.
## Exit codes
For a mixed run the highest-precedence code present wins, `130 > 4 > 3 > 2 >
1 > 0`. The report keeps every individual result. A setup assertion failure
is a test failure and skips its dependents. Missing setup infrastructure keeps
its configuration or infrastructure class.