Files
e2e/docs/reference/engine.mdx
T

2450 lines
97 KiB
Plaintext

---
title: e2e/engine
description: Build an engine with observations, actions, lifecycle hooks, and fixtures.
"og:image": "https://e2e.com/docs/images/og/reference/engine.png?v=d82cf186db"
"twitter:image": "https://e2e.com/docs/images/og/reference/engine.png?v=d82cf186db"
---
`e2e/engine` is the contract an engine implements to drive a user interface,
such as a browser or simulator. An engine implements shared types for nodes,
locators, actions, and errors. `defineEngine` validates and freezes the
implementation, derives its capabilities, and returns a handle for `targets`
in the config. The runner drives it over the
[engine protocol](/reference/engine-protocol), in its own process; see
[Engines](/engines). Use `test.extend` fixtures for resources outside the UI,
such as APIs or databases. `@e2e-dev/core/engine` is the same module: a
published engine imports it from there and peers on `@e2e-dev/core`, so it runs
under the runner and under [@e2e-dev/core](/core) alike.
This sketch assumes a `connect` adapter for the platform:
```ts title="src/index.ts"
import { defineEngine, resolveExpression, type EngineHandle } from 'e2e/engine';
import { connect } from './client.ts';
export function kiosk(options: { host: string }): EngineHandle {
const client = connect(options.host);
return defineEngine({
name: 'kiosk',
version: '1.0.0',
spiVersion: 1,
platform: 'kiosk',
async observe() {
const screen = await client.snapshot();
return {
root: { ref: { id: 'root', revision: '' }, role: 'window', children: screen.nodes },
viewport: screen.viewport,
};
},
async locate(expression) {
const screen = await client.snapshot();
return resolveExpression(expression, screen.nodes);
},
actions: ['tap'],
async perform(ref) {
await client.tap(ref.id);
},
});
}
```
```ts title="e2e.config.ts"
import type { E2EConfig } from 'e2e';
import { kiosk } from './src/index.ts';
export default {
targets: [{ name: 'kiosk', engine: kiosk({ host: 'kiosk.local' }) }],
} satisfies E2EConfig;
```
## Reference
### defineEngine
Validates an engine spec, freezes it, computes its
[capabilities](#capabilities), and returns a branded handle.
`isEngineHandle` tells such a handle from any other value.
```ts
import { defineEngine, isEngineHandle, type EngineHandle } from 'e2e/engine';
export function defineEngine<Fixtures extends object = {}>(spec: Engine): EngineHandle<Fixtures>;
// With a fixture beside screen, app, and platform, `spec.fixtures` must hold
// a factory returning each one Fixtures names.
export function isEngineHandle(value: unknown): value is EngineHandle;
/** A validated engine: branded, frozen, capabilities computed. */
export interface EngineHandle<Fixtures extends object = {}> extends Engine {
readonly capabilities: ReadonlySet<EngineCapability>;
}
```
Every function on the handle is bound to the spec it came from, so a class
instance is as valid a body as an object literal.
`Fixtures` types what the engine's [`fixtures`](#contributed-fixtures) contribute, such as
`{ browser: Browser }`. The compiler requires a factory in `fixtures` for each
fixture it names, returning that fixture; `screen`, `app`, and `platform` need
none, since every session holds them. Nothing checks it at runtime. With it, `launch` from [`@e2e-dev/core`](/core) types a session's fixtures
without a hand-written argument: return `defineEngine<{ device: Device }>(...)`
from the factory.
`defineEngine` throws `INVALID_CONFIG` when the spec breaks one of these
rules:
| Member | Rejected when |
| --- | --- |
| the spec | It is not an object. |
| `name` | Missing or empty. |
| `version` | Missing or empty. |
| `spiVersion` | Anything but `1`. |
| `platform` | Empty. |
| `workers` | Not a positive safe integer. |
| `observe`, `readStyle`, `locate`, `perform`, `performAt`, `prepare`, `finish`, `doctor`, `init`, `startAttempt`, `settleAttempt`, `endAttempt`, `dispose`, `validateApp` | Not a function. |
| any other top-level key | On an object literal, `app` included: declare the app on the target and check it in `validateApp`. A class instance's own fields are not checked. |
| `readStyle`, `perform`, `locate`, `performAt`, `keyboard` | Declared without `observe`. |
| `perform`, `actions` | One without the other. An `actions` list that is empty, repeats a kind, or names one outside `LOCATOR_ACTION_KINDS`. |
| `performAt`, `pointerActions` | One without the other. A `pointerActions` list that is empty, repeats a kind, or names one outside `POINTER_ACTION_KINDS`. |
| `tapModifiers` | Not a boolean, or `true` without `perform`. |
| `keyboard`, `state`, `artifacts`, `session` | An unknown key or a non-function member. `keyboard` without `type` and `press`; `state` without both `capture` and `restore`; `artifacts` without `screenshot`; `startVideo` without `stopVideo`, or the reverse. |
| `secrets` | Not an array of `secrets.get()` handles. |
| `fixtures` | Not an object, a fixture that is not a factory function, or a fixture name that is not a lower-camel identifier or shadows `agent`, `app`, `screen`, `platform`, or `session`. |
<Note>
`artifacts.startTrace` and `artifacts.stopTrace` were removed from the
contract. An engine published before their removal that declares them still
passes validation; the runner never calls them.
</Note>
### Engine
The body of one target. Every member but `name`, `version`, and
`spiVersion` is optional; the members an engine implements decide its
[capabilities](#capabilities).
| Member | Type |
| --- | --- |
| `name` | `string` |
| `version` | `string` |
| `spiVersion` | `1` |
| `platform` | `string` |
| `workers` | `number` |
| [`observe`](#observation) | method |
| [`readStyle`](#rendered-node-properties) | method |
| [`perform`, `actions`](#actions) | method, kinds |
| [`locate`](#capabilities) | method |
| [`performAt`, `pointerActions`](#pointer-actions) | method, kinds |
| [`tapModifiers`](#capabilities) | `boolean` |
| [`keyboard`](#keyboard) | `EngineKeyboard` |
| [`fixtures`](#contributed-fixtures) | factories |
| [`state`](#state) | `EngineStateCapability` |
| [`artifacts`](#artifacts) | `EngineArtifacts` |
| [`validateApp`](#app) | method |
| [`secrets`](#startattempt) | `Secret[]` |
| [`session`](#session) | `EngineSession` |
| [`prepare`, `finish`, `doctor`, `init`, `startAttempt`, `settleAttempt`, `endAttempt`, `dispose`](#lifecycle) | hooks |
```ts
/** The body of one target: typed, model-free, capability-graded. */
export interface Engine {
/** Engine name, for diagnostics. */
readonly name: string;
/**
* Required implementation version, recorded in reports and sessions.
* A saved session never restores across engine versions; a cached trace
* replays across them and re-finds its nodes.
*/
readonly version: string;
/**
* Contract version literal. Additive optional members never bump it;
* changed required semantics do.
*/
readonly spiVersion: EngineSpiVersion;
/**
* Platform label this engine drives (`web`, `ios`, `android`, or a label of
* the engine's own). Tests filter on it through `platforms` and tool packs
* are scoped by it; the harness attaches no meaning to any value. A target
* inherits it; a target that names a platform of its own must agree with it.
*/
readonly platform?: string;
/**
* Maximum workers for this target, such as the number of devices in a
* pool. The scheduler applies this cap even when `config.workers` is
* higher. Omit when the engine has no concurrency limit.
*/
readonly workers?: number;
/** capability: observation. */
observe?(context: OperationContext, options?: EngineObserveOptions): Promise<EngineSnapshot>;
/** capability: computed-style, requires observation. Read-only rendered properties of an observation ref. */
readStyle?(ref: NodeRef, properties: readonly ComputedStyleProperty[], context: OperationContext): Promise<ComputedStyleValues>;
/**
* Performs one action on a ref from the newest observation or `locate`,
* with the platform's actionability checks. Both agent tools and locator
* actions call this method. A `swipe` on the observation root swipes the
* viewport.
*
* For safe trace replay, throw a retryable
* `NODE_STALE`-coded error when a ref no longer binds, and an
* `ACTION_MAY_HAVE_COMMITTED`-coded error when input may have reached the
* app. The runner never repeats an action that may have committed.
*/
perform?(ref: NodeRef, action: LocatorAction, context: OperationContext): Promise<void>;
/**
* The action kinds `perform` honors, required with it. The harness offers
* the agent and the `screen` tier exactly these: a kind left out is never
* presented as a tool that can only decline, and `screen`/`Locator` methods
* for it fail with `UNSUPPORTED_CAPABILITY` before reaching the engine.
* `perform` still throws `UNSUPPORTED_CAPABILITY` for a declared kind one
* particular node cannot take (a `check` on a toggle whose state is unknown).
*/
readonly actions?: readonly LocatorActionKind[];
/**
* capability: location - requires observation. Deterministic locator
* resolution for the `screen`/`expect` tier: resolve one expression to the
* nodes it currently matches. The harness owns polling, strictness, and
* staleness; an engine resolves once, immediately.
*/
locate?(
expression: LocatorExpression,
context: OperationContext,
): Promise<readonly SemanticNode[]>;
/**
* capability: pointer - requires observation. Performs one pointer action
* at a viewport point, in the CSS pixels of `SemanticNode.rect`, with no
* node behind it: the agent's `tap_at`, `hover_at`, and `drag_at` verbs land
* here with the point the model named in a screenshot (a shape on a canvas, a pin on
* a map, a control in a system sheet or one the tree lists), and so do a test's
* `screen.tapAt`, `screen.swipe({ from, to })`, and `tap({ position })`.
* Dispatch the pointer at the point as given; the harness has clamped an
* agent's point to the viewport and validated a test's. Throw
* `UNSUPPORTED_CAPABILITY` for a declared kind the surface cannot deliver
* at this particular point.
*/
performAt?(point: ViewportPoint, action: PointerAction, context: OperationContext): Promise<void>;
/**
* The pointer action kinds `performAt` honors, required with it. The
* harness routes a point-addressed action to the engine only for a kind
* listed here; an undeclared kind fails with `UNSUPPORTED_CAPABILITY`
* before reaching the engine.
*/
readonly pointerActions?: readonly PointerActionKind[];
/**
* The surface honors `modifiers` on the `tap`, `doubleTap`, and
* `secondaryTap` actions `perform` receives. Without it the harness fails
* an action that carries modifiers with `UNSUPPORTED_CAPABILITY` before it
* reaches the engine.
*/
readonly tapModifiers?: boolean;
/** capability: keyboard - requires observation. Input to whatever holds focus; see `EngineKeyboard`. */
readonly keyboard?: EngineKeyboard;
/**
* Named deterministic surfaces this engine contributes to TestFixtures
* (a device fixture, a document fixture - anything). Keys become fixture and
* capability names; `requires: ['<name>']` gates at selection.
*/
readonly fixtures?: Readonly<Record<string, EngineFixtureFactory>>;
/** capability: state - opaque snapshot capture/restore for session reuse. */
readonly state?: EngineStateCapability;
/** capability: artifacts - screenshots under the attempt directory, and videos there or at a provider. */
readonly artifacts?: EngineArtifacts;
/**
* Checks the app a target declares against what this engine can drive, at
* config load, synchronously: throw `INVALID_CONFIG` naming
* `info.targetName` for a field the platform needs and the target left out
* (a URL, a bundle id) or one it cannot drive. The harness has already
* checked every field's shape.
*/
validateApp?(app: EngineAppDeclaration, info: EngineAppCheckInfo): void;
/**
* The `secrets.get()` handles the engine's options hold, such as a
* basic-auth password. The config load checks each names a configured
* secret (`INVALID_CONFIG` otherwise), and only these resolve through
* `EngineAttemptContext.resolveSecret`. The handles carry names only, so
* no value reaches the config digest.
*/
readonly secrets?: readonly Secret[];
/** Steering hooks: open, back, resize, restart, reset. */
readonly session?: EngineSession;
/**
* Runs once per target in the runner, after collection and before workers,
* app commands, and the run's clock start. A run with nothing to run (the
* test filters left nothing, or `--pass-with-no-tests` on an empty
* selection) prepares no target, and so calls no `finish`. Download
* dependencies or boot devices here; report progress through `info.log`.
* No launch timeout applies. A failure ends the run as an infrastructure
* error.
* The result can lower the target's worker cap, for example after
* discovering how many devices are available.
*/
prepare?(info: EnginePrepareInfo): Promise<void | EnginePrepareResult>;
/**
* Once per run and target in the runner process, after every worker of the
* target is gone, on every exit path, whenever `prepare` was called.
* Release what `prepare` acquired for the run (a leased cloud device).
* Runs within the cleanup budget, never cancelled by an interrupt; a
* failure is a cleanup-phase run error.
*/
finish?(info: EngineFinishInfo): Promise<void>;
/**
* Checks this machine for what a run on the target needs, for
* `e2e doctor`. Read-only and quick: never install, launch, lease, or
* start anything a check could instead look up, and never call `prepare`.
* A check that cannot tell reports nothing rather than guess. A throw is
* reported as one failed check of the target.
*/
doctor?(info: EngineDoctorInfo): Promise<readonly EngineDoctorCheck[]>;
/**
* Once per worker, before the first step; boot devices here, not in a step
* budget. The same handle can be booted again after `dispose`: a config-held
* handle outlives an in-process worker, so init MUST work on a disposed
* engine as it does on a fresh one.
*/
init?(info: EngineInitInfo): Promise<void>;
/** Before each attempt: set up per-test isolation. */
startAttempt?(context: EngineAttemptContext): Promise<void>;
/**
* After the body has settled, and again after teardown, before the verdict,
* within the cleanup budget: report a failure that landed on a path no step
* awaited (a callback the surface ran for the test: a request interceptor,
* a native dialog handler). Wait for one still running, then throw its
* error; the attempt fails with it the way a step would, an error already
* classified keeping its code. MUST report each failure once.
*/
settleAttempt?(context: EngineCleanupContext): Promise<void>;
/**
* After each attempt, within the cleanup budget: tear that isolation down.
* MUST be idempotent, and safe to call after a failed `startAttempt`.
*/
endAttempt?(context: EngineCleanupContext): Promise<void>;
/**
* Worker shutdown, within the cleanup budget; failure is a run error. Runs
* once `init` was called, even when it failed, so it MUST tolerate a
* partly initialized engine. A live
* `reuseSignal` permits preserving run-owned resources for a replacement,
* until that permission is revoked or this hook settles.
*/
dispose?(context: EngineDisposeContext): Promise<void>;
}
```
#### `OperationContext`
```ts
/** What every engine call carries: budget, ids, and who is acting. */
export interface OperationContext {
/**
* Aborts when the attempt, or the hook running the call, ends. It does not
* abort at `timeoutMs`: bound each call by that budget yourself.
*/
readonly signal: AbortSignal;
/**
* Remaining operation budget when the call starts. A waiting call's last
* locate or read can get more than its deadline leaves: as long as the
* slowest one it has seen.
*/
readonly timeoutMs: number;
/** The run id. */
readonly runId: string;
/** The attempt id. */
readonly attemptId: string;
/**
* Who asked: a test's own deterministic step, or the agent acting on a
* screen it observed. A deterministic step verifies its outcome with
* `expect`, so an engine may act as soon as the target holds still; an
* agent reads the screen right after acting, so an engine may wait for the
* transition to end first.
*/
readonly origin: 'test' | 'agent';
}
```
### Capabilities
The members an engine implements set its capabilities. A `screen`,
`Locator`, or `app` call that needs a missing one fails with
`UNSUPPORTED_CAPABILITY`. [Test APIs](#test-apis) lists what needs which
member. Three gaps end differently:
- A test that `requires` a fixture the engine lacks is skipped
(`capability-unavailable`).
- A selected test that uses a `test.setup` session, on an engine without
`state`, is a collection error before the setup runs.
- A `video` mode on an engine without video recording prints a notice and
records no video.
| Member | Capability | Requires | Unlocks |
| --- | --- | --- | --- |
| `observe` | `observation` | - | The judgment tier, prompt snapshots, trace replay, the viewport swipe's root. |
| `readStyle` | `computed-style` | `observe` | Allow-listed rendered properties for read-only project tools. |
| `perform` + `actions` | `actions` | `observe` | Node actions of the declared kinds, with platform actionability checks. |
| `locate` | `location` | `observe` | `screen`, `Locator`, `expect(locator)`. |
| `tapModifiers: true` | - | `perform` | `modifiers` on `tap`, `doubleTap`, and `secondaryTap`. |
| `performAt` + `pointerActions` | `pointer` | `observe` | One pointer action at a viewport point, with no node behind it. |
| `keyboard.{type,press,dismiss?}` | `keyboard` | `observe` | Input to whatever holds focus. |
| `fixtures.<name>` | `<name>` | - | A contributed fixture. |
| `state` | `state` | - | `test.setup` sessions and the `session:` option. |
| `artifacts` | `artifacts` | - | Screenshots and videos. |
| `validateApp` | - | - | A config-load check of the target's `app`. |
| `session` | - | - | The `app` steering methods and the agent's `navigate`, `back`, `resize_viewport`, `switch_tab`, and `close_tab` verbs. |
#### Rendered node properties
`readStyle(ref, properties, context)` returns computed CSS strings for the requested
properties without changing app state. Its `computed-style` capability requires
`observe`, and `ref` comes from the newest observation. Return a string for
every requested property. Throw retryable `NODE_STALE` if the node no longer
binds. The harness redacts values before returning them to tools.
```ts
export type ComputedStyleProperty =
| 'color' | 'background-color' | 'font-size' | 'font-weight' | 'font-style'
| 'text-align' | 'text-decoration-line' | 'display' | 'visibility' | 'opacity';
export type ComputedStyleValues = Readonly<Partial<Record<ComputedStyleProperty, string>>>;
```
`COMPUTED_STYLE_PROPERTIES` exports the allow-list as a frozen array.
`validateComputedStyleProperties(properties)` returns a deduplicated copy and
throws `INVALID_ARGUMENT` for an empty list or a property outside the
allow-list. Call it before page access.
See [Project tools](/tools#read-a-nodes-computed-style) for `ToolContext.read`.
#### Agent verbs
The agent offers the model only the verbs the engine's declarations unlock.
An undeclared verb is absent from the toolset.
| Verb | Needs |
| --- | --- |
| `tap` | the `tap` kind in `actions` |
| `double_tap` | `doubleTap` |
| `long_press` | `longPress` |
| `right_click` | `secondaryTap` |
| `hover` | `hover` |
| `type` | `fill`; without a target, `keyboard` |
| `type_secret` | `fill` |
| `press` | `press`; without a target, `keyboard` |
| `select` | `selectOption` |
| `check` | both `check` and `uncheck`, since it sets a state rather than flipping one |
| `scroll`, `scroll_to({ text })` | `swipe` |
| `scroll_to({ target })` | `scrollIntoView` |
| `drag` | `dragTo` |
| `upload` | `setInputFiles` |
| `navigate` | `session.open` |
| `back` | `session.back` |
| `resize_viewport` | `session.resize` |
| `switch_tab` | `session.switchTab` |
| `close_tab` | `session.closeTab` |
| `tap_at`, `hover_at` | the `tap` or `hover` kind, in `actions` or in `pointerActions` |
| `drag_at` | the `dragTo` kind, in `actions` or in `pointerActions` |
| `type_at` | `fill` or `keyboard` |
| `press_at` | `press` or `keyboard` |
| `select_at` | `selectOption` |
| `dismiss_keyboard` | `keyboard.dismiss` |
- `clear` and `focus` unlock no verb.
- `tap_at` and `hover_at` go through `performAt` (the `tap` and `hover`
pointer kinds). Without that kind they go through `perform` when the point
lands on a node the tree lists.
- `drag_at` goes through `performAt` with the `dragTo` pointer kind and the
second point as its `target`. Without that kind it goes through `perform`
when its two points land on two different listed nodes.
- `type_at`, `press_at`, and `select_at` go through `perform` when the point
lands on a node the tree lists. Elsewhere `type_at` and `press_at` fall
back to `keyboard`.
#### Test APIs
The test-facing APIs each engine member backs. A `screen` or `Locator`
method for an undeclared kind, or a call with `modifiers` and no
`tapModifiers`, fails with `UNSUPPORTED_CAPABILITY` before it reaches the
engine.
| API | Needs |
| --- | --- |
| `screen`, `Locator`, `expect(locator)` | `locate` |
| `screen` and `Locator` actions | `perform`, each only for a kind listed in `actions` |
| `screen.swipe()` | `perform` with `swipe` on the observation root |
| `modifiers` on a tap (`row.click({ modifiers: ['Shift'] })`) | `tapModifiers: true`; the keys are held for the click |
| `screen.tapAt`, `tap({ position })` | `performAt` with the `tap` pointer kind |
| `screen.doubleTapAt` | `performAt` with the `doubleTap` pointer kind |
| `screen.swipe({ from, to })` | `performAt` with the `swipeTo` pointer kind |
| `locator.pressSequentially` | the `focus` action and `keyboard`; the harness composes `focus` then `keyboard.type` |
| `test.setup` sessions, the `session:` option | `state` |
| `app.screenshot()`, assertion evidence, the failure screenshot | `artifacts` |
| every [`video` mode](/reference/config#recording) but `off`, from the config, a target, a test, or `--video` | `artifacts` |
| `app.open()` | `session.open`; on a target with no app URL, `session.restart`, or `session.openLink` for a path on a target with `app.linkBase` |
| `app.back()` | `session.back` |
| `app.restart()` | `session.restart` |
| `app.clearState()` | `session.reset` |
| `requires: ['<name>']` | `fixtures.<name>` |
#### `engine.capabilities`
The computed `ReadonlySet` of capability names. Selection, the report, and
`UNSUPPORTED_CAPABILITY` messages read it.
```ts
const engine = kiosk({ host: 'kiosk.local' });
engine.capabilities.has('location'); // true: the engine implements locate
```
Selection also grades a target by the kind of app it declares, not by its
engine. A target with `app.url` has `browser`, and one with `app.bundleId` or
`app.appPath` has `native-app`, so a test can `requires: ['native-app']`.
```ts
/**
* Capability names: the closed harness capabilities plus one name per
* contributed fixture. `actions` is `perform`; `location` is `locate`; both
* require observation, because their refs live in the observation's id space.
* `pointer` is `performAt`, an action addressed by a viewport point rather
* than a node, and requires observation because the point is read off its
* pixels.
*/
export type EngineCapability =
| 'observation'
| 'computed-style'
| 'actions'
| 'location'
| 'pointer'
| 'keyboard'
| 'state'
| 'artifacts'
| (string & {});
```
#### Keyboard
```ts
/**
* capability: keyboard - requires observation. Input to whatever holds focus,
* with no node behind it: the agent's `type` and `press` without a target, and
* the fallback of its point-addressed `type_at` and `press_at` when the point
* lands on nothing the tree lists (a field drawn on a canvas, an input a
* platform flattens out of its accessibility tree). Focus is the app's: the
* harness tapped first, or the model said the field already has it.
*/
export interface EngineKeyboard {
/**
* Types `text` into the focused field as a person typing would, through
* the platform's keyboard path: a browser dispatches the key events of each
* character, a device sends the text through its keyboard. Where the
* platform has key events the app must receive them; setting the value
* without keystrokes is not a compliant implementation, since
* `locator.pressSequentially` and the agent's `type` rely on the app
* reacting as it does to a user. With `replace`, clears the field first
* (select-all and delete on a keyboard surface; the platform's clear on a
* device). Throw `NOT_ACTIONABLE` when nothing that accepts text has focus,
* so the keystrokes are never silently discarded into the document body.
*/
type(text: string, options: { readonly replace: boolean }, context: OperationContext): Promise<void>;
/**
* Sends one key of the contract grammar (`Enter`, `Escape`, `Tab`,
* `Shift+Tab`, a character) to whatever holds focus. The harness has
* already validated the key's shape; refuse with `UNSUPPORTED_CAPABILITY`
* a key the surface cannot deliver.
*/
press(key: string, context: OperationContext): Promise<void>;
/** Hides an on-screen keyboard. Platforms without one leave it undefined. */
dismiss?(context: OperationContext): Promise<void>;
}
```
### Lifecycle
The hooks the runner calls around workers and attempts. Every hook is
optional. The `Engine` declaration's JSDoc above states each hook's contract.
`init` and `startAttempt` run within `launchTimeout`. `finish`,
`settleAttempt`, `endAttempt`, and `dispose` run within `cleanupTimeout`.
`prepare` has no timeout. Both budgets come from the
[config](/reference/config).
| Hook | Runs |
| --- | --- |
| [`prepare(info)`](#prepare) | Once per target, before any worker |
| [`finish(info)`](#finish) | After the target's last worker is gone |
| [`doctor(info)`](#doctor) | Once per target in `e2e doctor`, never in a run |
| [`init(info)`](#init) | Once per worker, before its first step |
| [`startAttempt(context)`](#startattempt) | Before each attempt |
| [`settleAttempt(context)`](#settleattempt) | After the body and after teardown |
| [`endAttempt(context)`](#endattempt) | After each attempt |
| [`dispose(context)`](#dispose) | Worker shutdown |
#### `prepare`
Runs once per target in the runner process, after collection and before
workers, app commands, and the run's clock start.
```ts
async prepare(info) {
if (info.slots === 0) return; // nothing runs on this target
info.log('booting devices');
const devices = await pool.boot(info.slots, { signal: info.signal });
return { workers: devices.length, env: { KIOSK_DEVICES: devices.join(',') } };
},
```
A failure ends the run before any test:
| Thrown | Becomes |
| --- | --- |
| A runner error class, such as a `ConfigurationError` for an option the engine cannot honour | The same code. |
| An `EngineError` with code `CANCELLED` | A cancellation. |
| Any other `EngineError`, or a plain `Error` | Infrastructure `ENGINE_FAILURE`. |
```ts
/**
* What `prepare` learned that the run must honour. `workers` replaces the
* engine's declared cap for this target and run: the surfaces the engine
* actually provisioned, `1` to `info.slots`. Declared `workers` stays the
* static answer for engines that know it up front.
*/
export interface EnginePrepareResult {
/** Lowers the target's worker cap for this run, `1` to `info.slots`. */
readonly workers?: number;
/**
* Variables every worker of this target is spawned with, on top of the
* run's environment: how a runner-side `prepare` hands what it provisioned
* (a device pool it discovered) to each worker's `init`. The run's own
* variables win on a clash.
*/
readonly env?: Readonly<Record<string, string>>;
}
/**
* Facts handed to `prepare`, once per run and target in the runner process,
* before any worker exists.
*/
export interface EnginePrepareInfo {
/** The run id. */
readonly runId: string;
/** The target being prepared. */
readonly targetName: string;
/** Directory relative paths in the config resolve against; see `EngineInitInfo.projectRoot`. */
readonly projectRoot: string;
/** The app the target declares, as `init` receives it: what a device installs or warms up here. */
readonly app: EngineAppInfo;
/**
* The worker slots the run will start for this target, `0` to `slots - 1`:
* the run's worker cap, the engine's declared `workers`, and the work units
* selected for the target, whichever is smallest; `0` when nothing runs on
* it while another target has work. A run with nothing to run prepares no
* target. Provision for exactly these slots (boot that many devices of a
* pool); `init` then finds them ready.
*/
readonly slots: number;
/**
* The run's environment: what every worker is started with. A host may hand
* the run an environment other than the runner process's own, so anything
* provisioned here that a worker later looks up by environment (a browser
* cache location) must be resolved and spawned against this, not
* `process.env`.
*/
readonly env: Readonly<Record<string, string | undefined>>;
/**
* Aborts on interrupt only. Provisioning has no budget: a first-run
* download is as long as the network makes it, and cutting it short would
* fail every test behind it.
*/
readonly signal: AbortSignal;
/** Whether the run asked for a visible surface (`--headed`), as `EngineInitInfo` declares it. */
readonly headed: boolean;
/**
* Reports one line of progress. The runner streams it as a `notice` run
* event, so it reaches the reporter and every host sink instead of being
* written to a worker's stderr underneath the live status block.
*/
readonly log: (line: string) => void;
}
```
#### `finish`
Runs once per run and target in the runner process, after every worker of
the target is gone. Release what `prepare` acquired for the run.
```ts
async finish(info) {
await pool.release({ signal: info.signal });
},
```
The `finish` hooks of all prepared targets share one `cleanupTimeout`.
```ts
/** Handed to `finish`, once per run and target after the last worker is gone, with the cleanup budget of one hook. */
export interface EngineFinishInfo extends EngineCleanupContext {
/** The run id. */
readonly runId: string;
/** The target being finished. */
readonly targetName: string;
/** The run's environment, the same `prepare` saw. */
readonly env: Readonly<Record<string, string | undefined>>;
/** Reports one line of progress, streamed as a `notice` run event like `prepare`'s. */
readonly log: (line: string) => void;
}
```
#### `doctor`
Runs once per target when [`e2e doctor`](/reference/cli#e2e-doctor) checks
the project, never during a run. Report what a run on this machine needs and
whether it's there. Each finding becomes a check under the target. Without the
hook, the target gets one `engine` check with status `skip`.
```ts
async doctor(info) {
const sdk = await findToolchain(info.env);
if (sdk === undefined) return [{ id: 'toolchain', status: 'fail', message: 'install the toolchain' }];
return [{ id: 'toolchain', status: 'pass', message: `toolchain ${sdk.version}` }];
},
```
`e2e doctor` gives the hook 30 seconds, starting a protocol engine's
server included. A throw, a timeout, a finding that isn't an
`EngineDoctorCheck`, or one with an id `e2e doctor` uses itself (`node`,
`config`, `engine`, `model`) becomes an `engine` check with status `fail`.
```ts
/** Facts handed to `doctor`, once per target in the process `e2e doctor` runs in. */
export interface EngineDoctorInfo {
/** The target being checked. */
readonly targetName: string;
/** Directory relative paths in the config resolve against; see `EngineInitInfo.projectRoot`. */
readonly projectRoot: string;
/** The app the target declares, as `prepare` receives it. */
readonly app: EngineAppInfo;
/** The environment a run would start with. */
readonly env: Readonly<Record<string, string | undefined>>;
/** Whether the run would show the surface (`--headed`), as `prepare` receives it. */
readonly headed: boolean;
/** Aborts 30 seconds in, when the checks fail as timed out, or when `e2e doctor` is interrupted; return within 5 seconds of it, when the connection closes. */
readonly signal: AbortSignal;
/** Reports one line of progress; `e2e doctor` prints it to stderr. */
readonly log: (line: string) => void;
}
/** One finding of `doctor`. */
export interface EngineDoctorCheck {
/** What was checked, in kebab case: `browser`, `toolchain`. */
readonly id: string;
/**
* `pass`; `warn` for something a run handles on its own, slower (a
* first-run download); `fail` for something that stops a run; `skip` for
* a check that does not apply to this target.
*/
readonly status: 'pass' | 'warn' | 'fail' | 'skip';
/** One line: what was found and, unless it passed, what to do. */
readonly message?: string;
}
```
#### `init`
Runs once per worker, before the first step. Boot devices here, not in a step
budget.
```ts
async init(info) {
const device = info.env.KIOSK_DEVICES?.split(',')[info.workerSlot];
info.log(`slot ${info.workerSlot}: ${device}`);
},
```
`init` never learns the app URL; URLs arrive through `session.open` only. A
failure is an infrastructure error.
```ts
/** Run identity and harness-resolved facts handed to `init`, once per worker before the first step. */
export interface EngineInitInfo {
/** The run id. */
readonly runId: string;
/** The target this worker serves. */
readonly targetName: string;
/**
* Directory relative paths in the config resolve against: the config file's
* directory, or the run's `cwd` for a programmatic config. An engine option
* naming a file (a build to install) resolves here, never against
* `process.cwd()`, which an in-process run does not change.
*/
readonly projectRoot: string;
/** The app the target declares: the origin and site policy of its URL, and what a device launches. */
readonly app: EngineAppInfo;
/**
* This worker's environment: the run's, plus what this target's `prepare`
* returned in its result's `env`. Read what `prepare` provisioned from
* here, never from `process.env`: an in-process worker shares its process
* with the runner and every other target.
*/
readonly env: Readonly<Record<string, string | undefined>>;
/** Whether the run asked for a visible surface (`--headed`). */
readonly headed: boolean;
/**
* This worker's 0-based slot among the target's workers: the lowest slot
* free when the worker was spawned, so a replacement worker takes over the
* slot of the one that exited. An engine with several devices hands each
* slot its own; `config.workers` bounds the slots a target can reach.
*/
readonly workerSlot: number;
/** Aborts on interrupt and when init exceeds the launch timeout; init must stop promptly. */
readonly signal: AbortSignal;
/**
* Reports one line of progress. The runner streams it as a `notice` run
* event, prefixed with the target and worker slot, so it reaches the
* reporter and every host sink instead of being written to a worker's
* stderr underneath the live status block. This is where a fact that only
* exists once the worker is up belongs: the URL a hosted browser or device
* can be watched at, which simulator a slot got, a version it detected.
* Meant for `init` itself; a line logged later is still delivered while
* the worker lives and dropped once it has exited. The line is shown as is
* in reporters and host sinks, so keep tokens and other secrets out of it
* (a viewer URL without its credential).
*/
readonly log: (line: string) => void;
}
```
#### `startAttempt`
Runs before each attempt, to set up the per-test isolation that
[`endAttempt`](#endattempt) tears down.
```ts
async startAttempt(context) {
const password = await context.resolveSecret(options.password);
await client.openContext({ password, signal: context.signal });
context.environment({ device: 'Kiosk 2', os: 'KioskOS 4.1' });
},
```
On failure or timeout, `context.signal` aborts and `endAttempt` runs.
<Warning>
A value from `resolveSecret` is protected as text only. Reports, logs, and
observations redact it and the attempt's text downloads are rewritten, but
a page that shows it outside a secure field is not masked in screenshots.
</Warning>
```ts
/**
* Per-attempt isolation context. An engine that must give each test a fresh
* surface (a fresh isolated context per test, a reset device) sets it up in
* `startAttempt` and tears it down in `endAttempt`.
*/
export interface EngineAttemptContext {
/** The attempt id. */
readonly attemptId: string;
/** Absolute directory every artifact of this attempt is written under. */
readonly artifactsDir: string;
/**
* The plaintext of a secret the engine declared in `secrets`, for an
* option the engine hands the app itself (basic-auth credentials). A
* provider runs again on every call. The value, and every value
* `options.derived` computes from it, join the attempt's redaction before
* this resolves, so reports, logs, and every observation redact them, and
* the attempt's text downloads are rewritten. The protection is
* text only: a page that shows the value outside a secure field is not
* masked in screenshots.
* A secret the engine did not declare is `SECRET_UNAVAILABLE`.
*/
readonly resolveSecret: (secret: Secret, options?: ResolveSecretOptions) => Promise<string>;
/**
* Aborts with the attempt, and the moment `startAttempt` fails or exceeds
* the launch timeout: a hook still running then must stop, because the
* harness ends the attempt's isolation right behind it and may retry.
*/
readonly signal: AbortSignal;
/**
* Reports something the app did on its own while the attempt ran: a
* console line, an uncaught exception, a request that failed. The harness
* redacts the text, keeps it with the step that was running (from an event
* callback outside any step, the step that started last; before the first
* step, with none), and shows it on the trace page. Call it from the moment `startAttempt` begins until
* `endAttempt`; entries past a per-attempt cap are dropped.
*/
readonly appLog: (entry: AppLogEntry) => void;
/**
* Reports where the app went on its own, as one line (`navigated to
* /login`, `the app opened a new tab at /help`): a navigation the engine
* did not start, a tab, a frame. Shown under the step that was running.
*/
readonly navigation: (line: string) => void;
/**
* Hands the harness a screen for the trace: the capture a `locate`
* matched against, or one read right after an action. A read after an
* action is on the action's path, so bound it and skip it where it is
* expensive; it never fails the action. The trace page shows how each
* step's screen changed. Refs in it are not valid for `perform`. Absent
* when the attempt keeps no trace.
*/
readonly screen?: (snapshot: EngineSnapshot) => void;
/**
* What the attempt runs on, such as `{ browser: 'chromium 141.0' }` or
* `{ device: 'iPhone 16', os: 'iOS 26.0' }`, for the trace page. Later
* calls add to and overwrite earlier facts.
*/
readonly environment: (facts: Readonly<Record<string, string>>) => void;
}
/** One line of what the app did during an attempt, as `EngineAttemptContext.appLog` takes it. */
export interface AppLogEntry {
/**
* Where it came from: the app's `console`, an `error` nothing caught, the
* `network`, or the `system` the app runs on (a crash, an OS log).
*/
readonly source: 'console' | 'error' | 'network' | 'system';
readonly level: 'error' | 'warning' | 'info';
/** One line, such as `GET /api/todos 500` or `TypeError: x is undefined`; the harness clips it. */
readonly text: string;
}
/** How `EngineAttemptContext.resolveSecret` treats what the engine makes of the value. */
export interface ResolveSecretOptions {
/**
* The forms of the value the app sees instead of the value itself, for the
* same redaction under the secret's name: the base64 `user:password` an
* `Authorization: Basic` header carries, which a page echoing its request
* headers would otherwise show.
*/
readonly derived?: (plaintext: string) => readonly string[] | Promise<readonly string[]>;
}
```
#### `settleAttempt`
Runs after the body has settled, and again after teardown, before the
verdict. It reports a failure on a path no step awaited, such as a request
interceptor or a native dialog handler.
```ts
async settleAttempt() {
const running = [...dialogHandlers];
dialogHandlers.clear();
await Promise.all(running); // rethrows a handler's failure
},
```
#### `endAttempt`
Runs after each attempt and tears down what `startAttempt` set up. A failure
marks cleanup failed; the test keeps its status.
```ts
async endAttempt(context) {
await client.closeContext({ signal: context.signal });
},
```
#### `dispose`
Runs one time per worker, including when `init` never ran.
Missing or aborted `context.reuseSignal` requires final cleanup. A live
signal permits keeping run-owned resources for a replacement worker.
An engine that keeps resources must observe revocation until its hook settles.
Final shutdown, a forced interrupt, or parent disconnect revokes that permission.
The cleanup budget still uses `context.signal` and `context.timeoutMs`.
Cleanup is best effort within that budget. The runner does not guarantee
cleanup after a worker exits or after disposal settles with reuse permission.
```ts
async dispose(context) {
await client.shutdown({ signal: context.signal });
},
```
```ts
/**
* Budget of one cleanup hook (`settleAttempt`, `endAttempt`, `dispose`). `signal` aborts when
* the budget is exhausted, so a hook that cannot finish in time stops instead
* of running on into the next attempt's setup.
*/
export interface EngineCleanupContext {
/** Aborts when the cleanup budget is spent. */
readonly signal: AbortSignal;
/** Remaining cleanup budget when the call starts. */
readonly timeoutMs: number;
}
/** Cleanup of one worker, with optional permission to leave run-owned resources for a replacement. */
export interface EngineDisposeContext extends EngineCleanupContext {
/**
* A live signal permits reusable resources owned by `prepare` and `finish`
* to survive this worker. Missing or aborted permission requires final
* cleanup. Observe revocation until disposal settles. This signal does
* not cancel cleanup, which remains bounded by `signal` and `timeoutMs`.
*/
readonly reuseSignal?: AbortSignal;
}
```
### Observation
`observe` returns one fresh semantic snapshot of the surface under test.
| Field | Type |
| --- | --- |
| `location` | `string` |
| `root` | `SemanticNode` |
| `viewport` | `ViewportSize` |
| [`truncated`](#truncated-trees) | `boolean` |
| `keyboardVisible` | `boolean` |
| `tabs` | `{ count, active }` |
| [`treeUnavailable`](#pixel-fallback) | `true` |
| `pixels` | `ObservationPixels` |
| `maskedRegionCount` | `number` |
`tabs` reports the open tabs when there is more than one: `count`, and the
zero-based place of the tab `root` reads, in the order the tabs opened. The
harness names the tab in every screen header and tells the model when a
tab opened or the screen moved to another tab. Omit it with one tab.
Return the surface as it is right now. The runner mints the revision and
stamps it onto every ref. When `pixels` are present, it checks that
`maskedRegionCount` covers every `secure` node. A valid empty tree may have a
childless root and include `pixels`.
```ts
/**
* One fresh semantic snapshot of the surface under test. The harness owns
* everything downstream: revision minting, secret redaction, and the
* observation byte budget apply to every engine equally.
*/
export interface EngineSnapshot {
/**
* Where the surface is, as an opaque address the platform understands: a
* URL on a document platform, the foreground screen or activity on a
* device, the front window on a desktop. The harness shows it to the model
* and to the report, anchors trace replay on it, and applies the origin
* policy to it only when it parses as a URL. Omit when the platform has no
* notion of a current location.
*/
readonly location?: string;
/**
* The observation tree under one root the engine minted. The root's id MUST
* be stable for the surface across observations: `perform(root, swipe)` is
* the viewport swipe, and the harness addresses it from an earlier
* observation, possibly one taken before a `restart` or `reset`, so the
* root swipe MUST NOT depend on a live handle from that observation. A
* platform with several top-level elements (a device's windows) wraps them
* in a root of its own.
*/
readonly root: SemanticNode;
/** The viewport `SemanticNode.rect` and `ViewportPoint` are measured in, in CSS pixels. */
readonly viewport: ViewportSize;
/**
* True when `root` leaves out nodes that are on the surface: the engine
* stopped reading at a node cap or a depth limit, or could not enter a
* child document. The harness tells the model the listing is incomplete
* and never calls such a screen unchanged. Omit or set false for a tree
* read whole.
*/
readonly truncated?: boolean;
/**
* Whether an on-screen keyboard is showing, as the engine measured it apart
* from `root`. Set `false` when it measured none, and omit it only when the
* engine did not measure it.
*/
readonly keyboardVisible?: boolean;
/**
* The tabs the surface has open, when it has more than one, such as a
* browser with a tab the app opened. Omit with one tab.
*/
readonly tabs?: { readonly count: number; readonly active: number };
/** Semantic capture failed; only fresh, independently masked pixels are evidence. */
readonly treeUnavailable?: true;
/** Masked pixels, when requested and producible; omitted otherwise. */
readonly pixels?: ObservationPixels;
/** Regions masked in `pixels`; the harness checks it covers every secure node. */
readonly maskedRegionCount?: number;
}
export interface EngineObserveOptions {
/** Requests masked viewport pixels for the same revision as the tree. */
readonly pixels?: boolean;
/** Permits independently masked pixels to replace a failed semantic capture. */
readonly pixelFallback?: boolean;
/** The pixels are compared against a stored screenshot: hold system chrome (a status bar's clock and battery) in a fixed state where the platform can. */
readonly comparable?: boolean;
}
```
#### Truncated trees
Set `truncated` when a node limit or an inaccessible subtree leaves content
out. The `truncated` JSDoc above says how the runner treats it.
#### Pixel fallback
When semantic capture times out, an engine may return masked pixels in place
of the tree, but only when `pixelFallback` is true and it can prove
screenshot masking without that capture.
```ts
return {
treeUnavailable: true,
root: { ref: rootRef },
viewport,
pixels,
};
```
- Return `treeUnavailable: true`, fresh `pixels`, and a `root` containing
only its stable `ref`. That ref supports viewport actions; it supplies no
semantic node evidence. Retire earlier node refs.
- The flag tells missing semantic evidence from a valid empty tree. Never
infer absence from it.
- Without permission, pixels, or independent masking proof, propagate the
capture failure. Cancellation, policy denial, and hard engine failures
propagate too.
- The runner permits fallback for `act` and for judgments with
`vision: true` or `'only'`. It includes the
screenshot even if the initial observation requested only a tree.
- A step that loses semantic evidence cannot record or finish by replaying a
trace.
- Engines whose masks depend on their semantic capture cannot provide this
fallback.
`EngineSnapshot` is an extendable interface, so TypeScript can't enforce
these rules. The runner checks the shape at the engine boundary, at runtime:
| Throws | When |
| --- | --- |
| `UNSUPPORTED_CAPABILITY` | A fallback without permitted pixels. |
| `INVALID_STATE` | A fallback whose root carries semantic fields. |
### Actions
`perform` carries out one `LocatorAction` on a node; `performAt` carries out
one `PointerAction` at a viewport point. An engine declares the kinds it
honors in `actions` and `pointerActions`.
| Kind | Fields | `actions` | `pointerActions` |
| --- | --- | --- | --- |
| `tap` | `modifiers?` on a node | yes | yes |
| `doubleTap` | `modifiers?` on a node | yes | yes |
| `secondaryTap` | `modifiers?` on a node | yes | yes |
| `longPress` | `durationMs?` | yes | yes |
| `fill` | `value`, `sensitive` | yes | - |
| `clear` | - | yes | - |
| `press` | `key` | yes | - |
| `check` | - | yes | - |
| `uncheck` | - | yes | - |
| `focus` | - | yes | - |
| `hover` | - | yes | yes |
| `scrollIntoView` | - | yes | - |
| `selectOption` | `value` | yes | - |
| `setInputFiles` | `paths`; empty clears the input | yes | - |
| `dragTo` | `target`: a `NodeRef` or a `ViewportPoint` | yes | yes |
| `swipe` | `direction`, `momentum?` | yes | yes |
| `swipeTo` | `target`, `durationMs?` | - | yes |
The grammar is closed. Anything beyond it is an agent-side `defineTool`.
#### Locator actions
```ts
async perform(ref, action, context) {
switch (action.kind) {
case 'tap':
return client.tap(ref.id, { signal: context.signal });
case 'swipe':
return client.scroll(ref.id === rootId ? undefined : ref.id, action.direction);
default:
throw new EngineError('UNSUPPORTED_CAPABILITY', `kiosk cannot ${action.kind}`, { retryable: false });
}
},
```
```ts
/**
* One deterministic action `perform` carries out on a node. The `swipe`
* kind on the observation's root node is the viewport swipe: the agent's
* `scroll` verb and `screen.swipe()` both arrive as `perform(root, swipe)`,
* so a surface implements scrolling once.
*
* `press` takes one key in the `Key` grammar below.
*/
export type LocatorAction =
| {
readonly kind: 'tap' | 'doubleTap' | 'secondaryTap';
/** Keys held for the click; only an engine that declares `tapModifiers` receives it. */
readonly modifiers?: readonly KeyModifier[];
}
| {
readonly kind:
| 'check'
| 'uncheck'
| 'clear'
| 'focus'
| 'hover'
| 'scrollIntoView';
}
| { readonly kind: 'longPress'; readonly durationMs?: number }
| { readonly kind: 'fill'; readonly value: string; readonly sensitive: boolean }
| { readonly kind: 'press'; readonly key: Key }
| { readonly kind: 'selectOption'; readonly value: SelectOption }
| { readonly kind: 'setInputFiles'; readonly paths: readonly string[] }
| { readonly kind: 'dragTo'; readonly target: NodeRef }
| {
readonly kind: 'swipe';
readonly direction: ScrollDirection;
readonly momentum?: Momentum;
};
/** Every action kind; what an engine declares in `actions`. */
export type LocatorActionKind = LocatorAction['kind'];
/** The closed list of action kinds, in contract order; the type is derived from it. */
export const LOCATOR_ACTION_KINDS = [
'tap',
'doubleTap',
'secondaryTap',
'longPress',
'fill',
'clear',
'press',
'check',
'uncheck',
'focus',
'hover',
'scrollIntoView',
'selectOption',
'setInputFiles',
'dragTo',
'swipe',
] as const satisfies readonly LocatorActionKind[];
```
#### Pointer actions
Pointer actions use viewport coordinates when no semantic node is available.
- `dragTo` ends at another point.
- `swipe` starts a scroll gesture at the given point.
- `swipeTo` is the one kind with no node form: the finger's path from the
point to `target`, a touch swipe on a device and a pointer drag on a
document platform. It differs from `dragTo`, where a platform holds before
it drags.
```ts
export type PointerAction =
| { readonly kind: 'tap' | 'doubleTap' | 'secondaryTap' | 'hover' }
| { readonly kind: 'longPress'; readonly durationMs?: number }
| { readonly kind: 'dragTo'; readonly target: ViewportPoint }
| {
readonly kind: 'swipe';
readonly direction: ScrollDirection;
readonly momentum?: Momentum;
}
| {
readonly kind: 'swipeTo';
readonly target: ViewportPoint;
readonly durationMs?: number;
};
/** Every pointer action kind; what an engine declares in `pointerActions`. */
export type PointerActionKind = PointerAction['kind'];
/** The closed list of pointer action kinds, in contract order; the type is derived from it. */
export const POINTER_ACTION_KINDS = [
'tap',
'doubleTap',
'secondaryTap',
'longPress',
'hover',
'dragTo',
'swipe',
'swipeTo',
] as const satisfies readonly PointerActionKind[];
```
#### Keys
The key grammar of `press`, shared by every engine, and its parser.
```ts
parseKey('Control+a'); // { modifiers: ['Control'], key: { kind: 'char', char: 'a' } }
parseKey('Shift+Tab'); // { modifiers: ['Shift'], key: { kind: 'named', name: 'Tab' } }
parseKey('Ctrl+a'); // undefined: Ctrl is not a modifier of the grammar
```
```ts
/**
* The key grammar of `press`, shared by every engine so a test's key names are
* portable: zero or more modifiers and one key, joined by `+`.
*
* - Modifiers: `Shift`, `Control`, `Alt`, `Meta`, `ControlOrMeta` (Control on
* Windows and Linux, Meta on macOS).
* - Named keys: those in `KEY_NAMES` (`Enter`, `Escape`, `Tab`, `Backspace`,
* `Delete`, `ArrowUp`..., `Home`, `End`, `PageUp`, `PageDown`, `Insert`,
* `Space`, `F1`..`F12`).
* - Any single printable character (`a`, `A`, `1`, `$`), typed as itself.
*
* `Control+a`, `Shift+Tab`, `Enter`, `$`. A platform without a key maps it
* or throws `UNSUPPORTED_CAPABILITY`; it never reinterprets the spelling.
* `parseKey` is the one parser, exported so no engine carries its own.
*/
export type Key = string;
export const KEY_MODIFIERS = ['Shift', 'Control', 'Alt', 'Meta', 'ControlOrMeta'] as const;
export const KEY_NAMES = [
'Enter',
'Escape',
'Tab',
'Backspace',
'Delete',
'Insert',
'Space',
'ArrowUp',
'ArrowDown',
'ArrowLeft',
'ArrowRight',
'Home',
'End',
'PageUp',
'PageDown',
'F1',
'F2',
'F3',
'F4',
'F5',
'F6',
'F7',
'F8',
'F9',
'F10',
'F11',
'F12',
] as const;
/** One parsed `press` key: its modifiers and either a named key or a single character. */
export interface ParsedKey {
/** Modifiers, in the order written. */
readonly modifiers: readonly KeyModifier[];
/** The key itself. */
readonly key: { readonly kind: 'named'; readonly name: KeyName } | { readonly kind: 'char'; readonly char: string };
}
export function parseKey(key: string): ParsedKey | undefined;
```
### App
The app under test is declared on the target (`targets: [{ engine, app }]`,
see [the app under test](/reference/config#the-app-under-test)); an engine
only drives it. The harness checks the shape of every field. Then it hands
the declaration to [`validateApp`](#validateapp) at config load.
| Target field | Reaches the engine as |
| --- | --- |
| `url` | `validateApp` only. The harness opens it through `session.open`. |
| the origin of `url`, on the run's port | `info.app.origin` in `prepare` and `init` |
| the site of `url` | `info.app.site` in `prepare` and `init` |
| `bundleId`, `appPath`, `launchArguments`, `permissions` | `validateApp`, and `info.app` in `prepare` and `init`, as the target declared them |
| `command`, `readyUrl` | Never. The runner starts the app. |
`CommandConfig` is exported from `e2e`, not `e2e/engine`.
#### `validateApp`
Checks the declared app at config load, synchronously.
```ts
validateApp(app, { targetName }) {
if (app.url === undefined) {
throw new ConfigurationError('INVALID_CONFIG', `target "${targetName}" needs app.url`);
}
},
```
- It runs for every target the engine serves, with `{}` for a target that
declares no `app`.
- `@e2e-dev/web` requires `url`; `@e2e-dev/mobile` requires `bundleId` or
`appPath` and refuses `url`.
| Throws | When |
| --- | --- |
| `INVALID_CONFIG` | A field the platform needs is missing, or the engine cannot drive one. Name `info.targetName` in the message. |
```ts
/**
* The app a target declares, as its engine reads it at config load: what it
* is and where it is served. `EngineAppInfo` (below) is what `prepare` and
* `init` receive; the declaration adds the URL, which a surface never gets.
*/
export type EngineAppDeclaration = EngineAppInfo & {
/** Base URL the target opens, as declared; see `TargetApp.url`. */
readonly url?: string;
};
/** Handed to `Engine.validateApp` beside the declaration. */
export interface EngineAppCheckInfo {
/** The target declaring the app, for the message of a refusal. */
readonly targetName: string;
}
/** A permission's state when the app launches: held, refused, or not asked for yet, so the OS asks again. */
export type AppPermissionState = 'grant' | 'deny' | 'reset';
```
#### `EngineAppInfo`
The resolved app, as `EnginePrepareInfo.app` and `EngineInitInfo.app`: the
origin and site of the URL, and the device fields as the target declared
them. Scope a credential the engine injects to the exact origin. Apply the
site to observed frames with [`sameSite`](#site-helpers).
```ts
if (new URL(request.url).origin === info.app.origin) request.headers.set('x-api-key', key);
```
```ts
/**
* The app the harness resolved from the target's declaration, handed to
* `prepare` and `init`: the origin and site policy for the decisions only a
* surface can make (which requests carry injected credentials, which nested
* documents enter an observation), and what a device launches.
*/
export interface EngineAppInfo {
/**
* The app URL's origin on the run's port (`scheme://host:port`, as
* `URL.origin` serializes it): where injected request credentials go.
* Absent when the app has no URL, and for a free port the run has not
* assigned (`e2e doctor`).
*/
readonly origin?: string;
/**
* The app URL's registrable domain, as read by `siteOf`. Use `sameSite`
* to scope observed child frames. Absent when the app has no URL. The
* runner passes navigation URLs separately through `session.open`.
*/
readonly site?: string;
/** The installed app to launch, as the target declared it. */
readonly bundleId?: string;
/** The build to install, as the target declared it; relative to `projectRoot`. */
readonly appPath?: string;
/** Arguments every fresh launch passes the app. */
readonly launchArguments?: readonly string[];
/** Permissions the app holds, by name; what each state means is the engine's (a device sets them before every fresh launch). */
readonly permissions?: Readonly<Record<string, AppPermissionState>>;
}
```
### Session
Steering hooks behind the universal `app` fixture and the agent's `navigate`,
`back`, `resize_viewport`, `switch_tab`, and `close_tab` verbs.
| Hook | Behind |
| --- | --- |
| `open(url)` | `app.open()`, the agent's `navigate` verb, the reopen after `restart` and `reset` |
| `openLink(url)` | `app.open(path)` on a target with `app.linkBase`: the joined link, its scheme checked, opened in the app under test. A target can declare `linkBase` only when its engine has this hook. |
| `back()` | `app.back()`, the agent's `back` verb |
| `resize(size)` | the agent's `resize_viewport` verb; `size` is `{ width, height }` in CSS pixels and holds for the rest of the attempt |
| `switchTab(index)` | the agent's `switch_tab` verb; `index` is zero-based in the order the tabs opened, as [`tabs`](#observation) counts them. An index with no open tab is `ACTION_FAILED`. |
| `closeTab(index)` | the agent's `close_tab` verb; `index` counts as for `switchTab`. Closing the tab shown shows another, such as the tab that opened it. An index with no open tab, or the last open tab, is `ACTION_FAILED`. |
| `restart()` | `app.restart()` |
| `reset()` | `app.clearState()` |
```ts
session: {
async back(context) { await client.back({ signal: context.signal }); },
async restart(context) { await client.relaunch(bundleId, { signal: context.signal }); },
},
```
```ts
/**
* Steering hooks behind the universal `app` fixture and the agent's
* `navigate` verb. The app itself is the target's `app`, handed over as data;
* node actions never live here, they are `perform`.
*
* `restart` and `reset` open nothing on an addressable surface: they end at
* a fresh surface with no location shown, and the harness reopens the app
* through `open` when the target has an address. A surface therefore never
* needs to know the app's URL, and an unreachable app after a restart is
* reported the same way as on first open. A surface without `open` (a
* device with a pinned app) relaunches the app inside `restart` and `reset`
* itself, since nothing else will.
*/
export interface EngineSession {
/**
* Opens one URL the harness already resolved against the base URL and the
* origin policy. Absent on a surface without addressable locations.
*/
open?(url: string, context: OperationContext): Promise<void>;
/**
* Opens a link in the app under test, as a deep link or a universal link
* would: `app.open(path)` on a target that declares `app.linkBase` lands
* here with the link joined and its scheme checked. A device engine's hook;
* a target whose engine lacks it cannot declare `linkBase`.
*/
openLink?(url: string, context: OperationContext): Promise<void>;
/** Navigates back once in the surface's history. */
back?(context: OperationContext): Promise<void>;
/** Sets the viewport to `size` in CSS pixels, for this page and every page the attempt opens after it. */
resize?(size: ViewportSize, context: OperationContext): Promise<void>;
/**
* Shows the open tab at `index`, zero-based in the order the tabs opened,
* as `EngineSnapshot.tabs` counts them: every observation and action after
* it reads and acts on that tab. An index with no open tab is an
* `ACTION_FAILED` the model reads. Declare it with `tabs` in snapshots.
*/
switchTab?(index: number, context: OperationContext): Promise<void>;
/**
* Closes the open tab at `index`, counted as `switchTab` counts them. When
* it was the tab shown, the engine shows another, such as the tab that
* opened it. An index with no open tab, or the last open tab, is an
* `ACTION_FAILED` the model reads.
*/
closeTab?(index: number, context: OperationContext): Promise<void>;
/** Recreates the execution context, keeping persisted state. */
restart?(context: OperationContext): Promise<void>;
/** Clears persisted client state and recreates the execution context. */
reset?(context: OperationContext): Promise<void>;
}
```
### Errors
An engine throws `EngineError` with one code of a closed set. The runner maps
each code to the error a test sees.
| Code | Runner mapping | Retryable |
| --- | --- | --- |
| `NODE_STALE` | re-resolve, then `LOCATOR_NOT_FOUND` | yes |
| `FRAME_NOT_FOUND` | re-resolve, then `LOCATOR_NOT_FOUND` | yes |
| `FRAME_AMBIGUOUS` | `LOCATOR_AMBIGUOUS` | no |
| `NOT_ACTIONABLE`, `ACTION_MAY_HAVE_COMMITTED`, `OPERATION_TIMEOUT` | `ACTION_FAILED` | no |
| `CANCELLED` | infrastructure `CANCELLED` | no |
| `UNSUPPORTED_CAPABILITY` | configuration `UNSUPPORTED_CAPABILITY` | no |
| `INVALID_STATE` | `APP_NOT_OPEN` | no |
| `ENGINE_FAILURE`, any non-`EngineError` | infrastructure `ENGINE_FAILURE` | no |
#### `EngineError`
```ts
throw new EngineError('NODE_STALE', `node ${ref.id} is gone`, { retryable: true });
```
```ts
export class EngineError extends Error {
constructor(code: EngineErrorCode, message: string, options: { retryable: boolean; cause?: unknown });
/** One of `ENGINE_ERROR_CODES`. */
readonly code: EngineErrorCode;
/** Whether the runner may re-resolve and retry; only `RETRYABLE_ENGINE_ERROR_CODES` can be. */
readonly retryable: boolean;
}
/** The closed engine error code set; the type is derived from it, so the two cannot drift. */
export const ENGINE_ERROR_CODES = [
'NODE_STALE',
'FRAME_NOT_FOUND',
'FRAME_AMBIGUOUS',
'NOT_ACTIONABLE',
'ACTION_MAY_HAVE_COMMITTED',
'OPERATION_TIMEOUT',
'CANCELLED',
'UNSUPPORTED_CAPABILITY',
'INVALID_STATE',
'ENGINE_FAILURE',
] as const;
/** One of `ENGINE_ERROR_CODES`. */
export type EngineErrorCode = (typeof ENGINE_ERROR_CODES)[number];
/**
* The only codes that may be retryable: both describe a repeatable read. Any
* other retryable claim, from this module's class or a foreign copy of it, is
* coerced to a non-retryable `ENGINE_FAILURE`.
*/
export const RETRYABLE_ENGINE_ERROR_CODES: ReadonlySet<EngineErrorCode> = new Set([
'NODE_STALE',
'FRAME_NOT_FOUND',
]);
```
#### Timeouts at a deadline
An `OPERATION_TIMEOUT` on a locate or read that started with under 100 ms, or
under its slowest completed locate or read, of a waiting call's deadline left,
after one that completed, ends the wait
instead:
| Waiting call | Fails with |
| --- | --- |
| an action, `waitFor` | `LOCATOR_NOT_FOUND`, with the timeout as its cause |
| an assertion | `ASSERTION_FAILED`, with the timeout as its cause |
A negated assertion passes there if its negation had held long enough by the
time the read was cut off. [`pollCondition`](#pollcondition) polls by these
rules.
#### Runner error classes
An `EngineError` a contributed fixture method throws is mapped by the table
above, so a fixture over a session that ended reports `APP_NOT_OPEN` the way
an observation would. A method can also fail a step with the runner's own
error classes, exported from the same entry.
```ts
throw new TestError('ASSERTION_FAILED', 'expected the receipt to print');
```
`E2EError` itself is not exported; construct one of the three.
```ts
/** The runner's error base: `category` sets the exit code, `code` names the cause. */
declare class E2EError extends Error {
readonly category: 'test' | 'configuration' | 'infrastructure' | 'internal' | 'interrupted';
readonly code: string;
readonly retryable: boolean;
}
/** A test failure: ASSERTION_FAILED, INVALID_LOCATOR, ACTION_FAILED, ... */
export class TestError extends E2EError {
constructor(code: string, message: string, options?: { retryable?: boolean; cause?: unknown });
}
/** POLICY_DENIED, APP_URL_REQUIRED, UNSUPPORTED_CAPABILITY, ... */
export class ConfigurationError extends E2EError {
constructor(code: string, message: string, options?: { cause?: unknown });
}
/** The surface itself failed: BROWSER_INSTALL_FAILED, ... */
export class InfrastructureError extends E2EError {
constructor(code: string, message: string, options?: { cause?: unknown });
}
```
### Shared semantics
Helpers that keep behavior consistent across engines. Every engine
reproduces these semantics; import them instead of carrying a copy.
| Export | Returns |
| --- | --- |
| [`matchesText`, `toTextPattern`, `describePattern`](#text-patterns) | `boolean`, `TextPattern`, `string` |
| [`resolveExpression`](#resolveexpression) | `SemanticNode[]` |
| [`urlMatches`](#urlmatches) | `boolean` |
| [`siteOf`, `sameSite`](#site-helpers) | `string`, `boolean` |
| [`isRefusedLinkScheme`](#isrefusedlinkscheme) | `boolean` |
| [`pollCondition`, `Deadline`](#pollcondition) | `Promise<void>`, `Deadline` |
| [`validateJsonValue`](#validatejsonvalue) | `void` |
| [`rejectUnknownKeys`](#rejectunknownkeys) | `void` |
| [`rejectUnknownOptions`](#rejectunknownoptions) | `void` |
| [`obj`](#obj) | `WithoutUndefined<T>` |
| [`raceAbort`, `withTimeout`, `withinCleanupBudget`](#timing-helpers) | `Promise<T>`, `Promise<T>`, `Promise<void>` |
| [`isSecret`](#issecret) | `boolean` |
| [`isProviderRecording`, `stopProviderRecording`](#provider-recordings) | `boolean`, `Promise<VideoSegment>` |
#### Text patterns
`TextPattern` matching for `locate` and text matchers.
```ts
export function toTextPattern(match: TextMatch, options?: { exact?: boolean }): TextPattern;
export function matchesText(actual: string, pattern: TextPattern): boolean;
export function describePattern(pattern: TextPattern): string;
```
```ts
const pattern = toTextPattern(/total/i);
matchesText('Order total', pattern); // true
describePattern(pattern); // '/total/i'
```
| Throws | When |
| --- | --- |
| `INVALID_ARGUMENT` | `toTextPattern` gets a `match` that is neither a string nor a `RegExp`. |
#### `resolveExpression`
The reference `locate`. Resolves one `LocatorExpression` over a semantic
tree, in document order.
```ts
export function resolveExpression(
expression: LocatorExpression,
nodes: readonly SemanticNode[],
options?: ResolveExpressionOptions,
): SemanticNode[];
```
```ts
async locate(expression) {
const screen = await client.snapshot();
return resolveExpression(expression, screen.nodes);
},
```
- `text` and `label` answer with the innermost match when a container echoes
a descendant's text.
- An engine whose tree is the whole truth (a device) calls it; a browser
engine reproduces it.
- A `selector` expression needs the `selector` option.
- A `frame` expression is `FRAME_NOT_FOUND`.
- A `union` expression matches either branch in document order, without duplicate nodes.
#### `urlMatches`
`toHaveURL`-style comparison of a current URL against an expected one.
```ts
export function urlMatches(
current: string,
expected: string | RegExp,
baseHref: string,
ignoreCase?: boolean,
): boolean;
```
```ts
urlMatches('https://shop.test/cart', '/cart', 'https://shop.test/'); // true
```
`ignoreCase` works as on the locator matchers.
| Throws | When |
| --- | --- |
| `INVALID_ARGUMENT` | `expected` is neither a string nor a `RegExp`, as in `toTextPattern`. |
#### Site helpers
`siteOf` reads a hostname's registrable domain, the form
`EngineAppInfo.site` takes. `sameSite` tells whether a URL is on a site.
```ts
export function siteOf(hostname: string): string;
export function sameSite(url: string | URL, site: string): boolean;
```
#### `isRefusedLinkScheme`
Whether a link's scheme is one no app link may take: `file:`, `data:`,
`javascript:`, `view-source:`, `blob:`, or `filesystem:`. An engine that
opens app links (`session.openLink`) refuses these with `POLICY_DENIED`, as
`app.open` does with `app.linkBase`.
```ts
export function isRefusedLinkScheme(protocol: string): boolean;
```
```ts
if (isRefusedLinkScheme(new URL(link).protocol)) throw new ConfigurationError('POLICY_DENIED', 'refused link scheme');
```
#### `pollCondition`
Assertion polling and the negation grace window.
```ts
export function pollCondition(options: PollConditionOptions): Promise<void>;
```
```ts
const deadline = new Deadline(context.timeouts.assertion);
await pollCondition({
deadline,
signal: context.signal,
negated: false,
evaluate: async () => (await read(context.operation(deadline.remaining()))) === 'Paid',
onTimeout: (cause) => new TestError('ASSERTION_FAILED', 'status never became Paid', { cause }),
});
```
- Bound each read `evaluate` makes by `deadline`
(`context.operation(deadline.remaining())`). A read on the action timeout
can outlast the assertion's own.
- An `OPERATION_TIMEOUT` from an `evaluate` ends the poll under the rules in
[Timeouts at a deadline](#timeouts-at-a-deadline). `onTimeout(cause)`
builds the failure with it.
#### `validateJsonValue`
Checks that a value is plain JSON.
```ts
export function validateJsonValue(value: unknown, label: string, rules?: JsonValueRules): void;
```
```ts
validateJsonValue(result, 'kiosk.evaluate result', { maxDepth: 32 });
```
- A top-level `undefined` passes.
- `rules.onSecret` replaces the `POLICY_DENIED` a `Secret` raises with an
error of its own; `rules.maxDepth` caps the nesting.
| Throws | When |
| --- | --- |
| `INVALID_ARGUMENT` | A non-finite number, a cycle, an object that is not a plain object or array, or a function, symbol, bigint, or `undefined` inside. The message names `label`. |
| `POLICY_DENIED` | A `Secret`, unless `rules.onSecret` throws its own. |
#### `rejectUnknownKeys`
Throws for the first own key of `value` outside `keys`: the check the runner
runs on its own config objects, for an engine factory's options.
```ts
export function rejectUnknownKeys(
label: string,
value: object,
keys: readonly string[],
code?: 'INVALID_CONFIG' | 'INVALID_ARGUMENT', // default: 'INVALID_CONFIG'
): void;
```
```ts
rejectUnknownKeys('web()', options, ['browser', 'viewport']);
// web() has unknown key "viewprt"; did you mean "viewport"?
```
- The message names the nearest known key when one is a plausible typo, and
the known keys otherwise.
- Pass `'INVALID_ARGUMENT'` as `code` for an object a test passes to a
fixture method.
#### `rejectUnknownOptions`
Refuses an option key a fixture method or matcher does not take, so a
JavaScript caller's unsupported option fails instead of running on the
default.
```ts
export function rejectUnknownOptions(
api: string,
options: object | undefined,
known: readonly string[],
code?: 'INVALID_ARGUMENT' | 'INVALID_LOCATOR', // default: 'INVALID_ARGUMENT'
): void;
```
```ts
rejectUnknownOptions('kiosk.print', options, ['copies']);
```
- Pass `INVALID_LOCATOR` as `code` for a locator option.
- `options` must be a plain object or `undefined`; a class instance or an
array is refused.
- Non-enumerable own keys count as keys.
#### `obj`
Drops `undefined`-valued keys, so a result built from optional inputs
satisfies exact optional property types.
```ts
obj({ a: 1, b: undefined, c: 3 }); // { a: 1, c: 3 }
```
#### Timing helpers
```ts
export function raceAbort<T>(work: Promise<T> | (() => Promise<T>), signal: AbortSignal, label: string): Promise<T>;
export function withTimeout<T>(promise: Promise<T>, timeoutMs: number, onTimeout: () => Error): Promise<T>;
export function stepTimeout(timeout: unknown, api: string): number | undefined;
export function withinCleanupBudget(
promise: Promise<unknown>,
budget: { readonly signal: AbortSignal; readonly timeoutMs: number },
): Promise<void>;
```
```ts
await raceAbort(() => client.tap(ref.id), context.signal, 'kiosk tap');
const operation = context.operation(stepTimeout(options?.timeout, 'kiosk.print'));
```
- `raceAbort` abandons the wait when `signal` aborts. Pass a thunk to check
cancellation before dispatch.
- `withTimeout` races a promise against a timeout and throws what
`onTimeout` builds.
- `stepTimeout` turns a call's own `timeout` option into a budget for
`context.operation`: `0` waits until the test timeout, or `cleanupTimeout` in `afterEach`, and a negative,
`NaN`, or infinite one is `INVALID_ARGUMENT`.
- `withinCleanupBudget` never rejects. It resolves when the promise settles,
failures included, when `signal` aborts, or when `timeoutMs` passes.
#### `isSecret`
Tells a `secrets.get()` handle from a string, for an option that takes
either.
```ts
const password = isSecret(options.password)
? await context.resolveSecret(options.password)
: options.password;
```
### State
Platform-neutral state capture and restore, behind `test.setup` sessions and
the `session:` option.
| Member | Does |
| --- | --- |
| `capture(context)` | Captures the surface's persisted state as an `EngineState`. |
| `restore(state, context)` | Replaces the surface's whole persisted state with `state`. |
```ts
state: {
async capture(context) {
return { format: 'kiosk-storage', version: 1, data: await client.exportStorage(context) };
},
async restore(state, context) {
await client.replaceStorage(state.data, context);
},
},
```
<Warning>
Snapshots are credentials: cookies, tokens, and storage that authenticate
the app under test. An engine must not persist, cache, or log a snapshot on
its own, must not echo snapshot contents in an error message, and must treat
`restore` as a replacement, never a merge, so a restored session cannot leak
into the next attempt.
</Warning>
```ts
/**
* An opaque, restorable snapshot of an engine's state. `data` is JSON the
* harness never inspects - a document platform's storage state, a device's app state, a
* desktop's window state all satisfy it identically. `format`/`version` let a
* engine reject a snapshot it can no longer read.
*/
export interface EngineState {
/** Engine-defined format name. */
readonly format: string;
/** Format version, for rejecting a snapshot the engine can no longer read. */
readonly version: number;
/** Opaque JSON the runner never inspects. */
readonly data: unknown;
/** Earliest moment the state is known to be invalid, if the engine knows one. */
readonly expiresAt?: string;
}
/**
* Platform-neutral state capture and restore. A setup test captures a
* snapshot; an ordinary test restores it at launch.
*
* Snapshots are credentials: cookies, tokens, and storage that authenticate
* the app under test. The duties split at the seam. The harness owns the
* snapshot from the moment `capture` returns: it encrypts it at rest with a
* per-run AES-256-GCM key that never touches disk, stores it under the run's
* private session directory, binds it to the run, target, engine, and app
* identity, and deletes it when the run ends; it never logs it, reports it,
* digests it, or sends it to a model. An engine MUST NOT persist, cache, or
* log a snapshot on its own, MUST NOT echo snapshot contents in an error
* message, and MUST treat `restore` as a replacement of the surface's whole
* persisted state, never a merge, so a restored session cannot leak into the
* next attempt.
*/
export interface EngineStateCapability {
/** Captures the surface's persisted state. */
capture(context: OperationContext): Promise<EngineState>;
/** Replaces the surface's persisted state with `state`. */
restore(state: EngineState, context: OperationContext): Promise<void>;
}
```
### Artifacts
Evidence capture: screenshots under the attempt directory, and videos there
or at a provider.
| Member | Returns |
| --- | --- |
| `screenshot(label, context)` | `Promise<string>` |
| [`startVideo(context)`](#videos) | `Promise<void>` |
| [`stopVideo(context)`](#videos) | `Promise<readonly VideoSegment[]>` |
`screenshot` is required and returns the path of the file it wrote.
```ts
artifacts: {
async screenshot(_label, context) {
// The label comes from test code, so it never becomes part of the path.
shots += 1;
const path = `screenshots/${String(shots).padStart(3, '0')}.png`;
await client.capture(join(artifactsDir, path), context);
return path;
},
},
```
```ts
/**
* Evidence capture. Paths are relative to the attempt artifact directory the
* engine received in `startAttempt`.
*/
export interface EngineArtifacts {
/** Captures a redacted screenshot; secure fields are masked at the source. */
screenshot(label: string | undefined, context: OperationContext): Promise<string>;
/**
* Starts recording the surface. Declared together with `stopVideo`. An
* attempt may start and stop more than once, never overlapping. A surface that has nothing to show yet (no page open) may
* defer the actual capture to the moment it does; the segments returned by
* `stopVideo` say when each one began.
*/
startVideo?(context: OperationContext): Promise<void>;
/**
* Stops recording and returns every segment written, in order. One segment
* is the common case; a surface whose recording is bound to a page returns
* one per page the attempt showed (a restart or a state reset opens a new
* one). An attempt that never showed anything returns none.
*/
stopVideo?(context: OperationContext): Promise<readonly VideoSegment[]>;
}
/**
* One recorded video of an attempt: a file the engine wrote into the attempt
* artifact directory, or a recording a hosted service keeps and links to.
*/
export type VideoSegment = VideoFile | VideoLink;
/** A video file in the attempt artifact directory. */
export interface VideoFile {
/** Path relative to the attempt artifact directory. */
readonly path: string;
/** When the segment started recording, as an ISO timestamp; its first frame is at or just after it. */
readonly startedAt: string;
}
/** A recording a hosted service keeps, by URL: the report links to it and the runner never downloads it. */
export interface VideoLink {
/** `http(s)` URL of the recording. */
readonly url: string;
/** Media type of what the URL serves: `video/mp4`, or `text/html` for a player page. */
readonly mediaType: string;
/** When the recording started, as an ISO timestamp; its first frame is at or just after it. */
readonly startedAt: string;
}
```
#### Videos
The runner calls `startVideo` for an attempt whose
[video mode](/reference/config#video) records, and for each
[`app.startRecording()`](/reference/app#startrecording), so one attempt may
start and stop several times. It moves the files a stop returns before the
next start, and never starts while a recording runs.
- `startVideo` runs after `startAttempt` and any session restore, within the
launch budget.
- `stopVideo` runs before `endAttempt`, within the cleanup budget.
- When the video mode is set explicitly, a `startVideo` that fails fails the
launch and a `stopVideo` that fails fails cleanup. With the default mode, a
failure of either drops the video quietly.
- The runner decides which recordings are kept and deletes a file an attempt
does not keep.
#### Provider recordings
An engine that leases its browser or device from a hosted service can let the
service record the attempt in place of its own capture. That is how
[`BrowserProvider.record`](/browser#recording-on-the-provider) and
[`DeviceProvider.record`](/mobile#hosted-devices) work. `e2e/engine` exports
the shared shape and `stopProviderRecording`, so every engine turns a
provider's recording into a `VideoSegment` the same way.
```ts
async stopVideo(context) {
if (recording === undefined) return [];
return [await stopProviderRecording(recording, { artifactsDir, provider: 'browser provider "kernel"', leaseId, signal: context.signal })];
},
```
- `isProviderRecording(value)` tells whether what a provider's `record`
resolved to is a recording: a parseable `startedAt` and a `stop`. Check it
before keeping one, so a malformed recording fails the start, named,
instead of the stop.
- `stopProviderRecording` creates the attempt's `video` directory and calls
`stop` with it. A file must be a plain name that exists in that directory;
a link must be an `http(s)` URL with a media type.
- It returns `{ path: 'video/<file>', startedAt }` or
`{ url, mediaType, startedAt }` for `stopVideo` to hand back.
- The target argument's type is exported as `ProviderRecordingTarget`.
| Throws | When |
| --- | --- |
| `ENGINE_FAILURE` | `stop` throws or returns anything else. The message names the provider and the lease. |
```ts
/** Handed to a provider's `record`, once per attempt that records video. */
export interface ProviderRecordContext {
readonly runId: string;
readonly targetName: string;
/** The attempt the recording covers. */
readonly attemptId: string;
/** The run's environment, the same the provider's `acquire` saw. */
readonly env: Readonly<Record<string, string | undefined>>;
/** Aborts when the attempt is cancelled or its launch budget is spent. */
readonly signal: AbortSignal;
}
/** A recording a provider's service makes of the browser or device it leased, as `record` started it. */
export interface ProviderRecording {
/** When the recording started, as an ISO timestamp; its first frame is at or just after it. */
readonly startedAt: string;
stop(context: ProviderRecordingStopContext): Promise<ProviderRecordingResult>;
}
/** Handed to `ProviderRecording.stop`, once per recording. */
export interface ProviderRecordingStopContext {
/** Absolute directory, already created, a recording written as a file goes into. */
readonly dir: string;
/** Aborts when the cleanup budget is spent. */
readonly signal: AbortSignal;
}
/**
* What a stopped recording became: a file the provider wrote into the
* directory it was given, by name, or a recording its service keeps, by
* `http(s)` URL and media type.
*/
export type ProviderRecordingResult =
| { readonly file: string }
| { readonly url: string; readonly mediaType: string };
export function isProviderRecording(value: unknown): value is ProviderRecording;
export function stopProviderRecording(
recording: ProviderRecording,
target: {
/** Absolute attempt artifact directory; a file recording goes into its `video` directory. */
readonly artifactsDir: string;
/** The provider as error messages name it: `browser provider "kernel"`. */
readonly provider: string;
/** The lease the recording covers, named in error messages. */
readonly leaseId: string;
readonly signal: AbortSignal;
},
): Promise<VideoSegment>;
```
#### Redaction
- Videos are unmasked and carry `redaction: 'incomplete'`.
- A `download` an engine attaches is the app's bytes. When the session knows
a secret value, the runner rewrites a text one (`.txt`, `.csv`, `.json`,
`.html`) through the ledger and labels it `complete`.
- Every other download keeps its bytes and carries
`redaction: 'incomplete'`.
### Contributed fixtures
An engine contributes named surfaces to the test fixtures through
`fixtures`. Each key becomes a fixture and a capability name.
```ts title="src/index.ts"
fixtures: {
printer: (context) =>
context.fixture(
'printer',
{
async print(text: string): Promise<void> {
await client.print(text, context.operation());
},
},
{ print: { kind: 'resource', label: (text) => text } },
),
},
```
```ts title="tests/receipt.e2e.ts"
import { test as base } from 'e2e';
const test = base.extend<{ printer: { print(text: string): Promise<void> } }>();
test('prints a receipt', { requires: ['printer'] }, async ({ printer }) => {
await printer.print('Order 42');
});
```
The factory runs once per attempt with an `EngineFixtureContext`:
| Member | Use |
| --- | --- |
| `targetName` | The target the fixture serves. |
| [`fixture(name, surface, operations)`](#declaring-operations) | Declares which methods are recorded steps. |
| `app` | The resolved app, plus `resolveUrl(url)`. |
| `timeouts` | The `test`, `action`, and `assertion` budgets. |
| `signal` | The running phase's signal. Read it per call. |
| `operation(timeoutMs?)` | A per-call `OperationContext`. |
| `attachArtifact(kind, relativePath)` | Registers a file the current step produced. |
| `attachViewport(viewport)` | Records the viewport the current step established. |
| `locator(expression)` | Mints a public locator from a raw expression. |
| `screen(scope)` | Mints a screen scope. |
| [`expectable(target, factory)`](#expectations) | Makes `expect(fixture)` return matchers. |
| Throws | When |
| --- | --- |
| `INVALID_CONFIG` | The factory returns a plain surface, not one declared through `context.fixture`, the first time a test reaches for it. |
| `APP_URL_REQUIRED` | `app.resolveUrl` on a target that declares no URL. |
| `POLICY_DENIED` | `app.resolveUrl` with any scheme but `http:` and `https:` (`about:blank` aside). |
```ts
/**
* A contributed fixture: any record of async methods, sync accessors, and
* nested namespaces, declared through context.fixture so its metadata controls
* labels, deadlines and verification. The factory must return the declared
* surface; a plain one is rejected with INVALID_CONFIG.
*/
export type EngineFixtureFactory = (context: EngineFixtureContext) => object;
/** Context handed to a contributed fixture factory, once per attempt. */
export interface EngineFixtureContext {
/** The target the fixture serves. */
readonly targetName: string;
/** Records the declared operations before invoking them; undeclared accessors retain their identity. */
fixture<T extends object>(name: string, surface: T, operations: FixtureOperations<T>): T;
readonly app: EngineAppInfo & {
/**
* Resolves a navigation target against the base URL. Throws
* `APP_URL_REQUIRED` when the target declares no URL and `POLICY_DENIED`
* for any scheme but `http:` and `https:` (`about:blank` aside), so a
* fixture never re-implements the rule the harness owns.
*/
resolveUrl(url: string): string;
};
/** Harness budgets, for fixtures that poll or navigate. */
readonly timeouts: {
readonly test: number;
readonly action: number;
readonly assertion: number;
};
/**
* The running phase's signal: the attempt's through `beforeEach` and the
* body, then a fresh one per `afterEach` hook so teardown can still drive
* the app after a body timeout. Read it per call; a fixture factory that
* captures it once keeps a signal that is dead by teardown.
*/
readonly signal: AbortSignal;
/** Per-call operation budget: the action timeout (or an explicit one) plus the attempt signal. */
operation(timeoutMs?: number): OperationContext;
/** Registers a file the current step produced under the attempt artifact directory. */
attachArtifact(
kind: 'screenshot' | 'video' | 'download' | 'log',
relativePath: string,
): void;
/** Records the viewport the current step established. */
attachViewport(viewport: { width: number; height: number; scale: number }): void;
/** Mints a public locator from a raw expression (a platform selector, say). */
locator(expression: LocatorExpression): Locator;
/** Mints a screen scope whose every query is wrapped by `scope` (a nested document, say). */
screen(scope: (expression: LocatorExpression) => LocatorExpression): Screen;
/**
* Attaches an expectation surface to a fixture object so `expect(fixture)`
* returns it. The harness records every matcher call as an assertion step
* named `expect.<matcher>`; the factory only decides what the matchers mean.
*/
expectable<T extends object, E extends object>(target: T, factory: () => E): T & Expectable<E>;
}
```
#### Declaring operations
`context.fixture(name, surface, operations)` declares which methods are
recorded steps. Each operation takes:
| Field | Values | Default |
| --- | --- | --- |
| `kind` | `'resource'` or `'assertion'` | required |
| `label` | `(...args) => string` | none |
| `timeout` | milliseconds, a function of the arguments, or `false` when the method owns its deadline | `actionTimeout` |
| `verifies` | `boolean` | `true` for assertions |
- Nested maps describe namespaces.
- Recording decorates the object in place, so create a fresh surface in each
factory invocation.
- The runner opens the step before calling a declared method, so attachments
and failures from synchronous and asynchronous work land on the same step.
Work arriving after its step has closed cannot attach to a later one.
- Type the fixture with `test.extend<{ name: Type }>()`.
```ts
/** Explicit recording policy for one async fixture method. Arguments enter reports only through label. */
export interface FixtureOperation<Args extends unknown[] = unknown[]> {
readonly kind: 'resource' | 'assertion';
readonly label?: (...args: Args) => string;
/** Action timeout by default; false when the method owns its assertion/navigation deadline. */
readonly timeout?: number | false | ((...args: Args) => number | undefined);
/** Assertions verify by default; waits may opt in as well. */
readonly verifies?: boolean;
}
/** Only declared async methods are recorded. Sync accessors pass through unchanged. */
export type FixtureOperations<T extends object> = {
readonly [Key in keyof T]?: T[Key] extends (...args: infer Args) => Promise<unknown>
? FixtureOperation<Args>
: T[Key] extends object ? FixtureOperations<T[Key]> : never;
};
```
#### Expectations
`expectable` attaches an expectation surface so `expect(fixture)` returns it.
Declare matcher recording with `context.fixture('expect', matchers,
operations)`.
```ts
const printer = context.expectable(surface, () =>
context.fixture('expect', matchers, { toHavePrinted: { kind: 'assertion' } }),
);
return context.fixture('printer', printer, { print: { kind: 'resource' } });
```
## Types
### Exported names
`EngineCleanupContext`, `EngineDisposeContext`, `OperationContext`,
`TextPattern`, `SemanticQuery`, `LocatorExpression`,
`NodeRef`, `SemanticNode`, `ViewportPoint`, `ViewportSize`,
`ObservationPixels`, `LocatorAction`, `LocatorActionKind`, `PointerAction`,
`PointerActionKind`, `ScrollDirection`, `Momentum`, `SelectOption`, `Key`,
`KeyModifier`, `KeyName`, `ParsedKey`, `ENGINE_SPI_VERSION`,
`LOCATOR_ACTION_KINDS`, `POINTER_ACTION_KINDS`, `KEY_MODIFIERS`,
`KEY_NAMES`, `parseKey`, `ENGINE_ERROR_CODES`, `RETRYABLE_ENGINE_ERROR_CODES`,
`Secret`, `isSecret`, `TextMatch`, `JsonValue`, `JsonValueRules`,
`ResolveExpressionOptions`, `PollConditionOptions`, `WithoutUndefined`,
`FixtureOperation`, `OBSERVED_NAME_LIMIT`, and `OBSERVED_TEXT_LIMIT` are
exported from `e2e/engine`. So are the [engine protocol](/reference/engine-protocol)'s
`connectEngine`, `serveEngine`, `describeEngine`, `spawnTransport`,
`socketTransport`, `loopbackTransport`, `ENGINE_METHODS`,
`ENGINE_PROTOCOL_VERSION`, and their types.
### `SemanticNode`
```ts
/** One node of an observation tree. */
export interface SemanticNode {
/** Node reference, valid against the observation it came from. */
readonly ref: NodeRef;
/** Semantic role. */
readonly role?: string;
/** Accessible name, cut at `OBSERVED_NAME_LIMIT`. */
readonly name?: string;
/** Visible text, cut at `OBSERVED_TEXT_LIMIT`. */
readonly text?: string;
/** Current input value; never reported for a secure field. */
readonly value?: string;
/**
* Text selected inside the node, for the focused field or editing host
* whose selection the platform reports; absent when the selection is
* collapsed, and never reported for a secure field. Cut at `OBSERVED_TEXT_LIMIT`.
*/
readonly selection?: string;
/**
* The node's test id, when the platform gives it one: the value of the
* project's test-id attribute on a document platform, an accessibility
* identifier on iOS, a resource id on Android, an automation id on a
* desktop. The `testId` query resolves against this field, so the harness
* never knows where a platform keeps it. An engine may also match the id
* without its platform's namespacing.
*/
readonly testId?: string;
/** What a field takes, deciding which secrets may fill it. */
readonly inputPurpose?: 'username' | 'password' | 'one-time-code' | 'generic-secret' | 'none';
/**
* Boolean states the platform reports. `pressed` is a toggle button's
* pressed state; `secure` marks a field whose value is never observed.
*/
readonly states?: Readonly<
Partial<
Record<
'checked' | 'disabled' | 'selected' | 'expanded' | 'pressed' | 'focused' | 'hidden' | 'secure',
boolean
>
>
>;
/** Heading level of a heading node (`1` through `6` on a document platform); absent elsewhere. */
readonly level?: number;
/** Platform attributes, what `getAttribute` reads. */
readonly attributes?: Readonly<Record<string, string>>;
/**
* The node's box in the top-level viewport's CSS pixels (`ViewportPoint`
* space), for every node, including those inside nested documents: an
* engine that measures a child document against its own viewport shifts
* the boxes by the boundary element's before reporting them.
*/
readonly rect?: {
readonly x: number;
readonly y: number;
readonly width: number;
readonly height: number;
};
/**
* The point an offset into the node is measured from, in `rect`'s space,
* when it is not `rect`'s top-left corner: a document element's padding
* box, inside its border, or the corner of a box a frame clipped `rect`
* from. `tap({ position })` adds the offset to it.
*/
readonly origin?: ViewportPoint;
/**
* Platform selector that addresses this node within its own document, when
* the platform has one. It is structural, not semantic: it survives the
* content changes that rename a node, and a runner may store it to re-find
* the node cheaply on a later run. It is never part of the node's identity,
* so whatever it resolves to is still checked before it is used.
*
* It SHOULD be anchored on an attribute naming the node or one of its
* ancestors, and SHOULD be absent rather than positional all the way to the
* document root: such a path is shifted by anything inserted above the node,
* so it does not survive to the later run it exists for.
*/
readonly selector?: string;
/**
* Enclosing nested-document chain as selectors of each boundary node,
* outermost first. Absent for nodes in the main document.
*/
readonly framePath?: readonly string[];
/** Child nodes. */
readonly children?: readonly SemanticNode[];
}
```
### Observed field limits
| Constant | Value | Cuts |
| --- | --- | --- |
| `OBSERVED_NAME_LIMIT` | 256 | `name` |
| `OBSERVED_TEXT_LIMIT` | 512 | `text`, and any `value` or `selection` the engine cuts |
- An engine that cuts an observed field cuts it at exactly its limit. A field
exactly at its limit may have been cut; a shorter one, or a longer one the
engine left whole, is complete.
- Keep the start and cut the end: the runner redacts the leading part of a
secret that a field exactly at its limit ends with, which a whole-value
match cannot see.
- Single-node reads are unbounded and always carry the full value.
### Queries
The `selector` expression kind carries a platform-native selector string;
`frame` scopes a query into a nested document.
```ts
/** The `screen` query kinds. */
export type QueryKind = 'role' | 'label' | 'placeholder' | 'text' | 'displayValue' | 'testId';
```
A `SemanticQuery` with `visible: true` must match only nodes whose
`states.hidden` is false. Use the same predicate the engine reports on
`SemanticNode`, so `getByText('x', { visible: true })` and `toBeVisible()`
never disagree. Apply it wherever the query sits in the expression, under a
scope, filter, or index too.
## Related
[Engines](/engines) compares the ways to build one.
[Writing an engine](/writing-an-engine) walks through `defineEngine`, and
[Engine servers](/engine-servers) runs an engine in another process,
machine, or language.
`@e2e-dev/web` ([reference](/reference/web)) and `@e2e-dev/mobile`
([reference](/reference/mobile)) implement this contract with the same
public `defineEngine`.