Files
e2e/docs/reference/engine.mdx
T

1138 lines
49 KiB
Plaintext

---
title: e2e/engine
description: The engine contract a target's surface implements, and the fixture context it contributes through.
---
`e2e/engine` is the authoring import for an engine. `defineEngine`
validates the body at config load, freezes it, computes its capability set,
and returns the handle a target accepts.
```ts
import { defineEngine, type EngineHandle } from 'e2e/engine';
export function defineEngine(spec: Engine): EngineHandle;
export function isEngineHandle(value: unknown): value is EngineHandle;
```
An engine drives one user interface: a browser, a simulator, a desktop shell.
The contract is platform-neutral vocabulary (semantic nodes, locator
expressions, action kinds, an error taxonomy). The world around the UI (an
inbox, an API, a database) is not an engine's to model; it enters a test as a
fixture through `test.extend`.
Validation fails with `INVALID_CONFIG` for:
- a missing or empty `version`, or a `spiVersion` other than `1`;
- an unknown top-level key on an object literal (a class instance's own
fields are not checked);
- `perform`, `locate`, `performAt`, or `keyboard` without `observe`;
- `perform` without `actions` or the reverse; an `actions` list that is
empty, repeats a kind, or names one outside `LOCATOR_ACTION_KINDS`;
- `performAt` without `pointerActions` or the reverse; a `pointerActions`
list that is empty, repeats a kind, or names one outside
`POINTER_ACTION_KINDS`;
- a `state`, `artifacts`, or `session` manifest with an unknown key or a
non-function member. `state` needs both `capture` and `restore`;
`artifacts` needs `screenshot`; `startTrace`/`stopTrace` and
`startVideo`/`stopVideo` come in pairs;
- an `app` declaration with an unknown key or a function;
- a fixture name that is not a lower-camel identifier or shadows `agent`,
`app`, `screen`, `platform`, or `session`.
Every function on the handle is bound to the spec it came from, so a class
instance is as valid a body as an object literal.
## Engine
```ts
/** The body of one target: typed, model-free, capability-graded. */
export interface Engine {
/** Engine name, for diagnostics. */
readonly name: string;
/**
* Implementation version, recorded as provenance and part of the trace
* cache identity: an engine that resolves nodes differently must not replay
* another version's traces, so an unversioned engine is not accepted.
*/
readonly version: string;
/**
* Contract version literal. Additive optional members never bump it;
* changed required semantics do.
*/
readonly spiVersion: EngineSpiVersion;
/**
* Platform label this engine drives (`web`, `ios`, `android`, or a label of
* the engine's own). Tests filter on it through `platforms` and tool packs
* are scoped by it; the harness attaches no meaning to any value. A target
* inherits it; a target that names a platform of its own must agree with it.
*/
readonly platform?: string;
/**
* The most workers this engine can serve at once for one target: one per
* surface it drives concurrently (a device pool's size; 1 for a single
* simulator). The scheduler never runs more workers for the target, whatever
* `config.workers` allows, so a device target shares a run with browser
* targets without being over-subscribed. Omit when there is no such bound.
*/
readonly workers?: number;
/** capability: observation. */
observe?(context: OperationContext, options?: EngineObserveOptions): Promise<EngineSnapshot>;
/**
* capability: actions - requires observation. Performs exactly one action
* on a ref this engine minted, from the newest observation or from
* `locate` (both live in one id space), with the platform's actionability
* checks. The agent's grammar verbs (tap, type, press, select, scroll) and
* the `screen` tier's actions both bottom out here, so a surface
* implements each action once. A `swipe` on the observation root is the
* viewport swipe.
*
* Error contract (load-bearing for trace replay): throw a retryable
* `NODE_STALE`-coded error when a ref no longer binds, and an
* `ACTION_MAY_HAVE_COMMITTED`-coded error when input may have reached the
* app - the harness never blindly repeats an uncertain mutation.
*/
perform?(ref: NodeRef, action: LocatorAction, context: OperationContext): Promise<void>;
/**
* The action kinds `perform` honors, required with it. The harness offers
* the agent and the `screen` tier exactly these: a kind left out is never
* presented as a tool that can only decline, and `screen`/`Locator` methods
* for it fail with `UNSUPPORTED_CAPABILITY` before reaching the engine.
* `perform` still throws `UNSUPPORTED_CAPABILITY` for a declared kind one
* particular node cannot take (a `check` on a toggle whose state is unknown).
*/
readonly actions?: readonly LocatorActionKind[];
/**
* capability: location - requires observation. Deterministic locator
* resolution for the `screen`/`expect` tier: resolve one expression to the
* nodes it currently matches. The harness owns polling, strictness, and
* staleness; an engine resolves once, immediately.
*/
locate?(
expression: LocatorExpression,
context: OperationContext,
): Promise<readonly SemanticNode[]>;
/**
* capability: pointer - requires observation. Performs one pointer action
* at a viewport point, in the CSS pixels of `SemanticNode.rect`, with no
* node behind it: the agent's `tap_at` verb lands here when the point the
* model named in a screenshot sits on nothing the tree lists (a shape on a
* canvas, a pin on a map, a control in a system sheet). Dispatch the
* pointer at the point as given; the harness has already clamped it to the
* viewport. Throw `UNSUPPORTED_CAPABILITY` for a declared kind the surface
* cannot deliver at this particular point.
*/
performAt?(point: ViewportPoint, action: PointerAction, context: OperationContext): Promise<void>;
/**
* The pointer action kinds `performAt` honors, required with it. The
* harness routes a point-addressed action to the engine only for a kind
* listed here; an undeclared kind fails with `UNSUPPORTED_CAPABILITY`
* before reaching the engine.
*/
readonly pointerActions?: readonly PointerActionKind[];
/** capability: keyboard - requires observation. Input to whatever holds focus; see `EngineKeyboard`. */
readonly keyboard?: EngineKeyboard;
/**
* Named deterministic surfaces this engine contributes to TestFixtures
* (a device fixture, a document fixture - anything). Keys become fixture and
* capability names; `requires: ['<name>']` gates at selection.
*/
readonly fixtures?: Readonly<Record<string, EngineFixtureFactory>>;
/** capability: state - opaque snapshot capture/restore for session reuse. */
readonly state?: EngineStateCapability;
/** capability: artifacts - screenshots and traces under the attempt directory. */
readonly artifacts?: EngineArtifacts;
/** The app under test, as data: url, identity, environment, command, services. */
readonly app?: EngineAppDeclaration;
/** Steering hooks: open, back, restart, reset. */
readonly session?: EngineSession;
/**
* Once per run and target, in the runner process, before any worker starts
* and outside every launch budget. Provision what the engine needs on this
* machine here (a first-run browser download, a toolchain fetch), so it
* happens once instead of per worker and its progress reaches the reporter
* through `info.log`. It runs after collection and before the run's clock
* starts: `plan` is emitted, the report's `startedAt` taken, and the app
* started only once every target is prepared, so a download is never part
* of a run's duration. Failure is infrastructure and ends the run before
* any test executes. The result may lower the target's worker cap for this
* run, for an engine that only learns its capacity here (a device pool
* discovered from the booted devices).
*/
prepare?(info: EnginePrepareInfo): Promise<void | EnginePrepareResult>;
/**
* Once per worker, before the first step; boot devices here, not in a step
* budget. The same handle can be booted again after `dispose`: a config-held
* handle outlives an in-process worker, so init MUST work on a disposed
* engine as it does on a fresh one.
*/
init?(info: EngineInitInfo): Promise<void>;
/** Before each attempt: set up per-test isolation. */
startAttempt?(context: EngineAttemptContext): Promise<void>;
/**
* After each attempt, within the cleanup budget: tear that isolation down.
* MUST be idempotent, and safe to call after a failed `startAttempt`.
*/
endAttempt?(context: EngineCleanupContext): Promise<void>;
/**
* Worker shutdown, within the cleanup budget; failure is a run error. Runs
* whether or not `init` ran, so it MUST tolerate a cold engine.
*/
dispose?(context: EngineCleanupContext): Promise<void>;
}
```
```ts
/** What every engine call carries: budget, ids, and who is acting. */
export interface OperationContext {
/** Aborts with the attempt or when the operation budget is spent. */
readonly signal: AbortSignal;
/** Remaining operation budget when the call starts. */
readonly timeoutMs: number;
/** The run id. */
readonly runId: string;
/** The attempt id. */
readonly attemptId: string;
/**
* Who asked: a test's own deterministic step, or the agent acting on a
* screen it observed. A deterministic step verifies its outcome with
* `expect`, so an engine may act as soon as the target holds still; an
* agent reads the screen right after acting, so an engine may wait for the
* transition to end first.
*/
readonly origin: 'test' | 'agent';
}
```
| Member | Capability | Requires | Unlocks |
| --- | --- | --- | --- |
| `observe` | `observation` | - | judgment tier, prompt snapshots, trace replay, the viewport swipe's root |
| `perform` + `actions` | `actions` | `observe` | the agent's grammar verbs (`tap`, `type`, `press`, `select`, `type_secret`, `scroll`, and `tap_at` when the point lands on a listed node) and the `screen`/`Locator` action set with platform actionability checks, each only for a kind listed in `actions` |
| `locate` | `location` | `observe` | `screen`, `Locator`, `expect(locator)` |
| `performAt` + `pointerActions` | `pointer` | `observe` | the agent's `tap_at` verb when the point lands on nothing the tree lists (needs the `tap` pointer kind): one pointer action at a viewport point in the CSS pixels of `SemanticNode.rect`, with no node behind it |
| `keyboard.{type,press,dismiss?}` | `keyboard` | `observe` | the agent's `type` and `press` without a target, the keyboard fallback of `type_at` and `press_at` on a point the tree lists nothing at, and `dismiss_keyboard` when `dismiss` exists: input to whatever holds focus, for a field drawn on a canvas or flattened out of a platform's tree |
| `fixtures.<name>` | `<name>` | - | a contributed fixture; `requires: ['<name>']` gates at selection |
| `state` | `state` | - | `test.setup` sessions and the `session:` option |
| `artifacts` | `artifacts` | - | `app.screenshot()`, assertion evidence, `artifacts: ['trace']`, `artifacts: ['video']` and `--video` |
| `app` | - | - | data only: the app under test, its origin policy, identity, and process |
| `session` | - | - | `app.open()` and the agent's `navigate` verb (`open`), `app.back()` (`back`), `app.restart()` (`restart`), `app.clearState()` (`reset`) |
The agent offers the model only what `actions` declares: `tap` needs the
`tap` kind, `type` and `type_secret` need `fill`, `press` needs `press`,
`select` needs `selectOption`, `scroll` needs `swipe`, `navigate` needs
`session.open`, `tap_at` needs the `tap` kind in `actions` or in
`pointerActions`. An undeclared verb is absent
from the toolset, and a `screen`/`Locator` method for an undeclared kind
fails with `UNSUPPORTED_CAPABILITY` before reaching the engine.
`engine.capabilities` is the computed `ReadonlySet` of capability names and is
what selection, the report, and `UNSUPPORTED_CAPABILITY` messages read.
```ts
/**
* Capability names: the closed harness capabilities plus one name per
* contributed fixture. `actions` is `perform`; `location` is `locate`; both
* require observation, because their refs live in the observation's id space.
* `pointer` is `performAt`, an action addressed by a viewport point rather
* than a node, and requires observation because the point is read off its
* pixels.
*/
export type EngineCapability =
| 'observation'
| 'actions'
| 'location'
| 'pointer'
| 'keyboard'
| 'state'
| 'artifacts'
| (string & {});
/**
* capability: keyboard - requires observation. Input to whatever holds focus,
* with no node behind it: the agent's `type` and `press` without a target, and
* the fallback of its point-addressed `type_at` and `press_at` when the point
* lands on nothing the tree lists (a field drawn on a canvas, an input a
* platform flattens out of its accessibility tree). Focus is the app's: the
* harness tapped first, or the model said the field already has it.
*/
export interface EngineKeyboard {
/**
* Types `text` into the focused field, through the platform's keyboard or
* text-input path. With `replace`, clears the field first (select-all and
* delete on a keyboard surface; the platform's clear on a device). Throw
* `NOT_ACTIONABLE` when nothing that accepts text has focus, so the
* keystrokes are never silently discarded into the document body.
*/
type(text: string, options: { readonly replace: boolean }, context: OperationContext): Promise<void>;
/**
* Sends one key of the contract grammar (`Enter`, `Escape`, `Tab`,
* `Shift+Tab`, a character) to whatever holds focus. The harness has
* already validated the key's shape; refuse with `UNSUPPORTED_CAPABILITY`
* a key the surface cannot deliver.
*/
press(key: string, context: OperationContext): Promise<void>;
/** Hides an on-screen keyboard. Platforms without one leave it undefined. */
dismiss?(context: OperationContext): Promise<void>;
}
```
## Lifecycle
| Hook | Budget |
| --- | --- |
| `prepare(info)` | none; only `info.signal` (the run's interrupt) cuts it short. Failure ends the run before any test |
| `init(info)` | `launchTimeout`; failure is infrastructure |
| `startAttempt(context)` | `launchTimeout`; on failure or timeout `context.signal` aborts and `endAttempt` runs |
| `endAttempt(context)` | `cleanupTimeout`; failure marks cleanup failed, the test keeps its status |
| `dispose(context)` | `cleanupTimeout`; failure is a run error |
```ts
/**
* What `prepare` learned that the run must honour. `workers` replaces the
* engine's declared cap for this target and run: the surfaces the engine
* actually provisioned, `1` to `info.slots`. Declared `workers` stays the
* static answer for engines that know it up front.
*/
export interface EnginePrepareResult {
/** Lowers the target's worker cap for this run, `1` to `info.slots`. */
readonly workers?: number;
/**
* Variables every worker of this target is spawned with, on top of the
* run's environment: how a runner-side `prepare` hands what it provisioned
* (a device pool it discovered) to each worker's `init`. The run's own
* variables win on a clash.
*/
readonly env?: Readonly<Record<string, string>>;
}
/**
* Facts handed to `prepare`, once per run and target in the runner process,
* before any worker exists.
*/
export interface EnginePrepareInfo {
/** The run id. */
readonly runId: string;
/** The target being prepared. */
readonly targetName: string;
/**
* The worker slots the run will start for this target, `0` to `slots - 1`:
* the run's worker cap, the engine's declared `workers`, and the work units
* selected for the target, whichever is smallest; `0` when nothing runs on
* it. Provision for exactly these slots (boot that many devices of a pool);
* `init` then finds them ready.
*/
readonly slots: number;
/**
* The run's environment: what every worker is started with. A host may hand
* the run an environment other than the runner process's own, so anything
* provisioned here that a worker later looks up by environment (a browser
* cache location) must be resolved and spawned against this, not
* `process.env`.
*/
readonly env: Readonly<Record<string, string | undefined>>;
/**
* Aborts on interrupt only. Provisioning has no budget: a first-run
* download is as long as the network makes it, and cutting it short would
* fail every test behind it.
*/
readonly signal: AbortSignal;
/**
* Reports one line of progress. The runner streams it as a `notice` run
* event, so it reaches the reporter and every host sink instead of being
* written to a worker's stderr underneath the live status block.
*/
readonly log: (line: string) => void;
}
/** Run identity and harness-resolved facts handed to `init`, once per worker before the first step. */
export interface EngineInitInfo {
/** The run id. */
readonly runId: string;
/** The target this worker serves. */
readonly targetName: string;
/**
* Directory relative paths in the config resolve against: the config file's
* directory, or the run's `cwd` for a programmatic config. An engine option
* naming a file (a build to install) resolves here, never against
* `process.cwd()`, which an in-process run does not change.
*/
readonly projectRoot: string;
/** The site policy resolved from the engine's `app` declaration. */
readonly app: EngineAppInfo;
/**
* This worker's environment: the run's, plus what this target's `prepare`
* returned in its result's `env`. Read what `prepare` provisioned from
* here, never from `process.env`: an in-process worker shares its process
* with the runner and every other target.
*/
readonly env: Readonly<Record<string, string | undefined>>;
/** Whether the run asked for a visible surface (`--headed`). */
readonly headed: boolean;
/**
* This worker's 0-based slot among the target's workers: the lowest slot
* free when the worker was spawned, so a replacement worker takes over the
* slot of the one that exited. An engine with several devices hands each
* slot its own; `config.workers` bounds the slots a target can reach.
*/
readonly workerSlot: number;
/** Aborts on interrupt and when init exceeds the launch timeout; init must stop promptly. */
readonly signal: AbortSignal;
}
/**
* The site policy the harness resolved from the engine's declaration, handed
* back at init, for the decisions only a surface can make: which nested
* documents enter an observation, which requests carry injected credentials.
*/
export interface EngineAppInfo {
/**
* The site of the app's `url`, its registrable domain as `siteOf` reads it:
* where the app's secrets, headers, and basic-auth credentials may go, and
* whose child frames an observation reads; `sameSite` applies it. Absent
* for an app without a URL, which is no policy at all. The URL itself stays
* with the harness: a surface never learns where the app is.
*/
readonly site?: string;
}
/**
* Per-attempt isolation context. An engine that must give each test a fresh
* surface (a fresh isolated context per test, a reset device) sets it up in
* `startAttempt` and tears it down in `endAttempt`.
*/
export interface EngineAttemptContext {
/** The attempt id. */
readonly attemptId: string;
/** Absolute directory every artifact of this attempt is written under. */
readonly artifactsDir: string;
/**
* Aborts with the attempt, and the moment `startAttempt` fails or exceeds
* the launch timeout: a hook still running then must stop, because the
* harness ends the attempt's isolation right behind it and may retry.
*/
readonly signal: AbortSignal;
}
/**
* Budget of one cleanup hook (`endAttempt`, `dispose`). `signal` aborts when
* the budget is exhausted, so a hook that cannot finish in time stops instead
* of running on into the next attempt's setup.
*/
export interface EngineCleanupContext {
/** Aborts when the cleanup budget is spent. */
readonly signal: AbortSignal;
/** Remaining cleanup budget when the call starts. */
readonly timeoutMs: number;
}
```
URLs arrive through `session.open` only; `init` never learns the app URL.
## Observation
```ts
/**
* One fresh semantic snapshot of the surface under test. The harness owns
* everything downstream: revision minting, secret redaction, and the
* observation byte budget apply to every engine equally.
*/
export interface EngineSnapshot {
/**
* Where the surface is, as an opaque address the platform understands: a
* URL on a document platform, the foreground screen or activity on a
* device, the front window on a desktop. The harness shows it to the model
* and to the report, anchors trace replay on it, and applies the origin
* policy to it only when it parses as a URL. Omit when the platform has no
* notion of a current location.
*/
readonly location?: string;
/**
* The observation tree under one root the engine minted. The root's id MUST
* be stable for the surface across observations: `perform(root, swipe)` is
* the viewport swipe, and the harness addresses it from an earlier
* observation, possibly one taken before a `restart` or `reset`, so the
* root swipe MUST NOT depend on a live handle from that observation. A
* platform with several top-level elements (a device's windows) wraps them
* in a root of its own.
*/
readonly root: SemanticNode;
/** The viewport `SemanticNode.rect` and `ViewportPoint` are measured in, in CSS pixels. */
readonly viewport: ViewportSize;
/**
* True when `root` leaves out nodes that are on the surface: the engine
* stopped reading at a node cap or a depth limit, or could not enter a
* child document. The harness tells the model the listing is incomplete
* and never calls such a screen unchanged. Omit or set false for a tree
* read whole.
*/
readonly truncated?: boolean;
/** Semantic capture failed; only fresh, independently masked pixels are evidence. */
readonly treeUnavailable?: true;
/** Masked pixels, when requested and producible; omitted otherwise. */
readonly pixels?: ObservationPixels;
/** Regions masked in `pixels`; the harness checks it covers every secure node. */
readonly maskedRegionCount?: number;
}
export interface EngineObserveOptions {
/** Requests masked viewport pixels for the same revision as the tree. */
readonly pixels?: boolean;
/** Permits independently masked pixels to replace a failed semantic capture. */
readonly pixelFallback?: boolean;
}
```
Return the surface as it is right now. The runner mints the revision, stamps
it onto every ref, redacts registered secret values, bounds the byte size, and
checks that `maskedRegionCount` covers every `secure` node when `pixels` are
present.
Set `truncated` when the tree is not the whole surface: a node cap stopped
the read, a frame or a subtree was out of reach. The runner appends a marker
to the listing the model reads, tells it nodes past the cut are on screen but
not listed, and refuses to report such a screen as unchanged between two
observations, since what was left out is unknown. A tree read whole omits it.
A valid empty tree may have a childless root and include `pixels`.
If semantic capture times out, an engine may recover only when `pixelFallback`
is true and it can prove screenshot masking without that capture. Return
`treeUnavailable: true`, fresh `pixels`, and a `root` containing only its
stable `ref`. That reference supports viewport actions; it supplies no
semantic node evidence. Retire earlier node references. The flag distinguishes
missing semantic evidence from a valid empty tree; never infer absence from
it. The runner validates this shape at the engine boundary and rejects a
fallback without permitted pixels as `UNSUPPORTED_CAPABILITY`, or one with
semantic root fields as `INVALID_STATE`. `EngineSnapshot` remains an
extendable interface; these dependent requirements are checked at runtime.
Without permission, pixels, or independent masking proof, propagate the
capture failure. Cancellation, policy denial, and hard engine failures must
also propagate.
The runner permits fallback for `act` and for judgments with `vision: true`
or `'only'`, before any secret fill. It includes the screenshot even if the
initial observation requested only a tree. A step that loses semantic
evidence cannot record or finish by replaying a trace. Engines whose masks
depend on their semantic capture cannot provide this fallback.
## Actions
```ts
/**
* One deterministic action `perform` carries out on a node. The `swipe`
* kind on the observation's root node is the viewport swipe: the agent's
* `scroll` verb and `screen.swipe()` both arrive as `perform(root, swipe)`,
* so a surface implements scrolling once.
*
* `press` takes one key in the `Key` grammar below.
*/
export type LocatorAction =
| {
readonly kind:
| 'tap'
| 'doubleTap'
| 'secondaryTap'
| 'check'
| 'uncheck'
| 'clear'
| 'focus'
| 'hover'
| 'scrollIntoView';
}
| { readonly kind: 'longPress'; readonly durationMs?: number }
| { readonly kind: 'fill'; readonly value: string; readonly sensitive: boolean }
| { readonly kind: 'press'; readonly key: Key }
| { readonly kind: 'selectOption'; readonly value: SelectOption }
| { readonly kind: 'setInputFiles'; readonly paths: readonly string[] }
| { readonly kind: 'dragTo'; readonly target: NodeRef }
| {
readonly kind: 'swipe';
readonly direction: ScrollDirection;
readonly momentum?: Momentum;
};
/** Every action kind; what an engine declares in `actions`. */
export type LocatorActionKind = LocatorAction['kind'];
/** The closed list of action kinds, in contract order; the type is derived from it. */
export const LOCATOR_ACTION_KINDS = [
'tap',
'doubleTap',
'secondaryTap',
'longPress',
'fill',
'clear',
'press',
'check',
'uncheck',
'focus',
'hover',
'scrollIntoView',
'selectOption',
'setInputFiles',
'dragTo',
'swipe',
] as const satisfies readonly LocatorActionKind[];
```
The grammar is closed. Anything beyond it is an agent-side `defineTool`.
### Pointer actions
The point-addressed half of the vocabulary, for a surface with pixels and
coordinates where the tree lists nothing. Same kinds, same meanings, no node:
`dragTo` ends at a second point and `swipe` is a scroll gesture started at
the point. An engine declares the kinds it honors in `pointerActions`, as it
declares `actions` for `perform`.
```ts
export type PointerAction =
| { readonly kind: 'tap' | 'doubleTap' | 'secondaryTap' | 'hover' }
| { readonly kind: 'longPress'; readonly durationMs?: number }
| { readonly kind: 'dragTo'; readonly target: ViewportPoint }
| {
readonly kind: 'swipe';
readonly direction: ScrollDirection;
readonly momentum?: Momentum;
};
/** Every pointer action kind; what an engine declares in `pointerActions`. */
export type PointerActionKind = PointerAction['kind'];
/** The closed list of pointer action kinds, in contract order; the type is derived from it. */
export const POINTER_ACTION_KINDS = [
'tap',
'doubleTap',
'secondaryTap',
'longPress',
'hover',
'dragTo',
'swipe',
] as const satisfies readonly PointerActionKind[];
```
### Keys
```ts
/**
* The key grammar of `press`, shared by every engine so a test's key names are
* portable: zero or more modifiers and one key, joined by `+`.
*
* - Modifiers: `Shift`, `Control`, `Alt`, `Meta`, `ControlOrMeta` (Control on
* Windows and Linux, Meta on macOS).
* - Named keys: those in `KEY_NAMES` (`Enter`, `Escape`, `Tab`, `Backspace`,
* `Delete`, `ArrowUp`..., `Home`, `End`, `PageUp`, `PageDown`, `Insert`,
* `Space`, `F1`..`F12`).
* - Any single printable character (`a`, `A`, `1`, `$`), typed as itself.
*
* `Control+a`, `Shift+Tab`, `Enter`, `$`. A platform without a key maps it
* or throws `UNSUPPORTED_CAPABILITY`; it never reinterprets the spelling.
* `parseKey` is the one parser, exported so no engine carries its own.
*/
export type Key = string;
export const KEY_MODIFIERS = ['Shift', 'Control', 'Alt', 'Meta', 'ControlOrMeta'] as const;
export const KEY_NAMES = [
'Enter',
'Escape',
'Tab',
'Backspace',
'Delete',
'Insert',
'Space',
'ArrowUp',
'ArrowDown',
'ArrowLeft',
'ArrowRight',
'Home',
'End',
'PageUp',
'PageDown',
'F1',
'F2',
'F3',
'F4',
'F5',
'F6',
'F7',
'F8',
'F9',
'F10',
'F11',
'F12',
] as const;
/** One parsed `press` key: its modifiers and either a named key or a single character. */
export interface ParsedKey {
/** Modifiers, in the order written. */
readonly modifiers: readonly KeyModifier[];
/** The key itself. */
readonly key: { readonly kind: 'named'; readonly name: KeyName } | { readonly kind: 'char'; readonly char: string };
}
export function parseKey(key: string): ParsedKey | undefined;
```
## App
`app` declares what the engine knows about the app under test. Every member
is data; a function here is `INVALID_CONFIG`.
```ts
/**
* What an engine declares about the app it drives. The app
* under test is the engine's to describe: a browser engine names a URL, a
* device engine a bundle id. The harness resolves the declaration once per
* target and owns everything built on it - navigation and origin policy,
* cache and session identity, the report's target record, and the app
* process it starts before the run.
*/
export interface EngineAppDeclaration {
/**
* Base URL of an addressable app: `app.open()` opens it and relative
* navigation resolves against it. WHATWG-normalized; no userinfo, query, or
* fragment; a missing scheme becomes `https://`, or `http://` for a
* loopback host. Plain HTTP is accepted for loopback hosts only. A URL on
* `127.0.0.1` or `[::1]` with port 0 asks the run for a free port,
* substituted wherever the declaration used it and handed to `command` and
* `services` as `{port}`.
*/
readonly url?: string;
/**
* Labels the target in the report and joins the cache and session identity
* digest; never gates a run. Defaults to `test` for loopback, `.localhost`,
* and `.test` hosts and for a surface without a URL, `production` otherwise.
*/
readonly environment?: 'test' | 'staging' | 'production';
/**
* Stable logical identity of the app under test, keying trace cache and
* session entries. Defaults to the URL's origin and base path, so an
* ephemeral per-deploy origin (a PR preview) cold-starts every entry; an
* explicit identity keys them by what the app *is* instead of where it is
* served this run. A surface without a URL has no default: declare one
* (a bundle id, say) or entries key on the target alone. Never share one
* identity across genuinely different apps: recorded traces would replay
* across them.
*/
readonly identity?: string;
/**
* Process the runner starts before the first test and stops on every exit
* path (a dev server). Structured, never shell-interpreted; the child
* inherits only `PATH`, `HOME`, the temp-directory variables, and
* `command.env`. Targets declaring the same command share one process,
* probed at the first declaring target's `readyUrl`.
*/
readonly command?: CommandConfig;
/** URL polled until `command` is ready (a 200-499 status); defaults to `url`. */
readonly readyUrl?: string;
/**
* Dependency processes the app needs before it can boot (a database
* container, a migration step), started in declaration order before any
* app command and torn down in reverse after it. Valid without `command`:
* the app may already be running, or be one of the services itself.
* Services declared identically by several targets start once; shared
* services must be declared in one order, and an explicit `name` must mean
* one process across the run.
*/
readonly services?: readonly ServiceConfig[];
}
```
Every field is documented under
[the app under test](/reference/config#the-app-under-test). The site comes
back to the engine as `EngineInitInfo.app.site`; `sameSite` from
`e2e/engine` applies it to a URL.
## Session
```ts
/**
* Steering hooks behind the universal `app` fixture and the agent's
* `navigate` verb. The app itself is described in `Engine.app`, as data;
* node actions never live here, they are `perform`.
*
* `restart` and `reset` open nothing on an addressable surface: they end at
* a fresh surface with no location shown, and the harness reopens the app
* through `open` when the target has an address. A surface therefore never
* needs to know the app's URL, and an unreachable app after a restart is
* reported the same way as on first open. A surface without `open` (a
* device with a pinned app) relaunches the app inside `restart` and `reset`
* itself, since nothing else will.
*/
export interface EngineSession {
/**
* Opens one URL the harness already resolved against the base URL and the
* origin policy. Absent on a surface without addressable locations.
*/
open?(url: string, context: OperationContext): Promise<void>;
/** Navigates back once in the surface's history. */
back?(context: OperationContext): Promise<void>;
/** Recreates the execution context, keeping persisted state. */
restart?(context: OperationContext): Promise<void>;
/** Clears persisted client state and recreates the execution context. */
reset?(context: OperationContext): Promise<void>;
}
```
`open` is behind `app.open()`, the agent's `navigate` verb, and the reopen
after `restart` and `reset`; `back`, `restart`, and `reset` are behind
`app.back()`, `app.restart()`, and `app.clearState()`. An absent hook makes
its method fail with `UNSUPPORTED_CAPABILITY`.
## Errors
```ts
export class EngineError extends Error {
constructor(code: EngineErrorCode, message: string, options: { retryable: boolean; cause?: unknown });
/** One of `ENGINE_ERROR_CODES`. */
readonly code: EngineErrorCode;
/** Whether the runner may re-resolve and retry; only `RETRYABLE_ENGINE_ERROR_CODES` can be. */
readonly retryable: boolean;
}
/** The closed engine error code set; the type is derived from it, so the two cannot drift. */
export const ENGINE_ERROR_CODES = [
'NODE_STALE',
'FRAME_NOT_FOUND',
'FRAME_AMBIGUOUS',
'NOT_ACTIONABLE',
'ACTION_MAY_HAVE_COMMITTED',
'OPERATION_TIMEOUT',
'CANCELLED',
'UNSUPPORTED_CAPABILITY',
'INVALID_STATE',
'ENGINE_FAILURE',
] as const;
/** One of `ENGINE_ERROR_CODES`. */
export type EngineErrorCode = (typeof ENGINE_ERROR_CODES)[number];
/**
* The only codes that may be retryable: both describe a repeatable read. Any
* other retryable claim, from this module's class or a foreign copy of it, is
* coerced to a non-retryable `ENGINE_FAILURE`.
*/
export const RETRYABLE_ENGINE_ERROR_CODES: ReadonlySet<EngineErrorCode> = new Set([
'NODE_STALE',
'FRAME_NOT_FOUND',
]);
```
| Code | Runner mapping | Retryable |
| --- | --- | --- |
| `NODE_STALE` | re-resolve, then `LOCATOR_NOT_FOUND` | yes |
| `FRAME_NOT_FOUND` | re-resolve, then `LOCATOR_NOT_FOUND` | yes |
| `FRAME_AMBIGUOUS` | `LOCATOR_AMBIGUOUS` | no |
| `NOT_ACTIONABLE`, `ACTION_MAY_HAVE_COMMITTED`, `OPERATION_TIMEOUT` | `ACTION_FAILED` | no |
| `CANCELLED` | infrastructure `CANCELLED` | no |
| `UNSUPPORTED_CAPABILITY` | configuration `UNSUPPORTED_CAPABILITY` | no |
| `INVALID_STATE` | `APP_NOT_OPEN` | no |
| `ENGINE_FAILURE`, any non-`EngineError` | infrastructure `ENGINE_FAILURE` | no |
A contributed fixture method fails a step with the runner's own error
classes, exported from the same entry:
```ts
export class TestError extends Error {} // a test failure: ASSERTION_FAILED, INVALID_LOCATOR, ACTION_FAILED, ...
export class ConfigurationError extends Error {} // POLICY_DENIED, APP_URL_REQUIRED, UNSUPPORTED_CAPABILITY, ...
export class InfrastructureError extends Error {} // the surface itself failed: BROWSER_INSTALL_FAILED, ...
```
## Shared semantics
Behavior every engine must reproduce exactly is exported rather than
re-implemented per engine:
| Export | Use |
| --- | --- |
| `matchesText(actual, pattern)`, `toTextPattern(match, { exact? })`, `describePattern(pattern)` | `TextPattern` matching for `locate` and text matchers |
| `urlMatches(current, expected, baseHref)` | `toHaveURL`-style comparison |
| `pollCondition({ deadline, signal, negated, evaluate, onTimeout })`, `Deadline` | Assertion polling and the negation grace window |
| `validateJsonValue(value, label)` | Rejects `undefined`, `NaN`, and cycles in a value a fixture returns |
| `obj`, `raceAbort`, `withTimeout`, `withinCleanupBudget` | See [Contributed fixtures](#contributed-fixtures) |
## State
```ts
/**
* An opaque, restorable snapshot of an engine's state. `data` is JSON the
* harness never inspects - a document platform's storage state, a device's app state, a
* desktop's window state all satisfy it identically. `format`/`version` let a
* engine reject a snapshot it can no longer read.
*/
export interface EngineState {
/** Engine-defined format name. */
readonly format: string;
/** Format version, for rejecting a snapshot the engine can no longer read. */
readonly version: number;
/** Opaque JSON the runner never inspects. */
readonly data: unknown;
/** Earliest moment the state is known to be invalid, if the engine knows one. */
readonly expiresAt?: string;
}
/**
* Platform-neutral state capture and restore. A setup test captures a
* snapshot; an ordinary test restores it at launch.
*
* Snapshots are credentials: cookies, tokens, and storage that authenticate
* the app under test. The duties split at the seam. The harness owns the
* snapshot from the moment `capture` returns: it encrypts it at rest with a
* per-run AES-256-GCM key that never touches disk, stores it under the run's
* private session directory, binds it to the run, target, engine, and app
* identity, and deletes it when the run ends; it never logs it, reports it,
* digests it, or sends it to a model. An engine MUST NOT persist, cache, or
* log a snapshot on its own, MUST NOT echo snapshot contents in an error
* message, and MUST treat `restore` as a replacement of the surface's whole
* persisted state, never a merge, so a restored session cannot leak into the
* next attempt.
*/
export interface EngineStateCapability {
/** Captures the surface's persisted state. */
capture(context: OperationContext): Promise<EngineState>;
/** Replaces the surface's persisted state with `state`. */
restore(state: EngineState, context: OperationContext): Promise<void>;
}
```
## Artifacts
```ts
/**
* Evidence capture. Paths are relative to the attempt artifact directory the
* engine received in `startAttempt`.
*/
export interface EngineArtifacts {
/** Captures a redacted screenshot; secure fields are masked at the source. */
screenshot(label: string | undefined, context: OperationContext): Promise<string>;
/** Starts recording an execution trace for the attempt. */
startTrace?(context: OperationContext): Promise<void>;
/**
* Stops the trace and returns its relative path, or every archive written,
* in order, when the trace had to be cut: a trace bound to one context
* closes as a segment when a restart or a state reset replaces the
* context, and a new one records on from there. The harness registers and
* redacts each returned archive.
*/
stopTrace?(context: OperationContext): Promise<string | readonly string[]>;
/**
* Starts recording the surface for the attempt. Declared together with
* `stopVideo`. A surface that has nothing to show yet (no page open) may
* defer the actual capture to the moment it does; the segments returned by
* `stopVideo` say when each one began.
*/
startVideo?(context: OperationContext): Promise<void>;
/**
* Stops recording and returns every segment written, in order. One segment
* is the common case; a surface whose recording is bound to a page returns
* one per page the attempt showed (a restart or a state reset opens a new
* one). An attempt that never showed anything returns none.
*/
stopVideo?(context: OperationContext): Promise<readonly VideoSegment[]>;
}
/** One recorded video file of an attempt. */
export interface VideoSegment {
/** Path relative to the attempt artifact directory. */
readonly path: string;
/** When the segment started recording, as an ISO timestamp; its first frame is at or just after it. */
readonly startedAt: string;
}
```
The runner starts the video, then the trace, right after `startAttempt` (and
after a session restore) under the launch budget, and stops them before
`endAttempt` under the cleanup budget. Video comes first so an engine that
records both through one screencast sizes it for the recording a person
watches. `stopVideo` returns every segment in order, one per page the
attempt showed. `stopTrace` returns the trace's path, or every archive in
order when the trace had to be cut because the context was replaced. When a
secret was filled in the attempt, every text entry of each archive is
rewritten through the secret redactor; if that fails the archives are deleted
and the attempt records `TRACE_WITHHELD`. Recordings mask nothing and are
recorded with `redaction: 'incomplete'`.
## Contributed fixtures
```ts
/**
* A contributed fixture: any record of async methods, sync accessors, and
* nested namespaces, declared through context.fixture so its metadata controls
* labels, deadlines and verification. The factory must return the declared
* surface; a plain one is rejected with INVALID_CONFIG.
*/
export type EngineFixtureFactory = (context: EngineFixtureContext) => object;
/** Context handed to a contributed fixture factory, once per attempt. */
export interface EngineFixtureContext {
/** The target the fixture serves. */
readonly targetName: string;
/** Records the declared operations before invoking them; undeclared accessors retain their identity. */
fixture<T extends object>(name: string, surface: T, operations: FixtureOperations<T>): T;
readonly app: EngineAppInfo & {
/**
* Resolves a navigation target against the base URL. Throws
* `APP_URL_REQUIRED` when the engine declared no URL and `POLICY_DENIED`
* for a `file:`, `data:`, or `javascript:` scheme, so a fixture never
* re-implements the rule the harness owns.
*/
resolveUrl(url: string): string;
};
/** Harness budgets, for fixtures that poll or navigate. */
readonly timeouts: {
readonly test: number;
readonly action: number;
readonly assertion: number;
};
/**
* The running phase's signal: the attempt's through `beforeEach` and the
* body, then a fresh one per `afterEach` hook so teardown can still drive
* the app after a body timeout. Read it per call; a fixture factory that
* captures it once keeps a signal that is dead by teardown.
*/
readonly signal: AbortSignal;
/** Per-call operation budget: the action timeout (or an explicit one) plus the attempt signal. */
operation(timeoutMs?: number): OperationContext;
/** Registers a file the current step produced under the attempt artifact directory. */
attachArtifact(
kind: 'screenshot' | 'trace' | 'video' | 'download' | 'log',
relativePath: string,
): void;
/** Records the viewport the current step established. */
attachViewport(viewport: { width: number; height: number; scale: number }): void;
/** Mints a public locator from a raw expression (a platform selector, say). */
locator(expression: LocatorExpression): Locator;
/** Mints a screen scope whose every query is wrapped by `scope` (a nested document, say). */
screen(scope: (expression: LocatorExpression) => LocatorExpression): Screen;
/**
* Attaches an expectation surface to a fixture object so `expect(fixture)`
* returns it. The harness records every matcher call as an assertion step
* named `expect.<matcher>`; the factory only decides what the matchers mean.
*/
expectable<T extends object, E extends object>(target: T, factory: () => E): T & Expectable<E>;
}
```
`context.fixture(name, surface, operations)` declares which methods are
recorded steps. Each operation names a `kind: 'resource' | 'assertion'`, an
optional `label(...args)`, an optional `timeout` (milliseconds, a function of
the arguments, or `false` when the method owns its deadline), and optional
`verifies`. The default timeout is `actionTimeout`; assertions verify by
default. Nested maps describe namespaces. Recording decorates the object in
place, so create a fresh surface in each factory invocation.
The runner opens the step before calling a declared method, so attachments
and failures from synchronous and asynchronous work land on the same step.
Work arriving after its step has closed cannot attach to a later one.
A factory must return the surface it declared through `context.fixture`; a
plain surface is `INVALID_CONFIG` the first time a test reaches for it.
`expectable` attaches an expectation surface so `expect(fixture)` returns it;
declare matcher recording with `context.fixture('expect', matchers,
operations)`. Type the fixture with `test.extend<{ name: Type }>()`.
The entrypoint also exports `obj(value)`, which drops `undefined`-valued keys
so a result built from optional inputs satisfies exact optional property
types, plus `raceAbort(work, signal, label)`, `withTimeout(promise,
timeoutMs, onTimeout)`, and `withinCleanupBudget(promise, { signal,
timeoutMs })`. Pass a thunk to `raceAbort` to check cancellation before
dispatch.
## Vocabulary
`OperationContext`, `TextPattern`, `SemanticQuery`, `LocatorExpression`,
`NodeRef`, `SemanticNode`, `ViewportPoint`, `ViewportSize`, `ObservationPixels`,
`LocatorAction`, `LocatorActionKind`, `PointerAction`, `PointerActionKind`,
`ScrollDirection`, `Momentum`, `SelectOption`, `Key`,
`KeyModifier`, `KeyName`, `ParsedKey`, `ENGINE_SPI_VERSION`,
`LOCATOR_ACTION_KINDS`, `POINTER_ACTION_KINDS`, `KEY_MODIFIERS`,
`KEY_NAMES`, `parseKey`, `ENGINE_ERROR_CODES`, `RETRYABLE_ENGINE_ERROR_CODES`,
`OBSERVED_NAME_LIMIT` and `OBSERVED_TEXT_LIMIT` are exported from
`e2e/engine`. The `selector` expression kind carries a
platform-native selector string; `frame` scopes a query into a nested
document.
```ts
/** One node of an observation tree. */
export interface SemanticNode {
/** Node reference, valid against the observation it came from. */
readonly ref: NodeRef;
/** Semantic role. */
readonly role?: string;
/** Accessible name, cut at `OBSERVED_NAME_LIMIT`. */
readonly name?: string;
/** Visible text, cut at `OBSERVED_TEXT_LIMIT`. */
readonly text?: string;
/** Current input value; never reported for a secure field. */
readonly value?: string;
/**
* The node's test id, when the platform gives it one: the value of the
* project's test-id attribute on a document platform, an accessibility
* identifier on iOS, a resource id on Android, an automation id on a
* desktop. The `testId` query resolves against exactly this field, so the
* harness never knows where a platform keeps it.
*/
readonly testId?: string;
/** What a field takes, deciding which secrets may fill it. */
readonly inputPurpose?: 'username' | 'password' | 'one-time-code' | 'generic-secret' | 'none';
/**
* Boolean states the platform reports. `pressed` is a toggle button's
* pressed state; `secure` marks a field whose value is never observed.
*/
readonly states?: Readonly<
Partial<
Record<
'checked' | 'disabled' | 'selected' | 'expanded' | 'pressed' | 'focused' | 'hidden' | 'secure',
boolean
>
>
>;
/** Heading level of a heading node (`1` through `6` on a document platform); absent elsewhere. */
readonly level?: number;
/** Platform attributes, what `getAttribute` reads. */
readonly attributes?: Readonly<Record<string, string>>;
/**
* The node's box in the top-level viewport's CSS pixels (`ViewportPoint`
* space), for every node, including those inside nested documents: an
* engine that measures a child document against its own viewport shifts
* the boxes by the boundary element's before reporting them.
*/
readonly rect?: {
readonly x: number;
readonly y: number;
readonly width: number;
readonly height: number;
};
/**
* Platform selector that addresses this node within its own document, when
* the platform has one. It is structural, not semantic: it survives the
* content changes that rename a node, and a runner may store it to re-find
* the node cheaply on a later run. It is never part of the node's identity,
* so whatever it resolves to is still checked before it is used.
*
* It SHOULD be anchored on an attribute naming the node or one of its
* ancestors, and SHOULD be absent rather than positional all the way to the
* document root: such a path is shifted by anything inserted above the node,
* so it does not survive to the later run it exists for.
*/
readonly selector?: string;
/**
* Enclosing nested-document chain as selectors of each boundary node,
* outermost first. Absent for nodes in the main document.
*/
readonly framePath?: readonly string[];
/** Child nodes. */
readonly children?: readonly SemanticNode[];
}
/** The `screen` query kinds. */
export type QueryKind = 'role' | 'label' | 'placeholder' | 'text' | 'displayValue' | 'testId';
```
A `SemanticQuery` with `visible: true` must match only nodes whose
`states.hidden` is false, by the same predicate the engine reports on
`SemanticNode`, so `getByText('x', { visible: true })` and `toBeVisible()`
never disagree. Apply it wherever the query sits in the expression, under a
scope, filter, or index too.