mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-22 05:00:35 +08:00
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2450 lines
97 KiB
Plaintext
2450 lines
97 KiB
Plaintext
---
|
|
title: e2e/engine
|
|
description: Build an engine with observations, actions, lifecycle hooks, and fixtures.
|
|
"og:image": "https://e2e.com/docs/images/og/reference/engine.png?v=d82cf186db"
|
|
"twitter:image": "https://e2e.com/docs/images/og/reference/engine.png?v=d82cf186db"
|
|
---
|
|
|
|
`e2e/engine` is the contract an engine implements to drive a user interface,
|
|
such as a browser or simulator. An engine implements shared types for nodes,
|
|
locators, actions, and errors. `defineEngine` validates and freezes the
|
|
implementation, derives its capabilities, and returns a handle for `targets`
|
|
in the config. The runner drives it over the
|
|
[engine protocol](/reference/engine-protocol), in its own process; see
|
|
[Engines](/engines). Use `test.extend` fixtures for resources outside the UI,
|
|
such as APIs or databases. `@e2e-dev/core/engine` is the same module: a
|
|
published engine imports it from there and peers on `@e2e-dev/core`, so it runs
|
|
under the runner and under [@e2e-dev/core](/core) alike.
|
|
|
|
This sketch assumes a `connect` adapter for the platform:
|
|
|
|
```ts title="src/index.ts"
|
|
import { defineEngine, resolveExpression, type EngineHandle } from 'e2e/engine';
|
|
import { connect } from './client.ts';
|
|
|
|
export function kiosk(options: { host: string }): EngineHandle {
|
|
const client = connect(options.host);
|
|
return defineEngine({
|
|
name: 'kiosk',
|
|
version: '1.0.0',
|
|
spiVersion: 1,
|
|
platform: 'kiosk',
|
|
async observe() {
|
|
const screen = await client.snapshot();
|
|
return {
|
|
root: { ref: { id: 'root', revision: '' }, role: 'window', children: screen.nodes },
|
|
viewport: screen.viewport,
|
|
};
|
|
},
|
|
async locate(expression) {
|
|
const screen = await client.snapshot();
|
|
return resolveExpression(expression, screen.nodes);
|
|
},
|
|
actions: ['tap'],
|
|
async perform(ref) {
|
|
await client.tap(ref.id);
|
|
},
|
|
});
|
|
}
|
|
```
|
|
|
|
```ts title="e2e.config.ts"
|
|
import type { E2EConfig } from 'e2e';
|
|
import { kiosk } from './src/index.ts';
|
|
|
|
export default {
|
|
targets: [{ name: 'kiosk', engine: kiosk({ host: 'kiosk.local' }) }],
|
|
} satisfies E2EConfig;
|
|
```
|
|
|
|
## Reference
|
|
|
|
### defineEngine
|
|
|
|
Validates an engine spec, freezes it, computes its
|
|
[capabilities](#capabilities), and returns a branded handle.
|
|
`isEngineHandle` tells such a handle from any other value.
|
|
|
|
```ts
|
|
import { defineEngine, isEngineHandle, type EngineHandle } from 'e2e/engine';
|
|
|
|
export function defineEngine<Fixtures extends object = {}>(spec: Engine): EngineHandle<Fixtures>;
|
|
// With a fixture beside screen, app, and platform, `spec.fixtures` must hold
|
|
// a factory returning each one Fixtures names.
|
|
export function isEngineHandle(value: unknown): value is EngineHandle;
|
|
|
|
/** A validated engine: branded, frozen, capabilities computed. */
|
|
export interface EngineHandle<Fixtures extends object = {}> extends Engine {
|
|
readonly capabilities: ReadonlySet<EngineCapability>;
|
|
}
|
|
```
|
|
|
|
Every function on the handle is bound to the spec it came from, so a class
|
|
instance is as valid a body as an object literal.
|
|
|
|
`Fixtures` types what the engine's [`fixtures`](#contributed-fixtures) contribute, such as
|
|
`{ browser: Browser }`. The compiler requires a factory in `fixtures` for each
|
|
fixture it names, returning that fixture; `screen`, `app`, and `platform` need
|
|
none, since every session holds them. Nothing checks it at runtime. With it, `launch` from [`@e2e-dev/core`](/core) types a session's fixtures
|
|
without a hand-written argument: return `defineEngine<{ device: Device }>(...)`
|
|
from the factory.
|
|
|
|
`defineEngine` throws `INVALID_CONFIG` when the spec breaks one of these
|
|
rules:
|
|
|
|
| Member | Rejected when |
|
|
| --- | --- |
|
|
| the spec | It is not an object. |
|
|
| `name` | Missing or empty. |
|
|
| `version` | Missing or empty. |
|
|
| `spiVersion` | Anything but `1`. |
|
|
| `platform` | Empty. |
|
|
| `workers` | Not a positive safe integer. |
|
|
| `observe`, `readStyle`, `locate`, `perform`, `performAt`, `prepare`, `finish`, `doctor`, `init`, `startAttempt`, `settleAttempt`, `endAttempt`, `dispose`, `validateApp` | Not a function. |
|
|
| any other top-level key | On an object literal, `app` included: declare the app on the target and check it in `validateApp`. A class instance's own fields are not checked. |
|
|
| `readStyle`, `perform`, `locate`, `performAt`, `keyboard` | Declared without `observe`. |
|
|
| `perform`, `actions` | One without the other. An `actions` list that is empty, repeats a kind, or names one outside `LOCATOR_ACTION_KINDS`. |
|
|
| `performAt`, `pointerActions` | One without the other. A `pointerActions` list that is empty, repeats a kind, or names one outside `POINTER_ACTION_KINDS`. |
|
|
| `tapModifiers` | Not a boolean, or `true` without `perform`. |
|
|
| `keyboard`, `state`, `artifacts`, `session` | An unknown key or a non-function member. `keyboard` without `type` and `press`; `state` without both `capture` and `restore`; `artifacts` without `screenshot`; `startVideo` without `stopVideo`, or the reverse. |
|
|
| `secrets` | Not an array of `secrets.get()` handles. |
|
|
| `fixtures` | Not an object, a fixture that is not a factory function, or a fixture name that is not a lower-camel identifier or shadows `agent`, `app`, `screen`, `platform`, or `session`. |
|
|
|
|
<Note>
|
|
`artifacts.startTrace` and `artifacts.stopTrace` were removed from the
|
|
contract. An engine published before their removal that declares them still
|
|
passes validation; the runner never calls them.
|
|
</Note>
|
|
|
|
### Engine
|
|
|
|
The body of one target. Every member but `name`, `version`, and
|
|
`spiVersion` is optional; the members an engine implements decide its
|
|
[capabilities](#capabilities).
|
|
|
|
| Member | Type |
|
|
| --- | --- |
|
|
| `name` | `string` |
|
|
| `version` | `string` |
|
|
| `spiVersion` | `1` |
|
|
| `platform` | `string` |
|
|
| `workers` | `number` |
|
|
| [`observe`](#observation) | method |
|
|
| [`readStyle`](#rendered-node-properties) | method |
|
|
| [`perform`, `actions`](#actions) | method, kinds |
|
|
| [`locate`](#capabilities) | method |
|
|
| [`performAt`, `pointerActions`](#pointer-actions) | method, kinds |
|
|
| [`tapModifiers`](#capabilities) | `boolean` |
|
|
| [`keyboard`](#keyboard) | `EngineKeyboard` |
|
|
| [`fixtures`](#contributed-fixtures) | factories |
|
|
| [`state`](#state) | `EngineStateCapability` |
|
|
| [`artifacts`](#artifacts) | `EngineArtifacts` |
|
|
| [`validateApp`](#app) | method |
|
|
| [`secrets`](#startattempt) | `Secret[]` |
|
|
| [`session`](#session) | `EngineSession` |
|
|
| [`prepare`, `finish`, `doctor`, `init`, `startAttempt`, `settleAttempt`, `endAttempt`, `dispose`](#lifecycle) | hooks |
|
|
|
|
```ts
|
|
/** The body of one target: typed, model-free, capability-graded. */
|
|
export interface Engine {
|
|
/** Engine name, for diagnostics. */
|
|
readonly name: string;
|
|
/**
|
|
* Required implementation version, recorded in reports and sessions.
|
|
* A saved session never restores across engine versions; a cached trace
|
|
* replays across them and re-finds its nodes.
|
|
*/
|
|
readonly version: string;
|
|
/**
|
|
* Contract version literal. Additive optional members never bump it;
|
|
* changed required semantics do.
|
|
*/
|
|
readonly spiVersion: EngineSpiVersion;
|
|
/**
|
|
* Platform label this engine drives (`web`, `ios`, `android`, or a label of
|
|
* the engine's own). Tests filter on it through `platforms` and tool packs
|
|
* are scoped by it; the harness attaches no meaning to any value. A target
|
|
* inherits it; a target that names a platform of its own must agree with it.
|
|
*/
|
|
readonly platform?: string;
|
|
/**
|
|
* Maximum workers for this target, such as the number of devices in a
|
|
* pool. The scheduler applies this cap even when `config.workers` is
|
|
* higher. Omit when the engine has no concurrency limit.
|
|
*/
|
|
readonly workers?: number;
|
|
/** capability: observation. */
|
|
observe?(context: OperationContext, options?: EngineObserveOptions): Promise<EngineSnapshot>;
|
|
/** capability: computed-style, requires observation. Read-only rendered properties of an observation ref. */
|
|
readStyle?(ref: NodeRef, properties: readonly ComputedStyleProperty[], context: OperationContext): Promise<ComputedStyleValues>;
|
|
/**
|
|
* Performs one action on a ref from the newest observation or `locate`,
|
|
* with the platform's actionability checks. Both agent tools and locator
|
|
* actions call this method. A `swipe` on the observation root swipes the
|
|
* viewport.
|
|
*
|
|
* For safe trace replay, throw a retryable
|
|
* `NODE_STALE`-coded error when a ref no longer binds, and an
|
|
* `ACTION_MAY_HAVE_COMMITTED`-coded error when input may have reached the
|
|
* app. The runner never repeats an action that may have committed.
|
|
*/
|
|
perform?(ref: NodeRef, action: LocatorAction, context: OperationContext): Promise<void>;
|
|
/**
|
|
* The action kinds `perform` honors, required with it. The harness offers
|
|
* the agent and the `screen` tier exactly these: a kind left out is never
|
|
* presented as a tool that can only decline, and `screen`/`Locator` methods
|
|
* for it fail with `UNSUPPORTED_CAPABILITY` before reaching the engine.
|
|
* `perform` still throws `UNSUPPORTED_CAPABILITY` for a declared kind one
|
|
* particular node cannot take (a `check` on a toggle whose state is unknown).
|
|
*/
|
|
readonly actions?: readonly LocatorActionKind[];
|
|
/**
|
|
* capability: location - requires observation. Deterministic locator
|
|
* resolution for the `screen`/`expect` tier: resolve one expression to the
|
|
* nodes it currently matches. The harness owns polling, strictness, and
|
|
* staleness; an engine resolves once, immediately.
|
|
*/
|
|
locate?(
|
|
expression: LocatorExpression,
|
|
context: OperationContext,
|
|
): Promise<readonly SemanticNode[]>;
|
|
/**
|
|
* capability: pointer - requires observation. Performs one pointer action
|
|
* at a viewport point, in the CSS pixels of `SemanticNode.rect`, with no
|
|
* node behind it: the agent's `tap_at`, `hover_at`, and `drag_at` verbs land
|
|
* here with the point the model named in a screenshot (a shape on a canvas, a pin on
|
|
* a map, a control in a system sheet or one the tree lists), and so do a test's
|
|
* `screen.tapAt`, `screen.swipe({ from, to })`, and `tap({ position })`.
|
|
* Dispatch the pointer at the point as given; the harness has clamped an
|
|
* agent's point to the viewport and validated a test's. Throw
|
|
* `UNSUPPORTED_CAPABILITY` for a declared kind the surface cannot deliver
|
|
* at this particular point.
|
|
*/
|
|
performAt?(point: ViewportPoint, action: PointerAction, context: OperationContext): Promise<void>;
|
|
/**
|
|
* The pointer action kinds `performAt` honors, required with it. The
|
|
* harness routes a point-addressed action to the engine only for a kind
|
|
* listed here; an undeclared kind fails with `UNSUPPORTED_CAPABILITY`
|
|
* before reaching the engine.
|
|
*/
|
|
readonly pointerActions?: readonly PointerActionKind[];
|
|
/**
|
|
* The surface honors `modifiers` on the `tap`, `doubleTap`, and
|
|
* `secondaryTap` actions `perform` receives. Without it the harness fails
|
|
* an action that carries modifiers with `UNSUPPORTED_CAPABILITY` before it
|
|
* reaches the engine.
|
|
*/
|
|
readonly tapModifiers?: boolean;
|
|
/** capability: keyboard - requires observation. Input to whatever holds focus; see `EngineKeyboard`. */
|
|
readonly keyboard?: EngineKeyboard;
|
|
/**
|
|
* Named deterministic surfaces this engine contributes to TestFixtures
|
|
* (a device fixture, a document fixture - anything). Keys become fixture and
|
|
* capability names; `requires: ['<name>']` gates at selection.
|
|
*/
|
|
readonly fixtures?: Readonly<Record<string, EngineFixtureFactory>>;
|
|
/** capability: state - opaque snapshot capture/restore for session reuse. */
|
|
readonly state?: EngineStateCapability;
|
|
/** capability: artifacts - screenshots under the attempt directory, and videos there or at a provider. */
|
|
readonly artifacts?: EngineArtifacts;
|
|
/**
|
|
* Checks the app a target declares against what this engine can drive, at
|
|
* config load, synchronously: throw `INVALID_CONFIG` naming
|
|
* `info.targetName` for a field the platform needs and the target left out
|
|
* (a URL, a bundle id) or one it cannot drive. The harness has already
|
|
* checked every field's shape.
|
|
*/
|
|
validateApp?(app: EngineAppDeclaration, info: EngineAppCheckInfo): void;
|
|
/**
|
|
* The `secrets.get()` handles the engine's options hold, such as a
|
|
* basic-auth password. The config load checks each names a configured
|
|
* secret (`INVALID_CONFIG` otherwise), and only these resolve through
|
|
* `EngineAttemptContext.resolveSecret`. The handles carry names only, so
|
|
* no value reaches the config digest.
|
|
*/
|
|
readonly secrets?: readonly Secret[];
|
|
/** Steering hooks: open, back, resize, restart, reset. */
|
|
readonly session?: EngineSession;
|
|
/**
|
|
* Runs once per target in the runner, after collection and before workers,
|
|
* app commands, and the run's clock start. A run with nothing to run (the
|
|
* test filters left nothing, or `--pass-with-no-tests` on an empty
|
|
* selection) prepares no target, and so calls no `finish`. Download
|
|
* dependencies or boot devices here; report progress through `info.log`.
|
|
* No launch timeout applies. A failure ends the run as an infrastructure
|
|
* error.
|
|
* The result can lower the target's worker cap, for example after
|
|
* discovering how many devices are available.
|
|
*/
|
|
prepare?(info: EnginePrepareInfo): Promise<void | EnginePrepareResult>;
|
|
/**
|
|
* Once per run and target in the runner process, after every worker of the
|
|
* target is gone, on every exit path, whenever `prepare` was called.
|
|
* Release what `prepare` acquired for the run (a leased cloud device).
|
|
* Runs within the cleanup budget, never cancelled by an interrupt; a
|
|
* failure is a cleanup-phase run error.
|
|
*/
|
|
finish?(info: EngineFinishInfo): Promise<void>;
|
|
/**
|
|
* Checks this machine for what a run on the target needs, for
|
|
* `e2e doctor`. Read-only and quick: never install, launch, lease, or
|
|
* start anything a check could instead look up, and never call `prepare`.
|
|
* A check that cannot tell reports nothing rather than guess. A throw is
|
|
* reported as one failed check of the target.
|
|
*/
|
|
doctor?(info: EngineDoctorInfo): Promise<readonly EngineDoctorCheck[]>;
|
|
/**
|
|
* Once per worker, before the first step; boot devices here, not in a step
|
|
* budget. The same handle can be booted again after `dispose`: a config-held
|
|
* handle outlives an in-process worker, so init MUST work on a disposed
|
|
* engine as it does on a fresh one.
|
|
*/
|
|
init?(info: EngineInitInfo): Promise<void>;
|
|
/** Before each attempt: set up per-test isolation. */
|
|
startAttempt?(context: EngineAttemptContext): Promise<void>;
|
|
/**
|
|
* After the body has settled, and again after teardown, before the verdict,
|
|
* within the cleanup budget: report a failure that landed on a path no step
|
|
* awaited (a callback the surface ran for the test: a request interceptor,
|
|
* a native dialog handler). Wait for one still running, then throw its
|
|
* error; the attempt fails with it the way a step would, an error already
|
|
* classified keeping its code. MUST report each failure once.
|
|
*/
|
|
settleAttempt?(context: EngineCleanupContext): Promise<void>;
|
|
/**
|
|
* After each attempt, within the cleanup budget: tear that isolation down.
|
|
* MUST be idempotent, and safe to call after a failed `startAttempt`.
|
|
*/
|
|
endAttempt?(context: EngineCleanupContext): Promise<void>;
|
|
/**
|
|
* Worker shutdown, within the cleanup budget; failure is a run error. Runs
|
|
* once `init` was called, even when it failed, so it MUST tolerate a
|
|
* partly initialized engine. A live
|
|
* `reuseSignal` permits preserving run-owned resources for a replacement,
|
|
* until that permission is revoked or this hook settles.
|
|
*/
|
|
dispose?(context: EngineDisposeContext): Promise<void>;
|
|
}
|
|
```
|
|
|
|
#### `OperationContext`
|
|
|
|
```ts
|
|
/** What every engine call carries: budget, ids, and who is acting. */
|
|
export interface OperationContext {
|
|
/**
|
|
* Aborts when the attempt, or the hook running the call, ends. It does not
|
|
* abort at `timeoutMs`: bound each call by that budget yourself.
|
|
*/
|
|
readonly signal: AbortSignal;
|
|
/**
|
|
* Remaining operation budget when the call starts. A waiting call's last
|
|
* locate or read can get more than its deadline leaves: as long as the
|
|
* slowest one it has seen.
|
|
*/
|
|
readonly timeoutMs: number;
|
|
/** The run id. */
|
|
readonly runId: string;
|
|
/** The attempt id. */
|
|
readonly attemptId: string;
|
|
/**
|
|
* Who asked: a test's own deterministic step, or the agent acting on a
|
|
* screen it observed. A deterministic step verifies its outcome with
|
|
* `expect`, so an engine may act as soon as the target holds still; an
|
|
* agent reads the screen right after acting, so an engine may wait for the
|
|
* transition to end first.
|
|
*/
|
|
readonly origin: 'test' | 'agent';
|
|
}
|
|
```
|
|
|
|
### Capabilities
|
|
|
|
The members an engine implements set its capabilities. A `screen`,
|
|
`Locator`, or `app` call that needs a missing one fails with
|
|
`UNSUPPORTED_CAPABILITY`. [Test APIs](#test-apis) lists what needs which
|
|
member. Three gaps end differently:
|
|
|
|
- A test that `requires` a fixture the engine lacks is skipped
|
|
(`capability-unavailable`).
|
|
- A selected test that uses a `test.setup` session, on an engine without
|
|
`state`, is a collection error before the setup runs.
|
|
- A `video` mode on an engine without video recording prints a notice and
|
|
records no video.
|
|
|
|
| Member | Capability | Requires | Unlocks |
|
|
| --- | --- | --- | --- |
|
|
| `observe` | `observation` | - | The judgment tier, prompt snapshots, trace replay, the viewport swipe's root. |
|
|
| `readStyle` | `computed-style` | `observe` | Allow-listed rendered properties for read-only project tools. |
|
|
| `perform` + `actions` | `actions` | `observe` | Node actions of the declared kinds, with platform actionability checks. |
|
|
| `locate` | `location` | `observe` | `screen`, `Locator`, `expect(locator)`. |
|
|
| `tapModifiers: true` | - | `perform` | `modifiers` on `tap`, `doubleTap`, and `secondaryTap`. |
|
|
| `performAt` + `pointerActions` | `pointer` | `observe` | One pointer action at a viewport point, with no node behind it. |
|
|
| `keyboard.{type,press,dismiss?}` | `keyboard` | `observe` | Input to whatever holds focus. |
|
|
| `fixtures.<name>` | `<name>` | - | A contributed fixture. |
|
|
| `state` | `state` | - | `test.setup` sessions and the `session:` option. |
|
|
| `artifacts` | `artifacts` | - | Screenshots and videos. |
|
|
| `validateApp` | - | - | A config-load check of the target's `app`. |
|
|
| `session` | - | - | The `app` steering methods and the agent's `navigate`, `back`, `resize_viewport`, `switch_tab`, and `close_tab` verbs. |
|
|
|
|
#### Rendered node properties
|
|
|
|
`readStyle(ref, properties, context)` returns computed CSS strings for the requested
|
|
properties without changing app state. Its `computed-style` capability requires
|
|
`observe`, and `ref` comes from the newest observation. Return a string for
|
|
every requested property. Throw retryable `NODE_STALE` if the node no longer
|
|
binds. The harness redacts values before returning them to tools.
|
|
|
|
```ts
|
|
export type ComputedStyleProperty =
|
|
| 'color' | 'background-color' | 'font-size' | 'font-weight' | 'font-style'
|
|
| 'text-align' | 'text-decoration-line' | 'display' | 'visibility' | 'opacity';
|
|
|
|
export type ComputedStyleValues = Readonly<Partial<Record<ComputedStyleProperty, string>>>;
|
|
```
|
|
|
|
`COMPUTED_STYLE_PROPERTIES` exports the allow-list as a frozen array.
|
|
`validateComputedStyleProperties(properties)` returns a deduplicated copy and
|
|
throws `INVALID_ARGUMENT` for an empty list or a property outside the
|
|
allow-list. Call it before page access.
|
|
See [Project tools](/tools#read-a-nodes-computed-style) for `ToolContext.read`.
|
|
|
|
#### Agent verbs
|
|
|
|
The agent offers the model only the verbs the engine's declarations unlock.
|
|
An undeclared verb is absent from the toolset.
|
|
|
|
| Verb | Needs |
|
|
| --- | --- |
|
|
| `tap` | the `tap` kind in `actions` |
|
|
| `double_tap` | `doubleTap` |
|
|
| `long_press` | `longPress` |
|
|
| `right_click` | `secondaryTap` |
|
|
| `hover` | `hover` |
|
|
| `type` | `fill`; without a target, `keyboard` |
|
|
| `type_secret` | `fill` |
|
|
| `press` | `press`; without a target, `keyboard` |
|
|
| `select` | `selectOption` |
|
|
| `check` | both `check` and `uncheck`, since it sets a state rather than flipping one |
|
|
| `scroll`, `scroll_to({ text })` | `swipe` |
|
|
| `scroll_to({ target })` | `scrollIntoView` |
|
|
| `drag` | `dragTo` |
|
|
| `upload` | `setInputFiles` |
|
|
| `navigate` | `session.open` |
|
|
| `back` | `session.back` |
|
|
| `resize_viewport` | `session.resize` |
|
|
| `switch_tab` | `session.switchTab` |
|
|
| `close_tab` | `session.closeTab` |
|
|
| `tap_at`, `hover_at` | the `tap` or `hover` kind, in `actions` or in `pointerActions` |
|
|
| `drag_at` | the `dragTo` kind, in `actions` or in `pointerActions` |
|
|
| `type_at` | `fill` or `keyboard` |
|
|
| `press_at` | `press` or `keyboard` |
|
|
| `select_at` | `selectOption` |
|
|
| `dismiss_keyboard` | `keyboard.dismiss` |
|
|
|
|
- `clear` and `focus` unlock no verb.
|
|
- `tap_at` and `hover_at` go through `performAt` (the `tap` and `hover`
|
|
pointer kinds). Without that kind they go through `perform` when the point
|
|
lands on a node the tree lists.
|
|
- `drag_at` goes through `performAt` with the `dragTo` pointer kind and the
|
|
second point as its `target`. Without that kind it goes through `perform`
|
|
when its two points land on two different listed nodes.
|
|
- `type_at`, `press_at`, and `select_at` go through `perform` when the point
|
|
lands on a node the tree lists. Elsewhere `type_at` and `press_at` fall
|
|
back to `keyboard`.
|
|
|
|
#### Test APIs
|
|
|
|
The test-facing APIs each engine member backs. A `screen` or `Locator`
|
|
method for an undeclared kind, or a call with `modifiers` and no
|
|
`tapModifiers`, fails with `UNSUPPORTED_CAPABILITY` before it reaches the
|
|
engine.
|
|
|
|
| API | Needs |
|
|
| --- | --- |
|
|
| `screen`, `Locator`, `expect(locator)` | `locate` |
|
|
| `screen` and `Locator` actions | `perform`, each only for a kind listed in `actions` |
|
|
| `screen.swipe()` | `perform` with `swipe` on the observation root |
|
|
| `modifiers` on a tap (`row.click({ modifiers: ['Shift'] })`) | `tapModifiers: true`; the keys are held for the click |
|
|
| `screen.tapAt`, `tap({ position })` | `performAt` with the `tap` pointer kind |
|
|
| `screen.doubleTapAt` | `performAt` with the `doubleTap` pointer kind |
|
|
| `screen.swipe({ from, to })` | `performAt` with the `swipeTo` pointer kind |
|
|
| `locator.pressSequentially` | the `focus` action and `keyboard`; the harness composes `focus` then `keyboard.type` |
|
|
| `test.setup` sessions, the `session:` option | `state` |
|
|
| `app.screenshot()`, assertion evidence, the failure screenshot | `artifacts` |
|
|
| every [`video` mode](/reference/config#recording) but `off`, from the config, a target, a test, or `--video` | `artifacts` |
|
|
| `app.open()` | `session.open`; on a target with no app URL, `session.restart`, or `session.openLink` for a path on a target with `app.linkBase` |
|
|
| `app.back()` | `session.back` |
|
|
| `app.restart()` | `session.restart` |
|
|
| `app.clearState()` | `session.reset` |
|
|
| `requires: ['<name>']` | `fixtures.<name>` |
|
|
|
|
#### `engine.capabilities`
|
|
|
|
The computed `ReadonlySet` of capability names. Selection, the report, and
|
|
`UNSUPPORTED_CAPABILITY` messages read it.
|
|
|
|
```ts
|
|
const engine = kiosk({ host: 'kiosk.local' });
|
|
engine.capabilities.has('location'); // true: the engine implements locate
|
|
```
|
|
|
|
Selection also grades a target by the kind of app it declares, not by its
|
|
engine. A target with `app.url` has `browser`, and one with `app.bundleId` or
|
|
`app.appPath` has `native-app`, so a test can `requires: ['native-app']`.
|
|
|
|
```ts
|
|
/**
|
|
* Capability names: the closed harness capabilities plus one name per
|
|
* contributed fixture. `actions` is `perform`; `location` is `locate`; both
|
|
* require observation, because their refs live in the observation's id space.
|
|
* `pointer` is `performAt`, an action addressed by a viewport point rather
|
|
* than a node, and requires observation because the point is read off its
|
|
* pixels.
|
|
*/
|
|
export type EngineCapability =
|
|
| 'observation'
|
|
| 'computed-style'
|
|
| 'actions'
|
|
| 'location'
|
|
| 'pointer'
|
|
| 'keyboard'
|
|
| 'state'
|
|
| 'artifacts'
|
|
| (string & {});
|
|
```
|
|
|
|
#### Keyboard
|
|
|
|
```ts
|
|
/**
|
|
* capability: keyboard - requires observation. Input to whatever holds focus,
|
|
* with no node behind it: the agent's `type` and `press` without a target, and
|
|
* the fallback of its point-addressed `type_at` and `press_at` when the point
|
|
* lands on nothing the tree lists (a field drawn on a canvas, an input a
|
|
* platform flattens out of its accessibility tree). Focus is the app's: the
|
|
* harness tapped first, or the model said the field already has it.
|
|
*/
|
|
export interface EngineKeyboard {
|
|
/**
|
|
* Types `text` into the focused field as a person typing would, through
|
|
* the platform's keyboard path: a browser dispatches the key events of each
|
|
* character, a device sends the text through its keyboard. Where the
|
|
* platform has key events the app must receive them; setting the value
|
|
* without keystrokes is not a compliant implementation, since
|
|
* `locator.pressSequentially` and the agent's `type` rely on the app
|
|
* reacting as it does to a user. With `replace`, clears the field first
|
|
* (select-all and delete on a keyboard surface; the platform's clear on a
|
|
* device). Throw `NOT_ACTIONABLE` when nothing that accepts text has focus,
|
|
* so the keystrokes are never silently discarded into the document body.
|
|
*/
|
|
type(text: string, options: { readonly replace: boolean }, context: OperationContext): Promise<void>;
|
|
/**
|
|
* Sends one key of the contract grammar (`Enter`, `Escape`, `Tab`,
|
|
* `Shift+Tab`, a character) to whatever holds focus. The harness has
|
|
* already validated the key's shape; refuse with `UNSUPPORTED_CAPABILITY`
|
|
* a key the surface cannot deliver.
|
|
*/
|
|
press(key: string, context: OperationContext): Promise<void>;
|
|
/** Hides an on-screen keyboard. Platforms without one leave it undefined. */
|
|
dismiss?(context: OperationContext): Promise<void>;
|
|
}
|
|
```
|
|
|
|
### Lifecycle
|
|
|
|
The hooks the runner calls around workers and attempts. Every hook is
|
|
optional. The `Engine` declaration's JSDoc above states each hook's contract.
|
|
|
|
`init` and `startAttempt` run within `launchTimeout`. `finish`,
|
|
`settleAttempt`, `endAttempt`, and `dispose` run within `cleanupTimeout`.
|
|
`prepare` has no timeout. Both budgets come from the
|
|
[config](/reference/config).
|
|
|
|
| Hook | Runs |
|
|
| --- | --- |
|
|
| [`prepare(info)`](#prepare) | Once per target, before any worker |
|
|
| [`finish(info)`](#finish) | After the target's last worker is gone |
|
|
| [`doctor(info)`](#doctor) | Once per target in `e2e doctor`, never in a run |
|
|
| [`init(info)`](#init) | Once per worker, before its first step |
|
|
| [`startAttempt(context)`](#startattempt) | Before each attempt |
|
|
| [`settleAttempt(context)`](#settleattempt) | After the body and after teardown |
|
|
| [`endAttempt(context)`](#endattempt) | After each attempt |
|
|
| [`dispose(context)`](#dispose) | Worker shutdown |
|
|
|
|
#### `prepare`
|
|
|
|
Runs once per target in the runner process, after collection and before
|
|
workers, app commands, and the run's clock start.
|
|
|
|
```ts
|
|
async prepare(info) {
|
|
if (info.slots === 0) return; // nothing runs on this target
|
|
info.log('booting devices');
|
|
const devices = await pool.boot(info.slots, { signal: info.signal });
|
|
return { workers: devices.length, env: { KIOSK_DEVICES: devices.join(',') } };
|
|
},
|
|
```
|
|
|
|
A failure ends the run before any test:
|
|
|
|
| Thrown | Becomes |
|
|
| --- | --- |
|
|
| A runner error class, such as a `ConfigurationError` for an option the engine cannot honour | The same code. |
|
|
| An `EngineError` with code `CANCELLED` | A cancellation. |
|
|
| Any other `EngineError`, or a plain `Error` | Infrastructure `ENGINE_FAILURE`. |
|
|
|
|
```ts
|
|
/**
|
|
* What `prepare` learned that the run must honour. `workers` replaces the
|
|
* engine's declared cap for this target and run: the surfaces the engine
|
|
* actually provisioned, `1` to `info.slots`. Declared `workers` stays the
|
|
* static answer for engines that know it up front.
|
|
*/
|
|
export interface EnginePrepareResult {
|
|
/** Lowers the target's worker cap for this run, `1` to `info.slots`. */
|
|
readonly workers?: number;
|
|
/**
|
|
* Variables every worker of this target is spawned with, on top of the
|
|
* run's environment: how a runner-side `prepare` hands what it provisioned
|
|
* (a device pool it discovered) to each worker's `init`. The run's own
|
|
* variables win on a clash.
|
|
*/
|
|
readonly env?: Readonly<Record<string, string>>;
|
|
}
|
|
|
|
/**
|
|
* Facts handed to `prepare`, once per run and target in the runner process,
|
|
* before any worker exists.
|
|
*/
|
|
export interface EnginePrepareInfo {
|
|
/** The run id. */
|
|
readonly runId: string;
|
|
/** The target being prepared. */
|
|
readonly targetName: string;
|
|
/** Directory relative paths in the config resolve against; see `EngineInitInfo.projectRoot`. */
|
|
readonly projectRoot: string;
|
|
/** The app the target declares, as `init` receives it: what a device installs or warms up here. */
|
|
readonly app: EngineAppInfo;
|
|
/**
|
|
* The worker slots the run will start for this target, `0` to `slots - 1`:
|
|
* the run's worker cap, the engine's declared `workers`, and the work units
|
|
* selected for the target, whichever is smallest; `0` when nothing runs on
|
|
* it while another target has work. A run with nothing to run prepares no
|
|
* target. Provision for exactly these slots (boot that many devices of a
|
|
* pool); `init` then finds them ready.
|
|
*/
|
|
readonly slots: number;
|
|
/**
|
|
* The run's environment: what every worker is started with. A host may hand
|
|
* the run an environment other than the runner process's own, so anything
|
|
* provisioned here that a worker later looks up by environment (a browser
|
|
* cache location) must be resolved and spawned against this, not
|
|
* `process.env`.
|
|
*/
|
|
readonly env: Readonly<Record<string, string | undefined>>;
|
|
/**
|
|
* Aborts on interrupt only. Provisioning has no budget: a first-run
|
|
* download is as long as the network makes it, and cutting it short would
|
|
* fail every test behind it.
|
|
*/
|
|
readonly signal: AbortSignal;
|
|
/** Whether the run asked for a visible surface (`--headed`), as `EngineInitInfo` declares it. */
|
|
readonly headed: boolean;
|
|
/**
|
|
* Reports one line of progress. The runner streams it as a `notice` run
|
|
* event, so it reaches the reporter and every host sink instead of being
|
|
* written to a worker's stderr underneath the live status block.
|
|
*/
|
|
readonly log: (line: string) => void;
|
|
}
|
|
```
|
|
|
|
#### `finish`
|
|
|
|
Runs once per run and target in the runner process, after every worker of
|
|
the target is gone. Release what `prepare` acquired for the run.
|
|
|
|
```ts
|
|
async finish(info) {
|
|
await pool.release({ signal: info.signal });
|
|
},
|
|
```
|
|
|
|
The `finish` hooks of all prepared targets share one `cleanupTimeout`.
|
|
|
|
```ts
|
|
/** Handed to `finish`, once per run and target after the last worker is gone, with the cleanup budget of one hook. */
|
|
export interface EngineFinishInfo extends EngineCleanupContext {
|
|
/** The run id. */
|
|
readonly runId: string;
|
|
/** The target being finished. */
|
|
readonly targetName: string;
|
|
/** The run's environment, the same `prepare` saw. */
|
|
readonly env: Readonly<Record<string, string | undefined>>;
|
|
/** Reports one line of progress, streamed as a `notice` run event like `prepare`'s. */
|
|
readonly log: (line: string) => void;
|
|
}
|
|
```
|
|
|
|
#### `doctor`
|
|
|
|
Runs once per target when [`e2e doctor`](/reference/cli#e2e-doctor) checks
|
|
the project, never during a run. Report what a run on this machine needs and
|
|
whether it's there. Each finding becomes a check under the target. Without the
|
|
hook, the target gets one `engine` check with status `skip`.
|
|
|
|
```ts
|
|
async doctor(info) {
|
|
const sdk = await findToolchain(info.env);
|
|
if (sdk === undefined) return [{ id: 'toolchain', status: 'fail', message: 'install the toolchain' }];
|
|
return [{ id: 'toolchain', status: 'pass', message: `toolchain ${sdk.version}` }];
|
|
},
|
|
```
|
|
|
|
`e2e doctor` gives the hook 30 seconds, starting a protocol engine's
|
|
server included. A throw, a timeout, a finding that isn't an
|
|
`EngineDoctorCheck`, or one with an id `e2e doctor` uses itself (`node`,
|
|
`config`, `engine`, `model`) becomes an `engine` check with status `fail`.
|
|
|
|
```ts
|
|
/** Facts handed to `doctor`, once per target in the process `e2e doctor` runs in. */
|
|
export interface EngineDoctorInfo {
|
|
/** The target being checked. */
|
|
readonly targetName: string;
|
|
/** Directory relative paths in the config resolve against; see `EngineInitInfo.projectRoot`. */
|
|
readonly projectRoot: string;
|
|
/** The app the target declares, as `prepare` receives it. */
|
|
readonly app: EngineAppInfo;
|
|
/** The environment a run would start with. */
|
|
readonly env: Readonly<Record<string, string | undefined>>;
|
|
/** Whether the run would show the surface (`--headed`), as `prepare` receives it. */
|
|
readonly headed: boolean;
|
|
/** Aborts 30 seconds in, when the checks fail as timed out, or when `e2e doctor` is interrupted; return within 5 seconds of it, when the connection closes. */
|
|
readonly signal: AbortSignal;
|
|
/** Reports one line of progress; `e2e doctor` prints it to stderr. */
|
|
readonly log: (line: string) => void;
|
|
}
|
|
|
|
/** One finding of `doctor`. */
|
|
export interface EngineDoctorCheck {
|
|
/** What was checked, in kebab case: `browser`, `toolchain`. */
|
|
readonly id: string;
|
|
/**
|
|
* `pass`; `warn` for something a run handles on its own, slower (a
|
|
* first-run download); `fail` for something that stops a run; `skip` for
|
|
* a check that does not apply to this target.
|
|
*/
|
|
readonly status: 'pass' | 'warn' | 'fail' | 'skip';
|
|
/** One line: what was found and, unless it passed, what to do. */
|
|
readonly message?: string;
|
|
}
|
|
```
|
|
|
|
#### `init`
|
|
|
|
Runs once per worker, before the first step. Boot devices here, not in a step
|
|
budget.
|
|
|
|
```ts
|
|
async init(info) {
|
|
const device = info.env.KIOSK_DEVICES?.split(',')[info.workerSlot];
|
|
info.log(`slot ${info.workerSlot}: ${device}`);
|
|
},
|
|
```
|
|
|
|
`init` never learns the app URL; URLs arrive through `session.open` only. A
|
|
failure is an infrastructure error.
|
|
|
|
```ts
|
|
/** Run identity and harness-resolved facts handed to `init`, once per worker before the first step. */
|
|
export interface EngineInitInfo {
|
|
/** The run id. */
|
|
readonly runId: string;
|
|
/** The target this worker serves. */
|
|
readonly targetName: string;
|
|
/**
|
|
* Directory relative paths in the config resolve against: the config file's
|
|
* directory, or the run's `cwd` for a programmatic config. An engine option
|
|
* naming a file (a build to install) resolves here, never against
|
|
* `process.cwd()`, which an in-process run does not change.
|
|
*/
|
|
readonly projectRoot: string;
|
|
/** The app the target declares: the origin and site policy of its URL, and what a device launches. */
|
|
readonly app: EngineAppInfo;
|
|
/**
|
|
* This worker's environment: the run's, plus what this target's `prepare`
|
|
* returned in its result's `env`. Read what `prepare` provisioned from
|
|
* here, never from `process.env`: an in-process worker shares its process
|
|
* with the runner and every other target.
|
|
*/
|
|
readonly env: Readonly<Record<string, string | undefined>>;
|
|
/** Whether the run asked for a visible surface (`--headed`). */
|
|
readonly headed: boolean;
|
|
/**
|
|
* This worker's 0-based slot among the target's workers: the lowest slot
|
|
* free when the worker was spawned, so a replacement worker takes over the
|
|
* slot of the one that exited. An engine with several devices hands each
|
|
* slot its own; `config.workers` bounds the slots a target can reach.
|
|
*/
|
|
readonly workerSlot: number;
|
|
/** Aborts on interrupt and when init exceeds the launch timeout; init must stop promptly. */
|
|
readonly signal: AbortSignal;
|
|
/**
|
|
* Reports one line of progress. The runner streams it as a `notice` run
|
|
* event, prefixed with the target and worker slot, so it reaches the
|
|
* reporter and every host sink instead of being written to a worker's
|
|
* stderr underneath the live status block. This is where a fact that only
|
|
* exists once the worker is up belongs: the URL a hosted browser or device
|
|
* can be watched at, which simulator a slot got, a version it detected.
|
|
* Meant for `init` itself; a line logged later is still delivered while
|
|
* the worker lives and dropped once it has exited. The line is shown as is
|
|
* in reporters and host sinks, so keep tokens and other secrets out of it
|
|
* (a viewer URL without its credential).
|
|
*/
|
|
readonly log: (line: string) => void;
|
|
}
|
|
```
|
|
|
|
#### `startAttempt`
|
|
|
|
Runs before each attempt, to set up the per-test isolation that
|
|
[`endAttempt`](#endattempt) tears down.
|
|
|
|
```ts
|
|
async startAttempt(context) {
|
|
const password = await context.resolveSecret(options.password);
|
|
await client.openContext({ password, signal: context.signal });
|
|
context.environment({ device: 'Kiosk 2', os: 'KioskOS 4.1' });
|
|
},
|
|
```
|
|
|
|
On failure or timeout, `context.signal` aborts and `endAttempt` runs.
|
|
|
|
<Warning>
|
|
A value from `resolveSecret` is protected as text only. Reports, logs, and
|
|
observations redact it and the attempt's text downloads are rewritten, but
|
|
a page that shows it outside a secure field is not masked in screenshots.
|
|
</Warning>
|
|
|
|
```ts
|
|
/**
|
|
* Per-attempt isolation context. An engine that must give each test a fresh
|
|
* surface (a fresh isolated context per test, a reset device) sets it up in
|
|
* `startAttempt` and tears it down in `endAttempt`.
|
|
*/
|
|
export interface EngineAttemptContext {
|
|
/** The attempt id. */
|
|
readonly attemptId: string;
|
|
/** Absolute directory every artifact of this attempt is written under. */
|
|
readonly artifactsDir: string;
|
|
/**
|
|
* The plaintext of a secret the engine declared in `secrets`, for an
|
|
* option the engine hands the app itself (basic-auth credentials). A
|
|
* provider runs again on every call. The value, and every value
|
|
* `options.derived` computes from it, join the attempt's redaction before
|
|
* this resolves, so reports, logs, and every observation redact them, and
|
|
* the attempt's text downloads are rewritten. The protection is
|
|
* text only: a page that shows the value outside a secure field is not
|
|
* masked in screenshots.
|
|
* A secret the engine did not declare is `SECRET_UNAVAILABLE`.
|
|
*/
|
|
readonly resolveSecret: (secret: Secret, options?: ResolveSecretOptions) => Promise<string>;
|
|
/**
|
|
* Aborts with the attempt, and the moment `startAttempt` fails or exceeds
|
|
* the launch timeout: a hook still running then must stop, because the
|
|
* harness ends the attempt's isolation right behind it and may retry.
|
|
*/
|
|
readonly signal: AbortSignal;
|
|
/**
|
|
* Reports something the app did on its own while the attempt ran: a
|
|
* console line, an uncaught exception, a request that failed. The harness
|
|
* redacts the text, keeps it with the step that was running (from an event
|
|
* callback outside any step, the step that started last; before the first
|
|
* step, with none), and shows it on the trace page. Call it from the moment `startAttempt` begins until
|
|
* `endAttempt`; entries past a per-attempt cap are dropped.
|
|
*/
|
|
readonly appLog: (entry: AppLogEntry) => void;
|
|
/**
|
|
* Reports where the app went on its own, as one line (`navigated to
|
|
* /login`, `the app opened a new tab at /help`): a navigation the engine
|
|
* did not start, a tab, a frame. Shown under the step that was running.
|
|
*/
|
|
readonly navigation: (line: string) => void;
|
|
/**
|
|
* Hands the harness a screen for the trace: the capture a `locate`
|
|
* matched against, or one read right after an action. A read after an
|
|
* action is on the action's path, so bound it and skip it where it is
|
|
* expensive; it never fails the action. The trace page shows how each
|
|
* step's screen changed. Refs in it are not valid for `perform`. Absent
|
|
* when the attempt keeps no trace.
|
|
*/
|
|
readonly screen?: (snapshot: EngineSnapshot) => void;
|
|
/**
|
|
* What the attempt runs on, such as `{ browser: 'chromium 141.0' }` or
|
|
* `{ device: 'iPhone 16', os: 'iOS 26.0' }`, for the trace page. Later
|
|
* calls add to and overwrite earlier facts.
|
|
*/
|
|
readonly environment: (facts: Readonly<Record<string, string>>) => void;
|
|
}
|
|
|
|
/** One line of what the app did during an attempt, as `EngineAttemptContext.appLog` takes it. */
|
|
export interface AppLogEntry {
|
|
/**
|
|
* Where it came from: the app's `console`, an `error` nothing caught, the
|
|
* `network`, or the `system` the app runs on (a crash, an OS log).
|
|
*/
|
|
readonly source: 'console' | 'error' | 'network' | 'system';
|
|
readonly level: 'error' | 'warning' | 'info';
|
|
/** One line, such as `GET /api/todos 500` or `TypeError: x is undefined`; the harness clips it. */
|
|
readonly text: string;
|
|
}
|
|
|
|
/** How `EngineAttemptContext.resolveSecret` treats what the engine makes of the value. */
|
|
export interface ResolveSecretOptions {
|
|
/**
|
|
* The forms of the value the app sees instead of the value itself, for the
|
|
* same redaction under the secret's name: the base64 `user:password` an
|
|
* `Authorization: Basic` header carries, which a page echoing its request
|
|
* headers would otherwise show.
|
|
*/
|
|
readonly derived?: (plaintext: string) => readonly string[] | Promise<readonly string[]>;
|
|
}
|
|
```
|
|
|
|
#### `settleAttempt`
|
|
|
|
Runs after the body has settled, and again after teardown, before the
|
|
verdict. It reports a failure on a path no step awaited, such as a request
|
|
interceptor or a native dialog handler.
|
|
|
|
```ts
|
|
async settleAttempt() {
|
|
const running = [...dialogHandlers];
|
|
dialogHandlers.clear();
|
|
await Promise.all(running); // rethrows a handler's failure
|
|
},
|
|
```
|
|
|
|
#### `endAttempt`
|
|
|
|
Runs after each attempt and tears down what `startAttempt` set up. A failure
|
|
marks cleanup failed; the test keeps its status.
|
|
|
|
```ts
|
|
async endAttempt(context) {
|
|
await client.closeContext({ signal: context.signal });
|
|
},
|
|
```
|
|
|
|
#### `dispose`
|
|
|
|
Runs one time per worker, including when `init` never ran.
|
|
|
|
Missing or aborted `context.reuseSignal` requires final cleanup. A live
|
|
signal permits keeping run-owned resources for a replacement worker.
|
|
An engine that keeps resources must observe revocation until its hook settles.
|
|
Final shutdown, a forced interrupt, or parent disconnect revokes that permission.
|
|
The cleanup budget still uses `context.signal` and `context.timeoutMs`.
|
|
|
|
Cleanup is best effort within that budget. The runner does not guarantee
|
|
cleanup after a worker exits or after disposal settles with reuse permission.
|
|
|
|
```ts
|
|
async dispose(context) {
|
|
await client.shutdown({ signal: context.signal });
|
|
},
|
|
```
|
|
|
|
```ts
|
|
/**
|
|
* Budget of one cleanup hook (`settleAttempt`, `endAttempt`, `dispose`). `signal` aborts when
|
|
* the budget is exhausted, so a hook that cannot finish in time stops instead
|
|
* of running on into the next attempt's setup.
|
|
*/
|
|
export interface EngineCleanupContext {
|
|
/** Aborts when the cleanup budget is spent. */
|
|
readonly signal: AbortSignal;
|
|
/** Remaining cleanup budget when the call starts. */
|
|
readonly timeoutMs: number;
|
|
}
|
|
|
|
/** Cleanup of one worker, with optional permission to leave run-owned resources for a replacement. */
|
|
export interface EngineDisposeContext extends EngineCleanupContext {
|
|
/**
|
|
* A live signal permits reusable resources owned by `prepare` and `finish`
|
|
* to survive this worker. Missing or aborted permission requires final
|
|
* cleanup. Observe revocation until disposal settles. This signal does
|
|
* not cancel cleanup, which remains bounded by `signal` and `timeoutMs`.
|
|
*/
|
|
readonly reuseSignal?: AbortSignal;
|
|
}
|
|
```
|
|
|
|
### Observation
|
|
|
|
`observe` returns one fresh semantic snapshot of the surface under test.
|
|
|
|
| Field | Type |
|
|
| --- | --- |
|
|
| `location` | `string` |
|
|
| `root` | `SemanticNode` |
|
|
| `viewport` | `ViewportSize` |
|
|
| [`truncated`](#truncated-trees) | `boolean` |
|
|
| `keyboardVisible` | `boolean` |
|
|
| `tabs` | `{ count, active }` |
|
|
| [`treeUnavailable`](#pixel-fallback) | `true` |
|
|
| `pixels` | `ObservationPixels` |
|
|
| `maskedRegionCount` | `number` |
|
|
|
|
`tabs` reports the open tabs when there is more than one: `count`, and the
|
|
zero-based place of the tab `root` reads, in the order the tabs opened. The
|
|
harness names the tab in every screen header and tells the model when a
|
|
tab opened or the screen moved to another tab. Omit it with one tab.
|
|
|
|
Return the surface as it is right now. The runner mints the revision and
|
|
stamps it onto every ref. When `pixels` are present, it checks that
|
|
`maskedRegionCount` covers every `secure` node. A valid empty tree may have a
|
|
childless root and include `pixels`.
|
|
|
|
```ts
|
|
/**
|
|
* One fresh semantic snapshot of the surface under test. The harness owns
|
|
* everything downstream: revision minting, secret redaction, and the
|
|
* observation byte budget apply to every engine equally.
|
|
*/
|
|
export interface EngineSnapshot {
|
|
/**
|
|
* Where the surface is, as an opaque address the platform understands: a
|
|
* URL on a document platform, the foreground screen or activity on a
|
|
* device, the front window on a desktop. The harness shows it to the model
|
|
* and to the report, anchors trace replay on it, and applies the origin
|
|
* policy to it only when it parses as a URL. Omit when the platform has no
|
|
* notion of a current location.
|
|
*/
|
|
readonly location?: string;
|
|
/**
|
|
* The observation tree under one root the engine minted. The root's id MUST
|
|
* be stable for the surface across observations: `perform(root, swipe)` is
|
|
* the viewport swipe, and the harness addresses it from an earlier
|
|
* observation, possibly one taken before a `restart` or `reset`, so the
|
|
* root swipe MUST NOT depend on a live handle from that observation. A
|
|
* platform with several top-level elements (a device's windows) wraps them
|
|
* in a root of its own.
|
|
*/
|
|
readonly root: SemanticNode;
|
|
/** The viewport `SemanticNode.rect` and `ViewportPoint` are measured in, in CSS pixels. */
|
|
readonly viewport: ViewportSize;
|
|
/**
|
|
* True when `root` leaves out nodes that are on the surface: the engine
|
|
* stopped reading at a node cap or a depth limit, or could not enter a
|
|
* child document. The harness tells the model the listing is incomplete
|
|
* and never calls such a screen unchanged. Omit or set false for a tree
|
|
* read whole.
|
|
*/
|
|
readonly truncated?: boolean;
|
|
/**
|
|
* Whether an on-screen keyboard is showing, as the engine measured it apart
|
|
* from `root`. Set `false` when it measured none, and omit it only when the
|
|
* engine did not measure it.
|
|
*/
|
|
readonly keyboardVisible?: boolean;
|
|
/**
|
|
* The tabs the surface has open, when it has more than one, such as a
|
|
* browser with a tab the app opened. Omit with one tab.
|
|
*/
|
|
readonly tabs?: { readonly count: number; readonly active: number };
|
|
/** Semantic capture failed; only fresh, independently masked pixels are evidence. */
|
|
readonly treeUnavailable?: true;
|
|
/** Masked pixels, when requested and producible; omitted otherwise. */
|
|
readonly pixels?: ObservationPixels;
|
|
/** Regions masked in `pixels`; the harness checks it covers every secure node. */
|
|
readonly maskedRegionCount?: number;
|
|
}
|
|
|
|
export interface EngineObserveOptions {
|
|
/** Requests masked viewport pixels for the same revision as the tree. */
|
|
readonly pixels?: boolean;
|
|
/** Permits independently masked pixels to replace a failed semantic capture. */
|
|
readonly pixelFallback?: boolean;
|
|
/** The pixels are compared against a stored screenshot: hold system chrome (a status bar's clock and battery) in a fixed state where the platform can. */
|
|
readonly comparable?: boolean;
|
|
}
|
|
```
|
|
|
|
#### Truncated trees
|
|
|
|
Set `truncated` when a node limit or an inaccessible subtree leaves content
|
|
out. The `truncated` JSDoc above says how the runner treats it.
|
|
|
|
#### Pixel fallback
|
|
|
|
When semantic capture times out, an engine may return masked pixels in place
|
|
of the tree, but only when `pixelFallback` is true and it can prove
|
|
screenshot masking without that capture.
|
|
|
|
```ts
|
|
return {
|
|
treeUnavailable: true,
|
|
root: { ref: rootRef },
|
|
viewport,
|
|
pixels,
|
|
};
|
|
```
|
|
|
|
- Return `treeUnavailable: true`, fresh `pixels`, and a `root` containing
|
|
only its stable `ref`. That ref supports viewport actions; it supplies no
|
|
semantic node evidence. Retire earlier node refs.
|
|
- The flag tells missing semantic evidence from a valid empty tree. Never
|
|
infer absence from it.
|
|
- Without permission, pixels, or independent masking proof, propagate the
|
|
capture failure. Cancellation, policy denial, and hard engine failures
|
|
propagate too.
|
|
- The runner permits fallback for `act` and for judgments with
|
|
`vision: true` or `'only'`. It includes the
|
|
screenshot even if the initial observation requested only a tree.
|
|
- A step that loses semantic evidence cannot record or finish by replaying a
|
|
trace.
|
|
- Engines whose masks depend on their semantic capture cannot provide this
|
|
fallback.
|
|
|
|
`EngineSnapshot` is an extendable interface, so TypeScript can't enforce
|
|
these rules. The runner checks the shape at the engine boundary, at runtime:
|
|
|
|
| Throws | When |
|
|
| --- | --- |
|
|
| `UNSUPPORTED_CAPABILITY` | A fallback without permitted pixels. |
|
|
| `INVALID_STATE` | A fallback whose root carries semantic fields. |
|
|
|
|
### Actions
|
|
|
|
`perform` carries out one `LocatorAction` on a node; `performAt` carries out
|
|
one `PointerAction` at a viewport point. An engine declares the kinds it
|
|
honors in `actions` and `pointerActions`.
|
|
|
|
| Kind | Fields | `actions` | `pointerActions` |
|
|
| --- | --- | --- | --- |
|
|
| `tap` | `modifiers?` on a node | yes | yes |
|
|
| `doubleTap` | `modifiers?` on a node | yes | yes |
|
|
| `secondaryTap` | `modifiers?` on a node | yes | yes |
|
|
| `longPress` | `durationMs?` | yes | yes |
|
|
| `fill` | `value`, `sensitive` | yes | - |
|
|
| `clear` | - | yes | - |
|
|
| `press` | `key` | yes | - |
|
|
| `check` | - | yes | - |
|
|
| `uncheck` | - | yes | - |
|
|
| `focus` | - | yes | - |
|
|
| `hover` | - | yes | yes |
|
|
| `scrollIntoView` | - | yes | - |
|
|
| `selectOption` | `value` | yes | - |
|
|
| `setInputFiles` | `paths`; empty clears the input | yes | - |
|
|
| `dragTo` | `target`: a `NodeRef` or a `ViewportPoint` | yes | yes |
|
|
| `swipe` | `direction`, `momentum?` | yes | yes |
|
|
| `swipeTo` | `target`, `durationMs?` | - | yes |
|
|
|
|
The grammar is closed. Anything beyond it is an agent-side `defineTool`.
|
|
|
|
#### Locator actions
|
|
|
|
```ts
|
|
async perform(ref, action, context) {
|
|
switch (action.kind) {
|
|
case 'tap':
|
|
return client.tap(ref.id, { signal: context.signal });
|
|
case 'swipe':
|
|
return client.scroll(ref.id === rootId ? undefined : ref.id, action.direction);
|
|
default:
|
|
throw new EngineError('UNSUPPORTED_CAPABILITY', `kiosk cannot ${action.kind}`, { retryable: false });
|
|
}
|
|
},
|
|
```
|
|
|
|
```ts
|
|
/**
|
|
* One deterministic action `perform` carries out on a node. The `swipe`
|
|
* kind on the observation's root node is the viewport swipe: the agent's
|
|
* `scroll` verb and `screen.swipe()` both arrive as `perform(root, swipe)`,
|
|
* so a surface implements scrolling once.
|
|
*
|
|
* `press` takes one key in the `Key` grammar below.
|
|
*/
|
|
export type LocatorAction =
|
|
| {
|
|
readonly kind: 'tap' | 'doubleTap' | 'secondaryTap';
|
|
/** Keys held for the click; only an engine that declares `tapModifiers` receives it. */
|
|
readonly modifiers?: readonly KeyModifier[];
|
|
}
|
|
| {
|
|
readonly kind:
|
|
| 'check'
|
|
| 'uncheck'
|
|
| 'clear'
|
|
| 'focus'
|
|
| 'hover'
|
|
| 'scrollIntoView';
|
|
}
|
|
| { readonly kind: 'longPress'; readonly durationMs?: number }
|
|
| { readonly kind: 'fill'; readonly value: string; readonly sensitive: boolean }
|
|
| { readonly kind: 'press'; readonly key: Key }
|
|
| { readonly kind: 'selectOption'; readonly value: SelectOption }
|
|
| { readonly kind: 'setInputFiles'; readonly paths: readonly string[] }
|
|
| { readonly kind: 'dragTo'; readonly target: NodeRef }
|
|
| {
|
|
readonly kind: 'swipe';
|
|
readonly direction: ScrollDirection;
|
|
readonly momentum?: Momentum;
|
|
};
|
|
|
|
/** Every action kind; what an engine declares in `actions`. */
|
|
export type LocatorActionKind = LocatorAction['kind'];
|
|
|
|
/** The closed list of action kinds, in contract order; the type is derived from it. */
|
|
export const LOCATOR_ACTION_KINDS = [
|
|
'tap',
|
|
'doubleTap',
|
|
'secondaryTap',
|
|
'longPress',
|
|
'fill',
|
|
'clear',
|
|
'press',
|
|
'check',
|
|
'uncheck',
|
|
'focus',
|
|
'hover',
|
|
'scrollIntoView',
|
|
'selectOption',
|
|
'setInputFiles',
|
|
'dragTo',
|
|
'swipe',
|
|
] as const satisfies readonly LocatorActionKind[];
|
|
```
|
|
|
|
#### Pointer actions
|
|
|
|
Pointer actions use viewport coordinates when no semantic node is available.
|
|
|
|
- `dragTo` ends at another point.
|
|
- `swipe` starts a scroll gesture at the given point.
|
|
- `swipeTo` is the one kind with no node form: the finger's path from the
|
|
point to `target`, a touch swipe on a device and a pointer drag on a
|
|
document platform. It differs from `dragTo`, where a platform holds before
|
|
it drags.
|
|
|
|
```ts
|
|
export type PointerAction =
|
|
| { readonly kind: 'tap' | 'doubleTap' | 'secondaryTap' | 'hover' }
|
|
| { readonly kind: 'longPress'; readonly durationMs?: number }
|
|
| { readonly kind: 'dragTo'; readonly target: ViewportPoint }
|
|
| {
|
|
readonly kind: 'swipe';
|
|
readonly direction: ScrollDirection;
|
|
readonly momentum?: Momentum;
|
|
}
|
|
| {
|
|
readonly kind: 'swipeTo';
|
|
readonly target: ViewportPoint;
|
|
readonly durationMs?: number;
|
|
};
|
|
|
|
/** Every pointer action kind; what an engine declares in `pointerActions`. */
|
|
export type PointerActionKind = PointerAction['kind'];
|
|
|
|
/** The closed list of pointer action kinds, in contract order; the type is derived from it. */
|
|
export const POINTER_ACTION_KINDS = [
|
|
'tap',
|
|
'doubleTap',
|
|
'secondaryTap',
|
|
'longPress',
|
|
'hover',
|
|
'dragTo',
|
|
'swipe',
|
|
'swipeTo',
|
|
] as const satisfies readonly PointerActionKind[];
|
|
```
|
|
|
|
#### Keys
|
|
|
|
The key grammar of `press`, shared by every engine, and its parser.
|
|
|
|
```ts
|
|
parseKey('Control+a'); // { modifiers: ['Control'], key: { kind: 'char', char: 'a' } }
|
|
parseKey('Shift+Tab'); // { modifiers: ['Shift'], key: { kind: 'named', name: 'Tab' } }
|
|
parseKey('Ctrl+a'); // undefined: Ctrl is not a modifier of the grammar
|
|
```
|
|
|
|
```ts
|
|
/**
|
|
* The key grammar of `press`, shared by every engine so a test's key names are
|
|
* portable: zero or more modifiers and one key, joined by `+`.
|
|
*
|
|
* - Modifiers: `Shift`, `Control`, `Alt`, `Meta`, `ControlOrMeta` (Control on
|
|
* Windows and Linux, Meta on macOS).
|
|
* - Named keys: those in `KEY_NAMES` (`Enter`, `Escape`, `Tab`, `Backspace`,
|
|
* `Delete`, `ArrowUp`..., `Home`, `End`, `PageUp`, `PageDown`, `Insert`,
|
|
* `Space`, `F1`..`F12`).
|
|
* - Any single printable character (`a`, `A`, `1`, `$`), typed as itself.
|
|
*
|
|
* `Control+a`, `Shift+Tab`, `Enter`, `$`. A platform without a key maps it
|
|
* or throws `UNSUPPORTED_CAPABILITY`; it never reinterprets the spelling.
|
|
* `parseKey` is the one parser, exported so no engine carries its own.
|
|
*/
|
|
export type Key = string;
|
|
|
|
export const KEY_MODIFIERS = ['Shift', 'Control', 'Alt', 'Meta', 'ControlOrMeta'] as const;
|
|
|
|
export const KEY_NAMES = [
|
|
'Enter',
|
|
'Escape',
|
|
'Tab',
|
|
'Backspace',
|
|
'Delete',
|
|
'Insert',
|
|
'Space',
|
|
'ArrowUp',
|
|
'ArrowDown',
|
|
'ArrowLeft',
|
|
'ArrowRight',
|
|
'Home',
|
|
'End',
|
|
'PageUp',
|
|
'PageDown',
|
|
'F1',
|
|
'F2',
|
|
'F3',
|
|
'F4',
|
|
'F5',
|
|
'F6',
|
|
'F7',
|
|
'F8',
|
|
'F9',
|
|
'F10',
|
|
'F11',
|
|
'F12',
|
|
] as const;
|
|
|
|
/** One parsed `press` key: its modifiers and either a named key or a single character. */
|
|
export interface ParsedKey {
|
|
/** Modifiers, in the order written. */
|
|
readonly modifiers: readonly KeyModifier[];
|
|
/** The key itself. */
|
|
readonly key: { readonly kind: 'named'; readonly name: KeyName } | { readonly kind: 'char'; readonly char: string };
|
|
}
|
|
|
|
export function parseKey(key: string): ParsedKey | undefined;
|
|
```
|
|
|
|
### App
|
|
|
|
The app under test is declared on the target (`targets: [{ engine, app }]`,
|
|
see [the app under test](/reference/config#the-app-under-test)); an engine
|
|
only drives it. The harness checks the shape of every field. Then it hands
|
|
the declaration to [`validateApp`](#validateapp) at config load.
|
|
|
|
| Target field | Reaches the engine as |
|
|
| --- | --- |
|
|
| `url` | `validateApp` only. The harness opens it through `session.open`. |
|
|
| the origin of `url`, on the run's port | `info.app.origin` in `prepare` and `init` |
|
|
| the site of `url` | `info.app.site` in `prepare` and `init` |
|
|
| `bundleId`, `appPath`, `launchArguments`, `permissions` | `validateApp`, and `info.app` in `prepare` and `init`, as the target declared them |
|
|
| `command`, `readyUrl` | Never. The runner starts the app. |
|
|
|
|
`CommandConfig` is exported from `e2e`, not `e2e/engine`.
|
|
|
|
#### `validateApp`
|
|
|
|
Checks the declared app at config load, synchronously.
|
|
|
|
```ts
|
|
validateApp(app, { targetName }) {
|
|
if (app.url === undefined) {
|
|
throw new ConfigurationError('INVALID_CONFIG', `target "${targetName}" needs app.url`);
|
|
}
|
|
},
|
|
```
|
|
|
|
- It runs for every target the engine serves, with `{}` for a target that
|
|
declares no `app`.
|
|
- `@e2e-dev/web` requires `url`; `@e2e-dev/mobile` requires `bundleId` or
|
|
`appPath` and refuses `url`.
|
|
|
|
| Throws | When |
|
|
| --- | --- |
|
|
| `INVALID_CONFIG` | A field the platform needs is missing, or the engine cannot drive one. Name `info.targetName` in the message. |
|
|
|
|
```ts
|
|
/**
|
|
* The app a target declares, as its engine reads it at config load: what it
|
|
* is and where it is served. `EngineAppInfo` (below) is what `prepare` and
|
|
* `init` receive; the declaration adds the URL, which a surface never gets.
|
|
*/
|
|
export type EngineAppDeclaration = EngineAppInfo & {
|
|
/** Base URL the target opens, as declared; see `TargetApp.url`. */
|
|
readonly url?: string;
|
|
};
|
|
|
|
/** Handed to `Engine.validateApp` beside the declaration. */
|
|
export interface EngineAppCheckInfo {
|
|
/** The target declaring the app, for the message of a refusal. */
|
|
readonly targetName: string;
|
|
}
|
|
|
|
/** A permission's state when the app launches: held, refused, or not asked for yet, so the OS asks again. */
|
|
export type AppPermissionState = 'grant' | 'deny' | 'reset';
|
|
```
|
|
|
|
#### `EngineAppInfo`
|
|
|
|
The resolved app, as `EnginePrepareInfo.app` and `EngineInitInfo.app`: the
|
|
origin and site of the URL, and the device fields as the target declared
|
|
them. Scope a credential the engine injects to the exact origin. Apply the
|
|
site to observed frames with [`sameSite`](#site-helpers).
|
|
|
|
```ts
|
|
if (new URL(request.url).origin === info.app.origin) request.headers.set('x-api-key', key);
|
|
```
|
|
|
|
```ts
|
|
/**
|
|
* The app the harness resolved from the target's declaration, handed to
|
|
* `prepare` and `init`: the origin and site policy for the decisions only a
|
|
* surface can make (which requests carry injected credentials, which nested
|
|
* documents enter an observation), and what a device launches.
|
|
*/
|
|
export interface EngineAppInfo {
|
|
/**
|
|
* The app URL's origin on the run's port (`scheme://host:port`, as
|
|
* `URL.origin` serializes it): where injected request credentials go.
|
|
* Absent when the app has no URL, and for a free port the run has not
|
|
* assigned (`e2e doctor`).
|
|
*/
|
|
readonly origin?: string;
|
|
/**
|
|
* The app URL's registrable domain, as read by `siteOf`. Use `sameSite`
|
|
* to scope observed child frames. Absent when the app has no URL. The
|
|
* runner passes navigation URLs separately through `session.open`.
|
|
*/
|
|
readonly site?: string;
|
|
/** The installed app to launch, as the target declared it. */
|
|
readonly bundleId?: string;
|
|
/** The build to install, as the target declared it; relative to `projectRoot`. */
|
|
readonly appPath?: string;
|
|
/** Arguments every fresh launch passes the app. */
|
|
readonly launchArguments?: readonly string[];
|
|
/** Permissions the app holds, by name; what each state means is the engine's (a device sets them before every fresh launch). */
|
|
readonly permissions?: Readonly<Record<string, AppPermissionState>>;
|
|
}
|
|
```
|
|
|
|
### Session
|
|
|
|
Steering hooks behind the universal `app` fixture and the agent's `navigate`,
|
|
`back`, `resize_viewport`, `switch_tab`, and `close_tab` verbs.
|
|
|
|
| Hook | Behind |
|
|
| --- | --- |
|
|
| `open(url)` | `app.open()`, the agent's `navigate` verb, the reopen after `restart` and `reset` |
|
|
| `openLink(url)` | `app.open(path)` on a target with `app.linkBase`: the joined link, its scheme checked, opened in the app under test. A target can declare `linkBase` only when its engine has this hook. |
|
|
| `back()` | `app.back()`, the agent's `back` verb |
|
|
| `resize(size)` | the agent's `resize_viewport` verb; `size` is `{ width, height }` in CSS pixels and holds for the rest of the attempt |
|
|
| `switchTab(index)` | the agent's `switch_tab` verb; `index` is zero-based in the order the tabs opened, as [`tabs`](#observation) counts them. An index with no open tab is `ACTION_FAILED`. |
|
|
| `closeTab(index)` | the agent's `close_tab` verb; `index` counts as for `switchTab`. Closing the tab shown shows another, such as the tab that opened it. An index with no open tab, or the last open tab, is `ACTION_FAILED`. |
|
|
| `restart()` | `app.restart()` |
|
|
| `reset()` | `app.clearState()` |
|
|
|
|
```ts
|
|
session: {
|
|
async back(context) { await client.back({ signal: context.signal }); },
|
|
async restart(context) { await client.relaunch(bundleId, { signal: context.signal }); },
|
|
},
|
|
```
|
|
|
|
```ts
|
|
/**
|
|
* Steering hooks behind the universal `app` fixture and the agent's
|
|
* `navigate` verb. The app itself is the target's `app`, handed over as data;
|
|
* node actions never live here, they are `perform`.
|
|
*
|
|
* `restart` and `reset` open nothing on an addressable surface: they end at
|
|
* a fresh surface with no location shown, and the harness reopens the app
|
|
* through `open` when the target has an address. A surface therefore never
|
|
* needs to know the app's URL, and an unreachable app after a restart is
|
|
* reported the same way as on first open. A surface without `open` (a
|
|
* device with a pinned app) relaunches the app inside `restart` and `reset`
|
|
* itself, since nothing else will.
|
|
*/
|
|
export interface EngineSession {
|
|
/**
|
|
* Opens one URL the harness already resolved against the base URL and the
|
|
* origin policy. Absent on a surface without addressable locations.
|
|
*/
|
|
open?(url: string, context: OperationContext): Promise<void>;
|
|
/**
|
|
* Opens a link in the app under test, as a deep link or a universal link
|
|
* would: `app.open(path)` on a target that declares `app.linkBase` lands
|
|
* here with the link joined and its scheme checked. A device engine's hook;
|
|
* a target whose engine lacks it cannot declare `linkBase`.
|
|
*/
|
|
openLink?(url: string, context: OperationContext): Promise<void>;
|
|
/** Navigates back once in the surface's history. */
|
|
back?(context: OperationContext): Promise<void>;
|
|
/** Sets the viewport to `size` in CSS pixels, for this page and every page the attempt opens after it. */
|
|
resize?(size: ViewportSize, context: OperationContext): Promise<void>;
|
|
/**
|
|
* Shows the open tab at `index`, zero-based in the order the tabs opened,
|
|
* as `EngineSnapshot.tabs` counts them: every observation and action after
|
|
* it reads and acts on that tab. An index with no open tab is an
|
|
* `ACTION_FAILED` the model reads. Declare it with `tabs` in snapshots.
|
|
*/
|
|
switchTab?(index: number, context: OperationContext): Promise<void>;
|
|
/**
|
|
* Closes the open tab at `index`, counted as `switchTab` counts them. When
|
|
* it was the tab shown, the engine shows another, such as the tab that
|
|
* opened it. An index with no open tab, or the last open tab, is an
|
|
* `ACTION_FAILED` the model reads.
|
|
*/
|
|
closeTab?(index: number, context: OperationContext): Promise<void>;
|
|
/** Recreates the execution context, keeping persisted state. */
|
|
restart?(context: OperationContext): Promise<void>;
|
|
/** Clears persisted client state and recreates the execution context. */
|
|
reset?(context: OperationContext): Promise<void>;
|
|
}
|
|
```
|
|
|
|
### Errors
|
|
|
|
An engine throws `EngineError` with one code of a closed set. The runner maps
|
|
each code to the error a test sees.
|
|
|
|
| Code | Runner mapping | Retryable |
|
|
| --- | --- | --- |
|
|
| `NODE_STALE` | re-resolve, then `LOCATOR_NOT_FOUND` | yes |
|
|
| `FRAME_NOT_FOUND` | re-resolve, then `LOCATOR_NOT_FOUND` | yes |
|
|
| `FRAME_AMBIGUOUS` | `LOCATOR_AMBIGUOUS` | no |
|
|
| `NOT_ACTIONABLE`, `ACTION_MAY_HAVE_COMMITTED`, `OPERATION_TIMEOUT` | `ACTION_FAILED` | no |
|
|
| `CANCELLED` | infrastructure `CANCELLED` | no |
|
|
| `UNSUPPORTED_CAPABILITY` | configuration `UNSUPPORTED_CAPABILITY` | no |
|
|
| `INVALID_STATE` | `APP_NOT_OPEN` | no |
|
|
| `ENGINE_FAILURE`, any non-`EngineError` | infrastructure `ENGINE_FAILURE` | no |
|
|
|
|
#### `EngineError`
|
|
|
|
```ts
|
|
throw new EngineError('NODE_STALE', `node ${ref.id} is gone`, { retryable: true });
|
|
```
|
|
|
|
```ts
|
|
export class EngineError extends Error {
|
|
constructor(code: EngineErrorCode, message: string, options: { retryable: boolean; cause?: unknown });
|
|
/** One of `ENGINE_ERROR_CODES`. */
|
|
readonly code: EngineErrorCode;
|
|
/** Whether the runner may re-resolve and retry; only `RETRYABLE_ENGINE_ERROR_CODES` can be. */
|
|
readonly retryable: boolean;
|
|
}
|
|
|
|
/** The closed engine error code set; the type is derived from it, so the two cannot drift. */
|
|
export const ENGINE_ERROR_CODES = [
|
|
'NODE_STALE',
|
|
'FRAME_NOT_FOUND',
|
|
'FRAME_AMBIGUOUS',
|
|
'NOT_ACTIONABLE',
|
|
'ACTION_MAY_HAVE_COMMITTED',
|
|
'OPERATION_TIMEOUT',
|
|
'CANCELLED',
|
|
'UNSUPPORTED_CAPABILITY',
|
|
'INVALID_STATE',
|
|
'ENGINE_FAILURE',
|
|
] as const;
|
|
|
|
/** One of `ENGINE_ERROR_CODES`. */
|
|
export type EngineErrorCode = (typeof ENGINE_ERROR_CODES)[number];
|
|
|
|
/**
|
|
* The only codes that may be retryable: both describe a repeatable read. Any
|
|
* other retryable claim, from this module's class or a foreign copy of it, is
|
|
* coerced to a non-retryable `ENGINE_FAILURE`.
|
|
*/
|
|
export const RETRYABLE_ENGINE_ERROR_CODES: ReadonlySet<EngineErrorCode> = new Set([
|
|
'NODE_STALE',
|
|
'FRAME_NOT_FOUND',
|
|
]);
|
|
```
|
|
|
|
#### Timeouts at a deadline
|
|
|
|
An `OPERATION_TIMEOUT` on a locate or read that started with under 100 ms, or
|
|
under its slowest completed locate or read, of a waiting call's deadline left,
|
|
after one that completed, ends the wait
|
|
instead:
|
|
|
|
| Waiting call | Fails with |
|
|
| --- | --- |
|
|
| an action, `waitFor` | `LOCATOR_NOT_FOUND`, with the timeout as its cause |
|
|
| an assertion | `ASSERTION_FAILED`, with the timeout as its cause |
|
|
|
|
A negated assertion passes there if its negation had held long enough by the
|
|
time the read was cut off. [`pollCondition`](#pollcondition) polls by these
|
|
rules.
|
|
|
|
#### Runner error classes
|
|
|
|
An `EngineError` a contributed fixture method throws is mapped by the table
|
|
above, so a fixture over a session that ended reports `APP_NOT_OPEN` the way
|
|
an observation would. A method can also fail a step with the runner's own
|
|
error classes, exported from the same entry.
|
|
|
|
```ts
|
|
throw new TestError('ASSERTION_FAILED', 'expected the receipt to print');
|
|
```
|
|
|
|
`E2EError` itself is not exported; construct one of the three.
|
|
|
|
```ts
|
|
/** The runner's error base: `category` sets the exit code, `code` names the cause. */
|
|
declare class E2EError extends Error {
|
|
readonly category: 'test' | 'configuration' | 'infrastructure' | 'internal' | 'interrupted';
|
|
readonly code: string;
|
|
readonly retryable: boolean;
|
|
}
|
|
|
|
/** A test failure: ASSERTION_FAILED, INVALID_LOCATOR, ACTION_FAILED, ... */
|
|
export class TestError extends E2EError {
|
|
constructor(code: string, message: string, options?: { retryable?: boolean; cause?: unknown });
|
|
}
|
|
/** POLICY_DENIED, APP_URL_REQUIRED, UNSUPPORTED_CAPABILITY, ... */
|
|
export class ConfigurationError extends E2EError {
|
|
constructor(code: string, message: string, options?: { cause?: unknown });
|
|
}
|
|
/** The surface itself failed: BROWSER_INSTALL_FAILED, ... */
|
|
export class InfrastructureError extends E2EError {
|
|
constructor(code: string, message: string, options?: { cause?: unknown });
|
|
}
|
|
```
|
|
|
|
### Shared semantics
|
|
|
|
Helpers that keep behavior consistent across engines. Every engine
|
|
reproduces these semantics; import them instead of carrying a copy.
|
|
|
|
| Export | Returns |
|
|
| --- | --- |
|
|
| [`matchesText`, `toTextPattern`, `describePattern`](#text-patterns) | `boolean`, `TextPattern`, `string` |
|
|
| [`resolveExpression`](#resolveexpression) | `SemanticNode[]` |
|
|
| [`urlMatches`](#urlmatches) | `boolean` |
|
|
| [`siteOf`, `sameSite`](#site-helpers) | `string`, `boolean` |
|
|
| [`isRefusedLinkScheme`](#isrefusedlinkscheme) | `boolean` |
|
|
| [`pollCondition`, `Deadline`](#pollcondition) | `Promise<void>`, `Deadline` |
|
|
| [`validateJsonValue`](#validatejsonvalue) | `void` |
|
|
| [`rejectUnknownKeys`](#rejectunknownkeys) | `void` |
|
|
| [`rejectUnknownOptions`](#rejectunknownoptions) | `void` |
|
|
| [`obj`](#obj) | `WithoutUndefined<T>` |
|
|
| [`raceAbort`, `withTimeout`, `withinCleanupBudget`](#timing-helpers) | `Promise<T>`, `Promise<T>`, `Promise<void>` |
|
|
| [`isSecret`](#issecret) | `boolean` |
|
|
| [`isProviderRecording`, `stopProviderRecording`](#provider-recordings) | `boolean`, `Promise<VideoSegment>` |
|
|
|
|
#### Text patterns
|
|
|
|
`TextPattern` matching for `locate` and text matchers.
|
|
|
|
```ts
|
|
export function toTextPattern(match: TextMatch, options?: { exact?: boolean }): TextPattern;
|
|
export function matchesText(actual: string, pattern: TextPattern): boolean;
|
|
export function describePattern(pattern: TextPattern): string;
|
|
```
|
|
|
|
```ts
|
|
const pattern = toTextPattern(/total/i);
|
|
matchesText('Order total', pattern); // true
|
|
describePattern(pattern); // '/total/i'
|
|
```
|
|
|
|
| Throws | When |
|
|
| --- | --- |
|
|
| `INVALID_ARGUMENT` | `toTextPattern` gets a `match` that is neither a string nor a `RegExp`. |
|
|
|
|
#### `resolveExpression`
|
|
|
|
The reference `locate`. Resolves one `LocatorExpression` over a semantic
|
|
tree, in document order.
|
|
|
|
```ts
|
|
export function resolveExpression(
|
|
expression: LocatorExpression,
|
|
nodes: readonly SemanticNode[],
|
|
options?: ResolveExpressionOptions,
|
|
): SemanticNode[];
|
|
```
|
|
|
|
```ts
|
|
async locate(expression) {
|
|
const screen = await client.snapshot();
|
|
return resolveExpression(expression, screen.nodes);
|
|
},
|
|
```
|
|
|
|
- `text` and `label` answer with the innermost match when a container echoes
|
|
a descendant's text.
|
|
- An engine whose tree is the whole truth (a device) calls it; a browser
|
|
engine reproduces it.
|
|
- A `selector` expression needs the `selector` option.
|
|
- A `frame` expression is `FRAME_NOT_FOUND`.
|
|
- A `union` expression matches either branch in document order, without duplicate nodes.
|
|
|
|
#### `urlMatches`
|
|
|
|
`toHaveURL`-style comparison of a current URL against an expected one.
|
|
|
|
```ts
|
|
export function urlMatches(
|
|
current: string,
|
|
expected: string | RegExp,
|
|
baseHref: string,
|
|
ignoreCase?: boolean,
|
|
): boolean;
|
|
```
|
|
|
|
```ts
|
|
urlMatches('https://shop.test/cart', '/cart', 'https://shop.test/'); // true
|
|
```
|
|
|
|
`ignoreCase` works as on the locator matchers.
|
|
|
|
| Throws | When |
|
|
| --- | --- |
|
|
| `INVALID_ARGUMENT` | `expected` is neither a string nor a `RegExp`, as in `toTextPattern`. |
|
|
|
|
#### Site helpers
|
|
|
|
`siteOf` reads a hostname's registrable domain, the form
|
|
`EngineAppInfo.site` takes. `sameSite` tells whether a URL is on a site.
|
|
|
|
```ts
|
|
export function siteOf(hostname: string): string;
|
|
export function sameSite(url: string | URL, site: string): boolean;
|
|
```
|
|
|
|
#### `isRefusedLinkScheme`
|
|
|
|
Whether a link's scheme is one no app link may take: `file:`, `data:`,
|
|
`javascript:`, `view-source:`, `blob:`, or `filesystem:`. An engine that
|
|
opens app links (`session.openLink`) refuses these with `POLICY_DENIED`, as
|
|
`app.open` does with `app.linkBase`.
|
|
|
|
```ts
|
|
export function isRefusedLinkScheme(protocol: string): boolean;
|
|
```
|
|
|
|
```ts
|
|
if (isRefusedLinkScheme(new URL(link).protocol)) throw new ConfigurationError('POLICY_DENIED', 'refused link scheme');
|
|
```
|
|
|
|
#### `pollCondition`
|
|
|
|
Assertion polling and the negation grace window.
|
|
|
|
```ts
|
|
export function pollCondition(options: PollConditionOptions): Promise<void>;
|
|
```
|
|
|
|
```ts
|
|
const deadline = new Deadline(context.timeouts.assertion);
|
|
await pollCondition({
|
|
deadline,
|
|
signal: context.signal,
|
|
negated: false,
|
|
evaluate: async () => (await read(context.operation(deadline.remaining()))) === 'Paid',
|
|
onTimeout: (cause) => new TestError('ASSERTION_FAILED', 'status never became Paid', { cause }),
|
|
});
|
|
```
|
|
|
|
- Bound each read `evaluate` makes by `deadline`
|
|
(`context.operation(deadline.remaining())`). A read on the action timeout
|
|
can outlast the assertion's own.
|
|
- An `OPERATION_TIMEOUT` from an `evaluate` ends the poll under the rules in
|
|
[Timeouts at a deadline](#timeouts-at-a-deadline). `onTimeout(cause)`
|
|
builds the failure with it.
|
|
|
|
#### `validateJsonValue`
|
|
|
|
Checks that a value is plain JSON.
|
|
|
|
```ts
|
|
export function validateJsonValue(value: unknown, label: string, rules?: JsonValueRules): void;
|
|
```
|
|
|
|
```ts
|
|
validateJsonValue(result, 'kiosk.evaluate result', { maxDepth: 32 });
|
|
```
|
|
|
|
- A top-level `undefined` passes.
|
|
- `rules.onSecret` replaces the `POLICY_DENIED` a `Secret` raises with an
|
|
error of its own; `rules.maxDepth` caps the nesting.
|
|
|
|
| Throws | When |
|
|
| --- | --- |
|
|
| `INVALID_ARGUMENT` | A non-finite number, a cycle, an object that is not a plain object or array, or a function, symbol, bigint, or `undefined` inside. The message names `label`. |
|
|
| `POLICY_DENIED` | A `Secret`, unless `rules.onSecret` throws its own. |
|
|
|
|
#### `rejectUnknownKeys`
|
|
|
|
Throws for the first own key of `value` outside `keys`: the check the runner
|
|
runs on its own config objects, for an engine factory's options.
|
|
|
|
```ts
|
|
export function rejectUnknownKeys(
|
|
label: string,
|
|
value: object,
|
|
keys: readonly string[],
|
|
code?: 'INVALID_CONFIG' | 'INVALID_ARGUMENT', // default: 'INVALID_CONFIG'
|
|
): void;
|
|
```
|
|
|
|
```ts
|
|
rejectUnknownKeys('web()', options, ['browser', 'viewport']);
|
|
// web() has unknown key "viewprt"; did you mean "viewport"?
|
|
```
|
|
|
|
- The message names the nearest known key when one is a plausible typo, and
|
|
the known keys otherwise.
|
|
- Pass `'INVALID_ARGUMENT'` as `code` for an object a test passes to a
|
|
fixture method.
|
|
|
|
#### `rejectUnknownOptions`
|
|
|
|
Refuses an option key a fixture method or matcher does not take, so a
|
|
JavaScript caller's unsupported option fails instead of running on the
|
|
default.
|
|
|
|
```ts
|
|
export function rejectUnknownOptions(
|
|
api: string,
|
|
options: object | undefined,
|
|
known: readonly string[],
|
|
code?: 'INVALID_ARGUMENT' | 'INVALID_LOCATOR', // default: 'INVALID_ARGUMENT'
|
|
): void;
|
|
```
|
|
|
|
```ts
|
|
rejectUnknownOptions('kiosk.print', options, ['copies']);
|
|
```
|
|
|
|
- Pass `INVALID_LOCATOR` as `code` for a locator option.
|
|
- `options` must be a plain object or `undefined`; a class instance or an
|
|
array is refused.
|
|
- Non-enumerable own keys count as keys.
|
|
|
|
#### `obj`
|
|
|
|
Drops `undefined`-valued keys, so a result built from optional inputs
|
|
satisfies exact optional property types.
|
|
|
|
```ts
|
|
obj({ a: 1, b: undefined, c: 3 }); // { a: 1, c: 3 }
|
|
```
|
|
|
|
#### Timing helpers
|
|
|
|
```ts
|
|
export function raceAbort<T>(work: Promise<T> | (() => Promise<T>), signal: AbortSignal, label: string): Promise<T>;
|
|
export function withTimeout<T>(promise: Promise<T>, timeoutMs: number, onTimeout: () => Error): Promise<T>;
|
|
export function stepTimeout(timeout: unknown, api: string): number | undefined;
|
|
export function withinCleanupBudget(
|
|
promise: Promise<unknown>,
|
|
budget: { readonly signal: AbortSignal; readonly timeoutMs: number },
|
|
): Promise<void>;
|
|
```
|
|
|
|
```ts
|
|
await raceAbort(() => client.tap(ref.id), context.signal, 'kiosk tap');
|
|
const operation = context.operation(stepTimeout(options?.timeout, 'kiosk.print'));
|
|
```
|
|
|
|
- `raceAbort` abandons the wait when `signal` aborts. Pass a thunk to check
|
|
cancellation before dispatch.
|
|
- `withTimeout` races a promise against a timeout and throws what
|
|
`onTimeout` builds.
|
|
- `stepTimeout` turns a call's own `timeout` option into a budget for
|
|
`context.operation`: `0` waits until the test timeout, or `cleanupTimeout` in `afterEach`, and a negative,
|
|
`NaN`, or infinite one is `INVALID_ARGUMENT`.
|
|
- `withinCleanupBudget` never rejects. It resolves when the promise settles,
|
|
failures included, when `signal` aborts, or when `timeoutMs` passes.
|
|
|
|
#### `isSecret`
|
|
|
|
Tells a `secrets.get()` handle from a string, for an option that takes
|
|
either.
|
|
|
|
```ts
|
|
const password = isSecret(options.password)
|
|
? await context.resolveSecret(options.password)
|
|
: options.password;
|
|
```
|
|
|
|
### State
|
|
|
|
Platform-neutral state capture and restore, behind `test.setup` sessions and
|
|
the `session:` option.
|
|
|
|
| Member | Does |
|
|
| --- | --- |
|
|
| `capture(context)` | Captures the surface's persisted state as an `EngineState`. |
|
|
| `restore(state, context)` | Replaces the surface's whole persisted state with `state`. |
|
|
|
|
```ts
|
|
state: {
|
|
async capture(context) {
|
|
return { format: 'kiosk-storage', version: 1, data: await client.exportStorage(context) };
|
|
},
|
|
async restore(state, context) {
|
|
await client.replaceStorage(state.data, context);
|
|
},
|
|
},
|
|
```
|
|
|
|
<Warning>
|
|
Snapshots are credentials: cookies, tokens, and storage that authenticate
|
|
the app under test. An engine must not persist, cache, or log a snapshot on
|
|
its own, must not echo snapshot contents in an error message, and must treat
|
|
`restore` as a replacement, never a merge, so a restored session cannot leak
|
|
into the next attempt.
|
|
</Warning>
|
|
|
|
```ts
|
|
/**
|
|
* An opaque, restorable snapshot of an engine's state. `data` is JSON the
|
|
* harness never inspects - a document platform's storage state, a device's app state, a
|
|
* desktop's window state all satisfy it identically. `format`/`version` let a
|
|
* engine reject a snapshot it can no longer read.
|
|
*/
|
|
export interface EngineState {
|
|
/** Engine-defined format name. */
|
|
readonly format: string;
|
|
/** Format version, for rejecting a snapshot the engine can no longer read. */
|
|
readonly version: number;
|
|
/** Opaque JSON the runner never inspects. */
|
|
readonly data: unknown;
|
|
/** Earliest moment the state is known to be invalid, if the engine knows one. */
|
|
readonly expiresAt?: string;
|
|
}
|
|
|
|
/**
|
|
* Platform-neutral state capture and restore. A setup test captures a
|
|
* snapshot; an ordinary test restores it at launch.
|
|
*
|
|
* Snapshots are credentials: cookies, tokens, and storage that authenticate
|
|
* the app under test. The duties split at the seam. The harness owns the
|
|
* snapshot from the moment `capture` returns: it encrypts it at rest with a
|
|
* per-run AES-256-GCM key that never touches disk, stores it under the run's
|
|
* private session directory, binds it to the run, target, engine, and app
|
|
* identity, and deletes it when the run ends; it never logs it, reports it,
|
|
* digests it, or sends it to a model. An engine MUST NOT persist, cache, or
|
|
* log a snapshot on its own, MUST NOT echo snapshot contents in an error
|
|
* message, and MUST treat `restore` as a replacement of the surface's whole
|
|
* persisted state, never a merge, so a restored session cannot leak into the
|
|
* next attempt.
|
|
*/
|
|
export interface EngineStateCapability {
|
|
/** Captures the surface's persisted state. */
|
|
capture(context: OperationContext): Promise<EngineState>;
|
|
/** Replaces the surface's persisted state with `state`. */
|
|
restore(state: EngineState, context: OperationContext): Promise<void>;
|
|
}
|
|
```
|
|
|
|
### Artifacts
|
|
|
|
Evidence capture: screenshots under the attempt directory, and videos there
|
|
or at a provider.
|
|
|
|
| Member | Returns |
|
|
| --- | --- |
|
|
| `screenshot(label, context)` | `Promise<string>` |
|
|
| [`startVideo(context)`](#videos) | `Promise<void>` |
|
|
| [`stopVideo(context)`](#videos) | `Promise<readonly VideoSegment[]>` |
|
|
|
|
`screenshot` is required and returns the path of the file it wrote.
|
|
|
|
```ts
|
|
artifacts: {
|
|
async screenshot(_label, context) {
|
|
// The label comes from test code, so it never becomes part of the path.
|
|
shots += 1;
|
|
const path = `screenshots/${String(shots).padStart(3, '0')}.png`;
|
|
await client.capture(join(artifactsDir, path), context);
|
|
return path;
|
|
},
|
|
},
|
|
```
|
|
|
|
```ts
|
|
/**
|
|
* Evidence capture. Paths are relative to the attempt artifact directory the
|
|
* engine received in `startAttempt`.
|
|
*/
|
|
export interface EngineArtifacts {
|
|
/** Captures a redacted screenshot; secure fields are masked at the source. */
|
|
screenshot(label: string | undefined, context: OperationContext): Promise<string>;
|
|
/**
|
|
* Starts recording the surface. Declared together with `stopVideo`. An
|
|
* attempt may start and stop more than once, never overlapping. A surface that has nothing to show yet (no page open) may
|
|
* defer the actual capture to the moment it does; the segments returned by
|
|
* `stopVideo` say when each one began.
|
|
*/
|
|
startVideo?(context: OperationContext): Promise<void>;
|
|
/**
|
|
* Stops recording and returns every segment written, in order. One segment
|
|
* is the common case; a surface whose recording is bound to a page returns
|
|
* one per page the attempt showed (a restart or a state reset opens a new
|
|
* one). An attempt that never showed anything returns none.
|
|
*/
|
|
stopVideo?(context: OperationContext): Promise<readonly VideoSegment[]>;
|
|
}
|
|
|
|
/**
|
|
* One recorded video of an attempt: a file the engine wrote into the attempt
|
|
* artifact directory, or a recording a hosted service keeps and links to.
|
|
*/
|
|
export type VideoSegment = VideoFile | VideoLink;
|
|
|
|
/** A video file in the attempt artifact directory. */
|
|
export interface VideoFile {
|
|
/** Path relative to the attempt artifact directory. */
|
|
readonly path: string;
|
|
/** When the segment started recording, as an ISO timestamp; its first frame is at or just after it. */
|
|
readonly startedAt: string;
|
|
}
|
|
|
|
/** A recording a hosted service keeps, by URL: the report links to it and the runner never downloads it. */
|
|
export interface VideoLink {
|
|
/** `http(s)` URL of the recording. */
|
|
readonly url: string;
|
|
/** Media type of what the URL serves: `video/mp4`, or `text/html` for a player page. */
|
|
readonly mediaType: string;
|
|
/** When the recording started, as an ISO timestamp; its first frame is at or just after it. */
|
|
readonly startedAt: string;
|
|
}
|
|
```
|
|
|
|
#### Videos
|
|
|
|
The runner calls `startVideo` for an attempt whose
|
|
[video mode](/reference/config#video) records, and for each
|
|
[`app.startRecording()`](/reference/app#startrecording), so one attempt may
|
|
start and stop several times. It moves the files a stop returns before the
|
|
next start, and never starts while a recording runs.
|
|
|
|
- `startVideo` runs after `startAttempt` and any session restore, within the
|
|
launch budget.
|
|
- `stopVideo` runs before `endAttempt`, within the cleanup budget.
|
|
- When the video mode is set explicitly, a `startVideo` that fails fails the
|
|
launch and a `stopVideo` that fails fails cleanup. With the default mode, a
|
|
failure of either drops the video quietly.
|
|
- The runner decides which recordings are kept and deletes a file an attempt
|
|
does not keep.
|
|
|
|
#### Provider recordings
|
|
|
|
An engine that leases its browser or device from a hosted service can let the
|
|
service record the attempt in place of its own capture. That is how
|
|
[`BrowserProvider.record`](/browser#recording-on-the-provider) and
|
|
[`DeviceProvider.record`](/mobile#hosted-devices) work. `e2e/engine` exports
|
|
the shared shape and `stopProviderRecording`, so every engine turns a
|
|
provider's recording into a `VideoSegment` the same way.
|
|
|
|
```ts
|
|
async stopVideo(context) {
|
|
if (recording === undefined) return [];
|
|
return [await stopProviderRecording(recording, { artifactsDir, provider: 'browser provider "kernel"', leaseId, signal: context.signal })];
|
|
},
|
|
```
|
|
|
|
- `isProviderRecording(value)` tells whether what a provider's `record`
|
|
resolved to is a recording: a parseable `startedAt` and a `stop`. Check it
|
|
before keeping one, so a malformed recording fails the start, named,
|
|
instead of the stop.
|
|
- `stopProviderRecording` creates the attempt's `video` directory and calls
|
|
`stop` with it. A file must be a plain name that exists in that directory;
|
|
a link must be an `http(s)` URL with a media type.
|
|
- It returns `{ path: 'video/<file>', startedAt }` or
|
|
`{ url, mediaType, startedAt }` for `stopVideo` to hand back.
|
|
- The target argument's type is exported as `ProviderRecordingTarget`.
|
|
|
|
| Throws | When |
|
|
| --- | --- |
|
|
| `ENGINE_FAILURE` | `stop` throws or returns anything else. The message names the provider and the lease. |
|
|
|
|
```ts
|
|
/** Handed to a provider's `record`, once per attempt that records video. */
|
|
export interface ProviderRecordContext {
|
|
readonly runId: string;
|
|
readonly targetName: string;
|
|
/** The attempt the recording covers. */
|
|
readonly attemptId: string;
|
|
/** The run's environment, the same the provider's `acquire` saw. */
|
|
readonly env: Readonly<Record<string, string | undefined>>;
|
|
/** Aborts when the attempt is cancelled or its launch budget is spent. */
|
|
readonly signal: AbortSignal;
|
|
}
|
|
|
|
/** A recording a provider's service makes of the browser or device it leased, as `record` started it. */
|
|
export interface ProviderRecording {
|
|
/** When the recording started, as an ISO timestamp; its first frame is at or just after it. */
|
|
readonly startedAt: string;
|
|
stop(context: ProviderRecordingStopContext): Promise<ProviderRecordingResult>;
|
|
}
|
|
|
|
/** Handed to `ProviderRecording.stop`, once per recording. */
|
|
export interface ProviderRecordingStopContext {
|
|
/** Absolute directory, already created, a recording written as a file goes into. */
|
|
readonly dir: string;
|
|
/** Aborts when the cleanup budget is spent. */
|
|
readonly signal: AbortSignal;
|
|
}
|
|
|
|
/**
|
|
* What a stopped recording became: a file the provider wrote into the
|
|
* directory it was given, by name, or a recording its service keeps, by
|
|
* `http(s)` URL and media type.
|
|
*/
|
|
export type ProviderRecordingResult =
|
|
| { readonly file: string }
|
|
| { readonly url: string; readonly mediaType: string };
|
|
|
|
export function isProviderRecording(value: unknown): value is ProviderRecording;
|
|
|
|
export function stopProviderRecording(
|
|
recording: ProviderRecording,
|
|
target: {
|
|
/** Absolute attempt artifact directory; a file recording goes into its `video` directory. */
|
|
readonly artifactsDir: string;
|
|
/** The provider as error messages name it: `browser provider "kernel"`. */
|
|
readonly provider: string;
|
|
/** The lease the recording covers, named in error messages. */
|
|
readonly leaseId: string;
|
|
readonly signal: AbortSignal;
|
|
},
|
|
): Promise<VideoSegment>;
|
|
```
|
|
|
|
#### Redaction
|
|
|
|
- Videos are unmasked and carry `redaction: 'incomplete'`.
|
|
- A `download` an engine attaches is the app's bytes. When the session knows
|
|
a secret value, the runner rewrites a text one (`.txt`, `.csv`, `.json`,
|
|
`.html`) through the ledger and labels it `complete`.
|
|
- Every other download keeps its bytes and carries
|
|
`redaction: 'incomplete'`.
|
|
|
|
### Contributed fixtures
|
|
|
|
An engine contributes named surfaces to the test fixtures through
|
|
`fixtures`. Each key becomes a fixture and a capability name.
|
|
|
|
```ts title="src/index.ts"
|
|
fixtures: {
|
|
printer: (context) =>
|
|
context.fixture(
|
|
'printer',
|
|
{
|
|
async print(text: string): Promise<void> {
|
|
await client.print(text, context.operation());
|
|
},
|
|
},
|
|
{ print: { kind: 'resource', label: (text) => text } },
|
|
),
|
|
},
|
|
```
|
|
|
|
```ts title="tests/receipt.e2e.ts"
|
|
import { test as base } from 'e2e';
|
|
|
|
const test = base.extend<{ printer: { print(text: string): Promise<void> } }>();
|
|
|
|
test('prints a receipt', { requires: ['printer'] }, async ({ printer }) => {
|
|
await printer.print('Order 42');
|
|
});
|
|
```
|
|
|
|
The factory runs once per attempt with an `EngineFixtureContext`:
|
|
|
|
| Member | Use |
|
|
| --- | --- |
|
|
| `targetName` | The target the fixture serves. |
|
|
| [`fixture(name, surface, operations)`](#declaring-operations) | Declares which methods are recorded steps. |
|
|
| `app` | The resolved app, plus `resolveUrl(url)`. |
|
|
| `timeouts` | The `test`, `action`, and `assertion` budgets. |
|
|
| `signal` | The running phase's signal. Read it per call. |
|
|
| `operation(timeoutMs?)` | A per-call `OperationContext`. |
|
|
| `attachArtifact(kind, relativePath)` | Registers a file the current step produced. |
|
|
| `attachViewport(viewport)` | Records the viewport the current step established. |
|
|
| `locator(expression)` | Mints a public locator from a raw expression. |
|
|
| `screen(scope)` | Mints a screen scope. |
|
|
| [`expectable(target, factory)`](#expectations) | Makes `expect(fixture)` return matchers. |
|
|
|
|
| Throws | When |
|
|
| --- | --- |
|
|
| `INVALID_CONFIG` | The factory returns a plain surface, not one declared through `context.fixture`, the first time a test reaches for it. |
|
|
| `APP_URL_REQUIRED` | `app.resolveUrl` on a target that declares no URL. |
|
|
| `POLICY_DENIED` | `app.resolveUrl` with any scheme but `http:` and `https:` (`about:blank` aside). |
|
|
|
|
```ts
|
|
/**
|
|
* A contributed fixture: any record of async methods, sync accessors, and
|
|
* nested namespaces, declared through context.fixture so its metadata controls
|
|
* labels, deadlines and verification. The factory must return the declared
|
|
* surface; a plain one is rejected with INVALID_CONFIG.
|
|
*/
|
|
export type EngineFixtureFactory = (context: EngineFixtureContext) => object;
|
|
|
|
/** Context handed to a contributed fixture factory, once per attempt. */
|
|
export interface EngineFixtureContext {
|
|
/** The target the fixture serves. */
|
|
readonly targetName: string;
|
|
/** Records the declared operations before invoking them; undeclared accessors retain their identity. */
|
|
fixture<T extends object>(name: string, surface: T, operations: FixtureOperations<T>): T;
|
|
readonly app: EngineAppInfo & {
|
|
/**
|
|
* Resolves a navigation target against the base URL. Throws
|
|
* `APP_URL_REQUIRED` when the target declares no URL and `POLICY_DENIED`
|
|
* for any scheme but `http:` and `https:` (`about:blank` aside), so a
|
|
* fixture never re-implements the rule the harness owns.
|
|
*/
|
|
resolveUrl(url: string): string;
|
|
};
|
|
/** Harness budgets, for fixtures that poll or navigate. */
|
|
readonly timeouts: {
|
|
readonly test: number;
|
|
readonly action: number;
|
|
readonly assertion: number;
|
|
};
|
|
/**
|
|
* The running phase's signal: the attempt's through `beforeEach` and the
|
|
* body, then a fresh one per `afterEach` hook so teardown can still drive
|
|
* the app after a body timeout. Read it per call; a fixture factory that
|
|
* captures it once keeps a signal that is dead by teardown.
|
|
*/
|
|
readonly signal: AbortSignal;
|
|
/** Per-call operation budget: the action timeout (or an explicit one) plus the attempt signal. */
|
|
operation(timeoutMs?: number): OperationContext;
|
|
/** Registers a file the current step produced under the attempt artifact directory. */
|
|
attachArtifact(
|
|
kind: 'screenshot' | 'video' | 'download' | 'log',
|
|
relativePath: string,
|
|
): void;
|
|
/** Records the viewport the current step established. */
|
|
attachViewport(viewport: { width: number; height: number; scale: number }): void;
|
|
/** Mints a public locator from a raw expression (a platform selector, say). */
|
|
locator(expression: LocatorExpression): Locator;
|
|
/** Mints a screen scope whose every query is wrapped by `scope` (a nested document, say). */
|
|
screen(scope: (expression: LocatorExpression) => LocatorExpression): Screen;
|
|
/**
|
|
* Attaches an expectation surface to a fixture object so `expect(fixture)`
|
|
* returns it. The harness records every matcher call as an assertion step
|
|
* named `expect.<matcher>`; the factory only decides what the matchers mean.
|
|
*/
|
|
expectable<T extends object, E extends object>(target: T, factory: () => E): T & Expectable<E>;
|
|
}
|
|
```
|
|
|
|
#### Declaring operations
|
|
|
|
`context.fixture(name, surface, operations)` declares which methods are
|
|
recorded steps. Each operation takes:
|
|
|
|
| Field | Values | Default |
|
|
| --- | --- | --- |
|
|
| `kind` | `'resource'` or `'assertion'` | required |
|
|
| `label` | `(...args) => string` | none |
|
|
| `timeout` | milliseconds, a function of the arguments, or `false` when the method owns its deadline | `actionTimeout` |
|
|
| `verifies` | `boolean` | `true` for assertions |
|
|
|
|
- Nested maps describe namespaces.
|
|
- Recording decorates the object in place, so create a fresh surface in each
|
|
factory invocation.
|
|
- The runner opens the step before calling a declared method, so attachments
|
|
and failures from synchronous and asynchronous work land on the same step.
|
|
Work arriving after its step has closed cannot attach to a later one.
|
|
- Type the fixture with `test.extend<{ name: Type }>()`.
|
|
|
|
```ts
|
|
/** Explicit recording policy for one async fixture method. Arguments enter reports only through label. */
|
|
export interface FixtureOperation<Args extends unknown[] = unknown[]> {
|
|
readonly kind: 'resource' | 'assertion';
|
|
readonly label?: (...args: Args) => string;
|
|
/** Action timeout by default; false when the method owns its assertion/navigation deadline. */
|
|
readonly timeout?: number | false | ((...args: Args) => number | undefined);
|
|
/** Assertions verify by default; waits may opt in as well. */
|
|
readonly verifies?: boolean;
|
|
}
|
|
|
|
/** Only declared async methods are recorded. Sync accessors pass through unchanged. */
|
|
export type FixtureOperations<T extends object> = {
|
|
readonly [Key in keyof T]?: T[Key] extends (...args: infer Args) => Promise<unknown>
|
|
? FixtureOperation<Args>
|
|
: T[Key] extends object ? FixtureOperations<T[Key]> : never;
|
|
};
|
|
```
|
|
|
|
#### Expectations
|
|
|
|
`expectable` attaches an expectation surface so `expect(fixture)` returns it.
|
|
Declare matcher recording with `context.fixture('expect', matchers,
|
|
operations)`.
|
|
|
|
```ts
|
|
const printer = context.expectable(surface, () =>
|
|
context.fixture('expect', matchers, { toHavePrinted: { kind: 'assertion' } }),
|
|
);
|
|
return context.fixture('printer', printer, { print: { kind: 'resource' } });
|
|
```
|
|
|
|
## Types
|
|
|
|
### Exported names
|
|
|
|
`EngineCleanupContext`, `EngineDisposeContext`, `OperationContext`,
|
|
`TextPattern`, `SemanticQuery`, `LocatorExpression`,
|
|
`NodeRef`, `SemanticNode`, `ViewportPoint`, `ViewportSize`,
|
|
`ObservationPixels`, `LocatorAction`, `LocatorActionKind`, `PointerAction`,
|
|
`PointerActionKind`, `ScrollDirection`, `Momentum`, `SelectOption`, `Key`,
|
|
`KeyModifier`, `KeyName`, `ParsedKey`, `ENGINE_SPI_VERSION`,
|
|
`LOCATOR_ACTION_KINDS`, `POINTER_ACTION_KINDS`, `KEY_MODIFIERS`,
|
|
`KEY_NAMES`, `parseKey`, `ENGINE_ERROR_CODES`, `RETRYABLE_ENGINE_ERROR_CODES`,
|
|
`Secret`, `isSecret`, `TextMatch`, `JsonValue`, `JsonValueRules`,
|
|
`ResolveExpressionOptions`, `PollConditionOptions`, `WithoutUndefined`,
|
|
`FixtureOperation`, `OBSERVED_NAME_LIMIT`, and `OBSERVED_TEXT_LIMIT` are
|
|
exported from `e2e/engine`. So are the [engine protocol](/reference/engine-protocol)'s
|
|
`connectEngine`, `serveEngine`, `describeEngine`, `spawnTransport`,
|
|
`socketTransport`, `loopbackTransport`, `ENGINE_METHODS`,
|
|
`ENGINE_PROTOCOL_VERSION`, and their types.
|
|
|
|
### `SemanticNode`
|
|
|
|
```ts
|
|
/** One node of an observation tree. */
|
|
export interface SemanticNode {
|
|
/** Node reference, valid against the observation it came from. */
|
|
readonly ref: NodeRef;
|
|
/** Semantic role. */
|
|
readonly role?: string;
|
|
/** Accessible name, cut at `OBSERVED_NAME_LIMIT`. */
|
|
readonly name?: string;
|
|
/** Visible text, cut at `OBSERVED_TEXT_LIMIT`. */
|
|
readonly text?: string;
|
|
/** Current input value; never reported for a secure field. */
|
|
readonly value?: string;
|
|
/**
|
|
* Text selected inside the node, for the focused field or editing host
|
|
* whose selection the platform reports; absent when the selection is
|
|
* collapsed, and never reported for a secure field. Cut at `OBSERVED_TEXT_LIMIT`.
|
|
*/
|
|
readonly selection?: string;
|
|
/**
|
|
* The node's test id, when the platform gives it one: the value of the
|
|
* project's test-id attribute on a document platform, an accessibility
|
|
* identifier on iOS, a resource id on Android, an automation id on a
|
|
* desktop. The `testId` query resolves against this field, so the harness
|
|
* never knows where a platform keeps it. An engine may also match the id
|
|
* without its platform's namespacing.
|
|
*/
|
|
readonly testId?: string;
|
|
/** What a field takes, deciding which secrets may fill it. */
|
|
readonly inputPurpose?: 'username' | 'password' | 'one-time-code' | 'generic-secret' | 'none';
|
|
/**
|
|
* Boolean states the platform reports. `pressed` is a toggle button's
|
|
* pressed state; `secure` marks a field whose value is never observed.
|
|
*/
|
|
readonly states?: Readonly<
|
|
Partial<
|
|
Record<
|
|
'checked' | 'disabled' | 'selected' | 'expanded' | 'pressed' | 'focused' | 'hidden' | 'secure',
|
|
boolean
|
|
>
|
|
>
|
|
>;
|
|
/** Heading level of a heading node (`1` through `6` on a document platform); absent elsewhere. */
|
|
readonly level?: number;
|
|
/** Platform attributes, what `getAttribute` reads. */
|
|
readonly attributes?: Readonly<Record<string, string>>;
|
|
/**
|
|
* The node's box in the top-level viewport's CSS pixels (`ViewportPoint`
|
|
* space), for every node, including those inside nested documents: an
|
|
* engine that measures a child document against its own viewport shifts
|
|
* the boxes by the boundary element's before reporting them.
|
|
*/
|
|
readonly rect?: {
|
|
readonly x: number;
|
|
readonly y: number;
|
|
readonly width: number;
|
|
readonly height: number;
|
|
};
|
|
/**
|
|
* The point an offset into the node is measured from, in `rect`'s space,
|
|
* when it is not `rect`'s top-left corner: a document element's padding
|
|
* box, inside its border, or the corner of a box a frame clipped `rect`
|
|
* from. `tap({ position })` adds the offset to it.
|
|
*/
|
|
readonly origin?: ViewportPoint;
|
|
/**
|
|
* Platform selector that addresses this node within its own document, when
|
|
* the platform has one. It is structural, not semantic: it survives the
|
|
* content changes that rename a node, and a runner may store it to re-find
|
|
* the node cheaply on a later run. It is never part of the node's identity,
|
|
* so whatever it resolves to is still checked before it is used.
|
|
*
|
|
* It SHOULD be anchored on an attribute naming the node or one of its
|
|
* ancestors, and SHOULD be absent rather than positional all the way to the
|
|
* document root: such a path is shifted by anything inserted above the node,
|
|
* so it does not survive to the later run it exists for.
|
|
*/
|
|
readonly selector?: string;
|
|
/**
|
|
* Enclosing nested-document chain as selectors of each boundary node,
|
|
* outermost first. Absent for nodes in the main document.
|
|
*/
|
|
readonly framePath?: readonly string[];
|
|
/** Child nodes. */
|
|
readonly children?: readonly SemanticNode[];
|
|
}
|
|
```
|
|
|
|
### Observed field limits
|
|
|
|
| Constant | Value | Cuts |
|
|
| --- | --- | --- |
|
|
| `OBSERVED_NAME_LIMIT` | 256 | `name` |
|
|
| `OBSERVED_TEXT_LIMIT` | 512 | `text`, and any `value` or `selection` the engine cuts |
|
|
|
|
- An engine that cuts an observed field cuts it at exactly its limit. A field
|
|
exactly at its limit may have been cut; a shorter one, or a longer one the
|
|
engine left whole, is complete.
|
|
- Keep the start and cut the end: the runner redacts the leading part of a
|
|
secret that a field exactly at its limit ends with, which a whole-value
|
|
match cannot see.
|
|
- Single-node reads are unbounded and always carry the full value.
|
|
|
|
### Queries
|
|
|
|
The `selector` expression kind carries a platform-native selector string;
|
|
`frame` scopes a query into a nested document.
|
|
|
|
```ts
|
|
/** The `screen` query kinds. */
|
|
export type QueryKind = 'role' | 'label' | 'placeholder' | 'text' | 'displayValue' | 'testId';
|
|
```
|
|
|
|
A `SemanticQuery` with `visible: true` must match only nodes whose
|
|
`states.hidden` is false. Use the same predicate the engine reports on
|
|
`SemanticNode`, so `getByText('x', { visible: true })` and `toBeVisible()`
|
|
never disagree. Apply it wherever the query sits in the expression, under a
|
|
scope, filter, or index too.
|
|
|
|
## Related
|
|
|
|
[Engines](/engines) compares the ways to build one.
|
|
[Writing an engine](/writing-an-engine) walks through `defineEngine`, and
|
|
[Engine servers](/engine-servers) runs an engine in another process,
|
|
machine, or language.
|
|
`@e2e-dev/web` ([reference](/reference/web)) and `@e2e-dev/mobile`
|
|
([reference](/reference/mobile)) implement this contract with the same
|
|
public `defineEngine`.
|