mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-22 07:55:36 +08:00
527 lines
25 KiB
Plaintext
527 lines
25 KiB
Plaintext
---
|
|
title: agent
|
|
description: Run agent actions, check conditions, and extract structured data.
|
|
---
|
|
|
|
Use `act` to interact with the app, `assert` to check the current screen,
|
|
`waitFor` to wait for a condition, and `extract` to read structured data.
|
|
Each call has its own timeout and model-call budget. The built-in agent
|
|
requires a configured model: without one, the first acquisition of the
|
|
`agent` fixture records one run-level `MODEL_UNAVAILABLE` and stops the run
|
|
(exit 2).
|
|
[Custom executors](/executors) can implement `act` and `assert` without one;
|
|
`waitFor` and `extract` still need a model.
|
|
|
|
```ts
|
|
export interface Agent {
|
|
/** Plans and executes a bounded multi-action flow; resolves with what the step did. */
|
|
act(instruction: string, options?: ActOptions): Promise<ActResult>;
|
|
/** Polls a natural-language condition until true or timed out. */
|
|
waitFor(condition: string, options?: WaitForOptions): Promise<void>;
|
|
/** Extracts and validates structured screen data. */
|
|
extract<Schema extends StandardSchemaV1>(
|
|
instruction: string,
|
|
options: ExtractOptions<Schema>,
|
|
): Promise<StandardSchemaV1.InferOutput<Schema>>;
|
|
/** Judges a natural-language assertion against a fresh observation. */
|
|
assert(assertion: string, options?: AssertOptions): Promise<void>;
|
|
}
|
|
```
|
|
|
|
## Budgets
|
|
|
|
Model calls count every model request including repair rounds. Action steps
|
|
count committed engine actions. Every timeout is additionally capped by the
|
|
remaining test timeout.
|
|
|
|
| Method | Model calls | Action steps | Default timeout |
|
|
| --- | ---: | ---: | --- |
|
|
| `act` | `agents.<name>.maxModelCalls`, 25 | `agents.<name>.maxSteps`, 25 | `config.timeout`, 120000 ms |
|
|
| `waitFor` | `agents.<name>.maxModelCalls`, 25 | 0 | `agents.<name>.judgmentTimeout`, 30000 ms |
|
|
| `extract` | 2 | 0 | `agents.<name>.judgmentTimeout`, 30000 ms |
|
|
| `assert` | 2 | 0 | `agents.<name>.judgmentTimeout`, 30000 ms |
|
|
|
|
`actionTimeout` limits engine operations, not model calls. Per-call budgets
|
|
must be positive integers no larger than the configured limits; a larger one
|
|
is `INVALID_ARGUMENT`. All durations are in milliseconds.
|
|
|
|
The `assert` row holds for the built-in agent. When the selected agent has a
|
|
custom `executor`, `assert` is dispatched to it as a step: its budget is the
|
|
agent's `maxSteps` and `maxModelCalls`, its default timeout is
|
|
`config.timeout`, and `vision` on the call is `UNSUPPORTED_CAPABILITY`;
|
|
`screenshot` still applies.
|
|
|
|
## agent.act
|
|
|
|
```ts
|
|
act(instruction: string, options?: ActOptions): Promise<ActResult>;
|
|
```
|
|
|
|
Plans and executes one bounded multi-action flow. A passing verdict resolves
|
|
with an `ActResult`; a failing or blocked one throws an `AgentError` whose
|
|
`code` and `blocked` flag say why.
|
|
|
|
`instruction` is 1 through 8192 UTF-8 bytes after NFC normalization. An
|
|
unknown `agent` name is `INVALID_ARGUMENT` before the step opens.
|
|
|
|
Values come from your code through `options.params`, never from the model. The
|
|
model sees a `Secret` as its name and purpose only. The runner checks the
|
|
field the model picked before it fills the value: it must be an enabled
|
|
editable input, and a password goes only into a password field.
|
|
`locator.fill` runs no such check, since your code chose the field.
|
|
|
|
```ts
|
|
await agent.act('invite {email} as an editor', { params: { email: 'ada@example.test' } });
|
|
|
|
const member = credentials.user('member');
|
|
const { summary, cache } = await agent.act('Sign in', {
|
|
params: { username: member.username, password: member.password },
|
|
});
|
|
```
|
|
|
|
The result carries the step's `summary`, the `modelCalls` and `actions` it
|
|
spent, and `cache`: how the [replay cache](/cache) took part
|
|
(`self-finalized`, `agent-concluded`, or `missed` with a `reason`; absent when
|
|
caching is off).
|
|
|
|
### Action tools
|
|
|
|
The built-in agent offers one tool per verb the target's engine declares.
|
|
Every tool takes node ids from the newest observation, returns what changed
|
|
on screen, and is recorded for the [replay cache](/cache) with the node it
|
|
acted on. A verb the engine does not declare is not offered.
|
|
|
|
| Tool | Engine action | What it does |
|
|
| --- | --- | --- |
|
|
| `tap({ target })` | `tap` | Taps or clicks one node: the gesture for a button, link, menu item, tab, checkbox, row, or field. |
|
|
| `double_tap({ target })` | `doubleTap` | Double-clicks one node, for an item that opens or enters editing on the second click. |
|
|
| `long_press({ target })` | `longPress` | Presses one node and holds, for a long-press menu or action. |
|
|
| `right_click({ target })` | `secondaryTap` | Right-clicks one node to open its context menu. |
|
|
| `hover({ target })` | `hover` | Moves the pointer over one node without pressing: hover menus, flyouts, tooltips, controls revealed on mouse-over. |
|
|
| `type({ target?, value, replace? })` | `fill`; without `target`, `keyboard` | Replaces the node's value; without a target, types into whatever has focus. |
|
|
| `type_secret({ target, name })` | `fill` | Fills a secret declared in the step's params. Offered only when the step declares one. |
|
|
| `press({ target?, key, times? })` | `press`; without `target`, `keyboard` | Sends one key to a node or to the focused control, up to 20 times in one call. Each press is one action; with a target, the first press addresses the node and the rest go to whatever has focus, so a repeated `Tab` walks on. |
|
|
| `select({ target, value })` | `selectOption` | Picks an option by its visible label. |
|
|
| `check({ target, checked? })` | `check` and `uncheck` | Sets a checkbox, switch, or radio to a state (`true` by default). A control already in that state is left alone, unlike a tap, which flips it. |
|
|
| `scroll({ direction, target?, times? })` | `swipe` | Scrolls the viewport or one scrollable node, up to 20 screens per call. |
|
|
| `scroll_to({ target })` or `scroll_to({ text, direction?, target? })` | `scrollIntoView`; with `text`, `swipe` | With `target` alone, brings one listed node into the viewport, for a node the tree lists below or above the fold. With `text`, pages the list (`target`, else the viewport) screen by screen in `direction` (down by default) until a node reading that text shows (as its whole label, or bounded by nothing of a word, so `Row 12` never stops at `Row 120`; inside the list when `target` names one, so a heading elsewhere reading the same does not end the paging), then brings it in when the engine can: for a row deep in a long or windowed list the tree has not rendered yet. One action however many screens, up to 800; a screen that stops moving or the cap is `LOCATOR_NOT_FOUND`. Each form is offered only with its action kind. |
|
|
| `drag({ target, to })` | `dragTo` | Drags one node and drops it on another: reordering, kanban columns, drop zones. |
|
|
| `upload({ target, files })` | `setInputFiles` | Attaches files to a file input. Paths are project-relative; see the policy below. |
|
|
| `navigate({ url })` | `session.open` | Opens a URL or an app-relative path. |
|
|
| `back()` | `session.back` | Goes back once: browser history on a page, the in-app back on a device. |
|
|
| `observe()` | | Looks again and reports what changed since the last screen the model received. |
|
|
| `dismiss_keyboard()` | `keyboard.dismiss` | Hides an on-screen keyboard. Device engines only. |
|
|
|
|
`scroll_to({ text })` waits for each scrolled screen to settle. On a device,
|
|
the next observation checks stability without waiting for a second scroll
|
|
effect. Engines with `scrollIntoView` still wait for the final adjustment
|
|
that brings the matched node into view. Replay follows the same rule.
|
|
|
|
`upload` takes the model's path but the runner decides. Each path is
|
|
resolved against the project root the way `locator.setInputFiles` resolves
|
|
it, and is refused with `POLICY_DENIED` when it lies outside the project
|
|
(through a symlink too) or names a hidden file or a file under a hidden
|
|
directory such as `.env`, `.git`, or `.e2e`. A path that names no regular
|
|
file is `INVALID_ARGUMENT`, an ordinary action failure the model reads and
|
|
corrects. Every refusal and the final allow is a `policy` event named
|
|
`upload.path` on the step. Nothing reaches the engine until every path passed.
|
|
|
|
The screen the model reads lists the text selected inside the focused field
|
|
as `selection="..."` after its value, when the engine reports one: the web
|
|
engine does for an input, a textarea, and a `contenteditable` editor, and a
|
|
secure field reports none. The runner drops it from a field whose value or
|
|
text holds a registered secret, and from a field cut at its length limit when
|
|
the selection is not in the part shown, so selected characters of a secret
|
|
never reach the model. A `press` of `Shift+ArrowLeft` shows what it selected, and
|
|
`times` presses the key several times in one call, so the agent extends a
|
|
selection to exactly the word it wants in one more turn before it formats or
|
|
replaces it.
|
|
|
|
The tap variants are separate tools rather than an option on `tap` on
|
|
purpose: offered as an optional `kind`, a model filled it on ordinary
|
|
buttons (a double-click on Save), and a double-click on a submit button
|
|
submits twice. A tool has to be chosen, and each one says what takes a
|
|
plain tap instead.
|
|
|
|
Mobile engines declare `hover`, `doubleTap`, `longPress`, `check`,
|
|
`uncheck`, `dragTo`, and `session.back`, so a device offers `hover`,
|
|
`double_tap`, `long_press`, `check`, `drag`, and `back`; they declare no
|
|
`secondaryTap`, `scrollIntoView`, or `setInputFiles`. Tab switching has no
|
|
verb yet: a new tab is invisible to the model.
|
|
|
|
### Pixel tools: screenshot and the point verbs
|
|
|
|
`act` reads the semantic tree and requests screenshots when it needs them.
|
|
A screen with no listed interactive nodes, such as a canvas, opens with a
|
|
screenshot. The same model reads the tree and screenshots; there is no
|
|
`vision` option for `act`.
|
|
|
|
The built-in agent offers these tools while no secret has been filled:
|
|
|
|
| Tool | What it does |
|
|
| --- | --- |
|
|
| `screenshot()` | Returns a masked viewport screenshot and includes a fresh screenshot after every later action in the step. |
|
|
| `tap_at({ x, y })` | Taps the control at the point by node id, or taps the coordinates directly when no control is listed. |
|
|
| `hover_at({ x, y })` | Hovers the control at the point by node id, or moves the pointer to the coordinates when no control is listed. Needs the `hover` action or pointer kind. |
|
|
| `type_at({ x, y, value, replace? })` | Replaces a listed input's value. For an unlisted input, taps to focus and types at the caret unless `replace` is set. Unlisted inputs require keyboard support. |
|
|
| `press_at({ x, y, key })` | Sends a key to the control at the point. If no control is listed, taps to focus and uses the engine's keyboard. |
|
|
| `select_at({ x, y, value })` | Selects an option by visible label in the control at the point. |
|
|
|
|
Coordinates refer to the latest screenshot. The model must take a screenshot
|
|
before using a point tool and should prefer node ids for listed controls.
|
|
The runner checks each point against the tree. A point on a listed control
|
|
uses the same policy checks and [replay cache](/cache) recording as an action
|
|
by id.
|
|
|
|
Once screenshots are in use, the post-action wait also detects pixel changes,
|
|
including canvas redraws. Screenshots have a maximum long side of 768 px;
|
|
older images are removed in batches, keeping the newest two.
|
|
|
|
On engines with a keyboard, including the browser and device engines, `type`
|
|
and `press` can omit `target` to act on the focused field. `type` inserts at
|
|
the caret; `replace: true` clears first. The browser engine rejects typing
|
|
when no editable field has focus. Device engines also offer `dismiss_keyboard`.
|
|
When an action closes an on-screen keyboard, the result says so: on a touch
|
|
screen that tap was often spent on closing it, and the model acts on the
|
|
control again.
|
|
|
|
If semantic capture times out, an engine that proves masking independently
|
|
can return a fresh screenshot with an explicit unavailable-tree warning.
|
|
Every `act` observation with unavailable semantics includes that image and
|
|
warning automatically. Earlier node ids are invalid, and the next successful
|
|
semantic capture presents the whole tree again. The screenshot shows the
|
|
current viewport; an unavailable tree does not prove that a control or
|
|
message is absent. After a secret fill there is no fallback: `act`, and a
|
|
judgment with `vision: true`, fails with `POLICY_DENIED` when semantic
|
|
capture times out before the step does.
|
|
|
|
If semantic capture fails, trace replay stops and the step cannot record a
|
|
new trace entry, even if the tree later recovers.
|
|
|
|
A secret filled during the step withholds every later screenshot
|
|
(`PIXEL_TAINTED`), and a step that opens on a tainted viewport offers none of
|
|
the six pixel tools. `tap_at` needs an engine whose `actions` or
|
|
`pointerActions` include `tap`, and `hover_at` one whose include `hover`;
|
|
without the pointer kind the point must land on a listed control. A
|
|
bare-point tap or hover replays from the [replay cache](/cache) only on a
|
|
viewport of the recorded size, or at the same place inside the listed node
|
|
that contained it. A step that sent pixels records `visionInput: true` and
|
|
`metrics.pixelBytes`.
|
|
|
|
A failed or blocked step throws an `AgentError`. A failed verdict is
|
|
`ACTION_FAILED` unless the executor names a more specific code, such as
|
|
`LOCATOR_NOT_FOUND` or `AUTHENTICATION_FAILED`. A blocked verdict, a step
|
|
that could not reach a product verdict, carries one of these codes:
|
|
|
|
| Blocked on | Codes |
|
|
| --- | --- |
|
|
| credentials | `AUTH_CREDENTIAL_UNAVAILABLE`, `AUTH_CREDENTIAL_INVALID`, `SECRET_UNAVAILABLE` |
|
|
| environment | `ENVIRONMENT_UNAVAILABLE`, `APP_UNREACHABLE`, `APP_ALREADY_RUNNING` |
|
|
| seed data | `SEED_DATA_MISSING` |
|
|
| test setup | `TEST_SETUP_FAILED`, `APP_NOT_OPEN`, `POLICY_DENIED` |
|
|
| automation | `MODEL_UNAVAILABLE`, `AUTOMATION_UNSUPPORTED`, `STEP_BUDGET_EXHAUSTED`, `STEP_TIMEOUT`, `CONTEXT_OVERFLOW` |
|
|
|
|
The runner's own stops are
|
|
`STEP_BUDGET_EXHAUSTED` and `STEP_TIMEOUT` when a budget or the deadline ran
|
|
out, `CANCELLED`, `STEP_NO_CONCLUSION` when the turns ran out without a
|
|
conclusion, `CONTEXT_OVERFLOW`, `MODEL_PROVIDER_FAILED`,
|
|
`MODEL_OUTPUT_INVALID` for a verdict outside the grammar, and `REPLAY_STALE`
|
|
under a strict cache. Every code is in [Errors](/reference/errors#agenterror).
|
|
|
|
`act` also throws `INVALID_ARGUMENT` for a malformed instruction, oversized
|
|
`params`, an unknown option, or a budget above the configured limit, and
|
|
`UNSUPPORTED_CAPABILITY` for `schema` or `vision`, which `act` does not take.
|
|
|
|
## agent.assert
|
|
|
|
```ts
|
|
assert(assertion: string, options?: AssertOptions): Promise<void>;
|
|
```
|
|
|
|
Checks one fresh observation without polling. The judge sees only the
|
|
assertion and current observation, never prior steps or `act` summaries.
|
|
It answers `holds`, `fails`, or `inconclusive`.
|
|
|
|
Throws `ASSERTION_FAILED` on a false judgment and `ASSERTION_INCONCLUSIVE`
|
|
when the screen did not show enough to decide. Both carry the explanation
|
|
and, when permitted, a screenshot path. An inconclusive judgment made from
|
|
the tree alone adds `pass vision: true when the answer is in pixels` to its
|
|
message, worded as a condition (`if the engine captures pixels, ...`) until
|
|
a step of the attempt has received pixels. There is no hint when the engine
|
|
declares no screenshot capture, a secret was filled, or an earlier step's
|
|
pixel request was degraded: `vision: true` could add no evidence then. The
|
|
hint is the built-in judge's; with a custom `executor`, `assert` carries the
|
|
executor's explanation alone. No evidence is captured with
|
|
`screenshot: false` or once a secret has been filled in the attempt.
|
|
`screenshot` controls failure evidence attached after the judgment; `vision`
|
|
controls what the model is shown before it. A custom `StepExecutor` judges
|
|
the assertion itself: it honors `screenshot`, since the runner takes that
|
|
evidence after the verdict, and rejects `vision` with
|
|
`UNSUPPORTED_CAPABILITY`, since the executor decides what its model sees.
|
|
|
|
```ts
|
|
await agent.assert('the dashboard shows a trial badge');
|
|
```
|
|
|
|
## agent.waitFor
|
|
|
|
```ts
|
|
waitFor(condition: string, options?: WaitForOptions): Promise<void>;
|
|
```
|
|
|
|
Observes and judges at once. After a false or inconclusive judgment it
|
|
waits out `interval`, then observes every 500 ms (or every `interval`, when
|
|
shorter) without a model call, and judges again only once the screen
|
|
differs from the one last judged, pixels included under `vision`. Succeeds
|
|
on the first judgment that holds.
|
|
|
|
Throws `STEP_TIMEOUT` on deadline expiry and `STEP_BUDGET_EXHAUSTED` on
|
|
budget exhaustion, each carrying the last judgment (with the same vision
|
|
hint as `agent.assert` when that round was inconclusive), and
|
|
`INVALID_ARGUMENT` for an out-of-range option or one `waitFor` does not
|
|
take.
|
|
|
|
```ts
|
|
await agent.waitFor('the upload progress reports completion', { interval: 500 });
|
|
```
|
|
|
|
## agent.extract
|
|
|
|
```ts
|
|
extract<Schema extends StandardSchemaV1>(
|
|
instruction: string,
|
|
options: ExtractOptions<Schema>,
|
|
): Promise<StandardSchemaV1.InferOutput<Schema>>;
|
|
```
|
|
|
|
Takes one fresh observation, requests structured output, validates it, and
|
|
returns the validated value. It performs no app actions.
|
|
|
|
When the schema has a JSON Schema projection (zod does), the model receives
|
|
its shape: types, fields, required fields, and allowed literals. Value rules
|
|
such as `min`, `max`, lengths, patterns, and formats stay with your schema,
|
|
which judges the value the model read. The model can also answer that the
|
|
screen does not show the data, which throws `ASSERTION_INCONCLUSIVE` with
|
|
what was missing, instead of returning a placeholder such as `""` or `0`.
|
|
To accept absence as an answer, say so in the instruction and the schema
|
|
(`'the phone number, or null when none is shown'` with
|
|
`z.string().nullable()`).
|
|
|
|
The budget is two model calls: one extraction and one repair round for
|
|
output that failed validation. Throws `INVALID_ARGUMENT` for a schema that is
|
|
not Standard Schema v1, `ASSERTION_INCONCLUSIVE` when the screen does not
|
|
show the data, and `MODEL_OUTPUT_INVALID` when validation still fails after
|
|
the repair.
|
|
|
|
```ts
|
|
const summary = await agent.extract('the workspace name and plan', {
|
|
schema: z.object({ name: z.string(), plan: z.string() }),
|
|
});
|
|
```
|
|
|
|
## Vision
|
|
|
|
`assert`, `waitFor`, and `extract` accept `vision`. Screenshots require a
|
|
judge model that accepts images.
|
|
|
|
| `vision` | Evidence sent to the judge |
|
|
| --- | --- |
|
|
| `false`, the default | Semantic tree |
|
|
| `true` | Semantic tree and a masked screenshot |
|
|
| `'only'` | Masked screenshot only |
|
|
|
|
Use `'only'` for visual checks, such as whether an overlay covers a form or a
|
|
chart looks correct. Screenshots show the current viewport, so scroll to the
|
|
relevant content first. `act` requests its own screenshots and does not take
|
|
this option.
|
|
|
|
Pixels are withheld and the tree sent alone when a secret fill has tainted
|
|
the viewport (`visionDegraded: "PIXEL_TAINTED"` on the step), when the engine
|
|
masked fewer regions than the secure fields it observed (`MASKING_UNPROVEN`),
|
|
or when the engine cannot capture pixels (`UNSUPPORTED_CAPABILITY`). In
|
|
`'only'` mode those cases fail with `POLICY_DENIED` instead.
|
|
|
|
With `vision: true` or `'only'`, a recoverable semantic-capture failure can
|
|
use a fresh screenshot when the engine proves masking independently. With
|
|
`vision: false`, or after a secret fill, the capture failure stops the
|
|
judgment before any model call. A missing tree cannot support a judgment
|
|
about an empty screen.
|
|
|
|
If capture fails before a judgment, the report records zero model calls and
|
|
omits `observationRevision` and `explanation`. The step error describes the
|
|
capture failure. An answer the closed grammar rejects is a `schema` event on
|
|
the step and earns one repair round; when that answer is rejected too, the
|
|
step fails with `MODEL_OUTPUT_INVALID` and its `explanation` is the rejection.
|
|
|
|
## Types
|
|
|
|
```ts
|
|
/**
|
|
* Parameters and budgets for one `act` call.
|
|
*/
|
|
export interface ActOptions extends AgentOption {
|
|
/**
|
|
* JSON-safe values the instruction refers to, at most 64 KiB and 32 levels
|
|
* deep. A `Secret` reaches the model by name only; the runner fills it. A
|
|
* value wrapped in `unique()` is different on every run, and the replay
|
|
* cache records a slot for it instead of the value.
|
|
*/
|
|
params?: AgentParams;
|
|
/** Step deadline in milliseconds; defaults to `config.timeout`. */
|
|
timeout?: number;
|
|
/** Action budget; defaults to `agents.<name>.maxSteps` and can only lower it. */
|
|
maxSteps?: number;
|
|
/** Model-call budget; defaults to `agents.<name>.maxModelCalls` and can only lower it. */
|
|
maxModelCalls?: number;
|
|
}
|
|
|
|
/** The configured agent (`agents.<name>`) one call runs with, in place of the test's. */
|
|
export interface AgentOption {
|
|
agent?: string;
|
|
}
|
|
|
|
/** `assert` options: one judgment plus one repair round, within `timeout`. */
|
|
export interface AssertOptions extends VisionOption, AgentOption {
|
|
/** Deadline in milliseconds; defaults to the agent's `judgmentTimeout`, 30000. */
|
|
timeout?: number;
|
|
/** Attach a redacted screenshot to the step; on by default, denied after a secret fill. */
|
|
screenshot?: boolean;
|
|
}
|
|
|
|
/** `waitFor` options: a judgment at most once per `interval` until `timeout`. */
|
|
export interface WaitForOptions extends VisionOption, AgentOption {
|
|
/** Deadline in milliseconds; defaults to the agent's `judgmentTimeout`, 30000. */
|
|
timeout?: number;
|
|
/** Least time between two judgments, in milliseconds; 100 through 60000, default 3000. */
|
|
interval?: number;
|
|
/** Judgment budget; defaults to `agents.<name>.maxModelCalls` and can only lower it. */
|
|
maxModelCalls?: number;
|
|
}
|
|
|
|
/** `extract` options: one extraction plus one repair round, validated against `schema`. */
|
|
export interface ExtractOptions<Schema extends StandardSchemaV1> extends VisionOption, AgentOption {
|
|
/** Any Standard Schema v1 validator; the output is validated against it, with one repair round. */
|
|
schema: Schema;
|
|
/** Deadline in milliseconds; defaults to the agent's `judgmentTimeout`, 30000. */
|
|
timeout?: number;
|
|
}
|
|
|
|
export interface VisionOption {
|
|
/** What the judge is shown; `false` by default. */
|
|
vision?: VisionMode;
|
|
}
|
|
|
|
/**
|
|
* Evidence for `assert`, `waitFor`, and `extract`.
|
|
*
|
|
* - `false`: the tree.
|
|
* - `true`: the tree and a screenshot, on every call.
|
|
* - `'only'`: the screenshot, and not the tree.
|
|
*
|
|
* When pixels cannot be safely captured, `true` falls back to the tree;
|
|
* `'only'` fails with `POLICY_DENIED`.
|
|
*/
|
|
export type VisionMode = boolean | 'only';
|
|
|
|
/** What one passing `act` step did, as the report records it. */
|
|
export interface ActResult {
|
|
/** The executor's one-line account of the step, or the replay's when the cache finished it. */
|
|
readonly summary: string;
|
|
/** How the replay cache took part; absent when caching is off for the step. */
|
|
readonly cache?: StepCacheInfo;
|
|
/** Model calls the step spent; 0 when a cached replay finished it. */
|
|
readonly modelCalls: number;
|
|
/** Grammar actions and mutating tool calls the step performed. */
|
|
readonly actions: number;
|
|
}
|
|
|
|
/**
|
|
* How the replay cache took part in one agent step. `self-finalized` means
|
|
* the full trace replayed and the step passed with zero model calls;
|
|
* `agent-concluded` means a replayed prefix handed the step to the executor;
|
|
* `missed` means the executor ran the step from the top.
|
|
*/
|
|
export interface StepCacheInfo {
|
|
mode: 'self-finalized' | 'agent-concluded' | 'missed';
|
|
/** The miss or hand-off reason token; absent on `self-finalized`. */
|
|
reason?: TraceReplayMissReason | ReplayHandOffReason;
|
|
/**
|
|
* On a `gap` at a typed value: the rule that made the value this run's
|
|
* data rather than the flow's. Absent for a gap at a project tool.
|
|
*/
|
|
derived?: DerivedReason;
|
|
/**
|
|
* Why a passing step that ran live was still not recorded, when the cause
|
|
* is the call itself: `param-collision` when a `unique()` value equals, is
|
|
* spelled inside, or is the encoded form of another param's value, so the
|
|
* recording could not tell which param a recorded input came from.
|
|
*/
|
|
notRecorded?: 'param-collision';
|
|
replayedActions: number;
|
|
totalActions: number;
|
|
}
|
|
|
|
type TraceReplayMissReason = 'retry' | 'no-entry' | 'invalid-entry' | 'truncated' | 'wrong-context';
|
|
export type ReplayHandOffReason =
|
|
| 'gap'
|
|
| 'target-not-found'
|
|
| 'target-ambiguous'
|
|
| 'viewport-changed'
|
|
| 'action-failed'
|
|
| 'action-uncertain'
|
|
| 'end-mismatch';
|
|
export type DerivedReason = 'pixels' | 'whole-node' | 'minted-token' | 'date';
|
|
|
|
/** Values an `act` instruction refers to. */
|
|
export type AgentParams = Readonly<Record<string, AgentParam>>;
|
|
|
|
/** A JSON-safe value, a `Secret`, or a `Unique` string. */
|
|
export type AgentParam =
|
|
| JsonPrimitive
|
|
| Secret
|
|
| Unique
|
|
| readonly AgentParam[]
|
|
| { readonly [key: string]: AgentParam };
|
|
```
|
|
|
|
Each `StepCacheInfo.reason` is explained under
|
|
[Why a step was not replayed](/cache#why-a-step-was-not-replayed).
|
|
|
|
`Unique` is what `unique()` returns: a value that differs on every run, which
|
|
the replay cache records as a slot and substitutes on replay. See
|
|
[Cache](/cache).
|
|
|
|
## unique
|
|
|
|
```ts
|
|
import { unique } from 'e2e';
|
|
|
|
/** A string param that differs on every run; the model sees `value`, the replay cache a slot. */
|
|
export interface Unique {
|
|
readonly value: string;
|
|
}
|
|
|
|
/**
|
|
* Marks a value that is different on every run. Mark values the flow types
|
|
* or looks for, not choices that steer it: two runs whose `unique()` values
|
|
* differ replay the same recording.
|
|
*/
|
|
export function unique(value: string): Unique;
|
|
```
|
|
|
|
Throws `INVALID_ARGUMENT` for a value that is not a string, is empty or only
|
|
whitespace, or contains the placeholder text `{{param:`.
|
|
|
|
```ts
|
|
await agent.act('create a company named {name}', {
|
|
params: { name: unique(`Acme ${Date.now()}`) },
|
|
});
|
|
```
|
|
|
|
## AgentError
|
|
|
|
Every failure is an `AgentError` with a closed `code`, an `explanation`, an
|
|
optional `screenshot` path, and a `blocked` flag. Check it with
|
|
`isAgentError(error)` from `e2e`, not `instanceof`. The class and
|
|
every code are in [Errors](/reference/errors#agenterror).
|