Files
e2e/docs/tools.mdx

169 lines
6.3 KiB
Plaintext

---
title: Project tools
description: Add tools for test APIs, data setup, and device operations.
"og:image": "https://e2e.com/docs/images/og/tools.png?v=8747047102"
"twitter:image": "https://e2e.com/docs/images/og/tools.png?v=8747047102"
---
A project tool is an AI SDK tool that the agent can call during a goal, such
as a request to your test API or a data setup step.
## Add a tool
Wrap the tool with `defineTool` and put it in the agent's `tools`:
```ts title="e2e.config.ts"
import type { E2EConfig } from 'e2e';
import { defineTool } from 'e2e/agent';
import { web } from '@e2e-dev/web';
import { gateway, tool } from 'ai';
import { z } from 'zod';
const APP_URL = process.env.APP_URL ?? 'http://localhost:3000';
const seedCart = defineTool(
tool({
description: 'Seed the cart with a SKU via the store test API',
inputSchema: z.object({ sku: z.string() }),
execute: async ({ sku }) => {
const response = await fetch(`${APP_URL}/api/test/cart?sku=${encodeURIComponent(sku)}`);
if (!response.ok) throw new Error(`Cart seeding failed: HTTP ${response.status}`);
return response.text();
},
}),
{ mutates: true },
);
export default {
targets: [{ engine: web(), app: { url: APP_URL } }],
agents: { default: { model: gateway('openai/gpt-6-luna-fast'), tools: { seedCart } } },
} satisfies E2EConfig;
```
The agent can now seed a cart before it checks out:
```ts
await agent.act('seed the cart with SKU {sku}, then check out', {
params: { sku: 'SHIRT-BLUE-M' },
});
await expect(screen.getByText('Order confirmed')).toBeVisible();
```
<Note>
The config load fails with `INVALID_CONFIG` for a tool named like a
grammar verb (`tap`, `screenshot`) or a tool the harness adds:
`complete_step`, `locate`, `run_code`, `test_lines`, `start_recording`,
`stop_recording`, or `report_finding`. It also fails for a value
`defineTool` didn't build, and for `tools` on an entry with a custom
`executor`.
</Note>
## Declare what a tool does
`defineTool` takes the AI SDK tool and these annotations:
| Annotation | Meaning |
| --- | --- |
| `mutates: true` | Changes state. The call takes a slot of the step's action budget before it runs, failed or not, and runs in order with the other actions. The [replay cache](/cache) can't replay it, so the agent runs that part of the step live. |
| `mutates: false` | Read-only. Runs without an action slot and may observe the screen or read node properties. |
| `platforms: ['ios', 'android']` | Offers the tool only on those platforms. Omit it to offer the tool on every target. |
## Observe from a tool
A read-only tool can observe the redacted screen through `getToolContext`.
This example saves a masked screenshot as report evidence when pixels are
available:
```ts
import { randomUUID } from 'node:crypto';
import { defineTool, getToolContext } from 'e2e/agent';
import { tool } from 'ai';
import { z } from 'zod';
const captureEvidence = defineTool(
tool({
description: 'Save a screenshot as evidence in the report',
inputSchema: z.object({}),
execute: async (_input, executionOptions) => {
const context = getToolContext(executionOptions);
const observation = await context.observe({ pixels: true });
const pixels = observation.pixels;
if (pixels === undefined) {
return { withheld: observation.pixelsWithheld ?? 'UNSUPPORTED_CAPABILITY' };
}
const id = await context.attachScreenshot(pixels, `evidence-${randomUUID()}`);
return { evidence: id };
},
}),
{ mutates: false },
);
```
- `observe()` returns a redacted text snapshot. `tree: true` adds the node
tree and `pixels: true` a masked screenshot, under the built-in agent's
rules.
- In a `mutates: true` tool, `observe()` fails with `POLICY_DENIED`.
Observe in a separate read-only tool.
- `attachScreenshot(pixels, label)` adds the image to the step's artifacts
and returns its artifact id. A reused label overwrites the earlier image.
## Read a node's computed style
In a read-only tool, use `context.read` to measure a listed node's rendered
style on the web engine:
```ts
const context = getToolContext(executionOptions);
await context.observe();
const result = await context.read({ id: nodeId }, {
style: ['color', 'font-weight', 'visibility', 'opacity'],
});
return result.style;
```
Pass the node's ID from the newest observation. The result contains only
requested properties, with secret values redacted as in the tree.
Values use the browser's computed CSS format, such as `rgb(255, 0, 0)` for red.
Request properties from the [computed-style allow-list](/reference/engine#rendered-node-properties).
Custom properties, generated content, selectors, and page code are rejected.
Reads use the step's deadline and run in order with observations and actions,
without taking an action slot. They measure the current value on the original
element, and do not refresh the observation or wait for a value to change.
Computed `visibility` and `opacity` describe the element's CSS, not whether
another element covers it or an ancestor makes it invisible.
| Error | When |
| --- | --- |
| `POLICY_DENIED` | A `mutates: true` tool calls `read`, or the engine has no `readStyle` method, as with the mobile engine. |
| `INVALID_ARGUMENT` | The options or property names are invalid. |
| `LOCATOR_NOT_FOUND` | No semantic observation exists, the latest one does not list the ID, or the element detached. Observe again before reading. |
| `STEP_TIMEOUT` | The read exhausted the step's remaining time, including an engine timeout at the deadline. Catching the read error cannot make the step pass. |
Use `requires: ['computed-style']` on a test to skip targets without style reads.
## Device toolkits
`@e2e-dev/mobile` has a ready tool pack, `mobileTools`, with `open_app`,
`swipe`, and `alert`. Spread it to add your own tools beside it:
```ts
tools: { ...mobileTools(iphone, pixel), seedCart },
```
See [Agent tools](/mobile#agent-tools) for the setup. A tool of your own
that needs a raw driver session gets the running device's client from
[`unstable_deviceSession`](/reference/mobile#unstable_devicesession).
## Next
<CardGroup cols={2}>
<Card title="Custom executors" icon="gears" href="/executors">
Replace the prompt or decision loop.
</Card>
<Card title="How agent steps work" icon="stairs" href="/agent-steps">
What the model sees and what it may do.
</Card>
</CardGroup>