Files
e2e/docs/tools.mdx
T
Oskar Kwaśniewski 4d488f8286 feat!: the app is declared on its target; engines only drive it (#698)
* feat!: the app is declared on its target; engines only drive it

The app under test moves from the engine's options to `targets: [{ engine,
app }]`: `url`, `bundleId`, `appPath`, `identity`, `environment`,
`launchArguments`, `permissions`, `command`, and `readyUrl`. `web()` and
`mobile()` refuse every moved option in one `INVALID_CONFIG` naming the
target block; `defineEngine({ app })` is gone and engines check what their
platform needs in `validateApp`. `requires: ['browser' | 'native-app']` is
derived from the target's app. Replay cache and session keys are byte for
byte what they were, pinned by target-identity.test.ts.

Dependency services are not part of this version: `services` anywhere, and
`web({ services })`, fail saying so. `app.command` on port 0 without a
command, and `reuseExisting` on a free port, are refused. An `e2e mcp`
open cancelled or disconnected mid-start stops what it started, and a hung
shared start is abandoned once no session waits on it.

* fix: review follow-ups: cancelled opens, abandoned starts, null targets, stale docs

* refactor: drop the moved-option messages; old app keys are unknown keys

* test: give the replaced-nodes suite a budget that survives a loaded runner

* test: size the replaced-nodes hook budget to its seven tests
2026-09-30 19:48:00 +02:00

151 lines
5.3 KiB
Plaintext

---
title: Project tools
description: Add tools for test APIs, data setup, and device operations.
---
Put tools in an agent's `tools` so the agent can call your test API or perform
other project-specific operations during a goal. Wrap each AI SDK tool with
`defineTool` to declare whether it changes state and which platforms support it.
```ts title="e2e.config.ts"
import type { E2EConfig } from 'e2e';
import { defineTool } from 'e2e/agent';
import { web } from '@e2e-dev/web';
import { gateway, tool } from 'ai';
import { z } from 'zod';
const APP_URL = process.env.APP_URL ?? 'http://localhost:3000';
const seedCart = defineTool(
tool({
description: 'Seed the cart with a SKU via the store test API',
inputSchema: z.object({ sku: z.string() }),
execute: async ({ sku }) => {
const response = await fetch(`${APP_URL}/api/test/cart?sku=${encodeURIComponent(sku)}`);
if (!response.ok) throw new Error(`Cart seeding failed: HTTP ${response.status}`);
return response.text();
},
}),
{ mutates: true },
);
export default {
targets: [{ engine: web(), app: { url: APP_URL } }],
agents: { default: { model: gateway('openai/gpt-6-luna-fast'), tools: { seedCart } } },
} satisfies E2EConfig;
```
The agent can now seed a cart before checking out:
```ts
await agent.act('seed the cart with SKU {sku}, then check out', {
params: { sku: 'SHIRT-BLUE-M' },
});
```
Built-in tool names are reserved. Loading the config fails with
`INVALID_CONFIG` for a project tool named like a grammar verb (`tap`,
`screenshot`), a tool the harness adds (`complete_step` in every step,
`locate`, `start_recording`, and `stop_recording` in an `e2e mcp` session,
`report_finding` under `e2e explore`), and for a value `defineTool` did not
build. Tools belong to
the built-in agent: an entry with a custom `executor` cannot also set
`tools`.
## Declare what a tool does
`defineTool` takes the AI SDK tool and these annotations:
| Annotation | Meaning |
| --- | --- |
| `mutates: true` | Changes state. The call counts against the step's action budget and runs in order with the other actions. The [replay cache](/cache) cannot replay it, so the agent runs that part of the step live. |
| `mutates: false` | Read-only. Runs without an action slot and may observe the screen. |
| `platforms: ['ios', 'android']` | Offers the tool only on those platforms. Omit this to offer it on every target. |
A failed call still consumes its action slot. The runner checks the budget
before dispatch, including when the model requests several calls at once.
## Observe from a tool
A read-only tool can observe the redacted screen. This example saves a
masked screenshot as report evidence when pixels are available:
```ts
import { randomUUID } from 'node:crypto';
import { defineTool, getToolContext } from 'e2e/agent';
import { tool } from 'ai';
import { z } from 'zod';
const captureEvidence = defineTool(
tool({
description: 'Save a screenshot as evidence in the report',
inputSchema: z.object({}),
execute: async (_input, executionOptions) => {
const context = getToolContext(executionOptions);
const observation = await context.observe({ pixels: true });
const pixels = observation.pixels;
if (pixels === undefined) {
return { withheld: observation.pixelsWithheld ?? 'UNSUPPORTED_CAPABILITY' };
}
const id = await context.attachScreenshot(pixels, `evidence-${randomUUID()}`);
return { evidence: id };
},
}),
{ mutates: false },
);
```
`observe()` returns a redacted text snapshot. Request `tree: true` for the
node tree or `pixels: true` for a masked screenshot. The built-in agent's
restrictions apply, including no pixels
[after a secret fill](/security#screenshots-after-a-secret-fill).
Observe in a separate read-only tool. A mutating tool already occupies the
action queue, so observing from inside it would wait on its own operation.
`attachScreenshot(pixels, label)` adds the image to the step's artifacts and
returns its artifact id. Use a unique label for each capture to avoid
overwriting an earlier image.
## Device toolkits
`@e2e-dev/mobile` ships a ready tool pack for its engine. Pass every device
engine to one `mobileTools` call, and spread the pack to add your own tools
beside it:
```ts
import type { E2EConfig } from 'e2e';
import { mobile } from '@e2e-dev/mobile';
import { mobileTools } from '@e2e-dev/mobile/tools';
import { gateway } from 'ai';
const iphone = mobile({ platform: 'ios' });
const pixel = mobile({ platform: 'android' });
export default {
targets: [
{ name: 'iphone', engine: iphone, app: { bundleId: 'Settings' } },
{ name: 'pixel', engine: pixel, app: { bundleId: 'com.android.settings' } },
],
agents: {
default: {
model: gateway('openai/gpt-6-luna-fast'),
tools: { ...mobileTools(iphone, pixel) /* , ...your own tools */ },
},
},
} satisfies E2EConfig;
```
The pack adds `open_app`, `swipe`, and `alert` on device targets.
Screenshots and point actions come from the engine's built-in tools.
See the [Mobile engine reference](/reference/mobile#agent-tools).
<CardGroup cols={2}>
<Card title="Custom executors" icon="gears" href="/executors">
Replace the prompt or decision loop.
</Card>
<Card title="How agent steps work" icon="stairs" href="/agent-steps">
What the model sees and what it may do.
</Card>
</CardGroup>