mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-21 03:05:26 +08:00
Claude Code loads .mcp.json from a plugin's root even when the manifest names none, so the file goes and Codex and Cursor read the server from mcp.json, the Agent Plugins file Copilot already uses. plugins:check fails if .mcp.json comes back.
295 lines
10 KiB
Plaintext
295 lines
10 KiB
Plaintext
---
|
||
title: Coding agents
|
||
description: Give a coding agent the skill, a live app session, and reports it can use to write and debug tests.
|
||
"og:image": "https://e2e.com/docs/images/og/coding-agents.png?v=c6990935d7"
|
||
"twitter:image": "https://e2e.com/docs/images/og/coding-agents.png?v=c6990935d7"
|
||
---
|
||
|
||
A coding agent such as Claude Code, Codex, or Cursor can write, run, and
|
||
debug e2e tests. It learns e2e from the skill, uses the app through the
|
||
`e2e mcp` server, and reads the trace page of a failed test.
|
||
|
||
## Install the skill
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e init
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm dlx e2e init
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e init
|
||
```
|
||
</CodeGroup>
|
||
|
||
| Path | Used by |
|
||
| --- | --- |
|
||
| `.agents/skills/e2e/` | Codex, Cursor, OpenCode, and other agents that read `.agents/skills` |
|
||
| `.claude/skills/e2e/` | Claude Code. A symlink to `.agents/skills/e2e/` when both are installed. |
|
||
|
||
The copy holds `SKILL.md` and topic references.
|
||
[`e2e upgrade`](/reference/cli#e2e-upgrade) updates it, also after you
|
||
upgrade with your package manager. Without `init`, `npx skills add e2e-inc/e2e`
|
||
installs the skill, and `npx e2e guide [topic]` prints it from the
|
||
installed package. The plugin below installs all three e2e skills for
|
||
every project.
|
||
|
||
`init` leaves your project instructions unchanged. For agents that need a
|
||
pointer, add this line:
|
||
|
||
```md title="AGENTS.md"
|
||
End-to-end tests use e2e; read .agents/skills/e2e/SKILL.md before writing or running one.
|
||
```
|
||
|
||
The package also ships these docs as `.mdx` pages for offline reading, in
|
||
`node_modules/e2e/docs` in a single-package project. A link such as
|
||
`/reference/cli` is `docs/reference/cli.mdx`.
|
||
|
||
<Accordion title="Install the Claude Code, Codex, or Cursor plugin">
|
||
|
||
The e2e repository is a plugin marketplace for Claude Code and Codex, and
|
||
for Grok Build, which reads the Claude Code files. Cursor loads the plugin
|
||
from its local plugins folder:
|
||
|
||
<CodeGroup>
|
||
```bash Claude
|
||
claude plugin marketplace add e2e-inc/e2e
|
||
claude plugin install e2e@e2e
|
||
```
|
||
|
||
```bash Codex
|
||
codex plugin marketplace add e2e-inc/e2e
|
||
codex plugin add e2e@e2e
|
||
```
|
||
|
||
```bash Cursor
|
||
repo=$(mktemp -d)
|
||
git clone --depth 1 https://github.com/e2e-inc/e2e "$repo"
|
||
mkdir -p ~/.cursor/plugins/local
|
||
rm -rf ~/.cursor/plugins/local/e2e
|
||
cp -R "$repo/plugins/e2e" ~/.cursor/plugins/local/
|
||
```
|
||
</CodeGroup>
|
||
|
||
Start a new session. In Cursor, run **Developer: Reload Window**. On Cursor
|
||
Teams and Enterprise, local plugins need the admin setting **Allow Local
|
||
Plugin Imports**, or an admin imports `e2e-inc/e2e` as a team
|
||
marketplace. The plugin adds the
|
||
`e2e`, `create-verification-skill`, and `migrate-to-e2e` skills, not the npm
|
||
package. In Codex and Cursor it also adds the [MCP server](/mcp-server); in
|
||
Claude Code and Grok Build, register it in the project. Invoke the other two under the plugin's
|
||
name, `/e2e:<skill>` in Claude Code and `$e2e:<skill>` in Codex, and as
|
||
`/<skill>` in Cursor.
|
||
|
||
The plugin's version follows e2e releases. To update it, run these and
|
||
start a new session. In Cursor, run the commands above again.
|
||
|
||
<CodeGroup>
|
||
```bash Claude
|
||
claude plugin marketplace update e2e
|
||
claude plugin update e2e@e2e
|
||
```
|
||
|
||
```bash Codex
|
||
codex plugin marketplace upgrade e2e
|
||
```
|
||
</CodeGroup>
|
||
|
||
</Accordion>
|
||
|
||
<Accordion title="Install in OpenClaw or Hermes Agent">
|
||
|
||
The three skills are published on [ClawHub](https://clawhub.ai/) under
|
||
`@e2e`. Hermes Agent reads the repository as a tap:
|
||
|
||
<CodeGroup>
|
||
```bash OpenClaw
|
||
openclaw skills install @e2e/e2e
|
||
openclaw skills install @e2e/create-verification-skill
|
||
openclaw skills install @e2e/migrate-to-e2e
|
||
```
|
||
|
||
```bash Hermes
|
||
hermes skills tap add e2e-inc/e2e
|
||
hermes skills install e2e-inc/e2e/e2e
|
||
```
|
||
</CodeGroup>
|
||
|
||
</Accordion>
|
||
|
||
## Let the agent use the app
|
||
|
||
`e2e mcp` gives the agent a live session on your app. It checks its
|
||
locators against the screen before it writes them, and checks its own UI
|
||
changes. `init` registers the server. See [MCP server](/mcp-server) for setup and
|
||
prompts.
|
||
|
||
## How an agent writes and runs a test
|
||
|
||
The skill tells the agent to read the config and an existing test first.
|
||
With `e2e mcp`, it walks the flow in a session and writes each step as it
|
||
takes it. Known fields and exact values get `screen` calls, from
|
||
locators that matched one node:
|
||
|
||
```ts title="tests/checkout.e2e.ts"
|
||
import { test, expect, credentials } from 'e2e';
|
||
|
||
test('a signed-in user reaches checkout', async ({ screen, app, agent }) => {
|
||
const user = credentials.user('customer');
|
||
await app.open('/login');
|
||
await screen.getByLabel('Email').fill(user.username);
|
||
await screen.getByLabel('Password').fill(user.password);
|
||
await screen.getByRole('button', 'Sign in').click();
|
||
await expect(screen.getByRole('heading', 'Welcome back')).toBeVisible();
|
||
|
||
await agent.act('Add the cheapest item on the page to the cart');
|
||
await expect(screen.getByRole('status', 'Cart')).toHaveText('1 item');
|
||
await agent.act('Open the cart');
|
||
await expect(screen.getByRole('heading', 'Your cart')).toBeVisible();
|
||
});
|
||
```
|
||
|
||
- **One `agent.act` per [atomic goal](/core-concepts#goals), then an
|
||
`expect`.** The `expect` fails when the agent did the wrong thing. A
|
||
locator matcher, or another
|
||
[verification check](/cache#when-a-recording-is-written), also lets the
|
||
cache replay the step.
|
||
- **No sleeps.** Locators poll, actions wait for their target, and `expect`
|
||
retries until its timeout.
|
||
- **Secrets through `credentials.user(name)`.** Declare them in the config.
|
||
Never write them in a test.
|
||
|
||
While it iterates, the agent runs only the test it works on, by the line
|
||
its `test(` opens on. With `--action-timeout 5000`, a wrong locator fails
|
||
after 5 s instead of 30 s:
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx e2e run tests/checkout.e2e.ts:3 --action-timeout 5000
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm exec e2e run tests/checkout.e2e.ts:3 --action-timeout 5000
|
||
```
|
||
|
||
```bash bun
|
||
bunx e2e run tests/checkout.e2e.ts:3 --action-timeout 5000
|
||
```
|
||
</CodeGroup>
|
||
|
||
The agent fixes CLI, config, and collection errors before it runs again.
|
||
It doesn't edit cache recordings.
|
||
|
||
The agent keeps the replay cache on, so a goal whose check passed replays
|
||
without model calls. It turns retries off, since a retry runs every goal
|
||
live. When the test passes, the agent drops `--action-timeout` and runs it
|
||
one time with `--no-cache` to check that the live agent reaches every
|
||
goal. See
|
||
[Rerun a test while you fix it](/cache#rerun-a-test-while-you-fix-it).
|
||
|
||
## How an agent reads a failure
|
||
|
||
By default ([`trace`](/reference/config#trace)), a run writes a trace page
|
||
for each failed, timed-out, interrupted, or flaky test, to the test's
|
||
directory under `.e2e/results/`. `.e2e` is the default `output`. The
|
||
terminal names the page under each failed test, as
|
||
`❯ trace .e2e/results/<test>/trace.md`.
|
||
|
||
The agent reads that page, not `.e2e/report.json`, which holds the same
|
||
facts for [scripts](/debugging). The page has the failing line and every
|
||
step with what it did: the node an action hit, the values an assertion
|
||
read, the cache's decision, console errors, and failed requests. It also
|
||
has the agent's last turns and the screen at the failure.
|
||
|
||
The agent fixes the test from that evidence. It fixes every line the
|
||
evidence covers, not only the failing one. A test stops at its first failed
|
||
step, so the agent also checks the lines after it before the next run.
|
||
|
||
To see a screen the page doesn't show, the agent opens it in an MCP session
|
||
by the shortest way, such as its URL on the web. It doesn't walk the flow
|
||
again from the start.
|
||
|
||
For model-call details, the agent adds `--ai-trace` and reads the trace
|
||
with [unbox-ai](/debugging#agent-steps). Pair it with `--no-cache` to
|
||
capture every model call in the flow. Don't open the raw trace in an
|
||
agent's context. It repeats prompts and can be several megabytes.
|
||
|
||
## Generate a verification skill for your app
|
||
|
||
The e2e skill teaches the runner, not your app. The
|
||
`create-verification-skill` skill writes a project-local
|
||
`.agents/skills/verify-<app>/` skill that does, and tests it one time
|
||
before it hands over.
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx skills add e2e-inc/e2e --skill create-verification-skill
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm dlx skills add e2e-inc/e2e --skill create-verification-skill
|
||
```
|
||
|
||
```bash bun
|
||
bunx skills add e2e-inc/e2e --skill create-verification-skill
|
||
```
|
||
</CodeGroup>
|
||
|
||
Then ask the agent for `/create-verification-skill`.
|
||
|
||
<Accordion title="What the generated skill holds">
|
||
- Launch and doctor commands from your config.
|
||
- The two ways to drive the app: a live `e2e mcp` session and a test.
|
||
- The evidence a proof keeps.
|
||
- A feature map with one `e2e explore` charter for each feature, so a
|
||
[bug bash](/bug-bash) fans out from the map.
|
||
|
||
The generated skill says only what is true of your app and links to the e2e
|
||
skill for the rest. It is based on
|
||
[create-verification-skill](https://github.com/cursor/plugins/tree/main/pstack/skills/create-verification-skill)
|
||
from poteto's pstack, with e2e in place of a hand-rolled harness.
|
||
</Accordion>
|
||
|
||
## Migrate an existing suite
|
||
|
||
The `migrate-to-e2e` skill moves a Playwright, Cypress, Selenium, Appium,
|
||
Detox, Maestro, or TesterArmy suite to e2e one slice per session. The old suite keeps
|
||
running beside the new tests, so you can stop after any session.
|
||
|
||
<CodeGroup>
|
||
```bash npm
|
||
npx skills add e2e-inc/e2e --skill migrate-to-e2e
|
||
```
|
||
|
||
```bash pnpm
|
||
pnpm dlx skills add e2e-inc/e2e --skill migrate-to-e2e
|
||
```
|
||
|
||
```bash bun
|
||
bunx skills add e2e-inc/e2e --skill migrate-to-e2e
|
||
```
|
||
</CodeGroup>
|
||
|
||
Then ask the agent for `/migrate-to-e2e`. It ports the slice you pick and
|
||
runs each ported test three times on every target. A `MIGRATION.md` file
|
||
records progress for the next session. Optionally, the agent plants a temporary bug
|
||
in the app, such as a wrong checkout total, and checks whether both
|
||
suites fail on it. The agent deletes an old test only when you confirm. It first reads the migration guide for your runner:
|
||
[Playwright](/migrate/playwright), [Cypress](/migrate/cypress),
|
||
[Selenium](/migrate/selenium), [Appium](/migrate/appium), [Detox](/migrate/detox),
|
||
[Maestro](/migrate/maestro), or [TesterArmy](/migrate/testerarmy).
|
||
|
||
## Next
|
||
|
||
<CardGroup cols={2}>
|
||
<Card title="CLI reference" icon="terminal" href="/reference/cli">
|
||
init, guide, mcp, and run, flag by flag.
|
||
</Card>
|
||
<Card title="Debugging a run" icon="bug" href="/debugging">
|
||
The report, the artifacts, and the agent flags.
|
||
</Card>
|
||
</CardGroup>
|