mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-21 03:05:26 +08:00
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
408 lines
14 KiB
Plaintext
408 lines
14 KiB
Plaintext
---
|
|
title: Subscriptions
|
|
description: Use your ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, your Claude Max or Team plan's API credits, or your Claude Code or Codex login for agent steps.
|
|
"og:image": "https://e2e.com/docs/images/og/subscriptions.png?v=174d49a0e9"
|
|
"twitter:image": "https://e2e.com/docs/images/og/subscriptions.png?v=174d49a0e9"
|
|
---
|
|
|
|
Run agent steps on a plan you already pay for. Sign in one time, then use
|
|
the plan's model in `e2e.config.ts`.
|
|
|
|
| Plan | How to connect |
|
|
| --- | --- |
|
|
| ChatGPT Plus or Pro | `npx e2e login openai` |
|
|
| GitHub Copilot | `npx e2e login github-copilot` |
|
|
| OpenCode Console (OpenCode Zen and OpenCode Go) | `npx e2e login opencode-console` |
|
|
| SuperGrok or X Premium+ | `npx e2e login spacexsi` |
|
|
| Claude Max or Team (Claude Code) | API key, paid from the plan's monthly API credits, not its usage limits |
|
|
| Claude Code or Codex login | [Run steps on your coding agent](#use-your-coding-agent) |
|
|
|
|
## Set up
|
|
|
|
For a new project, run `e2e init` and select your plan
|
|
([quickstart](/quickstart)). For an existing project, install the packages,
|
|
sign in, and set `agents: { default: agent }` in `e2e.config.ts`.
|
|
|
|
<Tabs>
|
|
<Tab title="ChatGPT">
|
|
<CodeGroup>
|
|
```bash npm
|
|
npm install -D ai @ai-sdk/openai
|
|
npx e2e login openai
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm add -D ai @ai-sdk/openai
|
|
pnpm exec e2e login openai
|
|
```
|
|
|
|
```bash bun
|
|
bun add -d ai @ai-sdk/openai
|
|
bunx e2e login openai
|
|
```
|
|
</CodeGroup>
|
|
|
|
```ts title="Model setup"
|
|
import { chatgpt } from 'e2e/oauth/chatgpt';
|
|
|
|
const agent = { model: chatgpt('gpt-6-luna') };
|
|
```
|
|
</Tab>
|
|
<Tab title="GitHub Copilot">
|
|
<CodeGroup>
|
|
```bash npm
|
|
npm install -D ai @ai-sdk/openai-compatible @ai-sdk/openai
|
|
npx e2e login github-copilot
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm add -D ai @ai-sdk/openai-compatible @ai-sdk/openai
|
|
pnpm exec e2e login github-copilot
|
|
```
|
|
|
|
```bash bun
|
|
bun add -d ai @ai-sdk/openai-compatible @ai-sdk/openai
|
|
bunx e2e login github-copilot
|
|
```
|
|
</CodeGroup>
|
|
|
|
```ts title="Model setup"
|
|
import { copilot } from 'e2e/oauth/copilot';
|
|
|
|
const agent = { model: copilot('claude-sonnet-5') };
|
|
```
|
|
|
|
Which models your plan serves is up to GitHub; see [Pick a model](#pick-a-model).
|
|
</Tab>
|
|
<Tab title="OpenCode Console">
|
|
<CodeGroup>
|
|
```bash npm
|
|
npm install -D ai @ai-sdk/openai-compatible @ai-sdk/openai @ai-sdk/anthropic @ai-sdk/google
|
|
npx e2e login opencode-console
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm add -D ai @ai-sdk/openai-compatible @ai-sdk/openai @ai-sdk/anthropic @ai-sdk/google
|
|
pnpm exec e2e login opencode-console
|
|
```
|
|
|
|
```bash bun
|
|
bun add -d ai @ai-sdk/openai-compatible @ai-sdk/openai @ai-sdk/anthropic @ai-sdk/google
|
|
bunx e2e login opencode-console
|
|
```
|
|
</CodeGroup>
|
|
|
|
```ts title="Model setup"
|
|
import { opencodeConsole } from 'e2e/oauth/opencode-console';
|
|
|
|
// OpenCode Zen, billed to the workspace's credits
|
|
const agent = { model: opencodeConsole('deepseek-v4.1-flash') };
|
|
// OpenCode Go, billed to your Go subscription
|
|
const goAgent = { model: opencodeConsole('go/deepseek-v4.1-flash') };
|
|
```
|
|
|
|
On the Anthropic route, a call that sets no `maxOutputTokens` asks for 8192: the
|
|
Anthropic SDK does not know the ids OpenCode serves there, and on its own it
|
|
limits the request to 4096 tokens and only warns.
|
|
</Tab>
|
|
<Tab title="SuperGrok">
|
|
<CodeGroup>
|
|
```bash npm
|
|
npm install -D ai @ai-sdk/xai
|
|
npx e2e login spacexsi
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm add -D ai @ai-sdk/xai
|
|
pnpm exec e2e login spacexsi
|
|
```
|
|
|
|
```bash bun
|
|
bun add -d ai @ai-sdk/xai
|
|
bunx e2e login spacexsi
|
|
```
|
|
</CodeGroup>
|
|
|
|
```ts title="Model setup"
|
|
import { grok } from 'e2e/oauth/grok';
|
|
|
|
const agent = { model: grok('grok-4') };
|
|
```
|
|
</Tab>
|
|
<Tab title="Claude Max or Team">
|
|
Claude plans have no sign-in for other apps. To run agent steps on Claude
|
|
Code itself, with its own login, see [your coding agent](#use-your-coding-agent). To use the
|
|
plan's API credits in a new project, select `None` for the model gateway in
|
|
`e2e init`. Then:
|
|
|
|
1. In claude.ai, link a Claude Console organization. On Max, go to
|
|
Settings > Billing. On Team, an Owner or Primary Owner goes to
|
|
Organization settings > Billing. See [Anthropic's guide](https://support.claude.com/en/articles/17154008-monthly-api-credits-for-max-and-team-plans).
|
|
2. In that organization, create an API key. Set it as `ANTHROPIC_API_KEY`.
|
|
3. Install the packages and set the model:
|
|
|
|
<CodeGroup>
|
|
```bash npm
|
|
npm install -D ai @ai-sdk/anthropic
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm add -D ai @ai-sdk/anthropic
|
|
```
|
|
|
|
```bash bun
|
|
bun add -d ai @ai-sdk/anthropic
|
|
```
|
|
</CodeGroup>
|
|
|
|
```ts title="Model setup"
|
|
import { anthropic } from '@ai-sdk/anthropic';
|
|
|
|
const agent = { model: anthropic('claude-sonnet-5-5') };
|
|
```
|
|
|
|
Set `agents: { default: agent }` in `e2e.config.ts`.
|
|
|
|
Unused credits expire at the end of each billing cycle. Other Claude plans
|
|
have no API credits. On those, use an [API provider](/models), or a Claude
|
|
model through your Copilot plan.
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Pick a model
|
|
|
|
`e2e models` lists the model ids your login can use:
|
|
|
|
<CodeGroup>
|
|
```bash npm
|
|
npx e2e models # every stored login
|
|
npx e2e models openai # one provider
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm exec e2e models # every stored login
|
|
pnpm exec e2e models openai # one provider
|
|
```
|
|
|
|
```bash bun
|
|
bunx e2e models # every stored login
|
|
bunx e2e models openai # one provider
|
|
```
|
|
</CodeGroup>
|
|
|
|
The id is not always the name the vendor's app shows.
|
|
|
|
With `OPENCODE_API_KEY` set, `e2e models opencode-console` lists the ids that
|
|
key serves, the same key `opencodeConsole()` uses in place of the login.
|
|
|
|
- **Copilot** marks `not enabled` (turn the model on in your Copilot
|
|
settings on GitHub, or ask the admin who manages your seat) and `no chat
|
|
or responses` (use a different model). It can't see what your plan
|
|
serves: on Copilot Free, GitHub serves models through
|
|
[auto model selection only](https://docs.github.com/en/copilot/get-started/plans),
|
|
so a model called by id is usually refused. The run then stops with
|
|
`MODEL_UNAVAILABLE` naming it; pick another id or an
|
|
[API provider](/models).
|
|
- **OpenCode Console** lists `go/` models only if you have Go. It leaves out
|
|
models your workspace turned off, and Zen's free models. Key
|
|
`providerOptions` by `opencode` for chat completions models, else by
|
|
`openai`, `anthropic`, or `google`.
|
|
|
|
## Sign-in help
|
|
|
|
- ChatGPT opens a browser. To use a device code, add `--device`.
|
|
- Copilot uses your GitHub CLI login. Without `gh`, pass `--client-id` with
|
|
your own GitHub OAuth app that has device flow on. For GitHub Enterprise,
|
|
add `--enterprise-url`.
|
|
- OpenCode Console shows a device code. Approve it and select the
|
|
workspace. The page shows e2e as an unknown client. To change the
|
|
workspace, run `npx e2e logout opencode-console` and sign in again.
|
|
- SuperGrok shows a device code and the URL where you enter it.
|
|
|
|
`npx e2e login` with no provider opens a picker that shows which accounts
|
|
are signed in. `npx e2e logout` removes a login. When a saved login stops
|
|
working, the step fails with `MODEL_PROVIDER_FAILED`, and the message gives
|
|
the `npx e2e login <provider>` command to run. See the
|
|
[CLI reference](/reference/cli#e2e-login) for every flag.
|
|
|
|
Model-call cancellation also applies while waiting for a token refresh.
|
|
|
|
When a login saved with `npx e2e login` expires, parallel workers refresh it
|
|
once between them. A worker waits up to 30 seconds for another worker's
|
|
refresh. After that its step fails with `MODEL_PROVIDER_FAILED`, unless the
|
|
step's own timeout ends it first, as `STEP_TIMEOUT`.
|
|
|
|
## CI
|
|
|
|
Use API keys in CI. Claude Max or Team already uses one. For OpenCode
|
|
Console, set `OPENCODE_API_KEY` to an OpenCode Go subscription API key, or to
|
|
a service account API key created in the Console (Service Accounts, then API
|
|
Keys). For ChatGPT, Copilot, and SuperGrok, use an [API provider](/models).
|
|
|
|
[`E2E_OAUTH_CREDENTIALS`](/reference/environment) can give a stored login to
|
|
CI, but e2e doesn't save refreshed tokens back to it. A login with a
|
|
rotating refresh token works there only for a limited time.
|
|
|
|
## Use your coding agent
|
|
|
|
`@e2e-dev/acp` runs `agent.act` and `agent.assert` on Claude Code or Codex
|
|
over the [Agent Client Protocol](https://agentclientprotocol.com). The
|
|
agent signs in with its own login or API key, so e2e needs no model
|
|
provider.
|
|
|
|
The agent gets `complete_step` and the built-in agent's action tools for
|
|
the verbs the target's engine supports. The runner records every action, so
|
|
steps replay from the [cache](/cache) without the agent. Your tests don't
|
|
change.
|
|
|
|
<Tabs>
|
|
<Tab title="Claude Code">
|
|
Install the package and Zed's Claude Code adapter. Sign in with `claude`, or
|
|
set `ANTHROPIC_API_KEY`.
|
|
|
|
<CodeGroup>
|
|
```bash npm
|
|
npm install -D @e2e-dev/acp @agentclientprotocol/claude-agent-acp
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm add -D @e2e-dev/acp @agentclientprotocol/claude-agent-acp
|
|
```
|
|
|
|
```bash bun
|
|
bun add -d @e2e-dev/acp @agentclientprotocol/claude-agent-acp
|
|
```
|
|
</CodeGroup>
|
|
|
|
```ts title="e2e.config.ts"
|
|
import type { E2EConfig } from 'e2e';
|
|
import { web } from '@e2e-dev/web';
|
|
import { claudeCode } from '@e2e-dev/acp';
|
|
|
|
export default {
|
|
targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
|
|
agents: { default: { executor: claudeCode({ model: 'sonnet' }) } },
|
|
} satisfies E2EConfig;
|
|
```
|
|
</Tab>
|
|
<Tab title="Codex">
|
|
Install the package and the Codex adapter. Sign in with `codex login`.
|
|
|
|
<CodeGroup>
|
|
```bash npm
|
|
npm install -D @e2e-dev/acp @agentclientprotocol/codex-acp
|
|
```
|
|
|
|
```bash pnpm
|
|
pnpm add -D @e2e-dev/acp @agentclientprotocol/codex-acp
|
|
```
|
|
|
|
```bash bun
|
|
bun add -d @e2e-dev/acp @agentclientprotocol/codex-acp
|
|
```
|
|
</CodeGroup>
|
|
|
|
```ts title="e2e.config.ts"
|
|
import type { E2EConfig } from 'e2e';
|
|
import { web } from '@e2e-dev/web';
|
|
import { codex } from '@e2e-dev/acp';
|
|
|
|
export default {
|
|
targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
|
|
agents: { default: { executor: codex() } },
|
|
} satisfies E2EConfig;
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
`claudeCode()` and `codex()` take these options:
|
|
|
|
| Option | What it does |
|
|
| --- | --- |
|
|
| `model` | A model the agent offers, such as `sonnet`. When the agent doesn't offer it, the session fails to start and the error lists the models it does offer. Omitted, the agent's default runs. |
|
|
| `env` | Variables added to the agent's environment. |
|
|
| `system` | Text appended to the rules the agent gets with the first step. |
|
|
| `name` | The executor name in reports. |
|
|
|
|
Unexpected forwarding failures stop the ACP session and appear in the runner's
|
|
secret-redacted failure output. The HTTP client receives a generic error or a
|
|
closed stream, without the exception text.
|
|
|
|
### What the agent can use
|
|
|
|
An action the agent takes with its own tools isn't recorded, can't replay,
|
|
and can reach files or the web. So e2e keeps the agent to the step tools:
|
|
|
|
- The agent process starts in your project. Its session opens in an empty
|
|
temporary directory, and e2e offers it no file system or terminal access.
|
|
- e2e allows permission requests for step tools and rejects all others.
|
|
- When a tool of the agent's own runs without asking, e2e stops the turn
|
|
and fails the step with `POLICY_DENIED`. Later step tool calls take no
|
|
action. The session closes, so the next step starts a fresh agent that
|
|
never saw what the tool read.
|
|
|
|
Each preset also turns off the agent's own tools that run without asking:
|
|
|
|
- **Claude Code**: no built-in tools, no user, project, or local settings,
|
|
no MCP servers but the step tools, no saved session, and no bypass of
|
|
permission prompts.
|
|
- **Codex**: read-only mode, no shell, web search, apps, plugins, browser,
|
|
or computer use. Every MCP server from your Codex config is off for these
|
|
sessions only. Codex can still start a subagent, which fails the step like
|
|
any tool of its own.
|
|
|
|
### Secrets
|
|
|
|
A secret param reaches the agent by name only. When the target's engine can
|
|
fill secrets, the agent calls `type_secret` and the runner types the value,
|
|
as for the built-in agent.
|
|
|
|
The agent's process gets your environment without the `E2E_SECRET_*` and
|
|
`E2E_USER_*` variables.
|
|
|
|
<Warning>
|
|
A secret in any other variable reaches the agent. Examples are `API_KEY`
|
|
behind `secrets: { apiKey: process.env.API_KEY }`, or the variable an
|
|
`E2E_SECRET_*` value was copied from. Keep secrets in `E2E_SECRET_*` and
|
|
`E2E_USER_*` only.
|
|
</Warning>
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="How a test runs">
|
|
One agent session serves one test attempt. The first agent step that the
|
|
cache can't replay starts it. Each later step is one more message in the
|
|
same conversation, and the session closes when the attempt ends, or
|
|
earlier on a `POLICY_DENIED` step.
|
|
|
|
Each message carries the step, its non-secret params, the steps done so
|
|
far, and the current screen. Action results say what changed on screen, and
|
|
a `screenshot` reaches the agent as an image.
|
|
|
|
An assertion can't change the screen. `observe`, `screenshot`, scrolling,
|
|
and hovering still work. The agent ends every step with `complete_step`:
|
|
`passed`, `failed`, or `blocked` with an error code, as the built-in agent
|
|
does.
|
|
</Accordion>
|
|
<Accordion title="Limits">
|
|
- The Agent Client Protocol doesn't report model requests. Each prompt
|
|
turn counts as one model call against `maxModelCalls`, with its tokens
|
|
and cost when the agent reports them. `maxSteps` and the step timeout
|
|
bound the turn.
|
|
- `waitFor` and `extract` use the built-in judgments, so they need the
|
|
agents entry's `model`.
|
|
- An agents entry with an `executor` takes no `tools`, so the agent gets no
|
|
project tools from `defineTool`.
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
## Next
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Models" icon="brain" href="/models">
|
|
Use an API key, a gateway, or a local model instead.
|
|
</Card>
|
|
<Card title="Agents and personas" icon="users" href="/agents">
|
|
Give tests different models and instructions.
|
|
</Card>
|
|
</CardGroup>
|