* docs: add a Next.js guide page
One page takes a Next.js developer from `e2e init` to CI: `next dev`
locally and `next start` on the build in CI from one config, the port,
`reuseExisting`, and the Next.js problems (`allowedDevOrigins` on
127.0.0.1, the route announcer that `getByRole('alert')` also matches, a
webpack watcher that reads `.e2e/`), plus the `create-next-app` starter
and the Vercel template. The hydration accordion moves here from web.mdx;
web, cache, and example-projects link to the page, with ja and zh
translations, and the e2e skill's setup reference carries the same
guidance.
Refs ENG-1378
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: move the Next.js page into a Frameworks group
Next.js is not a platform like web, mobile, or desktop, so the page moves
to docs/frameworks/nextjs.mdx under a new Frameworks navigation group. The
web page no longer points at it from the app URL paragraph.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(notte): add @e2e-dev/notte, a browser provider for Notte cloud browsers
notte() runs each test attempt in its own Notte cloud browser session and
stops it when the attempt ends. It leases in attempt scope only: a Notte
session refuses Target.createBrowserContext, which worker scope needs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(notte): downloads, sweep, and console links, verified on the testbed
- downloads.read copies each download from the session's file storage,
found by the site's suggested name. @e2e-dev/web now passes that name to
providers as BrowserDownloadContext.suggestedFilename (optional,
additive).
- sweep stops the sessions of workers that died, found through per-session
marker files under the system temp directory, since Notte sessions have
no tags.
- The progress line links the session's console page instead of the
token-bearing viewer URL.
- A start response without cdp_url falls back to the session's debug info.
- apps/testbed gets e2e.notte.config.ts and test:notte.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(notte): address review: fail-closed ledger, required suggestedFilename, paged downloads, shared sweep helper
* docs(notte): add notte() to the security table and skill setup
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
Manage local Moli CDP browsers through the web engine's BrowserProvider API.
Each worker gets an available loopback port, layout and resource loading,
startup cancellation, and process cleanup when its lease ends.
Wire the testbed, documentation, consumer skill, build and test commands,
and a patch changeset for @e2e-dev/moli.
Keep process-group shutdown active after an executable wrapper exits,
and surface process-inspection failures in the provider tests.
Verified locally:
- pnpm check
- 16 Moli provider unit tests
- Testbed on two workers: 16 passed, 1 expected skip; both browsers released
- Empty and whitespace CDP URLs use managed browsers; padded URLs attach
without stopping the existing server
- MCP login, secret fill, assertions, and screenshot
- Packed packages installed, typechecked, and run in an isolated project
* docs(ci): rename the Vercel page to Vercel previews
* docs(ci): move the Vercel previews page to /ci/vercel-previews
* docs(ci): capitalize Vercel Previews
* docs(ci): drop the redirect, the page is not shipped
* docs(ci): add a GitLab CI/CD page
* docs(ci): pin the CI device, bound the emulator boot, and note GitLab masking rules
* docs(ci): add the GitLab job to an existing pipeline, set the key with glab
* docs(ci): note --pass-with-no-tests for GitLab shards
* docs(ci): give each GitLab Android job its own emulator port, list the JDK
* docs: explain agentic and deterministic testing
* docs: refine agentic and deterministic testing guidance
* docs: clarify when to use agentic and deterministic tests
* docs: cover agentic assertions alongside actions
* improveoments
* docs: drop the sidebar icon from the agentic page
* docs: mention monitoring with real data
* docs: clarify what agent assertions check
* docs: clarify the agentic assertions heading
* docs: point agentic monitoring at test accounts and locator checks
* feat(engine): engine protocol for out-of-process engines
The engine contract as LSP-style JSON-RPC: connectEngine turns a server
into an engine handle, serveEngine serves any defineEngine handle, and
spawn, socket, and loopback transports reach it. Abort reasons, errors,
and pixels cross intact; the manifest has a JSON schema for servers in
other languages. E2E_ENGINE_LOOPBACK=1 runs every engine through the
protocol in process; the testbed runs that way in CI.
* feat(engine): run every in-process engine over the protocol, drop E2E_ENGINE_LOOPBACK
* refactor(engine): drive engine protocol from one method table
- one METHODS table feeds describeEngine, server and client
- client builds the engine from the manifest, defineEngine validates it
- harness holds only protocol clients (EngineClientHandle)
- signals scoped to attempts, drop $/release
- worker connections queue for MCP session handover
- track attempts for servers without startAttempt
- locate/stopVideo return arrays, keyboard/type takes options
* feat(engine): let connectEngine answer session hooks on the client
Host-side relaunch (simctl) without wrapping the client in defineEngine,
which made the harness serve it over loopback a second time.
* docs(engine): group engine docs into one Engines section
New Engines overview and Engine servers guide; the protocol reference
keeps signatures, messages and errors.
* docs(engine): complete config examples, drop leftover OG card
* fix(engine): harden protocol from review
- bound finish/dispose hooks by their budget, always close transport
- late endAttempt retires only its own attempt (client and server)
- reject malformed error responses instead of throwing from stream
- resolve a secret once; server derives redacted forms via attempt/deriveSecret
- carry error stack, errno, and causes across the wire
- serveEngine always answers startAttempt/endAttempt, so signals are freed without hooks
- detach signal listeners when a connection closes
- refuse a server speaking another protocol version
- docs: platform in connectEngine examples, message and error updates
* fix(engine): fixed diagnostic for malformed error response
* docs(migrate): add an overview page for the migration guides
* docs(migrate): apply review fixes to the overview page
* docs(migrate): note JavaScript tests on the overview page
* docs: add a Writing fixtures guide
* docs: list Writing fixtures after Project tools and fold its long example
* docs: show fixtures creating test data and leave sign-in to saved sessions
* docs: create fixture data with agent.act instead of a test API
* docs: cut the Writing fixtures page to the essentials
* docs: import expect from e2e in the fixtures example
* docs(ci): move the CI guide to Get started as Setting up CI
* docs(ci): keep Continuous integration as the title, Setting up CI in the sidebar
* docs(quickstart): keep the code-branch icon on the CI card
Mintlify hosts its own MCP server at <docs url>/mcp, and that endpoint
wins over a page with the same slug, so /docs/mcp served JSON instead of
the guide. Rename the page to mcp-server and point every link at it.
The footer reads "Built by TesterArmy | Powered by Mintlify" on one line,
GitHub alone on the left; on a phone the Mintlify credit takes a second,
right-aligned line. Mintlify's footer takes no markup, so the credit is a
website social restyled in style.css with the brand kit's mark and wordmark
(images/footer/). Each image is two url() layers, one path for mint dev
and one for production's /docs prefix, since Mintlify inlines the
stylesheet into the page.
The guide answers what most readers ask: how secrets stay out of the model,
what can still leak, and what runs with your permissions. Every rule moves to
docs/reference/security.mdx. smol host-loopback access moves to the smol
page's warning. Fixes facts that no longer matched src/: defineTool errors are
INVALID_CONFIG, the CI cache default skips a cache.store, start-screen replay
matches routes, and outbound connections now list login, models, init, and the
smol image pull.
* feat(decision): add Clef and Jev executors on the System One choice API
* feat(decision): default Clef transport to clef-flash
* feat(decision): generalize to decision models, address review
* fix(decision): reject non-string decision choices
* fix(decision): skip empty select labels and blank destinations
An empty option label or whitespace-only destination offered as a choice fails at dispatch with INVALID_ARGUMENT and aborts the step instead of letting the model re-aim. Omit them at enumeration. Also qualify the uncertain-action docs: blocking stops the selected action, while earlier confident actions in the step have already run.
* feat(decision): add opt-in vision for Clef, Jev stays text-only
The executor stays text-only by default. With vision true and a transport that declares it, masked viewport pixels ride along as System One images data URLs. Clef declares vision; Jev never receives pixels. Withheld or tainted viewports fall back to semantic text.
* docs(decision): qualify vision cost to calls carrying pixels
* fix(decision): untainted pixel fixtures and honest skill pixels line
The test helper takes a tainted flag so the image-send tests use the only state the runner returns pixels in; tainted captures stay withheld observations. The skill no longer claims pixels are unavailable: screenshots may inform node choices with vision true, without adding point actions.
* bench(testbed): luna vs clef-flash harness and first numbers
Same todo flow under two configs: default agent on gpt-6-luna via OpenCodeX and decisionExecutor on clef-flash. Luna side passed in 23s with 6 model calls; clef-flash pending credentials. Includes the probe script, the docs section, and the flow screenshot.
* docs(decision): fill clef-flash benchmark row
Two no-cache runs both blocked uncertain on the first action (a25 at p=0.619, conf=0.387 against the default 0.9 gates), answering in about a second with 1,543 input tokens.
* docs(decision): benchmark aggregates over 5 runs per side
Luna completed 5/5 flows at ~14s and ~25k input tokens per run; clef-flash blocked uncertain 5/5 at ~1s and ~1.5k tokens per run.
* bench(testbed): parametrized flow, gate sweep, wire-shape finding
Params fix the missing type choices; gates 0.6/0.5 and 0.51/0.3 still block. Direct experiment shows hierarchical structured questions take clef-flash from 0.57 to 0.89 confidence on the same screen.
* feat(decision): hierarchical operation-then-target on structured transports
Structured transports (Clef, Jev) now decide the operation first and the target within it, with object instructions and element records. Flat endpoints keep the single string choice. Live benchmark: first operation 0.909/0.802, first target 0.940/0.775, up from 0.747/0.558 flat.
* docs(decision): compare luna vs clef-flash vs clef with measured cost
* feat(decision): run decisions through AI SDK evaluate, gates opt-in
decisionExecutor now accepts any AI SDK evaluation model that answers
choice questions, such as typeSafeAi.evaluationModel('jev-latest'), and
decides through experimental_evaluate (one choice question per decision,
no retries, so the call budget stays exact). ai is an optional peer that
loads only on that path. evaluationModel() exposes clef(), jev(), and
systemOne() transports to experimental_evaluate, which covers Clef until
it has an AI SDK provider.
The official TypeSafe provider rounds probabilities to two places, so a
long distribution can sum to 0.99: validation now allows half a unit per
rounded option, the AI SDK rule, and jev() declares the two places.
Probability and confidence gates are off by default and stay
configurable. Measuring clef-flash without them surfaced three executor
bugs, each fixed here:
- a lone target is dispatched without a one-option question, which the
choice API rejects with HTTP 422;
- acting decisions receive the step's action history, and the repeat
guard counts the same action on the same screen across the whole step,
so an A-B oscillation blocks instead of burning the call budget;
- the operation and target questions state that typed text is not saved
until it is submitted.
The structured path also checks the call budget before the target
request (review feedback).
Testbed todo flow, 5 no-cache runs each: clef-flash 5/5 (9.3s mean, 12
calls), clef 5/5 (11.5s, 12 calls), luna unchanged at 5/5 (14.9s).
* bench(testbed): Laya through systemOne
Laya serves the System One API and accepts structured values, so it runs
through systemOne({ structured: true }) unchanged. On the todo flow its
first operation question chose unsupported at 0.29 / 0.10 on four runs,
answered by the multilingual checkpoint its router picked for an English
prompt; the fifth run and further probes hit insufficient_credits.
* fix(decision): address review on rounding, history, and evaluation ids
- validateDecision accepts a rounded distribution only when 1 lies between
the sums of each value's rounding interval, clipped to [0, 1]; a summed
per-option tolerance exceeded 1 with many options and admitted two
options at probability 1.
- Decisions send the 20 most recent history entries, each clipped to 240
characters, so long steps and large params stay within request limits.
- evaluationModel() builds its answer, confidence, and criteria maps with
null prototypes, so an id such as __proto__ stays an own entry.
- @ai-sdk/provider moves to dependencies: the public declarations import
its evaluation types. ai stays an optional peer.
* bench(testbed): pin Laya's English checkpoint
Laya's router sent the full English prompt to its multilingual checkpoint.
The API accepts model 'english' to pin the English one (its routing
metadata reports the explicit model), so the bench config sends it.
* docs(decision): benchmark Laya locally on M1, both checkpoints 0/5
* docs(decision): reconcile laya narrative with run evidence, money not tokens
* feat(decision): caller-overridable transport flags on clef() and jev()
* feat(decision): AI SDK evaluate executor with element table and text model
* fix(decision): empty-tree check and provider pin
* fix(decision): address review on secrets, options, errors, and docs
Answers the cubic threads on the evaluate-only executor: terminal checks reject an incomplete screen instead of judging it, secret markers are stripped at any depth, hidden/disabled options are never offered, select options share the 255-choice cap, select criteria name the option, out-of-range probabilities fail, text-model errors mirror the decision mapping, and the README/docs describe the evaluate-only API.
* fix(decision): count dropped select options and fix README
Counts enabled options inside rows dropped past the element cap as omitted, adds the E2EConfig type import to the README example, and qualifies when type is offered.
* fix(decision): settle terminal claims by the check and match codebase style
The second rejected done/failed claim now ends by the check verdict (fails -> ACTION_FAILED, anything else -> blocked), and the completion check honors the gates. The decision state carries the last 10 actions, check targets show their checked state, checked radios are never toggled, and options dropped by the per-question cap stay out of omitted. The docs example opens /todos in test code since navigate is not offered. Single quotes, template literals, and split long lines across the package.
* fix(decision): show filled fields to verdicts and pin the history window
Verdict views (assert and the completion check) build the element table with typing enabled, so a field the step filled keeps its row and value even when the step itself cannot type. The history test now asserts the window holds the newest ten actions.
* fix(decision): let completion checks see results and name the text key
Page text keeps a named node's text when it differs from the name, as formatNode does, so a status named Greeting that says Welcome back, admin! reaches the verdict. The completion check now asks whether the task is complete and sees the non-secret params and the actions with their typed values; rejected claims stay out. The text prompt names the single text key, asks for no code fence, and prefixes the request so JSON-object response modes accept it.
* fix(decision): keep secret handles out of replayed history and mark check inputs as data
Replayed summaries name a filled secret handle; seeding the history now replaces declared secret names with <secret>, so names reach the model only as secret question criteria. The completion check treats action descriptions, typed values, and params as data, and the guide says the check sees only non-secret values.
* fix(decision): redact only the handle in replayed secret-fill summaries
Matching the declared secret name anywhere in a replayed summary also erased labels and values equal to it, and missed handles the runner cut at 40 characters. Only the quoted handle after fill secret at the start of the summary becomes <secret> now.
* docs(decision): describe the one-request fan-out in the guide intro
* fix(decision): count a scroll that changes what is in view as progress
The fingerprint behind the stall guard hashed node content but not position, so a scroll that only moved the viewport read as page unchanged and the third one blocked the step. It now also hashes whether each node meets the viewport (membership, not coordinates), so a scroll that brings other nodes into view resets the count while one that moves nothing, as at the page bottom, still trips the guard.
* chore: prune @pnpm/exe from the lockfile after merging main
* fix(decision): match the supported Node range in engines
---------
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
* docs(mobile): document physical devices
Closes#766
* docs(mobile): phone requirements in setup and README
* docs(mobile): move physical devices to its own page
* docs: add a light mode that follows the system theme
The site was dark only (appearance.strict). It now follows the reader's
system and the navbar toggle switches it. The logo has a light and a dark
file, code blocks are one flat surface in both modes, and the Expo card
icon is drawn black and inverted in dark mode.
The light logo is a placeholder: the white PNG recolored to ink, which
inverts the mascot's shading.
* docs: update the light-mode logo
* docs: the video card thumbnail uses the surface color in both modes
--e2e-surface now has a light value on :root, so the hairline fallback
never applies.