Commit Graph
485 Commits
Author SHA1 Message Date
Satyajit Sahoo 36f5c58d9d feat(mcp): attach a session to a test paused at screen.pause()
With `open_session {attach: true}` on `e2e mcp`, a coding agent can attach
a session to the test that's paused in an `e2e run` in the server's
directory. The agent gets the same browser or device the test is using, on
the paused screen.

- The session is only detached by `close_session`, so the test stays
  paused.
- The session ends when the test is resumed or the run ends. The agent
  resumes the test with `e2e run --step next`, or ends the run with
  `e2e run --step stop`.
- The session is shown as one step of the test.
- A session also attaches to a test held at a step in an `e2e run --step`
  without a terminal. An `e2e run --step` in a terminal doesn't listen for
  commands, so a session can't be attached to it.
2026-11-19 12:46:29 +01:00
Oskar Kwaśniewski 0e9d58ec98 feat(mcp)!: remove the run_tests tool (#1967) 2026-11-19 13:31:59 +02:00
Satyajit Sahoo 34322c5416 feat(cli): pause at screen.pause() under --watch (#1905)
Under `e2e run --watch`, a test is now paused at each `screen.pause()` on
every run, as in `e2e run`.

While a pause is shown, the keys are read by the pause prompt and not by
watch mode:

- Press Enter to resume the test.
- Press `q` to cancel that run and stay in watch mode.

`e2e run --step next` and `stop` can be used from another shell, as for a
run without `--watch`.
2026-11-19 11:15:49 +00:00
Satyajit Sahoo 4893d1dd75 feat: add screen.pause() (#1904)
With `await screen.pause()`, a test is paused at that line, with the app
open, until someone resumes it.

- In a terminal, press Enter to resume the test.
- From another shell in the same directory, run `e2e run --step next`.
  A run without a terminal is resumed this way.
- Paused time doesn't count toward the test's timeout.
- With `CI` set, the test fails with `PAUSE_IN_CI`.
- Under `--step`, a test is still paused after `n` or `a`.
- A test isn't paused in an MCP `open_session {test}` session, under
  `openHost` or `run()`, or in a core program.
- A test isn't paused under `e2e run --watch` yet.
2026-11-19 11:00:39 +00:00
Milosz JankiewiczandOskar Kwaśniewski 5e9b82e6ae feat(oauth): name the SuperGrok provider spacexsi, with spacexai and grok as aliases (#1943)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-19 12:07:52 +02:00
bardropandOskar Kwaśniewski a213247170 feat(test): add test.each and test.for (#1882)
* feat(test): add test.each and test.for

* fix(test): read %i and %f titles like parseInt/parseFloat, as Vitest does

* fix(test): keep each titles on one line and spread only all-array tables

* fix(test): type unknown each rows loosely and keep only real cycles as [Circular]

An unknown row may be spread or passed whole at run time, so EachArgs
now types only the fixtures, last. %j only reads Node's circular-structure
TypeError as a cycle; a toJSON that throws names the row. Docs say %d
keeps BigInt's n suffix and shows Symbol as NaN, and that a string title
value keeps its line breaks.

* docs(test): note that %i keeps the BigInt n suffix

* fix(test): print undefined for %j of a function or Symbol, read Unicode $keys

- %j prints undefined for what JSON leaves out, as Vitest does, so a
  function's source never lands in a title
- $name and its path read Unicode identifier characters
- inline the test mode back into registrar
- docs: note that a Vitest test.each body gets no fixtures

* fix(test): read test.each fixtures from the parameter after the row

Fixtures are now set up on demand from what the body destructures (#1891).
A test.each row runs a wrapper, (fixtures) => body(...row, fixtures), so
registration read the wrapper's own parameter and refused every row with
"must destructure its first parameter". registerTest now takes the body
and the position of its fixtures parameter (the row's length when it is
spread, else 1), and fixtureNames reads that parameter.

* fix(test): keep Error rows on one line and locate rows through a helper

- An Error in a title shows as its name and message, not its stack, and
  whatever inspect still breaks is joined, so a title is always one line
- The source location reads the whole stack: the default 10 frames cut the
  test file off a row declared through a helper and a describe
- A tagged template table is a COLLECTION_ERROR instead of one test per
  template chunk
- A bound row body is told how to wrap itself with the row first
- tests: a run sets up only the fixtures a row body names; fixtureNames at
  a position
- docs, skill, changeset: a row body names its fixtures after the row

* fix(test): restore the stack limit on a throw, read only an own raw

- stackLocations puts Error.stackTraceLimit back in a finally
- Only an own raw property marks a template table
- The wrap hint for a bound row body names one argument per row item
- skill: rows are spread only when every row is an array

---------

Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-19 08:56:16 +00:00
Asheesh Kumar 243eaa40ec fix(html-reporter): keep the previous page when a run stops before its tests (#1915)
* fix(html-reporter): keep the previous page when a run stops before its tests

writeReport never checked run.reportPath, so a run that stopped before
its tests (NO_TESTS, a config error, an app that never started) replaced
report.html with an empty page beside the previous run's report.json and
printed an HTML report row under "Report (not written)". It now returns
early with no row, as the built-in junit and markdown reporters do, and
the FinishedRun.reportPath docs say it is also absent when the run stops
before its tests.

Fixes #1897.

* docs(runner): say run()'s reportPath is also undefined when a run stops before its tests

The same field FinishedRun.reportPath documents, in reference/runner.mdx
and the ja and zh pages. The changeset is one line.
2026-11-19 02:36:26 +00:00
Oskar Kwaśniewski 9d240ac954 feat(core): report launch() sessions in the test that opens them (#1935)
A session a test opens with launch() from @e2e-dev/core was invisible to
the test's report, and a secret passed only to launch() could reach it.

- secrets: every owner of values (config, open host, open session) shares
  them through one refcounted map on globalThis; ledgers redact what is
  shared beside them without storing it
- report: a launch session's steps are mirrored into the test's recorder
  under an actor name (launch-1), with model usage and video
- lifecycle: the attempt owns hosts and sessions the test opened, cuts
  their steps when the body ends, and closes them at its end
- reporters: list, trace page, markdown, junit and html name the session
- errors a session throws no longer carry a secret in details
2026-11-18 16:37:55 +00:00
Krzysztof MochandOskar Kwaśniewski 847a09e6db feat(test)!: set fixtures up on demand, once per worker, and in beforeAll (#1891)
* feat(test): set fixtures up once per worker and pass them to beforeAll

test.extend() takes [fn, { scope: 'worker' }]: one instance per worker and
target, torn down when the worker ends. Suite hooks receive worker fixtures.
Engines can declare isolation: 'worker'; @e2e-dev/mobile does, so its device,
app, and screen reach beforeAll, afterAll, and worker fixtures.

Refs ENG-1297

* feat(e2e)!: set up fixtures on demand

A test now sets up only the test.extend() fixtures its body and its
beforeEach and afterEach hooks name in their first parameter, the fixtures
those name, and the auto ones. A suite hook sets up the worker fixtures it
names and the auto worker fixtures. Names are read from the function's
source, as Playwright and Vitest read them.

- [fn, { auto: true }] and [fn, { scope: 'worker', auto: true }] run a
  fixture nothing names.
- A fixture can name the fixtures declared beside it in the same extend()
  call. Fixtures run in declaration order, each after the ones it names.
- The attempt is resolved as one unit before the session launches: a
  fixture only an afterEach names is set up before the first beforeEach.
  An afterEach that names a fixture whose setup failed is skipped.
- Fixture rules fail as COLLECTION_ERROR with the file and line of the
  code: a parameter that is not destructured or takes a rest property, and
  a cycle, when the file is collected; a name the target lacks, a test
  fixture named by a suite hook or a worker fixture, and a fixture named
  like an engine's, at selection, for the pairs that run on that target.
- On an engine with isolation: 'worker', a suite hook or a worker fixture
  teardown that names no engine-backed fixture opens no engine attempt.

BREAKING CHANGE: a fixture nothing names no longer runs unless it is
declared with auto: true, and a test body, hook, or fixture function that
takes the whole fixture object (async (fixtures) => {}) no longer loads.

Refs ENG-1297

* refactor(core): declare core fixtures in one table

The five core fixtures (agent, app, screen, platform, session) are one
table in @e2e-dev/core: name, scope, and factory. createFixtures builds
the built-ins object through one loop over that table and the engine's
factories, every fixture a lazy accessor behind the same gate as before.

The runner's fixture pool, test.extend() validation, and launch read the
names and scopes from the table, so the name lists and the per-name scope
checks are gone. A launch session exposes the core fixtures that do not
belong to a test: app, screen, platform.

No behaviour change. app and screen were built with the attempt and are
now built on first read, like agent and the engine's fixtures.

Refs ENG-1297

* feat(e2e): extend a built-in fixture through test.extend

A test.extend() fixture named like one the test already has extends it,
by Playwright's rule: inside its function its own name is the fixture it
extends (the built-in, or an earlier fixture of that name), and tests,
hooks, and other fixtures that name it receive what it hands to use().

  base.extend({ device: async ({ device }, use) => {
    await device.setPermission('camera', 'deny');
    await use(device);
  } })

- A plain function takes the scope the extended fixture has on the
  target: device is a worker fixture on mobile, so its override runs once
  per worker and reaches beforeAll; app is a test fixture on web and a
  worker fixture on mobile. A declared scope that differs is a
  COLLECTION_ERROR naming both.
- A worker-scoped override receives what a worker fixture receives for
  that built-in, the forwarder to the attempt that is running.
- On a target without the built-in nothing fails until code that runs
  there names it; then the target has no such fixture.
- Only user code sees the override: the attempt's own built-ins, which the
  runner and the worker fixtures' forwarders read, are untouched.
- Types: the override keeps the extended fixture's type and scope, through
  an extend() overload that takes overrides without type arguments.

A fixture named like a core fixture, an engine's, or an earlier one of the
chain was refused before.

Refs ENG-1297

* docs(fixtures): name built-in fixtures and give their scope per engine

The docs now use one term, built-in fixtures, for agent, app, screen,
platform, session, and the ones an engine contributes (browser, device),
where pages said core fixtures, contributed fixtures, or engine fixtures.

The test reference gets one table with each built-in fixture's scope on
web and on mobile, and says what the scope decides: a test fixture reaches
test bodies, beforeEach, and afterEach; a worker fixture also reaches
beforeAll, afterAll, and worker fixtures. The hooks, worker fixtures, and
override sections link to it, as do the fixtures guide and the engine
pages. The Japanese and Chinese pages carry the table and the same terms.

Refs ENG-1297

* docs(ja): translate the fixture scope and on-demand changes

* docs(zh): translate the fixture scope and on-demand changes

* fix(e2e): let a worker fixture name an extended app or screen

A fixture that extends `app` or `screen` was taken for a test fixture when
its test.extend() call was checked, before any target is known. A worker
fixture declared after it that names `app` then failed at import on every
target, though the same fixture is accepted without the override and the
plan resolves both to worker scope on an engine whose state outlives a
test.

The scope of a plain function is now unresolved until a target is known,
whatever it extends, as it already was for an engine's own fixtures.

Refs ENG-1297

* fix(e2e): fail a suite hook that left an engine step running

On an engine whose state outlives a test, a beforeAll or afterAll hook, or
a worker fixture's teardown, can start a device step and not await it. The
hook's attempt then closed under the step, and its failure arrived later
as a stray rejection charged to whichever test ran then.

The suite attempt now does what a test attempt does once its body settled:
the step is failed at its call and cancelled, its outcome is awaited within
the cleanup budget, and the hook fails with STEP_NOT_AWAITED (HOOK_FAILED
for a suite hook).

The suite attempt keeps no step records, and StepRecorder dropped the
record of the step it had just opened before marking it running, so there
was nothing to abandon: a step now counts as running before the retained
window is trimmed.

Refs ENG-1297

* refactor(e2e): keep the fixtures a function names beside it

Registration parsed each test body, hook, and fixture function for the
fixtures it names, dropped the result, and left it in a WeakMap that the
planner and the pool read back with placeholder subjects. A cache miss
there would have named the wrong code and no line.

The parse result is now data: `names` on the registered test, the hook,
and the fixture definition. The planner and the pool read it, the WeakMap
and the per-function wrappers are gone, and one describeUser() gives an
error its subject. The explore test, registered by hand, parses its body
where it builds the record.

Unit tests state the names instead of generating source text for them.

Refs ENG-1297

* refactor(core): resolve built-in fixtures in one place

The core table declared the core fixtures, but what a target's built-in
fixtures are was still assembled around it: core names plus engine names in
three files, the 'engine' scope resolved in e2e through a cast into the
table, `platform` hard-coded as the one fixture that needs no attempt, and
two structural copies of the target.

core now exports builtInFixtures(target): names, scope(name),
needsAttempt(name), attemptless(name), and the unknown-fixture message,
over one BuiltInTarget. `platform` declares in the table that the target
alone decides it. launch, the planner, and the pool read from it.

DECLARED_BUILT_INS is the same view before a target is known: a scope the
engine decides is 'unresolved'. The planner takes either one, so its
no-target mode and the `target === undefined` branches through scopeOf and
read are gone. The scope of a fixture that extends `app` before a target
is known is 'unresolved' by construction.

Refs ENG-1297

* refactor(e2e): hand each function the fixtures it names

Every test body, hook, and fixture function must destructure its first
parameter, so what it reads is known before it runs. bindFixtures still
built a Proxy-gated bag for arbitrary property reads: one value in three
stores, the gate run twice (once over a faked plan), a third derivation of
scope, and an own-name shadow that re-derived what the planner had resolved.
No test code could reach the gate's throwing branches; only unit tests did,
by reading the bag directly.

The plan now records, per function, what each name it reads resolved to: a
chain fixture or a built-in. The pool hands each function a plain object
with exactly those names, the value a chain fixture handed over, or a
built-in read off the attempt when the function destructures it, so
built-ins stay lazy. An override that names itself receives the fixture it
extends because that is what its own name resolved to.

- bound.call(user) replaces user.fn(bound.fixtures); ready(user) asks
  whether every chain fixture it names was provided.
- attemptlessBuiltIns is a function from a name to a value that throws the
  reason for a built-in that needs an attempt.
- The planner runs once: it records what each fixture names, and the order
  is one declaration-order walk over that record. A second validating pass
  only reordered.

Behaviour: a bound or native function, whose parameters cannot be read, is
a COLLECTION_ERROR; it used to parse as naming nothing and still got lazy
built-ins. The docs no longer promise hooks and the body one shared object.

Refs ENG-1297

* refactor(e2e): move the suite attempt out of TargetExecutor

dispose, openSuiteHooks, and openSuiteAttempt were 177 lines of a second
attempt lifecycle inside TargetExecutor, written as closures in one method.

run/suite-attempt.ts now owns it: a SuiteAttempt class with open, acquire,
within, and close, over a SuiteAttemptHost interface the executor satisfies
(the way serial.ts borrows it through SerialHost), plus the worker
fixtures' teardown pass. It is the realm's SuiteHookSession itself, so the
executor keeps a context object and two call sites.

The three branches that built an attemptless reader with a "not available
to the <label>" error are one NoAttempt value ('isolated', 'unnamed',
'interrupted', or the launch failure) and one mapper to the message.

execute.ts: 1244 lines to 1052.

Refs ENG-1297

* refactor(e2e): share the attempt's fixture sequence

A run's attempt and an `e2e mcp` session's test each planned the test's
fixtures, bound them, entered the worker pool, set them up, ran the
beforeEach hooks and the body, filtered the afterEach hooks that can still
run, and labelled the teardowns. This PR widened that copy, and
standalone-test.ts asked the realm for the same hooks four times.

planAttemptFixtures(target, test, hooks) plans before any session exists
and returns open(), which binds the plan to the attempt: enter(), run()
for setup, hooks, and body, and cleanup() for the labelled afterEach hooks
and teardowns. Both call sites keep their own phases, budgets, and error
handling around it.

execute.ts: 1052 lines to 1031, its size on main.

Refs ENG-1297

* refactor(e2e): open the suite-hook pass without a lazy thunk

withSuiteHooks handed its work a thunk that opened the pass on first use.
Opening one does no I/O: the suite attempt launches nothing until a hook
that names an engine fixture runs. The pass is now opened up front and
closed in finally, and the `opened` slot and the `hooks()` call form go.

Refs ENG-1297

* refactor(e2e): ask the registry which hooks apply to a test

The selection-time fixture check rebuilt the scope chain of a test and
filtered the file's hooks by it, which the realm already did its own way.
registry.ts now exports scopeChain(test) and hooksFor(registration, test),
and both use them.

Refs ENG-1297

* refactor(core): move the fixture types into types/fixtures.ts

types.ts grew from 1679 to 1798 lines in this PR. The fixture scope, the
worker fixtures, and what test.extend() takes (definitions and overrides)
are one coherent group that nothing else in the file defines against, so
they move to types/fixtures.ts and types.ts re-exports them. The public
surface is the same.

types.ts: 1798 lines to 1715.

Refs ENG-1297

* refactor(core): drop the redundant override branch in FixtureDefinitions

A fixture that extends one the base already has was typed three ways: a
conditional branch inside the `Extra` map of FixtureDefinitions, the
`& FixtureOverrides` intersection, and the extend(overrides) overload.

Compiling the type tests without each in turn: the intersection types an
override written beside new fixtures with type arguments, and the overload
types an override-only call without them; both fail the tests when
removed. The conditional branch does not: with it gone the tests pass, and
a base name in `Extra` is `never` again, as it was before overrides. It is
removed, and the reference's copy of the declaration follows.

Refs ENG-1297

* refactor(e2e): split fixture errors from the parser, share hookLog

fixture-names.ts parses a parameter list, and also held FixtureUseError and
declarationSite, which format an error. They move to fixture-errors.ts.
Its identifier pattern held two raw zero-width characters; they are
written as escapes now.

hookLog and LOG_MODULE were pasted into three integration tests; they are
one helper in tests/helpers/hook-log.ts.

Refs ENG-1297

* docs(e2e): name the scope rule in the selection check's summary

The summary of assertFixtureUse still listed a chain fixture named like an
engine's as an error. Such a fixture extends the built-in now; the rule
left is a declared scope that differs from the extended fixture's.

Refs ENG-1297

* refactor(e2e): take execute.ts under 1000 lines

Two more pieces that are not the attempt's own sequence leave the file.

- Holding steps for `e2e run --step`: reading the screen a hold starts on
  and wiring AttemptHolds to the runner's hook move to step-hold.ts, beside
  the holds they configure (holdAttemptSteps, readHeldScreen).
- The result of a finished test or setup unit is a pure function of the
  pair, the verdict, the attempts, and a beforeAll failure: unitResult in
  units.ts, beside pairResult.

execute.ts: 1031 lines to 956. It was 1031 on main and 1226 at the review.

Refs ENG-1297

* test(e2e): match the not-awaited message without a step label

Main's #1880 names an unlabeled step by its api alone ("gadget.hang
finished"), where the message used to quote an empty label. The suite-hook
test's pattern required a label between the two, so it failed on the merge
with main.

* feat(e2e): an override inherits auto from the fixture it extends

A fixture that extends another and does not state `auto` now has the
extended fixture's, the way it has its scope: an override of an auto
fixture is set up for every test, and one of a built-in fixture on demand.
An `auto` that differs from the extended fixture's is a COLLECTION_ERROR
naming both, worded like the scope mismatch. This matches Playwright.

It also fixes which fixtures the plan starts from. An auto fixture was a
root by its own flag, so with an auto base and its override in one chain
the base was set up and the override skipped, while the name resolved to
the override; an override that replaced the base without naming it still
left the base running. An auto fixture is now a root only where its name
stands for it: the base runs once through the override's own name, or not
at all when the override replaces it.

Refs ENG-1297

* fix(core): name the hook or fixture that left a step running

STEP_NOT_AWAITED said "the test body returned before ..." for a step a
beforeAll, afterAll or beforeEach hook, a fixture's setup or a worker
fixture's teardown left running. The recorder now keeps the owner each
step was started under, and the runner says which code runs: the pool
for fixtures, hooks and the body, the suite attempt for suite hooks, and
a hosted session for itself.

---------

Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-18 13:09:51 +00:00
Natasha Tran 902f044b49 feat: add a dot reporter (#1911)
* feat: add a dot reporter

* fix: flush dot output and separate list formatting

* docs: clarify buffered dot reporter output
2026-11-18 12:58:46 +00:00
Dery Ferdika f5cda16313 feat(html-reporter): filter a run's results by target (#1793) 2026-11-18 14:49:06 +02:00
Krzysztof Moch 6fbcb66f02 docs: show how to connect e2e mcp to OpenCode (#1926)
Refs ENG-1317
2026-11-18 10:02:37 +00:00
Kacper K d16ad2c2d5 feat(cli): add e2e report command to open HTML reports (#1892) 2026-11-18 08:39:35 +02:00
ParbezandOskar Kwaśniewski fabb0c07fb feat(expect): take an optional message on locator expectations (#1865)
* feat(expect): take an optional message on locator expectations

* docs(expect): name expect.soft(locator, message) and scope the message to assertion failures

* feat(expect): open an ambiguous locator's error and expect(screen) failures with the message

* fix(expect): open an ambiguous locator's toHaveScreenshot error with the message

labelAmbiguous moves to expect/heading.ts and wraps the screenshot capture, so
expect(locator, message).toHaveScreenshot() labels LOCATOR_AMBIGUOUS as the
polling matchers do. Tests cover it, a screen .not and the CI no-stored path
with a message. The reference drops a sentence about step names.

* docs(expect): update the ja and zh pages for the locator message

The translations landed on main after this branch was cut, so the English edits left docs/ja and docs/zh stale and docs:check failed on the code blocks of reference/expect.mdx. Bring both languages level with the English pages, the Playwright guide rows included, and rewrap one line of the English soft assertions paragraph.

---------

Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-17 21:30:02 +00:00
Samanyu DasandClaude Opus 5.5 d960d2c448 fix(engine): wait for a crashed engine server's exit before its connection closes (#1863)
* fix(engine): wait for a crashed server's exit before its connection closes

spawnTransport handed the child's stdout to the connection directly. When
a server exits mid-request, its stdout can end before Node delivers the
exit and the rest of its stderr, and the connection reads both for its
message the moment it closes. The request then failed with only "the
peer closed the connection", which made the engine-protocol crash test
flaky on CI (Node 22).

The connection now reads stdout through a stream that ends once the
child has closed, or 2 s after stdout ended, for a server that stays up
or a grandchild that holds its stderr.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(engine): pin the wait on close, and time only the failing request

A crashed server's child that shares its stderr now has a test, which
fails when the transport waits for the exit instead of the close. The
stdout-first test measures the request instead of a 1.5 s test timeout,
which failed under load. The cap is a cleared timer, the changeset and
docs name the signal case, and the fixture's modes are one option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): end a server's stdout from a generator, and test the signal and the cap

The input is now an async generator over the server's stdout that waits
for the close or the 2 s cap, so a stdout error reaches the connection as
an error event and the cap's timer is aborted, by construction. New tests
pin the signal case the docs name and the cap's length, and the crash
helper takes the fixture's mode type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(engine): say what a crash error holds after the cap, and what keeps a request open

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): cap the stdout wait with an unref'd delay, and pin the cap from both sides

The aborted delay's .catch could never run and nothing tested the
abort. An unref'd delay needs neither, the pattern step-control.ts
uses. The stays-up test now fails a cap under 1.9 s or over 3 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 21:01:20 +00:00
9602daa960 feat(notte): add @e2e-dev/notte, a browser provider for Notte cloud browsers (#1726)
* feat(notte): add @e2e-dev/notte, a browser provider for Notte cloud browsers

notte() runs each test attempt in its own Notte cloud browser session and
stops it when the attempt ends. It leases in attempt scope only: a Notte
session refuses Target.createBrowserContext, which worker scope needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(notte): downloads, sweep, and console links, verified on the testbed

- downloads.read copies each download from the session's file storage,
  found by the site's suggested name. @e2e-dev/web now passes that name to
  providers as BrowserDownloadContext.suggestedFilename (optional,
  additive).
- sweep stops the sessions of workers that died, found through per-session
  marker files under the system temp directory, since Notte sessions have
  no tags.
- The progress line links the session's console page instead of the
  token-bearing viewer URL.
- A start response without cdp_url falls back to the session's debug info.
- apps/testbed gets e2e.notte.config.ts and test:notte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(notte): address review: fail-closed ledger, required suggestedFilename, paged downloads, shared sweep helper

* docs(notte): add notte() to the security table and skill setup

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-17 19:38:50 +00:00
Oskar Kwaśniewski 51d304e736 fix(core): cancel running calls by name when a session closes (#1880) 2026-11-17 20:58:39 +02:00
Kayaan TharaniandOskar Kwaśniewski f659d64d80 feat(revyl): add @e2e-dev/revyl for mobile tests on Revyl devices (#1708)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-17 20:55:14 +02:00
Matt Van HornandMatt Van Horn 95c574adec fix(agent): strip unknown fields on no-input agent tools (#1835)
* fix(agent): strip unknown fields on no-input agent tools

A tool with an empty schema shape still uses the strict schema, which
keeps the provider-facing input closed, but a plain object is reduced to
an empty object before that check, so extra keys are dropped and the
call runs. Values that are not plain objects still fail validation.
Tools that declare fields still reject unknown keys, and screenshot does
too once a path field is present. The security and reference docs and a
patch changeset describe that split.

Fixes #1812

Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>

* fix(agent): trim schemaTool comment and cover array input

Cut the schemaTool JSDoc to why an empty schema strips, drop the
changeset's provider-schema sentence, and add an array to the values a
no-input tool rejects.

* docs(security): drop provider schema detail and tighten tool rules

---------

Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-11-17 11:23:33 +00:00
Samanyu DasandClaude Opus 5.5 1d0271b306 feat(test): add test.skipIf and test.runIf (#1870)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:19:26 +02:00
Oskar Kwaśniewski 6c4eddd0f9 feat(cli): ask for a GitHub star after init, upgrade, and update notice (#1874) 2026-11-17 13:17:51 +02:00
Asheesh Kumar d4fb5ccd49 feat(expect): add toHaveRole locator matcher (#1857) 2026-11-17 10:24:13 +02:00
Donny/강동윤andClaude Opus 5.5 aadf95506d fix(report): keep \r\n line breaks in serialized errors and the JUnit report (#1860)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 10:20:56 +02:00
Donough Liu 29a0261775 feat(moli): add managed local browser provider (#1843)
Manage local Moli CDP browsers through the web engine's BrowserProvider API.
Each worker gets an available loopback port, layout and resource loading,
startup cancellation, and process cleanup when its lease ends.

Wire the testbed, documentation, consumer skill, build and test commands,
and a patch changeset for @e2e-dev/moli.

Keep process-group shutdown active after an executable wrapper exits,
and surface process-inspection failures in the provider tests.

Verified locally:
- pnpm check
- 16 Moli provider unit tests
- Testbed on two workers: 16 passed, 1 expected skip; both browsers released
- Empty and whitespace CDP URLs use managed browsers; padded URLs attach
  without stopping the existing server
- MCP login, secret fill, assertions, and screenshot
- Packed packages installed, typechecked, and run in an isolated project
2026-11-17 07:03:27 +00:00
Oskar Kwaśniewski 705c049205 fix(web): leave descendants with their own role out of a name from content (#1853) 2026-11-16 20:33:09 +02:00
Donny/강동윤andClaude Opus 5.5 0cd14a0dfa perf(core): read a wait again at 20 and 50 ms, then every 100 ms from its start (#1834)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 19:29:29 +02:00
Donny/강동윤andClaude Opus 5.5 1b7c3a99d6 perf(agent): keep the act tool set the same across steps for the prompt cache (#1823)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 19:27:52 +02:00
Kacper KapuściakandOskar Kwaśniewski d593863b41 feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init (#1647)
* feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init

e2e init asks for a CI pipeline in a project with the web engine (--ci github-actions|gitlab|none). It writes one job for the project's package manager and Node version that installs Chromium, runs e2e run, and keeps the report, and names the secret or CI/CD variable for the model key. It never overwrites a pipeline file; beside an existing .gitlab-ci.yml it writes a file to include.

* fix(cli): handle cubic's findings on the init CI step

Exact GitLab stage names, a distinct job name beside an existing e2e job, loopback baseURL in an existing config, no writes through a symlinked .github or .gitlab, --ci none in the skill, gitlab in the telemetry comment, and the GitLab Node image's major version in the docs.

* refactor(cli): keep the init CI pipelines as YAML templates

The GitHub Actions workflow and the GitLab job live in packages/e2e/templates/ci/ as complete pipelines for npm at the repository root, linted as they are. init loads them with the yaml package's Document API and edits a copy for the project's package manager, folders, Node version, secrets, and target, keeping the comments. The pinned actions now live only in the templates.

* fix(cli): point setup-bun at the package.json of a project in a subfolder

* docs(ci): link the GitLab page from init and the GitLab template

* feat(cli): write the iOS and Android CI jobs in e2e init

init writes one job per platform the config's targets run on: web targets in the web job, iOS and Android targets in their own jobs, which build the app, boot a simulator or an emulator, and install the app. Targets come from the new config, or from loading an existing one after the confirmation and the install. Expo and bare React Native apps get their build; any other app gets a step to replace. GitHub's mobile jobs follow e2e-inc/e2e-react-native's workflow; GitLab's run on self-managed runners.

* fix(cli): reinstall an emulator system image that holds no image in the Android CI job

* fix(cli): handle cubic's findings on the mobile CI jobs

Upload the emulator log from the GitLab Android job, count peer and optional engine dependencies, ask nothing for a new config with no engine, key caches on Bun's bun.lockb when that is the lockfile, and run the symlink test with a Windows junction.

* fix(cli): put current Android command-line tools in place before the emulator image

The runner image's cmdline-tools can be older than the system image's package format, and android-emulator-runner uses them to create the AVD, which then fails with 'contains no system images'. The Android job downloads command-line tools 23.0, checks its SHA-256, and installs the image with it.

* fix(cli): import moved core modules from @e2e-dev/core in init

* fix(cli): count a pipeline as set up only when a command runs e2e run

init matched the text 'e2e run' anywhere in a workflow, so a comment or a step name such as 'our old e2e runner' made --ci write nothing. It now reads the commands (run, script, before_script, after_script) from the parsed YAML and matches e2e run as words of their own.

Also: --ci help and the InitOptions JSDoc say tests on every platform, and yaml is pinned exactly like the other dependencies.

* chore(ci): bump setup-node and upload-artifact pins in the init templates, docs, and skill

docs/ci/vercel-previews.mdx on main pins setup-node v7.1.0 and upload-artifact v7.0.2, the pins this repository's own workflows run. The templates, three docs pages, and the skill still had v7.0.0 and v7.0.1, so the pin test failed once main was merged in.

---------

Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-16 17:21:49 +00:00
Donny/강동윤andClaude Opus 5.5 3422abfb6a fix(agent): judge a waitFor screen change without waiting out the interval (#1819)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 19:22:00 +02:00
Donny/강동윤andClaude Opus 5.5 40cb356a1e fix(web,mobile): load the main entry without e2e installed (#1828)
`@e2e-dev/web` and `@e2e-dev/mobile` declare `e2e` an optional peer, but
their main entry imported it statically to build `test`. A program on
`@e2e-dev/core` alone that wrote `import { web } from '@e2e-dev/web'`
failed with ERR_MODULE_NOT_FOUND before any of its code ran.

The main entry now requires `e2e` only when it resolves. Without it, the
engine factory works and `test`, `describe`, and the hooks throw
RUNNER_NOT_INSTALLED when called. It uses createRequire, not a top-level
await, so a CommonJS file can still require() the entry.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 09:50:17 +00:00
dbe3dc54a9 feat(core): add launch, SessionState, and createAgent to @e2e-dev/core (#1734)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-16 09:38:07 +01:00
0a5ee16387 refactor: move e2e's core into @e2e-dev/core, and engines peer on it (#1737)
* refactor: move e2e's core into @e2e-dev/core, and engines peer on it

The engine contract, locators, agent, replay cache, secrets, and the
session half of a worker move from packages/e2e to packages/core, the new
@e2e-dev/core package, at the same paths. e2e depends on it at its exact
version (a changesets fixed group) and reaches its modules through
@e2e-dev/core/lib/*, which is not public API; e2e/engine and e2e/agent
re-export it. @e2e-dev/web and @e2e-dev/mobile peer on @e2e-dev/core and
import the contract from @e2e-dev/core/engine. Install smoke checks that one
copy of @e2e-dev/core is installed.

Most of this diff is git mv and import specifiers rewritten to
@e2e-dev/core/lib/<path>; no behavior changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: build and pack @e2e-dev/core where e2e is built or packed alone

The Bun job built e2e before the @e2e-dev/core it compiles against, and the
examples nightly installs e2e from a tarball whose exact @e2e-dev/core is
not on npm; it now packs @e2e-dev/core and overrides it in each example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(engines): name @e2e-dev/core/engine as the contract's source

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(e2e): list @e2e-dev/core beside an engine in init and upgrade

Engines peer on @e2e-dev/core. Yarn installs no peer the project does
not list, and under Plug'n'Play the engine then fails to load.

- init adds @e2e-dev/core with any engine
- upgrade adds a required @e2e-dev peer the project lacks, and keeps a
  package e2e depends on at the version e2e names

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-15 18:31:58 +00:00
Krzysztof Moch 2359860783 feat(cli): step through a test without a terminal (#1750) 2026-11-15 17:47:21 +01:00
Łukasz LechandOskar Kwaśniewski 0b4485989d feat(agent): read computed styles in project tools (#1633)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-15 15:49:17 +01:00
Karol Kulesza 74435e23b6 fix(mobile): keep userinfo out of openLink and openApp step labels (#1787)
* fix(mobile): keep userinfo out of openLink and openApp step labels

* fix(mobile): strip userinfo from slashless web links, tab-padded links and backslash userinfo in labels

* fix(mobile): label links through the URL parser, never echo an unparseable one
2026-11-14 16:20:12 +00:00
Karol Kulesza dd39974ace fix(mobile): refuse an unknown snapshot or videoTouches value in mobile() (#1784)
* fix(mobile): refuse an unknown snapshot or videoTouches value in mobile()

A typo such as snapshot: 'Interactive' captured the full tree and videoTouches: 'false' drew the touches, since the engine compares against the exact values and falls back to the default. Both now fail at config load with INVALID_CONFIG naming the accepted values, like settle and transition. The reference's Throws table also lists the existing platform check.

* fix(mobile): format a refused snapshot or videoTouches value without throwing

* fix(mobile): resolve snapshot and videoTouches once, like settle and transition
2026-11-14 16:20:06 +00:00
Nguyễn Văn Hiếu b56e8ed525 docs(web): clarify media emulation tab scope (#1772) 2026-11-14 10:23:18 +00:00
Karol Kulesza e167149af1 docs(mobile): say which device runs when no device is named (#1728)
* docs(mobile): say which device runs when no device is named

A paired iPhone or iPad counts as booted and joins the pool; with none booted, agent-device boots an iPhone simulator over an iPad, then the newest OS, then name order.

* docs(mobile): name paired devices and the Apple TV and Android TV tiers in the cold-pick order

* docs(mobile): say a reachable paired device counts as booted, fix changeset and wording

* docs(mobile): name reachable paired iPhones and iPads in the device pool
2026-11-13 22:16:02 +00:00
Karol Kulesza 800a93e62a docs(screen): say what doubleTapAt does on a device (#1733)
* docs(screen): say what doubleTapAt does on a device

* docs(screen): give doubleTapAt its own iOS tap gap

* docs(mobile): give doubleTap and doubleTapAt the same iOS gap
2026-11-13 22:15:53 +00:00
bardrop 80775424fd fix(config): reject timeouts above the timer limit (#1742) 2026-11-13 10:13:08 +00:00
Karol Kulesza 5ff74812a8 docs(skills): give AUTOMATION_UNSUPPORTED an example a device really lacks (#1732) 2026-11-13 08:49:29 +00:00
Brendan Irvine-BroqueandOskar Kwaśniewski 352b774b33 feat(browser-run): add Cloudflare hosted browsers (#1634)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-13 08:46:31 +00:00
Tim Shamilov 68407d2af2 fix(mobile): cancel presses when the response wait expires (#1685)
* fix(mobile): cancel pending press requests at the input deadline

* fix(mobile): preserve response grace when cancelling presses

* fix(mobile): share cancellation across an operation

* docs(mobile): clarify pending request tracking

* docs(mobile): scope cancellation in the reference

* docs(mobile): include all cancellable press paths

* docs(mobile): include uncheck in the release note

* fix(mobile): centralize cancellable press dispatch

* test(mobile): cover signals at every press dispatch path

* docs(mobile): clarify press cancellation and tighten checks
2026-11-13 07:31:34 +00:00
Karol Kulesza 0e25c1d33c docs(mobile): say a node under the soft keyboard is hidden (#1725)
* docs(mobile): say a node under the soft keyboard is hidden

* docs(mobile): list what stays visible under the soft keyboard
2026-11-12 22:18:52 +00:00
Karol Kulesza e044b8848b docs(mobile): list what fails before an appPath-only build is installed (#1724)
* docs(mobile): list what fails before an appPath-only build is installed

* docs(mobile): name the no-bundleId condition on restart and clearState
2026-11-12 22:18:44 +00:00
Krzysztof MochandOskar Kwaśniewski 17a2e464f0 feat(explore): explore what a pull request changed with --changed-since (#1711)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-12 17:42:01 +00:00
Krzysztof Moch 258ef9be7a feat(plugins): register the e2e MCP server in the plugin (#1717) 2026-11-12 17:23:48 +00:00
Donny/강동윤andClaude Opus 5.5 0920ef5f94 feat(web): tabs in the browser fixture, with switch_tab and close_tab agent verbs (#1710)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 17:00:57 +00:00
Oskar Kwaśniewski be460ecd91 fix(security): redact tool outputs and patch vulnerable dependencies (#1715) 2026-11-12 16:47:45 +00:00
Krzysztof Moch 565e685779 docs(cli): say --output must be inside the project root (#1721)
The --output row in the CLI reference and the bundled running guide never said the directory must be inside the project root and not the root itself, so a run with an output outside it failed INVALID_CONFIG with no warning in the docs. Fixes ENG-976.
2026-11-12 14:48:31 +00:00