133 Commits
Author SHA1 Message Date
Satyajit Sahoo 23aa697e26 feat(cli): show --step keys under the summary and view the screen and app log (#2015)
* feat(cli): show step keys under the summary and view the screen and app log

The --step and screen.pause() prompt now lists its keys below the live
summary, as --watch does, and the held step row uses a pause glyph.

Press s for the screen's accessibility tree or l for the attempt's app
log. Both open in a scrolling viewer on the alternate screen. The app log
is read from the worker during the hold through a new logs-request and
logs-reply message pair.

The mode without a terminal keeps its text.

* feat(cli): stop the terminal's Duration row while a step is held

The list reporter's Duration row counted wall time, so it kept ticking at
a --step prompt or a screen.pause(). The scheduler now reports when the
first hold opens and the last closes through a run-hold event, and the
row excludes that time, live and in the final summary.

report.json and the reports built from it keep wall time.

* feat(report): add run.durationMs without held time

The report only had the run's start and finish timestamps, so a run that
waited at a --step prompt or a screen.pause() looked as long as the wait.

run.durationMs is the run's duration with held time left out. The
terminal's final Duration row prints the same value. Markdown, the GitHub
comment, the HTML report, and telemetry read it in place of the timestamp
difference. startedAt and finishedAt still record wall time.

A simplification review also removed a few flags and options from the
step viewer and the hold path, and the docs keep each new fact in one
place.

* docs: trim the step viewer changeset and docs

The changeset is down to two sentences. The docs no longer say where the
keys are, since the terminal sample shows it, and the json reporter
section matches main again.

* fix(cli): print the viewer's text before the prompt is repainted

On p, the viewer closed first, so the live window was repainted and then
erased again for the printed text. That left a blank frame. The text is
now queued before the viewer closes, and the window is repainted once.

* fix(cli): number the step prompt by its trace position

The app log view numbered its lines by the recorder's step index, which
counts nested steps, while the prompt counted held steps only. A line
logged inside a nested step could show a different number than the
prompt.

The worker now sends each held step's position in the trace, so the
prompt, the app log view, and the trace page use one number.

The viewer's try blocks are removed too. Nothing received their rethrows,
and the guarded calls don't throw.

* fix(cli): count held time only while every running worker is held

With several workers, a screen.pause() in one test took the time other
tests spent running out of the run's duration. A run could read shorter
than one of its tests. An interval now counts as held only while every
running worker is held. The time limit keeps its rule.

Also from review:
- A routing state an older installed copy made is upgraded in place,
  so claimProcessOutput() no longer throws.
- The viewer holds writes only for terminal streams.
- Ctrl-Z closes the viewer and suspends.
- A failed screen read can be retried during the same hold.
- p is ignored while a read is pending.
- The app log view shows plain text, not Markdown code spans.
- The viewer's help row starts with q, and its headings use the
  redacted title and target.

* test(cli): cover an older dispatcher installed on the streams

The upgrade test seeded a state with no dispatchers. This one installs
an older copy's dispatchers first, then checks that writes are held
while the viewer is open.

The comment on that branch loses its FIXME prefix. It describes what the
code does.
2026-11-21 14:00:20 +01:00
Samanyu DasandClaude Opus 5.5 198d25f038 feat(test): add test.fixme (#1887)
* feat(test): add test.fixme

test.fixme takes the four call shapes test.skip takes. test.fixme(title, fn)
registers a skipped test whose body never runs. test.fixme(), test.fixme(reason),
and test.fixme(condition, reason) skip the running test from a body or hook.
Both report the test skipped with cause explicit and the reason "fixme", or
"fixme: <reason>" when a reason that is not blank is given.

The runtime form shares skipRunningTest with test.skip, which now takes the
API name so every error names test.fixme(). The registration form rejects a
serial group with an API-named message, as test.skipIf and test.runIf do.

Refs #1855

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(test): say test.fixme's reason and setup error need a non-blank reason and a true condition

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(test): route test.skip(title, fn) through registrar and mirror test.fixme docs in zh

test.skip(title, fn) in a serial group now names test.skip(), as test.fixme,
test.skipIf, and test.runIf do. docs/zh, which landed on main after this
branch was cut, gets the test.fixme section, the serial-group rows, the Types
block, and the Playwright guide rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(changeset): keep the test.fixme changeset to one sentence

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(report): report test.fixme with its own skip cause, fixme

A test.fixme test, registered or skipped from its body, now reports
skip.cause fixme in report-v1, beside explicit. The reason text is
unchanged. Every consumer that counted an explicit skip as one the test
gave itself counts fixme too: serial member accounting, Skipped After
Failure, the GitHub merge summary, and NO_TESTS, which says marked fixme.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(test): document the fixme skip cause, and pin the GitHub merge count

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(report): keep fixme out of --last-failed, and cover the fixme cause's other paths

A SelfSkipCause type names the causes a test gives itself, and
RUNTIME_SKIP_APIS carries each runtime skip API's cause, so a new one
cannot report half-wired. Tests now pin --last-failed leaving fixme out,
the fixme cause beside a failure that outranks it, and the explicit
fallback for a RuntimeSkip from an older copy. The docs align the
test.skip setup row and say the Playwright guide's { skip: 'fixme' }
workaround reports cause explicit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(changeset): tell users to update @e2e-dev/github with e2e for the fixme cause

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(types): give test.fixme(title, fn) worker fixtures, and name it in the executed docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: rerun checks after a flaky app-command-exit test on Node 24.8.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(migrate): name the serial-group exception for skip and fixme registrations

Playwright accepts test.fixme(title, fn) and test.skip(title, fn) inside a
serial describe, and e2e rejects them, so the "the same" cell names it.
Type a caught runtime skip's cause as unknown, so every reader goes through
runtimeSkipCause, and keep RuntimeSkip module-private. State the github
changeset as the symptom, and name test.fixme in the skip JSDoc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: shorten the migrate cell, and name test.fixme in failureBeforeSkip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: list test.fixme(title, details, body) as missing, and fix the github changeset

The PR comment never shows the executed count, so the changeset names the
merged report instead. The Playwright guide lists the details form under
the APIs e2e does not have yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(migrate): list Playwright's skip and fixme callback forms as missing

Playwright's test.skip(callback, description) and
test.fixme(callback, description) fail collection in e2e with
COLLECTION_ERROR. The guide now lists them, with the conditional form
inside the body or platforms on the test as the workaround, in en, ja,
and zh. The ja test reference says 直列実行 for serial groups, like the
rest of that page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(schema): put the fixme skip cause in both report-1 fixtures

The valid fixture's runtime skip, on the result and its attempt, now has
cause fixme. The invalid fixture's skip that names a filter now has cause
fixme, and a test checks that the schema rejects exactly that pair: the
rule that only a filtered skip names a filter fails at that cause, and
deleting the filter clears it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(test): skip as fixme after a timeout test.setTimeout raised

A test that raises its timeout from 300 ms to 10 s, waits 900 ms, and then
calls test.fixme() is reported skipped with cause fixme, on the result and
its attempt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 15:30:09 +00:00
Krzysztof Moch eddf577526 feat(reporters): build the built-in reporters on the public Reporter API (#1936)
* feat(reporters): build the built-in reporters on the public Reporter API

The built-in list, dot, json, junit, and markdown reporters now use only
the public Reporter contract: factories on e2e/reporters, printsTo for the
terminal stream a reporter prints to (one reporter per stream), live
onOutput, onStepPrompt, and onSummary hooks for any number of reporters,
and reportFiles for files the runner writes atomically. The list and dot
ids beside json print to stderr, so list,json now works as dot,json does.
@e2e-dev/html-reporter writes through reportFiles.

* docs(reporters): bring the Japanese and Chinese reporter pages up to date

Translates the reporter reference, the config reporters table, the
--reporter row, and the debugging output paragraph. In English, the name
section now says a reporter may share a built-in's name, and two lines
read less ambiguously.
2026-11-19 15:58:33 +00:00
Satyajit Sahoo 4893d1dd75 feat: add screen.pause() (#1904)
With `await screen.pause()`, a test is paused at that line, with the app
open, until someone resumes it.

- In a terminal, press Enter to resume the test.
- From another shell in the same directory, run `e2e run --step next`.
  A run without a terminal is resumed this way.
- Paused time doesn't count toward the test's timeout.
- With `CI` set, the test fails with `PAUSE_IN_CI`.
- Under `--step`, a test is still paused after `n` or `a`.
- A test isn't paused in an MCP `open_session {test}` session, under
  `openHost` or `run()`, or in a core program.
- A test isn't paused under `e2e run --watch` yet.
2026-11-19 11:00:39 +00:00
Milosz JankiewiczandOskar Kwaśniewski 5e9b82e6ae feat(oauth): name the SuperGrok provider spacexsi, with spacexai and grok as aliases (#1943)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-19 12:07:52 +02:00
Natasha Tran 902f044b49 feat: add a dot reporter (#1911)
* feat: add a dot reporter

* fix: flush dot output and separate list formatting

* docs: clarify buffered dot reporter output
2026-11-18 12:58:46 +00:00
Kacper K d16ad2c2d5 feat(cli): add e2e report command to open HTML reports (#1892) 2026-11-18 08:39:35 +02:00
Oskar Kwaśniewski 6c4eddd0f9 feat(cli): ask for a GitHub star after init, upgrade, and update notice (#1874) 2026-11-17 13:17:51 +02:00
Kacper KapuściakandOskar Kwaśniewski d593863b41 feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init (#1647)
* feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init

e2e init asks for a CI pipeline in a project with the web engine (--ci github-actions|gitlab|none). It writes one job for the project's package manager and Node version that installs Chromium, runs e2e run, and keeps the report, and names the secret or CI/CD variable for the model key. It never overwrites a pipeline file; beside an existing .gitlab-ci.yml it writes a file to include.

* fix(cli): handle cubic's findings on the init CI step

Exact GitLab stage names, a distinct job name beside an existing e2e job, loopback baseURL in an existing config, no writes through a symlinked .github or .gitlab, --ci none in the skill, gitlab in the telemetry comment, and the GitLab Node image's major version in the docs.

* refactor(cli): keep the init CI pipelines as YAML templates

The GitHub Actions workflow and the GitLab job live in packages/e2e/templates/ci/ as complete pipelines for npm at the repository root, linted as they are. init loads them with the yaml package's Document API and edits a copy for the project's package manager, folders, Node version, secrets, and target, keeping the comments. The pinned actions now live only in the templates.

* fix(cli): point setup-bun at the package.json of a project in a subfolder

* docs(ci): link the GitLab page from init and the GitLab template

* feat(cli): write the iOS and Android CI jobs in e2e init

init writes one job per platform the config's targets run on: web targets in the web job, iOS and Android targets in their own jobs, which build the app, boot a simulator or an emulator, and install the app. Targets come from the new config, or from loading an existing one after the confirmation and the install. Expo and bare React Native apps get their build; any other app gets a step to replace. GitHub's mobile jobs follow e2e-inc/e2e-react-native's workflow; GitLab's run on self-managed runners.

* fix(cli): reinstall an emulator system image that holds no image in the Android CI job

* fix(cli): handle cubic's findings on the mobile CI jobs

Upload the emulator log from the GitLab Android job, count peer and optional engine dependencies, ask nothing for a new config with no engine, key caches on Bun's bun.lockb when that is the lockfile, and run the symlink test with a Windows junction.

* fix(cli): put current Android command-line tools in place before the emulator image

The runner image's cmdline-tools can be older than the system image's package format, and android-emulator-runner uses them to create the AVD, which then fails with 'contains no system images'. The Android job downloads command-line tools 23.0, checks its SHA-256, and installs the image with it.

* fix(cli): import moved core modules from @e2e-dev/core in init

* fix(cli): count a pipeline as set up only when a command runs e2e run

init matched the text 'e2e run' anywhere in a workflow, so a comment or a step name such as 'our old e2e runner' made --ci write nothing. It now reads the commands (run, script, before_script, after_script) from the parsed YAML and matches e2e run as words of their own.

Also: --ci help and the InitOptions JSDoc say tests on every platform, and yaml is pinned exactly like the other dependencies.

* chore(ci): bump setup-node and upload-artifact pins in the init templates, docs, and skill

docs/ci/vercel-previews.mdx on main pins setup-node v7.1.0 and upload-artifact v7.0.2, the pins this repository's own workflows run. The templates, three docs pages, and the skill still had v7.0.0 and v7.0.1, so the pin test failed once main was merged in.

---------

Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-16 17:21:49 +00:00
0a5ee16387 refactor: move e2e's core into @e2e-dev/core, and engines peer on it (#1737)
* refactor: move e2e's core into @e2e-dev/core, and engines peer on it

The engine contract, locators, agent, replay cache, secrets, and the
session half of a worker move from packages/e2e to packages/core, the new
@e2e-dev/core package, at the same paths. e2e depends on it at its exact
version (a changesets fixed group) and reaches its modules through
@e2e-dev/core/lib/*, which is not public API; e2e/engine and e2e/agent
re-export it. @e2e-dev/web and @e2e-dev/mobile peer on @e2e-dev/core and
import the contract from @e2e-dev/core/engine. Install smoke checks that one
copy of @e2e-dev/core is installed.

Most of this diff is git mv and import specifiers rewritten to
@e2e-dev/core/lib/<path>; no behavior changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: build and pack @e2e-dev/core where e2e is built or packed alone

The Bun job built e2e before the @e2e-dev/core it compiles against, and the
examples nightly installs e2e from a tarball whose exact @e2e-dev/core is
not on npm; it now packs @e2e-dev/core and overrides it in each example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(engines): name @e2e-dev/core/engine as the contract's source

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(e2e): list @e2e-dev/core beside an engine in init and upgrade

Engines peer on @e2e-dev/core. Yarn installs no peer the project does
not list, and under Plug'n'Play the engine then fails to load.

- init adds @e2e-dev/core with any engine
- upgrade adds a required @e2e-dev peer the project lacks, and keeps a
  package e2e depends on at the version e2e names

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-15 18:31:58 +00:00
Krzysztof Moch 2359860783 feat(cli): step through a test without a terminal (#1750) 2026-11-15 17:47:21 +01:00
Krzysztof MochandOskar Kwaśniewski 17a2e464f0 feat(explore): explore what a pull request changed with --changed-since (#1711)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-12 17:42:01 +00:00
Krzysztof Moch 565e685779 docs(cli): say --output must be inside the project root (#1721)
The --output row in the CLI reference and the bundled running guide never said the directory must be inside the project root and not the root itself, so a run with an output outside it failed INVALID_CONFIG with no warning in the docs. Fixes ENG-976.
2026-11-12 14:48:31 +00:00
Donny/강동윤andClaude Sonnet 5.5 b5e47036ba feat(runner): start the tests a change is likeliest to break first (#1589)
* feat(runner): start the tests a change is likeliest to break first

A test filter can score the tests it keeps: `relevance` in its result maps test ids to a number from 0 to 1. With `--changed-since`, each target starts the files of the likeliest tests first, in bands at 75%, 50%, and 25%, and the target with the likeliest file goes first. Inside a band the longest-first order from the previous run still decides, with more than one worker.

`decisionFilter()` scores every test it hands back: 1 for a test whose file, setup file, or stored screenshots changed, the model's answer for one it judged, and 0.5 for one it did not. It requires e2e 0.23.1 or newer, since an older runner refuses the unknown `relevance` key and drops the whole filter.

The run prints `ranked N of M tests by relevance, likeliest first` when the scores change the order.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test(runner): write the failing fixture as a constant, not built from a parameter

CodeQL flagged `failingFile(title)` for putting `JSON.stringify` output into generated code. Only `gamma` used it, so the file is a constant now.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs(runner): say how targets, bands, and the ranking line are decided

Review of #1589 found the docs looser than the code: targets go by the band of their best file, not its exact score, with ties in config order; a rating the model did not give counts as 50% and the test takes the higher of its two ratings; the bands are half-open; the ranking line follows when the scores put tests in more than one band; and a filter that returns no scores leaves the order alone. One test title said an empty target goes last, but it keeps its place.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs(runner): score known areas on the fallback path and name files, not tests

The areas example now scores the tests whose area a change reaches even when another changed file maps to no area, keeping the no-exclusion fallback. The README, docs, and skill say the run starts the files of the likeliest tests, ordered by relevance band with collection order breaking ties on one worker. The timings test makes beta and gamma wait for each other before they sleep, so a slow second worker cannot start alpha ahead of gamma.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(runner): keep the scores of a filter that fails

A filter that threw left every test unscored, so a run with no AI Gateway
key (a pull request from a fork) or a model outage fell back to collection
order and lost the one thing the decision filter knows without the model:
which tests the change touches.

The error a filter throws may now carry `relevance`. The runner checks it
the way it checks a result's, merges it by the highest score, and still
warns, leaves nothing out, and records `outcome: failed`. Scores that fail
the check are dropped, and the note says why. A call past its deadline is
abandoned, and nothing it throws is read.

decisionFilter() puts the scores the change alone gives (1 for a test the
change touches, 0.5 for the rest) on the error it throws, as a property
that is not enumerable, so printing the error does not list every test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs(runner): point the ordering prose at the timings history

The timings history (#1639) replaced the previous report as where file
durations come from, so the prose that says what orders files inside a
relevance band now names it: the filters guide, the `workers` bullet in
the config reference, the CI guide, the skill's running reference, and the
wording of the integration test that pins the one-worker gate.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 17:54:12 +00:00
Oskar Kwaśniewski 3735887569 chore: move from tester-army to e2e-inc GitHub org (#1672) 2026-11-10 20:11:53 +01:00
Donny/강동윤andClaude Sonnet 5.5 b285835b29 feat(run): keep a timings history that CI can cache (#1639)
* feat(run): keep a timings history that CI can cache

`e2e run` already starts the longest files first, but it read the
previous run's durations out of report.json, which a fresh CI checkout
never has. Write a small versioned `<output>/timings.json` (timings-1)
at the end of every run instead: per target::file milliseconds, smoothed
with an exponential moving average, with files that are gone and targets
the config dropped pruned.

The file is advisory. A missing, corrupt, or schema-invalid file is no
history, and an absent one falls back to report.json as before, so a
restored cache can reorder files and never change a result.
`--timings <file>` moves it. `--last-failed` still reads report.json.

Documents the actions/cache recipe for GitHub Actions and the generic
restore/save rule in the CI guide. Duration-balanced --shard is left for
a separate decision.

Refs ENG-1111

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(run): time a file only when the run carried out all of it

A `--grep`, `--last-failed`, interrupted, or `--repeat-each` run summed
only part of a file's tests, or several repeats of them, into the
timings history, so a 60 s file could drop to about 40 s after a rerun
of one failing test. A file now has a timing only when none of its tests
was left out, cut short, or never reached, and repeats past the first do
not count. Adds the integration test for the runner wiring.

Refs ENG-1111

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(run): tighten the timings keys, count an interrupted retry as partial, and say when ordering applies

- A retry cut short after a failure leaves the result under the earlier
  verdict; read the attempts, and a serial group's, for an interrupt.
- timings-1 keys: a target name, then a project-relative POSIX file with
  no `..` segment, in the schema and the reader alike.
- Rerun cache key adds github.run_attempt, since a rerun shares run_id
  and actions/cache never saves over an existing key.
- Docs and help say the history orders files only with more than one
  worker, and that the file is `<output>/timings.json`.

Refs ENG-1111

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs(run): say files with no recorded time start before the timed ones

Refs ENG-1111

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(run): time a file whose other tests a platforms list keeps off the target

A test a `platforms` list excludes is `selected: false` with cause
`platform-unavailable`, the same flag a `--grep` or `--last-failed` sets, so a
file holding both an iOS-only and an Android-only test never got a time on
either target. Only a filter makes a file partial; a permanent platform
exclusion does not.

Refs ENG-1111

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 14:58:07 +01:00
Oskar Kwaśniewski 5847701fb5 feat(e2e): run the CLI on Bun 1.4.3 and newer, without Node.js (#1616) 2026-11-09 18:25:18 +01:00
Krzysztof Moch 93c93f5e25 feat(cli): e2e run --step holds each step until a key (#1540) 2026-11-09 18:06:53 +01:00
Donny/강동윤 3c902b0c86 fix(init): register the MCP server for Codex (#1586) 2026-11-06 10:08:03 +01:00
Kacper Kapuściak f185ca8ba2 docs: retire sidebar page icons (#1582)
* docs: retire sidebar page icons

The docs sidebar no longer shows icons. Drop icon: from every page's frontmatter, delete the grey silhouettes in docs/images/icons/, and make check-docs-logos fail on any frontmatter icon instead of requiring one on vendor pages. Card logos stay required. The authoring-docs skill trades its Page icons section for a short Vendor logos one.

* docs: require a migrate overview card for each migration guide

* docs: match only top-level icon keys in check-docs-logos
2026-11-05 18:23:59 +00:00
Kacper Kapuściak 7d7d0e1615 feat(cli): draw the new e2e logo in init and help (#1575)
* feat(cli): draw the new e2e logo in init and help

* docs(cli): soften the bar-to-gap comment

* docs(cli): state the init logo's width limit

* feat(cli): load the init logo top to bottom like a scan

* feat(cli): draw the init logo's bars as full-block rows

* feat(cli): scan the init logo in as thin lines, then fill the bars

* feat(cli): scan the init logo in row by row

* test(cli): require the logo scan to add one row at a time

* feat(cli): draw each logo bar as one full-block row

* feat(cli): reveal the init logo with a sweeping scan line

* feat(cli): shade the init logo in
2026-11-05 11:53:36 +00:00
Oskar Kwaśniewski 0c0469605a chore: replace e2e.tester.army with e2e.com (#1572) 2026-11-05 09:23:45 +01:00
Oskar Kwaśniewski f4be1c4568 feat(explore): allow runs up to 100 steps and 2 hours (#1523) 2026-11-03 19:34:35 +01:00
Samanyu DasandClaude Opus 5.5 b8485ba2e7 fix(report): count a serial member executed when its body ran (#1470)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 10:52:01 +01:00
Oskar Kwaśniewski 91bd61490d feat(cli): e2e doctor preflight with engine doctor hook (#1493) 2026-11-03 10:27:52 +01:00
Samanyu DasandClaude Opus 5.5 d10655f343 fix(run): print run errors on stderr when the list reporter is off (#1474)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 10:29:30 +01:00
Oskar Kwaśniewski d94593b0c1 fix(reporter): stream test lines when piped, close output with failed tests (#1482) 2026-10-31 10:25:39 +01:00
Oskar Kwaśniewski 94e6e76548 feat(agent)!: keep model screenshots after a secret fill (#1437) 2026-10-28 19:11:24 -07:00
4c1e2e83d2 docs(cli): say which feedback paths an opt-out variable stops (#1397)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-10-28 14:46:37 -07:00
e087ed2c47 fix(cli): describe what e2e init changes in existing files (#1398)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-10-28 14:27:25 -07:00
Samanyu DasandClaude Opus 5.5 7f0a2f4997 docs(reference): say e2e upgrade pins only the packages that move (#1396)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 16:36:31 -07:00
Kacper Kapuściak e9e8c5005a feat(cli): e2e upgrade updates the agent skill copies (#1394) 2026-10-26 14:51:34 -07:00
Kacper Kapuściak ce249b62b0 feat(cli): run the init wizard when e2e explore finds no config (#1401) 2026-10-26 18:37:16 +00:00
Kacper Kapuściak 0d3e403c2e feat(cli): select web and mobile together in e2e init (#1338)
The engine question is a multiselect and --engine takes a list. With both engines the config gets a target per engine and the examples split into tests/web.e2e.ts and tests/mobile.e2e.ts, each requiring its engine's fixture.
2026-10-26 13:07:52 +00:00
Kacper Kapuściak 6b04d18a62 docs(examples): add standalone Angular project (#1389) 2026-10-26 08:39:11 +00:00
Samanyu DasandClaude Opus 5.5 bfae325707 fix(cli): reject an empty --reporter instead of running with no reporters (#1351)
`--reporter ''`, `--reporter ','`, or `--reporter "$REPORTERS"` with the
variable unset parsed to an empty list that replaced every configured
reporter, so a failing run printed nothing and exited 1. It is now a usage
error, exit 2, like an empty --tag, --target, or --agent. Applies to run
and explore, which share the parser.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 22:18:37 +00:00
Oskar Kwaśniewski da13624d31 feat(runner): run only the tests a pull request can affect (#1206) 2026-10-23 12:29:15 -05:00
Oskar Kwaśniewski e62486d668 docs(quickstart): list the common config options (#1332) 2026-10-22 12:20:53 -04:00
Kacper Kapuściak bbb13bf7ff feat(e2e): start a project from an example app with e2e init --example (#1318) 2026-10-21 14:17:11 -04:00
Kacper Kapuściak 329e03f7a8 feat(cli): add --action-timeout and --assertion-timeout to e2e run (#1310) 2026-10-21 13:47:20 -04:00
Kacper Kapuściak 1071933813 feat(config): discover e2e.config.js and document a JavaScript setup (#1280) 2026-10-20 08:53:36 -04:00
Krzysztof Moch 3bd32ef8ef fix(e2e): launch a device's pinned app before explore and MCP look at it (#1252) 2026-10-19 15:59:54 +02:00
Kacper Kapuściak 332b934684 feat(cli): add npx e2e upgrade (#1240) 2026-10-19 15:28:46 +02:00
Oskar Kwaśniewski 7b6ac5327e feat(cli): print an update notice when a newer e2e is on npm (#1238) 2026-10-19 11:11:44 +02:00
Kacper Kapuściak c304924e53 docs(reference): rewrite the API reference in the Next.js shape (#1197)
* docs(reference): rewrite the API reference in the Next.js shape

Every reference page now opens with a lede and a minimal example, groups its
API under ## sections that start with a summary table linking to one ###
entry per member, and keeps the TypeScript declarations under ## Types.

* docs(reference): fix claims the review caught

Narrow the security summary rows to their sections, generate the screenshot
file name in the engine example instead of building it from the label, and
correct defaults, exit codes, and examples on config, cli, screen, web,
mobile, test, runner, and engine.

* docs(reference): address review findings

Scope error classes to runner errors, guard a zero-slot prepare in the engine
example, keep the cli #help-and-version anchor, and correct poll cancellation,
same-run screenshots, openLink schemes, type_secret, longPress durations,
runner signals and close failures, empty usernames, and the feedback exit
code. Make the reporter and extract examples self-contained.

* docs(reference): trim the summary tables to the minimum

Each group's summary table keeps the linked name and one or two short
columns (type, default, or a few-word description). Examples and details
live in the entries; facts that only a table cell held moved into its entry.
The expect overview is headed Reference instead of Entry points, and the
security rules table names what each section covers instead of restating
its claims.

* docs(reference): open every page's API under one Reference heading

Each page's first section after the lede is now ## Reference. A page with
one group lists its entries at ### right under it; a page with several keeps
each group as a ### with its summary table and its entries at ####.
Explanatory sections (behavior, budgets, setup, types) follow it. Linked
sub-parts that moved to ##### keep their anchors through explicit ids.

* docs(skills): describe the reference page shape in authoring-docs

Reference pages open their API under ## Reference, keep summary tables to
the name and one or two short columns, and nest entries at ### or ####
because Mintlify drops anchors below ####. ship-pr points reference edits
at these rules instead of the guide shortening pass.

* docs(reference): address cubic review on tables and skill

Keep readonly in the mobile device type, show the keyboard and mouse
object shapes, say a locator assertion times out as ASSERTION_FAILED, and
give reference pages a narrower shortening pass in authoring-docs.

* docs(reference): tighten the reference pages

Merge facts said twice into one place, move each declaration's JSDoc to a
single home, link timeout defaults, key grammar, and secure-field rules to
the pages that own them, and split long sentences. No fact is cut.

* docs(reference): address cubic review of the tightening pass

Qualify the GitHub token for fork pull requests, provider recording URLs,
duplicate device entries, MODEL_UNAVAILABLE timing, the APP_UNREACHABLE log
quote, UNKNOWN_TOOL versus UNSUPPORTED_CAPABILITY, node-less agent tools,
which model receives screenshots, passing assert evidence, and config-path
hosts; fix three garbled sentences.

* docs(reference): say device, not simulator, for worker isolation
2026-10-16 14:41:30 +00:00
Piotr Pasztor bf3c8e8ac1 fix(agent): stop the run once when the provider does not serve the model (#1153) 2026-10-16 14:44:25 +02:00
Jay Hemnani af0437bb8f fix(cli): report a skipped test in one shard only (#1113) 2026-10-16 12:25:56 +02:00
Oskar Kwaśniewski 236611fafb docs: fix packaged docs against source, ship snippets, check anchors (#1168)
* docs: fix packaged docs against source, ship snippets, check anchors

* docs: tighten review wording, harden anchor check fences and ids

* docs: address review on tool errors, CLI failure output, anchor check

* docs(security): base64 is decoded in text downloads

* docs(telemetry): shorten the telemetry page (#1174)

* docs(telemetry): shorten the telemetry page

* docs(telemetry): engine versions are semver or unversioned

* docs(migrate): shorten the Playwright migration guide (#1172)

* docs(debugging): shorten the debugging guide (#1170)

* docs(auth): merge repeated secret facts on the signing-in page (#1169)
2026-10-15 17:56:13 +02:00
Oskar Kwaśniewski babae32bda feat(cli): add e2e run --max-duration and maxDuration config (#1131)
* feat(cli): add e2e run --max-duration and maxDuration config

* fix(cli): keep e2e explore on its own --timeout, not maxDuration

* fix(cli): skip max-duration timer past setTimeout cap, cover config and explore paths

* refactor(cli): address max-duration review

- explore drops maxDuration from its config; runner no longer special-cases it
- scheduler records RUN_TIMEOUT from the skip reason, naming --max-duration or maxDuration
- stopEarly wakes the loop itself
- reject maxDuration above 2147483647 ms instead of skipping the timer
- trim cli docs, clarify ci wording, sync plugin
2026-10-14 19:09:31 +02:00
Oskar Kwaśniewski 27a6ed0d83 feat(feedback): send bugs without a repro to GitHub, refuse outdated reports (#1137)
* feat(feedback): send bugs without a repro to GitHub, refuse outdated reports

* fix(feedback): redact error code, qualify version check docs, valid skill example
2026-10-14 18:45:45 +02:00