Commit Graph
22 Commits
Author SHA1 Message Date
Krzysztof Moch eddf577526 feat(reporters): build the built-in reporters on the public Reporter API (#1936)
* feat(reporters): build the built-in reporters on the public Reporter API

The built-in list, dot, json, junit, and markdown reporters now use only
the public Reporter contract: factories on e2e/reporters, printsTo for the
terminal stream a reporter prints to (one reporter per stream), live
onOutput, onStepPrompt, and onSummary hooks for any number of reporters,
and reportFiles for files the runner writes atomically. The list and dot
ids beside json print to stderr, so list,json now works as dot,json does.
@e2e-dev/html-reporter writes through reportFiles.

* docs(reporters): bring the Japanese and Chinese reporter pages up to date

Translates the reporter reference, the config reporters table, the
--reporter row, and the debugging output paragraph. In English, the name
section now says a reporter may share a built-in's name, and two lines
read less ambiguously.
2026-11-19 15:58:33 +00:00
Krzysztof MochandOskar Kwaśniewski 17a2e464f0 feat(explore): explore what a pull request changed with --changed-since (#1711)
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-12 17:42:01 +00:00
Krzysztof Moch a13380728f feat(html-reporter): write the run as one HTML page (#1642)
* feat(html-reporter): write the run as one HTML page

* fix(html-reporter): match the headline to the listed tests, fence prompts safely, and stop on the reporter signal

* docs(html-reporter): code frames need the test file in the project

* test(e2e): give the JUnit attachment test's finished run a redact
2026-11-10 15:48:15 +01:00
Samanyu DasandClaude Opus 5.5 b8485ba2e7 fix(report): count a serial member executed when its body ran (#1470)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 10:52:01 +01:00
Oskar Kwaśniewski d181e2645f fix(github): give an e2e explore run its own pull request comment (#1438) 2026-10-28 17:45:34 -07:00
Oskar Kwaśniewski da13624d31 feat(runner): run only the tests a pull request can affect (#1206) 2026-10-23 12:29:15 -05:00
aa9699a688 fix(github): distinguish API throttling from token permissions (#1038)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-10-16 15:38:08 +02:00
Oskar Kwaśniewski ce4dac3737 fix(github): update only the reporter's own PR comment (#1161)
* fix(github): update only the reporter's own PR comment

* docs(changeset): say any bot author matches for installation tokens
2026-10-15 12:30:10 +02:00
Oskar Kwaśniewski 1b92a6ddfa feat: e2e trace pages replace the Playwright trace (#869) 2026-10-07 11:28:27 +02:00
Oskar Kwaśniewski 8657c8b710 test(mobile): fake clock in engine tests, drop the simulator suite CI never ran (#808)
* test(mobile): fake the clock and shrink PNGs in engine tests, trim duplicates

engine.test.ts 6.4s -> 0.4s: retry backoff and transition budgets on fake timers, small screenshots. Merge or drop tests restating constants or covered elsewhere; prose asserts in errors.test check the headline and recovery only.

* test(mobile): delete the simulator suite no workflow runs

E2E_AGENT_DEVICE_SIMULATOR was never set in CI; the mobile benchmark drives real simulators and emulators. Drops test:simulator and the integration project.

* test(mobile): cover duplicate refind, clipboard step label, unreadable secure screenshot

* test(eas): restore Date.now in afterEach, unique run ids, drop constant asserts

* test(github): drop the factory tautology and markdown asserts core owns

* test(mobile): keep the videoTouches and interactive snapshot option checks

* test(mobile): name the openApp permission refusal by what it checks
2026-10-04 10:10:57 +02:00
Oskar Kwaśniewski 3a9667c046 fix(run): keep --last-failed reruns honest: failure-limit skips, carried debts, kept evidence (#724)
* fix(run): keep --last-failed reruns honest: failure-limit skips, carried debts, kept evidence

--last-failed now reruns tests --max-failures skipped. A test or failed
suite hook another filter leaves out stays owed in the report's new
run.carried until a rerun runs it; the github fold shows carried rows and
errors and stays red while anything is carried. A rerun prunes artifacts/
to what the previous report names and writes its attempts under
artifacts/rerun-<n>/ instead of wiping the tree and reusing attempt paths.

* fix(run): carry a setup's hook debt when no selected test needs it

The report has no row for an unneeded setup, so carryForward now asks the
collection whether a test still exists instead of the report's rows.

* fix(run): reject malformed artifacts in --last-failed reports, tighten rerun docs

Distinct ids in the carried fixture row.

* fix(run): keep carrying tests of a file a narrowed rerun could not collect

* fix(github): a carried interrupted test reads interrupted, not failed, in the folded page
2026-10-03 16:36:53 +02:00
Oskar Kwaśniewski 7ab80bc610 fix(report)!: count interrupted tests apart from failures (#713)
* fix(report)!: count interrupted tests apart from failures

* fix(report): tell a failed test from its last failed attempt, whatever the cut retry left
2026-10-03 09:30:44 +02:00
Oskar Kwaśniewski 62bfe0a896 fix(github): long marker fields keep a digest so distinct keys never share a comment (#720)
* fix(github): long marker fields keep a digest so distinct keys never share a comment

* docs(changeset): note one-time duplicate for long marker fields
2026-10-02 18:17:26 +02:00
Oskar Kwaśniewski 67c29fc6b2 chore!: publish engines and reporter under the @e2e-dev npm scope (#602) 2026-09-28 19:05:05 +02:00
Oskar Kwaśniewski 61603e6c2c fix(github): a --last-failed rerun updates the pull request comment as one run (#537) 2026-09-25 17:09:09 +02:00
Oskar Kwaśniewski cdc41098c7 fix(report): markdown cells cannot mention, autolink, or start a block, and all-dot names are rejected (#448) 2026-09-23 10:53:49 +02:00
Oskar Kwaśniewski b71ecc0f09 fix(report): leave unselected results off the page and JUnit, link sources under the project path (#422) 2026-09-22 18:14:28 +02:00
Oskar Kwaśniewski 2e593dfb46 fix(agent): recover semantic timeouts with masked pixels (#317)
* fix(agent): recover semantic timeouts with masked pixels

Distinguish unavailable semantics from an empty tree so a proven fresh screenshot can remain useful without supporting an absence judgment. Reserve time for independent masking, reject stale references, and stop trace replay or recording after evidence degrades.

Respect judgment vision settings and attempt pixel taint. Keep device capture closed when accessibility bounds cannot prove masking. Report pre-model capture failures without inventing a revision or verdict.

* fix(agent): isolate fallback capture and evidence ownership

Reserve document node IDs before readers run and bound the entire semantic transaction, including metadata and handle readback. Quarantine late handles and publish each capture only while its page and generation remain current.

Carry semantic or pixel evidence through the internal session and agent paths. Preserve completed cache captures for one executor handoff, retire semantic presentation history on fallback, and keep public interfaces extendable.

* test(agent): align capture assertions with cache handoff

Count the reused cache observation once and wait for the fixture delayed control before asserting an unchanged screen after batched actions.

* docs(agent): clarify fallback without act vision modes

* test(agent): cover requested pixels after semantic fallback

* test(playwright): wait for recorded paint in lifecycle coverage

* test(playwright): flush a frame before checking video paint
2026-09-15 13:47:30 +02:00
Oskar Kwaśniewski c2c5df7df9 feat(e2e)!: the runner publishes as e2e; @e2edev/e2e is retired (#294) 2026-09-14 08:56:22 +02:00
Oskar Kwaśniewski 8811ba7de9 feat(report): a markdown reporter writes the run as summary.md, with an exploration's findings, and the GitHub comment renders from it (#284) 2026-09-12 15:54:10 +02:00
Oskar Kwaśniewski 8287bf930b feat(run)!: run one test as several agents; --agent takes a list (#262) 2026-09-11 10:03:35 +02:00
Oskar Kwaśniewski a0a4df01b4 feat(github): the GitHub reporter comments on the pull request from Actions (#246) 2026-09-10 14:47:36 +02:00