`@e2e-dev/web` and `@e2e-dev/mobile` declare `e2e` an optional peer, but
their main entry imported it statically to build `test`. A program on
`@e2e-dev/core` alone that wrote `import { web } from '@e2e-dev/web'`
failed with ERR_MODULE_NOT_FOUND before any of its code ran.
The main entry now requires `e2e` only when it resolves. Without it, the
engine factory works and `test`, `describe`, and the hooks throw
RUNNER_NOT_INSTALLED when called. It uses createRequire, not a top-level
await, so a CommonJS file can still require() the entry.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor: move e2e's core into @e2e-dev/core, and engines peer on it
The engine contract, locators, agent, replay cache, secrets, and the
session half of a worker move from packages/e2e to packages/core, the new
@e2e-dev/core package, at the same paths. e2e depends on it at its exact
version (a changesets fixed group) and reaches its modules through
@e2e-dev/core/lib/*, which is not public API; e2e/engine and e2e/agent
re-export it. @e2e-dev/web and @e2e-dev/mobile peer on @e2e-dev/core and
import the contract from @e2e-dev/core/engine. Install smoke checks that one
copy of @e2e-dev/core is installed.
Most of this diff is git mv and import specifiers rewritten to
@e2e-dev/core/lib/<path>; no behavior changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* ci: build and pack @e2e-dev/core where e2e is built or packed alone
The Bun job built e2e before the @e2e-dev/core it compiles against, and the
examples nightly installs e2e from a tarball whose exact @e2e-dev/core is
not on npm; it now packs @e2e-dev/core and overrides it in each example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(engines): name @e2e-dev/core/engine as the contract's source
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(e2e): list @e2e-dev/core beside an engine in init and upgrade
Engines peer on @e2e-dev/core. Yarn installs no peer the project does
not list, and under Plug'n'Play the engine then fails to load.
- init adds @e2e-dev/core with any engine
- upgrade adds a required @e2e-dev peer the project lacks, and keeps a
package e2e depends on at the version e2e names
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
* fix(mobile): keep userinfo out of openLink and openApp step labels
* fix(mobile): strip userinfo from slashless web links, tab-padded links and backslash userinfo in labels
* fix(mobile): label links through the URL parser, never echo an unparseable one
* fix(mobile): refuse an unknown snapshot or videoTouches value in mobile()
A typo such as snapshot: 'Interactive' captured the full tree and videoTouches: 'false' drew the touches, since the engine compares against the exact values and falls back to the default. Both now fail at config load with INVALID_CONFIG naming the accepted values, like settle and transition. The reference's Throws table also lists the existing platform check.
* fix(mobile): format a refused snapshot or videoTouches value without throwing
* fix(mobile): resolve snapshot and videoTouches once, like settle and transition
* docs(mobile): say which device runs when no device is named
A paired iPhone or iPad counts as booted and joins the pool; with none booted, agent-device boots an iPhone simulator over an iPad, then the newest OS, then name order.
* docs(mobile): name paired devices and the Apple TV and Android TV tiers in the cold-pick order
* docs(mobile): say a reachable paired device counts as booted, fix changeset and wording
* docs(mobile): name reachable paired iPhones and iPads in the device pool
* docs(screen): say what doubleTapAt does on a device
* docs(screen): give doubleTapAt its own iOS tap gap
* docs(mobile): give doubleTap and doubleTapAt the same iOS gap
The --output row in the CLI reference and the bundled running guide never said the directory must be inside the project root and not the root itself, so a run with an output outside it failed INVALID_CONFIG with no warning in the docs. Fixes ENG-976.
* fix(mobile): hide nodes under the soft keyboard and fail covered taps as actions
A node whose centre the soft keyboard covers is now hidden in the
projection, so scrollUntilVisible keeps scrolling to it, toBeVisible()
fails on it, and getByRole skips it. The band is agent-device's measured
frame, else the keyboard elements in the tree down to the screen bottom
(the iOS simulator capture publishes no frame). The keyboard's keys, dock,
and suggestion bar stay visible, and so does the focused field.
agent-device's target_covered and tap_keyboard_occludes_target refusals
that dispatched nothing now map to NOT_ACTIONABLE (ACTION_FAILED) with the
hint naming the cover, not ENGINE_FAILURE.
Part of ENG-1042.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(mobile): read the keyboard band off keys without a keyboard element
Also qualify the skill line and changeset: the keyboard's own nodes, the
focused field, and a container with a showing child stay visible, and only
a refusal that sent no tap becomes ACTION_FAILED.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* docs(mobile): say which platform each biometric sensor works on
* docs(mobile): cover physical iPhone and screen lock wording for biometrics
* docs(mobile): name physical iOS devices, not only iPhones, for biometrics
* docs(mobile): say that settle applies to the agent's actions only
The settle JSDoc and its reference copy said the wait follows any tap,
fill, scroll, or back, but a test's own steps never settle. The options
table and the package README already said so. The Detox and Maestro
guides pointed deterministic suites at settle; point them at transition.
* docs(mobile): qualify settle as locator actions and point agent steps at settle in the Detox guide
* docs(mobile): name device.back() as the one test step that settles
* docs(mobile): rewrap the settle option comment
* feat(mcp): tell the agent to read the writing-tests guide before it writes a test
The server instructions and every open_session result now say to read
e2e://guide/writing-tests (or npx e2e guide writing-tests) before writing
or editing a test.
ENG-1269
* test(mcp): assert the whole writing-guide line in the instructions; fix its docs placement
* feat(runner): start the tests a change is likeliest to break first
A test filter can score the tests it keeps: `relevance` in its result maps test ids to a number from 0 to 1. With `--changed-since`, each target starts the files of the likeliest tests first, in bands at 75%, 50%, and 25%, and the target with the likeliest file goes first. Inside a band the longest-first order from the previous run still decides, with more than one worker.
`decisionFilter()` scores every test it hands back: 1 for a test whose file, setup file, or stored screenshots changed, the model's answer for one it judged, and 0.5 for one it did not. It requires e2e 0.23.1 or newer, since an older runner refuses the unknown `relevance` key and drops the whole filter.
The run prints `ranked N of M tests by relevance, likeliest first` when the scores change the order.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* test(runner): write the failing fixture as a constant, not built from a parameter
CodeQL flagged `failingFile(title)` for putting `JSON.stringify` output into generated code. Only `gamma` used it, so the file is a constant now.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* docs(runner): say how targets, bands, and the ranking line are decided
Review of #1589 found the docs looser than the code: targets go by the band of their best file, not its exact score, with ties in config order; a rating the model did not give counts as 50% and the test takes the higher of its two ratings; the bands are half-open; the ranking line follows when the scores put tests in more than one band; and a filter that returns no scores leaves the order alone. One test title said an empty target goes last, but it keeps its place.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* docs(runner): score known areas on the fallback path and name files, not tests
The areas example now scores the tests whose area a change reaches even when another changed file maps to no area, keeping the no-exclusion fallback. The README, docs, and skill say the run starts the files of the likeliest tests, ordered by relevance band with collection order breaking ties on one worker. The timings test makes beta and gamma wait for each other before they sleep, so a slow second worker cannot start alpha ahead of gamma.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(runner): keep the scores of a filter that fails
A filter that threw left every test unscored, so a run with no AI Gateway
key (a pull request from a fork) or a model outage fell back to collection
order and lost the one thing the decision filter knows without the model:
which tests the change touches.
The error a filter throws may now carry `relevance`. The runner checks it
the way it checks a result's, merges it by the highest score, and still
warns, leaves nothing out, and records `outcome: failed`. Scores that fail
the check are dropped, and the note says why. A call past its deadline is
abandoned, and nothing it throws is read.
decisionFilter() puts the scores the change alone gives (1 for a test the
change touches, 0.5 for the rest) on the error it throws, as a property
that is not enumerable, so printing the error does not list every test.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* docs(runner): point the ordering prose at the timings history
The timings history (#1639) replaced the previous report as where file
durations come from, so the prose that says what orders files inside a
relevance band now names it: the filters guide, the `workers` bullet in
the config reference, the CI guide, the skill's running reference, and the
wording of the integration test that pins the one-worker gate.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* feat(mobile): export agent-device from @e2e-dev/mobile/agent-device
Custom tools that extend the mobile engine with agent-device calls have
to import `agent-device` directly. pnpm does not resolve that import
from a transitive dependency, so the project installs agent-device
itself and has to keep it at the exact version @e2e-dev/mobile pins,
because two versions do not share one daemon.
The new subpath re-exports what a tool commonly needs from the engine's
copy: `createAgentDeviceClient`, the error helpers (`AppError`,
`isAgentDeviceError`, `normalizeAgentDeviceError`), `centerOfRect`, and
the types for the client, snapshots, screenshots, targets, gestures,
text entry, alerts, and waits. `ensureDaemon` and `startAgentDevice`
stay out because the engine owns the daemon.
The install smoke imports the subpath from a fresh pnpm project with no
agent-device dependency and checks there is one agent-device install.
The e2e skill tells agents to use this import instead of adding
agent-device.
* docs(mobile): mark @e2e-dev/mobile/agent-device unstable, point tests to unstable_raw
A client built with `createAgentDeviceClient` does not get the engine's
daemon, lease, or per-worker session name, so the docs no longer say it
drives the engine's session. Tests use `device.unstable_raw.client`.
The subpath is agent-device's API at the pinned version, so it is
labeled unstable like `unstable_raw`.
* chore(mobile): shorten the agent-device subpath comments
* feat(mobile): give custom tools the running device session
A custom tool had no way to reach the engine's agent-device client, and
a client built from `createAgentDeviceClient` misses the engine's
daemon, lease, and per-worker session. `deviceSession(...engines)` on
`@e2e-dev/mobile/tools` returns the running attempt's `DeviceSession`,
the object a test reads from `device.unstable_raw`, using the lookup
`mobileTools` already had.
From review: `@e2e-dev/mobile/agent-device` drops the client factory and
keeps the types and error helpers. The skill line is scoped to these
imports, and the stale Limrun peer line is fixed. The export-map test
checks every entry's paths, the install smoke stops reading the `.pnpm`
layout, and the docs keep one full description in the reference.
* fix(mobile): spread the device selection in the deviceSession example
The docs example called `settings.update` without the session's
`selection`, unlike every engine call, so it could miss the worker's
device before an app is open. Also restores `createDeviceFixture`'s
JSDoc and tests the several-attempts refusal of `deviceSession`.
* refactor(mobile): keep raw sessions behind a neutral unstable API
* fix(mobile): type the unstable raw session client
---------
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
* test(core): reproduce app commands exiting after readiness
* fix(core): fail runs when app commands exit after readiness
* test(core): tighten app exit outcomes and diagnostic docs
* fix(core): scope app command exits to dependent targets
Retain all owners of deduplicated commands and route ready app exits through the scheduler. Ignore owners with no unfinished work, preserve healthy targets, and classify forced app cancellation before watchdog timeout synthesis.
Cover shared ownership, completed targets, teardown, readiness and interrupt ordering, navigation advice, and both watchdog/app-grace orderings.
* fix(core): clarify app readiness and output guidance
* fix(core): share app command interruption and cover buffered exits
Use the same per-worker interruption path for app exits and run interrupts.
Iterate snapshots so a synchronous transport exit cannot skip another worker.
Cover exits buffered between readiness and scheduling, serial-group exit
status, and returned-unit skip reasons. Keep the aggregate process hook
separate from the individual managed-process hook.
* refactor(core): return app-exit target set directly
The scheduler result contains only the target names used to distinguish
app cancellation from a user interrupt. Return that set directly instead
of wrapping it in a named single-field result and a caller temporary.
Keep the same completion and drain order.
* test(core): cover app-exit guards and process liveness
* test(core): expect app exits to quote short logs whole
Main's #1577 quotes a log of up to 40 lines whole and keeps the first
lines of a longer one, so an app exit after readiness now quotes the
test server's first line too. The test asserts that instead of the old
tail-only quote, and docs/web.mdx says unexpected exits quote the log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changeset): note that background launchers now fail
Apply suggestion from @kdy1
Co-authored-by: Donny/강동윤 <kdy.1997.dev@gmail.com>
* fix(core): keep reused app hints from promising command output
A refused navigation on a target whose app.command sets reuseExisting
now says the runner may use a process it did not start, whose output
app.command.log never holds. The docs scope the exit quote to e2e run,
list spawn and log-open failures under APP_UNREACHABLE, and name the
first declaring target in the shared-command error label.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Donny/강동윤 <kdy.1997.dev@gmail.com>
* fix(mobile): boot a shut-down device when none is booted and no device is named
With no `device` and nothing booted, the engine called agent-device's boot with only the platform, which fails with AMBIGUOUS_MATCH on any Mac with several simulators. Pass agent-device's opt-in `pick: 'newest'` (0.21.35+) when the selection names no device.
Fixes ENG-1037
* docs(mobile): name the mobile-over-TV rule in the cold boot pick
* docs(mobile): show the per-device prepare timing line and how to read it
* docs(mobile): say what the open step does and which failures are tolerated
* docs(mobile): say the app opens on both platforms and the order of the two lines
* fix(web): check a radio or checkbox through its own label when the label covers it
Element Plus hides the native input under its label's styled spans or
gives it no box, so Playwright's hit-target check refused it. check,
uncheck, and tap now click the label at the input's point when the
element at that point is the label's own content, or the input's first
label with a box when the input has none; anything else on top, a
disabled input, an input on top itself, or a boxless input whose labels
are all hidden is clicked as before.
One in-page read decides whether a label stands in, so a tap on anything
else pays that read and keeps locator.click, under one deadline. A
locator that never appears still fails NOT_ACTIONABLE within one timeout.
Fixes#1276
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(web): skip the label stand-in for disabled fieldsets, hidden labels, and links over the input
A control a disabled fieldset holds is refused by :disabled, not only
the attribute. A boxless control picks a label that is rendered
(checkVisibility), since visibility: hidden keeps its box. No label
stands in when the element at the click point, the input's centre or a
boxless label's centre, is a link, button, or other interactive
descendant of the label: the click would press that instead, and a tap
could follow a link. The locator's label resolution runs under the
action's remaining deadline.
Docs: uncheck reaches a checkbox only, the boxless case needs a visible
label, and a link or button in the label still fails.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(web): choose the label stand-in past loading masks and wait for an unrendered control
The stand-in was decided once from elementFromPoint, so a loading mask on
top, or a dialog not open yet, sent the click to the input, which its label
then covered until the deadline. The choice now reads the stack at the
input's point past anything outside the control and its labels, so
Playwright's checks on the label wait the mask out, and is polled while
nothing of the control is rendered, failing NOT_ACTIONABLE at the deadline.
One evaluateHandle returns the label itself, so the index and the second
resolve are gone. A point outside the label's box clicks the input as
before, and a tap on anything else returns before scrolling.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): click no control after a cancelled stand-in read, and dispose the null handle
The surface abandons a cancelled or timed-out operation without stopping
it, so a stand-in read still in flight went on to click. The signal is now
checked right before both click paths. The handle `pickStandIn` returns
for a control clicked as itself is disposed instead of living until the
page navigates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(web): type the label stand-in as undefined instead of a null cast
A `let` initialised to `null` narrows to `null`, and TypeScript does not
see the assignment inside the poll callback, so the old code needed an
`as` cast and a comment. Declared without an initialiser, the variable
keeps its declared type after the poll. No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(web): give the label refusal cases a 500ms deadline
Each "still refuses" case waited out the 2s deadline twice (check, then
tap). `perform` now takes a timeout, and those cases pass 500ms. The file
drops from about 102s to 48s locally.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(web): resolve the element target through ElementList, as an observed ref does
Since #1571 an observed ref's handle is made on first use from the walk's
in-page array. Building the check-actions element target the same way runs
every label stand-in case on the path an agent step takes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(web): cross-check a checkbox or radio its label draws over no box
Element Plus shrinks its native checkbox to a 0x0 box under the label it
draws. Chrome's accessibility tree lists the control, and the tree leaves
it out, so an agent sees the label text with no checkbox and no state
(#1593). This page fails the cross-check with five `chrome missing` lines
until the reader lists the labelled ones.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): list a checkbox or radio its own label draws over no box
The reader treats a box with no width or height as hidden, as
Playwright's `toBeVisible` does, and leaves hidden elements out of the
observation. Element Plus shrinks its native checkbox to 0x0 under the
box its label draws, so the agent saw the label text with no checkbox and
no checked state, and a tap that ticked it reported no change (#1593).
The walk now lists such a control while a label of its own renders, and
marks it not hidden, as it does a select's options. Chrome's accessibility
tree keeps the input, so the cross-check agrees on the labelled ones. A
shrunk control with no rendered label element still shows a person
nothing and stays out, recorded in expected.txt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(web): say the agent sees a checkbox its label draws over no box
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(web): pin that the observation shows a label-drawn checkbox the locator calls hidden
Agent code skips a node whose hidden state is true, so the listed control
must read as shown, while the locator path keeps Playwright's visibility.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(web): read a drawn-by-label control's labels the way the reader does, and say locators still call it hidden
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(web): limit the label-click claims to what the click checks
The hit check runs at the point the click lands, so a cover elsewhere over
the drawn box does not fail it, and uncheck refuses a native radio before
any click.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): list a label-drawn checkbox only under a label a tap can click
The reader listed a box-less checkbox under a `display: contents` label,
which counts as shown while a child paints. A tap clicks the label in its
place (`pickStandIn`), which needs a box the label never has, so the tap
failed `the control and its labels are not visible` where tapping the
label text used to work. The reader now counts only a label with a box.
Tests pin each guard of `isDrawnByLabel`: visibility, a slotted input in
skipped content, the checkbox or radio type, and the `display: contents`
label. A new action test taps every checkbox or radio an observation lists
over no box.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): leave out a label-drawn checkbox whose label holds a link
A tap on a box-less checkbox clicks its label, and `pickStandIn` refuses
when the label's centre lands on a link, so the click follows no link.
The reader still listed such a checkbox ("Accept the Terms and
Conditions" in Element Plus), and its tap failed `element is not
visible`. The reader now counts only a label that holds no link or
control besides its own, which is stricter than the tap, so it lists no
checkbox the tap refuses.
The action test now uses only box-less controls and checks each one has
no box. The docs say when the agent sees only the label text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(web): say the agent still sees a link inside a label-drawn checkbox's label
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(web): pin the clicked control and cover uncheck through a label
A locator that matches another control after the state read must still
click the control it read. Removing the pinned element from setChecked
fails the new case.
uncheck on an Element Plus checkbox clears it through its label, and on
an Element Plus radio it is refused with no click. Turning the label
stand-in off fails it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): leave out a label-drawn checkbox whose label is itself a control
A label with role="button", role="link", or contenteditable listed its
box-less checkbox, but a tap on it fails NOT_ACTIONABLE, because
pickStandIn's closest() check matches the label itself. The reader now
skips such a label too, so the agent taps the label, as on main.
The JSDoc, the cross-check reason, and docs/web.mdx now say the tap
refuses only when the label's centre lands on a link, and that the
reader's rule is stricter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
Add opt-in read-only assertion exploration with bounded scrolling and
model calls. Keep every viewport for the judgment and classify exhausted
assertions as inconclusive without changing default model selection.
The built-in agent picks a dedicated assertion loop by step kind: its
own rules, opening, vision policy, and inconclusiveOnOverflow. Tool
removal lives in the grammar's assertion filter. createVerdictTool(kind)
drops ASSERTION_INCONCLUSIVE from an act verdict instead of rejecting it,
so ACP serves one complete_step without validating by hand.
Co-authored-by: Ubuntu <ubuntu@ip-10-48-38-144.us-west-2.compute.internal>
* feat(html-reporter): write the run as one HTML page
* fix(html-reporter): match the headline to the listed tests, fence prompts safely, and stop on the reporter signal
* docs(html-reporter): code frames need the test file in the project
* test(e2e): give the JUnit attachment test's finished run a redact
* docs(security): list BrowserStack, TestMu AI devices, and E2B in outbound connections
* docs(security): say the e2b sandbox downloads Node only when absent
* feat(run): keep a timings history that CI can cache
`e2e run` already starts the longest files first, but it read the
previous run's durations out of report.json, which a fresh CI checkout
never has. Write a small versioned `<output>/timings.json` (timings-1)
at the end of every run instead: per target::file milliseconds, smoothed
with an exponential moving average, with files that are gone and targets
the config dropped pruned.
The file is advisory. A missing, corrupt, or schema-invalid file is no
history, and an absent one falls back to report.json as before, so a
restored cache can reorder files and never change a result.
`--timings <file>` moves it. `--last-failed` still reads report.json.
Documents the actions/cache recipe for GitHub Actions and the generic
restore/save rule in the CI guide. Duration-balanced --shard is left for
a separate decision.
Refs ENG-1111
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(run): time a file only when the run carried out all of it
A `--grep`, `--last-failed`, interrupted, or `--repeat-each` run summed
only part of a file's tests, or several repeats of them, into the
timings history, so a 60 s file could drop to about 40 s after a rerun
of one failing test. A file now has a timing only when none of its tests
was left out, cut short, or never reached, and repeats past the first do
not count. Adds the integration test for the runner wiring.
Refs ENG-1111
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(run): tighten the timings keys, count an interrupted retry as partial, and say when ordering applies
- A retry cut short after a failure leaves the result under the earlier
verdict; read the attempts, and a serial group's, for an interrupt.
- timings-1 keys: a target name, then a project-relative POSIX file with
no `..` segment, in the schema and the reader alike.
- Rerun cache key adds github.run_attempt, since a rerun shares run_id
and actions/cache never saves over an existing key.
- Docs and help say the history orders files only with more than one
worker, and that the file is `<output>/timings.json`.
Refs ENG-1111
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* docs(run): say files with no recorded time start before the timed ones
Refs ENG-1111
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(run): time a file whose other tests a platforms list keeps off the target
A test a `platforms` list excludes is `selected: false` with cause
`platform-unavailable`, the same flag a `--grep` or `--last-failed` sets, so a
file holding both an iOS-only and an Android-only test never got a time on
either target. Only a filter makes a file partial; a permanent platform
exclusion does not.
Refs ENG-1111
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* feat(mcp): open a session without a config by naming the engine
open_session takes engine, a package name or the factory call a config
would write as {package, export, options}, and app. An option can be
another such call, which is how an engine gets a device or browser
provider. Packages are found in the project, beside e2e, or in
~/.e2e/engines; ENGINE_NOT_INSTALLED names the install command.
@e2e-dev/web and @e2e-dev/mobile default-export their factory.
* fix(mcp): load import-only packages and refuse builtins as engines
Review fixes: resolve a package by its node_modules directory, so an
installed package without a loadable entry is reported as such; fall
back to the import condition; refuse Node builtin names; never read
the options object itself as a factory call; mobile() names the
platforms it takes when called without one.
* docs(mcp): say "no e2e config file", split the config error rows, pin the __proto__ key
Review fixes: a concrete app.url in the skill example, the string
shorthand in AGENTS.md, the engine-only wording in the security note,
and a test assertion that the __proto__ key survives as an own property.
* fix(report): name only the selected targets in the run header
`run-started` carried every configured target, so `e2e run --target web`
printed `targets: ios, android, web` while only `web` ran. It now carries
the targets `--target` selected, in config order. An unknown target ID
still fails the run with UNKNOWN_TARGET; the header then names every
configured target. The report's `run.targets` is unchanged.
Fixes#1658
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(report): select the run's targets once, before the header
The header and the run now share one `--target` selection instead of
selecting twice and discarding the first call's error. An unknown target
is still recorded as an UNKNOWN_TARGET collection failure inside the run,
after `run-started`. The reporters page says the header names every
configured target in that case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changeset): say when the run header still names every target
Without `--target`, or with an unknown target ID, the header and
`run-started` still name every configured target; the note now says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(report): link failure screenshots as GitLab attachments in JUnit XML (#1507)
A testcase whose test failed, timed out, passed only on a retry, or
failed as test.fail expected links its failure screenshot as
<system-out>[[ATTACHMENT|<path>]]</system-out>, which GitLab shows in
the pipeline test report. The path is relative to CI_PROJECT_DIR, else
the project root; a screenshot outside it gets no link.
- One systemOut(lines) renders every <system-out>: the flaky note, the
test.fail expected note, and the attachment, in that order.
- posixRelative(from, to) in report/write.ts replaces the copies in
junit.ts, markdown.ts, and traces.ts.
- failureScreenshot(told) in failure-text.ts, shared with the trace page.
* docs(skill): describe the junit screenshot attachment; test the errored case
- skills/e2e/references/running.md (and its plugin copy) says a junit
case links its failure screenshot for GitLab, as #1507 lists.
- A test covers an errored (<error>) case with a failure screenshot.
---------
Co-authored-by: Tingri Lohar Mohammed Mohsin <Mohsintl@users.noreply.github.com>
A process that fails early and then keeps logging - a file watcher, say -
buried the startup error under its own tail: the message quoted only the last
20 lines appended since the command started. It now quotes the first and last
20 lines with `[startup output omitted]` between them when the log is longer,
both windows still byte-bounded and passed through the same redactor, so the
error that explains the failure survives.
Fixes#1527