Commit Graph
36 Commits
Author SHA1 Message Date
Kacper KapuściakandOskar Kwaśniewski d593863b41 feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init (#1647)
* feat(cli): write a GitHub Actions or GitLab CI/CD pipeline in e2e init

e2e init asks for a CI pipeline in a project with the web engine (--ci github-actions|gitlab|none). It writes one job for the project's package manager and Node version that installs Chromium, runs e2e run, and keeps the report, and names the secret or CI/CD variable for the model key. It never overwrites a pipeline file; beside an existing .gitlab-ci.yml it writes a file to include.

* fix(cli): handle cubic's findings on the init CI step

Exact GitLab stage names, a distinct job name beside an existing e2e job, loopback baseURL in an existing config, no writes through a symlinked .github or .gitlab, --ci none in the skill, gitlab in the telemetry comment, and the GitLab Node image's major version in the docs.

* refactor(cli): keep the init CI pipelines as YAML templates

The GitHub Actions workflow and the GitLab job live in packages/e2e/templates/ci/ as complete pipelines for npm at the repository root, linted as they are. init loads them with the yaml package's Document API and edits a copy for the project's package manager, folders, Node version, secrets, and target, keeping the comments. The pinned actions now live only in the templates.

* fix(cli): point setup-bun at the package.json of a project in a subfolder

* docs(ci): link the GitLab page from init and the GitLab template

* feat(cli): write the iOS and Android CI jobs in e2e init

init writes one job per platform the config's targets run on: web targets in the web job, iOS and Android targets in their own jobs, which build the app, boot a simulator or an emulator, and install the app. Targets come from the new config, or from loading an existing one after the confirmation and the install. Expo and bare React Native apps get their build; any other app gets a step to replace. GitHub's mobile jobs follow e2e-inc/e2e-react-native's workflow; GitLab's run on self-managed runners.

* fix(cli): reinstall an emulator system image that holds no image in the Android CI job

* fix(cli): handle cubic's findings on the mobile CI jobs

Upload the emulator log from the GitLab Android job, count peer and optional engine dependencies, ask nothing for a new config with no engine, key caches on Bun's bun.lockb when that is the lockfile, and run the symlink test with a Windows junction.

* fix(cli): put current Android command-line tools in place before the emulator image

The runner image's cmdline-tools can be older than the system image's package format, and android-emulator-runner uses them to create the AVD, which then fails with 'contains no system images'. The Android job downloads command-line tools 23.0, checks its SHA-256, and installs the image with it.

* fix(cli): import moved core modules from @e2e-dev/core in init

* fix(cli): count a pipeline as set up only when a command runs e2e run

init matched the text 'e2e run' anywhere in a workflow, so a comment or a step name such as 'our old e2e runner' made --ci write nothing. It now reads the commands (run, script, before_script, after_script) from the parsed YAML and matches e2e run as words of their own.

Also: --ci help and the InitOptions JSDoc say tests on every platform, and yaml is pinned exactly like the other dependencies.

* chore(ci): bump setup-node and upload-artifact pins in the init templates, docs, and skill

docs/ci/vercel-previews.mdx on main pins setup-node v7.1.0 and upload-artifact v7.0.2, the pins this repository's own workflows run. The templates, three docs pages, and the skill still had v7.0.0 and v7.0.1, so the pin test failed once main was merged in.

---------

Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-16 17:21:49 +00:00
Kacper Kapuściak f185ca8ba2 docs: retire sidebar page icons (#1582)
* docs: retire sidebar page icons

The docs sidebar no longer shows icons. Drop icon: from every page's frontmatter, delete the grey silhouettes in docs/images/icons/, and make check-docs-logos fail on any frontmatter icon instead of requiring one on vendor pages. Card logos stay required. The authoring-docs skill trades its Page icons section for a short Vendor logos one.

* docs: require a migrate overview card for each migration guide

* docs: match only top-level icon keys in check-docs-logos
2026-11-05 18:23:59 +00:00
Oskar Kwaśniewski 0c0469605a chore: replace e2e.tester.army with e2e.com (#1572) 2026-11-05 09:23:45 +01:00
Oskar Kwaśniewski 1d671f6786 feat(report): link e2e site in markdown report footer (#1499) 2026-10-31 17:51:57 +01:00
Oskar Kwaśniewski d181e2645f fix(github): give an e2e explore run its own pull request comment (#1438) 2026-10-28 17:45:34 -07:00
Oskar Kwaśniewski da13624d31 feat(runner): run only the tests a pull request can affect (#1206) 2026-10-23 12:29:15 -05:00
aa9699a688 fix(github): distinguish API throttling from token permissions (#1038)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-10-16 15:38:08 +02:00
Kacper Kapuściak 897e368e5d docs(integrations): give TestMu AI its logo and fail docs:check on a vendor page without one (#1201)
* docs(integrations): give TestMu AI its logo and fail docs:check on a vendor page without one

The TestMu AI page shipped in #772 with no sidebar icon and a Font Awesome
card on the integrations overview. The authoring-docs rule existed, but
the PR was a package PR and never loaded the skill.

scripts/check-docs-logos.ts now fails pnpm docs:check when a page under
docs/integrations/, docs/ci/, or docs/migrate/ has no logo as its sidebar
icon, a card linking to it has none, or it has no overview card. The
authoring-docs and ship-pr skills state the rule and point to the check.

* docs(scripts): reject a logo path that leaves its directory; scope the overview-card rule in ship-pr
2026-10-16 11:48:29 +00:00
Oskar Kwaśniewski babae32bda feat(cli): add e2e run --max-duration and maxDuration config (#1131)
* feat(cli): add e2e run --max-duration and maxDuration config

* fix(cli): keep e2e explore on its own --timeout, not maxDuration

* fix(cli): skip max-duration timer past setTimeout cap, cover config and explore paths

* refactor(cli): address max-duration review

- explore drops maxDuration from its config; runner no longer special-cases it
- scheduler records RUN_TIMEOUT from the skip reason, naming --max-duration or maxDuration
- stopEarly wakes the loop itself
- reject maxDuration above 2147483647 ms instead of skipping the timer
- trim cli docs, clarify ci wording, sync plugin
2026-10-14 19:09:31 +02:00
Kacper Kapuściak 5d4053e8fa docs: render an Open Graph card for every docs page with @vercel/og (#1062) 2026-10-12 15:10:32 +02:00
Kacper Kapuściak 449fa93670 docs: rewrite and shorten the guide pages with the authoring-docs skill (#1019)
* docs(agent-steps): rewrite with the authoring-docs skill

* docs(agents): rewrite with the authoring-docs skill

* docs(api-testing): rewrite with the authoring-docs skill

* docs(authentication): rewrite with the authoring-docs skill

INVALID_CONFIG also fires for a password shorter than 6 characters.

* docs(browser): rewrite with the authoring-docs skill

* docs(bug-bash): rewrite with the authoring-docs skill

* docs(cache): rewrite with the authoring-docs skill

* docs(ci): rewrite with the authoring-docs skill

Rerun folders are rerun-<n>/, not rerun-1/. Merge exit codes into Gate pull requests.

* docs(ci/bitrise): rewrite with the authoring-docs skill

* docs(ci/codemagic): rewrite with the authoring-docs skill

* docs(ci/eas-workflows): rewrite with the authoring-docs skill

* docs(ci/github-actions): rewrite with the authoring-docs skill

* docs(coding-agents): rewrite with the authoring-docs skill

* docs(core-concepts): rewrite with the authoring-docs skill

* docs(debugging): rewrite with the authoring-docs skill

* docs(decision-models): rewrite with the authoring-docs skill

A point tap costs three decision calls and one text call, not four.

* docs(example-projects): rewrite with the authoring-docs skill

* docs(executors): rewrite with the authoring-docs skill

* docs(explore): rewrite with the authoring-docs skill

* docs(github): rewrite with the authoring-docs skill

* docs(index): rewrite with the authoring-docs skill

* docs(integrations/eas): rewrite with the authoring-docs skill

* docs(integrations/index): rewrite with the authoring-docs skill

* docs(integrations/kernel): rewrite with the authoring-docs skill

* docs(integrations/smol): rewrite with the authoring-docs skill

* docs(migrate/cypress): rewrite with the authoring-docs skill

* docs(migrate/detox): rewrite with the authoring-docs skill

* docs(migrate/maestro): rewrite with the authoring-docs skill

Fold the command map tables into accordions.

* docs(migrate/playwright): rewrite with the authoring-docs skill

List use and testMatch among the Playwright keys e2e rejects. Fold the mapping tables into accordions.

* docs(migrate/selenium): rewrite with the authoring-docs skill

* docs(mobile): rewrite with the authoring-docs skill

Fix the Android installApp troubleshooting entry: agent-device 0.21.22 parses the manifest itself and searches ~/Library/Android/sdk on macOS.

* docs(models): rewrite with the authoring-docs skill

* docs(quickstart): rewrite with the authoring-docs skill

* docs(security): rewrite with the authoring-docs skill

Name smol beside Kernel and EAS Simulators among the integrations that make outbound connections.

* docs(subscriptions): rewrite with the authoring-docs skill

npx e2e login with no provider opens a picker.

* docs(telemetry): rewrite with the authoring-docs skill

Add the source-checkout, unwritable-preferences, and CI opt-out rules.

* docs(tools): rewrite with the authoring-docs skill

Config load fails only when an entry sets both tools and a custom executor.

* docs(visual-testing): rewrite with the authoring-docs skill

* docs(web): rewrite with the authoring-docs skill

Fold the Next.js dev-origins section, merge the basicAuth examples, and fix the reuseExisting row: in CI e2e ignores it and only a server that already answers is APP_ALREADY_RUNNING. Drop the unrelated agents block from the command config example.

* docs(writing-an-engine): rewrite with the authoring-docs skill

Remove the origin-check claim for location; PR #290 removed origin policy.

* docs(models): keep process.loadEnvFile() as a way to load the pulled OIDC token

* docs(mobile): restore the imports in the several-devices and agent-tools configs

* docs(agent-steps): shorten

Prose words 841 -> 764.

* docs(agents): shorten

Prose words 789 -> 675.

* docs(api-testing): shorten

Prose words 270 -> 266.

* docs(authentication): shorten

Prose words 1000 -> 849.

* docs(browser): shorten

Prose words 1934 -> 1697.

* docs(bug-bash): shorten

Prose words 615 -> 585.

* docs(cache): shorten

Prose words 2793 -> 2374.

* docs(ci): shorten

Prose words 1338 -> 1182.

* docs(ci/bitrise): shorten

Prose words 321 -> 306.

* docs(ci/codemagic): shorten

Prose words 259 -> 250.

* docs(ci/eas-workflows): shorten

Prose words 431 -> 394.

* docs(ci/github-actions): shorten

Prose words 544 -> 453.

* docs(coding-agents): shorten

Prose words 841 -> 735.

* docs(core-concepts): shorten

Prose words 686 -> 620.

* docs(debugging): shorten

Prose words 1826 -> 1604.

* docs(decision-models): shorten

Prose words 1003 -> 942.

* docs(example-projects): shorten

Prose words 347 -> 288.

* docs(executors): shorten

Prose words 1297 -> 1114.

* docs(explore): shorten

Prose words 451 -> 413.

* docs(github): shorten

Prose words 723 -> 626.

* docs(index): shorten

Prose words 387 -> 333.

* docs(integrations/eas): shorten

Prose words 1293 -> 1119.

* docs(integrations/index): shorten

Prose words 174 -> 163.

* docs(integrations/kernel): shorten

Prose words 794 -> 620.

* docs(integrations/smol): shorten

Prose words 924 -> 855.

* docs(migrate/cypress): shorten

Prose words 957 -> 890.

* docs(migrate/detox): shorten

Prose words 1073 -> 1021.

* docs(migrate/maestro): shorten

Prose words 2464 -> 2246.

* docs(migrate/playwright): shorten

Prose words 4835 -> 4546.

* docs(migrate/selenium): shorten

Prose words 933 -> 887.

* docs(mobile): shorten

Prose words 2912 -> 2595.

* docs(models): shorten

Prose words 1187 -> 1048.

* docs(quickstart): shorten

Prose words 980 -> 946.

* docs(security): shorten

Prose words 587 -> 559.

* docs(subscriptions): shorten

Prose words 694 -> 651.

* docs(telemetry): shorten

Prose words 2158 -> 1999.

* docs(tools): shorten

Prose words 460 -> 397.

* docs(visual-testing): shorten

Prose words 583 -> 566.

* docs(web): shorten

Prose words 1262 -> 1104.

* docs(writing-an-engine): shorten

Prose words 1731 -> 1522.

* docs(agent-steps): address review findings

* docs(agents): address review findings

* docs(authentication): address review findings

* docs(cache): address review findings

* docs(ci): address review findings

* docs(coding-agents): address review findings

* docs(debugging): address review findings

* docs(example-projects): address review findings

* docs(executors): address review findings

* docs(index): address review findings

* docs(integrations/eas): address review findings

* docs(migrate/maestro): address review findings

* docs(subscriptions): address review findings

* docs(telemetry): address review findings

* docs(tools): address review findings

* docs(web): address review findings

* docs(writing-an-engine): address review findings
2026-10-10 12:55:15 +00:00
Kacper Kapuściak 00c6bba4e7 docs(ci): give each CI service its own sidebar page (#964) 2026-10-07 20:01:56 +02:00
Oskar Kwaśniewski 1b92a6ddfa feat: e2e trace pages replace the Playwright trace (#869) 2026-10-07 11:28:27 +02:00
Kacper Kapuściak 63482bd6fe docs: reorganize the sidebar into platforms and a deep dive (#870) 2026-10-05 20:39:06 +02:00
Kacper Kapuściak a3f0f9fd4d docs: collapse long code blocks outside the reference behind Show more (#868) 2026-10-05 17:07:06 +02:00
Oskar Kwaśniewski 3a9667c046 fix(run): keep --last-failed reruns honest: failure-limit skips, carried debts, kept evidence (#724)
* fix(run): keep --last-failed reruns honest: failure-limit skips, carried debts, kept evidence

--last-failed now reruns tests --max-failures skipped. A test or failed
suite hook another filter leaves out stays owed in the report's new
run.carried until a rerun runs it; the github fold shows carried rows and
errors and stays red while anything is carried. A rerun prunes artifacts/
to what the previous report names and writes its attempts under
artifacts/rerun-<n>/ instead of wiping the tree and reusing attempt paths.

* fix(run): carry a setup's hook debt when no selected test needs it

The report has no row for an unneeded setup, so carryForward now asks the
collection whether a test still exists instead of the report's rows.

* fix(run): reject malformed artifacts in --last-failed reports, tighten rerun docs

Distinct ids in the carried fixture row.

* fix(run): keep carrying tests of a file a narrowed rerun could not collect

* fix(github): a carried interrupted test reads interrupted, not failed, in the folded page
2026-10-03 16:36:53 +02:00
Oskar Kwaśniewski 7ab80bc610 fix(report)!: count interrupted tests apart from failures (#713)
* fix(report)!: count interrupted tests apart from failures

* fix(report): tell a failed test from its last failed attempt, whatever the cut retry left
2026-10-03 09:30:44 +02:00
Oskar Kwaśniewski 4d488f8286 feat!: the app is declared on its target; engines only drive it (#698)
* feat!: the app is declared on its target; engines only drive it

The app under test moves from the engine's options to `targets: [{ engine,
app }]`: `url`, `bundleId`, `appPath`, `identity`, `environment`,
`launchArguments`, `permissions`, `command`, and `readyUrl`. `web()` and
`mobile()` refuse every moved option in one `INVALID_CONFIG` naming the
target block; `defineEngine({ app })` is gone and engines check what their
platform needs in `validateApp`. `requires: ['browser' | 'native-app']` is
derived from the target's app. Replay cache and session keys are byte for
byte what they were, pinned by target-identity.test.ts.

Dependency services are not part of this version: `services` anywhere, and
`web({ services })`, fail saying so. `app.command` on port 0 without a
command, and `reuseExisting` on a free port, are refused. An `e2e mcp`
open cancelled or disconnected mid-start stops what it started, and a hung
shared start is abandoned once no session waits on it.

* fix: review follow-ups: cancelled opens, abandoned starts, null targets, stale docs

* refactor: drop the moved-option messages; old app keys are unknown keys

* test: give the replaced-nodes suite a budget that survives a loaded runner

* test: size the replaced-nodes hook budget to its seven tests
2026-09-30 19:48:00 +02:00
devin-ai-integration[bot]andoskar 256513dc0b docs: sync the package manager choice across every command snippet (#679)
Co-authored-by: oskar <oskar@okwasniewski.com>
2026-09-30 14:40:08 +02:00
Oskar Kwaśniewski bb51ec1fdb docs: trim guides to what a test author needs (#656)
* docs: trim guides to what a test author needs

Cut runner internals, give each repeated topic one owner page, rewrite
the densest paragraphs.

Owners now:
- screenshots after a secret fill: security#screenshots-after-a-secret-fill
  (auth, explore, debugging, agent-steps, tools, executors, engine link it)
- redaction rules and limits: security#redaction, #where-redaction-stops
- what the model sees/does in a step: agent-steps; security keeps the
  trust-boundary and POLICY_DENIED tables
- budgets and timeouts: reference/agent#budgets (agent-steps keeps cost levers)
- assert under a custom executor, executor model conflicts: executors
- exit codes: debugging; telemetry in CI: telemetry; explore credentials:
  explore; cache trust: security; personas: agents + authentication
- shard/last-failed edge cases: reference/cli; engine capabilities and
  lifecycle detail: reference/engine
- locator strictness: reference/screen; no sessions on a device: mobile

Other changes:
- security: redactor internals -> short user-facing summary
- cache: timing internals cut, reason table one line per cell plus a
  "Limits and gaps" subsection, committed-file changes as a list
- executors: tool-loop sample drives ctx.actions (portable, recorded)
- tools: simpler seed sample, mobileTools spread beside own tools
- writing-an-engine: locate in skeleton, minimal capability table, one-line
  lifecycle, own Keyboard heading, tool-pack pointer, GitHub test link
- models: `ai` prerequisite and key source up front, env-var switch moved
- migrate: shared rules to one line + link, framework-specific "migrate
  first", drop "Lives in" column, US spelling
- kernel: minimal setup sample, neutral replay size wording
- telemetry: source-checkout paragraph cut

* docs: address review on the guide trims

- tools: the seed sample reads APP_URL again
- ci: --last-failed reads report.json under output
- cypress, detox: strictness is for actions and single-node reads
- explore: artifacts, not evidence, land in the goal directory
2026-09-29 23:46:03 +02:00
Oskar Kwaśniewski 6b851bb604 docs: fix stale and wrong claims in guides (#653)
* docs: fix stale and wrong claims in guides

- output: --artifacts -> --output in bug-bash guide and skill; sessions and
  cache locations follow output / cache.dir in security; artifact path
  gains the agent segment in github
- ci: upload artifacts on !cancelled() in ci guide and skill
- mcp: several sessions (--max-sessions) in security
- replay cache naming: "Commit the replay cache" heading and anchors
  (old anchor kept, e2e init links the new one),
  trace replay -> replay cache in writing-an-engine and executors
- timeouts: STEP_TIMEOUT / STEP_BUDGET_EXHAUSTED advice names config
  timeout, judgmentTimeout, maxSteps, maxModelCalls (debugging, skill)
- mobile: app.open() in device examples and quickstart, mobileTools pack
  all mutating, Android toolbar anchor, drop agent-device release note
- executors: verdict code rules, missing ctx members and scrollUntil,
  a replay cache section; agent-steps passed+code rule
- migrate: swipe direction is the content's, `!` test exclusions,
  video row status, Kernel for Selenium Grid
- secrets: an unset env string fails config load, a provider's value is
  redacted only from its first fill; three secret sinks
- misc: Models link text, telemetry links outbound connections, integration
  config wording, web-only expect(web) in a cross-platform block,
  capabilities heading in engine reference, swipe direction in screen reference

* docs(skill): separate the credential redaction paragraph

* docs(reference): KERNEL_API_KEY names @e2e-dev/kernel
2026-09-29 23:38:52 +02:00
Oskar Kwaśniewski 67c29fc6b2 chore!: publish engines and reporter under the @e2e-dev npm scope (#602) 2026-09-28 19:05:05 +02:00
Oskar Kwaśniewski dceae62730 fix(report): run page reads top down: source under the title, evidence kinds linked without paths, folded attempt runs, one target named once (#538) 2026-09-25 19:13:45 +02:00
Oskar Kwaśniewski 61603e6c2c fix(github): a --last-failed rerun updates the pull request comment as one run (#537) 2026-09-25 17:09:09 +02:00
Oskar Kwaśniewski cdc41098c7 fix(report): markdown cells cannot mention, autolink, or start a block, and all-dot names are rejected (#448) 2026-09-23 10:53:49 +02:00
Oskar Kwaśniewski b71ecc0f09 fix(report): leave unselected results off the page and JUnit, link sources under the project path (#422) 2026-09-22 18:14:28 +02:00
Oskar Kwaśniewski 2b9361ffea feat(report): the folded test list is one table, a row per file above its tests (#413) 2026-09-22 13:11:39 +02:00
Oskar Kwaśniewski f5e96d0cbe feat!: rename engine packages to @e2edev/web and @e2edev/mobile (#392) 2026-09-21 19:52:32 +02:00
Oskar Kwaśniewski f09b69e90a docs: simplify onboarding and clarify model access (#346) 2026-09-18 11:52:06 +02:00
Oskar Kwaśniewski 4c76360cb6 feat(report): the run page is paragraphs a reader can skim; flaky tests and the file list fold away (#310) 2026-09-14 17:50:53 +02:00
Oskar Kwaśniewski c2c5df7df9 feat(e2e)!: the runner publishes as e2e; @e2edev/e2e is retired (#294) 2026-09-14 08:56:22 +02:00
Oskar Kwaśniewski 8811ba7de9 feat(report): a markdown reporter writes the run as summary.md, with an exploration's findings, and the GitHub comment renders from it (#284) 2026-09-12 15:54:10 +02:00
Oskar Kwaśniewski 3de7471f77 fix(collect): a test declared through a helper reports the helper call in the test file as its source, never a runner or node_modules frame (#277) 2026-09-12 11:12:58 +02:00
Oskar Kwaśniewski 07cae8787c docs(cli): run the CLI as npx e2e; drop --no-install everywhere (#275) 2026-09-12 11:07:20 +02:00
Oskar Kwaśniewski fcf4fe6ff7 docs: split guides and references by platform (#266) 2026-09-11 17:32:13 +02:00
Oskar Kwaśniewski a0a4df01b4 feat(github): the GitHub reporter comments on the pull request from Actions (#246) 2026-09-10 14:47:36 +02:00