Commit Graph
18 Commits
Author SHA1 Message Date
Oskar Kwaśniewski e62486d668 docs(quickstart): list the common config options (#1332) 2026-10-22 12:20:53 -04:00
Michał PierzchałaandClaude Opus 5.5 490a97e7cc fix(mobile): match device selectors with agent-device's own term rules (#1023)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:02:51 +02:00
Kacper Kapuściak 449fa93670 docs: rewrite and shorten the guide pages with the authoring-docs skill (#1019)
* docs(agent-steps): rewrite with the authoring-docs skill

* docs(agents): rewrite with the authoring-docs skill

* docs(api-testing): rewrite with the authoring-docs skill

* docs(authentication): rewrite with the authoring-docs skill

INVALID_CONFIG also fires for a password shorter than 6 characters.

* docs(browser): rewrite with the authoring-docs skill

* docs(bug-bash): rewrite with the authoring-docs skill

* docs(cache): rewrite with the authoring-docs skill

* docs(ci): rewrite with the authoring-docs skill

Rerun folders are rerun-<n>/, not rerun-1/. Merge exit codes into Gate pull requests.

* docs(ci/bitrise): rewrite with the authoring-docs skill

* docs(ci/codemagic): rewrite with the authoring-docs skill

* docs(ci/eas-workflows): rewrite with the authoring-docs skill

* docs(ci/github-actions): rewrite with the authoring-docs skill

* docs(coding-agents): rewrite with the authoring-docs skill

* docs(core-concepts): rewrite with the authoring-docs skill

* docs(debugging): rewrite with the authoring-docs skill

* docs(decision-models): rewrite with the authoring-docs skill

A point tap costs three decision calls and one text call, not four.

* docs(example-projects): rewrite with the authoring-docs skill

* docs(executors): rewrite with the authoring-docs skill

* docs(explore): rewrite with the authoring-docs skill

* docs(github): rewrite with the authoring-docs skill

* docs(index): rewrite with the authoring-docs skill

* docs(integrations/eas): rewrite with the authoring-docs skill

* docs(integrations/index): rewrite with the authoring-docs skill

* docs(integrations/kernel): rewrite with the authoring-docs skill

* docs(integrations/smol): rewrite with the authoring-docs skill

* docs(migrate/cypress): rewrite with the authoring-docs skill

* docs(migrate/detox): rewrite with the authoring-docs skill

* docs(migrate/maestro): rewrite with the authoring-docs skill

Fold the command map tables into accordions.

* docs(migrate/playwright): rewrite with the authoring-docs skill

List use and testMatch among the Playwright keys e2e rejects. Fold the mapping tables into accordions.

* docs(migrate/selenium): rewrite with the authoring-docs skill

* docs(mobile): rewrite with the authoring-docs skill

Fix the Android installApp troubleshooting entry: agent-device 0.21.22 parses the manifest itself and searches ~/Library/Android/sdk on macOS.

* docs(models): rewrite with the authoring-docs skill

* docs(quickstart): rewrite with the authoring-docs skill

* docs(security): rewrite with the authoring-docs skill

Name smol beside Kernel and EAS Simulators among the integrations that make outbound connections.

* docs(subscriptions): rewrite with the authoring-docs skill

npx e2e login with no provider opens a picker.

* docs(telemetry): rewrite with the authoring-docs skill

Add the source-checkout, unwritable-preferences, and CI opt-out rules.

* docs(tools): rewrite with the authoring-docs skill

Config load fails only when an entry sets both tools and a custom executor.

* docs(visual-testing): rewrite with the authoring-docs skill

* docs(web): rewrite with the authoring-docs skill

Fold the Next.js dev-origins section, merge the basicAuth examples, and fix the reuseExisting row: in CI e2e ignores it and only a server that already answers is APP_ALREADY_RUNNING. Drop the unrelated agents block from the command config example.

* docs(writing-an-engine): rewrite with the authoring-docs skill

Remove the origin-check claim for location; PR #290 removed origin policy.

* docs(models): keep process.loadEnvFile() as a way to load the pulled OIDC token

* docs(mobile): restore the imports in the several-devices and agent-tools configs

* docs(agent-steps): shorten

Prose words 841 -> 764.

* docs(agents): shorten

Prose words 789 -> 675.

* docs(api-testing): shorten

Prose words 270 -> 266.

* docs(authentication): shorten

Prose words 1000 -> 849.

* docs(browser): shorten

Prose words 1934 -> 1697.

* docs(bug-bash): shorten

Prose words 615 -> 585.

* docs(cache): shorten

Prose words 2793 -> 2374.

* docs(ci): shorten

Prose words 1338 -> 1182.

* docs(ci/bitrise): shorten

Prose words 321 -> 306.

* docs(ci/codemagic): shorten

Prose words 259 -> 250.

* docs(ci/eas-workflows): shorten

Prose words 431 -> 394.

* docs(ci/github-actions): shorten

Prose words 544 -> 453.

* docs(coding-agents): shorten

Prose words 841 -> 735.

* docs(core-concepts): shorten

Prose words 686 -> 620.

* docs(debugging): shorten

Prose words 1826 -> 1604.

* docs(decision-models): shorten

Prose words 1003 -> 942.

* docs(example-projects): shorten

Prose words 347 -> 288.

* docs(executors): shorten

Prose words 1297 -> 1114.

* docs(explore): shorten

Prose words 451 -> 413.

* docs(github): shorten

Prose words 723 -> 626.

* docs(index): shorten

Prose words 387 -> 333.

* docs(integrations/eas): shorten

Prose words 1293 -> 1119.

* docs(integrations/index): shorten

Prose words 174 -> 163.

* docs(integrations/kernel): shorten

Prose words 794 -> 620.

* docs(integrations/smol): shorten

Prose words 924 -> 855.

* docs(migrate/cypress): shorten

Prose words 957 -> 890.

* docs(migrate/detox): shorten

Prose words 1073 -> 1021.

* docs(migrate/maestro): shorten

Prose words 2464 -> 2246.

* docs(migrate/playwright): shorten

Prose words 4835 -> 4546.

* docs(migrate/selenium): shorten

Prose words 933 -> 887.

* docs(mobile): shorten

Prose words 2912 -> 2595.

* docs(models): shorten

Prose words 1187 -> 1048.

* docs(quickstart): shorten

Prose words 980 -> 946.

* docs(security): shorten

Prose words 587 -> 559.

* docs(subscriptions): shorten

Prose words 694 -> 651.

* docs(telemetry): shorten

Prose words 2158 -> 1999.

* docs(tools): shorten

Prose words 460 -> 397.

* docs(visual-testing): shorten

Prose words 583 -> 566.

* docs(web): shorten

Prose words 1262 -> 1104.

* docs(writing-an-engine): shorten

Prose words 1731 -> 1522.

* docs(agent-steps): address review findings

* docs(agents): address review findings

* docs(authentication): address review findings

* docs(cache): address review findings

* docs(ci): address review findings

* docs(coding-agents): address review findings

* docs(debugging): address review findings

* docs(example-projects): address review findings

* docs(executors): address review findings

* docs(index): address review findings

* docs(integrations/eas): address review findings

* docs(migrate/maestro): address review findings

* docs(subscriptions): address review findings

* docs(telemetry): address review findings

* docs(tools): address review findings

* docs(web): address review findings

* docs(writing-an-engine): address review findings
2026-10-10 12:55:15 +00:00
Oskar Kwaśniewski 3dc4ec9614 feat!: browser fixture, getByRole name arg, toMatchSchema, top-level hooks, separate credential namespace (#704)
* feat!: browser fixture, getByRole name arg, toMatchSchema, top-level hooks, separate credential namespace

API review feedback before launch:

- web fixture renamed browser (Web -> Browser, WebExpectation ->
  BrowserExpectation); requires: ['browser']; destructuring web hints the
  new name. web() factory and other Web* types unchanged.
- getByRole(role, name, options?) beside the object form; MCP locate
  writes the short form.
- expect(value).toMatchSchema(standardSchema): sync, returns the typed
  output; poll resolves to it, soft returns undefined after a kept failure.
- describe, beforeEach, afterEach, beforeAll, afterAll exported from e2e;
  web and mobile export typed beforeEach/afterEach.
- credentials and secrets are separate namespaces: a password handle is
  <name>.password, secrets.get() never returns it. Committed recordings
  migrated offline (old key hash verified, then recomputed).
- runtime test.skip keeps Playwright semantics; docs say it throws.

* docs(web): browser acts on one active tab; cookies, routes, dialogs cover the whole browser

* fix(secrets): judge a fill's purpose by the config, not the handle; no old-name hints

A config-time secrets.get() handle reads its purpose from the run's
registry again, and authorizeSecretFill checks the registered purpose, so a
handle claiming generic-secret cannot put a password into a plain field.
Drop the web fixture migration hint. Changesets name the on-screen marker
change for replay entries.

* docs(skill): browser fixture, getByRole name arg, toMatchSchema, top-level hooks, credential namespace on the rewritten skill

* fix(expect): negated toMatchSchema returns nothing, as its type says

* fix(expect,docs): soft double negation stays soft, null ~standard is INVALID_ARGUMENT, Playwright samples keep Playwright's getByRole

Also documents SoftExpectCall, SoftValueExpectation, and NegatedPollExpectation
on the expect reference, requires: ['browser'] in the skill's options table,
and tightens the poll and secret-purpose tests.

* docs(babysit): triage pattern for scripted renames that rewrite another library's code

* docs: engine options take a secrets entry, web package hook exports, vault example imports

* refactor(secrets)!: credentials hold their password, allSecrets feeds fills; engines export describe and every hook; BrowserRoute, BrowserResponse

From the maintainability review:
- ResolvedConfig.secrets is the config.secrets entries only; a credential
  carries its password; allSecrets is the one map fills, the ledger, and
  redaction read. The per-consumer purpose filters and the .password
  suffix parsing are gone, and e2e mcp builds handles from config.
- The config load and a running test name the credential a missing
  secrets.get() most likely meant.
- @e2e-dev/web and @e2e-dev/mobile export describe and all four hooks
  beside test, so a test file registers from one import.
- WebRoute and WebResponse are BrowserRoute and BrowserResponse.
- getByRole normalizes its arguments once; isTextMatch is the one
  string-or-RegExp check; poll reuses ValueMatcherName.

* revert(web): keep WebRoute and WebResponse; Browser names the fixture, Web the network and option types
2026-09-30 20:58:52 +02:00
Oskar Kwaśniewski 4d488f8286 feat!: the app is declared on its target; engines only drive it (#698)
* feat!: the app is declared on its target; engines only drive it

The app under test moves from the engine's options to `targets: [{ engine,
app }]`: `url`, `bundleId`, `appPath`, `identity`, `environment`,
`launchArguments`, `permissions`, `command`, and `readyUrl`. `web()` and
`mobile()` refuse every moved option in one `INVALID_CONFIG` naming the
target block; `defineEngine({ app })` is gone and engines check what their
platform needs in `validateApp`. `requires: ['browser' | 'native-app']` is
derived from the target's app. Replay cache and session keys are byte for
byte what they were, pinned by target-identity.test.ts.

Dependency services are not part of this version: `services` anywhere, and
`web({ services })`, fail saying so. `app.command` on port 0 without a
command, and `reuseExisting` on a free port, are refused. An `e2e mcp`
open cancelled or disconnected mid-start stops what it started, and a hung
shared start is abandoned once no session waits on it.

* fix: review follow-ups: cancelled opens, abandoned starts, null targets, stale docs

* refactor: drop the moved-option messages; old app keys are unknown keys

* test: give the replaced-nodes suite a budget that survives a loaded runner

* test: size the replaced-nodes hook budget to its seven tests
2026-09-30 19:48:00 +02:00
Oskar Kwaśniewski d228e22ea3 feat(agents)!: plain-object agents entries; createAgent and limits removed (#646)
* feat(agents)!: plain-object agents entries, createAgent and limits removed

Each agents entry is one AgentConfig object: model, judge, system, context,
tools, maxSteps, maxModelCalls, judgmentTimeout, maxObservationBytes,
maxInputTokens, providerOptions, or executor for a custom brain (system and
tools rejected beside it). No inheritance between agents.

- createAgent() kept as a stub throwing INVALID_CONFIG; bare StepExecutor or
  legacy createAgent value as an entry fails naming { executor } / the object
- timeout -> judgmentTimeout, maxTurns removed (entry and
  createToolLoopExecutor), unknown keys get did-you-mean
- top-level limits removed; maxModelTokensPerCall -> per-agent
  maxInputTokens, the rest internal constants; report-1 run.limits keeps
  its shape, filled from max across agents plus the constants
- tools validated at config load; built-in agent built from the entry at
  fixture time; explore and the MCP catalog read the entry's tools
- migrate apps, docs, docs examples, init scaffold, skills, mobile README

* docs: AGENTS.md names the agent entry's model, not createAgent

* refactor(agents)!: delete createAgent instead of a throwing stub

* docs: MODEL_UNAVAILABLE scoped to the built-in agent, fixture-acquisition timing
2026-09-29 19:51:02 +02:00
Oskar Kwaśniewski 67c29fc6b2 chore!: publish engines and reporter under the @e2e-dev npm scope (#602) 2026-09-28 19:05:05 +02:00
Oskar Kwaśniewski 920ac4ebf0 ci(mobile): iOS simulator and Android emulator gates for the mobile benchmark, with a Control Inventory scenario (#499) 2026-09-24 16:08:05 +02:00
Oskar Kwaśniewski 8802b0f0dc chore: pin openai/gpt-6-luna-fast as the example and suite model (#463) 2026-09-23 10:29:52 +02:00
Oskar Kwaśniewski f5e96d0cbe feat!: rename engine packages to @e2edev/web and @e2edev/mobile (#392) 2026-09-21 19:52:32 +02:00
Oskar Kwaśniewski f09b69e90a docs: simplify onboarding and clarify model access (#346) 2026-09-18 11:52:06 +02:00
Oskar Kwaśniewski c2c5df7df9 feat(e2e)!: the runner publishes as e2e; @e2edev/e2e is retired (#294) 2026-09-14 08:56:22 +02:00
Oskar Kwaśniewski ff33218345 docs: e2e is agentic testing first; quickstart for web and mobile; icons everywhere (#282) 2026-09-12 13:52:59 +02:00
Oskar Kwaśniewski fcf4fe6ff7 docs: split guides and references by platform (#266) 2026-09-11 17:32:13 +02:00
Oskar Kwaśniewski 2d0c6931c4 feat(agent)!: the model is an AI SDK instance the config constructs; init asks which gateway (#255) 2026-09-10 17:26:25 +02:00
Oskar Kwaśniewski c72b05e57f feat(config)!: agents by name replace agent, and --agent picks one per run (#244) 2026-09-10 11:59:30 +02:00
Oskar Kwaśniewski 17cde5a128 feat(agent)!: act takes one options bag and resolves with what the step did (#219)
* feat(agent)!: act takes one options bag and resolves with what the step did

* fix(agent): a legacy act call fails loudly; a retry names itself in step.cache

`act(instruction, params)` and `act(instruction, params, options)` from 0.7
arrived as an options bag of foreign keys, or with a third argument the
fixture dropped, and ran with their parameters silently ignored. Both now
fail with `INVALID_ARGUMENT` naming the move to `{ params }`, from a
JavaScript test as from a typed one; `schema` and `vision` keep their
`UNSUPPORTED_CAPABILITY` message. Unit-tested in act-validation.test.ts.

A step in a retry attempt records a trace but never replays one, and its
`step.cache` was absent, indistinguishable from caching being off. It now
reads `missed` with the new reason `retry`, in the report schema too.

The reference page's opening `Agent` interface catches up with the act
signature, the summary is described as the step's (the replay writes it when
the cache finishes the step), and the quickstart example passes its values
under `params`, which is what failed the docs typecheck in CI.
2026-09-09 12:23:55 +02:00
devin-ai-integration[bot]andoskar d8a2cfd3fe docs(quickstart): three commands to green with no key, typechecked snippets (#213)
Co-authored-by: oskar <oskar@okwasniewski.com>
2026-09-09 11:27:48 +02:00