mirror of
https://github.com/tester-army/e2e.git
synced 2026-11-21 03:05:26 +08:00
* fix(web): launch Chromium with its sandbox on by default
The web engine launched Chromium without chromiumSandbox, so Playwright added --no-sandbox to every browser process. The agent can follow the app to another origin, so a renderer exploit on a page it reaches ran unsandboxed as the user running e2e run, with the checkout, .env, and model keys in reach.
Chromium now launches with chromiumSandbox: true unless the config opts out with web({ chromiumSandbox: false }) or the run environment sets E2E_CHROMIUM_SANDBOX=0, for hosts where the sandbox cannot start (root, containers, Ubuntu 23.10+ with the AppArmor user-namespace restriction). Firefox and WebKit never receive the option. The flag resolves from the option, then the run's environment, then on; the process environment never decides a launch, per the EngineInitInfo.env contract. CI opts out explicitly because the ubuntu-2404 runners restrict unprivileged user namespaces.
Docs updated next to the no-origin-allowlist note (docs/reference/web.mdx, docs/security.mdx, SECURITY.md, skills/e2e/references/setup.md) with a .changeset entry. The smol VM's --no-sandbox is unchanged: the VM is the boundary there.
Verified: pnpm check green; web unit 354/354; web integration 379 passed with 3 known load flakes that pass in isolation; testbed CLI 6/6 fail with "Chromium sandboxing failed!" by default and 6/6 pass with the opt-out on this host.
Fixes #1613
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(web): qualify chromiumSandbox coverage as local-launch-only
Cubic review on the sandbox-default PR found three doc overstatements: SECURITY.md presented the sandbox as universal while connect() and browser providers bypass launch, the skill reference omitted that E2E_CHROMIUM_SANDBOX=0 only applies when the option is omitted, and the reference page implied any container cannot run the sandbox. All three are corrected in the SECURITY note, the reference page, the option JSDoc, and both skill copies.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* fix(web): friendly Chromium sandbox failure and unified sandbox resolution
A sandboxed launch on a host whose sandbox cannot start failed with Playwright's raw error and its full call log, which only hints at the option. acquireBrowser now catches that one failure on a local launch and throws an ENGINE_FAILURE naming both opt-outs, E2E_CHROMIUM_SANDBOX=0 and web({ chromiumSandbox: false }), with no call log and no cause chain to leak it back.
The sandbox flag resolves in one place now: resolveChromiumSandbox(configured, env) in browser-connection.ts replaces the split readChromiumSandboxFlag plus surface resolver, and the surface keeps a single field that init resolves. BrowserConnection.acquire takes an options object, so a relaunch retries with the same options. The option JSDoc shrinks to two lines pointing at the security docs.
Tests follow the restructure: the cdp-attach edits are gone (the flag never applied to attach), screenshot-caret and stalled-frame run sandboxed by default, resolved from the ambient environment so the CI opt-out still reaches a direct launch, and the workflow comments shrink to one line. New tests pin the trim and lowercase handling, the opt-out surviving a failed relaunch, and the friendly error text.
Verified: web unit 358/358 with the opt-out; a real launch on this host fails with the friendly error by default and passes with E2E_CHROMIUM_SANDBOX=0.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(web): minor bump for the sandbox default, CI opt-out, one home for the why
The sandbox default breaks Docker and Linux CI hosts, and the changeset said nothing about it: it is now a minor bump, opened as a maintainer-called release, telling users to set E2E_CHROMIUM_SANDBOX=0.
The CI docs name the opt-out where CI users find it: the GitHub Actions web workflow and the visual-testing screenshots workflow both set E2E_CHROMIUM_SANDBOX=0 on ubuntu-latest, matching this repo's own CI, with a note pointing at the security docs. The other CI pages run mobile targets only, so there is nothing to add there.
The why now lives in docs/security.mdx alone: the reference page, SECURITY.md, the option JSDoc, and the skill reference link to it instead of repeating it.
Verified: changeset-bumps, the docs check chain (anchors, MDX, examples, package managers, logos), and the skill tests all pass.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(web): qualify the Chromium sandbox opt-out as ubuntu-latest only
The visual-testing guide said Linux runners generally block the user
namespaces Chromium needs. The limitation is specific to the
ubuntu-latest runners used here, so name them and tell readers to leave
the sandbox on where it starts.
* fix(web): address review follow-ups on the Chromium sandbox default
Move the sandbox launch error out of the surface and into
BrowserConnection.launchLocal, where the browser name and the resolved
sandbox flag are already known, and drop the untested verb guard. The
error now names the environment opt-out only when the option did not set
the value, since an explicit option wins over the variable. The surface
keeps the option as a readonly field and resolves it into a private
field defaulting to true, the same pattern as headed, so startAttempt
without init no longer needs the fallback. The skill GitHub Actions
recipe runs ubuntu-latest with E2E_CHROMIUM_SANDBOX 0, the web reference
points at the new ENGINE_FAILURE instead of Playwright error text, and
the resolve helper loses its dead branches.
Web unit suite: 362/363 pass; the one failure is artifact-path, which
fails identically on the base head because a real Chromium cannot start
sandboxed in this container.
* fix(web): address re-review follow-ups on the Chromium sandbox default
Resolve the sandbox into one { enabled, fromOption } object and pass it through acquire into launchLocal, which now takes the options object instead of five positional args, so the launch and its error cannot disagree about which opt-out set the value. Add the waiter-joins-a-pending-relaunch case, where the re-acquire runs once and a flip cannot cancel out, and assert retryable is false on the friendly error. Document E2E_CHROMIUM_SANDBOX in the environment reference, point the dead [Security](/security) links at https://e2e.com/docs/security, and fix the changeset wording, since the error no longer always names both opt-outs. The acquire JSDoc now states the actual guarantee: a launch takes the value its own call passes.
Web unit suite: 364/364 pass with E2E_CHROMIUM_SANDBOX=0 (artifact-path fails without it, identically on the base head, because a real Chromium cannot start sandboxed in this container). Web typecheck clean, oxlint clean. The new waiter test fails against a flip mutant injected on the re-acquire and passes without it.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(web): mark E2E_CHROMIUM_SANDBOX as a value in the env reference
The entry was marked flag, but on this page 0 and false mean unset for flags, while 0 opts out of sandboxing here. Mark it value so readers do not apply the wrong flag semantics.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(ci): restore the timings-history section lost in the rebase
The rebase conflict resolution in docs/ci/github-actions.mdx kept only the tail of the new Cache the timings history section, dropping its heading, intro, and example, which broke the docs anchor check (docs/ci.mdx links to #cache-the-timings-history). Restored in full; the sandbox opt-out paragraph follows it unchanged.
Verified: the docs check chain passes locally (anchors, examples, package managers, logos, MDX).
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(security): soften sandbox containment wording
The security page stated a renderer exploit stays contained by the
sandbox, which reads as a guarantee. Say the sandbox helps contain it
and note it does not prevent sandbox escapes or Chromium
vulnerabilities, matching the review feedback on the Chromium sandbox
default.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(security): wrap the sandbox containment sentence at 80 columns
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* fix(web): import EngineError from @e2e-dev/core/engine
browser-connection imported EngineError from e2e/engine, so loading the
web engine without the e2e package failed with ERR_MODULE_NOT_FOUND.
That broke the core-only install smoke check, which loads each engine
main entry without e2e installed. Import from @e2e-dev/core/engine
instead, like every other file in the web engine.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* docs(security): re-wrap sandbox paragraph and soften changeset wording
Re-wrap the sandbox containment paragraph in docs/security.mdx at 80
columns so vulnerabilities; no longer sits on a line of its own, and
soften the changeset to match: the sandbox helps contain a renderer
exploit but does not prevent sandbox escapes. The changeset keeps only
the user-facing part: what changed and the two opt-outs.
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
* fix(web): default the Chromium sandbox by platform, drop the env opt-out
On for macOS and Windows, opt-in on Linux where root, most containers
and Ubuntu 23.10+ cannot start it. The option alone decides, so CI,
the test env forwarding and the docs recipes need no opt-out.
* test(web): drop the relaunch cases the waiter test already covers
---------
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
107 lines
4.8 KiB
Plaintext
107 lines
4.8 KiB
Plaintext
---
|
|
title: Security
|
|
description: What the model can see and do, how secrets stay out of it, and what you still need to watch.
|
|
"og:image": "https://e2e.com/docs/images/og/security.png?v=6c43d95c56"
|
|
"twitter:image": "https://e2e.com/docs/images/og/security.png?v=6c43d95c56"
|
|
---
|
|
|
|
e2e trusts the code you write and treats everything the app shows as
|
|
untrusted. App content can't grant the model tools, credentials, or a
|
|
larger budget.
|
|
|
|
## How secrets stay out of the model
|
|
|
|
A `Secret` is a handle with no plaintext accessor. The model sees only its
|
|
name and purpose. When the model asks to fill one, the runner checks the
|
|
field and types the value itself.
|
|
|
|
Every known secret value becomes `<secret:name>` in the model's text input, reports,
|
|
console output, and `.txt`, `.csv`, `.json`, `.html`, and `.htm` downloads.
|
|
|
|
## What can still leak
|
|
|
|
Redaction matches strings, and screenshots mask secure fields only:
|
|
|
|
- **Screenshots, the model's included.** A generic secret filled into a
|
|
plain input, or a value the app shows outside a secure field, is in the
|
|
image and reaches your model provider. Use test credentials, not
|
|
production secrets.
|
|
- **Video and downloads.** Video masks nothing, and a download other than a
|
|
redacted text file is kept as served. If the test typed a password, check
|
|
them before you share them.
|
|
- **`command.log`.** It holds your app's stdout and stderr verbatim. Keep
|
|
secrets out of your app's output.
|
|
- **Transformed values.** The last four characters of a key, or its base64,
|
|
pass through. Assert the outcome, not the secret.
|
|
- **Values not filled as a secret.** Only a configured secret's value is
|
|
redacted. A value a sign-in sets in a cookie is not.
|
|
- **Request credentials.** `headers` go on every request that starts at the
|
|
app's origin or one you list, even one a third-party page starts, and stay
|
|
on its redirects to any origin. `basicAuth` answers a `401` from those
|
|
origins, and from any origin on WebKit when one of those is on a default
|
|
port or has an IPv6 host.
|
|
- **Committed recordings.** Cache entries store typed values verbatim, except
|
|
secret fills. They aren't signed and replay with your permissions, so
|
|
review them as you would test code.
|
|
|
|
[Where redaction stops](/reference/security#where-redaction-stops) lists
|
|
every case.
|
|
|
|
## What the model sees and does
|
|
|
|
The model sees your instruction, the screen as an accessibility tree, the
|
|
redacted URL, and records of earlier steps in the test. It gets a masked
|
|
screenshot when it asks for one or a step sets `vision`.
|
|
|
|
The model acts only through tools. The runner checks each call against a
|
|
closed schema before it runs, and never evaluates model text as code. A
|
|
built-in tool rejects a key it does not declare unless it has no fields, in
|
|
which case it drops the keys and runs; a project tool follows its own
|
|
`defineTool` schema. A call that breaks a rule, such as a navigation to
|
|
`file:`, fails with `POLICY_DENIED`. There is no origin allowlist: the agent can
|
|
follow the app to another site and use the step's secrets there.
|
|
|
|
On macOS and Windows, Chromium launches with its OS sandbox on, which helps
|
|
contain a renderer exploit on a page the agent reaches. On Linux it is opt-in:
|
|
see [`chromiumSandbox`](/reference/web#chromiumsandbox).
|
|
|
|
## What runs with your permissions
|
|
|
|
Your tests, config, engines, tools, reporters, stores, secret providers, and
|
|
app commands run with your operating-system permissions, unsandboxed. Run
|
|
untrusted pull request code in an external sandbox with no secrets or write
|
|
tokens.
|
|
|
|
## What leaves your computer
|
|
|
|
A run connects to your model provider, the app under test, and, unless you
|
|
opt out, [telemetry](/telemetry) at `eu.i.posthog.com`. There is no crash
|
|
reporting. A CLI command on a terminal checks `registry.npmjs.org` for a
|
|
newer release at most once a day, `e2e feedback` checks before each
|
|
report unless `--allow-outdated`, and `e2e upgrade` reads the versions of
|
|
the e2e packages the project lists. Integrations such as Kernel, Expo Simulators, Limrun, and smol add
|
|
their own destinations, listed with the rest under
|
|
[Outbound connections](/reference/security#outbound-connections).
|
|
|
|
## Report a vulnerability
|
|
|
|
Report privately through
|
|
[GitHub security advisories](https://github.com/e2e-inc/e2e/security/advisories/new)
|
|
or by mail to security@e2e.com. The
|
|
[security policy](https://github.com/e2e-inc/e2e/blob/main/SECURITY.md)
|
|
states response times and supported versions.
|
|
|
|
## Next
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Security rules" icon="lock" href="/reference/security">
|
|
Every rule behind this page, as the code enforces it.
|
|
</Card>
|
|
<Card title="Authentication" icon="key" href="/authentication">
|
|
Pass secrets to tests and agent steps, and save sessions.
|
|
</Card>
|
|
<Card title="Continuous integration" icon="rotate" href="/ci">
|
|
Give model keys only to builds you trust.
|
|
</Card>
|
|
</CardGroup>
|