Files
e2e/docs/security.mdx
VarshithandOskar Kwaśniewski a06b931b8d fix(web): launch Chromium with its sandbox on where it can start (#1621)
* fix(web): launch Chromium with its sandbox on by default

The web engine launched Chromium without chromiumSandbox, so Playwright added --no-sandbox to every browser process. The agent can follow the app to another origin, so a renderer exploit on a page it reaches ran unsandboxed as the user running e2e run, with the checkout, .env, and model keys in reach.

Chromium now launches with chromiumSandbox: true unless the config opts out with web({ chromiumSandbox: false }) or the run environment sets E2E_CHROMIUM_SANDBOX=0, for hosts where the sandbox cannot start (root, containers, Ubuntu 23.10+ with the AppArmor user-namespace restriction). Firefox and WebKit never receive the option. The flag resolves from the option, then the run's environment, then on; the process environment never decides a launch, per the EngineInitInfo.env contract. CI opts out explicitly because the ubuntu-2404 runners restrict unprivileged user namespaces.

Docs updated next to the no-origin-allowlist note (docs/reference/web.mdx, docs/security.mdx, SECURITY.md, skills/e2e/references/setup.md) with a .changeset entry. The smol VM's --no-sandbox is unchanged: the VM is the boundary there.

Verified: pnpm check green; web unit 354/354; web integration 379 passed with 3 known load flakes that pass in isolation; testbed CLI 6/6 fail with "Chromium sandboxing failed!" by default and 6/6 pass with the opt-out on this host.

Fixes #1613

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(web): qualify chromiumSandbox coverage as local-launch-only

Cubic review on the sandbox-default PR found three doc overstatements: SECURITY.md presented the sandbox as universal while connect() and browser providers bypass launch, the skill reference omitted that E2E_CHROMIUM_SANDBOX=0 only applies when the option is omitted, and the reference page implied any container cannot run the sandbox. All three are corrected in the SECURITY note, the reference page, the option JSDoc, and both skill copies.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* fix(web): friendly Chromium sandbox failure and unified sandbox resolution

A sandboxed launch on a host whose sandbox cannot start failed with Playwright's raw error and its full call log, which only hints at the option. acquireBrowser now catches that one failure on a local launch and throws an ENGINE_FAILURE naming both opt-outs, E2E_CHROMIUM_SANDBOX=0 and web({ chromiumSandbox: false }), with no call log and no cause chain to leak it back.

The sandbox flag resolves in one place now: resolveChromiumSandbox(configured, env) in browser-connection.ts replaces the split readChromiumSandboxFlag plus surface resolver, and the surface keeps a single field that init resolves. BrowserConnection.acquire takes an options object, so a relaunch retries with the same options. The option JSDoc shrinks to two lines pointing at the security docs.

Tests follow the restructure: the cdp-attach edits are gone (the flag never applied to attach), screenshot-caret and stalled-frame run sandboxed by default, resolved from the ambient environment so the CI opt-out still reaches a direct launch, and the workflow comments shrink to one line. New tests pin the trim and lowercase handling, the opt-out surviving a failed relaunch, and the friendly error text.

Verified: web unit 358/358 with the opt-out; a real launch on this host fails with the friendly error by default and passes with E2E_CHROMIUM_SANDBOX=0.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(web): minor bump for the sandbox default, CI opt-out, one home for the why

The sandbox default breaks Docker and Linux CI hosts, and the changeset said nothing about it: it is now a minor bump, opened as a maintainer-called release, telling users to set E2E_CHROMIUM_SANDBOX=0.

The CI docs name the opt-out where CI users find it: the GitHub Actions web workflow and the visual-testing screenshots workflow both set E2E_CHROMIUM_SANDBOX=0 on ubuntu-latest, matching this repo's own CI, with a note pointing at the security docs. The other CI pages run mobile targets only, so there is nothing to add there.

The why now lives in docs/security.mdx alone: the reference page, SECURITY.md, the option JSDoc, and the skill reference link to it instead of repeating it.

Verified: changeset-bumps, the docs check chain (anchors, MDX, examples, package managers, logos), and the skill tests all pass.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(web): qualify the Chromium sandbox opt-out as ubuntu-latest only

The visual-testing guide said Linux runners generally block the user
namespaces Chromium needs. The limitation is specific to the
ubuntu-latest runners used here, so name them and tell readers to leave
the sandbox on where it starts.

* fix(web): address review follow-ups on the Chromium sandbox default

Move the sandbox launch error out of the surface and into
BrowserConnection.launchLocal, where the browser name and the resolved
sandbox flag are already known, and drop the untested verb guard. The
error now names the environment opt-out only when the option did not set
the value, since an explicit option wins over the variable. The surface
keeps the option as a readonly field and resolves it into a private
field defaulting to true, the same pattern as headed, so startAttempt
without init no longer needs the fallback. The skill GitHub Actions
recipe runs ubuntu-latest with E2E_CHROMIUM_SANDBOX 0, the web reference
points at the new ENGINE_FAILURE instead of Playwright error text, and
the resolve helper loses its dead branches.

Web unit suite: 362/363 pass; the one failure is artifact-path, which
fails identically on the base head because a real Chromium cannot start
sandboxed in this container.

* fix(web): address re-review follow-ups on the Chromium sandbox default

Resolve the sandbox into one { enabled, fromOption } object and pass it through acquire into launchLocal, which now takes the options object instead of five positional args, so the launch and its error cannot disagree about which opt-out set the value. Add the waiter-joins-a-pending-relaunch case, where the re-acquire runs once and a flip cannot cancel out, and assert retryable is false on the friendly error. Document E2E_CHROMIUM_SANDBOX in the environment reference, point the dead [Security](/security) links at https://e2e.com/docs/security, and fix the changeset wording, since the error no longer always names both opt-outs. The acquire JSDoc now states the actual guarantee: a launch takes the value its own call passes.

Web unit suite: 364/364 pass with E2E_CHROMIUM_SANDBOX=0 (artifact-path fails without it, identically on the base head, because a real Chromium cannot start sandboxed in this container). Web typecheck clean, oxlint clean. The new waiter test fails against a flip mutant injected on the re-acquire and passes without it.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(web): mark E2E_CHROMIUM_SANDBOX as a value in the env reference

The entry was marked flag, but on this page 0 and false mean unset for flags, while 0 opts out of sandboxing here. Mark it value so readers do not apply the wrong flag semantics.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(ci): restore the timings-history section lost in the rebase

The rebase conflict resolution in docs/ci/github-actions.mdx kept only the tail of the new Cache the timings history section, dropping its heading, intro, and example, which broke the docs anchor check (docs/ci.mdx links to #cache-the-timings-history). Restored in full; the sandbox opt-out paragraph follows it unchanged.

Verified: the docs check chain passes locally (anchors, examples, package managers, logos, MDX).

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(security): soften sandbox containment wording

The security page stated a renderer exploit stays contained by the
sandbox, which reads as a guarantee. Say the sandbox helps contain it
and note it does not prevent sandbox escapes or Chromium
vulnerabilities, matching the review feedback on the Chromium sandbox
default.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(security): wrap the sandbox containment sentence at 80 columns

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* fix(web): import EngineError from @e2e-dev/core/engine

browser-connection imported EngineError from e2e/engine, so loading the
web engine without the e2e package failed with ERR_MODULE_NOT_FOUND.
That broke the core-only install smoke check, which loads each engine
main entry without e2e installed. Import from @e2e-dev/core/engine
instead, like every other file in the web engine.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* docs(security): re-wrap sandbox paragraph and soften changeset wording

Re-wrap the sandbox containment paragraph in docs/security.mdx at 80
columns so vulnerabilities; no longer sits on a line of its own, and
soften the changeset to match: the sandbox helps contain a renderer
exploit but does not prevent sandbox escapes. The changeset keeps only
the user-facing part: what changed and the two opt-outs.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

* fix(web): default the Chromium sandbox by platform, drop the env opt-out

On for macOS and Windows, opt-in on Linux where root, most containers
and Ubuntu 23.10+ cannot start it. The option alone decides, so CI,
the test env forwarding and the docs recipes need no opt-out.

* test(web): drop the relaunch cases the waiter test already covers

---------

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Co-authored-by: Oskar Kwaśniewski <oskar@okwasniewski.com>
2026-11-20 11:25:39 +00:00

107 lines
4.8 KiB
Plaintext

---
title: Security
description: What the model can see and do, how secrets stay out of it, and what you still need to watch.
"og:image": "https://e2e.com/docs/images/og/security.png?v=6c43d95c56"
"twitter:image": "https://e2e.com/docs/images/og/security.png?v=6c43d95c56"
---
e2e trusts the code you write and treats everything the app shows as
untrusted. App content can't grant the model tools, credentials, or a
larger budget.
## How secrets stay out of the model
A `Secret` is a handle with no plaintext accessor. The model sees only its
name and purpose. When the model asks to fill one, the runner checks the
field and types the value itself.
Every known secret value becomes `<secret:name>` in the model's text input, reports,
console output, and `.txt`, `.csv`, `.json`, `.html`, and `.htm` downloads.
## What can still leak
Redaction matches strings, and screenshots mask secure fields only:
- **Screenshots, the model's included.** A generic secret filled into a
plain input, or a value the app shows outside a secure field, is in the
image and reaches your model provider. Use test credentials, not
production secrets.
- **Video and downloads.** Video masks nothing, and a download other than a
redacted text file is kept as served. If the test typed a password, check
them before you share them.
- **`command.log`.** It holds your app's stdout and stderr verbatim. Keep
secrets out of your app's output.
- **Transformed values.** The last four characters of a key, or its base64,
pass through. Assert the outcome, not the secret.
- **Values not filled as a secret.** Only a configured secret's value is
redacted. A value a sign-in sets in a cookie is not.
- **Request credentials.** `headers` go on every request that starts at the
app's origin or one you list, even one a third-party page starts, and stay
on its redirects to any origin. `basicAuth` answers a `401` from those
origins, and from any origin on WebKit when one of those is on a default
port or has an IPv6 host.
- **Committed recordings.** Cache entries store typed values verbatim, except
secret fills. They aren't signed and replay with your permissions, so
review them as you would test code.
[Where redaction stops](/reference/security#where-redaction-stops) lists
every case.
## What the model sees and does
The model sees your instruction, the screen as an accessibility tree, the
redacted URL, and records of earlier steps in the test. It gets a masked
screenshot when it asks for one or a step sets `vision`.
The model acts only through tools. The runner checks each call against a
closed schema before it runs, and never evaluates model text as code. A
built-in tool rejects a key it does not declare unless it has no fields, in
which case it drops the keys and runs; a project tool follows its own
`defineTool` schema. A call that breaks a rule, such as a navigation to
`file:`, fails with `POLICY_DENIED`. There is no origin allowlist: the agent can
follow the app to another site and use the step's secrets there.
On macOS and Windows, Chromium launches with its OS sandbox on, which helps
contain a renderer exploit on a page the agent reaches. On Linux it is opt-in:
see [`chromiumSandbox`](/reference/web#chromiumsandbox).
## What runs with your permissions
Your tests, config, engines, tools, reporters, stores, secret providers, and
app commands run with your operating-system permissions, unsandboxed. Run
untrusted pull request code in an external sandbox with no secrets or write
tokens.
## What leaves your computer
A run connects to your model provider, the app under test, and, unless you
opt out, [telemetry](/telemetry) at `eu.i.posthog.com`. There is no crash
reporting. A CLI command on a terminal checks `registry.npmjs.org` for a
newer release at most once a day, `e2e feedback` checks before each
report unless `--allow-outdated`, and `e2e upgrade` reads the versions of
the e2e packages the project lists. Integrations such as Kernel, Expo Simulators, Limrun, and smol add
their own destinations, listed with the rest under
[Outbound connections](/reference/security#outbound-connections).
## Report a vulnerability
Report privately through
[GitHub security advisories](https://github.com/e2e-inc/e2e/security/advisories/new)
or by mail to security@e2e.com. The
[security policy](https://github.com/e2e-inc/e2e/blob/main/SECURITY.md)
states response times and supported versions.
## Next
<CardGroup cols={2}>
<Card title="Security rules" icon="lock" href="/reference/security">
Every rule behind this page, as the code enforces it.
</Card>
<Card title="Authentication" icon="key" href="/authentication">
Pass secrets to tests and agent steps, and save sessions.
</Card>
<Card title="Continuous integration" icon="rotate" href="/ci">
Give model keys only to builds you trust.
</Card>
</CardGroup>