Files
Oskar Kwaśniewski d7a218af07 feat(config)!: config shape, part 1: recording modes, output root, test-glob exclusions, renames, secrets in engine options (#645)
* feat(config)!: remove specVersion from the config

The runner version is the format version. A config that still sets
specVersion fails INVALID_CONFIG telling the author to remove it. The
report's run.specVersion and the replay cache key keep their hard-coded
copies.

* feat(collect): "!" exclusions in the tests globs

An entry starting with "!" removes files an including glob matched,
in any order. Discovery never reads a directory an exclusion takes
whole, unless an including glob could reach a dot name beneath it.
A list of only exclusions is INVALID_CONFIG.

* refactor!: CacheStore and web({ screencast }) renames

TraceCacheStore is CacheStore, and the docs, skill, and CLI call the
cache the replay cache. web({ video: { size, quality } }) is
web({ screencast }) with WebScreencastOptions; web({ video }) fails
INVALID_CONFIG naming screencast.

* feat(secrets): secrets.get() in engine options, resolved per attempt

At config evaluation, before any run exists, secrets.get(name) returns
a reference by name. An engine declares the handles its options hold
in defineEngine({ secrets }); the config load fails INVALID_CONFIG for
a name no secret or credential has. The engine reads the value through
EngineAttemptContext.resolveSecret, which runs a provider fresh and
registers the value in the session and process ledgers before it
returns, without tainting the viewport. e2e/engine exports isSecret.

web({ basicAuth: { password } }) takes a Secret and declares it. Docs
show process.loadEnvFile('.env') in the config.

* feat(run)!: trace and video share recording modes; artifacts holds only store

One RecordingMode ('off' | 'on' | 'retain-on-failure' | 'on-first-retry' |
'on-all-retries') for trace and video, set on the config, a target, the CLI
(--trace [mode], --video [mode]) and a test, one precedence rule: config,
target, CLI, test. Trace defaults to on locally, on-first-retry in CI, with
a plan-time notice when a retry mode meets retries: 0. Serial members cannot
set trace.

Capability rule: a mode from the config root or a flag applies to targets
whose engine can record it, one notice naming the rest; a mode set on a
target or a test is required and fails with UNSUPPORTED_ARTIFACT.

artifacts.kinds (both forms) and artifacts.trace are removed and fail
INVALID_CONFIG naming the trace mode they meant. Failure screenshot and
screen text are always captured when the engine can. MCP sessions keep the
target's trace mode and trace only under 'on'.

* feat(run)!: output directory for every run result; ArtifactStore.putLink

output (default .e2e) and --output <dir> on run and explore hold report.json,
junit.xml, summary.md, ai-trace.json, artifacts/, sessions/ and the e2e mcp
session files; --last-failed reads <output>/report.json. The reportSibling
derivation from the artifacts dir is gone. --artifacts stays as a hidden
flag that fails with the --output it maps to.

A run clears <output>/artifacts at start. cache.dir stays independent. An
output that is the project root or outside it, is or sits in cache.dir,
would hold cache.dir under artifacts/, sessions/ or videos/, or holds a
tests glob root fails INVALID_CONFIG with the reason.

ArtifactStore gains optional putLink(link) for provider-hosted video links;
its ref lands beside the url. A passed retain-on-failure attempt's link is
never handed over, and a failing putLink never fails the run.

bench-ab and the mobile workflow move to --output.

* test(secrets): a hold's registry is probed by an unknown name, now that get() defers without one

* test(web): the download trigger's measured wait tolerates a timer firing early

* fix(secrets): rewrite the trace of an attempt whose engine holds a secret

A Playwright trace records the context options, basic-auth password in
plaintext, but a trace was rewritten only after a fill. resolveSecret now
marks the session so its trace goes through the redactor, screencast
frames kept since the value never reached the screen. The fake engine
writes a real trace archive, and the basic-auth fixture escapes its echo
and refuses a malformed header.

* fix(web): race the basic-auth secret with the attempt's signal

A provider still resolving after the launch timeout set the session after
the runner ended the attempt, so the retry failed with an attempt already
running. Resolution now runs first, raced with the signal.

* fix(mcp): a session notes a configured trace its engine cannot record

* test: a shadow-free fake trace archive and an ungarbled exclusion title

* docs: output paths, replay cache wording, and review corrections

--last-failed, --ai-trace, and the json reporter name <output> paths;
the skill says output never moves cache/ or the app's log; the
INVALID_GLOB row states the check the code makes; serial members list
only; the engine vocabulary names Secret and isSecret; the basic-auth
example uses a provider; the last trace cache wording is replay cache.

* chore(changeset): leave released notes alone; pending notes match the renames

Released CHANGELOG entries keep TraceCacheStore, the name they shipped
with; cache-store-rename.md announces the rename. The pending video and
provider recording changesets name on-all-retries and web({ screencast }).

* chore(bench): refuse a forwarded --output the harness would not clean up

* fix(secrets): a rewritten trace drops its screencast frames, engine-held secrets included
2026-09-29 19:51:02 +02:00
..