- Keep the project overview and development setup in `README.md`.
- Put system design in `ARCHITECTURE.md` and detailed contributor and
agent guidance in `AGENTS.md`; symlink `CLAUDE.md` to that source.
- Remove duplicate rules, memory files and unused Jekyll tooling.
Nightly cron only fires on the default branch, so the suite would have
gone unrun until after it merged, and the first real evidence it works
would have arrived on main. Moving it to pull_request means it is exercised
where it can still be fixed.
It stays a separate job rather than joining the deterministic suite's:
Brew-UI still skips these tests by name, so a live failure reads as a
live failure and never as a broken fixture-backed test. Concurrency now
keys on the pull request, since queueing every PR behind one run only made
sense while the group modelled a single shared machine.
The workflow file loses its `_nightly` name, and the docs that promised a
nightly schedule now say what actually happens.
Two gaps in a suite that retries. A failure's message says an element was
missing; it cannot say whether a dialog was covering the app, which is how
the last one actually failed. And a retry that turns red into green leaves
a run that reads as clean.
BrewUITestCase now attaches a screenshot of the whole screen to every
issue it records, so the picture lands in the result bundle CI uploads.
It goes in as PNG data rather than as the XCUIScreenshot, which cannot
cross out of the main actor into this nonisolated override. Both the
attachment and the plan keep screenshots always: the interesting one
belongs to a failed attempt of a test that ultimately passed, and
deleteOnSuccess prunes exactly that.
The plan now retries a failing test twice rather than once, and
annotate-flaky-tests reads the result bundle afterwards and names any test
that needed more than one run to pass — a ::warning:: annotation on GitHub,
plain text locally, never a failure by itself. It also tells the workflow
to keep the bundle for a green-but-retried run, so the evidence the
annotation points at still exists.
The live suite's comments restated what the code says and argued its own
design at length. What survives is the part a reader cannot recover: that
a login shell keeps exported variables, that a real inventory virtualizes
so an installed row may never render, that cleanup runs after the app is
terminated to stay clear of Homebrew's lock, and why the runner must be
signed. The reasoning behind the rest is in the commits and in
E2E/README.md, which is where it belongs.
The suite could not run at all: macOS put up "BrewUITests-Runner is
damaged and can't be opened", and xcodebuild sat there until it gave up
with "The test runner hung before establishing connection." The dialog is
the cause and the hang is only the consequence — a runner Gatekeeper
refuses to start never connects.
CODE_SIGNING_ALLOWED=NO, copied here from scripts/test-ui, does not
produce an unsigned runner. BrewUITests-Runner.app is a copy of Xcode's
XCTRunner.app template, which arrives signed by Apple; the build inserts
our .xctest into its PlugIns directory and then, with signing disallowed,
skips the CodeSign step. What launches is a bundle carrying Apple's
signature over contents that no longer match it, which codesign reports
as "code has no resources but signature indicates they must be present".
Allowing signing with identity "-" ad-hoc signs the runner as
sh.brew.BrewUITests.xctrunner with a valid seal. Ad-hoc needs no identity,
team or provisioning profile, so this is not a local-only accommodation —
it works on a bare CI runner too.
scripts/test-ui and the ui-test CI job still carry the flag and the same
latent defect; TROUBLESHOOTING.md now says so, so the next person to meet
this dialog reads it rather than re-derives it.
A canary nobody runs is a canary that reports nothing, so the suite gets a
schedule rather than only a plan: nightly on an ephemeral GitHub runner,
which ships Homebrew and has egress to formulae.brew.sh and ghcr.io, and
by hand before cutting a release — the moment you most want to know brew
has not shifted under the app. The release workflow is deliberately not
wired to it; that stays a decision someone makes.
Concurrency is keyed on the workflow rather than the ref, because two runs
would fight over one machine's brew state.
scripts/test-e2e mirrors scripts/test-ui, including its refusal to pass
-derivedDataPath, and leads with what it does to the machine it runs on:
this one installs and uninstalls a formula, and the script is the last
place that fact can be read before it happens.
Mike McQuaid