Files
Maximo GukandClaude Opus 5 6ae7c4f1cb Build "deploy" releases in parallel (#309)
* Build releases in parallel

Every worker bundle in a release is independent: each `wrangler deploy
--dry-run` reads only its own package and writes only its own outdir. They
ran one at a time anyway, and the frontend build sat in front of all of
them. Measured on a 10-core machine: 62.4s -> 12.3s.

The frontend now overlaps with the bundles too. Only a worker that serves
static assets reads workshop-frontend/dist, so it alone awaits the frontend
build -- read off `config.assets` rather than naming the router, so a second
asset-serving worker would be sequenced as well.

Nothing about the output depends on the interleaving: results come back in
`findDeployablePackages` order, and the blob writes and the manifest are
built from them in that order. Verified byte-identical against a sequential
baseline (modules, assets and manifest, modulo the `createdAt` timestamp),
at the default concurrency and at `--concurrency 1`.

`mapConcurrent` is a shared, tested helper rather than a `Promise.all` at
the call site for its failure behaviour: `Promise.all` settles on the first
rejection and leaves the rest bundling under a process that is already
exiting, so CI would report whichever package lost the race. Here every task
is awaited and all the failures are named together.

`run` captures output instead of inheriting the terminal -- a dozen wrangler
logs interleaved line by line would be unreadable -- and prints one labelled
block per command with its duration, in completion order. A failed command's
diagnostics are in its block, so the summary carries only labels and exit
codes rather than repeating them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Cancel in-flight builds on the first failure, and flush diagnostics before exiting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 15:36:48 -05:00

65 lines
3.2 KiB
TypeScript

// Runs an async operation over a list with a bounded number of them in flight.
//
// Written for the release build, where each package's `wrangler deploy --dry-run` is independent
// and saturates about one core. Two properties matter there, and they are why this is a shared,
// tested helper rather than a `Promise.all` at the call site:
//
// - Results keep the input's order, so a manifest built from them is byte-stable no matter which
// bundle happened to finish first.
// - A rejection does not abandon the tasks still running. `Promise.all` settles on the first
// failure and leaves the rest writing to disk under a process that is already exiting; here
// every task is awaited, and all the failures are reported together -- in CI that names every
// broken package rather than whichever one lost the race.
//
// The optional signal is the fail-fast complement: when the caller learns elsewhere that the whole
// run is doomed (the frontend build failed), aborting stops new tasks from being claimed while the
// wait-for-all guarantee on the ones already running still holds.
/**
* Applies `task` to every item, with at most `limit` running at a time, and returns the results in
* the input's order.
*
* Every task runs even if an earlier one rejects. If any did, this rejects once they have all
* settled: with that error if there was exactly one, or an `AggregateError` over them otherwise.
*
* An aborted `signal` stops further items from starting; tasks already running are still awaited.
* If the signal aborted at any point and no task failed, this rejects with `signal.reason`; task
* failures take precedence over the abort reason.
*/
export async function mapConcurrent<T, R>(
items: readonly T[],
limit: number,
task: (item: T, index: number) => Promise<R>,
signal?: AbortSignal,
): Promise<R[]> {
if (!Number.isInteger(limit) || limit < 1) {
throw new RangeError(`concurrency limit must be a positive integer, got ${limit}`);
}
const results: R[] = Array.from({ length: items.length });
const failures: unknown[] = [];
// Each runner claims the next index and works until the list is exhausted. The claim is a bare
// `next++` because JS runs it to completion before any other runner resumes.
let next = 0;
const runners = Array.from({ length: Math.min(limit, items.length) }, async () => {
for (let i = next++; i < items.length; i = next++) {
if (signal?.aborted) break;
try {
results[i] = await task(items[i], i);
} catch (error) {
failures.push(error);
}
}
});
await Promise.all(runners);
if (failures.length === 1) throw failures[0];
if (failures.length > 1) {
throw new AggregateError(failures, `${failures.length} of ${items.length} tasks failed`);
}
// Aborted is aborted, whether or not it cost us an item. Keying this off "we skipped something"
// instead reports success for the case where the signal lands once every item is already claimed
// and each task then finishes on its own merits -- no holes in `results`, nothing rejected, and a
// caller that asked to stop handed a full set of results as if it had not.
if (signal?.aborted) throw signal.reason;
return results;
}