The --layouts wrapper always forwarded to clang. A musl target links with
`zig cc`, and SCRIPTC_LINKER replaces the driver together with its `cc`
argument, so every layout build failed. The wrapper now forwards to
`zig cc` when SCRIPTC_TARGET names a musl triple.
The lanes run with CI=1, so the Node island tests require a libnode pack and
fail rather than skip without one. A lane Sandbox prepared from a snapshot
that carries a pack had no way to be told where it is, because the lane
environment pinned only CI, SCRIPTC_SAN and the shard variables. Forward an
explicitly configured SCRIPTC_LIBNODE — a directory inside the Sandbox, never
the host — through sandboxLaneEnv, which every remote lane command already
uses, and document it in the runner's help.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Taking the head entry of a Readable's buffer moved every remaining entry
down one slot, so draining a queue of n buffered chunks cost O(n²). The
buffer now keeps the index of its first entry: a take advances it, a push
at the tail moves the entries down only when at least half the slots are
free at the front (and grows otherwise), and unshift takes a freed front
slot when there is one. The drop, trace and boxing loops walk the entries
from that index.
The stream-buffered-drain runtime workload buffers 200,000 objects and
100,000 Buffers before draining them; corpus 8533 covers reads across
entries, unshift into a freed slot and onto a part-read head, setEncoding
after takes, and object-mode takes, unshifts and pushes.
Each build stages the runtime artifacts it links from the runtime pack and
hashes each one with SHA-256 unless the build cache records that it was
verified with the file identity it still has. A build with an empty cache
therefore hashed every artifact again: on Linux x64, 40 objects and 4.3 MB
for a Benchmarks Game program, about 25 ms of a 300-1100 ms build.
Staging now also reads and writes runtime-pack.verified.json beside the
pack. It lists each verified artifact with its digest and file identity
(device, inode, size, mtime and ctime), as the cache's record does:
- An artifact is trusted only when one of the two records lists it with
the manifest's digest and size and the identity its file has during the
read. Any change to the file means hashing it again.
- A missing, unreadable or malformed record, or a malformed entry, is
treated as absent.
- The record is written to a new file and renamed over the old one, so a
reader sees a whole record. A write merges the record as it is then.
- When the pack directory cannot be written, the cache's record is the only
one, as before.
The runtime pack packages do not publish the file. The native CLI build
removes the record its probe builds leave in the bundled pack, and
verify-native-cli.mjs rejects a distribution that has one.
The LLVM helper's emit reports the sums of the profile's counters on the
functions it annotated and on the functions whose control flow changed since
the profile was collected (annotated_counts and mismatched_counts), and the
partition cache keeps them with each partition's match. The annotated ones
are listed as the profile annotation leaves them, before block counts
recover any. Builds of the native compiler print the share of the profile's
counts that changed functions hold.
An executable with at least 4 MiB of program code is linked with 2 MiB
segment alignment and published with one write, so the page cache holds
its text in 2 MiB folios and the runtime advises huge pages for it. A run
then maps every 2 MiB text window it touches whole. Most runs of the
compiler are short: `scriptc --version` peaked at 76 MB and an unchanged
rebuild at 78 MB, against 20 MB and 26 MB with 4 KiB alignment, and cold
and edited builds of tsc-ts took the same time either way.
compile() takes an internal `hugeText: false`, which links with 4 KiB
alignment whatever the code size and is part of the executable cache key.
scripts/build-native-cli.mts passes it. Executables the compiler builds
keep huge text.
refresh-cli-layout.mjs kept the raw profile of every cold training build
until the end of the run, about 23 MB per build and 8.7 GB for the 399
builds. When the disk filled, the profile runtime wrote partial files, the
compiler still exited with 0, and reading a partial file threw a RangeError
past the end of its counters.
Each cold profile is now read and deleted when its build finishes, so the
work directory stays under 200 MB. The script throws when a training run
reports `LLVM Profile Error`, naming the build and the write error, and
when a profile is shorter than its header describes.
ICU builds with symbol renaming on, so its C API symbols carry the _78
version suffix, its data symbol is icudt78_dat, and its C++ symbols are
Itanium manglings that name namespace icu_78. icuSymbols() classifies
exactly those, and the gate's dynamic-tier check now fails a candidate
that links any of them without the pattern being allowed: a program that
does not lower to Intl links no ICU object at all.
The engine-free Intl functions (scr_intl_num_format_en_us in scr_lib.c,
scr_intl_segmenter_new in scr_grapheme.c) share the scr_intl_ prefix and
are deliberately not part of the test; any scr_intl.c function that does
link brings an ICU symbol with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A profile annotates a function only while its control flow hashes as it did
when the profile was collected, so compiler and source changes leave the
changed functions without counts. A --profile-use build now reads a
block-count file beside the profile (app.blocks beside app.profdata). Right
after the annotation before inlining, the helper aligns each uncounted
function's blocks with its recorded blocks by label kind and successor
count. Paired blocks take their recorded counts and their edges the recorded
edge counts, by target or, between targets whose labels changed, by
position. LLVM's flow inference fills in the rest, avoiding the edges the
function's own llvm.expect weights make unlikely. The helper then sets the
entry count, branch weights and hot or cold attribute as the profile
annotation does.
SCRIPTC_RECORD_PROFILE_BLOCKS names the file a --profile-use build writes
for the functions its profile annotates; such a build reuses no cached
artifacts. The block file keys every cache, the build report counts the
functions it annotated, and the staleness warning counts only the functions
left without counts. A single-module build with a context-sensitive profile
no longer counts annotated, mismatched and missing functions a second time
after inlining. scripts/refresh-cli-profile.mjs records the file in the
second phase and build-native-cli.mts reads it beside the profile.
scr_unwind_raise walks the stack with plans cached per return address.
The first raise through a return address finds the function's FDE
through the .eh_frame_hdr of the object that contains it, runs the CFI
up to the call, and finds the call site's landing pad in the LSDA; the
plan records the frame's CFA rule, where its callee-saved registers are
saved, and the landing pad. Later raises look the plans up, restore the
registers, and jump to the first landing pad. Plans are inserted under
a lock and read without one.
Landing pads continue the unwind with scr_unwind_resume instead of
`resume`, so the walk goes on from their frame. A frame whose CFI,
pointer encodings or personality the plans do not express, or a return
address without an FDE, leaves the unwind to libgcc's forced unwind
from the frame the walk started at, as before. Other Linux targets keep
libgcc's unwinder; their scr_unwind_resume calls _Unwind_Resume.
bench:runtime gains layered-errors, a workload whose errors are thrown
below small functions that hold references and caught per entry in a
loop, and the corpus gains unwind-landing-pad-chains, which pins landing
pads that continue the unwind, including in frames LLVM inlines into a
caller whose handler takes the exception.
The LLVM helper's runtime-unit now optimizes a unit itself (--opt-level),
with IR instrumentation (--profile-generate) or a profile (--profile-use),
through the same pipeline as program code. The Linux x64 pack's speed
units are compiled to unoptimized bitcode and optimized there with
packages/runtime-pack-common/profiles/linux-x64-gnu.profdata.gz, so one
LLVM writes and reads every profile. Release and library flavors are
unchanged. The pack manifest records how the profile matched the units.
Code the runtime profile finds cold is optimized for speed, not size: a
runtime function the training programs rarely call can be hot in another
program. scr_message.c is optimized without the profile.
scripts/refresh-runtime-profile.mjs trains the profile on the runtime
benchmark workloads and, optionally, a tsc-ts checkout, and keeps only the
records of functions the instrumented runtime units define
(merge-profiles --keep-functions). Tests check that the profile still
matches the runtime and names only runtime functions.
A program is nondeterministic only when every artifact that differs also
differs between two baseline builds; a deterministic difference in another
artifact still fails the gate. The runtime-only check now reads the symbols
of changed vendor archives as well as runtime objects.
`scripts/refresh-cli-profile.mjs --context-sensitive` runs the training twice. The second run uses a compiler that the first profile optimizes and that is instrumented again after inlining, and it merges those counts into the first profile. Builds with the result also order each function's blocks by the counts and split cold blocks off (context-sensitive profiles). `--base-profile` runs only the second phase. `scripts/build-native-cli.mts` builds such a compiler when `SCRIPTC_NATIVE_PROFILE_GENERATE` and `SCRIPTC_NATIVE_PROFILE_USE` are both set. The release workflow's cli-profile job collects the profile this way, with a 120-minute timeout.
Main moved the per-program report lines into zero-cost-compare.mjs and
added sharding and a baseline cache. The runtime-only and
nondeterministic verdicts now print through programLines, count as
compared in merged shard reports, and the runtime pack diff is skipped
when only recording a baseline cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under ASan, self-hosting-llvm-emitter, three-library, npm-static and npm
each take 20 to 40 minutes as one Vitest file, so a sharded run cannot
finish before them.
- self-hosting-llvm-emitter splits into its emitter-cases-and-programs
and fixed-point halves; a shard builds the seed once for the halves it
owns. three-library skips the optimization its shard does not own.
Both put their tests in a describe block, which skipUnownedTest
partitions.
- CI runs both in every shard's case-sharded side queue
(scripts/ci-test-plan.mjs); pnpm test:sandbox case-shards them with npm
and npm-static.
- npm.test.ts keeps a skipped placeholder in a shard that owns no
program, so an empty shard does not fail the file.
The sanitized native-driver bootstrap ran its instrumented compiler with
the default 8 MiB stack. Its frontend phase overflows it (SC3004) on main
as well, outside CI's ulimit for the emitter phase. The test now raises
the soft stack limit to the hard one for every sanitized compiler process
it starts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A unit's record holds its IR in the compact encoding, the entries it added
to the helper, lifted-function and other appendable lowerer tables, and the
reachability, generic demand, statement and flag events it made. A unit has
no record when it grew a table no record carries, changed a field, replaced
an entry it did not add, or emitted a diagnostic. Every lowerer collection
and scalar field is classified, and a test fails on one that is not.
SCRIPTC_LOWER_JOURNAL=serial records every unit and replaces its results
with the ones its record carries. scripts/recorded-lowering.mjs
(pnpm test:recorded-lowering) compares corpus programs' LLVM text with and
without the mode. SCRIPTC_LOWER_JOURNAL_DUMP=<dir> also turns the mode on
and appends each unit's record to <dir>/records.jsonl. Without either,
lowering is unchanged.
Linux links use ld.lld when clang finds it and the system linker
otherwise, and the two produce different executables. A baseline record
made with one must not settle candidates linked with the other.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI's pinned LLVM ships ld.lld, and the compiler links Linux executables
with it whenever clang finds it. The Sandbox image and the managed-image
bootstrap installed clang-22 without lld-22, so Sandbox builds linked
with GNU ld: static hello came out at 99 KB instead of 65 KB, and
runtime-tree-shaking failed only in Sandboxes. Install lld-22 in both
and check that clang finds it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Chris Tate