The bootstrap test builds its own seed when CI shares none, and that build
still applied the committed function layout, so its LLVM listed functions in
a different order from the native compiler's own emission. Both seed builds
now share one environment, which leaves the layout out.
The native bootstrap compares the seed's LLVM, which the Node-hosted
compiler emits, with the LLVM the native compiler emits for itself. The
seed's build applied the committed function layout and the native
compiler has none, so the two listed the same functions in different
orders. The layout only orders functions, so the seed is built without it.
With --cpu-dispatch, an x86-64 executable carries two copies of each
program function: the baseline code and a copy compiled for x86-64-v3.
main reads CPUID and XGETBV once. Calls between the program's functions
go to the caller's copy. A function that is also named outside the
copies (a function value, a call from main, a runtime callback or a
coroutine body) keeps a dispatcher under its own name, which tests the
level and tail-calls one copy. Functions without such a use get no
dispatcher, so their copies have the callers the original had and LLVM
inlines them the same way.
The executable still runs on any x86-64 CPU. SCRIPTC_CPU_LEVEL caps the
level at run time. The option is off by default, applies to executables
without a --target-cpu level, and is part of the executable cache key.
bench:runtime takes --baseline-cpu-dispatch and --candidate-cpu-dispatch,
and the LLVM differential suite runs each program at levels 0 and 3 with
SCRIPTC_CPU_DISPATCH=1.
A build with an empty cache type-checked scriptc's own declaration files,
the TypeScript libraries and scriptc's declarations, before reusing the
answer in later builds. That check was 18 ms of a 105 ms first build of
hello world.
- build-native-cli.mts has the built compiler build a module without
imports and a module with a type error, each in an empty cache, and
records the content keys of the declaration file sets the checker
reported no diagnostics in: the checker release, the compiler options,
and each file's name and sha256. They ship in
lib/declarations/lib-diagnostics.json, named by the toolchain manifest's
lib_diagnostics.
- A build whose cache has no answer computes the files' content key and,
when the shipped answers hold it, uses no diagnostics for those files
and stores that answer in the cache. The shipped answers apply only
while the checker is the distribution's own tsc-ts and its package
version is the release they were recorded with; a changed file has
another key.
- A compiler the build host cannot run ships without them.
The Linux helpers linked LLVM 22.1.8's release archive. The helper's code
generation is LLVM's code, so building that LLVM from its release sources
with ThinLTO and with a profile of the helper's own work makes code
generation 5-9% faster than the helper as released, with identical objects.
- native/llvm-codegen/llvm.profdata.gz is the profile, collected by
scripts/refresh-llvm-profile.mjs: it builds LLVM with IR instrumentation
and an instrumented helper, and runs that helper on tsc-ts, five Effect
fixtures, the runtime benchmark workloads and every 50th corpus program
(scripts/llvm-helper-inputs.mjs). It follows LLVM's sources, so it needs
a refresh when LLVM is bumped.
- scripts/build-llvm-dist.mjs builds LLVM with that profile and ThinLTO,
and packs it with a checksum. The llvm-dist workflow, run by hand, builds
it for x64 and arm64, requires a helper linked against it to give the
same objects as one linked against the release archive
(scripts/check-llvm-dist.mjs), prints both helpers' times and the
profile's mismatch count, and attaches the archive to the LLVM_DIST_TAG
release of this repository.
- The release workflow's Linux jobs, and the wasm32-wasi job, which also
uploads the x64 GNU helper, download that archive and build the helper
with SCRIPTC_LLVM_THINLTO=1: clang, lld and ThinLTO, and the profile for
the helper's own sources. Without the archive, or if that build fails,
the helper links the release archive as before.
- The four Linux helper packages share their build in
native/llvm-codegen/linux-helper.mjs. Without SCRIPTC_LLVM_THINLTO it
builds the same executable as before.
The release workflow publishes each package and moves on once npm accepts
the upload. A package whose processing on npm stalls was visible only on
npmjs.com.
A new npm-available job runs after "Publish to npm". It checks each package
of the release with `npm view <name>@<version> version --prefer-online`
every 2 minutes (scripts/wait-npm-available.mjs) until every package
resolves, and reports each package's time from upload to available in the
log and the job summary. When a package still does not resolve after 2
hours, the job fails and names each missing package and version. It only
reads from the registry.
The publish job lists every package of the release and records the time
npm accepted each upload, keeps those times with its upload receipts, and
passes both to the new job. Create GitHub Release now waits for the new
job, so the release is created and marked complete only once every
package resolves; a later release run checks again and skips uploads npm
already accepted.
Parallel changes kept taking the same next corpus number (6992 twice, the
Node island programs renumbered several times). The harness never needed
the numbers: it discovers every entry by glob, and 818 programs already
have descriptive names. AGENTS.md now makes a name, without a number, the
convention for new programs and keeps the numbered ones as they are.
The two scripts that sampled programs by number skipped named programs.
They now key a named program by a stable hash of its name, so they sample
it at the same rate; the numbered programs they pick are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A warm rebuild ran about 3.5 MB of the native compiler's code, but the code
was spread over its 69 MB text, so 64 KB fault-around mapped about 20 MB of
it. The emitter can now take a function layout: a list of functions in
tiers. It writes the listed definitions, generated helpers included, first
and by tier, each in a `layout-<tier>` partition group. The LLVM helper
compiles each partition's definitions in text order, so the listed ones
come first in their partitions, and a helper that places functions by
group keeps a tier's definitions in the same partitions. The rest keep
their order and their source-file groups.
- compile() takes a `functionLayout` file (`<tier> <symbol>` lines,
gzip-compressed when the name ends in .gz). Keys leave out module
numbers.
- build-native-cli.mts uses the committed packages/compiler/native/cli.layout.gz,
or the file SCRIPTC_NATIVE_LAYOUT names; an empty value and instrumented
builds use none.
- scripts/refresh-cli-layout.mjs builds an instrumented compiler, builds
the runtime workloads and a corpus slice cold and then unchanged, and
reads the functions each run executed from the raw profiles. Tier 1 is
`--version` and rebuilds that hit both caches; tier 2 is the other builds.
- The release workflow's cli-profile job also collects a layout on the
release commit, and the CLI builds use it. Names of closures and
generated helpers change with nearby edits, so a committed layout
matches fewer functions as main moves on.
- Forked lowering and emission workers reset the profile counters after
fork and write their profile before _exit, so instrumented builds also
count the work done in workers. This applies to refresh-cli-profile.mjs
too.
Executables the compiler produces are unchanged.
Without @types/node, every program loaded scriptc-node-fallback.d.ts whole:
the console and process globals and 47 Node modules, about 8,100 lines,
parsed, bound and checked in every cold build whether or not the program
imported them.
- The fallback file keeps the globals and the modules they name (console,
readline, readline/promises, perf_hooks, tty). Every other module, under
both spellings, has its own file in ambient/node-fallback/, which
references the files of the modules its declarations name.
- A module's file joins the program when one of the program's files names
the module in a string literal: imports, requires, import types and JSDoc
types all name modules that way. The entry is read before the first
program; the program-root discovery passes add modules that other files
name. Files are read as tsgo reads them, and the TypeScript libraries and
scriptc's own declarations are skipped.
- The checks that recognized the fallback file by its path recognize every
fallback file.
Its replace callbacks receive checked-dynamic values where the lib types
their parameters any: captures a match can leave unset, and results the
checker types any.
On the san 1/20 shard the sanitized nodes build shares the runner with the
corpus's other two workers and the side queue. It took 944 s and 860 s on
#2194's runs and passed 1200 s on main's 7fa88083, failing the test timeout.
The Linux san corpus now skips that one case
(SCRIPTC_THREE_SANITIZED_NODES_JOB), and a three_nodes_san job runs it alone
with the corpus's LeakSanitizer settings. The required test check waits for
the new job. Every other three.js case, the plain nodes build and the macOS
suites are unchanged.
The workload reads its file through readline over fs.createReadStream.
The stream and readline runtime units define dynamic-tier symbols on
their own: a program that only reads a file with createReadStream links
the same 32, as log-summary and inventory-report do.
readline split redirected input byte by byte, twice per chunk (once to
find whether the chunk ends a line, once to split it). Before splitting
it copied each chunk out of its Buffer and into a scratch buffer, decoded
it into a new string, and copied the unfinished line and the whole chunk
into a joined buffer. Now:
- one scan per chunk: memchr for a CR once, then LF to LF with memchr
when the chunk has none, and the LF, CR and CRLF split otherwise;
- the unfinished line joins only the first line of the chunk;
- an ASCII chunk with no partial UTF-8 sequence held from the previous
one is split where it is, and a chunk that is not shared memory is
read from its own storage. Lines are still copied into their strings
before the first is delivered.
Buffer.from(string, 'latin1' | 'ascii') encoded character by character;
an ASCII prefix, all of an ASCII string, is now copied with memcpy.
Line events, their order and the CRLF delay are unchanged. Corpus 8344
covers lines joined across chunks, a line longer than a chunk, empty
lines, mixed line endings and UTF-8 split across chunks. The new
line-reader workload reads a 67 MB log through readline over a file
stream.
The release workflow's new cli-profile job runs
scripts/refresh-cli-profile.mjs on Linux x64 and uploads the profile.
Every platform's native CLI build uses it through
SCRIPTC_NATIVE_PROFILE_USE, and the GitHub release attaches it as
scriptc-<version>-cli.profdata.gz. When the job fails, the CLI builds
use the committed profile.
build-native-cli.mts accepts a gzip-compressed profile in
SCRIPTC_NATIVE_PROFILE_USE when the name ends in .gz.
refresh-cli-profile.mjs creates the directory of --output before it
starts.
The native compiler is built with a committed optimization profile. A
function whose control flow changed since the profile was collected is
optimized as without one, so the profile loses its gain as compiler code
changes, with nothing to show it.
With --profile-use, the helper now counts what the profile annotation
reports while it applies the profile: functions that took the profile's
counts, functions whose control flow no longer matches (hash mismatch),
and functions the profile has no record of. It sums the counts over
partitions, keeps each partition's counts beside its cached summary, and
prints them as one line of JSON after a successful emit. The compiler
reads that line (an older helper prints none) and returns it as the
executable artifact's profileMatch.
build-native-cli prints the share of profiled functions that match, and
warns below 90% that the profile should be refreshed. On main aa1c26845, a
profile refreshed on that commit matches 92.7% of the compiler's profiled
functions, and the one committed about 200 commits earlier 87.2%; that one
kept about half of the fresh profile's gain on a cold tsc-ts build.
Emitted functions report an exception by returning with the thread's
pending flag set, and every caller tests the flag after each call that
may throw. On Linux executables (glibc and musl), functions reached only
by direct calls now propagate exceptions by unwinding the native stack
instead:
- unwindAbiFunctions selects the may-throw functions that are not async
or generators, constructors, virtually called or overridden methods,
closure targets, the entry, library exports, module initializers or
callees of coroutine bodies.
- A native unwind costs about 2 µs a raise and 0.1 µs a frame, so functions that
exceptions are likely to cross keep the flag: those with a `throw` of
their own, those up to three calls above one, and recursive functions,
counting calls through methods, closures and callbacks.
- Their propagating exits call scr_unwind_raise (scr_unwind.c), a forced
unwind through _Unwind_ForcedUnwind with the exception still in the
active cell.
- Direct calls to them are `invoke`s without a pending check. The
landing pad runs the frame's usual unwind path: it releases the frame's
values and then enters the enclosing try handler or resumes the unwind.
- Remaining plain calls to them (wrappers, adapters, runtime-facing entry
points) become invokes whose landing pad continues on the pending flag,
or resumes when the wrapper is itself invoked. Every other call is
marked `nounwind`, so inlining a raising function at an invoke keeps its
calls plain.
- When emitted text breaks these rules (a raising function's address is
taken, or an invoked function returns with the flag pending), the
module is emitted again with those functions on the pending flag.
Emitted code does not declare scr_unwind_raise `noreturn`, so sanitized
builds do not run the sanitizer's no-return hook, which needs stack
bounds that fiber stacks do not report. Instead, scr_unwind_raise clears
the shadow of the frames it unwinds.
Executables link a static unwinder: libgcc_eh with the host toolchain and
the glibc runtime packs, and Zig's libunwind with Zig and the musl packs.
macOS, Windows, Android, WASI, libraries and debug builds keep the
pending flag everywhere and emit the same code as before.
The throw-chains runtime workload measures exceptions caught 2, 5 and 20
recursive calls above the throw, through direct calls and a callback,
and a parser that backtracks by throwing.
Reapplies #2017, which #2038 reverted after effect's observability-otlp
fixture crashed in plain builds. The crash was the candidate-buffer
corruption that the previous commit fixes: the cached property sites
change when the collector runs, and with enough environment variables a
pass buffered a string box that was later pooled while still buffered.
The Linux test shards run their suites in a transient systemd scope with
MemoryMax=13G and no swap, below the runner's 16 GB. A process tree that
grows past the cap loses its largest process to the kernel, and the job
log names the failing test, where the runner itself ran out of memory
and left no log. Hosts without systemd run the suites directly.
The side-queue balancer also takes a table of memory-heavy files, with
their measured peak. They run whole in the cost-balanced task, and each
goes to the shard with the least memory-heavy work so far, after the
time-balanced files. decl-boundary.test.ts, which peaks at about 1.8 GB
alone and had no time measurement, so vitest's file sharding put it in
shard 11, is the first entry. It now runs in shard 1 (plain) and 6 (san).
Release builds compile for the x86-64 baseline, so LLVM vectorizes with
SSE2 only. `--target-cpu=<baseline|x86-64-v2|x86-64-v3|x86-64-v4|native>`
compiles the program's code for a higher level: every emitted function
carries the level as its `target-cpu`, so LLVM can use AVX2, BMI2, FMA or
AVX-512. The executable then runs only on CPUs of that level.
- `native` resolves to the highest level the build machine supports, read
from /proc/cpuinfo on Linux and `sysctl machdep.cpu` on macOS.
- The level is part of the executable cache key; `baseline` and builds
without the option produce the same output and keys as before.
- The runtime pack keeps its baseline code.
- Targets other than x86-64 refuse a level (SC3002). The CLI's cached
bootstrap route leaves builds with the option to the full compiler.
- `bench:runtime` takes `--baseline-target-cpu` and `--candidate-target-cpu`.
A program that declares a class reads and writes untyped properties
through helpers that first select the receiver's property bag. The bag
helpers return jsvals, so the helpers read the bag through js.toDyn;
keyed helpers test each class's key table between the key's coercion and
the read; and a program with subclasses normalizes the receiver
(value = %dyn.class.normalize(value)) at the start of each helper. The
inline-cache recognizer accepted none of these forms, so every property
site in such a program called its helper. It now looks through dyn/jsval
conversions of the receiver, accepts capsule branches before a keyed
helper's final read, and accepts a rebinding of the receiver to a bag of
itself. Plain objects and arrays then read and write through the site's
cache, as they do in programs without classes.
bench/dynamic-properties (new) exercises these sites.
tsc-ts 0.0.8's API server serves on the named pipe the Node host
connects over on Windows (tsc-ts --api --pipe), and the native compiler
already starts it over its standard streams there, so tsc-ts is the
default checker on every platform. SCRIPTC_CHECKER=tsgo still selects
tsgo.
The compiler depends on tsc-ts 0.0.8, which the release-age exclusions
admit with its platform packages and @scriptc/threads 0.4.2.
The Windows CLI smoke test runs with both checkers, and
verify-native-cli.mjs --run checks that a distribution's tsc-ts runs on
its platform (tsc-ts.exe on Windows).
getCipherInfo(nameOrNid[, options]) runs in a new runtime unit,
scr_cipher_info.c, over a table that scripts/gen-cipher-info.mjs generates
from Node 24.22's getCipherInfo. The table covers every cipher OpenSSL
names, including the ones without a native implementation. Lookups match
names case-insensitively and accept OpenSSL's nids. The unit applies
cipher.js's argument checks and GetCipherInfo's keyLength and ivLength
rules, and returns the info object in Node's key order. CipherInfo maps
to a checked-dynamic value, so TypeScript and JavaScript code receive
Node's object. Reading the nid Node leaves out for the SIV and SM4 GCM,
CCM and XTS ciphers, where the declared type gives number, throws a
TypeError.
createCipheriv and createDecipheriv support aes-128/192/256-ccm (and the
id-aesN-ccm names) over OpenSSL's ccm128 steps. They follow Node's tag and
IV lengths, the plaintextLength requirement, the message-length caps and
the one-update rule, and report a failed authentication from final().
They also support the RFC 3394 and RFC 5649 key wrap ciphers (aes-N-wrap,
aesN-wrap, id-aesN-wrap, the -pad forms and the -inv ciphers) over
wrap128.c's routines, with Node's lengths, integrity failures and
one-update rule.
The new names follow the existing ones in the cipher table, so the lookup
cost of the existing ciphers is unchanged.
A method call on a value of unknown type (untyped JavaScript, `any`)
picked its builtin by comparing the method name with strcmp against each
candidate, in long ladders per receiver kind: a string method could test
dozens of names before it matched.
- The name is now looked up once per call in a generated hash table
(scr_dyn_methods.h, from scripts/gen-dyn-methods.mjs), and each ladder
compares the resulting id. The table verifies the name's bytes, so a
name is matched by its content, as before.
- The worker event methods compare the first character before strcmp.
- bench/dynamic-methods calls string, array, Map, Set, number and own
methods from untyped JavaScript.
A typed teardown freed each block onto its free list as it went, and the
free lists, which are LIFO, handed a torn-down structure's blocks to the
next allocations in the reverse order. A structure that a loop rebuilds
the same way each time was therefore laid out mirrored every other
generation, and every walk over it, its teardown included, ran through
memory backwards, which the hardware prefetchers handle three to four times
worse. A program that rebuilds a tree of 2^21 - 1 object literals 12
times, summing each generation before dropping it, took 26 to 28 ns per
node to build, 24 to sum and 16 to drop in those generations, against
6.8, 6.3 and 5.3 in the others. With this change, every generation runs
at the second set of times, and the program takes 0.50 s instead of
1.18 s (Node: 0.91 s).
The teardown of a class or record type with children of its own type now
keeps the blocks it frees in a segment, linked through their first words
and built in order through a tail pointer. A node releases its own block
first, places it ahead of its children's when its address is the lower
(object literals allocate before their fields) and behind them otherwise
(constructor arguments build the children first), and the chain's first
teardown frees the whole segment onto the free list in one step. A
teardown past DIRECT_DESTROY_CHAIN appends through scr_rc_destroy_seg,
which keeps scr_rc_destroy's depth budget and queue. Blocks outside the
calling thread's arena are freed as before, and targets without the
inline allocation paths keep the old teardown.
The split tools ran oxfmt in the checkout they rewrite. oxfmt looks up
its configuration from that directory and from the formatted files'
directories, and runs an oxfmt.config.ts it finds there, so a branch that
rehome-split rewrites could run code with the maintainer's permissions.
The tools now format through format.mjs, which passes the tools
checkout's .oxfmtrc.jsonc with --config and disables the nested lookup,
so no configuration file of the rewritten checkout is read or run. With
no files to format it does nothing, where oxfmt would format the whole
directory.
The split-tools tests cover split-methods and move-state on a checkout
whose root and src directory commit an oxfmt.config.ts that records it
ran.
jsx-source.test.ts compares the JSX rewrite with TypeScript 5's
transpileModule output, so it imports typescript5 as the other test
oracles on the TypeScript 5 island allowlist do. scripts/test-ts7.mjs
did not list it, and the TypeScript 7 parity job failed since #1786.
Chris Tate