A union that is a Set element or a Map key was a tagged box in the whole
program, because the C map hashes and compares union keys by tag and
payload. One `Set<Node | undefined>` made every `Node | undefined` value
of the program a box, at every read.
A union whose arms all compare by identity (records, class instances,
arrays, functions), beside at most one undefined and one null arm, now
keeps its pointer representation in collections too. The map keys such a
union by reference: the pointer carries the identity, and NULL and the
null sentinel are two more distinct keys. Unions with a string or another
value arm keep their boxes, and library builds keep the earlier rule.
The tests of the box paths keep their unions boxed through a new
`boxedUnions` emission option instead of a Set of each union.
scr_unwind_raise walks the stack with plans cached per return address.
The first raise through a return address finds the function's FDE
through the .eh_frame_hdr of the object that contains it, runs the CFI
up to the call, and finds the call site's landing pad in the LSDA; the
plan records the frame's CFA rule, where its callee-saved registers are
saved, and the landing pad. Later raises look the plans up, restore the
registers, and jump to the first landing pad. Plans are inserted under
a lock and read without one.
Landing pads continue the unwind with scr_unwind_resume instead of
`resume`, so the walk goes on from their frame. A frame whose CFI,
pointer encodings or personality the plans do not express, or a return
address without an FDE, leaves the unwind to libgcc's forced unwind
from the frame the walk started at, as before. Other Linux targets keep
libgcc's unwinder; their scr_unwind_resume calls _Unwind_Resume.
bench:runtime gains layered-errors, a workload whose errors are thrown
below small functions that hold references and caught per entry in a
loop, and the corpus gains unwind-landing-pad-chains, which pins landing
pads that continue the unwind, including in frames LLVM inlines into a
caller whose handler takes the exception.
Values typed `F | undefined` for a function type F are now plain closure
pointers (nullable-pointer unions), so a keyed read of a
`Record<string, F>` or a Map.get no longer allocates a union box. Keyed
record reads, string-keyed Map.get and checked extractions lend the stored
closure as a receiver or callee when ReferenceWrites proves the call cannot
write the locations it was read from, and optional-chain receivers lend
under the same proof. Owned record overflow reads use the inline entry
lookup.
Indirect closure calls skip the ambient receiver bracket for closures
whose bodies never observe it (new ScrClosure.function_kind bit
SCR_FN_RECEIVER_FREE; signature adapters that only forward the receiver
copy it from their original), and a callValue that forwards `dyn.this`
needs no bracket at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
for-of, spread, Array.from, yield* and [Symbol.iterator]() over a string
walked a UTF-16 index. Each step mapped the index to a byte offset, built
the character, and measured its UTF-16 length to advance.
They now walk a byte cursor over the WTF-8 data:
- The cpAtByte intrinsic reads the character that starts at the cursor.
ASCII bytes take the one-character table inline; other characters call
scr_str_cp_at_byte. byteLength advances the cursor by the character's
byte count. A lone surrogate is its own three-byte character, so it
stays a one-unit character, as in Node.
- Spread and Array.from of a string call scr_str_code_points, which
counts the characters, allocates the array once and fills it.
- The one-character strings of U+0080..U+07FF are made immortal on first
use and shared, as the ASCII ones are static.
The cpAt intrinsic, which no lowering emits any more, is removed.
TypeScript types the parameters of an inline replace callback after the
match `any`. An unannotated parameter that is never reassigned, whose
every read means the same for a string or number as for `any`, and that
receives a capture every match sets, the offset or the subject now takes
that value's type. String-search callbacks get the same treatment for the
offset and the subject.
scr_regex_replace_fn calls the closure directly in more cases:
- checked-dynamic parameters, which receive the boxed value Node passes at
their position (a capture or undefined, the offset, the subject, the
groups object, then undefined);
- `string | undefined` capture parameters, which receive NULL when the
match leaves the capture unset;
- a checked-dynamic result, converted with ToString;
- up to six parameters.
Callbacks of the earlier typed form compile to the same call, with a
kinds argument of 0, and run the same per-match work.
readline split redirected input byte by byte, twice per chunk (once to
find whether the chunk ends a line, once to split it). Before splitting
it copied each chunk out of its Buffer and into a scratch buffer, decoded
it into a new string, and copied the unfinished line and the whole chunk
into a joined buffer. Now:
- one scan per chunk: memchr for a CR once, then LF to LF with memchr
when the chunk has none, and the LF, CR and CRLF split otherwise;
- the unfinished line joins only the first line of the chunk;
- an ASCII chunk with no partial UTF-8 sequence held from the previous
one is split where it is, and a chunk that is not shared memory is
read from its own storage. Lines are still copied into their strings
before the first is delivered.
Buffer.from(string, 'latin1' | 'ascii') encoded character by character;
an ASCII prefix, all of an ASCII string, is now copied with memcpy.
Line events, their order and the CRLF delay are unchanged. Corpus 8344
covers lines joined across chunks, a line longer than a chunk, empty
lines, mixed line endings and UTF-8 split across chunks. The new
line-reader workload reads a 67 MB log through readline over a file
stream.
Emitted functions report an exception by returning with the thread's
pending flag set, and every caller tests the flag after each call that
may throw. On Linux executables (glibc and musl), functions reached only
by direct calls now propagate exceptions by unwinding the native stack
instead:
- unwindAbiFunctions selects the may-throw functions that are not async
or generators, constructors, virtually called or overridden methods,
closure targets, the entry, library exports, module initializers or
callees of coroutine bodies.
- A native unwind costs about 2 µs a raise and 0.1 µs a frame, so functions that
exceptions are likely to cross keep the flag: those with a `throw` of
their own, those up to three calls above one, and recursive functions,
counting calls through methods, closures and callbacks.
- Their propagating exits call scr_unwind_raise (scr_unwind.c), a forced
unwind through _Unwind_ForcedUnwind with the exception still in the
active cell.
- Direct calls to them are `invoke`s without a pending check. The
landing pad runs the frame's usual unwind path: it releases the frame's
values and then enters the enclosing try handler or resumes the unwind.
- Remaining plain calls to them (wrappers, adapters, runtime-facing entry
points) become invokes whose landing pad continues on the pending flag,
or resumes when the wrapper is itself invoked. Every other call is
marked `nounwind`, so inlining a raising function at an invoke keeps its
calls plain.
- When emitted text breaks these rules (a raising function's address is
taken, or an invoked function returns with the flag pending), the
module is emitted again with those functions on the pending flag.
Emitted code does not declare scr_unwind_raise `noreturn`, so sanitized
builds do not run the sanitizer's no-return hook, which needs stack
bounds that fiber stacks do not report. Instead, scr_unwind_raise clears
the shadow of the frames it unwinds.
Executables link a static unwinder: libgcc_eh with the host toolchain and
the glibc runtime packs, and Zig's libunwind with Zig and the musl packs.
macOS, Windows, Android, WASI, libraries and debug builds keep the
pending flag everywhere and emit the same code as before.
The throw-chains runtime workload measures exceptions caught 2, 5 and 20
recursive calls above the throw, through direct calls and a callback,
and a parser that backtracks by throwing.
Reapplies #2017, which #2038 reverted after effect's observability-otlp
fixture crashed in plain builds. The crash was the candidate-buffer
corruption that the previous commit fixes: the cached property sites
change when the collector runs, and with enough environment variables a
pass buffered a string box that was later pooled while still buffered.
A checked cast into a union whose discriminant has four or more string
literals reads the discriminant field once (scr_dyn_discriminant_case)
and then tests only the arms whose literal it holds and the arms without
a literal, in the union's arm order. Every other arm's literal test would
fail, so the arm chosen and the error for a value no arm matches are
unchanged. A typed capsule, an accessor or an inherited discriminant
still takes the tests arm by arm. Unions with fewer literals emit the
same code as before.
A program that declares a class reads and writes untyped properties
through helpers that first select the receiver's property bag. The bag
helpers return jsvals, so the helpers read the bag through js.toDyn;
keyed helpers test each class's key table between the key's coercion and
the read; and a program with subclasses normalizes the receiver
(value = %dyn.class.normalize(value)) at the start of each helper. The
inline-cache recognizer accepted none of these forms, so every property
site in such a program called its helper. It now looks through dyn/jsval
conversions of the receiver, accepts capsule branches before a keyed
helper's final read, and accepts a rebinding of the receiver to a bag of
itself. Plain objects and arrays then read and write through the site's
cache, as they do in programs without classes.
bench/dynamic-properties (new) exercises these sites.
for-in over a pure index-signature record (`Record<string, number>`) skips
keys deleted during the walk by testing each snapshotted key for presence
before visiting it. The test built the overflow map's whole key list and
searched it with includes() on every visit, so a walk over n keys took
O(n^2) time: 200 walks over 2000 keys took 8.7 s, against 31 ms under
Node. The test is now recordOvfHas, the overflow map's hashed presence
check, which answers the same question in constant time.
The new benchmark workload dictionary-walks walks, prunes and merges
Record<string, number> dictionaries with for-in and Object.keys/values/
entries. It takes 7.1 s before this change and 23 ms after (Node 116 ms).
Corpus program 7011 pins for-in's visit order and its handling of keys
deleted, re-added and added during the walk.
A method call on a value of unknown type (untyped JavaScript, `any`)
picked its builtin by comparing the method name with strcmp against each
candidate, in long ladders per receiver kind: a string method could test
dozens of names before it matched.
- The name is now looked up once per call in a generated hash table
(scr_dyn_methods.h, from scripts/gen-dyn-methods.mjs), and each ladder
compares the resulting id. The table verifies the name's bytes, so a
name is matched by its content, as before.
- The worker event methods compare the first character before strcmp.
- bench/dynamic-methods calls string, array, Map, Set, number and own
methods from untyped JavaScript.
A row-major dot product, `const base = r * cols; for (let j = 0; j < cols;
j++) s += w[base + j] * v[j]`, kept one index test per `w` read: the
typed-array loop version guard covered an index `k * counter + terms + c`
only when every loop-invariant term had a proven integer range, and `base`
has none when `cols` is a parameter.
The guard now also accepts a pure loop-invariant term without a proven
range. It evaluates the term once and requires an exact integer (not -0)
between -2^32 and 2^32, then checks the first and last index as for any
affine index. In the fast copy the term has that range, so the index is
integer arithmetic and the reads take no index test. Any other value runs
the checked copy.
Adds a Float32Array matrix-vector product with four accumulators per row
to the typed-array-kernels workload.
The helpers of the typed-array callback methods (forEach, map, filter,
some, every, find, findIndex, findLast, findLastIndex, reduce,
reduceRight) called their callback through the closure's code pointer on
every element, and one helper served every call with the same signature,
so the callback was never inlined. A literal callback now gets a helper
of its own, as for the array methods, and the constant-callback census
accepts the typed-array helpers, so the helper calls that function
directly.
The visit is now a loop on its own counter, `for (let i = 0; i < a.length;
i++)` or down from the last index, instead of a loop on a counter
declared before it, so the counter is an integer and element reads need no
index test. reduce and reduceRight without an initial value read their
seed at the first visited index and start one past it.
Adds reduce, some and map callbacks to the typed-array-kernels workload.
A Uint8ClampedArray store called scr_bytes_to_u8_clamp for every element,
which kept loops of clamped stores scalar. The store now computes
ToUint8Clamp in line: NaN and values at most 0 store 0, values at least
255 store 255, and values between round through (v + 2^52) - 2^52, which
rounds a value in (0, 255) to the nearest integer, ties to even, as the
runtime function does.
In worker-enabled programs, every typed-array element access tests the
view's shared storage and branches between a relaxed atomic access and
the ordinary one, so loops over thread-owned views kept one branch per
access. A view's shared storage is fixed at construction, like its length,
so the typed-array loop version guard now tests it once for each invariant
receiver, and the fast copy reads and writes those views directly. In
worker programs a loop is versioned for this even when no bound needs a
guard. When any receiver is shared, the checked copy runs unchanged.
Programs without workers emit the same code as before.
Adds a Uint8ClampedArray brighten kernel to the typed-array-kernels
workload.
A loop such as `for (let i = 0; i < out.length; i++) out[i] = x[i] * y[i]`
keeps its integer induction, and the version guard now also receives the
loop's counted form, so it covers the accesses of `x` and `y` at the
counter. Before, only counted loops (`i < n`) passed a counter to the
guard, and loops to a length kept one bounds check per access of every
other typed array, which kept LLVM from vectorizing them.
Adds the typed-array-kernels runtime workload: element-wise loops over
Float64Array, Float32Array, Uint8Array and Int32Array, and RGBA to gray.
workloads.json listed every workload in one array, and its README table
listed them in one hand order, so changes that each added a workload
appended at the same place and conflicted. Each workload is now a JSON
file in benchmarks/runtime/workloads/ named after it, read through
benchmarks/runtime/workloads.mjs by the bench runner, the zero-cost
gate, the strace script and the dynamic-tier test, ordered by category
and then name. The loader still reads workloads.json in a checkout from
before this change, so the gate can compare against one. The README
table follows the same order.
Indexed reads of a non-ASCII string (charCodeAt, length) become loads
once the runtime holds its UTF-16 units, but emitted code read the
published scan receiver only where runtime state is ordinary global
storage, so worker executables called into the runtime for every read.
On ELF targets their thread-locals live in the executable's static TLS
block, so the receiver is now declared thread-local there and read with
one thread-pointer-relative load. The unit array was also limited to
receivers of 2^20 units; source texts of several megabytes, which
scanners read in full, now get one too.
Adds the unicode-lexer runtime benchmark workload: a lexer over a
2.7M-unit module whose comments and strings are in several languages.
A message encoder started its node array at 16 nodes of 184 bytes and
kept the array, with its unused capacity, until the message was
received. A port that queued many messages held about 3 KB of zeroed
memory per message: 200,000 queued numbers took about 590 MB, and the
page faults made postMessage cost about 1.3 µs. The node array now
starts at 4 nodes and is trimmed to the message's node count when
encoding ends.
scr_channel_post posted a wake event, and with it a pipe write and a
condition broadcast, to the receiving thread's mailbox for every
message. The owner's loop counts a port with queued messages as ready,
so only a message that finds the queue empty needs to wake it.
bench/channel-messages, added here, posts 200,000 messages (numbers and
small objects) on a MessageChannel before the port delivers any.
A worker message round trip cost about 12 µs, almost all of it in two
blocked waits: the poster signaled the waiting thread while it still held
the mailbox lock, so the woken thread went back to sleep on the lock, and
every wait paid a sleep and a wakeup. scr_mailbox_post and close now
signal after the unlock, and scr_mailbox_wait spins for up to 20 µs (or
the wait's timeout, if shorter) on a flag the queue keeps before it
blocks. A process that may run on only one CPU never spins.
A new bench:runtime workload, worker-messages, posts 100,000 round trips
between the main thread and a worker: 1296 -> 360 ms.
ThreadSlots<K, V> holds one value per key and thread, like a WeakMap that
each thread has its own copy of; under Node.js it is a WeakMap per name
and thread. Setting a value never writes the key, so values can be set on
published keys.
Classes used as keys carry a 32-bit slot word after their header words,
assigned on the first set for a key (with a compare-and-swap for a
published key) and never reused. Each family has one block per thread
that maps slot indices to values through pages of 1024 entries. get and
has are inline reads of the slot word, the page and the entry; set and
delete call scr_slots.c, which holds one reference to each value. A key
class's free paths release the key's values, and thread exit releases the
rest.
Frame storage for non-escaping records was limited to number, boolean and
inline-union fields. Records with string, array, record or class fields,
tuples, and same-shape spread updates (`{ ...state, count }`) were always
heap-allocated, even when every use only projected them.
Closed record and tuple shapes now qualify regardless of field types. The
frame owns the reference fields: they start empty in entry-addressed slots
and are registered before anything is stored, with the binding's lexical
scope for locals and with the statement frame for call-argument
temporaries, so an exception while the record is being built releases
exactly what was stored. Interned string literal fields need no owner.
A reassigned binding builds the new value apart, copies it over its slot
and then releases the replaced fields. Result-slot entries cover producers
returning such records, tuples and spreads of a binding, and store into
the zeroed caller storage without releasing an old value. A spread of a
binding is a projection for the lifetime analysis. Published shapes and
switch-case bindings of reference-owning shapes keep heap storage.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Atomics operations on 8-, 16- and 32-bit integer arrays are sequentially
consistent atomic instructions on the element, and no longer take the
process-wide shared-memory lock. Other reads and writes of shared storage
are relaxed atomic accesses: element reads and writes access the element,
and copies, fills, conversions, searches and sorts read shared bytes into
a private copy and write back only the range they change, so concurrent
accesses to the same bytes are free of data races. Non-shared storage
keeps its plain memory accesses. Atomics.wait and Atomics.notify queue
waiters in buckets chosen by location, each with its own lock and
condition variable, so waiting threads no longer serialize other Atomics.
Atomics.store answers +0 for a value that truncates to -0, as Node does.
The shared storage runtime test also runs under ThreadSanitizer.
Every region creation, release and pressure retry walked every condemned
region again. A released region that something still refers to was
re-walked on each of them, two or three times per build in a watch loop,
and each walk visits every object of the region.
An attempt that keeps a region now records the references counted to it.
Retries at creation, release and region entry attempt a kept region again
only when that count has changed, or once the thread has allocated 8 MiB
(or the region's size, if larger) in regions since the attempt. That
volume doubles after each attempt that finds the count unchanged, up to
64 times. The volume covers references from ordinary objects the walk
reaches, which can go away without changing the count. The schedule is
per region, so a region that another region's release unblocks is
attempted at once, and the full cycle collection before an attempt runs
only when some region is due.
Adds the region-builds runtime benchmark and a region test for the
schedule.
publish() used to remove each published object from this thread's
cycle-collector candidate buffers after the walk, which loaded every
object's collector header a second time. On graphs larger than the
cache that second pass took about a third of the publish.
scr_pub_mark now takes a buffered object out of its buffer when it marks
it, while the header is in cache: the slot is cleared and its index
recorded. When the publish ends, scr_cyc_pub_fill closes the empty slots
with entries from the ends of the buffers. A refused publish puts each
taken object back in its slot first. After the walk, the log is read only
for dyn containers, the RC audit and LeakSanitizer.
The emitted walkers test null and immortal references (parent links,
values published before) before calling scr_pub_push, and the string
walker checks for ASCII 32 bytes at a time.
Adds the published-graphs runtime benchmark, a cycle collector test for
the take, restore and fill hooks, and corpus program 5702.
@scriptc/experimental exports InternTable<K, V>: a named table that every
thread of a scriptc program shares and that each thread has its own copy
of under Node.js. Keys are strings or numbers with Map equality; values are
published (@scriptc/threads) when stored, and the first stored value of a
key is kept. Members are get, has, size, getOrInsert and
getOrInsertComputed.
The runtime table (scr_itable.c) splits each table into 64 segments.
Lookups take no lock and do not write; inserts take a segment's spin lock
for a probe and a few stores, and callbacks run without a lock. Number
tables keep keys and values in the index; string tables keep entries in
segment-owned chunks behind tagged slots. Replaced indexes are kept.
The compiler lowers the class to a synthetic class with one f64 handle
field and five intrinsics, with interned helpers per member and type and
an in-place lowering of getOrInsertComputed that creates a function
expression callback only on a miss. Stored values go through the publish
intrinsic, so their types get the write guard. scriptc refuses names that
are not string literals, one name opened with different type arguments,
and key or value types a table cannot store.
scr_publish walks small graphs in inline buffers and reads
SCRIPTC_PUBLISH_TRACE once, instead of allocating and calling getenv on
every call.
A candidate release now also runs a pass once the live cycle-headered
count has grown by 4096 objects since the last pass. A dead graph is
usually reachable from one candidate (its root), so the candidate count
alone let a few hundred dead graphs pile up before a pass, which then
walked them after they had left the cache.
markGray records the objects it grays and counts the ones still
externally referenced after the trial decrements. When none is, every
gray object is garbage, so the pass whitens the recorded set and skips
scan. markGray also records each edge it skips into an older generation,
so a restricted pass pins those targets without tracing the white set a
second time.
Adds the cyclic-graphs runtime benchmark and corpus programs 5700 and
5701.
Number-key reads of a map without the direct index called
scr_map_entry_f64 for every lookup. The internal sc_map_entry_f64 helper
now calls sc_map_entry_f64_hashed for such maps instead: it hashes the
key as scr_map_find_f64 does (SameValueZero key word, scr_key_mix, the
absent-key filter), scans the entries of a small map or probes a compact
bucket table, and calls the runtime lookup only for a table wider than
the compact limit. It stays out of line so the direct-index path stays
small at every call site.
The runtime fallback is called as a read-only call, so both helpers only
read memory and LLVM merges lookups of one key when nothing is stored in
between, such as a has() followed by get().
The runtime benchmark suite gains number-maps: a node table indexed by
dense ids, scope tables keyed by sparse ids, and get/has/set mixes.
In worker-enabled executables a typed-array access tests whether the view
is shared. The test used to come before the index check, and the shared
path called scr_bytes_get_or_nan/scr_bytes_set with the index as a
double, then converted the element back to an integer. Both paths now
use the index the inline check produced: the shared test follows the
check (a view's length never changes), and the shared path calls new
runtime entries that take the index as a size_t
(scr_bytes_shared_get_int, scr_bytes_shared_get_f64,
scr_bytes_shared_set). Integer elements join as one 32-bit value and
convert to a number after the join, so compares and ToInt32/ToUint32
of the result stay integer operations in every element kind.
The amortized worker cancellation poll moves from the loop header to
the loop's back edge (the continue target). In the header it was a loop
exit ahead of the body, which kept LLVM from rotating the loop and left
loads such as the view's length inside it.
Adds the typed-array-scan and typed-array-scan-workers runtime
benchmarks: Uint8Array, Uint16Array, Int32Array and Float64Array scan
loops in an ordinary and a worker-enabled executable.
A record built from unchecked element reads can hold undefined in a field
the checker calls present. Such a record kept its widened layout only in
an inferred binding: an annotated parameter or return, or an unannotated
return, converted it to the declared layout with a checked narrow, which
throws a TypeError for a missing element where Node passes undefined
through.
Record values that flow between bindings and the parameters and results
of functions and methods whose every call is visible now form flow
classes. A class takes the widened layout, with the union of its members'
optional fields, when it is closed: each value entering it is a fresh
object literal or comes from a member, each use of a member reads or
writes its fields, passes it to another member or only inspects it, and
every member declares the same shape. Members then share one layout, so
no conversion runs at the boundary and the record keeps its identity.
A class that reaches other storage keeps the declared layout for every
member, including the inferred bindings and the record parameters the
analysis widened before. Converting there copied the object, so writes
through a parameter or a pushed record did not reach the caller's object;
such a literal now converts once where it is created.
Chris Tate