A worker's messages and its uncaught 'error' event reach the parent on
separate channels in Node, so their relative order is not specified and
can differ under load. The fixture now records both and reports them when
the worker exits, which Node and native executables deliver after both.
Concurrent builds of different entries with the same file name share
the default .ll path in a common output directory. The helper read that
shared file in place, so one build could compile the other program's
LLVM text. The helper now always reads a private copy written in large
batches, as before partitioning.
The partition debug-information test also matches the Mach-O
__debug_info section name, so it checks the same property on macOS.
Compiler caches keyed by programs and lowering passes move onto the
objects that own their lifetime, and the field-creation pass recognizes
its own writes by their receiver temporaries. A source test keeps weak
collections out of the compiler. Records and arrays that reach an
untyped slot through a union now keep their identity, and empty object
literals are keyed by their native object, so both work as WeakMap and
WeakSet keys instead of failing at runtime.
Large executables were simplified and inlined as one module before the
optimizer split them, so most of a release build ran on a single core,
and any edit recompiled the whole program.
The LLVM helper now divides the program before optimization. It splits
the emitter's assembly text directly, so partitions also parse
concurrently, and falls back to LLVM's module splitter for any input
outside the emitter's grammar. Internal functions referenced by a single
function stay with that function, and globals stay with their first user.
Optimized partitions follow the ThinLTO model: each is simplified and
summarized on its own thread, a thin link imports small functions across
partitions for inlining and returns unexported hidden definitions to
internal linkage, then each partition is optimized and compiled on its
own thread. Unoptimized partitions compile independently, so dev builds
of large programs are parallel too.
A private per-helper cache directory stores simplified partitions under
the hash of their source and objects under the hash of the module after
importing, so a rebuild recompiles only the partitions an edit can
affect. To keep unchanged partitions identical across edits, each
partition numbers its metadata in first-use order, lists only the debug
compile units it uses, and orders declarations by name, and interned
string constants are named by their content rather than first-use order.
Partition counts are powers of two that depend only on the module size,
up to 64, so output stays the same on every host and functions keep
their partition across ordinary edits. LLVM text that arrives in parts is
written in large batches and handed to the helper in place instead of
being written a second time.
Two self-hosting tests compile large parts of the compiler under the
sanitizer, and on loaded macOS runners they regularly come close to their
limits. Backend analysis ranged from 165 s to over 300 s against a 300 s
limit with no code change behind the spread (134 s locally, base and
current alike), and the native-frontend compile ranged from 643 s to over
900 s against a 900 s limit. Backend analysis now allows 600 s, and the
native-frontend compile 1500 s within an 1800 s test limit.
Embedded engine calls entered from a fiber get a stack budget documented
as half the fiber stack, leaving the other half as margin. The budget
stayed at 128 KiB when fibers grew to 512 KiB, so it is now 256 KiB,
which restores that ratio.
The stack guard was a tentative definition, which some targets emit as a
common symbol. Localized library archives deliberately leave common
symbols shared, so on macOS the archive defined scr_stack_guard alongside
its declared exports. An explicit initializer makes it an ordinary
definition that localization hides like the rest of the runtime.
Async functions and generators run on their own fiber stacks. The new
stack-overflow guard keeps a quarter of each stack as a reserve for
unwinding and error reporting, which cut the depth they could recurse to
from 256 KiB to 192 KiB of frames. Moderately deep recursion inside an
async body or generator then threw RangeError where Node runs it, for
example 3000 frames of an unoptimized build on arm64 macOS.
Fibers now get 512 KiB, which leaves 384 KiB after the reserve. Pages are
still committed lazily, so idle and shallow fibers cost address space, not
memory. The corpus pins recursion inside async functions and generators
next to deeper recursion on the main stack.
Native executables used to terminate with a signal when recursion
exhausted the stack. Functions that can take part in unbounded recursion
now compare the stack pointer against a per-stack guard on entry and
throw Node's `RangeError: Maximum call stack size exceeded` through the
ordinary exception path, so catch handlers, finally blocks, worker error
events and the uncaught-error exit code behave as they do under Node.
- The checked set is every function on a cycle of the call graph, where
operations that can reenter compiled code indirectly (indirect calls,
callbacks, dynamic and library operations) reach closure targets and
class members. Non-recursive functions, including leaf callbacks, stay
check-free, and the may-throw analysis accounts for the new throw.
- The runtime installs the guard per context (main thread from the stack
rlimit, workers from their thread attributes) and per async fiber, with
a reserve for the throw, unwinding and error reporting. Worker threads
now get an explicit 8 MiB stack.
- Scalar call results pass through the unwind path as its dummy value, so
calls in tail position keep tail-call and recursion elimination.
Indexed reads that can miss lower through shared helpers (`%arr.idxOr`,
`%bytes.idxOr`), and printing a value that can be undefined goes through a
shared formatter (`%util.fmtv`). Each helper took the location of the read
or print that first created it, so its body carried that line's debug
locations. A breakpoint on such a line then resolved to every helper as
well, and the debugger stopped inside helper code where the user's
variables are not in scope.
The helpers are compiler code, so they now get a location without a
source position (helperLoc). They emit no debug info, while each call
still carries the user's location, so a breakpoint on the line stops only
at the read or print itself.
Class-typed fields that construction may read before assigning them now
observe undefined instead of reading an empty slot. A small analysis walks
field initializers and the constructor in evaluation order and stops at the
first point that can hand the instance to other code (a method call, an
argument, a callback, a base constructor that does); private helpers that
only assign fields are followed. Fields not assigned by then take the
deferred-init representation:
- the slot starts empty and reads check it inline, so assigned reads stay
borrowed and only an empty slot reaches the throw;
- equality, `typeof`, truthiness, `== null`, optional chaining, `as`
casts, undefined-admitting destinations and console output observe the
undefined itself;
- member reads, method calls, updates and writes through an unassigned
field throw Node's TypeErrors in Node's evaluation order.
Fields assigned before any exposure keep their plain representation.
Definite-assignment-asserted fields assigned through such private helpers
no longer need the deferred form either.
Corpus 4471 pins inheritance, callbacks, initializer order, private helpers
and member updates; unit tests cover the analysis and the emitted checks.
Virtual slots whose implementations all borrow a parameter now store the
borrowing body (`@sc_bf_...`), or a virtual adapter, instead of the owned
entry (`@sc_f_...`). Three inheritance codegen tests still required the
owned entry. They now accept either entry for the same implementation,
so they keep checking which method fills each slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The compiler stopped compiling itself. Two recent emitter additions use
forms the self-hosted subset refuses:
- LlEmitter tracked immortal temps in a WeakMap field keyed by block
builder. That field has no lowering, so LlEmitter itself was not
lowered. The set now lives on BlockBuilder, which owns the temp names.
- mentionsLocal probed IR nodes with `"localId" in node` and
`"captures" in node`. It now narrows by node kind, covering every
expression and statement kind that names a local or captures one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vtables store the borrowing body of a method whose slot borrows its
parameters, but that decision lived in private emitter state, so the
self-hosting layout fixture kept emitting owned entries and no longer
matched production (`@sc_f_...` vs `@sc_bf_...`).
Move the decision into VirtualBorrows (virtual-borrows.ts), used by the
emitter and by the fixture, which now runs the same call-lifetimes
analysis. The slot lookup narrows explicitly instead of combining a
union with `&&`/`||`, which the self-hosted subset rejects (SC1090). The
abstract-slot emission case now expects the borrowing body, since its
implementation borrows its receiver.
A ternary whose arms are string literals (`i === 0 ? '' : ','`) produced
an owned frame temp, so every evaluation ended with a scr_str_release call
on an immortal string. tsc-ts made 3.3M such calls (2.1M in typeListKey).
The result is an interned literal like a strLit operand: it is now marked
immortal instead of owned, so it is neither retained nor released wherever
it goes. Immortal temps are recorded per block builder, since temp names
are only unique within one.
Corpus 4564 stores, returns, captures, compares and concatenates such
choices, nested ones included.
tsc-ts self-check (two 11-run interleaved benches): 1.864 -> 1.803 s and
1.736 -> 1.706 s (-1.7..-3.3%), check -0.6% and -2.4%.
An element read (`c.nodes[i]`, `i in c.nodes`, hole tests) borrowed its
array only when it was an unchanged local; an array held in a parameter's
or local's field was retained and released around every read. tsc-ts's
getTypeAtFlowNode scans `c.sharedFlowNodes[i]` and reads the matching
`c.sharedFlowIncomplete[i]` / `c.sharedFlowTypes[i]`: 3.3M retain/release
pairs of 13.5M array releases in the self-check.
Element reads run no user code, so they now take their receiver through
emitStableReceiver like the scalar array intrinsics: a projection rooted in
a counted owner is borrowed while the index cannot remove a reference
edge; an effectful index (a call that may replace the array) keeps the
owned snapshot, so the read still sees the array named first.
Corpus 4563 covers reads through parameter and nested fields, holes and
`in`, and an index that replaces the array mid-expression.
tsc-ts self-check (median of 11, interleaved): 1.757 -> 1.715 s (-2.4%),
check 1.403 -> 1.369 s; binary -16 KB.
`s += x` (canonically `s = s + x`) on a string binding took a retained
snapshot of `s`, evaluated the suffix, released the binding's value so the
snapshot became unique, concatenated (appending in place), and released
the snapshot: a retain and two releases around every append. tsc-ts's
typeListKey alone made 3.4M such releases (plus 2.1M on literal temps).
When the binding is an unboxed local that the suffix does not mention,
nothing but this function's own expressions can read or write it, so
reading it after the suffix gives the left side's value. The old value now
moves into scr_str_concat_move / scr_str_concat_mixed_move, which consume
their head (appending in place when it is unique), and the result takes
its place in the binding: one runtime call instead of four. Boxed
bindings, suffixes that read the local, and expression-position updates
keep the general path.
Corpus 4562 covers aliases that must keep their value, literal heads,
number suffixes, throwing suffixes, self-reading suffixes, expression
position, captured bindings, and long ASCII and non-ASCII appends.
tsc-ts self-check (median of 9, interleaved): 1.854 -> 1.804 s (-2.7%),
check 1.471 -> 1.428 s.
Virtual dispatch kept the owned ABI: the caller retained every argument
and the method's owned entry released what its borrowing body did not
consume. For a visitor-style forEachChild that meant a retain and a
release of the receiver per call, and the release buffered every node the
first time a visitor reached it (1.3M cycle candidates on the tsc-ts
self-check came from forEachChild alone).
call-lifetimes proves per function which parameters its body borrows,
independent of the caller. When every implementation of a vtable slot has
a borrowing body for a parameter, the slot now borrows it: call sites pass
it like a borrowed argument of a direct call (an unchanged local as is,
anything else as an owned snapshot released after the call), and the
vtable stores the borrowing body. An implementation that borrows more
parameters than its slot gets a `.virtual` adapter that releases the
difference; closures, async methods and generators keep the owned slot.
Corpus 4561 covers visitors with early exits, a holder that drops its
own reference to the child it is visiting, slots narrowed by a captured
receiver and by a reassigned parameter (both adapters), results and
exceptions through borrowed virtual calls.
tsc-ts self-check (median of 9, interleaved): 1.788 -> 1.744 s (-2.5%),
check 1.438 -> 1.407 s; binary -55 KB.
Emitted releases of class and record objects already test `buffered`
before calling scr_cyc_on_release, but the runtime's own release paths
(arrays, maps, boxes, closures, unions, errors, promises, streams) called
it unconditionally, and every death of a headered object called
scr_cyc_on_dead, which returns at once for an object that was never
buffered. On the tsc-ts self-check 13M of 18.5M scr_cyc_on_release calls
found the object already buffered (10.7M from traced arrays,
3.2M from maps).
scr_cyc_released/scr_cyc_dead are the inline halves for runtime code,
and the emitted noinline slow paths test `buffered` before calling
scr_cyc_on_dead (cycleDeadLines). The out-of-line hooks keep their own
checks, so other callers are unaffected.
tsc-ts self-check (median of 7): 1.652 -> 1.643 s (-0.5%); in the profile
scr_cyc_on_release + scr_cyc_on_dead fall from 2.75% to under 0.5% self
time. Binary +16 KB.
A literal holding a value that may be missing at run time (and the item
arrays of push/unshift/toSpliced with such values) starts from an empty
array and stores each element with a state test. The empty array had no
dense storage, so the first store always took the runtime path and grew
the array. arrayLit gains an optional capacity hint (validated: an
integer between the element count and 1024, no observable effect) that
the emitter passes to the allocation; these builders set it to the
number of plain elements, so the stores stay on the inline dense path.
tsc-ts hits this in getOptionalType's `[t, missingOrUndefined]`.
A store of a reference-counted value always retained its receiver for the
duration of the store and released it afterwards, while scalar stores
already borrowed a stable receiver. The old value's release cannot drop a
receiver rooted in a counted owner: the owner's path to the receiver never
uses an edge leaving the receiver, and destructors run no user code. So
ref-valued (and nullable-pointer) stores now use emitStableReceiver too.
In a derived constructor `this` lowers to a sequence that throws unless
super() has returned, which no borrow check recognized: every field store,
the super() call (an upcast of `this`) and the discarded `this` result of
the super() statement retained and released the new object, and each
release left it buffered as a cycle candidate. Such throwing guards over
scalar conditions now read through to their result (guardedValue), class
casts of an unchanged local are borrowable call arguments, and a
statement-position sequence whose result is such a read drops it without a
retain (the sequence's locals still move to the statement frame).
Corpus 4560 covers released subtrees that point back at the receiver,
receivers reached through parameter fields and reassigned locals, base
constructors calling overridden methods, and super() chains.
tsc-ts self-check (median of 7, vs /tmp/tscts/tsc-int3): 1.697 -> 1.702 s
(neutral), binary 17,218,200 -> 16,768,696 bytes (-2.6%).
The self-hosted compiler refuses passing Map<string, IrLocal> where a structural Map<string, { boxed?: boolean }> is expected (SC1090), which broke the native bootstrap of local-array-reads.ts. The helper now takes the locals map at its real type.
A call argument or local read from an array may skip its retain when the
enclosing function preserves every heap edge and the array is a stable
borrowed parameter. Accept pure projections of such a parameter as well
(field and record reads, narrowing, casts): the caller-owned root keeps
each projected field, the array, and its elements alive for the frame.
The edge-preservation summary rejected `??` and Math library calls,
neither of which can remove a reference edge, which disqualified
tsc-ts's compareNodes and compareCodePoints and with them the symbol and
type sort comparators. compareSymbols now passes `declarations[0]` and
the symbol names to its callees without retaining and releasing them.
`a.splice(i, n, v)` where v may hold undefined at run time (a for-of
binding or unchecked read, as in reorderCandidates) always built a
present-undefined item array. It now evaluates receiver, start, count,
and v once, in order, and branches: a present v is spliced as a plain
element literal, a missing one keeps the item array. In statement
position the emitter now also looks through such sequences and
conditionals, so the present arm drops its removed elements with
scr_arr_splice_drop and allocates nothing.
`arr.length = n` always called scr_arr_set_len, even when n already is
the length. tsc-ts resets three shared flow-cache arrays to their saved
start after every flow walk, which is usually exactly that no-op. The
emitted code now compares n with the current length first: equality is
precisely the case the runtime accepts and leaves unchanged (-0 included,
NaN never), so only real resizes and invalid lengths reach the runtime.
A statement-position `a.splice(start, deleteCount, ...items)` built a
one-element (or n-element) items array, then scr_arr_splice_insert built
the removed-elements array that the caller released straight away: three
or more allocations per sorted insert in tsc-ts's insertType. When the
result is discarded and the items are plain evaluated values, the emitter
now evaluates receiver, start, count, and items in order and calls
scr_arr_splice_drop with the owned value slots. It applies the same
clamping and state shifts on dense receivers, releases removed values
only after the receiver is consistent again, and keeps the general
implementation for sparse receivers and sparse-sized results.
The runtime-optional `x instanceof C` lowering copied C into an owned
temporary before testing. For a capture-free class reference that value
is the immortal static class object, and the copy cost an out-of-line
scr_classobj_retain_v/release_v pair on every test (2,513 sites in
tsc-ts, mostly AST accessors such as getName and getText). The class
reference is now used directly.
String construction copied every span with a libc memcpy call, and
scr_str_new scanned the copied bytes for ASCII in a second pass. Spans of
at most 16 bytes (identifiers, map keys, number text) now move as two
overlapping fixed-size words, and scr_str_new tests those same words for
ASCII, so the fact is free for short strings; longer spans keep the bulk
copy and a chunked scan that stops at the first non-ASCII byte.
A scan that proves a heap string ASCII now also drops its index-cache
entry: a proven-ASCII string never owns one, because every UTF-16 mapper
answers it by identity and re-checks the header bit after extending an
index. Releasing such a string therefore skips the cache purge, and a
large text proven ASCII once (a JSON document's length, a source file)
no longer keeps every later string release walking the cache. Immortal
strings cannot record the bit and keep their entries as before.
`a.push(v)` where v may hold undefined at run time (an unchecked array
read or a hole-observing for-of binding, as in tsc-ts's filterType loop)
built a temporary one-element array, stored v or present undefined into
it, and spread it onto the receiver: an allocation, a growth, a copy and
two releases per append. Such pushes now evaluate every argument first,
then append each present value through the ordinary push (and its inline
dense fast path) and each missing one as a present-undefined slot at the
current length, which is what Node stores. Pure receivers and arguments
are re-read instead of copied into owned temporaries.
Call arguments read from arrays of nullable references now take the
inline dense lookup like other projections (the value is the element
pointer itself, so the path is a few loads), and scr_arr_peek_ref answers
dense in-range indices before the general lookup. compareSymbols'
`declarations[0]!` arguments no longer leave compiled code.
buf.write(string, 'utf16le' | 'latin1' | 'ascii') decoded one UTF-8
code point per step. tsc-ts converts every source file to UTF-16 this
way (toCodeUnits), so the per-character loop showed up in the profile.
Eight-byte ASCII runs whose units all fit the remaining budget are now
widened (utf16le) or copied directly; any other position, and a tail
that would cross the budget, takes the unchanged per-code-point path, so
truncation at a unit or surrogate boundary is exactly as before.
The number-key insert inlined a second copy of the hashed word insert and every index transition (1.7 KB on arm64). One out-of-line word insert now serves the generic dispatcher and the number path, and entering, growing and leaving the direct index stay out of line, leaving the insert at 0.5 KB.
JavaScript string reads use UTF-16 indices over scriptc's UTF-8 storage,
so charCodeAt, length, slice, substring and indexOf looked every receiver
up in a small global index cache (scr_sidx) on each call, and a fresh
short receiver evicted an entry and rescanned its bytes. In the tsc-ts
self-check that lookup and its mapping were the largest string cost.
Bit 31 of the capacity word now records that a string is proven all
ASCII, where units are bytes. A clear bit means "not known", so the fact
only needs to be exact where it is set: scr_str_new and its users prove
it while copying, concatenations combine their operands' bits, number
text, ASCII slices and repeat inherit it, emitted literals and runtime
statics carry it in their capacity word, and an index-cache scan that
proves ASCII records it. scr_str_hash_forget clears it with the cached
key hash, so every in-place mutation drops it; an in-place append that
keeps the string ASCII sets it again. Capacity stops at 2^31 - 1 bytes,
still above the UTF-8 size of any string V8 can represent.
Proven-ASCII receivers skip the index cache in every UTF-16 mapper. Sites
for length and charCodeAt call small internal helpers (string-reads.ts)
that LLVM inlines: a header load and, for an in-range index, one byte
load; everything else defers to the runtime with unchanged semantics.
For non-ASCII receivers the cache remembers its last hit (validated by
owner and tier), charCodeAt answers sequential scans from the cached
ASCII span before any other work, and appends that keep a string ASCII
no longer walk the cache.
`s + n`, multi-part concatenations with number operands, and `s += a + n`
materialized each number as a temporary heap string, and a compound
suffix as a second string, before copying them into the result. A new
runtime entry, scr_str_concat_mixed, takes string and number operands
together and formats numbers directly into the result. Given a head, it
follows scr_str_concat's left-operand rule: appended in place when the
caller's borrow is its only reference, it has room, and it is not also an
operand.
The self-concatenation assignment now flattens its suffix and appends
every operand to the snapshot in one call, so tsc-ts's typeListKey loop
(`key += (i === 0 ? '' : ',') + types[i].id`) appends in place without
allocating per element. Operands still evaluate left to right; converting
a number has no effects, so formatting it at the call keeps JavaScript
order. A corpus program pins Number::toString shapes, evaluation order,
self-referencing suffixes, aliases taken mid-append and Unicode text.
Number-key reads called scr_map_get_f64_ref and friends, which hashed the
key and retained reference values through the map's val_retain function
pointer. They now go through sc_map_entry_f64, a small internal IR helper
LLVM inlines at each site: with the runtime's direct index present, an
integral in-range key reads its slot and any other key is absent, with no
call; maps without the index defer to scr_map_entry_f64. The generated
code reads the entry's value slot itself and retains reference values
with the value type's own (inlinable) retain, the adapter the collection
was constructed with. Map.get with stack-boxed results, union and scalar
values, and Map/Set.has all use it.
The helper and its %ScrMapIx/%ScrMapEntry types follow the target's size
type, so wasm32 builds verify as well.
Maps and Sets with number keys used as id tables (tsc-ts keeps node and
symbol links in Map<number, Links>) hashed every key as a double. They now
keep a direct index, dense[k] = entry index + 1, while every live key is an
integer in [0, ndense) and the table stays within 8x the live count (64
slots minimum). The entries array, tombstones and iteration order are
unchanged; only the lookup index differs.
-0 lands on slot 0 (SameValueZero), and NaN, fractions, negatives and keys
beyond the table are absent by construction. Inserting a key the table
cannot hold moves the map to the bucket table; whenever that table would
grow, compact integer keys move it back, so maps whose ids start far from
zero switch over once their live count catches up. scr_map_entry_f64 and
layout asserts expose the index to generated code.
A randomized C test compares every operation with an ordered reference
model across linear, direct and hashed indexes, and a corpus program pins
the JavaScript-visible semantics around the transitions.
Only growth moves a map to the direct index: an append that
scr_map_reserve_append would absorb by compacting tombstones (a linear map
with a dead entry, a bucket table at most half live, no iteration active)
keeps the map's compact storage, so churn on a small number-keyed map no
longer allocates a table. While an iteration is active nothing compacts,
so a small map promotes to the direct index (integer keys) or the bucket
table (other numbers) without moving entry indices. The runtime map test's
small-storage script, which assumed bucket promotion and failed its two
live-iteration checks, now runs with integer and fractional keys and
checks both transitions, and the corpus program covers small maps growing
and clearing mid-iteration.
Chris Tate