These PRs were merged into their stack base branches instead of main. This merges perf/alloc-hugepages (with the CI fixes below) into main. Conflicts: order-parity.json (union of entries); scr_alloc.c/scr_alloc.test.ts, where main already has the worker exit trim (#858, adapted to the pre-hugepages allocator): resolved by taking the hugepages allocator and re-applying the original trim commits written on top of it (ea88b87c + 4816696d, clean apply on an identical base).
Conflicts, all resolved to this branch's side:
- emitter.ts (numeric-table get, map index probe), expr-bytes.ts, local-array-reads.ts, string-reads.ts: the base only respelled the size-to-double conversions in the old range checks (uitofp nneg, now sitofp); exactIndexLines replaces those checks entirely. The base's other hunks in these files (host inline truncation, the remaining sitofp conversions) merge as they are.
- int32-slots.ts: #837 and #843 both extracted dynamicFieldStores with identical bodies; keep this branch's signature, whose optional records/unions maps also serve #837's single-argument callers.
Since #837 a type is traced only if it lies in a cycle-capable component or reaches an intrinsic reference; a type that merely holds a cycle cannot be reached back from it. In both backend-analysis cases `record:outer` only holds `record:cycle`, so the Node oracle and the native compiler agree on ["record:cycle"] and the expectations still listed `record:outer`.
Brings in cc7c6cb7 (inline truncation decided for the host when no triple is given), which this branch predates; without it the native bootstrap's LLVM comparison with the Node seed fails on x86-64 Linux.
Three workers each hold and drop about 80 MB of small objects and exit;
three more then allocate as much again in the arenas the first ones
returned (on Linux the first ones hand those pages back when they exit).
Records the fixture's preflight/order baseline.
Publication (@scriptc/threads publish() and the zero-copy delivery of
deeply frozen plain-data graphs) makes objects immortal: nothing frees
them, and the RC audit already stops counting them. LeakSanitizer did not
know, so a program that drops its last reference to a published graph
reported every such object as a direct leak whenever leak detection was
on (SCRIPTC_SAN=1 runs on Linux without ASAN_OPTIONS=detect_leaks=0, such
as the documented local fallback). Both publication paths now mark each
object they make immortal with the weak __lsan_ignore_object hook in ASan
builds; ignored chunks are leak-scan roots, so the verdict on the rest of
the heap is unchanged. Other builds compile the hook away.
The sanitized lanes run with detect_leaks=0, so a harness test pins the
contract with LeakSanitizer on (Linux, SCRIPTC_SAN=1 lane).
Worker loops poll the exception alert every 64 iterations so that a loop calling nothing that tests still observes a terminate request. A loop whose condition, update or do-while condition evaluates a call followed by a pending test, or whose body evaluates one before any statement that can continue the loop, observes the request through that test. Corpus native-worker-terminate-tested-loops terminates workers spinning in each shape, including a continue path that keeps its poll.
Number-keyed maps index small integer keys through a direct table sized
by the largest key ever stored, and bucket tables are sized by the most
entries ever held; neither shrank. scr_map_clear memset both on every
call, even on an empty map, so a collection that was large once paid its
peak size on every later clear. tsc-ts' Relater.reset clears a
Set<number> on every relation check: on Sentry, 36% of the CPU was that
memset.
- clear() resets only the direct-index slots of the entries it removes
(the table is zero at every absent key), resets the bucket table only
when entries pointed into it and it is proportional to them, and frees
storage far larger than the entries removed, returning the map to its
initial state as V8 does. Clearing an empty map does no work.
- delete compacts once fewer than a quarter of at least 64 entries are
live (V8's shrink point), so iteration after mass deletion is O(live).
- Compaction shrinks oversized bucket tables, and after a delete or an
iteration it shrinks the direct index to the surviving keys or moves
keys that became sparse back to hashing. The append path keeps the
current index: it holds a slot of it.
Insertion order and live iteration are unchanged: nothing compacts while
an iteration is active.
The self-hosted compiler's compact IR is ~515-528M characters, a few
million below V8's maximum string length (2^29 - 24). serializeModule
joined it into one string, deserializeModule parsed one string, and the
native bootstrap comparison read cli.ir.json and cli.ll as utf8 strings,
so any compiler growth crashed --emit=ir and the bootstrap tests with
"RangeError: Invalid string length".
- serializeModuleParts returns the compact encoding as ordered parts
(header, one part per function, remainder). serializeModule(mod, true)
is their join, so the on-disk format is byte-identical.
- deserializeModuleBytes splits a UTF-8 Buffer at the root object's
"functions" array with a byte-level scanner and parses the header and
each function separately. Malformed input falls back to the
whole-document parse for its exact error.
- The Node API writes --emit=ir, --emit-ir sidecars and --emit=llvm
through writeTextParts (the former writeLlvmSource batching writer).
- The native bootstrap harness decodes IR per function and compares
LLVM as Buffers, removing ABI-check lines at the byte level.
- The self-hosting serialization fixture gains a "bytes" mode so the
scriptc-compiled serializer exercises both new entry points.
Self-hosted compiler IR (514.6 MB, macOS arm64, Node 24.12):
read old 5.85 s / 4.26 GB max RSS -> new 5.84 s / 1.36 GB
write old 2.05-2.53 s / 4.48 GB -> new 2.30-2.37 s / 2.10 GB
2x module (1.03 GB): old serialize and read throw; new round-trips.
The remaining per-string bound is the largest function (16.3M chars,
~33x headroom) and the header (2.3M chars).
`for (let i = start; i < end; i++)` kept a double counter when the start had
no proven integer range, as in a hash over `[start, end)` whose bounds come
from fields. Such a loop is now versioned like one with an unproven limit:
when the start is an exact integer of at most 2^53 - 1 in magnitude (compared
bit for bit, so -0 takes the original loop), the integer-counter copy runs.
The usual bounds on duplicated bodies apply.
`const DOT = 0x2e` at module scope was a global that every reader loaded (a
thread-local load in worker programs), hiding the value from integer ranges,
switch dispatch and LLVM. foldConstantGlobals rewrites reads of an immutable
number or boolean global that the program writes exactly once, with a literal,
and that has neither a TDZ sentinel nor an initialization flag (globals that
can be read early keep their checked load). The store stays, so namespace
objects still find the value. Library builds keep their globals.
A reference field that only its class family's constructors write, through
their own `this`, never changes outside them. While its object is held, the
field's value is held too, so `this.scanner.scan()` and
`const units = this.units` need no retain and release (and no collector
buffering on the release).
analyzeFinalFields finds such fields over the whole program. It looks through
the derived constructor's guards on `this`, and disqualifies fields that live
dynamic views can store into (the detection int32 slots already uses, now
shared as dynamicFieldStores), fields of runtime-provided or generic classes,
and every field in library builds. canBorrowCallArgument accepts a final field
of a borrowable object, except in constructors, async functions and
generators, and except for nullable-pointer fields, which have no box to lend.
A typed-array, inline ASCII charCodeAt, local array, constant-table or
number-map read whose index is not a proven integer decided validity with
two float compares against the converted length, then an fptoui/uitofp round
trip. On x86-64 both unsigned conversions expand to long sequences, and the
tsc-ts scanner pays them on every character it reads by position.
exactIndexLines replaces them with fptosi, freeze, sitofp, one float equality
and one unsigned integer compare against the length. A frozen out-of-range or
NaN conversion is an arbitrary integer that never converts back to the index
or is not below the length; below the length the round trip is exact, and -0
converts to 0 like ToPropertyKey(-0).
The cycle analysis gave a header to every type that could reach a cycle,
and its construction-only facts gave up on a whole hierarchy as soon as a
constructor let `this` escape or a type could cross into checked-dynamic
code. Four refinements, all derived from the final IR:
- A constructor's stores stay construction-only up to the first top-level
statement in which `this` escapes. Statements run in order, so earlier
stores complete while the object is private; stores from the escaping
statement on, and every store of a constructor whose ancestor let `this`
escape, are writes. Compound-assignment receiver temporaries count as
`this`. An all-immutable cycle is still impossible: each immutable edge
a -> b is stored before a escapes while b has escaped already.
- Crossing into checked-dynamic code only makes types mutable when the
program contains a runtime store into a dynamic value (int32-slots'
audit, now shared as dynamicFieldStores): typed-ref capsules write native
fields only through such a commit. Without one, hidden property bags
stay empty and add no edges. FFI operands and intrinsics other than
publish stay conservative.
- A type is traced iff it lies in a cycle-capable component or reaches an
intrinsic reference. A type that merely holds a cycle cannot be reached
back from it: its references are external counts.
- Reachability is per class: a slot of static type C reaches C's subtree,
not the whole hierarchy; header presence stays uniform per hierarchy.
tsc-ts (e469e33): traced classes 603 -> 469 of 700, records 64 -> 10 of
202, unions 344 -> 93 of 482; scr_cyc_alloc sites 659 -> 471. Mutable
hierarchy units 243 -> 4. Collector candidates on the self-check
(single-threaded) 6.02M -> 5.55M.
A function is edge-preserving when it cannot remove a reference edge; callers then pass field projections borrowed. tsc-ts lost the property for its whole type-ordering family because compareTypeLists reads a module binding behind an import-cycle TDZ check (module.tdzCheck), and for compareSymbols because getSymbolId bumps a shared id counter with Atomics.add (atomics.op). Fresh array literals without spreads and the array operations that only read, copy into a fresh array or append (push and pushSpread retain the new element; moving the slot storage releases nothing) were rejected too. All of these may allocate or throw, which the summary already allows. Removals (pop, shift, splice), truncation, overwrites and Atomics.wait keep the conservative answer.
compareTypes now passes its field arguments borrowed: 42 retains and 118 releases in its body become 4 and 4, and compareSymbols, compareTypeNames and binarySearchTypes need none. Corpus 4621 covers a comparator behind a real import-cycle TDZ error, an atomic id counter, appends that move the storage of the array owning a borrowed argument, loop bindings over appending and draining calls, and callees that shift or truncate the owning array.
An optional chain such as `resolved.members?.get(name)` took three references to its receiver: the owned receiver value, a +1 extraction into the bind slot, and a +1 copy for every chainRecv use, each released out of line (scr_map_release for maps). Now the bind slot borrows the payload whenever another owner outlives the body (the statement frame owns the receiver value), chainRecv reads are borrowable receivers and call arguments, and a receiver projected from a borrowable owner is not retained at all when the body preserves reference edges. Optional chains and chain receivers count as edge-preserving themselves; their children decide. Stack receivers, multi-arm unions and bodies that may remove an edge keep the owned forms.
Corpus 4620 covers chains whose body clears or replaces the receiver field, drops the last other reference, nested and missing links, call and element receivers, self-clearing optional calls, and values that outlive later stores; a mutation that ignores the body check is caught by ASan (heap-use-after-free).
TypeScript types `read ?? "text"` as number when the read is typed number,
and `() => xs[i]` as returning number. Declared functions already widened
their returns for both (the primitive result union and the optional
read), but arrow functions and function expressions kept the checker's
return, so the closure narrowed its result and threw "a 'string' value is
not representable" or "undefined is not representable" (the scanbox
`[string, () => unknown][]` probe table).
Unannotated TypeScript lambdas now get the same widened return as
declared functions. A const or global binding of such a lambda keeps the
checker's parameters and takes the widened result, and calls through the
binding count as optional or primitive-union results for the enclosing
function's return.
After `obj.field = bytes[i]` on a `number | undefined` field, TypeScript
narrows later reads to number, so the read extracted the number arm
unchecked and an out-of-range read printed 0 instead of undefined. Locals
declared `T | undefined` had the same problem inside functions
(`let x: number | undefined; x = xs[i]; x + 1` gave 1, not NaN).
The optional-read analysis now records instance fields declared exactly
`T | undefined` that receive a possibly missing value; their narrowed
reads keep the stored union, as runtime-optional record fields already
do, and the value propagates into parameters and returns like other
optional reads. Locals declared that way join the runtime-optional
storage tracking, so bare reads keep the union until a guard proves
presence. A member read through such a field throws Node's "Cannot read
properties of undefined" TypeError.
String.fromCharCode, String.fromCodePoint, the global isNaN and isFinite,
DataView offsets and values, and typed-array slice/subarray bounds took a
`number | undefined` value (an out-of-range read the checker types as
number) through a checked narrow, which threw "undefined is not
representable" or refused to compile. Node converts undefined with
ToNumber: code unit 0, a RangeError for fromCodePoint, isNaN true, offset
0; an undefined slice end is the length.
The Math argument conversion becomes a shared helper that also handles
field reads and any other value lowered as a number-or-undefined union.
The runtime answers like libuv 1.51's uv_available_parallelism, which
Node's os.availableParallelism() returns: on Linux the CPUs in the
process's affinity mask, lowered to a smaller cgroup (v2 cpu.max or v1
CFS) CPU quota; on Darwin hw.activecpu, hw.logicalcpu or hw.ncpu; on
Windows GetSystemInfo's processor count; at least 1. WASI answers 1, as
the dynamic island does.
Parallel programs (tsc-ts --checkers) need it to size their thread pools.
Corpus 4598 pins the value against the host's Node.
A static array passed where a dynamically typed array is expected (for
example a `number[]` given to an `(object | undefined)[]` or `any[]`
parameter) travels as a live typed reference. Every operation on that
reference refreshed its dyn view by rebuilding the entire array, so
`xs[i]`, `xs.length`, `xs[i] = v`, `i in xs`, `Object.hasOwn`, each
`for...of` step, each `for...in` key probe, and `push`/`pop`/`at` all
cost O(n), and any loop over the array was quadratic: 10k elements took
12 s and 20k took 53 s on Linux. Writes also re-converted and swapped the
whole native storage on every store.
The native array is already the source of truth: each refresh rebuilds
the view from it and keeps only the prototype and symbol properties. Live
array capsules now carry per-element hooks (ScrDynTypedArrayOps), emitted
per array type next to the existing snapshot adapter and bound when the
capsule is built. The runtime answers length and presence from the native
array and converts exactly one element per read, writes an in-range or
appended element in place, and steps array iterators without a refresh.
Holes and out-of-range reads, which can reach the prototype chain, keep
using the view. A value the element type rejects, a write that would
grow the array by more than one element, a multi-value push, and `at`
with a coercible argument also keep the snapshot path, so their
diagnostics and atomicity are unchanged.
The capsule payload stays 80 bytes: the key length and the two flags now
share one 32-bit word, which frees the slot for the hooks pointer on both
64- and 32-bit targets.
Linux, 100k elements: the index loop went from beyond a 120 s timeout to
7 ms; Node takes 0.5 ms.
Test: test_json.c live_array_ref_tests. Corpus:
4598-live-array-ref-element-access.
A typed-array read that may name a missing element lowers to a boxed
`number | undefined`, and the optional-value analysis widens every
parameter it reaches for all callers. Two patterns in tsc-ts's relater
boxed its hottest relation methods this way, failing the build's scalar
ABI check ("Boxed numeric arguments in Checker.isTypeAssignableTo"):
- `const regular = this.regularTypes[id]; const sourceId = regular >= 0
? regular : id;` The read is NaN-coded, but its occurrence in the true
arm was rebuilt as `number | undefined`, so `sourceId` and the
parameters of isSimpleTypeRelatedTo and structuredTypeRelatedTo were
boxed. A comparison with a number that is never NaN is false for
undefined and NaN alike, so an occurrence of a never-reassigned binding
under such a guard is a plain, non-NaN number. The analysis now treats
it as one wherever it flows.
- `for (let i = 0; i < sources.length; i++) isTypeAssignableTo(sources[i],
targets[i])` over two Uint32Arrays allocated with the same length. The
parameters also receive a JSON field that may be NaN, so NaN-coding
cannot carry them (`source === target` must tell NaN from undefined).
The reads are now proven present instead: the index is a non-negative
integer counter that only its own function writes, a dominating guard
bounds it by the array's length or by the length of a parallel typed
array allocated with the same length, and each array is a `const`
fixed-length typed array referenced only through element accesses and
`.length`, so no code can detach its buffer. A proven-present read
lowers to the plain element read and is never optional.
indexReadInBounds also accepts a read inside an `if` condition and takes
an optional same-length fact. Reads the proofs do not cover keep the
established lowering: an inclusive bound, a shorter parallel array, a
decreasing or fractional counter, a counter a closure writes, and an
array that escapes all still read undefined, as the new corpus program
typed-array-present-reads checks against Node.
A typed-array element read answers undefined for an index outside
[0, length), so the default lowering returns a heap-boxed
`number | undefined` and widens every binding, parameter and return the
read reaches. In the tsc-ts scanner that boxed every character read and
turned every character predicate (isLineBreak, isIdentifierPart, ...)
into a function taking a boxed union.
Most consumers cannot tell undefined from NaN: relational, arithmetic
and bitwise operators, truthiness, Math functions, String.fromCharCode,
typed-array indexing, switch over number cases, and equality with a
number that is never undefined. The new analysis (nan-coded-reads.ts)
follows reads through local bindings, parameters of directly called
functions and methods (no overrides, no value uses), and their returns.
A slot is NaN-coded when every value it receives is a NaN-coded read or
slot, or any other number (converted with ToNumber on entry), and either:
- it never holds a genuine NaN (integer elements, integer constants,
guarded integer arithmetic), so NaN stands for undefined exactly; any
consumer that takes the value whole (an argument, a stored or returned
value, a literal element, concatenation) gets the exact
`number | undefined` rebuilt, and the optional-read analysis treats
that occurrence as optional, so it lowers as before; or
- every consumer is NaN-insensitive.
Equality of two NaN-coded values that never hold a genuine NaN also
answers true for two NaNs, as undefined === undefined does. Any other
slot keeps the established union lowering; asserted reads (`t[i]!`) are
unchanged.
tsc-ts best-v1 (scriptc main + this commit vs main, Linux sandbox,
Xeon 2.5 GHz, interleaved, medians of 5, diagnostics identical):
| app | parse | total (single-threaded) | wall --checkers 8 |
| --- | --- | --- | --- |
| tRPC | 1.263 -> 0.866 s (-31.4%) | 3.035 -> 2.527 s (-16.7%) | -13.0% |
| Excalidraw | 2.637 -> 1.865 s (-29.3%) | 14.99 -> 14.04 s (-6.3%) | -16.2% |
| Playwright | 3.150 -> 2.202 s (-30.1%) | 12.93 -> 12.14 s (-6.1%) | -12.4% |
| TypeORM | 2.562 -> 2.036 s (-20.5%) | 10.93 -> 10.46 s (-4.3%) | -8.6% |
| self-check | 2.080 -> 1.529 s (-26.5%) | 7.06 -> 6.68 s (-5.4%) | -15.1% |
The tsc-ts IR has 646 instead of 697 scr_union_new_f64 sites (the rest
are cold), 22 instead of 168 typed-read helper calls, and 217 instead of
572 scr_union_get_f64 calls; the scanner's hot functions allocate no
union boxes. Binary size -0.2%.
A closure created above the `let`/`const` it captures holds the binding
in a TDZ box whose NULL payload slot marks "not yet initialized". A
union of one reference arm with `null` and/or `undefined` (`Set<T> |
null`, `string | undefined`, `C | null`) is represented as the bare arm
pointer with NULL standing for the unit arm, so storing that unit value
made the box indistinguishable from the temporal dead zone: every later
read or write threw "Cannot access 'x' before initialization" although
the declaration had run.
The self-hosted compiler trips on exactly this shape in
lowerer.ts (`let dynamicViews: Set<string> | null = null`, read by a
callback created earlier), which made scriptc 0.2.8 fail to build
tsc-ts's relater with SC3004. Node evaluates the source correctly; the
native code was wrong.
TDZ boxes of nullable-union bindings now start at the module's immortal
NULLABLE_ABSENT sentinel and the TDZ test compares against it. No value
of the binding can hold that sentinel, and every RC entry point and the
cycle collector already skip it, so box release, trace and the
initializing store need no other change.
Corpus: 4596-tdz-nullable-forward-capture.
The class method census marks a method name as replaceable when a
function-typed property write uses that name, so a scanner storing its
error callback in `this.onError` made every call of any class's
`onError` method go through the checked property bag. That bag is a
dynamic store naming any property, so the int32 slot analysis also gave up
int32 storage for every class field exposed to dynamic code.
A function-typed write now marks the name only when an instance of a class
that has the method can inhabit the receiver's type: an assignability check
against the instance types of the method's owners, with `this` in an
instance member read as its class. Generic or anonymous owners, `this`
outside instance members, type parameters, `any` and `unknown` receivers
keep the conservative answer. Writes through interfaces or subclasses that
can reach a method still replace it.
Test: class-method-mutations.test.ts. Corpus:
4597-callback-field-method-name.
A class extending Error without its own constructor (or one passing
options to super) initializes the builtin prefix through the
error.ctorOptions helper, whose message and options arguments are dynamic.
The int32 slot analysis did not list that helper among the dynamic helpers
that never store into a live class view, so a single such class anywhere in
a program counted as a dynamic store naming every property: every f64
field of every class exposed to dynamic code lost its int32 storage, and
the integer facts of the functions reading those fields went with it.
error.ctorOptions runs scr_error_init_options, the same initializer
error.newOptions (already listed) runs: it stamps the receiver's
runtime-owned error prefix and only reads `options.cause`. In the native
tsc-ts self-check, two such classes cost 7% total time and 10% check time.
Unit test: int32-slots.test.ts. Corpus: 4596-error-subclass-int-fields
pins Error subclasses forwarding message and options beside integer
fields.
A field shared by every class arm of a union could only be read or
written when all arms declared it with one IR type. Syntax-tree code
commonly narrows a node to several classes with `instanceof` where one
class declares the field optional and another required (`initializer:
Expression | undefined` beside `initializer: Expression`), or as a
subclass beside its base, and both forms were refused.
Reads now dispatch on the union tag, read each arm's own slot, and widen
it to the arm field type every other one converts into without a check
(a union wrap or class upcast); the caller narrows the result to the
checker's flow type as before. Writes evaluate the value once as the arm
field type that every other arm's storage accepts and widen it per arm.
Arms whose field types are unrelated keep the existing refusal.
Corpus: 4594-union-class-field-mixed-types.
A parameter typed as a base class becomes runtime-optional storage
(`Base | undefined`) when a caller can pass an array hole. After
`instanceof` narrowed it to several subclasses and a guard such as
`node.label === undefined` ended its active runtime-optional state, the
next `node.label` lowered the receiver as the storage union, which has no
retag to the checker's class union, and the shared field read or write was
refused. Union property reads and union field writes now convert such a
receiver with the checked extraction and class test that typed slots
already use.
Corpus: 4595-union-class-field-optional-receiver.
- Synchronize published packages and the compiler surface manifest at v0.2.8, and bump the runtime ABI marker to v9.
- Add release notes for worker graph sharing, catchable deep recursion, parallel code generation, runtime performance, and compatibility fixes.
The natively built emitter records an allocation stack for every object
under ASan, so emitting its own module takes several times longer than
the plain build and now runs close to the 300 s per-stage limit on shared
runners. Sanitized runs get a 900 s per-stage limit and a matching test
budget; plain runs keep the existing limits.
A shorthand property and an object literal passed as an argument no
longer widen the declared record layout of the binding or parameter
that receives them. A widened layout made every existing record that
flowed into that binding or parameter convert to a copy, so later
writes through the copy were lost. Those literals keep a missing
element in their own fields and convert, with a check, into a declared
record.
The debug-variables fixture iterates an array whose elements may be
undefined, so its loop binding keeps the union layout the debugger
check inspects.
Chris Tate