Commit Graph
1258 Commits
Author SHA1 Message Date
Malte Ubl 10e10061a6 runtime: keep a map's storage when clear() finds no entries
An empty map's tables are already empty, so clear() returns at once
instead of freeing oversized storage that the next fill would regrow.
Only a clear that removes entries frees storage far larger than them.
The compaction test in delete is marked unlikely.
2026-10-11 06:39:40 -07:00
Malte Ubl 714afdcfe5 runtime: size a cloned map's direct index for its own keys
new Map(m) and new Set(s) copied the source's direct-index size, which
follows the source's peak. The copy now covers its largest live key, and
keys that are sparse for their count hash instead.
2026-10-11 06:39:40 -07:00
Malte Ubl 175d79ff01 test: record the TS7 preflight/order baseline for 4598-map-set-clear-reuse 2026-10-11 06:39:40 -07:00
Malte Ubl 7956559ed1 runtime: make Map/Set clear and delete cost O(live), not O(peak)
Number-keyed maps index small integer keys through a direct table sized
by the largest key ever stored, and bucket tables are sized by the most
entries ever held; neither shrank. scr_map_clear memset both on every
call, even on an empty map, so a collection that was large once paid its
peak size on every later clear. tsc-ts' Relater.reset clears a
Set<number> on every relation check: on Sentry, 36% of the CPU was that
memset.

- clear() resets only the direct-index slots of the entries it removes
  (the table is zero at every absent key), resets the bucket table only
  when entries pointed into it and it is proportional to them, and frees
  storage far larger than the entries removed, returning the map to its
  initial state as V8 does. Clearing an empty map does no work.
- delete compacts once fewer than a quarter of at least 64 entries are
  live (V8's shrink point), so iteration after mass deletion is O(live).
- Compaction shrinks oversized bucket tables, and after a delete or an
  iteration it shrinks the direct index to the surviving keys or moves
  keys that became sparse back to hashing. The append path keeps the
  current index: it holds a slot of it.

Insertion order and live iteration are unchanged: nothing compacts while
an iteration is active.
2026-10-11 06:39:39 -07:00
Chris Tate 918389d62a Merge pull request #847 from vercel-labs/fix/stream-ir-serialization
Stream serialized IR so compiler-scale artifacts no longer hit V8's string limit
2026-10-10 23:21:07 -05:00
Malte Ubl 5b8661bf8c Keep native compiler IR and frontend cache entries out of single strings
Scriptc-compiled strings stop at 2^31 - 1 UTF-8 bytes and exceeding the
cap traps ("scriptc: out of memory") instead of throwing. In the native
compiler the self-hosted IR was one 515 MB string, and the frontend
cache entry JSON.stringify'd LLVM and IR together into one ~920 MB
string, about 2.3x below the trap.

- NativeExecutableInput.ir is compact IR as ~16M-char parts; every IR
  output is written with writeTextPartsSync.
- Frontend cache entries move to scriptc.native-frontend.v3: the input
  metadata carries placeholders and the LLVM and IR follow as separate
  members, so the entry is written as parts with an incremental digest
  (NativeCache.writeParts). Entries a native reader could not decode as
  one string are neither published nor read (NativeCache.read maxBytes),
  turning the old trap into a cache miss.
- SCRIPTC_BOOTSTRAP_IR_INPUT decodes the seed IR per function with
  deserializeModuleBytes.

The native LLVM text remains one string (344 MB for the compiler,
~6x below the native cap).
2026-10-10 20:39:35 -07:00
Malte Ubl 20b7da29e7 Stream serialized IR and LLVM instead of building one string
The self-hosted compiler's compact IR is ~515-528M characters, a few
million below V8's maximum string length (2^29 - 24). serializeModule
joined it into one string, deserializeModule parsed one string, and the
native bootstrap comparison read cli.ir.json and cli.ll as utf8 strings,
so any compiler growth crashed --emit=ir and the bootstrap tests with
"RangeError: Invalid string length".

- serializeModuleParts returns the compact encoding as ordered parts
  (header, one part per function, remainder). serializeModule(mod, true)
  is their join, so the on-disk format is byte-identical.
- deserializeModuleBytes splits a UTF-8 Buffer at the root object's
  "functions" array with a byte-level scanner and parses the header and
  each function separately. Malformed input falls back to the
  whole-document parse for its exact error.
- The Node API writes --emit=ir, --emit-ir sidecars and --emit=llvm
  through writeTextParts (the former writeLlvmSource batching writer).
- The native bootstrap harness decodes IR per function and compares
  LLVM as Buffers, removing ABI-check lines at the byte level.
- The self-hosting serialization fixture gains a "bytes" mode so the
  scriptc-compiled serializer exercises both new entry points.

Self-hosted compiler IR (514.6 MB, macOS arm64, Node 24.12):
  read  old 5.85 s / 4.26 GB max RSS -> new 5.84 s / 1.36 GB
  write old 2.05-2.53 s / 4.48 GB    -> new 2.30-2.37 s / 2.10 GB
  2x module (1.03 GB): old serialize and read throw; new round-trips.
The remaining per-string bound is the largest function (16.3M chars,
~33x headroom) and the header (2.3M chars).
2026-10-10 20:39:35 -07:00
Chris Tate b610c8d6ea Merge pull request #828 from vercel-labs/fix/lowerer-init-order
Keep null distinct from the TDZ in forward-captured nullable bindings
2026-10-10 17:20:25 -05:00
Malte Ubl 5d0b38ef87 Keep null distinct from the TDZ in forward-captured nullable bindings
A closure created above the `let`/`const` it captures holds the binding
in a TDZ box whose NULL payload slot marks "not yet initialized". A
union of one reference arm with `null` and/or `undefined` (`Set<T> |
null`, `string | undefined`, `C | null`) is represented as the bare arm
pointer with NULL standing for the unit arm, so storing that unit value
made the box indistinguishable from the temporal dead zone: every later
read or write threw "Cannot access 'x' before initialization" although
the declaration had run.

The self-hosted compiler trips on exactly this shape in
lowerer.ts (`let dynamicViews: Set<string> | null = null`, read by a
callback created earlier), which made scriptc 0.2.8 fail to build
tsc-ts's relater with SC3004. Node evaluates the source correctly; the
native code was wrong.

TDZ boxes of nullable-union bindings now start at the module's immortal
NULLABLE_ABSENT sentinel and the TDZ test compares against it. No value
of the binding can hold that sentinel, and every RC entry point and the
cycle collector already skip it, so box release, trace and the
initializing store need no other change.

Corpus: 4596-tdz-nullable-forward-capture.
2026-10-10 14:03:19 -07:00
Malte Ubl 810a0b1d56 Merge pull request #826 from vercel-labs/fix/tscts-gaps/04-callback-fields
tsc-ts -7%: keep native method calls when only an unrelated callback field shares the name
2026-10-10 14:03:48 -05:00
Malte Ubl b979bf5434 Merge pull request #825 from vercel-labs/fix/tscts-gaps/03-error-int32
tsc-ts -7%: keep int32 fields when an Error subclass forwards its constructor arguments
2026-10-10 14:03:32 -05:00
Malte Ubl 64206c72e7 Merge pull request #824 from vercel-labs/fix/tscts-gaps/02-union-fields
Read and write union class fields declared with different types
2026-10-10 14:03:07 -05:00
Malte Ubl 443b3dc666 Merge pull request #823 from vercel-labs/fix/tscts-gaps/01-optional-receivers
Narrow optional base-class receivers before shared union field access
2026-10-10 13:39:28 -05:00
Malte Ubl 3eeea34ce5 Keep native method calls when only an unrelated callback field shares the name
The class method census marks a method name as replaceable when a
function-typed property write uses that name, so a scanner storing its
error callback in `this.onError` made every call of any class's
`onError` method go through the checked property bag. That bag is a
dynamic store naming any property, so the int32 slot analysis also gave up
int32 storage for every class field exposed to dynamic code.

A function-typed write now marks the name only when an instance of a class
that has the method can inhabit the receiver's type: an assignability check
against the instance types of the method's owners, with `this` in an
instance member read as its class. Generic or anonymous owners, `this`
outside instance members, type parameters, `any` and `unknown` receivers
keep the conservative answer. Writes through interfaces or subclasses that
can reach a method still replace it.

Test: class-method-mutations.test.ts. Corpus:
4597-callback-field-method-name.
2026-10-10 09:01:04 -07:00
Malte Ubl 8d8d478f80 Keep int32 fields when an Error subclass forwards its constructor arguments
A class extending Error without its own constructor (or one passing
options to super) initializes the builtin prefix through the
error.ctorOptions helper, whose message and options arguments are dynamic.
The int32 slot analysis did not list that helper among the dynamic helpers
that never store into a live class view, so a single such class anywhere in
a program counted as a dynamic store naming every property: every f64
field of every class exposed to dynamic code lost its int32 storage, and
the integer facts of the functions reading those fields went with it.

error.ctorOptions runs scr_error_init_options, the same initializer
error.newOptions (already listed) runs: it stamps the receiver's
runtime-owned error prefix and only reads `options.cause`. In the native
tsc-ts self-check, two such classes cost 7% total time and 10% check time.

Unit test: int32-slots.test.ts. Corpus: 4596-error-subclass-int-fields
pins Error subclasses forwarding message and options beside integer
fields.
2026-10-10 09:01:03 -07:00
Malte Ubl 0d679c1b98 Read and write union class fields declared with different types
A field shared by every class arm of a union could only be read or
written when all arms declared it with one IR type. Syntax-tree code
commonly narrows a node to several classes with `instanceof` where one
class declares the field optional and another required (`initializer:
Expression | undefined` beside `initializer: Expression`), or as a
subclass beside its base, and both forms were refused.

Reads now dispatch on the union tag, read each arm's own slot, and widen
it to the arm field type every other one converts into without a check
(a union wrap or class upcast); the caller narrows the result to the
checker's flow type as before. Writes evaluate the value once as the arm
field type that every other arm's storage accepts and widen it per arm.
Arms whose field types are unrelated keep the existing refusal.

Corpus: 4594-union-class-field-mixed-types.
2026-10-10 09:01:03 -07:00
Malte Ubl b901ec78eb Narrow optional base-class receivers before shared union field access
A parameter typed as a base class becomes runtime-optional storage
(`Base | undefined`) when a caller can pass an array hole. After
`instanceof` narrowed it to several subclasses and a guard such as
`node.label === undefined` ended its active runtime-optional state, the
next `node.label` lowered the receiver as the storage union, which has no
retag to the checker's class union, and the shared field read or write was
refused. Union property reads and union field writes now convert such a
receiver with the checked extraction and class test that typed slots
already use.

Corpus: 4595-union-class-field-optional-receiver.
2026-10-10 09:01:02 -07:00
Chris Tate c7e82741f9 Merge pull request #822 from vercel-labs/ctate/prepare-v0-2-04de95
Prepare v0.2.8
v0.2.8
2026-10-10 10:45:36 -05:00
Chris Tate b40dbe9bb7 Show 0.2.8 in the coverage diagnostics example 2026-10-10 10:18:09 -05:00
Chris Tate c67e08a9cf Prepare v0.2.8
- Synchronize published packages and the compiler surface manifest at v0.2.8, and bump the runtime ABI marker to v9.
- Add release notes for worker graph sharing, catchable deep recursion, parallel code generation, runtime performance, and compatibility fixes.
2026-10-10 10:08:39 -05:00
Chris Tate 1bf547a122 Report a rejected declaration's own diagnostic (#818) 2026-10-10 10:01:46 -05:00
Chris Tate 9e0c6564d4 Format the LLVM emitter bootstrap test 2026-10-10 08:36:39 -05:00
Chris Tate 8e2197fe10 Merge main into ctate/report-rejected-declaration-a704c5 2026-10-10 08:07:08 -05:00
Chris Tate da2517a976 Keep record layouts stable for shorthand and literal-argument fields (#819) 2026-10-10 07:59:01 -05:00
Chris Tate ec5d419080 Give the sanitized LLVM emitter bootstrap room to emit itself (#820) 2026-10-10 04:00:46 -05:00
Chris Tate e8e644c7e0 Give the sanitized LLVM emitter bootstrap room to emit itself
The natively built emitter records an allocation stack for every object
under ASan, so emitting its own module takes several times longer than
the plain build and now runs close to the 300 s per-stage limit on shared
runners. Sanitized runs get a 900 s per-stage limit and a matching test
budget; plain runs keep the existing limits.
2026-10-10 03:58:05 -05:00
Chris Tate 2ff5711a54 Keep record layouts stable for shorthand and literal-argument fields
A shorthand property and an object literal passed as an argument no
longer widen the declared record layout of the binding or parameter
that receives them. A widened layout made every existing record that
flowed into that binding or parameter convert to a copy, so later
writes through the copy were lost. Those literals keep a missing
element in their own fields and convert, with a check, into a declared
record.

The debug-variables fixture iterates an array whose elements may be
undefined, so its loop binding keeps the union layout the debugger
check inspects.
2026-10-10 03:40:36 -05:00
Chris Tate 9302fc9c93 Merge main into ctate/report-rejected-declaration-a704c5 2026-10-10 01:54:50 -05:00
Chris Tate 6cf3d4744b Read holes in for-of loops and missing-element shorthand properties as undefined (#817) 2026-10-10 01:08:12 -05:00
Malte Ubl 1b6d0723e9 Merge pull request #814 from vercel-labs/perf/workers/03-typed-publication
@scriptc/threads: publish typed object graphs to workers by reference (parallel tsc-ts 4.72 s -> 0.97 s)
2026-10-10 01:06:23 -05:00
Malte Ubl 272aef3247 Release @scriptc/threads with the other packages
sync-versions stamps packages/threads with the CLI version (0.2.7 now),
the release workflow checks that version and publishes the package
before scriptc, and RELEASING.md lists it with the packages that always
release together. npm trusted publishing must be configured for
@scriptc/threads before the next release: a package without it fails
before upload.
2026-10-10 01:00:36 -05:00
Malte Ubl 6835e5a734 Document sharing published graphs between workers
The new docs page (/docs/threads) explains @scriptc/threads: a runnable
example that publishes a million-order ledger and sends it to four
workers, verified under Node.js 24 and as a scriptc executable; what
publish accepts and refuses, and that a refused publish leaves the graph
unchanged; the frozen-object TypeErrors of the write guard; a comparison
with Node's structured clone (constant-time sends, preserved classes and
identity, immortal objects) and the three rules that make a program
behave the same under both; the automatic sharing of deeply frozen
untyped data; and the cost model. The limitations page gains a Published
graphs section (immortality, class and identity differences, write-check
evaluation order, private fields) and links the new page from the
worker-threads section.

This is the first page added since the migration, so the route-test
fixture now accepts pages without a legacy URL, and the agent guide's
checklist for new pages names the fixture.
2026-10-10 01:00:36 -05:00
Malte Ubl 8406d3fcd9 Publish strings as leaves and record their ASCII fact first
A published string can never record facts in its header later, so
publication proves ASCII before marking (every UTF-16 index mapper
answers ASCII strings by identity) and primes the Map key hash only for
short strings: hashing every source text cost more than the rest of the
walk, and the map code already skips caching into immortal strings.
Strings are marked when pushed instead of going through the worklist.
SCRIPTC_PUBLISH_TRACE reports object counts and walk/commit times.
2026-10-10 01:00:36 -05:00
Malte Ubl bf508eb70f Publish typed object graphs to workers by reference (@scriptc/threads)
The new @scriptc/threads package exports publish(value) and
sharesPublishedGraphs. Node.js runs its JavaScript: publish validates
the whole graph (plain objects, arrays, class instances, Maps, Sets and
primitives other than symbols), then deep-freezes it, and published Maps
and Sets reject mutation. scriptc builds lower both exports to
intrinsics and never compile the package body.

In scriptc, publish makes the graph immutable and immortal: a
compiler-emitted walker per static type (llvm/publish.ts) marks every
object rc == SIZE_MAX through a runtime worklist (scr_publish.c), so
retains, releases and the collector skip it and nothing frees it, and
the objects leave the collector's buffers. Hierarchy members dispatch on
the concrete class; arrays, maps, sets, strings and dyn subgraphs have
runtime walkers. Static types over-approximate a graph, so an
unpublishable type gets a refusing walker that fails only when a value
is present, and every mark is logged so a refusal restores the whole
graph, exactly like the Node implementation's validate-first order.

A typed-ref capsule over a published object travels through postMessage
and workerData as a pointer (a fresh capsule in the receiver; vtables,
adapters and type keys are process-wide); immortal dyn graphs inside a
message are shared the same way. Iterating a published map no longer
writes its iteration depth.

Stores into the static types a program publishes (computed by
backend/publication.ts, a class with its whole hierarchy) carry a guard
that throws Node's frozen-object TypeErrors, and may-throw counts them.
The corpus program pins the contract against Node: shared and cyclic
class graphs with arrays, maps, sets and records read identically on
both threads, every write form fails with Node's message, unpublished
instances of the same classes stay writable, a refused publish leaves
its graph unchanged, and a worker-published graph reaches the main
thread. packages/threads/README.md documents the contract and the
differences (prototypes and identity across messages).

Each mark records its object kind, so a successful publish takes the
objects out of the RC audit's live counts (immortal objects are never
freed).

Dates (time values) and process streams (descriptors) are scalars in
scriptc, so their walkers refuse a date field or root directly instead
of pushing its bits as a pointer; the refusal corpus program pins the
statically typed refusals (date, function, RegExp, typed array and Error
values in class and record fields, and a date root) against Node.

Main lowers a discarded or union-typed `xs.pop()` to a length
truncation; those stores now carry a `pop` marker so a published array
fails with pop's own TypeError ("Cannot delete property 'N'", or the
length message for an empty array, whose `length = 0` store the emitter
drops unless the array type can be published).
2026-10-10 01:00:36 -05:00
Malte Ubl 1c5583bba8 Merge pull request #813 from vercel-labs/perf/workers/02-zero-copy-plain-data
Share deeply frozen plain data with workers without copying
2026-10-10 01:00:32 -05:00
Malte Ubl fd5615a274 Return from immortal releases on the inline path
The inline half of a class or record release sent NULL, last and
immortal references to the out-of-line slow release. Static literals
were the only immortal objects then; a published graph is released as
often as ordinary objects (tsc-ts checking a published program spent
more time in the no-op calls of Node releases than in any checker
function). The rarely taken branch now tests for immortal first; the
common decrement path is unchanged.
2026-10-10 00:57:58 -05:00
Malte Ubl 3487d17348 Publish deeply frozen plain-data graphs to workers without copying
A postMessage or workerData graph whose objects and arrays are all
frozen, ordinary and hold only primitives, strings and such containers
is now shared by pointer: every node and string becomes immortal
(rc == SIZE_MAX, so retains, releases and the cycle collector skip it
and nothing frees it), strings cache their key hash first, and the
nodes leave the sender's cycle buffers. Posting an already published
root is O(1). Anything else is cloned as before; SCRIPTC_PUBLISH=0
turns publication off and SCRIPTC_PUBLISH_TRACE=1 reports each choice.

With the receiver freezing what it receives (Node's clone is not
frozen) and no identity comparison across messages, the behavior
matches Node's structured clone. Published nodes leave the RC audit's live counts, since they are never
freed, and the small-integer dyn cache becomes process-wide (filled
before main): a published graph can outlive the thread that created its
nodes, and the thread-local table died with its thread.

The new corpus program pins that
contract both ways, including a worker-published graph read after the
worker exited and a shallowly frozen graph that must still be cloned.
2026-10-10 00:57:58 -05:00
Malte Ubl 33b040d9d7 Merge pull request #812 from vercel-labs/perf/workers/01-worker-tax
Worker tax 1.47x -> 1.04x (Linux), 1.91x -> 1.07x (macOS): per-thread allocator, TLS blocks, one exception alert
2026-10-10 00:57:54 -05:00
Chris Tate 1767fe7fa0 Merge main into perf/workers/01-worker-tax 2026-10-10 00:55:35 -05:00
Chris Tate cbbdf1560c Merge pull request #806 from vercel-labs/perf/tscts/10-checker-hot-paths
tsc-ts -8.6%: borrow checker hot paths (walks, wraps, casts, borrowed returns)
2026-10-10 00:33:08 -05:00
Chris Tate 69b1523466 Merge main into perf/tscts/10-checker-hot-paths 2026-10-10 00:24:43 -05:00
Chris Tate f992727487 Report a rejected declaration's own diagnostic when a discarded probe flushed it first
Deferred collection diagnostics now stay attached to their declaration until a
copy reaches the final report, so a discarded probe buffer or a successful
nested collection can no longer drop them. Uses of a binding whose type names
a rejected class report that class's diagnostic before the inherited-blocker
cascade.
2026-10-09 23:07:57 -05:00
Chris Tate 2d15471497 Read missing elements as NaN in every static Math argument
Positional arguments of Math.hypot now take the same undefined-to-NaN
conversion as Math.max, Math.min and the fixed-arity statics, and a
direct indexed read that can miss converts the same way as a binding
holding one.
2026-10-09 22:45:10 -05:00
Chris Tate 92642d775e Read holes in for-of loops and missing-element shorthand properties as undefined
A for-of loop over an array whose own facts show a hole or a stored
undefined binds T | undefined, and the typed parameters, locals, record
fields and Math arguments it reaches widen to hold it. Loops over arrays
the element-state analysis proves free of holes read each element
directly, without a per-element state test.

Object literals built from a missing-element read keep the property as
undefined in shorthand and long form, through spreads, nested records,
arrays of records, annotated locals and literal arguments to typed
parameters.
2026-10-09 22:05:36 -05:00
Chris Tate e74dac7d5c Make the worker stack-overflow fixture independent of event order (#816) 2026-10-09 21:21:48 -05:00
Chris Tate f388612dcd Make the worker stack-overflow fixture independent of event order
A worker's messages and its uncaught 'error' event reach the parent on
separate channels in Node, so their relative order is not specified and
can differ under load. The fixture now records both and reports them when
the worker exits, which Node and native executables deliver after both.
2026-10-09 21:19:20 -05:00
Chris Tate 198f27eea9 Keep the native compiler independent of weak keys (#810) 2026-10-09 20:37:13 -05:00
Chris Tate ead81b8788 Give each native helper build a private LLVM input (#811) 2026-10-09 20:34:49 -05:00
Chris Tate d08f94b7c3 Give each native helper build a private LLVM input
Concurrent builds of different entries with the same file name share
the default .ll path in a common output directory. The helper read that
shared file in place, so one build could compile the other program's
LLVM text. The helper now always reads a private copy written in large
batches, as before partitioning.

The partition debug-information test also matches the Mach-O
__debug_info section name, so it checks the same property on macOS.
2026-10-09 20:32:08 -05:00
Chris Tate 34cb02d6d1 Keep the native compiler independent of weak keys
Compiler caches keyed by programs and lowering passes move onto the
objects that own their lifetime, and the field-creation pass recognizes
its own writes by their receiver temporaries. A source test keeps weak
collections out of the compiler. Records and arrays that reach an
untyped slot through a union now keep their identity, and empty object
literals are keyed by their native object, so both work as WeakMap and
WeakSet keys instead of failing at runtime.
2026-10-09 20:26:57 -05:00