Commit Graph
213 Commits
Author SHA1 Message Date
Chris Tate ac64b1d7fd Compile regex literals to native matchers (#931) 2026-10-13 11:58:36 -05:00
Chris Tate 67f59798d2 Support node:console Console instances (#923) 2026-10-13 11:57:56 -05:00
Chris Tate bf40cf4255 Compile regex literals to native matchers 2026-10-13 11:30:19 -05:00
Chris Tate 2b280dd25c Merge main into node:console Console support 2026-10-13 10:54:25 -05:00
Chris Tate 1334a8acd2 Lend arguments to closure bodies and comparators 2026-10-13 10:35:18 -05:00
Chris Tate 8056aafa37 Support node:console Console instances
- Support `new Console(...)` over process stdio and writable streams
- Write `log`/`info`/`debug`/`error`/`warn` output like Node, including `ignoreErrors`
- Mark the `Console` class as partially supported in the compatibility data
2026-10-13 09:15:07 -05:00
Chris Tate 4f44241543 Clarify stream consumer support details
- Describe the consumers' Node Readable scope in the compatibility data
- Renumber the new corpus programs to avoid shared prefixes
2026-10-13 09:13:49 -05:00
Chris Tate 5a7fc7225b Support stream/consumers arrayBuffer
- Support `arrayBuffer()` from `node:stream/consumers`
- Print ArrayBuffer and SharedArrayBuffer values like Node
- Mark the stream consumers as partially supported in the compatibility data
2026-10-13 08:33:55 -05:00
Chris Tate 2c8dccd52b Merge main into the docs rewrite 2026-10-13 07:46:52 -05:00
Chris Tate 5df589706b Skip array state bytes below a hole-free prefix 2026-10-13 02:13:46 -05:00
Chris Tate e5ed06771f Use plain wording in the introduction and READMEs 2026-10-13 00:40:11 -05:00
Chris Tate d316552918 Rewrite docs for a coherent new-user flow 2026-10-13 00:34:50 -05:00
Chris Tate ef25cf119d Prepare v0.3.0 2026-10-12 15:57:06 -05:00
Chris Tate 7e6294ca0f Make scriptc compilation native-only 2026-10-12 12:25:30 -05:00
Chris Tate d75d7a22f0 Prepare v0.2.10 2026-10-12 10:31:02 -05:00
Chris Tate 043c656bbd Merge main into feat/program-lifetime-publication 2026-10-12 09:38:24 -05:00
Chris Tate 7cf57f9ed5 Merge main into feat/acyclic-contract 2026-10-12 08:43:05 -05:00
Chris Tate 85736170ae Merge main into feat/program-lifetime-publication 2026-10-12 08:13:15 -05:00
Chris Tate 0d2bcf9d97 Merge main into feat/acyclic-contract 2026-10-12 08:10:24 -05:00
Chris Tate 88b4e89e55 Merge main into feat/nothrow-try-catch 2026-10-12 08:07:54 -05:00
Chris Tate 83248651e2 Publish Immutable factory arrays only after provable index writes 2026-10-12 02:33:29 -05:00
Malte Ubl fa55ce6b07 Keep the RC entry points of array elements in one shared descriptor
Every SCR_ELEM_REF array stored its element type's retain, release and trace entry points in its own header (24 bytes), so ScrArr was 80 bytes: an 80-byte block for arrays of acyclic elements and a 96-byte block for cycle-capable ones. The three pointers now live in one constant ScrArrOps descriptor per element type, which the compiler emits once per module (@sc_arrops.*) and the runtime keeps static, and the header holds a pointer to it plus a 4-byte `cyc` flag beside `elem` that the inline RC fast paths test. ScrArr is 64 bytes: arrays take 64- and 80-byte blocks.

scr_arr_new_ref(retain, release, trace, cap) becomes scr_arr_new_ops(ops, cap); scr_set_to_arr_ref takes the result's descriptor from its caller instead of building an array from the set's key adapters. The backend names the %ScrArr fields in SCR_ARR_FIELD. The layout change bumps the runtime ABI marker to scr_runtime_abi_v13.
2026-10-11 19:46:48 -07:00
Malte Ubl 2ccb018004 Publish ProgramLifetime graphs with @scriptc/threads: program-lifetime instances count 2^63 until published
Program-lifetime instances were created with rc == SIZE_MAX, which every runtime contract reads as "published": the frozen-object guards throw on stores, postMessage and workerData share the object by pointer instead of cloning it, and the publish walker stops there. So programs that publish could not use ProgramLifetime (SC7007 refused it), and an unpublished instance posted to a worker would have been shared instead of cloned.

They are now created with the midpoint count 2^63 (2^31 on 32-bit targets). Only the class's empty retain and release helpers count references on these types, so nothing ever changes or frees that count; the midpoint keeps even an unpaired generic retain or release away from 0 and SIZE_MAX. Until the program publishes an instance it is an ordinary mutable, thread-local object to every runtime contract: stores succeed, postMessage clones it. Publication marks it SIZE_MAX like any object it reaches (the visited mark, logged and restored when a refusal undoes the publish), after which the frozen guards throw Node's TypeErrors and other threads read it by pointer. No header, extra word, runtime change or new instruction: the count itself is the published mark. The cycle analysis keeps the hierarchies untraced, so the collector never visits an instance, as for every untraced class.

SC7007 no longer refuses program-lifetime classes reachable from a publish() call; it still requires a root class in an executable. Without the marker nothing changes: tsc-ts best-v7 built with and without this change is byte-identical.

Tests: corpus 4634-contract-program-lifetime-publish (Node vs native): program-lifetime hierarchies with parent cycles, maps and sets of them, maps keyed by them and mortal scopes around them, published on the main thread and a worker and read on the other; writes after publication throw on both threads; unpublished instances stay writable; a refused publish leaves visited instances writable; a later publish reaches published and fresh instances; an unpublished instance in workerData is cloned. With the old SIZE_MAX count the program dies at its first constructor store. classes.test.ts covers the count on both pointer widths, the walker and the frozen guard. The contract-program-lifetime-publish diagnostics fixture is removed (the program now compiles). Docs: contracts and limitations pages, @scriptc/types.
2026-10-11 17:07:22 -07:00
Malte Ubl 3c26a85d9c Merge branch 'feat/contract-types' into feat/acyclic-contract 2026-10-11 16:32:31 -07:00
Malte Ubl 3d7e8e731c Merge branch 'feat/contract-types' into feat/nothrow-try-catch 2026-10-11 16:32:27 -07:00
Malte Ubl 5bfe1a6bfd Merge branch 'feat/contract-types' into feat/program-lifetime 2026-10-11 16:32:21 -07:00
Malte Ubl 15af413acd Merge branch 'main' into feat/contract-types 2026-10-11 16:32:00 -07:00
Malte Ubl 4dc638b328 Acyclic: compile explainCycle in the self-hosted compiler; MDX-safe docs
An empty step list literal is typed explicitly so the native compiler
maps it to the step array type, and the positions list avoids braces,
which the docs pipeline would parse as an MDX expression.
2026-10-11 16:30:25 -07:00
Malte Ubl ad2fcfbc27 try/catch discharges NoThrow: handlers take what they catch
The may-throw analysis now models handlers. A throwing operation or call
inside a `try` with a `catch`, or inside a try/catch whose `finally`
never completes normally (it returns, breaks or continues, discarding the
pending exception), adds no seed and no call edge. The catch and finally
bodies are analyzed like any other code, so rethrows and a finally that
re-raises stay exact. Worker programs (a termination request unwinds
through catch) and generator bodies (a `.return()` does too) keep the
conservative answer.

Unreachable statements no longer seed: the frontend's SC9002 trap after a
body that returns from both its try and its catch (or ends in a switch
with a default, or an infinite loop) is dead code. On tsc-ts this drops
14 functions from the may-throw set and 44 pending-exception checks.

A NoThrow function may now contain handled calls; the SC7001 hint names
the remedy that applies (catch it, remove the recursion, or the worker
exception).
2026-10-11 16:30:24 -07:00
Malte Ubl 3ce9370b35 Acyclic contract type: classes and object types that cannot lie on a cycle
`class C implements Acyclic`, `interface I extends Acyclic` and
`type T = Acyclic<{ ... }>` promise that no chain of references leads
from such an object back to itself. The check is the cycle analysis that
decides tracing, so a proven type carries no cycle-collector header. When
it does not hold, SC7007 shows the cycle: the path of fields, and why it
can close (a field assigned after construction, with the first such
write; an array, Map or Set; a function, dynamic or promise field; a
subclass sharing the layout). cycle-analysis.ts gains explainCycle over
the same graph with edge labels; computeTraced is unchanged.

Acyclic elsewhere is reported with SC7006. Docs, @scriptc/types,
corpus 4642, diagnostics snapshots and TypeScript baselines included.
2026-10-11 16:30:24 -07:00
Malte Ubl 5cc3babc73 Add the ProgramLifetime contract: classes whose instances live until exit skip reference counting and cycle collection
`class Node implements ProgramLifetime` (@scriptc/types, import type only, erased for Node) promises that instances of the class and of its subclasses are kept until the program exits. scriptc then creates them immortal: `new` stores rc = SIZE_MAX and allocates no collector header, the class's retain and release entry points are empty (so reference counting on every static type of the hierarchy folds away), and the cycle analysis treats the hierarchy as a leaf, which untraces types that only reached it. Immortal objects are already part of every RC, collector, weak-collection and publication contract (static literals, published graphs).

The promise only changes memory use; SC7007 refuses it where something else could observe it: on a class that is not the root of its hierarchy (including built-in bases), in library builds, and on classes whose instances can be published to other threads. SC7006 reports the marker outside a class's implements clause.

tsc-ts with nine roots marked (Node, NodeList, FlowNode, Symbol, Type, Signature, IndexInfo, SymbolLinks, NodeLinks): single-threaded wall -25.8% Excalidraw, -22.2% TypeORM, -25.9% self-check, -28.5% Sentry; peak RSS -6.5% to -16.5%; identical diagnostics.
2026-10-11 16:30:23 -07:00
Chris Tate fa11424e5f Prepare v0.2.9 2026-10-11 17:25:14 -05:00
Malte Ubl bb274adc5d Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/test/ts7/baselines/order-parity.json
2026-10-11 13:27:39 -07:00
Malte Ubl 947c9bcf3f Contract types: explain throwing paths, bind only declared functions, docs and baselines
- NoThrow diagnostics name compiler-resolved Node errors (class and message)
  and explain parameters the frontend compiles as T | undefined because a
  caller passes a possibly missing value (for example xs[i]!), naming that
  call.
- Synthetic dynamic-class helpers (%dyn.*) carry the location of the code
  that first needed them; they are no longer bound to annotations there.
- Int32 hints only parenthesize where needed; the NoThrow hint no longer
  suggests try/catch, which does not discharge the contract.
- Diagnostics snapshots and TypeScript preflight/order baselines for the new
  fixtures.
- Docs: /docs/contracts and a limitations entry.
2026-10-11 12:24:27 -07:00
Malte Ubl fc93d10871 Merge remote-tracking branch 'origin/perf/array-layout-main' into HEAD 2026-10-11 12:23:05 -07:00
Malte Ubl cc8392d26a Merge remote-tracking branch 'origin/perf/heap-layout-main' into HEAD 2026-10-11 12:22:56 -07:00
Malte Ubl c8430bbf0d Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/test/ts7/baselines/order-parity.json
2026-10-11 12:22:49 -07:00
Chris Tate 0ab65a79b0 Merge pull request #836 from vercel-labs/perf/parmem
Support os.availableParallelism() in static programs
2026-10-11 12:55:51 -05:00
Malte Ubl def6f86a0e Let collector types bypass the generations where they never die
Nearly every long-lived object becomes a cycle candidate once, soon after
it is built: the constructing frame's release leaves it held only by the
structure it was stored into. A nursery pass then walks it (about 75 ns per
object), it survives, and it is re-buffered for the mature level, whose
next pass walks it again. In tsc-ts single-threaded, nursery passes walked
6.0M objects on Excalidraw and 36.5M on Sentry and freed 7%; 99% of what
they walked was promoted. Most of that walking goes to types whose objects
never die in a young cycle: SymbolLinks, NodeLinks and Symbol were 2.0M of
Excalidraw's candidates and the collector freed none of them, while young
garbage comes from a few types (inference contexts and mappers, closures,
some arrays). Types with a collector yield under 0.5% are 42-55% of what
nursery passes walk.

So the collector now keeps per-thread statistics per type and level. Each
ScrCycType gains a `kind` index: emitted descriptors number themselves from
32 (FIRST_EMITTED_CYC_KIND / SCR_CYC_KIND_EMITTED), the runtime's own
families have fixed kinds below that, 0 keeps no statistics. For the
nursery and mature levels, a pass counts the candidates of each kind it
walks and the objects of each kind it frees. After every 8192 walked
candidates a kind bypasses that level for the next window if fewer than 1
in 128 of them were freed. A release that buffers an object at a level its
type bypasses promotes it past the level first, and a nursery pass that
re-buffers a survivor into a bypassed mature level moves it to the old
level. One candidate in 16 stays put, so the yield keeps being measured
and a type whose objects start dying young again is walked again after
one window.

Only the generation an object is judged in changes. Any assignment is
sound (a restricted pass counts edges from older objects as external), a
bypassing candidate stays buffered, and full passes walk everything they
did before, so the old level's bounds hold unchanged. Cyclic garbage made
only of objects of a bypassing type waits for the next pass at an older
level. Programs that never complete a window (8192 walked candidates of one
type) are collected exactly as before.

The descriptor layout is part of the emitted-code/runtime ABI, so the
program-object marker moves to scr_runtime_abi_v12.

Pass log (instrumented build, single-threaded, one run each, with the
preceding null-slot change): all collector passes Excalidraw 606 -> 313 ms
(nursery 435 -> 153), TypeORM 629 -> 383 ms, tsc-ts self-check 377 -> 215
ms, Sentry 7.61 -> 5.75 s (nursery 2.92 -> 1.04 s, mature 1.17 -> 1.02 s;
the full pass is unchanged at 3.6 s).

tsc-ts (Linux sandbox, interleaved, medians; 10 rounds, Sentry 5 rounds
pinned to 8 CPUs; both changes against int/best-v6): single-threaded
Excalidraw -8.0%, TypeORM -4.2%, self-check -4.3%, Sentry -5.3%;
--checkers 8 -12.2%, -0.8%, -2.3%, -1.6%. Peak RSS -0.8% to +0.6%.
Diagnostics identical.

Tests: test_cycle gains check_nursery_bypass, check_mature_bypass and
check_rebuffer_past_mature (a test-only entry point, scr_cyc_pass_at under
SCR_CYC_TESTING, runs passes at exact levels); each fails under the
corresponding mutation (no bypass, a sticky decision, no sampling, no freed
or walked counts, a bypassing candidate left unbuffered, survivors
re-buffered into a bypassed level). Corpus 4690 builds a long-lived
declaration graph large enough for both levels to bypass and lets objects
of the same classes die in cycles; under --sanitize the exit RC audit
proves they are reclaimed.
2026-10-11 09:39:57 -07:00
Malte Ubl 1a14263b5f Move rarely used array storage out of line
Every ScrArr carried sixteen words, seven of them (sparse slots, their
length and capacity, noncanonical numeric properties with length and
capacity, and match-result metadata) used only by sparse arrays, arrays
with numeric non-index properties and regex match results. They now live in
a ScrArrExt the array points at, allocated on first use: a dense array
shrinks from 128 to 80 bytes, and a cycle-capable one (with its 16-byte
collector header) from 144 to 96.

Runtime readers go through a shared empty ext; writers allocate it. The
backend mirror loses the seven fields, tagged-template arrays are emitted
with a NULL ext, live stream commits swap the ext pointer (which swapped all
seven fields before), and the cached typed-view refresh swaps sparse slots
and properties through scr_arr_swap_indexed_ext, keeping each array's
metadata as before. The program-object ABI marker moves to
scr_runtime_abi_v11.

A corpus program (4610-array-ext-dyn-refresh.ts, with its order baseline) pins numeric non-index properties of a dynamically typed array through its refreshed typed view, sparse slots written directly and through a live reference, truncation, and regex match metadata read directly, through `any` and after a copy.
2026-10-11 09:38:27 -07:00
Malte Ubl 6f488e13aa Shrink the cycle-collector header to 16 bytes
Every cycle-capable object carried a 32-byte header: trace and teardown function pointers, a 32-bit color, the buffered flag, the generation and a 64-bit candidate index. The two function pointers are the same for every object of a type, so the header now points at one constant ScrCycType { trace, free_fn } descriptor per type, and the remaining fields pack into the second word: u8 color, u8 gen, u16 buffered, u32 buf_index (candidate buffers stop at 2^31 entries). wasm32 pads before the descriptor, so every object-relative offset is the same on all targets.

Emitted shapes define their descriptor next to the trace and teardown functions; inline allocation stores one header word; the inline release reads buffered at obj-6 and the inline free reads the generation as an i8 at obj-7. Runtime families allocate with static descriptors. The program-object ABI marker moves to scr_runtime_abi_v10.

tsc-ts (interleaved, 5 rounds): peak RSS -8% to -11% on every app and mode; single-threaded wall -0.3% to -5.5% on top of the typed-rest-spread change.
2026-10-11 09:38:26 -07:00
Malte Ubl 6a9475e931 Free the address space after process.exit off the critical path (--detached-exit)
`scriptc build|run --detached-exit` opts an executable into handing the address-space teardown after process.exit to a helper process on Linux. The helper shares the address space (CLONE_VM), blocks signals, closes every descriptor but the read end of a pipe and waits for EOF on it. The kernel releases an exiting process's memory (exit_mm) before its files (exit_files), so EOF means the helper holds the last reference; it then exits and the kernel frees the pages there instead of before the parent sees the exit (30-40 ms per GB of 4 KiB pages; tsc-ts 20-110 ms). The same latency mold hides with --fork.

It is a build-time choice, not a run-time knob, because the helper keeps the executable mapped (overwriting the binary right after it exits can fail with ETXTBSY) and the memory charged to the cgroup for up to ~100 ms, and the teardown CPU time leaves the parent's rusage. The program object defines `scr_detached_exit = 1`, overriding the runtime's weak default of 0, so no runtime flavor or linker flag is added. The option is part of the early executable cache and route keys; executables only, no effect on other platforms.
2026-10-11 07:05:29 -07:00
Malte Ubl 4b325150fa Support os.availableParallelism() in static programs
The runtime answers like libuv 1.51's uv_available_parallelism, which
Node's os.availableParallelism() returns: on Linux the CPUs in the
process's affinity mask, lowered to a smaller cgroup (v2 cpu.max or v1
CFS) CPU quota; on Darwin hw.activecpu, hw.logicalcpu or hw.ncpu; on
Windows GetSystemInfo's processor count; at least 1. WASI answers 1, as
the dynamic island does.

Parallel programs (tsc-ts --checkers) need it to size their thread pools.
Corpus 4598 pins the value against the host's Node.
2026-10-10 14:45:28 -07:00
Chris Tate b40dbe9bb7 Show 0.2.8 in the coverage diagnostics example 2026-10-10 10:18:09 -05:00
Chris Tate c67e08a9cf Prepare v0.2.8
- Synchronize published packages and the compiler surface manifest at v0.2.8, and bump the runtime ABI marker to v9.
- Add release notes for worker graph sharing, catchable deep recursion, parallel code generation, runtime performance, and compatibility fixes.
2026-10-10 10:08:39 -05:00
Malte Ubl 6835e5a734 Document sharing published graphs between workers
The new docs page (/docs/threads) explains @scriptc/threads: a runnable
example that publishes a million-order ledger and sends it to four
workers, verified under Node.js 24 and as a scriptc executable; what
publish accepts and refuses, and that a refused publish leaves the graph
unchanged; the frozen-object TypeErrors of the write guard; a comparison
with Node's structured clone (constant-time sends, preserved classes and
identity, immortal objects) and the three rules that make a program
behave the same under both; the automatic sharing of deeply frozen
untyped data; and the cost model. The limitations page gains a Published
graphs section (immortality, class and identity differences, write-check
evaluation order, private fields) and links the new page from the
worker-threads section.

This is the first page added since the migration, so the route-test
fixture now accepts pages without a legacy URL, and the agent guide's
checklist for new pages names the fixture.
2026-10-10 01:00:36 -05:00
Chris Tate 3ea231d70d Optimize and generate native code in parallel partitions
Large executables were simplified and inlined as one module before the
optimizer split them, so most of a release build ran on a single core,
and any edit recompiled the whole program.

The LLVM helper now divides the program before optimization. It splits
the emitter's assembly text directly, so partitions also parse
concurrently, and falls back to LLVM's module splitter for any input
outside the emitter's grammar. Internal functions referenced by a single
function stay with that function, and globals stay with their first user.
Optimized partitions follow the ThinLTO model: each is simplified and
summarized on its own thread, a thin link imports small functions across
partitions for inlining and returns unexported hidden definitions to
internal linkage, then each partition is optimized and compiled on its
own thread. Unoptimized partitions compile independently, so dev builds
of large programs are parallel too.

A private per-helper cache directory stores simplified partitions under
the hash of their source and objects under the hash of the module after
importing, so a rebuild recompiles only the partitions an edit can
affect. To keep unchanged partitions identical across edits, each
partition numbers its metadata in first-use order, lists only the debug
compile units it uses, and orders declarations by name, and interned
string constants are named by their content rather than first-use order.

Partition counts are powers of two that depend only on the module size,
up to 64, so output stays the same on every host and functions keep
their partition across ordinary edits. LLVM text that arrives in parts is
written in large batches and handed to the helper in place instead of
being written a second time.
2026-10-09 19:51:47 -05:00
Chris Tate d1e7f9088c Give async bodies and generators 512 KiB stacks
Async functions and generators run on their own fiber stacks. The new
stack-overflow guard keeps a quarter of each stack as a reserve for
unwinding and error reporting, which cut the depth they could recurse to
from 256 KiB to 192 KiB of frames. Moderately deep recursion inside an
async body or generator then threw RangeError where Node runs it, for
example 3000 frames of an unoptimized build on arm64 macOS.

Fibers now get 512 KiB, which leaves 384 KiB after the reserve. Pages are
still committed lazily, so idle and shallow fibers cost address space, not
memory. The corpus pins recursion inside async functions and generators
next to deeper recursion on the main stack.
2026-10-09 17:33:59 -05:00
Chris Tate 3bcf761f42 Throw a catchable RangeError on deep recursion
Native executables used to terminate with a signal when recursion
exhausted the stack. Functions that can take part in unbounded recursion
now compare the stack pointer against a per-stack guard on entry and
throw Node's `RangeError: Maximum call stack size exceeded` through the
ordinary exception path, so catch handlers, finally blocks, worker error
events and the uncaught-error exit code behave as they do under Node.

- The checked set is every function on a cycle of the call graph, where
  operations that can reenter compiled code indirectly (indirect calls,
  callbacks, dynamic and library operations) reach closure targets and
  class members. Non-recursive functions, including leaf callbacks, stay
  check-free, and the may-throw analysis accounts for the new throw.
- The runtime installs the guard per context (main thread from the stack
  rlimit, workers from their thread attributes) and per async fiber, with
  a reserve for the throw, unwinding and error reporting. Worker threads
  now get an explicit 8 MiB stack.
- Scalar call results pass through the unwind path as its dummy value, so
  calls in tail position keep tail-call and recursion elimination.
2026-10-09 16:55:35 -05:00
Chris Tate cd28e4d6ca Answer undefined for out-of-range typed array reads like Node (#797) 2026-10-09 11:37:16 -05:00