scr_ctx_swap now stores the callee-saved registers, stack pointer, and
resume address into the outgoing ScrCtx and loads the incoming one,
instead of pushing a frame onto the fiber's own stack. The register file
lives where ucontext_t kept it: inside the heap ScrFiber for fibers and on
the C stack for a resumer, so a suspended fiber's register-held pointers
stay visible to heap scanners such as LeakSanitizer (fiber stacks are
anonymous mappings they do not scan). Fresh fibers are seeded by writing
the ScrCtx fields; only the top word of the new stack is touched.
Layout asserts pin the struct offsets the assembly uses.
The switch cost is unchanged or slightly lower (async-pipeline scale 10 on
macOS arm64: 0.17-0.19 s vs 0.20-0.21 s).
On x86_64 and AArch64 POSIX targets, fiber switches now save and restore
only the ABI's callee-saved state (scr_ctx_swap) instead of going through
swapcontext/getcontext/makecontext. swapcontext saves and restores the
process signal mask on every switch: one rt_sigprocmask syscall on glibc,
and sigprocmask + sigaltstack + getrlimit on Darwin, plus a 16 KiB memset
in Darwin's makecontext per spawn. Those dominated await-heavy programs
(async-pipeline: most samples in sigprocmask/__sigaltstack/__getrlimit on
macOS, 0.11 s of 0.23 s in the kernel on Linux). The runtime never uses a
context to change the signal mask. Other POSIX targets keep ucontext;
Windows and WASI are unchanged. The sanitizer fiber annotations wrap the
new switch exactly as they wrapped swapcontext.
The spare fiber-stack pool grows from 4 to 64 entries. With fan-out
(Promise.all over a few batches) finished and new fibers alternate in
waves wider than 4, so most spawns paid an mmap, a munmap, and fresh-page
faults. Spares are still unmapped before the loop sleeps.
Security review finding (#747): rn_first recursed into every split target
(split_next/split_goto/loop/loop_split). `visited` stops cycles but not
depth, so a flat runtime pattern with a long run of alternatives, such as
new RegExp("(?:" + "|".repeat(n) + "x)"), chains one split per
alternative and recursed once per alternative while the regex was being
constructed (scr_re_native_build runs on every compile), before any
subject is matched. Deep enough patterns could exhaust the C stack of the
thread compiling them.
The analysis is now iterative: split targets go on a heap worklist of
nops + 1 entries allocated next to `visited`. Every op is processed once,
so each split pushes at most one entry, and the accumulated first-byte set,
empty-match and anchor flags do not depend on the visiting order. If the
worklist cannot be allocated the native build fails and the regex keeps
using libregexp, like the other native-build failures. No other function in
scr_regex_native.h recurses on pattern structure.
Tests: corpus 3139 constructs runtime patterns with 30,000-alternative runs
(empty, consuming, word-bounded, capturing, nested-but-flat, global) and
matches ASCII subjects; it matches Node in the plain and sanitized lanes.
libregexp compiles every non-sticky pattern with a `.*?` search prefix and
interprets its bytecode one opcode at a time, so an unanchored search pays a
backtrack push/pop and ~7 dispatches per subject position, and each `x+`
iteration pushes a backtrack state. Profiles of regex-logs and
template-render showed 65-85% of samples inside lre_exec.
scr_regex_native.h translates the compiled bytecode body (the bytecode stays
the source of truth for parsing, flags, case folding and capture numbering)
into a small program for one-byte subjects:
- single-character opcodes become 128-entry tables computed by evaluating the
interpreter's own predicate (lre_canonicalize, lre_is_space, range pairs)
for each ASCII code unit, so /i and /u are exact on these subjects;
- quantifiers over one character become counted loops that backtrack by
count; greedy loops whose continuation cannot start with a byte of the
loop's class (or is just saves + match) are possessive;
- runs of literal bytes become one memcmp;
- splits, gotos, saves, capture resets, counted group loops and empty-check
registers keep libregexp's exact backtracking order and undo semantics;
- unanchored searches start only at bytes that can begin a match (memchr for
a single byte) and only at 0 for non-multiline `^` patterns.
Lookaround and back references, and all non-ASCII subjects, keep lre_exec.
scr_exec is the single dispatch point, so every match loop is shared. The
program is built with the bytecode and cached on ScrRegex (new `native` slot;
the emitter's %ScrRegex type and literal initializer gain one null pointer).
Tests: packages/runtime/test/test_regex_native.c fuzzes the matcher against
lre_exec (random patterns over the subset and its bail-outs, every start
index, all capture slots; ASan+UBSan), wired into regex.test.ts; a sweep of
8 seeds x 100k patterns (~575k translated, ~70.7M checks) found no mismatch.
tests/corpus/regex-native-matcher.ts pins each construct through the public
APIs on ASCII subjects and non-ASCII twins.
Squashes 1d210a03: build the fuzz test with -D_GNU_SOURCE on Linux; the
vendored cutils.h needs CLOCK_MONOTONIC, which strict -std=c11 hides on glibc.
Number() of an optional string element read no longer boxes string | undefined
into a dynamic value: plain-operand reads test the slot state and convert the
element with num.fromString, answering NaN for missing slots.
Read-only fallbacks (get_f64/bool/number, state, has, borrow) carry
memory(read, inaccessiblemem: readwrite) at the call site and the guard loads
the storage bases before branching, so LLVM hoists header loads out of
loops. Conditions on number[]/boolean[] element reads no longer box
T | undefined: numbers reuse getNumber (missing is NaN, falsy like undefined),
booleans test the slot state then read the value.
Strict element reads (scr_arr_get_f64/bool/ref, borrow), scr_arr_get_number,
scr_arr_state/has and element writes (scr_arr_set_f64/bool/ref) now check
inline for a canonical integer index inside dense storage (below cap, and
below len for reads, present-value state for strict reads) and load or
store the slot directly. Every other case calls the unchanged runtime entry
with a cold call-site attribute. Proven integers (exactInteger) skip the
double round-trip. for...of also reads the length inline.
Seventeen Order inputs whose shape disagrees with the target (wrong leaf
kinds at the top level, in nested records, record arrays and string
arrays; null for object and number; missing fields; object vs array at a
field and at the root; duplicate keys; truncated JSON; 1e400 and a lone
surrogate; extra nested members; reordered keys) plus an Item[] root.
The expected lines are the output of the checked-route-only build
(perf/harness); the fast path must decline or agree on every one.
`ScrSaState scr_sa;` was a tentative definition, which the runtime build
emits as a COMMON symbol. Library localization (abi.localize_runtime)
deliberately keeps COMMON symbols global (the ASan image-registration guard
relies on that), so every runtime-localized library archive exported
scr_sa next to its declared API, failing tests/harness/library-multi.test.ts
M1 ("external definitions equal the declared set exactly").
Initialize it explicitly so it is a regular zero-initialized definition that
localization demotes like every other runtime global. No other global added
by the stack is a tentative definition. The library-multi suite passes
(it fails M1 without this change).
The tree-shaking test failed on Linux x64 with "expected 68344 to be less
than 65536". Measured with the release helper/runtime-pack pipeline in a
Linux x64 sandbox (hello = console.log("hello", "world")):
PR file .text R+E segment
#743 64256 15794 0x3f11
#744 68416 16050 0x4011
#764 68520 16546 0x4221
The allocator adds 256 bytes of .text to hello: the inline scr_mem_free
fast path in the release functions hello already links (scr_str_release,
scr_arr_destroy, scr_arr_gc_free, scr_cyc_free, scr_dyn_dispose). Its
slow path and address-space reservation (scr_sa_slow, scr_sa_reserve)
are already stripped from hello. ELF segments are page aligned, and #743
left only 239 bytes before the executable segment's 16 KiB page, so 17
bytes over the boundary move every later segment, and the file, by 4 KiB.
Shrinking hello back under the old ceiling is not a reasonable trade.
At the top of the stack the executable segment is 545 bytes past the
page boundary. The extra bytes come from hot-path code the later PRs add
on purpose (number-to-string fast paths in scr_f64_to_str, inline-alloc
release paths, the direct fd write), all reached through console.log's
generic argument rendering. Outlining them to save one page would undo
the optimizations those PRs add.
The ceiling's documented intent is "several native pages of
linker-version slack" above a hello that the comment put at about 41KB.
That no longer held before this stack (64256 bytes on #743, 1.2 KiB of
slack). 80 KiB restores about three pages of slack over the stack's
largest hello and still catches lost section GC (the old always-linked
runtime was about 400KB). Symbol absence remains the primary
reachability contract. The Mach-O ceiling is unchanged.
Verified in the Linux x64 sandbox: pnpm test
tests/harness/runtime-tree-shaking.test.ts fails on #744 before this
change (68416 vs 65536) and passes after it, on #744 and on #764.
Upstream native workers (095b6193) build every runtime unit with
-DSCR_WORKERS and run one runtime instance per thread. The small-object
allocator state is process-global and unsynchronized, so worker builds
fall back to the system allocator, exactly like thread-instanced library
archives. Ordinary executables keep the allocator unchanged.
Move the allocator into scr_alloc.c with inline fast paths in scr_runtime.h;
strings, array headers/slot buffers, and emitted acyclic records/classes
(scr_rt_calloc/scr_rt_free) now use it. Adds scr_alloc.test.{c,ts}.
* Remove the large-array cliff past 2^20 elements
Arrays longer than 2^20 kept every element beyond the cutoff in a sorted
side store, and each indexed write to an existing entry deleted and
reinserted it, so fill + indexed read/write loops went quadratic
(n=1.2M: 10.4s on macOS arm64, 12.6s on Linux x64).
- Rewrite existing sparse entries in place.
- Let dense storage grow past the cutoff for contiguous extension of a
populated prefix, bulk range writes, and sparse tails that reach 3/16
density, migrating side-store entries so index < cap stays dense.
- Make the side store a deque with front slack so descending first-touch
writes and prefix migration are amortized O(1).
- Bound range-derived slice/splice capacity hints for hole runs.
scr_arr_new now reserves the live count, so a 2^20+3 element Set
converts into dense storage instead of a sparse tail; the white-box
test_lib check that pinned the old policy now asserts dense storage.
* Share immortal boolean union boxes
Maybe-undefined reads of boolean[] elements (`if (composite[i])` in a
sieve) box the element in a boolean|undefined union and free it at once:
one calloc/free pair per read, about a quarter of numeric-kernels samples
at large scales. Unions are immutable and boolean arms own no payload, so
low tags now return headerless immortal instances, the same convention as
the compiler's static unit-arm constants.
* Record preflight/order baselines for corpus 3138
Node is the reference the project aims to beat. A single cold oracle
launch overstated its time; it is now warmed and sampled in the same
interleaved loop as the compiled executables, and the report adds a
per-workload candidate/Node ratio and its geomean. --no-node skips it.
Fourteen application-shaped workloads (JSON, regex, parsing and class
dispatch, sorting, rendering, collection pipelines, async fan-out,
Map/Set graphs, allocation, numeric kernels, exceptions, CSV, and the
two multi-module build benchmark apps on large inputs).
bench:runtime builds each workload with one or two checkouts, refuses to
time any executable whose stdout, stderr, or exit status differs from
Node, interleaves baseline and candidate launches, and reports medians
with bootstrap 95% confidence intervals, binary size, and peak RSS.
- Preserve callback identity, receiver binding, and class references across unions and collections.
- Track record field presence independently of undefined values and improve native runtime behavior.
- Expand differential coverage and streamline CI bootstrap and compiler rebuild checks.
* Compose class references across unions and collections
- Compare class, function and native references held in differently represented unions with `===`, `!==` and `Object.is` while preserving subclass identity, and refuse comparisons that a structural union arm leaves undecidable
- Keep subclass values intact through base-typed array and Set insertion spreads, `concat`, and inferred class-predicate `filter`/`find`/`findLast`, and allow checked class refinement of captured optional values with several class arms
- Split the sanitized compiler self-rebuild CI job at the IR boundary, after giving sort helpers distinct IR nodes so serialized and in-memory IR emit the same LLVM
* Keep compact comparisons for plain union arms
- Compare a plain operand with a union that already holds its type through the existing inline union comparison, preserving established parameter representations
- Update the coverage expectation for the accepted class refinement
- Preserve narrowed class and generic values, abstract properties, and standalone caught-class identity.
- Support identity-preserving array concatenation and typed-array collection values, with focused differential and sanitizer coverage.
- Share revision-matched bootstrap seeds across CI jobs while retaining command, rebuild, packaging, and sanitizer checks.
- Keep narrowed class fields and shared union writes native.
- Support covariant returns and method extraction through abstract class views.
- Preserve missing numeric values through local, static, and namespace updates.
- Keep typed callbacks on native paths while preserving receivers, evaluation order, and method replacement.
- Preserve caught class identity, collection narrowing, and missing indexed values across typed code.
- Add standard terminal access and filesystem error metadata with differential coverage and versioned runtime compatibility.
- Release native workers, shared memory, and broader language and library support.
- Include runtime and compiler improvements in the release notes.
- Synchronize all published packages and the surface manifest at 0.2.6.
- Add native worker execution with isolated module state, structured messages, and lifecycle controls.
- Support shared buffers and integer atomics with coordinated runtime ownership and cancellation.
- Extend compiler, runtime-pack, differential, and compatibility coverage for native concurrency.
- Avoid materializing scalar string slices and preserve index metadata.
- Specialize known callbacks and numeric call boundaries.
- Lower proved integer remainders without changing number semantics.
- Preserve live class identity through interface receivers and object methods.
- Extend synchronous text reads and regular expression replacement callbacks.
- Keep unsupported generic dispatch boundaries explicit.
- Support concrete generic inheritance, forward references, and nullable substitutions.
- Bound recursive specialization demands while preserving independent concrete uses.
- Preserve class identity through checked assertions and contextual lowering.
Malte Ubl