The speed posture's inline scr_union_release fast path predates the
runtime change that buffers a surviving union box as a cycle candidate
only when its arm_trace can reach a cycle. It still enqueued every
surviving box: sound (an untraced box visits nothing and the runtime death
path reads the header), but it gave back that change's collector savings
under --optimization=speed. The fast path now loads arm_trace and returns
when it is NULL, exactly like scr_union_release; scr_runtime.h pins the
offset.
Re-lands perf/h30-inline-rc (00051ca0, 40398773, 07972360) behind the
speed posture. With inline RC on, every mirrored runtime RC family
retains inline and releases inline down to the rc == 1 / new cycle
candidate slow path (shapes.ts INLINE_RC_FAMILIES). Release emission is
unchanged: ShapeHost.rcHelpers is null unless LlvmTargetOptions.inlineRc
is set, and every retain/release (including capture boxes) stays a
runtime call; the %ScrMapRc type is only emitted with inline RC.
- index.ts and native/prepare.ts pass inlineRc for speed executables,
speed object/asm emission, and speed library profiles.
- scr_runtime.h keeps H30's static asserts for the mirrored header and
trace-slot offsets (no code change; the rebuilt macOS pack's manifest is
identical to the baseline's).
- Tests: shapes.test.ts pins release symbols and H30's speed fast paths
and header offsets; emitter.test.ts pins that release IR is unchanged
and speed routes releases through the helpers. The differential harness
gains `// @optimization: speed` (and a SCRIPTC_TEST_OPTIMIZATION=speed
lane); corpus 4032 pins every mirrored family under speed.
Release executables stay byte-identical to the stack tip (14/14
workloads, macOS).
Upstream native workers (095b6193) run one runtime instance per thread and
make the timer heap thread-local (SCR_TL). The timer slot index added for
O(1) handle lookups indexes that heap, so it becomes thread-local too;
scr_timers_teardown already frees it during worker context shutdown.
The RegExp constructor cache stores retained templates that live instances
borrow and that are never evicted. Its table is thread-local, and worker
executables leave it empty (constructions compile privately as before),
because a worker context has no teardown point after every borrowing
instance. Ordinary executables are unchanged.
stderr is unbuffered, so the piecewise stdio calls issued one write(2)
per argument, separator and newline (two syscalls per simple line), and
the line was not atomic against other writers sharing the descriptor.
Render the line first and emit it with a single fwrite, as Node writes
one chunk.
clearTimeout/clearInterval, refresh, ref/unref and hasRef scanned the
whole timer heap for the handle, so arming and clearing many timers was
quadratic (1e5 cancels: 1.0 s vs Node 46 ms). An id-to-slot hash kept
exact by every heap write makes the lookup O(1); heap order is unchanged.
exec/test rescanned the whole subject for ASCII (and re-converted a
non-ASCII subject to UTF-16) on every call, so lastIndex-driven exec
loops and sticky tokenizers were quadratic (1e5 matches: 7.9 s vs Node
43 ms). A single-entry cache keeps the last long subject's decoding; it
retains the subject so the address cannot be recycled and the string is
never appended in place. Only exec/test use it (no user code runs while
their input is live).
new RegExp recompiled its pattern on every construction (22x Node). A
bounded never-evicting cache keyed by (flags, source) shares bytecode
through the literal-template mechanism; invalid patterns still throw on
every construction.
- scr_net: scr_net_sock_writev_native sends ordered slices with one
sendmsg when the bytes would go out immediately (plain socket, nothing
buffered); every other state takes the per-slice write_raw path, so
buffering, transports, and teardown are unchanged.
- scr_http: end(data) before the head, write() before the head, chunk
framing, and the final chunk now leave as one write (head + body +
framing), as Node does. A typical response was a head segment plus a
body segment; it is now one segment and one syscall.
- scr_net: stop reading after a short read (libuv UV_HANDLE_READ_PARTIAL);
the level-triggered poller reports later bytes or EOF. Removes the
EAGAIN read per request (0.7 per request on http-api). TLS transports
keep draining.
- scr_http: Date header formatted once per second; head built in one
pass without snprintf; request headers parsed with geometric array
growth and one shared string when the name is already lowercase.
The bytes on the wire are unchanged (the runner digests every response
against Node).
On 64-bit little-endian targets ScrStr's capacity word is split into a
32-bit cap and a 32-bit cached Map key hash (header stays 24 bytes).
Heap strings fill the hash on first lookup; every in-place mutation path
(unique concat, split scratch reuse, regrow, spare reuse) clears it.
Immortal strings are never written: compiler literals and the runtime's
one-character/empty statics carry the hash precomputed (string-key-hash.ts
mirrors scr_key_hash; both sides pin the same vectors).
scr_key_hash seeds by length multiplicatively: XORing the length into the
low bits cancelled against the last byte ("ab" and "abc" collided).
Map string lookups inline the probe into each getter and reject linear
entries by hash; set specializes its probe/overwrite by lookup kind with a
shared append path.
Tests: runtime map.test.ts (pinned vectors, statics, in-place append
invalidation), string-key-hash.test.ts, corpus 3141-map-string-key-hash
(plain + sanitized; mutation-checked against both invalidation paths).
Emitted record and class constructors called scr_rt_calloc/scr_cyc_alloc
with a constant size, so the size class, the zeroing length and the
collector header setup were recomputed per object behind a call (plus an
empty scr_obj_alloc_note call and a variable-length memset). llvm/alloc.ts
now emits the fast paths inline: pop the class free list (clearing only
the bytes the helper does not store itself) or bump, with the old calls as
the cold slow path. Releases inline the weak-dispose hook check, the live
count, and the free-list push; blocks outside the reservation still go to
scr_rt_free.
The emitted IR does not know which runtime it links against (the sanitized
lane only differs at link time), so the runtime now defines scr_sa in
every build. Where the allocator is compiled out (ASan, SCR_RC_AUDIT,
wasm32, Windows, 32-bit, thread-instanced libraries) it stays all-zero and
the inline paths always take the slow path, which keeps the RC-audit
object notes. ScrSaState's layout is pinned with static asserts. Inline
emission is additionally gated off for 32-bit, WASI, Windows triples and
thread-instanced libraries.
Runtime callers with hot constant-size allocations (union boxes, arrays,
boxes, closures, dyn nodes) use new static inline twins
scr_cyc_alloc_inline/scr_cyc_free_inline; scr_cyc_live is exported.
The ABI guard now recognizes function-pointer extern data
(scr_weak_dispose_hook). New corpus program 4410 pins recycled-block
clearing, size classes, objects above the largest class, cycles and weak
observers.
On the x86_64 lane 41% of records-sort samples sat on the first load of
each record header (the comparator argument retain): a merge step touches
one element it has not seen before per comparison. Prefetching the
element four positions ahead on the advancing side (and during run
detection) overlaps that miss with the current comparator call: ~4%
faster on Linux (156 vs 162 ms median), neutral on macOS.
records-sort compares interned literals (`key === "department"`, equal
departments) and names that differ in their first bytes, yet every
comparison was an out-of-line call (and memcmp through a stub).
- sc_str_eq: identity, then byte length, then scr_str_eq.
- sc_str_cmp / sc_str_cmp_u16: identity; then the first differing byte
of one 8-byte word when both strings are that long, otherwise of the
first bytes; the UTF-16 order is decided inline only for ASCII bytes.
Everything else calls the runtime.
The helpers are internal module functions that LLVM inlines; they are
emitted whenever the runtime entry is declared.
The comparator sort kernel also holds one undefined-`this` window for the
whole sort (each emitted call restores the receiver stack it pushes) and
resolves the element retain function once.
After the union fix, records-sort spent most of its time in the IR merge
sort's element moves: every src[i] read retained the swapped `src` array
and the element (scr_arr_get_ref), every dst[k] write released the old
slot, and `dst` grew from an empty array.
scr_arr_sort_values runs the IR helper's exact algorithm (same comparator
call order) over borrowed slot copies of the private, compacted snapshot:
- comparator calls use the callValue ABI (owned element arguments, an
undefined `this` window, pending-exception check);
- the array keeps owning every element and is untouched while user code
runs, so the cycle collector and a throwing comparator see the original
snapshot; the sorted permutation is published only after success;
- f64 and reference-counted element types qualify (isSortValuesElement);
unions with an undefined arm and checked/island values keep the IR.
The new internal arrIntrinsic `sortValues` is validated, may throw, and
is not reference-preserving.
scr_obj_alloc_note/scr_obj_free_note exist for the SCR_RC_AUDIT live-object
audit and are empty in release and dev runtime packs, yet every emitted
record/class new and free helper called them. The LLVM emitter takes an
objectAudit option (default on for direct emission); the production
executable and library pipelines pass it only for --sanitize builds. Runtime
TUs compile their own call sites away outside SCR_RC_AUDIT while scr_object.c
keeps the no-op symbols for objects that still reference them.
scr_union_release buffered every union box left alive by a release as a
cycle-collector candidate, including boxes whose arm_trace is NULL (scalar
arms, and ref arms whose type the compiler proved acyclic, such as
TreeNode | null once TreeNode is constructor-only). Such a box's trace visits
nothing, so it can never be on a collectable cycle: skip the candidate
buffer for it. Deaths check the header's buffered bit inline before calling
scr_cyc_on_dead, because typed-ref cache commits swap arm contents between
boxes of one union type, so a box can have been buffered under an earlier
arm.
test_closure.c checks that string- and f64-armed boxes are never buffered,
that closure-armed boxes are, and that a closure -> box -> union -> closure
cycle is still collected under the RC audit.
Upstream native workers (095b6193) start each worker thread with a NULL
scr_als_active (SCR_ALS_SLOT() supplies the main slot) and give the fiber
epilogue its own context-termination path. The fiberless frame now goes
through SCR_ALS_SLOT(), and worker programs keep the fiber path, so
termination and AsyncLocalStorage behave exactly as upstream defines them.
Ordinary executables are unchanged.
scr_ctx_swap now stores the callee-saved registers, stack pointer, and
resume address into the outgoing ScrCtx and loads the incoming one,
instead of pushing a frame onto the fiber's own stack. The register file
lives where ucontext_t kept it: inside the heap ScrFiber for fibers and on
the C stack for a resumer, so a suspended fiber's register-held pointers
stay visible to heap scanners such as LeakSanitizer (fiber stacks are
anonymous mappings they do not scan). Fresh fibers are seeded by writing
the ScrCtx fields; only the top word of the new stack is touched.
Layout asserts pin the struct offsets the assembly uses.
The switch cost is unchanged or slightly lower (async-pipeline scale 10 on
macOS arm64: 0.17-0.19 s vs 0.20-0.21 s).
On x86_64 and AArch64 POSIX targets, fiber switches now save and restore
only the ABI's callee-saved state (scr_ctx_swap) instead of going through
swapcontext/getcontext/makecontext. swapcontext saves and restores the
process signal mask on every switch: one rt_sigprocmask syscall on glibc,
and sigprocmask + sigaltstack + getrlimit on Darwin, plus a 16 KiB memset
in Darwin's makecontext per spawn. Those dominated await-heavy programs
(async-pipeline: most samples in sigprocmask/__sigaltstack/__getrlimit on
macOS, 0.11 s of 0.23 s in the kernel on Linux). The runtime never uses a
context to change the signal mask. Other POSIX targets keep ucontext;
Windows and WASI are unchanged. The sanitizer fiber annotations wrap the
new switch exactly as they wrapped swapcontext.
The spare fiber-stack pool grows from 4 to 64 entries. With fan-out
(Promise.all over a few batches) finished and new fibers alternate in
waves wider than 4, so most spawns paid an mmap, a munmap, and fresh-page
faults. Spares are still unmapped before the loop sleeps.
Security review finding (#747): rn_first recursed into every split target
(split_next/split_goto/loop/loop_split). `visited` stops cycles but not
depth, so a flat runtime pattern with a long run of alternatives, such as
new RegExp("(?:" + "|".repeat(n) + "x)"), chains one split per
alternative and recursed once per alternative while the regex was being
constructed (scr_re_native_build runs on every compile), before any
subject is matched. Deep enough patterns could exhaust the C stack of the
thread compiling them.
The analysis is now iterative: split targets go on a heap worklist of
nops + 1 entries allocated next to `visited`. Every op is processed once,
so each split pushes at most one entry, and the accumulated first-byte set,
empty-match and anchor flags do not depend on the visiting order. If the
worklist cannot be allocated the native build fails and the regex keeps
using libregexp, like the other native-build failures. No other function in
scr_regex_native.h recurses on pattern structure.
Tests: corpus 3139 constructs runtime patterns with 30,000-alternative runs
(empty, consuming, word-bounded, capturing, nested-but-flat, global) and
matches ASCII subjects; it matches Node in the plain and sanitized lanes.
libregexp compiles every non-sticky pattern with a `.*?` search prefix and
interprets its bytecode one opcode at a time, so an unanchored search pays a
backtrack push/pop and ~7 dispatches per subject position, and each `x+`
iteration pushes a backtrack state. Profiles of regex-logs and
template-render showed 65-85% of samples inside lre_exec.
scr_regex_native.h translates the compiled bytecode body (the bytecode stays
the source of truth for parsing, flags, case folding and capture numbering)
into a small program for one-byte subjects:
- single-character opcodes become 128-entry tables computed by evaluating the
interpreter's own predicate (lre_canonicalize, lre_is_space, range pairs)
for each ASCII code unit, so /i and /u are exact on these subjects;
- quantifiers over one character become counted loops that backtrack by
count; greedy loops whose continuation cannot start with a byte of the
loop's class (or is just saves + match) are possessive;
- runs of literal bytes become one memcmp;
- splits, gotos, saves, capture resets, counted group loops and empty-check
registers keep libregexp's exact backtracking order and undo semantics;
- unanchored searches start only at bytes that can begin a match (memchr for
a single byte) and only at 0 for non-multiline `^` patterns.
Lookaround and back references, and all non-ASCII subjects, keep lre_exec.
scr_exec is the single dispatch point, so every match loop is shared. The
program is built with the bytecode and cached on ScrRegex (new `native` slot;
the emitter's %ScrRegex type and literal initializer gain one null pointer).
Tests: packages/runtime/test/test_regex_native.c fuzzes the matcher against
lre_exec (random patterns over the subset and its bail-outs, every start
index, all capture slots; ASan+UBSan), wired into regex.test.ts; a sweep of
8 seeds x 100k patterns (~575k translated, ~70.7M checks) found no mismatch.
tests/corpus/regex-native-matcher.ts pins each construct through the public
APIs on ASCII subjects and non-ASCII twins.
Squashes 1d210a03: build the fuzz test with -D_GNU_SOURCE on Linux; the
vendored cutils.h needs CLOCK_MONOTONIC, which strict -std=c11 hides on glibc.
`ScrSaState scr_sa;` was a tentative definition, which the runtime build
emits as a COMMON symbol. Library localization (abi.localize_runtime)
deliberately keeps COMMON symbols global (the ASan image-registration guard
relies on that), so every runtime-localized library archive exported
scr_sa next to its declared API, failing tests/harness/library-multi.test.ts
M1 ("external definitions equal the declared set exactly").
Initialize it explicitly so it is a regular zero-initialized definition that
localization demotes like every other runtime global. No other global added
by the stack is a tentative definition. The library-multi suite passes
(it fails M1 without this change).
Upstream native workers (095b6193) build every runtime unit with
-DSCR_WORKERS and run one runtime instance per thread. The small-object
allocator state is process-global and unsynchronized, so worker builds
fall back to the system allocator, exactly like thread-instanced library
archives. Ordinary executables keep the allocator unchanged.
Move the allocator into scr_alloc.c with inline fast paths in scr_runtime.h;
strings, array headers/slot buffers, and emitted acyclic records/classes
(scr_rt_calloc/scr_rt_free) now use it. Adds scr_alloc.test.{c,ts}.
* Remove the large-array cliff past 2^20 elements
Arrays longer than 2^20 kept every element beyond the cutoff in a sorted
side store, and each indexed write to an existing entry deleted and
reinserted it, so fill + indexed read/write loops went quadratic
(n=1.2M: 10.4s on macOS arm64, 12.6s on Linux x64).
- Rewrite existing sparse entries in place.
- Let dense storage grow past the cutoff for contiguous extension of a
populated prefix, bulk range writes, and sparse tails that reach 3/16
density, migrating side-store entries so index < cap stays dense.
- Make the side store a deque with front slack so descending first-touch
writes and prefix migration are amortized O(1).
- Bound range-derived slice/splice capacity hints for hole runs.
scr_arr_new now reserves the live count, so a 2^20+3 element Set
converts into dense storage instead of a sparse tail; the white-box
test_lib check that pinned the old policy now asserts dense storage.
* Share immortal boolean union boxes
Maybe-undefined reads of boolean[] elements (`if (composite[i])` in a
sieve) box the element in a boolean|undefined union and free it at once:
one calloc/free pair per read, about a quarter of numeric-kernels samples
at large scales. Unions are immutable and boolean arms own no payload, so
low tags now return headerless immortal instances, the same convention as
the compiler's static unit-arm constants.
* Record preflight/order baselines for corpus 3138
- Preserve callback identity, receiver binding, and class references across unions and collections.
- Track record field presence independently of undefined values and improve native runtime behavior.
- Expand differential coverage and streamline CI bootstrap and compiler rebuild checks.
- Release native workers, shared memory, and broader language and library support.
- Include runtime and compiler improvements in the release notes.
- Synchronize all published packages and the surface manifest at 0.2.6.
- Add native worker execution with isolated module state, structured messages, and lifecycle controls.
- Support shared buffers and integer atomics with coordinated runtime ownership and cancellation.
- Extend compiler, runtime-pack, differential, and compatibility coverage for native concurrency.
- Avoid materializing scalar string slices and preserve index metadata.
- Specialize known callbacks and numeric call boundaries.
- Lower proved integer remainders without changing number semantics.
- Preserve live class identity through interface receivers and object methods.
- Extend synchronous text reads and regular expression replacement callbacks.
- Keep unsupported generic dispatch boundaries explicit.
- Sort primitive arrays with stable native ordering and pure conversion keys.
- Preserve sparse semantics and argument effects while avoiding callback overhead.
- Verify Unicode ordering and ownership across native and WASI paths.
- Speed up string-keyed collections and wide object property lookup.
- Share scalable numeric-key ordering across typed and checked objects.
- Preserve property, iteration, and ownership behavior across native and WASI targets.
- Reduce temporary storage and per-byte overhead across Buffer conversions, bounded writes, and text encoding.
- Support runtime Buffer string encodings with consistent defaults, aliases, and catchable errors.
- Cover conversion boundaries, partial writes, malformed inputs, and retained outputs across native and WASI execution.
- Reuse bounded local spans and process Unicode index conversions in chunks.
- Share slice extraction and avoid unnecessary work in positioned searches.
- Preserve cached receiver state across suffix checks and cover navigation lifetimes.
- Lower literal switches to guarded native dispatch while preserving control flow and evaluation order.
- Reduce repeated AST lookup, traversal ownership, and source classification work.
- Share UTF-8 validation and decode spans directly into final string storage.
- Share regex input preparation and bound capture storage to each result.
- Batch replacement output and avoid rebuilding unchanged strings.
- Accelerate case conversion and normalization while preserving Unicode and ownership behavior.
- Share dense array storage and move owned ranges without rebuilding containers.
- Batch array construction and insertion while preserving evaluation order and bounded temporary storage.
- Lower indexed mutations natively and preserve typed-array reversal across numeric element types.
Malte Ubl