Worker runtimes reserve one region for 32 arenas of 40 class slices. When
the host refused the full region (32 arenas of 1 GiB slices, 1.28 TiB of
address space), the allocator kept 32 arenas and shrank every slice. Under
valgrind, which maps at most 56 GiB, slices dropped to 16 MiB, and every
class whose live blocks outgrew 16 MiB took the system allocator.
Callgrind profiles then counted system allocations that a native run
never makes: tsc-ts checking VS Code made 10.3 M small calloc calls under
valgrind and 0.06 M natively.
The reservation now tries arena counts from 32 down to 4 at each slice
size, from 1 GiB down, before smaller slices, and 2 and 1 arena only when
no slice size fits 4. Arenas that are not reserved stay claimed, so
threads beyond them use the system allocator, and the region bounds of
foreign frees use the arenas reserved. Hosts that map the full region keep
32 arenas of 1 GiB slices.
A runtime test limits RLIMIT_AS to 60 GiB and checks that the first thread
gets 256 MiB slices for 400,000 blocks of 56 bytes, that three more
threads get arenas and a fourth uses the system allocator, and that blocks
freed across threads keep their contents.
charCodeAt on a non-ASCII string mapped the UTF-16 index to a byte offset
through a string index entry, which the first read claimed and the
string's release purged. For a string of one character, such as each
character of a for-of loop that codePointAt(0) classifies, the entry cost
more than the read.
A non-ASCII receiver of up to 16 bytes is now decoded from its start on
each read, as scr_str_utf16_len already counts short receivers. Longer
receivers keep the index entry and its cursor.
A template literal of strings and numbers reserved 32 bytes for each
number, the longest number text, and kept that capacity for the result's
lifetime: an 8-byte name built from two numbers used a 96-byte block. The
numbers now format into a stack buffer first, so the result is allocated
once at its exact length.
A checked object (SCR_DYN_OBJ) now starts in layout mode: v.obj.layout names
its key sequence, a node of the thread's interned transition tree, and
v.obj.slots holds one 8-byte value per key, where every property used to take
a 48-byte entry. All properties of a layout-mode object are writable,
enumerable, configurable data properties. An accessor, a non-default
attribute, a delete, more than 64 keys, spent layout budgets, publication to
other threads, or any code that reads entries directly moves the object to
dictionary mode (today's entries) for good, so unported paths stay correct.
Layouts replace the dynamic tier's lazily computed hidden classes (ScrJsShape)
and keep their budgets. Inline caches key on the layout pointer and read the
slot directly; JSON.parse builds objects along layouts; stringify, key
enumeration, release, trace and the runtime read cache read slots in place.
The emitted index-signature checks read members through runtime accessors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runtime units outside the object core (scr_json.c, scr_js_object.c) read and
adjusted SCR_DYN_OBJ entries in place: v.obj.len, v.obj.entries[i] and the
48-byte ScrDynEntry fields. The single object model's generic objects
(stage S3) change that storage, so these units now go through own-property
accessors in scr_dyn.h:
- scr_dyn_obj_count, scr_dyn_obj_key_at, scr_dyn_obj_value_at and
scr_dyn_obj_prop_at (a ScrDynProp view: key, value, getter, setter and
attributes) walk the properties in creation order;
- scr_dyn_obj_index_of finds one by key;
- scr_dyn_obj_set_attrs_at restricts a member a builtin object just added;
- scr_dyn_own_symbol_prop and scr_dyn_copy_prop_at wrap the symbol-entry
and copy primitives.
The accessors are inline over today's entries, so behavior is unchanged.
21 units move (inspect, assert, util compare, console, http, http2, fetch,
tls, fs, os, lib, array, message, publish, region, symbol, worker events,
handle properties, builtin values, declaration adoption). A package test
keeps entry storage reads inside the core.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A limited pass (scr_cyc_collect_set, which freeing a LifetimeRegion runs
over the objects the region owned) takes only the candidates in its set.
Its walk can still reach other candidates, and one that is referenced from
outside is blackened by scan and promoted to the next generation while it
keeps its slot in its old generation's buffer. scr_cyc_on_dead finds a
candidate's buffer by its generation, so when that object died it removed
another buffer's entry and left its own slot pointing at the freed block,
which the next pass then read.
Such a candidate is no longer promoted: it keeps the generation of the
buffer it is in until a pass takes it.
The test kept one live 48-byte block every 1,000 allocations, which on a
system with 64 KiB pages leaves a live block on every page, so no page held
only free blocks and the resident-memory check failed. It now keeps one
block per four pages of blocks, from the system's page size.
The native compiler's frontend-dispose sweep frees about 450 MB on a cold
tsc-ts build, but the size-class allocator keeps the freed blocks on their
classes' free lists, which only later blocks of the same size reuse. Code
generation allocates other sizes, so its pages stacked on top of the freed
heap and set the build's peak resident memory.
- scr_heap_trim (scr_alloc.c) returns the pages of the calling thread's
size-class slices that hold only free blocks (MADV_DONTNEED) and the
system allocator's free pages (malloc_trim). In the single-threaded
allocator the blocks of a returned page leave the free list and the
class carves past its frontier instead; the remaining free blocks are
relinked in address order. Worker runtimes return the calling thread's
arena pages as a parking thread does.
- scr_collect_cycles_phase, the full sweep after a phase's data died,
calls it; the native compiler calls that sweep when it disposes the
frontend.
- test_heap_trim.c checks that the pages go back, that the blocks still
allocated keep their contents, and that new blocks overlap none of
them, with and without worker arenas (another thread allocating and
freeing remotely throughout).
Cold tsc-ts build, 12 interleaved rounds: peak RSS 1407 -> 1247 MB
(-160 MB, -11.4%, every round); wall 87.1 -> 85.9 s (median).
String storage can hold bytes that are not well-formed UTF-8: an HTTP
header value keeps the raw bytes a client sent. Encoding such a string
read past its end.
- scr_bytes_next_cp decoded any high byte as the lead of a sequence and
read up to three bytes after it without a bound. It now takes the
storage length, decodes only complete, well-formed sequences, and
returns any other byte as U+0000..U+00FF, advancing by one. That is
each such byte's value in Node, which decodes header values as latin1.
This covers latin1, ascii, binary and utf16le encoding, hex and base64
of non-ASCII text (which narrow through latin1), Buffer#write and
TextEncoder#encodeInto.
- base64 decoding read one byte past its input when the eight-character
groups ended exactly at the end.
The runtime's bytes unit test (built with ASan) encodes ill-formed
storage at every offset within a word and random high bytes, and
compares latin1, ascii and utf16le with a scalar reference. A corpus
program sends such header bytes to a server and encodes them as Node
does.
A program's nursery trigger starts at 256 candidates and doubles, up to
4096, after each scheduled mature or full pass whose dead set held more
mature and old objects than an eighth of what the nursery passes freed
since the previous one; it halves after one that held less than a
thirty-second of that. SCR_CYCLE_THRESHOLD and
scr_cyc_set_nursery_threshold pin the trigger as before. The
scheduled-age rule keeps counting collection points in the pinned or
default trigger.
A versioned loop over a non-ASCII string passes the range of indexes it
reads: a counted loop's start and limit, or a while loop's counter and
bound. The runtime converts the string at once when that range has at
least one unit per 16 bytes of it, and otherwise counts the loop as one
indexed read, so a loop over a line of a long source text no longer
keeps a UTF-16 copy of all of it. The publish call is weighted as
unlikely, since loops re-entered over one string skip it.
A number-keyed Map or Set with keys the direct index allows now moves to it
at its second entry rather than its fifth, so a map over a few small
integer keys (a histogram over 0..3) reads through the inline direct index
instead of the out-of-line linear scan.
The native compiler now starts with the cycle collector's nursery trigger
at 1024 new candidates instead of 256. scr_cyc_set_nursery_threshold sets
the trigger that SCR_CYCLE_THRESHOLD sets, on the pattern of
scr_cyc_set_growth_cap; the environment variable still takes precedence.
The compiler sets it from a constructor in its host object, before its
modules initialize. Set from main, the trigger leaves the build's
collection work where it was: the cost depends on the collector's first
passes. Programs keep the default trigger.
A cold tsc-ts build takes 4% less wall time (12 paired runs) and its peak
RSS goes from 1.41 GB to 1.31 GB. The program's IR is unchanged.
uintptr_t is 32 bits on wasm32, where the weak-collection hashes shifted it by 33 and 56 bits. The hashes now widen keys, owners and kinds to uint64_t before mixing; 64-bit targets compute the same values as before.
JSON.parse's SyntaxError messages were shaped like V8's but differed from
Node's in several ways:
- Every positional message now ends with V8's "(line L column C)", where
\n, \r and \r\n end lines.
- Positions are UTF-16 offsets, as in Node, not byte offsets.
- An object or array cut off after '{', a property name, a value or an
array element reports the token V8 expected ("Expected property name or
'}'", "Expected ':' after property name", "Expected ',' or '}' after
property value", "Expected ',' or ']' after array element") instead of
"Unexpected end of JSON input".
- After a comma in an object the message is "Expected double-quoted
property name".
- An unterminated string reports the end of the input, and a truncated
\u escape is a "Bad Unicode escape" there.
- A leading zero followed by a digit is an "Unexpected number".
- A misspelled literal names the first differing character, or the end of
the input.
- An unexpected character is named with the source as V8 does: whole when
it is under 21 units, otherwise ten units of context on each side; the
sources "NaN", "Infinity", "undefined" and "[object Object]" name only
themselves; a quote or a number there is an "Unexpected string" or
"Unexpected number".
The new corpus program 7993-json-parse-error-messages pins 76 inputs
against Node, including non-ASCII text before the error, line breaks, and
long sources with split surrogate pairs at the context edges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Materializing the checked view of a JavaScript class instance copies its
property bag through scr_dyn_copy_property_descriptors, which built a key
string and a descriptor object per property and defined each through
scr_dyn_define_property (about 2k instructions per property). Plain objects
on both sides now copy each entry directly with the attributes, value or
accessor pair and own-key order the definition gives; properties the target
already has, non-extensible targets and other receivers keep the general
path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A local number[] candidate whose stores are exact integers but not all
proven int32s (a distance array updated with dist[v] = dist[u]! + 1), and
that is only ever allocated by new Array(n).fill(v), now holds SCR_ELEM_I32
slots when n is at least 4096. A shorter fill creates doubles, which fit a
typical L1 data cache. The first store of a value that is not an int32
widens the array to doubles in place (scr_arr_widen_i32).
An innermost loop that accesses such arrays tests their kinds once on
entry (backend/llvm/int32-kind-loops.ts). Its fast copy reads and writes
them as int32 arrays; a store that would widen one leaves for a copy that
re-executes it with kind tests and finishes the loop on doubles. When the
arrays already hold doubles, a copy accesses them as double arrays. Other
accesses test the kind each time.
Only these arrays take kind tests: every other array, and every program
without one, emits the same code as before.
A local number[] that only compiled element operations reach, and whose
every stored element the int32 slot analysis proves an int32 (never -0),
is now stored as SCR_ELEM_I32: 4-byte int32 slots, with state bytes once
it holds a hole or undefined, as for every array. Element reads give
consumers an exact int32, so indexing with an element skips the double
round trip, and the slots take half the memory.
The runtime's dense slot accessors convert between the int32 slot and the
double, so its double-valued entry points serve int32 arrays unchanged;
growth, copies, fill and length changes use the 4-byte width.
ir/int32-arrays.ts proposes the candidates and inlines their element-reader
helper calls; the int32 slot analysis keeps the proven ones, and
selectInt32Arrays rewrites the module for exactly those, so a program
without an int32 array emits the same LLVM IR as before.
When the last reference to an array dropped, the runtime ran the generic
teardown: scr_rc_destroy's depth bookkeeping, an indirect call to the array
destroy, and a loop over the element slots. Arrays of numbers, booleans and
compact ProgramLifetime references release nothing, and neither do arrays of
other ProgramLifetime instances, whose release does nothing.
- scr_arr_release frees an acyclic array without an ext directly when its
elements are numbers, booleans or compact references, or its descriptor
has SCR_ARR_OPS_RELEASE_NOTHING. Every other array keeps the full
teardown.
- The backend sets SCR_ARR_OPS_RELEASE_NOTHING in the descriptor's flags
word on 64-bit targets for ProgramLifetime element types that are not
compact, plain or as the pointer arm of a nullable union, in programs
without lifetime regions. Registration copies it into a table indexed by
the descriptor's registry id before the id is published. Descriptors
without the flag read 0 and keep the full teardown.
Corpus 7401 drops arrays of ProgramLifetime instances, numbers and booleans
through every array operation, with holes and far indices, next to arrays of
ordinary objects, and matches Node.
The cycle collector widens the growth fraction that schedules full passes
up to 16 times the heap left by the last pass while passes find little.
Lowering a large program leaves hundreds of megabytes of cyclic garbage,
and whether a scheduled pass reclaims it before the build's peak depends
on where the growth rule fires: small changes to the compiler moved the
peak of a tsc-ts build between 1.66 and 2.05 GB.
The native compiler now sets the growth cap to 2 at startup, as
SCR_CYCLE_GROWTH_CAP=2 does, so a full pass runs once the heap has grown
to three times its size after the last one. scr_cyc_set_growth_cap sets
the cap for a program from its own code; the environment variable still
takes precedence, and child processes are not affected.
test_dh.c includes scr_dh.c and still called sd_generate_safe_prime
without its new top_two argument, so dh.test.ts failed to compile. It
now passes false, as the DiffieHellman paths do, and a new case checks
generatePrime's search: the sizes and add forms with one possible
result, top-two-bit safe-prime starts, and checks above OpenSSL's round
count.
An array starts without per-slot state bytes. While it has none, every
index below its length is a present value in dense storage (len <= cap,
no sparse entries) and its hole-free prefix bound `filled` equals its
length, so its slots take 8 bytes per element (1 for booleans) instead
of 9 (2). The first hole or present undefined below the length, a length
set past the elements, or a sparse entry gives it states
(scr_arr_add_states), which it keeps; SCR_ARR_STATES in `spare` records
it, and the element ops id moves up one bit.
Runtime readers take scr_arr_dense_state or scr_arr_scan_states, and
writers keep the elements contiguous from 0 or give the array states
before raising its length. A length set on an empty array with inline
slots reserves its slots and states in one allocation.
Emitted reads keep their shape: they load a state only at or past
`filled`, which an array without states never reaches. A store past
`filled` tests the flag and, without states, appends at the length; push
and pop write a state byte only when the array has states. map, fused
map and slice pipelines that keep holes, and concat's element copy
reserve the result's storage, fill it, and set its length afterwards,
so a result without holes is filled by appends and gets no states. The
typed-view cache refresh moves the flag with the storage it swaps, and
tagged-template arrays carry no state bytes.
An array constructed with a known element count of up to 16 (array
literals, copies, slices and the arrays the runtime builds) keeps its
slots in its own block, as empty arrays built for appends already do, so
it costs one allocation and one free.
scr_arr_make_ops is inlined into its two constructors, so the size class
of scr_arr_new_ops_small folds to a table index. It writes the element
ops id with the other header fields and records ops past the registry
in the array's ext after the storage is set up: the ext pointer precedes
the dense allocation, so inline slots must move out before the ext is
created.
A class carries the collector header when any class in its hierarchy can
be on a cycle, so instances of classes whose own fields hold nothing
cycle-capable were buffered as candidates and walked by every level. Such
an object has no traced children and is never on a cycle.
Their descriptors now name scr_cyc_no_children, and they are allocated
with buf set to SCR_CYC_ACYCLIC. That value has the buffered bit set, so
a release that leaves the object alive makes no call; scr_cyc_is_buffered
is false for it, and an emitted death or publication mark takes the object
out of a candidate buffer only when buf is below -1. A pass that walks
such an object as a child restores the value when it survives.
crypto.createDiffieHellman, crypto.getDiffieHellman and
crypto.createDiffieHellmanGroup return native DiffieHellman and
DiffieHellmanGroup handles backed by mbedTLS bignums. generateKeys,
computeSecret, getPrime, getGenerator, getPublicKey, getPrivateKey,
setPublicKey, setPrivateKey and verifyError follow Node 24.22:
- the modp1, modp2, modp5 and modp14 through modp18 groups, with the
private key widths of Node's OpenSSL 3 path (200 to 400 bits for
modp5 and up, [2, p - 2] for modp1 and modp2);
- explicit primes and generators in every argument form and encoding,
with keys of bits(p) - 2 bits, or below min(2^N, q) when the
parameters equal an OpenSSL named group (RFC 3526, RFC 7919,
RFC 5114);
- generated safe primes with OpenSSL's congruences for the generator;
- verifyError's DH_check bits, computed when it is first read;
- Node's argument, generator, modulus, key range, state and encoding
errors, including the OpenSSL errors' library and reason properties.
Safe-prime generation sieves candidates with the odd primes below a
bound that grows with the prime's length and tests them with
Miller-Rabin rounds over a variable-time Montgomery exponentiation that
only reads public values; key and secret exponentiations stay on
mbedTLS's constant-time path. Primes longer than 8192 bits, property
reads other than verifyError, property writes, Object.prototype methods,
and calls whose output encoding is typed any or matches no @types/node
overload are refused.
The runtime unit scr_dh.c links, with the mbedTLS archive, only into
programs that use DiffieHellman. wasm32-wasi builds refuse it. The ECDH
unit's argument and encoding helpers move to scr_crypto_args.h, which
both units include.
An array's element ops are a registry id in `spare` instead of a pointer
field: descriptors get their id on their first array (emitted descriptors
are writable for it), and a program with more element types than the id
holds keeps the rest in the array's out-of-line storage.
The out-of-line storage (sparse indices, noncanonical numeric properties,
match metadata) is no longer a header field. An array that has it keeps
its pointer in the 8 bytes before its dense allocation and sets
SCR_ARR_HAS_EXT; such an array always has a dense allocation of its own
(inline slots move out, an empty array gets the prefix alone), and every
allocation, reallocation and free of the dense storage accounts for the
prefix.
Loops over elements look the ops up once, and the state lookup below the
dense capacity reads no out-of-line storage.
fs.promises.rm(path, options) accepts the { recursive, force, maxRetries,
retryDelay } literal options that rmSync accepts. It runs the same removal
and rejects with its errors.
Removing a directory without recursive now throws Node's ERR_FS_EISDIR
SystemError from rmSync and fs.promises.rm. A retryDelay outside int32 >= 0
or a maxRetries outside uint32 throws ERR_OUT_OF_RANGE. rmSync checks these
options before the path, and fs.promises.rm checks the path first, as Node
does.
Fixes#1615
Co-authored-by: jlucaso1 <55464917+jlucaso1@users.noreply.github.com>
The frontend's syntax trees and program state refer to each other, so
reference counting frees almost none of them when a build disposes the
frontend. Whether a cycle pass reclaimed them before the build ended
depended on where the collector's schedule happened to be. Passes during
the frontend found its data live and widened the growth fractions, so the
next pass waited until the heap had grown well past its size at disposal,
and the dead frontend and the emitter's garbage stayed until the build
ended.
The native compiler now runs scr_collect_cycles_phase right after it
disposes the frontend, through a host function. The runtime entry point
sweeps, then restarts the growth fractions and the death rule's backoff
from the heap the sweep leaves, as after a productive pass. Under Node
the hook is unset.
Chris Tate