Commit Graph
187 Commits
Author SHA1 Message Date
Chris Tate ddc7fe8b0d Store number-key map entries in 16 bytes and size the direct index near the largest key 2026-10-20 23:01:43 -05:00
Chris Tate 69ef7da7f8 Run death-triggered full collections at the event loop's idle moments (#1522) 2026-10-20 21:48:02 -05:00
Chris Tate 44c586b1ee Run a waiting death pass at the idle moment even after the heap shrank 2026-10-20 21:44:46 -05:00
Chris Tate eac75404d5 Merge main 2026-10-20 19:19:07 -05:00
Chris Tate da0b70bc8d Leave an Error's code to the generic store, and allow test units to include scr_js.h
The generic store keeps a native Error's code slot in step with writes
through its checked view, as it does for name, message, stack and cause,
so the store cache leaves `code` to it. The header invariant accepts
scr_js_*.test.c units, which link the dynamic tier.
2026-10-20 18:10:27 -05:00
Chris Tate 7b510efacd Merge main 2026-10-20 14:13:21 -05:00
Chris Tate fe589fdddf Run SHA-384 and SHA-512 with AVX2, AVX-512VL or the arm64 SHA-512 instructions (#1464) 2026-10-20 08:53:46 -05:00
Chris Tate 32f2b5d28b Run death-triggered full collections at the event loop's idle moments 2026-10-20 08:50:23 -05:00
Chris Tate 1ef2004c9b Compare only the 48 bytes SHA-384 writes in the cross-kernel check 2026-10-20 08:46:56 -05:00
Chris Tate 1d6bd1ea04 Shrink the array header to 48 bytes
ScrArr stores len and cap as 32-bit fields and no longer stores the
state pointer. The dense states share the values' allocation and run
backward from its end: the state of slot i is at
data + cap * (slot width + 1) - 1 - i. Removing slots from the front
(shift, splice(0, n)) advances data and lowers cap, which leaves every
remaining state at its address; the head room size, needed only to free
or reallocate the block, is kept in the vacated slot before data. On
64-bit targets the header goes from 64 to 48 bytes, and from 36 to 32
bytes on wasm32. Array lengths were already limited to 2^32 - 1.

The emitted code loads len and cap as i32 and zero-extends them, and
computes state addresses from data and cap. Runtime loops compute the
end of the states once per call. Local array reads load data once for
both the state and value paths. A local or strict element read whose
index is a loop's induction variable reads slots above the hole-free
prefix through the runtime: an inline state check there makes LLVM keep
-1 - i as a second induction variable, which costs a register in the
loop.

Bulk operations do not read back states they just wrote: an append to
an array whose hole-free prefix reaches its length moves the prefix bound
to the new length, a copy from below the source's prefix writes value
states without reading the source's, and releasing a string array skips
the state checks below the prefix. Copies of up to eight slots and of
short state runs use overlapping word moves instead of a call. Emitted
element access computes the state address only on the state-checked
path. Without these, programs that build many small arrays with split or
slice ran 2-4% slower than with the separate state pointer.

Integer loops over an array's length compare the counter with the i32
length field in 32 bits; the counter never exceeds 2^32 - 1 there. The
array-length stability proof accepts Map and Set reads (get, has, size)
and checked union extractions (`x!`), which run no user code, so loops
that look up maps keep the dense fast copy, which tests the hole-free
prefix once before the loop.

Small constructions with the small allocator request 6 slots, so their
blocks stay 112 bytes, now with 7 inline slots (6 after a collector
header). With 96-byte blocks of 5 slots, programs that free and reuse
many small arrays ran 15-20% slower.

RUNTIME_ABI_VERSION goes from 22 to 23.
2026-10-20 00:21:45 -05:00
Chris Tate 72e22ed0ed Run SHA-384 and SHA-512 blocks with AVX2, AVX-512VL or the arm64 SHA-512 instructions
SHA-384, SHA-512, SHA-512/224 and SHA-512/256 (createHash, createHmac,
crypto.hash, pbkdf2, hkdf) process their blocks with the fastest kernel
the CPU supports:

- x86-64 with AVX2 and BMI2: the message schedule is computed two words
  at a time in vector registers, interleaved with the scalar rounds,
  which rotate with RORX.
- x86-64 with AVX-512VL: the same, with vector rotates and three-input
  XOR in the schedule.
- arm64 with the SHA-512 extension: SHA512H, SHA512H2, SHA512SU0 and
  SHA512SU1.
- Otherwise the portable blocks, now unrolled over a 16-word schedule
  ring.

The CPU and OS checks run once and are cached. The update and final
paths pass all their whole blocks to the kernel in one call.

test_sha_kernels.c checks the FIPS 180-4 SHA-384 and SHA-512 vectors
through every kernel the CPU supports and compares each kernel with the
portable blocks over every length from 0 to 1100 bytes and 1 MiB, whole
and in 1-, 7-, 64- and 100-byte updates. Corpus 5779 compares SHA-384,
SHA-512 and HMAC-SHA512 digests over many lengths and update sizes with
Node; tests/harness/corpus-costs/ records its plain and sanitized costs.
2026-10-19 23:22:23 -05:00
Chris Tate 7e0106722b Widen an unjudged backlog threshold with its level's growth fraction
The backlog threshold of a level backs off by doublings as backlog passes
find little to free, but only passes that had candidates count. A program
whose mature passes have no candidates while its heap grows keeps the base
threshold, a quarter of the live heap, while those same passes widen the
level's growth fraction. When it then reads the structure it built, the
mature buffer fills with live objects and the first backlog pass walks a
quarter of the live heap to find nothing. In tsc-ts's program build on
Sentry, that pass ran inside the loader walk and took 9 to 18 ms.

Until a pass at a level has had candidates, the backlog threshold now uses
the level's growth fraction doublings. The first pass with candidates
judges the level, and the threshold follows its own yield from then on.
2026-10-19 23:18:55 -05:00
Malte Ubl a10ce583d7 Merge remote-tracking branch 'origin/main' into opt/tier-s3-jsval 2026-10-19 05:43:49 -07:00
Chris Tate 8ea9dd3206 Merge main 2026-10-19 04:16:52 -05:00
Chris Tate b9ca34887f Merge main 2026-10-18 21:24:49 -05:00
Malte Ubl f18e804ffd Merge remote-tracking branch 'origin/main' into opt/tier-s3-jsval
# Conflicts:
#	packages/compiler/src/backend/llvm/shapes.ts
2026-10-18 15:58:49 -07:00
Malte UblandClaude Opus 5.5 89bca68cbe Compute with NaN-boxed jsval values in the dynamic tier
Stage S3 of the two-tier design: dynamic-tier functions compute with
jsval, a new IR type holding a 64-bit NaN-boxed word whose numbers,
booleans, undefined and null are immediates, so a number between two
dynamic operators is no longer a heap node.

- IR: the jsval type and js.* builtins (operators and the boundary
  conversions fromDyn/toDyn/fromF64/fromBool/fromStr/fromBigInt/toF64).
  The jsval pass rewrites dynamic-tier functions after tier assignment:
  operator chains, comparisons, number checks and truthiness of operator
  results run on jsvals; dyn locals become jsval locals when that saves
  boxing; dynamic functions called only from dynamic functions return a
  jsval. IR version 17 serializes each function's tier.
- Runtime: scr_js.h and scr_js_value.c behind the dynTier feature
  (invariant I3); static runtime units never include scr_js.h.
- Backend: number cases inline (immediate doubles and NUM cells), always
  inlined jsval retain/release; 32-bit targets keep boxed cells.
- Interprocedural analyses see jsval conversions as the dyn forms they
  replace, so no fact about a static function changes.

Static functions are untouched: typed programs and the self-hosted
compiler's static functions compile to byte-identical LLVM.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 15:11:32 -07:00
Malte UblandClaude Opus 5.5 98d4611ef6 Merge origin/main into opt/gc-closure-cycles
Conflicts with #1286's merged form and the death rule (74702ac4b):
- scr_cycle.c: keep the death rule's per-thread fields and drop the
  running buffered total this branch removed (the slow path sums the
  buffers); scheduling decisions route bypassed appended candidates and
  then note the live count's fall.
- test_cycle.c: keep both new checks (inline buffering, death brings a
  full pass).
- The self-capture corpus program moves from 5990 to 5985 (5990 is now
  5990-worker-uncloneable.mjs), with its order baseline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 13:49:20 -07:00
Malte UblandClaude Opus 5.5 d7fcc9287c Merge origin/main into opt/gc-nursery
main now has #1286 and later collector changes. Resolved in scr_cycle.c:
the scheduled collection point collects the nursery, then notes the
live-count fall (death rule). Semantic fixes for the new main code:
a full pass's markGray no longer overwrites a young object's `buf` (its
nursery index) with the component number, and a nursery collection
refreshes the death rule's live reading at its end so the young garbage
it frees does not count as deaths outside passes. Formats the runtime
pack matrix test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 13:14:30 -07:00
Malte UblandClaude Opus 5.5 989d0d6bc2 Buffer nursery cycle candidates on the release fast paths
Every release that left a headered object alive and unbuffered called
scr_cyc_on_release, which looked up the type's bypass statistics, pushed
the object, kept a running total and compared it with two triggers. The
nursery case now has a fast path that appends the object to the nursery
buffer itself, while the runtime-maintained window says the append can
neither make a collection due nor grow the buffer, and the live count is
below the allocation trigger:

- scr_rc.h exports the candidate buffers (scr_cyc_buffers: the three
  buffers, the window and the allocation trigger) where the collector
  state is ordinary globals, and scr_cyc_append. The runtime's own
  release paths (scr_cyc_released) append inline, and scr_cyc_on_release
  tries the append first.
- Optimized emitted code buffers through one sc_cyc_append helper per
  module, called with preserve_mostcc, so release fast paths keep the
  size of the plain call they had (inlining the append into every
  release fast path grew the self-hosted compiler's text by 4.6%) and
  their callers keep their registers. Its runtime call goes through a
  second preserve_mostcc function, which keeps the register saves off the
  append path.
- scr_cycle.c keeps the window: recomputed where buffers, capacity or the
  trigger change in bulk, lowered by one on older buffering and nursery
  deaths, 0 during passes. The running total is gone; the slow path sums
  the buffers.
- An appended candidate whose type bypasses the nursery would have been
  buffered past it. scr_cyc_route_bypassed moves such candidates before
  any scheduling decision or pass reads the buffers, consuming the bypass
  counts as scr_cyc_on_release does (skipped while no type bypasses).

test_cycle checks the window (the release that reaches the trigger still
collects) and the routing of inline-buffered candidates of a bypassed
type; its mature-bypass probe routes before reading generations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 11:51:00 -07:00
Malte UblandClaude Opus 5.5 19c742834e Add an experimental young-in nursery for compiled classes and records
SCRIPTC_YOUNG_NURSERY=1 at compile time makes an executable allocate
traced, program-owned class instances and records young and link the
collector unit built with SCR_YIN (a runtime pack variant of
scr_cycle.c; libraries never get it). The emitted constructor lists the
object in a per-thread nursery and marks its header: generation
SCR_CYC_YOUNG and its nursery index under the buffered bit, so every
existing release path treats it as a buffered candidate (no call while it
stays alive; scr_cyc_on_dead takes it out of the nursery at its death).

A nursery collection (whenever 4096 young objects are live, at collection
points, before full sweeps and limited passes) counts each young object's
references from other young objects, marks from the objects whose count is
below their reference count (referenced from outside the nursery: stacks,
globals, older objects, runtime structures, FFI, fibers, coroutine
frames), frees what stays unmarked, and gives up the dead set's outgoing
edges after the teardowns. The counts are recomputed at every collection
instead of maintained by a store barrier, so soundness rests only on the
invariants trial deletion already needs. Survivors join trial deletion's
nursery and outside roots become candidates; trial deletion walks young
objects it reaches as nursery-level objects. Per-type admission statistics
send types that do not die in young cycles back to trial deletion for a
while. Publication takes young objects out of the nursery. The RC audit
build checks nursery membership and counts at every collection.

With the flag off, the compiler emits and links exactly what it did
before (249 programs and the self-hosted compiler compared
byte-identical). Measured with the flag on: cg-rings -38%, cg-pairs -27%,
the runtime suite geomean -0.3% (published-graphs +3.4%), the self-hosted
compiler neutral; shapes whose cycles go through arrays or closures
(phase 2) pay without gaining.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 11:48:50 -07:00
Chris Tate 20e5a2d7c6 Write borrowed string slices into stack buffers (#1203) 2026-10-18 11:24:42 -05:00
Chris Tate 74702ac4be Run a full cycle pass when a structure dies outside passes
Full passes are scheduled by heap growth, and the fraction they wait for
widens while growth-triggered passes find nothing, up to 16 times the heap
after the last full pass. That keeps a large live heap that keeps growing
(a compiler checking a large program) from being walked again and again. But
growth alone cannot tell such a heap from one that keeps garbage: a program
that drops a large structure and builds another, as a compiler rebuilding in
watch mode does, holds the dead structures' cyclic parts until the heap has
grown to 17 times its size. Ten rebuilds of benchmarks/regions/compiler.ts
without lifetime regions reached 1.3 GB that way.

The acyclic parts of such a structure go by reference counting, and that
shows: the live count falls between passes. The collector now sums those
falls since the last full pass. Once they reach 1/32 of the heap that pass
left and have stopped (a structure torn down by reference counting falls
over several checks, and a pass in the middle would find its rest still
referenced), a full pass is due as soon as the heap has grown the base
quarter past it, whatever the growth fraction. One that finds fewer older
objects than had died doubles the fall the next one needs, up to 256 times,
and a productive one restores it.

A full pass that the growth rule did not choose (this one, a backlog pass, a
scheduled-age pass) now raises the growth baselines of the levels it walked
but does not lower them. After such a pass reclaims a dropped structure, the
heap regrowing to where it was is the program rebuilding what it had. From
the low point, growth passes at both levels would walk the rebuilt live heap
and find nothing. On the ten rebuilds that was a third of the collector's
added time. A heap that only grows never falls, so its schedule is
unchanged. The checks run where the release path and the event loop already
decide which pass to run.
2026-10-18 05:42:55 -05:00
Malte UblandClaude Opus 5.5 4aa5d6cbf8 Give dense arrays head room so shift, splice(0, n) and unshift runs cost amortized O(1)
shift() moved every remaining element down one slot, making queue drains
quadratic (100k elements: 0.85 s; 1M: no result in 30 s). Dense arrays of
8-byte slots with their own allocation now remove front elements by
advancing data/present and lowering cap; the vacated slots stay in the
allocation as head room. A flag in `spare` marks it and its size is
implied by the layout (present == data + cap + head), so every reader,
including inline element access in emitted code, is unchanged. Growth
moves the elements back to the start of the allocation (in place only
when that leaves half the moved count free, else geometrically), a
drained array starts over at the front, unshift reuses head room, and an
array unshift already had to move once reserves head room for half its
length. scr_arr_outline drops head room before compiled storage swaps.
Booleans, inline slots, sparse arrays and static arrays keep the old
behavior.

Scaling probes: shift/BFS/splice(0,1)/deque drains and unshift runs go
from exponent ~2.0 to ~1.0 (shift 1e5 847 ms -> 1.6 ms); runtime bench
geomean 1.0045 (neutral).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 12:13:53 -07:00
Malte UblandClaude Opus 5.5 06b0bb2931 Write borrowed string slices into stack buffers
A slice read only by lending consumers (string equality and ordering,
length/charCodeAt/indexOf/includes/startsWith/endsWith, Number/unary
plus/parseInt/parseFloat, typed string-keyed Map/Set get/has/set/add/
delete keys, string switch discriminants, and string parameters their
callee only lends) is written by scr_str_slice_scratch into a per-site
stack buffer as an immortal ASCII ScrStr marked as scratch (cap < len),
instead of a heap allocation. Stable locals initialized by such a slice
qualify when every read lends. Map/Set insertion stores a heap copy of a
scratch key, and probes cache the key hash in the scratch header, so the
counting idiom allocates only for new keys. Longer, non-ASCII, and
interned results keep the owned slice; coroutines and debug builds are
unchanged. Slice bounds are normalized without a libm trunc call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 12:13:11 -07:00
Chris Tate db94b2d257 Add fs.cpSync, fs.cp and fs/promises cp
fs.cpSync, fs.cp and fs/promises cp compile with every option:
dereference, errorOnExist, filter, force, mode, preserveTimestamps,
recursive and verbatimSymlinks. They follow Node's two implementations.
cpSync follows cp-sync.js over the fs binding: copy errors are plain
Errors with a code, std::error_code failures throw an ErrnoException,
and links inside a copied directory point at their canonical targets.
fs.cp and fs/promises cp follow cp.js, whose copy errors are
SystemErrors with code, info, errno, syscall and path.

Options and paths validate in Node's order with Node's errors. filter
runs for the top-level paths and each entry in Node's order; an async
filter must return a promise that is already settled, and a pending one
throws SC2020. The callback and promise forms complete the copy before
the callback runs or the promise settles. Windows builds throw SC2020.

The copy lives in the new scr_fs_cp.c. Directory walks keep their frames
on the heap, so deep trees also copy on a fiber's stack. File contents
copy with sendfile on Linux, and COPYFILE_FICLONE uses FICLONE
where it exists. ScrError gains the SystemError context and the
ErrnoException key order, with matching fields in the compiler's Error
layout. The stored fs.cp callback uses the same implementation.

Corpus 5567-5571 cover the sync, callback and promise forms, errors,
symlinks and 1500-level trees.
2026-10-16 21:06:22 -05:00
Chris Tate c268ec5c8b Hash SHA-1 and SHA-256 blocks with the SHA extensions on x86-64
On x86-64 CPUs with the SHA extensions (and SSSE3 and SSE4.1), SHA-1 and
SHA-256 process their blocks with SHA1RNDS4/SHA256RNDS2 and the
message-schedule instructions, several blocks per call. The CPU check
runs once and is cached; other CPUs, other architectures and wasm32 keep
the portable blocks, and both give the same digests. createHash,
crypto.hash, createHmac, pbkdf2, hkdf and every other caller of the two
digests use the new path.

sha-kernels.test.ts runs both block implementations over known vectors
and every length from 0 to 1100 bytes and 1 MiB, whole and in 1-, 7-,
64- and 100-byte updates. Corpus 5763 compares the digests, HMAC, PBKDF2
and Hash.copy() with Node across block-boundary lengths and update
sizes.
2026-10-16 08:43:18 -05:00
Chris Tate bb8af06bd0 Free large cycle-collector work vectors after a pass (#1114) 2026-10-16 08:20:54 -05:00
Chris Tate f19b2cffe1 Make regex exec results, parseInt of captures, and UTF-8 file reads cheaper (#1128) 2026-10-16 08:11:29 -05:00
Chris Tate 9aa2697acb Merge main 2026-10-16 07:54:59 -05:00
Chris Tate 9f2cd0850a Shrink Map and Set headers and small entry arrays (#1117) 2026-10-16 07:49:10 -05:00
Chris Tate 92032a1b11 Shrink Map and Set headers and small entry arrays
A Map or Set header held six adapter pointers, the union key table and
size_t counts: 168 bytes, 176 in its size class. The adapters and union
key kinds now live in an immutable per-type ScrMapOps descriptor that the
compiler emits once per collection type, a traced byte replaces the two
trace-pointer tests, and the counts and capacities are 32-bit. The header
is 88 bytes.

The entries array starts with one slot and grows to four, then doubles,
and maps of up to 8 entries scan their cached hashes instead of keeping a
bucket table. Number-key maps still move to the direct index from four
entries.

One million maps held at once (RSS per map): empty 188 -> 100 bytes,
1 entry 286 -> 125, 4 entries 286 -> 198, 8 entries 450 -> 295.
2026-10-16 07:27:17 -05:00
Chris Tate df74099468 Free released regions without a pass over the whole heap
Every release of a LifetimeRegion, and every retry at region creation, ran
scr_collect_cycles(): full cycle-collector passes over the thread's heap, to
a fixpoint, before checking any region, in case dead cycles of ordinary
objects held counted references into it. Freeing a region ran them again
whenever the owned ordinary objects left counts behind after the region
released its fields. Next to a large heap of candidates (a type checker
beside per-file emit regions) those passes cost far more than the regions.

A retry now checks the due regions first. A region whose references are all
explained frees without a pass; only when one would be kept does the heap
get the full pass, after which every due region is judged as before. The
first check leaves a region it cannot free as it was, so the backoff, the
kept report and SCRIPTC_REGION_DEBUG see only the check after the pass. It
keeps what its walk found: a region whose counted references the pass did not
change is judged from that, without walking it again. A pass that freed a dead
cycle referring only to an owned object leaves the count unchanged; such a
region waits for its next attempt.

After a region releases its instances' fields, what can still hold counts
is the owned ordinary objects' cycles. Each has a member that lost a
reference in those releases and is buffered as a candidate, and the release
walk entered every cycle member. The cycle collector gains a pass limited to
a set of candidates (scr_cyc_collect_set): it takes the set's members out of
the buffers up to a given generation, reading only the buffers' pointers, so
the candidates it leaves are not touched, and it walks at full depth without
moving the schedule's baselines. A dead object such a pass reaches through a
candidate it did not take leaves its buffer before it is freed. Freeing a
region runs these passes over the walk's objects from the youngest buffer
out, and the whole heap only if counts remain.

SCRIPTC_REGION_STATS also prints, per thread at exit, how many full passes
over the heap regions needed.
2026-10-16 07:01:01 -05:00
Chris Tate 485201d804 Free large cycle-collector work vectors after a pass
A collection pass copied every buffered candidate from the generation
root buffers into its candidate list, so it held two buffers of every
candidate, and each work vector kept its largest size for the rest of
the program.

A pass now takes the fullest root buffer as its candidate list and
appends the others; that generation gets the previous candidate buffer,
empty. The vectors a pass fills and leaves empty (pending, white,
promote, restore and cross-generation lists) record how many entries the
pass held at most. When a pass ends, any of them that holds more than
65536 entries is freed if the pass used less than a quarter of it, and so
is the candidate list's buffer when this pass's candidates filled less
than a quarter of it. Passes of a steady size keep reusing their
buffers; a spike in one pass no longer keeps its buffers for the rest of
the program.

When an older generation's buffer is the fullest, candidates are
visited, and a pass's garbage is freed, in a different order; the set
of objects a pass frees does not change.
2026-10-16 06:03:55 -05:00
Chris Tate 0d59abe143 Pool small Buffers as Node does
Buffer.from of strings, arrays and Uint8Arrays, Buffer.allocUnsafe and
Buffer.concat now return views into a per-thread pool slab of
Buffer.poolSize bytes (64 KiB by default) when the size is below
Buffer.poolSize >>> 1, following Node's allocate, fromStringFast and
fromArrayLike: 8-byte aligned offsets, a new slab when a request does not
fit, and the four-bytes-per-unit reservation for strings. Buffer.poolSize
can be read and set, and Buffer.allocUnsafeSlow returns an unpooled Buffer.
A slab lives while any of its views does. Its ArrayBuffer cannot be
transferred, and structuredClone of a typed array view copies the whole
ArrayBuffer and keeps the view's offset. Buffer.alloc, allocUnsafe and
allocUnsafeSlow validate sizes with Node's ERR_OUT_OF_RANGE error.

The first pooled allocation finds the startup slab, which has the
default 64 KiB size. A request it cannot hold gets a slab of the current
Buffer.poolSize, as in Node, and the room is checked after every new
slab, so a Buffer.poolSize raised before the first pooled allocation
cannot place a reservation past the end of a 64 KiB slab. A reservation
larger than a fresh slab traps instead of writing past it.

Corpus 5531 (typed) and 5532 (untyped) raise Buffer.poolSize before the
first pooled allocation and allocate just under the new threshold
through each pooled path, each case in a fresh child process, and also
shrink Buffer.poolSize and change it between allocations. The
node-types fenced fixture no longer lists Buffer.poolSize, which now
compiles.
2026-10-16 01:39:10 -05:00
Chris Tate a87bf8b4b0 Take published objects out of the candidate buffers during the walk
publish() used to remove each published object from this thread's
cycle-collector candidate buffers after the walk, which loaded every
object's collector header a second time. On graphs larger than the
cache that second pass took about a third of the publish.

scr_pub_mark now takes a buffered object out of its buffer when it marks
it, while the header is in cache: the slot is cleared and its index
recorded. When the publish ends, scr_cyc_pub_fill closes the empty slots
with entries from the ends of the buffers. A refused publish puts each
taken object back in its slot first. After the walk, the log is read only
for dyn containers, the RC audit and LeakSanitizer.

The emitted walkers test null and immortal references (parent links,
values published before) before calling scr_pub_push, and the string
walker checks for ASCII 32 bytes at a time.

Adds the published-graphs runtime benchmark, a cycle collector test for
the take, restore and fill hooks, and corpus program 5702.
2026-10-15 22:19:05 -05:00
Chris Tate d30d2619d7 Support crypto.createCipheriv and createDecipheriv (#986) 2026-10-15 21:30:30 -05:00
Chris Tate ab0d4f0cd3 Give small exact arrays room for one append and keep growth on constant slot sizes
Exact constructions (literals, slice, concat results) round their request
to 16 bytes before the size class, so arrays of 2, 4 and 6 slots get one
spare slot and an append right after a copy no longer reallocates. Arrays
of 8-byte slots with their own storage and no sparse entries grow below
the dense cutoff on a path with a constant slot size, and dense copies of
8-byte slots use a constant size.
2026-10-15 20:14:59 -05:00
Chris Tate b00d65b8be Merge main 2026-10-15 19:57:06 -05:00
Chris Tate 982ebac139 Accept typed-ref arguments in createCipheriv and the cipher methods
createCipheriv, createDecipheriv, update(), setAAD() and setAuthTag()
now read the value a typed-ref box holds when it is a string, view,
ArrayBuffer or KeyObject. ERR_INVALID_ARG_TYPE names the class of a
class instance that arrives through an untyped value, as Node does,
instead of Object. An unknown output encoding that is not a string is
rendered as Node's %s formats it (util.inspect at depth 0, or the
object's own toString); a function, or an instance of a class that
defines toString, throws a coded SC2020 error. Cipher programs link the
inspect unit for that rendering.

5250 passes each argument through `any`, including class instances,
records and arrays that Node rejects, and 5254 covers output encodings
that are not strings from JavaScript.
2026-10-15 18:47:02 -05:00
Chris Tate ce2e567922 Collect dropped cyclic graphs while they are in cache and skip redundant walks
A candidate release now also runs a pass once the live cycle-headered
count has grown by 4096 objects since the last pass. A dead graph is
usually reachable from one candidate (its root), so the candidate count
alone let a few hundred dead graphs pile up before a pass, which then
walked them after they had left the cache.

markGray records the objects it grays and counts the ones still
externally referenced after the trial decrements. When none is, every
gray object is garbage, so the pass whitens the recorded set and skips
scan. markGray also records each edge it skips into an older generation,
so a restricted pass pins those targets without tracing the white set a
second time.

Adds the cyclic-graphs runtime benchmark and corpus programs 5700 and
5701.
2026-10-15 18:26:48 -05:00
Chris Tate 59cdda3930 Merge main 2026-10-15 09:20:01 -05:00
Chris Tate 0b468e40a1 Add fs.globSync, fs.glob and fs/promises glob
The three glob entry points run a port of Node 24's
lib/internal/fs/glob.js walk in scr_glob.c: the same LIFO queue of
(path, patterns) items over the sorted directory listings, the same
lstat, stat, readdir, realpath and seen-pattern caches, the same result
set in insertion order, and the same exclusion checks, so matches come
out in Node's order. Patterns compile with the existing minimatch port.
glob_match is split into a compile step and a test step so the walk can
reuse each compiled pattern and test it segment by segment.
nocaseMagicOnly is now an option, because exclude patterns turn it off.

Patterns (a string or string array) and options (cwd as a path or
file: URL, exclude as patterns or a function, followSymlinks,
withFileTypes) are validated in Node's order with Node's errors,
including the internal assertion Node hits for a non-string cwd with
exclude patterns. withFileTypes results are Dirent rows: name,
parentPath and the type probes. parentPath follows Node's rule for each
form: the joined path for globSync, the resolved path for fs/promises
glob.

fs.glob checks the callback and validates synchronously, then delivers
string[] or the walk's error through the filesystem callback queue.
fs/promises glob lowers as the source of a for await loop. The walk runs
when the loop starts, and the matches found before an error bind before
the error is thrown, as Node's async iterator yields them first. The
older glob(3)-based fs.glob branch is removed.

Corpus 5448-5452 and a node-types fixture check the behavior against
Node.
2026-10-15 07:59:19 -05:00
Malte Ubl b275f47fb3 Read UTF-8 files straight into string storage
readFileSync(path | fd, "utf8") (any ASCII-case utf8/utf-8 spelling and
the empty default) used to read into a Buffer, scan it for UTF-8
validity, then copy it into a new string while scanning it again for
ASCII. It now reads directly into the final string allocation, scans
each read chunk for ASCII while it is still in cache, and validates the
remainder in place (scr_str_adopt_utf8_lossy). Only malformed input is
repaired into a new string, with the same WHATWG maximal-subpart U+FFFD
replacement as Buffer#toString; a BOM is kept.

The raw-string readers behind fs.promises.readFile(path, "utf8") and
readFileSync(fd, "utf8") now decode the same way, so malformed bytes and
encoded surrogates become U+FFFD as in Node instead of passing through.

The ASCII prefix scan tests 32 bytes per step (scr_bytes_ascii_span),
which also speeds up Buffer and TextDecoder UTF-8 decoding of long ASCII
runs.

The corpus program covers empty files, BOMs, every malformed-subpart
class, truncation at EOF, chunk-boundary offsets, large ASCII and
multibyte files, /proc pseudo-files, fd and promise reads; the runtime
unit test checks the in-place decode under ASan and the RC audit.
2026-10-15 04:46:55 -07:00
Chris Tate 0df4436960 Merge main 2026-10-15 04:02:33 -05:00
Chris Tate a08fbd2746 Merge main into createCipheriv support 2026-10-15 03:12:04 -05:00
Chris Tate b249cc6bae Regenerate the runtime header split on current main 2026-10-15 03:01:21 -05:00
Chris Tate 5336966743 Merge main into createCipheriv support
Keeps the cipher support together with the hash, random and
timingSafeEqual surface added on main.
2026-10-15 02:22:02 -05:00
Chris Tate 4d54f6c4d8 Split scr_runtime.h into per-module runtime headers
scr_runtime.h now includes 48 module headers (scr_core.h, scr_rc.h,
scr_string.h, scr_dyn.h, scr_http.h, ...) in dependency order and holds no
declarations. Each module header holds whole sections of the former file,
includes the headers whose names it uses, and compiles on its own.

Declarations, macros, struct layouts and inline bodies are unchanged. Five
blocks move to join their module: the class-hierarchy vtable header (into
scr_error.h, after the string core), the node:timers/promises pair (into
scr_async.h), the JSON.parse entry points and the JSON output buffer (into
scr_json.h, after the dyn declarations) and the EventTarget block (after
scr_json.h). No moved block uses or declares a name that a block it moves
past uses or declares. The string methods keep their position after the
class objects in scr_string_methods.h, because their struct ScrDyn
parameters rely on the file-scope struct ScrDyn declaration there.

The runtime ABI scan reads scr_runtime.h with its module headers expanded
in include order. A runtime test compiles every module header on its own.
scripts/runtime-header-hunks.mjs retargets a diff of the single-file
header onto the module headers.
2026-10-15 02:06:23 -05:00
Chris Tate 03e78caffc Initialize the worker exception alert and check macOS COMMON symbols on every host
scr_exc_alert, the process-wide exception alert of worker executables on
targets without the ELF per-thread pending word, was a tentative
definition. That branch compiles only for Mach-O, COFF and WASI, so the
macOS runtime build emitted it as a COMMON symbol, which library
localization keeps global, and the COMMON-symbol test failed on macOS
while passing on Linux.

Initialize it explicitly, and have the test also compile every runtime
source for aarch64-macos through zig cc when zig is on PATH and the host
is not macOS, so declarations in __APPLE__ branches are checked on Linux
too.
2026-10-15 01:54:52 -05:00