Commit Graph
389 Commits
Author SHA1 Message Date
Chris Tate 85439ae670 Merge main 2026-10-16 19:54:32 -05:00
Chris Tate 2e52dcd35c Add Readable.from over Sets, iterators and class iterables (#1074) 2026-10-16 19:30:21 -05:00
Chris Tate d9229b044f Inline small runtime functions into release executables (#1130) 2026-10-16 19:16:56 -05:00
Chris Tate 5a091f050f Run the zlib one-shot forms on the stream engine (#1139) 2026-10-16 19:11:18 -05:00
Malte Ubl f4ce6910c7 Inline small runtime functions into release executables
Release executables now import small runtime functions from runtime
bitcode, as speed executables do, so LLVM can inline them into program
code. Release units keep the runtime compiler's own objects, unchanged,
and additionally ship the bitcode of the same compilation reduced by the
helper's prune-import. Their unit-local symbols stay local, so only
bodies that need none of them can be imported, which keeps the import
small: about 1.3 MB per program.

Only unsanitized programs small enough for one codegen partition import.
Every partition reads the bitcode again; on the Effect fixtures (64
partitions) import cost +2..18% compile time and +4.3..4.9% size, so
partitioned release builds stay exactly as before. Speed is unchanged.

Measured against main in Linux Vercel Sandboxes, same-VM interleaved
medians (bench-runtime --layouts=4 --runs=16), on two VMs:

- bench-runtime (31 workloads): -1.3% (8 faster, 0 slower) and -1.0%
  (1 slower, typed-array-scan-workers +3.2%); size +0.0%
- js-compiled suite (22 programs): -2.2% and -1.8%; size -0.1%
- cold builds of all 53 programs, interleaved: +3.1%; helper time per
  build +40..235 ms
- Effect fixtures: byte-identical to main
2026-10-16 14:03:52 -07:00
Malte Ubl 8f5ec2d94f Report record width copies at assertions, assignments and array elements
SC6001 only looked at call arguments and typed initializers, and only at
strict field subsets: a target that adds optional properties, the empty
record type, `r as T`, a later `v = r` into a typed binding, and records
stored as elements of a typed array literal copied silently.

The scan now reports a copy whenever the property sets differ, covers
those sites, and reports a class instance that reaches a plain record
type which does not keep its identity (the remaining boundary: interfaces
of only generic methods). Every corpus program still reports no
divergence.
2026-10-16 14:02:02 -07:00
Malte Ubl 6c9c39b3ab Keep class instance identity through structural record views
A class instance passed, stored, returned or asserted as an anonymous
object type or an interface it does not `implements` was copied into a
native record, so writes, identity checks and inspection through the view
missed the instance (`function add(o: { count: number }, n: number)
{ o.count += n }` left a Counter at 0).

ClassProtocols now scans flow sites (arguments, typed initializers and
fields, defaults, returns, arrow bodies, yields, assignments, array
elements, property values, assertions through `unknown`, and annotated
callback parameters) and marks every program object type a class
instance flows into, then propagates the mark along flows between
structural types. Marked types are class-backed protocols: their values
are checked views that keep the original object, like implemented
interfaces already were. Only sites whose source carries identity query
the checker for their target, and a checker panic skips the site.
Assertions read or written member-wise in place keep their direct field
access, and `typeof C` is not an instance.

An assertion of a class instance to such a view boxes it like an
implicit flow, and `view.n++` on a number member lowers to the checked
expanded form.
2026-10-16 14:02:02 -07:00
Chris Tate 01f25f6b3e Lower util.promisify over the fs callback functions (#1138) 2026-10-16 09:37:58 -05:00
Chris Tate d533fd7ce9 Merge main 2026-10-16 09:33:08 -05:00
Chris Tate f4445f1540 Keep undeclared keys of checked JSON records; lower compound assignment with untyped operands (#1123) 2026-10-16 09:20:49 -05:00
Chris Tate ed5ab6796b Run the zlib one-shot forms on the stream engine
deflateSync, inflateSync, gzipSync, gunzipSync, deflateRawSync,
inflateRawSync and unzipSync, and their callback forms, accept Node's
options and run Node's processChunkSync over the zlib stream engine.

- Options (windowBits, level, memLevel, strategy, dictionary, chunkSize,
  finishFlush, maxOutputLength, rejectGarbageAfterEnd) are validated at
  run time with Node's ladder, shared with the stream classes; info is
  refused at run time.
- Errors carry Node's errno and code; maxOutputLength reports
  ERR_BUFFER_TOO_LARGE and rejectGarbageAfterEnd
  ERR_TRAILING_JUNK_AFTER_STREAM_END. The default-options decoders also
  read every gzip member.
- Untyped JavaScript inputs may be strings, TypedArrays, DataViews or
  ArrayBuffers; other values throw Node's ERR_INVALID_ARG_TYPE after the
  options are checked.
- Results are Buffers, as in Node: they convert to their text.

Corpus programs 5378, 5379 and 5415 cover the sync, callback and untyped
input forms.
2026-10-16 09:18:32 -05:00
Chris Tate daa5cc256f Lower util.promisify over the fs callback functions
promisify(fs.X) over a named or namespace import of node:fs lowers as
the fs/promises member of the same name for access, appendFile, chmod,
chown, copyFile, lchown, link, lstat, lutimes, mkdir, mkdtemp, readFile,
readdir, readlink, realpath, rename, rm, rmdir, stat, statfs, symlink,
truncate, unlink, utimes and writeFile. For these, Node's promisified
callback form settles with the callback's single result and rejects
with the same errors as the fs/promises call. Calls through the bound
function accept the forms the fs/promises lowering accepts and refuse
the others by name.

fs.open (an fd number where fs/promises.open gives a FileHandle),
fs.exists (a custom promisify) and fs.read/fs.write (multi-value
callbacks) remain refused.
2026-10-16 09:14:35 -05:00
Chris Tate 241006cff3 Add fs read and write streams (#1089) 2026-10-16 09:02:48 -05:00
Chris Tate 21c2340ef6 Support the dgram socket options and disconnect() (#1122) 2026-10-16 08:39:23 -05:00
Chris Tate 451e925b12 Merge main 2026-10-16 08:32:21 -05:00
Malte Ubl dff726715b Keep undeclared keys of checked JSON records
`const parsed: { score: number }[] = JSON.parse(doc)` printed a different
checksum from Node: the checked conversion (and the schema-directed
JSON.parse fast path) stored only the declared fields of each object in
its native record, so JSON.stringify(parsed) silently dropped `id`,
`name` and `active`, and every other reflective surface agreed.

A closed record converted from an ordinary object now keeps the source's
own properties beside it when the object has undeclared keys or holds the
declared ones out of declaration order. The runtime keeps a per-thread
table from the native record to a view of the source's own keys
(placeholders for declared keys, which the record still owns), released
through the native dispose hook when the record is freed:

- the checked record builder notes its source (scr_record_source_note);
- the schema parser parses undeclared members into the view instead of
  declining, so typed parses keep their one-pass fast path;
- typed JSON.stringify writes such records in source key order, declared
  fields through a generated per-shape field writer and the others through
  the dyn writer;
- record capsules adopt the view and refresh it in place, so untyped reads
  and writes see and persist undeclared properties;
- util.inspect and Object.keys/values/entries answer through the live view
  (values/entries check the extra values into their static result type and
  throw the checked-conversion TypeError when one does not fit).

Copies made by object spread, Object.assign, rest destructuring or
conversion to another record type still hold only the declared fields;
the limitations page documents this.
2026-10-16 06:31:51 -07:00
Chris Tate a5da46267b Merge main 2026-10-16 08:31:20 -05:00
Chris Tate a09990192e Add tlsSocket.getProtocol, getCipher, alpnProtocol and tls.DEFAULT_MIN_VERSION/MAX_VERSION (#1054) 2026-10-16 08:23:42 -05:00
Chris Tate 21c0e17178 Merge main 2026-10-16 08:17:52 -05:00
Chris Tate 0cad2fe551 Name the TLS ciphers limitations partial after its heading
The partial for the "TLS ciphers" section was tls-cipher.mdx, while the
limitations tooling names a section's partial after its heading's slug
(tls-ciphers). It is renamed, so a join and split of the page leaves the
docs unchanged.
2026-10-16 07:48:05 -05:00
Chris Tate 4adc3e6f2a Support the dgram socket options and disconnect()
dgram.Socket's setBroadcast, setTTL, setMulticastTTL,
setMulticastLoopback, setMulticastInterface, addMembership,
dropMembership, get/setRecvBufferSize, get/setSendBufferSize,
getSendQueueSize, getSendQueueCount and disconnect were refused at
compile time. They now lower to scr_dgram.c and match Node:

- The TTL setters and setMulticastLoopback return their argument. The
  value checks run first: a TTL outside 1-255 (0-255 for multicast,
  after ToInt32) throws "setTTL EINVAL" with code, errno and syscall.
- A socket that has not been bound has no descriptor yet, so the
  options throw EBADF, and the buffer size methods throw Node's
  ERR_SOCKET_BUFFER_SIZE SystemError.
- After close(), the members that read Node's null handle throw its
  TypeError ("Cannot read properties of null (reading 'setTTL')"), and
  setMulticastInterface and the membership calls throw "Not running".
- Buffer sizes must be uint32 values (ERR_SOCKET_BAD_BUFFER_SIZE), 0
  reads the current size, and sizes past the int32 range fail as EINVAL.
- IPv6 interfaces and groups go to the IPv6 socket options, so a udp4
  socket answers ENOPROTOOPT as in Node.
- disconnect() throws ERR_SOCKET_DGRAM_NOT_CONNECTED until 'connect' was
  emitted, and dissolves the association with an AF_UNSPEC connect.

addMembership and dropMembership before bind() throw SC2020, where
Node binds the socket to an ephemeral port first; the limitations page
documents it, along with the missing info object on the buffer size
SystemError. Corpus 5655 and 5656 and the node-types fixture
dgram-options.ts cover each method.
2026-10-16 07:34:12 -05:00
Chris Tate 60623086a3 Lower util.promisify(child_process.exec)
A const binding of promisify(exec) now lowers like promisify(execFile):
each call runs the command line with /bin/sh -c, as execSync does, and
settles with { stdout, stderr }. A non-zero exit, a missing command (the
shell's 127) or a timeout rejects with Node's "Command failed: <command>\n
<stderr>" message. The options are the execFile slice: encoding utf8,
cwd, env (inline spread included), timeout, and maxBuffer accepted.

Three gaps closed for both targets along the way:

- the { stdout, stderr } result maps as a record when it is held in a
  variable, passed or returned, instead of only when destructured at the
  await (it is an anonymous literal in @types/node and in the fallback);
- a namespace member is a recognized target (promisify(cp.exec));
- util.promisify(...) through a namespace or default import is
  recognized like the named import.
2026-10-16 07:32:06 -05:00
Chris Tate df74099468 Free released regions without a pass over the whole heap
Every release of a LifetimeRegion, and every retry at region creation, ran
scr_collect_cycles(): full cycle-collector passes over the thread's heap, to
a fixpoint, before checking any region, in case dead cycles of ordinary
objects held counted references into it. Freeing a region ran them again
whenever the owned ordinary objects left counts behind after the region
released its fields. Next to a large heap of candidates (a type checker
beside per-file emit regions) those passes cost far more than the regions.

A retry now checks the due regions first. A region whose references are all
explained frees without a pass; only when one would be kept does the heap
get the full pass, after which every due region is judged as before. The
first check leaves a region it cannot free as it was, so the backoff, the
kept report and SCRIPTC_REGION_DEBUG see only the check after the pass. It
keeps what its walk found: a region whose counted references the pass did not
change is judged from that, without walking it again. A pass that freed a dead
cycle referring only to an owned object leaves the count unchanged; such a
region waits for its next attempt.

After a region releases its instances' fields, what can still hold counts
is the owned ordinary objects' cycles. Each has a member that lost a
reference in those releases and is buffered as a candidate, and the release
walk entered every cycle member. The cycle collector gains a pass limited to
a set of candidates (scr_cyc_collect_set): it takes the set's members out of
the buffers up to a given generation, reading only the buffers' pointers, so
the candidates it leaves are not touched, and it walks at full depth without
moving the schedule's baselines. A dead object such a pass reaches through a
candidate it did not take leaves its buffer before it is freed. Freeing a
region runs these passes over the walk's objects from the youngest buffer
out, and the whole heap only if counts remain.

SCRIPTC_REGION_STATS also prints, per thread at exit, how many full passes
over the heap regions needed.
2026-10-16 07:01:01 -05:00
Chris Tate d28ec3e31e Merge main 2026-10-16 04:34:48 -05:00
Chris Tate 2e8b836a79 Merge main 2026-10-16 04:33:13 -05:00
Chris Tate 18e69b81c5 Merge main 2026-10-16 04:30:20 -05:00
Chris Tate eb48e1d2e4 Pool small Buffers as Node does (#1101) 2026-10-16 03:50:17 -05:00
Chris Tate b487571539 Merge main 2026-10-16 03:33:30 -05:00
Chris Tate 4858a24420 Merge main 2026-10-16 03:30:56 -05:00
Chris Tate d3dda8271d Leave the main thread's heap to the operating system at exit
When the main thread's script ends, the executable released every module
global and ran a full cycle collection before exiting, which takes time
proportional to everything the program still holds (about 15 ms for
1.5 million live objects) and frees memory the operating system reclaims
anyway. The main thread now skips those releases, the exit-time
collection and the collector's buffer cleanup. Exit listeners and output
flushing run as before. Worker threads still free their heaps when they
exit, since later threads reuse their arenas, and builds that check for
leaks (the RC audit, AddressSanitizer) free everything as before, as does
any process run with SCRIPTC_EXIT_FREE=1 for external leak checkers.

SCRIPTC_WORKER_STATS=1 reports, as each worker thread exits, how long it
took to start, to run its script and to exit.
2026-10-16 01:58:45 -05:00
Chris Tate 0d59abe143 Pool small Buffers as Node does
Buffer.from of strings, arrays and Uint8Arrays, Buffer.allocUnsafe and
Buffer.concat now return views into a per-thread pool slab of
Buffer.poolSize bytes (64 KiB by default) when the size is below
Buffer.poolSize >>> 1, following Node's allocate, fromStringFast and
fromArrayLike: 8-byte aligned offsets, a new slab when a request does not
fit, and the four-bytes-per-unit reservation for strings. Buffer.poolSize
can be read and set, and Buffer.allocUnsafeSlow returns an unpooled Buffer.
A slab lives while any of its views does. Its ArrayBuffer cannot be
transferred, and structuredClone of a typed array view copies the whole
ArrayBuffer and keeps the view's offset. Buffer.alloc, allocUnsafe and
allocUnsafeSlow validate sizes with Node's ERR_OUT_OF_RANGE error.

The first pooled allocation finds the startup slab, which has the
default 64 KiB size. A request it cannot hold gets a slab of the current
Buffer.poolSize, as in Node, and the room is checked after every new
slab, so a Buffer.poolSize raised before the first pooled allocation
cannot place a reservation past the end of a 64 KiB slab. A reservation
larger than a fresh slab traps instead of writing past it.

Corpus 5531 (typed) and 5532 (untyped) raise Buffer.poolSize before the
first pooled allocation and allocate just under the new threshold
through each pooled path, each case in a fresh child process, and also
shrink Buffer.poolSize and change it between allocations. The
node-types fenced fixture no longer lists Buffer.poolSize, which now
compiles.
2026-10-16 01:39:10 -05:00
Chris Tate 2c04163350 Merge main 2026-10-16 01:32:17 -05:00
Chris Tate ba72bb6420 Merge main 2026-10-16 00:50:23 -05:00
Chris Tate 30bb7d4703 Add mock.fn and t.mock.fn with Node's calls log and implementation controls (#1092) 2026-10-16 00:49:22 -05:00
Chris Tate 00bb57ac0a Merge main 2026-10-16 00:37:00 -05:00
Chris Tate 8d8e7a7737 Run Atomics on shared storage without a lock (#1090) 2026-10-16 00:27:39 -05:00
Chris Tate 42954a3266 Add mock.fn and t.mock.fn with Node's calls log and implementation controls
node:test's mock.fn(original[, implementation]) and t.mock.fn compile.
A Mock<F> is the function-with-properties record { %call, mock }: the
callable is a wrapper closure that shares the mock context with the mock
field. The context records each call's arguments, result, error and this,
and keeps the current implementation, the original and the one-time
implementations by call index. fn.mock supports, as in Node 24.22:

- calls, read as a copy of the log;
- callCount();
- mockImplementation(fn);
- mockImplementationOnce(fn[, onCall]), for the next call by default, a
  later registration for the same index replacing the earlier one;
- resetCalls();
- restore(), which brings back the original.

A throwing implementation records the error and rethrows it. Functions
with fixed parameters or a single rest array parameter are supported; a
union return type needs an undefined arm. mock.fn() without an original,
mock.method, the tracker's reset and restoreAll, and t.mock.method are
refused with SC2020.

The node:test fixture mock-fn compares the behavior with Node, and
tests/diagnostics/mock-fences.ts covers the refusals.
2026-10-16 00:22:22 -05:00
Chris Tate 7b83558f52 Run Atomics on shared storage without a lock
Atomics operations on 8-, 16- and 32-bit integer arrays are sequentially
consistent atomic instructions on the element, and no longer take the
process-wide shared-memory lock. Other reads and writes of shared storage
are relaxed atomic accesses: element reads and writes access the element,
and copies, fills, conversions, searches and sorts read shared bytes into
a private copy and write back only the range they change, so concurrent
accesses to the same bytes are free of data races. Non-shared storage
keeps its plain memory accesses. Atomics.wait and Atomics.notify queue
waiters in buckets chosen by location, each with its own lock and
condition variable, so waiting threads no longer serialize other Atomics.

Atomics.store answers +0 for a value that truncates to -0, as Node does.
The shared storage runtime test also runs under ThreadSanitizer.
2026-10-16 00:06:31 -05:00
Chris Tate de0917284d Merge main 2026-10-16 00:05:07 -05:00
Chris Tate 93dcd9865e Merge main 2026-10-16 00:01:13 -05:00
Chris Tate 524af28870 Add fs read and write streams
fs.createReadStream, fs.createWriteStream and FileHandle
createReadStream, createWriteStream and readLines return runtime
Readable and Writable streams (scr_fs_stream.c), as do new fs.ReadStream
and new fs.WriteStream in JavaScript sources. They follow
lib/internal/fs/streams.js on the stream engine:

- _construct opens the file, then emits 'open' (with the fd) and
  'ready'; _read reads up to end; _write retries short writes as
  writeAll does; _destroy waits for I/O in flight, then closes (with
  fsync first for the flush option). Every open, read, write, fsync and
  close runs on the shared worker pool.
- Options are checked in Node's order: getOptions, options.fd or the
  path, start, end, then highWaterMark, encoding, emitClose and
  autoClose. flags and mode are checked when the file opens, so their
  errors arrive as 'error' events.
- bytesRead, bytesWritten, pending, path and close(cb) compile, also
  through untyped values (close() there).

A FileHandle stream holds a reference on the handle; closing the stream
releases it and closes the handle unless autoClose is false.
FileHandle.close() follows Node's reference count: it waits for the
streams over the handle and runs their close(). readLines is a readline
Interface over createReadStream with crlfDelay: Infinity.

The stream engine gains Node's _construct (reads, buffered writes,
finish and _destroy wait for construction; destroy errors combine with
aggregateTwoErrors) and _writev, both internal. Writable.destroy() now
runs the buffered write callbacks and end() callbacks (errorBuffer),
which it dropped before. highWaterMark in a dynamic options record is
checked as getHighWaterMark does, and for await over a constructing
stream waits for construction.

Numeric flags, the fs, signal and windowsHandle options, non-UTF-8
write encodings, option functions that replace the file implementation,
and instanceof fs.ReadStream/WriteStream are refused.

Corpus 5546, 5547 and 5554 cover both lanes against Node 24.22.
2026-10-15 23:58:50 -05:00
Chris Tate 9b452b88de Merge main 2026-10-15 23:33:48 -05:00
Chris Tate 681f5ceee7 Add TestContext hooks, plan and fullName, and Node's todo and only rules
node:test changes, matching Node 24.22:

- t.before, t.after, t.beforeEach and t.afterEach lower. t.before runs
  when it is called; a throw fails the test and the subtests that follow,
  which skip their body and beforeEach hooks and run their afterEach hooks.
  t.beforeEach and t.afterEach wrap every descendant subtest. t.after runs
  after the test's afterEach chain, also when the body throws, and a throw
  fails the test.
- t.plan(count) counts the test's t.assert calls and subtests. A mismatch
  fails the test with Node's "plan expected N assertions but received M"
  string, unless an assertion failed first.
- t.fullName joins the names from the outermost suite or test with " > ".
- Tests and suites under a todo suite or test are todo, with a bare TODO
  directive, unless they are skipped or carry their own todo message. A
  todo suite reports its directive.
- describe.only, suite.only and { only: true } on a suite mark it. At each
  level, when a child is marked or contains a marked test or suite, only
  those children run.
- The failing-tests section lists failures in completion order, so
  subtests come before the test that ran them, and prints thrown strings
  quoted, as Node inspects them.

The fixtures ctx-hooks, ctx-plan, todo-suites, suite-variants,
only-nested and thrown-values compare the reporter output and exit code
with Node.
2026-10-15 23:16:04 -05:00
Chris Tate 386cfd0c65 Accept readline's history and completer options and add the history members
createInterface accepts history, historySize, removeHistoryDuplicates,
completer, tabSize and escapeCodeTimeout, as Node does for a non-terminal
interface: tabSize, escapeCodeTimeout and historySize are checked in the
constructor's order before the interface opens (ERR_OUT_OF_RANGE and
ERR_INVALID_ARG_VALUE with Node's messages), a completer that is not a
function throws ERR_INVALID_ARG_VALUE, and the values are kept on the
interface. Options may be written as shorthand properties.

rl.history answers the history option itself, or the interface's own
empty array, and historySize, historyIndex, crlfDelay, tabSize,
escapeCodeTimeout, line and terminal read as in Node. 'history'
listeners can be registered; without terminal mode Node records no
history, so they never fire.
2026-10-15 22:53:40 -05:00
Chris Tate 63b4dddeca Support array keys in InternTable and free replaced indexes (#1072) 2026-10-15 22:50:08 -05:00
Chris Tate 54727fc983 Merge main 2026-10-15 22:40:47 -05:00
Chris Tate 1b4103889b Add assert.deepEqual, partialDeepStrictEqual and the Assert class (#1079) 2026-10-15 22:22:57 -05:00
Chris Tate 7c09e1c020 Add assert.deepEqual, partialDeepStrictEqual and the Assert class 2026-10-15 22:01:38 -05:00
Chris Tate 647bc2a492 Add source functions, promise destinations and { signal } to pipeline stages
pipeline() accepts a source function (an async generator function called
with { signal }) and a destination async function, following
pipeline.js: the destination's promise value resolves the
stream/promises form and reaches the callback's second argument, and
pipeline() answers a PassThrough for it. A rejected destination destroys
that PassThrough with the reason and fails the pipeline.

Stages may take the { signal } argument: the pipeline creates its own
AbortController when one does, and aborts it when the pipeline finishes
or fails, as finishImpl does. Stage functions take the source and
{ signal } through the closure ABI, and the PassThrough and the value
are released with the pipeline.

The stream unit now links the emitter unit whenever it links, so a
pipeline of functions builds without constructing a stream. An async
generator destination and reading the PassThrough a function destination
returns are refused at compile time.
2026-10-15 21:50:01 -05:00
Chris Tate 77c9a2ff23 Merge main 2026-10-15 21:48:48 -05:00