Commit Graph
402 Commits
Author SHA1 Message Date
Chris Tate 5c39821726 Merge main 2026-10-17 02:30:43 -05:00
Chris Tate 6fa1f8ce01 Lower util.promisify(stream.finished) and util.promisify(stream.pipeline) (#1177) 2026-10-17 02:19:12 -05:00
Chris Tate 3e83b4a5c7 Lower util.promisify(stream.finished) and util.promisify(stream.pipeline)
Node's util.promisify returns the custom promisified forms of
stream.finished and stream.pipeline, which are stream/promises' functions
of the same name. A const binding of promisify over the node:stream named
imports or namespace members now registers that stream/promises member
as the projection, so calls through the binding lower as the
stream/promises call and accept the stream arguments and options it
accepts.

The fallback declarations gain the matching util.promisify overloads.
pipeline's overload comes first: its (...streams: unknown[]) => unknown
type is assignable to every other target's parameter type.
2026-10-17 01:51:57 -05:00
Chris Tate f7ad1f96a4 Lower Array.fromAsync over arrays
The ESNext array lib is now part of the type world, and Array.fromAsync(source,
mapFn?) compiles when the source is an array of values or of promises. Each
call lowers to an async helper with the spec's loop: elements are awaited
through the async-from-sync iterator, with the same microtask turns as a
`for await` loop over an array, and a mapping function's result is awaited
before it is stored. A throwing or rejecting mapping function and a rejected
element close the iterator before the returned promise rejects, on the turn
Node rejects it. Async generator sources, a thisArg, and arrays that mix
values and promises are refused.
2026-10-17 01:47:40 -05:00
Chris Tate 6d4843ea99 Dispose Disposable-typed values and expose SuppressedError's error and suppressed
`using` and `await using` now accept object literals and other values typed
Disposable, AsyncDisposable or unknown. The declaration reads the value's
[Symbol.dispose] method ([Symbol.asyncDispose] first for `await using`) and
throws Node's TypeErrors for values that are not objects and for missing or
non-callable methods; null and undefined dispose nothing. Scope exit calls the
method with the value as `this`, and `await using` awaits the result of an
async method but not of the sync fallback. A class instance reached through
such a value throws an SC2020 error, because its dynamic view has no methods.

SuppressedError is its own error class: a failing disposal over a pending
error throws one whose `error` is the disposal error and whose `suppressed` is
the pending error, both non-enumerable as in Node. `new SuppressedError(error,
suppressed, message?)` and `instanceof SuppressedError` compile, and
`extends SuppressedError` is refused.
2026-10-16 22:36:18 -05:00
Chris Tate d7a5568726 Revert "Lower child_process.exec's callback forms" (#1161) 2026-10-16 22:05:36 -05:00
Chris Tate db94b2d257 Add fs.cpSync, fs.cp and fs/promises cp
fs.cpSync, fs.cp and fs/promises cp compile with every option:
dereference, errorOnExist, filter, force, mode, preserveTimestamps,
recursive and verbatimSymlinks. They follow Node's two implementations.
cpSync follows cp-sync.js over the fs binding: copy errors are plain
Errors with a code, std::error_code failures throw an ErrnoException,
and links inside a copied directory point at their canonical targets.
fs.cp and fs/promises cp follow cp.js, whose copy errors are
SystemErrors with code, info, errno, syscall and path.

Options and paths validate in Node's order with Node's errors. filter
runs for the top-level paths and each entry in Node's order; an async
filter must return a promise that is already settled, and a pending one
throws SC2020. The callback and promise forms complete the copy before
the callback runs or the promise settles. Windows builds throw SC2020.

The copy lives in the new scr_fs_cp.c. Directory walks keep their frames
on the heap, so deep trees also copy on a fiber's stack. File contents
copy with sendfile on Linux, and COPYFILE_FICLONE uses FICLONE
where it exists. ScrError gains the SystemError context and the
ErrnoException key order, with matching fields in the compiler's Error
layout. The stored fs.cp callback uses the same implementation.

Corpus 5567-5571 cover the sync, callback and promise forms, errors,
symlinks and 1500-level trees.
2026-10-16 21:06:22 -05:00
Chris Tate d8c2b0f96d Lower child_process.exec's callback forms (#1161) 2026-10-16 20:57:36 -05:00
Chris Tate 6f4334be00 Accept Map arguments in the Set composition methods (#1164) 2026-10-16 20:51:52 -05:00
Chris Tate d53c1b8ba0 Merge main 2026-10-16 20:30:22 -05:00
Chris Tate 58842aa39b Accept Map arguments in the Set composition methods
union, intersection, difference, symmetricDifference, isSubsetOf,
isSupersetOf and isDisjointFrom now compile with a Map argument whose key type
is the Set's element type, as well as with a Set. A Map's set record reads its
keys, and its size, has and keys are the builtins, so the helper loops walk
and test the Map's keys in the order the spec gives. Other set-like arguments
are still refused.
2026-10-16 20:28:04 -05:00
Chris Tate 9681454556 Lower util.promisify(setTimeout) and util.promisify(setImmediate)
Node's util.promisify returns the custom promisified form of setTimeout
and setImmediate, which is timers/promises' function of the same name.
A const binding of promisify over the global functions, the node:timers
named imports or the node:timers namespace members now registers that
timers/promises member as the projection, so calls through the binding
lower as the timers/promises call: they fulfill with the given value
(undefined without one) after the delay or in the check phase.

The fallback declarations gain the matching util.promisify overloads.
2026-10-16 20:22:19 -05:00
Chris Tate cef618b553 Lower child_process.exec's callback forms
exec(command, callback) and exec(command, { encoding, maxBuffer },
callback) lower like execFile's callback slice. The command line runs
with /bin/sh -c, as execSync runs it, the call returns the live
ChildProcess, and the error-first callback receives the captured utf8
stdout and stderr. A non-zero exit, including the shell's 127 for a
missing command, reports Node's "Command failed: <command>\n<stderr>":
scr_exec_file takes a shell flag so the message shows the command line.

Under @types/node, child_process.ExecException maps to Error, as
ExecFileException does. Its code member is the child's numeric exit
status, which the runtime error's string code slot does not hold, so
reading it, testing it with `in`, or testing its typeof is a named
refusal.
2026-10-16 20:14:33 -05:00
Chris Tate 85439ae670 Merge main 2026-10-16 19:54:32 -05:00
Chris Tate 2e52dcd35c Add Readable.from over Sets, iterators and class iterables (#1074) 2026-10-16 19:30:21 -05:00
Chris Tate d9229b044f Inline small runtime functions into release executables (#1130) 2026-10-16 19:16:56 -05:00
Chris Tate 5a091f050f Run the zlib one-shot forms on the stream engine (#1139) 2026-10-16 19:11:18 -05:00
Malte Ubl f4ce6910c7 Inline small runtime functions into release executables
Release executables now import small runtime functions from runtime
bitcode, as speed executables do, so LLVM can inline them into program
code. Release units keep the runtime compiler's own objects, unchanged,
and additionally ship the bitcode of the same compilation reduced by the
helper's prune-import. Their unit-local symbols stay local, so only
bodies that need none of them can be imported, which keeps the import
small: about 1.3 MB per program.

Only unsanitized programs small enough for one codegen partition import.
Every partition reads the bitcode again; on the Effect fixtures (64
partitions) import cost +2..18% compile time and +4.3..4.9% size, so
partitioned release builds stay exactly as before. Speed is unchanged.

Measured against main in Linux Vercel Sandboxes, same-VM interleaved
medians (bench-runtime --layouts=4 --runs=16), on two VMs:

- bench-runtime (31 workloads): -1.3% (8 faster, 0 slower) and -1.0%
  (1 slower, typed-array-scan-workers +3.2%); size +0.0%
- js-compiled suite (22 programs): -2.2% and -1.8%; size -0.1%
- cold builds of all 53 programs, interleaved: +3.1%; helper time per
  build +40..235 ms
- Effect fixtures: byte-identical to main
2026-10-16 14:03:52 -07:00
Malte Ubl 8f5ec2d94f Report record width copies at assertions, assignments and array elements
SC6001 only looked at call arguments and typed initializers, and only at
strict field subsets: a target that adds optional properties, the empty
record type, `r as T`, a later `v = r` into a typed binding, and records
stored as elements of a typed array literal copied silently.

The scan now reports a copy whenever the property sets differ, covers
those sites, and reports a class instance that reaches a plain record
type which does not keep its identity (the remaining boundary: interfaces
of only generic methods). Every corpus program still reports no
divergence.
2026-10-16 14:02:02 -07:00
Malte Ubl 6c9c39b3ab Keep class instance identity through structural record views
A class instance passed, stored, returned or asserted as an anonymous
object type or an interface it does not `implements` was copied into a
native record, so writes, identity checks and inspection through the view
missed the instance (`function add(o: { count: number }, n: number)
{ o.count += n }` left a Counter at 0).

ClassProtocols now scans flow sites (arguments, typed initializers and
fields, defaults, returns, arrow bodies, yields, assignments, array
elements, property values, assertions through `unknown`, and annotated
callback parameters) and marks every program object type a class
instance flows into, then propagates the mark along flows between
structural types. Marked types are class-backed protocols: their values
are checked views that keep the original object, like implemented
interfaces already were. Only sites whose source carries identity query
the checker for their target, and a checker panic skips the site.
Assertions read or written member-wise in place keep their direct field
access, and `typeof C` is not an instance.

An assertion of a class instance to such a view boxes it like an
implicit flow, and `view.n++` on a number member lowers to the checked
expanded form.
2026-10-16 14:02:02 -07:00
Chris Tate 01f25f6b3e Lower util.promisify over the fs callback functions (#1138) 2026-10-16 09:37:58 -05:00
Chris Tate d533fd7ce9 Merge main 2026-10-16 09:33:08 -05:00
Chris Tate f4445f1540 Keep undeclared keys of checked JSON records; lower compound assignment with untyped operands (#1123) 2026-10-16 09:20:49 -05:00
Chris Tate ed5ab6796b Run the zlib one-shot forms on the stream engine
deflateSync, inflateSync, gzipSync, gunzipSync, deflateRawSync,
inflateRawSync and unzipSync, and their callback forms, accept Node's
options and run Node's processChunkSync over the zlib stream engine.

- Options (windowBits, level, memLevel, strategy, dictionary, chunkSize,
  finishFlush, maxOutputLength, rejectGarbageAfterEnd) are validated at
  run time with Node's ladder, shared with the stream classes; info is
  refused at run time.
- Errors carry Node's errno and code; maxOutputLength reports
  ERR_BUFFER_TOO_LARGE and rejectGarbageAfterEnd
  ERR_TRAILING_JUNK_AFTER_STREAM_END. The default-options decoders also
  read every gzip member.
- Untyped JavaScript inputs may be strings, TypedArrays, DataViews or
  ArrayBuffers; other values throw Node's ERR_INVALID_ARG_TYPE after the
  options are checked.
- Results are Buffers, as in Node: they convert to their text.

Corpus programs 5378, 5379 and 5415 cover the sync, callback and untyped
input forms.
2026-10-16 09:18:32 -05:00
Chris Tate daa5cc256f Lower util.promisify over the fs callback functions
promisify(fs.X) over a named or namespace import of node:fs lowers as
the fs/promises member of the same name for access, appendFile, chmod,
chown, copyFile, lchown, link, lstat, lutimes, mkdir, mkdtemp, readFile,
readdir, readlink, realpath, rename, rm, rmdir, stat, statfs, symlink,
truncate, unlink, utimes and writeFile. For these, Node's promisified
callback form settles with the callback's single result and rejects
with the same errors as the fs/promises call. Calls through the bound
function accept the forms the fs/promises lowering accepts and refuse
the others by name.

fs.open (an fd number where fs/promises.open gives a FileHandle),
fs.exists (a custom promisify) and fs.read/fs.write (multi-value
callbacks) remain refused.
2026-10-16 09:14:35 -05:00
Chris Tate 241006cff3 Add fs read and write streams (#1089) 2026-10-16 09:02:48 -05:00
Chris Tate 21c2340ef6 Support the dgram socket options and disconnect() (#1122) 2026-10-16 08:39:23 -05:00
Chris Tate 451e925b12 Merge main 2026-10-16 08:32:21 -05:00
Malte Ubl dff726715b Keep undeclared keys of checked JSON records
`const parsed: { score: number }[] = JSON.parse(doc)` printed a different
checksum from Node: the checked conversion (and the schema-directed
JSON.parse fast path) stored only the declared fields of each object in
its native record, so JSON.stringify(parsed) silently dropped `id`,
`name` and `active`, and every other reflective surface agreed.

A closed record converted from an ordinary object now keeps the source's
own properties beside it when the object has undeclared keys or holds the
declared ones out of declaration order. The runtime keeps a per-thread
table from the native record to a view of the source's own keys
(placeholders for declared keys, which the record still owns), released
through the native dispose hook when the record is freed:

- the checked record builder notes its source (scr_record_source_note);
- the schema parser parses undeclared members into the view instead of
  declining, so typed parses keep their one-pass fast path;
- typed JSON.stringify writes such records in source key order, declared
  fields through a generated per-shape field writer and the others through
  the dyn writer;
- record capsules adopt the view and refresh it in place, so untyped reads
  and writes see and persist undeclared properties;
- util.inspect and Object.keys/values/entries answer through the live view
  (values/entries check the extra values into their static result type and
  throw the checked-conversion TypeError when one does not fit).

Copies made by object spread, Object.assign, rest destructuring or
conversion to another record type still hold only the declared fields;
the limitations page documents this.
2026-10-16 06:31:51 -07:00
Chris Tate a5da46267b Merge main 2026-10-16 08:31:20 -05:00
Chris Tate a09990192e Add tlsSocket.getProtocol, getCipher, alpnProtocol and tls.DEFAULT_MIN_VERSION/MAX_VERSION (#1054) 2026-10-16 08:23:42 -05:00
Chris Tate 21c0e17178 Merge main 2026-10-16 08:17:52 -05:00
Chris Tate 0cad2fe551 Name the TLS ciphers limitations partial after its heading
The partial for the "TLS ciphers" section was tls-cipher.mdx, while the
limitations tooling names a section's partial after its heading's slug
(tls-ciphers). It is renamed, so a join and split of the page leaves the
docs unchanged.
2026-10-16 07:48:05 -05:00
Chris Tate 4adc3e6f2a Support the dgram socket options and disconnect()
dgram.Socket's setBroadcast, setTTL, setMulticastTTL,
setMulticastLoopback, setMulticastInterface, addMembership,
dropMembership, get/setRecvBufferSize, get/setSendBufferSize,
getSendQueueSize, getSendQueueCount and disconnect were refused at
compile time. They now lower to scr_dgram.c and match Node:

- The TTL setters and setMulticastLoopback return their argument. The
  value checks run first: a TTL outside 1-255 (0-255 for multicast,
  after ToInt32) throws "setTTL EINVAL" with code, errno and syscall.
- A socket that has not been bound has no descriptor yet, so the
  options throw EBADF, and the buffer size methods throw Node's
  ERR_SOCKET_BUFFER_SIZE SystemError.
- After close(), the members that read Node's null handle throw its
  TypeError ("Cannot read properties of null (reading 'setTTL')"), and
  setMulticastInterface and the membership calls throw "Not running".
- Buffer sizes must be uint32 values (ERR_SOCKET_BAD_BUFFER_SIZE), 0
  reads the current size, and sizes past the int32 range fail as EINVAL.
- IPv6 interfaces and groups go to the IPv6 socket options, so a udp4
  socket answers ENOPROTOOPT as in Node.
- disconnect() throws ERR_SOCKET_DGRAM_NOT_CONNECTED until 'connect' was
  emitted, and dissolves the association with an AF_UNSPEC connect.

addMembership and dropMembership before bind() throw SC2020, where
Node binds the socket to an ephemeral port first; the limitations page
documents it, along with the missing info object on the buffer size
SystemError. Corpus 5655 and 5656 and the node-types fixture
dgram-options.ts cover each method.
2026-10-16 07:34:12 -05:00
Chris Tate 60623086a3 Lower util.promisify(child_process.exec)
A const binding of promisify(exec) now lowers like promisify(execFile):
each call runs the command line with /bin/sh -c, as execSync does, and
settles with { stdout, stderr }. A non-zero exit, a missing command (the
shell's 127) or a timeout rejects with Node's "Command failed: <command>\n
<stderr>" message. The options are the execFile slice: encoding utf8,
cwd, env (inline spread included), timeout, and maxBuffer accepted.

Three gaps closed for both targets along the way:

- the { stdout, stderr } result maps as a record when it is held in a
  variable, passed or returned, instead of only when destructured at the
  await (it is an anonymous literal in @types/node and in the fallback);
- a namespace member is a recognized target (promisify(cp.exec));
- util.promisify(...) through a namespace or default import is
  recognized like the named import.
2026-10-16 07:32:06 -05:00
Chris Tate df74099468 Free released regions without a pass over the whole heap
Every release of a LifetimeRegion, and every retry at region creation, ran
scr_collect_cycles(): full cycle-collector passes over the thread's heap, to
a fixpoint, before checking any region, in case dead cycles of ordinary
objects held counted references into it. Freeing a region ran them again
whenever the owned ordinary objects left counts behind after the region
released its fields. Next to a large heap of candidates (a type checker
beside per-file emit regions) those passes cost far more than the regions.

A retry now checks the due regions first. A region whose references are all
explained frees without a pass; only when one would be kept does the heap
get the full pass, after which every due region is judged as before. The
first check leaves a region it cannot free as it was, so the backoff, the
kept report and SCRIPTC_REGION_DEBUG see only the check after the pass. It
keeps what its walk found: a region whose counted references the pass did not
change is judged from that, without walking it again. A pass that freed a dead
cycle referring only to an owned object leaves the count unchanged; such a
region waits for its next attempt.

After a region releases its instances' fields, what can still hold counts
is the owned ordinary objects' cycles. Each has a member that lost a
reference in those releases and is buffered as a candidate, and the release
walk entered every cycle member. The cycle collector gains a pass limited to
a set of candidates (scr_cyc_collect_set): it takes the set's members out of
the buffers up to a given generation, reading only the buffers' pointers, so
the candidates it leaves are not touched, and it walks at full depth without
moving the schedule's baselines. A dead object such a pass reaches through a
candidate it did not take leaves its buffer before it is freed. Freeing a
region runs these passes over the walk's objects from the youngest buffer
out, and the whole heap only if counts remain.

SCRIPTC_REGION_STATS also prints, per thread at exit, how many full passes
over the heap regions needed.
2026-10-16 07:01:01 -05:00
Chris Tate d28ec3e31e Merge main 2026-10-16 04:34:48 -05:00
Chris Tate 2e8b836a79 Merge main 2026-10-16 04:33:13 -05:00
Chris Tate 18e69b81c5 Merge main 2026-10-16 04:30:20 -05:00
Chris Tate eb48e1d2e4 Pool small Buffers as Node does (#1101) 2026-10-16 03:50:17 -05:00
Chris Tate b487571539 Merge main 2026-10-16 03:33:30 -05:00
Chris Tate 4858a24420 Merge main 2026-10-16 03:30:56 -05:00
Chris Tate d3dda8271d Leave the main thread's heap to the operating system at exit
When the main thread's script ends, the executable released every module
global and ran a full cycle collection before exiting, which takes time
proportional to everything the program still holds (about 15 ms for
1.5 million live objects) and frees memory the operating system reclaims
anyway. The main thread now skips those releases, the exit-time
collection and the collector's buffer cleanup. Exit listeners and output
flushing run as before. Worker threads still free their heaps when they
exit, since later threads reuse their arenas, and builds that check for
leaks (the RC audit, AddressSanitizer) free everything as before, as does
any process run with SCRIPTC_EXIT_FREE=1 for external leak checkers.

SCRIPTC_WORKER_STATS=1 reports, as each worker thread exits, how long it
took to start, to run its script and to exit.
2026-10-16 01:58:45 -05:00
Chris Tate 0d59abe143 Pool small Buffers as Node does
Buffer.from of strings, arrays and Uint8Arrays, Buffer.allocUnsafe and
Buffer.concat now return views into a per-thread pool slab of
Buffer.poolSize bytes (64 KiB by default) when the size is below
Buffer.poolSize >>> 1, following Node's allocate, fromStringFast and
fromArrayLike: 8-byte aligned offsets, a new slab when a request does not
fit, and the four-bytes-per-unit reservation for strings. Buffer.poolSize
can be read and set, and Buffer.allocUnsafeSlow returns an unpooled Buffer.
A slab lives while any of its views does. Its ArrayBuffer cannot be
transferred, and structuredClone of a typed array view copies the whole
ArrayBuffer and keeps the view's offset. Buffer.alloc, allocUnsafe and
allocUnsafeSlow validate sizes with Node's ERR_OUT_OF_RANGE error.

The first pooled allocation finds the startup slab, which has the
default 64 KiB size. A request it cannot hold gets a slab of the current
Buffer.poolSize, as in Node, and the room is checked after every new
slab, so a Buffer.poolSize raised before the first pooled allocation
cannot place a reservation past the end of a 64 KiB slab. A reservation
larger than a fresh slab traps instead of writing past it.

Corpus 5531 (typed) and 5532 (untyped) raise Buffer.poolSize before the
first pooled allocation and allocate just under the new threshold
through each pooled path, each case in a fresh child process, and also
shrink Buffer.poolSize and change it between allocations. The
node-types fenced fixture no longer lists Buffer.poolSize, which now
compiles.
2026-10-16 01:39:10 -05:00
Chris Tate 2c04163350 Merge main 2026-10-16 01:32:17 -05:00
Chris Tate ba72bb6420 Merge main 2026-10-16 00:50:23 -05:00
Chris Tate 30bb7d4703 Add mock.fn and t.mock.fn with Node's calls log and implementation controls (#1092) 2026-10-16 00:49:22 -05:00
Chris Tate 00bb57ac0a Merge main 2026-10-16 00:37:00 -05:00
Chris Tate 8d8e7a7737 Run Atomics on shared storage without a lock (#1090) 2026-10-16 00:27:39 -05:00
Chris Tate 42954a3266 Add mock.fn and t.mock.fn with Node's calls log and implementation controls
node:test's mock.fn(original[, implementation]) and t.mock.fn compile.
A Mock<F> is the function-with-properties record { %call, mock }: the
callable is a wrapper closure that shares the mock context with the mock
field. The context records each call's arguments, result, error and this,
and keeps the current implementation, the original and the one-time
implementations by call index. fn.mock supports, as in Node 24.22:

- calls, read as a copy of the log;
- callCount();
- mockImplementation(fn);
- mockImplementationOnce(fn[, onCall]), for the next call by default, a
  later registration for the same index replacing the earlier one;
- resetCalls();
- restore(), which brings back the original.

A throwing implementation records the error and rethrows it. Functions
with fixed parameters or a single rest array parameter are supported; a
union return type needs an undefined arm. mock.fn() without an original,
mock.method, the tracker's reset and restoreAll, and t.mock.method are
refused with SC2020.

The node:test fixture mock-fn compares the behavior with Node, and
tests/diagnostics/mock-fences.ts covers the refusals.
2026-10-16 00:22:22 -05:00