Commit Graph
3704 Commits
Author SHA1 Message Date
Chris Tate 612be233dc Keep int32 field storage when reflection reads values through unknown (#1099) 2026-10-16 03:37:46 -05:00
Chris Tate b02e8be154 Shrink the publish log and keep it between publishes (#1104) 2026-10-16 03:37:27 -05:00
Chris Tate bb01d06859 Read ahead on sockets without a consumer, so a peer's close reaches them
A plain socket with no 'data' listener, pipe or 'readable' consumer was
never read, so a peer that ended or closed the connection was never
noticed and the process stayed up. Node's stream reads ahead up to its
high-water mark (64 KiB) even without a consumer: a peer that ends
without sending data ends and closes the socket, unread bytes stay
buffered (bytesRead counts them) and keep the socket open, and a
listener added later receives them before 'end'. The runtime now does
the same, holding EOF back while unread bytes remain.
Corpus 5653 covers each case.
2026-10-16 03:36:39 -05:00
Chris Tate b487571539 Merge main 2026-10-16 03:33:30 -05:00
Chris Tate 7175b1126a Run each darwin corpus slice as two concurrent halves (#1102) 2026-10-16 03:32:20 -05:00
Chris Tate a697e5c816 Leave the main thread's heap to the operating system at exit (#1103) 2026-10-16 03:32:00 -05:00
Chris Tate 4858a24420 Merge main 2026-10-16 03:30:56 -05:00
Chris Tate 383f43bdeb Keep integer-slot checking working in the native compiler for library calls with bytes or array arguments
The global-effects summary typed each library-call argument as a
structural `{ kind: string }`. The native compiler extracted that one
record arm from the IrType union and threw on bytes, array and other
arms with more fields, so `scriptc build --lib` failed with SC3004 for
integer-slot profiles that call, for example, `Buffer.alloc` or
`String.fromCharCode`. The helper now takes `IrType`, and the bootstrap
test compares native and Node sidecars for such a profile.
2026-10-16 02:51:23 -05:00
Chris Tate 379c771e14 Shrink the publish log and keep it between publishes
publish() logs each object it marks, with the object's count before the
mark, so that a refused publish can restore the graph. At 24 bytes per
object the log was most of the walk's memory traffic. Each publish also
allocated the log anew, so a large one faulted in fresh pages every time,
and a thread that publishes one graph per file paid that on each file.

A log entry is now 16 bytes on 64-bit targets. The count is stored in 32
bits, with escapes for the program-lifetime count and for any other count
of 32 bits or more, which goes to a side array restored in order on
refusal. The dyn-container flag moves into the entry's pointer, and the
RC audit's object kinds into a side array of audit builds. The stack and
log of a thread's last publish are kept for its next one while they take
16 MiB or less, and freed at thread exit.

Adds corpus program 5704.
2026-10-16 02:00:49 -05:00
Chris Tate b5f73477fb v0.3.4 (#1100) v0.3.4 2026-10-16 02:00:37 -05:00
Chris Tate d3dda8271d Leave the main thread's heap to the operating system at exit
When the main thread's script ends, the executable released every module
global and ran a full cycle collection before exiting, which takes time
proportional to everything the program still holds (about 15 ms for
1.5 million live objects) and frees memory the operating system reclaims
anyway. The main thread now skips those releases, the exit-time
collection and the collector's buffer cleanup. Exit listeners and output
flushing run as before. Worker threads still free their heaps when they
exit, since later threads reuse their arenas, and builds that check for
leaks (the RC audit, AddressSanitizer) free everything as before, as does
any process run with SCRIPTC_EXIT_FREE=1 for external leak checkers.

SCRIPTC_WORKER_STATS=1 reports, as each worker thread exits, how long it
took to start, to run its script and to exit.
2026-10-16 01:58:45 -05:00
Chris Tate bf98a562d8 Run each darwin corpus slice as two concurrent halves
The macOS lane of pull requests and pushes to main runs the darwin suite in
three slices. Each slice runs differential.test.ts over a third of the
corpus while the darwin contracts run beside it. vitest runs one file's
cases in sequence, so a slice's corpus time is the sum of its case times:
in three green main runs, 587-891 s (plain) and 800-1,731 s (sanitized),
on 3-core runners whose contracts task ends after 46-180 s.

ciTestPlan now runs slice i of n as case shards 2i-1 and 2i of 2n in two
vitest processes: the first beside the side queue, the second in the side
queue after the darwin contracts. The halves of all slices are the case
shards of 2n slices, so every program still runs once.

The comment above darwinFiles now says which macOS suite each CI event
runs: pull requests and pushes to main run the darwin suite, and the
nightly schedule and manual runs with macos_full run the full suite.
2026-10-16 01:55:40 -05:00
Chris Tate 0d59abe143 Pool small Buffers as Node does
Buffer.from of strings, arrays and Uint8Arrays, Buffer.allocUnsafe and
Buffer.concat now return views into a per-thread pool slab of
Buffer.poolSize bytes (64 KiB by default) when the size is below
Buffer.poolSize >>> 1, following Node's allocate, fromStringFast and
fromArrayLike: 8-byte aligned offsets, a new slab when a request does not
fit, and the four-bytes-per-unit reservation for strings. Buffer.poolSize
can be read and set, and Buffer.allocUnsafeSlow returns an unpooled Buffer.
A slab lives while any of its views does. Its ArrayBuffer cannot be
transferred, and structuredClone of a typed array view copies the whole
ArrayBuffer and keeps the view's offset. Buffer.alloc, allocUnsafe and
allocUnsafeSlow validate sizes with Node's ERR_OUT_OF_RANGE error.

The first pooled allocation finds the startup slab, which has the
default 64 KiB size. A request it cannot hold gets a slab of the current
Buffer.poolSize, as in Node, and the room is checked after every new
slab, so a Buffer.poolSize raised before the first pooled allocation
cannot place a reservation past the end of a 64 KiB slab. A reservation
larger than a fresh slab traps instead of writing past it.

Corpus 5531 (typed) and 5532 (untyped) raise Buffer.poolSize before the
first pooled allocation and allocate just under the new threshold
through each pooled path, each case in a fresh child process, and also
shrink Buffer.poolSize and change it between allocations. The
node-types fenced fixture no longer lists Buffer.poolSize, which now
compiles.
2026-10-16 01:39:10 -05:00
Chris Tate 2c04163350 Merge main 2026-10-16 01:32:17 -05:00
Chris Tate bf050d1077 v0.3.4 2026-10-16 01:29:54 -05:00
Chris Tate 97afb3f281 Convert cyclic dyn values to recursive shapes (#1098) 2026-10-16 01:19:00 -05:00
Chris Tate a09af4f721 Report what keeps a region with SCRIPTC_REGION_DEBUG (#1095) 2026-10-16 01:12:28 -05:00
Chris Tate d62b19badb Keep int32 field storage when reflection reads values through unknown
Int32Slots keeps a class field as a double when its class reaches
checked-dynamic code and some dynamic store in the program can name the
field. A runtime helper that takes a dynamic value counts as a store of
every property unless it is in DYN_READ_ONLY. Object.keys, Object.values,
Object.entries, for-in, Object.getOwnPropertyNames, Reflect.ownKeys,
Object.hasOwn, Object.prototype.toString.call, toString, typeof and
console.log/util.inspect of an unknown value were not in that list, so one
of them anywhere in a program, reached or not, stored every int32 field of
every class reachable from a value passed to unknown as a double.

The helpers they lower to are added to DYN_READ_ONLY. None of their
runtime functions reaches scr_dyn_typed_ref_commit, the only caller of a
live view's commit, except through scr_dyn_typed_ref_materialize, which
defines properties only on its fresh snapshot (as for dyn.hasKey).
2026-10-16 01:10:45 -05:00
Chris Tate 455ca6c172 Revert "Return unions with reference arms in registers" (#1096) 2026-10-16 01:02:39 -05:00
Chris Tate 6f63520c98 Emit Atomics operations inline as atomic instructions (#1094) 2026-10-16 00:54:32 -05:00
Chris Tate c7c21fe4a5 Convert cyclic dyn values to recursive shapes
Checking or converting a dyn value whose objects or arrays refer back to
each other against a recursive interface (`raw.next = raw; raw as Node`)
recursed without end and overflowed the stack. Matchers and builders of
record and array types that reach themselves now track the dyn values
they visit:

- a matcher answers true for a (value, type) pair it is already checking
  further up the stack, so the outer check decides;
- a builder records the record or array it allocated for a dyn value
  before converting its members, and a later visit of the same value in
  the conversion answers it, so `n.next === n` holds after the cast and a
  value reached twice converts once.

Only values with more than one reference are tracked, since a value
reached twice has at least two; the reference a member read holds is not
counted. Capsules materialized during a conversion keep their first view,
so a nested visit cannot refresh a view an outer visit is reading, and
getters and proxy traps that a member read calls run outside the visit. The
cycle analysis treats record shapes that reach themselves and are targets
of a dynCheck as mutable, so the cyclic records a conversion builds are
collected.
2026-10-16 00:52:19 -05:00
Chris Tate c8999078ca Refresh the Effect fixture costs from CI measurements
tests/harness/effect-costs.json and effect4-costs.json were recorded on
2026-10-01 and give the fixtures' relative cost for balancedShardSelect
across the 16 Effect shards of each flavor. Against the fixture times of
three green main runs (38851650148, 38851726594, 38852266290), the old
tables put 1,171 s of plain and 1,295 s of sanitized fixture time on the
longest shard, against a mean of 1,027 and 1,111 s.

scripts/ci-effect-costs.mjs refreshes both tables from Effect job logs:
each fixture's plain and sanitized medians, summed in seconds, as the
tables recorded before. Fixtures without a measurement keep their value,
and fixtures that no longer exist are removed. With the refreshed tables,
the longest shard of the same measurements carries 1,082 s (plain) and
1,144 s (sanitized).
2026-10-16 00:50:37 -05:00
Chris Tate ba72bb6420 Merge main 2026-10-16 00:50:23 -05:00
Chris Tate 30bb7d4703 Add mock.fn and t.mock.fn with Node's calls log and implementation controls (#1092) 2026-10-16 00:49:22 -05:00
Chris Tate 2e3ae368d1 Deliver fs callbacks without a timer turn (#1093) 2026-10-16 00:49:02 -05:00
Chris Tate 6330e26eb8 Open a WriteStream's file when its encoding throws
Node's WriteStream constructor calls setDefaultEncoding after the
Writable constructor has scheduled _construct, so with encoding: 'buffer'
it throws ERR_UNKNOWN_ENCODING but the unreachable stream still opens its
file on a later tick, creating it, and with the default 'w' flags
truncating it. The runtime threw before scheduling the open; it now
constructs the stream first, as Node does.

Corpus 5547 ran that check on the file its other blocks write, so the
late truncation raced with the fd block's write and Node's own output
changed with the timing (it printed an empty file on macOS). The check
now uses its own file, and the program prints that the file was created.
2026-10-16 00:46:22 -05:00
Chris Tate 00bb57ac0a Merge main 2026-10-16 00:37:00 -05:00
Chris Tate 77d9970b8c Revert "Return unions with reference arms in registers" 2026-10-16 00:35:43 -05:00
Chris Tate a8d154f370 Report what keeps a region with SCRIPTC_REGION_DEBUG
A released region that something outside still refers to is kept, and
SCRIPTC_REGION_STATS only said how many references the walk could not
explain. In a build with --optimization=dev (or --sanitize), setting
SCRIPTC_REGION_DEBUG=1 (or a larger number, the line limit) makes each
attempt that keeps a region also print what keeps it:

- per chunk, the references to its objects that the walk did not find
  (from variables, other regions or objects it does not walk), with the
  classes the chunk holds, since counts are kept per chunk;
- each ordinary object the region owns that is referenced from outside,
  named by the region class and field it was reached through, with its
  reference count and the references the walk found.

The emitted region class descriptors gain the names of the fields their
visits report, in visit order. Debug walks give every owned object an
entry, so that each one records where the walk first reached it. The
bookkeeping is compiled only into unoptimized and RC-audit runtimes:
compiled in, it made the release walk 6.5% slower even with the variable
unset. Optimized runtimes walk as before and, when the variable is set,
say once that the reports need a dev build. Retains, releases and
allocation are unchanged.

Adds corpus program 5703.
2026-10-16 00:35:06 -05:00
Chris Tate 93bc902cd8 Emit Atomics operations inline as atomic instructions
Atomics.load, store, exchange, compareExchange, add, sub, and, or and xor
on an integer typed array compile to one sequentially consistent LLVM
atomic instruction on the element. An index that is not an integer in
range takes the runtime operation, which applies ToIndex and raises its
errors.
2026-10-16 00:31:52 -05:00
Chris Tate 8d8e7a7737 Run Atomics on shared storage without a lock (#1090) 2026-10-16 00:27:39 -05:00
Chris Tate 3005acc14f Deliver fs callbacks without a timer turn
The fs error-first callbacks (readFile, writeFile, stat, access, mkdir and
the other callback forms that complete synchronously) and fs.exists
delivered their callback through a zero-delay timer. Timers have a 1 ms
floor, so every callback waited about 1 ms: 5000 sequential readFile
callbacks took 5.5 s, where Node takes about 0.35 s.

The callbacks now go on a FIFO queue that the event loop drains like the
worker-pool completions: one callback per checkpoint, with nextTicks and
microtasks drained between callbacks. A turn delivers the callbacks that
were queued when its timers ran, as libuv's poll phase does, so a
callback that queues another leaves room for timers and immediates.

Corpus 5555 covers the order against the synchronous code and its
microtasks, and timers and immediates during a chain of 2000 callbacks.
2026-10-16 00:26:28 -05:00
Chris Tate d64d19bf1e Split the native frontend test across three files (#1088) 2026-10-16 00:25:32 -05:00
Chris Tate 42954a3266 Add mock.fn and t.mock.fn with Node's calls log and implementation controls
node:test's mock.fn(original[, implementation]) and t.mock.fn compile.
A Mock<F> is the function-with-properties record { %call, mock }: the
callable is a wrapper closure that shares the mock context with the mock
field. The context records each call's arguments, result, error and this,
and keeps the current implementation, the original and the one-time
implementations by call index. fn.mock supports, as in Node 24.22:

- calls, read as a copy of the log;
- callCount();
- mockImplementation(fn);
- mockImplementationOnce(fn[, onCall]), for the next call by default, a
  later registration for the same index replacing the earlier one;
- resetCalls();
- restore(), which brings back the original.

A throwing implementation records the error and rethrows it. Functions
with fixed parameters or a single rest array parameter are supported; a
union return type needs an undefined arm. mock.fn() without an original,
mock.method, the tracker's reset and restoreAll, and t.mock.method are
refused with SC2020.

The node:test fixture mock-fn compares the behavior with Node, and
tests/diagnostics/mock-fences.ts covers the refusals.
2026-10-16 00:22:22 -05:00
Chris Tate e97249791c Keep region counting out of the hot path of programs without regions (#1087) 2026-10-16 00:08:07 -05:00
Chris Tate 5528d85b81 Add the zlib stream classes (#999) 2026-10-16 00:07:48 -05:00
Chris Tate 7b83558f52 Run Atomics on shared storage without a lock
Atomics operations on 8-, 16- and 32-bit integer arrays are sequentially
consistent atomic instructions on the element, and no longer take the
process-wide shared-memory lock. Other reads and writes of shared storage
are relaxed atomic accesses: element reads and writes access the element,
and copies, fills, conversions, searches and sorts read shared bytes into
a private copy and write back only the range they change, so concurrent
accesses to the same bytes are free of data races. Non-shared storage
keeps its plain memory accesses. Atomics.wait and Atomics.notify queue
waiters in buckets chosen by location, each with its own lock and
condition variable, so waiting threads no longer serialize other Atomics.

Atomics.store answers +0 for a value that truncates to -0, as Node does.
The shared storage runtime test also runs under ThreadSanitizer.
2026-10-16 00:06:31 -05:00
Chris Tate de0917284d Merge main 2026-10-16 00:05:07 -05:00
Chris Tate 93dcd9865e Merge main 2026-10-16 00:01:13 -05:00
Chris Tate 524af28870 Add fs read and write streams
fs.createReadStream, fs.createWriteStream and FileHandle
createReadStream, createWriteStream and readLines return runtime
Readable and Writable streams (scr_fs_stream.c), as do new fs.ReadStream
and new fs.WriteStream in JavaScript sources. They follow
lib/internal/fs/streams.js on the stream engine:

- _construct opens the file, then emits 'open' (with the fd) and
  'ready'; _read reads up to end; _write retries short writes as
  writeAll does; _destroy waits for I/O in flight, then closes (with
  fsync first for the flush option). Every open, read, write, fsync and
  close runs on the shared worker pool.
- Options are checked in Node's order: getOptions, options.fd or the
  path, start, end, then highWaterMark, encoding, emitClose and
  autoClose. flags and mode are checked when the file opens, so their
  errors arrive as 'error' events.
- bytesRead, bytesWritten, pending, path and close(cb) compile, also
  through untyped values (close() there).

A FileHandle stream holds a reference on the handle; closing the stream
releases it and closes the handle unless autoClose is false.
FileHandle.close() follows Node's reference count: it waits for the
streams over the handle and runs their close(). readLines is a readline
Interface over createReadStream with crlfDelay: Infinity.

The stream engine gains Node's _construct (reads, buffered writes,
finish and _destroy wait for construction; destroy errors combine with
aggregateTwoErrors) and _writev, both internal. Writable.destroy() now
runs the buffered write callbacks and end() callbacks (errorBuffer),
which it dropped before. highWaterMark in a dynamic options record is
checked as getHighWaterMark does, and for await over a constructing
stream waits for construction.

Numeric flags, the fs, signal and windowsHandle options, non-UTF-8
write encodings, option functions that replace the file implementation,
and instanceof fs.ReadStream/WriteStream are refused.

Corpus 5546, 5547 and 5554 cover both lanes against Node 24.22.
2026-10-15 23:58:50 -05:00
Chris Tate 9acf71cfe3 Split the native frontend test across three files
self-hosting-native-frontend.test.ts was one test of 577 s (plain) and 607 s
(sanitized) in the cost table, and 748 s on san 1/20 in run 38835202089; once
the other long files were spread, it alone bounded the Linux test lane. It
builds the compiler frontend into a native executable, then checks 17 corpus
programs and four other cases with it. Locally the build took about 189 s and
the checks 297 s (14-20 s per program).

The build and the checks move to tests/harness/native-frontend.ts without
changes to what they check; useNativeFrontend builds the executable in a
beforeAll of the calling suite. self-hosting-native-frontend.test.ts,
-programs.test.ts and -checks.test.ts each build once and run programs 1-6,
7-12, and 13-17 plus the four other checks, each check as its own test. The
diagnostics, library contract and FFI checks use project directories of their
own instead of rewriting one shared main.ts.

scripts/ci-test-costs.json replaces the old file's entry with estimates for
the three files (the build share of its measured cost plus each file's share of
the checks); the next refresh measures them, since vitest's file time includes
beforeAll. The largest side-queue share per shard at 20 shards goes from 580 to
367 s (plain) and from 608 to 432 s (sanitized), for two more builds per flavor.
2026-10-15 23:46:14 -05:00
Chris Tate ebbc3d4519 Keep region counting out of the hot path of programs without regions
In a program that constructs a LifetimeRegion, the retain and release of
every ProgramLifetime class count references in the chunk that holds the
instance, once the process has created a region. Speed builds inline those
bodies into every use of an instance. A program that references
LifetimeRegion but never creates a region still carried the counting path
inline at every use, and its walks over ProgramLifetime graphs ran about
10% slower than the same program without the reference.

The flag test that skips counting is now marked likely, so the counting
path is laid out away from the callers' hot code. Programs that create
regions take the same path as before.
2026-10-15 23:45:56 -05:00
Chris Tate 399f8681a2 Let vitest order the corpus files again (#1083) 2026-10-15 23:45:31 -05:00
Chris Tate c6b6ac1d02 Split npm-static.test.ts across the Linux shards by measured test cost (#1084) 2026-10-15 23:39:13 -05:00
Chris Tate 7678b91406 Add TestContext hooks, plan and fullName, and Node's todo and only rules (#1086) 2026-10-15 23:38:17 -05:00
Chris Tate 9b452b88de Merge main 2026-10-15 23:33:48 -05:00
Chris Tate 9b9a54e71b Check the certificate against the host when servername is empty
An https.request with servername: "" cleared the TLS hostname, which
also turned off the certificate's name check, so a certificate for any
other name passed verification. Node's empty servername only stops SNI
from being sent; checkServerIdentity still checks the certificate
against the request's host.

The request now sends no SNI and the verify callback checks the leaf
certificate against the host with mbedTLS's name rules, so a mismatch
fails with ERR_TLS_CERT_ALTNAME_INVALID as it does without servername.
Corpus 5654 covers an empty servername against a certificate for the
host and one for another name, and the SNI names the server receives.
2026-10-15 23:33:11 -05:00
Chris Tate 681f5ceee7 Add TestContext hooks, plan and fullName, and Node's todo and only rules
node:test changes, matching Node 24.22:

- t.before, t.after, t.beforeEach and t.afterEach lower. t.before runs
  when it is called; a throw fails the test and the subtests that follow,
  which skip their body and beforeEach hooks and run their afterEach hooks.
  t.beforeEach and t.afterEach wrap every descendant subtest. t.after runs
  after the test's afterEach chain, also when the body throws, and a throw
  fails the test.
- t.plan(count) counts the test's t.assert calls and subtests. A mismatch
  fails the test with Node's "plan expected N assertions but received M"
  string, unless an assertion failed first.
- t.fullName joins the names from the outermost suite or test with " > ".
- Tests and suites under a todo suite or test are todo, with a bare TODO
  directive, unless they are skipped or carry their own todo message. A
  todo suite reports its directive.
- describe.only, suite.only and { only: true } on a suite mark it. At each
  level, when a child is marked or contains a marked test or suite, only
  those children run.
- The failing-tests section lists failures in completion order, so
  subtests come before the test that ran them, and prints thrown strings
  quoted, as Node inspects them.

The fixtures ctx-hooks, ctx-plan, todo-suites, suite-variants,
only-nested and thrown-values compare the reporter output and exit code
with Node.
2026-10-15 23:16:04 -05:00
Chris Tate dcd98ae5e5 Walk owned objects with a single reference without an entry (#1082) 2026-10-15 23:01:00 -05:00
Chris Tate 867f7944b4 Lower util.format %f (#1060) 2026-10-15 22:56:18 -05:00