Commit Graph
2159 Commits
Author SHA1 Message Date
Chris Tate 919b79b62a Revert "Keep frozen checked records frozen, and give sloppy-mode this the global object" 2026-10-27 19:20:37 -05:00
Chris Tate dfb7afcdd7 Load Node.js in-process on first use: libnode bridge and runtime loader (#1685) 2026-10-27 18:48:02 -05:00
Chris Tate e941d5d659 Run JavaScript WritableStream, TransformStream and pipes natively (#2437) 2026-10-27 18:43:35 -05:00
Chris Tate ccd87f44fc Keep the scr_map and three-library tests inside their timeouts (#2441) 2026-10-27 18:35:52 -05:00
Chris Tate ea8c1e6a34 Keep frozen checked records frozen, and give sloppy-mode this the global object (#2436) 2026-10-27 18:33:05 -05:00
Chris Tate 90471a180d Read a proxy's integrity from its target, and give sloppy this the program's global object
Records checked from a proxy in a program that freezes untyped values no
longer fail on the proxy's integrity query: a proxy is frozen exactly when
its target is. A sloppy-mode function called with undefined or null
receives the global object that globalThis reads, so its builtin members
such as console answer before anything else reads the global object. The
new corpus programs are named instead of numbered and record their CI
costs.
2026-10-27 17:54:43 -05:00
Chris Tate 4e98e5e9bd Call web stream methods past own overrides, and await a transformer's thenable start
A method value of a web stream class, such as a saved
WritableStream.prototype.close, called on an instance whose own property of
the same name wraps it, ran the own property again, so the wrapper recursed.
The classes' method values now call the class's method directly
(ScrDynHandleClass.method_invoke); member calls still look at own properties
first.

TransformStream resolved its start promise with a transformer start's
result without adopting a thenable, so both sides started at once and a
rejection was lost. The start promise now resolves as a promise's resolve
function does (scr_dyn_promise_resolve_with): a thenable adopts through its
resolve job, a throwing `then` read rejects, and the stream starts or errors
when it settles, in Node's order.

Lowerer.namesWebStreamClass, a cache made when first needed, is listed in
Lowerer.unjournaled.
2026-10-27 17:53:47 -05:00
Malte UblandClaude Opus 5.5 54aba6a606 Build the scr_map semantics test at -O0
The scr_map unit test compiles 23 runtime translation units with
ASan/UBSan, and that build is essentially the whole test: 90s of clang
against a 2.1s binary run. scr_hash.c alone takes 29s at -O1 because the
sanitizer instruments its unrolled digest rounds; scr_json.c and
scr_lib.c add 28s more. The test then normally ran within 20s of its
120s timeout, and whenever a lane shared the box it repeatedly timed out
in both the plain and sanitized lanes.

The link set cannot shrink: scr_map.c's dyn ops need scr_dyn_* from
scr_json.c, scr_lib.c supplies the remaining date/math/number/perf
symbols, and scr_lib.c needs scr_hash.c. The test asserts Map semantics
under sanitizers, not codegen, and its cost checks are relative bounds,
so build it at -O0: every assertion and sanitizer check still runs and
the build drops to about 15s. Eight concurrent copies now take 20-22s
each instead of 90s, with the timeout unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 15:26:28 -07:00
Chris Tate 6e5f21c580 Reuse keep-alive connections across fetch() requests (#2379) 2026-10-27 16:45:35 -05:00
Chris Tate 1a86f94239 Merge main 2026-10-27 16:45:14 -05:00
Chris Tate 38bc4ed67b Pad ASCII strings without indexing their UTF-16 units (#2392) 2026-10-27 16:44:40 -05:00
Chris Tate f945efecee Run JavaScript WritableStream, TransformStream and pipes natively
In JavaScript sources, WritableStream with its default writer and
controller, TransformStream with its controller, pipeTo and pipeThrough,
and TextEncoderStream and TextDecoderStream join the native web stream
classes, following Node 24's webstreams step by step: sinks and
transformers, backpressure and the writers' ready and closed promises,
abort with the controller's signal, the pipe's shared write requests and
its abort signal, and UTF-8 encoding and decoding across chunks settle in
Node's order. The streams take own properties, which a pipe does not use
for its own reads and writes, and the global object answers the classes
by computed name in programs that name them.

A JavaScript class that extends one of these classes keeps the SC1090
refusal, and TextDecoderStream refuses encodings other than UTF-8.

The node24-web-stream lifecycle, errors, queues, transforms, pipes,
pipe-errors, pipe-abort, tee and encoding cases now match Node.
2026-10-27 15:55:31 -05:00
Chris Tate 62d4cecd3a Keep frozen checked records frozen, and give sloppy-mode this the global object 2026-10-27 15:22:08 -05:00
Chris Tate 5dc4a4daa6 Map large executables' text with 2 MiB pages 2026-10-27 15:04:45 -05:00
Chris Tate cc6ee85fd7 Run JavaScript web streams on a native ReadableStream (#2411) 2026-10-27 14:20:02 -05:00
Chris Tate 349feb5e47 Capture stack traces with Error.captureStackTrace as Node does (#1809) 2026-10-27 13:49:06 -05:00
Chris Tate 1feed2a3ec Answer cross-layout record casts with views of the original record (#2419) 2026-10-27 13:47:06 -05:00
Chris Tate 4bd4c43efd Check Uint8Array chunks in clones of JavaScript ReadableStream bodies
Response.clone and Request.clone tee the body. The tee branches of a body
read from a JavaScript ReadableStream did not keep that body's chunk check,
so consuming the original or the clone took string chunks as text. The
check is now a flag on the body's stream that its tee branches inherit, and
consuming either branch rejects with Node's TypeError "Received
non-Uint8Array chunk".
2026-10-27 13:28:33 -05:00
Chris Tate 9bdb36ed46 Report published values as frozen, sealed and not extensible
Under Node.js, publish() deep-freezes everything it reaches, so
Object.isFrozen and Object.isSealed return true and Object.isExtensible
returns false for a published object, array, Map or Set. Native builds
already refused writes into published values, but answered these queries
as for an unpublished value: publication marks an object immortal and sets
no frozen bit.

- record.isFrozen, array.isFrozen and the extensibility test of Maps and
  Sets also read an immortal rc word as published, the test of the
  published write guards, for the types the program may publish
  (PublishedTypes). Other types emit the code they did.
- scr_dyn_is_frozen, scr_dyn_is_sealed and scr_dyn_is_extensible answer
  for a published object, array, Map or Set, or a checked view of one,
  before materializing the view.
2026-10-27 12:57:41 -05:00
Chris Tate 0c954ebfd6 Reserve fewer worker arenas before smaller class slices on hosts that limit address space
Worker runtimes reserve one region for 32 arenas of 40 class slices. When
the host refused the full region (32 arenas of 1 GiB slices, 1.28 TiB of
address space), the allocator kept 32 arenas and shrank every slice. Under
valgrind, which maps at most 56 GiB, slices dropped to 16 MiB, and every
class whose live blocks outgrew 16 MiB took the system allocator.
Callgrind profiles then counted system allocations that a native run
never makes: tsc-ts checking VS Code made 10.3 M small calloc calls under
valgrind and 0.06 M natively.

The reservation now tries arena counts from 32 down to 4 at each slice
size, from 1 GiB down, before smaller slices, and 2 and 1 arena only when
no slice size fits 4. Arenas that are not reserved stay claimed, so
threads beyond them use the system allocator, and the region bounds of
foreign frees use the arenas reserved. Hosts that map the full region keep
32 arenas of 1 GiB slices.

A runtime test limits RLIMIT_AS to 60 GiB and checks that the first thread
gets 256 MiB slices for 400,000 blocks of 56 bytes, that three more
threads get arenas and a fourth uses the system allocator, and that blocks
freed across threads keep their contents.
2026-10-27 10:43:05 -05:00
Chris Tate f12a2b2a23 Take JavaScript ReadableStreams as fetch bodies and in instanceof by value
A JavaScript ReadableStream given to Response, Request or fetch as a body
was converted to the string "[object ReadableStream]". The fetch unit now
reads such a stream through a reader of its own. A locked or disturbed
stream gives Node's TypeError, and the chunks must be Uint8Array
when the body is consumed ("Received non-Uint8Array chunk").

instanceof against a web stream class held in a variable or passed as an
argument walked no prototype for handles and answered false. It now starts
at the class prototype of the handle. The fetch unit's own streams, readers
and controllers, such as Response.body, are instances of the JavaScript
classes, so `response.body instanceof ReadableStream` is true again.

These paths, and the fetch reader functions' hand-off to web stream
readers, are in SCR_HANDLE_CLASSES variants of scr_json.c and scr_fetch.c,
which only programs with JavaScript web streams link; other programs that
use fetch build to the same size as on main.
2026-10-27 10:00:00 -05:00
Chris Tate 8f3a8e3e63 Merge main 2026-10-27 09:58:30 -05:00
Malte UblandClaude Opus 5.5 85d2a63410 Keep a view's viewed record in its record-source entry instead of trailing words
View target structs no longer grow: every stack copy of such a record grew
by the two words, which enlarged unoptimized frames enough for the
sanitized self-rebuild to run out of stack. The guard's cold path looks the
slot up through the view's record-source entry (scr_record_view_slot),
which now holds the viewed record and its offsets, and stack objects of
target types are allowed again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 06:39:05 -07:00
Malte UblandClaude Opus 5.5 47dfa71ab1 Keep the view guard to one value in unoptimized builds
Unoptimized builds give every value a frame slot, so the inline guard
grew the frames of large recursive functions enough for the sanitized
self-rebuild to run out of stack. Development builds now call
scr_record_view_field for the slot address; optimized builds test the view
mark as one range (the frozen bit plus a count) before the cold lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 06:39:04 -07:00
Malte UblandClaude Opus 5.5 3f772c0ee6 Look up view slots out of line behind a cold branch
The view guard's slot lookup moves into scr_record_view_slot, and the
branch to it is weighted as unlikely, so a field access of a view target
type adds one rc-word test to its hot path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 06:39:04 -07:00
Malte UblandClaude Opus 5.5 10185c2147 Mark record views with the frozen bit instead of testing a trailing word
A value of a view target type can point at another storage (a shared
field read of a shape union, for one), whose block ends before the view
words. The view mark is now the rc word's frozen bit, which no other value
of a view target has (targets are never freezable; an immortal word is no
view), and the trailing words are read only after it. View targets release
like freezable shapes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 06:35:26 -07:00
Malte UblandClaude Opus 5.5 fe34846a54 Keep tagged record shapes out of record views and declare the view entry point
A tagged shape's values share field reads with the other arms of its shape
union, which carry no view words, so tagged shapes keep their copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 06:35:26 -07:00
Malte UblandClaude Opus 5.5 6a30f02a69 Answer cross-layout record casts with views of the original record
A checked conversion of a capsule of record A into record type B, when A
has all of B's fields with the same types in any order, now answers a view:
a B object whose two trailing words hold A and A's offsets for B's fields.
Every field address of a B value tests the first word (recordViewFieldPtr,
the record clone and the JSON walker), so reads and writes through either
name reach the same storage. The view's record-source entry owns A's
capsule: boxing the view answers that capsule, so identity holds, B's
reflective surfaces and spreads see the whole object, and the entry dies
with the view, which releases A's capsule. The capsule finds its views
again through borrowed cast-list entries.

Only the target types of such conversions carry the words and the test.
They are planned from the IR before emission (record-views.ts): record
types a checked conversion can produce for which another record type in
the module has all their fields. Programs without checked conversions are
unchanged. This replaces the prefix pointer pass, whose narrower release
could not free the source's extra fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 06:35:26 -07:00
Chris Tate adbf74802f Free a worker's group-arena pool when the worker exits (#2413) 2026-10-27 07:54:31 -05:00
Chris Tate 635f732a34 Free a worker's group-arena pool when the worker exits
Each thread keeps up to 8 MB of arena chunks and up to 64 dropped arenas
for reuse (scr_arena.c). Nothing freed them when a worker thread finished,
so every worker that used a group arena left its pool behind: 40 workers
run one after another, each dropping a 12 MB arena, kept 328 MB resident.

scr_arena_thread_exit frees the calling thread's pooled arenas and chunks.
The worker thread calls it after its context cleanup, before it posts the
exit event. The new harness test runs 24 such workers in sequence and
checks that the resident set grows by less than 32 MiB over the last 20
(160 MiB before this change, 1 MiB after).
2026-10-27 07:17:08 -05:00
Chris Tate 9ca9c7af9d Run JavaScript web streams on a native ReadableStream
In JavaScript sources, ReadableStream, ReadableStreamDefaultReader,
ReadableStreamDefaultController, CountQueuingStrategy and
ByteLengthQueuingStrategy (globals and node:stream/web exports) are class
values over a new runtime unit, scr_webstreams.c, which follows Node 24's
webstreams step by step: underlying sources, queuing strategies, default
readers, tee, ReadableStream.from and async iteration settle their promises
in Node's order. scr_dyn_handle_class.c gives a dyn handle tag a
constructor value with a prototype, brand checks and Node's constructor
errors. Stream state carries cycle headers, so the collector reclaims
streams whose callbacks close over their controllers.

Promise steps run as native promise reactions, a variant of scr_async.c
that only programs with web streams link; every other program links the
default object. TypeScript sources keep their static web stream lowerings.
Byte streams, BYOB readers, pipeTo and pipeThrough throw SC2020.

node24-web-stream-readers and node24-web-stream-readable-lifecycle now
match Node.
2026-10-27 06:53:26 -05:00
Chris Tate f4a4cf8652 Merge main 2026-10-27 04:32:00 -05:00
Chris Tate 7bd12ea89f Add a representation report and a heap census for ranking value-layout work (#2390) 2026-10-27 04:01:41 -05:00
Chris Tate 60372c09a4 Declare the heap census writer in scr_lib.h
The LLVM backend can emit scr_census_write, which a census program
defines and the runtime calls through a weak reference, so the header
declares it as it does scr_detached_exit.
2026-10-27 03:28:32 -05:00
Chris Tate 429aa8273d Keep the heap census hook out of library archives
Libraries never build the census (heapCensusOf), but the runtime's weak
scr_census_write reference made library archives list an external
symbol outside their declared set. Declare and call it only in programs.
2026-10-27 02:49:05 -05:00
Chris Tate cb572a7b67 Pad ASCII strings without indexing their UTF-16 units
`toUpperCase` and `toLowerCase` now record that a result of their ASCII
path is ASCII. `padStart` and `padEnd` take the UTF-16 lengths of the
receiver and the fill the way `length` does: a proven-ASCII or short
string is counted directly instead of claiming a string-index entry. A
one-unit fill repeats without a division, a one-byte fill with memset,
and a result padded from an ASCII receiver and fill is recorded as ASCII.

`word.slice(0, 3).toUpperCase().padStart(4, "-")` previously built an
index entry for the three-byte receiver on every call.
2026-10-27 02:44:56 -05:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 6451e846c6 Reset the prefault helper's state in forked children (#2391) 2026-10-27 02:10:51 -05:00
Chris Tate 5a1bc2a6d5 Drain a compressed fetch body that is still buffered when the response ends
Since the transfer drops its socket at the response's 'end', resuming a
read no longer reached the decoder: sf_response_resume_if_ready returned
before sf_inflate_resume once the transfer held no paused socket. A
compressed response whose whole body arrived before it was read then kept
its remaining decoded output buffered, and later reads stayed pending. The
buffered input and decoder output now drain on demand after 'end' as well
as while the socket is paused.
2026-10-27 01:56:31 -05:00
Chris Tate 59ec7998fe Reset the prefault helper's state in forked children
fork() copies only the calling thread, so a child forked while the
allocator's prefault helper held scr_sa_pf_lock waited on that lock forever
at its first fault-in request, and a child that released the range the helper
was faulting in waited forever for the helper to finish it. Native builds'
emit workers hung this way, with the parent waiting in waitpid.

The helper's first start now registers pthread_atfork handlers. The prepare
handler takes the lock across the fork and the parent handler releases it.
The child handler resets the ring, the busy range, the waiter state and the
heap polling state, reinitializes the lock and both conditions, and marks the
helper not started, so the child's first request starts a helper of its own.

check_fork_while_prefaulting in scr_alloc.test.c forks while the helper
faults in 64 MiB, over 64 rounds; each child releases that range, faults in a
range of its own and must exit within five seconds. ThreadSanitizer builds
skip it.
2026-10-27 01:35:32 -05:00
Chris Tate 8b256639f4 Add a representation report and a heap census for ranking value-layout work 2026-10-27 01:31:00 -05:00
Chris Tate 47904df3c2 Decode short non-ASCII strings directly in charCodeAt
charCodeAt on a non-ASCII string mapped the UTF-16 index to a byte offset
through a string index entry, which the first read claimed and the
string's release purged. For a string of one character, such as each
character of a for-of loop that codePointAt(0) classifies, the entry cost
more than the read.

A non-ASCII receiver of up to 16 bytes is now decoded from its start on
each read, as scr_str_utf16_len already counts short receivers. Longer
receivers keep the index entry and its cursor.
2026-10-27 01:07:00 -05:00
Chris Tate 98cc35de09 Find the code segment that holds the runtime under musl too
The constructor took the program object with an empty name for the main
program, which is how glibc reports it. musl reports the main program
with its path, or "/proc/self/exe" in a static executable, so on musl
targets --large-pages found no segment and left the code file-mapped.
The constructor now takes the executable segment that holds its own code:
the runtime links into the executable, so that is the main program's
code under either C library.

scr_text_pages.test.ts runs its check against a static musl build made
with zig as well as the glibc build.
2026-10-26 23:55:27 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Chris Tate df77671e33 Merge main 2026-10-26 23:13:34 -05:00
Chris Tate 80b851d65f Write untyped stores into live records and keep one object across prefix casts (#2366) 2026-10-26 22:56:27 -05:00
Chris Tate 79f456d665 Keep the static hello binary small under GNU ld (#2373) 2026-10-26 22:36:03 -05:00
Chris Tate fefbf3ac35 Reuse keep-alive connections across fetch() requests
The native fetch dialed a new connection for every request and closed it
after the response, so a loop of requests to one origin paid a TCP
connect, and for https a TLS handshake, per request. Node's fetch keeps
connections alive through undici's dispatcher. A direct request now goes
through a keep-alive agent private to fetch (scr_http_fetch_request): an
idle connection for the same host, port and bare hostname serves the next
request, there is no limit on connections per origin, and an idle
connection closes after 4 s or the server's Keep-Alive hint less one
second, without keeping the process alive. Proxied requests still dial
their own connection.

When a response ends, the transfer resumes its socket if it had paused it
and stops pausing and resuming it, since the agent may hand the socket to
the next request. scr_http_request_agent_ex and http.Agent creation share
their agent logic with the new entry point.
2026-10-26 22:25:51 -05:00
Malte UblandClaude Opus 5.5 70c39cca49 Keep frozen records frozen for untyped code and ignore refused sloppy writes
Object.freeze of a fresh record or array literal now sets the native
frozen bit instead of compiling to the literal, so an untyped reference to
the value sees it frozen: Object.isFrozen answers true and strict-mode
writes throw Node's TypeErrors.

Untyped property writes in sloppy-mode code (IrModule.sloppyDynWrites,
decided per write by isStrictCode) call scr_dyn_key_set_sloppy and its
computed form, which drop exactly the TypeError a refused assignment raises
(a read-only or frozen property, a non-extensible object, a frozen array
element) while a setter's exception still propagates. Computed-write
dispatch helpers are shared only by writes of the same mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:16:30 -07:00
Chris Tate 80138011ed Allocate template literals at their exact length (#2369) 2026-10-26 21:55:50 -05:00