Commit Graph
2166 Commits
Author SHA1 Message Date
Malte Ubl caee8dfef2 Merge the restacked base into ni/i2-handles
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/index.ts
2026-10-27 05:41:01 -07:00
Malte UblandClaude Opus 5.5 02a322e712 Merge ni/13-libnode-packs (with ni/1-libnode-loader)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 05:40:40 -07:00
Chris Tate f4a4cf8652 Merge main 2026-10-27 04:32:00 -05:00
Chris Tate 7bd12ea89f Add a representation report and a heap census for ranking value-layout work (#2390) 2026-10-27 04:01:41 -05:00
Chris Tate 60372c09a4 Declare the heap census writer in scr_lib.h
The LLVM backend can emit scr_census_write, which a census program
defines and the runtime calls through a weak reference, so the header
declares it as it does scr_detached_exit.
2026-10-27 03:28:32 -05:00
Chris Tate 429aa8273d Keep the heap census hook out of library archives
Libraries never build the census (heapCensusOf), but the runtime's weak
scr_census_write reference made library archives list an external
symbol outside their declared set. Declare and call it only in programs.
2026-10-27 02:49:05 -05:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 6451e846c6 Reset the prefault helper's state in forked children (#2391) 2026-10-27 02:10:51 -05:00
Chris Tate 59ec7998fe Reset the prefault helper's state in forked children
fork() copies only the calling thread, so a child forked while the
allocator's prefault helper held scr_sa_pf_lock waited on that lock forever
at its first fault-in request, and a child that released the range the helper
was faulting in waited forever for the helper to finish it. Native builds'
emit workers hung this way, with the parent waiting in waitpid.

The helper's first start now registers pthread_atfork handlers. The prepare
handler takes the lock across the fork and the parent handler releases it.
The child handler resets the ring, the busy range, the waiter state and the
heap polling state, reinitializes the lock and both conditions, and marks the
helper not started, so the child's first request starts a helper of its own.

check_fork_while_prefaulting in scr_alloc.test.c forks while the helper
faults in 64 MiB, over 64 rounds; each child releases that range, faults in a
range of its own and must exit within five seconds. ThreadSanitizer builds
skip it.
2026-10-27 01:35:32 -05:00
Chris Tate 8b256639f4 Add a representation report and a heap census for ranking value-layout work 2026-10-27 01:31:00 -05:00
Chris Tate 47904df3c2 Decode short non-ASCII strings directly in charCodeAt
charCodeAt on a non-ASCII string mapped the UTF-16 index to a byte offset
through a string index entry, which the first read claimed and the
string's release purged. For a string of one character, such as each
character of a for-of loop that codePointAt(0) classifies, the entry cost
more than the read.

A non-ASCII receiver of up to 16 bytes is now decoded from its start on
each read, as scr_str_utf16_len already counts short receivers. Longer
receivers keep the index entry and its cursor.
2026-10-27 01:07:00 -05:00
Chris Tate 98cc35de09 Find the code segment that holds the runtime under musl too
The constructor took the program object with an empty name for the main
program, which is how glibc reports it. musl reports the main program
with its path, or "/proc/self/exe" in a static executable, so on musl
targets --large-pages found no segment and left the code file-mapped.
The constructor now takes the executable segment that holds its own code:
the runtime links into the executable, so that is the main program's
code under either C library.

scr_text_pages.test.ts runs its check against a static musl build made
with zig as well as the glibc build.
2026-10-26 23:55:27 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Chris Tate df77671e33 Merge main 2026-10-26 23:13:34 -05:00
Chris Tate 80b851d65f Write untyped stores into live records and keep one object across prefix casts (#2366) 2026-10-26 22:56:27 -05:00
Chris Tate 79f456d665 Keep the static hello binary small under GNU ld (#2373) 2026-10-26 22:36:03 -05:00
Malte UblandClaude Opus 5.5 70c39cca49 Keep frozen records frozen for untyped code and ignore refused sloppy writes
Object.freeze of a fresh record or array literal now sets the native
frozen bit instead of compiling to the literal, so an untyped reference to
the value sees it frozen: Object.isFrozen answers true and strict-mode
writes throw Node's TypeErrors.

Untyped property writes in sloppy-mode code (IrModule.sloppyDynWrites,
decided per write by isStrictCode) call scr_dyn_key_set_sloppy and its
computed form, which drop exactly the TypeError a refused assignment raises
(a read-only or frozen property, a non-extensible object, a frozen array
element) while a setter's exception still propagates. Computed-write
dispatch helpers are shared only by writes of the same mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:16:30 -07:00
Chris Tate 80138011ed Allocate template literals at their exact length (#2369) 2026-10-26 21:55:50 -05:00
Chris Tate ad29d598f5 Merge main into the untyped-store branch 2026-10-26 21:44:47 -05:00
Malte UblandClaude Opus 5.5 019fc51e31 Keep the generator tag reader and the glibc math unit out of hello
The static hello binary grew past the 96 KiB ceiling of the runtime
tree-shaking test on Linux hosts that link with GNU ld (99,040 B). CI does
not see it: its pinned LLVM ships ld.lld, which the compiler prefers, and
hello links to about 65 KB there. The Sandbox gate installs Ubuntu's
clang-22 without lld, so it links with GNU ld.

dd418b067 crossed the ceiling (98,032 -> 98,960 B). Its scr_glibc_math.c
defines floor, fmod, frexp and other glibc names as hidden functions so a
program can start without libm. With GNU ld that cannot work: ld decides
--as-needed before section GC, so the runtime's unreachable code keeps
libm needed, and it keeps every regular definition of a name a needed
shared library defines, hidden or not. Every executable carried the whole
unit (about 2.5 KB of code and unwind data) and still loaded libm. Links
without lld now leave the unit out; its functions answer what glibc's do.

192ea3247 installed scr_gen_tag_reader from a load-time constructor in
scr_async.c. A constructor is a link root on every format, so each
executable kept scr_gen_tag and, with GNU ld, all of scr_async.c's string
literals (about 1.1 KB). The generator runtime now installs the reader
when it creates a generator, the only time a reader can be needed.

Hello is now 93,928 B with GNU ld and 65,472 B with lld.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 19:35:52 -07:00
Chris Tate 48ccee523e Allocate template literals at their exact length
A template literal of strings and numbers reserved 32 bytes for each
number, the longest number text, and kept that capacity for the result's
lifetime: an 8-byte name built from two numbers used a 96-byte block. The
numbers now format into a stack buffer first, so the result is allocated
once at its exact length.
2026-10-26 21:01:36 -05:00
Chris Tate 87565abc16 Merge main 2026-10-26 20:57:59 -05:00
Chris Tate 1245b18469 Search ASCII strings from a start index without index conversion
String.prototype.indexOf took its byte-search fast path on a proven-ASCII
receiver only when the start index was not positive. A positive start went
through the general path: ToIntegerOrInfinity, the surrogate-needle check,
and UTF-16 to byte conversions in both directions, although byte offsets
are UTF-16 indexes in an ASCII string. The fast path now covers every start
index: a positive start truncates and clamps to the length, where only the
empty needle is found, and the search runs from that byte offset.
2026-10-26 18:22:22 -05:00
Malte Ubl 69ecfbe1fa Allocate the Node island handle tag by name
#2277 numbers new handle tags from their names, so the islands' tag no
longer picks a value that another branch can take.
2026-10-26 16:00:46 -07:00
Malte Ubl 437ad4ae38 Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD 2026-10-26 16:00:05 -07:00
Malte Ubl 4063f1e1f7 Merge remote-tracking branch 'origin/ni/islands-base' into HEAD 2026-10-26 15:59:32 -07:00
Malte Ubl 05584c5bf7 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into HEAD 2026-10-26 15:55:46 -07:00
Chris Tate 7e225c5d87 Merge main 2026-10-26 17:33:06 -05:00
Malte UblandClaude Opus 5.5 c2f9b49113 Write untyped stores into live records and pass prefix records by pointer
An untyped write of a present declared field of a live record capsule now
stores into the native record through a per-shape `set` hook (checked like
the commit, refusing to the snapshot path for values the field cannot hold),
so the capsule keeps answering from the record instead of turning into a
permanent snapshot. Object.assign from a plain object takes the same path,
and checked conversions and matches read a capsule through a fresh,
uncached snapshot.

A checked conversion of a record capsule into a record type whose storage
is a same-offset prefix of the capsule's record (and whose extra fields hold
no references) passes the pointer: both names keep one object, writes show
through either, and the record remembers its own capsule parameters, so
every later box of it and the narrower type's reflective surfaces answer
for the whole object (decision D3). Other cross-layout conversions keep the
refreshed copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 15:05:27 -07:00
Chris Tate 3c83fcb89c Allocate object groups that die together in arenas (#2348) 2026-10-26 16:54:31 -05:00
Chris Tate c5e8a876ec Store ordinary objects as a layout plus value slots (#2353) 2026-10-26 16:18:38 -05:00
Malte UblandClaude Opus 5.5 3ea7031ea1 Merge the ASan fix for nested entries on macOS
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 13:34:29 -07:00
Malte UblandClaude Opus 5.5 6748313cba Unpoison a nested entry's ending frame for ASan on macOS
The nested trampoline never returns (setcontext back), so the redzones
around its locals stayed poisoned in the shadow of Node's stack, and
Node's own frames tripped over them later (stack-buffer-underflow in
__bzero, sanitized corpus 6822/6823/6831/6832/6894 on macOS). Linux's
ASan clears a context's stack in its swapcontext interceptor; macOS's has
none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 13:34:14 -07:00
Chris Tate 133551d76e Build node:zlib on the zlib Node bundles
The runtime vendored zlib 1.3.1 and archived it at -Os, and host C-lane
builds linked the system libz, so deflate, raw deflate and gzip output
differed from Node's at every level. The vendored copy is now Node
v24.22.0's deps/zlib (Chromium's zlib, 1.3.2.1-motley-285e94b), built at
-O3 with the units and defines Node's zlib.gyp selects per target: the
SSSE3 Adler-32 and SSE2 inflate and slide-hash code on x86-64, the NEON
and ARMv8 CRC-32 code on arm64, and the portable code on wasm32. The SIMD
units check the CPU at run time and give the same output, so compressed
bytes match Node's on every target.

zlibInputs (runtime-pack-common/scripts/zlib-inputs.mjs for the runtime
packs, vendor-inputs.ts for Zig cross builds and the sanitized runtime)
holds the per-target table, and a unit test keeps the two identical.
Every C-lane driver now links the vendored objects instead of -lz.
Chromium's BSD license ships beside zlib's in the runtime packs.

Corpus 8405 compares compressed bytes with Node across levels,
strategies, memLevel, windowBits, dictionaries and flushing streams;
8406 covers chunked inflation with small input pieces and output chunks.
2026-10-26 15:12:51 -05:00
Malte Ubl 4c68e95384 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into mac/ni1 2026-10-26 12:50:45 -07:00
Malte UblandClaude Opus 5.5 704e6b4992 Merge the macOS island fixes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:50:41 -07:00
Malte UblandClaude Opus 5.5 bfbbd349a3 Merge the macOS island fixes, and make nested entries' contexts the same way
A nested entry below the mark makes its context with ni_make_context,
so macOS clears only the top slice of the range instead of the whole of
Node's stack below the mark on every nested call (17 ms per call, and
the cross-heap GC test ran past its timeout).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:50:40 -07:00
Malte UblandClaude Opus 5.5 b1005549c3 Merge the macOS island fixes of ni/1-libnode-loader
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:50:39 -07:00
Malte UblandClaude Opus 5.5 50385349d3 Merge the macOS island fixes of ni/13-libnode-packs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:50:39 -07:00
Malte UblandClaude Opus 5.5 df869f28b0 Run Node islands on macOS, and test them in the macOS CI lanes
macOS binds libnode's own calls to v8::base::Stack inside libnode, so the
bridge's definitions cannot give V8 Node's stack as they do on Linux; V8
took the main thread's stack start and its conservative scan faulted in
the gap above Node's stack. While Node starts on its stack, the main
thread's pthread record now names that stack, which V8 reads once and
keeps.

macOS's makecontext clears the whole stack it is given, so it gets only
the top slice of Node's stack (no 64 MiB pass per context).

CI: dev-pack.json pins mirrored darwin-arm64 and darwin-x64 libnode
builds of the pinned Node 24.22.0 from the libnode-packs workflow, the
dev-pack action serves either platform, and the macOS lanes provision it,
so island tests fail rather than skip on macOS CI. The arm64 LLVM job
runs the island harness; the Intel job runs it and the island corpus.
bridge-smoke loads the .dylib bridge on macOS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:50:38 -07:00
Chris Tate 2e627dab79 Merge main 2026-10-26 14:45:37 -05:00
Chris Tate f12b2b041d Cheaper trial deletion in the cycle collector 2026-10-26 14:34:49 -05:00
Chris Tate 9c7d552fdc Link the arena runtime into the runtime C tests 2026-10-26 13:26:29 -05:00
Malte Ubl 39a125ec38 Move the Node island handle tag to 84
Main's navigator global took tag 83.
2026-10-26 11:25:16 -07:00
Malte Ubl 3a5aab5b2f Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD
# Conflicts:
#	packages/compiler/src/cli/command.ts
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/frontend/lowering/builtins/module-resolution.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/compiler/src/index.ts
#	packages/runtime/src/scr_json.c
#	tests/harness/npm.test.ts
2026-10-26 11:22:14 -07:00
Malte Ubl f5b1e5f007 Merge remote-tracking branch 'origin/ni/islands-base' into HEAD
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/node-islands/libnode-pack.ts
#	packages/libnode-darwin-arm64/package.json
#	packages/libnode-darwin-x64/package.json
#	packages/libnode-linux-arm64-gnu/package.json
#	packages/libnode-linux-x64-gnu/package.json
#	packages/node/package.json
2026-10-26 11:20:55 -07:00
Malte Ubl c088613033 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/3-hosted-loop 2026-10-26 11:15:41 -07:00
Malte Ubl 5817be4d0f Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/ir/function-references.test.ts
2026-10-26 11:08:44 -07:00
Chris Tate 68e2556fda Merge main 2026-10-26 12:24:22 -05:00
Chris Tate 18755e0599 Allocate object groups that die together in arenas 2026-10-26 11:41:21 -05:00