Commit Graph
205 Commits
Author SHA1 Message Date
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Chris Tate df77671e33 Merge main 2026-10-26 23:13:34 -05:00
Chris Tate 7be9803076 Build node:zlib on the zlib Node bundles
Re-lands #2355, reverted in #2358. #2361 fixed the CI failure it exposed.
2026-10-26 18:40:07 -05:00
Chris Tate a82a0dda93 Revert "Build node:zlib on the zlib Node bundles"
This reverts #2355. The sanitized three.js nodes CI job has failed on every main commit since it landed.
2026-10-26 17:05:37 -05:00
Chris Tate c5e8a876ec Store ordinary objects as a layout plus value slots (#2353) 2026-10-26 16:18:38 -05:00
Chris Tate 133551d76e Build node:zlib on the zlib Node bundles
The runtime vendored zlib 1.3.1 and archived it at -Os, and host C-lane
builds linked the system libz, so deflate, raw deflate and gzip output
differed from Node's at every level. The vendored copy is now Node
v24.22.0's deps/zlib (Chromium's zlib, 1.3.2.1-motley-285e94b), built at
-O3 with the units and defines Node's zlib.gyp selects per target: the
SSSE3 Adler-32 and SSE2 inflate and slide-hash code on x86-64, the NEON
and ARMv8 CRC-32 code on arm64, and the portable code on wasm32. The SIMD
units check the CPU at run time and give the same output, so compressed
bytes match Node's on every target.

zlibInputs (runtime-pack-common/scripts/zlib-inputs.mjs for the runtime
packs, vendor-inputs.ts for Zig cross builds and the sanitized runtime)
holds the per-target table, and a unit test keeps the two identical.
Every C-lane driver now links the vendored objects instead of -lz.
Chromium's BSD license ships beside zlib's in the runtime packs.

Corpus 8405 compares compressed bytes with Node across levels,
strategies, memLevel, windowBits, dictionaries and flushing streams;
8406 covers chunked inflation with small input pieces and output chunks.
2026-10-26 15:12:51 -05:00
Malte Ubl cc091bac48 Merge remote-tracking branch 'origin/ni/i10-js-island-requires' into HEAD 2026-10-26 11:32:11 -07:00
Malte Ubl b2fc8c1cd8 Merge remote-tracking branch 'origin/ni/i6-subclasses' into HEAD 2026-10-26 11:28:52 -07:00
Malte Ubl a8067be282 Merge remote-tracking branch 'origin/ni/i3-exports' into HEAD 2026-10-26 11:26:23 -07:00
Malte Ubl 0d054133ad Merge remote-tracking branch 'origin/ni/i2-handles' into HEAD
# Conflicts:
#	packages/compiler/src/frontend/lowering/lower-classes.ts
2026-10-26 11:25:53 -07:00
Malte Ubl 3a5aab5b2f Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD
# Conflicts:
#	packages/compiler/src/cli/command.ts
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/frontend/lowering/builtins/module-resolution.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/compiler/src/index.ts
#	packages/runtime/src/scr_json.c
#	tests/harness/npm.test.ts
2026-10-26 11:22:14 -07:00
Malte UblandClaude Opus 5.5 dc1e8acc1d Store ordinary objects as a layout plus value slots
A checked object (SCR_DYN_OBJ) now starts in layout mode: v.obj.layout names
its key sequence, a node of the thread's interned transition tree, and
v.obj.slots holds one 8-byte value per key, where every property used to take
a 48-byte entry. All properties of a layout-mode object are writable,
enumerable, configurable data properties. An accessor, a non-default
attribute, a delete, more than 64 keys, spent layout budgets, publication to
other threads, or any code that reads entries directly moves the object to
dictionary mode (today's entries) for good, so unported paths stay correct.

Layouts replace the dynamic tier's lazily computed hidden classes (ScrJsShape)
and keep their budgets. Inline caches key on the layout pointer and read the
slot directly; JSON.parse builds objects along layouts; stringify, key
enumeration, release, trace and the runtime read cache read slots in place.
The emitted index-signature checks read members through runtime accessors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 09:18:53 -07:00
Chris Tate fade126bb5 Merge ni/i10-js-island-requires and fix live-handle tracing 2026-10-26 05:18:31 -05:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate 98ed422405 Merge ni/i6-subclasses and cascade ES-module routing 2026-10-26 01:00:45 -05:00
Chris Tate a21f4f6823 Localize merged program shards in process on Linux
A dev build splits a large program into LLVM shards, compiles them in
parallel and merges the objects with ld -r. On a native Linux host the
merge then ran objcopy --keep-global-symbols to turn the shards' promoted
symbols back into locals. A sanitized build of three.js's node materials
merges into a 660 MB object with 278k sections, and objcopy spent 425 s
of a 616 s build on it. localizeElfObject, which cross-ELF builds already
use, does the same demotion in about 2 s.

- Program-shard merges for executables, library archives and packed
  libraries use localizeElfObject after ld -r on native Linux. Library
  runtime localization keeps objcopy.
- localizeElfObject reads and writes extended section numbering: with
  SHN_LORESERVE or more sections the counts go in section 0, and symbol
  section indices from SHN_LORESERVE up go in .symtab_shndx. Before, it
  truncated e_shnum and read such indices as reserved values.
- The Linux shard-merge cache identity drops objcopy.
2026-10-25 22:55:22 -05:00
Chris Tate ec6db73f6b Merge ni/i3-exports 2026-10-25 22:15:54 -05:00
Chris Tate 8a71a20e32 Merge ni/i2-handles 2026-10-25 21:39:21 -05:00
Chris Tate 59e78f298a Name the package declaration and its patch when a declared-type check fails (#2268) 2026-10-25 21:20:34 -05:00
Chris Tate 28b76f7b25 Merge ni/i1-stack-bound 2026-10-25 21:08:37 -05:00
Chris Tate 8981b5c0a2 Resolve computed createRequire specifiers at run time (#2079) 2026-10-25 20:29:51 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Malte UblandClaude Opus 5.5 56e8417746 Name the package declaration and its patch when a declared-type check fails
A value that contradicts a type an npm package declares used to throw
"expected number at $, got undefined" and nothing else. Such a check now
names the declaration and the type patch that admits the value:

    profilekit@3.1.0 declares Profile.email as string (index.d.ts:7)
    if that declaration is wrong, patch it in package.json: ...

Origins are found for values a package function returns under a written
return type, for properties of records typed by a package-declared type
(record projections into such a type are keyed by it), and for
declaration-boundary casts, whose origin the projection records from the
declaration-world checker. Generic declarations get none: their types
come from the program's own instantiation.

Checked conversions carry dynCheck.origin, an index into
IrModule.declOrigins; the site's existing cold exception block calls
scr_type_origin_note (new runtime unit behind the typeOrigins feature),
which appends the note to the pending TypeError and derives the selector
suffix and the admitted kind from the failure. Union extractions get the
note per arm at compile time. The hot path is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 15:06:50 -07:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate fce38bcbcf Merge base branch and fix 2026-10-25 07:35:31 -05:00
Chris Tate 7d17f2a758 Merge base branch and fix 2026-10-25 07:13:36 -05:00
Chris Tate b7915b92a7 Merge main 2026-10-25 06:38:44 -05:00
Chris Tate 303bfcd922 Merge main 2026-10-25 05:25:41 -05:00
Chris Tate 0eef6251de Merge main 2026-10-25 02:28:32 -05:00
Chris Tate 0f7fd1bdb6 Merge base and fix 2026-10-25 02:27:42 -05:00
Chris Tate 0b4301bc3e Merge base and fix 2026-10-25 02:21:39 -05:00
Chris Tate f3750ef976 Merge base and fix 2026-10-25 02:16:24 -05:00
Chris Tate 8b0921243f Compile large sanitized dev programs in shards 2026-10-25 01:59:32 -05:00
Chris Tate 5d3e0c7a8b Support X509Certificate fields and methods, TLS certificates and tls.Server contexts (#2052) 2026-10-25 01:00:24 -05:00
Chris Tate e3bda96fd1 Merge main 2026-10-24 20:31:45 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate 97c3b04f73 Merge main and add the Node 24 util batch 2026-10-24 18:55:14 -05:00
Chris Tate 9e55835014 Compile more npm and CommonJS JavaScript dependencies statically (#2122) 2026-10-24 18:30:56 -05:00
Chris Tate 6c18c58708 Merge main 2026-10-24 18:21:25 -05:00
Chris Tate 4f0975cbbe Compile more npm and CommonJS JavaScript dependencies statically 2026-10-24 12:01:26 -05:00
Chris Tate 2967a1b8c0 Propagate exceptions by unwinding from directly called functions on Linux
Emitted functions report an exception by returning with the thread's
pending flag set, and every caller tests the flag after each call that
may throw. On Linux executables (glibc and musl), functions reached only
by direct calls now propagate exceptions by unwinding the native stack
instead:

- unwindAbiFunctions selects the may-throw functions that are not async
  or generators, constructors, virtually called or overridden methods,
  closure targets, the entry, library exports, module initializers or
  callees of coroutine bodies.
- A native unwind costs about 2 µs a raise and 0.1 µs a frame, so functions that
  exceptions are likely to cross keep the flag: those with a `throw` of
  their own, those up to three calls above one, and recursive functions,
  counting calls through methods, closures and callbacks.
- Their propagating exits call scr_unwind_raise (scr_unwind.c), a forced
  unwind through _Unwind_ForcedUnwind with the exception still in the
  active cell.
- Direct calls to them are `invoke`s without a pending check. The
  landing pad runs the frame's usual unwind path: it releases the frame's
  values and then enters the enclosing try handler or resumes the unwind.
- Remaining plain calls to them (wrappers, adapters, runtime-facing entry
  points) become invokes whose landing pad continues on the pending flag,
  or resumes when the wrapper is itself invoked. Every other call is
  marked `nounwind`, so inlining a raising function at an invoke keeps its
  calls plain.
- When emitted text breaks these rules (a raising function's address is
  taken, or an invoked function returns with the flag pending), the
  module is emitted again with those functions on the pending flag.

Emitted code does not declare scr_unwind_raise `noreturn`, so sanitized
builds do not run the sanitizer's no-return hook, which needs stack
bounds that fiber stacks do not report. Instead, scr_unwind_raise clears
the shadow of the frames it unwinds.

Executables link a static unwinder: libgcc_eh with the host toolchain and
the glibc runtime packs, and Zig's libunwind with Zig and the musl packs.
macOS, Windows, Android, WASI, libraries and debug builds keep the
pending flag everywhere and emit the same code as before.

The throw-chains runtime workload measures exceptions caught 2, 5 and 20
recursive calls above the throw, through direct calls and a callback,
and a parser that backtracks by throwing.
2026-10-24 11:03:34 -05:00
Chris Tate 1d5a7ff48a Merge main 2026-10-24 06:47:59 -05:00
Chris Tate dd4ae4e508 Merge main 2026-10-24 01:57:53 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate 1e2f9b2dfc Resolve computed createRequire specifiers at run time
In JavaScript dependencies, a createRequire require call with a computed
specifier, and require.resolve called with a computed request or read as
a value, now compile. The new runtime unit scr_require.c resolves the
specifier with Node 24's CommonJS algorithm (builtins, relative and
absolute paths with Node's extension search, package "main", "exports"
and self-reference, node_modules and NODE_PATH lookup) when the call runs:

- require.resolve returns the id or real path Node returns, and throws
  Node's MODULE_NOT_FOUND error when nothing resolves.
- require throws the same MODULE_NOT_FOUND error when nothing resolves,
  so packages that probe for optional addons, such as pi-tui, take their
  absent branch as in Node. When the specifier resolves to a native addon,
  a builtin or another file, Node would load that module: the program
  prints an SC2020 error naming it and exits with status 1, without
  running the catch clause around the call.

Coverage lists these calls as deferred to runtime. TypeScript sources keep
the compile-time refusal.

Refs #1777.
2026-10-23 23:52:50 -05:00
Chris Tate b3bea829b9 Fix the island CI failures and merge main 2026-10-23 19:19:21 -05:00
Chris Tate a9da37b82e zlib: Brotli compression and decompression over the vendored brotli 1.2.0 2026-10-23 18:11:49 -05:00
Chris Tate 072543f5b9 Support X509Certificate fields and methods, TLS certificates and tls.Server contexts 2026-10-23 18:01:19 -05:00
Malte UblandClaude Opus 5.5 f14997776b Box TextEncoder and TextDecoder records as checked-dynamic handles
A codec stored in untyped storage (hono's StreamingApi keeps
`this.encoder = new TextEncoder()` in a JavaScript class field) had no
dynamic representation and threw SC1101 at run time. The codec records
now cross 'unknown' slots as handles of two new tags that hold the record
itself, so identity and a decoder's streaming state survive the round
trip, and checked casts unbox the same record.

The handles answer encode, encodeInto and decode (with options objects),
the encoding, fatal and ignoreBOM properties, method values with Node's
ERR_INVALID_THIS, instanceof, String() tags and JSON's {}. The unit
scr_text_codec_dyn.c links only into programs that convert a codec to or
from an untyped value (the textCodecDyn runtime feature, found with the
dgram carrier walk, now shared).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 10:33:06 -07:00