Commit Graph
127 Commits
Author SHA1 Message Date
Malte Ubl 3a5aab5b2f Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD
# Conflicts:
#	packages/compiler/src/cli/command.ts
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/frontend/lowering/builtins/module-resolution.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/compiler/src/index.ts
#	packages/runtime/src/scr_json.c
#	tests/harness/npm.test.ts
2026-10-26 11:22:14 -07:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate a21f4f6823 Localize merged program shards in process on Linux
A dev build splits a large program into LLVM shards, compiles them in
parallel and merges the objects with ld -r. On a native Linux host the
merge then ran objcopy --keep-global-symbols to turn the shards' promoted
symbols back into locals. A sanitized build of three.js's node materials
merges into a 660 MB object with 278k sections, and objcopy spent 425 s
of a 616 s build on it. localizeElfObject, which cross-ELF builds already
use, does the same demotion in about 2 s.

- Program-shard merges for executables, library archives and packed
  libraries use localizeElfObject after ld -r on native Linux. Library
  runtime localization keeps objcopy.
- localizeElfObject reads and writes extended section numbering: with
  SHN_LORESERVE or more sections the counts go in section 0, and symbol
  section indices from SHN_LORESERVE up go in .symtab_shndx. Before, it
  truncated e_shnum and read such indices as reserved values.
- The Linux shard-merge cache identity drops objcopy.
2026-10-25 22:55:22 -05:00
Chris Tate 59e78f298a Name the package declaration and its patch when a declared-type check fails (#2268) 2026-10-25 21:20:34 -05:00
Chris Tate 28b76f7b25 Merge ni/i1-stack-bound 2026-10-25 21:08:37 -05:00
Chris Tate 8981b5c0a2 Resolve computed createRequire specifiers at run time (#2079) 2026-10-25 20:29:51 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Malte UblandClaude Opus 5.5 56e8417746 Name the package declaration and its patch when a declared-type check fails
A value that contradicts a type an npm package declares used to throw
"expected number at $, got undefined" and nothing else. Such a check now
names the declaration and the type patch that admits the value:

    profilekit@3.1.0 declares Profile.email as string (index.d.ts:7)
    if that declaration is wrong, patch it in package.json: ...

Origins are found for values a package function returns under a written
return type, for properties of records typed by a package-declared type
(record projections into such a type are keyed by it), and for
declaration-boundary casts, whose origin the projection records from the
declaration-world checker. Generic declarations get none: their types
come from the program's own instantiation.

Checked conversions carry dynCheck.origin, an index into
IrModule.declOrigins; the site's existing cold exception block calls
scr_type_origin_note (new runtime unit behind the typeOrigins feature),
which appends the note to the pending TypeError and derives the selector
suffix and the admitted kind from the failure. Union extractions get the
note per arm at compile time. The hot path is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 15:06:50 -07:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate b7915b92a7 Merge main 2026-10-25 06:38:44 -05:00
Chris Tate 303bfcd922 Merge main 2026-10-25 05:25:41 -05:00
Chris Tate 0eef6251de Merge main 2026-10-25 02:28:32 -05:00
Chris Tate f3750ef976 Merge base and fix 2026-10-25 02:16:24 -05:00
Chris Tate 8b0921243f Compile large sanitized dev programs in shards 2026-10-25 01:59:32 -05:00
Chris Tate 5d3e0c7a8b Support X509Certificate fields and methods, TLS certificates and tls.Server contexts (#2052) 2026-10-25 01:00:24 -05:00
Chris Tate e3bda96fd1 Merge main 2026-10-24 20:31:45 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate 97c3b04f73 Merge main and add the Node 24 util batch 2026-10-24 18:55:14 -05:00
Chris Tate 9e55835014 Compile more npm and CommonJS JavaScript dependencies statically (#2122) 2026-10-24 18:30:56 -05:00
Chris Tate 6c18c58708 Merge main 2026-10-24 18:21:25 -05:00
Chris Tate 4f0975cbbe Compile more npm and CommonJS JavaScript dependencies statically 2026-10-24 12:01:26 -05:00
Chris Tate 2967a1b8c0 Propagate exceptions by unwinding from directly called functions on Linux
Emitted functions report an exception by returning with the thread's
pending flag set, and every caller tests the flag after each call that
may throw. On Linux executables (glibc and musl), functions reached only
by direct calls now propagate exceptions by unwinding the native stack
instead:

- unwindAbiFunctions selects the may-throw functions that are not async
  or generators, constructors, virtually called or overridden methods,
  closure targets, the entry, library exports, module initializers or
  callees of coroutine bodies.
- A native unwind costs about 2 µs a raise and 0.1 µs a frame, so functions that
  exceptions are likely to cross keep the flag: those with a `throw` of
  their own, those up to three calls above one, and recursive functions,
  counting calls through methods, closures and callbacks.
- Their propagating exits call scr_unwind_raise (scr_unwind.c), a forced
  unwind through _Unwind_ForcedUnwind with the exception still in the
  active cell.
- Direct calls to them are `invoke`s without a pending check. The
  landing pad runs the frame's usual unwind path: it releases the frame's
  values and then enters the enclosing try handler or resumes the unwind.
- Remaining plain calls to them (wrappers, adapters, runtime-facing entry
  points) become invokes whose landing pad continues on the pending flag,
  or resumes when the wrapper is itself invoked. Every other call is
  marked `nounwind`, so inlining a raising function at an invoke keeps its
  calls plain.
- When emitted text breaks these rules (a raising function's address is
  taken, or an invoked function returns with the flag pending), the
  module is emitted again with those functions on the pending flag.

Emitted code does not declare scr_unwind_raise `noreturn`, so sanitized
builds do not run the sanitizer's no-return hook, which needs stack
bounds that fiber stacks do not report. Instead, scr_unwind_raise clears
the shadow of the frames it unwinds.

Executables link a static unwinder: libgcc_eh with the host toolchain and
the glibc runtime packs, and Zig's libunwind with Zig and the musl packs.
macOS, Windows, Android, WASI, libraries and debug builds keep the
pending flag everywhere and emit the same code as before.

The throw-chains runtime workload measures exceptions caught 2, 5 and 20
recursive calls above the throw, through direct calls and a callback,
and a parser that backtracks by throwing.
2026-10-24 11:03:34 -05:00
Chris Tate 1d5a7ff48a Merge main 2026-10-24 06:47:59 -05:00
Chris Tate dd4ae4e508 Merge main 2026-10-24 01:57:53 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate 1e2f9b2dfc Resolve computed createRequire specifiers at run time
In JavaScript dependencies, a createRequire require call with a computed
specifier, and require.resolve called with a computed request or read as
a value, now compile. The new runtime unit scr_require.c resolves the
specifier with Node 24's CommonJS algorithm (builtins, relative and
absolute paths with Node's extension search, package "main", "exports"
and self-reference, node_modules and NODE_PATH lookup) when the call runs:

- require.resolve returns the id or real path Node returns, and throws
  Node's MODULE_NOT_FOUND error when nothing resolves.
- require throws the same MODULE_NOT_FOUND error when nothing resolves,
  so packages that probe for optional addons, such as pi-tui, take their
  absent branch as in Node. When the specifier resolves to a native addon,
  a builtin or another file, Node would load that module: the program
  prints an SC2020 error naming it and exits with status 1, without
  running the catch clause around the call.

Coverage lists these calls as deferred to runtime. TypeScript sources keep
the compile-time refusal.

Refs #1777.
2026-10-23 23:52:50 -05:00
Chris Tate b3bea829b9 Fix the island CI failures and merge main 2026-10-23 19:19:21 -05:00
Chris Tate a9da37b82e zlib: Brotli compression and decompression over the vendored brotli 1.2.0 2026-10-23 18:11:49 -05:00
Chris Tate 072543f5b9 Support X509Certificate fields and methods, TLS certificates and tls.Server contexts 2026-10-23 18:01:19 -05:00
Malte UblandClaude Opus 5.5 f14997776b Box TextEncoder and TextDecoder records as checked-dynamic handles
A codec stored in untyped storage (hono's StreamingApi keeps
`this.encoder = new TextEncoder()` in a JavaScript class field) had no
dynamic representation and threw SC1101 at run time. The codec records
now cross 'unknown' slots as handles of two new tags that hold the record
itself, so identity and a decoder's streaming state survive the round
trip, and checked casts unbox the same record.

The handles answer encode, encodeInto and decode (with options objects),
the encoding, fatal and ignoreBOM properties, method values with Node's
ERR_INVALID_THIS, instanceof, String() tags and JSON's {}. The unit
scr_text_codec_dyn.c links only into programs that convert a codec to or
from an untyped value (the textCodecDyn runtime feature, found with the
dgram carrier walk, now shared).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 10:33:06 -07:00
Malte UblandClaude Opus 5.5 b653f4fd08 Merge origin/ni/i1-stack-bound into ni/i2-handles
Main's FinalizationRegistry took handle tag 72, so the Node island
handle moves to 73. Island handles keep their instanceof path in the
new scr_bytes_instanceof_at.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 06:31:43 -07:00
Malte Ubl 2494e7c1dc Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/src/backend/native/executable.ts
#	packages/compiler/src/frontend/lowering/lower-calls.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 05:53:42 -07:00
Malte UblandClaude Opus 5.5 14966de563 Move adoption into its own runtime unit behind the declAdopt feature
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:35:40 -07:00
Malte UblandClaude Opus 5.5 a1b6fda11c Run npm packages in Node islands: object handles and compiler routing (NI-4)
Node islands stack, part 2 of 11 (scriptc-perf/results/islands-prs/PLAN.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:49:52 -07:00
Malte Ubl 13b00dd78b Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/frontend/lowering/lower-modules.ts
#	tests/harness/differential.test.ts
#	tests/harness/linux-differential.test.ts
#	tests/harness/llvm-differential.test.ts
#	tests/harness/windows-differential.test.ts
2026-10-22 14:10:07 -07:00
Chris Tate 5c29845909 Support Readable.fromWeb over web ReadableStreams 2026-10-22 15:23:28 -05:00
Chris Tate 317ec98fa5 Close async generators on early for await exits and iterate untyped native streams 2026-10-22 12:31:21 -05:00
Chris Tate 75bcf9dfa7 Merge main 2026-10-22 06:55:45 -05:00
Chris Tate 2e29fd6a8c Support asymmetric keys, signatures and Ed25519 in node:crypto and WebCrypto (#1771) 2026-10-22 05:54:11 -05:00
Chris Tate c086610b94 Keep each build's translation unit private until the build ends (#1766) 2026-10-22 05:50:07 -05:00
Chris Tate 6e753cd5c8 Merge main 2026-10-22 04:08:52 -05:00
Chris Tate b5fb4115e1 Merge main 2026-10-22 03:33:29 -05:00
Chris Tate 1773ffbef3 WebCrypto: the crypto global, node:crypto's webcrypto, and secret-key SubtleCrypto 2026-10-22 02:52:01 -05:00
Chris Tateandsaboursepideh 1a240fbf10 Support asymmetric keys, signatures and Ed25519 in node:crypto and WebCrypto
Co-authored-by: saboursepideh <25549001+saboursepideh@users.noreply.github.com>
2026-10-22 02:49:33 -05:00
Chris Tate 2fbfc1b270 Keep each build's translation unit private until the build ends
The kept LLVM text and IR are named after the entry (`<stem>.ll`,
`<stem>.ir.json`) in the output directory, so two programs whose entries
share a name and are built into one directory share those files. The
executable cache published the translation unit by copying the shared
`.ll` after the native build, so when the other program wrote it in
between, the stamp recorded the other program's text, and a later cache
hit relinked the wrong program without any error.

A build now writes, compiles and caches private copies beside the shared
names and renames them into place when it ends. A cache hit that relinks
compiles a private copy of the cached translation unit, and leaves a
shared file that already holds those bytes as it is. The native build
reads the private copy and keeps the shared name as the spelling clang
sees, so native cache keys and debug names do not change.
2026-10-22 02:00:30 -05:00
Chris Tate 283a504fe0 Support crypto.createDiffieHellman and getDiffieHellman
crypto.createDiffieHellman, crypto.getDiffieHellman and
crypto.createDiffieHellmanGroup return native DiffieHellman and
DiffieHellmanGroup handles backed by mbedTLS bignums. generateKeys,
computeSecret, getPrime, getGenerator, getPublicKey, getPrivateKey,
setPublicKey, setPrivateKey and verifyError follow Node 24.22:

- the modp1, modp2, modp5 and modp14 through modp18 groups, with the
  private key widths of Node's OpenSSL 3 path (200 to 400 bits for
  modp5 and up, [2, p - 2] for modp1 and modp2);
- explicit primes and generators in every argument form and encoding,
  with keys of bits(p) - 2 bits, or below min(2^N, q) when the
  parameters equal an OpenSSL named group (RFC 3526, RFC 7919,
  RFC 5114);
- generated safe primes with OpenSSL's congruences for the generator;
- verifyError's DH_check bits, computed when it is first read;
- Node's argument, generator, modulus, key range, state and encoding
  errors, including the OpenSSL errors' library and reason properties.

Safe-prime generation sieves candidates with the odd primes below a
bound that grows with the prime's length and tests them with
Miller-Rabin rounds over a variable-time Montgomery exponentiation that
only reads public values; key and secret exponentiations stay on
mbedTLS's constant-time path. Primes longer than 8192 bits, property
reads other than verifyError, property writes, Object.prototype methods,
and calls whose output encoding is typed any or matches no @types/node
overload are refused.

The runtime unit scr_dh.c links, with the mbedTLS archive, only into
programs that use DiffieHellman. wasm32-wasi builds refuse it. The ECDH
unit's argument and encoding helpers move to scr_crypto_args.h, which
both units include.
2026-10-21 21:55:47 -05:00
Chris Tate 6d19198eed Merge main 2026-10-21 12:43:37 -05:00
Malte UblandClaude Opus 5.5 9a0a3f1562 Load Node.js in-process on first use: libnode bridge and runtime loader
Scope-B milestone NI-1, runtime side. Programs whose IR has a
`nodeIsland.*` operation link scr_node_island.c and the SCR_NODE_ISLANDS
build of scr_async.c; every other program links neither, byte for byte.

- scr_node_island.c dlopens the bridge of a libnode pack on the first call
  (RTLD_NOW | RTLD_GLOBAL), checks its bridge ABI and that it links exactly
  the Node version recorded at build time, starts one Node environment and
  runs the embedded bootstrap (scr_node_island_host.cjs). A run that never
  calls into Node never maps libnode. The pack directory is SCRIPTC_LIBNODE
  at run time, else the recorded path; a `$ORIGIN/` path resolves against
  the executable's directory (the beside packaging, and the hook for
  shipping Node with the program).
- scr_node_bridge.h is the C ABI between the runtime and the bridge; the
  runtime includes no Node or V8 header. The bridge
  (packages/libnode-common/bridge/scr_node_bridge.cc) initializes Node once
  with no default signal handling, no debug-signal handler or inspector,
  no stdio reinitialization and one platform worker, creates the
  environment from the built-in snapshot, routes Node's process exit to
  the runtime's exit sequence, promotes libnode to the global scope for
  native addons, and runs calls in a CallbackScope with a TryCatch.
  build-pack.mjs assembles a pack (libnode, the bridge, manifest.json) from
  a Node --shared build and the matching headers.
- The SCR_NODE_ISLANDS build of scr_async.c adds a loop slot through which
  the runtime's loop drives Node's until NI-3 hosts the runtime's queues on
  Node's libuv: it keeps the program alive while Node has ref'd work,
  sleeps on Node's backend fd up to its next timer, and runs one
  non-blocking turn per loop turn.
- The `nodeIsland` runtime feature (rule, link features, unit table, pack
  matrix row and scr_async.c variant), the nodeIsland.call/callSync
  libCalls, the install call in main, and the build-time record of the
  pack (node-islands/libnode.ts: --libnode=<pack>|beside or
  SCRIPTC_LIBNODE, SC8005 when none is usable, Node >=24.13.1).
- The zero-cost gate and the bench nm test require that no good-code
  executable defines a Node island symbol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:54:47 -07:00
Chris Tate f2fd1c85a2 Run Buffer methods on Buffers held as checked-dynamic values 2026-10-21 08:50:54 -05:00
Chris Tate 00840aaca6 Merge main 2026-10-21 00:08:02 -05:00