Commit Graph
92 Commits
Author SHA1 Message Date
Malte UblandClaude Opus 5.5 02a322e712 Merge ni/13-libnode-packs (with ni/1-libnode-loader)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 05:40:40 -07:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Chris Tate eeef447216 Merge ni/1-libnode-loader 2026-10-26 06:36:38 -05:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate 59e78f298a Name the package declaration and its patch when a declared-type check fails (#2268) 2026-10-25 21:20:34 -05:00
Chris Tate 8981b5c0a2 Resolve computed createRequire specifiers at run time (#2079) 2026-10-25 20:29:51 -05:00
Chris Tate c46faa93b2 Merge base branch 2026-10-25 19:32:26 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Malte UblandClaude Opus 5.5 56e8417746 Name the package declaration and its patch when a declared-type check fails
A value that contradicts a type an npm package declares used to throw
"expected number at $, got undefined" and nothing else. Such a check now
names the declaration and the type patch that admits the value:

    profilekit@3.1.0 declares Profile.email as string (index.d.ts:7)
    if that declaration is wrong, patch it in package.json: ...

Origins are found for values a package function returns under a written
return type, for properties of records typed by a package-declared type
(record projections into such a type are keyed by it), and for
declaration-boundary casts, whose origin the projection records from the
declaration-world checker. Generic declarations get none: their types
come from the program's own instantiation.

Checked conversions carry dynCheck.origin, an index into
IrModule.declOrigins; the site's existing cold exception block calls
scr_type_origin_note (new runtime unit behind the typeOrigins feature),
which appends the note to the pending TypeError and derives the selector
suffix and the admitted kind from the failure. Union extractions get the
note per arm at compile time. The hot path is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 15:06:50 -07:00
Chris Tate 34401a7d7b Merge base branch 2026-10-25 16:16:47 -05:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate b7915b92a7 Merge main 2026-10-25 06:38:44 -05:00
Chris Tate 303bfcd922 Merge main 2026-10-25 05:25:41 -05:00
Chris Tate 0eef6251de Merge main 2026-10-25 02:28:32 -05:00
Chris Tate 5d3e0c7a8b Support X509Certificate fields and methods, TLS certificates and tls.Server contexts (#2052) 2026-10-25 01:00:24 -05:00
Chris Tate e3bda96fd1 Merge main 2026-10-24 20:31:45 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate 97c3b04f73 Merge main and add the Node 24 util batch 2026-10-24 18:55:14 -05:00
Chris Tate d60cdcc843 Merge the base branch 2026-10-24 16:49:25 -05:00
Chris Tate 4f0975cbbe Compile more npm and CommonJS JavaScript dependencies statically 2026-10-24 12:01:26 -05:00
Chris Tate 1d5a7ff48a Merge main 2026-10-24 06:47:59 -05:00
Chris Tate dd4ae4e508 Merge main 2026-10-24 01:57:53 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate 1e2f9b2dfc Resolve computed createRequire specifiers at run time
In JavaScript dependencies, a createRequire require call with a computed
specifier, and require.resolve called with a computed request or read as
a value, now compile. The new runtime unit scr_require.c resolves the
specifier with Node 24's CommonJS algorithm (builtins, relative and
absolute paths with Node's extension search, package "main", "exports"
and self-reference, node_modules and NODE_PATH lookup) when the call runs:

- require.resolve returns the id or real path Node returns, and throws
  Node's MODULE_NOT_FOUND error when nothing resolves.
- require throws the same MODULE_NOT_FOUND error when nothing resolves,
  so packages that probe for optional addons, such as pi-tui, take their
  absent branch as in Node. When the specifier resolves to a native addon,
  a builtin or another file, Node would load that module: the program
  prints an SC2020 error naming it and exits with status 1, without
  running the catch clause around the call.

Coverage lists these calls as deferred to runtime. TypeScript sources keep
the compile-time refusal.

Refs #1777.
2026-10-23 23:52:50 -05:00
Chris Tate b3bea829b9 Fix the island CI failures and merge main 2026-10-23 19:19:21 -05:00
Chris Tate a9da37b82e zlib: Brotli compression and decompression over the vendored brotli 1.2.0 2026-10-23 18:11:49 -05:00
Chris Tate 072543f5b9 Support X509Certificate fields and methods, TLS certificates and tls.Server contexts 2026-10-23 18:01:19 -05:00
Malte UblandClaude Opus 5.5 f14997776b Box TextEncoder and TextDecoder records as checked-dynamic handles
A codec stored in untyped storage (hono's StreamingApi keeps
`this.encoder = new TextEncoder()` in a JavaScript class field) had no
dynamic representation and threw SC1101 at run time. The codec records
now cross 'unknown' slots as handles of two new tags that hold the record
itself, so identity and a decoder's streaming state survive the round
trip, and checked casts unbox the same record.

The handles answer encode, encodeInto and decode (with options objects),
the encoding, fatal and ignoreBOM properties, method values with Node's
ERR_INVALID_THIS, instanceof, String() tags and JSON's {}. The unit
scr_text_codec_dyn.c links only into programs that convert a codec to or
from an untyped value (the textCodecDyn runtime feature, found with the
dgram carrier walk, now shared).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 10:33:06 -07:00
Malte Ubl 72b4020624 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into HEAD 2026-10-23 05:55:50 -07:00
Malte Ubl 2494e7c1dc Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/src/backend/native/executable.ts
#	packages/compiler/src/frontend/lowering/lower-calls.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 05:53:42 -07:00
Malte UblandClaude Opus 5.5 14966de563 Move adoption into its own runtime unit behind the declAdopt feature
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:35:40 -07:00
Malte Ubl 694d34bb44 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/3-hosted-loop 2026-10-22 14:17:32 -07:00
Malte Ubl 13b00dd78b Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/frontend/lowering/lower-modules.ts
#	tests/harness/differential.test.ts
#	tests/harness/linux-differential.test.ts
#	tests/harness/llvm-differential.test.ts
#	tests/harness/windows-differential.test.ts
2026-10-22 14:10:07 -07:00
Chris Tate 75bcf9dfa7 Merge main 2026-10-22 06:55:45 -05:00
Chris Tateandsaboursepideh 1a240fbf10 Support asymmetric keys, signatures and Ed25519 in node:crypto and WebCrypto
Co-authored-by: saboursepideh <25549001+saboursepideh@users.noreply.github.com>
2026-10-22 02:49:33 -05:00
Chris Tate 283a504fe0 Support crypto.createDiffieHellman and getDiffieHellman
crypto.createDiffieHellman, crypto.getDiffieHellman and
crypto.createDiffieHellmanGroup return native DiffieHellman and
DiffieHellmanGroup handles backed by mbedTLS bignums. generateKeys,
computeSecret, getPrime, getGenerator, getPublicKey, getPrivateKey,
setPublicKey, setPrivateKey and verifyError follow Node 24.22:

- the modp1, modp2, modp5 and modp14 through modp18 groups, with the
  private key widths of Node's OpenSSL 3 path (200 to 400 bits for
  modp5 and up, [2, p - 2] for modp1 and modp2);
- explicit primes and generators in every argument form and encoding,
  with keys of bits(p) - 2 bits, or below min(2^N, q) when the
  parameters equal an OpenSSL named group (RFC 3526, RFC 7919,
  RFC 5114);
- generated safe primes with OpenSSL's congruences for the generator;
- verifyError's DH_check bits, computed when it is first read;
- Node's argument, generator, modulus, key range, state and encoding
  errors, including the OpenSSL errors' library and reason properties.

Safe-prime generation sieves candidates with the odd primes below a
bound that grows with the prime's length and tests them with
Miller-Rabin rounds over a variable-time Montgomery exponentiation that
only reads public values; key and secret exponentiations stay on
mbedTLS's constant-time path. Primes longer than 8192 bits, property
reads other than verifyError, property writes, Object.prototype methods,
and calls whose output encoding is typed any or matches no @types/node
overload are refused.

The runtime unit scr_dh.c links, with the mbedTLS archive, only into
programs that use DiffieHellman. wasm32-wasi builds refuse it. The ECDH
unit's argument and encoding helpers move to scr_crypto_args.h, which
both units include.
2026-10-21 21:55:47 -05:00
Malte UblandClaude Opus 5.5 eaefbdf44f Host the runtime's event loop on Node's libuv once Node loads (NI-3)
Scope-B milestone NI-3. When a program loads Node in-process, its pending
ticks, promise jobs, timers and immediates move onto Node's queues in
order (timers keep their deadlines relative to the moment Node began
loading; the work of a phase the switch interrupted finishes first) and
every later one is scheduled there, so the jobs of both sides interleave
as in Node. The runtime's I/O hooks are watched from Node's loop, and its
liveness is re-read before each poll phase, after each check phase and
after every microtask checkpoint. Node's loop runs on Node's stack, and
V8's limit follows whichever stack an entry comes from.

Island signal and exit listeners join the runtime's registries at their
position, so the runtime keeps owning every signal disposition; Node's own
'exit' emission is left to the runtime's exit sequence, and the exit code
island code leaves in process.exitCode becomes the program's. Bridge ABI 3.

Ordering programs (tests/corpus-node-islands-order, harness
node-island-order.test.ts), corpus programs 6804-6811 and the
node-island-process harness pin the behavior against Node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:31:51 -07:00
Chris Tate 6d19198eed Merge main 2026-10-21 12:43:37 -05:00
Malte UblandClaude Opus 5.5 9a0a3f1562 Load Node.js in-process on first use: libnode bridge and runtime loader
Scope-B milestone NI-1, runtime side. Programs whose IR has a
`nodeIsland.*` operation link scr_node_island.c and the SCR_NODE_ISLANDS
build of scr_async.c; every other program links neither, byte for byte.

- scr_node_island.c dlopens the bridge of a libnode pack on the first call
  (RTLD_NOW | RTLD_GLOBAL), checks its bridge ABI and that it links exactly
  the Node version recorded at build time, starts one Node environment and
  runs the embedded bootstrap (scr_node_island_host.cjs). A run that never
  calls into Node never maps libnode. The pack directory is SCRIPTC_LIBNODE
  at run time, else the recorded path; a `$ORIGIN/` path resolves against
  the executable's directory (the beside packaging, and the hook for
  shipping Node with the program).
- scr_node_bridge.h is the C ABI between the runtime and the bridge; the
  runtime includes no Node or V8 header. The bridge
  (packages/libnode-common/bridge/scr_node_bridge.cc) initializes Node once
  with no default signal handling, no debug-signal handler or inspector,
  no stdio reinitialization and one platform worker, creates the
  environment from the built-in snapshot, routes Node's process exit to
  the runtime's exit sequence, promotes libnode to the global scope for
  native addons, and runs calls in a CallbackScope with a TryCatch.
  build-pack.mjs assembles a pack (libnode, the bridge, manifest.json) from
  a Node --shared build and the matching headers.
- The SCR_NODE_ISLANDS build of scr_async.c adds a loop slot through which
  the runtime's loop drives Node's until NI-3 hosts the runtime's queues on
  Node's libuv: it keeps the program alive while Node has ref'd work,
  sleeps on Node's backend fd up to its next timer, and runs one
  non-blocking turn per loop turn.
- The `nodeIsland` runtime feature (rule, link features, unit table, pack
  matrix row and scr_async.c variant), the nodeIsland.call/callSync
  libCalls, the install call in main, and the build-time record of the
  pack (node-islands/libnode.ts: --libnode=<pack>|beside or
  SCRIPTC_LIBNODE, SC8005 when none is usable, Node >=24.13.1).
- The zero-cost gate and the bench nm test require that no good-code
  executable defines a Node island symbol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:54:47 -07:00
Chris Tate f2fd1c85a2 Run Buffer methods on Buffers held as checked-dynamic values 2026-10-21 08:50:54 -05:00
Chris Tate 00840aaca6 Merge main 2026-10-21 00:08:02 -05:00
Chris Tate a1c55207fc Merge main 2026-10-20 23:21:29 -05:00
Chris Tate 012fdb3fb0 Merge main 2026-10-20 21:52:19 -05:00
Chris Tate d59d8ff6f2 Merge main and count declaration writes to parameters 2026-10-20 21:05:19 -05:00
Malte UblandClaude Opus 5.5 da13375a8b Keep properties JavaScript adds to a child process, and await it as a thenable
execa 5 mixes then/catch/finally into a real ChildProcess with
Reflect.defineProperty, replaces kill and adds cancel by assignment, and
awaits the child.

- A new optional runtime unit, scr_dyn_handle_props.c, keeps those
  properties in a side table keyed by the native child. Wrapper ops for
  the child tag look there first for reads, member calls and writes.
  defineProperty and getOwnPropertyDescriptor use it, and await adopts a
  child with a callable then. Class methods read as values call the
  class's own method.
- In JavaScript files, writes onto typed children, reads and calls of
  added names, method values for bind/call/apply, and members no typed
  lowering claims go through the child's checked-dynamic box.
- Only programs whose JavaScript adds properties to children link the
  unit and install the table; everything else compiles as before.
- A setTimeout delay that may be a boolean converts with ToNumber, and
  `if (t.unref)` over a timer handle folds to true (execa's lib/kill.js).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:28:00 -07:00
Chris Tate 2582e46e76 Support crypto.createECDH and ECDH.convertKey
crypto.createECDH returns a native ECDH handle backed by mbedTLS for
prime192v1, secp224r1, prime256v1, secp384r1, secp521r1, secp192k1,
secp224k1, secp256k1, brainpoolP256r1, brainpoolP384r1 and
brainpoolP512r1. generateKeys, computeSecret, getPublicKey,
getPrivateKey, setPrivateKey, setPublicKey and ECDH.convertKey follow
Node 24: compressed, uncompressed and hybrid points, input and output
encodings, the private key range, key pair validation, and Node's error
codes and messages.

Curve names follow OpenSSL's short-name lookup: names of other curves
from crypto.getCurves() are refused at compile time when literal and
throw a coded SC2020 error when computed, other OpenSSL short names
throw ERR_CRYPTO_OPERATION_FAILED, and unknown names throw
ERR_CRYPTO_INVALID_CURVE. Calls whose arguments no @types/node overload
accepts, and computeSecret(view, encoding), are refused at compile time.

The runtime unit scr_ecdh.c links, with the mbedTLS archive, only into
programs that use ECDH. wasm32-wasi builds refuse it.
2026-10-20 19:17:05 -05:00
Chris Tate 40ed0e3aff Merge main 2026-10-20 18:16:00 -05:00
Malte UblandClaude Opus 5.5 2f50141c0c Compile the hot-shots StatsD client over UDP
hot-shots, a StatsD client compiled from its JavaScript, failed when a
program constructed a client. Each gap on its UDP path is closed
generically:

- Sloppy-mode functions with named parameters may read `arguments`
  where the aliasing of their parameters cannot show: only
  `arguments.length`, or element reads without parameter or element
  writes.
- An `any` slot admits any boxable function value, including functions
  with optional parameters (`C.prototype.m = function (...)` through an
  untyped receiver).
- In a JavaScript file TypeScript does not check, a prototype method
  takes its documented parameters as checked-dynamic values, as
  constructors already did.
- JavaScript dgram sends run completion callbacks, buffer lists,
  connected sends and the six-argument offset/length form.
- A dgram socket crosses into the checked-dynamic tree as a handle
  (SCR_DYNH_DGRAM_SOCKET) whose methods are function values, with
  untyped listeners that survive 'close'. The handle lives in its own
  runtime unit (scr_dgram_dyn.c, the dgramDyn feature), linked only
  when a program converts a socket to or from an untyped value.
- `String.prototype.concat` on strings, and `parseInt` over a possibly
  missing or untyped value in JavaScript.

Corpus programs 6590-6596 and the hot-shots-udp npm
case (hot-shots 8.5.0 vendored) match Node in both lanes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:56:59 -07:00
Malte UblandClaude Opus 5.5 e060eeae25 Iterate untyped values with for await, and delegate with yield* in async generators
- A JavaScript generator whose inferred return is `any` (it delegates to
  itself) is a generator with checked-dynamic channels.
- for await over a checked-dynamic value: a compiled async generator, the
  value's Symbol.asyncIterator result, or its synchronous iterator with
  awaited values; break and throws close it through an awaited return().
  A value that is not iterable throws V8's TypeError.
- yield* in async generators delegates through the same protocol and
  completes with the delegate's return value.

The runtime part is a new unit, scr_dyn_for_await.c, linked only by
programs that use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:17:52 -07:00