Commit Graph
103 Commits
Author SHA1 Message Date
Chris Tate 39216602ab Merge main 2026-10-24 02:39:41 -05:00
Chris Tate 7bf50dd9cc zlib: Brotli compression and decompression over the vendored brotli 1.2.0 (#2053) 2026-10-24 00:30:21 -05:00
Chris Tate e9833e3260 Add --target-cpu for x86-64 microarchitecture levels (#2025) 2026-10-23 20:00:33 -05:00
Chris Tate a9da37b82e zlib: Brotli compression and decompression over the vendored brotli 1.2.0 2026-10-23 18:11:49 -05:00
Chris Tate e3b42a311d Add --target-cpu to compile x86-64 programs for a microarchitecture level
Release builds compile for the x86-64 baseline, so LLVM vectorizes with
SSE2 only. `--target-cpu=<baseline|x86-64-v2|x86-64-v3|x86-64-v4|native>`
compiles the program's code for a higher level: every emitted function
carries the level as its `target-cpu`, so LLVM can use AVX2, BMI2, FMA or
AVX-512. The executable then runs only on CPUs of that level.

- `native` resolves to the highest level the build machine supports, read
  from /proc/cpuinfo on Linux and `sysctl machdep.cpu` on macOS.
- The level is part of the executable cache key; `baseline` and builds
  without the option produce the same output and keys as before.
- The runtime pack keeps its baseline code.
- Targets other than x86-64 refuse a level (SC3002). The CLI's cached
  bootstrap route leaves builds with the option to the full compiler.
- `bench:runtime` takes `--baseline-target-cpu` and `--candidate-target-cpu`.
2026-10-23 13:10:16 -05:00
Malte UblandClaude Opus 5.5 f14997776b Box TextEncoder and TextDecoder records as checked-dynamic handles
A codec stored in untyped storage (hono's StreamingApi keeps
`this.encoder = new TextEncoder()` in a JavaScript class field) had no
dynamic representation and threw SC1101 at run time. The codec records
now cross 'unknown' slots as handles of two new tags that hold the record
itself, so identity and a decoder's streaming state survive the round
trip, and checked casts unbox the same record.

The handles answer encode, encodeInto and decode (with options objects),
the encoding, fatal and ignoreBOM properties, method values with Node's
ERR_INVALID_THIS, instanceof, String() tags and JSON's {}. The unit
scr_text_codec_dyn.c links only into programs that convert a codec to or
from an untyped value (the textCodecDyn runtime feature, found with the
dgram carrier walk, now shared).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 10:33:06 -07:00
Chris Tate ccd45c748f Address the Node island report review findings 2026-10-23 12:21:50 -05:00
Malte UblandClaude Opus 5.5 14966de563 Move adoption into its own runtime unit behind the declAdopt feature
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:35:40 -07:00
Chris Tate 75bcf9dfa7 Merge main 2026-10-22 06:55:45 -05:00
Chris Tate 2e29fd6a8c Support asymmetric keys, signatures and Ed25519 in node:crypto and WebCrypto (#1771) 2026-10-22 05:54:11 -05:00
Chris Tate 6bf3b7d9df Install driver-built translation units before the native build
The private copy of the LLVM text had no extension. Builds that compile it
with the C driver (sanitize builds, targets without the LLVM helper) passed
it to clang directly when the native cache was off, and clang read it as a
linker input, so the build failed. The copy is now named with `.ll`.

Those builds also record the output directory in their native cache proof.
Renaming the private copies to the shared names after the native build
changed that directory, so every unchanged repeat missed the proof and
relinked the executable. The shared names are now installed before the
native build, which compiles and caches copies in a private directory
outside the output directory.
2026-10-22 05:44:28 -05:00
Chris Tateandsaboursepideh 1a240fbf10 Support asymmetric keys, signatures and Ed25519 in node:crypto and WebCrypto
Co-authored-by: saboursepideh <25549001+saboursepideh@users.noreply.github.com>
2026-10-22 02:49:33 -05:00
Chris Tate 2fbfc1b270 Keep each build's translation unit private until the build ends
The kept LLVM text and IR are named after the entry (`<stem>.ll`,
`<stem>.ir.json`) in the output directory, so two programs whose entries
share a name and are built into one directory share those files. The
executable cache published the translation unit by copying the shared
`.ll` after the native build, so when the other program wrote it in
between, the stamp recorded the other program's text, and a later cache
hit relinked the wrong program without any error.

A build now writes, compiles and caches private copies beside the shared
names and renames them into place when it ends. A cache hit that relinks
compiles a private copy of the cached translation unit, and leaves a
shared file that already holds those bytes as it is. The native build
reads the private copy and keeps the shared name as the spelling clang
sees, so native cache keys and debug names do not change.
2026-10-22 02:00:30 -05:00
Chris Tate 283a504fe0 Support crypto.createDiffieHellman and getDiffieHellman
crypto.createDiffieHellman, crypto.getDiffieHellman and
crypto.createDiffieHellmanGroup return native DiffieHellman and
DiffieHellmanGroup handles backed by mbedTLS bignums. generateKeys,
computeSecret, getPrime, getGenerator, getPublicKey, getPrivateKey,
setPublicKey, setPrivateKey and verifyError follow Node 24.22:

- the modp1, modp2, modp5 and modp14 through modp18 groups, with the
  private key widths of Node's OpenSSL 3 path (200 to 400 bits for
  modp5 and up, [2, p - 2] for modp1 and modp2);
- explicit primes and generators in every argument form and encoding,
  with keys of bits(p) - 2 bits, or below min(2^N, q) when the
  parameters equal an OpenSSL named group (RFC 3526, RFC 7919,
  RFC 5114);
- generated safe primes with OpenSSL's congruences for the generator;
- verifyError's DH_check bits, computed when it is first read;
- Node's argument, generator, modulus, key range, state and encoding
  errors, including the OpenSSL errors' library and reason properties.

Safe-prime generation sieves candidates with the odd primes below a
bound that grows with the prime's length and tests them with
Miller-Rabin rounds over a variable-time Montgomery exponentiation that
only reads public values; key and secret exponentiations stay on
mbedTLS's constant-time path. Primes longer than 8192 bits, property
reads other than verifyError, property writes, Object.prototype methods,
and calls whose output encoding is typed any or matches no @types/node
overload are refused.

The runtime unit scr_dh.c links, with the mbedTLS archive, only into
programs that use DiffieHellman. wasm32-wasi builds refuse it. The ECDH
unit's argument and encoding helpers move to scr_crypto_args.h, which
both units include.
2026-10-21 21:55:47 -05:00
Chris Tate fa7bc7c7a0 Warn about likely-hot functions that use untyped values
A build now names the functions of the program's own files that compile
to the dynamic tier and likely run often, so a slow program with a little
untyped code says where the time goes:

  warning SC6101: 3 functions that likely run often use untyped values, which runs slower than typed code:
    advance (nbody.js:22): untyped JavaScript file, in a loop
    ...

- The functions come from the tier assignment lowering already makes
  (Lowerer.dynamicFunctions). Without a profile, a function counts as
  likely hot when a dynamic operation runs inside one of its loops, when
  a loop calls it, or when it calls itself (lowering/dynamic-tier.ts).
- A function is named only when its source says why its values are
  untyped: an `any` (for a for...of variable, the value it iterates), an
  untyped JSON.parse result, or an untyped JavaScript file. Generated
  %dyn.* helpers, module top levels and functions that are dynamic for an
  operation without a static lowering are left out.
- At most five functions are named, most likely hot first, with the
  total; dynamic-tier code in npm packages is counted per package on one
  line. All of the warning's text is in diagnostics/dynamic-tier.ts.
- SC6101 opens the SC61xx band, performance notes, with a new
  "performance" category in the diagnostics envelope; SC60xx stay the
  divergences.
- scriptc build prints it after the divergence warnings, and the
  executable cache stores it with the entry, so a cached build (including
  the CLI's routed fast path) repeats it. scriptc coverage reports it, and
  a comment-only edit no longer replays a verdict that carries it, since
  its text holds line numbers.
- --no-dynamic-tier-warning leaves it out of build and coverage;
  coverage --fail-on=dynamic fails on it.
- Docs: coverage.mdx (Dynamic tier, Gating builds and CI, the envelope
  categories) and cli.mdx.
2026-10-21 18:53:43 -05:00
Chris Tate 23febaddf8 Collect the frontend's cycles when the native compiler disposes it (#1671) 2026-10-21 11:30:00 -05:00
Chris Tate e34e5edfd5 Run Buffer methods on Buffers held as checked-dynamic values (#1659) 2026-10-21 11:15:06 -05:00
Chris Tate 8fc36e61f2 Merge main 2026-10-21 09:17:28 -05:00
Chris Tate 050bbd852c Collect the frontend's cycles when the native compiler disposes it
The frontend's syntax trees and program state refer to each other, so
reference counting frees almost none of them when a build disposes the
frontend. Whether a cycle pass reclaimed them before the build ended
depended on where the collector's schedule happened to be. Passes during
the frontend found its data live and widened the growth fractions, so the
next pass waited until the heap had grown well past its size at disposal,
and the dead frontend and the emitter's garbage stayed until the build
ended.

The native compiler now runs scr_collect_cycles_phase right after it
disposes the frontend, through a host function. The runtime entry point
sweeps, then restarts the growth fractions and the death rule's backoff
from the heap the sweep leaves, as after a productive pass. Under Node
the hook is unset.
2026-10-21 09:09:19 -05:00
Chris Tate f2fd1c85a2 Run Buffer methods on Buffers held as checked-dynamic values 2026-10-21 08:50:54 -05:00
Chris Tate 0cc0674308 Prepare the first build of an executable in a frontend process
The frontend, lowering and LLVM emission of an optimized executable build run in a child
process when no earlier build of the same invocation emitted its native program object.
The parent passes the LLVM text to the native helper as the file the child wrote, so the
helper runs after the pages the frontend used are released. Later builds of the
invocation find most native partitions cached and peak in the frontend, so they, along
with development builds and builds with SCRIPTC_FRONTEND_PROCESS=0, prepare their input
in-process. The child runs with the parent's Node options without --eval, --print and
their code, or inspector, watch and test runner options. With SCRIPTC_TIMING=1, it
reports its phases on the parent's clock.
2026-10-21 08:23:18 -05:00
Chris Tate 8065be3ccd Add profile-guided optimization 2026-10-21 03:13:45 -05:00
Chris Tate 00840aaca6 Merge main 2026-10-21 00:08:02 -05:00
Chris Tate a1c55207fc Merge main 2026-10-20 23:21:29 -05:00
Chris Tate 012fdb3fb0 Merge main 2026-10-20 21:52:19 -05:00
Chris Tate d59d8ff6f2 Merge main and count declaration writes to parameters 2026-10-20 21:05:19 -05:00
Malte UblandClaude Opus 5.5 da13375a8b Keep properties JavaScript adds to a child process, and await it as a thenable
execa 5 mixes then/catch/finally into a real ChildProcess with
Reflect.defineProperty, replaces kill and adds cancel by assignment, and
awaits the child.

- A new optional runtime unit, scr_dyn_handle_props.c, keeps those
  properties in a side table keyed by the native child. Wrapper ops for
  the child tag look there first for reads, member calls and writes.
  defineProperty and getOwnPropertyDescriptor use it, and await adopts a
  child with a callable then. Class methods read as values call the
  class's own method.
- In JavaScript files, writes onto typed children, reads and calls of
  added names, method values for bind/call/apply, and members no typed
  lowering claims go through the child's checked-dynamic box.
- Only programs whose JavaScript adds properties to children link the
  unit and install the table; everything else compiles as before.
- A setTimeout delay that may be a boolean converts with ToNumber, and
  `if (t.unref)` over a timer handle folds to true (execa's lib/kill.js).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:28:00 -07:00
Chris Tate 2582e46e76 Support crypto.createECDH and ECDH.convertKey
crypto.createECDH returns a native ECDH handle backed by mbedTLS for
prime192v1, secp224r1, prime256v1, secp384r1, secp521r1, secp192k1,
secp224k1, secp256k1, brainpoolP256r1, brainpoolP384r1 and
brainpoolP512r1. generateKeys, computeSecret, getPublicKey,
getPrivateKey, setPrivateKey, setPublicKey and ECDH.convertKey follow
Node 24: compressed, uncompressed and hybrid points, input and output
encodings, the private key range, key pair validation, and Node's error
codes and messages.

Curve names follow OpenSSL's short-name lookup: names of other curves
from crypto.getCurves() are refused at compile time when literal and
throw a coded SC2020 error when computed, other OpenSSL short names
throw ERR_CRYPTO_OPERATION_FAILED, and unknown names throw
ERR_CRYPTO_INVALID_CURVE. Calls whose arguments no @types/node overload
accepts, and computeSecret(view, encoding), are refused at compile time.

The runtime unit scr_ecdh.c links, with the mbedTLS archive, only into
programs that use ECDH. wasm32-wasi builds refuse it.
2026-10-20 19:17:05 -05:00
Chris Tate 40ed0e3aff Merge main 2026-10-20 18:16:00 -05:00
Malte UblandClaude Opus 5.5 2f50141c0c Compile the hot-shots StatsD client over UDP
hot-shots, a StatsD client compiled from its JavaScript, failed when a
program constructed a client. Each gap on its UDP path is closed
generically:

- Sloppy-mode functions with named parameters may read `arguments`
  where the aliasing of their parameters cannot show: only
  `arguments.length`, or element reads without parameter or element
  writes.
- An `any` slot admits any boxable function value, including functions
  with optional parameters (`C.prototype.m = function (...)` through an
  untyped receiver).
- In a JavaScript file TypeScript does not check, a prototype method
  takes its documented parameters as checked-dynamic values, as
  constructors already did.
- JavaScript dgram sends run completion callbacks, buffer lists,
  connected sends and the six-argument offset/length form.
- A dgram socket crosses into the checked-dynamic tree as a handle
  (SCR_DYNH_DGRAM_SOCKET) whose methods are function values, with
  untyped listeners that survive 'close'. The handle lives in its own
  runtime unit (scr_dgram_dyn.c, the dgramDyn feature), linked only
  when a program converts a socket to or from an untyped value.
- `String.prototype.concat` on strings, and `parseInt` over a possibly
  missing or untyped value in JavaScript.

Corpus programs 6590-6596 and the hot-shots-udp npm
case (hot-shots 8.5.0 vendored) match Node in both lanes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:56:59 -07:00
Malte UblandClaude Opus 5.5 e060eeae25 Iterate untyped values with for await, and delegate with yield* in async generators
- A JavaScript generator whose inferred return is `any` (it delegates to
  itself) is a generator with checked-dynamic channels.
- for await over a checked-dynamic value: a compiled async generator, the
  value's Symbol.asyncIterator result, or its synchronous iterator with
  awaited values; break and throws close it through an awaited return().
  A value that is not iterable throws V8's TypeError.
- yield* in async generators delegates through the same protocol and
  completes with the delegate's return value.

The runtime part is a new unit, scr_dyn_for_await.c, linked only by
programs that use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:17:52 -07:00
Malte UblandClaude Opus 5.5 be9c18f0b3 Bind builtin static functions and read Buffer.alloc, fs.realpathSync and String methods as values
- bind/call/apply on builtin function values held as checked-dynamic
  functions (String.fromCharCode, JSON.stringify and the forms below).
- Function values for Buffer.byteLength (Node's runtime encoding ladder),
  Buffer.compare, Buffer.alloc (size check, then fill), JSON.parse,
  fs.realpathSync and fs.realpathSync.native (Node's options handling).
- A JavaScript string method read as a value answers String.prototype's
  function (smol-toml's `!!(''.isWellFormed)` probe).

The runtime entries live in a new unit, scr_builtin_values.c, linked only
by programs that read these values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 13:50:09 -07:00
Malte UblandClaude Opus 5.5 b723155820 Run buffer-crc32's and yazl's capability probes as Node does
- A JavaScript binding written with a value whose representation differs
  from its initializer's (buffer-crc32's Int32Array table) stores a
  checked-dynamic value.
- A top-level JavaScript function declaration its module reassigns
  (yazl's self-redefining bufferFrom) lives in a mutable module global,
  assigned when the module starts; reads, calls and writes use it.
- Buffer.from with a non-literal encoding follows Node's fromString rules
  at run time, and copies arrays crossing by reference; array and byte
  sources ignore the extra argument.
- `new Buffer(...)` in dependency sources, where Node never warns, is
  Node's construction.
- Fixed-width read/write methods on Buffers held as checked-dynamic
  values dispatch through a separate runtime unit (scr_bytes_dyn.c),
  linked only by programs that call them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 13:26:02 -07:00
Chris Tate a0894cc488 Give constructor functions native stream state over the stream classes (#1533) 2026-10-20 13:21:25 -05:00
Chris Tate 3751afa51f Merge main 2026-10-20 13:02:01 -05:00
Chris Tate 76227c86c8 Merge main 2026-10-20 12:07:23 -05:00
Chris Tate eea3bc5447 Specialize variadic generic wrappers to the wrapped function's signature (#1413) 2026-10-20 11:52:34 -05:00
Malte UblandClaude Opus 5.5 73e40f4196 Merge main
pipe() into an untyped destination now comes from main's pipeIntoDyn
(stream.unboxDyn); programs with constructor-function streams unbox
through scr_dyn_builtin_stream.c, which also takes those streams, so the
separate dyn.builtinStreamPipe entry point is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 09:49:43 -07:00
Malte UblandClaude Opus 5.5 507610b90d Give constructor functions native stream state over the stream classes
Bundled npm code (from2, bl, tar-stream, yazl, multistream) builds its
streams as ES5 constructor functions: util.inherits(Ctor, Readable) and
Readable.call(this, opts), with _read/_write/_transform on the
prototype. Readable.call(this) threw SC2020.

A new runtime unit, scr_dyn_builtin_stream.c (feature
dynBuiltinStream), gives the untyped stream constructors their stream
half. Base.call(this, opts) creates a runtime stream over the receiver
and keeps it in a hidden slot; the runtime calls the receiver's
underscore methods through its property lookup; the stream prototypes
carry Node's methods and accessors over the hidden stream; and
EventEmitter's methods register listeners with the runtime stream's
emitter, so runtime events and pipes reach JavaScript listeners with the
receiver as this. _readableState/_writableState are live views.

Stream class values now come from dyn.builtinStreamClass, so only
programs that use them link the stream half. Typed code sees these
objects as their runtime streams: checked casts to the stream classes
take them, readable.pipe() into an untyped destination pipes into them,
and runtime streams crossing into JavaScript answer the constructor
object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 09:46:05 -07:00
Chris Tate 8823556a84 Let constructor functions inherit from EventEmitter and the stream classes (#1512) 2026-10-20 11:32:22 -05:00
Chris Tate 324f173f65 Merge main 2026-10-20 11:23:45 -05:00
Chris Tate 25780d9231 Merge main 2026-10-20 11:03:09 -05:00
Chris Tate 8eb5ad28e5 Keep WebAssembly references and memory storage alive while they are in use; merge main
Externrefs a guest holds are borrowed pointers. An import's result was
kept only until the outermost call returned, which was before the export
call converted its results to JavaScript, so a result created by an
import was freed before conversion. A table or global slot released the
reference it replaced at once, although the guest may still hold it in a
local after reading the slot. The kept references are now released after
the outermost caller has converted its results (scr_wasm_leave), and a
table or global store during a call keeps the reference it replaces until
then.

Checked memories (explicit bounds checks: sanitized builds, Windows)
grew with realloc, which can move the storage that memory.buffer and its
views alias. They now reserve the address space for their largest size
(mmap on POSIX hosts, VirtualAlloc on Windows) and commit pages as they
grow, so the base never moves. Hosts without address-space reservation
grow in place only until a buffer has been handed out; after that,
growth that would move the storage fails.

Tests: webassembly-externref-lifetimes.mjs (import results through an
export, table and global references replaced by the guest and by
JavaScript), webassembly-memory-growth.test.ts (views written after
growth from JavaScript and from the guest), and translate.test.ts cases
for memargs naming memory 0 explicitly and for local.tee operand types,
which b3bcbaaf7 made consistent with validation. Under SCRIPTC_SAN=1 the
two runtime tests report heap-use-after-free without these changes.

Merges main; internal/compatibility/generated/node-v24-backlog.json is
regenerated with pnpm node-compat.
2026-10-20 10:58:34 -05:00
Malte UblandClaude Opus 5.5 c7b2620e5c Compile events.once with an AbortSignal option and over stream receivers
events.once(emitter, name, { signal }) lowers to events.onceSignal, served
by a new runtime unit (scr_events_once_signal.c, linked only through the
eventsOnceSignal feature). It follows Node 24's lib/events.js: an
undefined signal is the plain form, a value that is not an AbortSignal
rejects with ERR_INVALID_ARG_TYPE, an aborted signal rejects with an
AbortError whose cause is signal.reason, and otherwise a once,
resist-stop-propagation 'abort' listener removes the event and 'error'
listeners and rejects. The event or the error removes the abort listener.
Options other than an object literal with one signal stay refused.

events.once now accepts stream receivers: a stream's runtime-emitted
events use their fixed tuples, and its 'data' and 'keypress' events stay
refused. A NodeJS stream interface slot that holds a stdio handle requests
the general stream representation.

Node's once is an async function that returns the executor's promise, so
its result adopts that promise through a thenable job. The plain form
settled its result two turns early; both forms now settle on Node's turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 08:47:51 -07:00
Chris Tate 21c6505709 Match tuple rests and positional any[] parameters in function patterns, and merge main
- matchInferSignature reads a fixed-tuple rest as its positions, so
  (...args: [string, number]) needs two arguments, and a positional
  any[] pattern parameter matches only when the function's parameter
  there accepts an array, as TypeScript's assignability does
  (corpus 6398).
- Merge origin/main (200fe8d22); node-v24-backlog.json regenerated.
2026-10-20 10:37:43 -05:00
Chris Tate e0b131c0f9 Merge main into opt/variadic-generic-wrappers 2026-10-20 10:19:08 -05:00
Chris Tate da476df676 Merge main 2026-10-20 10:11:51 -05:00
Chris Tate 7d555c4cf3 Merge main 2026-10-20 09:04:02 -05:00
Malte UblandClaude Opus 5.5 dfd59e18ee Let NodeJS stream interface slots hold JavaScript stream objects
Under the general stream representation, NodeJS.ReadableStream and
NodeJS.WritableStream compile as the runtime Readable and Writable
classes, so a JavaScript stream object (a constructor-function stream, a
plain object with on/pipe, a stream from bundled npm code) passed through
a TypeScript function typed with them took the checked unwrap and threw.

A program that moves an untyped value into a runtime stream class slot,
or passes an untyped pipe() destination or pipeline stage, now lowers
once more with checked-dynamic stream slots: the two interfaces (and
intersections over them) map to dyn. A native stream rides its typed
capsule and keeps its identity; any other value is the JavaScript object
itself. Member calls dispatch per value: stream.dynNative tests for a
boxed native stream, which takes the ordinary class-method lowering;
other values call their own member through dynInvoke. pipe() into a
JavaScript destination follows Node's Readable.prototype.pipe listener
protocol (stream.pipeDyn). for await and pipeline stages keep the
not-supported error for values that are not native streams.

The runtime code lives in scr_stream_dyn.c behind a new streamDyn
feature, so programs without these libCalls link the same objects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 06:49:37 -07:00
Chris Tate ce5e4294ad Merge main 2026-10-20 08:49:25 -05:00