Commit Graph
149 Commits
Author SHA1 Message Date
Chris Tate 60be717d92 Merge main 2026-10-29 04:24:50 -05:00
Chris Tate 10479ce2e0 Link the scriptc CLI without huge text
An executable with at least 4 MiB of program code is linked with 2 MiB
segment alignment and published with one write, so the page cache holds
its text in 2 MiB folios and the runtime advises huge pages for it. A run
then maps every 2 MiB text window it touches whole. Most runs of the
compiler are short: `scriptc --version` peaked at 76 MB and an unchanged
rebuild at 78 MB, against 20 MB and 26 MB with 4 KiB alignment, and cold
and edited builds of tsc-ts took the same time either way.

compile() takes an internal `hugeText: false`, which links with 4 KiB
alignment whatever the code size and is part of the executable cache key.
scripts/build-native-cli.mts passes it. Executables the compiler builds
keep huge text.
2026-10-29 01:01:21 -05:00
Chris Tate 3e7528ca86 pi wave 4: path values, union delete, NonNullable unions, BigInt stats, optional-param function values, live accessor literals, clearer stdlib-alias errors (#2497) 2026-10-28 22:33:16 -05:00
Malte UblandClaude Opus 5.5 7176257136 fs.watchFile: poll a path from its own gated runtime unit
fs.watchFile(path[, options], listener) polls the resolved path every
`interval` milliseconds (5007 by default, `persistent` by default) and
calls its listener with (current, previous) Stats when a poll sees a
difference. The first poll only records the baseline, a path that cannot
be stat'd polls as all-zero Stats, and a stat failure that repeats for
the same errno is not re-reported. One poller per RESOLVED path is
shared by every listener on it, and fs.unwatchFile(path[, listener])
removes the last listener registered with that callback, or every
listener when called without one. The call is a statement, since Node's
StatWatcher has no lowering, and bigint: true is refused.

The implementation is scr_watch_file.c, a new gated runtime unit
switched by moduleUsesFsWatchFile, so programs that do not poll link
none of it. The feature travels the usual chain (runtime-features,
executable-features, the native link info, the executable cache keys,
the runtime-pack matrix) and the validator learns that
fs.unwatchFile's trailing listener slot may be omitted.

Evidence:
- tests/corpus/fs-watch-file-names.ts and fs-watch-file-polling.ts match
  Node on stdout, stderr and exit status in the plain and sanitized lanes
- 1564-fs-watch.ts, 1751-options-record-stance.ts and the `fs` corpus stay
  green; the new order baselines and corpus cost files are recorded

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 19:49:44 -07:00
Malte UblandClaude Opus 5.5 cf7814d19c Lower fs statSync/lstatSync/fstatSync with { bigint: true }
Node answers a BigIntStats when the stat call is passed `bigint: true`: the
same snapshot, read through a bigint member table. Every number member of
Stats reads as a bigint — the integers, the `*Ms` millisecond members (the
truncated nanosecond quotient, as the binding computes them) and the
BigIntStats-only `*Ns` nanosecond members (sec * 1e9 + nsec, exactly) —
while only the four time members stay Dates. scriptc refused the option
outright.

The representation is static, chosen by the option literal: the lowerer
validates `bigint` against the call's result type (so
`{ bigint: true, throwIfNoEntry: false }` still narrows
`BigIntStats | undefined`), refuses a runtime-valued option and an option
that disagrees with the destination, and keeps every other option key
fenced. fstatSync joins statSync/lstatSync on the same path, with the
descriptor evaluated before the options.

The IR gains a `bigintStats` handle kind beside `stats`: the same RC
family, the same non-traced leaf and non-dynamic handle treatment, so no
shape, tracing or dyn-check behavior moves for any program that never asks
for one. Its member table is a `bigstats.*` lib-call family whose Date
members reuse the existing stat reads.

The runtime support is its own unit, scr_stats_bigint.c: the snapshot
layout moved into scr_stats_private.h so the bigint getters can read the
same fields without scr_lib.c carrying their code. The unit is gated on the
existing `bigint` feature in both the runtime-pack matrix and the
compiler's from-source unit table (the sanitized lane links from that one),
and `-ffunction-sections` keeps a bigint program that never reads a member
from linking any of the getters. No typed program's IR or LLVM changes.

tests/corpus/7018-bigint-stats.ts pins the member table, the Date/bigint
split, ms/ns agreement, arithmetic on a member, lstat over a symlink,
fstatSync over a descriptor, the throwIfNoEntry union, and the plain Stats
answers of the un-optioned and `bigint: false` forms against Node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 17:58:01 -07:00
Chris Tate 061020b2f8 Add --exceptions=rare to unwind through recursive functions (#2481) 2026-10-28 15:17:22 -05:00
Chris Tate 377af617d3 Add --exceptions=rare to unwind through recursive functions
With --exceptions=rare, recursive functions and functions up to three
calls above a throw also propagate exceptions by unwinding on Linux, so
calls to them need no pending-flag test. Functions with a throw of their
own keep the flag. The default, common, keeps the current rules, and
builds without the flag emit the same code as before.

The option is part of the executable cache key; the bootstrap's cached
route leaves builds with it to the full compiler, and coverage and
library builds refuse it.
2026-10-28 14:28:28 -05:00
Malte UblandClaude Opus 5.5 753532a3da Merge origin/main into fix/pi-wp2-stdlib-node
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 11:33:39 -07:00
Malte Ubl e5f57e2093 Merge remote-tracking branch 'origin/main' into HEAD 2026-10-28 08:43:13 -07:00
Malte Ubl 29a31446d3 Merge remote-tracking branch 'origin/main' into HEAD 2026-10-28 08:43:06 -07:00
Malte UblandClaude Opus 5.5 39326c55be runtime: ship child.off/signalCode in a gated unit
Adding the six new entry points (scr_child_off_spawn/_exit/_close/_error,
scr_child_has_signal_code, scr_child_signal_code) to scr_child.c changed
the executable of every program that links the unit, although none of
them calls the new functions and IR and LLVM stayed identical: the extra
functions grow the unit's `.eh_frame` contribution, and the linker's
frame bookkeeping for the retained sections changes with it (the first
visible difference is the phdr sizes at byte offset 208). Appending them
after the unit's own functions does not help — the unit's frame section
still differs, so all 89 corpus programs that link scr_child.o got a
different binary.

Move the entries into scr_child_lifecycle.c, a unit of their own
selected only by programs that use them (feature childLifecycle, the
scr_dyn_handle_props.c pattern: the matrix's optional source list, the
RUNTIME_FEATURE_RULES rule for child.offSpawn/offExit/offClose/offError/
signalCode, and the link-feature plumbing through the executable cache,
CcOptions and RUNTIME_UNITS). scr_child.c keeps its original content and
layout; the new unit reads its registries through scr_child_private.h,
which holds the per-platform ScrChild layout both units compile against.

A program that does not call the new entries links exactly the units it
did before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:38:17 -07:00
Malte UblandClaude Opus 5.5 efc22a2fd3 Box user module namespaces as dynamic values
A `file:` module namespace handed to typed code now boxes as Node's
namespace object instead of the "converting typed values to 'unknown'"
fence: `const u: unknown = ns`, `take(ns)`, a `Promise<typeof
import("./m")>` widened to `Promise<unknown>` (or `Promise<unknown> |
undefined`), and `await import("./m")` in an `unknown` slot all see one
non-extensible null-prototype object per module whose own properties are
the module's exports in Node's sorted order. They are enumerable
read-only accessors whose reads call the compiled namespace getter, so
they stay live, and the object carries Symbol.toStringTag "Module",
Node's strict-mode write TypeErrors, the not-extensible add error and
the ToPrimitive throw. `node:` builtins have no compiled export table
and keep their SC1101 fence.

The object is built by a new gated runtime unit,
packages/runtime/src/scr_module_namespace_dyn.c, selected by the
`moduleNamespaceDyn` runtime feature (`module.namespaceDyn` in the IR).
It adds no field to ScrDynHandleOps and no branch to a shared unit: the
core reaches it through the ordinary SCR_DYN_OBJ paths, so key listing,
for-in, `in`, JSON, identity, property reads and assignment all reuse
the existing dispatch. The compiler reuses the getter lifted for
module.namespaceValue (existing programs' IR is unchanged) and adds
moduleNamespacePromiseDynValue, one lifted async mapper per module for
the promise slots.

Zero cost: a program that never boxes a namespace links exactly what it
did. The zero-cost gate reports every program identical, with
byte-identical IR and LLVM; the runtime pack's only difference is the
new unit (0 changed runtime functions), and among the namespace corpus
programs only 8412 links it.

Evidence: tests/corpus/8412-module-namespace-untyped is a differential
program against Node in both the plain and sanitized lanes, and the
diagnostics snapshots and TypeScript 7 baselines stay unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:37:45 -07:00
Chris Tate d9481023c3 Annotate functions a stale profile misses from its block counts
A profile annotates a function only while its control flow hashes as it did
when the profile was collected, so compiler and source changes leave the
changed functions without counts. A --profile-use build now reads a
block-count file beside the profile (app.blocks beside app.profdata). Right
after the annotation before inlining, the helper aligns each uncounted
function's blocks with its recorded blocks by label kind and successor
count. Paired blocks take their recorded counts and their edges the recorded
edge counts, by target or, between targets whose labels changed, by
position. LLVM's flow inference fills in the rest, avoiding the edges the
function's own llvm.expect weights make unlikely. The helper then sets the
entry count, branch weights and hot or cold attribute as the profile
annotation does.

SCRIPTC_RECORD_PROFILE_BLOCKS names the file a --profile-use build writes
for the functions its profile annotates; such a build reuses no cached
artifacts. The block file keys every cache, the build report counts the
functions it annotated, and the staleness warning counts only the functions
left without counts. A single-module build with a context-sensitive profile
no longer counts annotated, mismatched and missing functions a second time
after inlining. scripts/refresh-cli-profile.mjs records the file in the
second phase and build-native-cli.mts reads it beside the profile.
2026-10-28 04:50:01 -05:00
Chris Tate 7d670f0e2a Merge main 2026-10-27 18:53:59 -05:00
Chris Tate 1a86f94239 Merge main 2026-10-27 16:45:14 -05:00
Chris Tate 8f3a8e3e63 Merge main 2026-10-27 09:58:30 -05:00
Malte Ubl caee8dfef2 Merge the restacked base into ni/i2-handles
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/index.ts
2026-10-27 05:41:01 -07:00
Chris Tate e84d332264 Link AbortController without the fetch unit
AbortController and AbortSignal lowered to fetch.abort* entry points, so
a program that only used them linked the fetch unit with its socket,
HTTP, TLS and zlib dependencies. They now lower to the abort.* entry
points on every target, a new abortSignal feature links scr_abort.c (the
fetch unit's signal implementation built without its transport, which
WASI already ships) and the EventTarget unit, and programs that also
fetch keep the fetch unit's copy.
2026-10-27 07:05:47 -05:00
Chris Tate 9ca9c7af9d Run JavaScript web streams on a native ReadableStream
In JavaScript sources, ReadableStream, ReadableStreamDefaultReader,
ReadableStreamDefaultController, CountQueuingStrategy and
ByteLengthQueuingStrategy (globals and node:stream/web exports) are class
values over a new runtime unit, scr_webstreams.c, which follows Node 24's
webstreams step by step: underlying sources, queuing strategies, default
readers, tee, ReadableStream.from and async iteration settle their promises
in Node's order. scr_dyn_handle_class.c gives a dyn handle tag a
constructor value with a prototype, brand checks and Node's constructor
errors. Stream state carries cycle headers, so the collector reclaims
streams whose callbacks close over their controllers.

Promise steps run as native promise reactions, a variant of scr_async.c
that only programs with web streams link; every other program links the
default object. TypeScript sources keep their static web stream lowerings.
Byte streams, BYOB readers, pipeTo and pipeThrough throw SC2020.

node24-web-stream-readers and node24-web-stream-readable-lifecycle now
match Node.
2026-10-27 06:53:26 -05:00
Chris Tate f4a4cf8652 Merge main 2026-10-27 04:32:00 -05:00
Chris Tate f73722c2ef Add an opt-in --static flag for Linux executables (#2398) 2026-10-27 04:31:35 -05:00
Chris Tate d4185bcd8e Add an opt-in --static flag for Linux executables
- `scriptc build --static` (and the `staticLink` compile option) links glibc
  executables as static PIE on the build host's own glibc target; musl
  executables are already static and accept the flag.
- An executable that keeps glibc's NSS module loader (host, user and group
  name lookups) is refused with SC3002, since a static glibc executable
  loads those modules from the run host.
- Other targets, cross-compiled glibc targets, --sanitize and non-executable
  output are refused; static and dynamic builds use separate cache entries.
2026-10-27 03:34:16 -05:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 60d5dc8ab1 Count class-value constructions in the census, and key caches on both diagnostics
- The heap census also counts instances a class value constructs: the
  construct thunk that `newValue` calls counts them after its allocation,
  as `new` does at its own.
- A build with SCRIPTC_HEAP_CENSUS set is its own executable cache entry
  (`heapCensus` in the early cache's key and route key), and one with
  SCRIPTC_REPR_REPORT set does not read the early cache, since a hit
  emits no module and so writes no report. The CLI's startup route defers
  both to the full compiler.
- Test: a warm cache still builds the census executable, counting
  `new` and class-value constructions, and still writes the report.
2026-10-27 01:54:48 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Malte Ubl 3a5aab5b2f Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD
# Conflicts:
#	packages/compiler/src/cli/command.ts
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/frontend/lowering/builtins/module-resolution.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/compiler/src/index.ts
#	packages/runtime/src/scr_json.c
#	tests/harness/npm.test.ts
2026-10-26 11:22:14 -07:00
Malte Ubl 5817be4d0f Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/ir/function-references.test.ts
2026-10-26 11:08:44 -07:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate 0ba138b57e Merge main 2026-10-26 03:25:54 -05:00
Chris Tate 7e3d10fa13 Lay out the native compiler's functions by the code each kind of run executes (#2265) 2026-10-26 00:29:02 -05:00
Chris Tate d48e060076 Merge main 2026-10-25 22:34:40 -05:00
Chris Tate f591e8e6d7 Print how a --profile-use build's profile matched
`scriptc build --profile-use` prints one line on stderr with the number of
functions the profile annotated, the number whose control flow changed since
it was collected (mismatched), and the number it has no record of. When more
than 10% of the functions the profile describes are mismatched, it also prints
a warning to collect the profile again. The build still succeeds.

Both the compiler library's build and the native compiler's build report the
match. Cached builds report the match of the build that produced them: the
native codegen and object caches store it beside the cached objects, and the
executable cache stores it in its entry.
2026-10-25 22:14:25 -05:00
Chris Tate cc2266ab1d Add --cpu-dispatch for x86-64 executables (#2288) 2026-10-25 21:36:00 -05:00
Chris Tate 59e78f298a Name the package declaration and its patch when a declared-type check fails (#2268) 2026-10-25 21:20:34 -05:00
Chris Tate 28b76f7b25 Merge ni/i1-stack-bound 2026-10-25 21:08:37 -05:00
Chris Tate 8981b5c0a2 Resolve computed createRequire specifiers at run time (#2079) 2026-10-25 20:29:51 -05:00
Chris Tate 07c5e79ffd Add --cpu-dispatch: x86-64-v3 copies of the program's code chosen at startup
With --cpu-dispatch, an x86-64 executable carries two copies of each
program function: the baseline code and a copy compiled for x86-64-v3.
main reads CPUID and XGETBV once. Calls between the program's functions
go to the caller's copy. A function that is also named outside the
copies (a function value, a call from main, a runtime callback or a
coroutine body) keeps a dispatcher under its own name, which tests the
level and tail-calls one copy. Functions without such a use get no
dispatcher, so their copies have the callers the original had and LLVM
inlines them the same way.

The executable still runs on any x86-64 CPU. SCRIPTC_CPU_LEVEL caps the
level at run time. The option is off by default, applies to executables
without a --target-cpu level, and is part of the executable cache key.
bench:runtime takes --baseline-cpu-dispatch and --candidate-cpu-dispatch,
and the LLVM differential suite runs each program at levels 0 and 3 with
SCRIPTC_CPU_DISPATCH=1.
2026-10-25 20:00:16 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Malte UblandClaude Opus 5.5 56e8417746 Name the package declaration and its patch when a declared-type check fails
A value that contradicts a type an npm package declares used to throw
"expected number at $, got undefined" and nothing else. Such a check now
names the declaration and the type patch that admits the value:

    profilekit@3.1.0 declares Profile.email as string (index.d.ts:7)
    if that declaration is wrong, patch it in package.json: ...

Origins are found for values a package function returns under a written
return type, for properties of records typed by a package-declared type
(record projections into such a type are keyed by it), and for
declaration-boundary casts, whose origin the projection records from the
declaration-world checker. Generic declarations get none: their types
come from the program's own instantiation.

Checked conversions carry dynCheck.origin, an index into
IrModule.declOrigins; the site's existing cold exception block calls
scr_type_origin_note (new runtime unit behind the typeOrigins feature),
which appends the note to the pending TypeError and derives the selector
suffix and the admitted kind from the failure. Union extractions get the
note per arm at compile time. The hot path is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 15:06:50 -07:00
Chris Tate e7a088f904 Key the executable cache by the function layout's contents
The early executable cache key covered the profile but not the function
layout, so a build that added, changed or edited a layout could restore
the executable built with the previous order. The key now includes the
digest of the layout file's bytes. An unreadable layout fails the build
with SC3002 before the cache lookup.
2026-10-25 16:58:11 -05:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate b7915b92a7 Merge main 2026-10-25 06:38:44 -05:00
Chris Tate 303bfcd922 Merge main 2026-10-25 05:25:41 -05:00
Chris Tate 0eef6251de Merge main 2026-10-25 02:28:32 -05:00
Chris Tate f3750ef976 Merge base and fix 2026-10-25 02:16:24 -05:00
Chris Tate 2e052854c5 Merge main 2026-10-25 01:21:07 -05:00
Chris Tate 5d3e0c7a8b Support X509Certificate fields and methods, TLS certificates and tls.Server contexts (#2052) 2026-10-25 01:00:24 -05:00
Malte UblandClaude Opus 5.5 4a8a6567ca Add the shape overlay: typed JavaScript from analysis-proven facts (opt-in)
The shape overlay (DESIGN.md, approved route) rewrites the JavaScript of
opted-in npm packages with JSDoc types the shape analysis proves: record
typedefs for non-escaping literal sites (optional fields for spread
sources), class field, parameter, return and local types, receiver casts,
and branches the analysis proved unreachable replaced by checks. The
lowering trusts only annotations carrying the overlay's marker; package
JSDoc stays a hint (its type tags are neutralized in overlaid files).

- Inferred-fact checks: every checked conversion lowered in an overlaid
  file records its site, and a failure throws a distinct internal error
  ("scriptc internal error: an inferred type was contradicted at <file:line>
  … This is a bug in scriptc's shape analysis, not in the program's data")
  naming the opt-outs; SCRIPTC_SHAPE_CHECK_LOG makes checks log instead
  (validation mode).
- Opt-outs: `--no-shape-overlay`, and `"scriptc": {"packages": {"<name>":
  {"shapes": false}}}` per package.
- Build report: SC6103 counts the declarations typed, the checks inserted
  and the branches replaced.
- `x instanceof Promise` on plain data folds to false.

The overlay runs only with SCRIPTC_SHAPE_OVERLAY=1 while it is new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:24:51 -07:00
Chris Tate e3bda96fd1 Merge main 2026-10-24 20:31:45 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00