Commit Graph
260 Commits
Author SHA1 Message Date
Chris Tate d4c6aee135 Merge main 2026-10-29 20:04:10 -05:00
Chris Tate 03e461b842 Match macOS objects and paths in the native install, loop poll and profile tests (#2541) 2026-10-29 15:07:46 -05:00
Chris Tate bbec52d459 Give class methods and function values jsval entries for untyped calls 2026-10-29 06:54:29 -05:00
Chris Tate ea0cdd0418 Match macOS objects and paths in the native install, loop poll and profile tests
The nightly macOS lanes failed these tests on Linux-specific expectations:

- native-install: installNativeCli resolves the installation directory,
  and on macOS /tmp is a link to /private/tmp. The fixture root is now
  resolved too.
- counted-loop-emission: two tests counted polls of @scr_exc_flag, the
  thread's pending word that ELF executables read. Other targets poll the
  process-wide alert word. The tests now emit for x86_64 Linux, as the
  file's other ELF tests do.
- native-partitions: Mach-O objects define an ltmp<N> label at the start
  of each section, which the function lists now leave out.
- native-profile: profile temperature appears in ELF section names
  (.text.hot.); Mach-O sections carry no such prefix. The section name
  check, and the check that recovered block counts change the objects (they
  move f4 into the hot section), now run on ELF objects only.
2026-10-29 06:36:29 -05:00
Chris Tate a2e1251759 Prepare v0.4.6 release 2026-10-29 06:02:00 -05:00
Chris Tate 377af617d3 Add --exceptions=rare to unwind through recursive functions
With --exceptions=rare, recursive functions and functions up to three
calls above a throw also propagate exceptions by unwinding on Linux, so
calls to them need no pending-flag test. Functions with a throw of their
own keep the flag. The default, common, keeps the current rules, and
builds without the flag emit the same code as before.

The option is part of the executable cache key; the bootstrap's cached
route leaves builds with it to the full compiler, and coverage and
library builds refuse it.
2026-10-28 14:28:28 -05:00
Chris Tate d9481023c3 Annotate functions a stale profile misses from its block counts
A profile annotates a function only while its control flow hashes as it did
when the profile was collected, so compiler and source changes leave the
changed functions without counts. A --profile-use build now reads a
block-count file beside the profile (app.blocks beside app.profdata). Right
after the annotation before inlining, the helper aligns each uncounted
function's blocks with its recorded blocks by label kind and successor
count. Paired blocks take their recorded counts and their edges the recorded
edge counts, by target or, between targets whose labels changed, by
position. LLVM's flow inference fills in the rest, avoiding the edges the
function's own llvm.expect weights make unlikely. The helper then sets the
entry count, branch weights and hot or cold attribute as the profile
annotation does.

SCRIPTC_RECORD_PROFILE_BLOCKS names the file a --profile-use build writes
for the functions its profile annotates; such a build reuses no cached
artifacts. The block file keys every cache, the build report counts the
functions it annotated, and the staleness warning counts only the functions
left without counts. A single-module build with a context-sensitive profile
no longer counts annotated, mismatched and missing functions a second time
after inlining. scripts/refresh-cli-profile.mjs records the file in the
second phase and build-native-cli.mts reads it beside the profile.
2026-10-28 04:50:01 -05:00
Chris Tate d2595a4a70 Merge main 2026-10-27 18:49:24 -05:00
Chris Tate 1a86f94239 Merge main 2026-10-27 16:45:14 -05:00
Chris Tate 18196d4df0 Lay out code by context-sensitive profiles 2026-10-27 09:36:18 -05:00
Malte Ubl f9419b5d47 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into HEAD
# Conflicts:
#	packages/compiler/src/index.ts
2026-10-27 05:40:18 -07:00
Chris Tate f4a4cf8652 Merge main 2026-10-27 04:32:00 -05:00
Chris Tate f73722c2ef Add an opt-in --static flag for Linux executables (#2398) 2026-10-27 04:31:35 -05:00
Chris Tate 8cdf783023 Find glibc's NSS loader in the --static link map
The NSS check scanned the linked executable for the `libnss_` literal, so
a program with that text in a string of its own was refused. A --static
glibc link now writes a link map, and the executable is refused when the
map places code from libc.a's nss_module.o or nsswitch.o in it (lld's
kept input sections, or GNU ld's memory map with a nonzero size).
2026-10-27 04:00:21 -05:00
Chris Tate d4185bcd8e Add an opt-in --static flag for Linux executables
- `scriptc build --static` (and the `staticLink` compile option) links glibc
  executables as static PIE on the build host's own glibc target; musl
  executables are already static and accept the flag.
- An executable that keeps glibc's NSS module loader (host, user and group
  name lookups) is refused with SC3002, since a static glibc executable
  loads those modules from the run host.
- Other targets, cross-compiled glibc targets, --sanitize and non-executable
  output are refused; static and dynamic builds use separate cache entries.
2026-10-27 03:34:16 -05:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 60d5dc8ab1 Count class-value constructions in the census, and key caches on both diagnostics
- The heap census also counts instances a class value constructs: the
  construct thunk that `newValue` calls counts them after its allocation,
  as `new` does at its own.
- A build with SCRIPTC_HEAP_CENSUS set is its own executable cache entry
  (`heapCensus` in the early cache's key and route key), and one with
  SCRIPTC_REPR_REPORT set does not read the early cache, since a hit
  emits no module and so writes no report. The CLI's startup route defers
  both to the full compiler.
- Test: a warm cache still builds the census executable, counting
  `new` and class-value constructions, and still writes the report.
2026-10-27 01:54:48 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Chris Tate df77671e33 Merge main 2026-10-26 23:13:34 -05:00
Chris Tate 47fd397395 Expect the unwind table attribute in the --target-cpu test's attribute group 2026-10-26 22:24:00 -05:00
Malte Ubl 141279b74c Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/13-libnode-packs 2026-10-26 11:15:43 -07:00
Malte Ubl 5817be4d0f Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/ir/function-references.test.ts
2026-10-26 11:08:44 -07:00
Chris Tate 34c2f9851d Have the native CLI packages depend on their LLVM helper and runtime pack (#2333) 2026-10-26 11:47:34 -05:00
Chris Tate d5dc896576 Merge main 2026-10-26 10:53:46 -05:00
Chris Tate 5a14c0a3dc Merge main 2026-10-26 08:37:13 -05:00
Chris Tate d1a0bea84e Have the native CLI packages depend on their LLVM helper and runtime pack
Each @scriptc/cli-<platform> package bundled its platform's LLVM helper
and runtime pack, which are also published as @scriptc/llvm-<platform> and
@scriptc/runtime-<platform>. The CLI packages now depend on those packages
at the same version (workspace:*, which pnpm pack writes as the exact
version) and leave them out: cli-linux-x64-gnu goes from 115 MB to 60 MB
packed.

- build-native-cli.mts --platform-packages, which each CLI package's
  build:native passes, writes a manifest without llvm_package, runtime_pack
  and runtime_packs.
- loadNativeToolchain resolves a missing helper or host runtime pack as
  Node resolves the package from the compiler's directory: nested in the
  platform package, beside it (hoisted, or pnpm's virtual store) or higher
  up. It requires the compiler's version, and an absent or mismatched
  package fails with an error that names it.
- installNativeCli records where the package manager put both packages in
  the installed manifest, with the same checks.
- verify-native-cli.mjs requires the exact dependencies of a platform
  package and checks the resolved packages' versions.
- package-native-cli.mjs bundles the helper and runtime pack into the
  standalone archives, which stay self-contained.
- smoke-native-install.mjs installs the helper and runtime pack tarballs
  with the platform package; the bootstrap's exported package drops its
  bundled copies first.
- release.yml publishes the CLI packages only once every helper and
  runtime pack package resolves on npm, and scriptc only once the CLI
  packages resolve, each wait at most 25 minutes. A rerun skips the
  uploads npm already accepted and waits again.
2026-10-26 07:48:11 -05:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate 0ba138b57e Merge main 2026-10-26 03:25:54 -05:00
Chris Tate ca3217fe99 Reuse the completion callback a stream passes to its option callbacks
Each call of a typed or untyped write, writev, transform, flush, final,
construct or destroy option callback allocated a closure and a box holding
the stream for its `callback` argument. The emitted thunks now take it
from scr_stream_done_closure, which keeps the last one on the stream state and
hands it out again while the stream holds its only reference and it has
no own properties. A callback the user kept (stored, or passed to
setImmediate or a promise) is not reused: the next call gets a new
closure, and the kept one keeps its own stream reference.

The cached closure holds the stream, so the stream drops it when it
emits 'close' and traces it, which lets the cycle collector reclaim a
stream dropped before it closes.

RUNTIME_ABI_VERSION goes from 46 to 47 for scr_stream_done_closure.
2026-10-26 02:48:03 -05:00
Chris Tate d48e060076 Merge main 2026-10-25 22:34:40 -05:00
Chris Tate f591e8e6d7 Print how a --profile-use build's profile matched
`scriptc build --profile-use` prints one line on stderr with the number of
functions the profile annotated, the number whose control flow changed since
it was collected (mismatched), and the number it has no record of. When more
than 10% of the functions the profile describes are mismatched, it also prints
a warning to collect the profile again. The build still succeeds.

Both the compiler library's build and the native compiler's build report the
match. Cached builds report the match of the build that produced them: the
native codegen and object caches store it beside the cached objects, and the
executable cache stores it in its entry.
2026-10-25 22:14:25 -05:00
Chris Tate 23deb595ac Relocate the prebuilt lib diagnostics path when installing the native command
The native toolchain manifest's lib_diagnostics path is relative to the
platform package's dist/bin. installNativeCli copies the manifest into the
scriptc package's bin and relocates each asset path, but lib_diagnostics
was not in the list, so it pointed at a missing file under the scriptc
package and npm installs did not use the prebuilt lib diagnostics. It is
now relocated with the other assets.
2026-10-25 21:41:36 -05:00
Chris Tate cc2266ab1d Add --cpu-dispatch for x86-64 executables (#2288) 2026-10-25 21:36:00 -05:00
Chris Tate 59e78f298a Name the package declaration and its patch when a declared-type check fails (#2268) 2026-10-25 21:20:34 -05:00
Chris Tate 07c5e79ffd Add --cpu-dispatch: x86-64-v3 copies of the program's code chosen at startup
With --cpu-dispatch, an x86-64 executable carries two copies of each
program function: the baseline code and a copy compiled for x86-64-v3.
main reads CPUID and XGETBV once. Calls between the program's functions
go to the caller's copy. A function that is also named outside the
copies (a function value, a call from main, a runtime callback or a
coroutine body) keeps a dispatcher under its own name, which tests the
level and tail-calls one copy. Functions without such a use get no
dispatcher, so their copies have the callers the original had and LLVM
inlines them the same way.

The executable still runs on any x86-64 CPU. SCRIPTC_CPU_LEVEL caps the
level at run time. The option is off by default, applies to executables
without a --target-cpu level, and is part of the executable cache key.
bench:runtime takes --baseline-cpu-dispatch and --candidate-cpu-dispatch,
and the LLVM differential suite runs each program at levels 0 and 3 with
SCRIPTC_CPU_DISPATCH=1.
2026-10-25 20:00:16 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Chris Tate 30cf205e90 Run untyped pipeline function stages and later iterable stages
A pipeline stage that arrives as an untyped value now follows Node's
stage ladder for more shapes. As the first stage, a function is a source
function: the pipeline calls it with { signal } while it is set up and
pumps its result, which must be iterable (ERR_INVALID_RETURN_VALUE
otherwise), and a { readable, writable } pair pumps its readable side.

After the first stage, the pipeline call is lowered once per combination
of its first three untyped stages being functions or not, and a typeof
test picks the branch at run time. A function stage is called with the
previous stage and { signal }; before the last stage its result must be
async iterable, and as the last stage a promise resolves the pipeline's
value and an async iterable feeds the PassThrough pipeline() returns
(SCR_PIPE_DEST_DYN). Any other value that is not a stream becomes
Duplex.from of it, which the next stage reads from without the previous
stage being piped into it, and Duplex.from's ERR_INVALID_ARG_TYPE for
the values it rejects.

An untyped stage followed by an untyped function stage hands its value to
that stage through a cell created per pipeline call, so the function
receives the previous stage's own iterator, as Node's makeAsyncIterable
answers it, instead of an async generator over it.

Untyped web streams, pairs after the first stage, and function stages
that return a stream throw the not-supported error.

RUNTIME_ABI_VERSION goes from 45 to 46 for the new stream exports.
2026-10-25 18:10:32 -05:00
Malte UblandClaude Opus 5.5 7675b7d653 Add scriptc types patch and scriptc types list
`scriptc types patch <package> <selector> <type> [--range] [--dry-run]`
finds the installed package from the working directory, shows every
declaration the selector matches (declaration files, JSDoc, @types
package) with its type before and after, and adds the patch to the
nearest package.json: into an entry whose range covers the installed
version, or under ^<version>. Nothing is written when the selector
matches nothing. `scriptc types list` shows each patch and what it
matches. Both CLI hosts run the same implementation with their own
parser. The TypeError note for a wrong declaration now names the command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 16:00:29 -07:00
Chris Tate cbe4a9387b Merge main 2026-10-25 15:24:25 -05:00
Chris Tate aead30fc26 Mark emitted functions sanitize_address only in sanitized builds
The sanitized lane's -fsanitize=address link needs the attribute to
instrument the emitted functions. Without a sanitizer pass it is not
inert: LLVM does not shrink-wrap functions that carry it, so every call
saved and restored all of a function's callee-saved registers on entry,
including on paths that return before any call, and it blocks
speculative loads. The LLVM helper already dropped it for builds that
import runtime bitcode (speed); release builds kept it.

- The new sanitizeAttributes option (default true for direct emission)
  is set from --sanitize wherever objectAudit is.
- Without it and without a target CPU level, attribute group #0 holds a
  string attribute LLVM ignores, since a group cannot be empty.
2026-10-25 15:10:32 -05:00
Chris Tate 826519133f Merge main 2026-10-25 14:49:01 -05:00
Chris Tate 7a175c374f Prepare v0.4.5 release 2026-10-25 11:26:13 -05:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate 1a0d58d1eb Merge main and move to ABI 45 2026-10-25 06:31:07 -05:00
Chris Tate edae637526 Support named properties and Symbol.species on typed arrays held as untyped values (#2068) 2026-10-25 05:48:27 -05:00
Chris Tate 1b40ac0c64 Merge main 2026-10-25 05:15:58 -05:00
Chris Tate fa02fcd8ae Report which packages and modules would run in a Node island (#1691) 2026-10-25 04:32:22 -05:00
Chris Tate 4050c0fb5c Merge main and skip package.json files without a scriptc key 2026-10-25 03:58:49 -05:00
Chris Tate 1e799cb60a Answer net.Socket write() with Node's backpressure boolean, and fire 'drain'
A TCP socket's write() answered nothing in typed code and true through
untyped values, and a 'drain' listener never fired. Node 24 answers
false when the bytes the write leaves unsent reach the socket's
65536-byte high-water mark (writeOrBuffer computes it after the socket
writes), sets needDrain, and emits 'drain' when the write buffer
empties; writes to an ended or destroyed socket answer false.

The runtime's write entry points now return that answer, record
needDrain, and fire the socket's 'drain' listeners from the flush that
empties the buffer, unless the write half is ending or the socket is
gone. writableLength reads the unsent bytes and writableNeedDrain the
flag. on('drain') and once('drain') register on the typed path and
through untyped values, and socket.write() may be used as a value.
The fallback node:net declarations type write() as boolean, add the
'drain' event and both properties.

RUNTIME_ABI_VERSION goes from 43 to 44: the socket write entry points
return a boolean.
2026-10-25 03:40:30 -05:00
Chris Tate 2e052854c5 Merge main 2026-10-25 01:21:07 -05:00