Commit Graph
736 Commits
Author SHA1 Message Date
Chris Tate 3b520e4fb2 Merge main 2026-10-28 20:50:23 -05:00
Chris Tate 377af617d3 Add --exceptions=rare to unwind through recursive functions
With --exceptions=rare, recursive functions and functions up to three
calls above a throw also propagate exceptions by unwinding on Linux, so
calls to them need no pending-flag test. Functions with a throw of their
own keep the flag. The default, common, keeps the current rules, and
builds without the flag emit the same code as before.

The option is part of the executable cache key; the bootstrap's cached
route leaves builds with it to the full compiler, and coverage and
library builds refuse it.
2026-10-28 14:28:28 -05:00
Chris Tate 44a4cd9db1 Merge main 2026-10-28 13:19:00 -05:00
Chris Tate 8b27e9cd2f Name program modules by the entry's file name so profiles match in every checkout (#2462) 2026-10-28 06:43:48 -05:00
Chris Tate 35eda071b2 Give JavaScript fetch bodies the members of a web ReadableStream (#2459) 2026-10-28 06:41:21 -05:00
Chris Tate 23d95a99fd Name program modules by the entry's file name so profiles match in every checkout
Each partition's source file name was the program entry's absolute path,
and profiles name a module's local functions after it. Since programs keep
partition-local functions internal (#2449), a profile collected in one
directory missed those functions in builds from any other: a native
compiler profile collected on main named 27% of its records, holding 2.2%
of its counts, after the checkout's path. Modules now carry the entry's
file name. The partition cache key still includes the full path.
2026-10-28 05:44:38 -05:00
Chris Tate 3bf316de6b Annotate only mismatched functions from block counts
A block-count file annotated every function the profile left without
counts, including functions the profile has no record of. The recovered
count could then exceed the mismatched functions it annotated and hide the
stale-profile warning for mismatched functions left without counts. Only
functions the profile annotation tagged as mismatched are annotated now, so
the recovered count stays among the mismatched ones.
2026-10-28 05:10:57 -05:00
Chris Tate d9481023c3 Annotate functions a stale profile misses from its block counts
A profile annotates a function only while its control flow hashes as it did
when the profile was collected, so compiler and source changes leave the
changed functions without counts. A --profile-use build now reads a
block-count file beside the profile (app.blocks beside app.profdata). Right
after the annotation before inlining, the helper aligns each uncounted
function's blocks with its recorded blocks by label kind and successor
count. Paired blocks take their recorded counts and their edges the recorded
edge counts, by target or, between targets whose labels changed, by
position. LLVM's flow inference fills in the rest, avoiding the edges the
function's own llvm.expect weights make unlikely. The helper then sets the
entry count, branch weights and hot or cold attribute as the profile
annotation does.

SCRIPTC_RECORD_PROFILE_BLOCKS names the file a --profile-use build writes
for the functions its profile annotates; such a build reuses no cached
artifacts. The block file keys every cache, the build report counts the
functions it annotated, and the staleness warning counts only the functions
left without counts. A single-module build with a context-sensitive profile
no longer counts annotated, mismatched and missing functions a second time
after inlining. scripts/refresh-cli-profile.mjs records the file in the
second phase and build-native-cli.mts reads it beside the profile.
2026-10-28 04:50:01 -05:00
Chris Tate 17efed329f Give JavaScript fetch bodies the members of a web ReadableStream
In JavaScript, a Response or Request body answered getReader, cancel and
locked, and every other ReadableStream member was undefined: tee,
pipeTo, pipeThrough, values and for await threw TypeErrors where Node's
body is a ReadableStream.

In programs that link the web stream classes, the body's handle now
stands for a ReadableStream once any other member is used
(sf_stream_web_view): a stream with a zero high-water mark whose native
source reads the body (scr_webstream_readable_over). From then on every
member goes to it, the class's method values and getters run on it
through the alias's view (scr_dyn_handle_class_receiver), and text(),
clone(), new Response(body) and fetch uploads read the body through it,
so its lock and bodyUsed follow it as in Node. A body locked when the
view is made keeps the view locked until the lock is released. Once
only the view's source holds the body, the body lets the view go.

The compiler links the classes into a program whose JavaScript reads a
ReadableStream member that the fetch profile lists as unsupported by the
native stream, or iterates a ReadableStream with for await, and that
names none of the classes: %main calls webstream.install first.
Programs without web streams build to the same binaries as before.

Adds corpus programs fetch-body-web-stream-linked.mjs,
fetch-body-web-stream-unlinked.mjs and fetch-body-web-stream-network.mjs,
with their corpus costs and preflight/order baselines.
2026-10-28 03:44:57 -05:00
Chris Tate 41e7950ea9 Keep frozen checked records frozen, and give sloppy-mode this the global object
Typed records checked from frozen untyped values stay frozen, as the
objects they copy are. A sloppy-mode function called with undefined or null
as this receives the global object that globalThis reads.

The record check reads the source's integrity with a query that answers for
every value: a proxy answers as its target, and values without a modeled
integrity level, which Object.freeze refuses, are not frozen. Object.isFrozen
keeps its refusal of those values.
2026-10-27 23:07:13 -05:00
Chris Tate 82ba2ef6ed Merge main 2026-10-27 23:02:33 -05:00
Chris Tate 9e91b7d912 Merge main 2026-10-27 21:44:40 -05:00
Chris Tate 05810305cf Compile small release and speed programs in partitions 2026-10-27 20:49:11 -05:00
Chris Tate 21fa3a8ec7 Merge main 2026-10-27 20:22:19 -05:00
Chris Tate 919b79b62a Revert "Keep frozen checked records frozen, and give sloppy-mode this the global object" 2026-10-27 19:20:37 -05:00
Chris Tate e941d5d659 Run JavaScript WritableStream, TransformStream and pipes natively (#2437) 2026-10-27 18:43:35 -05:00
Chris Tate ea8c1e6a34 Keep frozen checked records frozen, and give sloppy-mode this the global object (#2436) 2026-10-27 18:33:05 -05:00
Chris Tate fa046b8757 Compile dev executables of more than 64 KiB in partitions
A dev executable was one partition up to 1 MiB of LLVM text, so every edit
compiled the whole program again. Unoptimized partitions are compiled on
their own, with nothing imported across them, so dividing a program does
not change its code's quality, and the helper reuses each partition an edit
does not change from the partition cache.

nativeProgramPartitions now takes whether the program is optimized. Dev
executables are divided from 64 KiB of LLVM text, with at least 32 KiB per
partition: four partitions up to 128 KiB, then at least 8 as before. Release
and speed executables keep their partition and code generation counts.
2026-10-27 17:32:28 -05:00
Chris Tate f945efecee Run JavaScript WritableStream, TransformStream and pipes natively
In JavaScript sources, WritableStream with its default writer and
controller, TransformStream with its controller, pipeTo and pipeThrough,
and TextEncoderStream and TextDecoderStream join the native web stream
classes, following Node 24's webstreams step by step: sinks and
transformers, backpressure and the writers' ready and closed promises,
abort with the controller's signal, the pipe's shared write requests and
its abort signal, and UTF-8 encoding and decoding across chunks settle in
Node's order. The streams take own properties, which a pipe does not use
for its own reads and writes, and the global object answers the classes
by computed name in programs that name them.

A JavaScript class that extends one of these classes keeps the SC1090
refusal, and TextDecoderStream refuses encodings other than UTF-8.

The node24-web-stream lifecycle, errors, queues, transforms, pipes,
pipe-errors, pipe-abort, tee and encoding cases now match Node.
2026-10-27 15:55:31 -05:00
Chris Tate 62d4cecd3a Keep frozen checked records frozen, and give sloppy-mode this the global object 2026-10-27 15:22:08 -05:00
Chris Tate 5dc4a4daa6 Map large executables' text with 2 MiB pages 2026-10-27 15:04:45 -05:00
Chris Tate cc6ee85fd7 Run JavaScript web streams on a native ReadableStream (#2411) 2026-10-27 14:20:02 -05:00
Chris Tate f12a2b2a23 Take JavaScript ReadableStreams as fetch bodies and in instanceof by value
A JavaScript ReadableStream given to Response, Request or fetch as a body
was converted to the string "[object ReadableStream]". The fetch unit now
reads such a stream through a reader of its own. A locked or disturbed
stream gives Node's TypeError, and the chunks must be Uint8Array
when the body is consumed ("Received non-Uint8Array chunk").

instanceof against a web stream class held in a variable or passed as an
argument walked no prototype for handles and answered false. It now starts
at the class prototype of the handle. The fetch unit's own streams, readers
and controllers, such as Response.body, are instances of the JavaScript
classes, so `response.body instanceof ReadableStream` is true again.

These paths, and the fetch reader functions' hand-off to web stream
readers, are in SCR_HANDLE_CLASSES variants of scr_json.c and scr_fetch.c,
which only programs with JavaScript web streams link; other programs that
use fetch build to the same size as on main.
2026-10-27 10:00:00 -05:00
Chris Tate 18196d4df0 Lay out code by context-sensitive profiles 2026-10-27 09:36:18 -05:00
Chris Tate f69ff780e5 Give a stop command that matches the shared checker server
The server's command line puts --cwd and --callbacks between --api and
--listen, so `pkill -f 'tsc-ts --api --listen'` matched no server that a
build started. The docs now give `pkill -f 'tsc-ts --api .*--listen'`, and
name the flags instead of writing them as one command.
2026-10-27 08:33:04 -05:00
Chris Tate 3b374f3201 Check with the shared tsc-ts server by default on Linux and macOS
Builds on Linux and macOS now connect to a shared `tsc-ts --api --listen`
server by default, which keeps the files it parsed and bound, so a later
build in the same working directory does not parse the TypeScript libs and
scriptc's declaration files again. SCRIPTC_CHECKER_SERVER=0 turns it off.
When CI is set to a value other than 0 or false, builds use their own tsc-ts
process unless SCRIPTC_CHECKER_SERVER=1.

- The server exits after 600 seconds without a build (was 300).
- Its name also digests the compiler executable's identity, so each scriptc
  installation and build connects only to servers it started.
- The server process runs in the root directory, so it does not keep the
  first build's directory in use; its --cwd argument is unchanged.
- When the connection to the server fails after it answered, through a
  failed transfer or an ended stream, the native CLI runs the command again
  in a new process with SCRIPTC_CHECKER_SERVER=0 and exits with its status.
- SCRIPTC_CHECKER_SERVER no longer changes the frontend cache key: the
  server's answers are the build's own checker's.

The CLI docs describe the background process, its socket directory, when it
exits and how to stop it.
2026-10-27 08:21:48 -05:00
Chris Tate 9ca9c7af9d Run JavaScript web streams on a native ReadableStream
In JavaScript sources, ReadableStream, ReadableStreamDefaultReader,
ReadableStreamDefaultController, CountQueuingStrategy and
ByteLengthQueuingStrategy (globals and node:stream/web exports) are class
values over a new runtime unit, scr_webstreams.c, which follows Node 24's
webstreams step by step: underlying sources, queuing strategies, default
readers, tee, ReadableStream.from and async iteration settle their promises
in Node's order. scr_dyn_handle_class.c gives a dyn handle tag a
constructor value with a prototype, brand checks and Node's constructor
errors. Stream state carries cycle headers, so the collector reclaims
streams whose callbacks close over their controllers.

Promise steps run as native promise reactions, a variant of scr_async.c
that only programs with web streams link; every other program links the
default object. TypeScript sources keep their static web stream lowerings.
Byte streams, BYOB readers, pipeTo and pipeThrough throw SC2020.

node24-web-stream-readers and node24-web-stream-readable-lifecycle now
match Node.
2026-10-27 06:53:26 -05:00
Chris Tate d4185bcd8e Add an opt-in --static flag for Linux executables
- `scriptc build --static` (and the `staticLink` compile option) links glibc
  executables as static PIE on the build host's own glibc target; musl
  executables are already static and accept the flag.
- An executable that keeps glibc's NSS module loader (host, user and group
  name lookups) is refused with SC3002, since a static glibc executable
  loads those modules from the run host.
- Other targets, cross-compiled glibc targets, --sanitize and non-executable
  output are refused; static and dynamic builds use separate cache entries.
2026-10-27 03:34:16 -05:00
Chris Tate 723975d01c Prove that the arrays InlineArray fields hold are never written
The constructor copies the array it is passed into the instance, and
other reads of the field give a new array over the elements, while under
Node.js the field holds that array itself. The contract check now proves
the two agree, and reports SC7009 where it cannot:

- The arrays fields hold are never written. A whole-program graph of the
  places arrays are held (locals, parameters, results, class and record
  fields, and the elements of arrays, Maps and Sets by where those are
  held) follows every array a field read may give, and reports a write to
  one, or one that reaches a dynamic value, a call through a function
  value or a library call other than printing and JSON.stringify without
  a replacer. Functions that return their argument give each caller its
  own argument back. The report names the field read.
- The array passed to the constructor is one nothing else keeps: a new
  array (a literal, a copy, or the result of a function that returns only
  such arrays or its argument), a local array at its last use (after calls
  that only read it or write it in place, or in a loop that then assigns
  the local a new array), a parameter whose every caller passes such an
  array, an array a field already holds, or a const array that is never
  written, which is then followed like a field's.

The review fixes from before stay: `new` evaluates its arguments into
locals unless they are all locals and constants, so that the allocation
reads the length after every argument has run; the constructor uses the
array parameter only to store it; and the dynamic view of an instance
reads the field as an array of its elements.
2026-10-27 02:38:15 -05:00
Chris Tate 618243caa8 Generate code concurrently for optimized executables that stay one partition (#2387) 2026-10-27 01:10:09 -05:00
Chris Tate 640001b260 Generate code concurrently for optimized executables that stay one partition
Optimized executables with at most 1 MiB of LLVM text were compiled as one
module on one thread. With more than 128 KiB of LLVM text they now pass
several outputs and `--split codegen` to the LLVM helper. The helper
optimizes the module whole, as for a single output, then divides its
functions in module order into runs of about the same instruction count and
generates code for each run on its own thread.

- The output count is 4 or 8 and depends only on the module's size
  (nativeCodegenParts), so artifacts never vary with the host. Node and
  native compiler builds pass the same outputs.
- The helper writes the optimized module to bitcode once with its use-list
  order. Every output after the first parses it and turns the other
  outputs' definitions into declarations; the first output does the same to
  the optimized module in place. Each function's machine code matches a
  single-output build except for jumps to a definition that another output
  also uses.
- Only local definitions that another output uses become hidden. Dead
  constant users are dropped before unused declarations are removed, so no
  output names a thread-local variable it does not use.
- When one run would hold more than three quarters of the instructions, the
  first output is generated from the optimized module as for a single
  output and the others are empty objects.
- Dev builds, partitioned builds and program-object validation are
  unchanged.
2026-10-27 00:22:17 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Chris Tate acf4c39f73 Keep the elements of InlineArray fields in their instances 2026-10-26 22:38:40 -05:00
Chris Tate 7e225c5d87 Merge main 2026-10-26 17:33:06 -05:00
Chris Tate 3c83fcb89c Allocate object groups that die together in arenas (#2348) 2026-10-26 16:54:31 -05:00
Chris Tate 18755e0599 Allocate object groups that die together in arenas 2026-10-26 11:41:21 -05:00
Chris Tate 2f13edd058 Merge main 2026-10-26 11:36:06 -05:00
Malte Ubl 572e7e3eac Merge remote-tracking branch 'origin/lf/l4-expressions' into lf/l5-polish
# Conflicts:
#	packages/compiler/src/frontend/lowering/lowerer.ts
2026-10-26 05:37:00 -07:00
Chris Tate fe244085d4 Run native async generators as stackless coroutines
An async generator whose body suspends only at its own awaits and yields
now compiles to an LLVM switched coroutine, as async functions of that
kind already do, instead of running on a fiber with its own stack. Its
steps run on the stack that resumes it (the loop's after an await), its
awaits suspend through scr_coro_await_prepare and scr_coro_hop_prepare,
and its yields store OUT through scr_coro_yield_* before suspending.

- scr_async_gen_new_stackless creates the generator. The stackless hop
  settles an async generator's state as the fiber hop does, and a step
  that would start below the stack guard throws a RangeError, which
  rejects its request.
- Calling an async generator's next, return or throw no longer counts as
  parking the caller: they queue a request and run the body to its next
  suspension, which returns to the caller. Bodies that consume async
  generators can now be stackless too.
- An untyped await (async.awaitDyn) has a native coroutine form,
  extracted by scr_coro_await_dyn_value, so async bodies that await
  untyped values can be stackless.
- An async generator's return through a finally region takes its await
  turn through the coroutine when the body is one.

Debug builds, libraries, worker programs and Windows executables keep
fibers, as for stackless async functions.
2026-10-26 07:28:50 -05:00
Chris Tate b8aaa5985a Merge main 2026-10-26 05:23:00 -05:00
Chris Tate 4553941d84 Merge main 2026-10-26 04:25:15 -05:00
Chris Tate acf19e2652 Merge main 2026-10-26 02:53:26 -05:00
Chris Tate 2d24d6ad80 Merge main 2026-10-26 00:41:23 -05:00
Chris Tate 21bce9e4fa Merge main 2026-10-26 00:21:24 -05:00
Chris Tate f591e8e6d7 Print how a --profile-use build's profile matched
`scriptc build --profile-use` prints one line on stderr with the number of
functions the profile annotated, the number whose control flow changed since
it was collected (mismatched), and the number it has no record of. When more
than 10% of the functions the profile describes are mismatched, it also prints
a warning to collect the profile again. The build still succeeds.

Both the compiler library's build and the native compiler's build report the
match. Cached builds report the match of the build that produced them: the
native codegen and object caches store it beside the cached objects, and the
executable cache stores it in its entry.
2026-10-25 22:14:25 -05:00
Malte UblandClaude Opus 5.5 1ae5f105ca Rank dynamic slots by their dynamic operations, name them in SC6101, and document them
Stage L5 of the localized-failure design.

- SC6101 names a dynamic slot as the cause of a likely-hot untyped function
  ('tool' is a dynamic slot, declared 'ToolDefinition', which does not
  compile).
- Coverage counts the dynamic operations each slot's values take part in,
  lists the costly slots first, and puts the count in the JSON envelope.
- A build whose only performance warnings are dynamic slots closes with a
  line about declared types instead of function types.
- Docs: a Dynamic slots section in the coverage guide, the limitations
  page bullet, and --fail-on=dynamic-slots in the CLI reference and usage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 19:48:18 -07:00
Chris Tate cc2266ab1d Add --cpu-dispatch for x86-64 executables (#2288) 2026-10-25 21:36:00 -05:00
Chris Tate 80ae660be6 Merge main 2026-10-25 21:22:05 -05:00
Chris Tate 59e78f298a Name the package declaration and its patch when a declared-type check fails (#2268) 2026-10-25 21:20:34 -05:00
Chris Tate 8981b5c0a2 Resolve computed createRequire specifiers at run time (#2079) 2026-10-25 20:29:51 -05:00