Any change to the runtime's C sources makes every executable differ, so
the gate failed even when every program's IR and LLVM were byte-identical.
The gate now classifies each difference outside --allow:
- runtime-only: IR and LLVM identical, the same LLVM helper, runtime pack
objects that differ between the checkouts (bitcode alone records the
checkout path, so it does not count), and every symbol the executable
gains, loses or resizes defined by a changed runtime object. It passes
with a note that lists the changed runtime functions the program links
(from the release and speed bitcode, through llvm-dis) and the size
change.
- nondeterministic: the baseline builds the program again, and its two
builds differ in the same artifacts. It passes with a note.
- differs: anything else; it fails as before.
--strict fails runtime-only and nondeterministic programs too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parallel changes kept taking the same next corpus number (6992 twice, the
Node island programs renumbered several times). The harness never needed
the numbers: it discovers every entry by glob, and 818 programs already
have descriptive names. AGENTS.md now makes a name, without a number, the
convention for new programs and keeps the numbered ones as they are.
The two scripts that sampled programs by number skipped named programs.
They now key a named program by a stable hash of its name, so they sample
it at the same rate; the numbered programs they pick are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parallel branches kept picking the same numbers: dyn handle tags, SC
diagnostic codes, and the values the LLVM backend copies from runtime
enums.
- Diagnostic codes: codes.ts lists every active, retired and reserved code.
ScrDiagnostic.code is typed by it, and codes.test.ts fails on a code
without a row, a retired code in use, an unused row, or rows out of
order. The Node island codes open PRs already use are reserved.
- Dyn handle tags: a new tag is added by name to SCR_DYNH_NEW_TAGS and the
enum numbers it after the largest pinned value, so branches never pick
a number. The pinned values are unchanged; value 63, never assigned, is
listed as unassigned, and the test fails on any other gap.
- The backend's copies of ScrDynKind, ScrMessageKind, ScrBytesElem, the
closure function_kind bits (now named SCR_CLOSURE_KIND_* in C) and the
%ScrArr spare flags are checked against their C declarations.
- scripts/registry-claims.mjs lists the codes and pinned tags open PRs
claim and reports collisions; it finds handle tag 81 claimed by three
open PRs today.
Every numeric value is unchanged, so IR, LLVM and executables are
byte-identical.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A warm rebuild ran about 3.5 MB of the native compiler's code, but the code
was spread over its 69 MB text, so 64 KB fault-around mapped about 20 MB of
it. The emitter can now take a function layout: a list of functions in
tiers. It writes the listed definitions, generated helpers included, first
and by tier, each in a `layout-<tier>` partition group. The LLVM helper
compiles each partition's definitions in text order, so the listed ones
come first in their partitions, and a helper that places functions by
group keeps a tier's definitions in the same partitions. The rest keep
their order and their source-file groups.
- compile() takes a `functionLayout` file (`<tier> <symbol>` lines,
gzip-compressed when the name ends in .gz). Keys leave out module
numbers.
- build-native-cli.mts uses the committed packages/compiler/native/cli.layout.gz,
or the file SCRIPTC_NATIVE_LAYOUT names; an empty value and instrumented
builds use none.
- scripts/refresh-cli-layout.mjs builds an instrumented compiler, builds
the runtime workloads and a corpus slice cold and then unchanged, and
reads the functions each run executed from the raw profiles. Tier 1 is
`--version` and rebuilds that hit both caches; tier 2 is the other builds.
- The release workflow's cli-profile job also collects a layout on the
release commit, and the CLI builds use it. Names of closures and
generated helpers change with nearby edits, so a committed layout
matches fewer functions as main moves on.
- Forked lowering and emission workers reset the profile counters after
fork and write their profile before _exit, so instrumented builds also
count the work done in workers. This applies to refresh-cli-profile.mjs
too.
Executables the compiler produces are unchanged.
Without @types/node, every program loaded scriptc-node-fallback.d.ts whole:
the console and process globals and 47 Node modules, about 8,100 lines,
parsed, bound and checked in every cold build whether or not the program
imported them.
- The fallback file keeps the globals and the modules they name (console,
readline, readline/promises, perf_hooks, tty). Every other module, under
both spellings, has its own file in ambient/node-fallback/, which
references the files of the modules its declarations name.
- A module's file joins the program when one of the program's files names
the module in a string literal: imports, requires, import types and JSDoc
types all name modules that way. The entry is read before the first
program; the program-root discovery passes add modules that other files
name. Files are read as tsgo reads them, and the TypeScript libraries and
scriptc's own declarations are skipped.
- The checks that recognized the fallback file by its path recognize every
fallback file.
With the overlay on, a lowering refusal no longer fails the build. The
annotations the refusal depends on are dropped and the program is loaded
and lowered again, so those declarations compile untyped, as they do
without the overlay:
- Each overlay annotation carries an id (`@scriptcShape #<pos>`).
- shape-inference/fallback.ts blames the annotations of the declarations
enclosing a refusal and of the declarations its expression names,
following unannotated functions' returns.
- Rounds that stop converging drop whole files' overlays, then the
overlay for the build. SC6103 reports what was dropped.
Also:
- The tsc gate's project world is built over the files as written; it
used to see the overlay's files, which failed the typecheck of three
npm cases.
- Arrays a builtin creates (match results, `new Array(n)`) are no longer
typed by the overlay: they may have holes, and the library types them.
- compile() accepts `shapeOverlay: true`.
tests/harness/npm-shape-overlay.test.ts runs the whole npm suite with the
overlay on (npm.test.ts's body moves to npm-suite.ts): every case builds
and runs as in the default build, and a refused build that compiles with
the overlay matches Node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its replace callbacks receive checked-dynamic values where the lib types
their parameters any: captures a match can leave unset, and results the
checker types any.
On the san 1/20 shard the sanitized nodes build shares the runner with the
corpus's other two workers and the side queue. It took 944 s and 860 s on
#2194's runs and passed 1200 s on main's 7fa88083, failing the test timeout.
The Linux san corpus now skips that one case
(SCRIPTC_THREE_SANITIZED_NODES_JOB), and a three_nodes_san job runs it alone
with the corpus's LeakSanitizer settings. The required test check waits for
the new job. Every other three.js case, the plain nodes build and the macOS
suites are unchanged.
The workload reads its file through readline over fs.createReadStream.
The stream and readline runtime units define dynamic-tier symbols on
their own: a program that only reads a file with createReadStream links
the same 32, as log-summary and inventory-report do.
readline split redirected input byte by byte, twice per chunk (once to
find whether the chunk ends a line, once to split it). Before splitting
it copied each chunk out of its Buffer and into a scratch buffer, decoded
it into a new string, and copied the unfinished line and the whole chunk
into a joined buffer. Now:
- one scan per chunk: memchr for a CR once, then LF to LF with memchr
when the chunk has none, and the LF, CR and CRLF split otherwise;
- the unfinished line joins only the first line of the chunk;
- an ASCII chunk with no partial UTF-8 sequence held from the previous
one is split where it is, and a chunk that is not shared memory is
read from its own storage. Lines are still copied into their strings
before the first is delivered.
Buffer.from(string, 'latin1' | 'ascii') encoded character by character;
an ASCII prefix, all of an ASCII string, is now copied with memcpy.
Line events, their order and the CRLF delay are unchanged. Corpus 8344
covers lines joined across chunks, a line longer than a chunk, empty
lines, mixed line endings and UTF-8 split across chunks. The new
line-reader workload reads a 67 MB log through readline over a file
stream.
The release workflow's new cli-profile job runs
scripts/refresh-cli-profile.mjs on Linux x64 and uploads the profile.
Every platform's native CLI build uses it through
SCRIPTC_NATIVE_PROFILE_USE, and the GitHub release attaches it as
scriptc-<version>-cli.profdata.gz. When the job fails, the CLI builds
use the committed profile.
build-native-cli.mts accepts a gzip-compressed profile in
SCRIPTC_NATIVE_PROFILE_USE when the name ends in .gz.
refresh-cli-profile.mjs creates the directory of --output before it
starts.
The native compiler is built with a committed optimization profile. A
function whose control flow changed since the profile was collected is
optimized as without one, so the profile loses its gain as compiler code
changes, with nothing to show it.
With --profile-use, the helper now counts what the profile annotation
reports while it applies the profile: functions that took the profile's
counts, functions whose control flow no longer matches (hash mismatch),
and functions the profile has no record of. It sums the counts over
partitions, keeps each partition's counts beside its cached summary, and
prints them as one line of JSON after a successful emit. The compiler
reads that line (an older helper prints none) and returns it as the
executable artifact's profileMatch.
build-native-cli prints the share of profiled functions that match, and
warns below 90% that the profile should be refreshed. On main aa1c26845, a
profile refreshed on that commit matches 92.7% of the compiler's profiled
functions, and the one committed about 200 commits earlier 87.2%; that one
kept about half of the fresh profile's gain on a cold tsc-ts build.
Emitted functions report an exception by returning with the thread's
pending flag set, and every caller tests the flag after each call that
may throw. On Linux executables (glibc and musl), functions reached only
by direct calls now propagate exceptions by unwinding the native stack
instead:
- unwindAbiFunctions selects the may-throw functions that are not async
or generators, constructors, virtually called or overridden methods,
closure targets, the entry, library exports, module initializers or
callees of coroutine bodies.
- A native unwind costs about 2 µs a raise and 0.1 µs a frame, so functions that
exceptions are likely to cross keep the flag: those with a `throw` of
their own, those up to three calls above one, and recursive functions,
counting calls through methods, closures and callbacks.
- Their propagating exits call scr_unwind_raise (scr_unwind.c), a forced
unwind through _Unwind_ForcedUnwind with the exception still in the
active cell.
- Direct calls to them are `invoke`s without a pending check. The
landing pad runs the frame's usual unwind path: it releases the frame's
values and then enters the enclosing try handler or resumes the unwind.
- Remaining plain calls to them (wrappers, adapters, runtime-facing entry
points) become invokes whose landing pad continues on the pending flag,
or resumes when the wrapper is itself invoked. Every other call is
marked `nounwind`, so inlining a raising function at an invoke keeps its
calls plain.
- When emitted text breaks these rules (a raising function's address is
taken, or an invoked function returns with the flag pending), the
module is emitted again with those functions on the pending flag.
Emitted code does not declare scr_unwind_raise `noreturn`, so sanitized
builds do not run the sanitizer's no-return hook, which needs stack
bounds that fiber stacks do not report. Instead, scr_unwind_raise clears
the shadow of the frames it unwinds.
Executables link a static unwinder: libgcc_eh with the host toolchain and
the glibc runtime packs, and Zig's libunwind with Zig and the musl packs.
macOS, Windows, Android, WASI, libraries and debug builds keep the
pending flag everywhere and emit the same code as before.
The throw-chains runtime workload measures exceptions caught 2, 5 and 20
recursive calls above the throw, through direct calls and a callback,
and a parser that backtracks by throwing.
Reapplies #2017, which #2038 reverted after effect's observability-otlp
fixture crashed in plain builds. The crash was the candidate-buffer
corruption that the previous commit fixes: the cached property sites
change when the collector runs, and with enough environment variables a
pass buffered a string box that was later pooled while still buffered.
The Linux test shards run their suites in a transient systemd scope with
MemoryMax=13G and no swap, below the runner's 16 GB. A process tree that
grows past the cap loses its largest process to the kernel, and the job
log names the failing test, where the runner itself ran out of memory
and left no log. Hosts without systemd run the suites directly.
The side-queue balancer also takes a table of memory-heavy files, with
their measured peak. They run whole in the cost-balanced task, and each
goes to the shard with the least memory-heavy work so far, after the
time-balanced files. decl-boundary.test.ts, which peaks at about 1.8 GB
alone and had no time measurement, so vitest's file sharding put it in
shard 11, is the first entry. It now runs in shard 1 (plain) and 6 (san).
Release builds compile for the x86-64 baseline, so LLVM vectorizes with
SSE2 only. `--target-cpu=<baseline|x86-64-v2|x86-64-v3|x86-64-v4|native>`
compiles the program's code for a higher level: every emitted function
carries the level as its `target-cpu`, so LLVM can use AVX2, BMI2, FMA or
AVX-512. The executable then runs only on CPUs of that level.
- `native` resolves to the highest level the build machine supports, read
from /proc/cpuinfo on Linux and `sysctl machdep.cpu` on macOS.
- The level is part of the executable cache key; `baseline` and builds
without the option produce the same output and keys as before.
- The runtime pack keeps its baseline code.
- Targets other than x86-64 refuse a level (SC3002). The CLI's cached
bootstrap route leaves builds with the option to the full compiler.
- `bench:runtime` takes `--baseline-target-cpu` and `--candidate-target-cpu`.
A program that declares a class reads and writes untyped properties
through helpers that first select the receiver's property bag. The bag
helpers return jsvals, so the helpers read the bag through js.toDyn;
keyed helpers test each class's key table between the key's coercion and
the read; and a program with subclasses normalizes the receiver
(value = %dyn.class.normalize(value)) at the start of each helper. The
inline-cache recognizer accepted none of these forms, so every property
site in such a program called its helper. It now looks through dyn/jsval
conversions of the receiver, accepts capsule branches before a keyed
helper's final read, and accepts a rebinding of the receiver to a bag of
itself. Plain objects and arrays then read and write through the site's
cache, as they do in programs without classes.
bench/dynamic-properties (new) exercises these sites.
tsc-ts 0.0.8's API server serves on the named pipe the Node host
connects over on Windows (tsc-ts --api --pipe), and the native compiler
already starts it over its standard streams there, so tsc-ts is the
default checker on every platform. SCRIPTC_CHECKER=tsgo still selects
tsgo.
The compiler depends on tsc-ts 0.0.8, which the release-age exclusions
admit with its platform packages and @scriptc/threads 0.4.2.
The Windows CLI smoke test runs with both checkers, and
verify-native-cli.mjs --run checks that a distribution's tsc-ts runs on
its platform (tsc-ts.exe on Windows).
