Shard 1 of "test (macOS arm64 LLVM artifacts, no clang)" ran the no-clang
object, helper parity, packaged helper and packed npm installation
contracts around its LLVM differential slice: about 12 minutes on top of
a 35-minute slice, 45 to 49 minutes against the job's 50-minute timeout.
The contracts now run in "test (macOS arm64 LLVM artifacts, no clang,
contracts)", which the all-lanes check requires like the shards, and
every shard runs only its differential slice.
The libnode-linux action becomes .github/actions/libnode, which serves
the Linux x64 and macOS lanes: the libnode-packs cache for the runner's
platform, else the development pack.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
macOS binds libnode's own calls to v8::base::Stack inside libnode, so the
bridge's definitions cannot give V8 Node's stack as they do on Linux; V8
took the main thread's stack start and its conservative scan faulted in
the gap above Node's stack. While Node starts on its stack, the main
thread's pthread record now names that stack, which V8 reads once and
keeps.
macOS's makecontext clears the whole stack it is given, so it gets only
the top slice of Node's stack (no 64 MiB pass per context).
CI: dev-pack.json pins mirrored darwin-arm64 and darwin-x64 libnode
builds of the pinned Node 24.22.0 from the libnode-packs workflow, the
dev-pack action serves either platform, and the macOS lanes provision it,
so island tests fail rather than skip on macOS CI. The arm64 LLVM job
runs the island harness; the Intel job runs it and the island corpus.
bridge-smoke loads the .dylib bridge on macOS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each @scriptc/cli-<platform> package bundled its platform's LLVM helper
and runtime pack, which are also published as @scriptc/llvm-<platform> and
@scriptc/runtime-<platform>. The CLI packages now depend on those packages
at the same version (workspace:*, which pnpm pack writes as the exact
version) and leave them out: cli-linux-x64-gnu goes from 115 MB to 60 MB
packed.
- build-native-cli.mts --platform-packages, which each CLI package's
build:native passes, writes a manifest without llvm_package, runtime_pack
and runtime_packs.
- loadNativeToolchain resolves a missing helper or host runtime pack as
Node resolves the package from the compiler's directory: nested in the
platform package, beside it (hoisted, or pnpm's virtual store) or higher
up. It requires the compiler's version, and an absent or mismatched
package fails with an error that names it.
- installNativeCli records where the package manager put both packages in
the installed manifest, with the same checks.
- verify-native-cli.mjs requires the exact dependencies of a platform
package and checks the resolved packages' versions.
- package-native-cli.mjs bundles the helper and runtime pack into the
standalone archives, which stay self-contained.
- smoke-native-install.mjs installs the helper and runtime pack tarballs
with the platform package; the bootstrap's exported package drops its
bundled copies first.
- release.yml publishes the CLI packages only once every helper and
runtime pack package resolves on npm, and scriptc only once the CLI
packages resolve, each wait at most 25 minutes. A rerun skips the
uploads npm already accepted and waits again.
The Linux helpers linked LLVM 22.1.8's release archive. The helper's code
generation is LLVM's code, so building that LLVM from its release sources
with ThinLTO and with a profile of the helper's own work makes code
generation 5-9% faster than the helper as released, with identical objects.
- native/llvm-codegen/llvm.profdata.gz is the profile, collected by
scripts/refresh-llvm-profile.mjs: it builds LLVM with IR instrumentation
and an instrumented helper, and runs that helper on tsc-ts, five Effect
fixtures, the runtime benchmark workloads and every 50th corpus program
(scripts/llvm-helper-inputs.mjs). It follows LLVM's sources, so it needs
a refresh when LLVM is bumped.
- scripts/build-llvm-dist.mjs builds LLVM with that profile and ThinLTO,
and packs it with a checksum. The llvm-dist workflow, run by hand, builds
it for x64 and arm64, requires a helper linked against it to give the
same objects as one linked against the release archive
(scripts/check-llvm-dist.mjs), prints both helpers' times and the
profile's mismatch count, and attaches the archive to the LLVM_DIST_TAG
release of this repository.
- The release workflow's Linux jobs, and the wasm32-wasi job, which also
uploads the x64 GNU helper, download that archive and build the helper
with SCRIPTC_LLVM_THINLTO=1: clang, lld and ThinLTO, and the profile for
the helper's own sources. Without the archive, or if that build fails,
the helper links the release archive as before.
- The four Linux helper packages share their build in
native/llvm-codegen/linux-helper.mjs. Without SCRIPTC_LLVM_THINLTO it
builds the same executable as before.
The release workflow publishes each package and moves on once npm accepts
the upload. A package whose processing on npm stalls was visible only on
npmjs.com.
A new npm-available job runs after "Publish to npm". It checks each package
of the release with `npm view <name>@<version> version --prefer-online`
every 2 minutes (scripts/wait-npm-available.mjs) until every package
resolves, and reports each package's time from upload to available in the
log and the job summary. When a package still does not resolve after 2
hours, the job fails and names each missing package and version. It only
reads from the registry.
The publish job lists every package of the release and records the time
npm accepted each upload, keeps those times with its upload receipts, and
passes both to the new job. Create GitHub Release now waits for the new
job, so the release is created and marked complete only once every
package resolves; a later release run checks again and skips uploads npm
already accepted.
A warm rebuild ran about 3.5 MB of the native compiler's code, but the code
was spread over its 69 MB text, so 64 KB fault-around mapped about 20 MB of
it. The emitter can now take a function layout: a list of functions in
tiers. It writes the listed definitions, generated helpers included, first
and by tier, each in a `layout-<tier>` partition group. The LLVM helper
compiles each partition's definitions in text order, so the listed ones
come first in their partitions, and a helper that places functions by
group keeps a tier's definitions in the same partitions. The rest keep
their order and their source-file groups.
- compile() takes a `functionLayout` file (`<tier> <symbol>` lines,
gzip-compressed when the name ends in .gz). Keys leave out module
numbers.
- build-native-cli.mts uses the committed packages/compiler/native/cli.layout.gz,
or the file SCRIPTC_NATIVE_LAYOUT names; an empty value and instrumented
builds use none.
- scripts/refresh-cli-layout.mjs builds an instrumented compiler, builds
the runtime workloads and a corpus slice cold and then unchanged, and
reads the functions each run executed from the raw profiles. Tier 1 is
`--version` and rebuilds that hit both caches; tier 2 is the other builds.
- The release workflow's cli-profile job also collects a layout on the
release commit, and the CLI builds use it. Names of closures and
generated helpers change with nearby edits, so a committed layout
matches fewer functions as main moves on.
- Forked lowering and emission workers reset the profile counters after
fork and write their profile before _exit, so instrumented builds also
count the work done in workers. This applies to refresh-cli-profile.mjs
too.
Executables the compiler produces are unchanged.
On the san 1/20 shard the sanitized nodes build shares the runner with the
corpus's other two workers and the side queue. It took 944 s and 860 s on
#2194's runs and passed 1200 s on main's 7fa88083, failing the test timeout.
The Linux san corpus now skips that one case
(SCRIPTC_THREE_SANITIZED_NODES_JOB), and a three_nodes_san job runs it alone
with the corpus's LeakSanitizer settings. The required test check waits for
the new job. Every other three.js case, the plain nodes build and the macOS
suites are unchanged.
The release workflow's new cli-profile job runs
scripts/refresh-cli-profile.mjs on Linux x64 and uploads the profile.
Every platform's native CLI build uses it through
SCRIPTC_NATIVE_PROFILE_USE, and the GitHub release attaches it as
scriptc-<version>-cli.profdata.gz. When the job fails, the CLI builds
use the committed profile.
build-native-cli.mts accepts a gzip-compressed profile in
SCRIPTC_NATIVE_PROFILE_USE when the name ends in .gz.
refresh-cli-profile.mjs creates the directory of --output before it
starts.
The sanitized self-hosting native-frontend build needs about 15 GB by
itself: its clang compile of the whole frontend under ASan peaks at
10.5 GB beside a 3.8 GB compile in Node. In the 14 GB scope, with the
shard's other work running beside it, the kernel killed that clang
intermittently, so SC5004 reported clang's warnings and no error (SAN
16/20 and 19/20). The scope now caps RAM at 15 GB, still below the
runner's 16 GB, and may still swap past it.
The Linux test shards run their suites in a transient systemd scope with
MemoryMax=13G and no swap, below the runner's 16 GB. A process tree that
grows past the cap loses its largest process to the kernel, and the job
log names the failing test, where the runner itself ran out of memory
and left no log. Hosts without systemd run the suites directly.
The side-queue balancer also takes a table of memory-heavy files, with
their measured peak. They run whole in the cost-balanced task, and each
goes to the shard with the least memory-heavy work so far, after the
time-balanced files. decl-boundary.test.ts, which peaks at about 1.8 GB
alone and had no time measurement, so vitest's file sharding put it in
shard 11, is the first entry. It now runs in shard 1 (plain) and 6 (san).
tsc-ts 0.0.8's API server serves on the named pipe the Node host
connects over on Windows (tsc-ts --api --pipe), and the native compiler
already starts it over its standard streams there, so tsc-ts is the
default checker on every platform. SCRIPTC_CHECKER=tsgo still selects
tsgo.
The compiler depends on tsc-ts 0.0.8, which the release-age exclusions
admit with its platform packages and @scriptc/threads 0.4.2.
The Windows CLI smoke test runs with both checkers, and
verify-native-cli.mjs --run checks that a distribution's tsc-ts runs on
its platform (tsc-ts.exe on Windows).
Its release pass alone takes about 15 minutes, so the dev pass is regularly
cancelled at the 30-minute limit on main and on PRs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sanitized seed compiles the compiler's 700 MB LLVM module with one clang
process under AddressSanitizer. The compile peaks near 14 GB while the Node
process that emitted the module still holds about 6 GB, which exceeds the
runner's 16 GB of memory. The job now enlarges swap to 24 GB before it builds
the seed, as the sanitized emitter job does.
Stack the libnode packs on the in-process loader (#1685) and use its
pack assembler and manifest format instead of a second one:
build-pack.mjs keeps its CLI and fields (schema, nodeVersion,
moduleVersion, bridgeAbi, platform, arch, files), adds the libnode and
bridge names, strips and (on macOS) re-signs the bridge and links it with
-z defs, and exports buildPack(), which the platform packs call with their
package, target and build inputs as extra manifest fields.
libnodeRecord falls back to the installed @scriptc/libnode-<platform>
pack, found from the entry's directory and then next to the compiler,
when neither --libnode nor SCRIPTC_LIBNODE names one.
The Linux test lanes restore the libnode build that the libnode-packs
workflow cached, assemble the linux-x64 pack from it in seconds
(build.mjs --from-cache never builds Node) and export SCRIPTC_LIBNODE, so
the Node island corpus and harness tests run instead of skipping.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the @scriptc/libnode-<platform> packs (linux-x64-gnu, linux-arm64-gnu,
darwin-arm64, darwin-x64): the pinned Node.js 24.22.0 built as a stripped
shared library with full ICU and its startup snapshot, the scriptc bridge
compiled against that release's headers, Node's LICENSE and a manifest
with the Node version, NODE_MODULE_VERSION, bridge ABI and file hashes.
packages/libnode-common holds the pin, the build (fetch and verify the
source tarball, configure --shared, build only the libnode target, strip,
stage with headers in a cache keyed by release, flags, platform and
toolchain; the bridge rebuilds in seconds against the cached headers),
a verifier, and a smoke test: a C program that loads a pack the way the
runtime does, starts Node from the snapshot and evaluates expressions
synchronously, asynchronously and throwing.
The libnode-packs workflow builds the packs on native runners, Linux in
the manylinux_2_28 images for a glibc 2.28 floor, caches each libnode
build, smoke-tests and uploads every pack. It never runs in the ordinary
build or test path. The compiler gains the pack lookup (SCRIPTC_LIBNODE,
then the installed package), validation, the nodeIslands.libnode
placements and the license notices for the Node island loader to use.
The bridge source and its C ABI header are the NI-1 bridge agent's,
included here so the packs build; the packs stay private until the
in-process island runtime ships.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Malte Ubl