`scripts/refresh-cli-profile.mjs --context-sensitive` runs the training twice. The second run uses a compiler that the first profile optimizes and that is instrumented again after inlining, and it merges those counts into the first profile. Builds with the result also order each function's blocks by the counts and split cold blocks off (context-sensitive profiles). `--base-profile` runs only the second phase. `scripts/build-native-cli.mts` builds such a compiler when `SCRIPTC_NATIVE_PROFILE_GENERATE` and `SCRIPTC_NATIVE_PROFILE_USE` are both set. The release workflow's cli-profile job collects the profile this way, with a 120-minute timeout.
Each @scriptc/cli-<platform> package bundled its platform's LLVM helper
and runtime pack, which are also published as @scriptc/llvm-<platform> and
@scriptc/runtime-<platform>. The CLI packages now depend on those packages
at the same version (workspace:*, which pnpm pack writes as the exact
version) and leave them out: cli-linux-x64-gnu goes from 115 MB to 60 MB
packed.
- build-native-cli.mts --platform-packages, which each CLI package's
build:native passes, writes a manifest without llvm_package, runtime_pack
and runtime_packs.
- loadNativeToolchain resolves a missing helper or host runtime pack as
Node resolves the package from the compiler's directory: nested in the
platform package, beside it (hoisted, or pnpm's virtual store) or higher
up. It requires the compiler's version, and an absent or mismatched
package fails with an error that names it.
- installNativeCli records where the package manager put both packages in
the installed manifest, with the same checks.
- verify-native-cli.mjs requires the exact dependencies of a platform
package and checks the resolved packages' versions.
- package-native-cli.mjs bundles the helper and runtime pack into the
standalone archives, which stay self-contained.
- smoke-native-install.mjs installs the helper and runtime pack tarballs
with the platform package; the bootstrap's exported package drops its
bundled copies first.
- release.yml publishes the CLI packages only once every helper and
runtime pack package resolves on npm, and scriptc only once the CLI
packages resolve, each wait at most 25 minutes. A rerun skips the
uploads npm already accepted and waits again.
The Linux helpers linked LLVM 22.1.8's release archive. The helper's code
generation is LLVM's code, so building that LLVM from its release sources
with ThinLTO and with a profile of the helper's own work makes code
generation 5-9% faster than the helper as released, with identical objects.
- native/llvm-codegen/llvm.profdata.gz is the profile, collected by
scripts/refresh-llvm-profile.mjs: it builds LLVM with IR instrumentation
and an instrumented helper, and runs that helper on tsc-ts, five Effect
fixtures, the runtime benchmark workloads and every 50th corpus program
(scripts/llvm-helper-inputs.mjs). It follows LLVM's sources, so it needs
a refresh when LLVM is bumped.
- scripts/build-llvm-dist.mjs builds LLVM with that profile and ThinLTO,
and packs it with a checksum. The llvm-dist workflow, run by hand, builds
it for x64 and arm64, requires a helper linked against it to give the
same objects as one linked against the release archive
(scripts/check-llvm-dist.mjs), prints both helpers' times and the
profile's mismatch count, and attaches the archive to the LLVM_DIST_TAG
release of this repository.
- The release workflow's Linux jobs, and the wasm32-wasi job, which also
uploads the x64 GNU helper, download that archive and build the helper
with SCRIPTC_LLVM_THINLTO=1: clang, lld and ThinLTO, and the profile for
the helper's own sources. Without the archive, or if that build fails,
the helper links the release archive as before.
- The four Linux helper packages share their build in
native/llvm-codegen/linux-helper.mjs. Without SCRIPTC_LLVM_THINLTO it
builds the same executable as before.
The release workflow publishes each package and moves on once npm accepts
the upload. A package whose processing on npm stalls was visible only on
npmjs.com.
A new npm-available job runs after "Publish to npm". It checks each package
of the release with `npm view <name>@<version> version --prefer-online`
every 2 minutes (scripts/wait-npm-available.mjs) until every package
resolves, and reports each package's time from upload to available in the
log and the job summary. When a package still does not resolve after 2
hours, the job fails and names each missing package and version. It only
reads from the registry.
The publish job lists every package of the release and records the time
npm accepted each upload, keeps those times with its upload receipts, and
passes both to the new job. Create GitHub Release now waits for the new
job, so the release is created and marked complete only once every
package resolves; a later release run checks again and skips uploads npm
already accepted.
A warm rebuild ran about 3.5 MB of the native compiler's code, but the code
was spread over its 69 MB text, so 64 KB fault-around mapped about 20 MB of
it. The emitter can now take a function layout: a list of functions in
tiers. It writes the listed definitions, generated helpers included, first
and by tier, each in a `layout-<tier>` partition group. The LLVM helper
compiles each partition's definitions in text order, so the listed ones
come first in their partitions, and a helper that places functions by
group keeps a tier's definitions in the same partitions. The rest keep
their order and their source-file groups.
- compile() takes a `functionLayout` file (`<tier> <symbol>` lines,
gzip-compressed when the name ends in .gz). Keys leave out module
numbers.
- build-native-cli.mts uses the committed packages/compiler/native/cli.layout.gz,
or the file SCRIPTC_NATIVE_LAYOUT names; an empty value and instrumented
builds use none.
- scripts/refresh-cli-layout.mjs builds an instrumented compiler, builds
the runtime workloads and a corpus slice cold and then unchanged, and
reads the functions each run executed from the raw profiles. Tier 1 is
`--version` and rebuilds that hit both caches; tier 2 is the other builds.
- The release workflow's cli-profile job also collects a layout on the
release commit, and the CLI builds use it. Names of closures and
generated helpers change with nearby edits, so a committed layout
matches fewer functions as main moves on.
- Forked lowering and emission workers reset the profile counters after
fork and write their profile before _exit, so instrumented builds also
count the work done in workers. This applies to refresh-cli-profile.mjs
too.
Executables the compiler produces are unchanged.
The release workflow's new cli-profile job runs
scripts/refresh-cli-profile.mjs on Linux x64 and uploads the profile.
Every platform's native CLI build uses it through
SCRIPTC_NATIVE_PROFILE_USE, and the GitHub release attaches it as
scriptc-<version>-cli.profdata.gz. When the job fails, the CLI builds
use the committed profile.
build-native-cli.mts accepts a gzip-compressed profile in
SCRIPTC_NATIVE_PROFILE_USE when the name ends in .gz.
refresh-cli-profile.mjs creates the directory of --output before it
starts.
Scope-B milestone NI-1, API side: the explicit call API that replaces the
build container's hand-written node subprocess (api patch 09).
- The new package @scriptc/node exports callSync(specifier, name, ...args)
and call(...). The module loads with require() from the calling module;
arguments and results cross as JSON copies; a thrown Error crosses as its
nearest intrinsic class with its message and string code, other thrown
values as JSON copies; callSync refuses a function that returns a
promise. Its Node implementation runs the calls in-process with the same
copies, so a program prints the same under Node and compiled.
- The compiler lowers calls to the named or namespace imports
(node-api-import.ts, lower-node-island.ts) to the nodeIsland.callSync
and nodeIsland.call libCalls: the calling module's real path, the
specifier, the export name and the boxed argument array, with the result
checked against the declared type like `JSON.parse(s) as T`. Arguments
and results whose types are not JSON data are refused at build time, and
the functions and the namespace refuse as values.
- Corpus programs 6800-6803 (sync and async results, errors, process.exit
from Node with the program's exit listeners, Node's pending timers
keeping the program alive) run under Node and as binaries that load
libnode; `// @requires-libnode` skips them where SCRIPTC_LIBNODE names no
pack (until the pack CI matrix publishes packs). The node-island harness
covers loading Node only on the first call, the beside packaging, a
missing pack, a pack of another Node version or bridge ABI, and the
executable's symbols.
- Release tooling publishes @scriptc/node with the other packages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The darwin-arm64, linux-x64-musl and linux-arm64-musl CLI builds failed with
"tsc-ts checker: ... has no @tsc-ts/<platform> executable":
- The lockfile had no entry for @tsc-ts/darwin-arm64@0.0.5, so the
darwin-arm64 install skipped it. The lockfile now lists all seven
platform packages.
- A CLI ships the tsc-ts executable of its target. The musl builds run on
glibc runners, where pnpm installs only the glibc packages. Those jobs now
add musl to pnpm's supported libc before installing.
sync-versions stamps packages/threads with the CLI version (0.2.7 now),
the release workflow checks that version and publishes the package
before scriptc, and RELEASING.md lists it with the packages that always
release together. npm trusted publishing must be configured for
@scriptc/threads before the next release: a package without it fails
before upload.
* Make native installs portable and smaller
- Support glibc 2.34 with portable compiler binaries and static LLVM helpers.
- Bundle the host runtime and discover optional cross-target packs from project dependencies.
- Add application build benchmarks covering latency, executable size, and memory.
* Keep bootstrap tools available in Sandbox validation
- Isolate Zig for native compiler bootstrap and glibc entropy contracts.
- Preserve the existing toolchain scope for cache and corpus tests.
* Fix portable compiler bootstrap linking
- Keep native support objects compatible with the destination host linker.
- Resolve installed targets using the host declared by the toolchain manifest.
- Resolve Windows toolchain paths during package verification and archive creation.
- Preserve completed native builds and accepted uploads across release tooling fixes.
- Preserve accepted uploads and continue past pending npm publications.
- Reuse completed native builds when only the release workflow changes.
- Require public packages and complete assets before finishing a release.
- Synchronize published compiler, runtime, and native platform packages at 0.2.0.
- Document the native CLI, LLVM migration, and compatibility and performance improvements.
- Keep release version checks scoped to published packages.
* feat: ship the self-hosted native scriptc CLI
- Run the shared compiler and CLI natively, including library builds and compile-time evaluation.
- Install native platform packages and ship relocatable standalone distributions.
- Preserve program semantics and reuse validated build artifacts for fast rebuilds.
* fix: complete native CLI integration and validation
- Preserve library diagnostics and run recursive compiler work on the main stack.
- Restore packaged helper permissions and exercise native npm and Wasm library builds.
- Recover interrupted Sandbox status probes within the original command deadline.
- Remove C emission, debugging options, and automatic backend fallback.
- Complete LLVM lowering and native bootstrap coverage across supported targets.
- Link executables and libraries with precompiled C runtime packs, including iOS and Android.
- Pin GNU runtime packs to glibc 2.36 Zig targets and reject incompatible imports.
- Provision verified Zig builds and exercise published artifacts on Debian Bookworm.
Fixes#349
Co-authored-by: Cruel <383999+Cruel@users.noreply.github.com>
* Expand LLVM native output targets
- Add owned helper and runtime-pack contracts for macOS x64, Linux glibc/musl, Windows x64, and WASI.
- Generalize LLVM target selection, package validation, linker plans, and release/CI matrices.
- Document linker boundaries and add focused native-output and runtime-pack coverage.
* Pin Windows LLVM helper CI to VS 2022
The official LLVM archive needs the Visual Studio 2022 CMake generator, which is not guaranteed by windows-latest.
* Align bootstrap cache with LLVM runtime-pack builds
Use the same target-specific helper and linker identity before and after compiler loading so routed executable cache hits remain lightweight.
* Install Windows LLVM CI tools with Chocolatey
windows-2022 includes Visual Studio but not winget; use its available Chocolatey bootstrap for CMake and Ninja.
* Prevent Zig version probes from leaving runtime-pack objects
Run runtime-pack compiler version probes in a private temporary directory and remove the accidental tracked WASI a.o file.
* Use 7-Zip for Windows LLVM setup
Extract the official LLVM development archive with two-stage 7-Zip instead of Windows tar.exe, which timed out while materializing the toolchain tree.
* Define ssize_t for the MSVC runtime pack
Clang's MSVC target does not expose POSIX ssize_t through sys/types.h; define the pointer-sized runtime type without affecting MinGW.
* Build the Windows runtime pack with Zig
Use the MinGW-compatible Windows sysroot required by the runtime while retaining COFF helper output and installing Zig in the Windows native CI lane.
* Give macOS LLVM differential shard time to finish
Shard 1 passed its setup and focused contracts but was cancelled during its cold LLVM differential slice at the 20-minute job limit.
- packages/compiler/surface-manifest.json is generated (pnpm manifest), committed, and shipped in @scriptc/compiler; entries project mechanically from the diagnostics registry, the unsupported-syntax dispatch tables, the stdlib/builtin lowering tables, and the supported-builtin-module list
- every non-static entry carries the SC code the compiler raises for it, and the version spine is the exact published release version
- FENCE_CODES joins the diagnostics registry so factory-minted construct fences are enumerable
- the sampling harness compiles listed-static probes and asserts each sampled non-static entry refuses with exactly its listed code, beside a byte-identical staleness guard
- the release workflow regenerates the manifest, fails on drift, and attaches it to the GitHub release
- RELEASING.md documents the lockstep three-package prepare flow and the marked-changelog convention
- CHANGELOG.md starts with an Unreleased section and the marked 0.0.1 entry that becomes the GitHub release body
- AGENTS.md states repo-wide build/test conventions and defers docs-site specifics to docs/AGENTS.md
- scripts/sync-versions.mjs stamps runtime and compiler from the CLI version; the workflow's sync check now hints at it
- release.yml gains a github-release job that tags v<version> after a successful publish, never gating npm
Chris Tate