Chris Tate
3ea231d70d
Large executables were simplified and inlined as one module before the optimizer split them, so most of a release build ran on a single core, and any edit recompiled the whole program. The LLVM helper now divides the program before optimization. It splits the emitter's assembly text directly, so partitions also parse concurrently, and falls back to LLVM's module splitter for any input outside the emitter's grammar. Internal functions referenced by a single function stay with that function, and globals stay with their first user. Optimized partitions follow the ThinLTO model: each is simplified and summarized on its own thread, a thin link imports small functions across partitions for inlining and returns unexported hidden definitions to internal linkage, then each partition is optimized and compiled on its own thread. Unoptimized partitions compile independently, so dev builds of large programs are parallel too. A private per-helper cache directory stores simplified partitions under the hash of their source and objects under the hash of the module after importing, so a rebuild recompiles only the partitions an edit can affect. To keep unchanged partitions identical across edits, each partition numbers its metadata in first-use order, lists only the debug compile units it uses, and orders declarations by name, and interned string constants are named by their content rather than first-use order. Partition counts are powers of two that depend only on the module size, up to 64, so output stays the same on every host and functions keep their partition across ordinary edits. LLVM text that arrives in parts is written in large batches and handed to the helper in place instead of being written a second time.
scriptc LLVM code-generation helper
This out-of-process helper owns LLVM assembly and object emission for
scriptc. It is built against exactly LLVM 22.1.8 and currently contains only
the AArch64 backend. The shipping @scriptc/llvm-darwin-arm64 package builds
and carries the executable; the compiler resolves that package directly and
never searches PATH for this program.
The ordinary workspace pnpm -r build does not rebuild this release artifact.
On macOS arm64, install CMake, Ninja, and Homebrew llvm@22, then build it
explicitly when working on native emission or preparing a package:
$ brew install cmake ninja llvm@22
$ pnpm --filter @scriptc/llvm-darwin-arm64 build:native
The protocol is intentionally small and versioned:
The packaged helper itself requires macOS 15 or newer because that is the minimum version of the pinned LLVM bottle it statically links. Its emitted assembly and objects separately target macOS 14 via the triple below.
scriptc-llvm-codegen version --format=json
scriptc-llvm-codegen emit --input app.ll --output app.o --filetype obj \
--target arm64-apple-macosx14.0.0 --opt-level 2 \
--relocation-model pic --diagnostic-format json --source-path app.ts
Emission uses LLVM 22's default per-module O2 pipeline, including coroutine lowering, verifies before and after optimization, and publishes through a private sibling file so a failed or interrupted request cannot truncate the requested output.
Repeating --output divides the program into that many partitions, which are compiled concurrently and linked together. The helper splits the emitter's LLVM assembly text directly, so partitions also parse concurrently; input outside the emitter's narrow grammar is parsed whole and split with LLVM instead. Internal definitions become hidden so partitions can reference each other. At opt-level 0 each partition is compiled independently. Optimized partitions follow the ThinLTO model: each is simplified and summarized on its own thread, a thin link decides which small functions every partition imports for inlining and which hidden definitions become internal again, and each partition is then optimized and compiled on its own thread. Partition placement depends only on the module and the partition count, so output never depends on the host.
--cache-dir <dir> names a private directory the caller owns for one helper build. Simplified partitions are stored under the hash of their exact source and objects under the hash of the exact module after importing, each with a trailing digest and published by rename, so a rebuild recompiles only the partitions an edit can affect.
--optimization=speed program builds pass --import-bitcode <unit.bc> once per runtime-pack unit the program links. Small runtime functions reachable from the program's runtime calls are imported as available_externally so LLVM can inline them; the runtime objects remain the only definitions. The helper never runs a sanitizer pass, so with import it also drops the emitter's inert sanitize_address attributes, which would otherwise block that inlining. Without --import-bitcode the module is optimized exactly as emitted.
Runtime-pack builds emit each unit of the executable speed flavor through the helper:
scriptc-llvm-codegen runtime-unit --input unit.opt.bc --object unit.o \
--bitcode unit.bc --target x86_64-unknown-linux-gnu --tag scr_array \
[--function-sections --data-sections]
The input is the runtime compiler's optimized bitcode. The command promotes unit-local symbols to hidden <name>.scrunit.<tag> globals and writes the object and the import bitcode from that one module, so imported bodies can reference the unit's private state and helpers.