trending/scriptc
1
0
Fork 0
mirror of https://github.com/vercel-labs/scriptc.git synced 2026-11-08 08:44:37 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge origin/main into harden/runtime-cflags

Browse Source 
Harden the vendored brotli archive like zlib and mbedtls: it also parses
untrusted compressed input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Malte Ubl
2026-10-27 05:36:11 -07:00
co-authored by Claude Opus 5.5
parent 2b43e6b5c6 f73722c2ef
commit e9495b9167
8320 changed files with 703980 additions and 150361 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yml
+210 -50
View File
@@ -4,21 +4,23 @@ on:
push:
branches: [main]
pull_request:
# Nightly full macOS sweep, including on days without a push to main.
# Nightly full macOS sweep. Pushes to main run no macOS jobs, and pull requests run them only when they change
# darwin code (scripts/ci-lanes.mjs).
schedule:
- cron: "17 7 * * *"
# A manual run; macos_full runs the complete suite on macOS as the nightly sweep does (for example on a
# release candidate commit).
# A manual run, for example on a release candidate commit. It runs every lane; macos_suite selects the complete
# suite on macOS, as the nightly sweep runs it, or the darwin suite that pull requests run.
workflow_dispatch:
inputs:
macos_full:
description: Run the complete test suite on macOS
type: boolean
default: false
macos_suite:
description: The test suite to run on macOS
type: choice
options: [full, darwin]
default: full
concurrency:
# A new push to a pull request cancels its previous run. Every push to main runs to completion in a group of
# its own commit, so each merged commit gets a full result (releases require a green run on their commit). The
# its own commit, so each merged commit gets a result (releases require a green run on their commit). The
# nightly sweep and manual runs neither cancel nor are cancelled by main pushes.
group: ci-${{ github.event_name == 'schedule' && 'schedule' || github.event_name == 'workflow_dispatch' && format('dispatch-{0}', github.run_id) || github.event.pull_request.number || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
@@ -59,7 +61,8 @@ jobs:
NODE
# Which expensive lanes a pull request needs (scripts/ci-lanes.mjs, run from the base commit against the pull
# request's diff, so a pull request cannot change its own selection). Pushes and the nightly sweep run every lane.
# request's diff, so a pull request cannot change its own selection). Pushes to main run every lane except macos,
# whose jobs the nightly sweep runs on main. The nightly sweep and manual runs run every lane.
lanes:
needs: validation_scope
if: ${{ needs.validation_scope.outputs.full == 'true' }}
@@ -86,7 +89,9 @@ jobs:
HEAD: ${{ github.event.pull_request.head.sha }}
run: |
every() { printf 'linux=true\nmacos=true\neffects=true\nbootstrap=true\n' | tee -a "$GITHUB_OUTPUT"; }
if [ "$EVENT" != pull_request ]; then
if [ "$EVENT" = push ]; then
printf 'linux=true\nmacos=false\neffects=true\nbootstrap=true\n' | tee -a "$GITHUB_OUTPUT"
elif [ "$EVENT" != pull_request ]; then
every
elif git show "$BASE:scripts/ci-lanes.mjs" > "$RUNNER_TEMP/ci-lanes.mjs" 2> /dev/null; then
node "$RUNNER_TEMP/ci-lanes.mjs" --base "$BASE" --head "$HEAD" --github-output
@@ -117,6 +122,8 @@ jobs:
SCRIPTC_TEST_WORKERS: "1"
- name: Verify mixed-source generated AST getters
run: pnpm generate:ts7-ast --check
- name: Verify the generated IR field readers
run: pnpm generate:ir-fields --check
native_macos:
needs: [validation_scope, lanes]
@@ -172,7 +179,9 @@ jobs:
SCRIPTC_SAN: ${{ matrix.flavor == 'san' && '1' || '' }}
# Match macOS, where Apple ASan has no LeakSanitizer: the RC audit owns
# leak detection, including intentional abandonment. Plain shards also
# build some sanitized probes, so this applies to both flavors.
# build some sanitized probes, so this applies to both flavors. The
# san shards' case-sharded corpus turns LeakSanitizer on
# (scripts/ci-test-plan.mjs, tests/harness/lsan.supp).
ASAN_OPTIONS: detect_leaks=0
# Within-file partition for the large case-driven harnesses
# (differential, llvm-differential, npm, server, Test262, coverage, and
@@ -209,33 +218,48 @@ jobs:
# Start the corpus immediately beside a serial side queue. One shared
# worker budget covers both processes; file and case shard axes stay
# separate so every fixture still runs exactly once in each lane.
# The tests run in a transient scope capped below the runner's 16 GB, so
# the runner itself keeps memory: past 15 GB of RAM the scope swaps, as
# the whole runner did before, and when RAM and swap run out the kernel
# kills the scope's largest process. A sanitized shard peaks near 16 GB,
# and the sanitized self-hosting native-frontend build alone needs 15 GB
# (its clang compile of the frontend takes 10.5 GB).
# OOMPolicy=continue leaves the rest of the scope running, so vitest
# reports the killed worker and the job log names the test. Node sizes
# its default heap from the memory it may use, which the scope lowers
# below 4 GB; the workers keep the 4 GB heap they had on the bare
# runner.
- name: Tests (corpus and integration suites)
run: |
ulimit -c 0
if [ -d /run/systemd/system ]; then
exec sudo --preserve-env systemd-run --scope --quiet -p MemoryMax=15G -p OOMPolicy=continue \
--uid="$(id -u)" --gid="$(id -g)" -- env PATH="$PATH" HOME="$HOME" NODE_OPTIONS="--max-old-space-size=4096" node scripts/ci-test.mjs
fi
node scripts/ci-test.mjs
tests_macos:
needs: [validation_scope, lanes, native_macos]
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.macos == 'true' }}
name: test (macOS, ${{ matrix.flavor }}, ${{ (github.event_name != 'schedule' && !inputs.macos_full) && 'darwin ' || '' }}${{ matrix.shard }}/${{ (github.event_name != 'schedule' && !inputs.macos_full) && '3' || '10' }})
name: test (macOS, ${{ matrix.flavor }}, ${{ (github.event_name != 'schedule' && inputs.macos_suite != 'full') && 'darwin ' || '' }}${{ matrix.shard }}/${{ (github.event_name != 'schedule' && inputs.macos_suite != 'full') && '3' || '10' }})
# arm64 macOS: same clang/libc the differential + number-formatting
# oracles were verified against. Pull requests and pushes to main run the
# darwin suite (scripts/ci-test-plan.mjs) in three slices; the nightly
# schedule and manual runs with macos_full run the complete suite in ten.
# The complete suite on every push needed more macOS runners than the
# repository has.
# oracles were verified against. Pull requests that change darwin code and
# manual runs with macos_suite set to darwin run the darwin suite
# (scripts/ci-test-plan.mjs) in three slices; the nightly schedule and
# other manual runs run the complete suite in ten. The complete suite on
# every push needed more macOS runners than the repository has.
runs-on: macos-15
strategy:
fail-fast: false
matrix:
flavor: [plain, san]
shard: ${{ fromJSON((github.event_name != 'schedule' && !inputs.macos_full) && '[1, 2, 3]' || '[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]') }}
shard: ${{ fromJSON((github.event_name != 'schedule' && inputs.macos_suite != 'full') && '[1, 2, 3]' || '[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]') }}
env:
# See the Linux tests job for the worker cap and case partitioning.
SCRIPTC_TEST_WORKERS: "4"
SCRIPTC_SAN: ${{ matrix.flavor == 'san' && '1' || '' }}
SCRIPTC_TEST_SHARD: ${{ matrix.shard }}/${{ (github.event_name != 'schedule' && !inputs.macos_full) && '3' || '10' }}
SCRIPTC_CI_SUITE: ${{ (github.event_name != 'schedule' && !inputs.macos_full) && 'darwin' || 'full' }}
SCRIPTC_TEST_SHARD: ${{ matrix.shard }}/${{ (github.event_name != 'schedule' && inputs.macos_suite != 'full') && '3' || '10' }}
SCRIPTC_CI_SUITE: ${{ (github.event_name != 'schedule' && inputs.macos_suite != 'full') && 'darwin' || 'full' }}
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
@@ -252,7 +276,7 @@ jobs:
path: ${{ runner.temp }}
- run: tar -xzf "$RUNNER_TEMP/native-darwin-arm64.tar.gz"
- run: pnpm build
- name: Tests (${{ (github.event_name != 'schedule' && !inputs.macos_full) && 'darwin suite' || 'corpus and integration suites' }})
- name: Tests (${{ (github.event_name != 'schedule' && inputs.macos_suite != 'full') && 'darwin suite' || 'corpus and integration suites' }})
run: node scripts/ci-test.mjs
native_linux:
@@ -324,6 +348,37 @@ jobs:
- run: tar -xzf "$RUNNER_TEMP/native-linux-x64-gnu.tar.gz"
- run: pnpm test tests/harness/effect.test.ts tests/harness/effect4.test.ts
# The sanitized three.js nodes build alone: beside the san corpus's other
# workers it took 860 to more than 1200 seconds of its shard
# (SCRIPTC_THREE_SANITIZED_NODES_JOB in scripts/ci-test-plan.mjs). It keeps
# the corpus's LeakSanitizer settings.
three_nodes_san:
needs: [validation_scope, lanes, native_linux]
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.linux == 'true' }}
name: test (Linux, san, three.js nodes)
runs-on: ubuntu-24.04
timeout-minutes: 40
env:
SCRIPTC_SAN: "1"
ASAN_OPTIONS: detect_leaks=1
LSAN_OPTIONS: suppressions=${{ github.workspace }}/tests/harness/lsan.supp:print_suppressions=0
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 11.1.3
- uses: actions/setup-node@v4
with:
node-version-file: .node-version
cache: pnpm
- run: pnpm install --frozen-lockfile
- uses: actions/download-artifact@v4
with:
name: native-linux-x64-gnu
path: ${{ runner.temp }}
- run: tar -xzf "$RUNNER_TEMP/native-linux-x64-gnu.tar.gz"
- run: pnpm test tests/harness/three.test.ts -t "'nodes' runs statically through LLVM on 'native'"
# Build each instrumented seed once. Downstream contracts retain the same
# compiler and source artifacts without repeating seed construction.
bootstrap_seed:
@@ -357,6 +412,20 @@ jobs:
target: wasm32-wasi
helper: "false"
- run: pnpm build
# The sanitized seed compiles the compiler's 700 MB module with one clang
# process under AddressSanitizer. It peaks near 14 GB while the Node
# process that emitted the module still holds about 6 GB. Together they
# exceed the runner's 16 GB of memory, so back them with swap.
- name: Enlarge swap for the sanitized seed
if: matrix.flavor == 'san'
run: |
sudo swapoff -a
sudo rm -f /mnt/swapfile
sudo fallocate -l 24G /mnt/swapfile
sudo chmod 600 /mnt/swapfile
sudo mkswap /mnt/swapfile
sudo swapon /mnt/swapfile
free -h
- name: Prepare bootstrap seed
run: node scripts/bootstrap-seed.mjs
env:
@@ -496,6 +565,32 @@ jobs:
- name: TypeScript 7 full parity sweep
run: pnpm test:ts7
# tsc-ts, the default checker, and tsgo (SCRIPTC_CHECKER=tsgo) serve the same TypeScript 7 API. Every corpus
# program must compile to the same LLVM IR, with the same diagnostics, with either checker.
checker_identity:
needs: [validation_scope, lanes]
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.linux == 'true' }}
name: test (tsgo and tsc-ts give the same IR)
runs-on: ubuntu-24.04
# The two corpus compiles run back to back and take close to 30 minutes on a
# slow hosted runner, so a 30-minute limit cancels otherwise healthy runs.
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 11.1.3
- uses: actions/setup-node@v4
with:
node-version-file: .node-version
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm build
- name: Corpus LLVM IR with tsgo and with tsc-ts (release)
run: pnpm test:checker-identity --optimization release
- name: Corpus LLVM IR with tsgo and with tsc-ts (dev)
run: pnpm test:checker-identity --optimization dev
llvm_artifacts_linux_x64:
needs: [validation_scope, lanes]
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.linux == 'true' }}
@@ -570,11 +665,10 @@ jobs:
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.macos == 'true' }}
name: test (macOS arm64 LLVM artifacts, no clang, ${{ matrix.shard }}/5)
runs-on: macos-15
# Shard 1 additionally runs package/install and link-parity contracts
# before its LLVM differential slice. The slice itself can take ~10
# minutes on a cold hosted runner, so 20 minutes cancels an otherwise
# healthy job before the differential completes.
timeout-minutes: 35
# Each LLVM differential slice takes 30 to 35 minutes on a hosted runner.
# The package/install and link-parity contracts run in their own job
# (llvm_artifacts_macos_arm64_contracts), so no slice carries them.
timeout-minutes: 50
strategy:
fail-fast: false
matrix:
@@ -584,6 +678,36 @@ jobs:
# keeping each macOS job below its timeout while the matrix union still
# exercises every LLVM-tier program.
SCRIPTC_TEST_SHARD: ${{ matrix.shard }}/5
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 11.1.3
- uses: actions/setup-node@v4
with:
node-version-file: .node-version
cache: pnpm
- run: pnpm install --frozen-lockfile
- uses: actions/download-artifact@v4
with:
name: native-darwin-arm64
path: ${{ runner.temp }}
- run: tar -xzf "$RUNNER_TEMP/native-darwin-arm64.tar.gz"
- run: pnpm build
- name: LLVM release/dev differential (${{ matrix.shard }}/5)
env:
SCRIPTC_TEST_WORKERS: "4"
run: pnpm test tests/harness/llvm-differential.test.ts
llvm_artifacts_macos_arm64_contracts:
needs: [validation_scope, lanes, native_macos]
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.macos == 'true' }}
name: test (macOS arm64 LLVM artifacts, no clang, contracts)
runs-on: macos-15
# The no-clang object, helper parity, packaged helper and packed npm
# installation contracts: about 12 minutes, which used to run in front
# of shard 1's differential slice and took that job to 45-50 minutes.
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
@@ -601,10 +725,8 @@ jobs:
- run: tar -xzf "$RUNNER_TEMP/native-darwin-arm64.tar.gz"
- run: pnpm build
- name: No-clang assembly/object contract
if: matrix.shard == 1
run: pnpm test packages/cli/test/native-output.test.ts
- name: Helper diagnostics and object/link parity
if: matrix.shard == 1
run: >-
pnpm test
packages/compiler/test/native-codegen-integration.test.ts
@@ -612,17 +734,11 @@ jobs:
packages/cli/test/native-link-info.test.ts
packages/cli/test/runtime-pack.test.ts
tests/harness/native-object-example.test.ts
- name: LLVM release/dev differential (${{ matrix.shard }}/5)
env:
SCRIPTC_TEST_WORKERS: "4"
run: pnpm test tests/harness/llvm-differential.test.ts
- name: Packaged helper contract
if: matrix.shard == 1
run: |
TARBALL=$(pnpm --dir packages/llvm-darwin-arm64 pack --pack-destination "$RUNNER_TEMP" --silent)
node scripts/verify-llvm-package.mjs "$RUNNER_TEMP/$(basename "$TARBALL")"
- name: Packed npm installation smoke
if: matrix.shard == 1
env:
# Release optimization is covered by the compiler bootstrap jobs.
# This lane verifies packaging and installation with a faster seed.
@@ -728,17 +844,22 @@ jobs:
run: >-
pnpm exec vitest run tests/harness/worktree-files.test.ts
--testNamePattern "workspace reset keeps Linux Sandbox paths POSIX"
# The Node host checks with tsc-ts, the default, over the named pipe it
# serves on Windows, and with tsgo.
- name: Windows scriptc run smoke
shell: pwsh
run: |
zig version
$output = & node packages/cli/dist/main.js run tests/corpus/001-hello.ts --backend llvm
if ($LASTEXITCODE -ne 0) {
exit $LASTEXITCODE
}
$actual = ($output -join "`n").Trim()
if ($actual -ne "hello world") {
throw "unexpected scriptc output: $actual"
foreach ($checker in @("tsc-ts", "tsgo")) {
$env:SCRIPTC_CHECKER = $checker
$output = & node packages/cli/dist/main.js run tests/corpus/001-hello.ts --backend llvm
if ($LASTEXITCODE -ne 0) {
exit $LASTEXITCODE
}
$actual = ($output -join "`n").Trim()
if ($actual -ne "hello world") {
throw "unexpected scriptc output with ${checker}: $actual"
}
}
- name: Windows PE subsystem selection
env:
@@ -842,14 +963,25 @@ jobs:
strategy:
fail-fast: false
matrix:
include:
- runtime: runtime-ios-arm64
runner: macos-15
- runtime: runtime-ios-simulator-arm64
runner: macos-15
- runtime: runtime-android-arm64
runner: ubuntu-24.04
runs-on: ${{ matrix.runner }}
runtime: [runtime-android-arm64]
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/mobile-runtime-pack
with:
runtime: ${{ matrix.runtime }}
# The iOS runtime packs build on macOS runners, so they run in the macos lane.
mobile_runtime_packs_ios:
needs: [validation_scope, lanes]
if: ${{ needs.validation_scope.outputs.full == 'true' && needs.lanes.outputs.macos == 'true' }}
name: test (${{ matrix.runtime }})
strategy:
fail-fast: false
matrix:
runtime: [runtime-ios-arm64, runtime-ios-simulator-arm64]
runs-on: macos-15
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
@@ -861,7 +993,31 @@ jobs:
# Require this check on the stack base. Upper layers must not pass before
# full validation, so untested groups cannot merge. Merge bottom-up.
test:
needs: [validation_scope, lanes, code_quality, native_macos, tests, tests_macos, native_linux, effects, bootstrap_seed, bootstrap, linux_host_clang, ts7_parity, llvm_artifacts_linux_x64, llvm_artifacts_macos_arm64, llvm_artifacts_macos_x64, windows_cli, llvm_artifacts_windows_x64, mobile_runtime_packs]
needs:
[
validation_scope,
lanes,
code_quality,
native_macos,
tests,
tests_macos,
native_linux,
effects,
three_nodes_san,
bootstrap_seed,
bootstrap,
linux_host_clang,
ts7_parity,
checker_identity,
llvm_artifacts_linux_x64,
llvm_artifacts_macos_arm64,
llvm_artifacts_macos_arm64_contracts,
llvm_artifacts_macos_x64,
windows_cli,
llvm_artifacts_windows_x64,
mobile_runtime_packs,
mobile_runtime_packs_ios,
]
# Run even after failed or cancelled lanes: skipped required jobs allow merging.
if: ${{ always() }}
runs-on: ubuntu-latest
@@ -881,16 +1037,20 @@ jobs:
lane tests_macos "${{ needs.tests_macos.result }}" "${{ needs.lanes.outputs.macos }}"
lane native_linux "${{ needs.native_linux.result }}" "${{ needs.lanes.outputs.linux }}"
lane effects "${{ needs.effects.result }}" "${{ needs.lanes.outputs.effects }}"
lane three_nodes_san "${{ needs.three_nodes_san.result }}" "${{ needs.lanes.outputs.linux }}"
lane bootstrap_seed "${{ needs.bootstrap_seed.result }}" "${{ needs.lanes.outputs.bootstrap }}"
lane bootstrap "${{ needs.bootstrap.result }}" "${{ needs.lanes.outputs.bootstrap }}"
lane linux_host_clang "${{ needs.linux_host_clang.result }}" "${{ needs.lanes.outputs.linux }}"
lane ts7_parity "${{ needs.ts7_parity.result }}" "${{ needs.lanes.outputs.linux }}"
lane checker_identity "${{ needs.checker_identity.result }}" "${{ needs.lanes.outputs.linux }}"
lane llvm_artifacts_linux_x64 "${{ needs.llvm_artifacts_linux_x64.result }}" "${{ needs.lanes.outputs.linux }}"
lane llvm_artifacts_macos_arm64 "${{ needs.llvm_artifacts_macos_arm64.result }}" "${{ needs.lanes.outputs.macos }}"
lane llvm_artifacts_macos_arm64_contracts "${{ needs.llvm_artifacts_macos_arm64_contracts.result }}" "${{ needs.lanes.outputs.macos }}"
lane llvm_artifacts_macos_x64 "${{ needs.llvm_artifacts_macos_x64.result }}" "${{ needs.lanes.outputs.macos }}"
lane windows_cli "${{ needs.windows_cli.result }}" "${{ needs.lanes.outputs.linux }}"
lane llvm_artifacts_windows_x64 "${{ needs.llvm_artifacts_windows_x64.result }}" "${{ needs.lanes.outputs.linux }}"
lane mobile_runtime_packs "${{ needs.mobile_runtime_packs.result }}" "${{ needs.lanes.outputs.linux }}"
lane mobile_runtime_packs_ios "${{ needs.mobile_runtime_packs_ios.result }}" "${{ needs.lanes.outputs.macos }}"
;;
false)
echo "::notice::Full CI is deferred until this PR becomes the lowest unmerged layer. Merge the stack from the bottom up."
.github/workflows/llvm-dist.yml
+93
View File
@@ -0,0 +1,93 @@
name: LLVM distribution
# Builds the LLVM that the Linux release helpers link
# (native/llvm-codegen/llvm-dist.mjs): LLVM from its release sources with
# ThinLTO and native/llvm-codegen/llvm.profdata.gz, for each Linux
# architecture. Each job builds a helper against it and one against LLVM's
# release archive, requires the two to give identical objects on this
# checkout's helper inputs (scripts/check-llvm-dist.mjs), prints both
# helpers' times, and attaches the archive to the LLVM_DIST_TAG release.
# The arm64 build uses the profile collected on x64; its job log reports how
# many functions the profile no longer matches. Run it after a profile refresh
# or an LLVM version change, with a new LLVM_DIST_TAG.
on:
workflow_dispatch:
permissions:
contents: read
concurrency: ${{ github.workflow }}
jobs:
build:
name: Build LLVM (${{ matrix.arch }})
if: github.repository == 'vercel-labs/scriptc'
strategy:
fail-fast: false
matrix:
include:
- arch: x64
runner: ubuntu-24.04
llvm_asset: LLVM-22.1.8-Linux-X64
- arch: arm64
runner: ubuntu-24.04-arm
llvm_asset: LLVM-22.1.8-Linux-ARM64
runs-on: ${{ matrix.runner }}
# About an hour for LLVM on 4 cores, and 5 minutes for each helper link.
timeout-minutes: 240
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 11.1.3
- uses: actions/setup-node@v4
with:
node-version: "24"
cache: pnpm
- name: Install LLVM's release archive and sources
run: |
sudo apt-get update
sudo apt-get install --yes ninja-build zlib1g-dev libzstd-dev
base=https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.8
curl -fL --retry 3 -o "$RUNNER_TEMP/llvm.tar.xz" "$base/${{ matrix.llvm_asset }}.tar.xz"
tar -xJf "$RUNNER_TEMP/llvm.tar.xz" -C "$RUNNER_TEMP"
mv "$RUNNER_TEMP/${{ matrix.llvm_asset }}" "$RUNNER_TEMP/llvm-22.1.8"
curl -fL --retry 3 -o "$RUNNER_TEMP/llvm-src.tar.xz" "$base/llvm-project-22.1.8.src.tar.xz"
tar -xJf "$RUNNER_TEMP/llvm-src.tar.xz" -C "$RUNNER_TEMP"
rm "$RUNNER_TEMP/llvm.tar.xz" "$RUNNER_TEMP/llvm-src.tar.xz"
echo "$RUNNER_TEMP/llvm-22.1.8/bin" >> "$GITHUB_PATH"
- run: pnpm install --frozen-lockfile
- run: pnpm --filter @scriptc/compiler --filter scriptc build
- name: Build the distribution
run: >-
node scripts/build-llvm-dist.mjs --source "$RUNNER_TEMP/llvm-project-22.1.8.src"
--prefix "$RUNNER_TEMP/dist/scriptc-llvm-22.1.8-linux-${{ matrix.arch }}"
--archive "$RUNNER_TEMP/scriptc-llvm-22.1.8-linux-${{ matrix.arch }}.tar.xz"
--work "$RUNNER_TEMP/llvm-dist-work"
- name: Build a helper against each LLVM
env:
HELPER: llvm-linux-${{ matrix.arch }}-gnu
run: |
LLVM_DIR="$RUNNER_TEMP/llvm-22.1.8/lib/cmake/llvm" pnpm --filter "@scriptc/$HELPER" build:native
cp "packages/$HELPER/bin/scriptc-llvm-codegen" "$RUNNER_TEMP/helper-release-archive"
LLVM_DIR="$RUNNER_TEMP/dist/scriptc-llvm-22.1.8-linux-${{ matrix.arch }}/lib/cmake/llvm" \
SCRIPTC_LLVM_THINLTO=1 pnpm --filter "@scriptc/$HELPER" build:native
cp "packages/$HELPER/bin/scriptc-llvm-codegen" "$RUNNER_TEMP/helper-profiled"
- name: Compare the helpers
run: >-
node scripts/check-llvm-dist.mjs --baseline "$RUNNER_TEMP/helper-release-archive"
--candidate "$RUNNER_TEMP/helper-profiled" --rounds 3 --work "$RUNNER_TEMP/llvm-check"
- name: Attach the archive to the release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ARCHIVE: ${{ runner.temp }}/scriptc-llvm-22.1.8-linux-${{ matrix.arch }}.tar.xz
run: |
tag=$(node --input-type=module -e "import { LLVM_DIST_TAG } from './native/llvm-codegen/llvm-dist.mjs'; console.log(LLVM_DIST_TAG)")
# Both architectures' jobs create the release if it is missing.
gh release view "$tag" > /dev/null 2>&1 ||
gh release create "$tag" --target "$GITHUB_SHA" --title "$tag" --latest=false \
--notes "LLVM 22.1.8 built with ThinLTO and the helper profile, for the Linux LLVM helpers (scripts/build-llvm-dist.mjs)." ||
gh release view "$tag" > /dev/null
gh release upload "$tag" "$ARCHIVE" "$ARCHIVE.sha256" --clobber
.github/workflows/prune-caches.yml
+41
View File
@@ -0,0 +1,41 @@
name: Prune caches
# Pull request caches are scoped to that pull request, so once it is closed no
# run can use them again. Deleting them keeps the repository under its cache
# size limit, so caches that main and open pull requests use are not evicted.
on:
pull_request:
types: [closed]
schedule:
- cron: "17 6 * * *"
workflow_dispatch:
permissions:
actions: write
pull-requests: read
jobs:
prune:
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Delete caches of closed pull requests
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
# Read both lists in full before deleting anything: deleting while the cache
# listing pages shifts later pages, and a failed request must stop the job
# instead of leaving open pull requests out of the list.
open=$(gh api --paginate "repos/$REPO/pulls?state=open&per_page=100" --jq '.[].number')
caches=$(gh api --paginate "repos/$REPO/actions/caches?per_page=100" \
--jq '.actions_caches[] | select(.ref | startswith("refs/pull/")) | "\(.id) \(.ref)"')
while read -r id ref; do
[ -n "$id" ] || continue
number=$(echo "$ref" | cut -d/ -f3)
if ! grep -qx "$number" <<<"$open"; then
gh api --method DELETE "repos/$REPO/actions/caches/$id" >/dev/null
echo "deleted cache $id ($ref)"
fi
done <<<"$caches"
.github/workflows/release.yml
+223 -12
View File
@@ -83,6 +83,7 @@ jobs:
spawnSync('git', ['diff', '--quiet', run.head_sha, 'HEAD', '--', '.',
':(exclude).github/workflows/release.yml', ':(exclude)scripts/verify-native-cli.mjs',
':(exclude)scripts/package-native-cli.mjs', ':(exclude)tests/harness/native-cli-packaging.test.ts',
':(exclude)scripts/wait-npm-available.mjs', ':(exclude)tests/harness/npm-available.test.ts',
]).status !== 0) continue;
const artifacts = api(`repos/${repo}/actions/runs/${run.id}/artifacts?per_page=100`).artifacts
.filter((artifact) => !artifact.expired && artifact.size_in_bytes > 0);
@@ -100,10 +101,71 @@ jobs:
if (receiptRun) appendFileSync(process.env.GITHUB_OUTPUT, `receipt_run=${receiptRun}\n`);
NODE
build-native-packages:
name: Build native package (${{ matrix.platform }})
# Collects the native compiler's function layout and profile on this commit
# (scripts/refresh-cli-layout.mjs, scripts/refresh-cli-profile.mjs), so every
# platform's CLI build optimizes and lays out the code it compiles. The
# committed ones fall behind main within a day or two. If this job fails,
# the CLI builds use the committed profile and layout.
cli-profile:
name: Collect the native compiler profile
needs: check-release
if: needs.check-release.outputs.should_release == 'true' && needs.check-release.outputs.reuse_artifacts != 'true'
runs-on: ubuntu-24.04
timeout-minutes: 75
continue-on-error: true
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 11.1.3
- uses: actions/setup-node@v4
with:
node-version: "24"
cache: pnpm
- uses: ./.github/actions/llvm-linux
- run: pnpm install --frozen-lockfile
- uses: ./.github/actions/zig
- uses: ./.github/actions/native-packages
with:
target: linux-x64-gnu
- run: pnpm --filter @scriptc/compiler --filter scriptc build
- name: Check out tsc-ts
run: |
git clone --quiet --depth 1 https://github.com/vercel-labs/tsc-ts "$RUNNER_TEMP/tsc-ts"
echo "tsc-ts $(git -C "$RUNNER_TEMP/tsc-ts" rev-parse HEAD)"
npm ci --prefix "$RUNNER_TEMP/tsc-ts"
# The layout comes first: the profile matches the functions of builds
# with the layout its instrumented compiler was built with, so the
# profile is collected with a second instrumented compiler built with
# this layout.
- name: Collect the function layout
run: >-
node scripts/refresh-cli-layout.mjs --jobs 4
--output "$RUNNER_TEMP/cli-profile/cli.layout.gz" --work "$RUNNER_TEMP/cli-layout-work"
- name: Collect the profile
env:
SCRIPTC_NATIVE_LAYOUT: ${{ runner.temp }}/cli-profile/cli.layout.gz
run: >-
node scripts/refresh-cli-profile.mjs --tsc-ts "$RUNNER_TEMP/tsc-ts" --jobs 4
--output "$RUNNER_TEMP/cli-profile/cli.profdata.gz" --work "$RUNNER_TEMP/cli-profile-work"
- uses: actions/upload-artifact@v4
with:
name: cli-profile
path: |
${{ runner.temp }}/cli-profile/cli.profdata.gz
${{ runner.temp }}/cli-profile/cli.layout.gz
retention-days: 7
if-no-files-found: error
build-native-packages:
name: Build native package (${{ matrix.platform }})
needs: [check-release, cli-profile]
# Runs when the profile job fails; the CLI builds then use the committed profile.
if: >-
!cancelled() && needs.check-release.outputs.should_release == 'true' &&
needs.check-release.outputs.reuse_artifacts != 'true'
strategy:
fail-fast: false
matrix:
@@ -124,6 +186,7 @@ jobs:
runtime: runtime-linux-x64-gnu
cli: cli-linux-x64-gnu
llvm_asset: LLVM-22.1.8-Linux-X64
llvm_dist: x64
use_zig: true
- platform: linux-arm64
runner: ubuntu-24.04-arm
@@ -131,6 +194,7 @@ jobs:
runtime: runtime-linux-arm64-gnu
cli: cli-linux-arm64-gnu
llvm_asset: LLVM-22.1.8-Linux-ARM64
llvm_dist: arm64
use_zig: true
- platform: windows-x64
runner: windows-2022
@@ -146,6 +210,7 @@ jobs:
seed_helper: llvm-linux-x64-gnu
cli_target: x86_64-linux-musl
llvm_asset: LLVM-22.1.8-Linux-X64
llvm_dist: x64
use_zig: true
- platform: linux-arm64-musl
runner: ubuntu-24.04-arm
@@ -155,12 +220,14 @@ jobs:
seed_helper: llvm-linux-arm64-gnu
cli_target: aarch64-linux-musl
llvm_asset: LLVM-22.1.8-Linux-ARM64
llvm_dist: arm64
use_zig: true
- platform: wasm32-wasi
runner: ubuntu-24.04
helper: llvm-linux-x64-gnu
runtime: runtime-wasm32-wasi
llvm_asset: LLVM-22.1.8-Linux-X64
llvm_dist: x64
use_zig: true
runs-on: ${{ matrix.runner }}
# The Intel macOS build took 24 minutes for 0.3.2 and passed 30 for 0.3.3.
@@ -191,6 +258,28 @@ jobs:
mv "$RUNNER_TEMP/${{ matrix.llvm_asset }}" "$RUNNER_TEMP/llvm-22.1.8"
echo "LLVM_DIR=$RUNNER_TEMP/llvm-22.1.8/lib/cmake/llvm" >> "$GITHUB_ENV"
echo "$RUNNER_TEMP/llvm-22.1.8/bin" >> "$GITHUB_PATH"
# The Linux helpers link LLVM built with ThinLTO and a profile of the
# helper's code generation (native/llvm-codegen/llvm-dist.mjs), from a
# release of this repository. Without that archive they link LLVM's
# release archive, which gives the same objects more slowly. The
# wasm32-wasi job also uploads the x64 GNU helper, so it selects the same.
- name: Select the profiled LLVM distribution (Linux)
if: matrix.llvm_dist != ''
env:
ARCH: ${{ matrix.llvm_dist }}
run: |
tag=$(node --input-type=module -e "import { LLVM_DIST_TAG } from './native/llvm-codegen/llvm-dist.mjs'; console.log(LLVM_DIST_TAG)")
asset=$(node --input-type=module -e "import { llvmDistAsset } from './native/llvm-codegen/llvm-dist.mjs'; console.log(llvmDistAsset(process.env.ARCH))")
base="https://github.com/$GITHUB_REPOSITORY/releases/download/$tag"
if curl -fsSL --retry 3 -o "$RUNNER_TEMP/$asset" "$base/$asset" &&
curl -fsSL --retry 3 -o "$RUNNER_TEMP/$asset.sha256" "$base/$asset.sha256" &&
(cd "$RUNNER_TEMP" && sha256sum --check "$asset.sha256"); then
tar -xJf "$RUNNER_TEMP/$asset" -C "$RUNNER_TEMP"
echo "SCRIPTC_HELPER_LLVM_DIR=$RUNNER_TEMP/${asset%.tar.xz}/lib/cmake/llvm" >> "$GITHUB_ENV"
echo "The helper links $asset from release $tag"
else
echo "::warning::$asset is not available from release $tag; the helper links LLVM's release archive"
fi
- uses: vercel-labs/setup-zig@v1
if: matrix.use_zig == true
with:
@@ -219,11 +308,25 @@ jobs:
Move-Item "$expanded\clang+llvm-22.1.8-x86_64-pc-windows-msvc" $llvm
echo "LLVM_DIR=$llvm\lib\cmake\llvm" >> $env:GITHUB_ENV
echo "$llvm\bin" >> $env:GITHUB_PATH
# A musl CLI ships tsc-ts's musl executable, and these jobs run on glibc
# runners, where pnpm installs only the glibc platform packages.
- name: Let pnpm install musl platform packages
if: endsWith(matrix.cli_target, '-musl')
run: |
printf 'supportedArchitectures:\n libc: [current, musl]\n' >> pnpm-workspace.yaml
- run: pnpm install --frozen-lockfile
- name: Build native packages (POSIX)
if: matrix.platform != 'windows-x64'
run: |
pnpm --filter @scriptc/${{ matrix.helper }} build:native
if [ -n "${SCRIPTC_HELPER_LLVM_DIR:-}" ]; then
LLVM_DIR="$SCRIPTC_HELPER_LLVM_DIR" SCRIPTC_LLVM_THINLTO=1 pnpm --filter @scriptc/${{ matrix.helper }} build:native || {
echo "::warning::the helper did not build against the profiled LLVM; it links LLVM's release archive"
rm -rf node_modules/.cache/scriptc-${{ matrix.helper }}
pnpm --filter @scriptc/${{ matrix.helper }} build:native
}
else
pnpm --filter @scriptc/${{ matrix.helper }} build:native
fi
if [ "${{ matrix.use_zig }}" = true ]; then
CC=zig AR=zig pnpm --filter @scriptc/${{ matrix.runtime }} build:native
else
@@ -237,6 +340,33 @@ jobs:
- name: Build host helper for cross-libc bootstrap
if: matrix.seed_helper != ''
run: pnpm --filter @scriptc/${{ matrix.seed_helper }} build:native
- name: Download the profile collected on this commit
id: cli-profile
if: matrix.cli != ''
continue-on-error: true
uses: actions/download-artifact@v4
with:
name: cli-profile
path: ${{ runner.temp }}/cli-profile
# Without the artifact, SCRIPTC_NATIVE_PROFILE_USE and SCRIPTC_NATIVE_LAYOUT
# stay unset and build-native-cli.mts uses the committed profile and layout.
- name: Select the native compiler profile
if: matrix.cli != ''
shell: bash
env:
COLLECTED: ${{ steps.cli-profile.outcome == 'success' && format('{0}/cli-profile/cli.profdata.gz', runner.temp) || '' }}
run: |
if [ -n "$COLLECTED" ]; then
echo "SCRIPTC_NATIVE_PROFILE_USE=$COLLECTED" >> "$GITHUB_ENV"
echo "The native CLI builds with the profile collected on $GITHUB_SHA"
layout="$(dirname "$COLLECTED")/cli.layout.gz"
if [ -f "$layout" ]; then
echo "SCRIPTC_NATIVE_LAYOUT=$layout" >> "$GITHUB_ENV"
echo "The native CLI builds with the function layout collected on $GITHUB_SHA"
fi
else
echo "::warning::No profile was collected on this commit; the native CLI builds with the committed profile"
fi
- name: Build native CLI distribution
if: matrix.cli != ''
env:
@@ -311,12 +441,15 @@ jobs:
(needs.check-release.outputs.reuse_artifacts == 'true' ||
(needs.build-native-packages.result == 'success' && needs.build-mobile-runtime-packs.result == 'success'))
runs-on: ubuntu-latest
timeout-minutes: 15
# Two waits for npm availability of at most 25 minutes each.
timeout-minutes: 70
environment: Release
permissions:
contents: read
actions: read
id-token: write
outputs:
uploads: ${{ steps.publish.outputs.uploads }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
@@ -336,7 +469,7 @@ jobs:
VERSION: ${{ needs.check-release.outputs.version }}
RECEIPT_RUN: ${{ needs.check-release.outputs.receipt_run }}
run: |
touch "$RUNNER_TEMP/npm-accepted.txt"
touch "$RUNNER_TEMP/npm-accepted.txt" "$RUNNER_TEMP/npm-accepted-times.txt"
# Include this run so re-running only failed jobs restores its receipts.
for run in "$GITHUB_RUN_ID" "$RECEIPT_RUN"; do
if [ -z "$run" ]; then continue; fi
@@ -345,6 +478,7 @@ jobs:
if [ "$count" -eq 0 ]; then continue; fi
gh run download "$run" --repo "$GITHUB_REPOSITORY" --pattern "npm-accepted-$VERSION-*" --dir "$RUNNER_TEMP/npm-receipts/$run"
find "$RUNNER_TEMP/npm-receipts/$run" -name npm-accepted.txt -exec cat {} + >> "$RUNNER_TEMP/npm-accepted.txt"
find "$RUNNER_TEMP/npm-receipts/$run" -name npm-accepted-times.txt -exec cat {} + >> "$RUNNER_TEMP/npm-accepted-times.txt"
done
- name: Setup pnpm
@@ -394,6 +528,7 @@ jobs:
done
- name: Publish to npm
id: publish
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -413,10 +548,14 @@ jobs:
# Upload acceptance and public availability are separate states.
# Save acceptance immediately so retries never depend on registry lag.
# npm-release.txt lists every package of the release, and
# npm-accepted-times.txt the time npm accepted each upload, for the
# npm-available job.
publish_dir() {
dir="$1"
packed="${2:-}"
name=$(node -p "require('./$dir/package.json').name")
echo "$name@$VERSION" >> "$RUNNER_TEMP/npm-release.txt"
if grep -Fxq "$name@$VERSION" "$RUNNER_TEMP/npm-accepted.txt"; then
echo "$name@$VERSION upload already accepted, skipping"
return 0
@@ -433,6 +572,7 @@ jobs:
publish_log="$RUNNER_TEMP/npm-publish.log"
if npm publish "$packed" $PROVENANCE --access public 2>&1 | tee "$publish_log"; then
echo "$name@$VERSION" >> "$RUNNER_TEMP/npm-accepted.txt"
echo "$name@$VERSION $(date +%s)" >> "$RUNNER_TEMP/npm-accepted-times.txt"
return 0
else
publish_status=${PIPESTATUS[0]}
@@ -455,8 +595,8 @@ jobs:
}
# pnpm pack resolves workspace dependencies; npm handles OIDC.
# Publish dependencies first. Registry propagation after an accepted
# upload must not block the remaining packages or the GitHub release.
# Publish dependencies first. The npm-available job waits for the
# packages that nothing in this release depends on.
publish_dir packages/runtime
publish_dir packages/runtime-darwin-arm64
publish_dir packages/llvm-darwin-arm64
@@ -476,31 +616,87 @@ jobs:
publish_dir packages/runtime-wasm32-wasi
publish_dir packages/runtime-win32-x64-msvc
publish_dir packages/llvm-win32-x64-msvc
# A native CLI package depends on its platform's LLVM helper and
# runtime pack packages, and scriptc on the native CLI packages.
# Each group publishes only once the packages it depends on
# resolve, so an install never finds a package whose dependencies
# npm is still processing. A rerun skips accepted uploads and waits
# again.
wait_available() {
awk 'FILENAME == ARGV[1] { accepted[$1] = $2; next } { print $1, accepted[$1] }' \
"$RUNNER_TEMP/npm-accepted-times.txt" "$RUNNER_TEMP/npm-release.txt" > "$RUNNER_TEMP/npm-wait.txt"
node scripts/wait-npm-available.mjs "$RUNNER_TEMP/npm-wait.txt" --interval 30 --timeout 1500
}
wait_available
for pkg in packages/cli-*; do publish_dir "$pkg"; done
publish_dir packages/compiler
publish_dir packages/experimental
publish_dir packages/threads
publish_dir packages/types
wait_available
publish_dir packages/cli
echo "All package uploads accepted; npm registry propagation may continue after this job." >> "$GITHUB_STEP_SUMMARY"
# Each package of the release with the time its upload was accepted,
# when a run recorded it.
{
echo "uploads<<NPM_UPLOADS"
awk 'FILENAME == ARGV[1] { accepted[$1] = $2; next } { print $1, accepted[$1] }' \
"$RUNNER_TEMP/npm-accepted-times.txt" "$RUNNER_TEMP/npm-release.txt"
echo "NPM_UPLOADS"
} >> "$GITHUB_OUTPUT"
- name: Save accepted upload receipts
if: always()
uses: actions/upload-artifact@v4
with:
name: npm-accepted-${{ needs.check-release.outputs.version }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/npm-accepted.txt
path: |
${{ runner.temp }}/npm-accepted.txt
${{ runner.temp }}/npm-accepted-times.txt
if-no-files-found: ignore
retention-days: 90
# Publish standalone native distributions and the surface manifest after
# npm publishing succeeds. The release body comes from the marked changelog.
github-release:
name: Create GitHub Release
# Waits until every package of the release resolves on the npm registry,
# and reports each package's time from upload to available. Fails with the
# packages that are still missing after 2 hours. It only reads from the
# registry; the next release run retries it, and publishing skips uploads
# npm already accepted.
npm-available:
name: Wait for npm availability
needs: [check-release, publish]
if: >-
!cancelled() && needs.check-release.outputs.should_release == 'true' && needs.publish.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 135
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "24"
registry-url: "https://registry.npmjs.org"
- name: Wait for every package to resolve
env:
UPLOADS: ${{ needs.publish.outputs.uploads }}
run: |
printf '%s\n' "$UPLOADS" > "$RUNNER_TEMP/npm-uploads.txt"
node scripts/wait-npm-available.mjs "$RUNNER_TEMP/npm-uploads.txt"
# Publish standalone native distributions and the surface manifest after
# every npm package resolves. The release body comes from the marked
# changelog.
github-release:
name: Create GitHub Release
needs: [check-release, publish, npm-available]
if: >-
!cancelled() && needs.check-release.outputs.should_release == 'true' && needs.npm-available.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write
@@ -552,6 +748,21 @@ jobs:
- name: Package standalone compilers
run: node scripts/package-native-cli.mjs packages /tmp/scriptc-release-assets
# The profile the native CLIs were built with. Runs that reuse native
# builds from before the cli-profile job, or whose profile job failed,
# have none; their CLIs used the committed profile.
- name: Download the native compiler profile
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RUN: ${{ needs.check-release.outputs.artifact_run }}
VERSION: ${{ needs.check-release.outputs.version }}
run: |
if gh run download "$RUN" --repo "$GITHUB_REPOSITORY" --name cli-profile --dir "$RUNNER_TEMP/cli-profile"; then
mv "$RUNNER_TEMP/cli-profile/cli.profdata.gz" "/tmp/scriptc-release-assets/scriptc-$VERSION-cli.profdata.gz"
else
echo "Run $RUN has no cli-profile artifact"
fi
- name: Create GitHub Release
run: |
VERSION="${{ needs.check-release.outputs.version }}"
.gitignore
+12
View File
@@ -8,8 +8,14 @@ node_modules/
!tests/fixtures/npm/node_modules/
!tests/corpus/*/node_modules/
!tests/fixtures/npm-static/node_modules/
!tests/coverage-fixtures/wasm/node_modules/
!tests/coverage-fixtures/node-islands*/node_modules/
!tests/coverage-fixtures/dynamic-tier/package/node_modules/
!tests/fixtures/npm-static/nested/inner/node_modules/
!tests/fixtures/npm-static/node_modules/*/node_modules/
!tests/fixtures/npm/store/*/node_modules/
!tests/fixtures/npm/node_modules/*/node_modules/
!tests/fixtures/npm/workspace/*/node_modules/
!tests/fixtures/npm/cases/*/node_modules/
!tests/library-mode/*/node_modules/
!tests/fixtures/provenance/node_modules/
@@ -28,10 +34,13 @@ output/
/packages/runtime-*/runtime-pack.json
/packages/runtime-*/.runtime-pack-*
!tests/fixtures/npm/node_modules/*/dist/
!tests/fixtures/npm/node_modules/@*/*/dist/
!tests/fixtures/npm/workspace/*/dist/
!tests/fixtures/npm/cases/workspace-copied/members/*/dist/
!tests/fixtures/npm/cases/workspace-copied/node_modules/*/dist/
!tests/fixtures/provenance/node_modules/*/dist/
!tests/fixtures/npm-static/node_modules/*/dist/
!tests/corpus/*/node_modules/*/dist/
.scriptc/
*.log
.DS_Store
@@ -45,3 +54,6 @@ tmp-*/
.env.local
.env*
.env*.local
# the vendored fs-extra corpus package: its lib/output module and jsonfile's own universalify
!tests/corpus/fs-extra-package/node_modules/fs-extra/lib/output/
!tests/corpus/fs-extra-package/node_modules/jsonfile/node_modules/
.oxlintrc.jsonc
+15 -1
View File
@@ -241,6 +241,7 @@
"packages/compiler/src/frontend/comptime-node.ts",
"packages/compiler/src/frontend/ts7/world-check.ts",
"packages/compiler/src/frontend/ts7/source-parser.test.ts",
"packages/compiler/src/frontend/jsx-source.test.ts",
"packages/compiler/src/frontend/npm-static-rewrite.ts",
"packages/compiler/src/frontend/npm-static-bundled-cjs.ts",
"packages/compiler/src/library/semantic-source.ts",
@@ -270,6 +271,7 @@
"packages/compiler/src/backend/llvm/**",
"packages/compiler/src/backend/loop-polls.test.ts",
"packages/compiler/src/backend/may-throw.test.ts",
"packages/compiler/src/backend/stack-observations.test.ts",
"packages/compiler/src/ir/**",
"packages/compiler/src/library/*.test.ts",
"packages/compiler/test/**",
@@ -281,10 +283,22 @@
},
{
// Ambient declarations describe an existing constructor and its prototype methods.
"files": ["packages/compiler/ambient/scriptc-node-fallback.d.ts"],
"files": [
"packages/compiler/ambient/scriptc-node-fallback.d.ts",
"packages/compiler/ambient/node-fallback/*.d.ts",
],
"rules": {
"typescript/no-unsafe-declaration-merging": "off",
},
},
{
// Each module's fallback declarations are a script that declares the
// module ambiently; it names the files of the modules it uses by
// reference, since a script cannot import them.
"files": ["packages/compiler/ambient/node-fallback/*.d.ts"],
"rules": {
"typescript/triple-slash-reference": "off",
},
},
],
}
AGENTS.md
+30 -5
View File
@@ -4,7 +4,7 @@ This is the only agent guide for the repository. Its rules apply everywhere, inc
## Markdown style
Write prose paragraphs and list items as single source lines and let editors and renderers wrap them naturally. Do not hard-wrap Markdown at a fixed column; preserve separate lines only where Markdown syntax or intentional formatting requires them, such as headings, lists, tables, blockquotes, and code blocks.
Write prose paragraphs and list items as single source lines and let editors and renderers wrap them naturally. Do not hard-wrap Markdown at a fixed column; preserve separate lines only where Markdown syntax or intentional formatting requires them, such as headings, lists, tables, blockquotes, and code blocks. The Limitations page partials under `docs/content/limitations/` are the exception: they keep one sentence per line (see Limitations page partials).
## Build and test
@@ -25,6 +25,8 @@ their supported native-clang contracts locally; other hosts retain those
checks in the Sandboxes. Both lanes green is the bar before shipping any
change.
`SCRIPTC_SANDBOX_NATIVE_REFERENCE` names a built checkout inside a prepared `SCRIPTC_SANDBOX_SNAPSHOT` or image: each lane copies that checkout's LLVM helper and runtime pack instead of rebuilding them when their source inputs are identical. Sandbox creation is rate limited per team in vCPUs per minute, so the runner retries a rate-limited creation with backoff. Larger Sandboxes (`SCRIPTC_SANDBOX_VCPUS` up to 32, with `SCRIPTC_TEST_WORKERS` at half the vCPUs) shorten the lanes most.
`VERCEL_OIDC_TOKEN` is the preferred credential. `VERCEL_TOKEN` remains
supported with explicit `VERCEL_TEAM_ID` and `VERCEL_PROJECT_ID`. The custom
`SCRIPTC_SANDBOX_IMAGE` is optional: without it, the gate starts from
@@ -46,8 +48,20 @@ SCRIPTC_TEST_WORKERS=4 SCRIPTC_SAN=1 pnpm test # sanitized lane
Corpus programs are differential tests against Node: every program runs under Node and as a compiled native binary, and stdout, stderr, and exit codes must match byte-for-byte. A new feature lands with corpus programs that pin its behavior both ways.
Name a new corpus program after what it tests, without a number: lowercase words joined by hyphens, such as `tests/corpus/intersection-restated-members.ts`, or a directory such as `tests/corpus/<name>/main.ts` for a multi-file program. Parallel changes kept taking the same next number; with names, two changes collide only when they add the same name, which git reports as a conflict. Leave existing numbered programs as they are: never rename or renumber them, and name a new program even when it extends a numbered family.
Record a new or remeasured program's CI cost (used to balance the test shards) in `tests/harness/corpus-costs/<program>.json`, not in `tests/harness/corpus-costs.json`; see the README there.
Before opening a PR that adds or changes corpus, diagnostic, or frontend fixtures, run `pnpm test:ts7 --baselines-only`. This fast completeness check uses the same fixture discovery as the parity suite; it does not type-check or replace behavioral parity. Generate missing preflight/order baselines, review their diagnostics and module order, and run the affected `order-parity.test.ts` batches before pushing. Keep existing baseline entries unchanged unless the corresponding behavior intentionally changes. The full `pnpm test:ts7` command also checks completeness before starting its sweep.
A change that must not cost good code anything, such as a compatibility fix that replaces a refusal, proves it with the zero-cost gate. The gate compiles the benchmark workloads, the self-hosted compiler, and the TypeScript corpus with two built checkouts. It requires byte-identical IR, LLVM, and release executables, and compile time within tolerance. `--allow=<regex>` names the change's pattern signature: programs whose sources match may differ, and the PR lists them. Any other difference fails the gate, which names the first differing function. The gate also enforces the two-tier runtime separation: no executable may gain symbols from `DYNAMIC_TIER_PREFIXES` in `scripts/zero-cost-compare.mjs`, and fully typed benchmarks may link none; extend that list as dynamic-tier units land. `--quick` checks only the benchmarks; `--cachegrind` adds per-function instruction counts for executables that differ. Run it in a Sandbox with both checkouts prepared:
```bash
node scripts/zero-cost-gate.mjs --baseline=/vercel/sandbox/prev --candidate=. --allow='\barguments\b'
```
The whole gate takes 30 to 75 minutes on one 8-vCPU Sandbox, so shard it across Sandboxes. `--shard=i/n` splits the bench and corpus programs; the self-hosted compiler runs in shard 1, or alone with `--set=self`, and its candidate-source build overlaps its comparison build on hosts with 40 GB or more. Each shard writes `--json`, and `node scripts/zero-cost-merge.mjs <shard.json>...` merges them into one verdict, recomputing the compile-time intervals over every shard. `--record-baseline-cache=<file>` builds only the baseline and records each program's artifact hashes; `--baseline-cache=<file>` then skips the baseline build of every program whose candidate artifacts hash the same, which halves the work for a change that leaves most programs identical. A record applies only to the built baseline checkout, toolchain, and paths it was made with, and is ignored otherwise; a program whose entry, relative imports, or optimization level changed since the record builds its baseline.
Test location follows scope:
- Co-locate white-box unit tests with implementation files under
@@ -58,6 +72,16 @@ Test location follows scope:
Keep existing tests in place unless a change already touches their organization;
new tests should follow this convention.
## Numbered registries
Parallel changes must never pick the same number by hand. Each registry has one source of truth and a fast check:
- Diagnostic codes: claim a code by adding its row, in numeric order, to `DIAGNOSTIC_CODES` in `packages/compiler/src/diagnostics/codes.ts` before any site uses it. A shipped construct's code moves to `RETIRED_DIAGNOSTIC_CODES` and is never reused; codes that open work already uses are held in `RESERVED_DIAGNOSTIC_CODES`. `codes.test.ts` fails on a code without a row, a retired code in use, an unused row, or rows out of order, and two claims of one code are a merge conflict or a duplicate key.
- Dyn handle tags: add a new tag by name only to `SCR_DYNH_NEW_TAGS` in `packages/runtime/src/scr_dyn.h`, in name order; the enum numbers it. Pin a tag to an explicit value in `SCR_DYNH_TAGS` only when the LLVM backend needs its number. The header does not compile when two tags share a value, and `dyn-handle-tags.test.ts` names the problem.
- Runtime enums and flag bits the backend mirrors (`ScrDynKind`, `ScrMessageKind`, `ScrBytesElem`, `SCR_CLOSURE_KIND_*`, the `%ScrArr` spare flags) are checked against their C declarations by `packages/compiler/src/backend/llvm/runtime-registries.test.ts`. A new mirrored enum or flag set gets a check there, and C code names flag bits instead of writing their values.
Before taking a number in a pinned registry, run `node scripts/registry-claims.mjs`: it lists the diagnostic codes and pinned handle tags that open pull requests claim and reports collisions; `--check` reports only collisions with the current branch and exits 1 when there is one.
## Where things live
- `packages/compiler` — the frontend (tsc API to IR), typed IR, validator, serializer, and LLVM backend.
@@ -132,11 +156,11 @@ Never hand-edit these files:
- `packages/compiler/surface-manifest.json`
- `internal/compatibility/generated/node-v24-internal.json`
- `internal/compatibility/generated/node-v24-backlog.json`
- `docs/src/generated/node-v24-compatibility.json`
- `docs/src/generated/node-v24-compatibility-meta.json`
Change compiler decision sources or the implementation-owned compatibility manifests, then regenerate. Use `pnpm manifest` when compiler decision tables change and `pnpm node-compat` whenever the Node pin, compatibility manifests, compiler surface manifest, changes.
Change compiler decision sources or the implementation-owned compatibility manifests, then regenerate. Use `pnpm manifest` when compiler decision tables change and `pnpm node-compat` whenever the Node pin, compatibility manifests, compiler surface manifest, changes. `pnpm node-compat --local` reclassifies the committed ledger without fetching Node's docs; it writes the same files as `pnpm node-compat` while the Node pin is unchanged.
Never resolve a merge conflict in these files by hand. Resolve the source conflicts, then run `node scripts/regen-generated.mjs`: it takes the merged-in branch's version of each conflicted generated file, regenerates all of them from the merged sources, and stages the ones that conflicted.
The public docs artifact must not expose repository test paths, internal source identifiers, engineering evidence bookkeeping, documentation/metadata headings, command-line or configuration-only entries, or rows that are N/A in the native tier. Those remain in the complete internal census under `internal/compatibility/generated/`.
@@ -151,7 +175,7 @@ For each API family:
5. Add differential evidence that executes under Node and as a compiled binary. Cover stdout, stderr, exit status, success behavior, and important error shapes. A load-only export check does not prove behavior.
6. Update the implementation-owned source or manifest. Positive `supported` and `partial` mappings must name existing test evidence.
7. Run `pnpm manifest` if compiler surface tables changed, then run `pnpm node-compat`. Never patch generated JSON or headers.
8. Inspect the internal ledger, public artifact, and backlog diff. Confirm the intended API family changed and unrelated rows did not.
8. Inspect the internal ledger and public artifact diff, and compare `pnpm node-compat:backlog` output before and after. Confirm the intended API family changed and unrelated rows did not.
9. Run focused tests, `pnpm node-compat:check`, and the full validation gate required for the change. If docs output changed, also run the docs gate.
Compatibility commands:
@@ -189,6 +213,7 @@ The docs site is a standalone pnpm workspace under `docs/`. It uses Next.js App
- A labeled bullet (`- **Crypto:** ...`) is `docs/content/limitations/<section>/<label>.mdx`, for example `builtin-apis/crypto.mdx`. Edit that file to change the bullet.
- The other text of a section (paragraphs, code blocks, unlabeled lists) is `docs/content/limitations/<section>.mdx`; text before the first heading is `introduction.mdx`.
- `<section>` and `<label>` are the heading and label text in lowercase with each run of other characters replaced by `-`.
- Write one sentence per line; a bullet's continuation lines are indented two spaces. Markdown joins the lines, and edits to different sentences don't conflict. `pnpm lint` fails on a line that holds two sentences, and `node docs/scripts/limitations-partials.mjs fix` splits such lines.
- To add a bullet, add its file and an include line at its position in `limitations.mdx`, with a blank line on each side. Consecutive bullet includes render as one list.
- To add a section, add the heading to `limitations.mdx` and include the section's partial below it.
CHANGELOG.md
+368 -2
View File
@@ -6,6 +6,374 @@ All notable changes to scriptc will be documented in this file.
<!-- release:start -->
## 0.4.5
### Highlights
- **Fixes:**
- Release builds read a typed array's length without a header load that could read a freed view.
- `net.Socket` `write()` returns Node's backpressure boolean and fires `'drain'`.
- Super calls in JavaScript method values call the declared base method.
- Calling a non-function through an asserted callee throws Node's TypeError.
- **Faster programs:**
- Exceptions propagate by unwinding from directly called functions on Linux, instead of status checks after every call.
- Typed-array loops prove their bounds and keep their views disjoint in more cases, including views over shared memory in worker programs; integer loops stay in integers across nested counted loops, pure helper calls, powers of two and byte reads.
- Lone array callbacks run as loops with their bodies inlined, and row sums run two rows per pass.
- Number-key maps look keys up on their integer view and write updates through the entry the read found; small number-key maps use a direct index.
- Strings: base64 decodes eight characters at a time; Buffers fill, swap byte order and decode `utf16le` a word at a time; readline splits input in one scan; string scans over non-ASCII strings use their UTF-16 units.
- Untyped code: property caches also cache absent properties, object literal members go through the store caches, calls on plain arrays skip the method ladder, and dynamic values dispose without key-cache walks.
- JSON parses into discriminated unions, and dynamic values convert into nested unions, in one pass.
- BigInt division settles quotients from the top limbs, and limb shifts are vectorized.
- **Faster builds:**
- Function bodies are emitted on up to eight workers, including in dev builds, and repeated work in LLVM module emission is cut.
- Linux executables link with lld when the driver finds it, and unchanged executables are reused without selecting the runtime pack.
- Dev builds reuse their frontend output after edits that keep token positions, and unchanged frontend inputs are checked by file identity before hashing.
- Lowering fetches only the syntax trees it reads, and the large compiler tables are built on first use.
- Release builds of the native compiler use a profile collected on the release commit.
- Type checking can use a tsc-ts server shared by later builds (opt-in).
- **Node.js parity:**
- Crypto: RSA key generation lengths, RSA-PSS keys, encrypted private keys, `X509Certificate` fields and methods, and TLS certificates and `tls.Server` contexts.
- zlib: Brotli compression and decompression.
- Dates are mutable objects with Node's setters and conversions; URL hosts and paths parse as the URL Standard does, including non-ASCII hosts with UTS #46.
- `Math`, `JSON` and `Reflect` are real objects in JavaScript; objects can inherit from JavaScript class prototypes; typed arrays held as untyped values support named properties and `Symbol.species`.
- Error messages: `JSON.parse` syntax errors use V8's messages and positions; `Object.getPrototypeOf` of an Error answers `Error.prototype`.
- `util.inspect` of dynamic classes, functions and `%o` arguments, and `Symbol.toStringTag` fields and getters, match Node.
- **npm packages:** more npm and CommonJS dependencies compile statically, including namespace-style packages with guarded declarations and `var` exports, and untyped Maps, Sets and arrays are typed from their uses.
<!-- release:end -->
## 0.4.4
### Highlights
- **Fixes for 0.4.3:**
- Unchanged rebuilds hit the frontend cache again. An unchanged rebuild of a 23-module program takes 93 ms instead of 1.26 s, and of tsc-ts (dev) 1.6 s instead of 58.2 s.
- The native compiler builds programs with several typed-array accesses at one masked index in a loop again, including tsc-ts's release build, which failed with SC3004.
- `x!` returned from a function, used as an arrow function's result, or passed as an argument keeps null or undefined when the destination type holds it, as JavaScript does. `x!` operands of `+`, `+=` and other conversions that hold undefined or null convert as Node does.
- Effect 4 programs whose generators yield clients built by other generators run again.
- The cache for untyped property access in programs that declare classes is reverted, fixing a crash in Effect 4 observability programs.
- **Faster:**
- Number compares and typed-array indexes use the exact integer of sums like `i + 1`: a heap sift-down over an `Int32Array` runs 17.5% faster.
- glibc's per-thread heaps use huge pages: the VS Code type-check runs 3.4% faster.
- Mid-size executables (over 1 MiB of LLVM text) build in at least eight partitions: the LLVM backend phase takes 34 to 74% less time for three.js programs of 1.4 to 7.8 MB.
- JavaScript class instances are boxed at their exact class, and stores into nested string-keyed dictionaries keep class method calls direct.
- `--target-cpu` selects an x86-64 microarchitecture level.
- **Node.js parity:**
- X25519 keys and `crypto.diffieHellman`; `existsSync` and `fs.promises` accept `file:` URLs; `Buffer.from` ranges over ArrayBuffers, the deprecated `Buffer()` constructor and `Buffer` as a value.
- Dates are objects with identity; `Promise.all` over different entry types settles on Node's turns; completion callbacks run in their call's AsyncLocalStorage context.
- TypedArray methods run on typed arrays held as untyped values, and unions of typed-array classes compile. Borrowed `TypedArray.prototype.set` calls compile and throw Node's TypeError for other targets.
- Split CommonJS `require` bindings, `__`-prefixed exports, string-key destructuring of `any` values, and record key order for values JavaScript built.
- **npm packages:** zod 4 and `@hono/zod-validator` values are projected at the declaration boundary.
## 0.4.3
### Highlights
- **Faster checking and smaller builds with tsc-ts 0.0.9:** scriptc checks with tsc-ts 0.0.9, and builds reuse the bundled declaration files' diagnostics instead of checking them again. Small IR builds are 10.6 to 16.0% faster and small executable builds 5.5 to 7.6% faster, with identical output. A native build of tsc-ts with this release checks 0.4 to 6.6% faster than with 0.4.2 and peaks 4 to 16% lower.
- **Integer arrays and loops:**
- Number arrays that only hold int32 values use 4-byte slots in locals, across direct calls and in module globals, and loop-carried locals written from their elements stay int32. A workload that walks arrays of cursors runs 54.5% faster.
- Loops with two counters closing in on each other (`for (let i = lo, j = hi; i < j; i++, j--)`) and element copies through `a[i]!` run without per-element checks, and two-counter `while` loops run on integer counters.
- Loops over typed arrays and charAt scans are versioned on their bounds, and ASCII strings are scanned as bytes.
- **Less work for untyped and dynamic code:**
- String-keyed record reads under `??` and `!` no longer allocate: a dictionary-walking workload makes no union boxes instead of 363,359 and runs 12 to 15% faster.
- Checked-dynamic method calls dispatch on method ids, observed class methods call natively while their slot is unchanged, and calls on bag-tracked class instances skip building their views.
- **Node.js parity:**
- `crypto.getCipherInfo`, AES-CCM, AES key wrap, the legacy `createCipheriv` modes, `checkPrime`/`generatePrime`, and Node's argument checks for `generateKeyPair`.
- Streams: `construct`, `writev`, `decodeStrings` and `defaultEncoding`, including `writev` on `Transform` and `PassThrough`.
- `FinalizationRegistry`, `new.target`, `String.prototype.at` on untyped values, `Error.stackTraceLimit`, `spawnSync`'s `env` option, and class constructors called through `call`/`apply` throw Node's TypeError.
- Records enumerate in the key order their object literal wrote, and objects converting to a union with an empty-record arm keep every field.
- **npm packages:** zod 3 compiles natively and zod 4's core runs, minified packages such as marked compile, chalk 5 and hot-shots 8 build from built-in shims, and `@hono/node-server` 2.x is built in. User code types npm package declarations at a checked, identity-preserving boundary.
- **Builds:** the cached build is reused after whitespace and comment edits, function bodies are emitted on forked workers, and the LLVM emitter reads its analyses from one recorded walk of each module.
## 0.4.2
### Highlights
- **Faster builds of large programs:**
- scriptc checks with tsc-ts 0.0.7, which answers contextual types and resolved signatures in batches, so lowering asks for them in one request per chunk instead of one per node. A native build of tsc-ts is 7% faster (110.5 → 103.5 s on 8 vCPUs, lowering 12% faster), with an identical executable.
- The native compiler is built with profile-guided optimization, so tsc-ts builds 6.9% faster cold and 10.7% faster after an edit.
- The native compiler's optimization profile is refreshed, so a native build of tsc-ts is a further 10% faster (124.7 → 111.7 s on 16 vCPUs), with lowering 17% faster.
- The class dispatch passes index their receivers by name instead of testing each one, and they reuse their per-function scans while a function's body is unchanged.
- Checker replies resolve objects the compiler already holds from their ids without decoding them.
- **Leaner builds:**
- The frontend of an optimized executable's first build runs in a child process, so the native code generator starts after its memory is released. The cold build of tsc-ts peaks at 2.21 GB instead of 3.30 GB.
- The compiler collects its heap as lowering and LLVM emission grow it.
- **WebCrypto:**
- The `crypto` global, `node:crypto`'s `webcrypto` and `crypto.subtle` are one object.
- `generateKey`, `importKey`, `exportKey`, `encrypt`, `decrypt`, `deriveBits`, `deriveKey`, `wrapKey` and `unwrapKey` cover HMAC, AES-GCM, AES-CBC, AES-CTR, AES-KW, PBKDF2, HKDF and JSON Web Keys.
- `crypto.subtle` signs and verifies with ECDSA, RSASSA-PKCS1-v1_5, RSA-PSS and Ed25519.
- All of it follows Node 24's argument conversion, algorithm normalization and `DOMException`s. jose's symmetric JWE paths run as in Node.
- **JSX:** `.tsx`, `.jsx` and JavaScript files with JSX compile under the project's `jsx` options.
- `react`, `react-jsx` and `react-jsxdev` lower to the factory calls TypeScript emits, with the same keys, fallbacks and source positions.
- `preserve` and `react-native` are refused with a named diagnostic.
- **Faster worker messages:**
- A message is encoded into one block sized to its contents.
- A typed record is posted from its fields.
- A `'message'` listener with a typed parameter builds it from the message, without a checked-dynamic copy.
- Messages between `MessageChannel` ports take 90 ms where they took 250 ms in 0.4.1 (Node: 337 ms), with a peak RSS of 45 MB instead of 115 MB.
- **Loops over typed arrays vectorize:** a loop to a typed array's length keeps one bounds test for every typed array it indexes.
- Index temporaries such as `const j = i * 4` are computed as integers.
- Uint8ClampedArray stores convert inline.
- Row-major index terms such as `r * cols + j` are tested once before the loop.
- Typed-array callback methods call a literal callback directly on an integer counter.
- An 896 × 896 matrix-vector product with four accumulators takes 55 ms instead of 396 ms (Node: 275 ms). `Uint8Array.map` is 4.5 times as fast and `Int32Array.some` 7.8 times as fast as in 0.4.1.
### Features
- **Asymmetric keys and signatures in `node:crypto`:**
- `generateKeyPair`, `generateKeyPairSync`, `createPrivateKey` and `createPublicKey` handle RSA, EC and Ed25519 keys in PEM, DER and JWK.
- `sign`, `verify`, `createSign` and `createVerify` sign and verify with those keys, with Node's output and errors.
- Ed25519 verification matches Node on 2,083 test vectors, including small-order and mixed-order keys.
- **`crypto.createDiffieHellman`, `crypto.getDiffieHellman` and `crypto.createDiffieHellmanGroup`:**
- The modp1, modp2 and modp5 to modp18 groups, explicit primes and generators, and generated safe primes are supported, with Node 24's key widths, `verifyError` bits, key range checks and errors.
- Secrets of 0, 1 and p − 1 are rejected, and so are peer keys outside the RFC 5114 subgroups.
- Primes longer than 8192 bits are refused.
- **TLS:**
- `tls.connect` and `tls.createServer` negotiate `ALPNProtocols`, and both sockets report `alpnProtocol`. `minVersion` and `maxVersion` bound the handshake, and `getProtocol()` reports the version it picked.
- `tls.createServer` takes an `SNICallback` that answers a `SecureContext` synchronously or later.
- `new tls.TLSSocket(socket, { isServer: true, ... })` and `tls.connect({ socket })` run the handshake on an open socket, such as an HTTP `CONNECT` tunnel.
- A client whose peer closes before the handshake completes fails with Node's `ECONNRESET`.
- **Streams:**
- Readable, Writable, Duplex, Transform and PassThrough streams take `objectMode`, `readableObjectMode` and `writableObjectMode`, and an object-mode side carries any value through `push`, `write`, `pipe`, `pipeline`, `for await` and `'data'` listeners.
- `_write` and `_transform` receive the encoding argument as Node passes it.
- `Cipheriv` and `Decipheriv` are Transforms: they can be piped, used as `pipeline()` stages, read and iterated.
- `pipeline()` accepts function stages anywhere among its streams, and an array, a string or a synchronous generator as its source.
- **Packages compiled from their JavaScript:**
- js-yaml and smol-toml compile statically and match Node: ES5 error constructors (`Object.create(Error.prototype)`, `Error.call(this)`, `Error.captureStackTrace`) and classes extending `Date` behave as in Node.
- TypeScript's UMD module wrapper compiles as the module body, and tsc's `__importDefault` and `__importStar` helpers around builtin modules compile as plain `require()` calls.
- Interfaces, type aliases and enums that only a package's `.d.ts` declares keep their types.
- **JavaScript sources:**
- Error constructors can be read as values, so `err.constructor === TypeError` works.
- `workerData` can be destructured.
- Unary `+` converts unions of strings, numbers, booleans, `null` and `undefined` as `Number()` does, so `+process.argv[2]` compiles.
- These compile and behave as in Node:
- chained readline listener registration;
- `Number.parseInt` without a radix;
- `match` and `search` with a possibly undefined regular expression;
- `Buffer.from` over an untyped `ArrayBuffer`, with its byte offset;
- Atomics on untyped arrays.
- Buffer methods run on Buffers held as untyped values.
- **`fs`:**
- The synchronous functions (`readFileSync`, `writeFileSync`, `statSync`, `readdirSync`, `mkdirSync`, `rmSync`, `openSync` and the others) accept `file:` URLs wherever they accept a path, with Node's conversion errors.
- `fs.promises.rm` takes `recursive`, `force`, `maxRetries` and `retryDelay`, and removing a directory without `recursive` throws Node's `ERR_FS_EISDIR`.
- **`TypedArray.of`** creates a typed array of its arguments, and a dynamic iterable spread into `Array.of`, `push` or `TypedArray.of` is drained through the iterator protocol.
- **Global `isFinite` converts its argument with ToNumber,** as `isNaN` does. The Math functions and `String.fromCharCode` evaluate every argument before converting any.
- **`Object.freeze`, `Object.seal` and `Object.preventExtensions` of a function** act on its own properties, and `Object.isFrozen`, `Object.isSealed` and `Object.isExtensible` answer for functions. @noble/hashes' frozen hashers work.
- **EventEmitters used through both typed and untyped references** share one listener list. An untyped `emit` reaches typed listeners, an untyped `emit('error')` follows Node's contract, and `listenerCount`, `eventNames`, `listeners`, `rawListeners`, `removeAllListeners`, `setMaxListeners` and `getMaxListeners` work on untyped values.
- **Template literals, `join`, `Array.prototype.toString` and the default `sort()`** call an object's own or inherited `toString`, as `String()` does, instead of printing `[object Object]`.
- **Classes and generics:**
- A generic factory declared to return a constructor type can return one of its local classes, including an anonymous class expression, and `class X extends factory(...)` extends it.
- Static methods of a function-local class compile.
- Constructor types declared in the program (`new () => unknown`, types that also name static members, construct overloads) hold class values.
- Members typed by a conditional type over a literal class type argument (`Field<string, "required">`) resolve.
- Object literals under an erased generic member may hold callbacks with optional or defaulted parameters.
- **Assignment and update targets wrapped in parentheses or non-null assertions** (`(x) = 1`, `h.sums[k]! += 10`) compile.
- **Untyped values keep what JavaScript code adds:**
- Properties that untyped code adds to a typed object literal stay on it.
- Named properties written to an array through an `unknown` view stay on the array, and `for...in` visits them after the indices.
- An `unknown` value asserted to a structural type with methods keeps its class's methods.
- The type inferred from an empty object literal holds any non-nullish value, as TypeScript's `{}` does.
- **A WebAssembly module the build finds but does not translate** rejects with a `CompileError` that names the reason, for example a proposal outside the translated feature set.
- **SC6101 names the functions that likely run often and use untyped values:**
- It shows up to five, most likely hot first, with why their values are untyped: an `any`, an untyped `JSON.parse` result or an untyped JavaScript file.
- `scriptc coverage --fail-on=dynamic` fails on it, and `--no-dynamic-tier-warning` leaves it out.
### Performance
- **Regular expressions skip start positions that their leading character sets rule out,** testing 16 positions at once. Searching 50 MB of text for nine case-insensitive alternations takes 0.83 s instead of 3.88 s (Node: 4.38 s).
- **Loop counters stay integers in more loops:**
- Counters that step beside a counted loop's counter (`for (let i = 0, p = 0; i < n; i++, p += 4)`) stay integers: a pixel-stride loop takes 114 ms instead of 340 ms (Node: 190 ms).
- So do two counters that close in on each other (`lo++, hi--`), 17% faster on a permutation-reversal loop.
- So do counters of loops bounded by a string's length or by an invariant limit other than an array's length, and while-loop counters that nested scans advance. A non-ASCII lexer is 9.5% faster.
- **Loops that call a known callback** test its closure once before the loop and call the function directly, so captured `let` bindings stay in registers. `forEach` sums with a captured accumulator are 25 to 55% faster.
- **A speculative integer loop whose check fails** finishes the iteration outside the ordinary loop, which stays optimizable, and a loop site that keeps failing stops trying. With the hole-free-prefix test that element reads now take first, a pairwise loop over an array of class instances with number fields is about 1.25 times as fast as in 0.4.1.
- **Typed arrays over a SharedArrayBuffer** read and write elements inline as relaxed atomic accesses. A numeric kernel over shared `Float64Array`s takes 0.78 s instead of 1.56 s (Node: 0.84 s).
- **`Math.pow(2, n)` for integer exponents** builds the result from its bits. A counter that masks with `Math.pow(2, length)` takes 11.0 s instead of 20.4 s.
- **Async calls that never suspend run in place,** and so does `await Promise.all` of such calls, with Node's microtask turns and rejection handling. An async pipeline takes 5.5 ms instead of 9.7 ms.
- **Strings:**
- `Number()` of a short integer string returns the integer directly.
- `split` builds its result in one inline block, and `indexOf` and `slice` on ASCII strings skip the UTF-16 index map. A log summary is 20% faster.
- `charAt` and `s[i]` on ASCII strings read the character from a table.
- Chains of one-character `replace` calls (the HTML-escape idiom) replace a block of 16 bytes at a time, 12.8% faster on a template renderer.
- **`JSON.parse` and `JSON.stringify`** scan whole words and append parsed typed arrays in bulk, 22% faster to parse and 10% faster to stringify records.
- **Smaller arrays, maps and program-lifetime instances:**
- The array header is 32 bytes instead of 48.
- Arrays of up to 16 known elements keep their slots in their own block.
- Arrays of a known final length get exactly that capacity.
- Small map entry arrays grow in finer steps.
- On Linux, `ProgramLifetime` instances take a 32-bit header and are referenced in 4 bytes.
- They lower the peak memory of a large TypeScript type checker: the 4-byte references alone take a large project's check from 4.13 GB to 3.75 GB, and the 32-bit header saves a further 5%.
- **Worker messages queued together are delivered in one batch** before immediates run. 20,000 messages interleaved with a worker's output drain in 0.5 s instead of 5.6 s (Node: 0.57 s).
- **The cycle collector** never buffers instances of classes whose fields hold nothing cycle-capable, and its per-object paths are shorter. A workload of cyclic graphs is about 5% faster.
- **Array helpers specialized for a callback that cannot throw** skip the pending-exception test, 4.5% faster on a functional pipeline.
- **Compile time:**
- A file's function declarations are walked once for their references, so lowering a 10 MB single-module bundle takes 69 s instead of 219 s.
- LLVM emission computes its module facts once, 6.9% faster on tsc-ts.
- AST walks visit subtrees by their node ranges and read node words through a `Uint32Array`.
- Program builds no longer import runtime bitcode, so builds through Node and through the native compiler produce identical executables.
### Fixes
- **Member reads and method calls on an array hole** held in a binding narrowed to one arm of a wide union throw Node's `Cannot read properties of undefined` TypeError.
- **`AsyncLocalStorage`:**
- A store entered before an async function's first await, or in a body that never suspends, is the caller's store after the call.
- Each synchronous generator step runs in its resumer's context, as in Node.
- **Generic arrow and function-expression bindings used before their declarations** behave as in Node, including the `ReferenceError` of a `let` or `const`.
- **Assignment expressions into a module `let` declared without an initializer** (`new Promise((r) => (resolve = r))`) store the value instead of throwing a `ReferenceError`.
- **The `extends` expression of a class defined inside a function** is evaluated each time the definition runs.
- **A `super()` call that leads an `if` condition's comma chain** compiles.
- **A catch binding tests `instanceof`** correctly for values thrown as `unknown`, rejected promises and runtime errors.
- **zlib callbacks receive `undefined` as their result when the operation fails,** as in Node, and a callback may assign a fallback to it.
- **`Readable.push` and `Writable.write` with `string | null` or `Buffer | null` chunks** no longer crash.
- **Object literals keep their key order** through compiler rewrites and on shapes a typed `JSON.parse` also builds, so `JSON.stringify`, `Object.keys` and `console.log` match Node.
- **`console.log` prints class instances with their class name** in programs that test `instanceof` on unknown values.
- **`String`, `Number` and `Boolean` passed to array methods** convert as their direct calls do, and nested array literals in a recursive JSON-style union build.
- **A dynamic method call that the program also reads through a prototype** finds the method when the instance's property bag does not hold it.
- **Storing an element on `Array.prototype`** fails with SC2020, because holes in native arrays do not read inherited elements.
- **A recursive type alias in a generic signature** no longer crashes the compiler.
- **A `Buffer` view of an untyped typed array** compiles.
- **JSX programs checked with tsgo** compile as they do with tsc-ts.
- **Two programs whose entries share a name and build into one directory** no longer share a cached translation unit, so a later cache hit always relinks the right program.
- **A warm build whose cached LLVM exceeds the engine's string limit** reads it in parts instead of failing with `Invalid string length`.
- **The same program compiles to the same executable wherever it sits on disk** and whatever order its functions lower in, and edits outside a declaration do not rename the helpers and closures inside it.
- **Libraries that use untyped byte arrays** link.
## 0.4.1
### Highlights
- **Profile-guided optimization:** `scriptc build --profile-generate` builds an instrumented executable, `scriptc merge-profiles` (with `--sparse`) merges the profiles its runs write, and `--profile-use` optimizes a release or speed build with the merged profile. tsc-ts built with a profile from its own type check checks other projects 1.15 to 1.2 times as fast, with the same diagnostics and memory. Builds without a profile are unchanged.
- **Faster cold builds of large programs:** large class-based programs build in partitions instead of as one module, so their cold builds are 25 to 32% faster, and an edit to one function recompiles only its partition.
- **npm packages:** more packages that pass builtins, classes and promises around as values now compile from their JavaScript. Among them are execa 5, hot-shots, @opentelemetry/api, async-sema, buffer-crc32, yazl, tar-stream, smol-toml and truncate-utf8-bytes. `undici` 5.x and 7.x compile from an implementation that ships with scriptc, and `isarray`, `safe-buffer` and `process-nextick-args` compile as the Node builtins they stand in for.
- **Dynamic-tier property access goes through per-site inline caches** keyed on object shapes. On constructor-and-prototype code, the code that reads several object shapes at one site and an AST interpreter, untyped code runs 1.4 to 3 times as fast as in 0.4.0. Typed programs compile to the same code as before.
- **Smaller arrays, class instances and number-keyed maps:**
- The array header is 48 bytes instead of 64.
- Small arrays start with three inline slots.
- Instances of program class hierarchies keep their class id in the reference-count word instead of a vtable pointer.
- `Map`s and `Set`s with number keys store 16-byte entries.
- Together they lower the peak memory of a large TypeScript type checker by 8 to 10%, with the same output and wall time.
### Features
- **`for await` iterates untyped values,** including async generators, `Symbol.asyncIterator` results and synchronous iterators, whose values are awaited. `yield*` delegates inside async generators. A JavaScript generator whose return type is inferred as `any` compiles, including one that delegates to itself.
- **`crypto.createECDH` and `ECDH.convertKey`** are supported for the 11 short Weierstrass curves mbedTLS implements, with Node 24's key formats, encodings and errors. Other curves are refused.
- **`undici` 5.x and 7.x:** `Client`, `Pool`, `Agent`, `request`, the global dispatcher, `fetch` through a dispatcher, the error classes and the `undici:` diagnostics channels follow each version's behavior. Each importing package compiles against its own installed version. The limitations page lists the parts that differ.
- **Packages marked as never loaded:** `"scriptc": { "packages": { "<name>": "unloaded" } }` in the entry's `package.json` makes `require("<name>")` throw Node's `MODULE_NOT_FOUND`, so optional dependencies take their absent branch.
- **`util.promisify` of an untyped function** returns a promise-returning wrapper at run time.
- **Destructuring with runtime computed keys** (`const { [name]: v } = source`) reads records and untyped values. A missing key gives `undefined` and the element default applies.
- **Import cycles compile when a class is used only after both modules finish initializing.** Reaching the class earlier throws Node's `ReferenceError: Cannot access 'X' before initialization`.
- **`await` of an untyped value adopts thenables,** including objects that define `then` at run time. `Promise.prototype` and `promise.constructor` can be read as values. A child process can be given new properties and awaited as a thenable.
- **Function values convert their results and arguments** when they are stored in a slot whose signature has the same structure but a different shape, such as a function returning `Promise<Message>` stored in a `() => Promise<Message | undefined>` slot.
- **Declarations from SDKs and shared type packages:** index signatures of `any`, `any[]` members, classes that a package only declares, and `Pick` views of classes compile.
- **Builtins as values:** `bind`, `call` and `apply` work on builtin functions, and `Buffer.byteLength`, `Buffer.compare`, `Buffer.alloc`, `JSON.parse`, `fs.realpathSync` and `String.prototype` methods can be read as values. `events` and `EventEmitter` can be used as untyped constructor values.
- **JavaScript sources:**
- Bitwise operators convert boolean operands as `ToNumber` does.
- `arguments.length` can be read in sloppy-mode functions with named parameters.
- A function value in an untyped slot may have any signature.
- Prototype methods take their JSDoc parameter types.
- A top-level function declaration that its module reassigns works.
- A `var` can be overwritten with a different kind of value.
- A default-export object literal can hold class values.
- **Index-signature records answer `"k" in record`** for literal keys. A fresh array literal spreads into an array of a wider element type. A `const` annotated with a type that resolves to `any` holds an untyped object.
- **`Set.prototype.add` returns the Set** when its result is used, so `s.add(a).add(b)` compiles.
- **`exec` works on regular expressions held as `any`,** and a numeric enum from a package can be read by a runtime key (`DiagLogLevel[level]`).
- **`dns.lookup` answers that are decided at run time** can be read as untyped values, `net.connect` accepts Node's `lookup` option, and `dns.lookup` passes the `ADDRCONFIG`, `V4MAPPED` and `ALL` hints.
- **`util.inspect`, `console.log` and `util.format` print program-declared `Error` subclasses** with Node's constructor-name header (`[Plain [Error]: m]`).
- **Typed arrays:** `forEach`, `map`, `filter`, `some`, `every`, `find`, `findIndex`, `findLast`, `findLastIndex`, `reduce` and `reduceRight` compile on every typed-array kind. `map` and `filter` return the receiver's kind (a Buffer for a Buffer), and callbacks receive the element, index and array. `BYTES_PER_ELEMENT` reads answer the element size, including on untyped values. Reads and `in` tests of members Node 24's typed-array and Buffer prototypes lack answer `undefined` and `false`, as in Node. Destructuring assignment into typed-array elements compiles.
- **`Math.max` and `Math.min` with spreads among other arguments** (`Math.max(...xs, 5)`) compile, with Node's NaN and ±0 rules.
- **Generic arrow and function-expression bindings inside functions** compile when they use nothing from the enclosing function's frame.
- **`const xs = []` inside a function** takes the element type its later writes give it, as TypeScript evolves it, instead of `number[]`. Bindings whose reads disagree or that are reassigned become dynamic arrays, as at module scope.
### Performance
- **Loop counters stay integers in more loops:** counters compared as `i + c` with a bound, and for-loop counters that are reset or copied inside the loop. A record-boundary scan takes 104 ms (Node: 260 ms), and a typed-array scan takes 318 ms (Node: 801 ms).
- **Indexed reads of non-ASCII strings are inline in worker executables,** and strings of up to 2^24 UTF-16 units keep their unit table. A lexer over a 2.7 MB non-ASCII source takes 190 ms (Node: 327 ms).
- **Worker messages:** a thread that finds its mailbox empty spins for up to 20 µs before it blocks. 100,000 round trips between the main thread and a worker take 360 ms, a quarter of Node's time.
- **Local-time `Date` getters cache the zone offset,** UTC getters reuse the last decomposition, and integers 0 to 99 convert to strings without allocating. Local-time date formatting takes 0.81 times Node's time.
- **Untyped code:** `%` and the bitwise operators run on integers for integral operands, `o.x += d` updates a number property in place, and array `length` reads answer from the element count.
- **Sorts by several keys** (comparators that chain field differences with `||`, including optional number fields with a default) sort by key in the runtime instead of calling the comparator, about 24% faster on a benchmark of such sorts.
- **Worker executables start their threads faster:** the allocator marks its memory for core dumps only when the system would write a core, which removes about 680 calls that hold the address-space lock as threads start. tsc-ts checks tRPC 6% faster at 16 checkers.
- **The native compiler walks IR without a closure per node,** about 3 to 10% faster in the frontend and LLVM emission phases of large builds.
- **Full cycle collections that the death rule requests run at the event loop's idle moments.** Watch-mode rebuilds of large projects are 6 to 11% faster.
### Fixes
- **`a?.b.c` on an `any` value short-circuits only the guarded step,** as in Node: it is `undefined` when `a` is nullish and throws when `a.b` is undefined.
- **A failed checked cast to an optional record names the deepest failing path,** for example `expected string at $.order.owner.id, got number`.
- **Spreading a non-iterable value held as `any`** throws Node's `x is not iterable` message.
- **`process.versions` entries that the runtime doesn't have read as `undefined`** in projects that use `@types/node`.
- **Functions stored in JavaScript object literals get Node's names,** taken from the property key.
- **`monitorEventLoopDelay().percentile()` returns 511** on an empty histogram, as Node does.
- **A CommonJS `var` statement that mixes `require()` declarators with other bindings** no longer crashes.
- **Untyped async methods called through overrides or untyped receivers return their promises,** which fixes internal compiler errors in programs that use zod's `parseAsync` this way.
- **TypeScript enum exports in compiled packages stay `undefined` until they are assigned,** as in Node.
- **Element writes accept a possibly undefined index** (typed-array writes, compound and update writes, element assignments used as values and destructuring targets) and convert it with ToNumber, as element reads already did.
## 0.4.0
### Breaking changes
- **tsc-ts is the default TypeScript 7 checker** on Linux and macOS. scriptc installs it with the compiler (`tsc-ts` 0.0.5 and its platform package for the host). Programs get the same diagnostics and generated code as with tsgo, and compiling takes less checker memory. `SCRIPTC_CHECKER=tsgo` selects tsgo, which stays the default on Windows. A build cache entry made with one checker isn't reused with the other.
- **`--npm-static=auto`, the default, compiles more packages:** packages typed by their DefinitelyTyped twin (`@types/<name>`), and untyped dependencies that another package's JavaScript imports. A program whose dependencies were refused before can now compile them, and a twin whose declarations disagree with the package's JavaScript fails the build at the use. Use `--npm-static <pkg,…>` to name the packages to compile.
- **Program type errors are reported as type errors** instead of as refusals of the npm package that contains them.
### Highlights
- **WebAssembly:** builds translate every WebAssembly module whose bytes they can see into native code, and `WebAssembly.compile`, `instantiate`, `validate`, `new WebAssembly.Module` and `new WebAssembly.Instance` run them. There is no interpreter. `WebAssembly.Memory`, `Table` and `Global`, `Module.imports`, `Module.exports` and `Module.customSections`, and function references from tables are supported, and modules can be sent to workers. `scriptc coverage` lists the modules a program loads, with `--wasm <file.wasm>` for modules loaded from paths the sources don't spell out.
- **Async functions that suspend only at their `await`s run as stackless coroutines** instead of on fibers with their own stack, so a suspended call keeps only the values live across its awaits.
- **Discriminated unions of records are the record pointer itself,** with no box around each value.
- **The dynamic tier computes with NaN-boxed values:** numbers, booleans, `undefined` and `null` between operators no longer allocate. `scriptc coverage --explain-dynamic` lists the functions that run in the dynamic tier and why, and `--tier=static-only` keeps a program fully static.
- **npm packages:** esbuild's bundled CommonJS chunks, CommonJS cycles and lazily initialized ES modules compile statically, as do classic CommonJS packages (late and JSON `require`s, require cycles), class hierarchies across a package's modules, and the packages unblocked by type-only imports, `@types` packages, `punycode` and `constants`. chalk 5 and the jose root entry compile.
### Features
- **`@scriptc/experimental` adds `ThreadSlots`,** one value per key and thread on shared objects, read without a lock.
- **`node:test` adds `mock.timers`, `getTestContext`, `t.assert.snapshot` and `t.assert.fileSnapshot`,** the `TestContext` values `filePath`, `attempt`, `passed`, `workerId` and `tags`, module-level `before` and `after` hooks, the `SuiteContext` and hook contexts, and unnamed `test` and `describe` calls. A timed-out test cancels its running subtests.
- **`node:worker_threads` adds `BroadcastChannel`,** `setEnvironmentData`/`getEnvironmentData`, `receiveMessageOnPort`, `markAsUntransferable`, `markAsUncloneable`, `threadName`, `resourceLimits.stackSizeMb`, the `stdout`/`stderr`/`stdin` options and streams, and `worker[Symbol.asyncDispose]`.
- **`node:net` adds `SocketAddress` and `BlockList`,** and `net.connect` resolves host names.
- **`node:http` pools keep-alive sockets in `Agent`s,** uses `globalAgent` for default requests, reads `http.request` options at run time, and speaks HTTP over Unix domain sockets. `fetch` tunnels through `HTTP_PROXY`/`HTTPS_PROXY` with `CONNECT`.
- **`DisposableStack`, `Float16Array`, `Error.isError`, `Promise.try` and the regular expression `v` flag** are supported.
- **Iterator helpers** run on generators, `Map` and `Set` iterators, string and typed array iterators, `Iterator.from` and untyped iterators.
- **`Object.groupBy` and `Map.groupBy`** group Sets, Maps and generators.
- **`v8`, `stream/web`, `dns/promises`, `EvalError` and `graceful-fs`** are available, and `Temporal` is reported as absent, as in Node 24.
- **`util.promisify`** works over the crypto callback functions, dns lookups, `child_process.exec`'s callback forms and callback-style functions.
- **`events.once`** takes an `AbortSignal` and works over streams.
- **`AsyncLocalStorage` stores** carry into timer, immediate, `process.nextTick` and microtask callbacks.
- **JavaScript classes compile like TypeScript classes** when their layout or JSDoc types allow it.
- **Classes that extend `EventEmitter` or a stream class can be used as values,** and so can `EventEmitter`, the stream classes and the error classes themselves. Constructor functions can inherit from them.
- **Class instances can be frozen and their fields redefined.**
- **Method overrides may differ from the method they override** in parameter and result types, as long as each value converts.
- **Library profiles add the `cstring` class.**
- **An experimental young-object nursery** for compiled classes and records is available at compile time with `SCRIPTC_YOUNG_NURSERY=1`.
### Performance
- **Dense arrays have head room,** so `shift`, `splice(0, n)` and `unshift` runs cost amortized O(1), and `pop` of a packed array is inline.
- **Index loops over arrays and typed arrays are versioned on one bounds guard,** with stable lengths proven and typed-array memory tagged for alias analysis.
- **Retain and release pairs are removed** through private-array precision and last-use moves, borrowed string slices go into stack buffers, and records, tuples and spread updates stay in frame storage.
- **SHA-384 and SHA-512 use AVX2, AVX-512VL or the arm64 SHA-512 instructions,** and MD5's block function is unrolled.
- **String character tests, optional-string `switch`es, `String.fromCharCode` and long-string `length` are faster,** and `Buffer.write` widens ASCII to UTF-16 sixteen bytes per step.
- **The cycle collector walks dead cycles once,** frees large dead cycle sets in address order and no longer creates self-capture cycles on the release fast path.
- **Workers return their memory** while the event loop waits and when they finish, and default-region memory is committed as it is used.
- **Lifetime regions carry across `await`,** and freed region chunks are reused.
- **Large closure environments compile in linear time,** and two frontend scans no longer repeat.
- **Programs keep stack frames only when they can read `stack`.** Before, a computed-key read on a dynamic value kept frames in every function; tsc-ts's own check is back to 0.3.6's speed.
- **The child-process runtime unit links only into programs that spawn.**
### Fixes
- **Node 24 parity:** `util.format`, `util.types` and `util.inspect`, `URLSearchParams`, `querystring` (null-prototype results), `string_decoder`, `os`, `events` and `timers/promises` match Node 24. Class instances, iterators, iterator helper objects and holes in arrays print as Node prints them, and `%d`/`%i` format dynamic arguments as Node does.
- **Silent wrong output is fixed** for object identity, `Object.prototype` values, `Date.parse` (V8's date grammar), stored builtins, constructor returns, `arr && arr[0]`, ES5 class names, builtin function values, uninitialized `let` writes, module-export `this` and `1 ** Infinity`.
- **Strings compare in UTF-16 code-unit order.**
- **Dates format with Node's local time zone names,** convert to local time once for their local getters, and follow `TZ` changes. `Date.prototype.toJSON` gives `null` for invalid dates.
- **Errors:** thrown Maps, Sets, typed arrays, functions and arrays keep their identity. Errors list their own name where they were created, errors held as `any` have their own `stack` accessor and convert with `Error.prototype.toString`, and errors that cross into dynamic code carry Node's error properties.
- **`matchAll` rows and `exec` results have `index`, `input` and `groups`,** and named groups keep their null prototype through optional chains.
- **Unhandled rejections are reported after each timer and immediate callback,** and async generator completions settle on Node's microtask turns.
- **Missing `Record` and index-signature keys read as `undefined`,** and generic functions that read a missing element return `undefined`.
- **Sockets:** `'data'` listeners get strings after `setEncoding`, socket and stream chunks are Buffers, and an idle pooled http socket that receives data is destroyed.
- **Diagnostic code frames window long source lines.**
- **npm crashes, internal errors and false TDZ errors** found while compiling popular packages are fixed.
## 0.3.6
### Features
@@ -38,8 +406,6 @@ All notable changes to scriptc will be documented in this file.
- **`Object.setPrototypeOf` returns primitive receivers,** and `createHmac` without a key throws Node's error.
- **Boxed records that hold an async function link the dynamic promise unit.**
<!-- release:end -->
## 0.3.5
### Features
Dockerfile.sandbox
+2
View File
@@ -26,6 +26,7 @@ RUN apt-get update \
libclang-rt-22-dev \
libpolly-22-dev \
libzstd-dev \
lld-22 \
llvm-22-dev \
ninja-build \
xz-utils \
@@ -33,6 +34,7 @@ RUN apt-get update \
&& ln -sf clang-22 /usr/bin/clang \
&& ln -sf clang++-22 /usr/bin/clang++ \
&& test "$(llvm-config-22 --version)" = 22.1.8 \
&& test -x "$(dirname "$(readlink -f /usr/bin/clang)")/ld.lld" \
&& rm -rf /var/lib/apt/lists/*
RUN curl --fail --silent --show-error --location \
RELEASING.md
+5 -1
View File
@@ -14,7 +14,11 @@ To prepare a release:
CI (`.github/workflows/release.yml`) compares the version in `packages/cli/package.json` to what `scriptc` has on npm. If it differs, it builds platform packages on matching macOS, Linux, and Windows runners, verifies the version spine, and publishes the runtime, helper, compiler, and CLI packages in dependency order. After the publish succeeds, a separate job creates the git tag `v<version>` and the GitHub release with the marked changelog entry as its body, and attaches `surface-manifest.json` — the machine-readable listing of the surface the static tier compiles at that version (stable per-entry ids, so two releases diff mechanically; see `packages/compiler/src/coverage/surface-manifest.ts` for the schema). The job regenerates the manifest from the tree and fails on any byte difference from the committed file before attaching, so the asset is always the manifest of the code being released. The same file ships inside the `@scriptc/compiler` package as `@scriptc/compiler/surface-manifest.json`.
The release job builds each pinned LLVM helper, precompiled runtime pack, and native CLI on its matching host. The native CLI bundles its TypeScript checker, LLVM helper, runtime pack, declarations, and comptime evaluator. npm postinstall installs the matching native executable; Node is needed for npm installation but not native compilation or execution. The user's toolchain performs the final platform link. Sanitized builds also use it to instrument program LLVM and compile the C runtime. Platform packages publish ahead of `@scriptc/compiler` and `scriptc`. The GitHub release attaches standalone compiler archives, their SHA-256 checksums, and the surface manifest after npm publication succeeds.
The release job builds each pinned LLVM helper, precompiled runtime pack, and native CLI on its matching host. The native CLI package bundles its TypeScript checker, declarations, and comptime evaluator, and depends on its platform's LLVM helper and runtime pack packages at the same version; the compiler resolves them beside itself. The standalone archives bundle the helper and runtime pack too. npm postinstall installs the matching native executable; Node is needed for npm installation but not native compilation or execution. The user's toolchain performs the final platform link. Sanitized builds also use it to instrument program LLVM and compile the C runtime. Platform packages publish ahead of `@scriptc/compiler` and `scriptc`. The GitHub release attaches standalone compiler archives, their SHA-256 checksums, and the surface manifest after npm publication succeeds.
Release builds of the native CLI optimize the compiler with a profile collected on the release commit. The `cli-profile` job runs `node scripts/refresh-cli-profile.mjs` on Linux x64, which builds an instrumented compiler, runs it on tsc-ts, a slice of the corpus, and the runtime benchmark workloads, and writes the merged profile. The profile keeps the functions that training never ran, with zero counts, so the CLI builds optimize them as cold code and place them after the code that ran. Every platform's CLI build uses that profile (`SCRIPTC_NATIVE_PROFILE_USE`), and the GitHub release attaches it as `scriptc-<version>-cli.profdata.gz`. Code the profile does not describe, or describes from older compiler sources, is optimized as without a profile. When the profile job fails, the CLI builds use the committed profile `packages/compiler/native/cli.profdata.gz`, which local and CI builds of the native compiler also use. Refresh the committed profile when compiler code has changed substantially since the last refresh: run `node scripts/refresh-cli-profile.mjs --tsc-ts <tsc-ts checkout>` on a quiet Linux x64 host and commit the new profile on its own.
Builds of the native compiler also order its functions by the committed layout `packages/compiler/native/cli.layout.gz`: the functions that `--version` and rebuilds of unchanged programs run come first, then the ones other builds run, ordered by the share of the layout's training builds that run each, so a run maps fewer pages of the executable. Functions the layout does not list keep their usual order. Function names of closures and generated helpers change with nearby edits, so the `cli-profile` job also collects a layout on the release commit (`scripts/refresh-cli-layout.mjs`), and the CLI builds use it (`SCRIPTC_NATIVE_LAYOUT`). Instrumented builds use the same layout as optimized builds, because a function laid out in a different partition than in the instrumented build no longer matches its profile record. The job therefore collects the layout first and then builds the instrumented compiler for the profile with that layout. Refresh the committed layout the same way, with `node scripts/refresh-cli-layout.mjs`, before the profile, and commit each on its own.
Publishing uses npm trusted publishing (OIDC) — there is no npm token secret.
Each published package must have a GitHub Actions trusted publisher for
benchmarks/regions/README.md
+1
View File
@@ -7,6 +7,7 @@ Programs that build generations of `ProgramLifetime` objects and free each gener
| `generations.ts` | Per-file syntax trees and binder symbols in one region per file, checker state (types, signatures, node and symbol links in paged tables, literal and union caches) in one region per generation. `edit` re-parses one file per generation, `all` re-parses every file. |
| `compiler.ts` | A tokenizer, recursive-descent parser, binder and checker over generated sources, rebuilt from scratch in one region per build, like a watch mode that does not reuse syntax trees. |
| `emit.ts` | A large live graph of ordinary objects with parent links, standing in for a type checker, and a per-file emit that builds its output in a region released right after. |
| `requests.ts` | Batches of concurrent async request handlers, each in its own region across its awaits: tokenize a JSON body, parse it into a syntax tree that points at its tokens, validate it into issues and an index, render a response, release the region. |
```console
$ node benchmarks/regions/run.mjs --baseline=../scriptc-main --runs=7
benchmarks/regions/emit.ts
+1 -1
View File
@@ -54,7 +54,7 @@ let level: Type[] = [new Type(id++, undefined)];
while (id < typeCount) {
const next: Type[] = [];
for (const t of level) for (let k = 0; k < 4 && id < typeCount; k++) next.push(new Type(id++, t));
roots.push(...next);
for (const t of next) roots.push(t);
level = next;
}
benchmarks/regions/requests.ts
+221
View File
@@ -0,0 +1,221 @@
// An async request handler: each request tokenizes a JSON body, awaits (as
// if reading more of it), parses the tokens into a syntax tree whose nodes
// point back at their tokens, awaits again (as if loading reference data),
// validates the tree into issues held in an index, and renders a response.
// Requests run in batches of concurrent handlers, so the handlers of one
// batch interleave at every await.
//
// requests <requests> <concurrency> <items-per-body> [plain]
//
// Each request runs in its own LifetimeRegion, released when its response
// is rendered; `plain` never constructs a region.
import { readFileSync } from "node:fs";
import type { ProgramLifetime } from "@scriptc/types";
import { LifetimeRegion } from "@scriptc/experimental";
const T_PUNCT = 0;
const T_STR = 1;
const T_NUM = 2;
const T_WORD = 3;
class Token implements ProgramLifetime {
readonly kind: number;
readonly text: string;
readonly pos: number;
constructor(kind: number, text: string, pos: number) {
this.kind = kind;
this.text = text;
this.pos = pos;
}
}
const N_OBJECT = 0;
const N_ARRAY = 1;
const N_STRING = 2;
const N_NUMBER = 3;
const N_LITERAL = 4;
class JsonNode implements ProgramLifetime {
readonly kind: number;
readonly token: Token;
parent: JsonNode | undefined = undefined;
readonly keys: string[] = [];
readonly children: JsonNode[] = [];
constructor(kind: number, token: Token) {
this.kind = kind;
this.token = token;
}
}
class Issue implements ProgramLifetime {
readonly path: string;
readonly node: JsonNode;
readonly message: string;
constructor(path: string, node: JsonNode, message: string) {
this.path = path;
this.node = node;
this.message = message;
}
}
// ---- request bodies (deterministic) ----
function body(id: number, items: number): string {
let seed = 7 + id * 7919;
const rand = (n: number): number => {
seed = (seed * 1103515245 + 12345) % 2147483648;
return seed % n;
};
const parts: string[] = [];
for (let i = 0; i < items; i++) {
const qty = rand(12) - 1;
const sku = rand(9) === 0 ? "" : `sku-${rand(5000)}`;
const tags = Array.from({ length: rand(4) }, (_, k) => `"t${k}${rand(10)}"`).join(",");
parts.push(`{"sku":"${sku}","qty":${qty},"price":${rand(10000) / 100},"gift":${rand(2) === 0},"tags":[${tags}]}`);
}
return `{"order":${id},"customer":"c${id % 97}","items":[${parts.join(",")}]}`;
}
// ---- handler ----
function tokenize(text: string): Token[] {
const tokens: Token[] = [];
let i = 0;
while (i < text.length) {
const c = text.charCodeAt(i);
const start = i;
if (c === 34) {
i++;
while (text.charCodeAt(i) !== 34) i++;
i++;
tokens.push(new Token(T_STR, text.slice(start + 1, i - 1), start));
} else if (c === 45 || (c >= 48 && c <= 57)) {
i++;
while (i < text.length && (text.charCodeAt(i) === 46 || (text.charCodeAt(i) >= 48 && text.charCodeAt(i) <= 57))) i++;
tokens.push(new Token(T_NUM, text.slice(start, i), start));
} else if (c >= 97 && c <= 122) {
while (i < text.length && text.charCodeAt(i) >= 97 && text.charCodeAt(i) <= 122) i++;
tokens.push(new Token(T_WORD, text.slice(start, i), start));
} else {
i++;
tokens.push(new Token(T_PUNCT, text[start]!, start));
}
}
return tokens;
}
class Parser {
private at = 0;
private readonly tokens: Token[];
constructor(tokens: Token[]) {
this.tokens = tokens;
}
value(): JsonNode {
const token = this.tokens[this.at++]!;
if (token.kind === T_STR) return new JsonNode(N_STRING, token);
if (token.kind === T_NUM) return new JsonNode(N_NUMBER, token);
if (token.kind === T_WORD) return new JsonNode(N_LITERAL, token);
const node = new JsonNode(token.text === "{" ? N_OBJECT : N_ARRAY, token);
const close = token.text === "{" ? "}" : "]";
while (this.tokens[this.at]!.text !== close) {
if (node.kind === N_OBJECT) {
node.keys.push(this.tokens[this.at]!.text);
this.at += 2; // the key and the colon
}
const child = this.value();
child.parent = node;
node.children.push(child);
if (this.tokens[this.at]!.text === ",") this.at++;
}
this.at++;
return node;
}
}
function member(node: JsonNode, key: string): JsonNode | undefined {
const i = node.keys.indexOf(key);
return i < 0 ? undefined : node.children[i];
}
function validate(root: JsonNode, issues: Issue[], index: Map<string, JsonNode>): number {
let total = 0;
const items = member(root, "items")!;
for (let i = 0; i < items.children.length; i++) {
const item = items.children[i]!;
const path = `items[${i}]`;
const sku = member(item, "sku")!;
if (sku.token.text === "") issues.push(new Issue(`${path}.sku`, sku, "empty sku"));
else if (index.has(sku.token.text)) issues.push(new Issue(`${path}.sku`, sku, `duplicate of ${index.get(sku.token.text)!.token.pos}`));
else index.set(sku.token.text, item);
const qty = Number(member(item, "qty")!.token.text);
if (qty <= 0) issues.push(new Issue(`${path}.qty`, member(item, "qty")!, "quantity must be positive"));
else total += qty * Number(member(item, "price")!.token.text);
const tags = member(item, "tags")!;
for (const tag of tags.children) if (tag.token.text.endsWith("0")) issues.push(new Issue(`${path}.tags`, tag, "reserved tag"));
}
return total;
}
const hop = (): Promise<void> => new Promise((resolve) => setImmediate(resolve));
async function handle(text: string): Promise<string> {
const tokens = tokenize(text);
await hop();
const root = new Parser(tokens).value();
await hop();
const issues: Issue[] = [];
const index = new Map<string, JsonNode>();
const total = validate(root, issues, index);
await hop();
const first = issues[0];
return `${member(root, "order")!.token.text} ${tokens.length} ${index.size} ${issues.length} ${total.toFixed(2)}` +
(first === undefined ? "" : ` ${first.path}@${first.node.token.pos}`);
}
// ---- driver ----
function rssMb(key: string): number {
const m = new RegExp(`${key}:\\s*(\\d+) kB`).exec(readFileSync("/proc/self/status", "utf8"));
return m === null ? 0 : Number(m[1]) / 1024;
}
const args = process.argv.slice(2);
const requests = Number(args[0] ?? "4000");
const concurrency = Number(args[1] ?? "16");
const items = Number(args[2] ?? "100");
const useRegions = args[3] !== "plain";
async function serve(id: number): Promise<string> {
const text = body(id, items);
if (!useRegions) return await handle(text);
const region = new LifetimeRegion();
const response = await region.run(() => handle(text));
region.release();
return response;
}
async function main(): Promise<void> {
let hash = 0;
let lines = 0;
let sample = "";
let rssFirst = 0;
const t0 = performance.now();
for (let next = 0; next < requests; next += concurrency) {
const batch: Promise<string>[] = [];
for (let k = next; k < next + concurrency && k < requests; k++) batch.push(serve(k));
for (const response of await Promise.all(batch)) {
for (let i = 0; i < response.length; i++) hash = (hash * 31 + response.charCodeAt(i)) % 1000000007;
lines++;
if (lines === 1) sample = response;
}
if (next === 0) rssFirst = rssMb("VmRSS");
}
const ms = performance.now() - t0;
console.log(`${lines} responses, hash ${hash}, first ${sample}`);
console.log(
`total ${ms.toFixed(1)} ms per-request ${((ms * 1000) / requests).toFixed(1)} us` +
` rss-first ${rssFirst.toFixed(1)} MB rss-last ${rssMb("VmRSS").toFixed(1)} MB peak ${rssMb("VmHWM").toFixed(1)} MB`,
);
}
main();
benchmarks/regions/run.mjs
+3 -1
View File
@@ -39,6 +39,8 @@ const CONFIGS = [
{ name: "compiler-one-shot", program: "compiler.ts", args: ["400", "60", "1"] },
// Per-file emit regions next to a large live heap of ordinary objects.
{ name: "emit-files", program: "emit.ts", args: ["400000", "200", "2000"] },
// Concurrent async request handlers, one region per request across its awaits.
{ name: "requests-async", program: "requests.ts", args: ["4000", "16", "100"] },
];
const work = join(tmpdir(), `scriptc-region-bench-${process.pid}`);
@@ -93,7 +95,7 @@ for (const config of CONFIGS) {
const lines = r.stdout.trim().split("\n");
outputs.set(name, lines[0].replace(/ gen0 .*/, ""));
const metrics = {};
for (const [, key, value] of lines.at(-1).matchAll(/([a-z0-9-]+) ([0-9.]+) (?:ms|MB)/g))
for (const [, key, value] of lines.at(-1).matchAll(/([a-z0-9-]+) ([0-9.]+) (?:ms|us|MB)/g))
metrics[key] = Number(value);
samples[name].push(metrics);
}
benchmarks/runtime/README.md
+39 -20
View File
@@ -4,30 +4,49 @@ Application-shaped workloads for measuring the speed of compiled executables. Ea
| Workload | Category | Exercises |
| --- | --- | --- |
| `json-records` | cpu | `JSON.stringify`, typed `JSON.parse`, Map aggregation |
| `regex-logs` | cpu | regex `exec` with groups, `test`, global `replace` |
| `ast-interp` | cpu | tokenizer, recursive-descent parser, class dispatch |
| `records-sort` | cpu | comparator sorts, group-by, `reduce`/`filter` |
| `template-render` | cpu | template literals, escaping, `map`/`join` |
| `functional-pipeline` | cpu | `filter`/`map`/`reduce`/`flatMap` chains, closures, small records |
| `async-pipeline` | cpu | `async`/`await`, `Promise.all` fan-out |
| `word-graph` | cpu | string-keyed Map/Set, BFS |
| `alloc-trees` | cpu | allocation-heavy trees, immutable object-spread updates |
| `cyclic-graphs` | cpu | object graphs with back references built and dropped: trees with parent and sibling links, rings of records sharing a long-lived registry, closures capturing their owner |
| `numeric-kernels` | cpu | sieve, hashing, `number[][]` matmul, `Float64Array` |
| `typed-array-scan`, `typed-array-scan-workers` | cpu | element-by-element scan loops over `Uint8Array`, `Uint16Array`, `Int32Array` and `Float64Array`; the second entry is a worker-enabled executable (it can start a worker), so each access also checks for shared storage |
| `intern-table`, `intern-table-map`, `intern-table-1`, `intern-table-map-1` | cpu | worker threads interning the same type-like objects under string keys and records under number keys, then hits and misses: one shared `InternTable` per kind of key, or a module-level `Map` per thread (the `-map` entries), with 8 threads or 1 (the `-1` entries); under Node.js every thread has its own tables |
| `ast-interp` | cpu | tokenizer, recursive-descent parser, class dispatch |
| `async-pipeline` | cpu | `async`/`await`, `Promise.all` fan-out |
| `atomics-contended` | cpu | four worker threads updating `SharedArrayBuffer` counters with `Atomics`: one shared counter, one counter per worker, a `compareExchange` loop, and `Atomics.wait`/`notify` handoffs |
| `value-objects` | cpu | small immutable records and class instances: vector helpers, method chains, particle updates |
| `region-builds` | cpu | a watch-mode build loop over `LifetimeRegion`: each build binds a program in a new region and releases the previous one, which a cache keeps referenced for two builds |
| `optional-fields` | cpu | records and class instances with `number \| undefined` and other scalar-union fields |
| `record-boundaries` | cpu | records built from unchecked array reads passed to and returned from annotated functions |
| `number-maps` | cpu | number-keyed Map/Set: a dense-id table, sparse ids in many small maps, get/has/set mixes |
| `published-graphs` | cpu | syntax trees with parent links and a symbol table, published with `@scriptc/threads` and then read |
| `validate-errors` | cpu | throw/catch on a fraction of inputs, small call chains |
| `channel-messages` | cpu | 200,000 MessageChannel messages in one thread, half numbers and half small objects, all posted before the port delivers any |
| `csv-numbers` | cpu | `split`, `Number`, `parseInt`, `toFixed` |
| `date-math` | cpu | schedules built with Date setters (local and UTC), weekday and month-end rules through the getters, Dates sorted by time value |
| `cyclic-graphs` | cpu | object graphs with back references built and dropped: trees with parent and sibling links, rings of records sharing a long-lived registry, closures capturing their owner |
| `dictionary-walks` | cpu | `Record<string, number>` dictionaries walked with for-in and `Object.keys`/`values`/`entries` while entries are added, updated and deleted |
| `dynamic-methods` | cpu | untyped JavaScript calling builtin methods on values it does not type: string `slice`/`indexOf`/`toUpperCase`/`padStart`, array `push`/`pop`/`includes`, `Map` and `Set`, and an object's own methods |
| `dynamic-properties` | cpu | untyped JavaScript reading and writing properties of values it does not type: records built by object literals in two key orders, nested records, and instances of a class and its subclass |
| `functional-pipeline` | cpu | `filter`/`map`/`reduce`/`flatMap` chains, closures, small records |
| `handler-tables` | cpu | dispatch tables of functions: `Record<string, fn>` read as `t[k]!(…)`, `t[k]?.(…)` and through a checked local, a `Map<string, fn>`, an array of handlers, and an object literal of methods; some handlers declare fewer parameters than the table type |
| `intern-table`, `intern-table-map`, `intern-table-1`, `intern-table-map-1` | cpu | worker threads interning the same type-like objects under string keys and records under number keys, then hits and misses: one shared `InternTable` per kind of key, or a module-level `Map` per thread (the `-map` entries), with 8 threads or 1 (the `-1` entries); under Node.js every thread has its own tables |
| `json-records` | cpu | `JSON.stringify`, typed `JSON.parse`, Map aggregation |
| `json-unions` | cpu | `JSON.parse` into discriminated unions: an event log with eight kinds and nested expression trees with nine, evaluated after the cast |
| `keyed-option-writes` | cpu | class methods whose names are also keys of option records written through element accesses: `enabled[name] = true` over a union of literal names and `totals["count"] = n` |
| `line-reader` | cpu | `readline` over a file stream: 'line' events for a 67 MB access log (every 50th line not ASCII), with a status count, a path tally and byte totals |
| `log-lines` | cpu | 200k immediate `console.log`/`process.stdout.write` lines into the runner's pipe |
| `log-summary`, `inventory-report` | cpu | the multi-module applications from `benchmarks/builds`, on 300k-line inputs |
| `number-maps` | cpu | number-keyed Map/Set: a dense-id table, sparse ids in many small maps, get/has/set mixes |
| `numeric-kernels` | cpu | sieve, hashing, `number[][]` matmul, `Float64Array` |
| `optional-fields` | cpu | records and class instances with `number \| undefined` and other scalar-union fields |
| `published-graphs` | cpu | syntax trees with parent links and a symbol table, published with `@scriptc/threads` and then read |
| `record-untyped-writes` | cpu | typed records written through untyped views: a counter helper taking `unknown` bumps fields of a stats record, and `Object.assign` merges patches into a settings record, with typed reads in between |
| `record-boundaries` | cpu | records built from unchecked array reads passed to and returned from annotated functions |
| `records-sort` | cpu | comparator sorts, group-by, `reduce`/`filter` |
| `regex-logs` | cpu | regex `exec` with groups, `test`, global `replace` |
| `regex-replacers` | cpu | regex `replace`/`replaceAll` with inline callbacks whose parameters have no annotations: dictionary placeholders, snake_case to camelCase, date reordering, optional captures, capitalization, and a string search that reads the offset |
| `region-builds` | cpu | a watch-mode build loop over `LifetimeRegion`: each build binds a program in a new region and releases the previous one, which a cache keeps referenced for two builds |
| `string-iteration` | cpu | code point iteration over text that mixes ASCII, Latin-1 letters, CJK and emoji: a for-of word tokenizer, spread reversal, and `Array.from` over words |
| `template-render` | cpu | template literals, escaping, `map`/`join` |
| `throw-chains` | cpu | exceptions caught 2, 5 and 20 recursive calls above the throw, through direct calls and through a callback, and a backtracking parser that throws on a failed alternative |
| `thread-slots`, `thread-slots-cache` | cpu | worker threads reading one published type graph from an `InternTable` and keeping per-thread state for each type they visit: a `ThreadSlots` family, or a direct-mapped cache by id in front of a `Map` by id (the `-cache` entry); under Node.js both are per thread |
| `typed-array-kernels` | cpu | element-wise loops that run to one typed array's length and index others at the same position: `Float64Array` axpy, `Float32Array` blend, `Uint8Array` mask, wrapping `Int32Array` add, `Uint8ClampedArray` brighten, RGBA to gray, a row-major `Float32Array` matrix-vector product, and `reduce`/`some`/`map` callbacks |
| `typed-array-scan`, `typed-array-scan-workers` | cpu | element-by-element scan loops over `Uint8Array`, `Uint16Array`, `Int32Array` and `Float64Array`; the second entry is a worker-enabled executable (it can start a worker), so each access also checks for shared storage |
| `unicode-lexer` | cpu | a hand-written lexer over a 2.7M-unit bundled module whose comments and strings are in several languages: `charCodeAt` scans, line starts |
| `union-node-sets` | cpu | syntax-tree passes over a discriminated union of object literals: a `Set` of visited nodes, a `Map` from node to value, conversions to a sub-union of leaves |
| `validate-errors` | cpu | throw/catch on a fraction of inputs, small call chains |
| `value-objects` | cpu | small immutable records and class instances: vector helpers, method chains, particle updates |
| `value-records` | cpu | records that hold strings, spread state updates and destructured tuples: token helpers, an immutable reducer, tuple returns |
| `word-graph` | cpu | string-keyed Map/Set, BFS |
| `worker-messages` | cpu | message round trips between the main thread and one worker: each posts a number and the other answers it, so every round trip is two messages and two wakeups of a waiting thread |
| `cli-config` | startup | a `deploy-plan` CLI: read a ~110 KB JSON manifest, validate, interpolate env values, order 80 services into dependency waves, print a summary |
| `http-api` | server | a `node:http` JSON API: routing, query parsing, a seeded catalog, per-cart state, typed JSON bodies, `ETag`/`If-None-Match`, auth and request-id headers |
@@ -43,7 +62,7 @@ Node is timed in the same interleaved loop and every row reports candidate/Node;
## Workload kinds
Entries in `workloads.json` are process workloads unless they set `kind`.
Each workload is a JSON file in `workloads/`, named after the workload, that holds its `entry`, `args` and options; adding a workload adds its file and its row in the table above, kept in category and then name order. Workloads are process workloads unless they set `kind`.
Process workloads time whole launches (wall time, startup included). An optional `launches` count makes each sample the median of that many sequential launches; `cli-config` uses 20 so that a startup-dominated command (a few milliseconds compiled, tens of milliseconds under Node) is measured with low noise. Its table row is labeled `(per launch)`. On a heavily loaded macOS host, whole batches of launches can run several times slower (samples become bimodal), so judge startup changes on the Linux lane or a quiet machine.
benchmarks/runtime/workloads.json
-249
View File
@@ -1,249 +0,0 @@
{
"workloads": [
{
"name": "json-records",
"entry": "workloads/json-records.ts",
"args": [
"4"
]
},
{
"name": "regex-logs",
"entry": "workloads/regex-logs.ts",
"args": [
"1"
]
},
{
"name": "ast-interp",
"entry": "workloads/ast-interp.ts",
"args": [
"5"
]
},
{
"name": "records-sort",
"entry": "workloads/records-sort.ts",
"args": [
"1"
]
},
{
"name": "template-render",
"entry": "workloads/template-render.ts",
"args": [
"1"
]
},
{
"name": "functional-pipeline",
"entry": "workloads/functional-pipeline.ts",
"args": [
"1"
]
},
{
"name": "async-pipeline",
"entry": "workloads/async-pipeline.ts",
"args": [
"1"
]
},
{
"name": "word-graph",
"entry": "workloads/word-graph.ts",
"args": [
"2"
]
},
{
"name": "alloc-trees",
"entry": "workloads/alloc-trees.ts",
"args": [
"1"
]
},
{
"name": "cyclic-graphs",
"entry": "workloads/cyclic-graphs.ts",
"args": [
"3"
]
},
{
"name": "numeric-kernels",
"entry": "workloads/numeric-kernels.ts",
"args": [
"0.5"
]
},
{
"name": "typed-array-scan",
"entry": "workloads/typed-array-scan.ts",
"args": [
"3"
]
},
{
"name": "typed-array-scan-workers",
"entry": "workloads/typed-array-scan-workers.ts",
"args": [
"3"
]
},
{
"name": "intern-table",
"entry": "workloads/intern-table.ts",
"args": [
"table",
"8",
"100000"
]
},
{
"name": "intern-table-map",
"entry": "workloads/intern-table.ts",
"args": [
"map",
"8",
"100000"
]
},
{
"name": "intern-table-1",
"entry": "workloads/intern-table.ts",
"args": [
"table",
"1",
"100000"
]
},
{
"name": "intern-table-map-1",
"entry": "workloads/intern-table.ts",
"args": [
"map",
"1",
"100000"
]
},
{
"name": "atomics-contended",
"entry": "workloads/atomics-contended.ts",
"args": [
"2"
]
},
{
"name": "value-objects",
"entry": "workloads/value-objects.ts",
"args": [
"2"
]
},
{
"name": "region-builds",
"entry": "workloads/region-builds.ts",
"args": [
"1"
]
},
{
"name": "optional-fields",
"entry": "workloads/optional-fields.ts",
"args": [
"10"
]
},
{
"name": "record-boundaries",
"entry": "workloads/record-boundaries.ts",
"args": [
"8"
]
},
{
"name": "number-maps",
"entry": "workloads/number-maps.ts",
"args": [
"4"
]
},
{
"name": "published-graphs",
"entry": "workloads/published-graphs.ts",
"args": [
"1"
]
},
{
"name": "validate-errors",
"entry": "workloads/validate-errors.ts",
"args": [
"1"
]
},
{
"name": "csv-numbers",
"entry": "workloads/csv-numbers.ts",
"args": [
"1"
]
},
{
"name": "log-lines",
"entry": "workloads/log-lines.ts",
"args": [
"4"
]
},
{
"name": "log-summary",
"entry": "../builds/log-summary/main.ts",
"input": "log-summary",
"args": [
"{input}",
"--min-status",
"200",
"--limit",
"12"
]
},
{
"name": "inventory-report",
"entry": "../builds/inventory-report/main.ts",
"input": "inventory-report",
"args": [
"{input}"
]
},
{
"name": "cli-config",
"entry": "workloads/cli-config.ts",
"input": "cli-config",
"launches": 20,
"args": [
"{input}",
"--env",
"production",
"--max-wave",
"12"
]
},
{
"name": "http-api",
"kind": "server",
"entry": "workloads/http-api.ts",
"args": [],
"client": {
"script": "clients/http-load.mjs",
"args": [
"--connections=64",
"--threads=2",
"--warmup=4000",
"--requests=40000"
]
}
}
]
}
benchmarks/runtime/workloads.mjs
+33
View File
@@ -0,0 +1,33 @@
/* The runtime benchmark workloads: one JSON file per workload in
* workloads/, named after the workload, next to the entry programs. A file
* holds the workload's entry, args and options; changes that add
* different workloads add different files. */
import { existsSync, readdirSync, readFileSync } from "node:fs";
import { join } from "node:path";
const CATEGORIES = ["cpu", "startup", "server"];
/* Server workloads run a listening program; startup-bound ones time
* several sequential launches per sample. */
export const workloadCategory = (w) =>
w.kind === "server" ? "server" : (w.launches ?? 1) > 1 ? "startup" : "cpu";
/* The workloads of the suite at `suite` (a benchmarks/runtime directory),
* by category and then name. A checkout from before the per-workload files
* lists them in workloads.json, in its own order. */
export function readWorkloads(suite) {
const legacy = join(suite, "workloads.json");
if (existsSync(legacy)) return JSON.parse(readFileSync(legacy, "utf8")).workloads;
const dir = join(suite, "workloads");
return readdirSync(dir)
.filter((file) => file.endsWith(".json"))
.map((file) => ({
name: file.slice(0, -".json".length),
...JSON.parse(readFileSync(join(dir, file), "utf8")),
}))
.sort(
(a, b) =>
CATEGORIES.indexOf(workloadCategory(a)) - CATEGORIES.indexOf(workloadCategory(b)) ||
(a.name < b.name ? -1 : a.name > b.name ? 1 : 0),
);
}
benchmarks/runtime/workloads/alloc-trees.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/alloc-trees.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/ast-interp.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/ast-interp.ts",
"args": [
"5"
]
}
benchmarks/runtime/workloads/async-pipeline.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/async-pipeline.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/atomics-contended.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/atomics-contended.ts",
"args": [
"2"
]
}
benchmarks/runtime/workloads/channel-messages.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/channel-messages.ts",
"args": [
"200000"
]
}
benchmarks/runtime/workloads/channel-messages.ts
+21
View File
@@ -0,0 +1,21 @@
// MessageChannel messages within one thread: the program posts the given
// number of messages, half of them numbers and half small objects, before
// the port delivers any, so all of them are queued at once. Argument: the
// number of messages.
import { MessageChannel } from "node:worker_threads";
const count = Number(process.argv[2] ?? "200000");
const { port1, port2 } = new MessageChannel();
let received = 0;
let sum = 0;
port1.on("message", (value: number | { id: number; name: string }) => {
sum += typeof value === "number" ? value : value.id + value.name.length;
if (++received === count) {
console.log(`${count} messages, sum ${sum}`);
port1.close();
}
});
for (let i = 0; i < count; i++) {
if (i % 2 === 0) port2.postMessage(i);
else port2.postMessage({ id: i, name: "item" });
}
benchmarks/runtime/workloads/cli-config.json
+12
View File
@@ -0,0 +1,12 @@
{
"entry": "workloads/cli-config.ts",
"input": "cli-config",
"launches": 20,
"args": [
"{input}",
"--env",
"production",
"--max-wave",
"12"
]
}
benchmarks/runtime/workloads/csv-numbers.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/csv-numbers.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/cyclic-graphs.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/cyclic-graphs.ts",
"args": [
"3"
]
}
benchmarks/runtime/workloads/date-math.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/date-math.ts",
"args": [
"40"
]
}
benchmarks/runtime/workloads/date-math.ts
+52
View File
@@ -0,0 +1,52 @@
// Calendar arithmetic on Dates: schedules built by adjusting Dates with the
// UTC and local setters, weekday and month-end rules read back through the
// getters, Dates compared and sorted by time value, and ISO text for a
// sample. Argument: the number of rounds.
process.env.TZ = "America/New_York";
const rounds = Number(process.argv[2] ?? "40");
let seed = 2026;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
/** The next business day at 09:00 local time on or after `from`. */
function nextBusinessDay(from: Date): Date {
const day = new Date(from.getTime());
day.setHours(9, 0, 0, 0);
if (day < from) day.setDate(day.getDate() + 1);
while (day.getDay() === 0 || day.getDay() === 6) day.setDate(day.getDate() + 1);
return day;
}
/** The last day of `date`'s month, in UTC. */
function monthEnd(date: Date): number {
const end = new Date(date.getTime());
end.setUTCDate(1);
end.setUTCMonth(end.getUTCMonth() + 1);
end.setUTCDate(0);
return end.getUTCDate();
}
let checksum = 0;
let sample = "";
for (let round = 0; round < rounds; round++) {
const due: Date[] = [];
const cursor = new Date(Date.UTC(2020 + (round % 7), 0, 1, 12));
for (let i = 0; i < 4000; i++) {
cursor.setUTCMinutes(cursor.getUTCMinutes() + Math.floor(random() * 4000));
const next = nextBusinessDay(cursor);
checksum += next.getDate() + monthEnd(next) + next.getHours();
if (i % 3 === 0) next.setMonth(next.getMonth() + 1, 15);
due.push(next);
}
due.sort((a, b) => +a - +b);
let previous = due[0]!;
for (const date of due) {
checksum += (+date - +previous) / 3600000;
previous = date;
}
sample = due[due.length >> 1]!.toISOString();
}
console.log(Math.round(checksum), sample);
benchmarks/runtime/workloads/dictionary-walks.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/dictionary-walks.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/dictionary-walks.ts
+53
View File
@@ -0,0 +1,53 @@
// String-keyed dictionaries (`Record<string, number>`) walked with for-in,
// Object.keys, Object.values and Object.entries while entries are added,
// updated and deleted: per-category totals, a pruning pass and a merge.
let seed = 777;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
const scale = Number(process.argv[2] ?? "1");
const categories = 4000;
const counts: Record<string, number> = {};
for (let i = 0; i < Math.floor(200000 * scale); i++) {
const key = "c" + Math.floor(random() * categories);
counts[key] = (counts[key] ?? 0) + 1;
}
let walked = 0;
let sum = 0;
for (let pass = 0; pass < 40; pass++) {
for (const key in counts) {
walked++;
sum += counts[key]!;
}
}
// Prune: delete the small categories during the walk, including keys the
// walk has not reached yet.
let pruned = 0;
for (const key in counts) {
const n = counts[key]!;
if (n < 45) {
delete counts[key];
pruned++;
}
const ahead = "c" + ((Number(key.slice(1)) + 1) % categories);
if (n > 60 && ahead in counts && counts[ahead]! < 45) {
delete counts[ahead];
pruned++;
}
}
// Merge into a second dictionary and read it back every way.
const merged: Record<string, number> = {};
for (const key of Object.keys(counts)) merged[key.toUpperCase()] = counts[key]! * 2;
let values = 0;
for (const v of Object.values(merged)) values += v;
let entries = 0;
for (const [key, v] of Object.entries(merged)) entries += key.length + v;
console.log(
`walked ${walked}, sum ${sum}, pruned ${pruned}, kept ${Object.keys(counts).length}, values ${values}, entries ${entries}`,
);
benchmarks/runtime/workloads/dynamic-methods.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/dynamic-methods.mjs",
"args": [
"300000"
]
}
benchmarks/runtime/workloads/dynamic-methods.mjs
+36
View File
@@ -0,0 +1,36 @@
// Untyped JavaScript calling builtin methods on values it never declares
// types for: string slice/indexOf/toUpperCase/startsWith/padStart, array
// push/pop/at/join/includes/indexOf, Map and Set get/set/has/add, number
// toFixed, and calls of an object's own methods. Argument: the number of
// iterations.
const rounds = Number(process.argv[2] ?? "300000");
const words = ["alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta"];
const counts = new Map();
const seen = new Set();
const stack = [];
const recent = [];
const tally = {
total: 0,
add(value) {
this.total += value;
return this.total;
},
};
let checksum = 0;
for (let i = 0; i < rounds; i++) {
const word = words[i % words.length];
const head = word.slice(0, 3).toUpperCase();
const key = head.padStart(4, "-");
if (word.indexOf("e") >= 0 && !word.startsWith("z")) checksum += 1;
counts.set(key, (counts.get(key) ?? 0) + 1);
seen.add(i % 97);
if (seen.has(i % 89)) checksum += 2;
stack.push(i);
if (stack.length > 32) checksum += stack.pop() % 3;
recent.push(word);
if (recent.length > 4) recent.shift();
if (recent.includes("gamma")) checksum += recent.indexOf("gamma");
checksum += tally.add(i % 5) % 11;
if (i % 1000 === 0) checksum += Number((i / 7).toFixed(2)) | 0;
}
console.log(checksum, counts.size, seen.size, stack.at(-1), recent.join(","), tally.total);
benchmarks/runtime/workloads/dynamic-properties.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/dynamic-properties.mjs",
"args": [
"1000000"
]
}
benchmarks/runtime/workloads/dynamic-properties.mjs
+56
View File
@@ -0,0 +1,56 @@
// Untyped JavaScript reading and writing properties of values it never
// declares types for: records built by object literals in two key orders,
// nested records, property updates, and instances of a class and its
// subclass read and written the same way. Argument: the number of
// iterations.
const rounds = Number(process.argv[2] ?? "1000000");
class Cursor {
constructor(x, y) {
this.x = x;
this.y = y;
}
step(dx, dy) {
this.x += dx;
this.y += dy;
return this;
}
}
class Marker extends Cursor {
constructor(x, y, label) {
super(x, y);
this.label = label;
}
}
function makeItem(i) {
if (i % 3 === 0)
return { id: i, price: i % 17, qty: i % 5, meta: { tag: i % 7, weight: i % 11 } };
return { id: i, qty: i % 5, price: i % 13, meta: { weight: i % 9, tag: i % 5 } };
}
function cost(item) {
return item.price * item.qty + item.meta.weight;
}
function restock(item, round) {
item.qty = (item.qty + round) % 7;
item.meta.tag = (item.meta.tag + 1) % 7;
}
function distance(a, b) {
return Math.abs(a.x - b.x) + Math.abs(a.y - b.y);
}
const items = [];
for (let i = 0; i < 1024; i++) items.push(makeItem(i));
const cursors = [];
for (let i = 0; i < 64; i++)
cursors.push(i % 2 === 0 ? new Cursor(i - 32, 7 - i) : new Marker(i - 32, 7 - i, "m" + i));
const origin = { x: 0, y: 0 };
let checksum = 0;
for (let r = 0; r < rounds; r++) {
const item = items[r & 1023];
checksum += cost(item);
restock(item, r & 3);
checksum += item.meta.tag;
const cursor = cursors[r & 63];
checksum += distance(cursor.step((r % 3) - 1, (r % 5) - 2), origin);
if (cursor.x > 100 || cursor.x < -100) cursor.x = 0;
if (cursor.y > 100 || cursor.y < -100) cursor.y = 0;
}
console.log(checksum, items[5].qty, items[6].meta.tag, cursors[3].x, cursors[3].y, cursors[3].label);
benchmarks/runtime/workloads/functional-pipeline.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/functional-pipeline.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/handler-tables.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/handler-tables.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/handler-tables.ts
+108
View File
@@ -0,0 +1,108 @@
// Dispatch tables of functions: a `Record<string, fn>` of opcode handlers
// called as `ops[name]!(…)`, `ops[name]?.(…)` and through a checked local,
// a `Map<string, fn>` of the same handlers, an array of handlers indexed by
// opcode number, and an object literal of methods. Some handlers declare
// fewer parameters than the table's function type.
type Op = (acc: number, arg: number) => number;
const ops: Record<string, Op> = {
add: (acc, arg) => acc + arg,
sub: (acc, arg) => acc - arg,
mul: (acc, arg) => (acc * arg) % 1000003,
neg: (acc) => -acc,
half: (acc) => Math.floor(acc / 2),
mix: (acc, arg) => (acc ^ arg) & 0xffff,
};
const byName = new Map<string, Op>(Object.entries(ops));
const byCode: Op[] = Object.values(ops);
const names = Object.keys(ops);
const methods = {
add(acc: number, arg: number): number {
return acc + arg;
},
sub(acc: number, arg: number): number {
return acc - arg;
},
mul(acc: number, arg: number): number {
return (acc * arg) % 1000003;
},
};
interface Insn {
name: string;
code: number;
arg: number;
}
function program(length: number): Insn[] {
const out: Insn[] = [];
let seed = 12345;
for (let i = 0; i < length; i++) {
seed = (seed * 1103515245 + 12345) % 2147483648;
const code = seed % (names.length + 1);
// One in seven instructions names an opcode no table holds.
out.push({ name: code < names.length ? names[code]! : "nop", code, arg: (seed >> 8) % 97 });
}
return out;
}
function runAsserted(insns: Insn[]): number {
let acc = 1;
for (const insn of insns) if (insn.name !== "nop") acc = ops[insn.name]!(acc, insn.arg);
return acc;
}
function runOptional(insns: Insn[]): number {
let acc = 1;
for (const insn of insns) acc = ops[insn.name]?.(acc, insn.arg) ?? acc + 1;
return acc;
}
function runChecked(insns: Insn[]): number {
let acc = 1;
for (const insn of insns) {
const handler = ops[insn.name];
if (handler) acc = handler(acc, insn.arg);
}
return acc;
}
function runMap(insns: Insn[]): number {
let acc = 1;
for (const insn of insns) {
const handler = byName.get(insn.name);
if (handler !== undefined) acc = handler(acc, insn.arg);
}
return acc;
}
function runArray(insns: Insn[]): number {
let acc = 1;
for (const insn of insns) if (insn.code < byCode.length) acc = byCode[insn.code]!(acc, insn.arg);
return acc;
}
function runMethods(insns: Insn[]): number {
let acc = 1;
for (const insn of insns) {
if (insn.code === 0) acc = methods.add(acc, insn.arg);
else if (insn.code === 1) acc = methods.sub(acc, insn.arg);
else if (insn.code === 2) acc = methods["mul"](acc, insn.arg);
}
return acc;
}
const rounds = Number(process.argv[2] ?? "4");
const insns = program(200_000);
const totals = [0, 0, 0, 0, 0, 0];
for (let round = 0; round < rounds * 10; round++) {
totals[0] = (totals[0]! + runAsserted(insns)) % 1e9;
totals[1] = (totals[1]! + runOptional(insns)) % 1e9;
totals[2] = (totals[2]! + runChecked(insns)) % 1e9;
totals[3] = (totals[3]! + runMap(insns)) % 1e9;
totals[4] = (totals[4]! + runArray(insns)) % 1e9;
totals[5] = (totals[5]! + runMethods(insns)) % 1e9;
}
console.log(totals.join(" "));
benchmarks/runtime/workloads/http-api.json
+14
View File
@@ -0,0 +1,14 @@
{
"kind": "server",
"entry": "workloads/http-api.ts",
"args": [],
"client": {
"script": "clients/http-load.mjs",
"args": [
"--connections=64",
"--threads=2",
"--warmup=4000",
"--requests=40000"
]
}
}
benchmarks/runtime/workloads/index-cursors.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/index-cursors.ts",
"args": [
"20000"
]
}
benchmarks/runtime/workloads/index-cursors.ts
+25
View File
@@ -0,0 +1,25 @@
// Arrays of indices walked by loop-carried cursors: each step reads the
// next index from the array and relinks the visited entries, so the
// cursor and every stored value are small integers read back from the
// array itself.
const N = 4096;
function relink(rounds: number): number {
const next: number[] = [];
for (let i = 0; i < N; i++) next.push(i + 1 < N ? i + 1 : 0);
let at = next[0]!;
let total = 0;
for (let r = 0; r < rounds; r++) {
for (let s = 0; s < N; s++) {
const t = next[at]!;
next[at] = next[t]!;
next[t] = at;
at = t;
total = (total + at) | 0;
}
}
return total;
}
const rounds = Number(process.argv[2] ?? "1");
console.log("relink", relink(rounds));
benchmarks/runtime/workloads/intern-table-1.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"table",
"1",
"100000"
]
}
benchmarks/runtime/workloads/intern-table-map-1.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"map",
"1",
"100000"
]
}
benchmarks/runtime/workloads/intern-table-map.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"map",
"8",
"100000"
]
}
benchmarks/runtime/workloads/intern-table.json
+8
View File
@@ -0,0 +1,8 @@
{
"entry": "workloads/intern-table.ts",
"args": [
"table",
"8",
"100000"
]
}
benchmarks/runtime/workloads/inventory-report.json
+7
View File
@@ -0,0 +1,7 @@
{
"entry": "../builds/inventory-report/main.ts",
"input": "inventory-report",
"args": [
"{input}"
]
}
benchmarks/runtime/workloads/json-records.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/json-records.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/json-unions.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/json-unions.ts",
"args": [
"6"
]
}
benchmarks/runtime/workloads/json-unions.ts
+155
View File
@@ -0,0 +1,155 @@
// JSON text parsed into discriminated unions: an event log with eight kinds
// of event and a nested expression tree with nine kinds of node, cast with
// `JSON.parse(text) as T`, then aggregated and evaluated. Argument: the
// number of rounds.
type Event =
| { kind: "click"; target: string; x: number; y: number }
| { key: string; kind: "key"; repeat: boolean }
| { kind: "scroll"; offset: number }
| { kind: "resize"; height: number; width: number }
| { kind: "focus"; target: string }
| { kind: "blur"; target: string }
| { kind: "load"; ms: number; url: string }
| { code: number; kind: "error"; message: string };
type Expr =
| { kind: "num"; value: number }
| { kind: "var"; name: string }
| { kind: "neg"; operand: Expr }
| { kind: "add"; left: Expr; right: Expr }
| { kind: "sub"; left: Expr; right: Expr }
| { kind: "mul"; left: Expr; right: Expr }
| { kind: "max"; items: Expr[] }
| { else: Expr; kind: "if"; test: Expr; then: Expr }
| { body: Expr; kind: "let"; name: string; value: Expr };
let seed = 12345;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
const targets = ["menu", "search", "button", "list", "footer"];
function eventText(i: number): string {
const target = targets[i % targets.length]!;
switch (i % 8) {
case 0:
return `{"kind":"click","target":"${target}","x":${i % 640},"y":${i % 480}}`;
case 1:
return `{"key":"k${i % 26}","kind":"key","repeat":${i % 3 === 0}}`;
case 2:
return `{"kind":"scroll","offset":${(i * 7) % 2000}}`;
case 3:
return `{"height":${400 + (i % 300)},"kind":"resize","width":${600 + (i % 500)}}`;
case 4:
return `{"kind":"focus","target":"${target}"}`;
case 5:
return `{"kind":"blur","target":"${target}"}`;
case 6:
return `{"kind":"load","ms":${i % 900},"url":"/page/${i % 50}"}`;
default:
return `{"code":${400 + (i % 100)},"kind":"error","message":"failed ${i % 13}"}`;
}
}
const names = ["a", "b", "c", "d"];
function exprText(depth: number): string {
const pick = depth <= 0 ? Math.floor(random() * 2) : Math.floor(random() * 9);
switch (pick) {
case 0:
return `{"kind":"num","value":${Math.floor(random() * 100)}}`;
case 1:
return `{"kind":"var","name":"${names[Math.floor(random() * names.length)]!}"}`;
case 2:
return `{"kind":"neg","operand":${exprText(depth - 1)}}`;
case 3:
case 4:
case 5: {
const kind = pick === 3 ? "add" : pick === 4 ? "sub" : "mul";
return `{"kind":"${kind}","left":${exprText(depth - 1)},"right":${exprText(depth - 1)}}`;
}
case 6:
return `{"items":[${exprText(depth - 1)},${exprText(depth - 2)},${exprText(depth - 2)}],"kind":"max"}`;
case 7:
return `{"else":${exprText(depth - 2)},"kind":"if","test":${exprText(depth - 1)},"then":${exprText(depth - 2)}}`;
default:
return `{"body":${exprText(depth - 1)},"kind":"let","name":"${names[Math.floor(random() * names.length)]!}","value":${exprText(depth - 2)}}`;
}
}
function evaluate(e: Expr, env: Map<string, number>): number {
switch (e.kind) {
case "num":
return e.value;
case "var":
return env.get(e.name) ?? 1;
case "neg":
return -evaluate(e.operand, env);
case "add":
return (evaluate(e.left, env) + evaluate(e.right, env)) % 1000003;
case "sub":
return (evaluate(e.left, env) - evaluate(e.right, env)) % 1000003;
case "mul":
return (evaluate(e.left, env) * evaluate(e.right, env)) % 1000003;
case "max": {
let best = -Infinity;
for (const item of e.items) best = Math.max(best, evaluate(item, env));
return best;
}
case "if":
return evaluate(e.test, env) > 0 ? evaluate(e.then, env) : evaluate(e.else, env);
case "let": {
const inner = new Map(env);
inner.set(e.name, evaluate(e.value, env));
return evaluate(e.body, inner);
}
}
}
const rounds = Number(process.argv[2] ?? "6");
const eventsText = `[${Array.from({ length: 60000 }, (_, i) => eventText(i)).join(",")}]`;
const treesText = `[${Array.from({ length: 400 }, () => exprText(9)).join(",")}]`;
let clicks = 0;
let keys = 0;
let scrolled = 0;
let area = 0;
let loadMs = 0;
let errors = 0;
let focus = 0;
let total = 0;
for (let round = 0; round < rounds; round++) {
const events = JSON.parse(eventsText) as Event[];
for (const event of events) {
switch (event.kind) {
case "click":
clicks += event.x + event.y;
break;
case "key":
if (event.repeat) keys++;
break;
case "scroll":
scrolled += event.offset;
break;
case "resize":
area += (event.width * event.height) % 1000;
break;
case "focus":
case "blur":
focus += event.target.length;
break;
case "load":
loadMs += event.ms;
break;
case "error":
errors += event.code % 7;
break;
}
}
const trees = JSON.parse(treesText) as Expr[];
const env = new Map([
["a", 3],
["b", 5],
]);
for (const tree of trees) total = (total + evaluate(tree, env)) % 1000003;
}
console.log(clicks, keys, scrolled, area, loadMs, errors, focus, total);
benchmarks/runtime/workloads/keyed-option-writes.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/keyed-option-writes.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/keyed-option-writes.ts
+76
View File
@@ -0,0 +1,76 @@
// Class methods whose names are also keys of option records written through
// element accesses: `enabled[name] = true` over a union of literal names and
// `totals["count"] = n`. The writes store booleans and numbers into plain
// records, so the methods keep their native calls.
type Step = "scale" | "offset" | "clamp";
const enabled: Record<Step, boolean> = { scale: false, offset: false, clamp: false };
function enable(name: Step): void {
enabled[name] = true;
}
const totals: Record<string, number> = {};
class Transform {
private readonly factor: number;
private readonly shift: number;
constructor(factor: number, shift: number) {
this.factor = factor;
this.shift = shift;
}
scale(x: number): number {
return enabled.scale ? x * this.factor : x;
}
offset(x: number): number {
return enabled.offset ? x + this.shift : x;
}
clamp(x: number): number {
if (!enabled.clamp) return x;
return x < 0 ? 0 : x > 1000 ? 1000 : x;
}
}
class Counter {
private seen = 0;
private sum = 0;
add(value: number): void {
this.seen++;
this.sum += value;
}
count(): number {
return this.seen;
}
mean(): number {
return this.seen === 0 ? 0 : this.sum / this.seen;
}
}
const scale = Number(process.argv[2] ?? "1");
const steps: Step[] = ["scale", "offset", "clamp"];
for (const step of steps) enable(step);
const transforms: Transform[] = [];
for (let i = 0; i < 16; i++) transforms.push(new Transform(1 + (i % 5) * 0.25, (i % 7) - 3));
const counters: Counter[] = [];
for (let i = 0; i < 8; i++) counters.push(new Counter());
let seed = 12345;
let checksum = 0;
const rounds = Math.floor(2_000_000 * scale);
for (let i = 0; i < rounds; i++) {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
const t = transforms[seed & 15]!;
const value = t.clamp(t.offset(t.scale((seed >>> 8) % 1200)));
const counter = counters[(seed >>> 4) & 7]!;
counter.add(value);
checksum += value;
if ((i & 65535) === 0) totals["count"] = counter.count();
}
let mean = 0;
let seen = 0;
for (const counter of counters) {
mean += counter.mean();
seen += counter.count();
}
console.log("seen", seen, "count", totals["count"], "mean", (mean / counters.length).toFixed(4), "checksum", checksum);
benchmarks/runtime/workloads/line-reader.json
+7
View File
@@ -0,0 +1,7 @@
{
"entry": "workloads/line-reader.ts",
"input": "line-reader",
"args": [
"{input}"
]
}
benchmarks/runtime/workloads/line-reader.ts
+34
View File
@@ -0,0 +1,34 @@
// Line-by-line processing of a log file with readline over a file stream:
// 'line' events for about 67 MB of access-log lines, with light work per
// line (a status count, a path tally, byte totals).
import { createReadStream } from "node:fs";
import { createInterface } from "node:readline";
const path = process.argv[2]!;
const statuses = new Map<string, number>();
const paths = new Map<string, number>();
let lines = 0;
let chars = 0;
let bytes = 0;
let longest = 0;
const input = createInterface({ input: createReadStream(path), crlfDelay: Infinity });
input.on("line", (line: string) => {
lines++;
chars += line.length;
if (line.length > longest) longest = line.length;
const quote = line.indexOf('"');
const space = line.indexOf(" ", quote);
const path = line.slice(space + 1, line.indexOf("/", line.indexOf("/", space) + 1));
paths.set(path, (paths.get(path) ?? 0) + 1);
const close = line.indexOf('"', space);
const status = line.slice(close + 2, close + 5);
statuses.set(status, (statuses.get(status) ?? 0) + 1);
bytes += Number(line.slice(close + 6, line.indexOf(" ", close + 6)));
});
input.on("close", () => {
console.log(`${lines} lines, ${chars} characters, longest ${longest}`);
console.log(`bytes ${bytes}`);
for (const status of [...statuses.keys()].sort()) console.log(`status ${status}: ${statuses.get(status)}`);
for (const prefix of [...paths.keys()].sort()) console.log(`path ${prefix}: ${paths.get(prefix)}`);
});
benchmarks/runtime/workloads/log-lines.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/log-lines.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/log-summary.json
+11
View File
@@ -0,0 +1,11 @@
{
"entry": "../builds/log-summary/main.ts",
"input": "log-summary",
"args": [
"{input}",
"--min-status",
"200",
"--limit",
"12"
]
}
benchmarks/runtime/workloads/number-maps.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/number-maps.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/numeric-kernels.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/numeric-kernels.ts",
"args": [
"0.5"
]
}
benchmarks/runtime/workloads/optional-fields.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/optional-fields.ts",
"args": [
"10"
]
}
benchmarks/runtime/workloads/published-graphs.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/published-graphs.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/record-boundaries.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/record-boundaries.ts",
"args": [
"8"
]
}
benchmarks/runtime/workloads/record-untyped-writes.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/record-untyped-writes.ts",
"args": [
"4"
]
}
benchmarks/runtime/workloads/record-untyped-writes.ts
+49
View File
@@ -0,0 +1,49 @@
// Typed records updated through untyped views: a stats record written by a
// generic counter helper that takes `unknown`, settings merged with
// Object.assign, and typed reads in between. Each write stores into the
// native record; typed code reads the same object.
type Stats = { hits: number; misses: number; bytes: number; name: string; ok: boolean };
type Settings = { host: string; port: number; debug: boolean; retries: number };
const scale = Number(process.argv[2] ?? "1");
function bump(target: unknown, key: string, by: number): void {
const record = target as Record<string, number>;
record[key] = record[key]! + by;
}
const stats: Stats[] = [];
for (let i = 0; i < 16; i++) stats.push({ hits: 0, misses: 0, bytes: 0, name: "s" + i, ok: true });
let seed = 12345;
let checksum = 0;
const rounds = Math.floor(1_000_000 * scale);
for (let i = 0; i < rounds; i++) {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
const s = stats[seed & 15]!;
if ((seed >>> 8) % 5 === 0) bump(s, "misses", 1);
else bump(s, "hits", 1);
bump(s, "bytes", (seed >>> 12) & 1023);
checksum += s.hits - s.misses;
}
const settings: Settings = { host: "localhost", port: 80, debug: false, retries: 0 };
const view: object = settings;
const patches: object[] = [
{ port: 8080 },
{ debug: true, retries: 3 },
{ host: "example.test" },
{ port: 443, debug: false },
];
for (let i = 0; i < Math.floor(200_000 * scale); i++) {
Object.assign(view, patches[i & 3]!);
checksum += settings.port + settings.retries;
}
let hits = 0;
let bytes = 0;
for (const s of stats) {
hits += s.hits;
bytes += s.bytes;
}
console.log("hits", hits, "bytes", bytes, "settings", JSON.stringify(settings), "checksum", checksum);
benchmarks/runtime/workloads/records-sort.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/records-sort.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/regex-logs.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/regex-logs.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/regex-replacers.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/regex-replacers.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/regex-replacers.ts
+60
View File
@@ -0,0 +1,60 @@
// Regex replace and replaceAll with inline callbacks, written as they
// usually are, without parameter annotations: template placeholders looked
// up in a dictionary, snake_case to camelCase, date reordering, optional
// version parts, capitalization and a string search with its offset.
let seed = 7;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
function pick<T>(items: readonly T[]): T {
return items[Math.floor(random() * items.length)]!;
}
const words = ["user", "order", "item", "cart", "price", "total", "status", "created", "updated", "id"];
const names = ["Ada", "Grace", "Linus", "Barbara", "Ken", "Margaret"];
const vars: Record<string, string> = {
name: "world",
city: "Lisbon",
day: "Tuesday",
plan: "pro",
count: "12",
};
const keys = [...Object.keys(vars), "missing"];
const scale = Number(process.argv[2] ?? "1");
const count = Math.floor(20000 * scale);
const templates: string[] = [];
const identifiers: string[] = [];
const dates: string[] = [];
const versions: string[] = [];
const sentences: string[] = [];
for (let i = 0; i < count; i++) {
templates.push(`Hello {${pick(keys)}}, your {${pick(keys)}} plan renews on {${pick(keys)}} in {${pick(keys)}}.`);
identifiers.push(`${pick(words)}_${pick(words)}_${pick(words)}${random() < 0.5 ? "_" + pick(words) : ""}`);
const year = 1990 + Math.floor(random() * 40);
const month = String(1 + Math.floor(random() * 12)).padStart(2, "0");
const day = String(1 + Math.floor(random() * 28)).padStart(2, "0");
dates.push(`from ${year}-${month}-${day} to ${year + 1}-${month}-${day} (${pick(names)})`);
versions.push(`v${i % 9}.${i % 13}${random() < 0.5 ? "." + (i % 7) : ""} and v${i % 5}.${i % 3}`);
sentences.push(`${pick(names)} said the ${pick(words)} and the ${pick(words)} were ready for ${pick(names)}`);
}
let length = 0;
let checksum = 0;
for (let round = 0; round < 4; round++) {
for (let i = 0; i < count; i++) {
const filled = templates[i]!.replace(/\{(\w+)\}/g, (_m, key) => vars[key] ?? `<${key}>`);
const camel = identifiers[i]!.replace(/_([a-z])/g, (_m, c) => c.toUpperCase());
const reordered = dates[i]!.replace(/(\d{4})-(\d{2})-(\d{2})/g, (_m, y, m, d) => `${d}/${m}/${y}`);
const padded = versions[i]!.replace(/v(\d+)\.(\d+)(?:\.(\d+))?/g, (_m, major, minor, patch) => `v${major}.${minor}.${patch ?? "0"}`);
const titled = sentences[i]!.replace(/\b[a-z]/g, (c) => c.toUpperCase());
const marked = sentences[i]!.replaceAll("the", (m, at) => (at % 2 === 0 ? m.toUpperCase() : m));
length += filled.length + camel.length + reordered.length + padded.length + titled.length + marked.length;
checksum = (checksum * 31 + camel.charCodeAt(camel.length - 1) + padded.charCodeAt(padded.length - 1)) >>> 0;
}
}
console.log(templates[0]!.replace(/\{(\w+)\}/g, (_m, key) => vars[key] ?? `<${key}>`));
console.log(identifiers[1]!.replace(/_([a-z])/g, (_m, c) => c.toUpperCase()));
console.log(versions[2]!.replace(/v(\d+)\.(\d+)(?:\.(\d+))?/g, (_m, major, minor, patch) => `v${major}.${minor}.${patch ?? "0"}`));
console.log("length", length, "checksum", checksum);
benchmarks/runtime/workloads/region-builds.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/region-builds.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/string-iteration.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/string-iteration.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/string-iteration.ts
+59
View File
@@ -0,0 +1,59 @@
// Code point iteration over mixed text: a word tokenizer over for-of, a
// character histogram, reversal through spread, and Array.from over
// words. The text mixes ASCII, Latin-1 letters, CJK and emoji.
let seed = 11;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
const pieces = [
"the", "quick", "brown", "fox", "jumps", "over", "lazy", "dog",
"café", "naïve", "über", "señor", "straße", "déjà", "vu",
"日本", "語", "テキスト", "中文", "한국어",
"😀", "🚀", "👍", "🎉",
];
const punctuation = [" ", " ", " ", ", ", ". ", "! ", "\n"];
const scale = Number(process.argv[2] ?? "1");
const lines: string[] = [];
for (let i = 0; i < Math.floor(4000 * scale); i++) {
let line = "";
const words = 6 + Math.floor(random() * 10);
for (let w = 0; w < words; w++) {
line += pieces[Math.floor(random() * pieces.length)]!;
line += punctuation[Math.floor(random() * punctuation.length)]!;
}
lines.push(line);
}
function isWordChar(c: string): boolean {
const code = c.codePointAt(0)!;
return (code >= 0x61 && code <= 0x7a) || (code >= 0x41 && code <= 0x5a) || code >= 0xc0;
}
let tokens = 0;
let tokenUnits = 0;
let reversedLength = 0;
let fromLength = 0;
const histogram = new Map<string, number>();
for (let round = 0; round < 30; round++) {
for (const line of lines) {
let word = "";
for (const c of line) {
if (isWordChar(c)) {
word += c;
continue;
}
if (word !== "") {
tokens++;
tokenUnits += word.length;
if (round === 0) histogram.set(word, (histogram.get(word) ?? 0) + 1);
fromLength += Array.from(word).length;
word = "";
}
}
reversedLength += [...line].reverse().join("").length;
}
}
const top = [...histogram].sort((a, b) => b[1] - a[1] || (a[0] < b[0] ? -1 : 1)).slice(0, 5);
console.log(top.map(([word, count]) => `${word}:${count}`).join(" "));
console.log("tokens", tokens, "units", tokenUnits, "reversed", reversedLength, "from", fromLength);
benchmarks/runtime/workloads/template-render.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/template-render.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/thread-slots-cache.json
+9
View File
@@ -0,0 +1,9 @@
{
"entry": "workloads/thread-slots.ts",
"args": [
"cache",
"8",
"50000",
"20"
]
}
benchmarks/runtime/workloads/thread-slots.json
+9
View File
@@ -0,0 +1,9 @@
{
"entry": "workloads/thread-slots.ts",
"args": [
"slots",
"8",
"50000",
"20"
]
}
benchmarks/runtime/workloads/thread-slots.ts
+122
View File
@@ -0,0 +1,122 @@
// Per-thread state for shared objects: worker threads read the same
// published type graph from an InternTable and keep per-thread state for
// each type they visit (a resolved flag and a visit count), then walk the
// graph many times. Arguments: the store ("slots" for a ThreadSlots family,
// "cache" for a direct-mapped cache by id in front of a Map by id, the
// layout programs use without slots), the thread count, the type count and
// the number of walks. Under Node.js both stores are per-thread.
import { Worker, isMainThread, parentPort, workerData } from "node:worker_threads";
import { InternTable, ThreadSlots } from "@scriptc/experimental";
class Type {
readonly id: number;
readonly members: readonly Type[];
constructor(id: number, members: readonly Type[]) {
this.id = id;
this.members = members;
}
}
class TypeState {
resolved = false;
visits = 0;
}
const types = new InternTable<number, Type>("bench.slot-types");
const states = new ThreadSlots<Type, TypeState>("bench.type-states");
const CACHE = 1024;
const cacheIds: number[] = new Array<number>(CACHE).fill(-1);
const cacheStates: (TypeState | undefined)[] = new Array<TypeState | undefined>(CACHE).fill(
undefined,
);
const byId = new Map<number, TypeState>();
function slotState(t: Type): TypeState {
let state = states.get(t);
if (state === undefined) {
state = new TypeState();
states.set(t, state);
}
return state;
}
function cachedState(t: Type): TypeState {
const slot = t.id & (CACHE - 1);
if (cacheIds[slot] === t.id) return cacheStates[slot]!;
let state = byId.get(t.id);
if (state === undefined) {
state = new TypeState();
byId.set(t.id, state);
}
cacheIds[slot] = t.id;
cacheStates[slot] = state;
return state;
}
function typeOf(id: number): Type {
return types.getOrInsertComputed(id, () => {
const members: Type[] = [];
if (id >= 3) members.push(typeOf(Math.floor(id / 3)));
if (id >= 7) members.push(typeOf(id - 7));
return new Type(id, members);
});
}
function run(useSlots: boolean, count: number, walks: number): string {
const all: Type[] = [];
for (let id = 0; id < count; id++) all.push(typeOf(id));
// Visits follow member edges from a pseudo-random start, as a checker's
// relation checks do: neighbours are often visited together.
let seed = 1;
let resolved = 0;
let visits = 0;
for (let w = 0; w < walks; w++) {
for (let i = 0; i < count; i++) {
seed = (seed * 48271) % 2147483647;
let t = all[seed % count]!;
for (let depth = 0; depth < 4; depth++) {
const state = useSlots ? slotState(t) : cachedState(t);
if (!state.resolved) {
state.resolved = true;
resolved++;
}
state.visits++;
visits++;
if (t.members.length === 0) break;
t = t.members[depth % t.members.length]!;
}
}
}
return `${resolved} resolved, ${visits} visits`;
}
if (isMainThread) {
const store = process.argv[2] ?? "slots";
const threads = Number(process.argv[3] ?? "8");
const count = Number(process.argv[4] ?? "50000");
const walks = Number(process.argv[5] ?? "20");
const results: string[] = [];
let pending = threads;
for (let i = 0; i < threads; i++) {
const worker = new Worker(new URL(import.meta.url), {
workerData: { useSlots: store === "slots", count, walks },
});
worker.on("message", (text: string) => {
results[i] = text;
});
worker.on("exit", () => {
if (--pending === 0) {
const same = results.every((r) => r === results[0]);
console.log(`${threads} threads: ${results[0]}, same in every thread ${same}`);
}
});
}
} else {
const { useSlots, count, walks } = workerData as {
useSlots: boolean;
count: number;
walks: number;
};
parentPort!.postMessage(run(useSlots, count, walks));
}
benchmarks/runtime/workloads/throw-chains.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/throw-chains.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/throw-chains.ts
+196
View File
@@ -0,0 +1,196 @@
// Exceptions that cross many frames: throws from helpers below a recursion,
// caught 2, 5 and 20 recursive calls up (direct calls, then calls through a
// callback), with each frame holding an array across its call, and a
// backtracking parser that uses exceptions for control flow.
class ParseError extends Error {
readonly at: number;
constructor(at: number, message: string) {
super(message);
this.at = at;
}
}
/** A thrown value that is not an Error (no stack to capture). */
class Stop {
readonly code: number;
constructor(code: number) {
this.code = code;
}
}
const held: number[][] = [];
function fail(n: number, plain: boolean): number {
if (plain) throw new Stop(n);
throw new RangeError("chain " + n);
}
function check(n: number, plain: boolean): number {
return (n & 3) === 0 ? fail(n, plain) : n;
}
function validate(n: number, plain: boolean): number {
return check(n, plain) + (n & 1);
}
function bottom(n: number, plain: boolean): number {
const box = [n, -1];
const r = validate(n, plain);
held[0] = box;
return r + box.length;
}
function walk(n: number, depth: number, plain: boolean): number {
const box = [n, depth];
const r = depth === 0 ? bottom(n, plain) : walk(n, depth - 1, plain);
held[depth] = box;
return r + box.length;
}
function chains(rounds: number): string {
const out: string[] = [];
for (const depth of [2, 5, 20])
for (const plain of [false, true]) {
let caught = 0;
let sum = 0;
for (let i = 0; i < rounds; i++) {
try {
sum += walk(i, depth - 2, plain);
} catch (error) {
caught++;
sum += error instanceof Stop ? error.code & 7 : 1;
}
}
out.push(`depth ${depth} ${plain ? "object" : "error"}: caught ${caught} sum ${sum}`);
}
return out.join("\n");
}
// The same chains through a callback: each level visits the next with a
// closure, as tree visitors do.
function visit(n: number, depth: number, each: (n: number, depth: number) => number): number {
const box = [n, depth];
const r = depth === 0 ? bottom(n, false) : each(n, depth - 1);
held[depth] = box;
return r + box.length;
}
function visits(rounds: number): string {
const out: string[] = [];
const each = (n: number, depth: number): number => visit(n, depth, each);
for (const depth of [2, 5, 20]) {
let caught = 0;
let sum = 0;
for (let i = 0; i < rounds; i++) {
try {
sum += visit(i, depth - 2, each);
} catch (error) {
caught++;
sum += error instanceof RangeError ? 1 : 0;
}
}
out.push(`visit depth ${depth}: caught ${caught} sum ${sum}`);
}
return out.join("\n");
}
// expr := term (("+" | "-") term)*; term := atom ("*" atom)*;
// atom := number | "(" expr ")" | name "(" expr ("," expr)* ")" | name.
// A call and a name start alike: the parser tries the call first and
// backtracks when it fails.
let text = "";
let pos = 0;
function peek(): string {
return pos < text.length ? text[pos]! : "";
}
function expect(c: string): void {
if (peek() !== c) throw new ParseError(pos, `expected ${c}`);
pos++;
}
function name(): string {
const start = pos;
while (pos < text.length && text.charCodeAt(pos) >= 97 && text.charCodeAt(pos) <= 122) pos++;
if (pos === start) throw new ParseError(pos, "expected a name");
return text.slice(start, pos);
}
function call(): number {
const callee = name();
expect("(");
let value = expr() + callee.length;
while (peek() === ",") {
pos++;
value += expr();
}
expect(")");
return value;
}
function atom(): number {
const c = peek();
if (c >= "0" && c <= "9") {
let value = 0;
while (peek() >= "0" && peek() <= "9") value = value * 10 + (text.charCodeAt(pos++) - 48);
return value;
}
if (c === "(") {
pos++;
const value = expr();
expect(")");
return value;
}
const start = pos;
try {
return call();
} catch (error) {
if (!(error instanceof ParseError)) throw error;
pos = start;
return name().length;
}
}
function term(): number {
let value = atom();
while (peek() === "*") {
pos++;
value = (value * atom()) % 1000003;
}
return value;
}
function expr(): number {
let value = term();
while (peek() === "+" || peek() === "-") {
const op = text[pos++];
value = op === "+" ? value + term() : value - term();
}
return value;
}
let seed = 4242;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
function source(depth: number): string {
const r = random();
if (depth > 6 || r < 0.25) return String(Math.floor(random() * 100));
if (r < 0.45) return "abcdefgh".slice(0, 1 + Math.floor(random() * 8));
if (r < 0.6) return `(${source(depth + 1)})`;
if (r < 0.8) return `${source(depth + 1)}${random() < 0.5 ? "+" : "*"}${source(depth + 1)}`;
return `f${"xyz".slice(0, Math.floor(random() * 3))}(${source(depth + 1)},${source(depth + 1)})`;
}
function parse(count: number): string {
let total = 0;
let failed = 0;
for (let i = 0; i < count; i++) {
// Every tenth input lacks its last character, which fails deep inside it.
const full = source(0);
text = i % 10 === 0 ? full.slice(0, -1) : full;
pos = 0;
try {
total = (total + expr()) % 1000003;
if (pos !== text.length) throw new ParseError(pos, "trailing input");
} catch (error) {
if (!(error instanceof ParseError)) throw error;
failed++;
}
}
return `parsed ${count}: total ${total} failed ${failed}`;
}
const scale = Number(process.argv[2] ?? "1");
console.log(chains(Math.floor(60000 * scale)));
console.log(visits(Math.floor(60000 * scale)));
console.log(parse(Math.floor(40000 * scale)));
benchmarks/runtime/workloads/typed-array-kernels.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/typed-array-kernels.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/typed-array-kernels.ts
+144
View File
@@ -0,0 +1,144 @@
// Element-wise loops over several typed arrays: each loop runs to one
// array's length and reads or writes the others at the same index. Also a
// row-major matrix-vector product with four accumulators per row.
function axpy(a: number, x: Float64Array, y: Float64Array, out: Float64Array): void {
for (let i = 0; i < out.length; i++) out[i] = a * x[i]! + y[i]!;
}
function blend(t: number, a: Float32Array, b: Float32Array, out: Float32Array): void {
for (let i = 0; i < out.length; i++) out[i] = a[i]! * (1 - t) + b[i]! * t;
}
function mask(data: Uint8Array, key: Uint8Array, out: Uint8Array): void {
for (let i = 0; i < out.length; i++) out[i] = data[i]! ^ key[i]!;
}
function addWrap(a: Int32Array, b: Int32Array, out: Int32Array): void {
for (let i = 0; i < out.length; i++) out[i] = (a[i]! + b[i]!) | 0;
}
function brighten(src: Uint8ClampedArray, gain: number, out: Uint8ClampedArray): void {
for (let i = 0; i < out.length; i++) out[i] = src[i]! * gain + 8;
}
function matvec(w: Float32Array, x: Float32Array, out: Float32Array, cols: number): void {
for (let r = 0; r < out.length; r++) {
const base = r * cols;
let s0 = 0;
let s1 = 0;
let s2 = 0;
let s3 = 0;
for (let j = 0; j < cols; j += 4) {
s0 += w[base + j]! * x[j]!;
s1 += w[base + j + 1]! * x[j + 1]!;
s2 += w[base + j + 2]! * x[j + 2]!;
s3 += w[base + j + 3]! * x[j + 3]!;
}
out[r] = s0 + s1 + s2 + s3;
}
}
// Callback methods over a whole typed array.
function callbacks(samples: Float32Array, limit: number, data: Uint8Array): number {
const energy = samples.reduce((sum, v) => sum + v * v, 0);
const loud = samples.some((v) => v > limit) ? 1 : 0;
const halved = data.map((v) => v >> 1);
return energy / samples.length + loud + halved[halved.length >> 1]!;
}
function gray(rgba: Uint8Array, out: Uint8Array): void {
for (let i = 0; i < out.length; i++) {
const j = i * 4;
out[i] = (rgba[j]! * 77 + rgba[j + 1]! * 150 + rgba[j + 2]! * 29) >> 8;
}
}
const scale = Number(process.argv[2] ?? "1");
const n = 1 << 16;
const x = new Float64Array(n);
const y = new Float64Array(n);
const d = new Float64Array(n);
const fa = new Float32Array(n);
const fb = new Float32Array(n);
const fo = new Float32Array(n);
const data = new Uint8Array(n);
const key = new Uint8Array(n);
const masked = new Uint8Array(n);
const ia = new Int32Array(n);
const ib = new Int32Array(n);
const io = new Int32Array(n);
const pixels = new Uint8ClampedArray(n);
const bright = new Uint8ClampedArray(n);
const rgba = new Uint8Array(n * 4);
const side = 256;
const weights = new Float32Array(side * side);
const vector = new Float32Array(side);
const product = new Float32Array(side);
const luma = new Uint8Array(n);
let state = 2463534242;
const next = (): number => {
state ^= state << 13;
state ^= state >>> 17;
state ^= state << 5;
return state >>> 0;
};
for (let i = 0; i < n; i++) {
x[i] = (next() % 20000) / 100 - 100;
y[i] = (next() % 20000) / 100;
fa[i] = (next() % 1000) / 7;
fb[i] = (next() % 1000) / 9;
data[i] = next() & 255;
key[i] = next() & 255;
ia[i] = next() | 0;
ib[i] = next() | 0;
pixels[i] = next() & 255;
}
for (let i = 0; i < rgba.length; i++) rgba[i] = next() & 255;
for (let i = 0; i < weights.length; i++) weights[i] = (next() % 2001) / 1000 - 1;
for (let i = 0; i < side; i++) vector[i] = (next() % 1001) / 1000;
const rounds = Math.round(1200 * scale);
let f64Sum = 0;
let f32Sum = 0;
let byteSum = 0;
let intSum = 0;
let brightSum = 0;
let lumaSum = 0;
let productSum = 0;
let callbackSum = 0;
for (let r = 0; r < rounds; r++) {
// Change one input element per round so no round repeats the last.
const at = (r * 7919) & (n - 1);
x[at] = r;
fa[at] = r;
data[at] = r & 255;
ia[at] = r * 65537;
pixels[at] = r & 255;
rgba[at * 4] = r & 255;
vector[r & (side - 1)] = (r % 1001) / 1000;
axpy(1.5, x, y, d);
blend((r & 15) / 16, fa, fb, fo);
mask(data, key, masked);
addWrap(ia, ib, io);
brighten(pixels, 0.75 + (r & 7) / 8, bright);
gray(rgba, luma);
matvec(weights, vector, product, side);
if ((r & 3) === 0) callbackSum += callbacks(fo, 140 + (r & 12), masked);
const probe = (r * 104729) & (n - 1);
f64Sum += d[probe]!;
f32Sum += fo[probe]!;
byteSum += masked[probe]!;
intSum = (intSum + io[probe]!) | 0;
brightSum += bright[probe]!;
lumaSum += luma[probe]!;
productSum += product[probe & (side - 1)]!;
}
console.log(`float64 ${f64Sum.toFixed(3)}`);
console.log(`float32 ${f32Sum.toFixed(3)}`);
console.log(`uint8 ${byteSum}`);
console.log(`int32 ${intSum}`);
console.log(`clamped ${brightSum}`);
console.log(`gray ${lumaSum}`);
console.log(`matvec ${productSum.toFixed(3)}`);
console.log(`callbacks ${callbackSum.toFixed(3)}`);
benchmarks/runtime/workloads/typed-array-scan-workers.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/typed-array-scan-workers.ts",
"args": [
"3"
]
}
benchmarks/runtime/workloads/typed-array-scan.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/typed-array-scan.ts",
"args": [
"3"
]
}
benchmarks/runtime/workloads/unicode-lexer.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/unicode-lexer.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/unicode-lexer.ts
+116
View File
@@ -0,0 +1,116 @@
// A lexer over a large bundled module whose comments and strings are in
// several languages: line starts, tokens by kind and comment text, read
// with charCodeAt as hand-written scanners do. Each pass relexes the whole
// text, as a bundler or an editor does after a change.
const comments = [
"// Berechnet die Größe des Puffers für übergroße Einträge",
"// Calcule la taille maximale avant la prochaine itération",
"// バッファのサイズを計算します",
"/* 计算缓冲区大小，然后返回结果 */",
"// Returns the cached value (fast path) — see the note above",
"/** Résumé: the état of the café's ñandú registry */",
];
const strings = ['"Grüße"', '"naïve façade"', '"東京"', '"ok ✓"', '"plain ascii"', "'mañana'"];
const names = ["buffer", "größe", "index", "resultat", "count", "value", "entry", "table"];
let seed = 11;
function random(): number {
seed = (Math.imul(seed, 1103515245) + 12345) >>> 0;
return seed / 4294967296;
}
function pick(list: readonly string[]): string {
return list[Math.floor(random() * list.length)]!;
}
function makeSource(lines: number): string {
const out: string[] = [];
for (let i = 0; i < lines; i++) {
const r = random();
if (r < 0.2) out.push(pick(comments));
else if (r < 0.6) out.push(` const ${pick(names)}${i} = ${pick(names)}(${Math.floor(random() * 1000)}, ${pick(strings)});`);
else if (r < 0.85) out.push(` if (${pick(names)} >= ${i % 97}) return ${pick(names)}.${pick(names)} + 1;`);
else out.push(`function ${pick(names)}${i}(a, b) { return a * b - ${i}; }`);
}
return out.join("\n");
}
interface Counts {
lines: number;
identifiers: number;
numbers: number;
strings: number;
comments: number;
commentChars: number;
punctuation: number;
}
function isIdentifierStart(ch: number): boolean {
return (ch >= 97 && ch <= 122) || (ch >= 65 && ch <= 90) || ch === 95 || ch === 36 || ch > 127;
}
function isIdentifierPart(ch: number): boolean {
return isIdentifierStart(ch) || (ch >= 48 && ch <= 57);
}
function lex(text: string): Counts {
const counts: Counts = { lines: 1, identifiers: 0, numbers: 0, strings: 0, comments: 0, commentChars: 0, punctuation: 0 };
let pos = 0;
const end = text.length;
while (pos < end) {
const ch = text.charCodeAt(pos);
if (ch === 10) {
counts.lines++;
pos++;
} else if (ch === 32 || ch === 9 || ch === 13) {
pos++;
} else if (ch === 47 && text.charCodeAt(pos + 1) === 47) {
const start = pos;
while (pos < end && text.charCodeAt(pos) !== 10) pos++;
counts.comments++;
counts.commentChars += pos - start;
} else if (ch === 47 && text.charCodeAt(pos + 1) === 42) {
const start = pos;
pos += 2;
while (pos < end && !(text.charCodeAt(pos) === 42 && text.charCodeAt(pos + 1) === 47)) {
if (text.charCodeAt(pos) === 10) counts.lines++;
pos++;
}
pos += 2;
counts.comments++;
counts.commentChars += pos - start;
} else if (ch === 34 || ch === 39) {
pos++;
while (pos < end && text.charCodeAt(pos) !== ch) pos++;
pos++;
counts.strings++;
} else if (ch >= 48 && ch <= 57) {
while (pos < end && text.charCodeAt(pos) >= 48 && text.charCodeAt(pos) <= 57) pos++;
counts.numbers++;
} else if (isIdentifierStart(ch)) {
while (pos < end && isIdentifierPart(text.charCodeAt(pos))) pos++;
counts.identifiers++;
} else {
counts.punctuation++;
pos++;
}
}
return counts;
}
function lineStarts(text: string): number[] {
const starts = [0];
for (let pos = 0; pos < text.length; pos++) if (text.charCodeAt(pos) === 10) starts.push(pos + 1);
return starts;
}
const scale = Number(process.argv[2] ?? "1");
const source = makeSource(60000);
const passes = Math.round(6 * scale);
let last = lex(source);
let total = last.identifiers + lineStarts(source).length;
for (let i = 1; i < passes; i++) {
last = lex(source);
total += last.identifiers + lineStarts(source).length;
}
console.log("units", source.length, "passes", passes, "total", total);
console.log(JSON.stringify(last));
benchmarks/runtime/workloads/union-node-sets.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/union-node-sets.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/union-node-sets.ts
+141
View File
@@ -0,0 +1,141 @@
// Syntax-tree passes over a discriminated union of object literals, the way
// compilers walk their IR: a DAG of shared subexpressions, a `Set` of
// visited nodes, a `Map` from node to its value, and conversions between
// the whole union and the sub-unions passes narrow it to.
type Num = { kind: "num"; value: number };
type Var = { kind: "var"; slot: number };
type Neg = { kind: "neg"; operand: Node };
type Add = { kind: "add"; left: Node; right: Node };
type Mul = { kind: "mul"; left: Node; right: Node };
type Pick = { kind: "pick"; cond: Node; then: Node; else_: Node };
type Leaf = Num | Var;
type Node = Leaf | Neg | Add | Mul | Pick;
function children(node: Node, visit: (child: Node) => void): void {
switch (node.kind) {
case "num":
case "var":
return;
case "neg":
visit(node.operand);
return;
case "add":
case "mul":
visit(node.left);
visit(node.right);
return;
case "pick":
visit(node.cond);
visit(node.then);
visit(node.else_);
return;
}
}
function leafValue(leaf: Leaf, env: number[]): number {
return leaf.kind === "num" ? leaf.value : env[leaf.slot]!;
}
function evaluate(node: Node, env: number[], memo: Map<Node, number>): number {
const cached = memo.get(node);
if (cached !== undefined) return cached;
let result: number;
switch (node.kind) {
case "num":
case "var":
result = leafValue(node, env);
break;
case "neg":
result = -evaluate(node.operand, env, memo);
break;
case "add":
result = (evaluate(node.left, env, memo) + evaluate(node.right, env, memo)) % 1000003;
break;
case "mul":
result = (evaluate(node.left, env, memo) * evaluate(node.right, env, memo)) % 1000003;
break;
case "pick":
result =
evaluate(node.cond, env, memo) % 2 === 0
? evaluate(node.then, env, memo)
: evaluate(node.else_, env, memo);
break;
}
memo.set(node, result);
return result;
}
function reachable(root: Node): number {
const seen = new Set<Node>();
const stack: Node[] = [root];
while (stack.length > 0) {
const node = stack.pop()!;
if (seen.has(node)) continue;
seen.add(node);
children(node, (child) => {
if (!seen.has(child)) stack.push(child);
});
}
return seen.size;
}
function leaves(root: Node): Leaf[] {
const out: Leaf[] = [];
const seen = new Set<Node>();
const walk = (node: Node): void => {
if (seen.has(node)) return;
seen.add(node);
if (node.kind === "num" || node.kind === "var") out.push(node);
else children(node, walk);
};
walk(root);
return out;
}
let seed = 2463534242;
function next(): number {
seed ^= seed << 13;
seed ^= seed >>> 17;
seed ^= seed << 5;
return seed >>> 0;
}
function build(size: number): Node {
const pool: Node[] = [];
for (let i = 0; i < 64; i++)
pool.push(i % 2 === 0 ? { kind: "num", value: i * 7 + 1 } : { kind: "var", slot: i % 16 });
// Operands come from the most recent nodes, so most of the DAG is
// reachable from the root and many nodes are shared.
const recent = (): Node => pool[pool.length - 1 - (next() % 48)]!;
for (let i = 0; i < size; i++) {
const a = recent();
const b = recent();
const r = next() % 10;
let node: Node;
if (r < 1) node = { kind: "neg", operand: a };
else if (r < 5) node = { kind: "add", left: a, right: b };
else if (r < 8) node = { kind: "mul", left: a, right: b };
else node = { kind: "pick", cond: a, then: b, else_: recent() };
pool.push(node);
}
return pool[pool.length - 1]!;
}
const scale = Number(process.argv[2] ?? "1");
const rounds = Math.max(1, Math.round(40 * scale));
let checksum = 0;
let nodes = 0;
let leafCount = 0;
for (let round = 0; round < rounds; round++) {
const root = build(20_000);
nodes += reachable(root);
const found = leaves(root);
leafCount += found.length;
for (let pass = 0; pass < 4; pass++) {
const env: number[] = [];
for (let i = 0; i < 16; i++) env.push((round * 31 + pass * 7 + i) % 97);
checksum = (checksum + evaluate(root, env, new Map())) % 1000003;
for (const leaf of found) checksum = (checksum + leafValue(leaf, env)) % 1000003;
}
}
console.log("rounds", rounds, "nodes", nodes, "leaves", leafCount, "checksum", checksum);
benchmarks/runtime/workloads/validate-errors.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/validate-errors.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/value-objects.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/value-objects.ts",
"args": [
"2"
]
}
benchmarks/runtime/workloads/value-records.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/value-records.ts",
"args": [
"1"
]
}
benchmarks/runtime/workloads/value-records.ts
+93
View File
@@ -0,0 +1,93 @@
// Records that hold references: a tokenizer that builds token records with
// string fields and passes them to helpers, an immutable reducer that
// updates its state with object spread, and tuple-returning helpers whose
// results are destructured at once. None of these temporaries outlives
// the statement or loop iteration that builds it.
interface Token {
kind: string;
text: string;
pos: number;
}
const KEYWORDS = ["let", "const", "if", "else", "for", "while", "return", "function"];
function classify(text: string): string {
return KEYWORDS.includes(text) ? "keyword" : "ident";
}
function token(kind: string, text: string, pos: number): Token {
return { kind, text, pos };
}
function weight(t: Token): number {
return t.kind === "keyword" ? t.text.length * 3 : t.text.length + (t.pos & 7);
}
function tokenize(source: string): number {
let sum = 0;
let start = -1;
for (let i = 0; i <= source.length; i++) {
const c = i < source.length ? source.charCodeAt(i) : 32;
const word = (c >= 97 && c <= 122) || (c >= 48 && c <= 57) || c === 95;
if (word) {
if (start < 0) start = i;
continue;
}
if (start >= 0) {
const text = source.slice(start, i);
const t = token(classify(text), text, start);
sum += weight(t);
start = -1;
}
if (c !== 32 && c !== 10) {
const punct = { kind: "punct", text: source[i]!, pos: i };
sum += weight(punct) + (punct.text === ";" ? 1 : 0);
}
}
return sum;
}
interface State {
mode: string;
depth: number;
count: number;
last: string;
}
function step(s: State, text: string): State {
if (text === "{") return { ...s, depth: s.depth + 1, last: text };
if (text === "}") return { ...s, depth: s.depth - 1, last: text };
return s.mode === "code" ? { ...s, count: s.count + 1 } : { ...s, mode: "code", last: text };
}
function reduce(texts: string[], rounds: number): number {
let total = 0;
for (let r = 0; r < rounds; r++) {
let state: State = { mode: "start", depth: 0, count: 0, last: "" };
for (const text of texts) {
state = step(state, text);
if (state.depth > 3) state = { ...state, depth: 3 };
}
total += state.count + state.depth + state.last.length + state.mode.length;
}
return total;
}
function divmod(a: number, b: number): [number, number] {
return [Math.floor(a / b), a % b];
}
function split(text: string, at: number): [string, string] {
return [text.slice(0, at), text.slice(at)];
}
function tuples(n: number, texts: string[]): number {
let sum = 0;
for (let i = 0; i < n; i++) {
const [q, r] = divmod(i * 31, 17);
const [head, tail] = split(texts[i % texts.length]!, r % 3);
sum += q + r + head.length * 2 + tail.length;
}
return sum;
}
const scaleArg = Number(process.argv[2] ?? "1");
const line = "function step ( s , text ) { if ( text ) { return s ; } let x = s ; while ( x ) { x = x ; } }\n";
const source = line.repeat(Math.floor(400 * scaleArg));
const texts = source.split(" ").filter((t) => t.length > 0);
let lexed = 0;
for (let i = 0; i < 240; i++) lexed += tokenize(source);
const reduced = reduce(texts, Math.floor(600 * scaleArg));
const pairs = tuples(Math.floor(2_000_000 * scaleArg), texts);
console.log("tokens", lexed, "reduce", reduced, "tuples", pairs);
benchmarks/runtime/workloads/word-graph.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/word-graph.ts",
"args": [
"2"
]
}
benchmarks/runtime/workloads/worker-messages.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/worker-messages.ts",
"args": [
"100000"
]
}
benchmarks/runtime/workloads/worker-messages.ts
+24
View File
@@ -0,0 +1,24 @@
// Message round trips between the main thread and one worker: the main
// thread posts a number and the worker answers it plus one, the given
// number of times. Each round trip is two messages and two wakeups of a
// waiting thread. Argument: the number of round trips.
import { Worker, isMainThread, parentPort } from "node:worker_threads";
if (isMainThread) {
const rounds = Number(process.argv[2] ?? "100000");
const worker = new Worker(new URL(import.meta.url));
let sent = 0;
let sum = 0;
worker.on("message", (value: number) => {
sum += value;
if (++sent === rounds) {
console.log(`${rounds} round trips, sum ${sum}`);
void worker.terminate();
} else worker.postMessage(sent);
});
worker.postMessage(0);
} else {
parentPort!.on("message", (value: number) => {
parentPort!.postMessage(value + 1);
});
}
benchmarks/scaling/strace-workloads.mjs
+2 -1
View File
@@ -7,6 +7,7 @@ import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { parseArgs } from "node:util";
import { readWorkloads } from "../runtime/workloads.mjs";
const here = dirname(fileURLToPath(import.meta.url));
const suite = resolve(here, "../runtime");
@@ -18,7 +19,7 @@ const { values } = parseArgs({
},
});
mkdirSync(values.out, { recursive: true });
const { workloads } = JSON.parse(readFileSync(join(suite, "workloads.json"), "utf8"));
const workloads = readWorkloads(suite);
const cli = join(values.scriptc, "packages/cli/dist/bootstrap.js");
// Same inputs as scripts/bench-runtime.mjs generates.
function input(kind) {
docs/content/docs/cli.mdx
+174 -7
View File
@@ -2,7 +2,7 @@
title: "CLI Reference"
---
The `scriptc` command has four subcommands: `build`, `run`, `coverage`, and `cache warm`. `scriptc --help` prints this summary and the options below, and `scriptc --version` prints the installed version.
The `scriptc` command has six subcommands: `build`, `run`, `coverage`, `types`, `cache warm`, and `merge-profiles`. `scriptc --help` prints this summary and the options below, and `scriptc --version` prints the installed version.
```console
$ scriptc --help
@@ -21,8 +21,22 @@ Usage:
exporting the profile symbols; a profile
with a sidecar section also gets the
contract sidecar JSON beside the archive
scriptc types patch <package> <selector> <type> [--range <r>] [--dry-run]
fix a wrong declaration of an installed
package: shows what the selector
matches and adds the type patch to
package.json ("| undefined" widens)
scriptc types list the project's type patches and what
they match
scriptc cache warm [runtime|tls…] verify and load precompiled runtime families
for the current compiler/SDK/target
scriptc merge-profiles -o <out.profdata> [--sparse] <profile…>
merge the raw profiles that
--profile-generate runs wrote (or
earlier merged profiles) into one
profile for --profile-use;
--sparse leaves out functions
that never ran (a smaller file)
```
## scriptc build
@@ -81,10 +95,38 @@ $ scriptc run fib.ts
Analyzes the program without building it and reports what compiles and what blocks the rest. See [Coverage Reports](/docs/coverage).
## scriptc types
Writes and lists [type patches](/docs/dependencies#fix-a-packages-types), the project's fixes for wrong declarations of npm packages.
<dl>
<dt><code>scriptc types patch &lt;package&gt; &lt;selector&gt; &lt;type&gt;</code></dt>
<dd>Finds the package installed for the working directory, prints every declaration the selector matches in its declaration files, its JSDoc and its <code>@types/</code> package with the type before and after, and adds the patch to the nearest <code>package.json</code>. The patch joins an entry whose range covers the installed version, or gets the range <code>^&lt;installed version&gt;</code>. Nothing is written when the selector matches nothing. <code>--range &lt;range&gt;</code> chooses the range, and <code>--dry-run</code> only prints.</dd>
<dt><code>scriptc types list</code></dt>
<dd>Prints each type patch in the nearest <code>package.json</code>, whether it applies to the installed version, and what it matches.</dd>
</dl>
```console
$ scriptc types list
commander@^15.0.0
HelpContext.helpWidth: | undefined
node_modules/commander/lib/command.js:2481 number -> number | undefined
```
## scriptc cache warm
Verifies and loads the installed precompiled runtime objects and vendored libraries, and prints the cache directory. Pass `runtime`, `tls`, or both to select which to warm. Ordinary builds don't need this step. With `--sanitize`, it builds the instrumented runtime objects using a C toolchain.
## scriptc merge-profiles
Merges the raw profiles that runs of a `--profile-generate` executable wrote, and any earlier merged profiles, into one profile for `--profile-use`. See [Profile-guided optimization](#profile-guided-optimization).
```console
$ scriptc merge-profiles -o app.profdata profiles/*.profraw
```
With `--sparse`, the profile leaves out the functions the runs never reached. The file is smaller, typically by about two thirds, and those functions are optimized as without a profile instead of as cold code.
## Options
<dl>
@@ -92,13 +134,13 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dd>Primary artifact path. An explicit path is exact. Defaults are <code>.scriptc/&lt;name&gt;.ir.json</code>, <code>.ll</code>, <code>.s</code>, <code>.o</code>, or the platform executable name.</dd>
<dt><code>--emit &lt;ir|llvm|asm|obj|exe&gt;</code></dt>
<dd>Select the primary artifact: serialized typed IR (<code>ir</code>), textual LLVM IR (<code>llvm</code>), assembly (<code>asm</code>), a relocatable object (<code>obj</code>), or an executable (<code>exe</code>, the default). Every kind except <code>exe</code> is produced without an external compiler, linker, or SDK.</dd>
<dd>Select the primary artifact: serialized typed IR (<code>ir</code>), textual LLVM IR (<code>llvm</code>), assembly (<code>asm</code>), a relocatable object (<code>obj</code>), or an executable (<code>exe</code>, the default). Every kind except <code>exe</code> is produced without an external compiler, linker, or SDK. Assembly and object output is compiled as one module; only executables are divided into partitions that compile in parallel (see <a href="/docs/how-it-works#build-parallelism">Build parallelism</a>).</dd>
<dt><code>--print &lt;native-link-info|diagnostics&gt;</code></dt>
<dd><code>native-link-info</code>: build an object (equivalent to <code>--emit=obj</code>) and print its machine-readable external link recipe as JSON instead of printing the artifact path. The document names the exact installed precompiled runtime pack and all link inputs, but does not invoke a linker. <code>diagnostics</code>: for <code>build</code> and <code>coverage</code>, print the versioned JSON diagnostics document described in <a href="/docs/coverage#machine-readable-diagnostics">Coverage Reports</a> instead of the human output.</dd>
<dt><code>--fail-on &lt;blockers|divergences&gt;</code></dt>
<dd>Coverage only. Exit with status 1 when the entry has blockers that would fail a build, or with <code>divergences</code>, also when it has Node divergence warnings. See <a href="/docs/coverage#gating-builds-and-ci">Gating builds and CI</a>.</dd>
<dt><code>--fail-on &lt;blockers|divergences|dynamic|dynamic-slots&gt;</code></dt>
<dd>Coverage only. Exit with status 1 when the entry has blockers that would fail a build, with <code>divergences</code>, also when it has Node divergence warnings, with <code>dynamic</code>, also when it has the <code>SC6101</code> warning for untyped code that likely runs often, or with <code>dynamic-slots</code>, also when it has <a href="/docs/coverage#dynamic-slots">dynamic slots</a> (<code>SC6104</code>). See <a href="/docs/coverage#gating-builds-and-ci">Gating builds and CI</a>.</dd>
<dt><code>--ffi &lt;file&gt;</code></dt>
<dd>Bind signature-only TypeScript declarations to native C ABI symbols and link the manifest's archive, object, and system-library inputs. See <a href="/docs/ffi">Native FFI</a>.</dd>
@@ -109,8 +151,11 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dt><code>--strip</code></dt>
<dd>Remove symbol and debug payload when linking an executable, including cross-compiled targets. This flag preserves code optimization.</dd>
<dt><code>--static</code></dt>
<dd>On Linux, link the C library and the other system libraries into the executable. The executable starts without the dynamic loader, which shortens its startup, and doesn't depend on the run host's glibc version. glibc executables link as static PIE, are about 1 MB larger, and need glibc's static libraries on the build machine (<code>libc6-dev</code> on Debian and Ubuntu, <code>glibc-static</code> on Fedora); they link only for the build machine's own glibc target. A program whose executable includes host, user, or group name lookups, which <code>dns.lookup</code>, <code>net</code> and <code>http</code> connections, <code>fetch</code>, and <code>os.userInfo()</code> use, is refused with <code>SC3002</code>: a static glibc executable loads the NSS modules for those lookups from the run host, which glibc supports only for the glibc version it was linked with. Build such a program without <code>--static</code>, or for an <code>&lt;arch&gt;-linux-musl</code> target. musl executables are always linked statically, and accept the flag. Valid only for executable builds.</dd>
<dt><code>--optimization &lt;release|dev|speed&gt;</code></dt>
<dd>Select native optimization for executable, assembly, or object output. The default is <code>release</code> (<code>-O2</code>), which emits string retains and the retains and releases of dynamic values inline and lets LLVM inline small runtime functions that keep no internal runtime state into executable code; programs large enough to be compiled in parallel partitions skip that inlining to keep build time and size in check. <code>dev</code> uses <code>-O0</code> and emits source locations for TypeScript and JavaScript breakpoints and stack frames. LLVM builds include source variable names and native storage descriptions; see <a href="/docs/limitations#native-debugging">Native debugging</a> for the supported representations. macOS executable builds also produce an adjacent <code>.dSYM</code> bundle. <code>speed</code> is <code>-O2</code> plus optimizations that trade binary size and build time for run-time speed: it lets LLVM inline more small runtime functions into program code, emits the remaining reference counting inline, and on x86-64 starts every runtime function on a cache-line boundary. Expect larger executables and longer cold builds than <code>release</code>; how much faster a program runs depends on how much of its time is spent in the runtime. Runtime inlining needs a runtime pack built with the LLVM helper; where a runtime pack lacks it, programs link the same runtime without it and <code>speed</code> links the release runtime and keeps inline reference counting. Selecting <code>speed</code> never changes what <code>release</code> builds produce, and the two never share cached artifacts. Library profiles accept <code>"optimization": "speed"</code> for inline reference counting.</dd>
<dd>Select native optimization for executable, assembly, or object output. The default is <code>release</code> (<code>-O2</code>), which emits string retains and the retains and releases of dynamic values inline. <code>dev</code> uses <code>-O0</code> and emits source locations for TypeScript and JavaScript breakpoints and stack frames. LLVM builds include source variable names and native storage descriptions; see <a href="/docs/limitations#native-debugging">Native debugging</a> for the supported representations. macOS executable builds also produce an adjacent <code>.dSYM</code> bundle. <code>speed</code> is <code>-O2</code> plus optimizations that trade binary size and build time for run-time speed: it emits the remaining reference counting inline and, on x86-64, starts every runtime function on a cache-line boundary. Expect larger executables and longer cold builds than <code>release</code>; how much faster a program runs depends on how much of its time is spent in the runtime. Where a runtime pack has no speed runtime, <code>speed</code> links the release runtime and keeps inline reference counting. Selecting <code>speed</code> never changes what <code>release</code> builds produce, and the two never share cached artifacts. Library profiles accept <code>"optimization": "speed"</code> for inline reference counting. In executables large enough to be divided into partitions, <code>dev</code> compiles each partition on its own, and <code>release</code> and <code>speed</code> add a step that lets partitions import small functions from each other; see <a href="/docs/how-it-works#build-parallelism">Build parallelism</a>.</dd>
<dt><code>--windows-subsystem &lt;console|gui&gt;</code></dt>
<dd>Select the subsystem of a Windows executable. <code>console</code> is the default; <code>gui</code> prevents Windows from opening a console window for the app. Valid only for executable builds targeting Windows.</dd>
@@ -118,8 +163,23 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dt><code>--detached-exit</code></dt>
<dd>On Linux, when the program calls <code>process.exit()</code>, hand the release of its memory to a helper process so the parent sees the exit sooner. The kernel otherwise frees the whole address space before the exit is reported, which takes tens of milliseconds per gigabyte. Opt in only where the trade-offs are acceptable: for up to about 100 ms after the exit, the helper keeps the executable mapped (overwriting the binary in place fails with "text file busy"), its memory stays charged to the cgroup, and the teardown CPU time no longer appears in the parent's resource usage. Valid only for executable builds; it has no effect on other platforms.</dd>
<dt><code>--large-pages</code></dt>
<dd>On Linux, when the program starts, move its code onto 2 MiB transparent huge pages, as Node's <code>--use-largepages=silent</code> does. A large program's code then takes a few instruction TLB entries instead of hundreds, and runs faster. The move applies only when the system grants huge pages to memory that asks for them (<code>/sys/kernel/mm/transparent_hugepage/enabled</code> is <code>always</code> or <code>madvise</code>), and only to the code between the first and the last 2 MiB boundary of the executable, so small programs keep their code as it is. Opt in only where the trade-offs are acceptable: the move takes about 3 ms per 10 MiB of code at every start, all of the moved code stays resident and is not shared with other processes running the same executable, and profilers that read symbols from the executable file report the moved code as anonymous memory. If a step of the move fails, the program runs with its code where it was. Valid only for executable builds; it has no effect on other platforms.</dd>
<dt><code>--target-cpu &lt;baseline|x86-64-v2|x86-64-v3|x86-64-v4|native&gt;</code></dt>
<dd>On x86-64 targets, compile the program's code for an x86-64 microarchitecture level. The default, <code>baseline</code>, runs on any x86-64 CPU. A higher level lets LLVM use more instructions, and the executable then runs only on CPUs that support that level. <code>native</code> uses the highest level the build machine supports. Valid for executable, assembly, and object output on x86-64 targets. See <a href="#target-cpu">Target CPU</a>.</dd>
<dt><code>--cpu-dispatch</code></dt>
<dd>On x86-64 targets, compile the program's code twice, for the baseline and for <code>x86-64-v3</code>, and choose the copy when the program starts. The executable runs on any x86-64 CPU and uses the <code>x86-64-v3</code> code on CPUs that support it. Not used with a <code>--target-cpu</code> level above <code>baseline</code>. Valid only for executable builds. See <a href="#runtime-cpu-dispatch">Runtime CPU dispatch</a>.</dd>
<dt><code>--profile-generate &lt;pattern&gt;</code></dt>
<dd>On Linux, build an instrumented executable for profile-guided optimization. Each run writes a raw profile to the path pattern, where <code>%p</code> expands to the process ID; the <code>LLVM_PROFILE_FILE</code> environment variable overrides the pattern at run time. A relative pattern resolves from the current working directory. Instrumented programs run slower. Valid only for executable builds with <code>--optimization=release</code> or <code>speed</code>.</dd>
<dt><code>--profile-use &lt;file.profdata&gt;</code></dt>
<dd>Optimize the executable with a profile from <code>scriptc merge-profiles</code>: code is laid out, inlined, and split by how the profiled runs used it. Code the profile does not describe, or describes from an older version of the program, is optimized as without a profile, so a stale profile only loses part of its gain. A new or changed profile compiles every partition of the program again. <code>scriptc build</code> prints how the profile matched the program's functions on stderr; see <a href="#profile-guided-optimization">Profile-guided optimization</a>. Valid only for executable builds with <code>--optimization=release</code> or <code>speed</code>.</dd>
<dt><code>--npm-static &lt;pkg[,pkg…]|auto&gt;</code></dt>
<dd>Name npm packages to compile even if they don't ship their own type declarations (repeatable). <code>auto</code> uses the default automatic selection. Packages that can't compile produce diagnostics. See <a href="/docs/dependencies#explicit-package-selection">npm Dependencies</a>.</dd>
<dd>Name npm packages to compile even if they don't ship their own type declarations (repeatable). A trailing <code>...</code>, as in <code>fs-extra...</code>, also names every package that package depends on. <code>auto</code> uses the default automatic selection. Packages that can't compile produce diagnostics. See <a href="/docs/dependencies#explicit-package-selection">npm Dependencies</a>.</dd>
<dt><code>--provenance-sources</code></dt>
<dd>Experimental. Compile npm dependencies from the source at the commit their provenance attestation names. Packages without a usable attestation compile from their installed files.</dd>
@@ -127,6 +187,18 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dt><code>--external-types &lt;specifier=file.d.ts&gt;</code></dt>
<dd>Coverage only. Map an exact bare module specifier to a local declaration file supplied by an embedder. Repeat the option for multiple modules. The declaration supplies checker types so application coverage can continue; runtime imports and values remain explicit <code>SC1010</code> blockers. Relative paths resolve from the current working directory. Accepted files end in <code>.d.ts</code>, <code>.d.mts</code>, or <code>.d.cts</code>.</dd>
<dt><code>--tier &lt;auto|static-only&gt;</code></dt>
<dd>The tier policy for code that has no static lowering. <code>auto</code> is the default. <code>static-only</code> guarantees that the program contains no dynamic-tier demotions: every such construct stays a compile error in TypeScript and a deferred runtime error in JavaScript, which is how both modes behave today. Use it in CI to keep a program fully static. Library mode is always static-only. See <a href="/docs/coverage#dynamic-tier">Dynamic tier</a>.</dd>
<dt><code>--explain-dynamic[=&lt;name&gt;]</code></dt>
<dd>Coverage only. After the report, list the functions that already operate on values typed only at runtime and the bindings, fields, functions, classes and modules that unsupported constructs would move to the dynamic tier, each with the diagnostic behind it. With a name such as <code>x</code>, <code>Point</code>, or <code>Point.x</code>, show only the matching units. See <a href="/docs/coverage#dynamic-tier">Dynamic tier</a>.</dd>
<dt><code>--no-dynamic-tier-warning</code></dt>
<dd>Build and coverage. Leave out <code>SC6101</code>, the warning that names functions that likely run often and use untyped values. Not allowed with <code>--fail-on=dynamic</code>. See <a href="/docs/coverage#untyped-code-that-likely-runs-often">Untyped code that likely runs often</a>.</dd>
<dt><code>--wasm &lt;file.wasm&gt;</code></dt>
<dd>Coverage only. Report a WebAssembly module the program loads from a path its sources do not spell out. The report already lists modules whose paths the compiler resolves, modules inlined as base64 literals, and <code>.wasm</code> files shipped by packages that use WebAssembly; a <code>scriptc.wasm</code> array in <code>package.json</code> adds files the same way as this flag. Each module is listed with its size, whether it validates, and the WebAssembly proposals it uses. Builds translate the modules they find to native code; a build reads the <code>scriptc.wasm</code> array but not this flag. Repeatable.</dd>
<dt><code>--sanitize</code></dt>
<dd>Build an executable with AddressSanitizer and a reference-count audit at exit, the same checks scriptc's own test suite uses. Requires a C compiler and is available only for host builds. Not supported with <code>--emit=asm</code> or <code>--emit=obj</code>.</dd>
@@ -150,7 +222,7 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dl>
<dt><code>SCRIPTC_CACHE_DIR</code></dt>
<dd>Override the persistent build-cache root. By default it is <code>$XDG_CACHE_HOME/scriptc/build</code> when set, <code>$HOME/Library/Caches/scriptc/build</code> on macOS, <code>%LOCALAPPDATA%\scriptc\cache\build</code> on Windows, and <code>$HOME/.cache/scriptc/build</code> elsewhere. Cached frontend results and LLVM artifacts are verified against their source, helper, target, and compilation settings. Native outputs also depend on the selected runtime pack and linker inputs. FFI objects, archives, and system libraries relink against their current dependencies. Sanitizer development builds additionally cache instrumented runtime objects and verify external compiler and header inputs. An existing POSIX override must already be private; otherwise caching is bypassed without changing the directory's permissions.</dd>
<dd>Override the persistent build-cache root. By default it is <code>$XDG_CACHE_HOME/scriptc/build</code> when set, <code>$HOME/Library/Caches/scriptc/build</code> on macOS, <code>%LOCALAPPDATA%\scriptc\cache\build</code> on Windows, and <code>$HOME/.cache/scriptc/build</code> elsewhere. Cached frontend results and LLVM artifacts, including the compiled partitions of large executables, are verified against their source, helper, target, and compilation settings. Native outputs also depend on the selected runtime pack and linker inputs. FFI objects, archives, and system libraries relink against their current dependencies. Sanitizer development builds additionally cache instrumented runtime objects and verify external compiler and header inputs. An existing POSIX override must already be private; otherwise caching is bypassed without changing the directory's permissions.</dd>
<dt><code>SCRIPTC_NO_CACHE</code></dt>
<dd>Set to <code>1</code> to bypass all build-cache reads and writes. An explicitly empty <code>SCRIPTC_CACHE_DIR</code> has the same effect.</dd>
@@ -169,6 +241,12 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dt><code>SCRIPTC_RUNTIME_PACK</code></dt>
<dd>A runtime pack directory to use instead of an installed <code>@scriptc/runtime-&lt;target&gt;</code> package.</dd>
<dt><code>SCRIPTC_CHECKER</code></dt>
<dd>The TypeScript 7 API server that checks the program: <code>tsc-ts</code> (the default), which scriptc includes, or <code>tsgo</code>, from the <code>typescript</code> package scriptc also includes. Both give the same diagnostics and generated code.</dd>
<dt><code>SCRIPTC_CHECKER_SERVER</code></dt>
<dd>Set to <code>1</code> on Linux and macOS to check with a tsc-ts server that later builds share. It takes effect with a tsc-ts release that serves <code>tsc-ts --api --listen</code>. The tsc-ts 0.0.9 that this version of scriptc includes does not, so each build checks with its own tsc-ts process, as without the variable. A build connects to the server for its working directory, starting it when none answers, and the server keeps the files it parsed, so the next build of the same program loads it faster. A build that finds the server in use by another build, or cannot start or reach it, checks with its own tsc-ts process. The server exits after 5 minutes without a build, or after a build when its resident memory is above 2048 MB (<code>TSC_TS_LISTEN_MEMORY</code> sets another limit in megabytes). Diagnostics and generated code are the same with and without it. It applies to the default checker, tsc-ts.</dd>
</dl>
```console
@@ -177,6 +255,91 @@ $ file fib-linux
fib-linux: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 2.0.0, with debug_info, not stripped
```
## Profile-guided optimization
A profile records how often each branch and call ran in representative runs of the program. With it, the optimizer inlines the hot calls, keeps hot paths together, and moves cold code out of the way. Profiling takes three steps:
```console
$ scriptc build app.ts --optimization=speed --profile-generate profiles/%p.profraw -o app-instrumented
$ ./app-instrumented typical-input.json
$ scriptc merge-profiles -o app.profdata profiles/*.profraw
$ scriptc build app.ts --optimization=speed --profile-use app.profdata
```
Train on workloads like the ones the program will run: code the training never reached is optimized as without a profile. The profile can be committed and reused as the program changes. Functions whose control flow changed since the profile was collected lose their profile, and the rest keep theirs, so regenerate it when much of the program has changed. [Build parallelism](/docs/how-it-works#build-parallelism) describes what a profile adds to build time.
`scriptc build --profile-use` prints one line on stderr with how the profile matched the program's functions:
```
scriptc: profile app.profdata: 9170 functions annotated, 0 mismatched, 28636 without a record
```
- **annotated:** functions optimized with the profile's counts.
- **mismatched:** functions whose control flow changed since the profile was collected. They are optimized without counts.
- **without a record:** functions the profile does not describe: new functions, and functions the profiled runs never reached.
- When the build reuses cached partitions whose counts were not recorded, the line ends with the number of those partitions.
When more than 10% of the functions the profile describes (annotated and mismatched) are mismatched, the build also prints a warning:
```
scriptc: warning: 3739 of the 8947 functions app.profdata describes (41.8%) changed since it was collected and are optimized without their counts; collect the profile again
```
The build still succeeds. Collecting the profile again with the current program and scriptc version restores the counts.
## Target CPU
By default, x86-64 executables run on any x86-64 CPU, so the compiler uses only the baseline instruction set (SSE2). `--target-cpu` compiles the program's code for a higher x86-64 microarchitecture level. Each level includes the ones below it:
<table>
<thead>
<tr>
<th>Level</th>
<th>Adds</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>x86-64-v2</code></td>
<td>SSE3, SSSE3, SSE4.1, SSE4.2, POPCNT</td>
</tr>
<tr>
<td><code>x86-64-v3</code></td>
<td>AVX, AVX2, BMI1, BMI2, F16C, FMA, LZCNT, MOVBE</td>
</tr>
<tr>
<td><code>x86-64-v4</code></td>
<td>AVX-512 (F, BW, CD, DQ, VL)</td>
</tr>
</tbody>
</table>
```console
$ scriptc build app.ts --target-cpu=x86-64-v3 -o app
```
Loops over typed arrays and numbers gain the most, because LLVM vectorizes them with wider registers. Some loops run slower at a higher level, so measure the program on the machines it will run on.
An executable built for a level runs only on CPUs that support that level. On an older CPU, it crashes with an illegal-instruction signal (`SIGILL`) when it reaches an instruction the CPU lacks. A named level is portable to every machine of that level. `native` selects the highest level the build machine supports, so use it only for executables that run on the machine that built them or on identical machines.
The level applies to the program's own code. The runtime pack keeps its baseline code. Builds for different levels never share cached artifacts, and `baseline` builds are the same as builds without the option.
### Runtime CPU dispatch
`--cpu-dispatch` keeps the executable portable and still uses the higher level where the CPU has it. The program's functions are compiled for the baseline and for `x86-64-v3`. At startup, the executable checks the CPU and the operating system's register support, and then runs the `x86-64-v3` code on CPUs that support every feature of that level. Calls between the program's functions stay within the chosen copy.
```console
$ scriptc build app.ts --optimization=speed --cpu-dispatch -o app
```
The second copy makes the executable larger and the first build of the program slower, since LLVM compiles the program's code twice. Unchanged rebuilds are not affected. Both copies compute the same results: neither contracts floating-point operations into fused multiply-adds.
The `SCRIPTC_CPU_LEVEL` environment variable caps the level the executable uses, for example to test the baseline code on a newer machine. It takes a level from `0` to `4`, `baseline`, or a level name such as `x86-64-v3`; any other value selects the baseline code.
```console
$ SCRIPTC_CPU_LEVEL=baseline ./app
```
## LLVM backend
scriptc generates LLVM IR and compiles it to native code. Executable builds link the result with the matching precompiled runtime pack. Use <code>--emit=llvm</code> to inspect the generated code.
@@ -213,6 +376,10 @@ For source breakpoints in Xcode or LLDB, build with <code>--optimization=dev</co
<td><code>scriptc run</code> on <code>wasm32-wasi</code></td>
<td>Node.js 24 or later on <code>PATH</code></td>
</tr>
<tr>
<td><code>--static</code> on glibc Linux</td>
<td>glibc's static libraries (<code>libc6-dev</code> on Debian and Ubuntu, <code>glibc-static</code> on Fedora)</td>
</tr>
<tr>
<td><code>--sanitize</code></td>
<td>A C compiler and LLVM toolchain</td>
docs/content/docs/contracts.mdx
+5 -3
View File
@@ -245,7 +245,7 @@ $ scriptc build scale.ts
1 error.
```
A copy such as `slice()` or a spread is checked as if it were the same array as its source. The results of callbacks are not followed, so an array built by `map` or `Array.from` with a mapping function cannot be `Int32[]`; build it with a loop that stores `value | 0`. scriptc refuses an `Int32[]` slot whose array can come from code it cannot see, such as `JSON.parse`, dynamic values or the callers of a function used as a value, and an array that can reach one and escapes to such code, such as a call through a function value (SC7003).
A copy such as `slice()` or a spread is checked as if it were the same array as its source. The results of a callback are followed only when scriptc copies it into the loop that runs the array method: a callback written at a `map` or `filter` call whose captured variables are not reassigned, such as `xs.map((x) => (x * 2) | 0)`. Other callback results are not followed, so for example an array built by `Array.from` with a mapping function cannot be `Int32[]`; build it with a loop that stores `value | 0`. scriptc refuses an `Int32[]` slot whose array can come from code it cannot see, such as `JSON.parse`, dynamic values or the callers of a function used as a value, and an array that can reach one and escapes to such code, such as a call through a function value (SC7003).
## Immutable
@@ -331,7 +331,7 @@ $ scriptc build graph.ts
## ProgramLifetime
`ProgramLifetime` marks a class whose instances a program keeps until it exits, such as the nodes of a syntax tree, the symbols of a symbol table or interned types. scriptc then never reference counts, collects or frees instances of the class and of its subclasses: they are never freed, every retain and release on them compiles to nothing, and the cycle collector neither walks them nor needs to trace the objects that point to them. Node.js behavior is unchanged; only memory use differs, because an instance the program drops is never reclaimed.
`ProgramLifetime` marks a class whose instances a program keeps until it exits, such as the nodes of a syntax tree, the symbols of a symbol table or interned types. scriptc then never reference counts, collects or frees instances of the class and of its subclasses: they are never freed, every retain and release on them compiles to nothing, and the cycle collector neither walks them nor needs to trace the objects that point to them. Node.js behavior is unchanged; only memory use differs, because an instance the program drops is never reclaimed. On Linux, the instances a worker thread creates are returned with the rest of its memory after it exits, unless the worker shared objects with other threads (see [cost](/docs/threads#cost)).
```ts:tree.ts
import type { ProgramLifetime } from "@scriptc/types";
@@ -349,6 +349,8 @@ class Identifier extends SyntaxNode {
}
```
On Linux, except in builds with sanitizers, the instances also take less memory. An executable keeps them in one address range below 32 GiB that it reserves at startup (only the pages in use take memory), and a field typed `C`, `C | undefined` or `C | null`, where `C` is a program-lifetime class, stores the instance's address in 4 bytes instead of 8, as does each element of an array of such a type and each value of a `Map` with `string` keys and such a value type. In the example, `parent` takes 4 bytes.
scriptc checks that the promise cannot change what the program computes (SC7008):
- the class is the root of its hierarchy, so every value of any class type in the hierarchy is program-lifetime; built-in bases such as `Error`, `EventEmitter` and streams are refused because they hold runtime state that is released with the object;
@@ -358,7 +360,7 @@ A class can be both `ProgramLifetime` and `Acyclic`. `Acyclic` is still checked
Instances can be published to other threads with `@scriptc/threads` like other class instances: `publish()` makes them immutable and shares them by reference, writes to a published instance throw the same `TypeError` as under Node.js, and instances that were not published stay writable and are cloned by `postMessage` and `workerData`.
Static code has no `WeakRef` or `FinalizationRegistry`, and a `WeakMap` or `WeakSet` entry whose key is never freed is never observable as removed, so nothing else can tell the difference.
Static code has no `WeakRef`, a `FinalizationRegistry` never runs its cleanup callbacks, and a `WeakMap` or `WeakSet` entry whose key is never freed is never observable as removed, so nothing else can tell the difference.
### Lifetime regions
docs/content/docs/coverage.mdx
+74 -6
View File
@@ -36,7 +36,10 @@ When a program has unsupported operations, the report adds a group for each root
<tbody>
<tr><td><code>statements analyzed</code></td><td>Executable statements in application code, excluding dependency implementation code.</td></tr>
<tr><td><code>compile statically</code></td><td>Statements that compile to native instructions.</td></tr>
<tr><td><code>in dynamic tier</code></td><td>Of those, the statements in functions that operate on values typed only at runtime, such as <code>any</code> values and untyped JavaScript. Shown when there are any. See <a href="#dynamic-tier">Dynamic tier</a>.</td></tr>
<tr><td><code>blockers</code></td><td>Unsupported operations with counts, diagnostic codes, reasons, locations, and hints.</td></tr>
<tr><td><code>runs at startup</code></td><td>Deferred operations in a JavaScript module's top-level code, which runs when the module loads: at startup for a module the program imports.</td></tr>
<tr><td><code>deferred to runtime</code></td><td>Other deferred operations in JavaScript code. Each throws only if it runs.</td></tr>
<tr><td><code>in unreached code</code></td><td>Blockers in code the entry never reaches. A build never compiles that code, so these cannot fail it.</td></tr>
<tr><td><code>differs from Node</code></td><td>Sites that compile but can behave differently from Node.js. See <a href="#node-divergences">Node divergences</a>.</td></tr>
<tr><td><code>npm package compilation</code></td><td>Native package compilation outcomes and refusal reasons.</td></tr>
@@ -45,12 +48,74 @@ When a program has unsupported operations, the report adds a group for each root
Blockers in unreached code and deferred operations in JavaScript packages are listed separately because they don't fail a build. A deferred operation throws a runtime error if it runs, so a successful build doesn't mean every API of a package works; see [unsupported code inside a package](/docs/dependencies#unsupported-code-inside-a-package).
If a function's signature can't compile, the statements in its body count as not compiling, so the percentage reflects how much of the program compiles.
If a function's signature can't compile, the statements in its body count as not compiling, so the percentage reflects how much of the program compiles. The report still lists the diagnostics of those statements; a use of a parameter whose type can't compile reports `SC2004` back to the signature.
<span id="programs-that-require-dynamic-execution" />
<span id="analyze-with-dynamic" />
<span id="embedded-nodejs-builtins" />
<span id="dynamic-tier" />
## Dynamic tier
Code that operates on values whose types are known only at runtime, such as `any` values, the result of an untyped `JSON.parse`, or untyped JavaScript, compiles into dynamic-tier functions. They are native code too, but they check types and look up properties at runtime, so they are slower than code with static types. The report's `in dynamic tier` line counts their statements.
`--explain-dynamic` lists those functions and the operation that makes each one dynamic. It also lists the units an unsupported construct would move to the dynamic tier: the binding, field, class, function, or module that has no static representation for the construct, with the diagnostic behind each one. These are the constructs the dynamic tier is planned to compile; today they remain blockers in TypeScript and deferred runtime errors in JavaScript. Add a name, such as `--explain-dynamic=Point`, to show only the matching units. With `--print=diagnostics`, the document gains a `tiers` object with the same information.
`--tier=static-only` keeps every unsupported construct a blocker instead of moving it to the dynamic tier; code that already operates on runtime-typed values compiles as before. Today `auto`, the default, behaves the same way.
### Untyped code that likely runs often
When functions of the program's own files use untyped values and likely run often, `scriptc build` and coverage report them as an `SC6101` warning. For the Benchmarks Game n-body program in plain JavaScript:
```console
$ scriptc build nbody.js -o nbody
warning SC6101: 3 functions that likely run often use untyped values, which runs slower than typed code:
advance (nbody.js:22): untyped JavaScript file, in a loop
energy (nbody.js:40): untyped JavaScript file, in a loop
offsetMomentum (nbody.js:12): untyped JavaScript file, in a loop
hint: add types to compile these statically; see scriptc coverage --explain-dynamic
The program compiled. Adding types to these functions lets scriptc compile them to faster code.
```
Without a profile, a dynamic-tier function counts as likely to run often when a dynamic operation runs on every iteration of one of its loops (a `for` loop's initializer and a `for...of` loop's iterable run once), when a loop calls it, or when it calls itself. A function is named only when its source says why its values are untyped: a value typed `any` (for a `for...of` variable, the value it iterates), an untyped `JSON.parse` result, or an untyped JavaScript file. Dynamic code that runs once, such as setup code that reads an `any` configuration, is not reported, and neither are functions that are dynamic for an operation with no static lowering, which more types would not change.
The warning names at most five functions, most likely to run often first, and gives the total. A function expression is named by its own name, else by the variable, property or class field it is assigned to, else as `anonymous function`. Dynamic-tier code in npm packages is summarized on one line with each package's function count. A build that comes from the cache repeats the warning. `--no-dynamic-tier-warning` leaves it out, and `--fail-on=dynamic` fails a coverage run on it.
## Dynamic slots
A declared type that has no native representation does not fail the build. The field, parameter, return value, variable or expression that holds it becomes a checked dynamic slot: the parts of the type that compile keep their native layout, and the failing parts hold checked-dynamic values. Coverage lists the slots under the statement counts, and `scriptc build` prints one `SC6104` warning per slot.
```console
$ cat plugins.ts
type Hook = (() => void) | AbortSignal | Map<string, (x: number) => string>;
interface Plugin {
name: string;
hook: Hook;
}
const plugins: Plugin[] = [{ name: "log", hook: () => console.log("hook ran") }];
for (const p of plugins) {
const hook = p.hook;
if (typeof hook === "function") hook();
console.log(p.name);
}
$ scriptc coverage plugins.ts
statements analyzed 6
compile statically 6 (100%)
in dynamic tier 1 (operate on values typed only at runtime; --explain-dynamic lists them)
dynamic slots 1 (declared types with no static lowering, held as checked-dynamic values; SC6104)
module global 'plugins' (declared 'Plugin[]') (dynamic pieces) plugins.ts:8:7
fully static — all analyzed statements compile to native code.
```
Here `Plugin` keeps its native `name` field, and only `hook` is checked-dynamic. Reading a dynamic value into typed code checks it against the type that reads it, so data that contradicts the declaration still throws a `TypeError`. Slots with the most dynamic operations are listed first. `--fail-on=dynamic-slots` fails a coverage run when any slot exists, and `--tier=static-only` reports the original errors instead.
## Diagnostic codes
Coverage and `scriptc build` report the same diagnostic codes. Common ones include:
@@ -65,10 +130,9 @@ Coverage and `scriptc build` report the same diagnostic codes. Common ones inclu
Some code compiles but can behave differently than under Node.js. Coverage reports the cases it can detect as `SC6xxx` warnings, and `scriptc build` prints the same warnings after a successful build. They never fail a build, but `--fail-on=divergences` can fail a CI job on them.
- `SC6001`: a record passed to, stored as, asserted as, or stored as an array element of a record type with different properties is copied, and the program then writes through the copy or compares it by identity.
- `SC6002`: an object literal's key order differs from its type's declaration order, and the program observes the order through `JSON.stringify`, `Object.keys`, `Object.values`, `Object.entries`, `for...in`, or console output.
- `SC6002`: values of one record type are built in key orders scriptc cannot track per value, so the type keeps its declaration order, an object literal's key order differs from it, and the program observes the order through `JSON.stringify`, `Object.keys`, `Object.values`, `Object.entries`, `for...in`, or console output.
- `SC6003`: `localeCompare` compares a string literal outside the built-in Latin-script collation.
- `SC6004`: a call passes a class instance to a function that casts it to an unrelated class.
- `SC6005`: a `Date` string literal uses a form the native parser does not accept.
## Gating builds and CI
@@ -78,7 +142,7 @@ By default, `scriptc coverage` exits with status 0 whenever the program type-che
$ scriptc coverage src/main.ts --fail-on=blockers
```
`--fail-on=blockers` exits with status 1 when the program has blockers that would fail `scriptc build`. Blockers in unreached code and deferred operations don't count. `--fail-on=divergences` also fails on Node divergences. A program that doesn't type-check always exits with status 1.
`--fail-on=blockers` exits with status 1 when the program has blockers that would fail `scriptc build`. Blockers in unreached code and deferred operations don't count. `--fail-on=divergences` also fails on Node divergences, `--fail-on=dynamic` also fails on the `SC6101` warning for [untyped code that likely runs often](#untyped-code-that-likely-runs-often), and `--fail-on=dynamic-slots` also fails on [dynamic slots](#dynamic-slots). A program that doesn't type-check always exits with status 1.
Coverage caches its result. Running it again on an unchanged program, or after edits that only touch comments, reuses the previous verdict. Set `SCRIPTC_NO_CACHE=1` to turn the cache off.
@@ -160,11 +224,11 @@ For editors, CI annotations, and other tools, `scriptc coverage --print=diagnost
</tr>
<tr>
<td><code>stats</code></td>
<td>Coverage only, absent after a preflight failure: the report's statement counts. <code>statementsDynamic</code> is always 0 and remains for schema compatibility.</td>
<td>Coverage only, absent after a preflight failure: the report's statement counts. <code>statementsStatic</code> counts every statement that compiles to native code; <code>statementsDynamic</code> is the part of those in dynamic-tier functions.</td>
</tr>
<tr>
<td><code>category</code></td>
<td><code>invalid-source</code> (the program has a TypeScript or JSON error), <code>unsupported</code> (valid code scriptc cannot compile yet), <code>divergence</code> (compiles, but can behave differently from Node.js), <code>environment</code> (project configuration, scriptc's type world, toolchain, or target), or <code>internal</code> (a compiler bug).</td>
<td><code>invalid-source</code> (the program has a TypeScript or JSON error), <code>unsupported</code> (valid code scriptc cannot compile yet), <code>divergence</code> (compiles, but can behave differently from Node.js), <code>performance</code> (compiles and behaves as under Node.js, but runs slower than typed code would), <code>environment</code> (project configuration, scriptc's type world, toolchain, or target), or <code>internal</code> (a compiler bug).</td>
</tr>
<tr>
<td><code>severity</code></td>
@@ -182,6 +246,10 @@ For editors, CI annotations, and other tools, `scriptc coverage --print=diagnost
<td><code>group</code></td>
<td>The <code>id</code> of the root-cause group the diagnostic belongs to. A group's <code>primary</code> site is its first in source order.</td>
</tr>
<tr>
<td><code>startup</code></td>
<td><code>true</code> on a <code>deferred</code> diagnostic in a module's top-level code, which runs when the module loads. Absent otherwise.</td>
</tr>
</tbody>
</table>
docs/content/docs/dependencies.mdx
+142 -5
View File
@@ -46,32 +46,169 @@ Your callback and the commander implementation both compile to native code. No e
scriptc finds packages in `node_modules` with Node.js's resolution rules, including the export conditions in `package.json`, and compiles their dependencies too. TypeScript workspace packages compile as part of your program, including `.js` import specifiers that resolve to `.ts` sources.
Type-only imports follow Node.js's type stripping. `import type` and `export type … from` statements are erased, so a package or module named only by them supplies types and isn't compiled or loaded; it can even be a package with no JavaScript, such as `type-fest`. A clause whose specifiers are all marked `type`, such as `import { type Options } from "./config.ts"`, still loads its module, as it does under Node.js.
## Which packages compile
scriptc compiles a package automatically when it:
- ships its own type declarations, and
- ships readable, unminified JavaScript.
- ships its own type declarations, or is typed by its DefinitelyTyped `@types/` package, or is a dependency of another package that has no declarations at all, and
- ships JavaScript that isn't a webpack bundle. Minified JavaScript compiles like unminified JavaScript.
The declarations check your program; the package's JavaScript compiles with the types scriptc infers from it. A program that doesn't typecheck against the inferred types, for example because an `@types/` package describes another release, fails the build at that use.
A package that doesn't meet these requirements, or that uses something scriptc can't compile, gets a package-specific diagnostic that explains why. The usual fixes are to switch to a version or alternative package that compiles, or to replace the call with code you write yourself.
`graceful-fs` is the exception: it compiles as `node:fs` instead of from its JavaScript, with one [documented difference](/docs/limitations#graceful-fs).
<span id="value-conversion" />
<span id="static-package-compilation-experimental" />
<span id="type-declarations-and-boundary-restrictions" />
## Explicit package selection
Some packages rely on separately installed declarations, such as an `@types/` package, or on types inferred from their JavaScript. To try compiling one of them, name it with `--npm-static`:
Some packages your program imports directly ship no declarations, so their types can only be inferred from their JavaScript. To try compiling one of them, name it with `--npm-static`:
```console
$ scriptc build app.ts --npm-static=three -o app >/dev/null
```
Naming a package skips the declaration requirement above, but the package still has to compile. `--npm-static=auto` uses the default automatic selection. The option accepts a comma-separated list and can be repeated.
Naming a package skips the declaration requirement above, but the package still has to compile. Add a trailing `...` to select a package together with every package it depends on, for example `--npm-static=semver...`. `--npm-static=auto` uses the default automatic selection. The option accepts a comma-separated list and can be repeated.
## Built-in packages
scriptc implements a few packages itself and compiles that implementation in place of the package's published JavaScript. Which implementation compiles depends on the version your program has installed, so an import resolves the same way it does under Node.js: a package that isn't installed is still a missing package, and a version scriptc doesn't implement compiles from its own files as usual.
`undici` 5.x and 7.x are built in. The implementation covers `request()`, `Dispatcher` (including subclasses that override `dispatch`), `Client`, `Pool` with its `stats`, `Agent` with per-origin `factory` functions and `connect` options, `buildConnector`, the `errors` classes and their `UND_ERR_*` codes, `setGlobalDispatcher` and `getGlobalDispatcher`, `fetch` and `Headers`, and the request events undici publishes on `node:diagnostics_channel`. Requests use HTTP/1.1 with keep-alive connection reuse and undici's headers, body and idle timeouts, and each version keeps its own behavior where 5.x and 7.x differ, such as the error an aborted request rejects with. The package's other exports aren't declared, so using one is a type error at build time. Known differences:
- A response body supports the readable-stream members undici's body mixin uses: `on`, `once`, `pause`, `resume`, `destroy`, `text()`, `json()`, `arrayBuffer()`, `bytes()`, `blob()` and `dump()`. It can't be piped or iterated with `for await`.
- Diagnostics channel messages carry the request but not the socket or connector.
- `fetch` with a dispatcher returns a response whose `url` is empty and whose `redirected` is `false`, offers only `gzip, deflate` encodings, and sends string and byte bodies. Without a dispatcher it is the built-in `fetch`.
- Request bodies are strings and bytes; stream, iterable and form-data bodies, `pipelining` above 1, `CONNECT` requests and Unix socket paths aren't supported.
`fs-extra` 8.x through 11.x is built in, and its declarations replace `@types/fs-extra`. The default import, a namespace import and named imports all read the same exports. fs's members are re-exported, and fs-extra's wrapped methods follow its rule: with a callback as the last argument they call back, otherwise they return a promise. The extra methods cover `copy`, `copySync`, `move`, `moveSync`, `remove`, `removeSync`, `emptyDir`, `ensureDir`, `mkdirs`, `mkdirp`, `ensureFile`, `pathExists`, `outputFile`, `readJson`, `readJsonSync`, `writeJson`, `writeJsonSync`, `outputJson` and their aliases, with fs-extra's error messages. Known differences:
- `ensureLink`, `ensureSymlink`, `createLink`, `createSymlink`, `gracefulify`, the promise forms of the descriptor-based methods, and the callback forms of `stat`, `lstat`, `readdir`, `mkdtemp`, `realpath` and `readJson` aren't declared, so using one is a type error at build time. Paths are strings, text is UTF-8, and write flags, string modes and `copyFile` mode flags aren't accepted.
- graceful-fs's retry queue, which fs-extra uses under Node.js, isn't part of the implementation, so a program that runs out of file descriptors fails at the first `EMFILE` error instead of retrying.
- An ES module that imports fs-extra can use any of its exports by name. Node.js only exposes the names its CommonJS lexer finds in fs-extra's source, so a name like `readFile` throws a `SyntaxError` there.
`@hono/node-server` 2.x is built in. `serve`, `createAdaptorServer`, `getRequestListener` and `RequestError` serve a fetch handler such as Hono's `app.fetch` over `node:http` with native `Request` and `Response` objects, and write each response the way the package does under Node.js: the same status line, header names, header order and content length, for buffered and streamed bodies, including the package's shortcut for a response that is still exactly what its constructor received. Request bodies stream to the handler as it reads them, and a body the handler leaves unread is drained after the response, up to 64 MiB or 500 ms, as the package does. Known differences:
- Only HTTP/1.1 servers are declared: the `createServer` and `serverOptions` options for `node:https` and `node:http2`, the `websocket` option and `upgradeWebSocket` aren't declared, so using one is a type error at build time. Subpath modules such as `@hono/node-server/serve-static` compile from the package's own files.
- The fetch handler must return a `Response` or a promise of one; the handler's second argument holds `incoming` and `outgoing`.
- The global `Request` and `Response` stay the built-in classes, so `overrideGlobalObjects` has no effect, and a `Response` reads its default text content type as `text/plain;charset=UTF-8` where the package's replacement reads `text/plain; charset=UTF-8`. Written responses carry the package's spelling.
- A streamed response is written without waiting for `drain`, and a failed handler answers 504 only for an error whose `name` is `TimeoutError`.
Small polyfill packages whose module is exactly a Node.js built-in on Node.js 24 compile as that built-in, at the `require()` that loads them, from your program's CommonJS files, from compiled packages, and from esbuild bundles whose module paths name the package and its version (as pnpm installs do). The package's own files aren't compiled.
<table>
<thead>
<tr><th>Package</th><th>Versions</th><th>Compiles as</th></tr>
</thead>
<tbody>
<tr><td><code>isarray</code></td><td>0.x, 1.x, 2.x</td><td><code>Array.isArray</code></td></tr>
<tr><td><code>safe-buffer</code></td><td>5.x</td><td><code>require("buffer")</code></td></tr>
<tr><td><code>process-nextick-args</code></td><td>2.x</td><td><code>process</code></td></tr>
<tr><td><code>inherits</code></td><td>2.x</td><td><code>require("util").inherits</code></td></tr>
<tr><td><code>util-deprecate</code></td><td>1.x</td><td><code>require("util").deprecate</code></td></tr>
<tr><td><code>readable-stream</code></td><td>2.x, 3.x, 4.x</td><td><code>require("stream")</code> and its classes</td></tr>
</tbody>
</table>
`readable-stream` compiles as Node.js's built-in streams, which is its own `READABLE_STREAM=disable` mode for 2.x and 4.x. That can differ from its default behavior under Node.js; see [limitations](/docs/limitations#readable-stream).
## Optional dependencies that never load
Some packages load an optional dependency inside `try`/`catch` and work without it, such as `node-fetch` with `encoding`. To keep such a dependency out of the native build even when it is installed, mark it unloaded in your entry's `package.json`:
```json:package.json
{
"scriptc": {
"packages": {
"encoding": "unloaded"
}
}
}
```
A `require()` of an unloaded package throws the same `MODULE_NOT_FOUND` error Node.js throws when the package isn't installed, and the package isn't compiled.
## CommonJS packages
CommonJS packages compile with Node.js's module semantics. Each `require()` evaluates its module at the call, once, and later calls return the cached exports. A package can assign `module.exports` and run code before it requires its dependencies. If that code reads a `const` or `let` require binding before its `require` runs, the read throws the same `ReferenceError` as Node.js. A `var` binding that earlier code can reach is reported at build time. A `require()` of a JSON file, such as a package's own `package.json`, returns one object parsed at build time, shared by every module that requires it. A require cycle compiles when each of its modules assigns its exports before requiring anything, as semver's classes and esbuild's per-file output do, and uses the other modules only inside functions that its top-level code can't call. Bundles built by esbuild compile with their bundled CommonJS modules and lazily initialized ES modules, each evaluated once, at its first use, as under Node.js. Other require cycles are reported at build time. The [limitations](/docs/limitations) page lists the supported `require` forms.
## Unsupported code inside a package
A JavaScript package can contain operations scriptc can't compile, often in functions your program never calls. These don't fail the build: each one compiles to code that throws a runtime error if it ever runs. Coverage reports list them as deferred sites, so review the list and test the parts of the package your program uses.
A JavaScript package can contain operations scriptc can't compile, often in functions your program never calls. These don't fail the build: each one compiles to code that throws a runtime error if it ever runs. Coverage reports list them as deferred sites, so review the list and test the parts of the package your program uses. Sites in a module's top-level code are listed under `runs at startup`: that code runs when the module loads, at startup for a module the program imports.
## Fix a package's types
scriptc trusts declared types: a package's `.d.ts` files, its `@types/` package, and the JSDoc types in its JavaScript. A value that contradicts its declared type throws a `TypeError` instead of flowing on, so a package whose declarations are wrong can stop a program that runs under Node.js. A common case is a member declared `string` or `number` that is sometimes `undefined`.
When the type comes from a package's declarations, the error names the declaration and the patch that would admit the value:
```text
Uncaught TypeError: expected string at $, got undefined
profilekit@3.1.0 declares Profile.email as string (index.d.ts:7)
if that declaration is wrong, patch it: scriptc types patch profilekit "Profile.email" "| undefined"
```
Fix such a declaration in your project with a type patch. `scriptc types patch` shows what a declaration path matches in the installed package and adds the patch to your `package.json`:
```console
$ scriptc types patch commander "HelpContext.helpWidth" "| undefined"
commander 15.0.0 (node_modules/commander)
lib/command.js:2481 number -> number | undefined
wrote package.json scriptc.typePatches["commander@^15.0.0"].declarations["HelpContext.helpWidth"]
```
The patch it writes looks like this:
```json:package.json
{
"scriptc": {
"typePatches": {
"commander@^15.0.0": {
"declarations": {
"HelpContext.helpWidth": "| undefined"
}
}
}
}
}
```
Each key names a package and a semver range of the installed versions the patch describes. `declarations` maps a declaration to a type:
<dl>
<dt>`"| undefined"`</dt>
<dd>A type that starts with `|` widens the declared type, here to `number | undefined`.</dd>
<dt>`"string | null"`</dt>
<dd>Any other type replaces the declared type.</dd>
</dl>
A declaration is named by its path: `Options.name` for a member of a type, `parse()` for a function's return type, `parse(text)` for a parameter, `list()[]` for an array's elements, and `Env[string]` for an index signature. `export=()` names the return type of the function the module exports, `default` its default export, and a file prefix such as `index.js:parse()` names a declaration inside one of the package's files. One patch applies to the package's own declarations, its JSDoc, and its `@types/` package. The patched type then holds everywhere, so your code has to handle the `undefined` that a widened declaration allows. An optional `"note"` string can link the upstream issue.
For larger fixes, replace a whole declaration file, or give a package without declarations its own:
```json:package.json
{
"scriptc": {
"typePatches": {
"some-client@^3.2.0": {
"files": { "dist/types/options.d.ts": "./type-patches/options.d.ts" }
},
"untyped-helper@1.x": {
"types": "./type-patches/untyped-helper.d.ts"
}
}
}
}
```
`files` maps a declaration file inside the package, or inside its `@types/` package written as `@types/<name>/<path>`, to a local `.d.ts` file. `types` supplies the package's declaration entry. Paths of local files are relative to the `package.json`, and a replaced file can import the package's other declaration files with the same relative paths as the file it replaces.
Patches change only declarations, never the package's code, and values are still checked against the patched types. A build warns when a patch doesn't apply as written: when no range covers the installed version, when a declaration path matches nothing, when the declaration already has the type, or when a path is ambiguous. Report wrong declarations to the package too, so the patch can be removed later.
## Provenance sources (experimental)
docs/content/docs/experimental.mdx
+151 -5
View File
@@ -73,9 +73,9 @@ scriptc region heap passes: 0
<dt><code>new LifetimeRegion()</code></dt>
<dd>Creates a region. Creating a region also retries the released regions that could not be freed yet.</dd>
<dt><code>region.run(fn)</code></dt>
<dd>Calls <code>fn</code> with the region as the current region and returns its result. <code>ProgramLifetime</code> instances created during the call belong to the region. Calls nest: a <code>run</code> inside the callback makes another region current until it returns, and the previous region is current again however the callback finishes, including when it throws.</dd>
<dd>Calls <code>fn</code> with the region as the current region and returns its result. <code>ProgramLifetime</code> instances created during the call belong to the region, and so do those that async functions called during it create after an <code>await</code> (see <a href="#async-work">Async work</a>). Calls nest: a <code>run</code> inside the callback makes another region current until it returns, and the previous region is current again however the callback finishes, including when it throws.</dd>
<dt><code>region.release()</code></dt>
<dd>Requests that the region be freed. scriptc frees it as soon as nothing outside the region refers to any of its instances, which may be at once, at a later release or region creation, or never. A region is never freed while a <code>run</code> call on it is active. After <code>release()</code>, <code>run</code> still calls its callback, and the instances it creates belong to the default region.</dd>
<dd>Requests that the region be freed. scriptc frees it as soon as nothing outside the region refers to any of its instances, which may be at once, at a later release or region creation, or never. A region is never freed while a <code>run</code> call on it is active, or while async work started inside it has not finished. After <code>release()</code>, <code>run</code> still calls its callback, and the instances it creates belong to the default region.</dd>
</dl>
`ProgramLifetime` instances created outside every `run` call belong to the default region, which is never freed. Ordinary objects (instances of classes without the contract, arrays, maps, closures) are reference counted as always; a region frees the ones that only its instances refer to.
@@ -92,7 +92,62 @@ A released region stays allocated, and its instances stay usable, while anything
- an instance of the default region, which is never freed;
- a promise, generator or `Error` that holds an instance, until it is no longer referenced.
Work that resumes after an `await` inside the callback runs outside the region, so the instances it creates belong to the default region and keep the instances they refer to allocated.
### Async work
An async function called inside `run` keeps the region for its whole run: the instances it creates after each `await` belong to the region too, as do those of the async functions it calls. A server can give every request its own region, even while requests interleave at their awaits:
```ts:requests.ts
import type { ProgramLifetime } from "@scriptc/types";
import { LifetimeRegion } from "@scriptc/experimental";
class Part implements ProgramLifetime {
readonly text: string;
previous: Part | undefined = undefined;
constructor(text: string) {
this.text = text;
}
}
const read = (): Promise<void> => new Promise((resolve) => setImmediate(resolve));
async function handle(id: number): Promise<string> {
let last: Part | undefined;
for (let i = 0; i < 1000; i++) {
if (i % 100 === 0) await read(); // the next chunk of the request body
const part = new Part(`${id}.${i}`);
part.previous = last;
last = part;
}
return `request ${id}: last part ${last!.text}`;
}
async function serve(id: number): Promise<string> {
const region = new LifetimeRegion();
try {
return await region.run(() => handle(id));
} finally {
region.release();
}
}
Promise.all([1, 2, 3].map(serve)).then((responses) => {
for (const response of responses) console.log(response);
});
```
Each request's 1000 parts are freed when the request releases its region:
```console
$ scriptc build requests.ts -o requests >/dev/null
$ SCRIPTC_REGION_STATS=1 ./requests >/dev/null
scriptc region 1: freed 1000 objects, 0 owned objects, 1 chunks (2.0 MiB); cycles 0.0 ms, walk 0.0 ms, free 0.0 ms
scriptc region 2: freed 1000 objects, 0 owned objects, 1 chunks (2.0 MiB); cycles 0.0 ms, walk 0.0 ms, free 0.0 ms
scriptc region 3: freed 1000 objects, 0 owned objects, 1 chunks (2.0 MiB); cycles 0.0 ms, walk 0.0 ms, free 0.0 ms
scriptc regions: 3 created, 3 released, 3 freed, 0 pinned, 0 kept
scriptc region heap passes: 0
```
A region released while async work started inside it is still pending stays allocated until that work has finished, and the work keeps creating its instances in the region. Callbacks that the event loop calls (timers, `queueMicrotask`, event listeners) and the bodies of async generators run outside the region; synchronous generators run in the region of the code that resumes them.
### Threads
@@ -102,7 +157,7 @@ Each thread has its own regions and its own default region. A region belongs to
Programs that never construct a `LifetimeRegion` compile exactly as they would without the package. In programs that do, every retain and release of a `ProgramLifetime` instance that belongs to a region updates a count of the region chunk that holds it. Instances of the default region are not counted, and until the program creates its first region, a retain or release only tests whether counting has started.
Freeing a region visits each of its instances and the ordinary objects they own. A region that the check would keep may be kept only by unreachable cycles of ordinary objects that refer into it, so before keeping it scriptc runs the cycle collector over the thread's heap and checks again.
Freeing a region visits each of its instances and the ordinary objects they own. Each thread keeps up to 8 MiB of the memory of freed regions for its next regions, so a program that frees a region per request or per batch does not map and fault in new pages every time. A region that the check would keep may be kept only by unreachable cycles of ordinary objects that refer into it, so before keeping it scriptc runs the cycle collector over the thread's heap and checks again.
`SCRIPTC_REGION_STATS=1` prints one line per release attempt, freed, `kept` (with the number of references the check could not explain) or `pinned` (published), and at exit a summary per thread and the number of those cycle-collector passes over the thread's heap.
@@ -183,8 +238,99 @@ No lock is held while a callback runs. Two threads that miss the same key at the
2. It does not compare identities or ids across threads.
3. A thread does not depend on another thread's inserts: `size` and `has` reflect every thread's inserts in scriptc and the calling thread's under Node.js.
State that a thread computes about a shared value later (lazily resolved members, caches) belongs in thread-local state, such as a module-level `Map` keyed by the value or an array indexed by an id the value carries: module state is per thread in both runtimes. In scriptc an array indexed by a dense id is the fastest of these.
State that a thread computes about a shared value later (lazily resolved members, caches) belongs in thread-local state, such as a [`ThreadSlots`](#threadslots) family, a module-level `Map` keyed by the value or an array indexed by an id the value carries: module state is per thread in both runtimes. In scriptc an array indexed by a dense id is the fastest of these.
### Memory
Lookups take no lock and write nothing. Inserts lock one of 64 segments of the table for a probe and a few stores. Stored values are immortal in scriptc builds: they stay allocated until the process exits, including a value a callback computed for a key that another thread stored first. Programs that never construct an `InternTable` do not run any of its code.
## ThreadSlots
Threads that share values through an `InternTable` often compute more about each shared value later: resolved members, cached results, flags. That state differs per thread, so it cannot live in the shared value, which is immutable. A `ThreadSlots` family holds one value per key and thread, like a `WeakMap` that each thread has its own copy of. A thread sees only the values it set, including for keys that every thread shares.
```ts:slots.ts
import { Worker, isMainThread, parentPort, threadId } from "node:worker_threads";
import { InternTable, ThreadSlots } from "@scriptc/experimental";
class Shape {
readonly name: string;
constructor(name: string) {
this.name = name;
}
}
class ShapeState {
uses = 0;
readonly thread: number;
constructor(thread: number) {
this.thread = thread;
}
}
const shapes = new InternTable<string, Shape>("example.slot-shapes");
// Module scope: every thread that evaluates this line opens the same family.
const states = new ThreadSlots<Shape, ShapeState>("example.shape-states");
function use(name: string): ShapeState {
const shape = shapes.getOrInsertComputed(name, () => new Shape(name));
let state = states.get(shape);
if (state === undefined) {
state = new ShapeState(threadId);
states.set(shape, state);
}
state.uses++;
return state;
}
if (isMainThread) {
for (let i = 0; i < 2; i++) {
new Worker(new URL(import.meta.url)).on("message", (text: string) => console.log(text));
}
} else {
use("square");
const state = use("square");
parentPort!.postMessage(`worker: ${state.uses} uses, own state ${state.thread === threadId}`);
}
```
In the scriptc executable both workers read the same `Shape` object from the table, and each keeps its own `ShapeState` for it. Both runtimes print the same:
```console
$ node slots.ts
worker: 2 uses, own state true
worker: 2 uses, own state true
$ scriptc build slots.ts -o slots >/dev/null
$ ./slots
worker: 2 uses, own state true
worker: 2 uses, own state true
```
### Slot methods
<dl>
<dt><code>new ThreadSlots&lt;K, V&gt;(name)</code></dt>
<dd>Opens the family called <code>name</code>. Within one thread every construction with the same name refers to the same family.</dd>
<dt><code>slots.get(key)</code>, <code>slots.has(key)</code></dt>
<dd>Read the calling thread's value for <code>key</code>.</dd>
<dt><code>slots.set(key, value)</code></dt>
<dd>Sets the calling thread's value for <code>key</code> and returns the slots. It never writes the key, so it works on published keys.</dd>
<dt><code>slots.delete(key)</code></dt>
<dd>Removes the calling thread's value for <code>key</code> and returns whether there was one.</dd>
</dl>
Under Node.js each family is a `WeakMap` per thread.
### Keys and values
scriptc requires:
- the name to be a string literal, and every construction of one name to use the same type arguments;
- the key type to be a class declared in the program; keys may be instances of its subclasses;
- the value type to be a class, an object type, an array, a `Map` or a `Set`;
- the methods to be called through a module-level `const` initialized with `new ThreadSlots<K, V>("name")`, as in the example. A family's storage is found by its name when the program is compiled.
### Slot cost
Each instance of a key class, and of every class in its hierarchy, carries a 4-byte slot index, 0 until a value is set for it. The first `set` for a key assigns the index; indices are never reused. `get` and `has` read the index, a page of the thread's family and the entry, with no lock, hash or atomic update. A family stores values in pages of 1024 entries, allocated as the thread sets values in them.
A family holds a reference to each of the thread's values. A value is released when it is replaced or deleted, when its key is freed, and when the thread exits. Keys that are freed with a [`LifetimeRegion`](#lifetimeregion) keep their values until the thread exits, a value that belongs to a region keeps the region allocated while the family holds it, and a value that refers to its own key keeps the key alive. Programs that never construct a `ThreadSlots` do not run any of its code, and their classes have no slot index.
docs/content/docs/how-it-works.mdx
+78 -3
View File
@@ -27,11 +27,86 @@ Runtime pack and linker -> executable
1. **Frontend:** TypeScript parses and checks the program against `es2025`, scriptc's ambient declarations, and available project declarations. `tsconfig.json` controls checker strictness. The compiler uses the checked types and narrowing information to lower supported operations; unsupported operations produce diagnostics. JavaScript files that TypeScript checks (a `// @ts-check` comment, or `checkJs` in `tsconfig.json`) report values that do not match their JSDoc types as errors, and their classes use those types the way TypeScript classes use annotations.
2. **Typed IR:** A validated, serializable intermediate representation connects the frontend and backend. Types are concrete: generic functions are specialized, unions use tags, and closures have explicit captures. `--emit=ir` writes this representation as JSON.
3. **LLVM backend:** `--emit=llvm` writes textual LLVM IR. The matching LLVM 22 helper generates assembly or objects on supported macOS, Linux, and Windows hosts, including output targeting WASI. Executable and library builds use this backend. Large executables, native and WASI alike, are divided into partitions that are optimized and compiled in parallel; optimized builds summarize every partition first so small functions can still be inlined across partitions. The partition count depends only on the program, so output is the same on every host, and a rebuild reuses the compiled partitions that an edit does not affect.
3. **LLVM backend:** `--emit=llvm` writes textual LLVM IR. The matching LLVM 22 helper generates assembly or objects on supported macOS, Linux, and Windows hosts, including output targeting WASI. Executable and library builds use this backend. Large executables, native and WASI alike, are divided into partitions that are optimized and compiled in parallel; see [Build parallelism](#build-parallelism).
4. **Linking:** Runtime packs contain the precompiled C runtime and vendored libraries. A hashed manifest selects the artifacts the program needs for its features, optimization mode, and (for libraries) state model. The target linker and SDK or sysroot produce the executable. Sanitizer builds use an instrumented C runtime.
`comptime` calls run during the frontend stage in an isolated evaluator inside the compiler, based on QuickJS. Their results are embedded in the program as constants.
## Build parallelism
Most of a build runs on one CPU core. The LLVM helper uses more cores when it compiles an executable in partitions, and when it generates code for an optimized executable of one partition.
### Frontend
Parsing, type checking, lowering, and LLVM IR generation run one after another on one thread. The type checker runs in a separate process that the compiler waits on. When none of a build's inputs has changed since an earlier build, the frontend's results come from the build cache and this stage is skipped. A change to any input runs the whole stage again.
### Partitions
The LLVM helper divides an executable's LLVM IR into partitions by its size. Up to 1 MiB of LLVM text is one partition, and up to 2 MiB is four. Larger programs get one partition per 4 MiB of LLVM text, rounded up to a power of two, and at least 8 and at most 64. `--emit=llvm` writes this text, so its file size shows how many partitions a program gets.
<table>
<thead>
<tr><th>LLVM text</th><th>Partitions</th></tr>
</thead>
<tbody>
<tr><td>Up to 1 MiB</td><td>1</td></tr>
<tr><td>Over 1 MiB, up to 2 MiB</td><td>4</td></tr>
<tr><td>Over 2 MiB, up to 32 MiB</td><td>8</td></tr>
<tr><td>Over 32 MiB, up to 64 MiB</td><td>16</td></tr>
<tr><td>Over 64 MiB, up to 128 MiB</td><td>32</td></tr>
<tr><td>Over 128 MiB</td><td>64</td></tr>
</tbody>
</table>
The count depends only on the size of the LLVM text, so a program has the same partitions, and builds the same executable, on every host. In programs with 32 partitions or more, functions from the same source file are placed in the same partition, so that their calls to each other stay inside it; a file larger than 1 MiB of LLVM text or than an average partition is divided into parts by function name. Other functions, and the functions of smaller programs, are placed by their name, and a small internal function goes with the one function that calls it. An edit leaves almost every other function in its partition.
These builds are not divided into partitions and compile on one thread: `--emit=asm`, `--emit=obj`, and `--print=native-link-info` output, library builds (`--lib`; `dev` library builds are compiled in pieces, one after another), and `--sanitize` builds, which compile the LLVM IR with the C compiler.
The helper compiles partitions on a pool with one thread per hardware thread of the host, so an executable with four partitions uses at most four cores in the backend.
How partitions are compiled depends on `--optimization`:
- `dev` compiles each partition on its own, all in parallel.
- `release` and `speed` optimize each partition and write a summary of its functions, all in parallel. Then a serial step reads every summary and decides which small functions each partition imports from the others, so that calls between partitions can still be inlined. Last, the partitions are optimized with their imports and compiled, again in parallel.
Reading the LLVM text and dividing it into partitions, and the import step, run on one thread.
### Code generation of one partition
A `release` or `speed` executable of one partition is optimized as one module on one thread. When its LLVM text is larger than 128 KiB, the helper then divides the optimized module for code generation and generates machine code for the parts in parallel, on the same pool. Each part holds a run of consecutive functions with about the same amount of code, so the executable keeps the functions in the same order. The module is optimized before it is divided, so each function's machine code is the same as from one thread, except that a jump to a function that another part also uses takes the longer form. When one function holds most of the code, the first part holds the whole module and the others are empty.
<table>
<thead>
<tr><th>LLVM text</th><th>Code generation parts</th></tr>
</thead>
<tbody>
<tr><td>Up to 128 KiB</td><td>1</td></tr>
<tr><td>Over 128 KiB, up to 256 KiB</td><td>4</td></tr>
<tr><td>Over 256 KiB, up to 1 MiB</td><td>8</td></tr>
</tbody>
</table>
Like partitions, the count depends only on the size of the LLVM text. A `dev` executable of one partition is compiled on one thread.
### Rebuilds
With two or more partitions, the helper keeps each partition's summary and object in the build cache (see `SCRIPTC_CACHE_DIR` in the [CLI reference](/docs/cli#environment-variables)). A rebuild after an edit compiles again only the partitions whose functions changed and, in `release` and `speed` builds, the partitions that import a changed function; the others come from the cache. Dividing the program, the import step, and linking run on every build. A cold build, with nothing cached, compiles every partition.
A program of one partition has no partition cache: a rebuild after an edit compiles the whole program again, and a rebuild that leaves the LLVM text unchanged takes the compiled code from the build cache.
When the LLVM text grows past a partition boundary (4, 8, 16 MiB, and so on), the partition count doubles, functions move to new partitions, and the next build compiles every partition again.
### Profile-guided optimization
Building with a profile adds these costs (see [Profile-guided optimization](/docs/cli#profile-guided-optimization)):
- `--profile-generate` builds an instrumented executable, which runs slower while it records the profile.
- The profile's contents are part of every partition's cache key. A new or changed profile compiles every partition again; builds without a profile keep their own cache entries.
- With a profile, the import step lets a hot call import a function of up to twice the usual size limit. LLVM's default limits for profiled calls are 10 and 100 times as large, and on large programs they tripled the build's time and memory without making the program measurably faster.
- The profile does not change which partition a function is in.
Collecting a profile takes an instrumented build, the training runs, and the merge. With a committed profile, later builds skip those steps and are one optimized build each.
## Inspect compiler output
The following examples use this program:
@@ -76,8 +151,8 @@ Program objects define `main` and leave selected `scr_*` runtime functions undef
## Native runtime
- **Memory:** Values use reference counting. Acyclic values are freed when their last reference is released, and a cycle collector runs at defined collection points. Some cycles involving opaque native values can remain allocated. When the main thread's script ends, the process exits without freeing what the program still holds, and the operating system reclaims it; worker threads free their heaps when they exit. Builds with `--sanitize`, and processes run with `SCRIPTC_EXIT_FREE=1` (for external leak checkers), free everything at exit.
- **Concurrency:** An `async` function that suspends only at its own `await` expressions compiles to a stackless coroutine whose heap frame holds the values live across those awaits; other suspending `async` functions and async generators use stackful fibers. The native event loop schedules microtasks, timers, and I/O, using kqueue on macOS and epoll on Linux.
- **Memory:** Values use reference counting. Acyclic values are freed when their last reference is released, and a cycle collector runs at defined collection points. When the compiler proves that a group of objects is released together, such as a tree that a recursive function builds and its caller only reads, the group is allocated in an arena and released at once, without freeing each object or collecting its cycles. Some cycles involving opaque native values can remain allocated. When the main thread's script ends, the process exits without freeing what the program still holds, and the operating system reclaims it; worker threads free their heaps when they exit. Builds with `--sanitize`, and processes run with `SCRIPTC_EXIT_FREE=1` (for external leak checkers), free everything at exit.
- **Concurrency:** An `async` function or async generator that suspends only at its own `await` and `yield` expressions compiles to a stackless coroutine whose heap frame holds the values live across them; other suspending `async` functions and async generators use stackful fibers, as do debug builds, libraries, worker programs and Windows executables. The native event loop schedules microtasks, timers, and I/O, using kqueue on macOS and epoll on Linux.
- **Networking:** Supported `net`, `http`, `https`, `tls`, `dgram`, and `dns` APIs use native implementations. TLS uses vendored mbedTLS.
- **Numbers:** Values use JavaScript's double-precision representation. Number-to-string formatting uses shortest-roundtrip output and is tested against Node.js.
- **Regular expressions and Unicode:** The runtime uses QuickJS's regular-expression and Unicode libraries, linked into programs that use them.
docs/content/docs/limitations.mdx
+48
View File
@@ -24,6 +24,8 @@ title: "Limitations"
<include>../limitations/language-features/async-iteration.mdx</include>
<include>../limitations/language-features/iterable-values.mdx</include>
<include>../limitations/language-features/generics.mdx</include>
<include>../limitations/language-features/function-values.mdx</include>
@@ -32,6 +34,10 @@ title: "Limitations"
<include>../limitations/language-features/dynamic-imports.mdx</include>
<include>../limitations/language-features/commonjs-require.mdx</include>
<include>../limitations/language-features/jsx.mdx</include>
### Types and class definitions
<include>../limitations/types-and-class-definitions/bigint.mdx</include>
@@ -42,10 +48,14 @@ title: "Limitations"
<include>../limitations/types-and-class-definitions/unions.mdx</include>
<include>../limitations/types-and-class-definitions/dynamic-slots.mdx</include>
<include>../limitations/types-and-class-definitions/null-and-optional-values.mdx</include>
<include>../limitations/types-and-class-definitions/static-field-updates.mdx</include>
<include>../limitations/types-and-class-definitions/array-subclasses.mdx</include>
<include>../limitations/types-and-class-definitions/local-class-expressions.mdx</include>
<include>../limitations/types-and-class-definitions/constructor-function-inheritance.mdx</include>
@@ -62,6 +72,8 @@ title: "Limitations"
<include>../limitations/builtin-apis/eventemitter.mdx</include>
<include>../limitations/builtin-apis/asynclocalstorage.mdx</include>
<include>../limitations/builtin-apis/streams-and-readline.mdx</include>
<include>../limitations/builtin-apis/commonjs-metadata.mdx</include>
@@ -74,8 +86,12 @@ title: "Limitations"
<include>../limitations/builtin-apis/legacy-url-api.mdx</include>
<include>../limitations/builtin-apis/deprecated-modules.mdx</include>
<include>../limitations/builtin-apis/filesystem-flags.mdx</include>
<include>../limitations/builtin-apis/filesystem-url-paths.mdx</include>
<include>../limitations/builtin-apis/copy.mdx</include>
<include>../limitations/builtin-apis/glob.mdx</include>
@@ -96,6 +112,8 @@ title: "Limitations"
<include>../limitations/builtin-apis/process-memory.mdx</include>
<include>../limitations/builtin-apis/v8.mdx</include>
<include>../limitations/builtin-apis/performance-entries.mdx</include>
<include>../limitations/builtin-apis/builtin-module-lookup.mdx</include>
@@ -116,6 +134,8 @@ title: "Limitations"
<include>../limitations/builtin-apis/synchronous-iteration.mdx</include>
<include>../limitations/builtin-apis/iterator-helpers.mdx</include>
<include>../limitations/builtin-apis/regular-expressions.mdx</include>
<include>../limitations/builtin-apis/object-freezing.mdx</include>
@@ -168,6 +188,10 @@ title: "Limitations"
<include>../limitations/buffer-construction.mdx</include>
### readable-stream
<include>../limitations/readable-stream.mdx</include>
### HTTP headers
<include>../limitations/http-headers.mdx</include>
@@ -200,10 +224,22 @@ title: "Limitations"
<include>../limitations/tls-ciphers.mdx</include>
### TLS certificates
<include>../limitations/tls-certificates.mdx</include>
### TLS sessions
<include>../limitations/tls-sessions.mdx</include>
### Console and process streams
<include>../limitations/console-and-process-streams.mdx</include>
### graceful-fs
<include>../limitations/graceful-fs.mdx</include>
### DNS lookup errors
<include>../limitations/dns-lookup-errors.mdx</include>
@@ -216,10 +252,18 @@ title: "Limitations"
<include>../limitations/udp-socket-options.mdx</include>
### Unix domain sockets and request options
<include>../limitations/unix-domain-sockets.mdx</include>
### Process arguments and errors
<include>../limitations/process-arguments-and-errors.mdx</include>
### Module path globals
<include>../limitations/module-path-globals.mdx</include>
<span id="embedded-javascript-engine" />
## Runtime code evaluation
@@ -234,6 +278,10 @@ title: "Limitations"
<include>../limitations/native-addons.mdx</include>
## WebAssembly modules
<include>../limitations/webassembly-modules.mdx</include>
## WASI target
<include>../limitations/wasi-target.mdx</include>
docs/content/docs/platforms.mdx
+1 -1
View File
@@ -49,7 +49,7 @@ $ file fib-linux
fib-linux: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 2.0.0, with debug_info, not stripped
```
`x86_64-linux-gnu.2.36` works the same way. For Alpine and other musl-based systems, use `aarch64-linux-musl` or `x86_64-linux-musl`; these produce statically linked executables.
`x86_64-linux-gnu.2.36` works the same way. For Alpine and other musl-based systems, use `aarch64-linux-musl` or `x86_64-linux-musl`; these produce statically linked executables. On a glibc-based Linux machine, [`--static`](/docs/cli#options) links a static executable for the machine's own target.
Linux executables use the native Linux implementations of the event loop (epoll), servers, TLS (with the system's CA bundle), and `fs.watch`, and they are tested against Node.js on Linux, including Alpine.
docs/content/docs/threads.mdx
+2
View File
@@ -234,6 +234,8 @@ Without `@scriptc/threads`, a scriptc executable also shares a message of untype
Starting a worker thread takes tens of microseconds plus the worker's own module initialization, since every worker initializes the program's modules again, as in Node.js. A worker that exits frees its heap first, which takes time proportional to what it still holds. `SCRIPTC_WORKER_STATS=1` prints, as each worker exits, how long it took to start, to run its script and to exit.
On Linux, a worker that exits without having shared an object it allocated with another thread returns all of its memory, including objects it never freed, such as `ProgramLifetime` instances, once no other worker thread is running. A worker shares objects when, for example, it sends a published value or frozen plain data, which other threads read by reference, or stores a value in an `InternTable`. The memory of a worker that shared stays allocated, as on other platforms. Set `SCR_SA_DISCARD=0` in the environment to keep the memory of every worker.
`publish` visits each object of the graph once and marks it immortal, so publication takes time proportional to the graph. Published objects then cost nothing on later sends, and reference counting and the cycle collector skip them. Stores into the types a program publishes carry one extra load and branch.
Threads that add to one shared collection of published values while they run can use an [`InternTable`](/docs/experimental#interntable) from `@scriptc/experimental`.
docs/content/docs/wasm.mdx
+3 -1
View File
@@ -96,11 +96,13 @@ After initialization, <code>counter_step(2)</code> returns <code>2</code>, and a
<tr><td><code>i64</code>, <code>u64</code></td><td>JavaScript BigInt. The existing library integer checks and proofs restrict values to the exact JavaScript number range.</td></tr>
<tr><td><code>string</code>, <code>bytes</code> parameters</td><td>Two i32 arguments: byte offset and byte length. Strings use UTF-8. Entry parameters are copied into runtime-owned values.</td></tr>
<tr><td><code>string</code>, <code>bytes</code> results</td><td>Two extra i32 arguments identify writable four-byte output slots. The function writes the result pointer and byte length there, in little-endian order.</td></tr>
<tr><td><code>cstring</code> parameters</td><td>One i32 argument: the byte offset of NUL-terminated UTF-8 text. The text is copied into a runtime-owned string; malformed UTF-8 becomes U+FFFD. A zero pointer traps with <code>SC4012</code>.</td></tr>
<tr><td><code>cstring</code> results</td><td>An i32 result: the byte offset of NUL-terminated UTF-8 text in the result arena. A result containing U+0000 traps with <code>SC4015</code> instead of being truncated.</td></tr>
<tr><td><code>void</code></td><td>No result.</td></tr>
</tbody>
</table>
The native library rules about directions still apply: u8/u32/i32 are export parameters and callback scalars; i64/u64 are export-only. Callback results must be scalars or void. Callback string/byte parameters are borrowed only for the duration of that callback; copy them before retaining them in the host.
The native library rules about directions still apply: u8/u32/i32 are export parameters and callback scalars; i64/u64 are export-only. Callback results must be scalars or void. Callback string/byte parameters are borrowed only for the duration of that callback; copy them before retaining them in the host. A <code>cstring</code> callback parameter is one i32 pointer to NUL-terminated text, borrowed the same way; a value containing U+0000 traps with <code>SC4015</code>. The <code>cstring</code> class requires <code>"profile_format": 2</code>. In native libraries it is a <code>const char *</code> with the same rules.
Allocate host input buffers and output slots with <code>scriptc_alloc(size)</code>; a zero result indicates allocation failure. Release each allocation exactly once with <code>scriptc_free(pointer)</code>. Only free pointers returned by <code>scriptc_alloc</code>. These allocations remain live across application calls, initialization, and collection until freed.

Some files were not shown because too many files have changed in this diff Show More