Commit Graph
72 Commits
Author SHA1 Message Date
Chris Tate 377af617d3 Add --exceptions=rare to unwind through recursive functions
With --exceptions=rare, recursive functions and functions up to three
calls above a throw also propagate exceptions by unwinding on Linux, so
calls to them need no pending-flag test. Functions with a throw of their
own keep the flag. The default, common, keeps the current rules, and
builds without the flag emit the same code as before.

The option is part of the executable cache key; the bootstrap's cached
route leaves builds with it to the full compiler, and coverage and
library builds refuse it.
2026-10-28 14:28:28 -05:00
Chris Tate d2595a4a70 Merge main 2026-10-27 18:49:24 -05:00
Chris Tate 1a86f94239 Merge main 2026-10-27 16:45:14 -05:00
Malte Ubl f9419b5d47 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into HEAD
# Conflicts:
#	packages/compiler/src/index.ts
2026-10-27 05:40:18 -07:00
Chris Tate f4a4cf8652 Merge main 2026-10-27 04:32:00 -05:00
Chris Tate f73722c2ef Add an opt-in --static flag for Linux executables (#2398) 2026-10-27 04:31:35 -05:00
Chris Tate d4185bcd8e Add an opt-in --static flag for Linux executables
- `scriptc build --static` (and the `staticLink` compile option) links glibc
  executables as static PIE on the build host's own glibc target; musl
  executables are already static and accept the flag.
- An executable that keeps glibc's NSS module loader (host, user and group
  name lookups) is refused with SC3002, since a static glibc executable
  loads those modules from the run host.
- Other targets, cross-compiled glibc targets, --sanitize and non-executable
  output are refused; static and dynamic builds use separate cache entries.
2026-10-27 03:34:16 -05:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 60d5dc8ab1 Count class-value constructions in the census, and key caches on both diagnostics
- The heap census also counts instances a class value constructs: the
  construct thunk that `newValue` calls counts them after its allocation,
  as `new` does at its own.
- A build with SCRIPTC_HEAP_CENSUS set is its own executable cache entry
  (`heapCensus` in the early cache's key and route key), and one with
  SCRIPTC_REPR_REPORT set does not read the early cache, since a hit
  emits no module and so writes no report. The CLI's startup route defers
  both to the full compiler.
- Test: a warm cache still builds the census executable, counting
  `new` and class-value constructions, and still writes the report.
2026-10-27 01:54:48 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Malte Ubl 141279b74c Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/13-libnode-packs 2026-10-26 11:15:43 -07:00
Malte Ubl 5817be4d0f Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/ir/function-references.test.ts
2026-10-26 11:08:44 -07:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate d48e060076 Merge main 2026-10-25 22:34:40 -05:00
Chris Tate cc2266ab1d Add --cpu-dispatch for x86-64 executables (#2288) 2026-10-25 21:36:00 -05:00
Chris Tate 07c5e79ffd Add --cpu-dispatch: x86-64-v3 copies of the program's code chosen at startup
With --cpu-dispatch, an x86-64 executable carries two copies of each
program function: the baseline code and a copy compiled for x86-64-v3.
main reads CPUID and XGETBV once. Calls between the program's functions
go to the caller's copy. A function that is also named outside the
copies (a function value, a call from main, a runtime callback or a
coroutine body) keeps a dispatcher under its own name, which tests the
level and tail-calls one copy. Functions without such a use get no
dispatcher, so their copies have the callers the original had and LLVM
inlines them the same way.

The executable still runs on any x86-64 CPU. SCRIPTC_CPU_LEVEL caps the
level at run time. The option is off by default, applies to executables
without a --target-cpu level, and is part of the executable cache key.
bench:runtime takes --baseline-cpu-dispatch and --candidate-cpu-dispatch,
and the LLVM differential suite runs each program at levels 0 and 3 with
SCRIPTC_CPU_DISPATCH=1.
2026-10-25 20:00:16 -05:00
Malte UblandClaude Opus 5.5 7675b7d653 Add scriptc types patch and scriptc types list
`scriptc types patch <package> <selector> <type> [--range] [--dry-run]`
finds the installed package from the working directory, shows every
declaration the selector matches (declaration files, JSDoc, @types
package) with its type before and after, and adds the patch to the
nearest package.json: into an entry whose range covers the installed
version, or under ^<version>. Nothing is written when the selector
matches nothing. `scriptc types list` shows each patch and what it
matches. Both CLI hosts run the same implementation with their own
parser. The TypeError note for a wrong declaration now names the command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 16:00:29 -07:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate 1b40ac0c64 Merge main 2026-10-25 05:15:58 -05:00
Malte UblandClaude Opus 5.5 4a8a6567ca Add the shape overlay: typed JavaScript from analysis-proven facts (opt-in)
The shape overlay (DESIGN.md, approved route) rewrites the JavaScript of
opted-in npm packages with JSDoc types the shape analysis proves: record
typedefs for non-escaping literal sites (optional fields for spread
sources), class field, parameter, return and local types, receiver casts,
and branches the analysis proved unreachable replaced by checks. The
lowering trusts only annotations carrying the overlay's marker; package
JSDoc stays a hint (its type tags are neutralized in overlaid files).

- Inferred-fact checks: every checked conversion lowered in an overlaid
  file records its site, and a failure throws a distinct internal error
  ("scriptc internal error: an inferred type was contradicted at <file:line>
  … This is a bug in scriptc's shape analysis, not in the program's data")
  naming the opt-outs; SCRIPTC_SHAPE_CHECK_LOG makes checks log instead
  (validation mode).
- Opt-outs: `--no-shape-overlay`, and `"scriptc": {"packages": {"<name>":
  {"shapes": false}}}` per package.
- Build report: SC6103 counts the declarations typed, the checks inserted
  and the branches replaced.
- `x instanceof Promise` on plain data folds to false.

The overlay runs only with SCRIPTC_SHAPE_OVERLAY=1 while it is new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:24:51 -07:00
Chris Tate 1559fb1d22 Merge main 2026-10-24 04:03:19 -05:00
Chris Tate 39216602ab Merge main 2026-10-24 02:39:41 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate f0448238f1 Keep libnode pack manifest paths inside the bundle 2026-10-23 14:06:42 -05:00
Chris Tate e3b42a311d Add --target-cpu to compile x86-64 programs for a microarchitecture level
Release builds compile for the x86-64 baseline, so LLVM vectorizes with
SSE2 only. `--target-cpu=<baseline|x86-64-v2|x86-64-v3|x86-64-v4|native>`
compiles the program's code for a higher level: every emitted function
carries the level as its `target-cpu`, so LLVM can use AVX2, BMI2, FMA or
AVX-512. The executable then runs only on CPUs of that level.

- `native` resolves to the highest level the build machine supports, read
  from /proc/cpuinfo on Linux and `sysctl machdep.cpu` on macOS.
- The level is part of the executable cache key; `baseline` and builds
  without the option produce the same output and keys as before.
- The runtime pack keeps its baseline code.
- Targets other than x86-64 refuse a level (SC3002). The CLI's cached
  bootstrap route leaves builds with the option to the full compiler.
- `bench:runtime` takes `--baseline-target-cpu` and `--candidate-target-cpu`.
2026-10-23 13:10:16 -05:00
Chris Tate 1562d08609 Merge the loader branch, key the build by the libnode pack, and pick musl packs 2026-10-23 13:09:08 -05:00
Chris Tate c0e24cdb3f Embed island modules, bound V8's stack scan in-thread, and keep Node's environment out 2026-10-23 12:52:57 -05:00
Malte Ubl 680af076cf Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/index.ts
2026-10-21 20:07:52 -07:00
Malte UblandClaude Opus 5.5 e61fbf8c0b Merge origin/main into ni-2-node-island-report
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:07:41 -07:00
Chris Tate fa7bc7c7a0 Warn about likely-hot functions that use untyped values
A build now names the functions of the program's own files that compile
to the dynamic tier and likely run often, so a slow program with a little
untyped code says where the time goes:

  warning SC6101: 3 functions that likely run often use untyped values, which runs slower than typed code:
    advance (nbody.js:22): untyped JavaScript file, in a loop
    ...

- The functions come from the tier assignment lowering already makes
  (Lowerer.dynamicFunctions). Without a profile, a function counts as
  likely hot when a dynamic operation runs inside one of its loops, when
  a loop calls it, or when it calls itself (lowering/dynamic-tier.ts).
- A function is named only when its source says why its values are
  untyped: an `any` (for a for...of variable, the value it iterates), an
  untyped JSON.parse result, or an untyped JavaScript file. Generated
  %dyn.* helpers, module top levels and functions that are dynamic for an
  operation without a static lowering are left out.
- At most five functions are named, most likely hot first, with the
  total; dynamic-tier code in npm packages is counted per package on one
  line. All of the warning's text is in diagnostics/dynamic-tier.ts.
- SC6101 opens the SC61xx band, performance notes, with a new
  "performance" category in the diagnostics envelope; SC60xx stay the
  divergences.
- scriptc build prints it after the divergence warnings, and the
  executable cache stores it with the entry, so a cached build (including
  the CLI's routed fast path) repeats it. scriptc coverage reports it, and
  a comment-only edit no longer replays a verdict that carries it, since
  its text holds line numbers.
- --no-dynamic-tier-warning leaves it out of build and coverage;
  coverage --fail-on=dynamic fails on it.
- Docs: coverage.mdx (Dynamic tier, Gating builds and CI, the envelope
  categories) and cli.mdx.
2026-10-21 18:53:43 -05:00
Chris Tate 692569a804 Collect the compiler's heap as lowering and LLVM emission grow it 2026-10-21 15:50:32 -05:00
Malte UblandClaude Opus 5.5 091b8f1934 Report which packages and modules would run in a Node island
Adds the report-only Node island policy, configuration and detector of
the scope-B plan (milestone NI-2). Coverage now lists the npm packages a
build would run in a Node island and why, gap islands first, without
changing code generation.

- project-config.ts reads the `scriptc` key of the entry's package.json
  with a schema; an unknown value in `scriptc.packages` or
  `scriptc.nodeIslands` is SC8004 instead of being ignored. Package shims
  and the WebAssembly scanner read through it.
- node-islands/: origin (user, npm, shim, builtin; package instances),
  rules R1.1-R2.5 and the four classes, a syntactic pre-scan for R1/R2
  markers, classification of refusals and runtime fences (reason ids at
  the refusal sites that need them), the policy (auto, explicit, off;
  forced node and native packages; marked user modules) and the report
  with island closure.
- coverage: a "Node islands" section, `--explain-island=<pkg>`, a
  `nodeIslands` field in the diagnostics envelope, and `--fail-on`
  `node-islands`, `eager-node-islands` and `island-gaps`.
- CLI: `--node-islands`, `--node-island`, `--native`, `--node` (validated,
  report-only) and `--strict-user-js` (SC8101-SC8105 for R2 constructs in
  user JavaScript).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:22:35 -07:00
Malte UblandClaude Opus 5.5 9a0a3f1562 Load Node.js in-process on first use: libnode bridge and runtime loader
Scope-B milestone NI-1, runtime side. Programs whose IR has a
`nodeIsland.*` operation link scr_node_island.c and the SCR_NODE_ISLANDS
build of scr_async.c; every other program links neither, byte for byte.

- scr_node_island.c dlopens the bridge of a libnode pack on the first call
  (RTLD_NOW | RTLD_GLOBAL), checks its bridge ABI and that it links exactly
  the Node version recorded at build time, starts one Node environment and
  runs the embedded bootstrap (scr_node_island_host.cjs). A run that never
  calls into Node never maps libnode. The pack directory is SCRIPTC_LIBNODE
  at run time, else the recorded path; a `$ORIGIN/` path resolves against
  the executable's directory (the beside packaging, and the hook for
  shipping Node with the program).
- scr_node_bridge.h is the C ABI between the runtime and the bridge; the
  runtime includes no Node or V8 header. The bridge
  (packages/libnode-common/bridge/scr_node_bridge.cc) initializes Node once
  with no default signal handling, no debug-signal handler or inspector,
  no stdio reinitialization and one platform worker, creates the
  environment from the built-in snapshot, routes Node's process exit to
  the runtime's exit sequence, promotes libnode to the global scope for
  native addons, and runs calls in a CallbackScope with a TryCatch.
  build-pack.mjs assembles a pack (libnode, the bridge, manifest.json) from
  a Node --shared build and the matching headers.
- The SCR_NODE_ISLANDS build of scr_async.c adds a loop slot through which
  the runtime's loop drives Node's until NI-3 hosts the runtime's queues on
  Node's libuv: it keeps the program alive while Node has ref'd work,
  sleeps on Node's backend fd up to its next timer, and runs one
  non-blocking turn per loop turn.
- The `nodeIsland` runtime feature (rule, link features, unit table, pack
  matrix row and scr_async.c variant), the nodeIsland.call/callSync
  libCalls, the install call in main, and the build-time record of the
  pack (node-islands/libnode.ts: --libnode=<pack>|beside or
  SCRIPTC_LIBNODE, SC8005 when none is usable, Node >=24.13.1).
- The zero-cost gate and the bench nm test require that no good-code
  executable defines a Node island symbol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:54:47 -07:00
Chris Tate 8065be3ccd Add profile-guided optimization 2026-10-21 03:13:45 -05:00
Chris Tate 6c07441c24 Check programs with tsc-ts by default
scriptc's frontend talks to a TypeScript 7 API server. tsc-ts serves the
same API as tsgo, and it is now the default checker; SCRIPTC_CHECKER=tsgo
selects tsgo, which scriptc still includes.

- @scriptc/compiler depends on tsc-ts 0.0.3. The native distribution
  ships its platform package as lib/tsc-ts, named by a new tsc_ts field
  of the toolchain manifest; a manifest without it takes the executable
  of an installed tsc-ts package.
- The executable cache key includes the tsc-ts executable's identity, in
  the compiler and in the CLI's startup route.
- tsc-ts keeps the syntax trees it builds for the life of its process,
  so the source parser starts a new server after 256 KB of parsed
  source instead of letting one server grow with every file.
- A CI job compiles every corpus program with both checkers, in release
  and dev, and requires the same LLVM IR and diagnostics
  (scripts/checker-identity.mjs).
- Tests that compare with the TypeScript 7 SDK, the checker-panic
  diagnostics, and the protocol tests run against tsgo.
2026-10-18 19:55:52 -05:00
Chris Tate 7e6294ca0f Make scriptc compilation native-only 2026-10-12 12:25:30 -05:00
Malte Ubl 6a9475e931 Free the address space after process.exit off the critical path (--detached-exit)
`scriptc build|run --detached-exit` opts an executable into handing the address-space teardown after process.exit to a helper process on Linux. The helper shares the address space (CLONE_VM), blocks signals, closes every descriptor but the read end of a pipe and waits for EOF on it. The kernel releases an exiting process's memory (exit_mm) before its files (exit_files), so EOF means the helper holds the last reference; it then exits and the kernel frees the pages there instead of before the parent sees the exit (30-40 ms per GB of 4 KiB pages; tsc-ts 20-110 ms). The same latency mold hides with --fork.

It is a build-time choice, not a run-time knob, because the helper keeps the executable mapped (overwriting the binary right after it exits can fail with ETXTBSY) and the memory charged to the cgroup for up to ~100 ms, and the teardown CPU time leaves the parent's rusage. The program object defines `scr_detached_exit = 1`, overriding the runtime's weak default of 0, so no runtime flavor or linker flag is added. The option is part of the early executable cache and route keys; executables only, no effect on other platforms.
2026-10-11 07:05:29 -07:00
Chris Tate 3e41bcbfd0 Make compile blockers and Node differences actionable (#786) 2026-10-09 10:30:31 -05:00
Malte Ubl 2372b75ec2 Add an opt-in --optimization=speed posture
Speed is a third native optimization posture next to release (default,
-O2) and dev (-O0). It is optimized exactly like release at the C/LLVM
level and is the switch for optimizations that trade executable size and
build time for run-time speed, so the default release build stays inside
the binary-size budget.

- backend/optimization.ts: the NativeOptimization type, its -O class, and
  cache-key helpers. Release keeps its historical key shapes (absent).
- CLI: build/run/cache warm accept speed; usage text names the trade.
  Bootstrap's routed executable cache mirrors the posture.
- Early executable cache keys and native-feature validation include speed,
  so release, dev, and speed entries never satisfy one another.
- Runtime packs: an optional executable `speed` flavor (-O2, only beside
  release). Speed links it when present and the release objects otherwise;
  library modes link their release flavor. The sanitizer lane follows the
  same flavor key.
- Library profiles accept "speed".
- WASI release-only linker flags apply to speed too.
- bench-runtime: --baseline-optimization/--candidate-optimization and a
  per-contender cold_build_ms.

Release executables are byte-identical to the stack tip for all 14
benchmark workloads (macOS arm64, sha256 of executables built from both
trees).
2026-10-08 15:46:34 -05:00
Chris Tate c08ab955ae Enforce linting and consistent source formatting
- Enforce oxlint and oxfmt for owned JavaScript and TypeScript with independent CI checks.
- Preserve compiler import boundaries and protected fixtures while cleaning up lint findings.
- Standardize source formatting and validate import, fixture, and embedded-source protections.
2026-10-05 17:51:24 -05:00
Chris Tate 4bd14bc101 perf: defer native driver discovery on CLI cache misses (#639) 2026-10-03 13:43:33 -05:00
Chris Tate ee9f0d64eb feat: ship the self-hosted native scriptc CLI (#585)
* feat: ship the self-hosted native scriptc CLI

- Run the shared compiler and CLI natively, including library builds and compile-time evaluation.
- Install native platform packages and ship relocatable standalone distributions.
- Preserve program semantics and reuse validated build artifacts for fast rebuilds.

* fix: complete native CLI integration and validation

- Preserve library diagnostics and run recursive compiler work on the main stack.
- Restore packaged helper permissions and exercise native npm and Wasm library builds.
- Recover interrupted Sandbox status probes within the original command deadline.
2026-09-30 02:17:56 -05:00
Chris Tate c7b11a9242 Compile embeddable Wasm modules with host imports
- Emit callable Wasm reactors with explicit exports, host callbacks, and independent instance memory.
- Compile three.js scene and geometry workloads into Wasm without an embedded JavaScript engine.
- Document initialization, memory ownership, and external browser hosting.
2026-09-30 01:09:15 -05:00
Chris Tate 3f66c8e20b Make LLVM the sole production backend
- Remove C emission, debugging options, and automatic backend fallback.
- Complete LLVM lowering and native bootstrap coverage across supported targets.
- Link executables and libraries with precompiled C runtime packs, including iOS and Android.
2026-09-29 20:54:09 -05:00
Chris Tateandtimokosonen 0e6a13d041 fix(compiler): support source breakpoints in dev builds (#481)
- Emit source locations for native TypeScript and JavaScript debugging across C and LLVM backends.
- Preserve macOS debug symbols through linking and cache reuse, with regression coverage and Xcode guidance.

Fixes #461

Co-authored-by: timokosonen <325538238+timokosonen@users.noreply.github.com>
2026-09-26 22:21:11 -05:00
Chris Tate 0c109d93f8 Add opt-in executable stripping (#425)
- Link smaller executables across native targets with an explicit --strip flag.

- Keep stripped and default builds separate in the cache and cover their behavior.
2026-09-25 09:12:58 -05:00
Chris Tate 9f48bc720a Preserve artifacts when switching build modes (#415)
- Keep generated outputs from earlier build modes, including cache hits.
- Cover WASI/native switching and document accumulated output files.
2026-09-24 23:45:17 -05:00
Chris TateandMateus Mamede Lage b6bd80c391 fix(windows): support GUI executables (#380)
* fix(windows): support GUI executables

- Add Windows subsystem selection across executable link paths and cache keys.
- Verify PE subsystem output and add Windows CI coverage.

Fixes #259

Co-authored-by: Mateus Mamede Lage <23098414+mmamedel@users.noreply.github.com>

* fix(ci): restore cross-platform test parity

- Make the Windows subsystem test use a portable loader URL.
- Sync TS7 order-parity baselines with current corpus fixtures.

---------

Co-authored-by: Mateus Mamede Lage <23098414+mmamedel@users.noreply.github.com>
2026-09-22 20:40:11 -05:00
Chris Tate 2bd2e6f54c Refactor container lowering into focused modules (#325)
* refactor(compiler): split container lowering modules

* chore: clear remaining lint errors
2026-09-14 21:33:20 -05:00
Chris Tate c41f1a8be9 Expand LLVM native output targets (#280)
* Expand LLVM native output targets

- Add owned helper and runtime-pack contracts for macOS x64, Linux glibc/musl, Windows x64, and WASI.
- Generalize LLVM target selection, package validation, linker plans, and release/CI matrices.
- Document linker boundaries and add focused native-output and runtime-pack coverage.

* Pin Windows LLVM helper CI to VS 2022

The official LLVM archive needs the Visual Studio 2022 CMake generator, which is not guaranteed by windows-latest.

* Align bootstrap cache with LLVM runtime-pack builds

Use the same target-specific helper and linker identity before and after compiler loading so routed executable cache hits remain lightweight.

* Install Windows LLVM CI tools with Chocolatey

windows-2022 includes Visual Studio but not winget; use its available Chocolatey bootstrap for CMake and Ninja.

* Prevent Zig version probes from leaving runtime-pack objects

Run runtime-pack compiler version probes in a private temporary directory and remove the accidental tracked WASI a.o file.

* Use 7-Zip for Windows LLVM setup

Extract the official LLVM development archive with two-stage 7-Zip instead of Windows tar.exe, which timed out while materializing the toolchain tree.

* Define ssize_t for the MSVC runtime pack

Clang's MSVC target does not expose POSIX ssize_t through sys/types.h; define the pointer-sized runtime type without affecting MinGW.

* Build the Windows runtime pack with Zig

Use the MinGW-compatible Windows sysroot required by the runtime while retaining COFF helper output and installing Zig in the Windows native CI lane.

* Give macOS LLVM differential shard time to finish

Shard 1 passed its setup and focused contracts but was cancelled during its cold LLVM differential slice at the 20-minute job limit.
2026-09-01 13:41:34 -05:00