Commit Graph
20 Commits
Author SHA1 Message Date
Chris Tate f591e8e6d7 Print how a --profile-use build's profile matched
`scriptc build --profile-use` prints one line on stderr with the number of
functions the profile annotated, the number whose control flow changed since
it was collected (mismatched), and the number it has no record of. When more
than 10% of the functions the profile describes are mismatched, it also prints
a warning to collect the profile again. The build still succeeds.

Both the compiler library's build and the native compiler's build report the
match. Cached builds report the match of the build that produced them: the
native codegen and object caches store it beside the cached objects, and the
executable cache stores it in its entry.
2026-10-25 22:14:25 -05:00
Chris Tate cc2266ab1d Add --cpu-dispatch for x86-64 executables (#2288) 2026-10-25 21:36:00 -05:00
Chris Tate 07c5e79ffd Add --cpu-dispatch: x86-64-v3 copies of the program's code chosen at startup
With --cpu-dispatch, an x86-64 executable carries two copies of each
program function: the baseline code and a copy compiled for x86-64-v3.
main reads CPUID and XGETBV once. Calls between the program's functions
go to the caller's copy. A function that is also named outside the
copies (a function value, a call from main, a runtime callback or a
coroutine body) keeps a dispatcher under its own name, which tests the
level and tail-calls one copy. Functions without such a use get no
dispatcher, so their copies have the callers the original had and LLVM
inlines them the same way.

The executable still runs on any x86-64 CPU. SCRIPTC_CPU_LEVEL caps the
level at run time. The option is off by default, applies to executables
without a --target-cpu level, and is part of the executable cache key.
bench:runtime takes --baseline-cpu-dispatch and --candidate-cpu-dispatch,
and the LLVM differential suite runs each program at levels 0 and 3 with
SCRIPTC_CPU_DISPATCH=1.
2026-10-25 20:00:16 -05:00
Malte UblandClaude Opus 5.5 7675b7d653 Add scriptc types patch and scriptc types list
`scriptc types patch <package> <selector> <type> [--range] [--dry-run]`
finds the installed package from the working directory, shows every
declaration the selector matches (declaration files, JSDoc, @types
package) with its type before and after, and adds the patch to the
nearest package.json: into an entry whose range covers the installed
version, or under ^<version>. Nothing is written when the selector
matches nothing. `scriptc types list` shows each patch and what it
matches. Both CLI hosts run the same implementation with their own
parser. The TypeError note for a wrong declaration now names the command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 16:00:29 -07:00
Malte UblandClaude Opus 5.5 c46f70e451 Fix wrong npm declarations locally with type patches
A project can now patch a package's declarations in package.json:
"scriptc": { "typePatches": { "<package>@<range>": { "declarations":
{ "<selector>": "<type>" } } } }. A leading "|" widens the declared type,
any other type replaces it. Selectors name declarations by path
(Type.member, fn(), fn(param), list()[], Env[string], export=(), default,
file.js:local()) and apply to the package's .d.ts, its JSDoc and its
@types twin alike.

Patches edit file text where every consumer reads it: the program host's
source reads (both checker worlds) and the npm-static passes that read
package files themselves. Only type nodes and JSDoc type tags change, and
no edit adds or removes a line. Installed copies are selected by version;
an @types twin by the runtime package's installed version. Patches that do
not apply as written warn (SC8030-SC8033) and the warnings repeat on
executable cache hits. Without the setting nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 10:27:52 -07:00
Chris Tate 39216602ab Merge main 2026-10-24 02:39:41 -05:00
Chris Tate e3b42a311d Add --target-cpu to compile x86-64 programs for a microarchitecture level
Release builds compile for the x86-64 baseline, so LLVM vectorizes with
SSE2 only. `--target-cpu=<baseline|x86-64-v2|x86-64-v3|x86-64-v4|native>`
compiles the program's code for a higher level: every emitted function
carries the level as its `target-cpu`, so LLVM can use AVX2, BMI2, FMA or
AVX-512. The executable then runs only on CPUs of that level.

- `native` resolves to the highest level the build machine supports, read
  from /proc/cpuinfo on Linux and `sysctl machdep.cpu` on macOS.
- The level is part of the executable cache key; `baseline` and builds
  without the option produce the same output and keys as before.
- The runtime pack keeps its baseline code.
- Targets other than x86-64 refuse a level (SC3002). The CLI's cached
  bootstrap route leaves builds with the option to the full compiler.
- `bench:runtime` takes `--baseline-target-cpu` and `--candidate-target-cpu`.
2026-10-23 13:10:16 -05:00
Chris Tate ccd45c748f Address the Node island report review findings 2026-10-23 12:21:50 -05:00
Malte UblandClaude Opus 5.5 e61fbf8c0b Merge origin/main into ni-2-node-island-report
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:07:41 -07:00
Chris Tate fa7bc7c7a0 Warn about likely-hot functions that use untyped values
A build now names the functions of the program's own files that compile
to the dynamic tier and likely run often, so a slow program with a little
untyped code says where the time goes:

  warning SC6101: 3 functions that likely run often use untyped values, which runs slower than typed code:
    advance (nbody.js:22): untyped JavaScript file, in a loop
    ...

- The functions come from the tier assignment lowering already makes
  (Lowerer.dynamicFunctions). Without a profile, a function counts as
  likely hot when a dynamic operation runs inside one of its loops, when
  a loop calls it, or when it calls itself (lowering/dynamic-tier.ts).
- A function is named only when its source says why its values are
  untyped: an `any` (for a for...of variable, the value it iterates), an
  untyped JSON.parse result, or an untyped JavaScript file. Generated
  %dyn.* helpers, module top levels and functions that are dynamic for an
  operation without a static lowering are left out.
- At most five functions are named, most likely hot first, with the
  total; dynamic-tier code in npm packages is counted per package on one
  line. All of the warning's text is in diagnostics/dynamic-tier.ts.
- SC6101 opens the SC61xx band, performance notes, with a new
  "performance" category in the diagnostics envelope; SC60xx stay the
  divergences.
- scriptc build prints it after the divergence warnings, and the
  executable cache stores it with the entry, so a cached build (including
  the CLI's routed fast path) repeats it. scriptc coverage reports it, and
  a comment-only edit no longer replays a verdict that carries it, since
  its text holds line numbers.
- --no-dynamic-tier-warning leaves it out of build and coverage;
  coverage --fail-on=dynamic fails on it.
- Docs: coverage.mdx (Dynamic tier, Gating builds and CI, the envelope
  categories) and cli.mdx.
2026-10-21 18:53:43 -05:00
Malte UblandClaude Opus 5.5 091b8f1934 Report which packages and modules would run in a Node island
Adds the report-only Node island policy, configuration and detector of
the scope-B plan (milestone NI-2). Coverage now lists the npm packages a
build would run in a Node island and why, gap islands first, without
changing code generation.

- project-config.ts reads the `scriptc` key of the entry's package.json
  with a schema; an unknown value in `scriptc.packages` or
  `scriptc.nodeIslands` is SC8004 instead of being ignored. Package shims
  and the WebAssembly scanner read through it.
- node-islands/: origin (user, npm, shim, builtin; package instances),
  rules R1.1-R2.5 and the four classes, a syntactic pre-scan for R1/R2
  markers, classification of refusals and runtime fences (reason ids at
  the refusal sites that need them), the policy (auto, explicit, off;
  forced node and native packages; marked user modules) and the report
  with island closure.
- coverage: a "Node islands" section, `--explain-island=<pkg>`, a
  `nodeIslands` field in the diagnostics envelope, and `--fail-on`
  `node-islands`, `eager-node-islands` and `island-gaps`.
- CLI: `--node-islands`, `--node-island`, `--native`, `--node` (validated,
  report-only) and `--strict-user-js` (SC8101-SC8105 for R2 constructs in
  user JavaScript).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:22:35 -07:00
Chris Tate 8065be3ccd Add profile-guided optimization 2026-10-21 03:13:45 -05:00
Malte UblandClaude Opus 5.5 daf4ff6a2b Decode and validate the WebAssembly modules a program can load, and report them in coverage
`scriptc coverage` now lists every WebAssembly module whose bytes the
build can see, with its size, whether it validates, its function, import
and export counts, and the WebAssembly proposals it uses. Nothing is
translated yet; builds are unchanged.

- packages/compiler/src/wasm/binary.ts decodes and validates a module in
  plain TypeScript (the native compiler compiles it too) at the feature
  level scriptc means to translate: the MVP plus sign-extension,
  non-trapping float-to-int, bulk memory, multi-value, mutable globals,
  reference types and extended constant expressions. Where WebAssembly
  3.0 and V8 relaxed a 2.0 rule (global initializers reading defined
  globals, memory indices), it follows V8, the behavior oracle. Modules
  that use SIMD, threads, exception handling, tail calls, function
  references or GC, memory64 or multiple memories still decode, and are
  reported as needing that proposal instead of as invalid.
- packages/compiler/src/wasm/discover.ts finds modules from four sources:
  paths the compiler resolves from the source text (require.resolve,
  createRequire(...).resolve, import.meta.resolve,
  new URL(..., import.meta.url), __dirname joins), inline base64 and byte-array
  literals, the .wasm files of reached packages whose code mentions
  WebAssembly, and a `scriptc.wasm` list in package.json or the new
  coverage-only `--wasm <file>` flag. Modules are reported once, by
  SHA-256, with every source that found them; module files are input
  probes of the coverage verdict cache. The JSON envelope
  (--print=diagnostics) carries the same records under `wasmModules`.

Evidence: packages/compiler/test/wasm-spec.test.ts runs the decoder over
the 3,017 distinct binary modules of the WebAssembly 2.0 core test suite
(without SIMD) and the extended-const proposal's tests, generated by
gen-wasm-spec-cases.mjs with wabt's wast2json, and requires V8's verdict
for each; 8,000 seeded mutations of those modules must also agree with
V8, as must undici's inline llhttp modules (scalar valid, SIMD reported)
and es-module-lexer's. src/wasm/binary.test.ts checks each feature and
each error class against V8. On the survey's real modules the decoder
agrees with V8 for quickjs-wasi (637 KB, 36 ms) and photon (1.9 MB,
121 ms). tests/coverage-fixtures/wasm pins the four discovery sources.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 15:19:39 -07:00
Malte UblandClaude Opus 5.5 577b0135a9 Add execution tiers to the IR and a report-only tier analysis
Stage S2 of the two-tier design. IrFunction gains a tier (absent means
static); lowering assigns it from function bodies, placing functions that
use the dyn vocabulary in the dynamic tier, and the validator enforces
invariant I1: a static function performs no dynamic operation, only
boundary operations (dynFrom, dynCheck, kind tests) and moves. Serialized
IR keeps the field out in this stage and loaders derive it again, so every
IR, LLVM and executable artifact is unchanged.

Every committed refusal and runtime fence becomes a recorded demotion
request attributed to the binding, field, function instance, class or
module the dynamic tier would take over, while the compiler still refuses
and fences as before. `scriptc coverage` reports dynamic-tier statements
(statementsDynamic), `--explain-dynamic[=<name>]` prints the demotion
chains, and `--tier=auto|static-only` is wired through build, run and
coverage for stage S6; both modes compile identically today.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:12:27 -07:00
Chris Tate 7e6294ca0f Make scriptc compilation native-only 2026-10-12 12:25:30 -05:00
Malte Ubl 6a9475e931 Free the address space after process.exit off the critical path (--detached-exit)
`scriptc build|run --detached-exit` opts an executable into handing the address-space teardown after process.exit to a helper process on Linux. The helper shares the address space (CLONE_VM), blocks signals, closes every descriptor but the read end of a pipe and waits for EOF on it. The kernel releases an exiting process's memory (exit_mm) before its files (exit_files), so EOF means the helper holds the last reference; it then exits and the kernel frees the pages there instead of before the parent sees the exit (30-40 ms per GB of 4 KiB pages; tsc-ts 20-110 ms). The same latency mold hides with --fork.

It is a build-time choice, not a run-time knob, because the helper keeps the executable mapped (overwriting the binary right after it exits can fail with ETXTBSY) and the memory charged to the cgroup for up to ~100 ms, and the teardown CPU time leaves the parent's rusage. The program object defines `scr_detached_exit = 1`, overriding the runtime's weak default of 0, so no runtime flavor or linker flag is added. The option is part of the early executable cache and route keys; executables only, no effect on other platforms.
2026-10-11 07:05:29 -07:00
Chris Tate 3e41bcbfd0 Make compile blockers and Node differences actionable (#786) 2026-10-09 10:30:31 -05:00
Malte Ubl 2372b75ec2 Add an opt-in --optimization=speed posture
Speed is a third native optimization posture next to release (default,
-O2) and dev (-O0). It is optimized exactly like release at the C/LLVM
level and is the switch for optimizations that trade executable size and
build time for run-time speed, so the default release build stays inside
the binary-size budget.

- backend/optimization.ts: the NativeOptimization type, its -O class, and
  cache-key helpers. Release keeps its historical key shapes (absent).
- CLI: build/run/cache warm accept speed; usage text names the trade.
  Bootstrap's routed executable cache mirrors the posture.
- Early executable cache keys and native-feature validation include speed,
  so release, dev, and speed entries never satisfy one another.
- Runtime packs: an optional executable `speed` flavor (-O2, only beside
  release). Speed links it when present and the release objects otherwise;
  library modes link their release flavor. The sanitizer lane follows the
  same flavor key.
- Library profiles accept "speed".
- WASI release-only linker flags apply to speed too.
- bench-runtime: --baseline-optimization/--candidate-optimization and a
  per-contender cold_build_ms.

Release executables are byte-identical to the stack tip for all 14
benchmark workloads (macOS arm64, sha256 of executables built from both
trees).
2026-10-08 15:46:34 -05:00
Chris Tate c08ab955ae Enforce linting and consistent source formatting
- Enforce oxlint and oxfmt for owned JavaScript and TypeScript with independent CI checks.
- Preserve compiler import boundaries and protected fixtures while cleaning up lint findings.
- Standardize source formatting and validate import, fixture, and embedded-source protections.
2026-10-05 17:51:24 -05:00
Chris Tate ee9f0d64eb feat: ship the self-hosted native scriptc CLI (#585)
* feat: ship the self-hosted native scriptc CLI

- Run the shared compiler and CLI natively, including library builds and compile-time evaluation.
- Install native platform packages and ship relocatable standalone distributions.
- Preserve program semantics and reuse validated build artifacts for fast rebuilds.

* fix: complete native CLI integration and validation

- Preserve library diagnostics and run recursive compiler work on the main stack.
- Restore packaged helper permissions and exercise native npm and Wasm library builds.
- Recover interrupted Sandbox status probes within the original command deadline.
2026-09-30 02:17:56 -05:00