Commit Graph
66 Commits
Author SHA1 Message Date
Chris Tate 377af617d3 Add --exceptions=rare to unwind through recursive functions
With --exceptions=rare, recursive functions and functions up to three
calls above a throw also propagate exceptions by unwinding on Linux, so
calls to them need no pending-flag test. Functions with a throw of their
own keep the flag. The default, common, keeps the current rules, and
builds without the flag emit the same code as before.

The option is part of the executable cache key; the bootstrap's cached
route leaves builds with it to the full compiler, and coverage and
library builds refuse it.
2026-10-28 14:28:28 -05:00
Chris Tate d9481023c3 Annotate functions a stale profile misses from its block counts
A profile annotates a function only while its control flow hashes as it did
when the profile was collected, so compiler and source changes leave the
changed functions without counts. A --profile-use build now reads a
block-count file beside the profile (app.blocks beside app.profdata). Right
after the annotation before inlining, the helper aligns each uncounted
function's blocks with its recorded blocks by label kind and successor
count. Paired blocks take their recorded counts and their edges the recorded
edge counts, by target or, between targets whose labels changed, by
position. LLVM's flow inference fills in the rest, avoiding the edges the
function's own llvm.expect weights make unlikely. The helper then sets the
entry count, branch weights and hot or cold attribute as the profile
annotation does.

SCRIPTC_RECORD_PROFILE_BLOCKS names the file a --profile-use build writes
for the functions its profile annotates; such a build reuses no cached
artifacts. The block file keys every cache, the build report counts the
functions it annotated, and the staleness warning counts only the functions
left without counts. A single-module build with a context-sensitive profile
no longer counts annotated, mismatched and missing functions a second time
after inlining. scripts/refresh-cli-profile.mjs records the file in the
second phase and build-native-cli.mts reads it beside the profile.
2026-10-28 04:50:01 -05:00
Chris Tate 7d670f0e2a Merge main 2026-10-27 18:53:59 -05:00
Chris Tate 1a86f94239 Merge main 2026-10-27 16:45:14 -05:00
Chris Tate 18196d4df0 Lay out code by context-sensitive profiles 2026-10-27 09:36:18 -05:00
Malte Ubl caee8dfef2 Merge the restacked base into ni/i2-handles
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/index.ts
2026-10-27 05:41:01 -07:00
Chris Tate d4185bcd8e Add an opt-in --static flag for Linux executables
- `scriptc build --static` (and the `staticLink` compile option) links glibc
  executables as static PIE on the build host's own glibc target; musl
  executables are already static and accept the flag.
- An executable that keeps glibc's NSS module loader (host, user and group
  name lookups) is refused with SC3002, since a static glibc executable
  loads those modules from the run host.
- Other targets, cross-compiled glibc targets, --sanitize and non-executable
  output are refused; static and dynamic builds use separate cache entries.
2026-10-27 03:34:16 -05:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 140990d257 Add --large-pages: move executable code onto transparent huge pages
A large program's hot code spans hundreds of 4 KiB pages, more than the
instruction TLB holds. tsc-ts has 12 MB of code, and 90% of its
main-thread samples at --checkers 1 fall in 274 distinct code pages.

`scriptc build --large-pages` links the new runtime unit
scr_text_pages.c, whose constructor moves the executable's code onto
2 MiB transparent huge pages before main. It copies the executable
segment, from its first to its last 2 MiB boundary, into anonymous memory
advised MADV_HUGEPAGE, makes the copy read-only and executable, and moves
it over the original range with one mremap, so addresses and instructions
stay the same. The move runs only on Linux, only when THP is "always" or
"madvise", and only in executables. If a step fails, the code stays
file-mapped. Like Node's --use-largepages, it is off by default: without
the flag the unit is not linked, and executables are unchanged.

The move costs about 3 ms per 10 MiB of code at startup, and all of the
moved code stays resident as private memory. The flag is part of the
executable cache key and the runtime-pack selection, and is refused for
non-executable output, like --detached-exit.
2026-10-26 23:41:46 -05:00
Malte Ubl 437ad4ae38 Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD 2026-10-26 16:00:05 -07:00
Chris Tate 2e627dab79 Merge main 2026-10-26 14:45:37 -05:00
Malte Ubl 3a5aab5b2f Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD
# Conflicts:
#	packages/compiler/src/cli/command.ts
#	packages/compiler/src/executable/executable-cache.ts
#	packages/compiler/src/frontend/lowering/builtins/module-resolution.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/compiler/src/index.ts
#	packages/runtime/src/scr_json.c
#	tests/harness/npm.test.ts
2026-10-26 11:22:14 -07:00
Chris Tate de3a75ff81 Push localized failure down to the failing component of a declared type (#2307) 2026-10-26 13:10:05 -05:00
Malte Ubl 5817be4d0f Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/ir/function-references.test.ts
2026-10-26 11:08:44 -07:00
Malte Ubl 572e7e3eac Merge remote-tracking branch 'origin/lf/l4-expressions' into lf/l5-polish
# Conflicts:
#	packages/compiler/src/frontend/lowering/lowerer.ts
2026-10-26 05:37:00 -07:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate 0ba138b57e Merge main 2026-10-26 03:25:54 -05:00
Chris Tate 7a8ec3f3d2 Hold locals and module globals whose declared type does not compile as checked dynamic slots (#2292) 2026-10-26 02:49:06 -05:00
Chris Tate 6817a24581 Merge lf/l0-report and main 2026-10-25 23:49:52 -05:00
Chris Tate d48e060076 Merge main 2026-10-25 22:34:40 -05:00
Chris Tate f591e8e6d7 Print how a --profile-use build's profile matched
`scriptc build --profile-use` prints one line on stderr with the number of
functions the profile annotated, the number whose control flow changed since
it was collected (mismatched), and the number it has no record of. When more
than 10% of the functions the profile describes are mismatched, it also prints
a warning to collect the profile again. The build still succeeds.

Both the compiler library's build and the native compiler's build report the
match. Cached builds report the match of the build that produced them: the
native codegen and object caches store it beside the cached objects, and the
executable cache stores it in its entry.
2026-10-25 22:14:25 -05:00
Malte UblandClaude Opus 5.5 1ae5f105ca Rank dynamic slots by their dynamic operations, name them in SC6101, and document them
Stage L5 of the localized-failure design.

- SC6101 names a dynamic slot as the cause of a likely-hot untyped function
  ('tool' is a dynamic slot, declared 'ToolDefinition', which does not
  compile).
- Coverage counts the dynamic operations each slot's values take part in,
  lists the costly slots first, and puts the count in the JSON envelope.
- A build whose only performance warnings are dynamic slots closes with a
  line about declared types instead of function types.
- Docs: a Dynamic slots section in the coverage guide, the limitations
  page bullet, and --fail-on=dynamic-slots in the CLI reference and usage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 19:48:18 -07:00
Chris Tate cc2266ab1d Add --cpu-dispatch for x86-64 executables (#2288) 2026-10-25 21:36:00 -05:00
Chris Tate 28b76f7b25 Merge ni/i1-stack-bound 2026-10-25 21:08:37 -05:00
Malte UblandClaude Opus 5.5 ac31a15a62 Hold locals and module globals whose declared type does not compile as checked dynamic slots
Stage L1 of the localized-failure design.

A local, loop variable or module global whose declared TypeScript type has
no static lowering used to reject its declaration statement, and every
later use of the binding failed with it. Where that failure is about to
enter the build (the statement's poison in a TypeScript file, or the
module global's registration), the failing type now joins the lowerer's
degraded types and the unit lowers again: the type maps to the checked
dynamic wherever its strict mapping fails, so the slot holds values by
reference (typed capsules, live records and arrays, function thunks) and
typed reads out of it are checked. Each such slot is an SC6104
performance warning, listed under coverage's statement counts and in the
JSON envelope; coverage --fail-on=dynamic-slots fails on them, and
--tier=static-only keeps today's refusals.

A program that compiles today reaches neither commit point, so nothing
degrades and its output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:35:38 -07:00
Chris Tate 07c5e79ffd Add --cpu-dispatch: x86-64-v3 copies of the program's code chosen at startup
With --cpu-dispatch, an x86-64 executable carries two copies of each
program function: the baseline code and a copy compiled for x86-64-v3.
main reads CPUID and XGETBV once. Calls between the program's functions
go to the caller's copy. A function that is also named outside the
copies (a function value, a call from main, a runtime callback or a
coroutine body) keeps a dispatcher under its own name, which tests the
level and tail-calls one copy. Functions without such a use get no
dispatcher, so their copies have the callers the original had and LLVM
inlines them the same way.

The executable still runs on any x86-64 CPU. SCRIPTC_CPU_LEVEL caps the
level at run time. The option is off by default, applies to executables
without a --target-cpu level, and is part of the executable cache key.
bench:runtime takes --baseline-cpu-dispatch and --candidate-cpu-dispatch,
and the LLVM differential suite runs each program at levels 0 and 3 with
SCRIPTC_CPU_DISPATCH=1.
2026-10-25 20:00:16 -05:00
Malte UblandClaude Opus 5.5 7675b7d653 Add scriptc types patch and scriptc types list
`scriptc types patch <package> <selector> <type> [--range] [--dry-run]`
finds the installed package from the working directory, shows every
declaration the selector matches (declaration files, JSDoc, @types
package) with its type before and after, and adds the patch to the
nearest package.json: into an entry whose range covers the installed
version, or under ^<version>. Nothing is written when the selector
matches nothing. `scriptc types list` shows each patch and what it
matches. Both CLI hosts run the same implementation with their own
parser. The TypeError note for a wrong declaration now names the command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 16:00:29 -07:00
Chris Tate cbe4a9387b Merge main 2026-10-25 15:24:25 -05:00
Chris Tate 826519133f Merge main 2026-10-25 14:49:01 -05:00
Malte UblandClaude Opus 5.5 c46f70e451 Fix wrong npm declarations locally with type patches
A project can now patch a package's declarations in package.json:
"scriptc": { "typePatches": { "<package>@<range>": { "declarations":
{ "<selector>": "<type>" } } } }. A leading "|" widens the declared type,
any other type replaces it. Selectors name declarations by path
(Type.member, fn(), fn(param), list()[], Env[string], export=(), default,
file.js:local()) and apply to the package's .d.ts, its JSDoc and its
@types twin alike.

Patches edit file text where every consumer reads it: the program host's
source reads (both checker worlds) and the npm-static passes that read
package files themselves. Only type nodes and JSDoc type tags change, and
no edit adds or removes a line. Installed copies are selected by version;
an @types twin by the runtime package's installed version. Patches that do
not apply as written warn (SC8030-SC8033) and the warnings repeat on
executable cache hits. Without the setting nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 10:27:52 -07:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate 1b40ac0c64 Merge main 2026-10-25 05:15:58 -05:00
Chris Tate f3750ef976 Merge base and fix 2026-10-25 02:16:24 -05:00
Malte UblandClaude Opus 5.5 4a8a6567ca Add the shape overlay: typed JavaScript from analysis-proven facts (opt-in)
The shape overlay (DESIGN.md, approved route) rewrites the JavaScript of
opted-in npm packages with JSDoc types the shape analysis proves: record
typedefs for non-escaping literal sites (optional fields for spread
sources), class field, parameter, return and local types, receiver casts,
and branches the analysis proved unreachable replaced by checks. The
lowering trusts only annotations carrying the overlay's marker; package
JSDoc stays a hint (its type tags are neutralized in overlaid files).

- Inferred-fact checks: every checked conversion lowered in an overlaid
  file records its site, and a failure throws a distinct internal error
  ("scriptc internal error: an inferred type was contradicted at <file:line>
  … This is a bug in scriptc's shape analysis, not in the program's data")
  naming the opt-outs; SCRIPTC_SHAPE_CHECK_LOG makes checks log instead
  (validation mode).
- Opt-outs: `--no-shape-overlay`, and `"scriptc": {"packages": {"<name>":
  {"shapes": false}}}` per package.
- Build report: SC6103 counts the declarations typed, the checks inserted
  and the branches replaced.
- `x instanceof Promise` on plain data folds to false.

The overlay runs only with SCRIPTC_SHAPE_OVERLAY=1 while it is new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:24:51 -07:00
Chris Tate e6790def72 Merge main 2026-10-24 20:06:31 -05:00
Chris Tate 39216602ab Merge main 2026-10-24 02:39:41 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate e3b42a311d Add --target-cpu to compile x86-64 programs for a microarchitecture level
Release builds compile for the x86-64 baseline, so LLVM vectorizes with
SSE2 only. `--target-cpu=<baseline|x86-64-v2|x86-64-v3|x86-64-v4|native>`
compiles the program's code for a higher level: every emitted function
carries the level as its `target-cpu`, so LLVM can use AVX2, BMI2, FMA or
AVX-512. The executable then runs only on CPUs of that level.

- `native` resolves to the highest level the build machine supports, read
  from /proc/cpuinfo on Linux and `sysctl machdep.cpu` on macOS.
- The level is part of the executable cache key; `baseline` and builds
  without the option produce the same output and keys as before.
- The runtime pack keeps its baseline code.
- Targets other than x86-64 refuse a level (SC3002). The CLI's cached
  bootstrap route leaves builds with the option to the full compiler.
- `bench:runtime` takes `--baseline-target-cpu` and `--candidate-target-cpu`.
2026-10-23 13:10:16 -05:00
Chris Tate ccd45c748f Address the Node island report review findings 2026-10-23 12:21:50 -05:00
Chris Tate fe452bdb03 Merge main and harden the Node island loader 2026-10-23 11:04:36 -05:00
Malte Ubl 17218d8c38 Merge remote-tracking branch 'origin/ni/i1-stack-bound' into HEAD 2026-10-23 08:07:47 -07:00
Malte Ubl 2d518aa680 Merge remote-tracking branch 'origin/ni-2-node-island-report' into HEAD 2026-10-23 08:06:55 -07:00
Malte Ubl 433cbaf3ac Merge remote-tracking branch 'origin/main' into HEAD 2026-10-23 07:54:16 -07:00
Chris Tate 967f2ec85d Compile minified npm entries and untyped regex arguments
Refs #1777

npm packages are no longer refused up front for minified JavaScript. A
bundle whose interop helpers lost their names is still refused by the
bundle recognizers, and a package whose attempt fails is still refused on
its own.

A `var` read in a later declarator of the statement that stores it
(`var a = {...}, b = {...a}`) runs after the store. A read in a function
body is now located at the body's own statement instead of the body, so
JavaScript `var` bindings read after their store in a function keep their
type instead of gaining undefined.

`replace` and `replaceAll` with an untyped search value and a callback
call the regex callback replacement for a RegExp and the string-search
helper otherwise. That helper now accepts untyped callback parameters,
passes undefined to parameters after the subject, and converts number,
boolean and untyped results with ToString.

In JavaScript, `||` and `&&` over primitives of different types whose
result tsc types `any` evaluate in the checked domain.
2026-10-23 05:55:39 -05:00
Malte UblandClaude Opus 5.5 a1b6fda11c Run npm packages in Node islands: object handles and compiler routing (NI-4)
Node islands stack, part 2 of 11 (scriptc-perf/results/islands-prs/PLAN.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:49:52 -07:00
Malte Ubl 56abb3d77a Merge ni-2-node-island-report (#1691) into the Node islands base 2026-10-22 14:45:33 -07:00
Malte Ubl 680af076cf Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/index.ts
2026-10-21 20:07:52 -07:00
Malte UblandClaude Opus 5.5 e61fbf8c0b Merge origin/main into ni-2-node-island-report
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:07:41 -07:00
Chris Tate fa7bc7c7a0 Warn about likely-hot functions that use untyped values
A build now names the functions of the program's own files that compile
to the dynamic tier and likely run often, so a slow program with a little
untyped code says where the time goes:

  warning SC6101: 3 functions that likely run often use untyped values, which runs slower than typed code:
    advance (nbody.js:22): untyped JavaScript file, in a loop
    ...

- The functions come from the tier assignment lowering already makes
  (Lowerer.dynamicFunctions). Without a profile, a function counts as
  likely hot when a dynamic operation runs inside one of its loops, when
  a loop calls it, or when it calls itself (lowering/dynamic-tier.ts).
- A function is named only when its source says why its values are
  untyped: an `any` (for a for...of variable, the value it iterates), an
  untyped JSON.parse result, or an untyped JavaScript file. Generated
  %dyn.* helpers, module top levels and functions that are dynamic for an
  operation without a static lowering are left out.
- At most five functions are named, most likely hot first, with the
  total; dynamic-tier code in npm packages is counted per package on one
  line. All of the warning's text is in diagnostics/dynamic-tier.ts.
- SC6101 opens the SC61xx band, performance notes, with a new
  "performance" category in the diagnostics envelope; SC60xx stay the
  divergences.
- scriptc build prints it after the divergence warnings, and the
  executable cache stores it with the entry, so a cached build (including
  the CLI's routed fast path) repeats it. scriptc coverage reports it, and
  a comment-only edit no longer replays a verdict that carries it, since
  its text holds line numbers.
- --no-dynamic-tier-warning leaves it out of build and coverage;
  coverage --fail-on=dynamic fails on it.
- Docs: coverage.mdx (Dynamic tier, Gating builds and CI, the envelope
  categories) and cli.mdx.
2026-10-21 18:53:43 -05:00
Malte UblandClaude Opus 5.5 091b8f1934 Report which packages and modules would run in a Node island
Adds the report-only Node island policy, configuration and detector of
the scope-B plan (milestone NI-2). Coverage now lists the npm packages a
build would run in a Node island and why, gap islands first, without
changing code generation.

- project-config.ts reads the `scriptc` key of the entry's package.json
  with a schema; an unknown value in `scriptc.packages` or
  `scriptc.nodeIslands` is SC8004 instead of being ignored. Package shims
  and the WebAssembly scanner read through it.
- node-islands/: origin (user, npm, shim, builtin; package instances),
  rules R1.1-R2.5 and the four classes, a syntactic pre-scan for R1/R2
  markers, classification of refusals and runtime fences (reason ids at
  the refusal sites that need them), the policy (auto, explicit, off;
  forced node and native packages; marked user modules) and the report
  with island closure.
- coverage: a "Node islands" section, `--explain-island=<pkg>`, a
  `nodeIslands` field in the diagnostics envelope, and `--fail-on`
  `node-islands`, `eager-node-islands` and `island-gaps`.
- CLI: `--node-islands`, `--node-island`, `--native`, `--node` (validated,
  report-only) and `--strict-user-js` (SC8101-SC8105 for R2 constructs in
  user JavaScript).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:22:35 -07:00