When the key is a parameter of a function that never escapes, every call
passes a string literal key of the table, and the body neither assigns the
parameter nor uses arguments, the dispatch needs no arm for other keys. In
nsbench the fallback boxed the Vec probe into a dynamic call, so every Vec
carried the property bag and lived on the heap; without it the workload
inlines into one loop. SCRIPTC_SHAPE_EXHAUSTIVE_TABLES=0 opts out.
nsbench (n=20000, overlay on): 36 -> 8.2 ms (Node about 21 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- static-objects: a primitive member stored once by an unconditional
top-level statement of the object's own file (app.math.EPSILON = 1e-9)
reads as its literal where the read provably runs after the store: inside
functions when the statements before the store run no code, and in later
top-level statements of that file. SCRIPTC_SHAPE_FOLD_STORES=0 opts out.
- JavaScript functions read the ambient receiver at entry only when this
occurs in their parameters or body outside nested non-arrow functions.
- npm case static-stores pins both, with the overlay on and off.
nsbench (n=20000, overlay on): 64.4 -> 36 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- analyze.ts keeps argsFrom in a Map the analyzer owns (compiler sources keep
caches on their owners, not in weak collections).
- constant-graphs.test.ts: lib.math is stored after lib was created, so the
report shows it as undefined|{...} with a graph of its own.
- maybe-promise-fences.ts: `record instanceof Promise` lowers to false since
the base's neverPromise lowering; the snapshot no longer lists the refusal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
static-objects, static-tables, concat-dynamic-args, evolving-let-index and
the three shape-* cases landed in this stack without preflight/order
baselines, so the TypeScript 7 parity job failed on every PR above them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The self-hosted compiler refused the truthiness test on an
IrType | null | "dyn" value (SC2003, no layout for the union result).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- memo.ts carries a build's overlay verdicts (refused annotations and files,
overlay off, the default build's deferred refusals) to the next build of the
same JavaScript, in the build cache.
- Builds whose only refusals are deferred lower the program without the
overlay first, and drop only what the added refusals blame; when the overlay
goes off, that lowering is the build's.
- Method families join only methods a call reaches together (receiver unions,
extends chains): ZodString's _addCheck returns ZodString, so zod 3 needs no
reload.
- Refusals in a class's fields or at new C blame annotations that name C.
- Rewrites that only fold constant reads compose with the typed overlay
planned on the same analysis (one rebuild and analysis instead of two).
- Shape analysis prefetches symbols only; input tracking notes a file's text
once per tracker.
- A declaration a project type patch edited gets no overlay annotation
(setShapeOverlayProtectedSpans, for the type-patches layer).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The overlay validation suites (npm, npm-static, three, Effect 3 and 4) now
pass with the overlay on. Each cause had an npm repro or suite case that
failed with the overlay and passed without it:
Analysis facts:
- Symbol members: Symbol.for/keyFor are functions, well-known symbols are
symbols, other members unknown.
- Reads that may answer undefined: index reads past the end (unless the
enclosing for loop proves them in bounds of an array that never shrinks or
has holes), computed reads of objects, keys an object literal lacks at
creation, optional-chain short circuits, `let x;` before assignment.
- Object.create(proto) inherits proto members; arguments past the declared
parameters are unknown; `new` of a primitive answers nothing.
- Arithmetic on bigints answers bigints (number|bigint is never printed).
Overlay:
- Package JSDoc is a declaration: it stays as written and the overlay adds
only types the JSDoc does not declare.
- Objects held by module-level bindings (and what they hold) are never typed
as records: a typed return would copy them.
- Only checks of overlay facts carry the inferred-fact message.
- static-objects ignores nullish receivers outside optional chains; folded
literal keys are unparenthesized.
Fallback (the overlay never adds a failure the default build lacks):
- the overlay state is kept per load (a later load reset it, so the
fallback never ran);
- deferred refusals in unreached code count;
- a preflight, deferred refusals or IR the program without the overlay does
not have make the build compile without the overlay.
CI: the shape_overlay job builds its own native packages; a manual run with
scope shape_overlay runs it alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three JavaScript lowering bugs threw a checked conversion where Node answers:
- concat with a checked-dynamic argument lowered it as an element, so an
array argument threw. The argument now spreads when it is an array at run
time and is appended otherwise.
- `let c; while ((c = str[i++]) !== "]" && c)`: the checker types c from the
index read, without the undefined a read past the end answers. Evolving-let
inference ignores index-read writes, and reads of a JS `let x;` statement
binding keep the actual value.
- Strict equality on a flow-narrowed dynamic JS name compares unchecked.
npm cases concat-dynamic-args and evolving-let-index match Node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
renderShapeOverlay rebuilt the whole source text once per JSDoc range, so
a file with many annotations cost time quadratic in its size. The ranges are
leading trivia of declarations, equal or disjoint, so the text is now built
once from the ranges in order.
Compile cost (Hono fixture: 53 s -> 99 s with the overlay before):
- The overlay is planned once per program text: a reload that only drops
refused annotations renders the cached plan (overlay.ts plan/render
split) and reuses the cached structural rewrites, instead of analyzing,
reporting and planning again.
- The overlay's analyses run without per-call-site clones, which only the
constant-graph report uses so far (SCRIPTC_SHAPE_SITE_CLONES opts in).
- A failing round also drops what the deferred refusals blame, saving a
reload; deferred refusals no longer blame every annotation of their
enclosing function (most of them exist without the overlay).
- A parameter whose `typeof` tests the proven values never pass stays
untyped: the typed parameter would narrow to never in that branch,
which the lowering refuses (zod 4's Doc.write).
- Deferred refusals in methods blame the arguments of the method's call
sites (implicit instantiations), like those in functions.
Package JSDoc: the overlay neutralized the package's JSDoc in every
overlaid file, so declarations it does not type lost their documented
types and the lowering instantiated them per call site instead. In
commander, this changed which refusal `--help` reaches: the run passed
the refusal the default build stops at and failed later, in
Command.helpInformation, at a checked conversion of the package's own
documented contract (a help width that is undefined off a TTY). Nothing
the analysis proved was contradicted (SCRIPTC_SHAPE_CHECK_LOG logs no
inferred-fact failure). Only the leading JSDoc of the declarations the
overlay annotates is neutralized now; commander's overlay build fails
exactly where its default build does, and the OVERLAY_KNOWN entries go.
The overlay npm suite (134 cases), the shape tests, lint and format pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Static-object specialization's stage 3 prerequisite
(DESIGN-constant-objects.md): a function called from module
initialization (top-level code outside loops, or another such clone) is
analyzed once per call site, up to 64 sites per function and 4 nested
clones deep. The objects and arrays it creates are one allocation per
call site, and each clone runs at most once, so they are singletons: the
schemas zod's and valibot's builders create at init become constant
objects instead of "init-factory" near misses.
Stage 1 report (SCRIPTC_DEBUG_SHAPES=const):
- zod 3.25 (z.object bench): constant objects 7 -> 48 (fields 57 -> 155),
init-factory near misses 12 -> 1, foldable reads 22 -> 72.
- valibot 1.1: constant objects 2 -> 5, init-factory 6 -> 0.
The overlay's annotations and the zod and namespace benchmarks are
unchanged. The overlay counts union arms rather than atoms, so the
per-site allocations of one record shape stay one typedef.
SCRIPTC_SHAPE_SITE_CLONES sets the per-function limit (0: off).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The analysis exceeded its 20M-step budget on the Hono fixture (32.5 s)
and on moment 2.30 (60 s, two files), so it reported no facts for them.
- Widening: a value node holding more than 32 atoms (merged code that
every value flows through, such as moment's utilities) becomes ⊤. Its
values escape, so the facts stay sound, and later atoms escape instead
of multiplying constraints. SCRIPTC_SHAPE_WIDEN sets the limit.
- A member a string, number or boolean prototype lacks reads undefined
instead of creating a builtin atom per name (Hono's request and route
objects meet strings in shared helpers).
- Members of builtin instances (`RegExp.instance`) no longer chain into
static builtin paths.
- Symbol resolution is memoized per identifier.
Hono: 1.16M steps in 2.4 s with complete facts (55 constant objects).
moment: 115k steps in 0.4 s. zod 3 and the namespace benchmark are
unchanged (same steps and overlay annotations).
SCRIPTC_DEBUG_SHAPES_PROFILE prints where the solver's steps go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Static-object specialization for namespace-style JavaScript, with the
shape overlay on (shape-inference/static-objects.ts):
- Functions stored in object literals that a top-level statement creates
(`ns.registry = { run: function () {} }`) are hoisted to module
functions, like those stored by assignment.
- `table[key](...)` on exactly one constant object whose members are all
hoisted functions, with a string key, compares the key against each
member and calls its function directly; any other key reads the table
as written. The members do not use `this`, the table has no accessors,
and comparing a string runs no code.
- A read of a primitive member written in the object's own literal
(`Kind = { CIRCLE: 1 }`) is replaced by the literal. Members stored
after the object was created stay reads (they could be read first).
- A read in another JavaScript file of a hoisted namespace member refers
to it through the defining module's exports: the member joins the
module's `module.exports = { ... }` literal, and the reading file binds
the module right after the top-level statement that requires it. This
applies when the exports object is constant and nothing enumerates,
reflects on, or lets it escape, and the module does not require the
reading file.
The overlay's cross-file class types and member function types are now
on by default (SCRIPTC_SHAPE_CROSS_FILE=0 and
SCRIPTC_SHAPE_MEMBER_FUNCTIONS=0 turn them off).
The analysis records computed callee reads as computed reads.
New npm case static-tables (with a JavaScript consumer) and
tests/harness/static-tables.test.ts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the overlay on, a lowering refusal no longer fails the build. The
annotations the refusal depends on are dropped and the program is loaded
and lowered again, so those declarations compile untyped, as they do
without the overlay:
- Each overlay annotation carries an id (`@scriptcShape #<pos>`).
- shape-inference/fallback.ts blames the annotations of the declarations
enclosing a refusal and of the declarations its expression names,
following unannotated functions' returns.
- Rounds that stop converging drop whole files' overlays, then the
overlay for the build. SC6103 reports what was dropped.
Also:
- The tsc gate's project world is built over the files as written; it
used to see the overlay's files, which failed the typecheck of three
npm cases.
- Arrays a builtin creates (match results, `new Array(n)`) are no longer
typed by the overlay: they may have holes, and the library types them.
- compile() accepts `shapeOverlay: true`.
tests/harness/npm-shape-overlay.test.ts runs the whole npm suite with the
overlay on (npm.test.ts's body moves to npm-suite.ts): every case builds
and runs as in the default build, and a refused build that compiles with
the overlay matches Node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the shape overlay on, a first round of structural rewrites runs before
the typed overlay (shape-inference/static-objects.ts, stage 2b of
DESIGN-constant-objects.md), proven by the constant object graphs:
- An ES5 constructor stored by a top-level statement whose prototype is a
constant object of plain methods becomes a class declaration with those
methods, when the program cannot tell: the function is only ever
constructed and never escapes, its prototype is never replaced, nothing
enumerates instances with for-in or reflects on the prototype or the
function, nothing reads their names or sources, the bodies are
strict-mode safe, and only definitions run between the constructor and
its last method.
- A constant namespace member holding a `this`-free function expression
becomes a function declaration, and reads of the member in the same file
whose receiver is exactly that constant object refer to it directly.
The rounds rerun the analysis on the rebuilt program, the overlay now also
covers the program's own JavaScript files, and the build report counts the
rewrites. A JavaScript switch whose case tests lower to another type than
the discriminant compares with === instead of being refused.
Namespace benchmark (n=20000): 1.55 s -> 0.42-0.47 s, output identical to
Node (typed TypeScript twin 85 ms, Node 21 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two refusals kept pre-module JavaScript packages from compiling:
- `var ns = ns || {}` (and `?? {}`): the read in the variable's own
initializer made the checker type the namespace `any`, and requirers
then failed to reach the binding (SC1030). In a module, the read sees
the hoisted `undefined` whenever nothing else writes the variable, so a
new package-source normalization respells the declaration as its
default (`var ns = ({})`) when no other declaration, assignment, update,
function, class, parameter or local of that name exists in the file and
no earlier top-level statement mentions it.
- `module.exports = { ns }` with a `var` binding was refused as exporting
a mutable binding by reference. A `var` (or `let`) whose initialized
declaration precedes the export statement at top level and that nothing
reassigns exports exactly the value Node copies, so it now aliases like
a const.
The namespace benchmark (results/design-shape-inference/nsbench) now
compiles and matches Node; the new npm case legacy-namespaces pins a
namespace library with guarded declarations, constructor functions with
prototype methods, enums, a revealing module and a handler registry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shape overlay (DESIGN.md, approved route) rewrites the JavaScript of
opted-in npm packages with JSDoc types the shape analysis proves: record
typedefs for non-escaping literal sites (optional fields for spread
sources), class field, parameter, return and local types, receiver casts,
and branches the analysis proved unreachable replaced by checks. The
lowering trusts only annotations carrying the overlay's marker; package
JSDoc stays a hint (its type tags are neutralized in overlaid files).
- Inferred-fact checks: every checked conversion lowered in an overlaid
file records its site, and a failure throws a distinct internal error
("scriptc internal error: an inferred type was contradicted at <file:line>
… This is a bug in scriptc's shape analysis, not in the program's data")
naming the opt-outs; SCRIPTC_SHAPE_CHECK_LOG makes checks log instead
(validation mode).
- Opt-outs: `--no-shape-overlay`, and `"scriptc": {"packages": {"<name>":
{"shapes": false}}}` per package.
- Build report: SC6103 counts the declarations typed, the checks inserted
and the branches replaced.
- `x instanceof Promise` on plain data folds to false.
The overlay runs only with SCRIPTC_SHAPE_OVERLAY=1 while it is new.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The constant-graph pass treats a function whose call edges all run once
at initialization as init-only. Some calls recorded no edge or a
once-only one:
- `new F()` of an ES5 constructor and `f.apply(thisArg, args)` bound the
arguments without a call edge, so a function also called directly once
was init-only even when a loop constructed or applied it.
- Callbacks of Array.from, toSorted, replace and replaceAll run once per
element, comparison or match, but those methods were missing from the
repeating callback set.
- JSON.parse revivers, JSON.stringify replacers and toJSON methods run
once per value; their edges at the JSON call site now count as
repeated, including toJSON calls resolved after the call's handling.
Objects these functions write or allocate were reported as constant, and
their reads as foldable. constant-graphs.test.ts covers each case.
SCRIPTC_DEBUG_SHAPES=const reports the constant object graphs shape
inference finds (DESIGN-constant-objects.md, stage 1): objects created by
code that runs at most once (module top level, IIFEs, functions reached
only through one call site from such code) whose fields are written only
there, that never meet a reflective or computed write, and that never
escape. Namespaces (`var ns = ns || {}; ns.util = {}`), revealing
modules, `module.exports` tables, and ES5 constructors' prototype objects
qualify. Each near miss names its rule: allocated repeatedly, created by an
init factory (one object per call site), escapes, reflective or computed
write, literal form, written after init, or lazy cache. The report counts
the member reads and calls a later stage can fold.
The analysis now models ES5 constructors (a function's prototype object,
instances reading through it) and links CommonJS require() and export
bindings to the analyzed module's exports, which it previously read as
nothing. Code generation is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
planNpmStaticReexports visited a file again each time another of its
export names was demanded. Every visit resolved the file's import and
export declarations again and walked its whole syntax tree for import()
and require() calls. The purity check kept only pure trees and the failing
root, so once a walk reached an impure file, the next namespace re-export
walked the same files again.
Each file's edges are now collected once per plan; the calls come from
moduleSourceCandidates, which fork and worker discovery already fill. A
repeated visit re-checks only the file's namespace re-exports, the only
edges whose pruning depends on the demand. Purity is decided per strongly
connected component of the module graph and kept for every file, and a
walk that reaches an impure file marks every file on its stack impure.
The pruned re-exports and the module order are unchanged.
The copy of a dense-array, byte-window, counted, callback or typed-array
versioned loop that runs when its guard fails no longer gives its loops
speculative integer or int32 kind copies, which emitted each loop again for
a path that rarely runs. Proven integer loops, which emit one copy, are kept
in it.
Linux executables linked through the clang driver with its default linker,
GNU ld. lld links the same objects in about half the time.
- Native and Node-hosted Linux exe links pass -fuse-ld=lld when ld.lld is
beside the driver's executable (where clang looks first) or on PATH, and
the default driver is in use (no SCRIPTC_LINKER, no sanitizer link).
- A link with lld that fails runs again with the driver's default linker,
whose error is the one reported when that fails too.
- Named loads also read what was written under that name or an unknown key where an own field
shadows a builtin: an instance's constructor, a function's call, apply, bind, prototype, name
and length, a class's prototype and name, an array's methods. Named loads of arrays read what
was written under unknown keys, which computed stores on arrays now record.
- Computed stores into builtins, promises and collections escape, as named stores there do.
- JSON.stringify calls toJSON on arrays, functions and classes, and on values written under
unknown keys. A JSON.parse reviver's results are part of the result and of later calls' values.
- Reflect.construct(Proxy, ...) and globalThis.Proxy record a proxy operation, as new Proxy does.
A child process has these members only when it was created with an IPC
channel (fork, or an "ipc" stdio entry), and keeps them after the channel
closes and after the child exits. The 'in' test on native handles answered
true for every declared member, so a child spawned without IPC reported
them and entered IPC-only branches. The three names now answer from the
runtime: scr_child_ipc_present reports whether the child was created with
a channel.
Corpus 8290 tests a forked and a spawned child before and after
disconnect and exit.
SCRIPTC_DEBUG_SHAPES=1 runs a whole-program allocation-site points-to
analysis over opted-in npm packages' JavaScript and the user files that
call into them, and prints which object and array literals a native
record would represent exactly, the inferred class field types, the
census of writes and reflection on untracked receivers, and the hot
functions' parameter and return types. Report only: nothing reaches the
lowering, and the analysis runs only when the variable is set.
Design: results/design-shape-inference/DESIGN.md (scriptc-perf).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
m.set(k, f(m.get(k))) and a get followed by a set of the same key, with a
number or boolean value, now reuse the entry the read resolved, as
string-keyed maps already do: the write overwrites the value in place
when the map is the same and the key has the read's bits, and takes the
ordinary set otherwise. Before, the write probed the table again: the two
lookups were not merged, because each sits behind its own direct-index
test.
A warm native build validates its frontend cache entry by reading and
SHA-256 hashing every source and declaration file the frontend read.
A tracked read now also records the file's identity (device, inode,
size, modification and change time) when the identity is the same before
and after the read and the file had been unchanged for two seconds. A
later check accepts a file whose identity is unchanged without reading
it, and hashes it otherwise. A write always gives a file a newer change
time, which tools cannot set back, so an unchanged identity means the
content hashed with it. The equivalent-edit check uses the same identity
for the unchanged sources.
A mapped type with no members was refused (SC2001) to keep `Partial<T>`
inside a generic body, where keyof T is still unknown, from interning a
wrong empty record. That also refused mapped types the checker resolved
from concrete inputs whose key filter keeps no key: `Record<never, T>`,
`Partial<{}>`, and a catalog type such as
`{ [K in KeyOf<Data, "image"> as Id<K>]: Model }` for data without image
models. When the mapped type comes through an alias whose arguments mention
no type parameter (or a module-level alias without arguments), it is the
empty object type and lowers like `{}`.
pi-ai declares its per-provider image and classifier model catalogs this
way; most providers have none, which refused 105 declaration groups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`value instanceof Boolean` (or Number, String, BigInt, Symbol) died at run
time in JavaScript with Node's TypeError for a function without a prototype
object, because those constructors are compiled as plain conversion
closures; in TypeScript the right-hand side was refused (SC1090). A
compiled program never creates a boxed primitive (`new Boolean(...)` has
no lowering) and a primitive is never an instance, so the test is false
for every value. It now folds to false after evaluating the operand.
typebox's value guards (IsBoolean, IsNumber, IsString) test this way, so
pi's tool-schema construction and validation died at its first guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A class field typed AbortController or AbortSignal (plain, optional,
nullable, or in an array) maps to checked-dynamic storage, and the
class-field rule refused it (SC1090) although the same handles already
live in locals, Map values, and closures. They now take the checked-dynamic
slot, like the fetch Headers, Request, and Response handles.
controller?.abort() on a nullable controller was refused as a method call
(also for locals): the static abort lowering ran only without an optional
chain. It now runs again once the chain has narrowed the receiver.
Found by pi's TUI Editor (autocompleteAbort: AbortController | undefined).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A native build reuses its stale frontend cache entry when every changed
source is token-equivalent to the text the entry was built from, but dev
builds never did: their debug information records source lines and
columns, which a token-equivalent edit can move.
Builds with debug information (dev builds without --strip) now reuse the
entry when every changed source is token-equivalent and every token keeps
its line and column (tokenPositionEquivalentSource): edits to a comment's
text within its line, comments and whitespace after the last token, and
trailing whitespace. Edits that move a token still rebuild.
The token scanner also reads a `/` after a non-null assertion as division
(`data[i]! / 7`). It used to read it as the start of a regular expression
and give up on the file, so comment edits in files with that pattern
always rebuilt. A `!` after a line break, or after an operator or a
keyword such as `return`, still starts a negated regular expression, and
a `!` after `)`, `}`, `++` or `--` gives up.
