Commit Graph
5474 Commits
Author SHA1 Message Date
Malte Ubl 5817be4d0f Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/ir/function-references.test.ts
2026-10-26 11:08:44 -07:00
Chris Tate 58197ed89d Close the largest canonical-mapping gaps the shadow census found (#2340) 2026-10-26 12:42:03 -05:00
Chris Tate 9b4a6ffdf1 Regenerate the IR field readers for dynCheck's inferred site (#2347) 2026-10-26 12:08:27 -05:00
Chris Tate 5b29eb1bae Merge main 2026-10-26 12:07:21 -05:00
Chris Tate 91014dfb66 Read TypeScript 7.1's binary AST format beside 7.0's (#2346) 2026-10-26 11:59:28 -05:00
Chris Tate 34c2f9851d Have the native CLI packages depend on their LLVM helper and runtime pack (#2333) 2026-10-26 11:47:34 -05:00
Chris Tate 6dbdf39120 Track a stream's 'data' listeners instead of looking them up per chunk (#2344) 2026-10-26 11:38:04 -05:00
Chris Tate 341ddd41ae Regenerate the IR field readers for dynCheck's inferred site 2026-10-26 11:34:57 -05:00
Chris Tate de6cdf1156 Add the shape overlay: typed JavaScript from analysis-proven facts (opt-in) (#2176) 2026-10-26 11:33:47 -05:00
Chris Tate ce1f3c4d2b Read TypeScript 7.1's binary AST format beside 7.0's
The AST reader accepts protocol 9, TypeScript 7.1's encoding, as well as
protocol 5 (7.0's), so scriptc can read either checker's responses:

- the 64-byte header of protocol 9;
- its node kinds, which count 7.1's SourceKeyword: each kind above it
  reads as 7.0's, and node handles name the response's own kinds;
- a module declaration's child mask, whose attributes slot moves the
  body to slot 3.

A protocol 9 response with a node 7.0 has no form for (a SourceKeyword
node, import attributes on a module declaration, or an `import source`
clause) throws an AstDecodeError that names it.
2026-10-26 11:27:36 -05:00
Chris Tate a5971450ec Tear down self-recursive records and classes in one loop (#2343) 2026-10-26 11:14:35 -05:00
Chris Tate b1b6b98c0d Compare canonical mapping with mapType in an opt-in shadow mode (#2334) 2026-10-26 11:03:38 -05:00
Chris Tate cc5c81fb4d Keep frontend snapshots stable when a file is read before and after it settles (#2342) 2026-10-26 11:02:20 -05:00
Chris Tate e7f3ab29c2 Track a stream's 'data' listeners instead of looking them up per chunk
Each chunk a readable side delivered looked up its 'data' listeners by
name twice: once to decide whether a flowing stream emits directly, and
again in emit('data'), which scans the stream's event names even when a
piped stream has no 'data' listener at all. The stream now keeps a
data_listeners flag, set from the emitter registry's empty/nonempty
transitions through scr_emitter_observe, and reads it on those paths;
emit('data') is skipped when the stream has no 'data' listener.
2026-10-26 10:59:32 -05:00
Chris Tate 09a73ab6ed Fix Array.prototype calls on native arrays, typeof on ambient names, and net connections to Unix sockets (#2326) 2026-10-26 10:56:58 -05:00
Chris Tate 6e4fdf95d0 Link the exported bootstrap package's helper and runtime pack dependencies
The native bootstrap's package checks copy the seed's distribution into a
package directory outside the workspace and, since the CLI packages depend
on their LLVM helper and runtime pack, drop the bundled copies. Nothing in
that directory resolved @scriptc/llvm-linux-x64-gnu, so
verify-native-cli.mjs failed with 'Cannot find module'. The test now links
both packages into the directory's node_modules, as pnpm links a workspace
package's dependencies.
2026-10-26 10:56:27 -05:00
Chris Tate 36273adc8f Unroll counted loops by the period of a guard on the counter's low bits (#2341) 2026-10-26 10:56:09 -05:00
Chris Tate d5dc896576 Merge main 2026-10-26 10:53:46 -05:00
Chris Tate 48c521dc27 Address review feedback 2026-10-26 10:43:59 -05:00
Chris Tate 9b96314c20 Tear down self-recursive records and classes in one loop
The teardown of a type with children of its own type called itself for
each child: a call, its register saves and the segment bookkeeping per
node, 45 instructions per node in a binary tree.

A record or standalone class whose every reference field holds its own
type and that has no collector header now tears down in one loop. A child
whose last reference the dead node held goes onto a pending list linked
through its own count word, which it no longer needs, and the loop takes
the list's first node next, so nodes are visited parent first. Each node
frees its block as before (dispose observers, the RC audit, blocks of
other arenas), the calling thread's blocks go into the teardown segment
in visit order, and the segment goes onto the free list in one step, so a
structure built parent first takes the blocks in its allocation order.
Shared and immortal children keep their release. The loop needs no depth
budget. Other teardowns keep the recursive form.
2026-10-26 10:42:41 -05:00
Chris Tate dec2b8a454 Record one probe for two reads of a file that saw the same content
A read within IDENTITY_SETTLE_MS of a file's last write records no
identity, and a later read of the same bytes in the same frontend does.
FrontendInputTracker.record compared the two probes as JSON and marked
the snapshot unstable. A build or coverage run that read its entry
before and after that point then published no cache entry: the exact
repeat of a build lowered again, and a coverage verdict was not replayed.
Two file probes with the same digest now merge into one that keeps the
settled identity; any other difference still marks the snapshot
unstable.
2026-10-26 10:33:02 -05:00
Chris Tate 6fa88c7243 Observe every top-level mapping in the canonical shadow 2026-10-26 10:32:41 -05:00
Chris Tate 928c30e908 Bind thisArg in the array callback methods of dynamic arrays
forEach, map, filter, some, every, find, findIndex, findLast,
findLastIndex and flatMap on a dynamic array, and through
Array.prototype.<method>.call on a native array, called the callback
without its thisArg, so `this` in the callback was not the second
argument. These methods now bind `this` to the thisArg, or undefined
when there is none, once per call, around all of the method's callbacks.
2026-10-26 10:19:57 -05:00
Chris Tate be293e3dfb Unroll counted loops by the period of a guard on the counter's low bits
A counted loop whose body branches on `(i & M) === C`, with M one of 1, 3
and 7 (a check every second, fourth or eighth iteration), now asks LLVM to
unroll it by M + 1 (llvm.loop.unroll.count on its back edge). In each
unrolled copy the counter's low bits are known, so the guard folds to a
constant and only one copy per period keeps the check. Before, LLVM tested
the guard on every iteration and, when the guarded condition was cheap,
evaluated that condition on every iteration too.

The loop must step by 1 from a multiple of the period, have no nested
loop, and stay under 4096 IR nodes once unrolled. Loops with the same
period share one loop ID node, which the module emits only when a back
edge refers to it.
2026-10-26 10:19:25 -05:00
Chris Tate 26b2fdcc67 Read IR nodes' fields by name in three llvm-emit passes (#2336) 2026-10-26 10:17:48 -05:00
Chris Tate 1947bf66c7 Release a substitution's operands when it returns (#2337) 2026-10-26 10:17:28 -05:00
Chris Tate c1ad72e1a4 Build the candidate's own compiler source in the zero-cost gate (#2339) 2026-10-26 10:15:57 -05:00
Malte UblandClaude Opus 5.5 812aa3a7bb Close the largest canonical-mapping gaps the shadow census found
- 'this' in a class body maps through the nominal table (it is the class).
- An unbound type parameter maps through the lowerer's IR-level binding,
  which is how generic class instances bind their parameters.
- An intersection that refines a class (a mixin instance, 'T & Class') is
  that class, not a merged record.
- An object whose members do not map structurally (an interface over Map
  or Set, method-only interfaces) keeps its per-type recognition.

On a 160-program sample of the corpus (generic classes, interfaces, Maps),
the shadow census goes from 78.7% equal (2,936 canonical failures, 943
divergences) to 98.0% equal (25 failures, 343 divergences).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:21:05 -07:00
Malte UblandClaude Opus 5.5 5aa6699b3d Build the candidate's own compiler source in the zero-cost gate
The self program compiles the compiler source of the checkout the programs
come from (the baseline by default), so a change adding compiler code the
self-hosted frontend cannot lower passed the gate and failed CI's
self-hosting tests. With the self set, the gate now also builds the
candidate's packages/compiler/src/native/cli.ts with the candidate (LLVM
only, not compared) and fails when that build fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:09:02 -07:00
Chris Tate 823ebd50dc Resolve each shipped declaration path once (#2335) 2026-10-26 08:42:47 -05:00
Chris Tate 5a14c0a3dc Merge main 2026-10-26 08:37:13 -05:00
Chris Tate b1614d6e75 Release a substitution's operands when it returns
The temps a regex replace or replaceAll consumed (the subject that a
previous substitution in a chain produced, a computed replacement) were
released at the end of the statement. A chain of substitutions over a
large string therefore held every intermediate result until the last one
returned. The operands now live in a frame of their own that is released
right after the call, so a chain holds at most its current subject and
result.
2026-10-26 08:30:42 -05:00
Chris Tate f9b5533ee6 Read IR nodes' fields by name in three llvm-emit passes 2026-10-26 08:13:15 -05:00
Chris Tate f1298287d6 Resolve each shipped declaration path once
isFallbackDtsPath runs for every type lowering maps (isStdlibFile), and
since the fallback declarations moved into per-module files (#2256) it
asked declarationPath for the fallback module directory each time. Without
an installed declaration root, each answer resolved
@scriptc/compiler/scriptc-node-fallback.d.ts through Node's module
resolver. Lowering three.js's buffers-animation fixture went from 6.3 s to
43 s. declarationPath now keeps its answers by name until the declaration
root changes.
2026-10-26 07:58:58 -05:00
Chris Tate 52a228d0f5 Keep declarations a removed branch shares with another 2026-10-26 07:50:59 -05:00
Malte UblandClaude Opus 5.5 f03741b03e Keep the branded-primitive rule compilable by the self-hosted frontend
Once shadow mode imports the normalizer, the self-hosted frontend lowers
it, and it cannot compare union-typed canonical nodes by identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 05:49:39 -07:00
Chris Tate d1a0bea84e Have the native CLI packages depend on their LLVM helper and runtime pack
Each @scriptc/cli-<platform> package bundled its platform's LLVM helper
and runtime pack, which are also published as @scriptc/llvm-<platform> and
@scriptc/runtime-<platform>. The CLI packages now depend on those packages
at the same version (workspace:*, which pnpm pack writes as the exact
version) and leave them out: cli-linux-x64-gnu goes from 115 MB to 60 MB
packed.

- build-native-cli.mts --platform-packages, which each CLI package's
  build:native passes, writes a manifest without llvm_package, runtime_pack
  and runtime_packs.
- loadNativeToolchain resolves a missing helper or host runtime pack as
  Node resolves the package from the compiler's directory: nested in the
  platform package, beside it (hoisted, or pnpm's virtual store) or higher
  up. It requires the compiler's version, and an absent or mismatched
  package fails with an error that names it.
- installNativeCli records where the package manager put both packages in
  the installed manifest, with the same checks.
- verify-native-cli.mjs requires the exact dependencies of a platform
  package and checks the resolved packages' versions.
- package-native-cli.mjs bundles the helper and runtime pack into the
  standalone archives, which stay self-contained.
- smoke-native-install.mjs installs the helper and runtime pack tarballs
  with the platform package; the bootstrap's exported package drops its
  bundled copies first.
- release.yml publishes the CLI packages only once every helper and
  runtime pack package resolves on npm, and scriptc only once the CLI
  packages resolve, each wait at most 25 minutes. A rerun skips the
  uploads npm already accepted and waits again.
2026-10-26 07:48:11 -05:00
Malte UblandClaude Opus 5.5 5c14941617 Bound canonical keys and keep the shadow from breaking a compile
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 05:40:13 -07:00
Malte UblandClaude Opus 5.5 9820569fc0 Compare canonical mapping with mapType in an opt-in shadow mode
Stage N1 of the localized-failure design (canonical type normalization).

mapCanonical (frontend/canon-map.ts) maps a canonical node to IR from its
structural form: primitives, literals, units, the top types, arrays, flat
unions and plain records. Nominal objects, callables, index-signature
shapes, tuples, bigint and symbol keep their per-type recognitions through
mapType on the node's checker type.

Under SCRIPTC_CANON_SHADOW=1, every successful top-level mapType also runs
canon and mapCanonical in throwaway registries and appends a JSON line per
comparison (equal, diverge, canon-fail) to SCRIPTC_CANON_SHADOW_OUT. The
census decides which rule families can later move to the normalizer. The
hook is absent otherwise, so no output changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 05:40:12 -07:00
Chris Tate a59df23c71 Merge main 2026-10-26 07:21:12 -05:00
Chris Tate 641f779fa3 Store identical runtime artifacts once and strip the copied CLI executables (#2301) 2026-10-26 07:06:31 -05:00
Chris Tate b221a13369 Reuse the completion callback a stream passes to its option callbacks (#2323) 2026-10-26 07:00:49 -05:00
Chris Tate de17158254 Merge main 2026-10-26 06:51:26 -05:00
Chris Tate 636c7631b3 Keep program partitions local to the code an edit changes (#2328) 2026-10-26 06:35:05 -05:00
Chris Tate 067f146e87 Read builtin globals through the global object, and support Node's navigator (#2242) 2026-10-26 06:16:08 -05:00
Chris Tate a758ff1b4e Select sanitized runtime variants by unit, and skip stripping without llvm-strip
The native sanitizer build chose the variants to compile by artifact path.
Since the runtime pack stores identical objects once, two variants of a
unit can share a path, so it compiled such a unit for both variants and
linked it twice (duplicate symbols in the sanitized native bootstrap).
selectedRuntimeUnits now returns each selected unit with its variant;
selectRuntimePackArtifacts and the sanitizer both use it.

Staging the native CLI's assets ran llvm-strip unconditionally and failed
on hosts without it. Without llvm-strip on PATH, tsc-ts now keeps its
symbols and the build prints a warning; the release hosts install LLVM.
2026-10-26 06:02:01 -05:00
Chris Tate 7ae99f63bc Give stream option callbacks Node's callback identities
Node passes a stream's one onwrite function to every _write and _writev
call, and a new function to every _transform call. A write or writev
option callback now receives the same closure on every call
(scr_stream_write_closure), kept or not, so a Set, WeakSet or WeakMap
finds it again. A reused transform, flush, final, construct or destroy
callback is first forgotten by weak collections, as at its free, so a
WeakSet or WeakMap that recorded the previous call's callback does not
find it on the next call.
2026-10-26 05:52:57 -05:00
Chris Tate e559baacde Skip repeated probes in lowering (#2330) 2026-10-26 05:47:59 -05:00
Chris Tate 66b6ca53ce Load a Node module's fallback declarations only when the program names it (#2256) 2026-10-26 05:37:36 -05:00
Chris Tate b8aaa5985a Merge main 2026-10-26 05:23:00 -05:00