Commit Graph
71 Commits
Author SHA1 Message Date
Chris Tate 5765d89227 Check the frontend's types against tsc-ts/api
The frontend calls tokenStartOfNode, getTextOfNode, getSourceFileOfNode and
forEachChild instead of AstNode's position and traversal methods. Node lists
answer hasTrailingComma(), and a declaration's modifiers are a modifier list
with their flags.

Symbol, Type, Signature, IndexInfo, TypePredicate, Program, TypeChecker and
Diagnostic have tsc-ts/api's members and types. Compiler code reads symbol
names with symbolName(symbol).

tsc-ts goes to 0.0.5. api-conformance.ts asserts that AstNode, AstNodeList,
SourceFile and each node interface are assignable to tsc-ts/api's interface of
the same name, and names the members where the wire format differs.
2026-10-20 13:41:39 -05:00
Chris Tate 7d555c4cf3 Merge main 2026-10-20 09:04:02 -05:00
Chris Tate 8d4737dfdb Merge main 2026-10-19 20:27:03 -05:00
Chris Tate 37f54ff157 Merge main 2026-10-19 17:14:32 -05:00
Malte Ubl a10ce583d7 Merge remote-tracking branch 'origin/main' into opt/tier-s3-jsval 2026-10-19 05:43:49 -07:00
Chris Tate 9b79eb8d32 Read declarations from symbols and signatures
symbol.declarations, symbol.valueDeclaration and signature.declaration
return nodes, as tsc-ts's Symbol and Signature do, instead of the
facade's declarationsOf, valueDeclarationOf and signatureDeclaration
resolving their handles. Each symbol and signature resolves its handles
in its own project on first read and keeps the nodes. The handles stay
available as declarationHandles, valueDeclarationHandle and
declarationHandle.

A program's checker and its symbols belong to the same project, so the
nodes are the ones the facade returned. The frontend's 432 calls of the
three facade methods become property reads, and the methods and their
caches go.
2026-10-19 07:41:28 -05:00
Malte UblandClaude Opus 5.5 05d200ae13 Assign tiers in the union-conversions fixture before serializing its IR
The fixture builds and serializes an IR module by hand. Since IR version
17 serializes function tiers instead of deriving them on load, the loaded
module's main, which performs dynamic operations, validated as a static
function. Assign tiers as lowering does before serializing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 05:33:41 -07:00
Chris Tate 3c749a2895 Read AST lists through their nodes
The frontend's node lists become node lists, { pos, end, nodes }, the
shape of tsc-ts's NodeList, so that the frontend can later read tsc-ts's
own nodes. AstNode's list getters return an AstNodeList, ts.NodeArray<T>
is that class, and the frontend reads a list's elements through nodes.
The lists already carried their positions on the wire.

The class is the type, not a structural interface, because native code
represents a class and a record differently.

A decoded file keeps each list's nodes array, as before, and each read
of a list field returns a new AstNodeList over it, so the file holds no
more objects than before. nodes keeps its identity; a list object is not
the same object on each read.

The change is mechanical: a type-aware rewrite of each list field read,
plus hand edits where lists are read through casts, generic accessors
or test mocks. The generated getters and types follow from
generate-ts7-ast-schema.mjs. AstFile and Ts7Session drop their unused
listMetadata hook.
2026-10-19 06:33:33 -05:00
Malte UblandClaude Opus 5.5 89bca68cbe Compute with NaN-boxed jsval values in the dynamic tier
Stage S3 of the two-tier design: dynamic-tier functions compute with
jsval, a new IR type holding a 64-bit NaN-boxed word whose numbers,
booleans, undefined and null are immediates, so a number between two
dynamic operators is no longer a heap node.

- IR: the jsval type and js.* builtins (operators and the boundary
  conversions fromDyn/toDyn/fromF64/fromBool/fromStr/fromBigInt/toF64).
  The jsval pass rewrites dynamic-tier functions after tier assignment:
  operator chains, comparisons, number checks and truthiness of operator
  results run on jsvals; dyn locals become jsval locals when that saves
  boxing; dynamic functions called only from dynamic functions return a
  jsval. IR version 17 serializes each function's tier.
- Runtime: scr_js.h and scr_js_value.c behind the dynTier feature
  (invariant I3); static runtime units never include scr_js.h.
- Backend: number cases inline (immediate doubles and NUM cells), always
  inlined jsval retain/release; 32-bit targets keep boxed cells.
- Interprocedural analyses see jsval conversions as the dyn forms they
  replace, so no fact about a static function changes.

Static functions are untouched: typed programs and the self-hosted
compiler's static functions compile to byte-identical LLVM.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 15:11:32 -07:00
Malte UblandClaude Opus 5.5 700f59c1b2 Serve Temporal's absence, v8, stream/web, dns/promises, EvalError and graceful-fs
T10 and T12 of the zero-cost compatibility plan, plus the maintainer
decisions:

- Programs that name Temporal type-check against TS 7's temporal lib
  (added only for them) and lower to Node 24's answers: typeof is
  "undefined", globalThis.Temporal is undefined, other uses throw
  ReferenceError at the root read.
- node:v8 is importable; getHeapStatistics answers allocator-derived
  figures in Node's shape; other members fence with the by-design reason.
  vm's refusal says why.
- node:stream/web exports the web stream globals themselves.
- node:dns/promises and dns.promises: lookup and lookupService as
  promises, in the link-gated scr_dns_promises.c unit.
- eval and the Function constructor throw Node's EvalError under
  --disallow-code-generation-from-strings; the corpus harness runs the
  Node oracle with that flag for `// @disallow-code-generation`.
- graceful-fs is served as node:fs.

Every new lowering replaces a refusal, and the new runtime code links only
when used, so other programs compile to identical bytes (zero-cost gate).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 12:11:05 -07:00
Chris Tate 92032a1b11 Shrink Map and Set headers and small entry arrays
A Map or Set header held six adapter pointers, the union key table and
size_t counts: 168 bytes, 176 in its size class. The adapters and union
key kinds now live in an immutable per-type ScrMapOps descriptor that the
compiler emits once per collection type, a traced byte replaces the two
trace-pointer tests, and the counts and capacities are 32-bit. The header
is 88 bytes.

The entries array starts with one slot and grows to four, then doubles,
and maps of up to 8 entries scan their cached hashes instead of keeping a
bucket table. Number-key maps still move to the direct index from four
entries.

One million maps held at once (RSS per map): empty 188 -> 100 bytes,
1 entry 286 -> 125, 4 entries 286 -> 198, 8 entries 450 -> 295.
2026-10-16 07:27:17 -05:00
Chris Tate 416311f2c5 Append to strings in place for fields, concat chains and character codes
The runtime appends to a string in place when the left operand is
uniquely owned, but three common forms never got there:

- `obj.field += s` on a plain class or record field kept a second
  reference to the old value, so every append copied the whole string.
  The compound assignment now reads a stored string field into a mutable
  local, stores "" into the field (no code runs in between: both operands
  are already strings), runs the self-append `old = old + rhs` and stores
  the result.
- `s = s + a + b` copied per append: the self-append recognized only
  `s = s + x`. A concat chain whose leftmost operand is the target now
  takes the remaining operands as the suffix, in the same order.
- `s += String.fromCharCode(c)` built a one-character string first.
  scr_str_concat_move_char_code appends the character, in place for ASCII.

Self-appends also borrow the suffix instead of retaining it, and a
discarded compound assignment no longer retains and releases its result.
2026-10-15 06:17:38 -05:00
Chris Tate f3ace69760 Store scalar-union fields inline in records and objects 2026-10-14 13:02:38 -05:00
Chris Tate e932714bfa Shrink the cycle-collector object header to 8 bytes 2026-10-14 04:53:07 -05:00
Chris Tate 7e6294ca0f Make scriptc compilation native-only 2026-10-12 12:25:30 -05:00
Malte Ubl f4eafed2b5 Number collector types in the self-hosting layout fixture
Emitted ScrCycType descriptors now carry collector statistics indices from ShapeHost.nextCycKind, counted from FIRST_EMITTED_CYC_KIND. The llvm-layouts.ts host lacked the hook, so its class descriptors and vtables stamped kind 0 and the native layout emission diverged from the production emitter's.
2026-10-11 09:39:58 -07:00
Malte Ubl 4fd8d5d3b5 Give the self-hosting layout fixture the cycle header's buffered offset
The 16-byte collector header replaced ClassHost.cycleColorOffset with cycleBufferedOffset (6 bytes behind every object on every target); llvm-layouts.ts still set the removed property and lacked the required one.
2026-10-11 09:38:27 -07:00
Malte Ubl 20b7da29e7 Stream serialized IR and LLVM instead of building one string
The self-hosted compiler's compact IR is ~515-528M characters, a few
million below V8's maximum string length (2^29 - 24). serializeModule
joined it into one string, deserializeModule parsed one string, and the
native bootstrap comparison read cli.ir.json and cli.ll as utf8 strings,
so any compiler growth crashed --emit=ir and the bootstrap tests with
"RangeError: Invalid string length".

- serializeModuleParts returns the compact encoding as ordered parts
  (header, one part per function, remainder). serializeModule(mod, true)
  is their join, so the on-disk format is byte-identical.
- deserializeModuleBytes splits a UTF-8 Buffer at the root object's
  "functions" array with a byte-level scanner and parses the header and
  each function separately. Malformed input falls back to the
  whole-document parse for its exact error.
- The Node API writes --emit=ir, --emit-ir sidecars and --emit=llvm
  through writeTextParts (the former writeLlvmSource batching writer).
- The native bootstrap harness decodes IR per function and compares
  LLVM as Buffers, removing ABI-check lines at the byte level.
- The self-hosting serialization fixture gains a "bytes" mode so the
  scriptc-compiled serializer exercises both new entry points.

Self-hosted compiler IR (514.6 MB, macOS arm64, Node 24.12):
  read  old 5.85 s / 4.26 GB max RSS -> new 5.84 s / 1.36 GB
  write old 2.05-2.53 s / 4.48 GB    -> new 2.30-2.37 s / 2.10 GB
  2x module (1.03 GB): old serialize and read throw; new round-trips.
The remaining per-string bound is the largest function (16.3M chars,
~33x headroom) and the header (2.3M chars).
2026-10-10 20:39:35 -07:00
Malte Ubl fcdbe88494 Share virtual-slot borrowing with the self-hosted layout stage
Vtables store the borrowing body of a method whose slot borrows its
parameters, but that decision lived in private emitter state, so the
self-hosting layout fixture kept emitting owned entries and no longer
matched production (`@sc_f_...` vs `@sc_bf_...`).

Move the decision into VirtualBorrows (virtual-borrows.ts), used by the
emitter and by the fixture, which now runs the same call-lifetimes
analysis. The slot lookup narrows explicitly instead of combining a
union with `&&`/`||`, which the self-hosted subset rejects (SC1090). The
abstract-slot emission case now expects the borrowing body, since its
implementation borrows its receiver.
2026-10-09 10:10:26 -07:00
Malte Ubl d58a5c3abc Give the self-hosting layout fixture the int32 slots
emitLlvmLayouts now takes the whole-program int32 slot facts, but the
self-hosting layout fixture never passed them and only imported the
Int32Slots type, so the self-compiled stage could not represent
`Int32Slots | undefined` (SC2009). Compute them with analyzeInt32Slots
like the emitter does, so the fixture compiles statically and its native
output mirrors production int32 field storage.
2026-10-09 10:10:20 -07:00
Malte Ubl fb23f3bf17 Give the self-hosting layout fixture the nullable-union host members
WalkerHost now carries the nullable-union registry and the record-field ABSENT test; the natively compiled layout/metadata fixture builds them the same way the emitter does.
2026-10-09 11:59:24 -05:00
Malte Ubl 213cf287b9 Emit inline retain/release fast paths under --optimization=speed (H30)
Re-lands perf/h30-inline-rc (00051ca0, 40398773, 07972360) behind the
speed posture. With inline RC on, every mirrored runtime RC family
retains inline and releases inline down to the rc == 1 / new cycle
candidate slow path (shapes.ts INLINE_RC_FAMILIES). Release emission is
unchanged: ShapeHost.rcHelpers is null unless LlvmTargetOptions.inlineRc
is set, and every retain/release (including capture boxes) stays a
runtime call; the %ScrMapRc type is only emitted with inline RC.

- index.ts and native/prepare.ts pass inlineRc for speed executables,
  speed object/asm emission, and speed library profiles.
- scr_runtime.h keeps H30's static asserts for the mirrored header and
  trace-slot offsets (no code change; the rebuilt macOS pack's manifest is
  identical to the baseline's).
- Tests: shapes.test.ts pins release symbols and H30's speed fast paths
  and header offsets; emitter.test.ts pins that release IR is unchanged
  and speed routes releases through the helpers. The differential harness
  gains `// @optimization: speed` (and a SCRIPTC_TEST_OPTIMIZATION=speed
  lane); corpus 4032 pins every mirrored family under speed.

Release executables stay byte-identical to the stack tip (14/14
workloads, macOS).
2026-10-08 15:46:34 -05:00
Malte Ubl c0e2d097eb Update self-hosting emission cases for the inline allocator
The full emitter inlines the small-object allocator on 64-bit host targets,
so the self-hosting layout fixture's host enables inlineAlloc for 64-bit
requests (as emitLlvmModule does), and the recursive-records case no longer
asserts the width-specific free path (64-bit pushes onto the allocator's
free list; 32-bit calls scr_cyc_free).

(Split from the stack's later "Update self-hosting emission cases for
acyclic proofs and inline allocation"; the acyclic-proof half moves to the
construction-only acyclic change.)
2026-10-08 15:36:07 -05:00
Chris Tate 1f10e83720 Promote canary to main (#738)
- Preserve callback identity, receiver binding, and class references across unions and collections.
- Track record field presence independently of undefined values and improve native runtime behavior.
- Expand differential coverage and streamline CI bootstrap and compiler rebuild checks.
2026-10-08 11:00:13 -05:00
Chris Tate 095b619330 Add native workers and shared memory
- Add native worker execution with isolated module state, structured messages, and lifecycle controls.
- Support shared buffers and integer atomics with coordinated runtime ownership and cancellation.
- Extend compiler, runtime-pack, differential, and compatibility coverage for native concurrency.
2026-10-07 19:07:02 -05:00
Chris Tate b8ce03cc4c Streamline source discovery and checker metadata (#716) 2026-10-07 00:19:01 -05:00
Chris Tate 25b7fd266a Streamline structured text output
- Share bulk string writes and JSON formatting paths.
- Append scalar values directly and streamline exact integer formatting.
- Cover output boundaries and clarify runtime-pack identity failures.
2026-10-06 17:25:57 -05:00
Chris Tate 88e6e6dcd0 Streamline class callbacks and compiler source access
- Defer empty callback storage while preserving method and prototype updates.
- Reuse source and node-handle information within compiler snapshots.
- Coalesce small checker requests and avoid response copies.
2026-10-06 08:10:34 -05:00
Chris Tate d575a5cfda Centralize builtin contracts and IR analysis ownership
- Derive builtin call names from shared signatures and keep call semantics with their ABI contracts.
- Separate runtime dependency detection, exception metadata, and library policy from the core IR while preserving existing exports.
- Enforce the frontend signature boundary and validate native compilation and policy behavior.
2026-10-05 18:36:03 -05:00
Chris Tate 665db543e2 Organize compiler conversions and builtin dispatch
- Separate conversion planning, helper generation, and timer lowering into focused modules.
- Share array, closure, record projection, and builtin dispatch logic, including timer namespace calls.
- Validate shared helpers with native correctness and differential coverage.
2026-10-05 16:21:22 -05:00
Chris Tate 98dd1b3f0f Optimize native file processing (#688)
- Read binary and text files into owned storage across filesystem entry points.
- Extend integer inference through module initialization and filtered processing loops.
- Preserve shared bindings, byte views, and filesystem error behavior.
2026-10-05 12:59:31 -05:00
Chris Tate 4427f1fb5a Expand native collection keys and construction
- Support boolean, bigint, and mixed value keys in native Maps and Sets.
- Reuse key hashes and copy collections directly into compact storage.
- Preserve collection identity, ordering, iteration, and ownership behavior.
2026-10-05 09:32:14 -05:00
Chris Tate 7a3794d8dd fix: emit TypeScript runtime package sources (#641) 2026-10-03 15:43:26 -05:00
Chris Tate 62ab952f02 Speed up native compiler lowering (#623)
* Speed up native compiler traversal and union handling

* Index live deferred initializer placeholders per worklist pass

* Record preflight baseline for deferred registry corpus
2026-10-02 22:14:59 -05:00
Chris Tate 2069684b40 Expand static Effect support across common APIs (#598)
* Expand static Effect support across common APIs

* Preserve compiler parity across expanded Effect patterns

* Fix compiler regressions exposed by Effect coverage

* Bound shared type conversion traversal

* Clarify conversion contracts and tidy fixtures

* Run every Effect fixture in CI shards

* Preserve inferred array assignments in concurrent Effect programs

* Record inferred array fixture compiler ordering
2026-10-01 02:49:31 -05:00
Chris Tate 18c9a4233b perf: shorten native bootstrap and cache CI toolchains
- Reuse decoded AST nodes, parent references, and recursive class type analysis during compilation.
- Split expression and library validation into smaller functions while preserving diagnostics.
- Shorten sanitizer bootstrap and reuse verified CI toolchains.
2026-09-30 15:12:04 -05:00
Chris Tate eb7be34d9a perf: reduce compiler work and reuse CI native builds
- Reduce frontend, runtime, and linking overhead for native builds.
- Add compact compiler artifacts and reproducible build timing benchmarks.
- Reuse native CI packages and consolidate self-hosting validation.
2026-09-30 11:08:13 -05:00
Chris Tate ee9f0d64eb feat: ship the self-hosted native scriptc CLI (#585)
* feat: ship the self-hosted native scriptc CLI

- Run the shared compiler and CLI natively, including library builds and compile-time evaluation.
- Install native platform packages and ship relocatable standalone distributions.
- Preserve program semantics and reuse validated build artifacts for fast rebuilds.

* fix: complete native CLI integration and validation

- Preserve library diagnostics and run recursive compiler work on the main stack.
- Restore packaged helper permissions and exercise native npm and Wasm library builds.
- Recover interrupted Sandbox status probes within the original command deadline.
2026-09-30 02:17:56 -05:00
Chris Tate f818e8fb89 Extend static three.js support to raycasting
- Compile mesh, line, and point picking with perspective and orthographic camera rays.
- Preserve JavaScript override arguments, array identity, and native subclass values across checked calls.
- Specialize package class helpers and handle unsupported JSDoc namepaths.
2026-09-30 00:35:35 -05:00
Chris Tate 3f66c8e20b Make LLVM the sole production backend
- Remove C emission, debugging options, and automatic backend fallback.
- Complete LLVM lowering and native bootstrap coverage across supported targets.
- Link executables and libraries with precompiled C runtime packs, including iOS and Android.
2026-09-29 20:54:09 -05:00
Chris Tate 0d9d946dde Bootstrap a standalone native compiler (#572)
- Build static programs through the TypeScript 7 frontend, C and LLVM emitters, and native linker without a JavaScript host
- Share runtime-pack verification and link planning across native and Node compiler hosts
- Handle large C output and deep logical-expression validation during compiler bootstrapping
2026-09-29 09:21:59 -05:00
Chris Tate 3f56576644 feat(compiler): rebuild the frontend with native compiler stages (#568)
- Share frontend loading, package selection, contracts, and FFI parsing between Node and native clients.
- Preserve collection layouts and optional method returns while lowering long nullish chains without recursive stack growth.
- Rebuild the frontend from TypeScript through native lowering and C emission with matching output.
2026-09-29 07:45:18 -05:00
Chris Tate 3af05d9fbc feat(compiler): lower the complete frontend statically
- Complete frontend lowering with zero rejected statements, dynamic islands, or skipped functions.
- Support contextual construction, presence guards, and union record copies used by the compiler.
- Add native CommonJS resolution and cross-platform file URL handling while preserving Node host behavior.
2026-09-28 23:49:13 -05:00
Chris Tate 37a19cbb3b feat(compiler): run type mapping natively
- Preserve discriminated union identities across optional storage and literal conversions.
- Run the production type mapper against native TypeScript 7, including scriptc's own IR types.
- Correct tuple reference metadata and void returns through finally for native compilation.
2026-09-28 21:37:37 -05:00
Chris Tate 6aa0d0015b feat(compiler): compile union conversions natively
- Dispatch refined recursive unions through validated literal discriminants.
- Run production retagging and checked extraction helpers as native compiler stages.
- Preserve payload layouts, evaluation order, and narrowing errors across both backends.
2026-09-28 20:42:45 -05:00
Chris Tate 39db8c0a99 feat(compiler): make lowering contexts natively compilable
- Run production scope and closure capture logic with native TypeScript 7 symbols.
- Give lowering caches explicit pass ownership and stable symbol identities.
- Support reference collection arrays and optional Set seeds on both native backends.
2026-09-28 19:48:46 -05:00
Chris Tate 2a23ddc37d feat(compiler): run program preflight with native frontend services
- Compile production program loading across module graphs, packages, and configuration
- Preserve optional defaults and support runtime URL bases and regex split captures
- Isolate host execution services and initialize lowering tables outside module cycles
2026-09-28 17:31:11 -05:00
Chris Tate e517a251ae feat(compiler): preserve frontend structural values in native code
- Preserve class property views, opaque record identity, and nullable structural conversions
- Run package metadata transforms and TypeScript option merges as native code
- Serialize live typed records with opaque payloads and JSON hooks
2026-09-28 15:49:12 -05:00
Chris Tate b40a6f6079 feat(compiler): lower frontend collections and string transforms natively
- Keep generic bindings and short-circuit writes scoped to their native execution context.
- Compile Map and Set copies, tuple callbacks and searches, and string-pattern replacement.
- Exercise production frontend registries without Node and preserve mutation, identity, and evaluation order.
2026-09-28 14:38:05 -05:00
Chris Tate 5a495dc02c feat(compiler): own frontend services in native clients
- Share native parsing, declaration projection, and npm capability analysis through explicit frontend services.
- Scope module analysis caches and parser connections to their owning program lifecycle.
- Preserve global fetch reads in destructuring defaults across embedded npm modules.
2026-09-28 13:48:53 -05:00