Commit Graph
40 Commits
Author SHA1 Message Date
Chris Tate e5ed06771f Use plain wording in the introduction and READMEs 2026-10-13 00:40:11 -05:00
Chris Tate d316552918 Rewrite docs for a coherent new-user flow 2026-10-13 00:34:50 -05:00
Chris Tate 7e6294ca0f Make scriptc compilation native-only 2026-10-12 12:25:30 -05:00
Malte Ubl fa55ce6b07 Keep the RC entry points of array elements in one shared descriptor
Every SCR_ELEM_REF array stored its element type's retain, release and trace entry points in its own header (24 bytes), so ScrArr was 80 bytes: an 80-byte block for arrays of acyclic elements and a 96-byte block for cycle-capable ones. The three pointers now live in one constant ScrArrOps descriptor per element type, which the compiler emits once per module (@sc_arrops.*) and the runtime keeps static, and the header holds a pointer to it plus a 4-byte `cyc` flag beside `elem` that the inline RC fast paths test. ScrArr is 64 bytes: arrays take 64- and 80-byte blocks.

scr_arr_new_ref(retain, release, trace, cap) becomes scr_arr_new_ops(ops, cap); scr_set_to_arr_ref takes the result's descriptor from its caller instead of building an array from the set's key adapters. The backend names the %ScrArr fields in SCR_ARR_FIELD. The layout change bumps the runtime ABI marker to scr_runtime_abi_v13.
2026-10-11 19:46:48 -07:00
Malte Ubl def6f86a0e Let collector types bypass the generations where they never die
Nearly every long-lived object becomes a cycle candidate once, soon after
it is built: the constructing frame's release leaves it held only by the
structure it was stored into. A nursery pass then walks it (about 75 ns per
object), it survives, and it is re-buffered for the mature level, whose
next pass walks it again. In tsc-ts single-threaded, nursery passes walked
6.0M objects on Excalidraw and 36.5M on Sentry and freed 7%; 99% of what
they walked was promoted. Most of that walking goes to types whose objects
never die in a young cycle: SymbolLinks, NodeLinks and Symbol were 2.0M of
Excalidraw's candidates and the collector freed none of them, while young
garbage comes from a few types (inference contexts and mappers, closures,
some arrays). Types with a collector yield under 0.5% are 42-55% of what
nursery passes walk.

So the collector now keeps per-thread statistics per type and level. Each
ScrCycType gains a `kind` index: emitted descriptors number themselves from
32 (FIRST_EMITTED_CYC_KIND / SCR_CYC_KIND_EMITTED), the runtime's own
families have fixed kinds below that, 0 keeps no statistics. For the
nursery and mature levels, a pass counts the candidates of each kind it
walks and the objects of each kind it frees. After every 8192 walked
candidates a kind bypasses that level for the next window if fewer than 1
in 128 of them were freed. A release that buffers an object at a level its
type bypasses promotes it past the level first, and a nursery pass that
re-buffers a survivor into a bypassed mature level moves it to the old
level. One candidate in 16 stays put, so the yield keeps being measured
and a type whose objects start dying young again is walked again after
one window.

Only the generation an object is judged in changes. Any assignment is
sound (a restricted pass counts edges from older objects as external), a
bypassing candidate stays buffered, and full passes walk everything they
did before, so the old level's bounds hold unchanged. Cyclic garbage made
only of objects of a bypassing type waits for the next pass at an older
level. Programs that never complete a window (8192 walked candidates of one
type) are collected exactly as before.

The descriptor layout is part of the emitted-code/runtime ABI, so the
program-object marker moves to scr_runtime_abi_v12.

Pass log (instrumented build, single-threaded, one run each, with the
preceding null-slot change): all collector passes Excalidraw 606 -> 313 ms
(nursery 435 -> 153), TypeORM 629 -> 383 ms, tsc-ts self-check 377 -> 215
ms, Sentry 7.61 -> 5.75 s (nursery 2.92 -> 1.04 s, mature 1.17 -> 1.02 s;
the full pass is unchanged at 3.6 s).

tsc-ts (Linux sandbox, interleaved, medians; 10 rounds, Sentry 5 rounds
pinned to 8 CPUs; both changes against int/best-v6): single-threaded
Excalidraw -8.0%, TypeORM -4.2%, self-check -4.3%, Sentry -5.3%;
--checkers 8 -12.2%, -0.8%, -2.3%, -1.6%. Peak RSS -0.8% to +0.6%.
Diagnostics identical.

Tests: test_cycle gains check_nursery_bypass, check_mature_bypass and
check_rebuffer_past_mature (a test-only entry point, scr_cyc_pass_at under
SCR_CYC_TESTING, runs passes at exact levels); each fails under the
corresponding mutation (no bypass, a sticky decision, no sampling, no freed
or walked counts, a bypassing candidate left unbuffered, survivors
re-buffered into a bypassed level). Corpus 4690 builds a long-lived
declaration graph large enough for both levels to bypass and lets objects
of the same classes die in cycles; under --sanitize the exit RC audit
proves they are reclaimed.
2026-10-11 09:39:57 -07:00
Malte Ubl 1a14263b5f Move rarely used array storage out of line
Every ScrArr carried sixteen words, seven of them (sparse slots, their
length and capacity, noncanonical numeric properties with length and
capacity, and match-result metadata) used only by sparse arrays, arrays
with numeric non-index properties and regex match results. They now live in
a ScrArrExt the array points at, allocated on first use: a dense array
shrinks from 128 to 80 bytes, and a cycle-capable one (with its 16-byte
collector header) from 144 to 96.

Runtime readers go through a shared empty ext; writers allocate it. The
backend mirror loses the seven fields, tagged-template arrays are emitted
with a NULL ext, live stream commits swap the ext pointer (which swapped all
seven fields before), and the cached typed-view refresh swaps sparse slots
and properties through scr_arr_swap_indexed_ext, keeping each array's
metadata as before. The program-object ABI marker moves to
scr_runtime_abi_v11.

A corpus program (4610-array-ext-dyn-refresh.ts, with its order baseline) pins numeric non-index properties of a dynamically typed array through its refreshed typed view, sparse slots written directly and through a live reference, truncation, and regex match metadata read directly, through `any` and after a copy.
2026-10-11 09:38:27 -07:00
Malte Ubl 6f488e13aa Shrink the cycle-collector header to 16 bytes
Every cycle-capable object carried a 32-byte header: trace and teardown function pointers, a 32-bit color, the buffered flag, the generation and a 64-bit candidate index. The two function pointers are the same for every object of a type, so the header now points at one constant ScrCycType { trace, free_fn } descriptor per type, and the remaining fields pack into the second word: u8 color, u8 gen, u16 buffered, u32 buf_index (candidate buffers stop at 2^31 entries). wasm32 pads before the descriptor, so every object-relative offset is the same on all targets.

Emitted shapes define their descriptor next to the trace and teardown functions; inline allocation stores one header word; the inline release reads buffered at obj-6 and the inline free reads the generation as an i8 at obj-7. Runtime families allocate with static descriptors. The program-object ABI marker moves to scr_runtime_abi_v10.

tsc-ts (interleaved, 5 rounds): peak RSS -8% to -11% on every app and mode; single-threaded wall -0.3% to -5.5% on top of the typed-rest-spread change.
2026-10-11 09:38:26 -07:00
Chris Tate c67e08a9cf Prepare v0.2.8
- Synchronize published packages and the compiler surface manifest at v0.2.8, and bump the runtime ABI marker to v9.
- Add release notes for worker graph sharing, catchable deep recursion, parallel code generation, runtime performance, and compatibility fixes.
2026-10-10 10:08:39 -05:00
Chris Tate 1f10e83720 Promote canary to main (#738)
- Preserve callback identity, receiver binding, and class references across unions and collections.
- Track record field presence independently of undefined values and improve native runtime behavior.
- Expand differential coverage and streamline CI bootstrap and compiler rebuild checks.
2026-10-08 11:00:13 -05:00
Chris Tate 095b619330 Add native workers and shared memory
- Add native worker execution with isolated module state, structured messages, and lifecycle controls.
- Support shared buffers and integer atomics with coordinated runtime ownership and cancellation.
- Extend compiler, runtime-pack, differential, and compatibility coverage for native concurrency.
2026-10-07 19:07:02 -05:00
Chris Tate e2d58f60d2 Expand static support for common Effect 4 patterns (#600)
* Expand static compilation for common Effect 4 patterns

* Run every Effect 4 fixture slice in CI

* Preserve existing corpus behavior across Effect 4 lowering

* Preserve Effect platform callbacks and native stream state

* Record new corpus baselines and retain child stdin refusals

* Retire the obsolete stateful regex diagnostic

* Preserve promise reference count layout for cycle collection
2026-10-01 17:36:29 -05:00
Chris Tate 0845c7ff56 Make native installs portable and smaller (#599)
* Make native installs portable and smaller

- Support glibc 2.34 with portable compiler binaries and static LLVM helpers.
- Bundle the host runtime and discover optional cross-target packs from project dependencies.
- Add application build benchmarks covering latency, executable size, and memory.

* Keep bootstrap tools available in Sandbox validation

- Isolate Zig for native compiler bootstrap and glibc entropy contracts.
- Preserve the existing toolchain scope for cache and corpus tests.

* Fix portable compiler bootstrap linking

- Keep native support objects compatible with the destination host linker.
- Resolve installed targets using the host declared by the toolchain manifest.
2026-10-01 10:46:16 -05:00
Chris Tate c257368c5e Compile Effect generators statically (#589)
* Compile Effect generators statically

* test: declare the dormant array write's expected fence
2026-09-30 14:25:29 -05:00
Chris Tate ee9f0d64eb feat: ship the self-hosted native scriptc CLI (#585)
* feat: ship the self-hosted native scriptc CLI

- Run the shared compiler and CLI natively, including library builds and compile-time evaluation.
- Install native platform packages and ship relocatable standalone distributions.
- Preserve program semantics and reuse validated build artifacts for fast rebuilds.

* fix: complete native CLI integration and validation

- Preserve library diagnostics and run recursive compiler work on the main stack.
- Restore packaged helper permissions and exercise native npm and Wasm library builds.
- Recover interrupted Sandbox status probes within the original command deadline.
2026-09-30 02:17:56 -05:00
Chris Tate c7b11a9242 Compile embeddable Wasm modules with host imports
- Emit callable Wasm reactors with explicit exports, host callbacks, and independent instance memory.
- Compile three.js scene and geometry workloads into Wasm without an embedded JavaScript engine.
- Document initialization, memory ownership, and external browser hosting.
2026-09-30 01:09:15 -05:00
Chris Tate 3f66c8e20b Make LLVM the sole production backend
- Remove C emission, debugging options, and automatic backend fallback.
- Complete LLVM lowering and native bootstrap coverage across supported targets.
- Link executables and libraries with precompiled C runtime packs, including iOS and Android.
2026-09-29 20:54:09 -05:00
Chris Tate 31332427b6 Compile local class factories and spread calls natively
- Preserve fresh constructor identity and shared captures across local class instances.
- Carry constructors through unknown values without a JavaScript engine.
- Support runtime-length spreads into fixed and rest parameters.
2026-09-28 20:11:25 -05:00
Chris Tate f7d217f0d5 Support native ReferenceError, EvalError, and URIError (#501)
* Support additional native Error subclasses

* fix error identity across island callback bridge

* record new Error family parity entry
2026-09-27 20:27:50 -05:00
Chris Tate ffe5e27563 feat(compiler): run the IR validator natively (#493)
- Compile the production validator and exercise native IR construction and validation on C and LLVM.
- Preserve Error causes, wider callbacks, and discriminated JSON layouts in static programs.
- Version the IR and runtime ABI and cover the new compiler paths with differential and sanitizer tests.
2026-09-27 13:11:56 -05:00
Chris Tate c41f1a8be9 Expand LLVM native output targets (#280)
* Expand LLVM native output targets

- Add owned helper and runtime-pack contracts for macOS x64, Linux glibc/musl, Windows x64, and WASI.
- Generalize LLVM target selection, package validation, linker plans, and release/CI matrices.
- Document linker boundaries and add focused native-output and runtime-pack coverage.

* Pin Windows LLVM helper CI to VS 2022

The official LLVM archive needs the Visual Studio 2022 CMake generator, which is not guaranteed by windows-latest.

* Align bootstrap cache with LLVM runtime-pack builds

Use the same target-specific helper and linker identity before and after compiler loading so routed executable cache hits remain lightweight.

* Install Windows LLVM CI tools with Chocolatey

windows-2022 includes Visual Studio but not winget; use its available Chocolatey bootstrap for CMake and Ninja.

* Prevent Zig version probes from leaving runtime-pack objects

Run runtime-pack compiler version probes in a private temporary directory and remove the accidental tracked WASI a.o file.

* Use 7-Zip for Windows LLVM setup

Extract the official LLVM development archive with two-stage 7-Zip instead of Windows tar.exe, which timed out while materializing the toolchain tree.

* Define ssize_t for the MSVC runtime pack

Clang's MSVC target does not expose POSIX ssize_t through sys/types.h; define the pointer-sized runtime type without affecting MinGW.

* Build the Windows runtime pack with Zig

Use the MinGW-compatible Windows sysroot required by the runtime while retaining COFF helper output and installing Zig in the Windows native CI lane.

* Give macOS LLVM differential shard time to finish

Shard 1 passed its setup and focused contracts but was cancelled during its cold LLVM differential slice at the 20-minute job limit.
2026-09-01 13:41:34 -05:00
Chris Tate 1cec922d95 Separate native linking from C compilation (#278)
* Separate native linking from C compilation

 - Split the helper-object runtime-pack path into explicit link planning and platform linking.
 - Reserve SCRIPTC_LINKER for object-only executable links while retaining a deprecated SCRIPTC_CC compatibility path.
 - Add no-legacy-C coverage and document the linker/SDK boundary.

* Preserve runtime pack link errors

* Fix linker cache invalidation and warnings

* Fix C executable cache routing

- Keep explicit C builds on compiler-backed cache identities.
- Cover linker-only environment changes across exact C repeats.
2026-08-31 20:29:59 -05:00
Chris Tate 64b2a285d5 feat(runtime): ship precompiled macOS artifacts (#248)
* feat(runtime): ship precompiled macOS artifacts

- Add a versioned macOS arm64 runtime pack with deterministic feature selection and artifact verification.
- Link LLVM helper objects against release-built runtime and vendor inputs without compiling user-machine C.
- Wire package publishing, cache validation, documentation, and full-gate coverage for the new path.

* fix(runtime): harden precompiled artifact handling

* fix(runtime): make precompiled artifacts reproducible

* fix(runtime): reject opaque linkers from executable cache

* fix(runtime): normalize precompiled archive metadata

* fix(runtime): harden precompiled artifact caching

* fix(runtime): close executable cache link race

* fix(runtime): stage verified pack artifacts

* fix(runtime): bracket helper cache inputs

* fix(runtime): preserve executable link identity

* fix(runtime): trace selected linker dependencies
2026-08-28 09:29:21 -05:00
Chris Tate 73efea6ab3 feat(cli): expose native object link info (#247)
* feat(cli): expose native object link info

- add versioned, cache-independent native link metadata for program objects

- document the experimental runtime ABI and provide C-driver and Apple linker examples

- verify ABI mismatch, FFI parity, packed installs, and external linking in CI

* fix(example): validate native runtime pack identity
2026-08-27 13:47:09 -05:00
Chris Tate 401338f437 feat(compiler): add native LLVM object emission (#242)
* feat(compiler): add native LLVM object emission

- ship the pinned macOS arm64 LLVM helper and platform package
- add assembly/object CLI outputs, caching, diagnostics, and runtime ABI checks
- validate helper artifacts through CI, packaging, docs, and differential coverage

* fix(compiler): harden native LLVM emission

* fix(compiler): harden LLVM helper packaging

* fix(compiler): stabilize LLVM helper package size

* fix(compiler): harden native helper validation

* fix(compiler): harden LLVM helper verification

* fix(compiler): harden native object emission
2026-08-27 10:30:40 -05:00
Chris Tate 77fad3c4cc feat(cli): add source-level output kinds (#241)
* feat(cli): add source-level output kinds

- add typed IR, C, and LLVM primary artifacts without native toolchain discovery
- centralize output paths, compatibility validation, and stale artifact cleanup
- document and test the new CLI contract

* fix(cli): harden source output contracts

* fix(cli): harden source output compatibility
2026-08-26 15:26:24 -05:00
Chris Tate 97585a8b5d Update description (#234) 2026-08-24 01:27:26 -05:00
Chris Tate 741e8cd3f6 Split volatile library identity code (#174)
* Split volatile library identity code

- Emit build identity getters from a tiny standalone C translation unit
- Cache generated library program objects independently of volatile identity and archives
- Preserve toolchain, runtime, and translation-unit dependency invalidation coverage

* Preserve library identity during localization

* Keep library identity sources private

* Preserve library identity in public emission

* Preserve identity in kept library units
2026-08-19 23:36:22 -05:00
Chris Tate 06d50244d0 Reuse IR across comment-only library edits (#171)
* Reuse IR across comment-only library edits

- Restore validated lowered IR when TypeScript edits change only non-semantic comments
- Rebase source locations and regenerate exact-source sidecar and build identities
- Add semantic invalidation, integrity, and real-cache coverage

* Fix semantic cache lexical validation

* Use canonical IR version in semantic cache

* Fix semantic cache lexical rebasing

* Fix semantic cache Unicode identifiers
2026-08-19 13:02:47 -05:00
Chris Tate d1a1da97a2 Cache unchanged library frontends (#169)
* Cache unchanged library frontends

- Restore generated library artifacts before TypeScript and lowering on exact repeats
- Track source, configuration, package-resolution, and missing-candidate inputs for safe invalidation
- Add integrity, replacement, and cache-disable coverage plus cache documentation

* Fix early frontend cache invalidation

* Fix early cache input race tracking

* Fix early frontend cache output tracking

* Key frontend cache by Node version
2026-08-18 11:04:08 -05:00
Chris TateandAlbert 31f268be2b feat: support wasm32-wasi as a production target (#133)
* feat: support wasm32-wasi as a production target

- Add production LLVM WASI codegen with a 32-bit ABI, coroutines, and dynamic modules
- Integrate WASI builds and execution into the CLI with capability diagnostics
- Add differential coverage and document the fourth production target

Co-authored-by: Albert <57402359+bulatovv@users.noreply.github.com>

* fix: address wasm target review findings

* fix WASI guest paths and fetch alias detection

* fix WASI async bridge and fetch detection

---------

Co-authored-by: Albert <57402359+bulatovv@users.noreply.github.com>
2026-08-11 14:12:59 -05:00
Chris Tate 0d5bc0dd0d feat(coverage): resolve external host type surfaces (#80)
* feat(coverage): resolve external host type surfaces

- Add repeatable --external-types mappings for checker-only module resolution.
- Keep host runtime values as explicit blockers with tests and documentation.

* fix: validate external type mappings

* fix: fence external host runtime edges

* fix(compiler): skip empty external type lookup

* fix(compiler): honor external type boundaries
2026-08-02 19:41:35 -05:00
Chris Tate 68263610e3 fix(compiler): prevent stale native cache hits 2026-08-02 13:59:06 -05:00
Chris Tate a1215f6ef9 fix(compiler): bypass cache for opaque wrappers 2026-08-02 13:00:17 -05:00
Chris Tate 24904ddff7 fix native cache invalidation and safety 2026-08-02 12:03:56 -05:00
Chris Tate 1f1f7d42a7 fix native cache invalidation 2026-08-01 15:39:44 -05:00
Chris Tate 6b996ea4a8 fix: harden persistent compilation cache 2026-08-01 15:39:44 -05:00
Chris Tate c23741ba75 fix: relink ambient system libraries 2026-08-01 15:39:44 -05:00
Chris Tate a578a9ab9b Persist native compilation cache
- Enable bounded platform-native caching for production executable and library builds.

- Reuse library runtime objects across edits and avoid duplicate no-npm frontend loading.

- Cover cache defaults, invalidation, bypass behavior, and artifact equivalence; document controls.
2026-08-01 15:39:43 -05:00
Chris Tate bed63d3223 feat: add outbound native FFI
- Add a strict versioned manifest and direct C ABI lowering for both backends.
- Link declared native inputs through the CLI and compiler APIs with precise diagnostics.
- Cover the ABI under plain and sanitized integration tests and document its contracts.
2026-07-25 09:56:48 -05:00
Chris Tate 23d5918a53 init 2026-07-22 18:08:04 -05:00