Commit Graph
44 Commits
Author SHA1 Message Date
Chris Tate 431a8d6874 Merge main 2026-10-19 21:39:39 -05:00
Chris Tate 2119198c55 Merge main 2026-10-19 20:53:44 -05:00
Chris Tate 8d4737dfdb Merge main 2026-10-19 20:27:03 -05:00
Chris Tate 4770b3ffa7 Fix pi's small compile gaps (#1348) 2026-10-19 19:50:07 -05:00
Chris Tate a31b1fe23e Merge main 2026-10-19 17:17:48 -05:00
Chris Tate 178646baf0 Merge main 2026-10-19 08:31:43 -05:00
Chris Tate 3d4f181cb7 Merge main 2026-10-19 08:15:47 -05:00
Malte Ubl a10ce583d7 Merge remote-tracking branch 'origin/main' into opt/tier-s3-jsval 2026-10-19 05:43:49 -07:00
Malte UblandClaude Opus 5.5 2b8cd22423 Add WebCrypto HMAC: crypto.subtle importKey, sign, verify and digest
crypto.subtle.importKey('raw', ...) for HMAC keys, HMAC sign and verify,
and the SHA-1/SHA-2 digests compile through the crypto global, with
CryptoKey values as native handles. scr_webcrypto.c is only the WebCrypto
layer (Web IDL conversion, algorithm normalization, the CryptoKey handle,
promise wrapping and Node's TypeErrors and DOMExceptions) over node:crypto's
digest core and deferred promise queue. It links only into programs that
call these methods (the new webcrypto link feature). Other algorithms, the
jwk format and the other SubtleCrypto methods are SC2020 refusals.

The real jose@6.2.12 HS256 path (SignJWT, jwtVerify) now runs natively. It
also needed four compiler fixes: object patterns of positions and length
over arrays, the Uint8Array base64/hex feature probes, a compile-time
String.fromCharCode.apply, and structuredClone of a parameter whose default
is {}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 05:34:26 -07:00
Malte UblandClaude Opus 5.5 78184a9869 WebAssembly W3–W4: translate modules to native code at build time, the JavaScript API over them, and Module transfer to workers
A build now translates every WebAssembly module whose bytes it can see
(W2's discovery) into LLVM IR with a TypeScript translator
(packages/compiler/src/wasm/module.ts, translate.ts) and appends it to the
program module, so the existing LLVM helper optimizes, partitions and
caches it with the program. The program also emits scr_wasm_registry, a
portable table of module descriptors (scr_wasm_abi.h). At run time,
WebAssembly.compile, instantiate, validate, new Module and new Instance
find the translated module by the SHA-256 of its bytes; bytes the build did
not see reject with a CompileError. There is no interpreter and no JIT.

The runtime (scr_wasm.c, link-gated by the new `wasm` feature, handle tags
inside W0's headroom) implements the JavaScript API once, over
descriptors, with Node's semantics and messages: exports (functions,
memories, tables, globals), imports (functions, immutable globals, and
memories, tables and globals other instances export), every trap as
WebAssembly.RuntimeError, the CompileError, LinkError and RuntimeError
classes, compile and instantiate settling on a later turn, and
globalThis.WebAssembly as a value. On 64-bit Linux and macOS, memories sit
in 8 GiB guard reservations and a SIGSEGV/SIGBUS handler turns faults into
traps or stack exhaustion; async fibers get a guard page on first use.
Sanitizer builds and Windows check bounds and the stack explicitly. A
WebAssembly.Module crosses to workers through W0's transfer hook (W4).

W1's refusals become lowerings; the Memory, Table and Global constructors,
Module.imports/exports/customSections and the streaming functions still
refuse with SC2020. The globals: WebAssembly compatibility row moves to
partial.

Evidence: differential corpus programs for the API, TypeScript use and the
codemode worker shape, plus five programs executing about 9,300 commands
of the WebAssembly 2.0 spec suite; all match Node in the plain and
sanitized lanes. quickjs-wasi runs through its own glue, matching Node
(apart from a pre-existing non-wasm fence) and on par with or faster than
V8. The zero-cost gate leaves every non-wasm program byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 05:33:11 -07:00
Chris Tate 4f1a304163 Merge main 2026-10-18 20:12:11 -05:00
Malte Ubl f18e804ffd Merge remote-tracking branch 'origin/main' into opt/tier-s3-jsval
# Conflicts:
#	packages/compiler/src/backend/llvm/shapes.ts
2026-10-18 15:58:49 -07:00
Malte UblandClaude Opus 5.5 8ec0d9e0b9 Link the emitWarning replacement and extended stdio only where used
The process.emitWarning replacement and the stdin encoding, string data
listener and stdio 'resize' support changed runtime functions every
program with the dyn-async or events unit links (the shared warning
paths, stdin delivery, the process stream object) and grew their
thread-local state. That code now compiles only into new runtime-pack
variants, SCR_WARNING_REPLACE of scr_async_dyn.c and SCR_STDIO_EXTENDED
of scr_events.c, which the link selects when the program assigns
process.emitWarning or reaches setEncoding, a string 'data' listener or
'resize'. The default objects are byte-identical to before, so other
executables do not change. The development and sanitizer builds pass the
same defines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 15:57:42 -07:00
Malte UblandClaude Opus 5.5 89bca68cbe Compute with NaN-boxed jsval values in the dynamic tier
Stage S3 of the two-tier design: dynamic-tier functions compute with
jsval, a new IR type holding a 64-bit NaN-boxed word whose numbers,
booleans, undefined and null are immediates, so a number between two
dynamic operators is no longer a heap node.

- IR: the jsval type and js.* builtins (operators and the boundary
  conversions fromDyn/toDyn/fromF64/fromBool/fromStr/fromBigInt/toF64).
  The jsval pass rewrites dynamic-tier functions after tier assignment:
  operator chains, comparisons, number checks and truthiness of operator
  results run on jsvals; dyn locals become jsval locals when that saves
  boxing; dynamic functions called only from dynamic functions return a
  jsval. IR version 17 serializes each function's tier.
- Runtime: scr_js.h and scr_js_value.c behind the dynTier feature
  (invariant I3); static runtime units never include scr_js.h.
- Backend: number cases inline (immediate doubles and NUM cells), always
  inlined jsval retain/release; 32-bit targets keep boxed cells.
- Interprocedural analyses see jsval conversions as the dyn forms they
  replace, so no fact about a static function changes.

Static functions are untouched: typed programs and the self-hosted
compiler's static functions compile to byte-identical LLVM.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 15:11:32 -07:00
Malte UblandClaude Opus 5.5 d7fcc9287c Merge origin/main into opt/gc-nursery
main now has #1286 and later collector changes. Resolved in scr_cycle.c:
the scheduled collection point collects the nursery, then notes the
live-count fall (death rule). Semantic fixes for the new main code:
a full pass's markGray no longer overwrites a young object's `buf` (its
nursery index) with the component number, and a nursery collection
refreshes the death rule's live reading at its end so the young garbage
it frees does not count as deaths outside passes. Formats the runtime
pack matrix test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 13:14:30 -07:00
Malte UblandClaude Opus 5.5 147013d7b9 Merge origin/main into opt/npm-small-fixes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 12:58:58 -07:00
Malte UblandClaude Opus 5.5 19c742834e Add an experimental young-in nursery for compiled classes and records
SCRIPTC_YOUNG_NURSERY=1 at compile time makes an executable allocate
traced, program-owned class instances and records young and link the
collector unit built with SCR_YIN (a runtime pack variant of
scr_cycle.c; libraries never get it). The emitted constructor lists the
object in a per-thread nursery and marks its header: generation
SCR_CYC_YOUNG and its nursery index under the buffered bit, so every
existing release path treats it as a buffered candidate (no call while it
stays alive; scr_cyc_on_dead takes it out of the nursery at its death).

A nursery collection (whenever 4096 young objects are live, at collection
points, before full sweeps and limited passes) counts each young object's
references from other young objects, marks from the objects whose count is
below their reference count (referenced from outside the nursery: stacks,
globals, older objects, runtime structures, FFI, fibers, coroutine
frames), frees what stays unmarked, and gives up the dead set's outgoing
edges after the teardowns. The counts are recomputed at every collection
instead of maintained by a store barrier, so soundness rests only on the
invariants trial deletion already needs. Survivors join trial deletion's
nursery and outside roots become candidates; trial deletion walks young
objects it reaches as nursery-level objects. Per-type admission statistics
send types that do not die in young cycles back to trial deletion for a
while. Publication takes young objects out of the nursery. The RC audit
build checks nursery membership and counts at every collection.

With the flag off, the compiler emits and links exactly what it did
before (249 programs and the self-hosted compiler compared
byte-identical). Measured with the flag on: cg-rings -38%, cg-pairs -27%,
the runtime suite geomean -0.3% (published-graphs +3.4%), the self-hosted
compiler neutral; shapes whose cycles go through arrays or closures
(phase 2) pay without gaining.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 11:48:50 -07:00
Malte UblandClaude Opus 5.5 577b0135a9 Add execution tiers to the IR and a report-only tier analysis
Stage S2 of the two-tier design. IrFunction gains a tier (absent means
static); lowering assigns it from function bodies, placing functions that
use the dyn vocabulary in the dynamic tier, and the validator enforces
invariant I1: a static function performs no dynamic operation, only
boundary operations (dynFrom, dynCheck, kind tests) and moves. Serialized
IR keeps the field out in this stage and loaders derive it again, so every
IR, LLVM and executable artifact is unchanged.

Every committed refusal and runtime fence becomes a recorded demotion
request attributed to the binding, field, function instance, class or
module the dynamic tier would take over, while the compiler still refuses
and fences as before. `scriptc coverage` reports dynamic-tier statements
(statementsDynamic), `--explain-dynamic[=<name>]` prints the demotion
chains, and `--tier=auto|static-only` is wired through build, run and
coverage for stage S6; both modes compile identically today.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:12:27 -07:00
Malte UblandClaude Opus 5.5 e79380ce17 Name the runtime package in npm refusals and add transitive --npm-static
- A package typed only by an @types twin was judged under the twin's name
  and refused with "its import resolves to '@types/x/index.d.ts', not to
  runtime JavaScript". Selection now resolves the runtime package, and the
  refusal names it with the --npm-static option that compiles it anyway.
- Refusals the declaration and shipped-JS heuristics make carry the same
  remedy as a diagnostic hint; a dependency of a named package suggests
  the transitive form.
- `--npm-static=pkg...` (and `"pkg..."` in a library profile) selects the
  package and every package its files import, nested require() calls
  included, without the eligibility heuristics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 19:39:21 -07:00
Chris Tate c9fe456a96 Support crypto.createCipheriv and createDecipheriv 2026-10-14 15:14:14 -05:00
Chris Tate a4af77ee8d Add EventTarget, Event and CustomEvent 2026-10-14 02:58:57 -05:00
Chris Tate 7e6294ca0f Make scriptc compilation native-only 2026-10-12 12:25:30 -05:00
Malte Ubl 5d4509702f wip: contract types prototype 2026-10-11 11:17:17 -07:00
Malte Ubl 6a9475e931 Free the address space after process.exit off the critical path (--detached-exit)
`scriptc build|run --detached-exit` opts an executable into handing the address-space teardown after process.exit to a helper process on Linux. The helper shares the address space (CLONE_VM), blocks signals, closes every descriptor but the read end of a pipe and waits for EOF on it. The kernel releases an exiting process's memory (exit_mm) before its files (exit_files), so EOF means the helper holds the last reference; it then exits and the kernel frees the pages there instead of before the parent sees the exit (30-40 ms per GB of 4 KiB pages; tsc-ts 20-110 ms). The same latency mold hides with --fork.

It is a build-time choice, not a run-time knob, because the helper keeps the executable mapped (overwriting the binary right after it exits can fail with ETXTBSY) and the memory charged to the cgroup for up to ~100 ms, and the teardown CPU time leaves the parent's rusage. The program object defines `scr_detached_exit = 1`, overriding the runtime's weak default of 0, so no runtime flavor or linker flag is added. The option is part of the early executable cache and route keys; executables only, no effect on other platforms.
2026-10-11 07:05:29 -07:00
Chris Tate 3e41bcbfd0 Make compile blockers and Node differences actionable (#786) 2026-10-09 10:30:31 -05:00
Malte Ubl 2372b75ec2 Add an opt-in --optimization=speed posture
Speed is a third native optimization posture next to release (default,
-O2) and dev (-O0). It is optimized exactly like release at the C/LLVM
level and is the switch for optimizations that trade executable size and
build time for run-time speed, so the default release build stays inside
the binary-size budget.

- backend/optimization.ts: the NativeOptimization type, its -O class, and
  cache-key helpers. Release keeps its historical key shapes (absent).
- CLI: build/run/cache warm accept speed; usage text names the trade.
  Bootstrap's routed executable cache mirrors the posture.
- Early executable cache keys and native-feature validation include speed,
  so release, dev, and speed entries never satisfy one another.
- Runtime packs: an optional executable `speed` flavor (-O2, only beside
  release). Speed links it when present and the release objects otherwise;
  library modes link their release flavor. The sanitizer lane follows the
  same flavor key.
- Library profiles accept "speed".
- WASI release-only linker flags apply to speed too.
- bench-runtime: --baseline-optimization/--candidate-optimization and a
  per-contender cold_build_ms.

Release executables are byte-identical to the stack tip for all 14
benchmark workloads (macOS arm64, sha256 of executables built from both
trees).
2026-10-08 15:46:34 -05:00
Chris Tate 095b619330 Add native workers and shared memory
- Add native worker execution with isolated module state, structured messages, and lifecycle controls.
- Support shared buffers and integer atomics with coordinated runtime ownership and cancellation.
- Extend compiler, runtime-pack, differential, and compatibility coverage for native concurrency.
2026-10-07 19:07:02 -05:00
Chris Tate 52cca919f5 Refactor native toolchain ownership and shared build operations
- Separate driver, fingerprint, artifact, and build responsibilities.
- Share shard selection, trace parsing, cache staging, and stamp publication.
- Preserve cache invalidation and enforce implementation imports.
2026-10-05 20:48:48 -05:00
Chris Tate c08ab955ae Enforce linting and consistent source formatting
- Enforce oxlint and oxfmt for owned JavaScript and TypeScript with independent CI checks.
- Preserve compiler import boundaries and protected fixtures while cleaning up lint findings.
- Standardize source formatting and validate import, fixture, and embedded-source protections.
2026-10-05 17:51:24 -05:00
Chris Tate 4bd14bc101 perf: defer native driver discovery on CLI cache misses (#639) 2026-10-03 13:43:33 -05:00
Chris Tate a07998f667 Expand static three.js workflows and host integration (#621)
* Expand static three.js workflows and host integration

* Preserve JavaScript class state in static three.js workflows

* Preserve inferred JavaScript behavior across static workflows

* Preserve inferred callables and native compatibility boundaries

* Retain annotated JavaScript factory exports and bootstrap-safe call spreads

* Preserve Buffer and child callback identities in static packages

* Clarify JavaScript symbol spread support
2026-10-02 19:56:34 -05:00
Chris Tate eb7be34d9a perf: reduce compiler work and reuse CI native builds
- Reduce frontend, runtime, and linking overhead for native builds.
- Add compact compiler artifacts and reproducible build timing benchmarks.
- Reuse native CI packages and consolidate self-hosting validation.
2026-09-30 11:08:13 -05:00
Chris Tate ee9f0d64eb feat: ship the self-hosted native scriptc CLI (#585)
* feat: ship the self-hosted native scriptc CLI

- Run the shared compiler and CLI natively, including library builds and compile-time evaluation.
- Install native platform packages and ship relocatable standalone distributions.
- Preserve program semantics and reuse validated build artifacts for fast rebuilds.

* fix: complete native CLI integration and validation

- Preserve library diagnostics and run recursive compiler work on the main stack.
- Restore packaged helper permissions and exercise native npm and Wasm library builds.
- Recover interrupted Sandbox status probes within the original command deadline.
2026-09-30 02:17:56 -05:00
Chris Tate 3f66c8e20b Make LLVM the sole production backend
- Remove C emission, debugging options, and automatic backend fallback.
- Complete LLVM lowering and native bootstrap coverage across supported targets.
- Link executables and libraries with precompiled C runtime packs, including iOS and Android.
2026-09-29 20:54:09 -05:00
Chris Tateandtimokosonen 0e6a13d041 fix(compiler): support source breakpoints in dev builds (#481)
- Emit source locations for native TypeScript and JavaScript debugging across C and LLVM backends.
- Preserve macOS debug symbols through linking and cache reuse, with regression coverage and Xcode guidance.

Fixes #461

Co-authored-by: timokosonen <325538238+timokosonen@users.noreply.github.com>
2026-09-26 22:21:11 -05:00
Chris Tate 0c109d93f8 Add opt-in executable stripping (#425)
- Link smaller executables across native targets with an explicit --strip flag.

- Keep stripped and default builds separate in the cache and cover their behavior.
2026-09-25 09:12:58 -05:00
Chris Tate 9f48bc720a Preserve artifacts when switching build modes (#415)
- Keep generated outputs from earlier build modes, including cache hits.
- Cover WASI/native switching and document accumulated output files.
2026-09-24 23:45:17 -05:00
Chris TateandMateus Mamede Lage b6bd80c391 fix(windows): support GUI executables (#380)
* fix(windows): support GUI executables

- Add Windows subsystem selection across executable link paths and cache keys.
- Verify PE subsystem output and add Windows CI coverage.

Fixes #259

Co-authored-by: Mateus Mamede Lage <23098414+mmamedel@users.noreply.github.com>

* fix(ci): restore cross-platform test parity

- Make the Windows subsystem test use a portable loader URL.
- Sync TS7 order-parity baselines with current corpus fixtures.

---------

Co-authored-by: Mateus Mamede Lage <23098414+mmamedel@users.noreply.github.com>
2026-09-22 20:40:11 -05:00
Chris Tate 1600935411 Add static-native BigInt support (#346)
* feat: add static BigInt support

* fix: prevent BigInt width overflow on WASI

* fix: harden BigInt formatting sizes

* test: record BigInt WASI parity baseline
2026-09-18 00:21:12 -05:00
Chris Tate 469efdd7f0 refactor: clarify compiler module boundaries (#232)
* refactor: clarify compiler module boundaries

- Rename backend, frontend, IR, and cache modules for explicit ownership.
- Split native cache/vendor and frontend shared concerns into focused modules.
- Expand compiler context names and refresh imports, tests, and documentation.

* fix: preserve renamed compiler interfaces
2026-08-23 20:16:13 -05:00
Chris Tate c6210788a7 refactor: share early cache primitives (#228)
- Extract common cache hashing, paths, payload I/O, and feature validation.
- Preserve library and executable stamp schemas and on-disk identities.
2026-08-23 17:02:51 -05:00
Chris Tate 5138c1683a perf: add executable dev object caching (#191)
- Add an explicit executable dev posture using native -O0 builds
- Split large LLVM programs into parallel, independently cached objects
- Preserve release cache compatibility and safe single-TU fallbacks
2026-08-21 13:03:31 -05:00
Chris Tate 2805c3ddb4 perf: lazy-load the compiler from the CLI (#189)
* perf: lazy-load the compiler from the CLI

- Serve help and version from a lightweight bootstrap and route exact executable cache hits before loading the compiler graph.
- Validate routed hits against compiler, frontend, toolchain, and native dependency proofs with portable cache publication.

* fix: repair lazy executable cache routes
2026-08-21 10:29:34 -05:00
Chris Tateandvercel[bot] f237e44c73 perf: cache complete executable builds early (#188)
* perf: cache complete executable builds early

- Restore exact executable builds before TypeScript frontend and lowering work.
- Validate frontend, native dependency, mode, target, and FFI identities before reuse.
- Cover cross-process hits, corruption repair, invalidation, and native fallback behavior.

* Update tests/harness/README.md

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

* fix: harden early executable cache identity

* fix: fingerprint effective compiler for early cache

---------

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
2026-08-21 08:41:51 -05:00