- Lower typed-array reads, writes, and lengths directly by IR element kind.
- Preserve coercion, bounds, and receiver lifetime semantics across C and LLVM.
- Cover emission, ABI layout, and Node-differential behavior.
Co-authored-by: morler <478444+morler@users.noreply.github.com>
* fix(compiler): support implicit top-level await modules
- Match Node 24 syntax detection by treating top-level await as an ES module marker.
- Add differential corpus coverage for TypeScript files without explicit imports or exports.
* fix Node ESM classification for top-level await
* fix ESM require diagnostics and classification caching
* fix require binding and package type precedence
* fix: align top-level await module detection with Node
* runtime: pin the ScrCycHdr layout the LLVM backend hardcodes
scr_cyc_mark_live is a static inline, so there is no symbol for the LLVM
backend to call — it open-codes the retain hook as a raw `store i32 0` at
obj-16 and reaches the cycle header at obj-32. Three sites emit that
(llvm/shapes.ts, llvm/classes.ts, llvm/emitter.ts), which makes the C
struct's size and its `color` offset a real ABI between the runtime header
and hand-written IR. The stored VALUE is part of that ABI too: the emitted
store is the literal 0, not the enumerator, so SCR_CYC_BLACK's numeric value
is pinned as surely as the offsets are.
Nothing enforced any of it. llvm-runtime-abi.test.ts mechanically checks that
every `declare @scr_*` agrees with its C prototype, but that is the FUNCTION
ABI; a struct-offset skew is invisible to it, to the type system, and to the
C compiler. Narrowing `color` and putting another field in the freed bytes
compiles clean and silently zeroes that field on every retain; reordering the
color enum compiles clean and makes every retain write the wrong color.
Assert the four properties the emitted IR actually depends on, so the next
edit to this struct or that enum fails the build instead of miscompiling.
The three layout assertions bake in 64-bit pointers and size_t, which every
current target has. A 32-bit port would trip them — correctly: the backend's
hardcoded -16/-32 would need fixing too.
* runtime: collect cycles generationally
A trial-deletion pass costs O(objects reachable from its candidates), and a
candidate inside a large live structure reaches all of it. Collecting every
256 candidates therefore made total collection work quadratic in a growing
live heap: the V8 splay benchmark spent 749ms per iteration, ~99% of it in
the collector re-walking a tree that was entirely live and never garbage.
Make the collector generational, in the shape CPython uses for this same
algorithm. Each header carries a generation; a pass names the oldest one it
will walk and skips anything above it, exactly as it already skips NULL and
immortals; survivors are promoted so later nursery passes never re-walk
them. A pass then costs what was allocated since the last one rather than
the whole heap.
Restricting the walk stays sound because trial deletion is already
conservative in the safe direction: an edge from a skipped older object is
never trial-deleted, so its target keeps that count and reads as externally
referenced — the same answer the collector gives for a stack reference.
What it gives up is cycles that span generations, and the full pass (which
skips nothing, and so is exactly the old algorithm) is what finds those.
Three things that are load-bearing and were not obvious:
- A restricted pass may spare a candidate ONLY because the edge keeping it
alive came from a generation it refused to walk. The candidate buffer is
the collector's only root set and markRoots consumes the entry, so
dropping such a survivor retires the one record that the object is a
possible cycle root — and a dead cycle whose members were all spared that
way is then invisible to every later pass, full ones included. So a
restricted pass hands its survivors back as candidates. A full pass needs
none of this: it walked every edge, so rc > 0 there is a genuine outside
reference and Bacon-Rajan's own reasoning retires the candidate.
- Live data that is unlinked INTO a dead cycle grows no counter: those
objects were tallied when they were allocated, and they are not in the
nursery. A heap-growth trigger alone therefore never fires for a program
that churns its long-lived structures, so the mature candidate buffer's
own size is a second full-pass trigger.
- Restricting the walk leaves one edge accounted by NOBODY. markGray does
not trial-delete an edge into a skipped generation, and the teardown
contract has free_fn release exactly the UNTRACED children — because every
traced edge was supposed to have been decremented by markGray. So freeing
a white object that holds a traced edge into an older generation drops the
edge while the target keeps the count, and that phantom reference reads as
an external one to every later pass, full ones included: the target and
everything it links to would never be reclaimable again.
The white set pays those edges off explicitly, in two halves, because the
walk cannot do it inline — a release there could free a survivor that
scr_promote still points at, or cascade into the white set mid-teardown.
Each target is PINNED with a retain while `gen` still holds the values the
walk filtered on, and after the teardowns the pin is dropped and the edge
given up for real. Two decrements per entry, because the pin is one of
them; a single release would only undo the pin and leave the phantom edge
where it was. The pin is also what makes the second decrement safe, since
a teardown's releases of untraced children can reach a cycle-headered
object — that is how they re-buffer survivors. One entry per skipped edge,
so several edges sharing a target settle it once each. Nothing pinned can
be in the white set: a skipped older object's edges were never
trial-deleted, so anything it references was scan-blacked rather than
whitened, by the same conservatism above.
The event loop's between-turns collection now runs one scheduled pass
instead of scr_collect_cycles(); a full sweep per turn would walk the whole
live heap every turn, which is the cost the generations exist to avoid.
scr_collect_cycles() stays the exit / session-reset sweep, where the RC
audit runs straight after and wants nothing reclaimable left.
757-cycle-cross-generation.ts covers the third point: a live list mutated
until its nodes promote, then dead nursery cycles holding traced edges into
it. Unaccounted, the sanitized lane reports 60 objects live at exit — the
whole list, held by phantom counts.
Measured, best-of-7 against the previous collector:
splay 620.45ms -> 1.45ms (428x)
deltablue 1.25 -> 0.81 (1.55x)
richards 1.196 -> 1.162 (1.03x)
crypto/navier-stokes/raytrace/regexp unchanged
an LRU cache (200k entries, 200k ops) 32137ms -> 93ms (345x)
a large live cache + 500 event-loop turns 3095ms -> 776ms (4.0x)
The trade is space: floating cyclic garbage is now a bounded fraction of the
live heap instead of a fixed candidate count, so peak RSS on
cyclic-garbage-heavy programs runs ~1.4x the old collector's. Bounded and
flat in run length, not a leak — verified by holding a 200k-object live heap
while churning 200k dead cycles per round for 30 rounds.
Test lanes show byte-identical failure sets against the previous collector:
Linux sanitized 164, Linux plain 528, macOS differential corpus 66 — all
pre-existing on those machines, none introduced or fixed here.
* runtime: bound scheduled mature cycle backlog
* runtime: rearm cycle trigger after candidate deaths
- Lower the HTTPS options row through the runtime ABI with callback ownership and loop liveness.
- Pin HTTP and HTTPS response callbacks with differential and tier-regression coverage.
Co-authored-by: morler <478444+morler@users.noreply.github.com>
- Add static Array and Uint8Array copying, sorting, and join surfaces.
- Lower Uint8Array spread, destructuring, and iterator methods across C and LLVM.
- Pin Node 24 semantics with differential coverage and runtime size gating.
- Lower number-array and typed-array value iterators statically.
- Preserve stored iterator cursor and exhaustion state across loops.
- Add Node differential coverage for arrays and Uint8Array.
- Lower to the stable array sort helper over a shallow snapshot.
- Publish Array.prototype.toSorted in the static surface manifest.
- Cover stability, copy semantics, and evaluation order differentially.
- Lower asynchronous ESM initialization with Node-compatible dependency ordering and caching.
- Finish top-level promises consistently across the C and LLVM runtimes.
- Add differential coverage for imports, rejection, and unsettled evaluation.
- Lower Number.prototype.toFixed through the compiler IR and native backends.
- Match JavaScript rounding, range errors, and numeric edge cases in the runtime.
- Add differential coverage and update surface metadata and diagnostics.
- `http.get("http://host/path")` and its https spelling now compile. The declarations only described the options-record form, so the first thing a from-scratch client writes was a type error — the shape worked under --dynamic, where the engine does not consult our declarations, which is how it read as a hello-world failure.
- A URL object is accepted the same way, as its href through the same parse. An unparsable input is the WHATWG "Invalid URL" TypeError and a scheme that is not the calling module's is ERR_INVALID_PROTOCOL, both catchable, matching Node rather than silently upgrading the dial.
- `request` and `get` take one signature with a union first parameter instead of Node's two overloads: `tls ? https.request : http.request` unions two function types, and resolving a call on that union collapses each side to a single signature, so an overload set misresolves in either declaration order. The lowering dispatches on the argument's own type.
- Both rows run on the LLVM backend, and the tls CA-store entries joined it too — the omission there was stale, so 2557-tls-ca-store moved off the C lane with identical output.
- Corpus 2644 and 2645 cover both schemes against loopback servers, so neither needs the network.
- tsc builds `u || d`'s result by DROPPING the left's falsy arms (`process.env.X || null` is `string | null`, `|| 3000` is `string | number`), so coercing the left into that result before the truthiness test retags an arm the test is about to rule out — and threw where Node yields the default.
- `orDefault` gains an optional `retag`: the truthy side hands the whole left box to a union→union helper whose stranded-arm throw is unreachable by construction, since truthiness has already ruled those arms out. Left evaluates once, right stays lazy, arm count is free.
- A ternary in logical-left position no longer adopts its contextual union over its own, for the same reason the eager coercion was wrong: tsc's assignability proof does not hold for that operand. Narrow to the union choice — contextual array element typing and the unmappable-own rescue are untouched.
- 2595 chained its two in-flight `fs.exists` stats: registering both after the awaited rejections orders them against the rejections, not against each other, and Node itself returns them reversed about one run in forty.
- Island values held in `unknown` arm the routed operation set (SCR_DYN_JSVAL): keyed reads and writes, calls and method dispatch, Object statics, and `??` all run in the engine, so a wrapped value answers exactly what Node answers instead of meeting a fence.
- Scalars normalize at the wrap, composites stay engine values by reference, and the round trip preserves identity — a value that crosses into the checked-dynamic tree and back is the same engine value.
- deepStrictEqual over a mixed engine-vs-native pair fails with the reason stated rather than guessing at an answer.
- Corpus 2582–2585 and an npm routed-row fixture pin the behavior; the dyncheck expectations move with it.
- The retired "DOM" nickname is swept out of the sources and corpus in the same pass (29 comment sites) — the world is the checked-dynamic tree, its values are dyn values.
- The crypto introspection statics (getFips and the cipher/hash/curve name lists) bake at the call site, and the crypto constant table joins http2's under a generalized per-module baked-constants lowering.
- The CA-store introspection surface lands in its own unit (scr_tls_ca.c) behind its own link gate, so a getCACertificates-only binary never pulls mbedTLS: getCACertificates, rootCertificates, setDefaultCACertificates.
- http2.createSecureServer grows the eager request handler and the runtime options record, riding the same server-options walk as tls.createServer; string-typed 'data' listeners decode.
- process once/off for 'unhandledRejection' plus the sibling 'rejectionHandled' registry, removal by closure identity per the warning-registry stance.
- Merge resolution unions this lane with main's fs ladders, createRequire erasure, and dyn secure-context work; the runtime's retired "DOM" nickname is spelled "dyn" throughout.
- Add Agent connection accounting and HTTP/socket compatibility surfaces
- Support dynamic Buffer at and slice access
- Pin behavior with differential corpus coverage
- The written-binding decline yields when the declared type has no mapping or the write's RHS roots at an ambient-undefined name
- A class extending a collection-poisoned base flushes the base's diagnostics eagerly: the heritage evaluates at module init, so deferring the compiler's own fence manufactured a silent wrong output
- Both pipeline lowerings catch the arg-type marker the finished paths already caught; provably-invalid stages throw Node's synchronous ERR_INVALID_ARG_TYPE in every position
- Accepted-but-unlowered sources fence naming what Node accepts; finished(null) joins the isNodeStream ladder
- for-of, destructuring, and spread over engine-held dyn values ride the engine's own iterator protocol with V8's exact non-iterable spellings; for-of over dyn lowers generally
- Static promises cross into the engine as thenables; Promise.all over mixed and non-literal entries runs the engine's own combinator; engine-array fulfillments exit by reference
- The re-minted silent-wrong-answer path is closed: island-literal members whose values poison fence at the read with the captured diagnostic
- The checked-dynamic tree replaces the old nickname across comments and internal docs
- Node serves the builtin for both spellings — a builtin wins over a same-named npm package for the bare specifier — so both key the same lowering tables
- The stale prefix-only caution predated the createRequire erasure; the hint's claim and the behavior now agree
- trapDeclRootOf's written-binding decline now requires a mappable declared type: ordinary storage only buys anything when it compiles — an unmappable recursive call shape traded the compiling no-storage trap for a guaranteed SC2009 on a value that never exists
- bindingEverWritten skips statement-position writes whose RHS chain roots at an ambient-undefined name: that statement IS the RHS root's throw (lowerExprStatement's first claim) and never touches the target's storage, so it cannot justify the decline
- A poisoned base a top-level class extends flushes its deferred diagnostics eagerly (collectClassShape): the heritage evaluates at module init and the base's fence is the compiler's own — Node defines the base fine — so the deferred trap compiled a binary that refused at startup where Node runs; leaf and derived-only poisoned classes keep the JS deferral
- Fixtures: corpus 2632 (unmappable written trap binding) and 2633 (ambient-rooted rewrite), diagnostics js-extends-poisoned-base (the eager report names the base's real blocker), errors.test pins for both deferral edges that must NOT report eagerly; order-parity baselines recorded for the new entries
- lowerStreamArg's arg-type marker escaped both pipeline lowerings uncaught (only finished caught it), so a compile-time non-stream stage — pipeline('', dst, cb) — crashed the CLI with a raw StreamArgTypeThrow; both pipeline paths now catch and answer it as the call's value, matching finished
- The ladder splits by API to match Node exactly: finished keeps the isNodeStream clause (null now included), while pipeline stages render makeAsyncIterable's "body" clause — the same text in source, middle, and destination positions — for numbers, booleans, null, and pair-less records
- Shapes Node accepts but scriptc can't lower yet never take a throw Node never throws: strings and arrays (iterables) and records carrying a readable/writable member (the duplex-pair form) keep the pointed fence, whose hint now names those inputs
- tests/corpus/2632-stream-pipeline-arg-ladders.cjs pins the ladder in all three positions plus finished(null) and a live pipe, byte-identical through both backends in the plain and sanitized lanes
- text, json, and buffer settle real promises over the readable machinery with Node's settle timing and rejection set; arrayBuffer and blob fence with their representation reasons
- createRequire over this file's URL is compile-time plumbing: builtin specs become namespace imports, relative JSON bakes as validated documents, npm packages ride the require-condition graph, unresolvable names throw Node's catchable MODULE_NOT_FOUND
- module.builtinModules bakes as the compat target's exact list
# Conflicts:
# packages/compiler/src/frontend/lowering/lower-modules.ts
# packages/compiler/src/frontend/lowering/lowerer.ts
- Buffer slice and subarray become true aliasing views over the retained owner — mutating through a view was silently computing wrong bytes
- Each stream tick posts a one-to-one marker on the user tick queue, so stream emissions interleave with process.nextTick in enqueue order, Node's order by construction; the station remains for teardown
- Stream emitters model the pre-created event-key shape governing eventNames order; push honors defaultEncoding with Node's unknown-encoding errors
# Conflicts:
# packages/compiler/test/ts7/baselines/order-parity.json
- The forwarding shape — emit(event, ...args) whose rest appears only as super.emit(event, ...args) — lowers per dispatching event as a real hierarchy method, vtable-participating
- error events keep the throw-when-unhandled path through overrides; super.emit resolves to the nearest override ancestor, JS's chain
- Receivers above every override route through interned instanceOf dispatch; other emitter members and stream-rooted classes keep their fences with reasons stated
# Conflicts:
# packages/compiler/test/ts7/baselines/order-parity.json
- The ambient-trap probe resolves as a side-effect-free probe: deferred collection diagnostics stay deferred, and the merged-namespace fence lands as a diagnostic instead of crashing the collect walk
- deadUnmappableBinding and the nullish family claim statement-position declarators only; for-init heads and catch bindings stay locals
- Trap classification declines written bindings; the seven 0.0.10 pass-to-fail fences restore byte-identical to 0.0.9 behavior
- ambientUndefVarRootOf resolves as a side-effect-free PROBE (the collect-phase guard): bare resolution flushed reached declarations' deferred collection diagnostics onto the build — failing three JS programs at compile time where the JS-input design defers fences to runtime (checkJsFiles_noErrorLocation, multipleDeclarations, jsdocTemplateClass), fencing tsxDefaultImports' enum-carrying static field through an import alias, and double-reporting the checker-panic fixture — and threw the cross-block merged-namespace fence's PoisonError out of every probe site
- Trap classification declines WRITTEN bindings (trapDeclRootOf): the no-storage trap fenced later assignments ("assignment to 't1' (not a writable local or module global)", literalWideningWithCompoundLikeAssignments) where ordinary storage compiles them; the initializer read still lowers to the root's ReferenceError, so the writes never run either way
- The nullish no-storage family claims only the EMPTY interned record shape: a field-bearing record binding (`const value: { inner: number | string } = null as any`) keeps real storage — its comma-chain and call-argument reads have no no-storage lowering (narrowCommaOperatorNestedWithinLHS, varianceProblingAndZeroOrderIndexSignatureRelationsAlign)
- classFieldSuperAccessibleJs2 reverts with the shared mechanism to its 0.0.9 state: the build passes and the binary still refuses at runtime with the same SC1090 (the honest JS runtime fence, not silent wrong output)
- Fixtures: corpus 2609 (trap binding with later writes), 2610 (field-bearing nullish record), 2611 (JS deferred-collection probe), 2612 (enum static field through a default import); checker-panic-infinity snapshot returns to its single pre-0.0.10 report
- The loadPlugins shape end-to-end: Promise.all(plugins.map((p) => loadPlugin(p))) — a NON-literal argument the checker spells any[] — lowers to the ENGINE's own Promise.all over the lifted array, and the INFERRED Promise<any[]> return coerces through the island→static promise bridge with the new SCR_ISLP_JSVAL_ARR settle payload: the fulfillment exits Array.isArray-gated BY REFERENCE at the settle, a lying fulfillment rejecting with the boundary TypeError (trust-but-verify at the settle).
- coerceToExpected's jsval branch gains the promise-slot arm generally: island values flowing into Promise<void>/Promise<any>/Promise<any[]>-typed slots bridge instead of fencing (the passing-a-package-promise-into-promise-typed-parameters fence retires for these inners); other inners keep the named fence.
- Corpus 2605 extended with the non-literal all + inferred-return rows, byte-identical to Node under both backends and SAN-clean; dyncheck/errors/island suites green.
- collectGlobals' trap/nullish/dead classification probes catch PoisonError per the per-probe recovery discipline: the cross-block merged-namespace SC1090 fence reached through the ambient-root walk recorded its diagnostic and then crashed the CLI mid-discovery (5 merged-namespace corpus tests, ice bucket)
- On a poisoned probe, JS files defer the recorded diagnostics to runtime fences (deferJsCollectionDiags) and the binding falls through to ordinary registration; the statement lowering re-fences at its own site
- The no-storage binding families (nullish, dead-unmappable) only claim statement-position declarators: an uninitialized for-init let (`for (let x; false;) {}`) was claimed as dead-unmappable, violating lowerVarDeclList's contract that a for-init declarator always lowers ("lowerer bug: for-init declarator resolved to a global", 2 corpus tests)
- Fixtures: corpus 2608 (for-init heads with the masking variants), diagnostics merged-namespace-collect-probe (the collection-reached fence stays an honest SC1090)
- jsvalIn (and the island-literal member path behind it) accepts STATIC promises whose fulfillment is in the reverse bridge's payload domain — void/number/boolean/string/'any', plus the NEW Promise<any[]> payload (SCR_ISLP_JSVAL_ARR: a native array of engine cells re-enters as a fresh engine array over the SAME engine values). scr_jsval_from_promise emits from jsMarshal in both backends; the IR validator admits the arm.
- Promise.all with MIXED entries where some entry is an island value (the withPlugins shape: Promise.all([loadBuiltinPlugins(), loadPlugins(plugins)])) lowers to the ENGINE's own Promise.all — island entries pass through, static promises bridge as thenables, and the combined result stays an island value the static side awaits through the existing island→static bridge. All-static tuples keep the typed fence hint.
- Corpus 2605 (unawaited static promise in an 'any' slot answers typeof "object" and awaits engine-side; the mixed-all + .flat() chain) is byte-identical to Node under both backends; SAN clean. Composite fulfillments (Promise<{...}[]>) stay fenced with the promise type named — the payload-domain residue.
- The ratified rename rider (world-unification §7 lane 5): "DOM" collided with the browser DOM and, post-lane-1, the structure is no longer a document tree. Comments and doc prose now say "the checked-dynamic tree" for the noun and "dyn" adjectivally; runtime symbol names (scr_dyn_*), the error.domClone libCall, and the real web DOMException family are unchanged.
- The few user-facing strings carrying the nickname move with it (the structuredClone cyclic message, the dc subscriber/publish fences, two emitter ICE texts); diagnostics snapshots regenerated (96 pass). One private compiler constant (DOM_PROTO_METHOD_NAMES -> DYN_PROTO_METHOD_NAMES) rides along.
- Mechanical sweep, 142 files, comments/prose only — no behavior change.
- The iteration pack's JSVAL arm is ARMED (scr_dyn_iter_pack, scr_dyn_arr_push_spread, scr_dyn_pack_push_spread_iter route through the new iter_drain ops row): the ENGINE's iterator protocol drains into a fresh DOM array — engine arrays, generators, Maps, Sets, and Symbol.iterator implementations step exactly as Node runs them, elements wrapping back scalar-normalized.
- for-of over checked-dynamic values LOWERS now (both backends): the source packs once through dyn.iterPack and a hidden index loop binds each element as a dyn value (new dyn.arrLen/dyn.arrAt accessors) — an eager snapshot, the matchAll stance. Pattern heads, pre-declared and var heads, and labeled break/continue all ride the desugar. Non-iterables throw V8's for-of CallPrinter spellings, threaded into the engine-side guard.
- The validated exit gains the jsval-element-array arm (scr_jsval_exit_jsval_arr, both backends): an 'any' value exits into 'any[]'- and 'any[] | undefined'-declared slots Array.isArray-gated, elements BY REFERENCE (identity crosses, the spine is a snapshot) — the withPlugins loadPlugins(plugins) boundary, the driver frontier's first named poison.
- Corpus 2604 and npm case jsval-iterate (package-minted Set/Map/generator/Symbol.iterator) are byte-identical to Node under both backends; dyncheck's jsval-iterate pin flips from the retired island fence to the armed answers; SAN clean over the drain/wrap/exit churn.
Chris Tate