Commit Graph
252 Commits
Author SHA1 Message Date
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 47904df3c2 Decode short non-ASCII strings directly in charCodeAt
charCodeAt on a non-ASCII string mapped the UTF-16 index to a byte offset
through a string index entry, which the first read claimed and the
string's release purged. For a string of one character, such as each
character of a for-of loop that codePointAt(0) classifies, the entry cost
more than the read.

A non-ASCII receiver of up to 16 bytes is now decoded from its start on
each read, as scr_str_utf16_len already counts short receivers. Longer
receivers keep the index entry and its cursor.
2026-10-27 01:07:00 -05:00
Chris Tate df77671e33 Merge main 2026-10-26 23:13:34 -05:00
Chris Tate 48ccee523e Allocate template literals at their exact length
A template literal of strings and numbers reserved 32 bytes for each
number, the longest number text, and kept that capacity for the result's
lifetime: an 8-byte name built from two numbers used a 96-byte block. The
numbers now format into a stack buffer first, so the result is allocated
once at its exact length.
2026-10-26 21:01:36 -05:00
Chris Tate 3c83fcb89c Allocate object groups that die together in arenas (#2348) 2026-10-26 16:54:31 -05:00
Chris Tate c5e8a876ec Store ordinary objects as a layout plus value slots (#2353) 2026-10-26 16:18:38 -05:00
Chris Tate f12b2b041d Cheaper trial deletion in the cycle collector 2026-10-26 14:34:49 -05:00
Chris Tate 9c7d552fdc Link the arena runtime into the runtime C tests 2026-10-26 13:26:29 -05:00
Malte UblandClaude Opus 5.5 dc1e8acc1d Store ordinary objects as a layout plus value slots
A checked object (SCR_DYN_OBJ) now starts in layout mode: v.obj.layout names
its key sequence, a node of the thread's interned transition tree, and
v.obj.slots holds one 8-byte value per key, where every property used to take
a 48-byte entry. All properties of a layout-mode object are writable,
enumerable, configurable data properties. An accessor, a non-default
attribute, a delete, more than 64 keys, spent layout budgets, publication to
other threads, or any code that reads entries directly moves the object to
dictionary mode (today's entries) for good, so unported paths stay correct.

Layouts replace the dynamic tier's lazily computed hidden classes (ScrJsShape)
and keep their budgets. Inline caches key on the layout pointer and read the
slot directly; JSON.parse builds objects along layouts; stringify, key
enumeration, release, trace and the runtime read cache read slots in place.
The emitted index-signature checks read members through runtime accessors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 09:18:53 -07:00
Malte UblandClaude Opus 5.5 7ac0542aa9 Read ordinary objects' properties through accessors outside the object core
Runtime units outside the object core (scr_json.c, scr_js_object.c) read and
adjusted SCR_DYN_OBJ entries in place: v.obj.len, v.obj.entries[i] and the
48-byte ScrDynEntry fields. The single object model's generic objects
(stage S3) change that storage, so these units now go through own-property
accessors in scr_dyn.h:

- scr_dyn_obj_count, scr_dyn_obj_key_at, scr_dyn_obj_value_at and
  scr_dyn_obj_prop_at (a ScrDynProp view: key, value, getter, setter and
  attributes) walk the properties in creation order;
- scr_dyn_obj_index_of finds one by key;
- scr_dyn_obj_set_attrs_at restricts a member a builtin object just added;
- scr_dyn_own_symbol_prop and scr_dyn_copy_prop_at wrap the symbol-entry
  and copy primitives.

The accessors are inline over today's entries, so behavior is unchanged.
21 units move (inspect, assert, util compare, console, http, http2, fetch,
tls, fs, os, lib, array, message, publish, region, symbol, worker events,
handle properties, builtin values, declaration adoption). A package test
keeps entry storage reads inside the core.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 05:39:34 -07:00
Chris Tate c3b265f68c Merge main 2026-10-26 04:39:20 -05:00
Chris Tate feb4501013 Keep a candidate's generation when a limited cycle pass reaches it
A limited pass (scr_cyc_collect_set, which freeing a LifetimeRegion runs
over the objects the region owned) takes only the candidates in its set.
Its walk can still reach other candidates, and one that is referenced from
outside is blackened by scan and promoted to the next generation while it
keeps its slot in its old generation's buffer. scr_cyc_on_dead finds a
candidate's buffer by its generation, so when that object died it removed
another buffer's entry and left its own slot pointing at the freed block,
which the next pass then read.

Such a candidate is no longer promoted: it keeps the generation of the
buffer it is in until a pass takes it.
2026-10-26 00:24:05 -05:00
Chris Tate 307f9e0009 Keep one heap-trim test block per four pages of any size
The test kept one live 48-byte block every 1,000 allocations, which on a
system with 64 KiB pages leaves a live block on every page, so no page held
only free blocks and the resident-memory check failed. It now keeps one
block per four pages of blocks, from the system's page size.
2026-10-25 22:47:07 -05:00
Chris Tate 5b672087c6 Return the freed heap pages to the system after the frontend is disposed
The native compiler's frontend-dispose sweep frees about 450 MB on a cold
tsc-ts build, but the size-class allocator keeps the freed blocks on their
classes' free lists, which only later blocks of the same size reuse. Code
generation allocates other sizes, so its pages stacked on top of the freed
heap and set the build's peak resident memory.

- scr_heap_trim (scr_alloc.c) returns the pages of the calling thread's
  size-class slices that hold only free blocks (MADV_DONTNEED) and the
  system allocator's free pages (malloc_trim). In the single-threaded
  allocator the blocks of a returned page leave the free list and the
  class carves past its frontier instead; the remaining free blocks are
  relinked in address order. Worker runtimes return the calling thread's
  arena pages as a parking thread does.
- scr_collect_cycles_phase, the full sweep after a phase's data died,
  calls it; the native compiler calls that sweep when it disposes the
  frontend.
- test_heap_trim.c checks that the pages go back, that the blocks still
  allocated keep their contents, and that new blocks overlap none of
  them, with and without worker arenas (another thread allocating and
  freeing remotely throughout).

Cold tsc-ts build, 12 interleaved rounds: peak RSS 1407 -> 1247 MB
(-160 MB, -11.4%, every round); wall 87.1 -> 85.9 s (median).
2026-10-25 22:36:11 -05:00
Chris Tate cbe4a9387b Merge main 2026-10-25 15:24:25 -05:00
Chris Tate cab0968510 Keep holes and values written through unknown[] views in typed arrays (#1866) 2026-10-25 14:01:03 -05:00
Chris Tate 90605110a5 Adapt the nursery trigger to where cyclic garbage dies (#2231) 2026-10-25 14:00:41 -05:00
Chris Tate e8a64267cb Call toJSON on foreign elements and merge main 2026-10-25 13:13:02 -05:00
Chris Tate e24ed12933 Decode string storage within its length in Buffer encodings
String storage can hold bytes that are not well-formed UTF-8: an HTTP
header value keeps the raw bytes a client sent. Encoding such a string
read past its end.

- scr_bytes_next_cp decoded any high byte as the lead of a sequence and
  read up to three bytes after it without a bound. It now takes the
  storage length, decodes only complete, well-formed sequences, and
  returns any other byte as U+0000..U+00FF, advancing by one. That is
  each such byte's value in Node, which decodes header values as latin1.
  This covers latin1, ascii, binary and utf16le encoding, hex and base64
  of non-ASCII text (which narrow through latin1), Buffer#write and
  TextEncoder#encodeInto.
- base64 decoding read one byte past its input when the eight-character
  groups ended exactly at the end.

The runtime's bytes unit test (built with ASan) encodes ill-formed
storage at every offset within a word and random high bytes, and
compares latin1, ascii and utf16le with a scalar reference. A corpus
program sends such header bytes to a server and encodes them as Node
does.
2026-10-25 12:48:21 -05:00
Chris Tate b0c01dc341 Adapt the nursery trigger to where cyclic garbage dies
A program's nursery trigger starts at 256 candidates and doubles, up to
4096, after each scheduled mature or full pass whose dead set held more
mature and old objects than an eighth of what the nursery passes freed
since the previous one; it halves after one that held less than a
thirty-second of that. SCR_CYCLE_THRESHOLD and
scr_cyc_set_nursery_threshold pin the trigger as before. The
scheduled-age rule keeps counting collection points in the pinned or
default trigger.
2026-10-25 12:48:07 -05:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate 63a99cc4d9 Convert a loop's string receiver to UTF-16 at once only when the loop can read enough of it
A versioned loop over a non-ASCII string passes the range of indexes it
reads: a counted loop's start and limit, or a while loop's counter and
bound. The runtime converts the string at once when that range has at
least one unit per 16 bytes of it, and otherwise counts the loop as one
indexed read, so a loop over a line of a long source text no longer
keeps a UTF-16 copy of all of it. The publish call is weighted as
unlikely, since loops re-entered over one string skip it.
2026-10-25 07:37:50 -05:00
Chris Tate b3c98139e4 Move small number-key maps to the direct index from the second key
A number-keyed Map or Set with keys the direct index allows now moves to it
at its second entry rather than its fifth, so a map over a few small
integer keys (a histogram over 0..3) reads through the inline direct index
instead of the out-of-line linear scan.
2026-10-25 05:48:40 -05:00
Chris Tate 7141af5cc9 Merge main 2026-10-25 00:07:23 -05:00
Chris Tate 37031fa7c2 Convert a loop's string receiver to UTF-16 at once, and publish it only when the scan holds another string (#2180) 2026-10-24 23:34:50 -05:00
Chris Tate 6a29947998 Run the native compiler's nursery passes every 1024 candidates
The native compiler now starts with the cycle collector's nursery trigger
at 1024 new candidates instead of 256. scr_cyc_set_nursery_threshold sets
the trigger that SCR_CYCLE_THRESHOLD sets, on the pattern of
scr_cyc_set_growth_cap; the environment variable still takes precedence.

The compiler sets it from a constructor in its host object, before its
modules initialize. Set from main, the trigger leaves the build's
collection work where it was: the cost depends on the collector's first
passes. Programs keep the default trigger.

A cold tsc-ts build takes 4% less wall time (12 paired runs) and its peak
RSS goes from 1.41 GB to 1.31 GB. The program's IR is unchanged.
2026-10-24 22:59:02 -05:00
Chris Tate add5f4869b Convert a loop's string receiver to UTF-16 at once, and publish it only when the scan holds another string 2026-10-24 22:53:15 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate 9d08b301fc Merge main 2026-10-24 17:50:19 -05:00
Chris Tate 12e16cdb32 Find weak collection entries through hashed indexes (#2004) 2026-10-24 16:40:37 -05:00
Chris Tate 190fb8d58e Mix weak-collection hashes in 64 bits on 32-bit targets
uintptr_t is 32 bits on wasm32, where the weak-collection hashes shifted it by 33 and 56 bits. The hashes now widen keys, owners and kinds to uint64_t before mixing; 64-bit targets compute the same values as before.
2026-10-24 16:01:33 -05:00
Chris Tate 02c2c89fd6 Merge main 2026-10-24 14:18:51 -05:00
Malte UblandClaude Opus 5.5 d037c66e8c Report JSON.parse syntax errors with V8's messages and positions
JSON.parse's SyntaxError messages were shaped like V8's but differed from
Node's in several ways:
- Every positional message now ends with V8's "(line L column C)", where
  \n, \r and \r\n end lines.
- Positions are UTF-16 offsets, as in Node, not byte offsets.
- An object or array cut off after '{', a property name, a value or an
  array element reports the token V8 expected ("Expected property name or
  '}'", "Expected ':' after property name", "Expected ',' or '}' after
  property value", "Expected ',' or ']' after array element") instead of
  "Unexpected end of JSON input".
- After a comma in an object the message is "Expected double-quoted
  property name".
- An unterminated string reports the end of the input, and a truncated
  \u escape is a "Bad Unicode escape" there.
- A leading zero followed by a digit is an "Unexpected number".
- A misspelled literal names the first differing character, or the end of
  the input.
- An unexpected character is named with the source as V8 does: whole when
  it is under 21 units, otherwise ten units of context on each side; the
  sources "NaN", "Infinity", "undefined" and "[object Object]" name only
  themselves; a quote or a number there is an "Unexpected string" or
  "Unexpected number".

The new corpus program 7993-json-parse-error-messages pins 76 inputs
against Node, including non-ASCII text before the error, line breaks, and
long sources with split surrogate pairs at the context edges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:55:05 -07:00
Chris Tate 73389fd513 Merge main 2026-10-24 06:53:30 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate f986d76d3b Bound Brotli parameter names in error messages 2026-10-23 22:57:22 -05:00
Chris Tate 0ec4fb6de6 Merge main 2026-10-23 14:35:27 -05:00
Chris Tate fe452bdb03 Merge main and harden the Node island loader 2026-10-23 11:04:36 -05:00
Malte Ubl 2494e7c1dc Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/src/backend/native/executable.ts
#	packages/compiler/src/frontend/lowering/lower-calls.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 05:53:42 -07:00
Chris Tate 029845e9c0 Keep values written through unknown[] views in typed arrays 2026-10-23 04:07:06 -05:00
Malte UblandClaude Opus 5.5 342cf3a063 Copy class-view property bags entry by entry
Materializing the checked view of a JavaScript class instance copies its
property bag through scr_dyn_copy_property_descriptors, which built a key
string and a descriptor object per property and defined each through
scr_dyn_define_property (about 2k instructions per property). Plain objects
on both sides now copy each entry directly with the attributes, value or
accessor pair and own-key order the definition gives; properties the target
already has, non-extensible targets and other receivers keep the general
path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:29:22 -07:00
Chris Tate fe406429eb Keep int32 slots for long number arrays with checked integer stores
A local number[] candidate whose stores are exact integers but not all
proven int32s (a distance array updated with dist[v] = dist[u]! + 1), and
that is only ever allocated by new Array(n).fill(v), now holds SCR_ELEM_I32
slots when n is at least 4096. A shorter fill creates doubles, which fit a
typical L1 data cache. The first store of a value that is not an int32
widens the array to doubles in place (scr_arr_widen_i32).

An innermost loop that accesses such arrays tests their kinds once on
entry (backend/llvm/int32-kind-loops.ts). Its fast copy reads and writes
them as int32 arrays; a store that would widen one leaves for a copy that
re-executes it with kind tests and finishes the loop on doubles. When the
arrays already hold doubles, a copy accesses them as double arrays. Other
accesses test the kind each time.

Only these arrays take kind tests: every other array, and every program
without one, emits the same code as before.
2026-10-22 21:37:12 -05:00
Chris Tate 7b79d73a55 Store local number arrays of proven int32s in 4-byte slots
A local number[] that only compiled element operations reach, and whose
every stored element the int32 slot analysis proves an int32 (never -0),
is now stored as SCR_ELEM_I32: 4-byte int32 slots, with state bytes once
it holds a hole or undefined, as for every array. Element reads give
consumers an exact int32, so indexing with an element skips the double
round trip, and the slots take half the memory.

The runtime's dense slot accessors convert between the int32 slot and the
double, so its double-valued entry points serve int32 arrays unchanged;
growth, copies, fill and length changes use the 4-byte width.

ir/int32-arrays.ts proposes the candidates and inlines their element-reader
helper calls; the int32 slot analysis keeps the proven ones, and
selectInt32Arrays rewrites the module for exactly those, so a program
without an int32 array emits the same LLVM IR as before.
2026-10-22 20:52:38 -05:00
Chris Tate d4a7b82dd8 Free arrays whose elements need no release without a teardown
When the last reference to an array dropped, the runtime ran the generic
teardown: scr_rc_destroy's depth bookkeeping, an indirect call to the array
destroy, and a loop over the element slots. Arrays of numbers, booleans and
compact ProgramLifetime references release nothing, and neither do arrays of
other ProgramLifetime instances, whose release does nothing.

- scr_arr_release frees an acyclic array without an ext directly when its
  elements are numbers, booleans or compact references, or its descriptor
  has SCR_ARR_OPS_RELEASE_NOTHING. Every other array keeps the full
  teardown.
- The backend sets SCR_ARR_OPS_RELEASE_NOTHING in the descriptor's flags
  word on 64-bit targets for ProgramLifetime element types that are not
  compact, plain or as the pointer arm of a nullable union, in programs
  without lifetime regions. Registration copies it into a table indexed by
  the descriptor's registry id before the id is published. Descriptors
  without the flag read 0 and keep the full teardown.

Corpus 7401 drops arrays of ProgramLifetime instances, numbers and booleans
through every array operation, with holes and far indices, next to arrays of
ordinary objects, and matches Node.
2026-10-22 19:22:55 -05:00
Chris Tate 6fe5612e48 Merge main 2026-10-22 18:05:53 -05:00
Chris Tate 36db0613cc Bound the compiler's cyclic garbage to twice its live heap (#1863) 2026-10-22 16:46:32 -05:00
Chris Tate ee865bcb0d Support the legacy createCipheriv modes and crypto.checkPrime/generatePrime (#1832) 2026-10-22 16:32:12 -05:00
Malte Ubl 13b00dd78b Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/frontend/lowering/lower-modules.ts
#	tests/harness/differential.test.ts
#	tests/harness/linux-differential.test.ts
#	tests/harness/llvm-differential.test.ts
#	tests/harness/windows-differential.test.ts
2026-10-22 14:10:07 -07:00
Chris Tate f5a3faf18d Bound the compiler's cyclic garbage to twice its live heap
The cycle collector widens the growth fraction that schedules full passes
up to 16 times the heap left by the last pass while passes find little.
Lowering a large program leaves hundreds of megabytes of cyclic garbage,
and whether a scheduled pass reclaims it before the build's peak depends
on where the growth rule fires: small changes to the compiler moved the
peak of a tsc-ts build between 1.66 and 2.05 GB.

The native compiler now sets the growth cap to 2 at startup, as
SCR_CYCLE_GROWTH_CAP=2 does, so a full pass runs once the heap has grown
to three times its size after the last one. scr_cyc_set_growth_cap sets
the cap for a program from its own code; the environment variable still
takes precedence, and child processes are not affected.
2026-10-22 14:25:55 -05:00
Chris Tate d83c676705 Store compact values of string-key maps in 4-byte slots on Linux 2026-10-22 14:08:00 -05:00