Counting array appends as edge-preserving makes `keep(item, items) { items.push(item); return item; }` a borrowed-return function: it preserves every heap edge, borrows every reference parameter and returns one of them. Its borrowing body now retains only the appended element and returns the parameter +0, and the owned adapter retains the result before releasing its parameters. An append never releases an element (the slow path stores past the old length), so the caller's owner keeps the result alive.
These PRs were merged into their stack base branches instead of main. This merges perf/alloc-hugepages (with the CI fixes below) into main. Conflicts: order-parity.json (union of entries); scr_alloc.c/scr_alloc.test.ts, where main already has the worker exit trim (#858, adapted to the pre-hugepages allocator): resolved by taking the hugepages allocator and re-applying the original trim commits written on top of it (ea88b87c + 4816696d, clean apply on an identical base).
Since #837 a type is traced only if it lies in a cycle-capable component or reaches an intrinsic reference; a type that merely holds a cycle cannot be reached back from it. In both backend-analysis cases `record:outer` only holds `record:cycle`, so the Node oracle and the native compiler agree on ["record:cycle"] and the expectations still listed `record:outer`.
Brings in cc7c6cb7 (inline truncation decided for the host when no triple is given), which this branch predates; without it the native bootstrap's LLVM comparison with the Node seed fails on x86-64 Linux.
Brings in cc7c6cb7 (inline truncation decided for the host when no triple is given), which this branch predates; without it the native bootstrap's LLVM comparison with the Node seed fails on x86-64 Linux.
Conflicts, all resolved to this branch's side:
- emitter.ts (numeric-table get, map index probe), expr-bytes.ts, local-array-reads.ts, string-reads.ts: the base only respelled the size-to-double conversions in the old range checks (uitofp nneg, now sitofp); exactIndexLines replaces those checks entirely. The base's other hunks in these files (host inline truncation, the remaining sitofp conversions) merge as they are.
- int32-slots.ts: #837 and #843 both extracted dynamicFieldStores with identical bodies; keep this branch's signature, whose optional records/unions maps also serve #837's single-argument callers.
The reference-write analysis borrows an argument when neither the callee nor a later argument writes a location it depends on. A later argument that only throws a fresh Error writes no array element, so `read(items[0], fail())` now borrows the element from the borrowed parameter's array and has nothing to release on any edge; the test pins that. The exception-edge release is now exercised by a later argument that pops the array before throwing, which still retains the snapshot and releases it before the exceptional return.
Two expectations predate the reference-write analysis and return moves on this branch. `keep(item, items) { items.push(item); return item; }` now qualifies as a borrowed-return function: its write set (array elements) misses the walked parameter, so the body retains only the appended element, returns the parameter +0 while the caller still owns it, and the owned adapter retains the result before releasing its parameters. In `snapshot`, `return saved` moves the saved alias's owned scope entry into the result rather than retaining it again and releasing the local at scope exit, leaving one retain.
The `nneg` flag on `uitofp` is LLVM 19 syntax. Sanitized builds hand the textual module to the host clang, and Ubuntu 24.04's clang 18 rejects it ("expected type"), which failed every Linux sanitized Effect shard. These operands are size_t values below 2^63, so `sitofp` converts them identically, is accepted by every supported LLVM, and still lowers to one cvtsi2sd on x86-64 (LLVM 19+ canonicalizes it to `uitofp nneg` where it can prove the sign).
Since #837 a type is traced only if it lies in a cycle-capable component or reaches an intrinsic reference; a type that merely holds a cycle cannot be reached back from it. In both backend-analysis cases `record:outer` only holds `record:cycle`, so the Node oracle and the native compiler agree on ["record:cycle"] and the expectations still listed `record:outer`.
Brings in cc7c6cb7 (inline truncation decided for the host when no triple is given), which this branch predates; without it the native bootstrap's LLVM comparison with the Node seed fails on x86-64 Linux.
The custom Dockerfile.sandbox image builds the LLVM helper against the same LLVM 22 CMake exports as the managed bootstrap, which reference the Polly libraries; without libpolly-22-dev CMake configure fails.
Three workers each hold and drop about 80 MB of small objects and exit;
three more then allocate as much again in the arenas the first ones
returned (on Linux the first ones hand those pages back when they exit).
Records the fixture's preflight/order baseline.
A script thread that exits parks its allocator arena with every freed
block still resident, so the memory of a finished worker stays with the
process until another thread claims the arena. On Linux, a parking thread
that carved at least 16 MiB now hands back every page of its arena that
holds only free blocks (MADV_DONTNEED), relinks the remaining free blocks
in address order, and keeps the returned page runs as zeroed regions that
the next owner carves before the slice frontier. It also trims glibc
malloc. SCR_SA_STATS=1 reports what each park returned; SCR_SA_TRIM=0
turns it off.
Every worker mailbox is a pipe. Once a process passes 64 descriptors the kernel grows its descriptor table, and while other threads share the table expand_fdtable first waits a full RCU grace period. tsc-ts on a 16-vCPU host stalled 11-25 ms on the thread starting its 14th checker (16 parse threads plus 13 checkers), on the critical path. Worker builds now grow the table to 256 entries at load, while the process has one thread, which never waits; the table never shrinks.
Publication (@scriptc/threads publish() and the zero-copy delivery of
deeply frozen plain-data graphs) makes objects immortal: nothing frees
them, and the RC audit already stops counting them. LeakSanitizer did not
know, so a program that drops its last reference to a published graph
reported every such object as a direct leak whenever leak detection was
on (SCRIPTC_SAN=1 runs on Linux without ASAN_OPTIONS=detect_leaks=0, such
as the documented local fallback). Both publication paths now mark each
object they make immortal with the weak __lsan_ignore_object hook in ASan
builds; ignored chunks are leak-scan roots, so the verdict on the rest of
the heap is unchanged. Other builds compile the hook away.
The sanitized lanes run with detect_leaks=0, so a harness test pins the
contract with LeakSanitizer on (Linux, SCRIPTC_SAN=1 lane).
An empty map's tables are already empty, so clear() returns at once
instead of freeing oversized storage that the next fill would regrow.
Only a clear that removes entries frees storage far larger than them.
The compaction test in delete is marked unlikely.
new Map(m) and new Set(s) copied the source's direct-index size, which
follows the source's peak. The copy now covers its largest live key, and
keys that are sparse for their count hash instead.
Number-keyed maps index small integer keys through a direct table sized
by the largest key ever stored, and bucket tables are sized by the most
entries ever held; neither shrank. scr_map_clear memset both on every
call, even on an empty map, so a collection that was large once paid its
peak size on every later clear. tsc-ts' Relater.reset clears a
Set<number> on every relation check: on Sentry, 36% of the CPU was that
memset.
- clear() resets only the direct-index slots of the entries it removes
(the table is zero at every absent key), resets the bucket table only
when entries pointed into it and it is proportional to them, and frees
storage far larger than the entries removed, returning the map to its
initial state as V8 does. Clearing an empty map does no work.
- delete compacts once fewer than a quarter of at least 64 entries are
live (V8's shrink point), so iteration after mass deletion is O(live).
- Compaction shrinks oversized bucket tables, and after a delete or an
iteration it shrinks the direct index to the surviving keys or moves
keys that became sparse back to hashing. The append path keeps the
current index: it holds a slot of it.
Insertion order and live iteration are unchanged: nothing compacts while
an iteration is active.
DynHost.liveDynRefAdapter was declared to return { snapshot; commit }
while LlEmitter returns its cached LlStreamTypedRefAdapter. Under Node
LlDyn received that same object and bindLiveArrayOps found arrayOps; the
native compiler width-copied the record into the narrower shape, dropped
arrayOps and emitted no scr_dyn_typed_ref_bind_array call, so native and
Node builds of the compiler emitted different LLVM. Declare the real
return type so both pass the shared record.
bindLiveArrayOps took an anonymous `{ declare(line: string): void }` and
four callers passed the LlEmitter class instance. The self-hosted
compiler projects a class instance into a record only from plain fields,
not methods, so the self-hosting IR check rejected each call with SC2002.
Type the parameter as ShapeHost and pass `host.shapeHost`, as the
neighbouring vAdapters/typedRefConstructor calls already do. The emitted
LLVM is unchanged.
Full passes are scheduled by heap growth, and the fraction they wait for
widens while growth-triggered passes keep finding nothing. Widening went one
doubling per pass after two passes in a row had asked for it, so a heap that
grows for a whole run never reached the cap of 16 that #833 introduced: on
tsc-ts, runs with seven full passes widened to 9x at the sixth pass, and the
seventh still walked 2.7-3.1M objects (294-393 ms) late in the run.
The old level now widens two doublings per qualifying pass, never past what
the pass's yield asks for, so passes that keep finding nothing reach the cap
on the fourth pass instead of the sixth, while the heap is still small.
Narrowing is unchanged: a pass that finds old garbage moves the fraction
straight to what its yield asks for, and widening still waits for two passes
in a row. The mature level keeps a doubling per pass: a mature pass walks only
what was promoted since the last one, so its total walking hardly depends on
how passes are spaced, and narrower spacing lets less young garbage float.
The memory bound is #833's: once passes stop finding garbage, the heap may
grow to 17 times its size after the last full pass before the next. It is
now reached after four such passes instead of six. Jumping straight to the
cap after two was also measured; it reaches the bound at heaps of a few
thousand objects and is not better on tsc-ts at --checkers 8.
test_cycle pins the schedule from where an old-garbage ring narrows the
fraction: the pass after it does not widen, the second pass in a row that
finds nothing reaches the cap (one doubling per pass fails here at caps 1
and 16), and a capped pass comes exactly at the cap. cycle.test.ts also runs
SCR_CYCLE_GROWTH_CAP=8.
Measured with tsc-ts origin/parallel-check e469e33 (--optimization=speed)
against the parent commit, interleaved in Linux Vercel Sandboxes (8 vCPU),
medians; diagnostics and exit codes identical. The build is the next
commit's with its parallel walk off (SCR_CYCLE_PARALLEL=0), which runs this
commit's collector. RSS in MB:
single-threaded (10 rounds) wall RSS
Excalidraw 7.94 -> 7.32 -7.8% 1263 -> 1280
Playwright 7.09 -> 6.90 -2.7% 1457 -> 1470
TypeORM 6.67 -> 6.39 -4.1% 1014 -> 1046
tsc-ts self-check 4.33 -> 4.03 -6.8% 626 -> 622
--checkers 8 (10 rounds, second host)
Excalidraw 2.56 -> 2.55 -0.6% 2120 -> 2200
Playwright 2.46 -> 2.39 -3.0% 2649 -> 2686
TypeORM 1.96 -> 1.85 -5.4% 1270 -> 1315
tsc-ts self-check 1.04 -> 1.04 +0.5% 742 -> 744
Pass log (instrumented build, one single-threaded run each): full-pass time
337 -> 89 ms (self-check), 426 -> 130 ms (TypeORM), 323 -> 104 ms
(Excalidraw); the largest full pass now walks 1.1-1.3M objects early in the
run, and total collector time drops by 27-35%.
Chris Tate