Commit Graph
36 Commits
Author SHA1 Message Date
Chris Tate 78dbd98b1a Merge main 2026-10-29 16:39:31 -05:00
Chris Tate 9632906b4d Take from a Readable's buffered queue by moving its head index
Taking the head entry of a Readable's buffer moved every remaining entry
down one slot, so draining a queue of n buffered chunks cost O(n²). The
buffer now keeps the index of its first entry: a take advances it, a push
at the tail moves the entries down only when at least half the slots are
free at the front (and grows otherwise), and unshift takes a freed front
slot when there is one. The drop, trace and boxing loops walk the entries
from that index.

The stream-buffered-drain runtime workload buffers 200,000 objects and
100,000 Buffers before draining them; corpus 8533 covers reads across
entries, unshift into a freed slot and onto a part-read head, setEncoding
after takes, and object-mode takes, unshifts and pushes.
2026-10-29 14:59:10 -05:00
Malte UblandClaude Opus 5.5 7cf37d9dc9 Keep ICU symbols out of non-Intl binaries in the zero-cost gate
ICU builds with symbol renaming on, so its C API symbols carry the _78
version suffix, its data symbol is icudt78_dat, and its C++ symbols are
Itanium manglings that name namespace icu_78. icuSymbols() classifies
exactly those, and the gate's dynamic-tier check now fails a candidate
that links any of them without the pattern being allowed: a program that
does not lower to Intl links no ICU object at all.

The engine-free Intl functions (scr_intl_num_format_en_us in scr_lib.c,
scr_intl_segmenter_new in scr_grapheme.c) share the scr_intl_ prefix and
are deliberately not part of the test; any scr_intl.c function that does
link brings an ICU symbol with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 19:49:11 -07:00
Chris Tate d5ac31f812 Raise through cached unwind plans on x86-64 Linux
scr_unwind_raise walks the stack with plans cached per return address.
The first raise through a return address finds the function's FDE
through the .eh_frame_hdr of the object that contains it, runs the CFI
up to the call, and finds the call site's landing pad in the LSDA; the
plan records the frame's CFA rule, where its callee-saved registers are
saved, and the landing pad. Later raises look the plans up, restore the
registers, and jump to the first landing pad. Plans are inserted under
a lock and read without one.

Landing pads continue the unwind with scr_unwind_resume instead of
`resume`, so the walk goes on from their frame. A frame whose CFI,
pointer encodings or personality the plans do not express, or a return
address without an FDE, leaves the unwind to libgcc's forced unwind
from the frame the walk started at, as before. Other Linux targets keep
libgcc's unwinder; their scr_unwind_resume calls _Unwind_Resume.

bench:runtime gains layered-errors, a workload whose errors are thrown
below small functions that hold references and caught per entry in a
loop, and the corpus gains unwind-landing-pad-chains, which pins landing
pads that continue the unwind, including in frames LLVM inlines into a
caller whose handler takes the exception.
2026-10-28 00:22:04 -05:00
Chris Tate b16c739d34 Classify zero-cost gate differences instead of failing every runtime change (#2271) 2026-10-27 19:27:25 -05:00
Chris Tate 08fbecb7e7 Excuse only reproduced differences as nondeterministic, and read vendor archive symbols
A program is nondeterministic only when every artifact that differs also
differs between two baseline builds; a deterministic difference in another
artifact still fails the gate. The runtime-only check now reads the symbols
of changed vendor archives as well as runtime objects.
2026-10-27 17:55:44 -05:00
Malte UblandClaude Opus 5.5 4d5127944e Merge main into the zero-cost triage branch
Main moved the per-program report lines into zero-cost-compare.mjs and
added sharding and a baseline cache. The runtime-only and
nondeterministic verdicts now print through programLines, count as
compared in merged shard reports, and the runtime pack diff is skipped
when only recording a baseline cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 05:38:08 -07:00
Chris Tate df77671e33 Merge main 2026-10-26 23:13:34 -05:00
Chris Tate ad29d598f5 Merge main into the untyped-store branch 2026-10-26 21:44:47 -05:00
Chris Tate e58def0299 Update tests for the record-ops store hook and the untyped-writes benchmark
- forin-keys.test.ts matches the record-ops table with its eighth field, the
  store hook, which is null for a record no untyped code writes.
- bench/record-untyped-writes writes typed records through `unknown` and
  `Object.assign` views, so it links dynamic-tier symbols; DYNAMIC_WORKLOADS
  lists it.
- The sanitized self-hosting emission test gets a 600 s timeout. It takes
  about half of the default 300 s on an idle 16-core host, with or without
  these changes, and exceeded the default on a loaded CI shard.
2026-10-26 21:04:31 -05:00
Chris Tate cd606e96ff Key baseline cache entries by their program's inputs
A recorded baseline entry now holds a hash of its program's inputs: the
entry file and its relative imports (paths and contents) and the
optimization level. --baseline-cache uses an entry only when the program
still hashes the same, so a program or an import that changed since the
record builds its baseline, instead of a recorded build error or artifact
hash of the old sources settling the comparison as both-fail or identical.
Records take schema 2; older records are ignored.
2026-10-26 19:53:49 -05:00
Malte UblandClaude Opus 5.5 d0047c813f Shard the zero-cost gate and reuse baseline work across runs
The full gate takes 30 to 75 minutes on one Sandbox, which made it the
longest step of a validation round. It can now be split and merged:

- --shard=i/n splits the bench and corpus programs (the self-hosted
  compiler runs in shard 1, or alone with --set=self), and
  scripts/zero-cost-merge.mjs merges the shards' --json reports into one
  verdict, recomputing the compile-time intervals over every shard.
- --record-baseline-cache records the baseline's artifact hashes once;
  --baseline-cache skips the baseline build of every program whose
  candidate artifacts hash the same. The record is tied to the built
  baseline checkout, toolchain and paths.
- The candidate-source self build overlaps the comparison builds on
  hosts with room for it.

pnpm test:sandbox can reuse a prepared snapshot's LLVM helper and runtime
pack (SCRIPTC_SANDBOX_NATIVE_REFERENCE) when their inputs are identical,
and retries rate-limited Sandbox creation with backoff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 17:43:44 -07:00
Malte UblandClaude Opus 5.5 0d27008899 Classify zero-cost gate differences instead of failing every runtime change
Any change to the runtime's C sources makes every executable differ, so
the gate failed even when every program's IR and LLVM were byte-identical.
The gate now classifies each difference outside --allow:

- runtime-only: IR and LLVM identical, the same LLVM helper, runtime pack
  objects that differ between the checkouts (bitcode alone records the
  checkout path, so it does not count), and every symbol the executable
  gains, loses or resizes defined by a changed runtime object. It passes
  with a note that lists the changed runtime functions the program links
  (from the release and speed bitcode, through llvm-dis) and the size
  change.
- nondeterministic: the baseline builds the program again, and its two
  builds differ in the same artifacts. It passes with a note.
- differs: anything else; it fails as before.

--strict fails runtime-only and nondeterministic programs too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 15:26:18 -07:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate 5247d2dfdd List bench/regex-replacers among the dynamic-tier workloads
Its replace callbacks receive checked-dynamic values where the lib types
their parameters any: captures a match can leave unset, and results the
checker types any.
2026-10-25 07:28:48 -05:00
Chris Tate ad26bed418 List line-reader among the benchmark workloads that link the dynamic tier
The workload reads its file through readline over fs.createReadStream.
The stream and readline runtime units define dynamic-tier symbols on
their own: a program that only reads a file with createReadStream links
the same 32, as log-summary and inventory-report do.
2026-10-25 02:29:47 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate 29a78af70b Merge main 2026-10-24 13:13:07 -05:00
Chris Tate 2967a1b8c0 Propagate exceptions by unwinding from directly called functions on Linux
Emitted functions report an exception by returning with the thread's
pending flag set, and every caller tests the flag after each call that
may throw. On Linux executables (glibc and musl), functions reached only
by direct calls now propagate exceptions by unwinding the native stack
instead:

- unwindAbiFunctions selects the may-throw functions that are not async
  or generators, constructors, virtually called or overridden methods,
  closure targets, the entry, library exports, module initializers or
  callees of coroutine bodies.
- A native unwind costs about 2 µs a raise and 0.1 µs a frame, so functions that
  exceptions are likely to cross keep the flag: those with a `throw` of
  their own, those up to three calls above one, and recursive functions,
  counting calls through methods, closures and callbacks.
- Their propagating exits call scr_unwind_raise (scr_unwind.c), a forced
  unwind through _Unwind_ForcedUnwind with the exception still in the
  active cell.
- Direct calls to them are `invoke`s without a pending check. The
  landing pad runs the frame's usual unwind path: it releases the frame's
  values and then enters the enclosing try handler or resumes the unwind.
- Remaining plain calls to them (wrappers, adapters, runtime-facing entry
  points) become invokes whose landing pad continues on the pending flag,
  or resumes when the wrapper is itself invoked. Every other call is
  marked `nounwind`, so inlining a raising function at an invoke keeps its
  calls plain.
- When emitted text breaks these rules (a raising function's address is
  taken, or an invoked function returns with the flag pending), the
  module is emitted again with those functions on the pending flag.

Emitted code does not declare scr_unwind_raise `noreturn`, so sanitized
builds do not run the sanitizer's no-return hook, which needs stack
bounds that fiber stacks do not report. Instead, scr_unwind_raise clears
the shadow of the frames it unwinds.

Executables link a static unwinder: libgcc_eh with the host toolchain and
the glibc runtime packs, and Zig's libunwind with Zig and the musl packs.
macOS, Windows, Android, WASI, libraries and debug builds keep the
pending flag everywhere and emit the same code as before.

The throw-chains runtime workload measures exceptions caught 2, 5 and 20
recursive calls above the throw, through direct calls and a callback,
and a parser that backtracks by throwing.
2026-10-24 11:03:34 -05:00
Chris Tate 427e383b5e Cache untyped property access in programs that declare classes (reland)
Reapplies #2017, which #2038 reverted after effect's observability-otlp
fixture crashed in plain builds. The crash was the candidate-buffer
corruption that the previous commit fixes: the cached property sites
change when the collector runs, and with enough environment variables a
pass buffered a string box that was later pooled while still buffered.
2026-10-24 08:22:30 -05:00
Chris Tate 36daee3f61 Fix the CI failure 2026-10-23 23:16:58 -05:00
Chris Tate 9eb14207a6 Revert "Cache untyped property access in programs that declare classes (reland)"
This reverts #2017 (adb8690f1), which made main CI fail.
2026-10-23 15:53:36 -05:00
Chris Tate bd36b77a8f Cache untyped property access in programs that declare classes (reland) 2026-10-23 11:57:22 -05:00
Chris Tate a10c183b13 Revert "Cache untyped property access in programs that declare classes (#1995)" 2026-10-23 09:05:55 -05:00
Chris Tate 8eedab9fde Cache untyped property access in programs that declare classes
A program that declares a class reads and writes untyped properties
through helpers that first select the receiver's property bag. The bag
helpers return jsvals, so the helpers read the bag through js.toDyn;
keyed helpers test each class's key table between the key's coercion and
the read; and a program with subclasses normalizes the receiver
(value = %dyn.class.normalize(value)) at the start of each helper. The
inline-cache recognizer accepted none of these forms, so every property
site in such a program called its helper. It now looks through dyn/jsval
conversions of the receiver, accepts capsule branches before a keyed
helper's final read, and accepts a rebinding of the receiver to a bag of
itself. Plain objects and arrays then read and write through the site's
cache, as they do in programs without classes.

bench/dynamic-properties (new) exercises these sites.
2026-10-23 08:41:40 -05:00
Malte Ubl 2494e7c1dc Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/src/backend/native/executable.ts
#	packages/compiler/src/frontend/lowering/lower-calls.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 05:53:42 -07:00
Chris Tate 0314d9db3e Dispatch checked-dynamic method calls on method ids 2026-10-22 23:52:25 -05:00
Chris Tate dc2b238df7 Revert "Dispatch checked-dynamic method calls on method ids (#1851)" 2026-10-22 18:02:51 -05:00
Chris Tate 7cce36354f Dispatch checked-dynamic method calls on method ids
A method call on a value of unknown type (untyped JavaScript, `any`)
picked its builtin by comparing the method name with strcmp against each
candidate, in long ladders per receiver kind: a string method could test
dozens of names before it matched.

- The name is now looked up once per call in a generated hash table
  (scr_dyn_methods.h, from scripts/gen-dyn-methods.mjs), and each ladder
  compares the resulting id. The table verifies the name's bytes, so a
  name is matched by its content, as before.
- The worker event methods compare the first character before strcmp.
- bench/dynamic-methods calls string, array, Map, Set, number and own
  methods from untyped JavaScript.
2026-10-22 11:21:10 -05:00
Malte UblandClaude Opus 5.5 9a0a3f1562 Load Node.js in-process on first use: libnode bridge and runtime loader
Scope-B milestone NI-1, runtime side. Programs whose IR has a
`nodeIsland.*` operation link scr_node_island.c and the SCR_NODE_ISLANDS
build of scr_async.c; every other program links neither, byte for byte.

- scr_node_island.c dlopens the bridge of a libnode pack on the first call
  (RTLD_NOW | RTLD_GLOBAL), checks its bridge ABI and that it links exactly
  the Node version recorded at build time, starts one Node environment and
  runs the embedded bootstrap (scr_node_island_host.cjs). A run that never
  calls into Node never maps libnode. The pack directory is SCRIPTC_LIBNODE
  at run time, else the recorded path; a `$ORIGIN/` path resolves against
  the executable's directory (the beside packaging, and the hook for
  shipping Node with the program).
- scr_node_bridge.h is the C ABI between the runtime and the bridge; the
  runtime includes no Node or V8 header. The bridge
  (packages/libnode-common/bridge/scr_node_bridge.cc) initializes Node once
  with no default signal handling, no debug-signal handler or inspector,
  no stdio reinitialization and one platform worker, creates the
  environment from the built-in snapshot, routes Node's process exit to
  the runtime's exit sequence, promotes libnode to the global scope for
  native addons, and runs calls in a CallbackScope with a TryCatch.
  build-pack.mjs assembles a pack (libnode, the bridge, manifest.json) from
  a Node --shared build and the matching headers.
- The SCR_NODE_ISLANDS build of scr_async.c adds a loop slot through which
  the runtime's loop drives Node's until NI-3 hosts the runtime's queues on
  Node's libuv: it keeps the program alive while Node has ref'd work,
  sleeps on Node's backend fd up to its next timer, and runs one
  non-blocking turn per loop turn.
- The `nodeIsland` runtime feature (rule, link features, unit table, pack
  matrix row and scr_async.c variant), the nodeIsland.call/callSync
  libCalls, the install call in main, and the build-time record of the
  pack (node-islands/libnode.ts: --libnode=<pack>|beside or
  SCRIPTC_LIBNODE, SC8005 when none is usable, Node >=24.13.1).
- The zero-cost gate and the bench nm test require that no good-code
  executable defines a Node island symbol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:54:47 -07:00
Chris Tate cb75c51eda Fail the zero-cost gate when it compares nothing
A benchmark workload or the self-hosted compiler that fails to build
with both checkouts now fails the gate, and so does a run in which no
program built with both. Before this, a run whose builds all failed the
same way, such as with a helper of another version, reported PASS. A
build error now prints in full under its program; its first line is
often a warning.
2026-10-21 03:45:36 -05:00
Chris Tate 50f93d0be0 Merge main 2026-10-20 19:05:22 -05:00
Chris Tate 3d00b5b837 Size encoded messages to their nodes and wake a channel's owner once per batch
A message encoder started its node array at 16 nodes of 184 bytes and
kept the array, with its unused capacity, until the message was
received. A port that queued many messages held about 3 KB of zeroed
memory per message: 200,000 queued numbers took about 590 MB, and the
page faults made postMessage cost about 1.3 µs. The node array now
starts at 4 nodes and is trimmed to the message's node count when
encoding ends.

scr_channel_post posted a wake event, and with it a pipe write and a
condition broadcast, to the receiving thread's mailbox for every
message. The owner's loop counts a port with queued messages as ready,
so only a message that finds the queue empty needs to wake it.

bench/channel-messages, added here, posts 200,000 messages (numbers and
small objects) on a MessageChannel before the port delivers any.
2026-10-19 23:24:56 -05:00
Malte UblandClaude Opus 5.5 7ea35785ee Check the bench workloads' dynamic-tier symbols in CI
The zero-cost gate's I3 check (a fully typed bench workload defines no
dynamic-tier runtime symbol) runs only when someone runs the gate, and on
Darwin it never matched: Mach-O names carry a leading underscore, so no
symbol had a DYNAMIC_TIER_PREFIXES prefix.

DYNAMIC_WORKLOADS and the nm function-name reader move to
zero-cost-compare.mjs, which now drops the Mach-O underscore. A harness
test builds every bench program on Linux and fails when a workload outside
DYNAMIC_WORKLOADS links a dynamic-tier symbol, or when a listed workload
no longer links any, so the list shrinks as leaks close.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 06:44:29 -07:00
Chris Tate 29c8ad237c Throw when nm cannot list an executable's symbols in the zero-cost gate
The gate read an empty symbol list when nm could not start, was killed by
a signal, or exited with a non-zero status, so the dynamic-tier symbol
check and the executable comparison passed without any symbols. nmOutput
now throws with the binary, the reason and nm's stderr.
2026-10-18 11:36:13 -05:00
Malte UblandClaude Opus 5.5 8e6986c561 Add the zero-cost gate
scripts/zero-cost-gate.mjs compiles a fixed good-code set (the runtime
benchmark workloads, the self-hosted compiler, and the TypeScript corpus)
with a baseline and a candidate checkout. It requires byte-identical IR,
LLVM and release executables, and names the first differing IR function,
LLVM function or executable symbol when something changes. Programs that
match the change's pattern signature (--allow) may differ. It also checks
compile time (paired builds, bootstrap interval), binary size, optional
per-function cachegrind instruction counts, and the two-tier I3 symbol
rule: no executable gains dynamic-tier runtime symbols, and fully typed
benchmark workloads define none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 08:17:51 -07:00