Commit Graph
209 Commits
Author SHA1 Message Date
Chris Tate efca230de9 Merge ni/i4-deferred and apply parity and security fixes 2026-10-25 22:26:54 -05:00
Chris Tate ec6db73f6b Merge ni/i3-exports 2026-10-25 22:15:54 -05:00
Chris Tate c9116749c1 Merge ni/3-hosted-loop 2026-10-25 20:18:43 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Malte UblandClaude Opus 5.5 1733a4c84b Name new corpus programs instead of numbering them
Parallel changes kept taking the same next corpus number (6992 twice, the
Node island programs renumbered several times). The harness never needed
the numbers: it discovers every entry by glob, and 818 programs already
have descriptive names. AGENTS.md now makes a name, without a number, the
convention for new programs and keeps the numbered ones as they are.

The two scripts that sampled programs by number skipped named programs.
They now key a named program by a stable hash of its name, so they sample
it at the same rate; the numbered programs they pick are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 15:17:09 -07:00
Chris Tate 27954618c7 Merge base branch 2026-10-25 16:31:56 -05:00
Chris Tate 7b24b410aa Merge main 2026-10-25 11:30:45 -05:00
Chris Tate 18fd46be1c Merge main 2026-10-25 10:43:36 -05:00
Chris Tate 5247d2dfdd List bench/regex-replacers among the dynamic-tier workloads
Its replace callbacks receive checked-dynamic values where the lib types
their parameters any: captures a match can leave unset, and results the
checker types any.
2026-10-25 07:28:48 -05:00
Chris Tate 6157fa93ab Merge base branch and fix 2026-10-25 07:02:48 -05:00
Chris Tate c2e6c962e8 Build the sanitized three.js nodes fixture in its own CI job
On the san 1/20 shard the sanitized nodes build shares the runner with the
corpus's other two workers and the side queue. It took 944 s and 860 s on
#2194's runs and passed 1200 s on main's 7fa88083, failing the test timeout.

The Linux san corpus now skips that one case
(SCRIPTC_THREE_SANITIZED_NODES_JOB), and a three_nodes_san job runs it alone
with the corpus's LeakSanitizer settings. The required test check waits for
the new job. Every other three.js case, the plain nodes build and the macOS
suites are unchanged.
2026-10-25 06:49:46 -05:00
Chris Tate ad26bed418 List line-reader among the benchmark workloads that link the dynamic tier
The workload reads its file through readline over fs.createReadStream.
The stream and readline runtime units define dynamic-tier symbols on
their own: a program that only reads a file with createReadStream links
the same 32, as log-summary and inventory-report do.
2026-10-25 02:29:47 -05:00
Chris Tate 0f7fd1bdb6 Merge base and fix 2026-10-25 02:27:42 -05:00
Chris Tate db2cb5c99e Split readline input in one scan and copy ASCII strings into buffers whole
readline split redirected input byte by byte, twice per chunk (once to
find whether the chunk ends a line, once to split it). Before splitting
it copied each chunk out of its Buffer and into a scratch buffer, decoded
it into a new string, and copied the unfinished line and the whole chunk
into a joined buffer. Now:

- one scan per chunk: memchr for a CR once, then LF to LF with memchr
  when the chunk has none, and the LF, CR and CRLF split otherwise;
- the unfinished line joins only the first line of the chunk;
- an ASCII chunk with no partial UTF-8 sequence held from the previous
  one is split where it is, and a chunk that is not shared memory is
  read from its own storage. Lines are still copied into their strings
  before the first is delivered.

Buffer.from(string, 'latin1' | 'ascii') encoded character by character;
an ASCII prefix, all of an ASCII string, is now copied with memcpy.

Line events, their order and the CRLF delay are unchanged. Corpus 8344
covers lines joined across chunks, a line longer than a chunk, empty
lines, mixed line endings and UTF-8 split across chunks. The new
line-reader workload reads a 67 MB log through readline over a file
stream.
2026-10-25 01:35:28 -05:00
Chris Tate e6790def72 Merge main 2026-10-24 20:06:31 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate 8b4f83ac02 Collect the native compiler profile on the release commit (#2138) 2026-10-24 18:25:03 -05:00
Chris Tate a5c7a4e983 Report how well the native compiler's profile matches its functions (#2117) 2026-10-24 17:39:30 -05:00
Chris Tate 40faf6a9f1 Collect the native compiler profile on the release commit
The release workflow's new cli-profile job runs
scripts/refresh-cli-profile.mjs on Linux x64 and uploads the profile.
Every platform's native CLI build uses it through
SCRIPTC_NATIVE_PROFILE_USE, and the GitHub release attaches it as
scriptc-<version>-cli.profdata.gz. When the job fails, the CLI builds
use the committed profile.

build-native-cli.mts accepts a gzip-compressed profile in
SCRIPTC_NATIVE_PROFILE_USE when the name ends in .gz.
refresh-cli-profile.mjs creates the directory of --output before it
starts.
2026-10-24 14:16:32 -05:00
Chris Tate 29a78af70b Merge main 2026-10-24 13:13:07 -05:00
Chris Tate 396c6944ec Report how well the native compiler's profile matches its functions
The native compiler is built with a committed optimization profile. A
function whose control flow changed since the profile was collected is
optimized as without one, so the profile loses its gain as compiler code
changes, with nothing to show it.

With --profile-use, the helper now counts what the profile annotation
reports while it applies the profile: functions that took the profile's
counts, functions whose control flow no longer matches (hash mismatch),
and functions the profile has no record of. It sums the counts over
partitions, keeps each partition's counts beside its cached summary, and
prints them as one line of JSON after a successful emit. The compiler
reads that line (an older helper prints none) and returns it as the
executable artifact's profileMatch.

build-native-cli prints the share of profiled functions that match, and
warns below 90% that the profile should be refreshed. On main aa1c26845, a
profile refreshed on that commit matches 92.7% of the compiler's profiled
functions, and the one committed about 200 commits earlier 87.2%; that one
kept about half of the fresh profile's gain on a cold tsc-ts build.
2026-10-24 11:20:47 -05:00
Chris Tate 2967a1b8c0 Propagate exceptions by unwinding from directly called functions on Linux
Emitted functions report an exception by returning with the thread's
pending flag set, and every caller tests the flag after each call that
may throw. On Linux executables (glibc and musl), functions reached only
by direct calls now propagate exceptions by unwinding the native stack
instead:

- unwindAbiFunctions selects the may-throw functions that are not async
  or generators, constructors, virtually called or overridden methods,
  closure targets, the entry, library exports, module initializers or
  callees of coroutine bodies.
- A native unwind costs about 2 µs a raise and 0.1 µs a frame, so functions that
  exceptions are likely to cross keep the flag: those with a `throw` of
  their own, those up to three calls above one, and recursive functions,
  counting calls through methods, closures and callbacks.
- Their propagating exits call scr_unwind_raise (scr_unwind.c), a forced
  unwind through _Unwind_ForcedUnwind with the exception still in the
  active cell.
- Direct calls to them are `invoke`s without a pending check. The
  landing pad runs the frame's usual unwind path: it releases the frame's
  values and then enters the enclosing try handler or resumes the unwind.
- Remaining plain calls to them (wrappers, adapters, runtime-facing entry
  points) become invokes whose landing pad continues on the pending flag,
  or resumes when the wrapper is itself invoked. Every other call is
  marked `nounwind`, so inlining a raising function at an invoke keeps its
  calls plain.
- When emitted text breaks these rules (a raising function's address is
  taken, or an invoked function returns with the flag pending), the
  module is emitted again with those functions on the pending flag.

Emitted code does not declare scr_unwind_raise `noreturn`, so sanitized
builds do not run the sanitizer's no-return hook, which needs stack
bounds that fiber stacks do not report. Instead, scr_unwind_raise clears
the shadow of the frames it unwinds.

Executables link a static unwinder: libgcc_eh with the host toolchain and
the glibc runtime packs, and Zig's libunwind with Zig and the musl packs.
macOS, Windows, Android, WASI, libraries and debug builds keep the
pending flag everywhere and emit the same code as before.

The throw-chains runtime workload measures exceptions caught 2, 5 and 20
recursive calls above the throw, through direct calls and a callback,
and a parser that backtracks by throwing.
2026-10-24 11:03:34 -05:00
Chris Tate 427e383b5e Cache untyped property access in programs that declare classes (reland)
Reapplies #2017, which #2038 reverted after effect's observability-otlp
fixture crashed in plain builds. The crash was the candidate-buffer
corruption that the previous commit fixes: the cached property sites
change when the collector runs, and with enough environment variables a
pass buffered a string box that was later pooled while still buffered.
2026-10-24 08:22:30 -05:00
Chris Tate 7278c3c467 Read a union's discriminant once in checked casts (#2073) 2026-10-24 04:51:36 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate e3fb35cc68 Cap the Linux test step's memory and move memory-heavy side files apart (#2030) 2026-10-23 23:48:45 -05:00
Chris Tate 36daee3f61 Fix the CI failure 2026-10-23 23:16:58 -05:00
Chris Tate b6a3756e8f Merge main into the Node island runtime 2026-10-23 15:25:48 -05:00
Chris Tate 7cdfe16c9b Cap the Linux test step's memory and spread memory-heavy side files
The Linux test shards run their suites in a transient systemd scope with
MemoryMax=13G and no swap, below the runner's 16 GB. A process tree that
grows past the cap loses its largest process to the kernel, and the job
log names the failing test, where the runner itself ran out of memory
and left no log. Hosts without systemd run the suites directly.

The side-queue balancer also takes a table of memory-heavy files, with
their measured peak. They run whole in the cost-balanced task, and each
goes to the shard with the least memory-heavy work so far, after the
time-balanced files. decl-boundary.test.ts, which peaks at about 1.8 GB
alone and had no time measurement, so vitest's file sharding put it in
shard 11, is the first entry. It now runs in shard 1 (plain) and 6 (san).
2026-10-23 14:24:38 -05:00
Chris Tate e3b42a311d Add --target-cpu to compile x86-64 programs for a microarchitecture level
Release builds compile for the x86-64 baseline, so LLVM vectorizes with
SSE2 only. `--target-cpu=<baseline|x86-64-v2|x86-64-v3|x86-64-v4|native>`
compiles the program's code for a higher level: every emitted function
carries the level as its `target-cpu`, so LLVM can use AVX2, BMI2, FMA or
AVX-512. The executable then runs only on CPUs of that level.

- `native` resolves to the highest level the build machine supports, read
  from /proc/cpuinfo on Linux and `sysctl machdep.cpu` on macOS.
- The level is part of the executable cache key; `baseline` and builds
  without the option produce the same output and keys as before.
- The runtime pack keeps its baseline code.
- Targets other than x86-64 refuse a level (SC3002). The CLI's cached
  bootstrap route leaves builds with the option to the full compiler.
- `bench:runtime` takes `--baseline-target-cpu` and `--candidate-target-cpu`.
2026-10-23 13:10:16 -05:00
Chris Tate 4dcb83c418 Free a typed teardown's blocks in the order it met them (#1849) 2026-10-23 12:09:37 -05:00
Chris Tate a10c183b13 Revert "Cache untyped property access in programs that declare classes (#1995)" 2026-10-23 09:05:55 -05:00
Chris Tate 8eedab9fde Cache untyped property access in programs that declare classes
A program that declares a class reads and writes untyped properties
through helpers that first select the receiver's property bag. The bag
helpers return jsvals, so the helpers read the bag through js.toDyn;
keyed helpers test each class's key table between the key's coercion and
the read; and a program with subclasses normalizes the receiver
(value = %dyn.class.normalize(value)) at the start of each helper. The
inline-cache recognizer accepted none of these forms, so every property
site in such a program called its helper. It now looks through dyn/jsval
conversions of the receiver, accepts capsule branches before a keyed
helper's final read, and accepts a rebinding of the receiver to a bag of
itself. Plain objects and arrays then read and write through the site's
cache, as they do in programs without classes.

bench/dynamic-properties (new) exercises these sites.
2026-10-23 08:41:40 -05:00
Malte Ubl 4bbbf3e079 Merge remote-tracking branch 'origin/ni/i4-deferred' into HEAD 2026-10-23 06:33:28 -07:00
Malte Ubl 67a0656781 Merge remote-tracking branch 'origin/ni/i3-exports' into HEAD
# Conflicts:
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 06:32:59 -07:00
Malte Ubl 2acfd9e723 Merge remote-tracking branch 'origin/ni/3-hosted-loop' into HEAD
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/frontend/pipeline.ts
2026-10-23 05:57:22 -07:00
Malte Ubl 2494e7c1dc Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/src/backend/native/executable.ts
#	packages/compiler/src/frontend/lowering/lower-calls.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 05:53:42 -07:00
Chris Tate 19e0fe8bcd Merge main and read cipher options through getters and prototypes 2026-10-23 05:33:14 -05:00
Chris Tate 0314d9db3e Dispatch checked-dynamic method calls on method ids 2026-10-22 23:52:25 -05:00
Chris Tate f030e5ad44 Check with tsc-ts 0.0.8 on every platform
tsc-ts 0.0.8's API server serves on the named pipe the Node host
connects over on Windows (tsc-ts --api --pipe), and the native compiler
already starts it over its standard streams there, so tsc-ts is the
default checker on every platform. SCRIPTC_CHECKER=tsgo still selects
tsgo.

The compiler depends on tsc-ts 0.0.8, which the release-age exclusions
admit with its platform packages and @scriptc/threads 0.4.2.

The Windows CLI smoke test runs with both checkers, and
verify-native-cli.mjs --run checks that a distribution's tsc-ts runs on
its platform (tsc-ts.exe on Windows).
2026-10-22 22:36:11 -05:00
Malte UblandClaude Opus 5.5 5fa44d9243 Share modules between a program and its Node islands through loader hooks (NI-6)
Node islands stack, part 5 of 11 (scriptc-perf/results/islands-prs/PLAN.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:49:53 -07:00
Malte UblandClaude Opus 5.5 2f8e8b2b92 Defer evaluating Node island modules until the program uses them (NI-7)
Node islands stack, part 4 of 11 (scriptc-perf/results/islands-prs/PLAN.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:49:53 -07:00
Chris Tate dc2b238df7 Revert "Dispatch checked-dynamic method calls on method ids (#1851)" 2026-10-22 18:02:51 -05:00
Chris Tate 289a6f619d Merge main 2026-10-22 17:39:40 -05:00
Chris Tate 5195e2b63a Merge main 2026-10-22 17:19:37 -05:00
Chris Tate 846552b0e2 Support crypto.getCipherInfo, AES-CCM and the AES key wrap ciphers
getCipherInfo(nameOrNid[, options]) runs in a new runtime unit,
scr_cipher_info.c, over a table that scripts/gen-cipher-info.mjs generates
from Node 24.22's getCipherInfo. The table covers every cipher OpenSSL
names, including the ones without a native implementation. Lookups match
names case-insensitively and accept OpenSSL's nids. The unit applies
cipher.js's argument checks and GetCipherInfo's keyLength and ivLength
rules, and returns the info object in Node's key order. CipherInfo maps
to a checked-dynamic value, so TypeScript and JavaScript code receive
Node's object. Reading the nid Node leaves out for the SIV and SM4 GCM,
CCM and XTS ciphers, where the declared type gives number, throws a
TypeError.

createCipheriv and createDecipheriv support aes-128/192/256-ccm (and the
id-aesN-ccm names) over OpenSSL's ccm128 steps. They follow Node's tag and
IV lengths, the plaintextLength requirement, the message-length caps and
the one-update rule, and report a failed authentication from final().

They also support the RFC 3394 and RFC 5649 key wrap ciphers (aes-N-wrap,
aesN-wrap, id-aesN-wrap, the -pad forms and the -inv ciphers) over
wrap128.c's routines, with Node's lengths, integrity failures and
one-update rule.

The new names follow the existing ones in the cipher table, so the lookup
cost of the existing ciphers is unchanged.
2026-10-22 17:18:54 -05:00
Chris Tate 8bc4586b9a Select AST children by wire slot in the generated getters 2026-10-22 16:56:13 -05:00
Malte Ubl fb0d7a99a8 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/13-libnode-packs 2026-10-22 14:17:35 -07:00
Malte Ubl 13b00dd78b Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/frontend/lowering/lower-modules.ts
#	tests/harness/differential.test.ts
#	tests/harness/linux-differential.test.ts
#	tests/harness/llvm-differential.test.ts
#	tests/harness/windows-differential.test.ts
2026-10-22 14:10:07 -07:00
Chris Tate 7cce36354f Dispatch checked-dynamic method calls on method ids
A method call on a value of unknown type (untyped JavaScript, `any`)
picked its builtin by comparing the method name with strcmp against each
candidate, in long ladders per receiver kind: a string method could test
dozens of names before it matched.

- The name is now looked up once per call in a generated hash table
  (scr_dyn_methods.h, from scripts/gen-dyn-methods.mjs), and each ladder
  compares the resulting id. The table verifies the name's bytes, so a
  name is matched by its content, as before.
- The worker event methods compare the first character before strcmp.
- bench/dynamic-methods calls string, array, Map, Set, number and own
  methods from untyped JavaScript.
2026-10-22 11:21:10 -05:00