Commit Graph
108 Commits
Author SHA1 Message Date
Chris Tate b9587f361d Reuse lowered units across builds behind SCRIPTC_INCR_LOWERING=1 (#2521) 2026-10-29 15:11:51 -05:00
Chris Tate 1784445e31 Split the Linux LLVM artifacts job and take its glibc packages from native_linux (#2536) 2026-10-29 15:08:06 -05:00
Chris Tate 593673f87a Provide the libnode pack to the shape overlay npm lanes
The scheduled shape overlay lanes for npm and npm-static failed: on Linux
x64 CI the npm suites' Node island cases require SCRIPTC_LIBNODE
(tests/harness/libnode.ts), and this job did not run
.github/actions/libnode. The job now runs it for those two suites, as the
other Linux x64 and macOS test lanes do.
2026-10-29 06:38:22 -05:00
Chris Tate 2fee369fe9 Split the Linux LLVM artifacts job and take its glibc packages from native_linux 2026-10-29 04:14:27 -05:00
Chris Tate 16ea77108b Reuse lowered units across builds behind SCRIPTC_INCR_LOWERING=1 2026-10-28 23:57:15 -05:00
Chris Tate 1278729267 Merge main 2026-10-27 23:21:09 -05:00
Chris Tate 248030cdd8 Merge main 2026-10-27 19:47:24 -05:00
Malte Ubl f9419b5d47 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into HEAD
# Conflicts:
#	packages/compiler/src/index.ts
2026-10-27 05:40:18 -07:00
Malte Ubl 7f0488a411 Merge remote-tracking branch 'origin/main' into perf/static-objects 2026-10-27 05:35:56 -07:00
Chris Tate 659a74acc4 Merge main 2026-10-27 02:27:10 -05:00
Chris Tate 5e6e1a9d85 Run the macOS arm64 no-clang contracts in their own job
Shard 1 of "test (macOS arm64 LLVM artifacts, no clang)" ran the no-clang
object, helper parity, packaged helper and packed npm installation
contracts around its LLVM differential slice: about 12 minutes on top of
a 35-minute slice, 45 to 49 minutes against the job's 50-minute timeout.
The contracts now run in "test (macOS arm64 LLVM artifacts, no clang,
contracts)", which the all-lanes check requires like the shards, and
every shard runs only its differential slice.
2026-10-26 22:21:48 -05:00
Malte UblandClaude Opus 5.5 5f3e97998d Merge the macOS island fixes of ni/1-libnode-loader
The libnode-linux action becomes .github/actions/libnode, which serves
the Linux x64 and macOS lanes: the libnode-packs cache for the runner's
platform, else the development pack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:50:39 -07:00
Malte UblandClaude Opus 5.5 df869f28b0 Run Node islands on macOS, and test them in the macOS CI lanes
macOS binds libnode's own calls to v8::base::Stack inside libnode, so the
bridge's definitions cannot give V8 Node's stack as they do on Linux; V8
took the main thread's stack start and its conservative scan faulted in
the gap above Node's stack. While Node starts on its stack, the main
thread's pthread record now names that stack, which V8 reads once and
keeps.

macOS's makecontext clears the whole stack it is given, so it gets only
the top slice of Node's stack (no 64 MiB pass per context).

CI: dev-pack.json pins mirrored darwin-arm64 and darwin-x64 libnode
builds of the pinned Node 24.22.0 from the libnode-packs workflow, the
dev-pack action serves either platform, and the macOS lanes provision it,
so island tests fail rather than skip on macOS CI. The arm64 LLVM job
runs the island harness; the Intel job runs it and the island corpus.
bridge-smoke loads the .dylib bridge on macOS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:50:38 -07:00
Malte Ubl 141279b74c Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/13-libnode-packs 2026-10-26 11:15:43 -07:00
Malte Ubl 5817be4d0f Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/ir/function-references.test.ts
2026-10-26 11:08:44 -07:00
Chris Tate 06f7b8631f Merge perf/shape-overlay-cost 2026-10-26 11:53:34 -05:00
Chris Tate f9b5533ee6 Read IR nodes' fields by name in three llvm-emit passes 2026-10-26 08:13:15 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Malte Ubl b04fea812d Merge remote-tracking branch 'origin/fix/js-dynamic-lowering' into HEAD
# Conflicts:
#	packages/compiler/src/frontend/pipeline.ts
#	packages/compiler/src/frontend/shape-inference/overlay.ts
#	packages/compiler/src/frontend/shape-inference/static-objects.ts
#	packages/compiler/test/shape-overlay-render.test.ts
2026-10-25 13:46:26 -07:00
Malte UblandClaude Opus 5.5 5455a4d817 Make the shape overlay's facts sound on the real-package suites
The overlay validation suites (npm, npm-static, three, Effect 3 and 4) now
pass with the overlay on. Each cause had an npm repro or suite case that
failed with the overlay and passed without it:

Analysis facts:
- Symbol members: Symbol.for/keyFor are functions, well-known symbols are
  symbols, other members unknown.
- Reads that may answer undefined: index reads past the end (unless the
  enclosing for loop proves them in bounds of an array that never shrinks or
  has holes), computed reads of objects, keys an object literal lacks at
  creation, optional-chain short circuits, `let x;` before assignment.
- Object.create(proto) inherits proto members; arguments past the declared
  parameters are unknown; `new` of a primitive answers nothing.
- Arithmetic on bigints answers bigints (number|bigint is never printed).

Overlay:
- Package JSDoc is a declaration: it stays as written and the overlay adds
  only types the JSDoc does not declare.
- Objects held by module-level bindings (and what they hold) are never typed
  as records: a typed return would copy them.
- Only checks of overlay facts carry the inferred-fact message.
- static-objects ignores nullish receivers outside optional chains; folded
  literal keys are unparenthesized.

Fallback (the overlay never adds a failure the default build lacks):
- the overlay state is kept per load (a later load reset it, so the
  fallback never ran);
- deferred refusals in unreached code count;
- a preflight, deferred refusals or IR the program without the overlay does
  not have make the build compile without the overlay.

CI: the shape_overlay job builds its own native packages; a manual run with
scope shape_overlay runs it alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 09:20:03 -07:00
Chris Tate c2e6c962e8 Build the sanitized three.js nodes fixture in its own CI job
On the san 1/20 shard the sanitized nodes build shares the runner with the
corpus's other two workers and the side queue. It took 944 s and 860 s on
#2194's runs and passed 1200 s on main's 7fa88083, failing the test timeout.

The Linux san corpus now skips that one case
(SCRIPTC_THREE_SANITIZED_NODES_JOB), and a three_nodes_san job runs it alone
with the corpus's LeakSanitizer settings. The required test check waits for
the new job. Every other three.js case, the plain nodes build and the macOS
suites are unchanged.
2026-10-25 06:49:46 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate bac7a80377 Raise slow CI job time limits and prune closed pull request caches (#1994) 2026-10-24 17:33:56 -05:00
Chris Tate f8f3194e0c Merge main 2026-10-24 13:13:52 -05:00
Chris Tate 29a78af70b Merge main 2026-10-24 13:13:07 -05:00
Chris Tate 1559fb1d22 Merge main 2026-10-24 04:03:19 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate 937db54656 Raise the capped Linux test scope to 15 GB of RAM
The sanitized self-hosting native-frontend build needs about 15 GB by
itself: its clang compile of the whole frontend under ASan peaks at
10.5 GB beside a 3.8 GB compile in Node. In the 14 GB scope, with the
shard's other work running beside it, the kernel killed that clang
intermittently, so SC5004 reported clang's warnings and no error (SAN
16/20 and 19/20). The scope now caps RAM at 15 GB, still below the
runner's 16 GB, and may still swap past it.
2026-10-24 01:42:23 -05:00
Chris Tate 68a9dea3fa Give the capped Linux test workers the heap they had before the cap 2026-10-24 00:42:45 -05:00
Chris Tate e4079abd41 Keep swap under the test memory cap and continue past OOM kills 2026-10-23 21:39:03 -05:00
Chris Tate 4a48f466ee Merge the Node island runtime branch and stamp the node packs with 0.4.3 2026-10-23 15:32:55 -05:00
Chris Tate b6a3756e8f Merge main into the Node island runtime 2026-10-23 15:25:48 -05:00
Chris Tate 7cdfe16c9b Cap the Linux test step's memory and spread memory-heavy side files
The Linux test shards run their suites in a transient systemd scope with
MemoryMax=13G and no swap, below the runner's 16 GB. A process tree that
grows past the cap loses its largest process to the kernel, and the job
log names the failing test, where the runner itself ran out of memory
and left no log. Hosts without systemd run the suites directly.

The side-queue balancer also takes a table of memory-heavy files, with
their measured peak. They run whole in the cost-balanced task, and each
goes to the shard with the least memory-heavy work so far, after the
time-balanced files. decl-boundary.test.ts, which peaks at about 1.8 GB
alone and had no time measurement, so vitest's file sharding put it in
shard 11, is the first entry. It now runs in shard 1 (plain) and 6 (san).
2026-10-23 14:24:38 -05:00
Chris Tate 1562d08609 Merge the loader branch, key the build by the libnode pack, and pick musl packs 2026-10-23 13:09:08 -05:00
Chris Tate c0e24cdb3f Embed island modules, bound V8's stack scan in-thread, and keep Node's environment out 2026-10-23 12:52:57 -05:00
Chris Tate 8283eeeea9 Merge main 2026-10-23 12:05:51 -05:00
Malte Ubl ce06a3377d Merge remote-tracking branch 'origin/main' into ci/tsgo-timeout 2026-10-23 09:01:36 -07:00
Chris Tate d22250f830 Raise slow CI job time limits and prune closed pull request caches 2026-10-23 08:05:50 -05:00
Malte Ubl 9b3c7bb520 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into HEAD 2026-10-23 05:56:23 -07:00
Malte Ubl 2494e7c1dc Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/src/backend/native/executable.ts
#	packages/compiler/src/frontend/lowering/lower-calls.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 05:53:42 -07:00
Chris Tate f030e5ad44 Check with tsc-ts 0.0.8 on every platform
tsc-ts 0.0.8's API server serves on the named pipe the Node host
connects over on Windows (tsc-ts --api --pipe), and the native compiler
already starts it over its standard streams there, so tsc-ts is the
default checker on every platform. SCRIPTC_CHECKER=tsgo still selects
tsgo.

The compiler depends on tsc-ts 0.0.8, which the release-age exclusions
admit with its platform packages and @scriptc/threads 0.4.2.

The Windows CLI smoke test runs with both checkers, and
verify-native-cli.mjs --run checks that a distribution's tsc-ts runs on
its platform (tsc-ts.exe on Windows).
2026-10-22 22:36:11 -05:00
Malte UblandClaude Opus 5.5 862d5e836d Give the tsgo/tsc-ts IR parity job 60 minutes
Its release pass alone takes about 15 minutes, so the dev pass is regularly
cancelled at the 30-minute limit on main and on PRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 16:12:37 -07:00
Malte Ubl fb0d7a99a8 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/13-libnode-packs 2026-10-22 14:17:35 -07:00
Malte Ubl 13b00dd78b Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/frontend/lowering/lower-modules.ts
#	tests/harness/differential.test.ts
#	tests/harness/linux-differential.test.ts
#	tests/harness/llvm-differential.test.ts
#	tests/harness/windows-differential.test.ts
2026-10-22 14:10:07 -07:00
Chris Tate 5aae5531e0 Give the sanitized bootstrap seed job swap
The sanitized seed compiles the compiler's 700 MB LLVM module with one clang
process under AddressSanitizer. The compile peaks near 14 GB while the Node
process that emitted the module still holds about 6 GB, which exceeds the
runner's 16 GB of memory. The job now enlarges swap to 24 GB before it builds
the seed, as the sanitized emitter job does.
2026-10-22 14:30:37 -05:00
Malte UblandClaude Opus 5.5 75aa8509f6 Assemble libnode packs with NI-1's build-pack.mjs and run island tests in CI
Stack the libnode packs on the in-process loader (#1685) and use its
pack assembler and manifest format instead of a second one:
build-pack.mjs keeps its CLI and fields (schema, nodeVersion,
moduleVersion, bridgeAbi, platform, arch, files), adds the libnode and
bridge names, strips and (on macOS) re-signs the bridge and links it with
-z defs, and exports buildPack(), which the platform packs call with their
package, target and build inputs as extra manifest fields.

libnodeRecord falls back to the installed @scriptc/libnode-<platform>
pack, found from the entry's directory and then next to the compiler,
when neither --libnode nor SCRIPTC_LIBNODE names one.

The Linux test lanes restore the libnode build that the libnode-packs
workflow cached, assemble the linux-x64 pack from it in seconds
(build.mjs --from-cache never builds Node) and export SCRIPTC_LIBNODE, so
the Node island corpus and harness tests run instead of skipping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 17:52:22 -07:00
Malte UblandClaude Opus 5.5 1081315a49 Run Node on its own stack; provide a libnode pack to the Linux CI test lanes
Calls into Node now run on one dedicated 64 MB stack with a guard page,
reached by a context switch, and the bridge sets V8's stack limit on it
(bridge ABI 2). Before, a call from an async fiber (512 KB stacks without a
guard page) failed to bootstrap Node, which the dev-mode LLVM differential
showed. The runtime's stack guard follows the switch, and the switch
carries ASan's fiber annotations.

The Node island tests load Node from a libnode pack. Until the NI-13 packs
are published, .github/actions/libnode-dev-pack assembles one for the
Linux x64 test lanes (plain and sanitized): a pinned, checksum-verified
prebuilt libnode of the repository's Node release (24.22.0) and Node's
official headers (packages/libnode-common/dev-pack.json), with the bridge
built against them by build-pack.mjs. The pack is cached by its inputs and
the bridge source, so only a cache miss downloads and builds; the normal
build path never does.

tests/harness/libnode.ts makes a missing pack a failure in those lanes:
island tests skip only off Linux x64 CI without a configured pack. The
bridge smoke test (packages/libnode-common/test/bridge-smoke.c) loads the
pack the way scr_node_island.c does and makes synchronous, throwing and
asynchronous calls through the bridge with the runtime's bootstrap.

IR, LLVM and object outputs of programs that call into Node no longer need
a pack at build time; only executables record one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 09:58:32 -07:00
Chris Tate f3c4ba0c20 Run macOS CI jobs nightly and for pull requests that change darwin code
Pushes to main now run every lane except macos, so a merged commit no
longer queues the macOS native build, the darwin test slices and the
macOS LLVM artifact jobs. The nightly sweep and manual runs still run
every lane. Pull requests keep running the macOS lane when
scripts/ci-lanes.mjs selects it.

- The iOS runtime pack builds move from mobile_runtime_packs (linux lane)
  to mobile_runtime_packs_ios (macos lane), since they run on macOS
  runners. The Android pack stays in the linux lane.
- The macos_full dispatch input is replaced by macos_suite (full or
  darwin, default full). Manual runs run the complete macOS suite unless
  macos_suite is darwin.
- ci-lanes.mjs also selects the macos lane for paths naming kqueue, the
  runtime's platform header, and every file of the darwin suite in
  scripts/ci-test-plan.mjs (adding tests/harness/differential.test.ts
  and the darwin runtime pack's archive test).
- The required test check counts a skipped macOS job as passing when the
  macos lane is off, as it already did for pull requests.
2026-10-20 13:43:22 -05:00
Chris Tate ce5e4294ad Merge main 2026-10-20 08:49:25 -05:00
Chris Tate a495dc35a6 Run LeakSanitizer over the sanitized corpus on Linux CI
The Linux test jobs set ASAN_OPTIONS=detect_leaks=0 for both flavors and
left leak detection to the RC audit, which does not see runtime tables
allocated with calloc. A worker thread's record-source table leaked that
way, and only local runs reported it. The san shards' case-sharded corpus
task now sets detect_leaks=1 and an LSAN_OPTIONS suppressions file,
tests/harness/lsan.supp, which starts empty. Plain shards, the side queue,
the Effect jobs and macOS, whose ASan has no LeakSanitizer, keep leak
detection off.
2026-10-20 00:38:58 -05:00