Commit Graph
115 Commits
Author SHA1 Message Date
Chris Tate c9116749c1 Merge ni/3-hosted-loop 2026-10-25 20:18:43 -05:00
Chris Tate c3ac3f6882 Merge main 2026-10-25 18:52:38 -05:00
Chris Tate 27954618c7 Merge base branch 2026-10-25 16:31:56 -05:00
Chris Tate c2e6c962e8 Build the sanitized three.js nodes fixture in its own CI job
On the san 1/20 shard the sanitized nodes build shares the runner with the
corpus's other two workers and the side queue. It took 944 s and 860 s on
#2194's runs and passed 1200 s on main's 7fa88083, failing the test timeout.

The Linux san corpus now skips that one case
(SCRIPTC_THREE_SANITIZED_NODES_JOB), and a three_nodes_san job runs it alone
with the corpus's LeakSanitizer settings. The required test check waits for
the new job. Every other three.js case, the plain nodes build and the macOS
suites are unchanged.
2026-10-25 06:49:46 -05:00
Chris Tate e6790def72 Merge main 2026-10-24 20:06:31 -05:00
Chris Tate e0f1de0a2b Merge main 2026-10-24 20:00:43 -05:00
Chris Tate 8b4f83ac02 Collect the native compiler profile on the release commit (#2138) 2026-10-24 18:25:03 -05:00
Chris Tate bac7a80377 Raise slow CI job time limits and prune closed pull request caches (#1994) 2026-10-24 17:33:56 -05:00
Chris Tate e5d386add7 List caches and open pull requests in full before pruning 2026-10-24 15:27:31 -05:00
Chris Tate 40faf6a9f1 Collect the native compiler profile on the release commit
The release workflow's new cli-profile job runs
scripts/refresh-cli-profile.mjs on Linux x64 and uploads the profile.
Every platform's native CLI build uses it through
SCRIPTC_NATIVE_PROFILE_USE, and the GitHub release attaches it as
scriptc-<version>-cli.profdata.gz. When the job fails, the CLI builds
use the committed profile.

build-native-cli.mts accepts a gzip-compressed profile in
SCRIPTC_NATIVE_PROFILE_USE when the name ends in .gz.
refresh-cli-profile.mjs creates the directory of --output before it
starts.
2026-10-24 14:16:32 -05:00
Chris Tate 29a78af70b Merge main 2026-10-24 13:13:07 -05:00
Chris Tate bf2f1c45d7 Merge main 2026-10-24 01:52:57 -05:00
Chris Tate 937db54656 Raise the capped Linux test scope to 15 GB of RAM
The sanitized self-hosting native-frontend build needs about 15 GB by
itself: its clang compile of the whole frontend under ASan peaks at
10.5 GB beside a 3.8 GB compile in Node. In the 14 GB scope, with the
shard's other work running beside it, the kernel killed that clang
intermittently, so SC5004 reported clang's warnings and no error (SAN
16/20 and 19/20). The scope now caps RAM at 15 GB, still below the
runner's 16 GB, and may still swap past it.
2026-10-24 01:42:23 -05:00
Chris Tate 68a9dea3fa Give the capped Linux test workers the heap they had before the cap 2026-10-24 00:42:45 -05:00
Chris Tate e4079abd41 Keep swap under the test memory cap and continue past OOM kills 2026-10-23 21:39:03 -05:00
Chris Tate b6a3756e8f Merge main into the Node island runtime 2026-10-23 15:25:48 -05:00
Chris Tate 7cdfe16c9b Cap the Linux test step's memory and spread memory-heavy side files
The Linux test shards run their suites in a transient systemd scope with
MemoryMax=13G and no swap, below the runner's 16 GB. A process tree that
grows past the cap loses its largest process to the kernel, and the job
log names the failing test, where the runner itself ran out of memory
and left no log. Hosts without systemd run the suites directly.

The side-queue balancer also takes a table of memory-heavy files, with
their measured peak. They run whole in the cost-balanced task, and each
goes to the shard with the least memory-heavy work so far, after the
time-balanced files. decl-boundary.test.ts, which peaks at about 1.8 GB
alone and had no time measurement, so vitest's file sharding put it in
shard 11, is the first entry. It now runs in shard 1 (plain) and 6 (san).
2026-10-23 14:24:38 -05:00
Chris Tate c0e24cdb3f Embed island modules, bound V8's stack scan in-thread, and keep Node's environment out 2026-10-23 12:52:57 -05:00
Chris Tate 8283eeeea9 Merge main 2026-10-23 12:05:51 -05:00
Malte Ubl ce06a3377d Merge remote-tracking branch 'origin/main' into ci/tsgo-timeout 2026-10-23 09:01:36 -07:00
Chris Tate d22250f830 Raise slow CI job time limits and prune closed pull request caches 2026-10-23 08:05:50 -05:00
Malte Ubl 2acfd9e723 Merge remote-tracking branch 'origin/ni/3-hosted-loop' into HEAD
# Conflicts:
#	packages/compiler/src/diagnostics/diagnostic.ts
#	packages/compiler/src/frontend/pipeline.ts
2026-10-23 05:57:22 -07:00
Malte Ubl 2494e7c1dc Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	packages/compiler/src/backend/native/executable.ts
#	packages/compiler/src/frontend/lowering/lower-calls.ts
#	packages/compiler/src/frontend/lowering/lowerer.ts
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
2026-10-23 05:53:42 -07:00
Chris Tate f030e5ad44 Check with tsc-ts 0.0.8 on every platform
tsc-ts 0.0.8's API server serves on the named pipe the Node host
connects over on Windows (tsc-ts --api --pipe), and the native compiler
already starts it over its standard streams there, so tsc-ts is the
default checker on every platform. SCRIPTC_CHECKER=tsgo still selects
tsgo.

The compiler depends on tsc-ts 0.0.8, which the release-age exclusions
admit with its platform packages and @scriptc/threads 0.4.2.

The Windows CLI smoke test runs with both checkers, and
verify-native-cli.mjs --run checks that a distribution's tsc-ts runs on
its platform (tsc-ts.exe on Windows).
2026-10-22 22:36:11 -05:00
Malte UblandClaude Opus 5.5 862d5e836d Give the tsgo/tsc-ts IR parity job 60 minutes
Its release pass alone takes about 15 minutes, so the dev pass is regularly
cancelled at the 30-minute limit on main and on PRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 16:12:37 -07:00
Malte Ubl fb0d7a99a8 Merge remote-tracking branch 'origin/ni/1-libnode-loader' into ni/13-libnode-packs 2026-10-22 14:17:35 -07:00
Malte Ubl 13b00dd78b Merge remote-tracking branch 'origin/main' into ni/1-libnode-loader
# Conflicts:
#	packages/compiler/src/frontend/lowering/lower-modules.ts
#	tests/harness/differential.test.ts
#	tests/harness/linux-differential.test.ts
#	tests/harness/llvm-differential.test.ts
#	tests/harness/windows-differential.test.ts
2026-10-22 14:10:07 -07:00
Chris Tate 5aae5531e0 Give the sanitized bootstrap seed job swap
The sanitized seed compiles the compiler's 700 MB LLVM module with one clang
process under AddressSanitizer. The compile peaks near 14 GB while the Node
process that emitted the module still holds about 6 GB, which exceeds the
runner's 16 GB of memory. The job now enlarges swap to 24 GB before it builds
the seed, as the sanitized emitter job does.
2026-10-22 14:30:37 -05:00
Malte UblandClaude Opus 5.5 75aa8509f6 Assemble libnode packs with NI-1's build-pack.mjs and run island tests in CI
Stack the libnode packs on the in-process loader (#1685) and use its
pack assembler and manifest format instead of a second one:
build-pack.mjs keeps its CLI and fields (schema, nodeVersion,
moduleVersion, bridgeAbi, platform, arch, files), adds the libnode and
bridge names, strips and (on macOS) re-signs the bridge and links it with
-z defs, and exports buildPack(), which the platform packs call with their
package, target and build inputs as extra manifest fields.

libnodeRecord falls back to the installed @scriptc/libnode-<platform>
pack, found from the entry's directory and then next to the compiler,
when neither --libnode nor SCRIPTC_LIBNODE names one.

The Linux test lanes restore the libnode build that the libnode-packs
workflow cached, assemble the linux-x64 pack from it in seconds
(build.mjs --from-cache never builds Node) and export SCRIPTC_LIBNODE, so
the Node island corpus and harness tests run instead of skipping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 17:52:22 -07:00
Malte UblandClaude Opus 5.5 e9c6b68c07 Build prebuilt libnode packs for Node islands
Add the @scriptc/libnode-<platform> packs (linux-x64-gnu, linux-arm64-gnu,
darwin-arm64, darwin-x64): the pinned Node.js 24.22.0 built as a stripped
shared library with full ICU and its startup snapshot, the scriptc bridge
compiled against that release's headers, Node's LICENSE and a manifest
with the Node version, NODE_MODULE_VERSION, bridge ABI and file hashes.

packages/libnode-common holds the pin, the build (fetch and verify the
source tarball, configure --shared, build only the libnode target, strip,
stage with headers in a cache keyed by release, flags, platform and
toolchain; the bridge rebuilds in seconds against the cached headers),
a verifier, and a smoke test: a C program that loads a pack the way the
runtime does, starts Node from the snapshot and evaluates expressions
synchronously, asynchronously and throwing.

The libnode-packs workflow builds the packs on native runners, Linux in
the manylinux_2_28 images for a glibc 2.28 floor, caches each libnode
build, smoke-tests and uploads every pack. It never runs in the ordinary
build or test path. The compiler gains the pack lookup (SCRIPTC_LIBNODE,
then the installed package), validation, the nodeIslands.libnode
placements and the license notices for the Node island loader to use.

The bridge source and its C ABI header are the NI-1 bridge agent's,
included here so the packs build; the packs stay private until the
in-process island runtime ships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 17:52:22 -07:00
Malte UblandClaude Opus 5.5 51e9812493 Call into Node.js from compiled programs with @scriptc/node
Scope-B milestone NI-1, API side: the explicit call API that replaces the
build container's hand-written node subprocess (api patch 09).

- The new package @scriptc/node exports callSync(specifier, name, ...args)
  and call(...). The module loads with require() from the calling module;
  arguments and results cross as JSON copies; a thrown Error crosses as its
  nearest intrinsic class with its message and string code, other thrown
  values as JSON copies; callSync refuses a function that returns a
  promise. Its Node implementation runs the calls in-process with the same
  copies, so a program prints the same under Node and compiled.
- The compiler lowers calls to the named or namespace imports
  (node-api-import.ts, lower-node-island.ts) to the nodeIsland.callSync
  and nodeIsland.call libCalls: the calling module's real path, the
  specifier, the export name and the boxed argument array, with the result
  checked against the declared type like `JSON.parse(s) as T`. Arguments
  and results whose types are not JSON data are refused at build time, and
  the functions and the namespace refuse as values.
- Corpus programs 6800-6803 (sync and async results, errors, process.exit
  from Node with the program's exit listeners, Node's pending timers
  keeping the program alive) run under Node and as binaries that load
  libnode; `// @requires-libnode` skips them where SCRIPTC_LIBNODE names no
  pack (until the pack CI matrix publishes packs). The node-island harness
  covers loading Node only on the first call, the beside packaging, a
  missing pack, a pack of another Node version or bridge ABI, and the
  executable's symbols.
- Release tooling publishes @scriptc/node with the other packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 09:59:55 -07:00
Malte UblandClaude Opus 5.5 1081315a49 Run Node on its own stack; provide a libnode pack to the Linux CI test lanes
Calls into Node now run on one dedicated 64 MB stack with a guard page,
reached by a context switch, and the bridge sets V8's stack limit on it
(bridge ABI 2). Before, a call from an async fiber (512 KB stacks without a
guard page) failed to bootstrap Node, which the dev-mode LLVM differential
showed. The runtime's stack guard follows the switch, and the switch
carries ASan's fiber annotations.

The Node island tests load Node from a libnode pack. Until the NI-13 packs
are published, .github/actions/libnode-dev-pack assembles one for the
Linux x64 test lanes (plain and sanitized): a pinned, checksum-verified
prebuilt libnode of the repository's Node release (24.22.0) and Node's
official headers (packages/libnode-common/dev-pack.json), with the bridge
built against them by build-pack.mjs. The pack is cached by its inputs and
the bridge source, so only a cache miss downloads and builds; the normal
build path never does.

tests/harness/libnode.ts makes a missing pack a failure in those lanes:
island tests skip only off Linux x64 CI without a configured pack. The
bridge smoke test (packages/libnode-common/test/bridge-smoke.c) loads the
pack the way scr_node_island.c does and makes synchronous, throwing and
asynchronous calls through the bridge with the runtime's bootstrap.

IR, LLVM and object outputs of programs that call into Node no longer need
a pack at build time; only executables record one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 09:58:32 -07:00
Chris Tate faf3ced3f7 Install tsc-ts's platform packages for every release target
The darwin-arm64, linux-x64-musl and linux-arm64-musl CLI builds failed with
"tsc-ts checker: ... has no @tsc-ts/<platform> executable":

- The lockfile had no entry for @tsc-ts/darwin-arm64@0.0.5, so the
  darwin-arm64 install skipped it. The lockfile now lists all seven
  platform packages.
- A CLI ships the tsc-ts executable of its target. The musl builds run on
  glibc runners, where pnpm installs only the glibc packages. Those jobs now
  add musl to pnpm's supported libc before installing.
2026-10-20 16:50:41 -05:00
Chris Tate f3c4ba0c20 Run macOS CI jobs nightly and for pull requests that change darwin code
Pushes to main now run every lane except macos, so a merged commit no
longer queues the macOS native build, the darwin test slices and the
macOS LLVM artifact jobs. The nightly sweep and manual runs still run
every lane. Pull requests keep running the macOS lane when
scripts/ci-lanes.mjs selects it.

- The iOS runtime pack builds move from mobile_runtime_packs (linux lane)
  to mobile_runtime_packs_ios (macos lane), since they run on macOS
  runners. The Android pack stays in the linux lane.
- The macos_full dispatch input is replaced by macos_suite (full or
  darwin, default full). Manual runs run the complete macOS suite unless
  macos_suite is darwin.
- ci-lanes.mjs also selects the macos lane for paths naming kqueue, the
  runtime's platform header, and every file of the darwin suite in
  scripts/ci-test-plan.mjs (adding tests/harness/differential.test.ts
  and the darwin runtime pack's archive test).
- The required test check counts a skipped macOS job as passing when the
  macos lane is off, as it already did for pull requests.
2026-10-20 13:43:22 -05:00
Chris Tate ce5e4294ad Merge main 2026-10-20 08:49:25 -05:00
Chris Tate a495dc35a6 Run LeakSanitizer over the sanitized corpus on Linux CI
The Linux test jobs set ASAN_OPTIONS=detect_leaks=0 for both flavors and
left leak detection to the RC audit, which does not see runtime tables
allocated with calloc. A worker thread's record-source table leaked that
way, and only local runs reported it. The san shards' case-sharded corpus
task now sets detect_leaks=1 and an LSAN_OPTIONS suppressions file,
tests/harness/lsan.supp, which starts empty. Plain shards, the side queue,
the Effect jobs and macOS, whose ASan has no LeakSanitizer, keep leak
detection off.
2026-10-20 00:38:58 -05:00
Chris Tate 6c07441c24 Check programs with tsc-ts by default
scriptc's frontend talks to a TypeScript 7 API server. tsc-ts serves the
same API as tsgo, and it is now the default checker; SCRIPTC_CHECKER=tsgo
selects tsgo, which scriptc still includes.

- @scriptc/compiler depends on tsc-ts 0.0.3. The native distribution
  ships its platform package as lib/tsc-ts, named by a new tsc_ts field
  of the toolchain manifest; a manifest without it takes the executable
  of an installed tsc-ts package.
- The executable cache key includes the tsc-ts executable's identity, in
  the compiler and in the CLI's startup route.
- tsc-ts keeps the syntax trees it builds for the life of its process,
  so the source parser starts a new server after 256 KB of parsed
  source instead of letting one server grow with every file.
- A CI job compiles every corpus program with both checkers, in release
  and dev, and requires the same LLVM IR and diagnostics
  (scripts/checker-identity.mjs).
- Tests that compare with the TypeScript 7 SDK, the checker-panic
  diagnostics, and the protocol tests run against tsgo.
2026-10-18 19:55:52 -05:00
Chris Tate 870d3153ae CI: run the darwin suite on macOS for pushes to main; the full suite nightly and on demand
Since pushes to main run to completion, each ran the complete suite on macOS:
20 test jobs and about 350 macOS job-minutes per push (run 38831529977), which
at about four pushes an hour exceeded the repository's macOS runners. On
2026-10-16 at 02:40 UTC 41 macOS jobs were running, 95 were queued (the oldest
for 35 minutes), and main runs took over an hour.

Pushes to main now run the darwin suite on macOS, as pull requests do: 6 jobs
and about 100 minutes (run 38835202089). The nightly schedule still runs the
complete suite, and a manual run (workflow_dispatch) with macos_full set runs
it on demand, for example on a release candidate commit. Manual runs get a
concurrency group of their own, so they neither cancel nor wait for the push
run of the same commit. The policy test evaluates the macOS matrix expressions
for each event.
2026-10-15 21:54:04 -05:00
Chris Tate 97eb372fe1 CI: run the macOS, Effect and bootstrap lanes on pull requests only when their areas change (#1071) 2026-10-15 21:23:10 -05:00
Chris Tate 5f954cfbb0 CI: run the macOS, Effect and bootstrap lanes on pull requests only when their areas change 2026-10-15 20:27:04 -05:00
Chris Tate 0ae0e9e3b9 Stop the vitest RPC timeout from failing green jobs, and run every push to main to completion 2026-10-15 19:16:04 -05:00
Chris Tate 8fded2ce9a Allow 45 minutes for native release builds 2026-10-15 12:47:20 -05:00
Chris Tate 090ccabb72 Add LifetimeRegion to @scriptc/experimental 2026-10-13 21:12:20 -05:00
Chris Tate 15ad4b8bc3 Keep expected aborts fast on Ubuntu runners 2026-10-12 19:00:42 -05:00
Chris Tate 644722e696 Run the PR test shards on Linux 2026-10-12 15:21:39 -05:00
Malte Ubl 15af413acd Merge branch 'main' into feat/contract-types 2026-10-11 16:32:00 -07:00
Chris Tate 3bf0d0b9a5 Give the sanitized emitter job swap and a larger stack
The sanitized native emitter peaks near 13 GB while it emits the
compiler and then compiles the 348 MB module with clang under
AddressSanitizer, which together exceed the 16 GB runner. Back the
rebuild-emit job with a 24 GiB swap file and lift its stack limit,
since the instrumented emitter's recursion can exceed 8 MiB.
2026-10-11 16:45:12 -05:00
Malte Ubl 5d4509702f wip: contract types prototype 2026-10-11 11:17:17 -07:00
Malte Ubl 272aef3247 Release @scriptc/threads with the other packages
sync-versions stamps packages/threads with the CLI version (0.2.7 now),
the release workflow checks that version and publishes the package
before scriptc, and RELEASING.md lists it with the packages that always
release together. npm trusted publishing must be configured for
@scriptc/threads before the next release: a package without it fails
before upload.
2026-10-10 01:00:36 -05:00
Chris TateandChris Tate 08f4140293 Harden stacked CI scope and merge validation (#771)
Co-authored-by: Chris Tate <366502+ctate@users.noreply.github.com>
2026-10-08 14:57:21 -05:00