Commit Graph
14 Commits
Author SHA1 Message Date
Chris Tate 3ea231d70d Optimize and generate native code in parallel partitions
Large executables were simplified and inlined as one module before the
optimizer split them, so most of a release build ran on a single core,
and any edit recompiled the whole program.

The LLVM helper now divides the program before optimization. It splits
the emitter's assembly text directly, so partitions also parse
concurrently, and falls back to LLVM's module splitter for any input
outside the emitter's grammar. Internal functions referenced by a single
function stay with that function, and globals stay with their first user.
Optimized partitions follow the ThinLTO model: each is simplified and
summarized on its own thread, a thin link imports small functions across
partitions for inlining and returns unexported hidden definitions to
internal linkage, then each partition is optimized and compiled on its
own thread. Unoptimized partitions compile independently, so dev builds
of large programs are parallel too.

A private per-helper cache directory stores simplified partitions under
the hash of their source and objects under the hash of the module after
importing, so a rebuild recompiles only the partitions an edit can
affect. To keep unchanged partitions identical across edits, each
partition numbers its metadata in first-use order, lists only the debug
compile units it uses, and orders declarations by name, and interned
string constants are named by their content rather than first-use order.

Partition counts are powers of two that depend only on the module size,
up to 64, so output stays the same on every host and functions keep
their partition across ordinary edits. LLVM text that arrives in parts is
written in large batches and handed to the helper in place instead of
being written a second time.
2026-10-09 19:51:47 -05:00
Malte Ubl 60c30d6ebd Import runtime bitcode into speed programs (H02)
Re-lands perf/h02-runtime-lto (6d373317, b62448f3) behind the speed
posture.

- Runtime packs gain an executable `speed` flavor whenever the host LLVM
  helper can run. Its static units are emitted through the helper's
  `runtime-unit` command (promoted unit-local symbols, object and import
  bitcode from one module); its SCR_DYNAMIC variants reference the release
  objects. Release, dev, and library flavors and the vendor archives are
  built exactly as before (the rebuilt macOS pack's flavors and archives
  are identical to the baseline's).
- Speed program builds load the speed flavor's bitcode and pass it to the
  helper (`--import-bitcode`); release builds never do.
  loadRuntimeBitcode only reads a real speed flavor, never release.
- The helper drops the emitter's inert sanitize_address attributes only
  when it imports, so release modules reach the pipeline unchanged.
- Tests: runtime-pack.test.ts covers speed-variant bitcode selection,
  verification, and that release-flavor bitcode is never imported;
  native-codegen.test.ts covers helper arguments and cache keys.

Release executables stay byte-identical to the stack tip with the rebuilt
helper (14/14 workloads, macOS).
2026-10-08 15:46:34 -05:00
Chris Tate e936346215 Parallelize large native builds and adapt cycle collection (#722) 2026-10-07 10:47:31 -05:00
Chris Tate 668eb78124 Optimize native array loops and preserve live tuple iteration
- Reduce allocation and reference-management overhead in native array loops.
- Enable vectorization while preserving numeric behavior.
- Preserve live tuple iteration and cover reference lifetimes and missing values.
2026-10-04 10:38:10 -05:00
Chris Tate 0845c7ff56 Make native installs portable and smaller (#599)
* Make native installs portable and smaller

- Support glibc 2.34 with portable compiler binaries and static LLVM helpers.
- Bundle the host runtime and discover optional cross-target packs from project dependencies.
- Add application build benchmarks covering latency, executable size, and memory.

* Keep bootstrap tools available in Sandbox validation

- Isolate Zig for native compiler bootstrap and glibc entropy contracts.
- Preserve the existing toolchain scope for cache and corpus tests.

* Fix portable compiler bootstrap linking

- Keep native support objects compatible with the destination host linker.
- Resolve installed targets using the host declared by the toolchain manifest.
2026-10-01 10:46:16 -05:00
Chris Tate 3f66c8e20b Make LLVM the sole production backend
- Remove C emission, debugging options, and automatic backend fallback.
- Complete LLVM lowering and native bootstrap coverage across supported targets.
- Link executables and libraries with precompiled C runtime packs, including iOS and Android.
2026-09-29 20:54:09 -05:00
Chris Tate 6700381ff5 fix(release): build portable musl LLVM helpers (#304)
- Build musl helper packages as static host executables matching LLVM’s glibc and libstdc++ ABI.
- Preserve musl code-generation targets and Zig-built runtime packs.
- Add pre-merge Alpine execution and package verification coverage.
2026-09-13 18:23:41 -05:00
Chris Tate c5362177d9 fix(release): use bfd for musl LLVM links (#303) 2026-09-13 15:33:03 -05:00
Chris Tate 0d96facc27 fix(release): use external lld for musl helpers (#302) 2026-09-13 11:36:14 -05:00
Chris Tate ce04d825fa fix(release): flatten musl LLVM link graph (#301)
* fix(release): flatten musl LLVM link graph

* fix(release): order musl LLVM archives safely
2026-09-13 10:47:55 -05:00
Chris Tate 6396598965 fix(release): satisfy musl LLVM dependency targets (#300) 2026-09-13 08:13:16 -05:00
Chris Tate 3339cf9521 fix(release): repair native package CI toolchains (#299) 2026-09-13 01:12:55 -05:00
Chris Tate c41f1a8be9 Expand LLVM native output targets (#280)
* Expand LLVM native output targets

- Add owned helper and runtime-pack contracts for macOS x64, Linux glibc/musl, Windows x64, and WASI.
- Generalize LLVM target selection, package validation, linker plans, and release/CI matrices.
- Document linker boundaries and add focused native-output and runtime-pack coverage.

* Pin Windows LLVM helper CI to VS 2022

The official LLVM archive needs the Visual Studio 2022 CMake generator, which is not guaranteed by windows-latest.

* Align bootstrap cache with LLVM runtime-pack builds

Use the same target-specific helper and linker identity before and after compiler loading so routed executable cache hits remain lightweight.

* Install Windows LLVM CI tools with Chocolatey

windows-2022 includes Visual Studio but not winget; use its available Chocolatey bootstrap for CMake and Ninja.

* Prevent Zig version probes from leaving runtime-pack objects

Run runtime-pack compiler version probes in a private temporary directory and remove the accidental tracked WASI a.o file.

* Use 7-Zip for Windows LLVM setup

Extract the official LLVM development archive with two-stage 7-Zip instead of Windows tar.exe, which timed out while materializing the toolchain tree.

* Define ssize_t for the MSVC runtime pack

Clang's MSVC target does not expose POSIX ssize_t through sys/types.h; define the pointer-sized runtime type without affecting MinGW.

* Build the Windows runtime pack with Zig

Use the MinGW-compatible Windows sysroot required by the runtime while retaining COFF helper output and installing Zig in the Windows native CI lane.

* Give macOS LLVM differential shard time to finish

Shard 1 passed its setup and focused contracts but was cancelled during its cold LLVM differential slice at the 20-minute job limit.
2026-09-01 13:41:34 -05:00
Chris Tate 401338f437 feat(compiler): add native LLVM object emission (#242)
* feat(compiler): add native LLVM object emission

- ship the pinned macOS arm64 LLVM helper and platform package
- add assembly/object CLI outputs, caching, diagnostics, and runtime ABI checks
- validate helper artifacts through CI, packaging, docs, and differential coverage

* fix(compiler): harden native LLVM emission

* fix(compiler): harden LLVM helper packaging

* fix(compiler): stabilize LLVM helper package size

* fix(compiler): harden native helper validation

* fix(compiler): harden LLVM helper verification

* fix(compiler): harden native object emission
2026-08-27 10:30:40 -05:00