Make native installs portable and smaller (#599)

* Make native installs portable and smaller

- Support glibc 2.34 with portable compiler binaries and static LLVM helpers.
- Bundle the host runtime and discover optional cross-target packs from project dependencies.
- Add application build benchmarks covering latency, executable size, and memory.

* Keep bootstrap tools available in Sandbox validation

- Isolate Zig for native compiler bootstrap and glibc entropy contracts.
- Preserve the existing toolchain scope for cache and corpus tests.

* Fix portable compiler bootstrap linking

- Keep native support objects compatible with the destination host linker.
- Resolve installed targets using the host declared by the toolchain manifest.

tests/harness/README.md

@@ -71,7 +71,9 @@ Full-suite runs (`vitest run` with no filters) take an advisory machine-wide loc
 
 ### Development build benchmark
 
-After rebuilding the workspace, `pnpm bench:builds` measures the built CLI on a generated 17-module, 256-function TypeScript program using `--optimization=dev`. It reports one build with an empty scriptc cache, five exact rebuilds, and five rebuilds after changing an imported module. Every run uses a fresh private cache and verifies each binary's stdout, stderr, and exit status against Node, outside the timed build. Temporary files are removed on completion. `pnpm bench:builds --iterations=3` changes the sample count; progress goes to stderr and the JSON result, including individual samples and medians, goes to stdout (use `pnpm --silent bench:builds` when capturing JSON).
+After rebuilding the workspace, `pnpm bench:builds` measures the built CLI on a generated 17-module, 256-function TypeScript program using `--optimization=dev`. It reports five builds with an empty scriptc cache, five exact rebuilds, and five rebuilds after changing an imported module. Every invocation uses a private cache and verifies each binary's stdout, stderr, and exit status against Node, outside the timed build. Temporary files are removed on completion. `pnpm bench:builds --iterations=3` changes the sample count; progress goes to stderr and the JSON result, including individual samples and medians, goes to stdout (use `pnpm --silent bench:builds` when capturing JSON).
+
+`--workload=log-summary` exercises a five-module request-log processor; `--workload=inventory-report` exercises a fifteen-module inventory report with parsing, validation, filtering, aggregation, and ranking. Both process 10,000 records and change an imported value that appears in the output for each edit. `--compiler=/path/to/scriptc` measures an installed native compiler. Results also include executable size and median end-to-end run time, including startup and application work. `--memory` adds peak process RSS from `/usr/bin/time` on macOS and Linux; this is not the sum of concurrent processes' memory. `--launches=20` controls the measured runs after three warmups. Empty-cache builds clear scriptc's cache, not the operating system's file cache.
 
 This is a local latency benchmark, not a timing assertion in CI. Compare the same Node version, target, compiler installation, and machine load; the empty-cache sample does not flush OS filesystem or Node bytecode caches. Unset `SCRIPTC_TARGET` because the benchmark executes the resulting host binary. Use representative application measurements alongside this small module-graph baseline when choosing further optimizations.
 

tests/harness/native-cli-packaging.test.ts

@@ -76,7 +76,8 @@ test.each([true, false])("verify and archive a native distribution across packag
   const manifest = JSON.parse(readFileSync(join(unpacked, "bin", f.executable + ".json"), "utf8"));
   expect(manifest.ts7).toBe(f.manifest.ts7);
   const runtime = manifest.runtime_packs[0];
-  expect(JSON.parse(readFileSync(join(unpacked, "bin", runtime.path, "runtime-pack.json"), "utf8")).target.name).toBe(f.manifest.target);
+  expect(JSON.parse(readFileSync(join(unpacked, "bin", runtime.path.replaceAll("\\", "/"), "runtime-pack.json"), "utf8")).target.name).toBe(f.manifest.target);
+  expect(manifest.runtime_packs).toEqual(f.manifest.runtime_packs);
   expect(readFileSync(f.manifestPath)).toEqual(original);
 });
 

tests/harness/self-hosting-native-driver.test.ts

@@ -1,5 +1,5 @@
 import { execFile, spawnSync } from "node:child_process";
-import { chmodSync, existsSync, mkdtempSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
+import { chmodSync, cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, renameSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
 import { tmpdir } from "node:os";
 import { delimiter, join, resolve } from "node:path";
 import { promisify } from "node:util";
@@ -45,6 +45,15 @@ test("the production CLI relocates, builds programs, and rebuilds itself with No
         // before the sanitizer can exercise the compiler.
         ...options, env: { ...process.env, SCRIPTC_SAN: sanitize ? "1" : "", SCRIPTC_NATIVE_EMIT_IR: "1" },
       }));
+    // CI reuses this already-built seed for npm and older-libc installation
+    // smoke tests, avoiding another full compiler build on the critical path.
+    const packageDirectory = process.env["SCRIPTC_BOOTSTRAP_PACKAGE_DIR"];
+    if (!sanitize && packageDirectory !== undefined) {
+      if (process.platform !== "linux" || process.arch !== "x64") throw new Error("bootstrap package export requires the Linux x64 GNU CI host");
+      mkdirSync(packageDirectory, { recursive: true });
+      cpSync(join(root, "packages/cli-linux-x64-gnu/package.json"), join(packageDirectory, "package.json"));
+      cpSync(distribution, join(packageDirectory, "dist"), { recursive: true });
+    }
     // All compiler assets must survive moving the complete distribution.
     const relocated = join(directory, "relocated");
     renameSync(distribution, relocated);
@@ -162,9 +171,14 @@ test("the production CLI relocates, builds programs, and rebuilds itself with No
       const linker = join(directory, "zigcc");
       writeFileSync(linker, `#!/bin/sh\nexec '${absoluteCommand("zig").replaceAll("'", "'\\''")}' cc "$@"\n`, { mode: 0o755 });
       const output = join(directory, "three.wasm");
+      // Install the cross-target pack after relocating the native compiler.
+      // Its resolver must discover project dependencies without Node on PATH.
+      const scope = join(directory, "node_modules/@scriptc");
+      mkdirSync(scope, { recursive: true });
+      symlinkSync(join(root, "packages/runtime-wasm32-wasi"), join(scope, "runtime-wasm32-wasi"), "dir");
       const built = await exec(probe, ["build", "--lib", "--profile", threeProfile, "-o", output], {
-        ...nativeOptions, env: { ...nativeOptions.env, SCRIPTC_TARGET: "wasm32-wasi", SCRIPTC_LINKER: linker,
-          SCRIPTC_RUNTIME_PACK: join(root, "packages/runtime-wasm32-wasi"), SCRIPTC_NO_CACHE: "1" },
+        ...nativeOptions, cwd: directory, env: { ...nativeOptions.env, SCRIPTC_TARGET: "wasm32-wasi", SCRIPTC_LINKER: linker,
+          SCRIPTC_RUNTIME_PACK: undefined, SCRIPTC_NO_CACHE: "1" },
       });
       expect(built.stdout.trim()).toBe(output);
       expect(comparableStderr(built.stderr)).toBe("");