mirror of
https://github.com/vercel-labs/scriptc.git
synced 2026-10-02 00:25:34 +08:00
fix(compiler): guide native addon users to C ABI FFI (#484)
- Diagnose native addon requires precisely across local path forms and build modes. - Document and validate migration from Node-API wrappers to statically linked C ABI functions. Fixes #471 Co-authored-by: Hagege Ruben <20857346+HagegeR@users.noreply.github.com>
This commit is contained in:
co-authored by
Hagege Ruben
parent
2d74c03adf
commit
41020ddb39
@@ -2,9 +2,24 @@
|
||||
|
||||
Outbound FFI lets statically compiled TypeScript call C ABI symbols directly. A strict JSON manifest connects a signature-only TypeScript declaration to a native symbol and supplies the archives or objects that resolve it at link time. There is no runtime symbol lookup and no JavaScript engine at the boundary.
|
||||
|
||||
## A complete example
|
||||
## Replacing a Node-API addon
|
||||
|
||||
Declare the native function in TypeScript. The declaration gives the type checker its ordinary source-level signature; it emits no JavaScript body.
|
||||
A Node-API addon exposes JavaScript functions through callbacks that take `napi_env` and `napi_callback_info`. To use its native operation through FFI, extract that operation into a plain C ABI function. Build the function into an object file or static archive, and bind its C signature in the manifest. Keep the Node-API registration and JavaScript value conversion in a separate wrapper if you also publish a Node addon. Renaming a `.node` file or linking an archive that still calls `napi_*` does not remove its dependency on Node. See [native addon limitations](/limitations#native-addons).
|
||||
|
||||
For example, an addon client might call a small numeric helper like this:
|
||||
|
||||
```ts:addon-client.ts
|
||||
import { createRequire } from "node:module";
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const native = require("./native.node") as {
|
||||
scale(value: number): number;
|
||||
};
|
||||
|
||||
console.log(native.scale(21));
|
||||
```
|
||||
|
||||
Replace the addon import and method call with a signature-only TypeScript declaration and a direct call. The declaration gives the type checker its ordinary source-level signature; it emits no JavaScript body.
|
||||
|
||||
```ts:main.ts
|
||||
declare function nativeScale(value: number): number;
|
||||
@@ -135,6 +150,8 @@ For ordinary outbound parameters, string and byte pointers are borrowed only for
|
||||
|
||||
For C++, export the symbol with `extern "C"` so it keeps the manifest's unmangled C name.
|
||||
|
||||
For operating-system helpers, wrap the system call in a fixed-signature C function too. For example, Linux's variadic `prctl` needs a wrapper that supplies its operation and trailing arguments. A wrapper returning C `int32_t` uses the manifest's `i32` return class and a TypeScript `number`; check its error result in TypeScript before using the value. Build the object or archive for the same target as the scriptc binary.
|
||||
|
||||
## Callbacks and context pointers
|
||||
|
||||
Format 2 adds call-scoped C function-pointer parameters. It describes the function pointer and opaque context as independent ABI entries, in their actual positions—matching the model used by C, Rust's `extern "C" fn` plus `*mut c_void`, and Zig's `*const fn (...) callconv(.c)` plus `*anyopaque`.
|
||||
|
||||
@@ -94,6 +94,12 @@ A static-tier program otherwise produces byte-identical stdout and the same exit
|
||||
- **Top-level `await` in embedded ESM packages** is not supported yet. It does compile in your program's own ESM graph and in npm packages compiled through `--npm-static`; the remaining limit is package code running inside the `--dynamic` island.
|
||||
- **`--npm-static` and `--provenance-sources` are experimental** — see [npm Dependencies](/dependencies) for the maturity notes.
|
||||
|
||||
## Native addons
|
||||
|
||||
Node-API (N-API) and V8 `.node` addons require Node's addon runtime, which scriptc does not embed in either static or `--dynamic` builds. Direct `createRequire` calls to local addons are rejected at compile time. Embedded npm packages receive a catchable `ERR_DLOPEN_FAILED` when they attempt to load an addon, allowing packages with JavaScript fallbacks to select them.
|
||||
|
||||
Use [Native FFI](/ffi#replacing-a-node-api-addon) to call the underlying native operation through a plain C ABI function linked from an object or static archive. The guide includes a complete replacement for a small Node-API helper.
|
||||
|
||||
## WASI target limits
|
||||
|
||||
The production <code>wasm32-wasi</code> target supports the complete executable language tier through LLVM: async/await, promises, generators, timers and other portable event-loop work, stdin/readline, filesystem callbacks and promises, and the <code>--dynamic</code> island. Portable WASI Preview 1 has no socket, process-spawn, OS-signal, network-interface, or filesystem-notification capabilities, so networking/fetch, child processes, signal APIs, <code>os.networkInterfaces()</code>, and <code>fs.watch</code> are rejected before linking with <code>SC3002</code>. <code>--sanitize</code>, native FFI, and library-mode archive builds are also unavailable. Filesystem access is bounded by the host's preopens; <code>scriptc run</code> exposes the current working directory and <code>/tmp</code>. See [Platform Support](/platforms) for build and run details.
|
||||
|
||||
@@ -620,6 +620,19 @@ export function noLoweringDiag(
|
||||
};
|
||||
}
|
||||
|
||||
/** A resolved native addon needs Node's embedding ABI, even when the
|
||||
* caller opts into scriptc's dynamic engine. The migration uses the
|
||||
* underlying native operation through an ordinary C ABI declaration. */
|
||||
export function nativeAddonDiag(specifier: string, loc: SrcLoc): ScrDiagnostic {
|
||||
return {
|
||||
code: "SC2020",
|
||||
message: `native addon '${specifier}' requires Node's addon runtime, which is unavailable in static and --dynamic builds`,
|
||||
loc,
|
||||
milestone: "later",
|
||||
hint: "expose the native operation as a plain C ABI function in an object or static archive, then bind a signature-only TypeScript declaration with --ffi; Node-API callbacks themselves require Node (https://scriptc.dev/ffi#replacing-a-node-api-addon)",
|
||||
};
|
||||
}
|
||||
|
||||
/** Records are monomorphic structs, so a value's shape must match the
|
||||
* expected shape exactly OR narrow through the width-copy family
|
||||
* (recordWidthPlan / the overflow capture: target fields copied off the
|
||||
|
||||
@@ -4,7 +4,7 @@ import { InternalCompilerError } from "../../errors.js";
|
||||
* methods), JSON.parse/stringify, process properties/methods and
|
||||
* process.env access, and console.log detection. */
|
||||
import { builtinModules } from "node:module";
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { dirname, isAbsolute, resolve } from "node:path";
|
||||
import * as ts from "../ts7/adapter.js";
|
||||
import type { Lowerer } from "./lowerer.js";
|
||||
import { PoisonError, dynUndefinedExpr, ladderFenceExpr, nodeThrowExpr, own } from "./lowerer.js";
|
||||
@@ -14,7 +14,7 @@ import { probeNodeRequireRefusal } from "../npm.js";
|
||||
import { isNpmStaticPackage } from "../npm-static.js";
|
||||
import { trackedReadFile } from "../input-tracker.js";
|
||||
import { requireResolvePathsRuntime, resolveImportMetaRuntime, resolveRequireRuntime, type RuntimeResolveError, type RuntimeResolveResult } from "../runtime-resolve.js";
|
||||
import { invalidJsonModuleDiag, requiresDynamicImportDiag } from "../../diagnostics/diagnostic.js";
|
||||
import { invalidJsonModuleDiag, nativeAddonDiag, requiresDynamicImportDiag } from "../../diagnostics/diagnostic.js";
|
||||
import {
|
||||
BuiltinModuleFn,
|
||||
builtinModuleFnOf,
|
||||
@@ -651,15 +651,26 @@ function lowerBuiltinOptionalDefault(
|
||||
"imports-field specifiers have no require lowering yet — import the target statically",
|
||||
);
|
||||
}
|
||||
if (isRelativeSpecifier(spec) || spec.startsWith("/")) {
|
||||
if (isRelativeSpecifier(spec) || isAbsolute(spec)) {
|
||||
if (!spec.endsWith(".json")) {
|
||||
// Only refine the existing refusal path: Node resolution detects
|
||||
// extensionless addons and directory entries without executing
|
||||
// them, and avoids mistaking a .node-named JS directory for one.
|
||||
// This probe marks frontend inputs unstable, but every branch
|
||||
// here refuses compilation, so successful-build caches keep their
|
||||
// existing dependency proofs.
|
||||
const resolved = resolveRequireRuntime(cr.baseFile.fileName, spec, lowerer.targetPlatform);
|
||||
if (resolved.ok && resolved.value.endsWith(".node")) {
|
||||
lowerer.pushDiag(nativeAddonDiag(spec, loc));
|
||||
throw new PoisonError();
|
||||
}
|
||||
lowerer.noLowering(
|
||||
`createRequire's require of '${spec}'`,
|
||||
call,
|
||||
"relative program modules lower when they resolve into the compiled graph; relative .json documents bake at build time",
|
||||
);
|
||||
}
|
||||
const abs = spec.startsWith("/")
|
||||
const abs = isAbsolute(spec)
|
||||
? spec
|
||||
: resolve(dirname(cr.baseFile.fileName), spec);
|
||||
const text = trackedReadFile(abs);
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
* the module artifacts (globals, embedded npm tables) the IR module carries. */
|
||||
import * as ts from "../ts7/adapter.js";
|
||||
import type { Lowerer } from "./lowerer.js";
|
||||
import { dirname as dirnamePath, resolve as resolvePath } from "node:path";
|
||||
import { dirname as dirnamePath, isAbsolute, resolve as resolvePath } from "node:path";
|
||||
import { NpmGraphBuilder, packageNameOfPath, probeNodeImportRefusal, probeNodeRequireRefusal } from "../npm.js";
|
||||
import { isNpmStaticPackage } from "../npm-static.js";
|
||||
import { isJsSourceFileName } from "../tsc-codes.js";
|
||||
@@ -549,7 +549,7 @@ export function appendForkModules(
|
||||
spec !== null &&
|
||||
canonicalBuiltinModule(spec) === null &&
|
||||
!isRelativeSpecifier(spec) &&
|
||||
!spec.startsWith("/") &&
|
||||
!isAbsolute(spec) &&
|
||||
!spec.startsWith("#") &&
|
||||
probeNodeRequireRefusal(cr.baseFile.fileName, spec) === null
|
||||
) {
|
||||
|
||||
@@ -490,7 +490,10 @@ function createRequireProgramRoots7(program: ts.Program): string[] {
|
||||
target = npm.typesFile;
|
||||
}
|
||||
}
|
||||
if (target === null || target.endsWith(".json")) return "skip";
|
||||
// A directory's package.json may point at a native addon. Keep it
|
||||
// out of TypeScript's source roots so the require call can report
|
||||
// the native-addon boundary instead of an unsupported-file error.
|
||||
if (target === null || target.endsWith(".json") || target.endsWith(".node")) return "skip";
|
||||
const normalized = tsgoPath(resolve(target));
|
||||
if (!known.has(normalized)) roots.add(normalized);
|
||||
return "skip";
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, expect, test } from "vitest";
|
||||
import { compile } from "../src/index.js";
|
||||
|
||||
const dirs: string[] = [];
|
||||
afterEach(() => {
|
||||
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function fixture() {
|
||||
const dir = mkdtempSync(join(tmpdir(), "scriptc-native-addon-"));
|
||||
dirs.push(dir);
|
||||
mkdirSync(join(dir, "native"));
|
||||
writeFileSync(join(dir, "native", "addon.node"), "fixture only; never execute native addons during compilation\n");
|
||||
writeFileSync(join(dir, "native", "package.json"), JSON.stringify({ main: "addon.node" }));
|
||||
return dir;
|
||||
}
|
||||
|
||||
for (const dynamic of [false, true]) {
|
||||
test.for(["relative", "extensionless", "directory", "absolute", "inline"] as const)(
|
||||
`createRequire explains native addon migration: %s (dynamic=${dynamic})`,
|
||||
async (form) => {
|
||||
const dir = fixture();
|
||||
const spec = form === "absolute" ? join(dir, "native", "addon.node")
|
||||
: form === "extensionless" ? "./native/addon"
|
||||
: form === "directory" ? "./native" : "./native/addon.node";
|
||||
const call = form === "inline"
|
||||
? `createRequire(import.meta.url)(${JSON.stringify(spec)})`
|
||||
: `require(${JSON.stringify(spec)})`;
|
||||
const source = [
|
||||
'import { createRequire } from "node:module";',
|
||||
'const require = createRequire(import.meta.url);',
|
||||
`const native = ${call} as { value: number };`,
|
||||
"console.log(native.value);",
|
||||
].join("\n");
|
||||
const entry = join(dir, "main.mts");
|
||||
writeFileSync(entry, source);
|
||||
const result = await compile(entry, {
|
||||
dynamic, outputKind: "ir", outDir: dir, outPath: join(dir, "out.json"),
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
const diagnostic = result.diagnostics.find((diag) => diag.code === "SC2020" && diag.message.includes("native addon"));
|
||||
expect(diagnostic, JSON.stringify(result.diagnostics)).toBeDefined();
|
||||
expect(diagnostic?.message).toContain(spec);
|
||||
expect(diagnostic?.hint).toContain("--ffi");
|
||||
expect(diagnostic?.hint).toContain("C ABI");
|
||||
expect(diagnostic?.hint).toContain("https://scriptc.dev/ffi#replacing-a-node-api-addon");
|
||||
expect(source.slice(diagnostic?.loc.start, diagnostic?.loc.end)).toBe(call);
|
||||
},
|
||||
);
|
||||
|
||||
test(`a JavaScript directory named .node is still a program module (dynamic=${dynamic})`, async () => {
|
||||
const dir = fixture();
|
||||
mkdirSync(join(dir, "javascript.node"));
|
||||
writeFileSync(join(dir, "javascript.node", "index.js"), "exports.value = 42;\n");
|
||||
const entry = join(dir, "main.mts");
|
||||
writeFileSync(entry, [
|
||||
'import { createRequire } from "node:module";',
|
||||
'const require = createRequire(import.meta.url);',
|
||||
'const javascript = require("./javascript.node") as { value: number };',
|
||||
'console.log(javascript.value);',
|
||||
].join("\n"));
|
||||
const result = await compile(entry, {
|
||||
dynamic, outputKind: "ir", outDir: dir, outPath: join(dir, "out.json"),
|
||||
});
|
||||
expect(result.ok, result.ok ? "" : JSON.stringify(result.diagnostics)).toBe(true);
|
||||
});
|
||||
}
|
||||
|
||||
test("a missing .node path keeps the unresolved-module diagnostic", async () => {
|
||||
const dir = fixture();
|
||||
const entry = join(dir, "main.mts");
|
||||
writeFileSync(entry, [
|
||||
'import { createRequire } from "node:module";',
|
||||
'const require = createRequire(import.meta.url);',
|
||||
'require("./missing.node");',
|
||||
].join("\n"));
|
||||
const result = await compile(entry, { outputKind: "ir", outDir: dir, outPath: join(dir, "out.json") });
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
expect(result.diagnostics.some((diag) => diag.code === "SC2020" && diag.message.includes("./missing.node"))).toBe(true);
|
||||
expect(result.diagnostics.some((diag) => diag.message.includes("native addon"))).toBe(false);
|
||||
});
|
||||
Reference in New Issue
Block a user