trending/scriptc
1
0
Fork 0
mirror of https://github.com/vercel-labs/scriptc.git synced 2026-10-28 06:53:42 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into opt/bc-satisfies-key-order

Browse Source 
# Conflicts:
#	packages/compiler/src/ir/traverse.ts
This commit is contained in:
Malte Ubl
2026-10-21 07:57:00 -07:00
parent 3b4ddd4c8d 0237d70df0
commit d3d39af3fe
466 changed files with 20468 additions and 92853 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+6 -5
View File
@@ -4,7 +4,7 @@ This is the only agent guide for the repository. Its rules apply everywhere, inc
## Markdown style
Write prose paragraphs and list items as single source lines and let editors and renderers wrap them naturally. Do not hard-wrap Markdown at a fixed column; preserve separate lines only where Markdown syntax or intentional formatting requires them, such as headings, lists, tables, blockquotes, and code blocks.
Write prose paragraphs and list items as single source lines and let editors and renderers wrap them naturally. Do not hard-wrap Markdown at a fixed column; preserve separate lines only where Markdown syntax or intentional formatting requires them, such as headings, lists, tables, blockquotes, and code blocks. The Limitations page partials under `docs/content/limitations/` are the exception: they keep one sentence per line (see Limitations page partials).
## Build and test
@@ -140,10 +140,11 @@ Never hand-edit these files:
- `packages/compiler/surface-manifest.json`
- `internal/compatibility/generated/node-v24-internal.json`
- `internal/compatibility/generated/node-v24-backlog.json`
- `docs/src/generated/node-v24-compatibility.json`
Change compiler decision sources or the implementation-owned compatibility manifests, then regenerate. Use `pnpm manifest` when compiler decision tables change and `pnpm node-compat` whenever the Node pin, compatibility manifests, compiler surface manifest, changes.
Change compiler decision sources or the implementation-owned compatibility manifests, then regenerate. Use `pnpm manifest` when compiler decision tables change and `pnpm node-compat` whenever the Node pin, compatibility manifests, compiler surface manifest, changes. `pnpm node-compat --local` reclassifies the committed ledger without fetching Node's docs; it writes the same files as `pnpm node-compat` while the Node pin is unchanged.
Never resolve a merge conflict in these files by hand. Resolve the source conflicts, then run `node scripts/regen-generated.mjs`: it takes the merged-in branch's version of each conflicted generated file, regenerates all of them from the merged sources, and stages the ones that conflicted.
The public docs artifact must not expose repository test paths, internal source identifiers, engineering evidence bookkeeping, documentation/metadata headings, command-line or configuration-only entries, or rows that are N/A in the native tier. Those remain in the complete internal census under `internal/compatibility/generated/`.
@@ -158,7 +159,7 @@ For each API family:
5. Add differential evidence that executes under Node and as a compiled binary. Cover stdout, stderr, exit status, success behavior, and important error shapes. A load-only export check does not prove behavior.
6. Update the implementation-owned source or manifest. Positive `supported` and `partial` mappings must name existing test evidence.
7. Run `pnpm manifest` if compiler surface tables changed, then run `pnpm node-compat`. Never patch generated JSON or headers.
8. Inspect the internal ledger, public artifact, and backlog diff. Confirm the intended API family changed and unrelated rows did not.
8. Inspect the internal ledger and public artifact diff, and compare `pnpm node-compat:backlog` output before and after. Confirm the intended API family changed and unrelated rows did not.
9. Run focused tests, `pnpm node-compat:check`, and the full validation gate required for the change. If docs output changed, also run the docs gate.
Compatibility commands:
@@ -196,7 +197,7 @@ The docs site is a standalone pnpm workspace under `docs/`. It uses Next.js App
- A labeled bullet (`- **Crypto:** ...`) is `docs/content/limitations/<section>/<label>.mdx`, for example `builtin-apis/crypto.mdx`. Edit that file to change the bullet.
- The other text of a section (paragraphs, code blocks, unlabeled lists) is `docs/content/limitations/<section>.mdx`; text before the first heading is `introduction.mdx`.
- `<section>` and `<label>` are the heading and label text in lowercase with each run of other characters replaced by `-`.
- Write one sentence per line; a bullet's continuation lines are indented two spaces. Markdown joins the lines, and edits to different sentences don't conflict.
- Write one sentence per line; a bullet's continuation lines are indented two spaces. Markdown joins the lines, and edits to different sentences don't conflict. `pnpm lint` fails on a line that holds two sentences, and `node docs/scripts/limitations-partials.mjs fix` splits such lines.
- To add a bullet, add its file and an include line at its position in `limitations.mdx`, with a blank line on each side. Consecutive bullet includes render as one list.
- To add a section, add the heading to `limitations.mdx` and include the section's partial below it.
benchmarks/runtime/README.md
+1
View File
@@ -32,6 +32,7 @@ Application-shaped workloads for measuring the speed of compiled executables. Ea
| `value-objects` | cpu | small immutable records and class instances: vector helpers, method chains, particle updates |
| `value-records` | cpu | records that hold strings, spread state updates and destructured tuples: token helpers, an immutable reducer, tuple returns |
| `word-graph` | cpu | string-keyed Map/Set, BFS |
| `worker-messages` | cpu | message round trips between the main thread and one worker: each posts a number and the other answers it, so every round trip is two messages and two wakeups of a waiting thread |
| `cli-config` | startup | a `deploy-plan` CLI: read a ~110 KB JSON manifest, validate, interpolate env values, order 80 services into dependency waves, print a summary |
| `http-api` | server | a `node:http` JSON API: routing, query parsing, a seeded catalog, per-cart state, typed JSON bodies, `ETag`/`If-None-Match`, auth and request-id headers |
benchmarks/runtime/workloads/worker-messages.json
+6
View File
@@ -0,0 +1,6 @@
{
"entry": "workloads/worker-messages.ts",
"args": [
"100000"
]
}
benchmarks/runtime/workloads/worker-messages.ts
+24
View File
@@ -0,0 +1,24 @@
// Message round trips between the main thread and one worker: the main
// thread posts a number and the worker answers it plus one, the given
// number of times. Each round trip is two messages and two wakeups of a
// waiting thread. Argument: the number of round trips.
import { Worker, isMainThread, parentPort } from "node:worker_threads";
if (isMainThread) {
const rounds = Number(process.argv[2] ?? "100000");
const worker = new Worker(new URL(import.meta.url));
let sent = 0;
let sum = 0;
worker.on("message", (value: number) => {
sum += value;
if (++sent === rounds) {
console.log(`${rounds} round trips, sum ${sum}`);
void worker.terminate();
} else worker.postMessage(sent);
});
worker.postMessage(0);
} else {
parentPort!.on("message", (value: number) => {
parentPort!.postMessage(value + 1);
});
}
docs/content/docs/cli.mdx
+38 -2
View File
@@ -2,7 +2,7 @@
title: "CLI Reference"
---
The `scriptc` command has four subcommands: `build`, `run`, `coverage`, and `cache warm`. `scriptc --help` prints this summary and the options below, and `scriptc --version` prints the installed version.
The `scriptc` command has five subcommands: `build`, `run`, `coverage`, `cache warm`, and `merge-profiles`. `scriptc --help` prints this summary and the options below, and `scriptc --version` prints the installed version.
```console
$ scriptc --help
@@ -23,6 +23,13 @@ Usage:
contract sidecar JSON beside the archive
scriptc cache warm [runtime|tls…] verify and load precompiled runtime families
for the current compiler/SDK/target
scriptc merge-profiles -o <out.profdata> [--sparse] <profile…>
merge the raw profiles that
--profile-generate runs wrote (or
earlier merged profiles) into one
profile for --profile-use;
--sparse leaves out functions
that never ran (a smaller file)
```
## scriptc build
@@ -85,6 +92,16 @@ Analyzes the program without building it and reports what compiles and what bloc
Verifies and loads the installed precompiled runtime objects and vendored libraries, and prints the cache directory. Pass `runtime`, `tls`, or both to select which to warm. Ordinary builds don't need this step. With `--sanitize`, it builds the instrumented runtime objects using a C toolchain.
## scriptc merge-profiles
Merges the raw profiles that runs of a `--profile-generate` executable wrote, and any earlier merged profiles, into one profile for `--profile-use`. See [Profile-guided optimization](#profile-guided-optimization).
```console
$ scriptc merge-profiles -o app.profdata profiles/*.profraw
```
With `--sparse`, the profile leaves out the functions the runs never reached. The file is smaller, typically by about two thirds, and those functions are optimized as without a profile instead of as cold code.
## Options
<dl>
@@ -118,6 +135,12 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dt><code>--detached-exit</code></dt>
<dd>On Linux, when the program calls <code>process.exit()</code>, hand the release of its memory to a helper process so the parent sees the exit sooner. The kernel otherwise frees the whole address space before the exit is reported, which takes tens of milliseconds per gigabyte. Opt in only where the trade-offs are acceptable: for up to about 100 ms after the exit, the helper keeps the executable mapped (overwriting the binary in place fails with "text file busy"), its memory stays charged to the cgroup, and the teardown CPU time no longer appears in the parent's resource usage. Valid only for executable builds; it has no effect on other platforms.</dd>
<dt><code>--profile-generate &lt;pattern&gt;</code></dt>
<dd>On Linux, build an instrumented executable for profile-guided optimization. Each run writes a raw profile to the path pattern, where <code>%p</code> expands to the process ID; the <code>LLVM_PROFILE_FILE</code> environment variable overrides the pattern at run time. A relative pattern resolves from the current working directory. Instrumented programs run slower. Valid only for executable builds with <code>--optimization=release</code> or <code>speed</code>.</dd>
<dt><code>--profile-use &lt;file.profdata&gt;</code></dt>
<dd>Optimize the executable with a profile from <code>scriptc merge-profiles</code>: code is laid out, inlined, and split by how the profiled runs used it. Code the profile does not describe, or describes from an older version of the program, is optimized as without a profile, so a stale profile only loses part of its gain. Valid only for executable builds with <code>--optimization=release</code> or <code>speed</code>.</dd>
<dt><code>--npm-static &lt;pkg[,pkg…]|auto&gt;</code></dt>
<dd>Name npm packages to compile even if they don't ship their own type declarations (repeatable). A trailing <code>...</code>, as in <code>fs-extra...</code>, also names every package that package depends on. <code>auto</code> uses the default automatic selection. Packages that can't compile produce diagnostics. See <a href="/docs/dependencies#explicit-package-selection">npm Dependencies</a>.</dd>
@@ -128,7 +151,7 @@ Verifies and loads the installed precompiled runtime objects and vendored librar
<dd>Coverage only. Map an exact bare module specifier to a local declaration file supplied by an embedder. Repeat the option for multiple modules. The declaration supplies checker types so application coverage can continue; runtime imports and values remain explicit <code>SC1010</code> blockers. Relative paths resolve from the current working directory. Accepted files end in <code>.d.ts</code>, <code>.d.mts</code>, or <code>.d.cts</code>.</dd>
<dt><code>--tier &lt;auto|static-only&gt;</code></dt>
<dd>The tier policy for code that has no static lowering. <code>auto</code> is the default. <code>static-only</code> guarantees that the program contains no dynamic-tier demotions: every such construct stays a compile error in TypeScript and a deferred runtime error in JavaScript, which is how both modes behave today, and a checked cast whose value does not match its type throws instead of continuing with the value. Use it in CI to keep a program fully static. Library mode is always static-only. See <a href="/docs/coverage#dynamic-tier">Dynamic tier</a>.</dd>
<dd>The tier policy for code that has no static lowering. <code>auto</code> is the default. <code>static-only</code> guarantees that the program contains no dynamic-tier demotions: every such construct stays a compile error in TypeScript and a deferred runtime error in JavaScript, which is how both modes behave today. Use it in CI to keep a program fully static. Library mode is always static-only. See <a href="/docs/coverage#dynamic-tier">Dynamic tier</a>.</dd>
<dt><code>--explain-dynamic[=&lt;name&gt;]</code></dt>
<dd>Coverage only. After the report, list the functions that already operate on values typed only at runtime and the bindings, fields, functions, classes and modules that unsupported constructs would move to the dynamic tier, each with the diagnostic behind it. With a name such as <code>x</code>, <code>Point</code>, or <code>Point.x</code>, show only the matching units. See <a href="/docs/coverage#dynamic-tier">Dynamic tier</a>.</dd>
@@ -189,6 +212,19 @@ $ file fib-linux
fib-linux: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 2.0.0, with debug_info, not stripped
```
## Profile-guided optimization
A profile records how often each branch and call ran in representative runs of the program. With it, the optimizer inlines the hot calls, keeps hot paths together, and moves cold code out of the way. Profiling takes three steps:
```console
$ scriptc build app.ts --optimization=speed --profile-generate profiles/%p.profraw -o app-instrumented
$ ./app-instrumented typical-input.json
$ scriptc merge-profiles -o app.profdata profiles/*.profraw
$ scriptc build app.ts --optimization=speed --profile-use app.profdata
```
Train on workloads like the ones the program will run: code the training never reached is optimized as without a profile. The profile can be committed and reused as the program changes. Functions whose control flow changed since the profile was collected lose their profile, and the rest keep theirs, so regenerate it when much of the program has changed.
## LLVM backend
scriptc generates LLVM IR and compiles it to native code. Executable builds link the result with the matching precompiled runtime pack. Use <code>--emit=llvm</code> to inspect the generated code.
docs/content/docs/coverage.mdx
+1 -1
View File
@@ -60,7 +60,7 @@ Code that operates on values whose types are known only at runtime, such as `any
`--explain-dynamic` lists those functions and the operation that makes each one dynamic. It also lists the units an unsupported construct would move to the dynamic tier: the binding, field, class, function, or module that has no static representation for the construct, with the diagnostic behind each one. These are the constructs the dynamic tier is planned to compile; today they remain blockers in TypeScript and deferred runtime errors in JavaScript. Add a name, such as `--explain-dynamic=Point`, to show only the matching units. With `--print=diagnostics`, the document gains a `tiers` object with the same information.
`--tier=static-only` keeps every unsupported construct a blocker instead of moving it to the dynamic tier; code that already operates on runtime-typed values compiles as before. It also makes a checked cast whose value does not match its type throw, where `auto`, the default, continues with the value in the dynamic tier (see [runtime type checks](/docs/limitations#runtime-type-checks)). Otherwise `auto` behaves the same way today.
`--tier=static-only` keeps every unsupported construct a blocker instead of moving it to the dynamic tier; code that already operates on runtime-typed values compiles as before. Today `auto`, the default, behaves the same way.
## Diagnostic codes
docs/content/docs/index.mdx
+8 -7
View File
@@ -68,24 +68,25 @@ Programs are checked against the `es2025` standard library, scriptc's own declar
## Runtime type checks
Compiled code relies on types, so scriptc checks a value whose type is only known at runtime before using it as a specific type. Parsed JSON that matches the type you cast it to uses the type's native layout. A value that does not match continues as the value it is, as under Node.js, which does not check the `as Config` assertion:
Compiled code relies on types, so scriptc checks a value whose type is only known at runtime before using it as a specific type. For example, parsed JSON must match the type you cast it to:
```ts:cast.ts
type Config = { port: number };
for (const text of ['{"port": 8080}', '{"port": "eighty"}']) {
const config = JSON.parse(text) as Config;
console.log(config.port, typeof config.port);
try {
const config = JSON.parse('{"port": "eighty"}') as Config;
console.log(config.port);
} catch (error) {
if (error instanceof Error) console.log(error.message);
}
```
```console
$ scriptc run cast.ts
8080 number
eighty string
expected number at $.port, got string
```
See [runtime type checks](/docs/limitations#runtime-type-checks) for where a mismatch still throws.
Under Node.js, the same `as Config` assertion performs no check. See [runtime type checks](/docs/limitations#runtime-type-checks) for details.
## Build-time evaluation
docs/content/limitations/builtin-apis/arraybuffers-and-typed-arrays.mdx
+1
View File
@@ -1,6 +1,7 @@
- **ArrayBuffers and typed arrays:** Fixed-length ArrayBuffers support shared typed-array and DataView storage.
Resizing, transferring, and BigInt arrays are unsupported.
`Float16Array` stores IEEE 754 half-precision values, rounding each stored number to nearest-even as Node does; `Math.f16round`, `DataView.prototype.getFloat16` and `setFloat16`, and `util.types.isFloat16Array` compile.
The typed-array callback methods (`forEach`, `map`, `filter`, `some`, `every`, `find`, `findIndex`, `findLast`, `findLastIndex`, `reduce`, `reduceRight`) compile; `map` and `filter` return the receiver's kind, a Buffer for a Buffer, and a `thisArg` argument is unsupported.
Numeric typed arrays and Buffers preserve their kind and shared storage through `unknown`, including when nested in records and arrays.
Numeric typed-array constructors can be stored and used with `new`, including constructors read from an array's `constructor` property.
JavaScript numeric-array parameters and returns preserve typed-array storage.
docs/content/limitations/builtin-apis/crypto.mdx
+4 -1
View File
@@ -12,5 +12,8 @@
Decrypting chacha20-poly1305 without an authentication tag fails in `final()`; Node 24.22 returns unverified plaintext.
After `update()` fails, for example past the AES-GCM limit of 2^36 - 32 bytes per message, every later call on that cipher throws; Node still lets `final()` run.
An output encoding that is a function or an instance of a class defining `toString` throws a coded SC2020 error.
Public-key operations remain unsupported.
`createECDH` and `ECDH.convertKey` support prime192v1, secp224r1, prime256v1, secp384r1, secp521r1, secp192k1, secp224k1, secp256k1, brainpoolP256r1, brainpoolP384r1, and brainpoolP512r1, on native targets, through `generateKeys()`, `computeSecret()`, `getPublicKey()`, `getPrivateKey()`, `setPrivateKey()`, and `setPublicKey()`, in compressed, uncompressed, and hybrid forms; the other curves `crypto.getCurves()` lists and wasm32-wasi builds are refused.
A call on an ECDH value whose arguments no @types/node overload accepts (`getPublicKey("HEX")`) is refused at compile time, as is `computeSecret(view, encoding)`: Node reads that encoding as the input encoding and answers a Buffer where @types/node declares a string.
ECDH property reads and writes throw a coded SC2020 error.
Other public-key operations remain unsupported.
Native derivations finish before their deferred callbacks, so computation blocks the caller.
docs/content/limitations/builtin-function-values.mdx
+5
View File
@@ -8,10 +8,15 @@ Some builtins can be stored and passed as functions:
In JavaScript, `setTimeout`, `setInterval`, `setImmediate`, `clearTimeout`, `clearInterval`, `clearImmediate`, `queueMicrotask`, and `process.nextTick` are function values wherever they are stored, including conditional and logical expressions and assignments: `typeof` answers `"function"`, identity holds across spellings, and calls schedule as the direct calls do, returning the same numeric timer handles.
Calls through these values bypass `node:test` mock timers.
Other builtin globals stored in JavaScript remain opaque identity values.
In JavaScript, `require("events")` and the `EventEmitter` class assigned to an untyped binding or stored in an object (`ReleaseEmitter = EventEmitter`) are the untyped `EventEmitter` constructor that constructor functions inherit from (see below): `new` through it builds such an emitter object, and identity and `instanceof` agree with the class.
A class can extend `require("events")` directly.
Other immutable aliases of table-backed Node builtins support direct calls, but cannot be passed elsewhere.
`util.promisify` supports compile-time conversions of `child_process.execFile`, `child_process.exec`, `setTimeout` and `setImmediate` (to the `timers/promises` functions, as in Node), `stream.finished` and `stream.pipeline` (to the `stream/promises` functions, as in Node), the `zlib` callback one-shots `deflate`, `inflate`, `deflateRaw`, `inflateRaw`, `gzip`, `gunzip` and `unzip` (with or without options), the `crypto` callback functions `randomBytes`, `pbkdf2`, `scrypt`, `hkdf`, `randomFill`, `randomInt` and `generateKey`, `dns.lookup` (settling with `{ address, family }`, or the address array with `all: true`) and `dns.lookupService` (settling with `{ hostname, service }`), and the `fs` callback functions that have an `fs/promises` counterpart (`access`, `appendFile`, `chmod`, `chown`, `copyFile`, `lchown`, `link`, `lstat`, `lutimes`, `mkdir`, `mkdtemp`, `readFile`, `readdir`, `readlink`, `realpath`, `rename`, `rm`, `rmdir`, `stat`, `statfs`, `symlink`, `truncate`, `unlink`, `utimes`, and `writeFile`); calls through the converted function lower as the `fs/promises` call.
`util.promisify` over an untyped (`any`) function returns a native wrapper that follows Node's callback rules and carries the original's `name` and `length`.
Such a function with other own properties, `util.promisify.custom` among them, throws `SC2020` where `util.promisify` runs, and the wrapper does not report Node's DEP0174 warning when the original returns a promise.
`util.styleText` supports standard formats and aliases, `#RGB` and `#RRGGBB` hex colors, arrays of formats, nested ANSI resets, and explicit color enabling with `{ validateStream: false }`.
Native calls accept standard process streams; custom native streams and mutations to `util.inspect.colors` remain unsupported.
docs/content/limitations/language-features/function-values.mdx
+3
View File
@@ -2,3 +2,6 @@
A function stored in a slot with a different signature remains the same function for equality, array searches, `length`, and own properties.
Typed rest-parameter functions can be stored and called indirectly, including through a slot with fixed parameters, where the slot's arguments arrive in the rest array.
Builtin function values have API-specific restrictions; see [Builtin function values](#builtin-function-values).
- **JavaScript parameters:** In a JavaScript file TypeScript does not check, a prototype method (`C.prototype.m = function (...) {}`) takes whatever values its callers pass; its JSDoc types document the usual values without constraining them.
A sloppy-mode function with named parameters may read `arguments` only where the aliasing of its parameters cannot show: it reads `arguments.length`, or reads elements and never writes a parameter or an element.
Other uses of `arguments` in such a function are refused with `SC1090`; ES modules, `'use strict'` code, classes, and functions with default, rest, or destructured parameters have no such restriction.
docs/content/limitations/readable-stream.mdx
+6 -1
View File
@@ -1 +1,6 @@
`readable-stream` 2.x, 3.x and 4.x compile as Node.js's built-in `stream` module. For 2.x and 4.x, that is the package's own behavior when the `READABLE_STREAM` environment variable is `disable`. Without that variable, Node.js runs the package's JavaScript copy of the stream classes, so a native build can differ from Node.js wherever a program depends on that difference: `instanceof` checks against the built-in classes, class identity, error messages, and timing details. The package's `lib/_stream_*.js` files also compile as the matching built-in classes, although Node.js loads the package's own implementations through them in every environment. `readable-stream` 3.x is a copy of Node.js 10's streams. Its own `disable` mode throws on Node.js 24, so no setting makes Node.js run it against the built-in classes; a native build maps the members its entry exports (the stream classes, `Stream`, `finished` and `pipeline`) to the built-in ones.
`readable-stream` 2.x, 3.x and 4.x compile as Node.js's built-in `stream` module.
For 2.x and 4.x, that is the package's own behavior when the `READABLE_STREAM` environment variable is `disable`.
Without that variable, Node.js runs the package's JavaScript copy of the stream classes, so a native build can differ from Node.js wherever a program depends on that difference: `instanceof` checks against the built-in classes, class identity, error messages, and timing details.
The package's `lib/_stream_*.js` files also compile as the matching built-in classes, although Node.js loads the package's own implementations through them in every environment.
`readable-stream` 3.x is a copy of Node.js 10's streams.
Its own `disable` mode throws on Node.js 24, so no setting makes Node.js run it against the built-in classes; a native build maps the members its entry exports (the stream classes, `Stream`, `finished` and `pipeline`) to the built-in ones.
docs/content/limitations/runtime-type-checks.mdx
+1 -8
View File
@@ -1,12 +1,5 @@
Casts from values whose type is only known at runtime validate the value.
A value that matches the type uses the type's native layout.
When the cast initializes a `const` or `let` binding declared in a function or block, and the binding is only read by the code after the cast, a value that does not match continues as the value it is, as under Node.js: with `JSON.parse(s) as Config` and `{"port": "eighty"}`, `config.port` is the string `"eighty"`, missing properties read `undefined`, and extra properties stay visible to `Object.keys`, `JSON.stringify`, and `util.inspect`.
That code then runs on dynamic values, which is slower than native records.
A statement of it that has no dynamic lowering throws a `TypeError` naming the construct and its location when such a value reaches it.
In the other cases a mismatch throws a catchable `TypeError` identifying the offending path, such as `expected number at $.port, got string`: casts that initialize module-level bindings or appear in other positions, and a mismatched value that leaves the code after its cast through a typed parameter, field, return value, or outer binding, where it is checked against that type.
A value that matches there continues, copied if it is an object.
`--tier=static-only` makes every mismatch throw at the cast.
For example, `JSON.parse(s) as Config` throws a catchable error if the data does not match `Config`, identifying the offending path, such as `expected number at $.port, got string`.
A TypeScript assertion alone does not perform this validation when run under Node.js.
Casts between program classes are checked too: casting an instance to a class it does not extend throws a `TypeError`, where Node.js reads the members structurally.
docs/content/limitations/udp-socket-options.mdx
+7
View File
@@ -1,2 +1,9 @@
`socket.addMembership()` and `socket.dropMembership()` on a `dgram` socket that is not bound yet throw `SC2020`; Node binds such a socket to an ephemeral port first, so bind the socket before joining a group.
The `SystemError` that the buffer size methods throw (`ERR_SOCKET_BUFFER_SIZE`, for example from `getRecvBufferSize()` before `bind()`) has Node's `message`, `code`, `errno` and `syscall`, but no `info` object.
In JavaScript, `socket.send()` also accepts lists of buffers, sends on a connected socket, and takes a completion callback, which receives `null` and the byte count, or the send's failure instead of an `'error'` event.
The callback runs after the socket's pending events, where Node runs it on the next tick.
In TypeScript, `send()` with a callback, a list, or on a connected socket fails to compile with `SC2020`.
JavaScript can read a socket's `on`, `once`, `addListener`, `removeListener`, `off`, `emit`, `send`, `close`, `bind`, `connect`, `disconnect`, `address`, `ref`, and `unref` as function values, for example `socket.emit.bind(socket)`; reading other members as values is refused with `SC1090`.
Listeners registered that way, or with untyped parameters, take only the `message`, `error`, `close`, `listening`, and `connect` events.
When a socket closes, listeners registered from TypeScript are released, so an `emit()` from JavaScript after `'close'` reaches only listeners registered from JavaScript.
docs/content/limitations/unix-domain-sockets.mdx
+7 -2
View File
@@ -1,2 +1,7 @@
`server.listen(path)` and the `socketPath` option of `http.request()` and `https.request()` use Unix domain sockets on Linux and macOS. On Windows, where Node maps these paths to named pipes, they fail with `ENOTSUP`. `server.address()` on a server listening on a Unix socket returns the socket path, as in Node, when the result is typed `string | AddressInfo | null` (the @types/node declaration). Where it is typed as the `{ address, family, port }` record alone, which cannot hold the path, the call throws an error for such a server. `listen(path, backlog, callback)` is refused at compile time; pass `listen(path, callback)`.
When the options of `http.request()` come from a variable, or from an object literal that spreads one, they are read at run time with Node's validation errors. The `auth`, `createConnection`, `family`, `hints`, `insecureHTTPParser`, `joinDuplicateHeaders`, `localAddress`, `localPort`, `lookup` and `uniqueHeaders` options then throw a runtime error when they hold a value, and so do `setHost: false` and `setDefaultHeaders: false`.
`server.listen(path)` and the `socketPath` option of `http.request()` and `https.request()` use Unix domain sockets on Linux and macOS.
On Windows, where Node maps these paths to named pipes, they fail with `ENOTSUP`.
`server.address()` on a server listening on a Unix socket returns the socket path, as in Node, when the result is typed `string | AddressInfo | null` (the @types/node declaration).
Where it is typed as the `{ address, family, port }` record alone, which cannot hold the path, the call throws an error for such a server.
`listen(path, backlog, callback)` is refused at compile time; pass `listen(path, callback)`.
When the options of `http.request()` come from a variable, or from an object literal that spreads one, they are read at run time with Node's validation errors.
The `auth`, `createConnection`, `family`, `hints`, `insecureHTTPParser`, `joinDuplicateHeaders`, `localAddress`, `localPort`, `lookup` and `uniqueHeaders` options then throw a runtime error when they hold a value, and so do `setHost: false` and `setDefaultHeaders: false`.
docs/scripts/limitations-partials.mjs
+54 -2
View File
@@ -21,7 +21,13 @@
* node scripts/limitations-partials.mjs split <file> [page] write the page and partials from one file
* node scripts/limitations-partials.mjs merge <base> <theirs>
* three-way merge a branch's single-file page (<theirs>, forked from
* <base>) into the current partials, then split the result */
* <base>) into the current partials, then split the result
* node scripts/limitations-partials.mjs check fail on a partial line that holds two sentences
* node scripts/limitations-partials.mjs fix put each such sentence on its own line
*
* One sentence per line keeps changes to different sentences on different
* lines, so concurrent edits merge. Markdown joins the lines with a space,
* so the rendered page is the same. */
import { execFileSync } from "node:child_process";
import { mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
@@ -139,6 +145,41 @@ function write(text, page = defaultPage) {
console.error(`${relative(docsRoot, page)}: ${partials.size} partials in ${relative(docsRoot, partialsDir)}/`);
}
// A sentence ends at `.`, `!` or `?` (after any closing `)`, `"` or `*`)
// followed by a space and a capital letter, a code span, a link or bold
// text. Abbreviations don't end one.
const sentenceEnd = /(?<!\b(?:e\.g|i\.e|etc|vs|cf))([.!?][)"*]*) (?=[A-Z`[*])/g;
/** The partial text with each sentence of a prose line on its own line;
* a bullet's sentences continue with the two-space indent. Code fences,
* headings, tables, HTML and include lines are left alone, as are code
* spans and link targets. */
export function splitSentences(text) {
let inFence = false;
const out = [];
for (const line of text.split("\n")) {
if (/^\s*(```|~~~)/.test(line)) inFence = !inFence;
if (inFence || /^\s*(```|~~~|#|<|\||$)/.test(line)) {
out.push(line);
continue;
}
const indent = /^\s*(?:[-*]|\d+\.) /.test(line) ? " " : line.match(/^\s*/)[0];
// Mask code spans (all but the opening backtick, which can start a
// sentence) and link targets, so their periods never split.
const masked = line.replace(/`[^`]*`|\]\([^)]*\)/g, (m) => m[0] + "\0".repeat(m.length - 1));
const pieces = [];
let start = 0;
for (const match of masked.matchAll(sentenceEnd)) {
const end = match.index + match[1].length;
pieces.push(line.slice(start, end));
start = end + 1;
}
pieces.push(line.slice(start));
out.push(pieces[0], ...pieces.slice(1).map((piece) => indent + piece));
}
return out.join("\n");
}
function listPartials(dir = partialsDir) {
return readdirSync(dir, { withFileTypes: true }).flatMap((entry) =>
entry.isDirectory() ? listPartials(join(dir, entry.name)) : [join(dir, entry.name)],
@@ -175,9 +216,20 @@ if (command === "join") {
} finally {
rmSync(work, { recursive: true, force: true });
}
} else if (command === "check" || command === "fix") {
let unsplit = 0;
for (const file of listPartials().sort()) {
const text = readFileSync(file, "utf8");
const split = splitSentences(text);
if (split === text) continue;
unsplit++;
if (command === "fix") writeFileSync(file, split);
else console.error(`${relative(docsRoot, file)}: put each sentence on its own line (node docs/scripts/limitations-partials.mjs fix)`);
}
if (command === "check" && unsplit > 0) process.exitCode = 1;
} else if (command === "list") {
for (const file of listPartials().sort()) console.log(relative(docsRoot, file));
} else {
console.error("usage: limitations-partials.mjs join [page] | split <file> [page] | merge <base> <theirs> | list");
console.error("usage: limitations-partials.mjs join [page] | split <file> [page] | merge <base> <theirs> | check | fix | list");
process.exitCode = 2;
}
docs/src/generated/node-v24-compatibility.json
+86 -590
View File
File diff suppressed because it is too large Load Diff
internal/compatibility/README.md
+2 -2
View File
@@ -5,14 +5,14 @@ This workspace package owns scriptc's Node.js parity system.
- `node-v24.json` pins the Node release tag, commit, and canonical docs/source inputs.
- `static-support.json` overlays dedicated compiler paths and their internal test evidence on the compiler's generated surface manifest.
- `generated/node-v24-internal.json` is the engineering ledger, including implementation evidence and repository test paths.
- `generated/node-v24-backlog.json` is the internal tier-specific work queue. It separates verification, implementation, explicit-refusal replacement, and partial-surface audit work, and prioritizes stable Node APIs above experimental, deprecated, and legacy APIs.
- `pnpm node-compat:backlog` derives the internal tier-specific work queue from the ledger; it is not committed. It separates verification, implementation, explicit-refusal replacement, and partial-surface audit work, and prioritizes stable Node APIs above experimental, deprecated, and legacy APIs.
- `docs/src/generated/node-v24-compatibility.json` is the stripped public artifact consumed by the docs website. It contains compiler-relevant runtime APIs, statuses, verification bases, and user-facing details; documentation, metadata, command-line/configuration entries, and rows that are N/A for native execution remain internal.
Statuses are evidence-aware: `supported` and `partial` require test evidence; `refused` requires an explicit compiler diagnostic or native throwing stub; `not-implemented` is the actionable backlog for an owned API family with no implementation match; `by-design` records an architectural exclusion; `unreviewed` still needs classification; and `not-applicable` is not a runtime API for that execution tier.
Run `pnpm node-compat:backlog` from the repository root for a summary. The filters `--tier`, `--action`, `--priority`, `--chapter`, and `--status` compose; use `--format=json` or `--format=tsv` for tooling.
Run `pnpm node-compat` from the repository root after changing the Node pin or the support manifest. `pnpm node-compat:check` is the offline drift/evidence gate; `pnpm --filter @internal/compatibility check:upstream` additionally verifies the pinned upstream inputs over the network.
Run `pnpm node-compat` from the repository root after changing the Node pin or the support manifest. `pnpm node-compat --local` does the same without network access while the pin is unchanged, by reclassifying the committed ledger rows; after a merge, `node scripts/regen-generated.mjs` regenerates every generated file instead of a hand merge. `pnpm node-compat:check` is the offline drift/evidence gate; `pnpm --filter @internal/compatibility check:upstream` additionally verifies the pinned upstream inputs over the network.
## npm packages that compile as builtins
internal/compatibility/generated/node-v24-backlog.json
-88685
View File
File diff suppressed because it is too large Load Diff
internal/compatibility/generated/node-v24-internal.json
+180 -652
View File
File diff suppressed because it is too large Load Diff
internal/compatibility/src/backlog.mjs
+4 -1
View File
@@ -2,9 +2,12 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { buildBacklog } from "./generate.mjs";
const packageRoot = fileURLToPath(new URL("../", import.meta.url));
const backlog = JSON.parse(readFileSync(`${packageRoot}generated/node-v24-backlog.json`, "utf8"));
const backlog = buildBacklog(
JSON.parse(readFileSync(`${packageRoot}generated/node-v24-internal.json`, "utf8")),
);
const args = process.argv.slice(2);
function option(name) {
internal/compatibility/src/generate.mjs
+57 -23
View File
@@ -13,6 +13,8 @@
*
* pnpm node-compat fetch latest-v24.x and rewrite the snapshot
* pnpm node-compat --check fetch and fail if the snapshot would change
* pnpm node-compat --local rewrite the snapshot without fetching: reclassify
* the committed rows from the current sources
* pnpm node-compat:check offline implementation/profile drift check
*/
@@ -22,12 +24,15 @@ import { fileURLToPath } from "node:url";
const packageRoot = fileURLToPath(new URL("../", import.meta.url));
const root = fileURLToPath(new URL("../../../", import.meta.url));
const internalOutputPath = `${packageRoot}generated/node-v24-internal.json`;
const backlogOutputPath = `${packageRoot}generated/node-v24-backlog.json`;
const publicOutputPath = `${root}docs/src/generated/node-v24-compatibility.json`;
// The docs build derives this file's values from the public artifact; a
// merge that keeps a branch's copy would bring back a file every
// compatibility change rewrites.
const retiredOutputPaths = [`${root}docs/src/generated/node-v24-compatibility-meta.json`];
// Files derived from the ledger elsewhere: the docs build computes the
// metadata from the public artifact, and `pnpm node-compat:backlog` builds
// the backlog in memory. A merge that keeps a branch's copy would bring
// back a file every compatibility change rewrites.
const retiredOutputPaths = [
`${root}docs/src/generated/node-v24-compatibility-meta.json`,
`${packageRoot}generated/node-v24-backlog.json`,
];
const compilerManifestPath = `${root}packages/compiler/surface-manifest.json`;
const compilerCompatPath = `${packageRoot}static-support.json`;
const pinPath = `${packageRoot}node-v24.json`;
@@ -447,12 +452,6 @@ function summarizeTier(rows, key, chapterPolicy) {
return tier("unreviewed", "", { source: "derived:descendants" });
}
function statusCounts(rows, key) {
const counts = Object.fromEntries(STATUSES.map((status) => [status, 0]));
for (const row of rows) if (row.scope === "api") counts[row[key].status] += 1;
return counts;
}
function verifiedAnchorIds(html) {
return new Set(
[...html.matchAll(/<a class="mark" href="#[^"]+" id="([^"]+)">#/g)].map((match) => match[1]),
@@ -524,7 +523,6 @@ function buildSnapshot({ allJson, indexMarkdown, allSha256, indexSha256 }) {
title: link.title,
entries: chapterRows.length,
apiEntries: chapterRows.filter((row) => row.scope === "api").length,
static: statusCounts(chapterRows, "static"),
});
rows.push(...chapterRows);
}
@@ -589,6 +587,12 @@ function publicDetail(tier) {
if (/^compiler-dedicated:assert\.doesNot(?:Throw|Reject)$/.test(source)) {
return "Supports callbacks or native promises with messages, Error classes, regular expressions, and validators accepting unknown values; object expectations and complete AssertionError metadata remain unsupported.";
}
if (/^surface-manifest:node-builtin\.crypto\.createECDH$/.test(source)) {
return "Supports prime192v1, secp224r1, prime256v1, secp384r1, secp521r1, secp192k1, secp224k1, secp256k1, brainpoolP256r1, brainpoolP384r1 and brainpoolP512r1 with Node's curve, key, point, format and encoding errors; other curves Node knows are refused.";
}
if (/^compiler-dedicated:(?:ECDH|ecdh\.)/.test(source)) {
return "Matches Node's key formats, point encodings, private-key range checks, key-pair checks and errors on the supported curves; ECDH.convertKey is supported, and property reads and writes are refused.";
}
if (/^compiler-dedicated:(?:Cipheriv|Decipheriv|cipher\.|decipher\.)/.test(source)) {
return "Matches Node's update() output lengths, padding, authentication tags, string encodings, and state errors for the supported ciphers; the stream.Transform interface is refused.";
}
@@ -867,7 +871,7 @@ function backlogAction(tier) {
return null;
}
function buildBacklog(snapshot) {
export function buildBacklog(snapshot) {
const tasks = [];
const tierItems = { static: {} };
const priorities = { high: 0, normal: 0, low: 0 };
@@ -963,7 +967,6 @@ function projectPublicMatrix(snapshot) {
title: chapter.title,
entries: chapterRows.length,
apiEntries: chapterRows.length,
static: statusCounts(chapterRows, "static"),
});
rows.push(...chapterRows);
}
@@ -1012,6 +1015,34 @@ function publicSnapshot(snapshot) {
};
}
/** The committed ledger with every row's classification recomputed from
* the current sources. The rows themselves come from the pinned Node docs,
* so while the pin is unchanged this is what a fetching run writes. */
function reclassify(snapshot) {
if (snapshot.nodeVersion !== pin.version || snapshot.nodeCommit !== pin.commit) {
throw new Error(
`the ledger targets Node ${snapshot.nodeVersion}, the pin ${pin.version}; run 'pnpm node-compat' to fetch the new docs`,
);
}
const ctx = classificationContext();
const ancestors = [];
for (const row of snapshot.rows) {
ancestors.length = row.depth;
const parentSignature = ancestors[row.depth - 1]?.signature;
row.static = classifyStatic(row, null, ctx, parentSignature);
ancestors[row.depth] = row;
}
for (const chapter of snapshot.chapters) {
const chapterRows = snapshot.rows.filter((row) => row.chapter === chapter.slug);
chapterRows[0].static = summarizeTier(
chapterRows,
"static",
ctx.compilerCompat.chapterPolicies[chapter.slug],
);
}
return snapshot;
}
function checkLocal() {
for (const path of retiredOutputPaths) {
if (existsSync(path))
@@ -1064,9 +1095,6 @@ function checkLocal() {
throw new Error(`${row.id} exposes a row that is not applicable to native compilation`);
}
}
if (readFileSync(backlogOutputPath, "utf8") !== render(buildBacklog(snapshot))) {
throw new Error("the Node compatibility backlog is stale; run 'pnpm node-compat'");
}
if (readFileSync(publicOutputPath, "utf8") !== render(publicSnapshot(snapshot))) {
throw new Error("the docs compatibility artifact is stale; run 'pnpm node-compat'");
}
@@ -1111,6 +1139,15 @@ async function fetchText(url) {
async function main() {
const args = new Set(process.argv.slice(2));
if (args.has("--check-local")) return checkLocal();
if (args.has("--local")) {
const snapshot = reclassify(JSON.parse(readFileSync(internalOutputPath, "utf8")));
writeFileSync(internalOutputPath, render(snapshot));
writeFileSync(publicOutputPath, render(publicSnapshot(snapshot)));
console.log(
`reclassified the compatibility ledger and public artifact (${snapshot.rows.length} source rows, ${projectPublicMatrix(snapshot).rows.length} public matrix rows, ${pin.tag})`,
);
return;
}
const [allText, indexMarkdown] = await Promise.all([
fetchText(pin.allJsonUrl),
@@ -1132,14 +1169,11 @@ async function main() {
);
verifyAnchors(snapshot, htmlByChapter);
const internalRendered = render(snapshot);
const backlogRendered = render(buildBacklog(snapshot));
const publicRendered = render(publicSnapshot(snapshot));
if (args.has("--check")) {
if (readFileSync(internalOutputPath, "utf8") !== internalRendered)
throw new Error("internal Node compatibility snapshot is stale; run 'pnpm node-compat'");
if (readFileSync(backlogOutputPath, "utf8") !== backlogRendered)
throw new Error("Node compatibility backlog is stale; run 'pnpm node-compat'");
if (readFileSync(publicOutputPath, "utf8") !== publicRendered)
throw new Error("public Node compatibility snapshot is stale; run 'pnpm node-compat'");
console.log(
@@ -1148,11 +1182,11 @@ async function main() {
return;
}
writeFileSync(internalOutputPath, internalRendered);
writeFileSync(backlogOutputPath, backlogRendered);
writeFileSync(publicOutputPath, publicRendered);
console.log(
`wrote compatibility ledger, backlog, and public artifact (${snapshot.rows.length} source rows, ${projectPublicMatrix(snapshot).rows.length} public matrix rows, ${pin.tag})`,
`wrote compatibility ledger and public artifact (${snapshot.rows.length} source rows, ${projectPublicMatrix(snapshot).rows.length} public matrix rows, ${pin.tag})`,
);
}
await main();
// backlog.mjs imports buildBacklog; only running this file generates.
if (process.argv[1] === fileURLToPath(import.meta.url)) await main();
internal/compatibility/src/update.mjs
+1 -2
View File
@@ -2,8 +2,7 @@
import { spawnSync } from "node:child_process";
const check = process.argv.slice(2).includes("--check");
const args = check ? ["--check"] : [];
const args = process.argv.slice(2).filter((arg) => arg === "--check" || arg === "--local");
for (const script of ["generate.mjs"]) {
const result = spawnSync(process.execPath, [`src/${script}`, ...args], { stdio: "inherit" });
internal/compatibility/static-support.json
+3
View File
@@ -44,6 +44,7 @@
"node-builtin.console": ["tests/corpus/console-imports.cjs", "tests/corpus/2440-console-inspect-args.ts", "tests/corpus/console-native-value.js", "tests/harness/npm-static.test.ts"],
"node-builtin.crypto.createCipheriv": ["tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/fixtures/node-types/cipher.ts"],
"node-builtin.crypto.createDecipheriv": ["tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/fixtures/node-types/cipher.ts"],
"node-builtin.crypto.createECDH": ["tests/corpus/6700-ecdh-shared-secret.ts", "tests/corpus/6701-ecdh-formats.ts", "tests/corpus/6702-ecdh-errors.js", "tests/corpus/6703-ecdh-private-keys.ts", "tests/corpus/6704-ecdh-set-public-key.cjs", "tests/corpus/6705-ecdh-handles.ts", "tests/fixtures/node-types/ecdh.ts"],
"node-builtin.crypto": ["tests/corpus/1358-crypto-random.ts", "tests/corpus/1534-crypto-hash-chain.ts", "tests/corpus/2556-crypto-introspection.ts", "tests/corpus/2890-crypto-static-utility.ts", "tests/harness/npm-static.test.ts", "tests/corpus/node24-crypto-sha2.ts", "tests/corpus/node24-crypto-kdf.ts", "tests/corpus/node24-crypto-kdf-errors.ts", "tests/corpus/node24-crypto-kdf-options.ts"],
"node-builtin.dgram": ["tests/corpus/2597-dgram-send-ladders.cjs"],
"node-builtin.diagnostics_channel": ["tests/corpus/2214-dc-bind-store.cjs"],
@@ -410,6 +411,8 @@
{ "symbols": ["assert.deepEqual", "assert.notDeepEqual"], "status": "partial", "evidence": ["tests/corpus/5513-assert-deep-equal-loose.js", "tests/corpus/5514-assert-deep-equal-typed.ts", "tests/corpus/5516-assert-deep-equal-edges.js"] },
{ "symbols": ["assert", "assert.ok", "assert.equal", "assert.notEqual", "assert.strictEqual", "assert.notStrictEqual", "assert.deepStrictEqual", "assert.notDeepStrictEqual", "assert.fail", "assert.match", "assert.doesNotMatch", "assert.throws", "assert.rejects", "assert.ifError"], "status": "partial", "evidence": ["tests/corpus/1600-assert-passing.ts", "tests/corpus/1604-assert-deep-structures.ts", "tests/corpus/1609-assert-async.ts", "tests/corpus/1721-assert-throws-regex-class.ts", "tests/corpus/2936-static-loose-equality.ts", "tests/corpus/5515-assert-deep-strict-typed-messages.ts"] },
{ "chapter": "crypto", "symbols": ["Cipheriv", "cipher.update", "cipher.final", "cipher.setAAD", "cipher.getAuthTag", "cipher.setAutoPadding", "Decipheriv", "decipher.update", "decipher.final", "decipher.setAAD", "decipher.setAuthTag", "decipher.setAutoPadding"], "status": "partial", "evidence": ["tests/corpus/5250-cipher-aes-gcm.ts", "tests/corpus/5251-cipher-aes-cbc.ts", "tests/corpus/5252-cipher-aes-ctr.ts", "tests/corpus/5253-cipher-chacha20-poly1305.ts", "tests/corpus/5254-cipher-errors.js", "tests/corpus/5255-cipher-encodings.cjs", "tests/corpus/5256-cipher-deprecation.cjs", "tests/corpus/5257-cipher-vectors.ts", "tests/corpus/5258-cipher-handles.js", "tests/corpus/5259-cipher-chunked.ts", "tests/fixtures/node-types/cipher.ts"] },
{ "chapter": "crypto", "symbols": ["ECDH", "ecdh.computeSecret", "ecdh.generateKeys", "ecdh.getPrivateKey", "ecdh.getPublicKey", "ecdh.setPrivateKey", "ecdh.setPublicKey"], "status": "partial", "evidence": ["tests/corpus/6700-ecdh-shared-secret.ts", "tests/corpus/6701-ecdh-formats.ts", "tests/corpus/6702-ecdh-errors.js", "tests/corpus/6703-ecdh-private-keys.ts", "tests/corpus/6704-ecdh-set-public-key.cjs", "tests/corpus/6705-ecdh-handles.ts", "tests/fixtures/node-types/ecdh.ts"] },
{ "chapter": "errors", "symbols": ["ERR_CRYPTO_ECDH_INVALID_FORMAT", "ERR_CRYPTO_ECDH_INVALID_PUBLIC_KEY"], "status": "partial", "evidence": ["tests/corpus/6702-ecdh-errors.js"] },
{ "symbols": ["crypto.createHash", "Hash", "hash.update", "hash.digest", "hash.copy", "crypto.createHmac", "Hmac", "hmac.update", "hmac.digest", "crypto.hash", "crypto.pbkdf2", "crypto.pbkdf2Sync"], "status": "partial", "evidence": ["tests/corpus/5760-crypto-hash-stream.ts", "tests/corpus/5768-crypto-hash-unknown.ts", "tests/corpus/5769-crypto-hash-unknown.cjs", "tests/corpus/5761-crypto-hash-stream-states.ts", "tests/corpus/5762-crypto-hash-stream.cjs", "tests/corpus/2890-crypto-static-utility.ts", "tests/corpus/node24-crypto-sha2.ts", "tests/corpus/5435-crypto-hash-algorithms.ts", "tests/corpus/5436-crypto-hash-names.js", "tests/corpus/5437-crypto-hmac-algorithms.ts", "tests/corpus/5438-crypto-kdf-digests.ts", "tests/corpus/5440-crypto-hash-encodings.js", "tests/corpus/5441-crypto-hash-output-length.ts", "tests/harness/npm-static.test.ts"] },
{ "symbols": ["crypto.randomFill", "crypto.randomFillSync", "crypto.randomUUID", "crypto.randomUUIDv7"], "status": "supported", "evidence": ["tests/corpus/5442-crypto-random-values.ts", "tests/corpus/5443-crypto-random-arguments.js", "tests/corpus/1358-crypto-random.ts"] },
{ "symbols": ["crypto.createSecretKey", "crypto.generateKey", "crypto.generateKeySync", "KeyObject", "keyObject.export", "keyObject.equals", "keyObject.symmetricKeySize", "keyObject.type"], "status": "partial", "evidence": ["tests/corpus/5445-crypto-secret-key-objects.ts", "tests/corpus/5446-crypto-secret-key-errors.js"] },
native/llvm-codegen/CMakeLists.txt
+3 -1
View File
@@ -80,6 +80,7 @@ add_executable(scriptc-llvm-codegen
src/emit.cpp
src/main.cpp
src/partition.cpp
src/profile.cpp
src/runtime-import.cpp
src/runtime-unit.cpp
src/target.cpp
@@ -180,7 +181,7 @@ if(SCRIPTC_STATIC_LINUX_BUILD)
endif()
endfunction()
set(SCRIPTC_LLVM_LINK_ROOTS ${SCRIPTC_LLVM_TARGET_LIBS} BitReader BitWriter IRReader LTO Linker Passes TransformUtils)
set(SCRIPTC_LLVM_LINK_ROOTS ${SCRIPTC_LLVM_TARGET_LIBS} BitReader BitWriter IRReader LTO Linker Passes ProfileData TransformUtils)
foreach(component IN LISTS SCRIPTC_LLVM_LINK_ROOTS)
if(TARGET "LLVM${component}")
scriptc_collect_llvm_target("LLVM${component}")
@@ -209,6 +210,7 @@ else()
LTO
Linker
Passes
ProfileData
Support
Target
TransformUtils
native/llvm-codegen/src/emit.cpp
+105 -11
View File
@@ -30,6 +30,10 @@
#include "llvm/PassInfo.h"
#include "llvm/PassRegistry.h"
#include "llvm/Passes/PassBuilder.h"
#include "llvm/ProfileData/InstrProfReader.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/PGOOptions.h"
#include "llvm/Support/VirtualFileSystem.h"
#include "llvm/Support/Caching.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/MemoryBuffer.h"
@@ -53,6 +57,10 @@ using namespace llvm;
namespace scriptc {
static constexpr size_t MaxPartitions = 64;
// The thin link's import allowance for hot and hottest call edges, as a
// multiple of the usual size limit, in builds with a profile. See
// limitProfileImports().
static constexpr const char *ProfileImportMultiplier = "2";
std::optional<EmitOptions> parseEmitOptions(int Argc, char **Argv) {
EmitOptions Options;
@@ -82,11 +90,18 @@ std::optional<EmitOptions> parseEmitOptions(int Argc, char **Argv) {
Options.ImportBitcode.push_back(Value.str());
else if (Arg == "--cache-dir")
Options.CacheDirectory = Value.str();
else if (Arg == "--profile-generate")
Options.ProfileGenerate = Value.str();
else if (Arg == "--profile-use")
Options.ProfileUse = Value.str();
else
return std::nullopt;
}
if (Options.Input.empty() || Options.Outputs.empty())
return std::nullopt;
// A build either collects a profile or uses one.
if (!Options.ProfileGenerate.empty() && !Options.ProfileUse.empty())
return std::nullopt;
return Options;
}
@@ -123,6 +138,57 @@ PipelineTuningOptions tuningFor(OptimizationLevel Level) {
return Tuning;
}
// The profile-guided optimization a build asks for: IR instrumentation that
// writes raw profiles to the ProfileGenerate pattern, or annotation from the
// indexed ProfileUse profile. Functions the profile does not describe, or
// whose control flow changed since it was collected, are optimized as without
// one, so a stale profile loses only its gain.
std::optional<PGOOptions> profileOptions(const EmitOptions &Options) {
if (!Options.ProfileGenerate.empty())
return PGOOptions(Options.ProfileGenerate, "", "", "",
PGOOptions::IRInstr);
if (!Options.ProfileUse.empty())
return PGOOptions(Options.ProfileUse, "", "", "", PGOOptions::IRUse);
return std::nullopt;
}
// With a profile, the thin link imports the callees of hot call edges up to
// 10 times, and of the hottest edges up to 100 times, the usual size limit,
// into every partition that calls them. On large programs that triples the
// build's time and memory for no measurable run-time gain over a much smaller
// allowance, so profile builds use that one. The importer's limits are
// process-wide options; the helper sets them once.
void limitProfileImports() {
static const bool Limited = [] {
auto Registered = cl::getRegisteredOptions();
for (StringRef Name :
{"import-hot-multiplier", "import-critical-multiplier"})
if (cl::Option *Option = Registered.lookup(Name))
Option->addOccurrence(0, Name, ProfileImportMultiplier);
return true;
}();
(void)Limited;
}
// Reads the profile's header once, before any partition needs it: it must
// hold IR-level counts.
std::optional<EmitFailure> inspectProfile(const EmitOptions &Options) {
if (Options.ProfileUse.empty())
return std::nullopt;
Expected<std::unique_ptr<IndexedInstrProfReader>> Reader =
IndexedInstrProfReader::create(Options.ProfileUse,
*vfs::getRealFileSystem());
if (!Reader)
return EmitFailure{"profile_read_failed",
Options.ProfileUse + ": " +
toString(Reader.takeError())};
if (!(*Reader)->isIRLevelProfile())
return EmitFailure{"profile_read_failed",
Options.ProfileUse +
": not an IR-level instrumentation profile"};
return std::nullopt;
}
// Owns the analysis managers a pipeline needs for one module.
struct Pipeline {
LoopAnalysisManager LAM;
@@ -131,8 +197,9 @@ struct Pipeline {
ModuleAnalysisManager MAM;
PassBuilder PB;
Pipeline(TargetMachine &Machine, OptimizationLevel Level)
: PB(&Machine, tuningFor(Level)) {
Pipeline(TargetMachine &Machine, OptimizationLevel Level,
std::optional<PGOOptions> Profile = std::nullopt)
: PB(&Machine, tuningFor(Level), std::move(Profile)) {
PB.registerModuleAnalyses(MAM);
PB.registerCGSCCAnalyses(CGAM);
PB.registerFunctionAnalyses(FAM);
@@ -319,9 +386,9 @@ std::optional<EmitFailure> publish(StringRef TemporaryPath,
std::optional<EmitFailure>
emitModule(Module &M, TargetMachine &Machine, OptimizationLevel Level,
CodeGenFileType Type, StringRef OutputPath,
std::vector<SmallString<256>> &Temporaries) {
Pipeline P(Machine, Level);
std::optional<PGOOptions> Profile, CodeGenFileType Type,
StringRef OutputPath, std::vector<SmallString<256>> &Temporaries) {
Pipeline P(Machine, Level, std::move(Profile));
P.PB.buildPerModuleDefaultPipeline(Level).run(M, P.MAM);
if (std::optional<EmitFailure> Failure =
verify(M, "post_optimization_verification_failed"))
@@ -462,7 +529,7 @@ compileUnoptimizedPartitions(const EmitOptions &Options,
if (!Mod)
return Failures.set({"partition_failed", toString(Mod.takeError())});
// The O0 pipeline still lowers coroutines and always-inline calls.
Pipeline P(*Machine, OptimizationLevel::O0);
Pipeline P(*Machine, OptimizationLevel::O0, profileOptions(Options));
P.PB.buildPerModuleDefaultPipeline(OptimizationLevel::O0)
.run(**Mod, P.MAM);
raw_svector_ostream Output(Objects[I]);
@@ -522,7 +589,9 @@ compileOptimizedPartitions(const EmitOptions &Options,
SmallVector<char, 0> Bitcode;
{
raw_svector_ostream Stream(Bitcode);
Pipeline P(*Machine, Level);
// Profile instrumentation or annotation happens here, before the thin
// link; the post-link optimization reads the annotated weights.
Pipeline P(*Machine, Level, profileOptions(Options));
ModulePassManager MPM = P.PB.buildThinLTOPreLinkDefaultPipeline(Level);
MPM.addPass(BitcodeWriterPass(Stream, /*ShouldPreserveUseListOrder=*/false,
/*EmitSummaryIndex=*/true,
@@ -620,8 +689,14 @@ compileOptimizedPartitions(const EmitOptions &Options,
std::vector<std::string> Names(Count);
for (size_t I = 0; I < Count; ++I)
Names[I] = "partition-" + std::to_string(I);
if (!Options.ProfileUse.empty())
limitProfileImports();
lto::LTO Link(std::move(Config),
lto::createInProcessThinBackend(hardware_concurrency()));
// Profile instrumentation gives every partition its own weak copy of the
// variables the profile runtime reads (the profile format and file name).
// One copy prevails and stays visible so the runtime's defaults do not.
StringSet<> ProfileSymbols;
for (size_t I = 0; I < Count; ++I) {
Expected<std::unique_ptr<lto::InputFile>> Input = lto::InputFile::create(
MemoryBufferRef(Summaries[I]->getBuffer(), Names[I]));
@@ -635,7 +710,10 @@ compileOptimizedPartitions(const EmitOptions &Options,
// external before partitioning are visible to the runtime objects
// linked beside them; the rest may become internal again.
bool Public = Plan.PublicSymbols.contains(Symbol.getIRName());
Resolution.Prevailing = true;
bool Profile = Symbol.getIRName().starts_with("__llvm_profile_");
if (Profile)
Public = ProfileSymbols.insert(Symbol.getIRName()).second;
Resolution.Prevailing = !Profile || Public;
Resolution.VisibleToRegularObj = Public;
Resolution.FinalDefinitionInLinkageUnit = !Public;
}
@@ -672,7 +750,19 @@ compilePartitions(const EmitOptions &Options, const PartitionPlan &Plan,
// helper build; source content and partition count arrive per module.
std::string Salt = hashParts({"scriptc-partition-v1", LLVM_VERSION_STRING,
Options.Target, Options.OptLevel,
Options.RelocationModel, Options.SourcePath});
Options.RelocationModel, Options.SourcePath,
"profile-generate", Options.ProfileGenerate});
// A profile changes every partition it describes, so its bytes key them.
if (!Options.ProfileUse.empty()) {
ErrorOr<std::unique_ptr<MemoryBuffer>> Profile =
MemoryBuffer::getFile(Options.ProfileUse, /*IsText=*/false,
/*RequiresNullTerminator=*/false);
if (!Profile)
return EmitFailure{"profile_read_failed",
Options.ProfileUse + ": " +
Profile.getError().message()};
Salt = hashParts({Salt, "profile-use", (*Profile)->getBuffer()});
}
for (const std::string &Path : Options.ImportBitcode) {
ErrorOr<std::unique_ptr<MemoryBuffer>> Bitcode =
MemoryBuffer::getFile(Path, /*IsText=*/false,
@@ -805,6 +895,9 @@ int emit(const EmitOptions &Options) {
return reportError("invalid_import",
"runtime bitcode import requires an optimized build",
Options.DiagnosticFormat);
if (std::optional<EmitFailure> Failure = inspectProfile(Options))
return reportError(Failure->Code, Failure->Message,
Options.DiagnosticFormat);
if (Options.Outputs.size() > 1)
return emitPartitions(Options);
@@ -864,8 +957,9 @@ int emit(const EmitOptions &Options) {
? CodeGenFileType::ObjectFile
: CodeGenFileType::AssemblyFile;
std::vector<SmallString<256>> Temporaries;
std::optional<EmitFailure> Failure = emitModule(
*Mod, *Machine, Level, Type, Options.Outputs[0], Temporaries);
std::optional<EmitFailure> Failure =
emitModule(*Mod, *Machine, Level, profileOptions(Options), Type,
Options.Outputs[0], Temporaries);
if (!Failure)
Failure = publish(Temporaries[0], Options.Outputs[0]);
if (Failure) {
native/llvm-codegen/src/emit.h
+6
View File
@@ -20,6 +20,12 @@ struct EmitOptions {
std::vector<std::string> ImportBitcode;
// Private directory for reusable partition artifacts; empty disables reuse.
std::string CacheDirectory;
// Instruments the program for IR profiling. The instrumented executable
// writes raw profiles to this path pattern (LLVM_PROFILE_FILE overrides it
// at run time). Empty: no instrumentation.
std::string ProfileGenerate;
// Indexed profile (.profdata) that guides optimization. Empty: none.
std::string ProfileUse;
};
std::optional<EmitOptions> parseEmitOptions(int Argc, char **Argv);
native/llvm-codegen/src/main.cpp
+7 -3
View File
@@ -1,5 +1,6 @@
#include "diagnostics.h"
#include "emit.h"
#include "profile.h"
#include "runtime-unit.h"
#include "target.h"
@@ -77,6 +78,8 @@ int main(int Argc, char **Argv) {
return scriptc::runtimeUnit(Argc, Argv);
if (Argc >= 2 && StringRef(Argv[1]) == "prune-import")
return scriptc::pruneImport(Argc, Argv);
if (Argc >= 2 && StringRef(Argv[1]) == "merge-profiles")
return scriptc::mergeProfiles(Argc, Argv);
if (Argc >= 2 && StringRef(Argv[1]) == "emit") {
std::optional<scriptc::EmitOptions> Options =
scriptc::parseEmitOptions(Argc, Argv);
@@ -86,10 +89,11 @@ int main(int Argc, char **Argv) {
"emit requires --input <file> --output <file> and accepts "
"--filetype <obj|asm> --target <triple> --opt-level <0|1|2|3|s|z> "
"--relocation-model pic --diagnostic-format json "
"--source-path <path> --import-bitcode <file> --cache-dir <dir>");
"--source-path <path> --import-bitcode <file> --cache-dir <dir> "
"--profile-generate <pattern> --profile-use <file.profdata>");
return scriptc::emit(*Options);
}
return scriptc::reportError("usage",
"expected 'version --format=json', 'emit', 'runtime-unit', or "
"'prune-import'");
"expected 'version --format=json', 'emit', 'runtime-unit', "
"'prune-import', or 'merge-profiles'");
}
native/llvm-codegen/src/partition.cpp
+28 -1
View File
@@ -359,7 +359,34 @@ std::optional<ProgramPartitions> partitionProgramText(StringRef Text,
} else if (Line.front() == '@') {
Symbol S;
S.Header = Line;
if (!parseGlobal(Line, S) || !Add(std::move(S)))
if (!parseGlobal(Line, S))
return std::nullopt;
// The emitter writes an interface dispatch table one element per
// line: the definition opens its aggregate initializer at the end of
// the line, and a line that is only the closing bracket ends it.
StringRef Opening = Line.rtrim();
if (S.Kind == SymbolKind::Variable &&
(Opening.ends_with(" [") || Opening.ends_with(" {"))) {
StringRef Close = Opening.back() == '[' ? "]" : "}";
size_t Cursor = Next;
while (true) {
if (Cursor >= Text.size())
return std::nullopt;
size_t LineEnd = Text.find('\n', Cursor);
if (LineEnd == StringRef::npos)
LineEnd = Text.size();
StringRef Element = Text.slice(Cursor, LineEnd);
if (Element == Close) {
S.Body = Text.slice(Next, LineEnd);
Next = LineEnd + 1;
break;
}
if (!Element.starts_with(" "))
return std::nullopt;
Cursor = LineEnd + 1;
}
}
if (!Add(std::move(S)))
return std::nullopt;
} else if (Line.starts_with("declare ")) {
Symbol S;
native/llvm-codegen/src/profile.cpp
+103
View File
@@ -0,0 +1,103 @@
#include "profile.h"
#include "diagnostics.h"
#include "llvm/ADT/SmallString.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/ProfileData/InstrProfReader.h"
#include "llvm/ProfileData/InstrProfWriter.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/VirtualFileSystem.h"
#include "llvm/Support/raw_ostream.h"
#include <string>
#include <vector>
using namespace llvm;
namespace scriptc {
int mergeProfiles(int Argc, char **Argv) {
std::string Output;
bool Sparse = false;
std::vector<std::string> Inputs;
for (int I = 2; I < Argc; ++I) {
StringRef Arg(Argv[I]);
if (Arg == "--output" && I + 1 < Argc)
Output = Argv[++I];
else if (Arg == "--sparse")
Sparse = true;
else if (!Arg.starts_with("--"))
Inputs.push_back(Arg.str());
else
return reportError("usage", "merge-profiles requires --output <file> "
"and one or more input profiles, and "
"accepts --sparse");
}
if (Output.empty() || Inputs.empty())
return reportError("usage", "merge-profiles requires --output <file> and "
"one or more input profiles");
// Sparse profiles leave out the functions that never ran, which then
// optimize as without a profile instead of as cold.
InstrProfWriter Writer(Sparse);
IntrusiveRefCntPtr<vfs::FileSystem> FS = vfs::getRealFileSystem();
for (const std::string &Path : Inputs) {
Expected<std::unique_ptr<InstrProfReader>> Reader =
InstrProfReader::create(Path, *FS);
if (!Reader)
return reportError("profile_read_failed",
Path + ": " + toString(Reader.takeError()));
if (Error E = Writer.mergeProfileKind((*Reader)->getProfileKind()))
return reportError("profile_merge_failed",
Path + ": " + toString(std::move(E)));
std::string RecordError;
for (NamedInstrProfRecord &Record : **Reader) {
Writer.addRecord(std::move(Record), /*Weight=*/1, [&](Error E) {
if (RecordError.empty())
RecordError = toString(std::move(E));
else
consumeError(std::move(E));
});
}
if ((*Reader)->hasError())
return reportError("profile_read_failed",
Path + ": " + toString((*Reader)->getError()));
if (!RecordError.empty())
return reportError("profile_merge_failed", Path + ": " + RecordError);
}
// Publish through a uniquely named sibling file, which createUniqueFile
// creates exclusively, as emission does.
SmallString<256> Temporary(Output);
Temporary += ".tmp-%%%%%%";
int Fd = -1;
if (std::error_code EC = sys::fs::createUniqueFile(Temporary, Fd, Temporary))
return reportError("output_open_failed", EC.message());
{
raw_fd_ostream Stream(Fd, /*shouldClose=*/true);
Error WriteError = Writer.write(Stream);
Stream.flush();
std::string StreamError;
if (Stream.has_error()) {
StreamError = Stream.error().message();
Stream.clear_error();
}
if (WriteError) {
sys::fs::remove(Temporary);
return reportError("profile_write_failed",
toString(std::move(WriteError)));
}
if (!StreamError.empty()) {
sys::fs::remove(Temporary);
return reportError("output_write_failed", StreamError);
}
}
if (std::error_code EC = sys::fs::rename(Temporary, Output)) {
sys::fs::remove(Temporary);
return reportError("output_publish_failed", EC.message());
}
outs() << "{\"ok\":true}\n";
return 0;
}
} // namespace scriptc
native/llvm-codegen/src/profile.h
+10
View File
@@ -0,0 +1,10 @@
#pragma once
namespace scriptc {
// `merge-profiles --output <file.profdata> [--sparse] <input>...`: merges raw
// (.profraw) or indexed (.profdata) IR profiles into one indexed profile.
// --sparse leaves out functions that never ran.
int mergeProfiles(int Argc, char **Argv);
} // namespace scriptc
package.json
+1 -1
View File
@@ -23,7 +23,7 @@
"test:sandbox": "node scripts/sandbox-test.mjs",
"test:sandbox:image": "node scripts/sandbox-image.mjs",
"test:watch": "vitest",
"lint": "oxlint --deny-warnings packages scripts internal/compatibility/src tests/harness vitest.config.ts",
"lint": "node docs/scripts/limitations-partials.mjs check && oxlint --deny-warnings packages scripts internal/compatibility/src tests/harness vitest.config.ts",
"lint:fix": "pnpm lint --fix",
"format": "oxfmt --write \"{packages,scripts,internal/compatibility/src,tests/harness}/**/*.{js,mjs,cjs,ts,mts,cts,tsx}\" vitest.config.ts .oxlintrc.jsonc .oxfmtrc.jsonc",
"format:check": "oxfmt --check \"{packages,scripts,internal/compatibility/src,tests/harness}/**/*.{js,mjs,cjs,ts,mts,cts,tsx}\" vitest.config.ts .oxlintrc.jsonc .oxfmtrc.jsonc",
packages/cli/src/bootstrap.ts
+4
View File
@@ -66,6 +66,10 @@ async function tryFastPath(): Promise<number | null> {
values["emit-ir"] ||
values.lib ||
values["provenance-sources"] ||
// The full compiler keys these builds by the profile's bytes.
values["profile-generate"] !== undefined ||
values["profile-use"] !== undefined ||
values.sparse ||
(values["external-types"] ?? []).length > 0
)
return null;
packages/cli/src/main.ts
+2
View File
@@ -6,6 +6,7 @@ import {
analyzeCached,
compile,
compileLibrary,
mergeProfiles,
resolveProvenanceSources,
sourceTargetPlatform,
warmNativeCaches,
@@ -33,6 +34,7 @@ process.exitCode = await runCli(process.argv.slice(2), {
resolveProvenanceSources,
sourceTargetPlatform,
warmNativeCaches,
mergeProfiles,
run: (binary) =>
new Promise<number>((resolveExit) => {
let child;
packages/cli/test/output-options.test.ts
+3
View File
@@ -48,6 +48,9 @@ describe("output option compatibility", () => {
[{ emit: "llvm", strip: true }, /strip/],
[{ emit: "obj", detachedExit: true }, /detached-exit/],
[{ emit: "llvm", detachedExit: true }, /detached-exit/],
[{ emit: "obj", profileGenerate: "/tmp/%p.profraw" }, /profile-generate/],
[{ emit: "llvm", profileUse: "/tmp/a.profdata" }, /profile-use/],
[{ profileGenerate: "/tmp/%p.profraw", profileUse: "/tmp/a.profdata" }, /cannot be combined/],
] as const)("rejects %j", (override, message) => {
const result = resolveOutputOptions("build", { ...BASE, ...override });
expect(result).toEqual({ ok: false, message: expect.stringMatching(message) });
packages/cli/test/profile.test.ts
+120
View File
@@ -0,0 +1,120 @@
import { execFile } from "node:child_process";
import { createRequire } from "node:module";
import { mkdtemp, readdir, rm, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { promisify } from "node:util";
import { expect, test } from "vitest";
const execFileAsync = promisify(execFile);
const require = createRequire(import.meta.url);
const repoRoot = join(import.meta.dirname, "../../..");
const cliEntry = join(repoRoot, "packages/cli/src/main.ts");
const tsxLoader = join(dirname(require.resolve("tsx/package.json")), "dist/loader.mjs");
const PROGRAM = `const n = Number(process.argv[2] ?? "10");
let total = 0;
for (let i = 0; i < n; i++) total += i % 3 === 0 ? i : 1;
console.log(\`total \${total}\`);
if (process.argv[3] === "exit") process.exit(3);
// Never runs in the profiled runs.
function depth(k: number): number {
return k <= 0 ? 0 : 1 + depth(k - 1);
}
if (process.argv[3] === "depth") console.log(depth(n));
`;
async function scriptc(args: string[], env: NodeJS.ProcessEnv) {
return execFileAsync(process.execPath, ["--import", tsxLoader, cliEntry, ...args], {
env,
maxBuffer: 4 * 1024 * 1024,
});
}
test.skipIf(process.platform !== "linux")(
"instrumented runs write profiles that merge and optimize the program, stale or not",
async () => {
const dir = await mkdtemp(join(tmpdir(), "scriptc-profile-"));
const env = { ...process.env, SCRIPTC_CACHE_DIR: join(dir, "cache") };
const entry = join(dir, "main.ts");
const raw = join(dir, "raw");
try {
await writeFile(entry, PROGRAM);
const instrumented = join(dir, "instrumented");
await scriptc(
["build", entry, "--profile-generate", join(raw, "%p.profraw"), "-o", instrumented],
env,
);
expect((await execFileAsync(instrumented, ["1000"])).stdout).toBe("total 167499\n");
// process.exit leaves through _Exit, which skips the profile runtime's
// atexit writer; the runtime writes the profile itself.
await expect(execFileAsync(instrumented, ["1000", "exit"])).rejects.toMatchObject({
code: 3,
stdout: "total 167499\n",
});
const profiles = (await readdir(raw)).filter((name) => name.endsWith(".profraw"));
expect(profiles).toHaveLength(2);
const merged = join(dir, "app.profdata");
const merge = await scriptc(
["merge-profiles", "-o", merged, ...profiles.map((name) => join(raw, name))],
env,
);
expect(merge.stdout).toBe(`${merged}\n`);
const optimized = join(dir, "optimized");
await scriptc(["build", entry, "--profile-use", merged, "-o", optimized], env);
expect((await execFileAsync(optimized, ["1000"])).stdout).toBe("total 167499\n");
// A sparse profile leaves out the runtime functions the runs never
// reached.
const sparse = join(dir, "sparse.profdata");
await scriptc(
["merge-profiles", "--sparse", "-o", sparse, ...profiles.map((name) => join(raw, name))],
env,
);
expect((await stat(sparse)).size).toBeLessThan((await stat(merged)).size);
await scriptc(["build", entry, "--profile-use", sparse, "-o", optimized], env);
expect((await execFileAsync(optimized, ["1000"])).stdout).toBe("total 167499\n");
// A profile from an older version of the program still builds it.
await writeFile(entry, PROGRAM + 'console.log("edited");\n');
await scriptc(["build", entry, "--profile-use", merged, "-o", optimized], env);
expect((await execFileAsync(optimized, ["1000"])).stdout).toBe("total 167499\nedited\n");
} finally {
await rm(dir, { recursive: true, force: true });
}
},
300_000,
);
test("profile options are refused where they cannot apply", async () => {
const dir = await mkdtemp(join(tmpdir(), "scriptc-profile-refusal-"));
const env = { ...process.env, SCRIPTC_CACHE_DIR: join(dir, "cache") };
const entry = join(dir, "main.ts");
try {
await writeFile(entry, PROGRAM);
for (const [args, message] of [
[["build", entry, "--emit=llvm", "--profile-use", "a.profdata"], "--emit=exe"],
[
["build", entry, "--profile-generate", "%p.profraw", "--profile-use", "a.profdata"],
"cannot be combined",
],
[["coverage", entry, "--profile-use", "a.profdata"], "build/run option"],
[["merge-profiles", "a.profraw"], "merge-profiles takes -o"],
[["merge-profiles", "-o", "out.profdata"], "merge-profiles takes -o"],
[["build", entry, "--sparse"], "--sparse is a merge-profiles option"],
[
["build", entry, "--optimization=dev", "--profile-use", join(dir, "a.profdata")],
"optimized build",
],
[["build", entry, "--profile-use", join(dir, "missing.profdata")], "cannot read"],
] as const) {
await expect(scriptc([...args], env)).rejects.toMatchObject({
stderr: expect.stringContaining(message),
});
}
} finally {
await rm(dir, { recursive: true, force: true });
}
}, 120_000);
packages/compiler/ambient/scriptc-node-fallback.d.ts
Vendored
+28
View File
@@ -2933,6 +2933,34 @@ declare module "crypto" {
iv: BinaryLike | null,
options?: { authTagLength?: number },
): Decipheriv;
export type ECDHKeyFormat = "compressed" | "uncompressed" | "hybrid";
export class ECDH {
private constructor();
static convertKey(
key: BinaryLike,
curve: string,
inputEncoding?: BinaryToTextEncoding,
outputEncoding?: "latin1" | "hex" | "base64" | "base64url",
format?: ECDHKeyFormat,
): Buffer | string;
generateKeys(): Buffer;
generateKeys(encoding: BinaryToTextEncoding, format?: ECDHKeyFormat): string;
computeSecret(otherPublicKey: ArrayBufferView): Buffer;
computeSecret(otherPublicKey: string, inputEncoding: BinaryToTextEncoding): Buffer;
computeSecret(otherPublicKey: ArrayBufferView, outputEncoding: BinaryToTextEncoding): string;
computeSecret(
otherPublicKey: string,
inputEncoding: BinaryToTextEncoding,
outputEncoding: BinaryToTextEncoding,
): string;
getPrivateKey(): Buffer;
getPrivateKey(encoding: BinaryToTextEncoding): string;
getPublicKey(encoding?: null, format?: ECDHKeyFormat): Buffer;
getPublicKey(encoding: BinaryToTextEncoding, format?: ECDHKeyFormat): string;
setPrivateKey(privateKey: ArrayBufferView): void;
setPrivateKey(privateKey: string, encoding: BinaryToTextEncoding): void;
}
export function createECDH(curveName: string): ECDH;
export interface ScryptOptions {
N?: number;
r?: number;
packages/compiler/shims/undici/core/client.ts
+204 -69
View File
@@ -328,15 +328,18 @@ export class Client extends DispatcherBase {
}
__onTimeout(): void {
this.__timerCancel = null;
this.__timerRefresh = null;
const socket = this.__socket;
const parser = this.__parser;
// A body timeout that fires while the response is paused keeps its
// timer: resuming refreshes it, as undici's parser does.
if (this.__timerType === TIMEOUT_BODY && parser !== null && parser.paused) return;
this.__timerCancel = null;
this.__timerRefresh = null;
if (socket === null || parser === null) return;
if (this.__timerType === TIMEOUT_HEADERS) {
this.__destroySocket(socket, new HeadersTimeoutError());
} else if (this.__timerType === TIMEOUT_BODY) {
if (!parser.paused) this.__destroySocket(socket, new BodyTimeoutError());
this.__destroySocket(socket, new BodyTimeoutError());
} else if (this.__timerType === TIMEOUT_IDLE) {
this.__destroySocket(socket, new InformationalError("socket idle timeout"));
}
@@ -782,14 +785,50 @@ function errorCode(err: Error): string {
const CRLF = Buffer.from("\r\n");
/** The value of an ASCII hex digit, or -1. */
function hexValue(c: number): number {
// Chunk-size line states (llhttp's chunk_size* and chunk_extension* nodes).
const SIZE_FIRST = 0;
const SIZE_DIGITS = 1;
const SIZE_AFTER = 2;
const SIZE_EXT = 3;
const SIZE_EXT_NAME = 4;
const SIZE_EXT_VALUE = 5;
const SIZE_EXT_QUOTED = 6;
const SIZE_EXT_PAIR = 7;
const SIZE_EXT_QUOTED_DONE = 8;
const SIZE_CR = 9;
function hexDigit(c: number): number {
if (c >= 0x30 && c <= 0x39) return c - 0x30;
if (c >= 0x41 && c <= 0x46) return c - 0x37;
if (c >= 0x61 && c <= 0x66) return c - 0x57;
return -1;
}
/** tchar: ALPHA, DIGIT and !#$%&'*+-.^_`|~. */
const TOKEN_BYTES = tokenBytes();
function tokenBytes(): Uint8Array {
const table = new Uint8Array(256);
const marks = "!#$%&'*+-.^_`|~";
for (let i = 0; i < marks.length; i++) table[marks.charCodeAt(i)] = 1;
for (let c = 0x30; c <= 0x39; c++) table[c] = 1;
for (let c = 0x41; c <= 0x5a; c++) {
table[c] = 1;
table[c + 0x20] = 1;
}
return table;
}
/** qdtext: HTAB, SP and 0x21-0xff but '"' and '\\'. */
function isQuotedByte(c: number): boolean {
return c === 0x09 || c === 0x20 || (c >= 0x21 && c !== 0x22 && c !== 0x5c);
}
/** The byte after '\\' in a quoted-pair: HTAB, SP, VCHAR or obs-text. */
function isQuotedPairByte(c: number): boolean {
return c === 0x09 || c === 0x20 || (c >= 0x21 && c !== 0x7f);
}
/** An incremental HTTP/1.1 response parser (the subset of llhttp undici
* relies on): status line, raw header pairs, Content-Length, chunked and
* read-until-close bodies, trailers, and pause/resume back-pressure. */
@@ -816,14 +855,9 @@ class ResponseParser {
bytesRead: number;
trailers: Buffer[];
skipBody: boolean;
/** The chunk-size line read so far: its value, its digit count, whether
* an extension or whitespace follows the digits, and whether its CR was
* read. Extensions are skipped as they arrive, so the line keeps no
* bytes. */
sizeState: number;
sizeDigits: number;
chunkSize: number;
chunkDigits: number;
chunkExtension: boolean;
chunkCR: boolean;
constructor(client: Client, socket: Socket) {
this.client = client;
@@ -846,10 +880,9 @@ class ResponseParser {
this.bytesRead = 0;
this.trailers = [];
this.skipBody = false;
this.sizeState = SIZE_FIRST;
this.sizeDigits = 0;
this.chunkSize = 0;
this.chunkDigits = 0;
this.chunkExtension = false;
this.chunkCR = false;
}
reset(): void {
@@ -886,6 +919,7 @@ class ResponseParser {
resume(): void {
if (!this.paused) return;
if (this.client.__timerType === TIMEOUT_BODY) this.client.__refreshTimer();
this.paused = false;
this.socket.resume();
this.run();
@@ -903,18 +937,154 @@ class ResponseParser {
}
}
fail(message: string, code: string): void {
// undici 7.x wraps llhttp's reason and leaves the code unset (its
// createError indexes the error-name table with the error number);
// 5.x reports the reason and its HPE_ code.
const error =
/** A parse error with llhttp's reason: undici 7 puts it inside its
* protocol message and, as its error table maps names to numbers only,
* sets no code; undici 5 uses the reason as the message and the code. */
fail(reason: string, code: string): void {
const err =
this.client.__profile.major >= 7
? new HTTPParserError(`Response does not match the HTTP/1.1 protocol (${message})`)
: new HTTPParserError(message, code, "");
this.client.__onParseError(this.socket, error);
? new HTTPParserError(`Response does not match the HTTP/1.1 protocol (${reason})`)
: new HTTPParserError(reason, code, "");
this.client.__onParseError(this.socket, err);
this.state = 99;
}
/** Reads the chunk-size line one byte at a time as llhttp does, so a line
* that never ends holds no memory: hex digits (past 16 significant ones
* the size overflows), then extensions, checked and dropped, then CRLF.
* The 7.x profile follows llhttp 9 (undici 7), the 5.x profile llhttp 8
* in its lenient mode (undici 5). Returns true when the line is complete
* and chunkSize holds its size; false when it needs more bytes or failed. */
chunkSizeLine(): boolean {
const buf = this.buf;
if (this.sizeState === SIZE_FIRST) {
// The common line, hex digits then CRLF in this buffer, in one pass;
// any other line takes the byte loop below from its start.
let size = 0;
let digits = 0;
let j = 0;
for (; j < buf.length && j < 32; j++) {
const d = hexDigit(buf[j]!);
if (d === -1) break;
if (digits > 0 || d !== 0) {
digits++;
size = size * 16 + d;
}
}
if (j > 0 && digits <= 16 && j + 1 < buf.length && buf[j] === 0x0d && buf[j + 1] === 0x0a) {
this.chunkSize = size;
this.buf = buf.subarray(j + 2);
return true;
}
}
const v7 = this.client.__profile.major >= 7;
let i = 0;
while (i < buf.length) {
const c = buf[i]!;
const state = this.sizeState;
if (state === SIZE_FIRST || state === SIZE_DIGITS) {
const d = hexDigit(c);
if (d === -1) {
if (state === SIZE_FIRST) {
this.fail("Invalid character in chunk size", "INVALID_CHUNK_SIZE");
return false;
}
this.sizeState = SIZE_AFTER;
continue;
}
if (state === SIZE_FIRST) {
this.chunkSize = 0;
this.sizeDigits = 0;
this.sizeState = SIZE_DIGITS;
}
if (this.sizeDigits > 0 || d !== 0) {
if (this.sizeDigits === 16) {
this.fail("Chunk size overflow", "INVALID_CHUNK_SIZE");
return false;
}
this.sizeDigits++;
this.chunkSize = this.chunkSize * 16 + d;
}
} else if (state === SIZE_AFTER) {
if (c === 0x0d) this.sizeState = SIZE_CR;
else if (c === 0x3b) this.sizeState = SIZE_EXT;
else if (c === 0x0a && v7) {
this.fail("Missing expected CR after chunk size", "CR_EXPECTED");
return false;
} else {
this.fail("Invalid character in chunk size", "INVALID_CHUNK_SIZE");
return false;
}
} else if (state === SIZE_EXT) {
if (c === 0x20 || c === 0x0d) {
this.fail("Invalid character in chunk extensions", "STRICT");
return false;
}
this.sizeState = SIZE_EXT_NAME;
continue;
} else if (state === SIZE_EXT_NAME) {
if (c === 0x3d) this.sizeState = SIZE_EXT_VALUE;
else if (c === 0x3b) this.sizeState = SIZE_EXT;
else if (c === 0x0d) this.sizeState = SIZE_CR;
else if (c === 0x0a && v7) {
this.fail("Missing expected CR after chunk extension name", "CR_EXPECTED");
return false;
} else if (TOKEN_BYTES[c] !== 1) {
this.fail("Invalid character in chunk extensions name", "STRICT");
return false;
}
} else if (state === SIZE_EXT_VALUE) {
if (c === 0x22) this.sizeState = SIZE_EXT_QUOTED;
// llhttp 8 goes on with the chunk-size node after a value's ';'.
else if (c === 0x3b) this.sizeState = v7 ? SIZE_EXT : SIZE_AFTER;
else if (c === 0x0d) this.sizeState = SIZE_CR;
else if (c === 0x0a && v7) {
this.fail("Missing expected CR after chunk extension value", "CR_EXPECTED");
return false;
} else if (TOKEN_BYTES[c] !== 1) {
this.fail("Invalid character in chunk extensions value", "STRICT");
return false;
}
} else if (state === SIZE_EXT_QUOTED) {
if (c === 0x22) this.sizeState = SIZE_EXT_QUOTED_DONE;
else if (c === 0x5c && v7) this.sizeState = SIZE_EXT_PAIR;
else if (!isQuotedByte(c)) {
this.fail("Invalid character in chunk extensions quoted value", "STRICT");
return false;
}
} else if (state === SIZE_EXT_PAIR) {
if (!isQuotedPairByte(c)) {
this.fail("Invalid quoted-pair in chunk extensions quoted value", "STRICT");
return false;
}
this.sizeState = SIZE_EXT_QUOTED;
} else if (state === SIZE_EXT_QUOTED_DONE) {
if (c === 0x3b) this.sizeState = SIZE_EXT;
else if (c === 0x0d) this.sizeState = SIZE_CR;
else if (c === 0x0a && v7) {
this.fail("Missing expected CR after chunk extension value", "CR_EXPECTED");
return false;
} else {
this.fail("Invalid character in chunk extensions quote value", "STRICT");
return false;
}
} else {
// SIZE_CR: llhttp 9 needs the LF; llhttp 8's lenient mode skips
// whatever byte follows the CR.
if (c !== 0x0a && v7) {
this.fail("Expected LF after chunk size", "STRICT");
return false;
}
this.buf = buf.subarray(i + 1);
this.sizeState = SIZE_FIRST;
return true;
}
i++;
}
this.buf = Buffer.alloc(0);
return false;
}
line(): Buffer | null {
const idx = this.buf.indexOf(CRLF);
if (idx === -1) return null;
@@ -941,7 +1111,12 @@ class ResponseParser {
const text = line.toString("latin1");
const m = /^HTTP\/1\.([01]) (\d{3})(?: (.*))?$/.exec(text);
if (m === null) {
this.fail("Response does not match the HTTP/1.1 protocol", "INVALID_CONSTANT");
const v7 = this.client.__profile.major >= 7;
if (!text.startsWith("HTTP/")) {
this.fail(v7 ? "Expected HTTP/, RTSP/ or ICE/" : "Expected HTTP/", "INVALID_CONSTANT");
} else {
this.fail(v7 ? "Invalid status code" : "Invalid response status", "INVALID_STATUS");
}
return;
}
this.versionMinor = Number(m[1]);
@@ -1055,55 +1230,15 @@ class ResponseParser {
if (!this.complete()) return;
}
} else if (this.state === 3) {
// Chunk size line: hex digits, then optional whitespace and
// extensions, then CRLF. It is read byte by byte as it arrives.
if (this.buf.length === 0) {
if (this.more()) continue;
// Chunk size line; a partial one continues with the next arrival.
if (!this.chunkSizeLine()) {
if (this.state === 3 && this.more()) continue;
return;
}
let i = 0;
let done = false;
for (; i < this.buf.length && !done; i++) {
const c = this.buf[i]!;
if (this.chunkCR) {
if (c !== 0x0a) {
this.fail("Expected LF after chunk size", "STRICT");
return;
}
done = true;
} else if (c === 0x0d) {
if (this.chunkDigits === 0) {
this.fail("Invalid character in chunk size", "INVALID_CHUNK_SIZE");
return;
}
this.chunkCR = true;
} else if (this.chunkExtension) {
continue;
} else if (hexValue(c) >= 0) {
this.chunkSize = this.chunkSize * 16 + hexValue(c);
this.chunkDigits++;
if (this.chunkSize > Number.MAX_SAFE_INTEGER) {
this.fail("Chunk size overflow", "INVALID_CHUNK_SIZE");
return;
}
} else if (this.chunkDigits > 0 && (c === 0x3b || c === 0x20 || c === 0x09)) {
this.chunkExtension = true;
} else {
this.fail("Invalid character in chunk size", "INVALID_CHUNK_SIZE");
return;
}
}
this.buf = this.buf.subarray(i);
if (!done) continue;
const size = this.chunkSize;
this.chunkSize = 0;
this.chunkDigits = 0;
this.chunkExtension = false;
this.chunkCR = false;
if (size === 0) {
if (this.chunkSize === 0) {
this.state = 5;
} else {
this.remaining = size;
this.remaining = this.chunkSize;
this.state = 4;
}
} else if (this.state === 4) {
packages/compiler/shims/undici/core/diagnostics-types.ts
+1 -1
View File
@@ -8,7 +8,7 @@ export declare namespace DiagnosticsChannel {
completed: boolean;
method?: string;
path: string;
headers: string;
headers: string | string[];
addHeader(key: string, value: string): Request;
}
interface Response {
packages/compiler/shims/undici/core/dispatcher.ts
+6
View File
@@ -38,6 +38,12 @@ export interface ResponseData {
const defaultProfile: Profile = { major: 7 };
export class Dispatcher extends EventEmitter {
// An explicit constructor: the implicit one forwards its arguments to
// EventEmitter's options-taking constructor, which stays a runtime fence.
constructor() {
super();
}
dispatch(_options: DispatchOptions, _handler: DispatchHandlers): boolean {
throw new Error("not implemented");
}
packages/compiler/shims/undici/core/fetch.ts
+277 -55
View File
@@ -2,14 +2,16 @@
// program-installed global dispatcher) it is the runtime's own fetch, which
// implements the same WHATWG algorithm. With one, the request goes through
// that dispatcher (its connector, lookup and pool), following undici's
// fetch for request headers, redirects and errors; the response body is
// buffered into a native Response.
// fetch for request headers, redirects and errors. The promise resolves
// when the response headers arrive, and the Response body streams from the
// connection with undici's back-pressure (see dispatchOnce).
//
// Known gaps: a Response built here reports `url` as "" and `redirected` as
// false (the runtime's Response has no settable URL yet), and https requests
// offer `gzip, deflate` rather than undici's `br, gzip, deflate` (the runtime
// has no Brotli decoder).
import { gunzipSync, inflateSync } from "node:zlib";
import type { Transform } from "node:stream";
import * as zlib from "node:zlib";
import type { Dispatcher } from "./dispatcher.ts";
import { getGlobalDispatcher, hasCustomGlobalDispatcher } from "./api.ts";
import type { DispatchHandlers, Profile } from "./types.ts";
@@ -115,7 +117,10 @@ interface RawResponse {
status: number;
statusText: string;
headers: string[];
body: Buffer;
/** null when the response has no body to expose: a null-body status or
* HEAD (its bytes, if any, are read and dropped), or a redirect fetch
* follows or rejects (the request is aborted, closing its connection). */
body: ReadableStream<Uint8Array> | null;
}
const redirectStatuses = [301, 302, 303, 307, 308];
@@ -125,7 +130,7 @@ export function undiciFetch(
input: string | URL,
init: UndiciRequestInit | undefined,
createGlobal: () => Dispatcher,
_profile: Profile,
profile: Profile,
): Promise<Response> {
const explicit = init !== undefined ? init.dispatcher : undefined;
if (explicit === undefined && !hasCustomGlobalDispatcher()) {
@@ -139,7 +144,7 @@ export function undiciFetch(
});
}
const dispatcher = explicit ?? getGlobalDispatcher(createGlobal);
return fetchThrough(dispatcher, input, init ?? {});
return fetchThrough(dispatcher, input, init ?? {}, profile);
}
function normalizeMethod(method: string): string {
@@ -153,6 +158,7 @@ async function fetchThrough(
dispatcher: Dispatcher,
input: string | URL,
init: UndiciRequestInit,
profile: Profile,
): Promise<Response> {
let url = typeof input === "string" ? new URL(input) : new URL(input.href);
let method = normalizeMethod(init.method ?? "GET");
@@ -197,7 +203,7 @@ async function fetchThrough(
let raw: RawResponse;
try {
raw = await dispatchOnce(dispatcher, url, method, pairs, body, signal);
raw = await dispatchOnce(dispatcher, url, method, pairs, body, signal, redirectMode, profile);
} catch (err) {
if (signal !== null && signal.aborted) throw abortReason(signal);
throw new TypeError("fetch failed", { cause: err });
@@ -243,15 +249,11 @@ async function fetchThrough(
continue;
}
const nullBody = nullBodyStatuses.includes(raw.status) || method === "HEAD";
return new Response(
nullBody ? null : decode(raw.body, responseHeaders.get("content-encoding")),
{
status: raw.status,
statusText: raw.statusText,
headers: responseHeaders,
},
);
return new Response(raw.body, {
status: raw.status,
statusText: raw.statusText,
headers: responseHeaders,
});
}
}
@@ -324,20 +326,28 @@ function headerListOf(
return list;
}
function decode(body: Buffer, encoding: string | null): Buffer {
if (encoding === null || body.length === 0) return body;
let out = body;
const codings = encoding
.toLowerCase()
.split(",")
.map((c) => c.trim())
.reverse();
/** undici 7's limit on a response's content codings, applied in both
* profiles: undici 5 has none, and builds one decoder per listed coding. */
const maxContentEncodings = 5;
/** Whether every coding has a decoder; otherwise the body passes through
* as sent. */
function decodable(codings: string[]): boolean {
if (codings.length === 0) return false;
for (const coding of codings) {
if (coding === "gzip" || coding === "x-gzip") out = gunzipSync(out);
else if (coding === "deflate") out = inflateSync(out);
else return body;
if (coding !== "gzip" && coding !== "x-gzip" && coding !== "deflate") return false;
}
return out;
return true;
}
/** A decoder for one coding. undici 7 decodes with Z_SYNC_FLUSH, which
* accepts a truncated stream; undici 5 uses zlib's defaults. */
function decoderFor(coding: string, profile: Profile): Transform {
const options =
profile.major >= 7
? { flush: zlib.constants.Z_SYNC_FLUSH, finishFlush: zlib.constants.Z_SYNC_FLUSH }
: {};
return coding === "deflate" ? zlib.createInflate(options) : zlib.createGunzip(options);
}
function abortReason(signal: AbortSignal): Error {
@@ -347,6 +357,16 @@ function abortReason(signal: AbortSignal): Error {
: new DOMException("This operation was aborted", "AbortError");
}
/** Unread bytes held beside the response stream before the connection
* pauses: Node's Readable highWaterMark, which undici's fetch body uses. */
const bodyHighWaterMark = 64 * 1024;
/** Dispatches one request and resolves when its response headers arrive.
* The body streams as undici's fetch reads one: up to bodyHighWaterMark
* unread bytes wait beside the stream, past that the connection (or the
* last decoder) pauses until the reader pulls again; cancelling the stream
* aborts the request, and an error after the headers errors the stream
* with TypeError('terminated'). */
function dispatchOnce(
dispatcher: Dispatcher,
url: URL,
@@ -354,50 +374,252 @@ function dispatchOnce(
headers: string[],
body: Buffer | null,
signal: AbortSignal | null,
redirectMode: "follow" | "manual" | "error",
profile: Profile,
): Promise<RawResponse> {
return new Promise<RawResponse>((resolve, reject) => {
let status = 0;
let statusText = "";
let rawHeaders: string[] = [];
const chunks: Buffer[] = [];
let settled = false;
let resolved = false;
let done = false;
let dump = false;
let abortFn: ((err?: Error) => void) | null = null;
const onAbort = (): void => {
// The response body: its stream controller, the unread bytes beside it,
// and how to restart the paused producer.
let controller: ReadableStreamDefaultController<Uint8Array> | null = null;
let hasBody = false;
const queue: Buffer[] = [];
let queued = 0;
let ended = false;
let closed = false;
let onDrain: (() => void) | null = null;
// The decoders, first-applied last; their listeners hold this scope, so
// releaseDecoders drops them once the body ends, fails or is cancelled.
let decoding = false;
let decoders: Transform[] = [];
let first: Transform | null = null;
let encodedBytes = 0;
let resumeConnection: (() => void) | null = null;
const abortRequest = (err: Error): void => {
const abort = abortFn;
if (abort !== null && signal !== null) abort(abortReason(signal));
if (abort !== null && !done) abort(err);
};
const onAbort = (): void => {
if (signal === null) return;
const reason = abortReason(signal);
// After the headers undici 7 errors the body with the signal's
// reason; undici 5 aborts its fetch controller without one, so the
// body gets the controller's default AbortError.
errorBody(
profile.major >= 7 ? reason : new DOMException("The operation was aborted.", "AbortError"),
);
abortRequest(reason);
};
const stopListening = (): void => {
if (signal !== null) signal.removeEventListener("abort", onAbort);
};
if (signal !== null) signal.addEventListener("abort", onAbort);
const finish = (err: Error | null): void => {
if (settled) return;
settled = true;
if (signal !== null) signal.removeEventListener("abort", onAbort);
if (err !== null) reject(err);
else resolve({ status, statusText, headers: rawHeaders, body: Buffer.concat(chunks) });
const releaseDecoders = (): void => {
const chain = decoders;
decoders = [];
first = null;
for (const decoder of chain) {
decoder.removeAllListeners();
decoder.destroy();
}
};
const closeBody = (): void => {
closed = true;
controller!.close();
stopListening();
};
const errorBody = (err: unknown): void => {
if (!hasBody || closed) return;
closed = true;
queue.length = 0;
queued = 0;
controller!.error(err);
stopListening();
releaseDecoders();
};
const endBody = (): void => {
if (closed) return;
ended = true;
if (queue.length === 0) closeBody();
};
/** Takes a chunk; false when the producer should pause until onDrain. */
const pushBody = (chunk: Buffer): boolean => {
if (closed) return true;
if (queue.length === 0 && (controller!.desiredSize ?? 0) > 0) {
controller!.enqueue(new Uint8Array(chunk));
return true;
}
queue.push(chunk);
queued += chunk.length;
return queued < bodyHighWaterMark;
};
const pullBody = (): void => {
if (closed) return;
const next = queue.shift();
if (next !== undefined) {
queued -= next.length;
controller!.enqueue(new Uint8Array(next));
}
if (queue.length === 0 && ended) {
closeBody();
return;
}
const drain = onDrain;
if (drain !== null && queued < bodyHighWaterMark) {
onDrain = null;
drain();
}
};
const cancelBody = (reason: unknown): void => {
if (closed) return;
closed = true;
queue.length = 0;
queued = 0;
stopListening();
releaseDecoders();
abortRequest(
reason instanceof Error
? reason
: new DOMException("The operation was aborted.", "AbortError"),
);
};
const failBody = (err: Error): void => {
errorBody(new TypeError("terminated", { cause: err }));
abortRequest(err);
};
const handler: DispatchHandlers = {
onConnect: (abort: (err?: Error) => void): void => {
abortFn = abort;
if (signal !== null && signal.aborted) abort(abortReason(signal));
},
onHeaders: (
statusCode: number,
raw: Buffer[],
_resume: () => void,
text: string,
): boolean => {
onHeaders: (statusCode: number, raw: Buffer[], resume: () => void, text: string): boolean => {
if (statusCode < 200) return true;
status = statusCode;
statusText = text;
rawHeaders = [];
for (const b of raw) rawHeaders.push(b.toString("latin1"));
resolved = true;
resumeConnection = resume;
const rawHeaders: string[] = [];
let location = false;
let encoding: string | null = null;
for (let i = 0; i + 1 < raw.length; i += 2) {
const name = raw[i]!.toString("latin1");
const value = raw[i + 1]!.toString("latin1");
rawHeaders.push(name, value);
const lower = name.toLowerCase();
if (lower === "location") location = true;
else if (lower === "content-encoding")
encoding = encoding === null ? value : `${encoding}, ${value}`;
}
const follows =
redirectStatuses.includes(statusCode) && location && redirectMode !== "manual";
if (follows || nullBodyStatuses.includes(statusCode) || method === "HEAD") {
dump = true;
stopListening();
resolve({ status: statusCode, statusText: text, headers: rawHeaders, body: null });
// A redirect that is followed or rejected never reads its body:
// abort the request so the connection is released instead of
// draining a body that may never end.
if (follows) abortRequest(new DOMException("The operation was aborted.", "AbortError"));
return true;
}
const codings =
encoding === null
? []
: encoding
.toLowerCase()
.split(",")
.map((c) => c.trim());
if (codings.length > maxContentEncodings) {
dump = true;
stopListening();
reject(
new Error(
`too many content-encodings in response: ${codings.length}, maximum allowed is ${maxContentEncodings}`,
),
);
abortRequest(new DOMException("The operation was aborted.", "AbortError"));
return true;
}
hasBody = true;
const stream = new ReadableStream<Uint8Array>({
start: (c: ReadableStreamDefaultController<Uint8Array>): void => {
controller = c;
},
pull: (): void => pullBody(),
cancel: (reason: unknown): void => cancelBody(reason),
});
if (decodable(codings)) {
// Decoders run last-applied coding first, piped into each other;
// the last one feeds the body.
decoding = true;
let last: Transform | null = null;
for (let i = codings.length - 1; i >= 0; i--) {
const decoder = decoderFor(codings[i]!, profile);
decoder.on("error", (err: Error) => failBody(err));
if (last === null) first = decoder;
else last.pipe(decoder);
decoders.push(decoder);
last = decoder;
}
const output = last!;
output.on("data", (chunk: Buffer) => {
if (!pushBody(chunk)) {
output.pause();
onDrain = (): void => {
output.resume();
};
}
});
output.on("end", () => {
releaseDecoders();
endBody();
});
first!.on("drain", () => resume());
}
resolve({ status: statusCode, statusText: text, headers: rawHeaders, body: stream });
return true;
},
onData: (chunk: Buffer): boolean => {
chunks.push(chunk);
return true;
if (dump || !hasBody || closed) return true;
if (decoding) {
const decoder = first;
if (decoder === null) return true;
encodedBytes += chunk.length;
return decoder.write(chunk);
}
if (pushBody(chunk)) return true;
onDrain = resumeConnection;
return false;
},
onComplete: (_trailers: string[] | null): void => {
done = true;
if (!hasBody) {
stopListening();
return;
}
// zlib rejects an empty stream; undici ends such a body unread.
const decoder = first;
if (decoder !== null && encodedBytes > 0) {
decoder.end();
} else {
releaseDecoders();
endBody();
}
},
onError: (err: Error): void => {
done = true;
if (!resolved) {
stopListening();
reject(err);
return;
}
errorBody(new TypeError("terminated", { cause: err }));
},
onComplete: (_trailers: string[] | null): void => finish(null),
onError: (err: Error): void => finish(err),
};
dispatcher.dispatch(
{
packages/compiler/shims/undici/core/request.ts
+34 -8
View File
@@ -32,8 +32,10 @@ export class Request {
error: Error | null;
abortFn: ((err?: Error) => void) | null;
handler: DispatchHandlers;
/** The request as diagnostics_channel subscribers see it: one boxed
* value per request, so its identity is stable across publications. */
profile: Profile;
/** The request as diagnostics_channel subscribers see it: one untyped
* object per request, so its identity is stable across publications.
* The request keeps its completed and headers fields current. */
diagnostic: unknown;
constructor(origin: string, opts: DispatchOptions, handler: DispatchHandlers, profile: Profile) {
@@ -91,6 +93,7 @@ export class Request {
}
this.completed = false;
this.profile = profile;
this.aborted = false;
this.idempotent = opts.idempotent ?? (method === "HEAD" || method === "GET");
this.blocking = opts.blocking ?? (profile.major >= 7 ? method !== "HEAD" : false);
@@ -131,18 +134,29 @@ export class Request {
}
diagnosticView(): unknown {
let view = this.diagnostic;
if (view === null) {
view = {
if (this.diagnostic === null) {
// addHeader adds a header as undici's Request.addHeader does (the
// tracing instrumentations' header injection).
const view: any = {
origin: this.origin,
completed: this.completed,
method: this.method,
path: this.path,
headers: this.headers,
headers: this.diagnosticHeaders(),
addHeader: (key: string, value: string): unknown => {
this.addHeader(key, value);
return view;
},
};
this.diagnostic = view;
}
return view;
return this.diagnostic;
}
/** The diagnostics view's headers: undici 7 publishes its header array,
* undici 5 the raw block. */
diagnosticHeaders(): string | string[] {
return this.profile.major >= 7 ? this.headerPairs.slice() : this.headers;
}
/** The DiagnosticsChannel.Request view's raw header block. */
@@ -155,7 +169,9 @@ export class Request {
}
addHeader(key: string, value: string): Request {
this.headerPairs.push(key, value);
processHeader(this, key, value, this.profile);
const view: any = this.diagnostic;
if (view !== null) view.headers = this.diagnosticHeaders();
return this;
}
@@ -207,6 +223,8 @@ export class Request {
onComplete(trailers: Buffer[]): void {
this.completed = true;
const view: any = this.diagnostic;
if (view !== null) view.completed = true;
if (channels.trailers.hasSubscribers)
channels.trailers.publish({ request: this.diagnosticView(), trailers });
if (this.handler.onComplete !== undefined) {
@@ -227,10 +245,18 @@ export class Request {
function processHeader(request: Request, key: string, val: string, profile: Profile): void {
const lower = key.toLowerCase();
// undici 7 checks the name and every value before the special cases;
// undici 5 checks the first Host value before storing it. Either way a
// CR or LF in a value never reaches the request head.
if (profile.major >= 7) {
if (!tokenRegExp.test(lower)) throw new InvalidArgumentError("invalid header key");
if (headerCharRegex.test(val)) throw new InvalidArgumentError(`invalid ${key} header`);
}
if (lower === "host") {
// The client writes the Host value into the request head as given.
if (headerCharRegex.test(val)) throw new InvalidArgumentError(`invalid ${key} header`);
if (request.host === null) {
if (headerCharRegex.test(val)) throw new InvalidArgumentError(`invalid ${key} header`);
request.host = val;
return;
}
packages/compiler/shims/undici/core/timers.ts
+2
View File
@@ -35,7 +35,9 @@ function onTick(): void {
}
}
fastTimers.length = len;
// An idle tick stops; the next timer schedules a new one.
if (fastTimers.length !== 0) refreshTimeout();
else refreshTick = null;
}
function refreshTimeout(): void {
packages/compiler/src/backend/executable-features.ts
+5
View File
@@ -27,6 +27,8 @@ export function executableLinkFeatures(mod: IrModule): NativeLinkFeatures {
dynAsync: features.dynAsync,
...(features.dynTier ? { dynTier: true } : {}),
...(features.streamDyn ? { streamDyn: true } : {}),
...(features.handleProps ? { handleProps: true } : {}),
...(features.dynForAwait ? { dynForAwait: true } : {}),
...(features.eventsOnceSignal ? { eventsOnceSignal: true } : {}),
...(features.dynBufferNum ? { dynBufferNum: true } : {}),
...(features.builtinValues ? { builtinValues: true } : {}),
@@ -36,6 +38,7 @@ export function executableLinkFeatures(mod: IrModule): NativeLinkFeatures {
bigint: features.bigint,
searchParams: features.searchParams,
qs: features.qs,
utilPromisify: features.utilPromisify,
dynBuiltinClass: features.dynBuiltinClass,
dynBuiltinStream: features.dynBuiltinStream,
parseArgs: features.parseArgs,
@@ -45,12 +48,14 @@ export function executableLinkFeatures(mod: IrModule): NativeLinkFeatures {
http2: features.http2,
dgram: features.dgram,
dnsPromises: features.dnsPromises,
dgramDyn: features.dgramDyn,
watch: features.fsWatch,
foreignFfi: hasForeignFfiCallback(mod.ffiImports ?? []),
nodeTest: features.nodeTest,
tls: features.tls,
tlsCa: features.tlsCa,
cipher: features.cipher,
ecdh: features.ecdh,
webcrypto: features.webcrypto,
perfLoop: features.perfLoop,
};
packages/compiler/src/backend/library-pack.ts
+3
View File
@@ -42,6 +42,7 @@ export async function compilePackedLibrary(
bigint: options.bigint ?? false,
searchParams: options.searchParams ?? false,
qs: false,
utilPromisify: false,
dynBuiltinClass: false,
dynBuiltinStream: false,
parseArgs: false,
@@ -51,11 +52,13 @@ export async function compilePackedLibrary(
http2: false,
dgram: false,
dnsPromises: false,
dgramDyn: false,
watch: false,
foreignFfi: false,
nodeTest: false,
tls: false,
cipher: false,
ecdh: false,
webcrypto: false,
perfLoop: false,
tlsCa: false,
packages/compiler/src/backend/link-plan-core.ts
+4
View File
@@ -7,6 +7,7 @@ import {
type NativeTargetSpec,
type WindowsSubsystem,
} from "./targets.js";
import { profileGenerateLinkerArgs } from "./native-profile.js";
export interface ExecutableLinkInputs {
inputs: string[];
@@ -28,6 +29,8 @@ export function executableLinkInputs(options: {
optimization: NativeOptimization;
strip?: boolean;
windowsSubsystem?: WindowsSubsystem;
/** Instrumented for profile-guided optimization. */
profileGenerate?: boolean;
}): ExecutableLinkInputs {
if (options.ffiFrameworks.length !== 0 && options.target.platform !== "darwin")
throw new Error("FFI frameworks require a Darwin target");
@@ -63,6 +66,7 @@ export function executableLinkInputs(options: {
...executableOptimizationLinkerArgs(options.target.platform, options.optimization),
...executableStripLinkerArgs(options.target.platform, options.strip ?? false),
...windowsSubsystemLinkerArgs(options.target.platform, options.windowsSubsystem),
...profileGenerateLinkerArgs(options.profileGenerate ?? false),
],
};
}
packages/compiler/src/backend/link-plan.ts
+2
View File
@@ -40,6 +40,7 @@ export async function createNativeLinkPlan(options: {
optimization: NativeOptimization;
strip?: boolean;
windowsSubsystem?: WindowsSubsystem;
profileGenerate?: boolean;
programObjectDependencies?: readonly NativeArtifactDependency[];
env?: NodeJS.ProcessEnv;
resolver?: (specifier: string) => string;
@@ -72,6 +73,7 @@ export async function createNativeLinkPlan(options: {
...(options.windowsSubsystem === undefined
? {}
: { windowsSubsystem: options.windowsSubsystem }),
...(options.profileGenerate === true ? { profileGenerate: true } : {}),
}),
dependencyPaths: [...runtimePack.dependencyPaths, ...(options.ffi?.libraries ?? [])],
programObjectDependencies: [...(options.programObjectDependencies ?? [])],
packages/compiler/src/backend/llvm/dyn.ts
+24 -16
View File
@@ -86,6 +86,7 @@ export const DYN_HANDLE_TAG_NUM: Record<string, number> = {
child: 16,
fileHandle: 17,
dir: 44,
dgramSocket: 62,
cryptoHash: 52,
cryptoHmac: 53,
};
@@ -96,6 +97,15 @@ export const DYN_HANDLE_TAG_NUM: Record<string, number> = {
const DYN_HANDLE_BOX_FN: Record<string, string> = {
cryptoHash: "scr_crypto_hash_to_dyn",
cryptoHmac: "scr_crypto_hash_to_dyn",
dgramSocket: "scr_dgram_to_dyn",
};
/** Handle kinds whose box holds a unit-owned wrapper rather than the
* handle itself (a dgram socket's box carries its untyped listeners):
* the unit's own function unwraps it, with the tag check and TypeError
* of scr_dyn_handle_unbox. */
const DYN_HANDLE_UNBOX_FN: Record<string, string> = {
dgramSocket: "scr_dgram_from_dyn",
};
export const DYN_KIND = {
@@ -1448,29 +1458,22 @@ export class LlDyn {
* differently), runs the checked-dynamic route — scr_json_parse plus the
* dynCheck builder — which reports the exact error. Returns +1, or null
* with the pending flag set. */
jsonParseHelper(t: IrType, fallback = false): string | null {
jsonParseHelper(t: IrType): string | null {
if (t.kind !== "record" && t.kind !== "array") return null;
const key = typeKey(t);
const cacheKey = fallback ? `fallback:${key}` : key;
const existing = this.jsonParsers.get(cacheKey);
const existing = this.jsonParsers.get(key);
if (existing) return existing;
if (!this.jsonSchemaSupported(t, new Set())) return null;
const host = this.host;
const name = `sc_jp_${this.jsonParsers.size}`;
this.jsonParsers.set(cacheKey, name);
this.jsonParsers.set(key, name);
const schema = this.jsonSchema(t);
const check = this.dynCheckHelper(t);
host.declare(`declare ptr @scr_json_parse_schema(ptr, ptr)`);
host.declare(`declare ptr @scr_json_parse(ptr)`);
host.declare(`declare void @scr_dyn_release(ptr)`);
// A fallback site (dynCheck.fallback): a value that fails validation
// parks as the checked value (scr_dyn_check_recover) and the helper
// returns the dummy with no exception pending. Syntax errors still
// throw.
if (fallback) host.declare(`declare zeroext i1 @scr_dyn_check_recover(ptr)`);
const recover = fallback ? [` call zeroext i1 @scr_dyn_check_recover(ptr %d)`] : [];
this.defs.push(
`define internal ptr @${name}(ptr %text) ${FN_ATTRS} { ; JSON.parse as ${llvmCommentText(key)}${fallback ? " (fallback)" : ""}`,
`define internal ptr @${name}(ptr %text) ${FN_ATTRS} { ; JSON.parse as ${llvmCommentText(key)}`,
`entry:`,
` %fast = call ptr @scr_json_parse_schema(ptr %text, ptr @${schema})`,
` %hit = icmp ne ptr %fast, null`,
@@ -1485,7 +1488,6 @@ export class LlDyn {
` ret ptr null`,
`check:`,
` %v = call ptr @${check}(ptr %d, ptr null)`,
...recover,
` call void @scr_dyn_release(ptr %d)`,
` ret ptr %v`,
`}`,
@@ -2500,11 +2502,17 @@ export class LlDyn {
// TypeError on any other kind or tag).
const h = DYN_HANDLE_KINDS.get(t.kind);
if (h) {
host.declare(`declare ptr @scr_dyn_handle_unbox(ptr, i32, ptr, ptr)`);
const r = B.tmp();
B.line(
`${r} = call ptr @scr_dyn_handle_unbox(ptr %d, i32 ${DYN_HANDLE_TAG_NUM[t.kind]}, ptr %path, ptr ${want})`,
);
const unbox = DYN_HANDLE_UNBOX_FN[t.kind];
if (unbox) {
host.declare(`declare ptr @${unbox}(ptr, ptr, ptr)`);
B.line(`${r} = call ptr @${unbox}(ptr %d, ptr %path, ptr ${want})`);
} else {
host.declare(`declare ptr @scr_dyn_handle_unbox(ptr, i32, ptr, ptr)`);
B.line(
`${r} = call ptr @scr_dyn_handle_unbox(ptr %d, i32 ${DYN_HANDLE_TAG_NUM[t.kind]}, ptr %path, ptr ${want})`,
);
}
B.terminate(`ret ptr ${r}`);
break;
}
packages/compiler/src/backend/llvm/emit-function.ts
+5 -2
View File
@@ -27,7 +27,7 @@ import { findMapEntryFusions } from "./map-entry-fusion.js";
import { findLocalStackUnions } from "./stack-unions.js";
import { findLocalStackObjects, findReassignedStackObjects } from "./stack-objects.js";
import { findLocalUnionStorage } from "./local-union-storage.js";
import { INT32_RANGE, analyzeIntegerRanges } from "../../ir/integer-ranges.js";
import { INT32_RANGE, analyzeIntegerRangeFacts } from "../../ir/integer-ranges.js";
import { findBytesBounds } from "./bytes-bounds.js";
import { findInt32Locals } from "./int32-locals.js";
import { findIntegerViews } from "./integer-views.js";
@@ -59,6 +59,7 @@ export function emitFunction(emitter: LlEmitter, fn: IrFunction): string {
const B = new BlockBuilder();
emitter.B = B;
emitter.currentFunctionName = fn.name;
emitter.currentTier = fn.tier;
// Fiber switches restore the active exception cell before control
// returns to a synchronous frame (emitPendingCheck).
B.excCellInvariant = fn.async !== true && fn.generator === undefined;
@@ -220,7 +221,9 @@ export function emitFunction(emitter: LlEmitter, fn: IrFunction): string {
emitter.localUnionStorage = new Map();
emitter.walkBorrows = emitter.walkBorrowsByFunction.get(fn.name) ?? new Set();
const slotFacts = emitter.int32Slots.facts(fn.name);
emitter.integerRanges = analyzeIntegerRanges(numericFn, slotFacts);
const rangeFacts = analyzeIntegerRangeFacts(numericFn, slotFacts);
emitter.integerRanges = rangeFacts.ranges;
emitter.integerLoopRanges = rangeFacts.loopRanges;
emitter.int32Elements = slotFacts.elements ?? null;
emitter.bytesBounds = findBytesBounds(numericFn, emitter.integerRanges);
// Debug builds keep every number local a double for the debugger's
packages/compiler/src/backend/llvm/emit-module.ts
+82 -50
View File
@@ -1,6 +1,7 @@
/* LlEmitter's module-level output: module parts, helper definitions, library declarations and FFI callback definitions. */
import { emitWasmModules } from "./wasm-modules.js";
import { type LlEmitter, ffiCallbackDummyLl, llStrBytes } from "./emitter.js";
import { JS_IC_WORDS } from "./js-ic.js";
import {
mangleAsyncSpawn,
mangleClassObj,
@@ -34,6 +35,8 @@ import { emitClassMembershipHelper } from "./classes.js";
import { type RegexMatcherCode, compileRegexMatcher } from "./regex-matchers.js";
import {
MAP_ITER_DECL,
MAP_ITER_NUM_DECL,
SCR_MAP_NUM_DEAD,
NUMBER_MAP_ENTRY_DECL,
STRING_MAP_ENTRY_DECL,
} from "./map-number-lookup.js";
@@ -449,25 +452,29 @@ export function emitModuleParts(emitter: LlEmitter): string[] {
// The ScrMap prefix { rc, key_kind, val_kind, traced } — the inline
// RC fast paths test the traced byte (shapes.ts).
...(emitter.rcHelpers === null ? [] : [`%ScrMapRc = type { ${emitter.sizeType}, i8, i8, i8 }`]),
// The complete ScrMap and its { key, val, hash } entry, read by the
// inline number-key lookup (sc_map_entry_f64) and the inline
// iteration steps (sc_map_iter_entry): field 8 is the i32 entry
// The complete ScrMap and its entries, read by the inline number-key
// lookup (sc_map_entry_f64) and the inline iteration steps
// (sc_map_iter_entry, sc_map_iter_entry_num): field 8 is the i32 entry
// count, field 10 the i64 absent-key filter, entries is field 11,
// nbuckets and buckets fields 13 and 14, the direct index and its i32
// length fields 15 and 16.
...(emitter.decls.has(NUMBER_MAP_ENTRY_DECL) ||
emitter.decls.has(STRING_MAP_ENTRY_DECL) ||
emitter.decls.has(MAP_ITER_DECL)
emitter.decls.has(MAP_ITER_DECL) ||
emitter.decls.has(MAP_ITER_NUM_DECL)
? [
`%ScrMapIx = type { ${emitter.sizeType}, i8, i8, i8, i8, i32, ptr, ptr, i32, i32, i64, ptr, i32, i32, ptr, ptr, i32, i32 }`,
]
: []),
// String-keyed reference reads load the value slot of the entry
// scr_map_entry_str returns (map-number-lookup.ts).
// The { key, val, hash } entry of every map but a number-key one, whose
// entry is { key, val } (scr_map.h ScrMapNumEntry). String-keyed
// reference reads load the value slot of the entry scr_map_entry_str
// returns (map-number-lookup.ts).
...(emitter.decls.has(NUMBER_MAP_ENTRY_DECL) ||
emitter.decls.has(STRING_MAP_ENTRY_DECL) ||
emitter.decls.has(MAP_ITER_DECL)
? [`%ScrMapEntry = type { i64, i64, i64 }`]
emitter.decls.has(MAP_ITER_DECL) ||
emitter.decls.has(MAP_ITER_NUM_DECL)
? [`%ScrMapEntry = type { i64, i64, i64 }`, `%ScrMapNumEntry = type { i64, i64 }`]
: []),
// The runtime error prefix { rc, vt, name, message, code, cause } and the
// class-object shape { rc, pre, post, ctor, name } — field reads on
@@ -559,6 +566,8 @@ export function emitModuleParts(emitter: LlEmitter): string[] {
out.push(
`@sc_dyn_read_ic_${site} = internal ${tl}global { ptr, ${emitter.sizeType}, ${emitter.sizeType}, ptr } zeroinitializer`,
);
for (let site = 0; site < emitter.jsIcSiteCount; site++)
out.push(`@sc_js_ic_${site} = internal ${tl}global [${JS_IC_WORDS} x i64] zeroinitializer`);
}
for (const table of emitter.constantNumericTables.values()) {
const n = table.values.length;
@@ -1083,7 +1092,7 @@ export function helperDefs(emitter: LlEmitter): string[] {
sz === "i32" ? ` %e = add i32 %e32, 0` : ` %e = zext i32 %e32 to ${sz}`,
` %ep = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 11`,
` %entries = load ptr, ptr %ep`,
` %found = getelementptr inbounds %ScrMapEntry, ptr %entries, ${sz} %e`,
` %found = getelementptr inbounds %ScrMapNumEntry, ptr %entries, ${sz} %e`,
` ret ptr %found`,
`miss:`,
` ret ptr null`,
@@ -1135,7 +1144,8 @@ export function helperDefs(emitter: LlEmitter): string[] {
` %ents = load ptr, ptr %eps`,
` %linear = icmp eq ${sz} %nb, 0`,
` br i1 %linear, label %scan, label %buckets`,
// Small maps keep no bucket table: scan the cached entry hashes.
// Small maps keep no bucket table: scan the entries' keys. A number-key
// entry has no cached hash; a dead entry's key matches no key.
`scan:`,
` %nep = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 8`,
...(sz === "i64"
@@ -1147,14 +1157,10 @@ export function helperDefs(emitter: LlEmitter): string[] {
` %sdone = icmp eq ${sz} %si, %ne`,
` br i1 %sdone, label %miss, label %scan.entry`,
`scan.entry:`,
` %se = getelementptr inbounds %ScrMapEntry, ptr %ents, ${sz} %si`,
` %seh = getelementptr inbounds %ScrMapEntry, ptr %se, i32 0, i32 2`,
` %sh = load i64, ptr %seh`,
` %se = getelementptr inbounds %ScrMapNumEntry, ptr %ents, ${sz} %si`,
` %sk = load i64, ptr %se`,
` %shq = icmp eq i64 %sh, %h`,
` %skq = icmp eq i64 %sk, %kb`,
` %sq = and i1 %shq, %skq`,
` br i1 %sq, label %scan.found, label %scan.next`,
` br i1 %skq, label %scan.found, label %scan.next`,
`scan.found:`,
` ret ptr %se`,
`scan.next:`,
@@ -1189,14 +1195,10 @@ export function helperDefs(emitter: LlEmitter): string[] {
` %bix = and i32 %b, 16777215`,
` %bix1 = sub i32 %bix, 1`,
sz === "i32" ? ` %bi = add i32 %bix1, 0` : ` %bi = zext i32 %bix1 to ${sz}`,
` %pe = getelementptr inbounds %ScrMapEntry, ptr %ents, ${sz} %bi`,
` %peh = getelementptr inbounds %ScrMapEntry, ptr %pe, i32 0, i32 2`,
` %ph = load i64, ptr %peh`,
` %pe = getelementptr inbounds %ScrMapNumEntry, ptr %ents, ${sz} %bi`,
` %pk = load i64, ptr %pe`,
` %phq = icmp eq i64 %ph, %h`,
` %pkq = icmp eq i64 %pk, %kb`,
` %pq = and i1 %phq, %pkq`,
` br i1 %pq, label %probe.found, label %probe.next`,
` br i1 %pkq, label %probe.found, label %probe.next`,
`probe.found:`,
` ret ptr %pe`,
`probe.next:`,
@@ -1214,36 +1216,66 @@ export function helperDefs(emitter: LlEmitter): string[] {
}
// Map/Set iteration steps read the entries array inline: an index is
// live when it is below the entry count and its entry's cached hash is
// nonzero (a tombstone's is zero), as scr_map_iter_live decides. The
// entry count is read again at every step, so entries appended during
// the iteration are visited.
if (emitter.decls.has(MAP_ITER_DECL)) {
// nonzero (a tombstone's is zero), as scr_map_iter_live decides, or for
// a number-key map when its key is not SCR_MAP_NUM_DEAD. The entry count
// is read again at every step, so entries appended during the iteration
// are visited.
if (emitter.decls.has(MAP_ITER_DECL) || emitter.decls.has(MAP_ITER_NUM_DECL)) {
const sz = emitter.sizeType;
if (emitter.decls.has(MAP_ITER_DECL))
defs.push(
`define internal ptr @sc_map_iter_entry(ptr %m, double %i) ${FN_ATTRS} {`,
`entry:`,
` %np = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 8`,
` %n = load i32, ptr %np`,
` %nf = uitofp i32 %n to double`,
` %lo = fcmp oge double %i, 0.0`,
` %hi = fcmp olt double %i, %nf`,
` %in = and i1 %lo, %hi`,
` br i1 %in, label %load, label %none`,
`load:`,
` %ix = fptoui double %i to ${sz}`,
` %ep = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 11`,
` %entries = load ptr, ptr %ep`,
` %e = getelementptr inbounds %ScrMapEntry, ptr %entries, ${sz} %ix`,
` %hp = getelementptr inbounds %ScrMapEntry, ptr %e, i32 0, i32 2`,
` %h = load i64, ptr %hp`,
` %live = icmp ne i64 %h, 0`,
` br i1 %live, label %yes, label %none`,
`yes:`,
` ret ptr %e`,
`none:`,
` ret ptr null`,
`}`,
``,
);
if (emitter.decls.has(MAP_ITER_NUM_DECL))
defs.push(
`define internal ptr @sc_map_iter_entry_num(ptr %m, double %i) ${FN_ATTRS} {`,
`entry:`,
` %np = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 8`,
` %n = load i32, ptr %np`,
` %nf = uitofp i32 %n to double`,
` %lo = fcmp oge double %i, 0.0`,
` %hi = fcmp olt double %i, %nf`,
` %in = and i1 %lo, %hi`,
` br i1 %in, label %load, label %none`,
`load:`,
` %ix = fptoui double %i to ${sz}`,
` %ep = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 11`,
` %entries = load ptr, ptr %ep`,
` %e = getelementptr inbounds %ScrMapNumEntry, ptr %entries, ${sz} %ix`,
` %key = load i64, ptr %e`,
` %live = icmp ne i64 %key, ${SCR_MAP_NUM_DEAD}`,
` br i1 %live, label %yes, label %none`,
`yes:`,
` ret ptr %e`,
`none:`,
` ret ptr null`,
`}`,
``,
);
defs.push(
`define internal ptr @sc_map_iter_entry(ptr %m, double %i) ${FN_ATTRS} {`,
`entry:`,
` %np = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 8`,
` %n = load i32, ptr %np`,
` %nf = uitofp i32 %n to double`,
` %lo = fcmp oge double %i, 0.0`,
` %hi = fcmp olt double %i, %nf`,
` %in = and i1 %lo, %hi`,
` br i1 %in, label %load, label %none`,
`load:`,
` %ix = fptoui double %i to ${sz}`,
` %ep = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 11`,
` %entries = load ptr, ptr %ep`,
` %e = getelementptr inbounds %ScrMapEntry, ptr %entries, ${sz} %ix`,
` %hp = getelementptr inbounds %ScrMapEntry, ptr %e, i32 0, i32 2`,
` %h = load i64, ptr %hp`,
` %live = icmp ne i64 %h, 0`,
` br i1 %live, label %yes, label %none`,
`yes:`,
` ret ptr %e`,
`none:`,
` ret ptr null`,
`}`,
``,
`define internal double @sc_map_iter_count(ptr %m) ${FN_ATTRS} {`,
`entry:`,
` %np = getelementptr inbounds %ScrMapIx, ptr %m, i32 0, i32 8`,
packages/compiler/src/backend/llvm/emit-statements.ts
+93 -8
View File
@@ -56,7 +56,9 @@ import {
SPECULATIVE_RANGE,
emitSafeIntegerCheck,
emitSpeculativeInteger,
planProvenIntegerLoop,
planSpeculativeLoop,
type ProvenLoopPlan,
speculativeReads,
speculativeWriteTarget,
} from "./speculative-integers.js";
@@ -1802,7 +1804,9 @@ export function emitLoopCopies(
)
: null;
if (!plan) {
emitCopy(exit);
const proven = allowed ? planProvenLoop(emitter, loop) : null;
if (proven) emitProvenLoop(emitter, proven, exit, emitCopy);
else emitCopy(exit);
return;
}
const B = emitter.B;
@@ -1881,7 +1885,7 @@ export function emitLoopCopies(
const leave = B.newLabel("spec.leave");
emitCopy(leave);
B.startBlock(leave);
emitter.writeBackSpeculativeGlobals(deferred);
emitter.writeBackDeferred(deferred);
B.br(exit);
}
} finally {
@@ -1890,10 +1894,91 @@ export function emitLoopCopies(
}
}
/** A proven integer loop (speculative-integers.ts) for this loop, or null.
* A shadowed local's double is stale until the loop is left, which only
* the loop's exit, a return and an exception do: the loop is planned only
* where an exception leaves the function (no enclosing try or finally, no
* coroutine frame), so no handler reads the stale double. */
function planProvenLoop(
emitter: LlEmitter,
loop: Extract<IrStmt, { kind: "while" | "doWhile" | "for" }>,
): ProvenLoopPlan | null {
if (
loop.kind !== "for" ||
!emitter.speculativeLoopsEnabled ||
emitter.speculation !== null ||
emitter.speculationResume !== null ||
emitter.tryStack.length > 0 ||
emitter.finallyStack.length > 0 ||
emitter.currentCoro !== null
)
return null;
const cursors = emitter.integerLoopRanges.get(loop);
if (!cursors) return null;
return planProvenIntegerLoop(
loop,
cursors,
(id) => emitter.currentLocals.has(id) && emitter.speculativeLocal(id),
(id) => emitter.integerLoopBindings.has(id) || emitter.int32Locals.has(id),
(e) => {
const range = emitter.integerRanges.get(e);
return (
range !== undefined &&
range !== null &&
range.min >= -Number.MAX_SAFE_INTEGER &&
range.max <= Number.MAX_SAFE_INTEGER
);
},
);
}
/** Emit a proven integer loop: its shadowed locals start from their
* doubles (integers in their loop ranges on entry), its planned writes
* compute integers into the shadows only, and leaving the loop through its
* exit writes each double back. */
function emitProvenLoop(
emitter: LlEmitter,
plan: ProvenLoopPlan,
exit: string,
emitCopy: (exit: string) => void,
): void {
const B = emitter.B;
for (const [id, range] of plan.locals) {
const shadow = B.slot();
B.entryAllocas.push(
`${shadow} = alloca i64 ; integer ${emitter.currentLocals.get(id)?.name ?? id}`,
);
const value = B.tmp(),
integer = B.tmp();
B.line(`${value} = load double, ptr ${emitter.binding(id).slot}`);
B.line(`${integer} = fptosi double ${value} to i64`);
B.line(`store i64 ${integer}, ptr ${shadow}`);
emitter.integerLoopBindings.set(id, { slot: shadow, type: "i64", signed: true, range });
}
const deferred = new Set<string>();
for (const id of plan.locals.keys()) deferred.add(id);
emitter.speculation = { writes: plan.writes, resume: new Map(), deferred };
try {
const leave = B.newLabel("integer.leave");
emitCopy(leave);
B.startBlock(leave);
emitter.writeBackDeferred(deferred);
B.br(exit);
} finally {
emitter.speculation = null;
for (const id of plan.locals.keys()) emitter.integerLoopBindings.delete(id);
}
}
/** A speculated write in the fast copy: compute the value as a safe
* integer, store it to the shadow and its exact double to the local.
* Any failed check re-executes the statement in the ordinary copy. */
export function emitSpeculativeWrite(emitter: LlEmitter, s: IrStmt, resume: string): void {
* Any failed check re-executes the statement in the ordinary copy. A
* proven integer loop's write (no `resume`) has nothing to check. */
export function emitSpeculativeWrite(
emitter: LlEmitter,
s: IrStmt,
resume: string | undefined,
): void {
const B = emitter.B;
const target = speculativeWriteTarget(s)!;
const binding = emitter.integerLoopBindings.get(target.localId)!;
@@ -1901,7 +1986,7 @@ export function emitSpeculativeWrite(emitter: LlEmitter, s: IrStmt, resume: stri
let bails = false;
emitter.declare("declare i1 @llvm.expect.i1(i1, i1)");
const bail = (condition: string): void => {
if (condition === "false") return;
if (condition === "false" || resume === undefined) return;
bails = true;
const expected = B.tmp();
B.line(`${expected} = call i1 @llvm.expect.i1(i1 ${condition}, i1 false)`);
@@ -1930,7 +2015,7 @@ export function emitSpeculativeWrite(emitter: LlEmitter, s: IrStmt, resume: stri
B.line(`${biased} = add i64 ${result}, ${Number.MAX_SAFE_INTEGER}`);
B.line(`${outside} = icmp ugt i64 ${biased}, ${2 * Number.MAX_SAFE_INTEGER}`);
bail(outside);
} else result = emitSpeculativeInteger(emitter, target.value, bail, reads);
} else result = emitSpeculativeInteger(emitter, target.value, bail, reads, resume === undefined);
// Speculable values read only numbers past this point.
if (emitter.frames.pop()!.length > 0)
throw new InternalCompilerError("llvm emitter bug: speculative write owns a temporary");
@@ -1945,8 +2030,8 @@ export function emitSpeculativeWrite(emitter: LlEmitter, s: IrStmt, resume: stri
const done = B.newLabel("spec.done");
B.br(done);
B.startBlock(failed);
emitter.writeBackSpeculativeGlobals(deferred);
B.br(resume);
emitter.writeBackDeferred(deferred);
B.br(resume!);
B.startBlock(done);
}
packages/compiler/src/backend/llvm/emitter.ts
+22 -3
View File
@@ -1,5 +1,6 @@
import { wasmBoundsChecks } from "./wasm-modules.js";
import { jsTruthy } from "./jsval.js";
import { JsIcSites } from "../../ir/js-ic.js";
import { type ArraySearchKind } from "./array-search.js";
import { emitStringInputs } from "./string-lifetimes.js";
import { emitThreadSlotsIntrinsic } from "./thread-slots.js";
@@ -919,6 +920,12 @@ export class LlEmitter {
readonly recordCloneShapes = new Set<string>();
readonly tracedShapes: Set<string>;
readonly tracedUnions: Set<string>;
/** Inline-cache sites of dynamic-tier functions (js-ic.ts): one
* zero-initialized ScrJsIc per site (scr_js.h). */
jsIcSiteCount = 0;
private jsIcSitesMemo: JsIcSites | undefined;
/** The tier of the function being emitted. */
currentTier: IrFunction["tier"] = undefined;
/** The class graph (buildClassGraph): preorder numbering, hierarchy
* membership, virtual slot lists. */
readonly classMeta: Map<string, LlClassMeta>;
@@ -1037,6 +1044,8 @@ export class LlEmitter {
* not a constructor (where they change), not async or a generator. */
finalFieldBorrows = false;
integerRanges: IntegerRanges = new Map();
/** The current function's bounded for-loop cursors (integer-ranges.ts). */
integerLoopRanges: ReadonlyMap<IrStmt, ReadonlyMap<string, IntegerRange>> = new Map();
/** Number arrays of the current function whose every present element is
* an int32 (Int32[] contracts), or null when the module has none. */
int32Elements: ((arr: IrExpr) => boolean) | null = null;
@@ -2687,7 +2696,8 @@ export class LlEmitter {
return emitLoopCopies(this, loop, allowed, exit, emitCopy);
}
writeBackSpeculativeGlobals(deferred: Set<string>): void {
/** Store the double of each deferred shadow back to its binding. */
writeBackDeferred(deferred: Set<string>): void {
const B = this.B;
for (const id of deferred) {
const integer = B.tmp(),
@@ -2766,7 +2776,7 @@ export class LlEmitter {
}
return this.globalWriters.get(id) ?? new Set<string>();
}
private emitSpeculativeWrite(s: IrStmt, resume: string): void {
private emitSpeculativeWrite(s: IrStmt, resume: string | undefined): void {
return emitSpeculativeWrite(this, s, resume);
}
@@ -4460,7 +4470,7 @@ export class LlEmitter {
this.B.startBlock(resume.label);
}
if (this.speculation?.writes.has(s)) {
this.emitSpeculativeWrite(s, this.speculation.resume.get(s)!);
this.emitSpeculativeWrite(s, this.speculation.resume.get(s));
return;
}
const previous = this.B.debugLocation;
@@ -5555,6 +5565,15 @@ export class LlEmitter {
return emitExpr(this, e);
}
/** The helper classification of this module's inline-cache sites. */
get jsIcSites(): JsIcSites {
return (this.jsIcSitesMemo ??= new JsIcSites(this.mod));
}
jsIcSite(): string {
return `@sc_js_ic_${this.jsIcSiteCount++}`;
}
/** A fresh read-site cache for one constant-key dyn read. */
dynReadIcSite(): string {
return `@sc_dyn_read_ic_${this.dynReadIcSites++}`;
packages/compiler/src/backend/llvm/expr-async.ts
+1 -7
View File
@@ -374,7 +374,7 @@ export function emitSerializationExpr(
// falls back to exactly this route whenever the fast path declines.
const parsed = e.value;
if (!e.preserveRefs && parsed.kind === "libCall" && parsed.fn === "json.parse") {
const fused = host.dyn.jsonParseHelper(e.type, e.fallback === true);
const fused = host.dyn.jsonParseHelper(e.type);
if (fused) {
const text = emitBorrowedInput(host, parsed.args[0]!);
const r = B.tmp();
@@ -400,12 +400,6 @@ export function emitSerializationExpr(
B.line(
`${t} = call ${ty === "i1" ? "zeroext i1" : ty} @${helper}(ptr ${dynV.name}, ptr null)`,
);
// A fallback site: a value that fails validation parks as the
// checked value instead of throwing (dynCheck.fallback).
if (e.fallback) {
host.declare(`declare zeroext i1 @scr_dyn_check_recover(ptr)`);
B.line(`call zeroext i1 @scr_dyn_check_recover(ptr ${dynV.name})`);
}
const out = host.own({ name: t, type: e.type });
host.emitPendingCheck();
return out;
packages/compiler/src/backend/llvm/expr-calls.ts
+7
View File
@@ -53,6 +53,7 @@ import {
inputSuffixWrites,
} from "./borrowed-inputs.js";
import type { Writes } from "./reference-writes.js";
import { emitJsIcCall, jsIcSiteOf } from "./js-ic.js";
import { emitClassMembership, emitDowncastGuard } from "./narrow-guards.js";
export function emitCallExpr(
@@ -88,6 +89,12 @@ export function emitCallExpr(
case "call": {
const callee = host.fnByName.get(e.callee);
if (!callee) throw new InternalCompilerError(`llvm emitter bug: unknown callee ${e.callee}`);
if (!plainCall && !awaitEntry && !resultSlot && !stackResult) {
// A property helper called from dynamic-tier code is an inline
// cache site (two-tier design S4).
const site = jsIcSiteOf(host, e);
if (site) return emitJsIcCall(host, e, callee, site);
}
if (callee.fastRead === "index" && !plainCall && !awaitEntry && !resultSlot) {
const indexed = emitNumericIndexRead(host, e, callee);
if (indexed) return indexed;
packages/compiler/src/backend/llvm/expr-containers.ts
+25 -9
View File
@@ -31,6 +31,7 @@ import {
emitMapIterCount,
emitMapIterEntry,
emitMapIterSlot,
mapEntryLayout,
emitMapIterStart,
emitNumberMapGetRef,
emitNumberMapGetScalar,
@@ -1430,7 +1431,7 @@ export function emitMapLikeIntrinsic(
return { name: emitMapIterStart(host, r.name), type: e.type };
case "iterLive": {
const i = host.emitExpr(e.args[0]!);
const entry = emitMapIterEntry(host, r.name, i.name);
const entry = emitMapIterEntry(host, r.name, i.name, mapEntryLayout(key));
const t = B.tmp();
B.line(`${t} = icmp ne ptr ${entry}, null`);
return { name: t, type: e.type };
@@ -1441,10 +1442,16 @@ export function emitMapLikeIntrinsic(
if (kAcc !== "ref") {
const retTy = kAcc === "f64" ? "double" : kAcc === "bool" ? "zeroext i1" : "ptr";
const symbol = `scr_map_iter_key_${kAcc}`;
const t = emitMapIterSlot(host, r.name, i.name, 0, kAcc === "str" ? "ref" : kAcc, key, {
decl: `declare ${retTy} @${symbol}(ptr, double)`,
symbol,
});
const t = emitMapIterSlot(
host,
r.name,
i.name,
0,
kAcc === "str" ? "ref" : kAcc,
key,
{ decl: `declare ${retTy} @${symbol}(ptr, double)`, symbol },
mapEntryLayout(key),
);
return host.own({ name: t, type: e.type });
}
const retTy = "ptr";
@@ -1459,10 +1466,19 @@ export function emitMapLikeIntrinsic(
const i = host.emitExpr(e.args[0]!);
const retTy = vAcc === "f64" ? "double" : vAcc === "bool" ? "i1" : "ptr";
const symbol = `scr_map_iter_val_${vAcc}`;
const t = emitMapIterSlot(host, r.name, i.name, 1, vAcc, receiverType.value, {
decl: `declare ${vAcc === "bool" ? "zeroext i1" : retTy} @${symbol}(ptr, double)`,
symbol,
});
const t = emitMapIterSlot(
host,
r.name,
i.name,
1,
vAcc,
receiverType.value,
{
decl: `declare ${vAcc === "bool" ? "zeroext i1" : retTy} @${symbol}(ptr, double)`,
symbol,
},
mapEntryLayout(key),
);
return host.own({ name: t, type: e.type });
}
case "iterEnter":
packages/compiler/src/backend/llvm/expr-dynamic.ts
+11 -4
View File
@@ -28,6 +28,7 @@ import { elemAccess, vAdapters } from "./shapes.js";
import { LlvmUnsupportedError } from "./unsupported.js";
import type { LlvmEmitterContext, ExprOf, LlValue } from "./expr-context.js";
import { emitUnionWiden } from "./expr-records.js";
import { emitJsIcMethod, jsIcMethods } from "./js-ic.js";
import { DYN_BUFFER_NUM_METHODS } from "../../ir/runtime-features.js";
export function emitDynamicExpr(
@@ -207,10 +208,16 @@ export function emitDynamicExpr(
// pending exception.
const recv = emitBorrowedInput(host, e.recv);
host.declare(`declare ptr @scr_dyn_prepare_method(ptr, ptr)`);
const prepared = B.tmp();
B.line(
`${prepared} = call ptr @scr_dyn_prepare_method(ptr ${recv.name}, ptr ${host.cstr(e.method)})`,
);
let prepared: string;
if (jsIcMethods(host)) {
// Dynamic-tier code reads the method through the site's cache.
prepared = emitJsIcMethod(host, e, recv.name);
} else {
prepared = B.tmp();
B.line(
`${prepared} = call ptr @scr_dyn_prepare_method(ptr ${recv.name}, ptr ${host.cstr(e.method)})`,
);
}
host.own({ name: prepared, type: { kind: "dyn" } });
host.emitPendingCheck();
const args = e.args.map((a) => emitBorrowedInput(host, a));
packages/compiler/src/backend/llvm/js-ic.ts
+779
View File
@@ -0,0 +1,779 @@
/* Inline-cache lowering of the dynamic tier (two-tier design stage S4;
* runtime scr_js_object.c, sites ir/js-ic.ts).
*
* A property site in a dynamic-tier function owns one zeroed thread-local
* ScrJsIc record (`@sc_js_ic_N`). The code at the site is fixed:
*
* load ordinary object whose shape is entry 0's own-property shape:
* read entry `slot` inline unless it is an accessor; otherwise
* scr_js_load_probe (polymorphic, prototype hits, misses)
* store ordinary object whose shape is entry 0's replace shape, outside
* an intact parse region, writable data entry: replace inline;
* otherwise scr_js_store_probe (adds, polymorphic, misses)
* keyed dense array element at an integral number key: inline;
* otherwise scr_js_keyed_probe (string keys of ordinary objects)
* method dynInvoke's method read: scr_js_load_probe
*
* A probe that cannot answer from plain data leaves everything untouched,
* and the site calls its helper (or the generic runtime entry), which
* defines the operation's semantics; nothing is ever patched.
*
* Layout facts (64-bit only; 32-bit targets keep the helper calls):
* ScrDyn kind +8; v.obj {len +16, entries +32, region +72,
* shape +80}; v.arr {len +16, items +32, presence +56};
* v.num +16
* ScrDynEntry 48 bytes: value +16, accessor bit 0 of +40, writable +41
* ScrJsIc entries of 40 bytes {shape, target, holder, holder_shape,
* slot}; 168 bytes in all */
import { BOOL, DYN, F64, STRING, type IrExpr, type IrFunction } from "../../ir/ir.js";
import { jsIcMethodSites, type JsIcSite } from "../../ir/js-ic.js";
import { mangleBorrowedFunction } from "../mangle.js";
import { emitBorrowedInput } from "./borrowed-inputs.js";
import { DYN_KIND } from "./dyn.js";
import type { ExprOf, LlValue, LlvmEmitterContext } from "./expr-context.js";
import { JS_BITS } from "./jsval-encoding.js";
import { jsImmediates } from "./jsval.js";
type Host = LlvmEmitterContext;
/** Words of one ScrJsIc record. */
export const JS_IC_WORDS = 21;
const ENTRY_STRIDE = 48;
/** The cache site a call is, when it is emitted in a dynamic-tier function
* of a 64-bit target. */
export function jsIcSiteOf(host: Host, e: ExprOf<"call">): JsIcSite | null {
if (!jsImmediates(host) || host.currentTier !== "dynamic") return null;
if (host.int32Abis.has(e.callee)) return null;
return host.jsIcSites.site(e.callee);
}
/** Whether method dispatch in the current function reads through a cache. */
export function jsIcMethods(host: Host): boolean {
return (
jsImmediates(host) &&
host.currentTier === "dynamic" &&
jsIcMethodSites(host.currentFunctionName)
);
}
function gep(host: Host, base: string, offset: number): string {
const p = host.B.tmp();
host.B.line(`${p} = getelementptr inbounds i8, ptr ${base}, i64 ${offset}`);
return p;
}
function load(host: Host, type: string, base: string, offset: number): string {
const v = host.B.tmp();
host.B.line(`${v} = load ${type}, ptr ${offset === 0 ? base : gep(host, base, offset)}`);
return v;
}
/** Branches to `yes` when the dyn cell `d` has kind `kind`. */
function kindIs(host: Host, d: string, kind: number, yes: string, no: string): void {
const k = load(host, "i32", d, 8);
const is = host.B.tmp();
host.B.line(`${is} = icmp eq i32 ${k}, ${kind}`);
host.B.condBr(is, yes, no);
}
/* How many cache entries a site tests inline, and whether those tests
* cover prototype hits. Measured (two-tier S4): testing all four entries
* inline made polymorphic reads 7-10% faster (the design prototype's W2/W3)
* and monomorphic method code no faster, for 65-225 more bytes per site;
* the prototype hit made method-heavy code 3-6% faster for 15-90 bytes.
* So a site tests entry 0, own or prototype, and its probe does the rest.
* (0 makes every access probe: with a runtime built with SCR_JS_VERIFY,
* that checks every cached shape against its object.) */
const INLINE_WAYS: number = 1;
const INLINE_PROTO = true;
/** The address of entry `slot` of object `d`'s entries. */
function entryAt(host: Host, d: string, slot: string): string {
const B = host.B;
const entries = load(host, "ptr", d, 32);
const offset = B.tmp();
const entry = B.tmp();
B.line(`${offset} = mul i64 ${slot}, ${ENTRY_STRIDE}`);
B.line(`${entry} = getelementptr inbounds i8, ptr ${entries}, i64 ${offset}`);
return entry;
}
/** Branches to `miss` when the entry at `entry` is an accessor. */
function dataEntry(host: Host, entry: string, miss: string): void {
const B = host.B;
const flags = load(host, "i8", entry, 40);
const accessor = B.tmp();
const isAccessor = B.tmp();
B.line(`${accessor} = and i8 ${flags}, 1`);
B.line(`${isAccessor} = icmp ne i8 ${accessor}, 0`);
const data = B.newLabel("ic.data");
B.condBr(isAccessor, miss, data);
B.startBlock(data);
}
/** The inline part of a load site, entered for an ordinary object `d`:
* tests the first `ways` cache entries (own properties, and prototype hits
* when `proto`); each hit converts the (borrowed) value with `result` and
* branches to `join`, recording its phi input. Anything else branches to
* `miss`. */
function inlineLoad(
host: Host,
d: string,
ic: string,
ways: number,
proto: boolean,
result: (cell: string) => string,
incoming: string[],
join: string,
miss: string,
): void {
const B = host.B;
if (ways === 0) {
B.br(miss);
return;
}
const shape = load(host, "ptr", d, 80);
const present = B.tmp();
B.line(`${present} = icmp ne ptr ${shape}, null`);
let next = B.newLabel("ic.w");
B.condBr(present, next, miss);
for (let way = 0; way < ways; way++) {
const base = way * 40;
B.startBlock(next);
next = way + 1 < ways ? B.newLabel("ic.w") : miss;
const cached = load(host, "ptr", ic, base);
const same = B.tmp();
B.line(`${same} = icmp eq ptr ${shape}, ${cached}`);
const matched = B.newLabel("ic.m");
B.condBr(same, matched, next);
B.startBlock(matched);
const target = load(host, "ptr", ic, base + 8);
const own = B.tmp();
B.line(`${own} = icmp eq ptr ${target}, null`);
const ownLabel = B.newLabel("ic.own");
const protoLabel = proto ? B.newLabel("ic.proto") : miss;
B.condBr(own, ownLabel, protoLabel);
const hit = (holder: string): void => {
const slot = load(host, "i64", ic, base + 32);
const entry = entryAt(host, holder, slot);
dataEntry(host, entry, miss);
const value = load(host, "ptr", entry, 16);
const out = result(value);
incoming.push(`[ ${out}, %${B.currentLabel()} ]`);
B.br(join);
};
B.startBlock(ownLabel);
hit(d);
if (!proto) continue;
B.startBlock(protoLabel);
const holder = load(host, "ptr", ic, base + 16);
const prototype = load(host, "ptr", d, 96);
const linked = B.tmp();
B.line(`${linked} = icmp eq ptr ${prototype}, ${holder}`);
const isLinked = B.newLabel("ic.pl");
B.condBr(linked, isLinked, miss);
B.startBlock(isLinked);
const holderObject = B.newLabel("ic.po");
kindIs(host, holder, DYN_KIND.OBJ, holderObject, miss);
B.startBlock(holderObject);
const holderShape = load(host, "ptr", holder, 80);
const current = B.tmp();
B.line(`${current} = icmp eq ptr ${holderShape}, ${target}`);
const holderHit = B.newLabel("ic.ph");
B.condBr(current, holderHit, miss);
B.startBlock(holderHit);
hit(holder);
}
}
/** The inline part of a store site that replaces an own data property,
* entered for an ordinary object `d` outside an intact parse region:
* tests the first `ways` cache entries; a hit calls `store` with the
* address of the entry's value word, which must end its block by
* branching to the site's join. */
function inlineReplace(
host: Host,
d: string,
ic: string,
ways: number,
store: (slot: string) => void,
miss: string,
): void {
const B = host.B;
if (ways === 0) {
B.br(miss);
return;
}
const shape = load(host, "ptr", d, 80);
const present = B.tmp();
B.line(`${present} = icmp ne ptr ${shape}, null`);
let next = B.newLabel("ic.w");
B.condBr(present, next, miss);
for (let way = 0; way < ways; way++) {
const base = way * 40;
B.startBlock(next);
next = way + 1 < ways ? B.newLabel("ic.w") : miss;
const cached = load(host, "ptr", ic, base);
const target = load(host, "ptr", ic, base + 8);
const same = B.tmp();
const replace = B.tmp();
const ok = B.tmp();
B.line(`${same} = icmp eq ptr ${shape}, ${cached}`);
B.line(`${replace} = icmp eq ptr ${target}, ${shape}`);
B.line(`${ok} = and i1 ${same}, ${replace}`);
const matched = B.newLabel("ic.m");
B.condBr(ok, matched, next);
B.startBlock(matched);
const slot = load(host, "i64", ic, base + 32);
const entry = entryAt(host, d, slot);
const flags = load(host, "i8", entry, 40);
const writable = load(host, "i8", entry, 41);
const accessor = B.tmp();
const data = B.tmp();
const canWrite = B.tmp();
const writes = B.tmp();
B.line(`${accessor} = and i8 ${flags}, 1`);
B.line(`${data} = icmp eq i8 ${accessor}, 0`);
B.line(`${canWrite} = icmp ne i8 ${writable}, 0`);
B.line(`${writes} = and i1 ${data}, ${canWrite}`);
const hit = B.newLabel("ic.st");
B.condBr(writes, hit, miss);
B.startBlock(hit);
store(gep(host, entry, 16));
}
}
/** A property value (a dyn cell, `owned` or borrowed from its entry) as
* the site's result type, owned. A jsval result reads a number cell as an
* immediate, so arithmetic on it takes no reference and a later store can
* update an unshared number cell in place; any other cell is carried under
* the cell tag. */
function asResult(host: Host, cell: string, e: IrExpr, owned: boolean | string): string {
const B = host.B;
/** Retains the cell unless the caller owns it. */
const hold = (): string => {
if (owned === true) return cell;
if (owned === false) return host.retainFrame(cell, DYN);
const borrowed = B.newLabel("icr.b");
const done = B.newLabel("icr.h");
const from = B.currentLabel();
B.condBr(owned, done, borrowed);
B.startBlock(borrowed);
const retained = host.retainFrame(cell, DYN);
const retainedEnd = B.currentLabel();
B.br(done);
B.startBlock(done);
const held = B.tmp();
B.line(`${held} = phi ptr [ ${cell}, %${from} ], [ ${retained}, %${retainedEnd} ]`);
return held;
};
if (e.type.kind !== "jsval") return hold();
const number = B.newLabel("icr.num");
const other = B.newLabel("icr.cell");
const join = B.newLabel("icr.j");
kindIs(host, cell, DYN_KIND.NUM, number, other);
B.startBlock(number);
const d = load(host, "double", cell, 16);
const raw = B.tmp();
const nan = B.tmp();
const bits = B.tmp();
B.line(`${raw} = bitcast double ${d} to i64`);
B.line(`${nan} = fcmp uno double ${d}, ${d}`);
B.line(`${bits} = select i1 ${nan}, i64 ${JS_BITS.canonicalNan}, i64 ${raw}`);
const immediate = B.tmp();
B.line(`${immediate} = inttoptr i64 ${bits} to ptr`);
if (owned === true) {
host.releaseFrame(cell, DYN);
} else if (owned !== false) {
const release = B.newLabel("icr.r");
const released = B.newLabel("icr.rd");
B.condBr(owned, release, released);
B.startBlock(release);
host.releaseFrame(cell, DYN);
B.br(released);
B.startBlock(released);
}
const numberEnd = B.currentLabel();
B.br(join);
B.startBlock(other);
const held = hold();
const cellBits = B.tmp();
const word = B.tmp();
const tagged = B.tmp();
B.line(`${cellBits} = ptrtoint ptr ${held} to i64`);
B.line(`${word} = or i64 ${cellBits}, ${JS_BITS.cellTag}`);
B.line(`${tagged} = inttoptr i64 ${word} to ptr`);
const otherEnd = B.currentLabel();
B.br(join);
B.startBlock(join);
const out = B.tmp();
B.line(`${out} = phi ptr [ ${immediate}, %${numberEnd} ], [ ${tagged}, %${otherEnd} ]`);
return out;
}
/** The helper call of a site's slow path, with the site's evaluated
* (borrowed) arguments. Ordinary parameters take a reference; borrowed
* ones lend it. Returns the owned result, or "" for a void helper. */
function helperCall(host: Host, e: ExprOf<"call">, callee: IrFunction, args: LlValue[]): string {
const B = host.B;
const borrowed = host.callLifetimes.borrowed.get(e.callee);
const argList = args
.map((a, i) => {
const type = callee.params[i]!.type;
const value = borrowed?.has(i) ? a.name : host.retainFrame(a.name, a.type);
return `${host.llType(type)} ${value}`;
})
.join(", ");
const target = `@${borrowed ? mangleBorrowedFunction(e.callee) : host.callTarget(e.callee)}`;
// The helper is the site's cold path: keep its body out of line.
if (e.type.kind === "void") {
B.line(`call void ${target}(${argList}) noinline cold`);
if (host.mayThrow.has(e.callee)) host.emitPendingCheck();
return "";
}
let t = B.tmp();
B.line(`${t} = call ${host.llType(e.type)} ${target}(${argList}) noinline cold`);
if (host.mayThrow.has(e.callee)) host.emitPendingCheck();
if (borrowed && host.borrowedReturns.has(e.callee)) t = host.retainFrame(t, e.type);
return t;
}
function keyString(host: Host, key: string, e: IrExpr): string {
return host.emitExpr({ kind: "strLit", value: key, type: STRING, loc: e.loc }).name;
}
/** Emits a call to a classified helper as a cache site. */
export function emitJsIcCall(
host: Host,
e: ExprOf<"call">,
callee: IrFunction,
site: JsIcSite,
): LlValue {
switch (site.kind) {
case "get":
return emitGet(host, e, callee, site.key);
case "set":
return emitSet(host, e, callee, site.key);
case "keyed":
return emitKeyed(host, e, callee);
}
}
function emitGet(host: Host, e: ExprOf<"call">, callee: IrFunction, key: string): LlValue {
const B = host.B;
const receiver = emitBorrowedInput(host, e.args[0]!);
const name = keyString(host, key, e);
const ic = host.jsIcSite();
const object = B.newLabel("icg.obj");
const probe = B.newLabel("icg.probe");
const slow = B.newLabel("icg.slow");
const join = B.newLabel("icg.j");
const incoming: string[] = [];
if (key === "length") {
// An array's length is its element count (the generic read answers
// it before any property table).
const array = B.newLabel("icg.arr");
const other = B.newLabel("icg.other");
kindIs(host, receiver.name, DYN_KIND.ARR, array, other);
B.startBlock(array);
const len = load(host, "i64", receiver.name, 16);
const count = B.tmp();
B.line(`${count} = uitofp i64 ${len} to double`);
incoming.push(`[ ${numberResult(host, count, e)}, %${B.currentLabel()} ]`);
B.br(join);
B.startBlock(other);
}
// Inline hits (borrowed values) and probe hits (owned ones) share one
// conversion to the result type.
const hit = B.newLabel("icg.hit");
const hits: string[] = [];
kindIs(host, receiver.name, DYN_KIND.OBJ, object, probe);
B.startBlock(object);
inlineLoad(host, receiver.name, ic, INLINE_WAYS, INLINE_PROTO, (cell) => cell, hits, hit, probe);
B.startBlock(probe);
host.declare(`declare ptr @scr_js_load_probe(ptr, ptr, ptr)`);
const probed = B.tmp();
B.line(`${probed} = call ptr @scr_js_load_probe(ptr ${receiver.name}, ptr ${name}, ptr ${ic})`);
const found = B.tmp();
B.line(`${found} = icmp ne ptr ${probed}, null`);
const probeEnd = B.currentLabel();
B.condBr(found, hit, slow);
B.startBlock(hit);
const cell = B.tmp();
const owned = B.tmp();
const ownedIn = hits.map((h) => h.replace(/^\[ [^,]+,/, "[ false,"));
B.line(`${cell} = phi ptr ${[...hits, `[ ${probed}, %${probeEnd} ]`].join(", ")}`);
B.line(`${owned} = phi i1 ${[...ownedIn, `[ true, %${probeEnd} ]`].join(", ")}`);
{
const out = asResult(host, cell, e, owned);
incoming.push(`[ ${out}, %${B.currentLabel()} ]`);
}
B.br(join);
B.startBlock(slow);
const result = helperCall(host, e, callee, [receiver]);
incoming.push(`[ ${result}, %${B.currentLabel()} ]`);
B.br(join);
B.startBlock(join);
const out = B.tmp();
B.line(`${out} = phi ptr ${incoming.join(", ")}`);
return host.own({ name: out, type: e.type });
}
/** A number as the site's result type: an immediate for a jsval result,
* a fresh cell (as the generic read makes) for a dyn one. */
function numberResult(host: Host, d: string, e: IrExpr): string {
const B = host.B;
if (e.type.kind === "jsval") {
// Counts are never NaN: the bits are the word.
const bits = B.tmp();
const word = B.tmp();
B.line(`${bits} = bitcast double ${d} to i64`);
B.line(`${word} = inttoptr i64 ${bits} to ptr`);
return word;
}
host.declare(`declare ptr @scr_dyn_new_num(double)`);
const cell = B.tmp();
B.line(`${cell} = call ptr @scr_dyn_new_num(double ${d})`);
return cell;
}
function emitSet(host: Host, e: ExprOf<"call">, callee: IrFunction, key: string): LlValue {
const B = host.B;
const receiver = emitBorrowedInput(host, e.args[0]!);
// A value the jsval pass left as `js.toDyn(x)` is stored from the jsval
// itself: a number replaces the value of an unshared number cell in
// place instead of boxing a new one.
const stored = e.args[1]!;
const word =
stored.kind === "libCall" && stored.fn === "js.toDyn"
? host.emitExpr(stored.args[0]!).name
: null;
const value = word === null ? emitBorrowedInput(host, stored) : null;
const name = keyString(host, key, e);
const ic = host.jsIcSite();
const object = B.newLabel("ics.obj");
const probe = B.newLabel("ics.probe");
const slow = B.newLabel("ics.slow");
const join = B.newLabel("ics.j");
// The cell the probe and the helper store when the value is a jsval;
// the statement's frame releases it.
let boxedSlot: string | null = null;
if (word !== null) {
boxedSlot = B.slot();
B.entryAllocas.push(`${boxedSlot} = alloca ptr`);
B.line(`store ptr null, ptr ${boxedSlot}`);
host.ownSlot(boxedSlot, DYN);
}
kindIs(host, receiver.name, DYN_KIND.OBJ, object, probe);
B.startBlock(object);
const region = load(host, "ptr", receiver.name, 72);
const outside = B.tmp();
B.line(`${outside} = icmp eq ptr ${region}, null`);
const unshared = B.newLabel("ics.r");
B.condBr(outside, unshared, probe);
B.startBlock(unshared);
host.declare(`declare ptr @scr_js_to_dyn_borrow(ptr)`);
inlineReplace(
host,
receiver.name,
ic,
INLINE_WAYS,
(slot) => {
const old = B.tmp();
B.line(`${old} = load ptr, ptr ${slot}`);
if (word === null) {
B.line(`store ptr ${host.retainHeap(value!.name, DYN)}, ptr ${slot}`);
host.releaseHeap(old, DYN);
B.br(join);
return;
}
const bits = B.tmp();
const isDouble = B.tmp();
B.line(`${bits} = ptrtoint ptr ${word} to i64`);
B.line(`${isDouble} = icmp ult i64 ${bits}, ${JS_BITS.limit}`);
const number = B.newLabel("ics.dbl");
const box = B.newLabel("ics.box");
B.condBr(isDouble, number, box);
B.startBlock(number);
const oldKind = load(host, "i32", old, 8);
const oldRc = load(host, "i64", old, 0);
const isNumber = B.tmp();
const unshared = B.tmp();
const inPlace = B.tmp();
B.line(`${isNumber} = icmp eq i32 ${oldKind}, ${DYN_KIND.NUM}`);
B.line(`${unshared} = icmp eq i64 ${oldRc}, 1`);
B.line(`${inPlace} = and i1 ${isNumber}, ${unshared}`);
const update = B.newLabel("ics.inplace");
B.condBr(inPlace, update, box);
B.startBlock(update);
const d = B.tmp();
B.line(`${d} = bitcast i64 ${bits} to double`);
B.line(`store double ${d}, ptr ${gep(host, old, 16)}`);
B.br(join);
B.startBlock(box);
const cell = B.tmp();
B.line(`${cell} = call ptr @scr_js_to_dyn_borrow(ptr ${word})`);
B.line(`store ptr ${cell}, ptr ${slot}`);
host.releaseHeap(old, DYN);
B.br(join);
},
probe,
);
B.startBlock(probe);
let cell: string;
if (word !== null) {
cell = B.tmp();
B.line(`${cell} = call ptr @scr_js_to_dyn_borrow(ptr ${word})`);
B.line(`store ptr ${cell}, ptr ${boxedSlot}`);
} else {
cell = value!.name;
}
host.declare(`declare zeroext i1 @scr_js_store_probe(ptr, ptr, ptr, ptr)`);
const done = B.tmp();
B.line(
`${done} = call zeroext i1 @scr_js_store_probe(ptr ${receiver.name}, ptr ${name}, ptr ${cell}, ptr ${ic})`,
);
B.condBr(done, join, slow);
B.startBlock(slow);
helperCall(host, e, callee, [receiver, { name: cell, type: DYN }]);
B.br(join);
B.startBlock(join);
return { name: "", type: e.type };
}
/** A keyed site's key when it is a static number boxed for the helper
* (`dynFrom(n)`, or `dynFrom(u)` of a local union with a number arm): the
* site indexes with the double and boxes the key only on a miss. */
interface NumberKey {
/** i1: whether `value` holds the key (always for a static number). */
present: string;
value: string;
/** Emits the key's cell (+1), exactly as the helper argument boxes it. */
box: () => string;
}
function numberKey(host: Host, key: IrExpr): NumberKey | null {
if (key.kind !== "dynFrom" || key.liveRef !== undefined || key.fnName !== undefined) return null;
const source = key.value;
if (source.type.kind === "f64") {
const value = host.emitExpr(source).name;
// Request the converter where the boxing it replaces did: converters
// are numbered in request order.
const conv = host.dyn.toDynHelper(source.type);
return {
present: "true",
value,
box: () => {
const cell = host.B.tmp();
host.B.line(`${cell} = call ptr @${conv}(double ${value})`);
return cell;
},
};
}
if (source.type.kind !== "union" || source.kind !== "varRef") return null;
const unionId = source.type.unionId;
const tag = host.unionsById.get(unionId)?.arms.findIndex((arm) => arm.kind === "f64") ?? -1;
if (tag < 0) return null;
const B = host.B;
const present = host.emitExpr({
kind: "unionIsTag",
unionId,
tag,
negated: false,
value: source,
type: BOOL,
loc: key.loc,
}).name;
const slot = B.slot();
B.entryAllocas.push(`${slot} = alloca double`);
B.line(`store double 0.0, ptr ${slot}`);
const number = B.newLabel("ick.k");
const done = B.newLabel("ick.kd");
B.condBr(present, number, done);
B.startBlock(number);
// Temps of a conditional block are released in it.
host.frames.push([]);
const d = host.emitExpr({
kind: "unionNarrow",
unionId,
tag,
value: source,
type: F64,
loc: key.loc,
});
B.line(`store double ${d.name}, ptr ${slot}`);
host.releaseTemps(host.frames.pop()!);
B.br(done);
B.startBlock(done);
const value = B.tmp();
B.line(`${value} = load double, ptr ${slot}`);
return {
present,
value,
box: () => {
// The local is a plain read: boxing it again evaluates nothing else.
host.frames.push([]);
const boxed = host.emitExpr(key);
host.toFrame(boxed);
host.releaseTemps(host.frames.pop()!);
return boxed.name;
},
};
}
function emitKeyed(host: Host, e: ExprOf<"call">, callee: IrFunction): LlValue {
const B = host.B;
const receiver = emitBorrowedInput(host, e.args[0]!);
const keyExpr = e.args[1]!;
const numeric = numberKey(host, keyExpr);
const key = numeric === null ? emitBorrowedInput(host, keyExpr) : null;
const ic = host.jsIcSite();
const array = B.newLabel("ick.arr");
const number = B.newLabel("ick.num");
const probe = B.newLabel("ick.probe");
const slow = B.newLabel("ick.slow");
const join = B.newLabel("ick.j");
const incoming: string[] = [];
// The boxed key of the miss paths when the site indexes with a double;
// the statement's frame releases it.
let keySlot: string | null = null;
if (numeric !== null) {
keySlot = B.slot();
B.entryAllocas.push(`${keySlot} = alloca ptr`);
B.line(`store ptr null, ptr ${keySlot}`);
host.ownSlot(keySlot, DYN);
}
kindIs(host, receiver.name, DYN_KIND.ARR, array, probe);
B.startBlock(array);
let index: string;
if (numeric === null) {
kindIs(host, key!.name, DYN_KIND.NUM, number, probe);
B.startBlock(number);
index = load(host, "double", key!.name, 16);
} else {
B.condBr(numeric.present, number, probe);
B.startBlock(number);
index = numeric.value;
}
const len = load(host, "i64", receiver.name, 16);
const i = B.tmp();
const back = B.tmp();
const integral = B.tmp();
const inRange = B.tmp();
const nonNegative = B.tmp();
const ok1 = B.tmp();
const ok2 = B.tmp();
// fptoui of a negative or out-of-range double is poison: test the
// double's range before trusting the integer.
B.line(`${nonNegative} = fcmp oge double ${index}, 0.0`);
const lenD = B.tmp();
B.line(`${lenD} = uitofp i64 ${len} to double`);
B.line(`${inRange} = fcmp olt double ${index}, ${lenD}`);
B.line(`${ok1} = and i1 ${nonNegative}, ${inRange}`);
const ranged = B.newLabel("ick.rng");
B.condBr(ok1, ranged, probe);
B.startBlock(ranged);
B.line(`${i} = fptoui double ${index} to i64`);
B.line(`${back} = uitofp i64 ${i} to double`);
B.line(`${integral} = fcmp oeq double ${back}, ${index}`);
const presence = load(host, "ptr", receiver.name, 56);
const dense = B.tmp();
B.line(`${dense} = icmp eq ptr ${presence}, null`);
B.line(`${ok2} = and i1 ${integral}, ${dense}`);
const hit = B.newLabel("ick.hit");
B.condBr(ok2, hit, probe);
B.startBlock(hit);
const items = load(host, "ptr", receiver.name, 32);
const itemPtr = B.tmp();
B.line(`${itemPtr} = getelementptr inbounds ptr, ptr ${items}, i64 ${i}`);
const item = B.tmp();
B.line(`${item} = load ptr, ptr ${itemPtr}`);
const itemEnd = B.currentLabel();
const shared = B.newLabel("ick.shared");
B.br(shared);
B.startBlock(probe);
let keyCell: string;
if (numeric === null) {
keyCell = key!.name;
} else {
keyCell = numeric.box();
B.line(`store ptr ${keyCell}, ptr ${keySlot}`);
}
host.declare(`declare ptr @scr_js_keyed_probe(ptr, ptr, ptr)`);
const probed = B.tmp();
B.line(
`${probed} = call ptr @scr_js_keyed_probe(ptr ${receiver.name}, ptr ${keyCell}, ptr ${ic})`,
);
const found = B.tmp();
B.line(`${found} = icmp ne ptr ${probed}, null`);
const probeEnd = B.currentLabel();
B.condBr(found, shared, slow);
B.startBlock(shared);
{
const cell = B.tmp();
const owned = B.tmp();
B.line(`${cell} = phi ptr [ ${item}, %${itemEnd} ], [ ${probed}, %${probeEnd} ]`);
B.line(`${owned} = phi i1 [ false, %${itemEnd} ], [ true, %${probeEnd} ]`);
const out = asResult(host, cell, e, owned);
incoming.push(`[ ${out}, %${B.currentLabel()} ]`);
}
B.br(join);
B.startBlock(slow);
const result = helperCall(host, e, callee, [receiver, { name: keyCell, type: DYN }]);
incoming.push(`[ ${result}, %${B.currentLabel()} ]`);
B.br(join);
B.startBlock(join);
const out = B.tmp();
B.line(`${out} = phi ptr ${incoming.join(", ")}`);
return host.own({ name: out, type: e.type });
}
/** dynInvoke's method read on an ordinary object through the site's
* cache, else scr_dyn_prepare_method. Returns the owned method (or NULL,
* which scr_dyn_invoke_prepared answers with the full dispatch). */
export function emitJsIcMethod(host: Host, e: ExprOf<"dynInvoke">, receiver: string): string {
const B = host.B;
const method = e.method;
const ic = host.jsIcSite();
const name = keyString(host, method, e);
host.declare(`declare ptr @scr_js_load_probe(ptr, ptr, ptr)`);
host.declare(`declare ptr @scr_dyn_prepare_method(ptr, ptr)`);
const object = B.newLabel("icm.obj");
const probe = B.newLabel("icm.probe");
const slow = B.newLabel("icm.slow");
const join = B.newLabel("icm.j");
const incoming: string[] = [];
kindIs(host, receiver, DYN_KIND.OBJ, object, probe);
B.startBlock(object);
inlineLoad(
host,
receiver,
ic,
INLINE_WAYS,
INLINE_PROTO,
(cell) => host.retainFrame(cell, DYN),
incoming,
join,
probe,
);
B.startBlock(probe);
const probed = B.tmp();
B.line(`${probed} = call ptr @scr_js_load_probe(ptr ${receiver}, ptr ${name}, ptr ${ic})`);
const found = B.tmp();
B.line(`${found} = icmp ne ptr ${probed}, null`);
incoming.push(`[ ${probed}, %${B.currentLabel()} ]`);
B.condBr(found, join, slow);
B.startBlock(slow);
const prepared = B.tmp();
B.line(
`${prepared} = call ptr @scr_dyn_prepare_method(ptr ${receiver}, ptr ${host.cstr(method)}) cold`,
);
incoming.push(`[ ${prepared}, %${B.currentLabel()} ]`);
B.br(join);
B.startBlock(join);
const out = B.tmp();
B.line(`${out} = phi ptr ${incoming.join(", ")}`);
return out;
}
packages/compiler/src/backend/llvm/jsval.ts
+148 -12
View File
@@ -205,6 +205,105 @@ function call(host: Host, ret: string, fn: string, args: [string, string][]): st
return r;
}
/** `x % y` on two doubles: an integer remainder when both are integers of
* magnitude below 2^31 and y is nonzero (the result keeps the dividend's
* sign, so a zero result of a negative dividend, including -0, is -0),
* else frem (libm's fmod). */
function remainder(host: Host, x: string, y: string): string {
const B = host.B;
host.declare(`declare double @llvm.fabs.f64(double)`);
const ax = B.tmp();
const ay = B.tmp();
const smallX = B.tmp();
const smallY = B.tmp();
const small = B.tmp();
B.line(`${ax} = call double @llvm.fabs.f64(double ${x})`);
B.line(`${ay} = call double @llvm.fabs.f64(double ${y})`);
B.line(`${smallX} = fcmp olt double ${ax}, 2147483648.0`);
B.line(`${smallY} = fcmp olt double ${ay}, 2147483648.0`);
B.line(`${small} = and i1 ${smallX}, ${smallY}`);
const ints = B.newLabel("js.rem.i");
const exact = B.newLabel("js.rem.x");
const float = B.newLabel("js.rem.f");
const join = B.newLabel("js.rem.j");
B.condBr(small, ints, float);
B.startBlock(ints);
const xi = B.tmp();
const yi = B.tmp();
const xb = B.tmp();
const yb = B.tmp();
const xInt = B.tmp();
const yInt = B.tmp();
const nonzero = B.tmp();
const both = B.tmp();
const ok = B.tmp();
B.line(`${xi} = fptosi double ${x} to i64`);
B.line(`${yi} = fptosi double ${y} to i64`);
B.line(`${xb} = sitofp i64 ${xi} to double`);
B.line(`${yb} = sitofp i64 ${yi} to double`);
B.line(`${xInt} = fcmp oeq double ${xb}, ${x}`);
B.line(`${yInt} = fcmp oeq double ${yb}, ${y}`);
B.line(`${nonzero} = icmp ne i64 ${yi}, 0`);
B.line(`${both} = and i1 ${xInt}, ${yInt}`);
B.line(`${ok} = and i1 ${both}, ${nonzero}`);
B.condBr(ok, exact, float);
B.startBlock(exact);
const r = B.tmp();
const rd = B.tmp();
const xBits = B.tmp();
const negative = B.tmp();
const zero = B.tmp();
const negZero = B.tmp();
const intResult = B.tmp();
B.line(`${r} = srem i64 ${xi}, ${yi}`);
B.line(`${rd} = sitofp i64 ${r} to double`);
B.line(`${xBits} = bitcast double ${x} to i64`);
B.line(`${negative} = icmp slt i64 ${xBits}, 0`);
B.line(`${zero} = icmp eq i64 ${r}, 0`);
B.line(`${negZero} = and i1 ${negative}, ${zero}`);
B.line(`${intResult} = select i1 ${negZero}, double -0.0, double ${rd}`);
const exactEnd = B.currentLabel();
B.br(join);
B.startBlock(float);
const fr = B.tmp();
B.line(`${fr} = frem double ${x}, ${y}`);
const floatEnd = B.currentLabel();
B.br(join);
B.startBlock(join);
const out = B.tmp();
B.line(`${out} = phi double [ ${intResult}, %${exactEnd} ], [ ${fr}, %${floatEnd} ]`);
return out;
}
const BITWISE_OPS: Readonly<Record<string, string>> = {
"&": "and",
"|": "or",
"^": "xor",
"<<": "shl",
">>": "ashr",
">>>": "lshr",
};
/** A double's ToInt32 when its magnitude is below 2^63: truncation toward
* zero, then the low 32 bits; branches to `slow` otherwise (NaN and the
* infinities included). */
function int32Operand(host: Host, d: string, slow: string): string {
const B = host.B;
host.declare(`declare double @llvm.fabs.f64(double)`);
const magnitude = B.tmp();
const small = B.tmp();
B.line(`${magnitude} = call double @llvm.fabs.f64(double ${d})`);
B.line(`${small} = fcmp olt double ${magnitude}, 9223372036854775808.0`);
const ok = B.newLabel("js.int");
B.condBr(small, ok, slow);
B.startBlock(ok);
const wide = B.tmp();
const i = B.tmp();
B.line(`${wide} = fptosi double ${d} to i64`);
B.line(`${i} = trunc i64 ${wide} to i32`);
return i;
}
const ARITH_OPS: Readonly<Record<string, string>> = {
"-": "fsub",
"*": "fmul",
@@ -334,8 +433,8 @@ export function emitJsLibCall(host: Host, e: LibCallExpr): LlValue {
(slow) => {
const x = a.number(slow);
const y = b.number(slow);
const r = B.tmp();
B.line(`${r} = ${inst} double ${x}, ${y}`);
const r = op === "%" ? remainder(host, x, y) : B.tmp();
if (op !== "%") B.line(`${r} = ${inst} double ${x}, ${y}`);
return boxDouble(host, r);
},
runtime,
@@ -410,17 +509,54 @@ export function emitJsLibCall(host: Host, e: LibCallExpr): LlValue {
return { name: out, type: e.type };
}
case "js.bitwise": {
const a = host.emitExpr(e.args[0]!).name;
const b = host.emitExpr(e.args[1]!).name;
const op = literalOp(e.args[2]!);
const inst = op !== null ? BITWISE_OPS[op] : undefined;
if (inst === undefined) {
const a = host.emitExpr(e.args[0]!).name;
const b = host.emitExpr(e.args[1]!).name;
const opArg = host.emitExpr(e.args[2]!).name;
const r = call(host, "ptr", "scr_js_bitwise", [
["ptr", a],
["ptr", b],
["ptr", opArg],
]);
const out = jsval(r);
host.emitPendingCheck();
return out;
}
// Two numbers of magnitude below 2^63: ToInt32 is truncation and
// wrapping, so the operation runs on i32; anything else (NaN, the
// infinities, larger magnitudes) takes the runtime.
const a = operand(host, e.args[0]!);
const b = operand(host, e.args[1]!);
const opArg = host.emitExpr(e.args[2]!).name;
const r = call(host, "ptr", "scr_js_bitwise", [
["ptr", a],
["ptr", b],
["ptr", opArg],
]);
const out = jsval(r);
host.emitPendingCheck();
return out;
const out = withSlowPath(
host,
"ptr",
(slow) => {
const x = int32Operand(host, a.number(slow), slow);
const y = int32Operand(host, b.number(slow), slow);
const r = B.tmp();
const d = B.tmp();
if (op === "<<" || op === ">>" || op === ">>>") {
const count = B.tmp();
B.line(`${count} = and i32 ${y}, 31`);
B.line(`${r} = ${inst} i32 ${x}, ${count}`);
} else {
B.line(`${r} = ${inst} i32 ${x}, ${y}`);
}
B.line(`${d} = ${op === ">>>" ? "uitofp" : "sitofp"} i32 ${r} to double`);
return boxDouble(host, d);
},
() =>
call(host, "ptr", "scr_js_bitwise", [
["ptr", a.word()],
["ptr", b.word()],
["ptr", opArg],
]),
true,
);
return jsval(out);
}
default:
throw new InternalCompilerError(`llvm emitter bug: unknown jsval builtin ${e.fn}`);
packages/compiler/src/backend/llvm/lib-dispatch.ts
-1
View File
@@ -865,7 +865,6 @@ export function emitLibCall(host: LlvmEmitterContext, e: LibCallExpr): LlValue {
case "blob":
case "wasm":
case "record":
case "checkedCast":
case "disposable":
case "punycode":
case "constants":
packages/compiler/src/backend/llvm/lib-process.ts
+4 -3
View File
@@ -60,12 +60,13 @@ export function emitChildProcessLibCall(host: LlvmEmitterContext, e: LibCallExpr
if (e.fn === "cp.spawnDynamic") host.emitPendingCheck();
return out;
}
if (e.fn === "child.onDyn") {
if (e.fn === "child.onDyn" || e.fn === "child.propsOnDyn") {
const child = host.emitExpr(e.args[0]!);
const event = host.emitExpr(e.args[1]!);
const cb = host.emitExpr(e.args[2]!);
host.declare("declare void @scr_child_on_dyn(ptr, ptr, ptr)");
B.line(`call void @scr_child_on_dyn(ptr ${child.name}, ptr ${event.name}, ptr ${cb.name})`);
const sym = e.fn === "child.onDyn" ? "scr_child_on_dyn" : "scr_child_on_dyn_hp";
host.declare(`declare void @${sym}(ptr, ptr, ptr)`);
B.line(`call void @${sym}(ptr ${child.name}, ptr ${event.name}, ptr ${cb.name})`);
host.emitPendingCheck();
return { name: "", type: e.type };
}
packages/compiler/src/backend/llvm/lib-shared.ts
+13 -3
View File
@@ -82,6 +82,7 @@ export const LIB_FN_SYMS: Record<string, string> = {
"util.inherits": "scr_util_inherits",
"util.deprecate": "scr_util_deprecate",
"util.callbackify": "scr_util_callbackify",
"util.promisifyDyn": "scr_util_promisify",
"util.debuglog": "scr_util_debuglog",
"util.typeIs": "scr_dyn_util_type_is",
"util.typeValue": "scr_dyn_util_type_value",
@@ -544,6 +545,7 @@ export const LIB_FN_SYMS: Record<string, string> = {
"stats.birthtime": "scr_stats_birthtime",
"text.decode": "scr_text_decode",
"bytes.bufferSource": "scr_bytes_buffer_source",
"bytes.copyBrand": "scr_bytes_copy_brand",
"text.decodeBufferSource": "scr_text_decode_buffer_source",
"text.decodeOptions": "scr_text_decode_options",
"text.decodeStream": "scr_text_decode_stream",
@@ -573,6 +575,8 @@ export const LIB_FN_SYMS: Record<string, string> = {
"crypto.hashStream": "scr_crypto_hash_stream",
"crypto.hashStreamPiped": "scr_crypto_hash_stream_piped",
"crypto.cipherNew": "scr_crypto_cipher_new",
"crypto.ecdhNew": "scr_crypto_ecdh_new",
"crypto.ecdhConvertKey": "scr_crypto_ecdh_convert_key",
"crypto.hmacNewStr": "scr_crypto_hmac_new_str",
"crypto.hmacNewBytes": "scr_crypto_hmac_new_bytes",
"crypto.hashUpdateStr": "scr_crypto_hash_update_str",
@@ -651,9 +655,6 @@ export const LIB_FN_SYMS: Record<string, string> = {
"insp.circular": "scr_insp_circular",
// Undeclared source properties kept beside a checked record (scr_json.c).
"record.sourceHas": "scr_record_source_has",
// Checked-cast fallback reads (scr_json.c); take answers +1.
"checkedCast.fallbackTaken": "scr_dyn_check_fallback_taken",
"checkedCast.fallbackTake": "scr_dyn_check_fallback_take",
// WHATWG URL + URLSearchParams (scr_url.c / scr_url_params.c):
// constructions +1; url.new, the fileURLToPath pair, the win32
// pathToFileURL flavor, and sp.fromPairs throw catchably (may-throw).
@@ -952,6 +953,8 @@ export const LIB_FN_SYMS: Record<string, string> = {
"dyn.iterPack": "scr_dyn_iter_pack",
"dyn.destructureRest": "scr_dyn_destructure_rest",
"dyn.iterator": "scr_dyn_iterator",
"dyn.asyncIteratorOpen": "scr_dyn_async_iterator_open",
"dyn.asyncIteratorWasSync": "scr_dyn_async_iterator_was_sync",
"dyn.arrayFromIterator": "scr_dyn_array_from_iterator",
"dyn.iteratorResult": "scr_dyn_iterator_result",
"dyn.iteratorCanStep": "scr_dyn_iterator_can_step",
@@ -974,6 +977,10 @@ export const LIB_FN_SYMS: Record<string, string> = {
"dyn.promiseAllSettled": "scr_dyn_promise_all_settled",
"dyn.promiseAny": "scr_dyn_promise_any",
"dyn.promiseResolve": "scr_dyn_promise_resolve",
"child.propsInstall": "scr_dyn_handle_props_install",
"child.propsDefine": "scr_dyn_define_property_hp",
"child.propsDescriptor": "scr_dyn_get_own_property_descriptor_hp",
"child.propsResolve": "scr_dyn_promise_resolve_hp",
"dyn.typeof": "scr_dyn_typeof",
"dyn.objectTag": "scr_dyn_object_tag_of",
"dyn.freeze": "scr_dyn_freeze",
@@ -1555,6 +1562,8 @@ export const USES_TIMERS_LIB_FNS = new Set<string>([
"dyn.builtinStreamClass",
// Decipheriv.setAuthTag reports DEP0182 on the next tick.
"crypto.cipherNew",
// ecdh.setPublicKey reports DEP0031 on the next tick.
"crypto.ecdhNew",
// WebCrypto results settle through the deferred crypto queue.
"webcrypto.sign",
"webcrypto.verify",
@@ -1567,6 +1576,7 @@ export const USES_TIMERS_LIB_FNS = new Set<string>([
// The wrappers schedule warnings and callbacks on the tick queue.
"util.deprecate",
"util.callbackify",
"util.promisifyDyn",
"util.debuglog",
"worker.new",
"worker.parentPort",
packages/compiler/src/backend/llvm/map-number-lookup.ts
+60 -6
View File
@@ -35,7 +35,8 @@ export function emitNumberMapEntry(host: LlvmEmitterContext, map: string, key: s
return entry;
}
/** Load a found entry's value slot as `type` (double, i64 or ptr). */
/** Load a found entry's value slot as `type` (double, i64 or ptr). The value
* is at byte 8 in both entry layouts (%ScrMapEntry, %ScrMapNumEntry). */
export function emitNumberMapSlot(host: LlvmEmitterContext, entry: string, type: string): string {
const B = host.B;
const slot = B.tmp(),
@@ -190,13 +191,65 @@ export function emitNumberMapSetScalar(
/** Declared by the inline Map/Set iteration steps; its presence makes the
* emitter define sc_map_iter_entry and sc_map_iter_count (emitter.ts). */
export const MAP_ITER_DECL = "declare zeroext i1 @scr_map_iter_live(ptr, double)";
/** Declared by the iteration steps over a number-key Map/Set; its presence
* makes the emitter define sc_map_iter_entry_num and sc_map_iter_count. */
export const MAP_ITER_NUM_DECL = "declare double @scr_map_iter_count(ptr)";
/** scr_map.h SCR_MAP_NUM_DEAD: a dead number-key entry's key word. */
export const SCR_MAP_NUM_DEAD = "9218868437227405313";
/** The entry layout a Map or Set of key type `key` stores:
* - "num": number keys, whose entries are { key, val } (scr_map.h ScrMapNumEntry);
* - "entry": every other typed key, whose entries are { key, val, hash };
* - "either": checked (`unknown`) keys. A `Map<unknown, unknown>` or
* `Set<unknown>` cast from a checked value keeps the collection's own
* storage (scr_dyn_native_collection_check), which can have number keys. */
export type MapEntryLayout = "num" | "entry" | "either";
export function mapEntryLayout(key: IrType): MapEntryLayout {
return key.kind === "f64" ? "num" : key.kind === "dyn" ? "either" : "entry";
}
/** The entry at an iteration index, or null for a tombstone or an index at
* or past the entry count (the scr_map_iter_live test). */
export function emitMapIterEntry(host: LlvmEmitterContext, map: string, index: string): string {
* or past the entry count (the scr_map_iter_live test). With the "either"
* layout, the map's key kind (%ScrMapIx field 1, SCR_MAP_KEY_F64 = 0) picks
* the step at run time. */
export function emitMapIterEntry(
host: LlvmEmitterContext,
map: string,
index: string,
layout: MapEntryLayout,
): string {
const B = host.B;
const step = (num: boolean): string => {
host.declare(num ? MAP_ITER_NUM_DECL : MAP_ITER_DECL);
const entry = B.tmp();
const helper = num ? "sc_map_iter_entry_num" : "sc_map_iter_entry";
B.line(`${entry} = call ptr @${helper}(ptr ${map}, double ${index})`);
return entry;
};
if (layout !== "either") return step(layout === "num");
host.declare(MAP_ITER_DECL);
const entry = host.B.tmp();
host.B.line(`${entry} = call ptr @sc_map_iter_entry(ptr ${map}, double ${index})`);
const kindPtr = B.tmp(),
kind = B.tmp(),
isNum = B.tmp();
B.line(`${kindPtr} = getelementptr inbounds %ScrMapIx, ptr ${map}, i32 0, i32 1`);
B.line(`${kind} = load i8, ptr ${kindPtr}`);
B.line(`${isNum} = icmp eq i8 ${kind}, 0`);
const numLabel = B.newLabel("mapit.num"),
entryLabel = B.newLabel("mapit.entry"),
join = B.newLabel("mapit.layout");
B.condBr(isNum, numLabel, entryLabel);
B.startBlock(numLabel);
const numEntry = step(true);
const numEnd = B.currentLabel();
B.br(join);
B.startBlock(entryLabel);
const otherEntry = step(false);
const entryEnd = B.currentLabel();
B.br(join);
B.startBlock(join);
const entry = B.tmp();
B.line(`${entry} = phi ptr [ ${numEntry}, %${numEnd} ], [ ${otherEntry}, %${entryEnd} ]`);
return entry;
}
@@ -230,9 +283,10 @@ export function emitMapIterSlot(
access: "f64" | "bool" | "ref",
type: IrType,
runtime: { decl: string; symbol: string },
layout: MapEntryLayout,
): string {
const B = host.B;
const entry = emitMapIterEntry(host, map, index);
const entry = emitMapIterEntry(host, map, index, layout);
const found = B.tmp();
B.line(`${found} = icmp ne ptr ${entry}, null`);
const hit = B.newLabel("mapit.hit"),
packages/compiler/src/backend/llvm/speculative-integers.test.ts
+72
View File
@@ -2,6 +2,7 @@ import { expect, test } from "vitest";
import { F64, type IrExpr, type IrStmt } from "../../ir/ir.js";
import {
isSpeculableValue,
planProvenIntegerLoop,
planSpeculativeLoop,
speculativeReads,
} from "./speculative-integers.js";
@@ -194,3 +195,74 @@ test("loops that only count and compare keep one copy", () => {
]);
expect(planSpeculativeLoop(scaled as never, all)?.locals.has("n")).toBe(true);
});
test("bounded cursors with proven writes are shadowed in a proven integer loop", () => {
// for (i = 0; i < n; i++) { if (a < b) run = run + 1; else run = 0;
// if (run > longest) longest = run; }
const sum = bin("+", ref("run"), num(1));
const step = assign("run", sum);
const reset = assign("run", num(0));
const copy = assign("longest", ref("run"));
const forLoop = (body: IrStmt[]): IrStmt => ({
kind: "for",
init: null,
cond: bin("<", ref("i"), ref("n")),
update: assign("i", bin("+", ref("i"), num(1))),
body,
loc,
});
const branch = (then: IrStmt[], else_: IrStmt[] | null): IrStmt => ({
kind: "if",
cond: bin("<", ref("a"), ref("b")),
then,
else_,
loc,
});
const body = [branch([step], [reset]), branch([copy], null)];
const range = { min: 0, max: 100 };
const cursors = new Map([
["run", range],
["longest", range],
]);
const plan = planProvenIntegerLoop(forLoop(body) as never, cursors, all, () => false, all)!;
expect([...plan.locals]).toEqual([
["run", range],
["longest", range],
]);
expect(plan.writes).toEqual(new Set([step, reset, copy]));
// A binding the loop does not bound keeps its double.
const only = planProvenIntegerLoop(
forLoop(body) as never,
new Map([["run", range]]),
all,
() => false,
all,
)!;
expect([...only.locals.keys()]).toEqual(["run"]);
// An unproven sum leaves `run` unshadowed, and its copy reads `run`
// where nothing proves it integral.
const unproven = (e: IrExpr): boolean => e !== sum && e.kind !== "varRef";
expect(
planProvenIntegerLoop(forLoop(body) as never, cursors, all, () => false, unproven),
).toBeNull();
// An expression-position write refuses the binding.
const inner: IrStmt = {
kind: "exprStmt",
expr: { kind: "incDec", op: "+", prefix: false, localId: "longest", type: F64, loc },
loc,
};
expect([
...planProvenIntegerLoop(
forLoop([...body, inner]) as never,
cursors,
all,
() => false,
all,
)!.locals.keys(),
]).toEqual(["run"]);
// A labeled jump may leave past the write-back.
const jump: IrStmt = { kind: "break", label: "outer", loc };
expect(
planProvenIntegerLoop(forLoop([...body, jump]) as never, cursors, all, () => false, all),
).toBeNull();
});
packages/compiler/src/backend/llvm/speculative-integers.ts
+100 -6
View File
@@ -3,7 +3,7 @@ import { everyExprChild, everyStmtChild } from "../../ir/traverse.js";
import { f64Lit } from "./common.js";
import type { LlValue, LlvmEmitterContext } from "./expr-context.js";
import { exactInteger, widenInteger } from "./integer-values.js";
import { withinInt32 } from "../../ir/integer-ranges.js";
import { type IntegerRange, withinInt32 } from "../../ir/integer-ranges.js";
/** Speculative integer loops.
*
@@ -410,6 +410,97 @@ export function planSpeculativeLoop(
return { locals, writes: speculated, observerFree };
}
/** A proven integer loop. The bindings integer-ranges.ts bounds over a for
* loop (`loopRanges`) hold an integer on entry and at every write, so the
* loop keeps them in i64 shadow slots without a second copy or checks, and
* writes their doubles once, when it is left. */
export interface ProvenLoopPlan {
/** Shadowed bindings and their loop ranges. */
locals: Map<string, IntegerRange>;
/** Statement-level writes computed as integers. */
writes: Set<IrStmt>;
}
/** Plan a proven integer loop. `cursors` are the loop's bounded bindings,
* `eligible` the locals a shadow may stand for, `integral` those the emitter
* already holds as integers and `proven` the expressions the range analysis
* proves integral. A shadowed binding's double is stale until the loop is
* left, so every exit has to pass the write-back: the caller plans only
* loops where an exception leaves the function, and a labeled jump (which
* may leave past the write-back) keeps the loop ordinary. */
export function planProvenIntegerLoop(
loop: SpeculativeLoop,
cursors: ReadonlyMap<string, IntegerRange>,
eligible: (localId: string) => boolean,
integral: (localId: string) => boolean,
proven: (e: IrExpr) => boolean,
): ProvenLoopPlan | null {
if (cursors.size === 0) return null;
const writes = new Map<string, IrStmt[]>();
const refused = new Set<string>();
let rejected = false;
const expr = (e: IrExpr): boolean => {
if (e.kind === "assignExpr" || e.kind === "incDec") refused.add(e.localId);
return everyExprChild(e, expr, nested);
};
const nested = (s: IrStmt): boolean => {
if (s.kind === "assign" || s.kind === "varDecl") refused.add(s.localId);
return everyStmtChild(s, expr, nested);
};
const stmt = (s: IrStmt): boolean => {
if (rejected) return false;
if (
!SIMPLE_STATEMENTS.has(s.kind) ||
((s.kind === "break" || s.kind === "continue") && s.label !== undefined)
) {
rejected = true;
return false;
}
const target = s.kind === "varDecl" ? null : speculativeWriteTarget(s);
if (target) {
if (target.value !== null && cursors.has(target.localId)) {
const list = writes.get(target.localId) ?? [];
list.push(s);
writes.set(target.localId, list);
} else refused.add(target.localId);
return target.value === null || expr(target.value);
}
if (s.kind === "varDecl") refused.add(s.localId);
return everyStmtChild(s, expr, stmt);
};
if (loop.cond) expr(loop.cond);
if (loop.kind === "for" && loop.update) nested(loop.update);
for (const s of loop.body) stmt(s);
if (rejected) return null;
const locals = new Map<string, IntegerRange>();
for (const [id, list] of writes) {
if (refused.has(id) || !eligible(id)) continue;
const exact = list.every((s) => {
const value = speculativeWriteTarget(s)!.value!;
return isSpeculableValue(value, false) && proven(value);
});
if (exact) locals.set(id, cursors.get(id)!);
}
// Every binding a shadowed write reads is shadowed, already an integer,
// or proven integral where it is read.
const grounded = (e: IrExpr): boolean =>
e.kind === "varRef"
? locals.has(e.localId) || integral(e.localId) || proven(e)
: everyExprChild(e, grounded, () => false);
for (let changed = true; changed;) {
changed = false;
for (const id of [...locals.keys()])
if (!writes.get(id)!.every((s) => grounded(speculativeWriteTarget(s)!.value!))) {
locals.delete(id);
changed = true;
}
}
if (locals.size === 0) return null;
const planned = new Set<IrStmt>();
for (const id of locals.keys()) for (const s of writes.get(id)!) planned.add(s);
return { locals, writes: planned };
}
const SAFE = Number.MAX_SAFE_INTEGER;
/** The range every speculative shadow holds. */
@@ -501,11 +592,14 @@ export function emitSpeculativeInteger(
e: IrExpr,
bail: (condition: string) => void,
reads: ReadonlyMap<IrExpr, LlValue> = new Map(),
integral = false,
): string {
const B = host.B;
// A value the range analysis already proves takes the ordinary integer
// emission, which needs no checks.
const proven = e.kind === "bin" ? host.integerRanges.get(e) : undefined;
// emission, which needs no checks. A proven integer loop's write
// (`integral`) is proven throughout and computes its operands as integers
// instead: the ordinary emission of a sum past int32 is a double.
const proven = e.kind === "bin" && !integral ? host.integerRanges.get(e) : undefined;
if (proven && proven.min >= -SAFE && proven.max <= SAFE && speculativeReads(e).length === 0)
return speculativeLeaf(host, host.emitExpr(e), e, bail);
switch (e.kind) {
@@ -533,8 +627,8 @@ export function emitSpeculativeInteger(
}
case "bin": {
if (e.op !== "+" && e.op !== "-" && e.op !== "*" && e.op !== "%") break;
const a = emitSpeculativeInteger(host, e.left, bail, reads);
const b = emitSpeculativeInteger(host, e.right, bail, reads);
const a = emitSpeculativeInteger(host, e.left, bail, reads, integral);
const b = emitSpeculativeInteger(host, e.right, bail, reads, integral);
const result = B.tmp();
if (e.op === "+" || e.op === "-") {
// With one operand's range known, the other is checked against the
@@ -590,7 +684,7 @@ export function emitSpeculativeInteger(
}
case "unary": {
if (e.op !== "-") break;
const a = emitSpeculativeInteger(host, e.operand, bail, reads);
const a = emitSpeculativeInteger(host, e.operand, bail, reads, integral);
const zero = B.tmp(),
result = B.tmp();
// -0 is not an integer result.
packages/compiler/src/backend/native-codegen.test.ts
+28
View File
@@ -192,6 +192,34 @@ test("runtime bitcode imports reach the helper and key the native cache", async
expect(calls[1]).toContain("--import-bitcode /pack/a.bc --import-bitcode /pack/b.bc");
});
test("profiles reach the helper and key the native cache by their bytes", async () => {
const pkg = await fakePackage();
const base = request(pkg.root, pkg.packageJson, join(pkg.root, "plain.o"));
await emitNativeArtifact(base);
await emitNativeArtifact({
...base,
outputPath: join(pkg.root, "instrumented.o"),
profile: { generate: "/profiles/%p.profraw" },
});
const used = {
...base,
outputPath: join(pkg.root, "used.o"),
profile: { use: { path: "/profiles/a.profdata", digest: "a".repeat(64) } },
};
await emitNativeArtifact(used);
await emitNativeArtifact({
...used,
outputPath: join(pkg.root, "changed.o"),
profile: { use: { path: "/profiles/a.profdata", digest: "b".repeat(64) } },
});
await emitNativeArtifact({ ...used, outputPath: join(pkg.root, "again.o") });
const calls = (await readFile(pkg.log, "utf8")).trim().split("\n");
expect(calls).toHaveLength(4);
expect(calls[0]).not.toContain("--profile-");
expect(calls[1]).toContain("--profile-generate /profiles/%p.profraw");
expect(calls[2]).toContain("--profile-use /profiles/a.profdata");
});
test("chunked LLVM input preserves emitted bytes and the native cache identity", async () => {
const pkg = await fakePackage();
const first = request(pkg.root, pkg.packageJson, join(pkg.root, "chunks.o"));
packages/compiler/src/backend/native-codegen.ts
+34
View File
@@ -27,6 +27,11 @@ import {
type NativeTargetSpec,
} from "./targets.js";
import { compilerReleaseVersion } from "../library/sidecar.js";
import {
nativeProfileHelperArgs,
nativeProfileKey,
type NativeProfileOptions,
} from "./native-profile.js";
import { writeTextParts } from "../text-parts.js";
import {
@@ -86,6 +91,8 @@ export interface NativeCodegenOptions {
partitions?: number;
/** Runtime bitcode whose small functions the helper imports for inlining. */
importBitcode?: { paths: readonly string[]; digests: readonly string[] };
/** Profile-guided optimization: instrumentation or an indexed profile. */
profile?: NativeProfileOptions;
}
/** Writes LLVM text in large batches; see `writeTextParts`. */
@@ -298,6 +305,7 @@ function cacheKey(
? ""
: `\0import:${options.importBitcode.digests.join(",")}`,
)
.update(options.profile === undefined ? "" : `\0${nativeProfileKey(options.profile)}`)
.digest("hex");
}
@@ -319,6 +327,31 @@ async function installVerifiedCache(source: string, destination: string): Promis
}
}
/** `scriptc merge-profiles`: merges raw or indexed IR profiles into one
* indexed profile for `--profile-use`. */
export async function mergeNativeProfiles(
outputPath: string,
inputPaths: readonly string[],
sparse = false,
): Promise<void> {
const target = nativeCodegenTarget();
if (target === null) {
throw new NativeCodegenError(
"SC3002",
nativeCodegenTargetRefusal() ?? "profile merging is unsupported for this target",
"unsupported_target",
);
}
const helper = await resolveHelper(target);
await invoke(helper.binaryPath, [
"merge-profiles",
"--output",
outputPath,
...(sparse ? ["--sparse"] : []),
...inputPaths,
]);
}
export async function emitNativeArtifact(
options: NativeCodegenOptions,
): Promise<NativeCodegenArtifact> {
@@ -411,6 +444,7 @@ export async function emitNativeArtifact(
options.sourcePath,
...(options.importBitcode?.paths ?? []).flatMap((path) => ["--import-bitcode", path]),
...(partitionCache === null ? [] : ["--cache-dir", partitionCache]),
...nativeProfileHelperArgs(options.profile),
]);
for (const stage of stages) {
const emitted = await stat(stage).catch(() => null);
packages/compiler/src/backend/native-link-info-core.ts
+12
View File
@@ -46,6 +46,12 @@ export interface NativeLinkFeatures {
/** Checked-dynamic stream slots: the scr_stream_dyn.c unit. Absent (not
* false) for programs without them. */
streamDyn?: boolean;
/** Own properties JavaScript adds to children: scr_dyn_handle_props.c.
* Absent (false) for programs without them. */
handleProps?: boolean;
/** for await over checked-dynamic values: scr_dyn_for_await.c. Absent
* (false) for programs without it. */
dynForAwait?: boolean;
/** events.once with options: scr_events_once_signal.c. Absent (not
* false) for programs without it. */
eventsOnceSignal?: boolean;
@@ -61,6 +67,7 @@ export interface NativeLinkFeatures {
bigint: boolean;
searchParams: boolean;
qs: boolean;
utilPromisify: boolean;
dynBuiltinClass: boolean;
dynBuiltinStream: boolean;
parseArgs: boolean;
@@ -71,6 +78,8 @@ export interface NativeLinkFeatures {
dgram: boolean;
/** node:dns/promises: scr_dns_promises.c. */
dnsPromises: boolean;
/** dgram sockets as untyped values: scr_dgram_dyn.c. */
dgramDyn: boolean;
watch: boolean;
foreignFfi: boolean;
nodeTest: boolean;
@@ -78,6 +87,9 @@ export interface NativeLinkFeatures {
/** crypto.createCipheriv/createDecipheriv: scr_cipher.c plus the mbedTLS
* archive. */
cipher: boolean;
/** crypto.createECDH and ECDH.convertKey: scr_ecdh.c plus the mbedTLS
* archive. */
ecdh: boolean;
/** perf_hooks event-loop metrics: scr_perf_loop.c. */
perfLoop: boolean;
tlsCa: boolean;
packages/compiler/src/backend/native-profile.test.ts
+73
View File
@@ -0,0 +1,73 @@
import { expect, test } from "vitest";
import { executableLinkInputs } from "./link-plan-core.js";
import {
nativeProfileHelperArgs,
nativeProfileKey,
nativeProfileRefusal,
profileGenerateLinkerArgs,
} from "./native-profile.js";
import { NATIVE_TARGETS } from "./targets.js";
const BASE = {
outputKind: "exe",
optimization: "speed",
sanitize: false,
platform: "linux",
} as const;
test("profiles are accepted for optimized executables", () => {
expect(nativeProfileRefusal(BASE)).toBeNull();
expect(nativeProfileRefusal({ ...BASE, profileGenerate: "/p/%p.profraw" })).toBeNull();
expect(nativeProfileRefusal({ ...BASE, profileUse: "/p/a.profdata" })).toBeNull();
expect(
nativeProfileRefusal({ ...BASE, optimization: "release", profileUse: "/p/a.profdata" }),
).toBeNull();
expect(
nativeProfileRefusal({ ...BASE, platform: "darwin", profileUse: "/p/a.profdata" }),
).toBeNull();
});
test.each([
[{ profileGenerate: "/p/%p.profraw", profileUse: "/p/a.profdata" }, /cannot be combined/],
[{ profileUse: "/p/a.profdata", outputKind: "obj" }, /--profile-use is only supported/],
[{ profileGenerate: "/p/%p.profraw", sanitize: true }, /--sanitize/],
[{ profileUse: "/p/a.profdata", optimization: "dev" }, /optimized build/],
[{ profileGenerate: "/p/%p.profraw", platform: "darwin" }, /Linux executables/],
[{ profileGenerate: "/p/%p.profraw", platform: "wasi" }, /Linux executables/],
] as const)("refuses %j", (override, message) => {
expect(nativeProfileRefusal({ ...BASE, ...override })).toMatch(message);
});
test("helper arguments and cache keys name the profile", () => {
expect(nativeProfileHelperArgs(undefined)).toEqual([]);
expect(nativeProfileKey(undefined)).toBe("");
expect(nativeProfileHelperArgs({ generate: "/p/%p.profraw" })).toEqual([
"--profile-generate",
"/p/%p.profraw",
]);
expect(nativeProfileKey({ generate: "/p/%p.profraw" })).toBe("profile-generate:/p/%p.profraw");
const use = { use: { path: "/p/a.profdata", digest: "d".repeat(64) } };
expect(nativeProfileHelperArgs(use)).toEqual(["--profile-use", "/p/a.profdata"]);
expect(nativeProfileKey(use)).toBe(`profile-use:${"d".repeat(64)}`);
});
test("only the instrumented link adds the profile runtime", () => {
expect(profileGenerateLinkerArgs(false)).toEqual([]);
expect(profileGenerateLinkerArgs(true)).toEqual(["-fprofile-generate"]);
const target = NATIVE_TARGETS.find((candidate) => candidate.platform === "linux")!;
const inputs = {
target,
programObject: "/p/program.o",
ffiLibraries: [],
ffiSystemLibraries: [],
ffiFrameworks: [],
runtimeObjects: [],
runtimeArchives: [],
runtimeSystemLibraries: [],
optimization: "speed",
} as const;
expect(executableLinkInputs(inputs).driverFlags).not.toContain("-fprofile-generate");
expect(executableLinkInputs({ ...inputs, profileGenerate: true }).driverFlags).toContain(
"-fprofile-generate",
);
});
packages/compiler/src/backend/native-profile.ts
+57
View File
@@ -0,0 +1,57 @@
/** Profile-guided optimization for helper-compiled executables.
*
* An instrumented build (`--profile-generate`) counts how often each branch
* and call runs and writes the counts when the program exits. `scriptc
* merge-profiles` turns those raw profiles into one indexed profile, and an
* optimized build (`--profile-use`) lays out, inlines, and splits code by it.
* The helper applies both in the pre-link pipeline of every partition.
*/
import type { NativeOptimization } from "./optimization.js";
export interface NativeProfileOptions {
/** Raw profile path pattern written by the instrumented executable. */
generate?: string;
/** Indexed profile and the digest of its bytes, which key every cache. */
use?: { path: string; digest: string };
}
/** Why a request cannot use profiles, or null when it can. */
export function nativeProfileRefusal(options: {
profileGenerate?: string;
profileUse?: string;
outputKind: string;
optimization: NativeOptimization;
sanitize: boolean;
platform: string;
}): string | null {
const generate = options.profileGenerate !== undefined;
const use = options.profileUse !== undefined;
if (!generate && !use) return null;
const flag = generate ? "--profile-generate" : "--profile-use";
if (generate && use) return "--profile-generate and --profile-use cannot be combined";
if (options.outputKind !== "exe") return `${flag} is only supported for executable output`;
if (options.sanitize) return `${flag} cannot be combined with --sanitize`;
if (options.optimization === "dev")
return `${flag} needs an optimized build (--optimization=release or speed)`;
if (generate && options.platform !== "linux")
return "--profile-generate is only supported for Linux executables";
return null;
}
export function nativeProfileHelperArgs(profile: NativeProfileOptions | undefined): string[] {
if (profile?.generate !== undefined) return ["--profile-generate", profile.generate];
if (profile?.use !== undefined) return ["--profile-use", profile.use.path];
return [];
}
/** Cache-key part; empty without a profile so other keys are unchanged. */
export function nativeProfileKey(profile: NativeProfileOptions | undefined): string {
if (profile?.generate !== undefined) return `profile-generate:${profile.generate}`;
if (profile?.use !== undefined) return `profile-use:${profile.use.digest}`;
return "";
}
/** The instrumented link adds the compiler driver's profile runtime. */
export function profileGenerateLinkerArgs(generate: boolean): string[] {
return generate ? ["-fprofile-generate"] : [];
}
packages/compiler/src/backend/native-tools.ts
+25
View File
@@ -8,6 +8,7 @@ import {
validateNativeCodegenVersion,
type NativeCodegenOutputKind,
} from "./native-codegen-core.js";
import { nativeProfileHelperArgs, type NativeProfileOptions } from "./native-profile.js";
export function runNativeTool(
executable: string,
@@ -45,6 +46,8 @@ export function emitNativeObject(options: {
partitions?: number;
/** Private directory where the helper reuses partition artifacts. */
partitionCache?: string;
/** Profile-guided optimization: instrumentation or an indexed profile. */
profile?: NativeProfileOptions;
}): string[] {
verifyNativeHelper(options);
const outputs = nativePartitionPaths(options.outputPath, options.partitions ?? 1);
@@ -68,11 +71,33 @@ export function emitNativeObject(options: {
...(options.partitionCache === undefined || outputs.length < 2
? []
: ["--cache-dir", options.partitionCache]),
...nativeProfileHelperArgs(options.profile),
]);
for (const output of outputs) requireNativeArtifact(output);
return outputs;
}
/** Merges raw or indexed IR profiles into one indexed profile. */
export function mergeNativeProfiles(options: {
executable: string;
packageRoot: string;
compilerVersion: string;
target: NativeTargetSpec;
helper: NativeHelperSpec;
outputPath: string;
inputPaths: readonly string[];
sparse: boolean;
}): void {
verifyNativeHelper(options);
runNativeTool(options.executable, [
"merge-profiles",
"--output",
options.outputPath,
...(options.sparse ? ["--sparse"] : []),
...options.inputPaths,
]);
}
/** Verify installed identity even when a caller can reuse a cached object. */
export function verifyNativeHelper(options: {
executable: string;
packages/compiler/src/backend/native/contracts.ts
+15
View File
@@ -116,6 +116,10 @@ export interface CcOptions {
/** Checked-dynamic stream slots (stream.dynNative / stream.pipeDyn):
* compiles scr_stream_dyn.c. */
streamDyn?: boolean;
/** Own properties JavaScript adds to children: compiles scr_dyn_handle_props.c. */
handleProps?: boolean;
/** for await over checked-dynamic values: compiles scr_dyn_for_await.c. */
dynForAwait?: boolean;
/** events.once with options (events.onceSignal): compiles
* scr_events_once_signal.c. */
eventsOnceSignal?: boolean;
@@ -163,6 +167,10 @@ export interface CcOptions {
* line (escape-only programs ride the always-linked component encoder
* and never flip this). */
qs?: boolean;
/** The program promisifies an untyped function (util.promisify over
* `any`): compiles scr_util_promisify.c, which also references the
* dyn-async and dyn-invoke units. */
utilPromisify?: boolean;
/** The program uses EventEmitter or a stream class as an untyped
* constructor value (a constructor function's base): compiles
* scr_dyn_builtin_class.c, which also references the inspection,
@@ -203,6 +211,10 @@ export interface CcOptions {
* compiles scr_dns_promises.c over the dgram unit, so callback-only
* binaries keep their exact link line. */
dnsPromises?: boolean;
/** A dgram socket converts to or from an untyped value: compiles
* scr_dgram_dyn.c over the dgram unit, so typed dgram binaries keep
* their exact link line. */
dgramDyn?: boolean;
/** The program uses fs.watch (moduleUsesFsWatch on the IR): compiles
* scr_watch.c into the binary — the net gating precedent, so watch-free
@@ -234,6 +246,9 @@ export interface CcOptions {
/** crypto.createCipheriv/createDecipheriv: compiles scr_cipher.c and links
* the mbedTLS archive (shared with tls). */
cipher?: boolean;
/** crypto.createECDH and ECDH.convertKey: compiles scr_ecdh.c and links
* the mbedTLS archive (shared with tls). */
ecdh?: boolean;
/** crypto.subtle importKey/sign/verify/digest: compiles scr_webcrypto.c. */
webcrypto?: boolean;
/** perf_hooks event-loop metrics: compiles scr_perf_loop.c. */
packages/compiler/src/backend/native/executable.ts
+17 -3
View File
@@ -113,6 +113,7 @@ interface RuntimeUnitContext {
tls: boolean;
tlsCa: boolean;
cipher: boolean;
ecdh: boolean;
lreObjects: readonly string[];
zlibObjects: readonly string[];
tlsArchive: string | null;
@@ -184,7 +185,7 @@ const RUNTIME_UNITS: readonly RuntimeUnit[] = [
// Builtin functions read as values in JavaScript dependencies.
unit((u) => u.opts.builtinValues === true, "scr_builtin_values.c"),
unit((u) => u.opts.dc, "scr_dc.c"),
unit((u) => u.opts.dynTier, "scr_js_value.c"),
unit((u) => u.opts.dynTier, "scr_js_value.c", "scr_js_object.c"),
unit((u) => u.opts.eventTarget || u.nativeFetch, "scr_event_target.c"),
unit((u) => u.opts.wasm, "scr_wasm.c"),
unit(
@@ -245,6 +246,8 @@ const RUNTIME_UNITS: readonly RuntimeUnit[] = [
(u) => u.opts.parseArgs && u.opts.inspect && u.opts.dynAsync && u.opts.dynInvoke && u.regex,
"scr_util_callable.c",
),
// util.promisify over an untyped function: dyn promises and calls.
unit((u) => u.opts.utilPromisify && u.opts.dynAsync && u.opts.dynInvoke, "scr_util_promisify.c"),
unit(
(u) =>
u.opts.dynBuiltinClass &&
@@ -272,6 +275,10 @@ const RUNTIME_UNITS: readonly RuntimeUnit[] = [
// Checked-dynamic stream slots test values and call JavaScript stream
// members through the dyn-invoke unit.
unit((u) => u.opts.streamDyn === true, "scr_stream_dyn.c"),
// Own properties JavaScript adds to children.
unit((u) => u.opts.handleProps === true, "scr_dyn_handle_props.c"),
// for await over checked-dynamic values.
unit((u) => u.opts.dynForAwait === true, "scr_dyn_for_await.c"),
// FileHandle streams reference both the stream unit and FileHandle.
unit((u) => u.opts.stream && u.opts.fileHandle, "scr_fs_stream_fh.c"),
// The readiness-poller backends (scr_platform.h): kqueue on macOS/BSD,
@@ -285,6 +292,7 @@ const RUNTIME_UNITS: readonly RuntimeUnit[] = [
unit((u) => u.opts.http2, "scr_http2.c"),
unit((u) => u.opts.dgram, "scr_dgram.c"),
unit((u) => u.opts.dgram && u.opts.dnsPromises, "scr_dns_promises.c"),
unit((u) => u.opts.dgram && u.opts.dgramDyn, "scr_dgram_dyn.c"),
unit((u) => u.opts.watch, "scr_watch.c"),
unit((u) => u.opts.foreignFfi, "scr_ffi_queue.c"),
unit((u) => u.opts.nodeTest, "scr_test.c"),
@@ -299,6 +307,7 @@ const RUNTIME_UNITS: readonly RuntimeUnit[] = [
join(vendorTlsDir(), "include"),
...(u.tls ? [u.rt(join(u.rtDir, "scr_tls.c"))] : []),
...(u.cipher ? [u.rt(join(u.rtDir, "scr_cipher.c"))] : []),
...(u.ecdh ? [u.rt(join(u.rtDir, "scr_ecdh.c"))] : []),
u.tlsArchive,
// mbedTLS's win32 entropy poll is BCryptGenRandom (bcrypt.h).
// The unconditional win32 libs above do not carry it.
@@ -334,6 +343,9 @@ const RUNTIME_DEFINES: readonly {
when: (opts: CcOptions) => boolean | undefined;
}[] = [
{ define: "SCR_TEXT_DECODER_LEGACY", when: (opts) => opts.textDecoderLegacy },
// Checked objects carry the dynamic tier's shapes (scr_json.c's SCR_JS_TIER build); only
// programs with dynamic-tier code select it.
{ define: "SCR_JS_TIER", when: (opts) => opts.dynTier === true },
{ define: "SCR_WARNING_REPLACE", when: (opts) => opts.warningReplace },
{ define: "SCR_STDIO_EXTENDED", when: (opts) => opts.stdioExtended },
{ define: "SCR_YIN", when: (opts) => opts.youngNursery },
@@ -360,9 +372,10 @@ export async function compileCInternal(
const http = (opts.http ?? false) || nativeFetch;
const tls = (opts.tls ?? false) || nativeFetch;
const tlsCa = (opts.tlsCa ?? false) || tls;
// Symmetric ciphers link the same mbedTLS archive as TLS.
// Symmetric ciphers and ECDH link the same mbedTLS archive as TLS.
const cipher = opts.cipher ?? false;
const mbedtls = tls || cipher;
const ecdh = opts.ecdh ?? false;
const mbedtls = tls || cipher || ecdh;
const driver = resolveCc();
const darwinDebugSymbols = needsDarwinDebugSymbols(
targetPlatform(driver),
@@ -719,6 +732,7 @@ export async function compileCInternal(
tls,
tlsCa,
cipher,
ecdh,
lreObjects,
zlibObjects,
tlsArchive,
packages/compiler/src/backend/native/library.ts
+2 -1
View File
@@ -112,7 +112,7 @@ export async function compileLibArchive(opts: LibArchiveOptions): Promise<void>
...(opts.assert && opts.bigint ? ["scr_bigint_assert.c"] : []),
...(opts.emitter ? ["scr_events_emitter.c", "scr_dyn_handle.c"] : []),
...(opts.dynInvoke ? ["scr_dyn_invoke.c"] : []),
...(opts.dynTier === true ? ["scr_js_value.c"] : []),
...(opts.dynTier === true ? ["scr_js_value.c", "scr_js_object.c"] : []),
...(opts.zlib ? ["scr_zlib.c"] : []),
...(opts.copying ? ["scr_copying.c"] : []),
];
@@ -129,6 +129,7 @@ export async function compileLibArchive(opts: LibArchiveOptions): Promise<void>
"-DSCR_LIB",
...(opts.threadInstances ? ["-DSCR_THREAD_INSTANCES"] : []),
...(opts.textDecoderLegacy ? ["-DSCR_TEXT_DECODER_LEGACY"] : []),
...(opts.dynTier === true ? ["-DSCR_JS_TIER"] : []),
"-I",
rtDir,
...(regex ? ["-I", vendorEngineDir()] : []),
packages/compiler/src/backend/runtime-pack-core.ts
+4
View File
@@ -85,6 +85,8 @@ export interface RuntimeFeatureSet extends NativeLinkFeatures {
wasm: boolean;
dynTier: boolean;
streamDyn: boolean;
handleProps: boolean;
dynForAwait: boolean;
eventsOnceSignal: boolean;
dynBufferNum: boolean;
builtinValues: boolean;
@@ -280,6 +282,8 @@ export function effectiveRuntimeFeatures(
wasm: features.wasm === true,
dynTier: features.dynTier === true,
streamDyn: features.streamDyn === true,
handleProps: features.handleProps === true,
dynForAwait: features.dynForAwait === true,
eventsOnceSignal: features.eventsOnceSignal === true,
dynBufferNum: features.dynBufferNum === true,
builtinValues: features.builtinValues === true,
packages/compiler/src/backend/runtime-pack-targets.test.ts
+3
View File
@@ -107,6 +107,7 @@ const features = {
bigint: false,
searchParams: false,
qs: false,
utilPromisify: false,
dynBuiltinClass: false,
dynBuiltinStream: false,
parseArgs: false,
@@ -116,12 +117,14 @@ const features = {
http2: false,
dgram: false,
dnsPromises: false,
dgramDyn: false,
watch: false,
foreignFfi: false,
nodeTest: false,
tls: false,
tlsCa: false,
cipher: false,
ecdh: false,
webcrypto: false,
perfLoop: false,
};
packages/compiler/src/backend/runtime-pack.test.ts
+3
View File
@@ -44,6 +44,7 @@ const BASE: NativeLinkFeatures = {
bigint: false,
searchParams: false,
qs: false,
utilPromisify: false,
dynBuiltinClass: false,
dynBuiltinStream: false,
parseArgs: false,
@@ -53,12 +54,14 @@ const BASE: NativeLinkFeatures = {
http2: false,
dgram: false,
dnsPromises: false,
dgramDyn: false,
watch: false,
foreignFfi: false,
nodeTest: false,
tls: false,
tlsCa: false,
cipher: false,
ecdh: false,
webcrypto: false,
perfLoop: false,
};
packages/compiler/src/backend/target-diagnostics.ts
+7
View File
@@ -208,6 +208,13 @@ export function moduleWasiUnavailableSurface(
};
return false;
}
if (node.fn === "crypto.ecdhNew" || node.fn === "crypto.ecdhConvertKey") {
found = {
surface: "ECDH (the wasm32-wasi runtime does not include the mbedTLS library)",
loc,
};
return false;
}
if (node.fn === "os.networkInterfaces") {
found = {
surface: "network-interface enumeration (WASI Preview 1 has no interface API)",
packages/compiler/src/backend/warm-cache.ts
+3
View File
@@ -50,6 +50,7 @@ export async function warmNativeCaches(
bigint: false,
searchParams: false,
qs: false,
utilPromisify: false,
dynBuiltinClass: false,
dynBuiltinStream: false,
parseArgs: false,
@@ -59,11 +60,13 @@ export async function warmNativeCaches(
http2: false,
dgram: false,
dnsPromises: false,
dgramDyn: false,
watch: false,
foreignFfi: false,
nodeTest: false,
tls: false,
cipher: false,
ecdh: false,
webcrypto: false,
perfLoop: false,
tlsCa: false,
packages/compiler/src/cli/command.ts
+68 -1
View File
@@ -95,6 +95,9 @@ async function main(args: string[], host: CliHost): Promise<number> {
values.strip ||
values["windows-subsystem"] !== undefined ||
values["detached-exit"] ||
values["profile-generate"] !== undefined ||
values["profile-use"] !== undefined ||
values.sparse ||
(values["npm-static"] ?? []).length > 0 ||
values["provenance-sources"] ||
externalTypeArgs.length > 0 ||
@@ -135,6 +138,50 @@ async function main(args: string[], host: CliHost): Promise<number> {
}
return 0;
}
if (command === "merge-profiles") {
const inputs = positionals.slice(1);
const others =
values.lib ||
values.backend !== undefined ||
values.emit !== undefined ||
values.print !== undefined ||
values["fail-on"] !== undefined ||
values.optimization !== undefined ||
values.ffi !== undefined ||
values.profile !== undefined ||
values.strip ||
values.sanitize ||
values["windows-subsystem"] !== undefined ||
values["detached-exit"] ||
values["profile-generate"] !== undefined ||
values["profile-use"] !== undefined ||
(values["npm-static"] ?? []).length > 0 ||
values["provenance-sources"] ||
externalTypeArgs.length > 0 ||
(values.wasm ?? []).length > 0 ||
values["emit-ir"] ||
values.tier !== undefined ||
values["explain-dynamic"] !== undefined ||
!values["keep-llvm"];
if (values.out === undefined || inputs.length === 0 || others) {
fail(
`scriptc merge-profiles takes -o <out.profdata>, optionally --sparse, and one or more profiles\n\n${USAGE}`,
);
}
const outputPath = resolve(values.out);
try {
await host.mergeProfiles(
outputPath,
inputs.map((input) => resolve(input)),
values.sparse,
);
} catch (error) {
fail(`scriptc: ${error instanceof Error ? error.message : String(error)}`);
}
process.stdout.write(`${outputPath}\n`);
return 0;
}
if (values.sparse) fail(`--sparse is a merge-profiles option\n\n${USAGE}`);
if (command !== "build" && command !== "run" && command !== "coverage") {
fail(`unknown command "${command}"\n\n${USAGE}`);
}
@@ -162,13 +209,15 @@ async function main(args: string[], host: CliHost): Promise<number> {
values.ffi !== undefined ||
values["windows-subsystem"] !== undefined ||
values["detached-exit"] ||
values["profile-generate"] !== undefined ||
values["profile-use"] !== undefined ||
(values["npm-static"] ?? []).length > 0 ||
externalTypeArgs.length > 0 ||
values.tier !== undefined ||
values["explain-dynamic"] !== undefined
) {
fail(
"scriptc build --lib takes no --backend/--emit/--print/--optimization/--strip/--windows-subsystem/--detached-exit/--npm-static/--ffi/--external-types/--tier/--explain-dynamic: the profile pins the emission and optimization, npm imports are judged automatically, outbound FFI belongs to executable builds, external type mappings belong to coverage, and library artifacts are static-tier only",
"scriptc build --lib takes no --backend/--emit/--print/--optimization/--strip/--windows-subsystem/--detached-exit/--profile-generate/--profile-use/--npm-static/--ffi/--external-types/--tier/--explain-dynamic: the profile pins the emission and optimization, npm imports are judged automatically, outbound FFI belongs to executable builds, external type mappings belong to coverage, and library artifacts are static-tier only",
);
}
const profilePath = resolve(profileArg);
@@ -214,6 +263,12 @@ async function main(args: string[], host: CliHost): Promise<number> {
if (command === "coverage" && values["detached-exit"]) {
fail(`--detached-exit is a build/run option\n\n${USAGE}`);
}
if (command === "coverage" && values["profile-generate"] !== undefined) {
fail(`--profile-generate is a build/run option\n\n${USAGE}`);
}
if (command === "coverage" && values["profile-use"] !== undefined) {
fail(`--profile-use is a build/run option\n\n${USAGE}`);
}
if (
values.print !== undefined &&
values.print !== "native-link-info" &&
@@ -323,6 +378,10 @@ async function main(args: string[], host: CliHost): Promise<number> {
...(values.optimization === undefined ? {} : { optimization: values.optimization }),
strip: values.strip,
detachedExit: values["detached-exit"],
...(values["profile-generate"] === undefined
? {}
: { profileGenerate: values["profile-generate"] }),
...(values["profile-use"] === undefined ? {} : { profileUse: values["profile-use"] }),
...(windowsSubsystem === undefined ? {} : { windowsSubsystem }),
...(values.ffi === undefined ? {} : { ffi: values.ffi }),
});
@@ -425,6 +484,14 @@ async function main(args: string[], host: CliHost): Promise<number> {
...(values.strip ? { strip: true } : {}),
...(windowsSubsystem !== undefined ? { windowsSubsystem } : {}),
...(values["detached-exit"] ? { detachedExit: true } : {}),
// The instrumented program resolves a relative pattern against its own
// working directory; resolve it here, where the user wrote it.
...(values["profile-generate"] === undefined
? {}
: { profileGenerate: resolve(values["profile-generate"]) }),
...(values["profile-use"] === undefined
? {}
: { profileUse: resolve(values["profile-use"]) }),
...(npmStatic !== undefined ? { npmStatic } : {}),
...(ffiProfilePath !== undefined ? { ffiProfilePath } : {}),
...(printNativeLinkInfo ? { nativeLinkInfo: true } : {}),
packages/compiler/src/cli/host.ts
+3
View File
@@ -27,5 +27,8 @@ export interface CliHost {
cacheRoot: string;
profiles: { profile: NativeCacheWarmProfile; elapsedMs: number }[];
}>;
/** Merges raw or indexed profiles into one indexed profile; a sparse one
* leaves out functions that never ran. */
mergeProfiles: (outputPath: string, inputPaths: string[], sparse: boolean) => Promise<void>;
run: (binary: string) => Promise<number>;
}
packages/compiler/src/cli/output-options.ts
+11
View File
@@ -11,6 +11,8 @@ export interface OutputOptionValues {
strip?: boolean;
windowsSubsystem?: string;
detachedExit?: boolean;
profileGenerate?: string;
profileUse?: string;
ffi?: string;
}
@@ -73,6 +75,15 @@ export function resolveOutputOptions(
if (values.detachedExit && emit !== "exe") {
return { ok: false, message: `--detached-exit is only supported with --emit=exe` };
}
if (values.profileGenerate !== undefined && values.profileUse !== undefined) {
return { ok: false, message: `--profile-generate and --profile-use cannot be combined` };
}
if (values.profileGenerate !== undefined && emit !== "exe") {
return { ok: false, message: `--profile-generate is only supported with --emit=exe` };
}
if (values.profileUse !== undefined && emit !== "exe") {
return { ok: false, message: `--profile-use is only supported with --emit=exe` };
}
if (emit === "ir" && backend !== undefined) {
return {
ok: false,
packages/compiler/src/cli/usage.ts
+25 -3
View File
@@ -15,6 +15,13 @@ Usage:
contract sidecar JSON beside the archive
scriptc cache warm [runtime|tls…] verify and load precompiled runtime families
for the current compiler/SDK/target
scriptc merge-profiles -o <out.profdata> [--sparse] <profile…>
merge the raw profiles that
--profile-generate runs wrote (or
earlier merged profiles) into one
profile for --profile-use;
--sparse leaves out functions
that never ran (a smaller file)
Options:
-o, --out <path> primary output path (default: .scriptc/<name><suffix>)
@@ -46,6 +53,19 @@ Options:
a helper process so the parent sees the exit sooner. The
helper keeps the executable mapped and its memory charged
for up to ~100 ms after exit (opt in; --emit=exe only)
--profile-generate <pattern>
Linux: instrument the executable for profile-guided
optimization. Each run writes a raw profile to the
pattern (%p expands to the process id; the
LLVM_PROFILE_FILE environment variable overrides it).
Instrumented programs run slower (--emit=exe only)
--profile-use <file.profdata>
optimize with a merged profile (scriptc merge-profiles):
code is laid out, inlined and split by how the profiled
runs used it. Code the profile does not describe, or
describes from an older version of the program, is
optimized as without one (release or speed; --emit=exe
only)
--keep-llvm keep generated LLVM IR beside the executable (default)
--no-keep-llvm delete generated LLVM IR after compiling
--emit-ir also write IR beside an executable or library archive;
@@ -73,9 +93,8 @@ Options:
--tier <auto|static-only>
tier policy (default: auto). static-only keeps every
construct without a static lowering a compile error, so
the program has no dynamic-tier demotions, and makes a
checked cast whose value does not fit its type throw
instead of continuing with the value
the program has no dynamic-tier demotions. Both modes
currently compile identically
--explain-dynamic[=<name>]
coverage only: list the functions that operate on dynamic
values and the bindings, fields, functions, classes and
@@ -100,6 +119,9 @@ export const CLI_OPTIONS = {
strip: { type: "boolean", default: false },
"windows-subsystem": { type: "string" },
"detached-exit": { type: "boolean", default: false },
"profile-generate": { type: "string" },
"profile-use": { type: "string" },
sparse: { type: "boolean", default: false },
"keep-llvm": { type: "boolean", default: true },
"emit-ir": { type: "boolean", default: false },
sanitize: { type: "boolean", default: false },
packages/compiler/src/compile-types.ts
+9
View File
@@ -36,6 +36,15 @@ export interface CompileBaseOptions {
* executable mapped and the memory charged for up to ~100 ms after exit.
* Executables only; no effect on other platforms. */
detachedExit?: boolean;
/** Instrument the executable for profile-guided optimization. Each run
* writes a raw profile to this path pattern (`%p` expands to the process
* id; `LLVM_PROFILE_FILE` overrides the pattern at run time). Linux
* executables only. */
profileGenerate?: string;
/** Optimize the executable with this indexed profile (`scriptc
* merge-profiles`). Code the profile does not describe, or describes
* from an older version of the program, is optimized as without one. */
profileUse?: string;
/** Eligible packages compile automatically. Explicit names override automatic selection heuristics. */
npmStatic?: readonly string[] | "auto";
packages/compiler/src/contracts/check.ts
+1 -4
View File
@@ -889,10 +889,7 @@ class Int32Contracts {
const value = s.kind === "varDecl" ? s.init : s.value;
if (note && value && !int32(value))
fail(note, value, value.loc, "the value stored into");
} else if (
s.kind === "forOf" ||
(s.kind === "tryCatch" && s.catchLocalId !== undefined)
) {
} else if (s.kind === "forOf" || (s.kind === "tryCatch" && s.catchLocalId !== null)) {
const id = s.kind === "forOf" ? s.localId : s.catchLocalId!;
const note = localNote(id);
if (note)
packages/compiler/src/executable/executable-cache.test.ts
+5
View File
@@ -45,6 +45,7 @@ const native: EarlyExecutableNativeFeatures = {
bigint: false,
searchParams: false,
qs: false,
utilPromisify: false,
dynBuiltinClass: false,
dynBuiltinStream: false,
parseArgs: false,
@@ -54,12 +55,14 @@ const native: EarlyExecutableNativeFeatures = {
http2: false,
dgram: false,
dnsPromises: false,
dgramDyn: false,
watch: false,
foreignFfi: false,
nodeTest: false,
tls: false,
tlsCa: false,
cipher: false,
ecdh: false,
webcrypto: false,
perfLoop: false,
};
@@ -220,6 +223,8 @@ test("early executable keys isolate compile modes, paths, implementation, checke
{ ...f.options, optimization: "dev" },
{ ...f.options, optimization: "speed" },
{ ...f.options, detachedExit: true },
{ ...f.options, profile: "profile-generate:/tmp/%p.profraw" },
{ ...f.options, profile: "profile-use:digest" },
{ ...f.options, checker: "tsc-ts-executable-identity" },
{ ...f.options, npmStatic: "auto" },
{ ...f.options, npmStatic: ["commander"] },
packages/compiler/src/executable/executable-cache.ts
+13
View File
@@ -67,6 +67,8 @@ export interface EarlyExecutableNativeFeatures {
dynAsync: boolean;
dynTier?: boolean;
streamDyn?: boolean;
handleProps?: boolean;
dynForAwait?: boolean;
eventsOnceSignal?: boolean;
dynBufferNum?: boolean;
builtinValues?: boolean;
@@ -76,6 +78,7 @@ export interface EarlyExecutableNativeFeatures {
bigint: boolean;
searchParams: boolean;
qs: boolean;
utilPromisify: boolean;
dynBuiltinClass: boolean;
dynBuiltinStream: boolean;
parseArgs: boolean;
@@ -85,11 +88,13 @@ export interface EarlyExecutableNativeFeatures {
http2: boolean;
dgram: boolean;
dnsPromises: boolean;
dgramDyn: boolean;
watch: boolean;
foreignFfi: boolean;
nodeTest: boolean;
tls: boolean;
cipher: boolean;
ecdh: boolean;
perfLoop: boolean;
tlsCa: boolean;
webcrypto: boolean;
@@ -125,6 +130,9 @@ export interface EarlyExecutableCacheOptions {
windowsSubsystem?: "gui";
/** Omitted unless --detached-exit is set. */
detachedExit?: true;
/** Omitted without --profile-generate/--profile-use; otherwise the
* pattern, or the digest of the profile's bytes. */
profile?: string;
/** Omitted for the default tsgo checker; with SCRIPTC_CHECKER=tsc-ts, the
* tsc-ts executable's file identity. */
checker?: string;
@@ -197,6 +205,7 @@ const BOOLEAN_NATIVE_KEYS = [
"bigint",
"searchParams",
"qs",
"utilPromisify",
"dynBuiltinClass",
"dynBuiltinStream",
"parseArgs",
@@ -206,12 +215,14 @@ const BOOLEAN_NATIVE_KEYS = [
"http2",
"dgram",
"dnsPromises",
"dgramDyn",
"watch",
"foreignFfi",
"nodeTest",
"tls",
"tlsCa",
"cipher",
"ecdh",
"webcrypto",
"perfLoop",
] as const satisfies readonly (keyof EarlyExecutableNativeFeatures)[];
@@ -246,6 +257,7 @@ function cacheKey(options: EarlyExecutableCacheOptions): string {
...(options.strip ? ["strip"] : []),
...(options.windowsSubsystem === "gui" ? ["windows-subsystem-gui"] : []),
...(options.detachedExit ? ["detached-exit"] : []),
...(options.profile !== undefined ? [options.profile] : []),
...(options.checker !== undefined ? [`checker:${options.checker}`] : []),
...(options.youngNursery ? ["young-nursery"] : []),
options.npmStatic === null
@@ -287,6 +299,7 @@ function routeKey(options: Omit<EarlyExecutableRouteOptions, "nativeEnvironment"
if (options.strip) hash.update("strip\0");
if (options.windowsSubsystem === "gui") hash.update("windows-subsystem-gui\0");
if (options.detachedExit) hash.update("detached-exit\0");
if (options.profile !== undefined) hash.update(`${options.profile}\0`);
if (options.checker !== undefined) hash.update(`checker:${options.checker}\0`);
if (options.youngNursery) hash.update("young-nursery\0");
hash
packages/compiler/src/frontend/arguments-usage.test.ts
+34
View File
@@ -0,0 +1,34 @@
import { afterAll, expect, test } from "vitest";
import { argumentsAliasingUnobservable } from "./arguments-usage.js";
import * as ts from "./ts7/adapter.js";
import { closeSourceParser, parseSourceFile } from "./ts7/source-parser-node.js";
afterAll(closeSourceParser);
function unobservable(source: string): boolean {
const file = parseSourceFile("scan.js", source, "js");
const fn = file.statements.nodes.find(ts.isFunctionDeclaration);
if (fn === undefined) throw new Error("no function declaration");
return argumentsAliasingUnobservable(fn);
}
test("element reads stay unobservable while nothing writes a parameter", () => {
expect(unobservable("function f(a, b) { return arguments[0] + arguments.length; }")).toBe(true);
expect(unobservable("function f(a) { var a; return arguments[0]; }")).toBe(true);
expect(unobservable("function f(a) { var b = 1; return arguments[0] + b; }")).toBe(true);
});
test("a parameter written by assignment or by a declaration makes element reads observable", () => {
for (const body of [
"a = 5;",
"a++;",
"var a = 5;",
"for (var a in o);",
"for (var a of o);",
"var { a } = o;",
"var [a] = o;",
"function a() {}",
]) {
expect(unobservable(`function f(a, o) { ${body} return arguments[0]; }`), body).toBe(false);
}
});
packages/compiler/src/frontend/arguments-usage.ts
+103
View File
@@ -33,3 +33,106 @@ export function bodyReadsArguments(fn: ts.Node): boolean {
});
return found;
}
/** Whether an expression is assigned, updated or deleted. */
function isWritten(expr: ts.Node): boolean {
let node = expr;
while (ts.isParenthesizedExpression(node.parent)) node = node.parent;
const parent = node.parent;
if (ts.isBinaryExpression(parent))
return (
parent.left === node &&
parent.operatorToken.kind >= ts.SyntaxKind.FirstAssignment &&
parent.operatorToken.kind <= ts.SyntaxKind.LastAssignment
);
if (ts.isPrefixUnaryExpression(parent) || ts.isPostfixUnaryExpression(parent))
return (
parent.operator === ts.SyntaxKind.PlusPlusToken ||
parent.operator === ts.SyntaxKind.MinusMinusToken
);
if (ts.isForInStatement(parent) || ts.isForOfStatement(parent))
return parent.initializer === node;
if (
ts.isArrayLiteralExpression(parent) ||
ts.isSpreadElement(parent) ||
ts.isShorthandPropertyAssignment(parent) ||
(ts.isPropertyAssignment(parent) && parent.initializer === node)
)
return isWritten(parent);
if (ts.isObjectLiteralExpression(parent) || ts.isSpreadAssignment(parent))
return isWritten(parent);
return ts.isDeleteExpression(parent);
}
/** Whether an identifier assigns its binding where it declares it: a
* `var` with an initializer or in a for-in/for-of head, a destructured
* element, or a function declaration's name. A redeclared parameter keeps
* its binding, so each of these writes the parameter. */
function isDeclarationWrite(id: ts.Identifier): boolean {
const parent = id.parent;
if (ts.isVariableDeclaration(parent) && parent.name === id) {
if (parent.initializer !== undefined) return true;
const head = parent.parent?.parent;
return (
(ts.isForInStatement(head) || ts.isForOfStatement(head)) && head.initializer === parent.parent
);
}
if (ts.isBindingElement(parent)) return parent.name === id;
return ts.isFunctionDeclaration(parent) && parent.name === id;
}
/** Whether sloppy-mode `arguments` in `fn`'s own body behaves exactly like
* an unmapped copy of the call's arguments. A mapped arguments object
* aliases the named parameters, so it differs from a copy only when one
* side is written after the other is read: reading `arguments.length`
* never observes the aliasing, and element reads cannot observe it while
* nothing writes an element or a parameter. The object itself must not
* escape, since another holder could write through it. */
export function argumentsAliasingUnobservable(fn: ts.SignatureDeclaration): boolean {
if (fn.body === undefined) return false;
let elementReads = false;
let escapes = false;
ts.walkPreorder(fn.body, (node) => {
if (ts.isTypeNode(node)) return "skip";
if (ts.isFunctionLike(node) && !ts.isArrowFunction(node) && node !== fn) return "skip";
if (!ts.isIdentifier(node) || node.text !== "arguments" || !isArgumentsRead(node))
return undefined;
const parent = node.parent;
if (
ts.isPropertyAccessExpression(parent) &&
parent.expression === node &&
parent.name.text === "length" &&
!isWritten(parent)
)
return undefined;
if (ts.isElementAccessExpression(parent) && parent.expression === node && !isWritten(parent)) {
elementReads = true;
return undefined;
}
escapes = true;
return "stop";
});
if (escapes) return false;
if (!elementReads) return true;
const names = new Set<string>();
for (const p of fn.parameters.nodes) {
if (!ts.isIdentifier(p.name)) return false;
names.add(p.name.text);
}
// Any write to a parameter's name, including from nested closures and
// shadowing bindings, conservatively counts as a parameter write.
let paramWritten = false;
ts.walkPreorder(fn.body, (node) => {
if (ts.isTypeNode(node)) return "skip";
if (
ts.isIdentifier(node) &&
names.has(node.text) &&
(isWritten(node) || isDeclarationWrite(node))
) {
paramWritten = true;
return "stop";
}
return undefined;
});
return !paramWritten;
}
packages/compiler/src/frontend/lowering/builtins/builtin-class-dyn.ts
+65 -1
View File
@@ -1,8 +1,18 @@
import { numLit } from "../../../ir/build.js";
import { BOOL, DYN, RUNTIME_EMITTER_CLASS, type IrExpr, type SrcLoc } from "../../../ir/ir.js";
import {
BOOL,
DYN,
RUNTIME_EMITTER_CLASS,
type IrExpr,
type IrStmt,
type SrcLoc,
} from "../../../ir/ir.js";
import * as ts from "../../ts7/adapter.js";
import { locOf } from "../../program.js";
import type { Lowerer } from "../lowerer.js";
import type { ClassInfo } from "../lower-classes.js";
import { classInstanceOf } from "../class-dynamic-dispatch.js";
import { isSafeToDiscard, isSafeToRepeat } from "../expressions/evaluation-safety.js";
/* EventEmitter and the stream classes as untyped constructor values, the
* bases of JavaScript constructor functions:
@@ -157,3 +167,57 @@ export function builtinClassInstanceOf(value: IrExpr, kind: number, loc: SrcLoc)
loc,
};
}
/** `value instanceof Base` for an untyped value that may also hold a
* compiled instance: an object built by a constructor function inherits
* the untyped constructor's prototype, and an instance of a program class
* extending Base boxes as a typed reference the class test recognizes.
* With `callee`, the right-hand side is an untyped value and the class
* test applies only when it is Base's untyped constructor. */
export function builtinClassInstanceOfAny(
lowerer: Lowerer,
value: IrExpr,
kind: number,
info: ClassInfo,
loc: SrcLoc,
callee?: IrExpr,
): IrExpr {
const stmts: IrStmt[] = [];
const once = (expr: IrExpr, name: string, snapshot = false): IrExpr => {
if (!snapshot && isSafeToRepeat(expr)) return expr;
const local = lowerer.declareHiddenLocal(name, DYN);
stmts.push({ kind: "varDecl", localId: local.id, init: expr, loc });
return { kind: "varRef", localId: local.id, type: DYN, loc };
};
// The left operand evaluates first: a variable read keeps its value in
// a local when the right-hand side can assign the variable.
const tested = once(value, "%instanceValue", callee !== undefined && !isSafeToDiscard(callee));
const ctor = callee ? once(callee, "%instanceCallee") : null;
const compiled = classInstanceOf(lowerer, tested, info, loc);
const result: IrExpr = {
kind: "logical",
op: "||",
left: ctor
? { kind: "libCall", fn: "dyn.functionInstanceOf", args: [tested, ctor], type: BOOL, loc }
: builtinClassInstanceOf(tested, kind, loc),
right: ctor
? {
kind: "logical",
op: "&&",
left: {
kind: "dynScalarEq",
left: ctor,
right: builtinClassValue(kind, loc),
type: BOOL,
loc,
},
right: compiled,
type: BOOL,
loc,
}
: compiled,
type: BOOL,
loc,
};
return stmts.length ? { kind: "seqExpr", stmts, result, type: BOOL, loc } : result;
}
packages/compiler/src/frontend/lowering/builtins/callable-aliases.ts
+6 -2
View File
@@ -6,7 +6,7 @@ import { registerHttpClientFnBinding } from "../lower-server.js";
import { isPromisifiedZlibMember } from "./compression.js";
import { isPromisifiedCryptoMember } from "./crypto.js";
import { isPromisifiedDnsMember } from "../lower-dgram.js";
import { callbackPromisifyShape } from "./promisify-callback.js";
import { callbackPromisifyShape, untypedPromisifyArgument } from "./promisify-callback.js";
/** Compile-time util.promisify projections. The immutable result binding
* owns no runtime slot: calls rewrite to the target's promise lowering,
@@ -167,7 +167,11 @@ export function registerPromisifiedBuiltinDecl(
: null;
// A typed callback-last function value: the call lowers to a native
// promisified function, and the binding holds it like any closure.
if (!projection && callbackPromisifyShape(lowerer, e, locOf(e)) !== null) return false;
if (
!projection &&
(callbackPromisifyShape(lowerer, e, locOf(e)) !== null || untypedPromisifyArgument(lowerer, e))
)
return false;
if (!projection) {
lowerer.noLowering(
"util.promisify of this target",
packages/compiler/src/frontend/lowering/builtins/crypto.ts
+3 -1
View File
@@ -1,5 +1,6 @@
import { lowerBuiltinByteInput, errorFirstBytesCallback, errorFirstCallback } from "./arguments.js";
import { isKeyObjectValue } from "./key-object.js";
import { lowerCreateEcdh } from "./ecdh.js";
import { boolLit, dynUndefinedExpr, nodeThrowExpr, varRef } from "../../../ir/build.js";
import * as ts from "../../ts7/adapter.js";
import { type Lowerer, own } from "../lowerer.js";
@@ -408,6 +409,7 @@ export function lowerCryptoModuleCall(
loc,
};
}
if (bi.member === "createECDH") return lowerCreateEcdh(lowerer, expr);
if (bi.member === "createHmac") {
if (args.length < 1 || args.length > 3 || args.some(ts.isSpreadElement)) {
lowerer.noLowering(
@@ -896,7 +898,7 @@ const CIPHER_METHODS = new Set([
/** V8's spelling of a callee in "is not a function" messages: call
* arguments render as "(...)". */
function v8CalleeText(node: ts.Expression): string {
export function v8CalleeText(node: ts.Expression): string {
if (ts.isPropertyAccessExpression(node))
return `${v8CalleeText(node.expression)}.${node.name.text}`;
if (ts.isCallExpression(node)) return `${v8CalleeText(node.expression)}(...)`;
packages/compiler/src/frontend/lowering/builtins/ecdh.ts
+220
View File
@@ -0,0 +1,220 @@
import * as ts from "../../ts7/adapter.js";
import { dynUndefinedExpr, nodeThrowExpr, varRef } from "../../../ir/build.js";
import { type Lowerer } from "../lowerer.js";
import { locOf } from "../../program.js";
import { DYN, type IrExpr, type IrStmt } from "../../../ir/ir.js";
import { builtinImportOf } from "./module-bindings.js";
import { v8CalleeText } from "./crypto.js";
import { CRYPTO_CURVES } from "../crypto-tables.js";
/* node:crypto's ECDH is a native checked-dynamic handle (scr_ecdh.c): the
* ECDH type maps to dyn, its methods dispatch through the handle, and
* these helpers lower createECDH, the static ECDH.convertKey and the
* method calls. */
/** Whether `node` names node:crypto's ECDH class. */
function isEcdhClass(lowerer: Lowerer, node: ts.Expression): boolean {
const builtin = ts.isIdentifier(node)
? builtinImportOf(lowerer, node)
: ts.isPropertyAccessExpression(node) && !node.questionDotToken
? lowerer.builtinMemberOf(node)
: null;
return builtin?.module === "crypto" && builtin.member === "ECDH";
}
/** Whether a checker type is node:crypto's ECDH. */
export function isEcdhType(lowerer: Lowerer, type: ts.Type): boolean {
const symbol = type.getAliasSymbol() ?? type.getSymbol();
return !!symbol && ts.symbolName(symbol) === "ECDH" && lowerer.isStdlibSymbol(symbol);
}
/** The call's arguments as checked-dynamic values in hidden locals, the
* first `count` of them padded with undefined, so every argument is
* evaluated left to right before the runtime checks them. Surplus
* arguments are evaluated and dropped. */
function dynArguments(
lowerer: Lowerer,
call: ts.CallExpression,
count: number,
stmts: IrStmt[],
): IrExpr[] {
const args = call.arguments.nodes;
const loc = locOf(call);
const values: IrExpr[] = [];
for (let i = 0; i < count; i++) {
const input: IrExpr = args[i]
? lowerer.lowerExprExpecting(args[i]!, DYN)
: dynUndefinedExpr(loc);
const saved = lowerer.declareHiddenLocal("%ecdhInput", DYN);
stmts.push({ kind: "varDecl", localId: saved.id, init: input, loc });
values.push(varRef(saved.id, DYN, loc));
}
for (const arg of args.slice(count))
stmts.push({ kind: "exprStmt", expr: lowerer.lowerExpr(arg), loc });
return values;
}
/** The curves scr_ecdh.c implements, by their OpenSSL short names. */
const IMPLEMENTED_CURVES = new Set([
"prime192v1",
"secp224r1",
"prime256v1",
"secp384r1",
"secp521r1",
"secp192k1",
"secp224k1",
"secp256k1",
"brainpoolP256r1",
"brainpoolP384r1",
"brainpoolP512r1",
]);
/** A literal curve name that Node knows (crypto.getCurves()) but the
* runtime does not implement is refused at compile time; computed names
* throw a coded SC2020 error at run time, and unknown names keep Node's
* errors. */
function checkCurveName(lowerer: Lowerer, node: ts.Expression | undefined, use: string): void {
if (node === undefined) return;
const type = lowerer.typeOf(node);
if (!type.isStringLiteralType()) return;
if (IMPLEMENTED_CURVES.has(type.value) || !CRYPTO_CURVES.includes(type.value)) return;
lowerer.noLowering(
`${use} with curve '${type.value}'`,
node,
"the supported curves are prime192v1, secp224r1, prime256v1, secp384r1, secp521r1, secp192k1, " +
"secp224k1, secp256k1, brainpoolP256r1, brainpoolP384r1, and brainpoolP512r1",
);
}
/** crypto.createECDH(curveName): a native ECDH handle. */
export function lowerCreateEcdh(lowerer: Lowerer, call: ts.CallExpression): IrExpr {
if (call.arguments.nodes.some(ts.isSpreadElement))
lowerer.noLowering("crypto.createECDH with spread arguments", call);
checkCurveName(lowerer, call.arguments.nodes[0], "crypto.createECDH");
const loc = locOf(call);
const stmts: IrStmt[] = [];
const values = dynArguments(lowerer, call, 1, stmts);
return {
kind: "seqExpr",
stmts,
result: { kind: "libCall", fn: "crypto.ecdhNew", args: values, type: DYN, loc },
type: DYN,
loc,
};
}
const ECDH_METHODS = new Set([
"generateKeys",
"computeSecret",
"getPublicKey",
"getPrivateKey",
"setPrivateKey",
"setPublicKey",
]);
/** computeSecret(view, encoding): Node reads the second argument as the
* input encoding, which a view key ignores, so the call answers a Buffer
* where @types/node declares the string of an output encoding. The
* form is refused at compile time rather than failing the string check
* at run time. */
function refuseViewKeyWithEncoding(lowerer: Lowerer, call: ts.CallExpression, name: string): void {
if (name !== "computeSecret") return;
const args = call.arguments.nodes;
if (args.length !== 2) return;
const encoding = lowerer.typeOf(args[1]!);
if ((encoding.flags & (ts.TypeFlags.Undefined | ts.TypeFlags.Null)) !== 0) return;
if ((lowerer.typeOf(args[0]!).flags & ts.TypeFlags.StringLike) !== 0) return;
lowerer.noLowering(
"ECDH.computeSecret(key, encoding) with a Buffer or view key",
call,
"Node reads the second argument as the key's input encoding and answers a Buffer; " +
"pass the output encoding third: computeSecret(key, undefined, encoding)",
);
}
/** A call whose arguments the checker's chosen @types/node overload does
* not accept: the checker then types the result by an overload Node may
* not follow (getPublicKey("HEX") answers a string where the encoding-less
* overload declares a Buffer), so the call is refused rather than checked
* against that type at run time. Arguments typed any are accepted. */
function refuseUnmatchedOverload(lowerer: Lowerer, call: ts.CallExpression, what: string): void {
const signature = lowerer.checker.getResolvedSignature(call);
if (signature === undefined) return;
const params = signature.getParameters();
const args = call.arguments.nodes;
for (let i = 0; i < args.length && i < params.length; i++) {
const argType = lowerer.typeOf(args[i]!);
if ((argType.flags & ts.TypeFlags.Any) !== 0) continue;
const paramType = lowerer.checker.getTypeOfSymbolAtLocation(params[i]!, call);
if (lowerer.checker.isTypeAssignableTo(argType, paramType)) continue;
lowerer.noLowering(
`${what} with an argument its declared overloads do not accept`,
args[i]!,
`pass ${lowerer.checker.typeToString(paramType)} here, as @types/node declares`,
);
}
}
/** `ECDH.convertKey(...)` and calls on native ECDH handles. Methods
* dispatch through the handle with checked-dynamic arguments. */
export function lowerEcdhCall(lowerer: Lowerer, call: ts.CallExpression): IrExpr | null {
if (call.questionDotToken || !ts.isPropertyAccessExpression(call.expression)) return null;
const access = call.expression;
if (access.questionDotToken) return null;
const name = access.name.text;
if (name === "convertKey" && isEcdhClass(lowerer, access.expression)) {
if (call.arguments.nodes.some(ts.isSpreadElement))
lowerer.noLowering("ECDH.convertKey with spread arguments", call);
refuseUnmatchedOverload(lowerer, call, "ECDH.convertKey");
checkCurveName(lowerer, call.arguments.nodes[1], "ECDH.convertKey");
const loc = locOf(call);
const stmts: IrStmt[] = [];
const values = dynArguments(lowerer, call, 5, stmts);
return {
kind: "seqExpr",
stmts,
result: { kind: "libCall", fn: "crypto.ecdhConvertKey", args: values, type: DYN, loc },
type: DYN,
loc,
};
}
if (!isEcdhType(lowerer, lowerer.typeOf(access.expression))) return null;
const loc = locOf(call);
if (!ECDH_METHODS.has(name)) {
// Node reads the member as undefined and throws a TypeError once the
// receiver and arguments are evaluated.
const evaluated = [access.expression, ...call.arguments.nodes].map((node): IrStmt => ({
kind: "exprStmt",
expr: lowerer.lowerExpr(node),
loc,
}));
return {
kind: "seqExpr",
stmts: evaluated,
result: nodeThrowExpr(1, "", `${v8CalleeText(access)} is not a function`, DYN, loc),
type: DYN,
loc,
};
}
if (call.arguments.nodes.some(ts.isSpreadElement))
lowerer.noLowering(`ECDH.${name} with spread arguments`, call);
refuseViewKeyWithEncoding(lowerer, call, name);
// setPublicKey has no @types/node declaration.
if (name !== "setPublicKey") refuseUnmatchedOverload(lowerer, call, `ECDH.${name}`);
const invoke: IrExpr = {
kind: "dynInvoke",
recv: lowerer.lowerExprExpecting(access.expression, DYN),
method: name,
calleeName: v8CalleeText(access),
args: call.arguments.nodes.map((arg) => lowerer.lowerExprExpecting(arg, DYN)),
type: DYN,
loc,
};
// The arguments match a declared overload, so a Buffer or string result
// takes its static representation.
if (name === "setPrivateKey" || name === "setPublicKey") return invoke;
const result = lowerer.irTypeOf(call);
return result.kind === "bytes" || result.kind === "string"
? lowerer.coerceInto(call, invoke, result)
: invoke;
}
packages/compiler/src/frontend/lowering/builtins/promisify-callback.ts
+20 -1
View File
@@ -3,6 +3,7 @@ import { type Lowerer } from "../lowerer.js";
import { varRef } from "../../../ir/build.js";
import {
BOOL,
DYN,
type IrExpr,
type IrLocal,
type IrParam,
@@ -90,6 +91,14 @@ function callbackShape(
return { fnT, resultT, args, callbackT, callbackParamT: callbackParamT!, inner, error };
}
/** Whether `promisify(fn)`'s one argument is typed `any`: the function is
* only known at run time, so the promisified function is built there. */
export function untypedPromisifyArgument(lowerer: Lowerer, call: ts.CallExpression): boolean {
if (call.questionDotToken || call.arguments.nodes.length !== 1) return false;
const argNode = call.arguments.nodes[0]!;
return !ts.isSpreadElement(argNode) && (lowerer.typeOf(argNode).flags & ts.TypeFlags.Any) !== 0;
}
/** The callback shape of a `promisify(fn)` call, or null. */
export function callbackPromisifyShape(
lowerer: Lowerer,
@@ -131,7 +140,17 @@ export function lowerCallbackPromisify(
loc: SrcLoc,
): IrExpr | null {
const shape = callbackPromisifyShape(lowerer, call, loc);
if (shape === null) return null;
if (shape === null) {
// An untyped (`any`) function: Node's promisify at run time.
if (!untypedPromisifyArgument(lowerer, call)) return null;
return {
kind: "libCall",
fn: "util.promisifyDyn",
args: [lowerer.lowerExprExpecting(call.arguments.nodes[0]!, DYN)],
type: DYN,
loc,
};
}
const argNode = call.arguments.nodes[0]!;
let fn = lowerer.lowerExpr(argNode);
if (!typeEquals(fn.type, shape.fnT)) fn = lowerer.coerceToExpected(fn, shape.fnT);
packages/compiler/src/frontend/lowering/builtins/reflection.ts
+27
View File
@@ -36,6 +36,33 @@ export function lowerReflectCall(
}
if (call.questionDotToken) return null;
const reflectMember = lowerer.stdlibGlobalMember(access, "Reflect");
// Reflect.getOwnPropertyDescriptor and Reflect.defineProperty over
// checked objects (execa's Promise mixin): the Object operations, with
// defineProperty answering true. A non-object target throws the Object
// operation's TypeError.
if (
(reflectMember === "getOwnPropertyDescriptor" && call.arguments.nodes.length === 2) ||
(reflectMember === "defineProperty" && call.arguments.nodes.length === 3)
) {
if (call.arguments.nodes.some(ts.isSpreadElement)) return null;
const loc = locOf(call);
const args = call.arguments.nodes.map((arg) => lowerer.lowerExprExpecting(arg, DYN));
if (reflectMember === "getOwnPropertyDescriptor")
return { kind: "libCall", fn: "dyn.getOwnPropertyDescriptor", args, type: DYN, loc };
return {
kind: "seqExpr",
stmts: [
{
kind: "exprStmt",
expr: { kind: "libCall", fn: "dyn.defineProperty", args, type: DYN, loc },
loc,
},
],
result: { kind: "boolLit", value: true, type: BOOL, loc },
type: BOOL,
loc,
};
}
if (reflectMember === "get" || reflectMember === "set") {
const required = reflectMember === "get" ? 2 : 3;
if (
packages/compiler/src/frontend/lowering/builtins/typed-array-members.ts
+163
View File
@@ -0,0 +1,163 @@
/* The members Node 24 gives typed arrays and Buffers: every string-keyed
* property name on their prototype chains (Object.getOwnPropertyNames of
* each prototype), Object.prototype included. A typed array or Buffer in a
* compiled program is always one of these built-in kinds (programs cannot
* subclass them, add own properties to them or change their prototypes), so
* a name outside these lists reads as undefined there, as in Node. */
/** %TypedArray%.prototype and Object.prototype. */
const TYPED_ARRAY_CHAIN = new Set<string>([
"__defineGetter__",
"__defineSetter__",
"__lookupGetter__",
"__lookupSetter__",
"__proto__",
"at",
"buffer",
"byteLength",
"byteOffset",
"constructor",
"copyWithin",
"entries",
"every",
"fill",
"filter",
"find",
"findIndex",
"findLast",
"findLastIndex",
"forEach",
"hasOwnProperty",
"includes",
"indexOf",
"isPrototypeOf",
"join",
"keys",
"lastIndexOf",
"length",
"map",
"propertyIsEnumerable",
"reduce",
"reduceRight",
"reverse",
"set",
"slice",
"some",
"sort",
"subarray",
"toLocaleString",
"toReversed",
"toSorted",
"toString",
"valueOf",
"values",
"with",
]);
/** Each element kind's own prototype adds only BYTES_PER_ELEMENT. */
const ELEMENT_KIND_MEMBER = "BYTES_PER_ELEMENT";
/** Buffer.prototype, beyond Uint8Array.prototype's chain. */
const BUFFER_PROTOTYPE = new Set<string>([
"asciiSlice",
"asciiWrite",
"base64Slice",
"base64Write",
"base64urlSlice",
"base64urlWrite",
"compare",
"copy",
"equals",
"hexSlice",
"hexWrite",
"inspect",
"latin1Slice",
"latin1Write",
"offset",
"parent",
"readBigInt64BE",
"readBigInt64LE",
"readBigUInt64BE",
"readBigUInt64LE",
"readBigUint64BE",
"readBigUint64LE",
"readDoubleBE",
"readDoubleLE",
"readFloatBE",
"readFloatLE",
"readInt16BE",
"readInt16LE",
"readInt32BE",
"readInt32LE",
"readInt8",
"readIntBE",
"readIntLE",
"readUInt16BE",
"readUInt16LE",
"readUInt32BE",
"readUInt32LE",
"readUInt8",
"readUIntBE",
"readUIntLE",
"readUint16BE",
"readUint16LE",
"readUint32BE",
"readUint32LE",
"readUint8",
"readUintBE",
"readUintLE",
"swap16",
"swap32",
"swap64",
"toJSON",
"ucs2Slice",
"ucs2Write",
"utf8Slice",
"utf8Write",
"write",
"writeBigInt64BE",
"writeBigInt64LE",
"writeBigUInt64BE",
"writeBigUInt64LE",
"writeBigUint64BE",
"writeBigUint64LE",
"writeDoubleBE",
"writeDoubleLE",
"writeFloatBE",
"writeFloatLE",
"writeInt16BE",
"writeInt16LE",
"writeInt32BE",
"writeInt32LE",
"writeInt8",
"writeIntBE",
"writeIntLE",
"writeUInt16BE",
"writeUInt16LE",
"writeUInt32BE",
"writeUInt32LE",
"writeUInt8",
"writeUIntBE",
"writeUIntLE",
"writeUint16BE",
"writeUint16LE",
"writeUint32BE",
"writeUint32LE",
"writeUint8",
"writeUintBE",
"writeUintLE",
]);
/** Whether every value a typed array of this element kind can hold has the
* member `name` (true), none has it (false), or it depends on the value
* (null): a Uint8Array may be a Buffer, whose prototype adds members. */
export function typedArrayHasMember(name: string, mayBeBuffer: boolean): boolean | null {
if (name === ELEMENT_KIND_MEMBER || TYPED_ARRAY_CHAIN.has(name)) return true;
return mayBeBuffer && BUFFER_PROTOTYPE.has(name) ? null : false;
}
/** Whether `name` is absent from the prototype chain of every value a
* typed array of this element kind can hold. */
export function typedArrayLacksMember(name: string, mayBeBuffer: boolean): boolean {
return typedArrayHasMember(name, mayBeBuffer) === false;
}
packages/compiler/src/frontend/lowering/call-signatures.ts
+119 -7
View File
@@ -5,9 +5,10 @@ import {
} from "./typed-arguments.js";
import { InternalCompilerError } from "../../errors.js";
import * as ts from "../ts7/adapter.js";
import { bodyReadsArguments } from "../arguments-usage.js";
import { argumentsAliasingUnobservable, bodyReadsArguments } from "../arguments-usage.js";
import type { Lowerer } from "./lowerer.js";
import {
canBoxFuncIntoDyn,
DYN,
type IrExpr,
type IrFunction,
@@ -143,6 +144,26 @@ export function generatorMeta(
return { yieldT: type.yieldT, nextT: type.nextT, resultType };
}
/** A function expression stored as a prototype method: `C.prototype.name =
* function (...) {}` or `C.prototype[key] = function (...) {}`
* (parentheses allowed around the function). */
export function prototypeAssignedFunction(fn: ts.Node | undefined): boolean {
if (fn === undefined || !ts.isFunctionExpression(fn)) return false;
let node: ts.Node = fn;
while (node.parent !== undefined && ts.isParenthesizedExpression(node.parent)) node = node.parent;
const parent = node.parent;
if (
parent === undefined ||
!ts.isBinaryExpression(parent) ||
parent.right !== node ||
parent.operatorToken.kind !== ts.SyntaxKind.EqualsToken ||
!(ts.isPropertyAccessExpression(parent.left) || ts.isElementAccessExpression(parent.left))
)
return false;
const owner = parent.left.expression;
return ts.isPropertyAccessExpression(owner) && owner.name.text === "prototype";
}
/** A parameter with a type annotation or a JSDoc `@param`/`@type` tag. */
export function documentedParameter(param: ts.ParameterDeclaration): boolean {
return (
@@ -199,6 +220,20 @@ export function paramShape(lowerer: Lowerer, param: ts.ParameterDeclaration): Pa
? { type: DYN, mode: "omittable", bodyType: DYN }
: { type: DYN, mode: param.questionToken ? "omittable" : "required" };
}
if (
isJsSourceFile(ts.getSourceFileOfNode(param)) &&
!param.dotDotDotToken &&
prototypeAssignedFunction(param.parent) &&
!checkedJsDocumentedParameter(lowerer, param)
) {
// A prototype method (`C.prototype.m = function (a) {}`) is called
// through dynamic dispatch, where callers pass whatever they hold. As
// for constructors, JSDoc in a file TypeScript does not check
// documents the usual types without constraining callers.
return param.initializer
? { type: DYN, mode: "omittable", bodyType: DYN }
: { type: DYN, mode: param.questionToken ? "omittable" : "required" };
}
const executor = param.parent;
const construction = executor?.parent;
const nativeExecutor =
@@ -640,6 +675,21 @@ export function requireExactArityValue(
? lowerer.mapTypeOf(lowerer.typeOf(contextual))
: null;
if (mapped && typeEquals(mapped, funcType)) return;
// An 'any' slot (`C.prototype.m = function (a, b) {…}` through an
// untyped receiver) boxes the value into a checked-dynamic function:
// the box completes omitted arguments with undefined and checks every
// argument into the completed ABI, so any boxable signature passes.
if (
target !== undefined &&
(target.flags & ts.TypeFlags.Any) !== 0 &&
canBoxFuncIntoDyn(
funcType,
(id) => lowerer.shapes.get(id),
(id) => lowerer.unions.get(id),
)
) {
return;
}
// An object-literal METHOD has no contextual type of its own: the
// literal's contextual type names the member slot it fills. An
// overloaded member's slot holds its erased function, which spells the
@@ -808,7 +858,19 @@ export function collectSignatureInner(lowerer: Lowerer, decl: ts.FunctionDeclara
decl.parameters.nodes.length > 0 ? "arguments" : "dynRest",
);
const nameBlame: ts.Node = decl.name ?? decl;
const returnType = lowerer.declaredReturnType(decl, nameBlame);
let returnType = lowerer.declaredReturnType(decl, nameBlame);
// A JavaScript generator whose inferred return the checker leaves `any`
// (it delegates to itself: `yield* walkTree(sub)`) is still a generator:
// its channels are checked-dynamic values.
if (isGenerator && returnType.kind === "dyn" && isJsSourceFile(ts.getSourceFileOfNode(decl))) {
returnType = {
kind: "generator",
...(isAsync ? { async: true as const } : {}),
yieldT: DYN,
retT: DYN,
nextT: DYN,
};
}
if (isAsync && !isGenerator && returnType.kind !== "promise") {
lowerer.badType(nameBlame, lowerer.typeOf(nameBlame));
}
@@ -839,7 +901,8 @@ export function collectSignatureInner(lowerer: Lowerer, decl: ts.FunctionDeclara
/** Arrows inherit arguments. Ordinary JS functions receive a hidden
* array unless a declared rest parameter already owns that contract.
* Class methods are strict; other parameterized forms require ES modules
* because sloppy-mode arguments alias named parameters. */
* because sloppy-mode arguments alias named parameters, unless the body
* cannot observe that aliasing (argumentsAliasingUnobservable). */
function appendArgumentsParameter(
lowerer: Lowerer,
node: ts.SignatureDeclaration,
@@ -864,7 +927,8 @@ function appendArgumentsParameter(
if (
!classMethod &&
!isNodeEsmFile(ts.getSourceFileOfNode(node), lowerer.program) &&
!unmappedArguments(node)
!unmappedArguments(node) &&
!argumentsAliasingUnobservable(node)
) {
lowerer.unsupported(
"SC1090",
@@ -1058,9 +1122,21 @@ export function lambdaSignature(
// Union-inferred returns adopt too (a mixed-return body inferring a
// SUB-union of the slot's union — adopting is a no-op when the two
// A synchronous object-literal method adopts its slot's return the same
// way: its slot is the property of that name on the literal's
// contextual type. A literal under `satisfies` keeps its own type (the
// constraint is only checked), so its methods keep their inferred
// returns: callers of the value see those, not the constraint's.
const literalMethod =
ts.isMethodDeclaration(node) &&
ts.isObjectLiteralExpression(node.parent) &&
!node.asteriskToken &&
node.modifiers?.nodes.some((m) => m.kind === ts.SyntaxKind.AsyncKeyword) !== true &&
!isSatisfiesOperand(node.parent);
if (
(ts.isArrowFunction(node) || ts.isFunctionExpression(node)) &&
(!isAsyncLike || ret.kind === "promise")
((ts.isArrowFunction(node) || ts.isFunctionExpression(node)) &&
(!isAsyncLike || ret.kind === "promise")) ||
literalMethod
) {
// The slot's signature: the contextual type stripped of its nullish
// parts (an OPTIONAL callback field's context is the whole
@@ -1068,7 +1144,10 @@ export function lambdaSignature(
// declared in USER code: stdlib callback slots (flatMap's
// `U | readonly U[]`, sort comparators) have intrinsic lowerings
// that inspect the INFERRED type, so they must not widen.
const ctxType = lowerer.checker.getContextualType(node);
const ctxType =
literalMethod && ts.isMethodDeclaration(node) && node.parent
? objectLiteralMethodSlot(lowerer, node, node.parent)
: lowerer.checker.getContextualType(node);
const ctxSigs = ctxType
? lowerer.checker.getCallSignatures(lowerer.checker.getNonNullableType(ctxType))
: [];
@@ -1136,6 +1215,7 @@ export function lambdaSignature(
const inferredLiteral =
literalSlot !== undefined && (literalType === null || literalType.kind === "dyn");
if (
!literalMethod &&
!isAsyncLike &&
!node.type &&
!inferredLiteral &&
@@ -1151,3 +1231,35 @@ export function lambdaSignature(
const funcType = funcTypeFromParamShapes(shapes, ret);
return { shapes, funcType };
}
/** Whether the object literal `literal` is the operand of a `satisfies`
* expression, directly or as a property value of literals that are. */
function isSatisfiesOperand(literal: ts.Node): boolean {
let n: ts.Node | undefined = literal;
while (n !== undefined) {
const parent: ts.Node | undefined = n.parent;
if (parent === undefined) return false;
if (ts.isParenthesizedExpression(parent)) n = parent;
else if (ts.isPropertyAssignment(parent) && parent.initializer === n) n = parent.parent;
else return ts.isSatisfiesExpression(parent) && parent.expression === n;
}
return false;
}
/** The type of the property an object-literal method fills on the
* literal's contextual type; undefined when the literal has no contextual
* type or the method has a computed name. */
function objectLiteralMethodSlot(
lowerer: Lowerer,
node: ts.MethodDeclaration,
literal: ts.Node,
): ts.Type | undefined {
if (!ts.isIdentifier(node.name) && !ts.isStringLiteral(node.name)) return undefined;
const literalType = lowerer.checker.getContextualType(literal);
if (!literalType) return undefined;
const property = lowerer.checker.getPropertyOfType(
lowerer.checker.getNonNullableType(literalType),
node.name.text,
);
return property ? lowerer.checker.getTypeOfSymbol(property) : undefined;
}
packages/compiler/src/frontend/lowering/checked-fallback.ts
-861
View File
@@ -1,861 +0,0 @@
/* Checked-cast fallback: a value that fails a checked cast keeps running.
*
* `JSON.parse(text) as T` (and `const x: T = anyValue`, `await res.json()
* as T`) is a checked conversion: the static code after it reads `T`'s
* native layout, so a value that does not fit `T` used to throw a
* TypeError at the cast. Node never checks: the program continues with the
* value it has. This module makes the failing case continue too, without
* changing the code that runs when the value fits.
*
* The unit is a FALLBACK SITE: a function-local binding whose only write
* is the checked cast (the ROOT), declared with `const`/`let` in a function
* body. The code that can observe the root is the statements after the
* cast in each enclosing statement list up to the root's declaration (the
* TAILS; every other reference disqualifies the site). Each tail is lowered
* twice:
*
* - the static lowering, exactly as before, used while the cast held;
* - the dynamic lowering, used after a failure: the root reads its checked
* dynamic value (a shadow local), and every expression derived from it is
* typed `any` (fallbackTainted), so the region lowers through scriptc's
* ordinary `any` path, which has Node's semantics: missing members read
* undefined, values keep their real types, extra members stay visible,
* and objects keep their identity.
*
* `if (flag) dynamic else static` selects the lowering after each tail's
* first statement; the cast sets the flag. A derived value that flows into
* a typed slot outside the region (a typed parameter, a field, an outer
* binding) passes the same checked conversion as any `any` value does
* there: it continues when it fits and throws that conversion's TypeError
* at that use when it does not. A statement of the dynamic lowering that
* has no `any` lowering compiles to a runtime fence that throws at that
* statement only when a failed value reaches it.
*
* Code without a fallback site is unchanged; code with one runs its static
* tail unchanged behind one flag test per tail, and the cast reads one
* thread-local word after the check (checkedCast.fallbackTaken).
* `--tier=static-only` (and library mode) opens no sites: casts throw. */
import * as ts from "../ts7/adapter.js";
import {
BOOL,
DYN,
type IrExpr,
type IrLocal,
type IrStmt,
type IrType,
type SrcLoc,
} from "../../ir/ir.js";
import { isJsSourceFile, locOf } from "../program.js";
import { PoisonError, type Lowerer } from "./lowerer.js";
export interface FallbackSite {
root: ts.Symbol;
/** The root's declared name (a module-level root is a module binding). */
name: ts.Identifier;
/** The statement holding the checked cast (the declaration itself, or
* the `root = value as T` expression statement). */
boundaryStmt: ts.Statement;
/** Statements after which a tail starts, innermost first; removed as
* their lists split. */
anchors: Set<ts.Statement>;
flag: IrLocal;
shadow: IrLocal;
armed: boolean;
/** An initializer-less root's flag and shadow declarations, and the
* statement list they were emitted into. */
decls: { stmts: IrStmt[]; out: IrStmt[] } | null;
}
export interface FallbackTaint {
symbols: Set<ts.Symbol>;
memo: Map<ts.Node, boolean>;
}
/** Tails larger than this keep the throwing cast: every node is lowered
* twice. */
const MAX_TAIL_NODES = 20000;
/** A site arms only while fewer than this many splits enclose its tail
* or share its split point: each one lowers the tail again, so a tail is
* lowered at most 2^n times. */
const MAX_NESTED_SPLITS = 2;
type StatementList = { statements: { nodes: readonly ts.Statement[] } };
function isStatementList(node: ts.Node | undefined): boolean {
return (
node !== undefined && (ts.isBlock(node) || ts.isModuleBlock(node) || ts.isSourceFile(node))
);
}
function listOf(node: ts.Node): readonly ts.Statement[] {
return (node as unknown as StatementList).statements.nodes;
}
function strip(e: ts.Expression): ts.Expression {
let v = e;
while (ts.isParenthesizedExpression(v)) v = v.expression;
return v;
}
function enclosingFunction(node: ts.Node): ts.Node | undefined {
let n: ts.Node | undefined = node.parent;
while (n !== undefined && !ts.isFunctionLike(n) && !ts.isSourceFile(n)) n = n.parent;
return n;
}
/** `value as T` / `<T>value` over an `any` or `unknown` value, or a
* dynamic value flowing into an annotated declaration. */
function isCheckedCast(lowerer: Lowerer, e: ts.Expression, declared: ts.Type | null): boolean {
const dynamicFlags = ts.TypeFlags.Any | ts.TypeFlags.Unknown;
const dynamic = (x: ts.Expression): boolean =>
(lowerer.checker.getTypeAtLocation(x).flags & dynamicFlags) !== 0;
const v = strip(e);
if (ts.isAsExpression(v) || ts.isTypeAssertion(v)) {
const target = lowerer.checker.getTypeFromTypeNode(v.type);
if (target.flags & dynamicFlags) return false;
return dynamic(strip(v.expression));
}
return declared !== null && (declared.flags & dynamicFlags) === 0 && dynamic(v);
}
function writesSymbol(lowerer: Lowerer, n: ts.Node, sym: ts.Symbol): ts.Node | null {
const names = (e: ts.Node): boolean =>
ts.isIdentifier(e) && lowerer.resolveValueSymbol(e) === sym;
const targetNames = (t: ts.Expression): boolean => {
const e = strip(t);
if (names(e)) return true;
if (ts.isArrayLiteralExpression(e) || ts.isObjectLiteralExpression(e)) {
let found = false;
const walk = (x: ts.Node): void => {
if (names(x)) found = true;
else ts.forEachChild(x, walk);
};
walk(e);
return found;
}
return false;
};
if (ts.isBinaryExpression(n)) {
const k = n.operatorToken.kind;
if (
k >= ts.SyntaxKind.FirstAssignment &&
k <= ts.SyntaxKind.LastAssignment &&
targetNames(n.left)
)
return n;
} else if (
(ts.isPrefixUnaryExpression(n) || ts.isPostfixUnaryExpression(n)) &&
(n.operator === ts.SyntaxKind.PlusPlusToken || n.operator === ts.SyntaxKind.MinusMinusToken) &&
targetNames(n.operand as ts.Expression)
) {
return n;
} else if (
(ts.isForOfStatement(n) || ts.isForInStatement(n)) &&
!ts.isVariableDeclarationList(n.initializer) &&
targetNames(n.initializer as ts.Expression)
) {
return n;
}
return null;
}
/** Hoisted or registered-once declarations, and `super()` calls (which
* initialize the instance once), that a second lowering would duplicate. */
function tailIsRelowerable(stmts: readonly ts.Statement[]): boolean {
let ok = true;
let count = 0;
const visit = (n: ts.Node): void => {
if (!ok) return;
if (++count > MAX_TAIL_NODES) {
ok = false;
return;
}
if (
ts.isFunctionDeclaration(n) ||
ts.isClassDeclaration(n) ||
ts.isClassExpression(n) ||
ts.isModuleDeclaration(n) ||
ts.isEnumDeclaration(n) ||
(ts.isCallExpression(n) && n.expression.kind === ts.SyntaxKind.SuperKeyword) ||
(ts.isVariableDeclarationList(n) &&
(n.flags & (ts.NodeFlags.Let | ts.NodeFlags.Const | ts.NodeFlags.Using)) === 0)
) {
ok = false;
return;
}
ts.forEachChild(n, visit);
};
for (const s of stmts) visit(s);
return ok;
}
interface Candidate {
root: ts.Symbol;
name: ts.Identifier;
boundaryStmt: ts.Statement;
anchors: ts.Statement[];
}
/** The fallback site a declaration statement starts, or null. */
function fallbackCandidate(lowerer: Lowerer, stmt: ts.VariableStatement): Candidate | null {
const list = stmt.declarationList;
if ((list.flags & (ts.NodeFlags.Let | ts.NodeFlags.Const)) === 0) return null;
if ((list.flags & ts.NodeFlags.Using) !== 0) return null;
if (list.declarations.nodes.length !== 1) return null;
if (ts.canHaveModifiers(stmt) && (ts.getModifiers(stmt)?.length ?? 0) > 0) return null;
const decl = list.declarations.nodes[0]!;
if (!ts.isIdentifier(decl.name)) return null;
if (isJsSourceFile(ts.getSourceFileOfNode(stmt))) return null;
// Cheap gates before any scan: a checked-cast initializer, or a typed
// declaration without one.
const declared = decl.type !== undefined ? lowerer.checker.getTypeFromTypeNode(decl.type) : null;
if (
decl.initializer !== undefined
? !isCheckedCast(lowerer, decl.initializer, declared)
: declared === null
)
return null;
const rootList = stmt.parent;
// Module-level bindings are module state (and module statements lower
// one at a time): roots live in blocks.
if (rootList === undefined || !ts.isBlock(rootList)) return null;
const fn = enclosingFunction(stmt);
if (fn === undefined) return null;
const root = lowerer.checker.getSymbolAtLocation(decl.name);
if (root === undefined) return null;
// A root in a module-level block is scanned across its file.
const body = ts.isSourceFile(fn) ? fn : (fn as ts.FunctionLikeDeclaration).body;
if (body === undefined) return null;
// The only write: the initializer, or one `root = value as T` statement.
let boundaryStmt: ts.Statement | null = null;
let boundaryWrite: ts.Node | null = null;
const writes: ts.Node[] = [];
const findWrites = (n: ts.Node): void => {
const w = writesSymbol(lowerer, n, root);
if (w) writes.push(w);
ts.forEachChild(n, findWrites);
};
findWrites(body);
if (decl.initializer !== undefined) {
if (writes.length > 0) return null;
boundaryStmt = stmt;
} else {
if (writes.length !== 1) return null;
const w = writes[0]!;
if (
!ts.isBinaryExpression(w) ||
w.operatorToken.kind !== ts.SyntaxKind.EqualsToken ||
!ts.isIdentifier(strip(w.left)) ||
!ts.isExpressionStatement(w.parent) ||
!isCheckedCast(lowerer, w.right, declared)
)
return null;
if (enclosingFunction(w) !== fn) return null;
boundaryStmt = w.parent;
boundaryWrite = w;
}
// The split points: the statement holding the cast in each enclosing
// statement list, up to the root's list.
const anchors: ts.Statement[] = [];
let cur: ts.Node = boundaryStmt;
for (;;) {
const parent: ts.Node | undefined = cur.parent;
if (parent === undefined || ts.isFunctionLike(parent)) return null;
if (ts.isCaseClause(parent) || ts.isDefaultClause(parent)) return null;
if (isStatementList(parent)) {
anchors.push(cur as ts.Statement);
if (parent === rootList) break;
if (ts.isSourceFile(parent)) return null;
}
cur = parent;
}
const anchorIndex = new Map<ts.Node, { list: ts.Node; index: number }>();
for (const a of anchors)
anchorIndex.set(a, { list: a.parent!, index: listOf(a.parent!).indexOf(a) });
const tails: ts.Statement[] = [];
for (const a of anchors) {
const { list, index } = anchorIndex.get(a)!;
tails.push(...listOf(list).slice(index + 1));
}
if (!tailIsRelowerable(tails)) return null;
// Every other reference sits in a tail.
const inTail = (ref: ts.Node): boolean => {
let n: ts.Node = ref;
while (n.parent !== undefined && n !== body) {
const parent = n.parent;
if (isStatementList(parent)) {
for (const a of anchors) {
if (a.parent !== parent) continue;
const at = anchorIndex.get(a)!.index;
const mine = listOf(parent).indexOf(n as ts.Statement);
if (mine > at) return true;
if (mine < at) return false;
}
}
n = parent;
}
return false;
};
let referencesOk = true;
const scanRefs = (n: ts.Node): void => {
if (!referencesOk) return;
if (ts.isIdentifier(n) && n !== decl.name && lowerer.resolveValueSymbol(n) === root) {
const isBoundaryTarget =
boundaryWrite !== null &&
ts.isBinaryExpression(boundaryWrite) &&
strip(boundaryWrite.left) === n;
if (!isBoundaryTarget && !inTail(n)) referencesOk = false;
return;
}
ts.forEachChild(n, scanRefs);
};
scanRefs(body);
if (!referencesOk) return null;
return { root, name: decl.name, boundaryStmt, anchors };
}
function declareSiteLocals(
lowerer: Lowerer,
candidate: Candidate,
loc: SrcLoc,
): { site: FallbackSite; decls: IrStmt[] } {
const name = ts.symbolName(candidate.root);
const flag = lowerer.declareHiddenLocal(`%fb.${name}`, BOOL);
flag.mutable = true;
const shadow = lowerer.declareHiddenLocal(`%fbv.${name}`, DYN);
shadow.mutable = true;
const decls: IrStmt[] = [
{
kind: "varDecl",
localId: flag.id,
init: { kind: "boolLit", value: false, type: BOOL, loc },
loc,
},
{ kind: "varDecl", localId: shadow.id, init: null, loc },
];
const site: FallbackSite = {
root: candidate.root,
name: candidate.name,
boundaryStmt: candidate.boundaryStmt,
anchors: new Set(candidate.anchors),
flag,
shadow,
armed: false,
decls: null,
};
return { site, decls };
}
/** Before an initializer-less typed declaration lowers: open the fallback
* site its later `root = value as T` statement starts, declaring the flag
* (false) and the shadow that holds a failed value where the root is
* declared. Declarations with an initializer open when they arm. */
export function openFallbackSite(lowerer: Lowerer, stmt: ts.Statement, out: IrStmt[]): void {
if (!ts.isVariableStatement(stmt) || lowerer.fnStack.length === 0) return;
if (lowerer.tierMode === "static-only") return;
const decls = stmt.declarationList.declarations.nodes;
if (decls.length !== 1 || decls[0]!.initializer !== undefined || decls[0]!.type === undefined)
return;
const candidate = fallbackCandidate(lowerer, stmt);
if (candidate === null) return;
const opened = declareSiteLocals(lowerer, candidate, locOf(stmt));
out.push(...opened.decls);
opened.site.decls = { stmts: opened.decls, out };
lowerer.fallbackSites.set(candidate.root, opened.site);
}
/** A site that will not arm: drop its declarations again, so the program
* lowers exactly as it did without the site. */
function closeUnarmed(lowerer: Lowerer, site: FallbackSite): void {
const decls = site.decls;
if (decls !== null) {
for (const d of decls.stmts) {
const at = decls.out.indexOf(d);
if (at >= 0) decls.out.splice(at, 1);
}
}
const locals = lowerer.ctx.locals;
for (const local of [site.flag, site.shadow]) {
const at = locals.indexOf(local);
if (at >= 0) locals.splice(at, 1);
}
}
const FALLBACK_TARGETS = new Set(["record", "array", "f64", "string", "bool"]);
/** A plain check of a record, array or scalar target that holds no class
* instance: casts to program classes keep their checked semantics (a
* documented scriptc boundary: a class cast checks the instance's class). */
function fallbackCheck(lowerer: Lowerer, check: IrExpr & { kind: "dynCheck" }): boolean {
if (check.preserveRefs || check.exact || !FALLBACK_TARGETS.has(check.type.kind)) return false;
return !holdsClassInstance(lowerer, check.type, new Set());
}
function holdsClassInstance(lowerer: Lowerer, t: IrType, seen: Set<string>): boolean {
switch (t.kind) {
case "object":
case "classval":
return true;
case "array":
return holdsClassInstance(lowerer, t.elem, seen);
case "record": {
if (seen.has(t.shapeId)) return false;
seen.add(t.shapeId);
const shape = lowerer.shapes.get(t.shapeId);
if (shape === undefined) return true;
if (shape.indexValue !== undefined && holdsClassInstance(lowerer, shape.indexValue, seen))
return true;
return shape.fields.some((f) => holdsClassInstance(lowerer, f.type, seen));
}
case "union": {
const def = lowerer.unions.get(t.unionId);
return def === undefined || def.arms.some((arm) => holdsClassInstance(lowerer, arm, seen));
}
default:
return false;
}
}
/** After a statement lowers: when it holds a fallback site's cast, mark the
* check as a fallback check and read its outcome into the flag and the
* shadow. A cast whose lowering is not a plain check keeps throwing. */
export function armFallbackSite(lowerer: Lowerer, stmt: ts.Statement, lowered: IrStmt[]): void {
// `const root = value as T`: the check is the declaration's initializer.
if (
ts.isVariableStatement(stmt) &&
lowerer.fnStack.length > 0 &&
lowerer.tierMode !== "static-only" &&
lowered.some((s) => s.kind === "varDecl" && s.init?.kind === "dynCheck")
) {
const decls = stmt.declarationList.declarations.nodes;
const decl = decls.length === 1 ? decls[0]! : undefined;
if (decl?.initializer !== undefined && ts.isIdentifier(decl.name)) {
const symbol = lowerer.checker.getSymbolAtLocation(decl.name);
const local = symbol ? lowerer.bindingIn(lowerer.ctx, symbol) : null;
const at =
local === null
? -1
: lowered.findIndex(
(s) => s.kind === "varDecl" && s.localId === local.id && s.init?.kind === "dynCheck",
);
const write = at >= 0 ? lowered[at]! : null;
const init = write?.kind === "varDecl" ? write.init : null;
if (init?.kind === "dynCheck" && fallbackCheck(lowerer, init)) {
const candidate = fallbackCandidate(lowerer, stmt);
if (candidate !== null) {
const opened = declareSiteLocals(lowerer, candidate, locOf(stmt));
if (armSite(lowerer, opened.site, lowered, local!.id))
lowered.splice(at, 0, ...opened.decls);
else closeUnarmed(lowerer, opened.site);
}
}
}
}
if (lowerer.fallbackSites.size === 0) return;
for (const site of [...lowerer.fallbackSites.values()]) {
if (site.boundaryStmt !== stmt || site.armed) continue;
lowerer.fallbackSites.delete(site.root);
const id = lowerer.bindingIn(lowerer.ctx, site.root)?.id;
if (id === undefined || !armSite(lowerer, site, lowered, id)) closeUnarmed(lowerer, site);
}
}
function armSite(lowerer: Lowerer, site: FallbackSite, lowered: IrStmt[], id: string): boolean {
const at = lowered.findIndex(
(s) =>
(s.kind === "varDecl" && s.localId === id && s.init?.kind === "dynCheck") ||
(s.kind === "assign" && s.localId === id && s.value.kind === "dynCheck"),
);
if (at < 0) return false;
const write = lowered[at]!;
let check: IrExpr | null = null;
if (write.kind === "varDecl") check = write.init;
else if (write.kind === "assign") check = write.value;
if (check === null || check.kind !== "dynCheck" || !fallbackCheck(lowerer, check)) return false;
// Every tail must split: each split point's list is lowering whole.
const whole = [...site.anchors].every((anchor) => {
const real = listOf(anchor.parent!);
const index = real.indexOf(anchor);
return lowerer.activeStmtLists.some(
(entry) =>
entry.stmts[entry.index] === anchor &&
entry.stmts.length - entry.index === real.length - index,
);
});
if (!whole) return false;
// Splits around this site (sequential casts each split the code after
// them) and sites sharing a split point multiply the lowerings.
let nested = lowerer.fallbackSplitDepth;
for (const other of lowerer.fallbackArmed) {
for (const a of other.anchors) if (site.anchors.has(a)) nested++;
}
if (nested >= MAX_NESTED_SPLITS) return false;
check.fallback = true;
const loc = check.loc;
lowered.splice(
at + 1,
0,
{
kind: "assign",
localId: site.flag.id,
value: { kind: "libCall", fn: "checkedCast.fallbackTaken", args: [], type: BOOL, loc },
loc,
},
{
kind: "if",
cond: { kind: "varRef", localId: site.flag.id, type: BOOL, loc },
then: [
{
kind: "assign",
localId: site.shadow.id,
value: { kind: "libCall", fn: "checkedCast.fallbackTake", args: [], type: DYN, loc },
loc,
},
],
else_: null,
loc,
},
);
site.armed = true;
lowerer.fallbackArmed.push(site);
return true;
}
/** Armed sites whose tail starts after `stmt`, consumed. */
export function fallbackSplitsAt(lowerer: Lowerer, stmt: ts.Statement): FallbackSite[] {
if (lowerer.fallbackArmed.length === 0) return [];
const here: FallbackSite[] = [];
for (const site of lowerer.fallbackArmed) {
if (site.anchors.delete(stmt)) here.push(site);
}
if (here.length > 0)
lowerer.fallbackArmed = lowerer.fallbackArmed.filter((s) => s.anchors.size > 0);
return here;
}
/** The tail after a split point: `if (flag) <dynamic> else <static>` per
* site. The static lowering runs first and is exactly the lowering the
* tail had without a site. */
export function lowerFallbackTail(
lowerer: Lowerer,
rest: readonly ts.Statement[],
sites: readonly FallbackSite[],
): IrStmt[] {
if (sites.length === 0) return lowerer.lowerStmts(rest);
// An empty tail (the cast ends its block) needs no split.
if (rest.length === 0) return [];
const [site, ...others] = sites as [FallbackSite, ...FallbackSite[]];
const at = locOf(rest.length > 0 ? rest[0]! : site.boundaryStmt);
let statics: IrStmt[];
let dynamic: IrStmt[];
lowerer.fallbackSplitDepth++;
try {
statics = scoped(lowerer, () => lowerFallbackTail(lowerer, rest, others));
dynamic = lowerDynamicTail(lowerer, rest, site, others);
} finally {
lowerer.fallbackSplitDepth--;
}
return [
{
kind: "if",
cond: { kind: "varRef", localId: site.flag.id, type: BOOL, loc: at },
then: dynamic,
else_: statics,
loc: at,
},
];
}
function scoped<T>(lowerer: Lowerer, lower: () => T, frame = new Map<ts.Symbol, IrLocal>()): T {
lowerer.scopes.push(frame);
try {
return lower();
} finally {
lowerer.scopes.pop();
}
}
function lowerDynamicTail(
lowerer: Lowerer,
rest: readonly ts.Statement[],
site: FallbackSite,
others: readonly FallbackSite[],
): IrStmt[] {
const saved = {
taint: lowerer.fallbackTaint,
sink: lowerer.diagSink,
stats: lowerer.suppressStats,
sites: new Map(lowerer.fallbackSites),
armed: lowerer.fallbackArmed,
scopes: lowerer.scopes.length,
};
lowerer.fallbackTaint = fallbackTaintOf(lowerer, saved.taint, site.root, rest);
lowerer.diagSink = [];
lowerer.suppressStats = true;
lowerer.fallbackArmed = [];
try {
return scoped(
lowerer,
() => lowerFallbackTail(lowerer, rest, others),
new Map([[site.root, site.shadow]]),
);
} catch (error) {
if (!(error instanceof PoisonError) && process.env.SCRIPTC_FALLBACK_DEBUG) throw error;
while (lowerer.scopes.length > saved.scopes) lowerer.scopes.pop();
const loc = rest.length > 0 ? locOf(rest[0]!) : locOf(site.boundaryStmt);
return [
{
kind: "runtimeFence",
code: "SC1090",
message: `a value that failed its checked cast reached code with no dynamic lowering`,
loc,
},
];
} finally {
lowerer.fallbackTaint = saved.taint;
lowerer.diagSink = saved.sink;
lowerer.suppressStats = saved.stats;
lowerer.fallbackSites = saved.sites;
lowerer.fallbackArmed = saved.armed;
}
}
/* ── taint: which expressions of a dynamic tail read a failed value ── */
/** True when `node`'s value derives from a failed checked value in the
* dynamic lowering being built: it is typed `any` there. */
export function fallbackTainted(lowerer: Lowerer, node: ts.Node): boolean {
const taint = lowerer.fallbackTaint;
if (taint === null) return false;
const known = taint.memo.get(node);
if (known !== undefined) return known;
taint.memo.set(node, false);
const result = computeTainted(lowerer, taint, node);
taint.memo.set(node, result);
return result;
}
function computeTainted(lowerer: Lowerer, taint: FallbackTaint, node: ts.Node): boolean {
const t = (n: ts.Node): boolean => fallbackTainted(lowerer, n);
if (ts.isIdentifier(node)) {
const sym = lowerer.resolveValueSymbol(node);
return sym !== null && taint.symbols.has(sym);
}
if (
ts.isParenthesizedExpression(node) ||
ts.isNonNullExpression(node) ||
ts.isAsExpression(node) ||
ts.isTypeAssertion(node) ||
ts.isSatisfiesExpression(node) ||
ts.isAwaitExpression(node)
)
return t(node.expression);
if (ts.isPropertyAccessExpression(node) || ts.isElementAccessExpression(node))
return t(node.expression);
if (ts.isCallExpression(node)) return t(node.expression);
if (ts.isConditionalExpression(node)) return t(node.whenTrue) || t(node.whenFalse);
if (ts.isBinaryExpression(node)) {
const k = node.operatorToken.kind;
switch (k) {
case ts.SyntaxKind.QuestionQuestionToken:
case ts.SyntaxKind.BarBarToken:
case ts.SyntaxKind.AmpersandAmpersandToken:
case ts.SyntaxKind.PlusToken:
return t(node.left) || t(node.right);
case ts.SyntaxKind.CommaToken:
case ts.SyntaxKind.EqualsToken:
return t(node.right);
case ts.SyntaxKind.PlusEqualsToken:
case ts.SyntaxKind.QuestionQuestionEqualsToken:
case ts.SyntaxKind.BarBarEqualsToken:
case ts.SyntaxKind.AmpersandAmpersandEqualsToken:
return t(node.left) || t(node.right);
default:
return false;
}
}
// A literal beside a failed value (`value ?? {}`, `ok ? value : []`)
// is built dynamically too: its contextual type is the static one.
if (ts.isObjectLiteralExpression(node) || ts.isArrayLiteralExpression(node)) {
// A literal spreading a failed value is built dynamically, and so is
// one holding a failed value that initializes a binding (the binding
// turns dynamic) or sits in such a literal. A literal flowing straight
// into a typed slot keeps its layout: each failed member is checked
// into its field, so members that fit continue.
let holds = false;
if (ts.isObjectLiteralExpression(node)) {
for (const p of node.properties.nodes) {
if (ts.isSpreadAssignment(p) && t(p.expression)) return true;
if (
(ts.isPropertyAssignment(p) && t(p.initializer)) ||
(ts.isShorthandPropertyAssignment(p) && t(p.name))
)
holds = true;
}
} else {
for (const e of node.elements.nodes)
if (t(ts.isSpreadElement(e) ? e.expression : e)) holds = true;
}
if (holds) {
// The outermost literal of a nest decides for all of it.
let top: ts.Node = node;
for (;;) {
const host: ts.Node | undefined = top.parent;
if (host !== undefined && ts.isPropertyAssignment(host) && host.parent !== undefined) {
top = host.parent;
continue;
}
if (host !== undefined && ts.isArrayLiteralExpression(host)) {
top = host;
continue;
}
break;
}
// An untyped destination (`console.log`, `JSON.stringify`) takes it
// dynamically.
const contextual = lowerer.checker.getContextualType(top);
if (
contextual !== undefined &&
(contextual.flags & (ts.TypeFlags.Any | ts.TypeFlags.Unknown)) !== 0
)
return true;
const host = top.parent;
if (host !== undefined && ts.isVariableDeclaration(host) && host.initializer === top)
return true;
}
let child: ts.Node = node;
let parent = node.parent;
while (parent !== undefined && ts.isParenthesizedExpression(parent)) {
child = parent;
parent = parent.parent;
}
if (parent === undefined) return false;
if (ts.isBinaryExpression(parent)) {
const k = parent.operatorToken.kind;
if (
k === ts.SyntaxKind.QuestionQuestionToken ||
k === ts.SyntaxKind.BarBarToken ||
k === ts.SyntaxKind.AmpersandAmpersandToken
)
return t(parent.left === child ? parent.right : parent.left);
}
if (ts.isConditionalExpression(parent) && parent.condition !== child)
return t(parent.whenTrue === child ? parent.whenFalse : parent.whenTrue);
return false;
}
if (ts.isPrefixUnaryExpression(node) || ts.isPostfixUnaryExpression(node)) {
if (
node.operator === ts.SyntaxKind.PlusPlusToken ||
node.operator === ts.SyntaxKind.MinusMinusToken
)
return t(node.operand);
return false;
}
return false;
}
/** The taint of a dynamic tail: the root, and every binding declared in the
* tail that a tainted value is written to (a fixpoint over the tail,
* nested functions included). Parameters of callbacks passed to a tainted
* callee are tainted too: the dynamic call passes them dynamic values. */
function fallbackTaintOf(
lowerer: Lowerer,
outer: FallbackTaint | null,
root: ts.Symbol,
region: readonly ts.Statement[],
): FallbackTaint {
const symbols = new Set(outer?.symbols ?? []);
symbols.add(root);
const taint: FallbackTaint = { symbols, memo: new Map() };
const saved = lowerer.fallbackTaint;
lowerer.fallbackTaint = taint;
try {
growTaint(lowerer, taint, region);
} finally {
lowerer.fallbackTaint = saved;
}
return taint;
}
function growTaint(lowerer: Lowerer, taint: FallbackTaint, region: readonly ts.Statement[]): void {
const symbols = taint.symbols;
const declared = new Set<ts.Symbol>();
const collectDeclared = (n: ts.Node): void => {
if (
(ts.isVariableDeclaration(n) || ts.isParameter(n) || ts.isBindingElement(n)) &&
ts.isIdentifier(n.name)
) {
const s = lowerer.checker.getSymbolAtLocation(n.name);
if (s) declared.add(s);
}
ts.forEachChild(n, collectDeclared);
};
for (const s of region) collectDeclared(s);
const bindNames = (name: ts.BindingName): boolean => {
let changed = false;
const visit = (n: ts.Node): void => {
if (ts.isIdentifier(n)) {
const s = lowerer.checker.getSymbolAtLocation(n);
if (s && !symbols.has(s)) {
symbols.add(s);
changed = true;
}
return;
}
if (ts.isBindingElement(n)) {
if (n.name !== undefined) visit(n.name);
return;
}
ts.forEachChild(n, visit);
};
visit(name);
return changed;
};
for (let round = 0; round < 32; round++) {
let changed = false;
taint.memo.clear();
const visit = (n: ts.Node): void => {
if (
ts.isVariableDeclaration(n) &&
n.initializer !== undefined &&
fallbackTainted(lowerer, n.initializer)
) {
if (bindNames(n.name)) changed = true;
} else if (ts.isBinaryExpression(n)) {
const k = n.operatorToken.kind;
if (
k >= ts.SyntaxKind.FirstAssignment &&
k <= ts.SyntaxKind.LastAssignment &&
ts.isIdentifier(strip(n.left))
) {
const s = lowerer.resolveValueSymbol(strip(n.left) as ts.Identifier);
if (s && declared.has(s) && !symbols.has(s) && fallbackTainted(lowerer, n.right)) {
symbols.add(s);
changed = true;
}
}
} else if (
ts.isForOfStatement(n) &&
ts.isVariableDeclarationList(n.initializer) &&
fallbackTainted(lowerer, n.expression)
) {
for (const d of n.initializer.declarations.nodes) if (bindNames(d.name)) changed = true;
} else if (ts.isCallExpression(n) && fallbackTainted(lowerer, n.expression)) {
for (const arg of n.arguments.nodes) {
const a = strip(arg);
if (ts.isArrowFunction(a) || ts.isFunctionExpression(a)) {
for (const p of a.parameters.nodes) if (bindNames(p.name)) changed = true;
}
}
}
ts.forEachChild(n, visit);
};
for (const s of region) visit(s);
if (!changed) break;
}
taint.memo.clear();
}
packages/compiler/src/frontend/lowering/coercions/function-adapters.ts
+111 -81
View File
@@ -13,6 +13,7 @@ import {
UNDEFINED_T,
} from "../../../ir/ir.js";
import { typeKey } from "../../type-mapper.js";
import { newFnCtx } from "../function-context.js";
import type { Lowerer } from "../lowerer.js";
import { maybePromiseUnionOf } from "../maybe-promise.js";
@@ -292,6 +293,10 @@ export function funcCoerceAdapter(
const actual = toT.params[i]!;
const expected = fromT.params[i]!;
if (lowerer.coercibleValue(actual, expected)) continue;
// A record argument the parameter's shape is a structural width of
// (members missing from the argument are optional there): the
// record width conversion.
if (lowerer.widthLiftPlan(actual, expected)) continue;
if (actual.kind === "union" && expected.kind === "dyn") {
const arms =
lowerer.unions.get(actual.unionId)?.arms.filter((arm) => lowerer.dynConvertible(arm)) ?? [];
@@ -355,6 +360,19 @@ export function funcCoerceAdapter(
(id) => lowerer.unions.get(id),
);
if (checkedRet) strandRet = false;
// A Promise result whose payload widens into the slot's (a record with
// extra, optional or nested members, `Promise<Narrow>` into
// `Promise<Wide>`): the result keeps its identity and converts its
// payload when it settles (coerceToExpected's boxed-settlement rule).
if (
strandRet &&
fromT.ret.kind === "promise" &&
toT.ret.kind === "promise" &&
lowerer.promisePayloadWidens(fromT.ret, toT.ret)
)
strandRet = false;
// A record result the slot's result is a structural width of.
if (strandRet && lowerer.widthLiftPlan(fromT.ret, toT.ret)) strandRet = false;
const key = `fnadapt:${typeKey(fromT)}:${typeKey(toT)}`;
const existing = lowerer.coercions.retags.get(key);
if (existing) return existing;
@@ -377,95 +395,107 @@ export function funcCoerceAdapter(
},
loc,
});
// The body lowers in the adapter's own function context: conversions
// that need scratch locals (a union of record results lifted into one
// record slot) declare them in the adapter, not in the function that
// asked for it.
const context = newFnCtx(false, null, null, toT.ret);
let body: IrStmt[];
if (strandParams) {
body = [
strandThrow(
`a '${lowerer.fmt(fromT)}' function invoked through a '${lowerer.fmt(toT)}' slot (the parameter types cannot convert — the checker's loose function compatibility admitted the assignment, but the call has no exact lowering)`,
),
];
} else {
const args = fromT.params.map((pt, i) => {
const aRef: IrExpr = { kind: "varRef", localId: `a.${i}`, type: toT.params[i]!, loc };
const partial = partialDynParams.get(i);
if (partial && aRef.type.kind === "union") {
const source = lowerer.unions.get(aRef.type.unionId)!;
const supported =
partial.kind === "union" ? lowerer.unions.get(partial.unionId)!.arms : [partial];
const rejected = new Set(
source.arms.flatMap((arm, tag) =>
supported.some((accepted) => typeEquals(accepted, arm)) ? [] : [tag],
),
);
const narrow =
partial.kind === "union"
? lowerer.unionRetagHelper(aRef.type.unionId, partial.unionId, loc, rejected)
: lowerer.narrowedArmHelper(aRef.type.unionId, partial, loc);
if (!narrow)
throw new InternalCompilerError(
"lowerer bug: partial callable parameter stopped narrowing",
);
return lowerer.coerceToExpected(
{ kind: "call", callee: narrow, args: [aRef], type: partial, loc },
pt,
);
}
const narrowed = narrowedParams.get(i);
const helper =
narrowed && aRef.type.kind === "union" && pt.kind === "union"
? lowerer.unionRetagHelper(aRef.type.unionId, pt.unionId, loc, narrowed)
: null;
const converted: IrExpr = helper
? { kind: "call", callee: helper, args: [aRef], type: pt, loc }
: lowerer.coerceToExpected(aRef, pt);
if (!typeEquals(converted.type, pt))
throw new InternalCompilerError("lowerer bug: probed fn-adapter param stopped coercing");
return converted;
});
const call: IrExpr = {
kind: "callValue",
callee: { kind: "varRef", localId: "f.0", type: fromT, loc },
receiver: { kind: "libCall", fn: "dyn.this", args: [], type: DYN, loc },
args,
type: fromT.ret,
loc,
};
if (toT.ret.kind === "void") {
lowerer.fnStack.push(context);
try {
if (strandParams) {
body = [
{ kind: "exprStmt", expr: call, loc },
{ kind: "return", value: null, loc },
];
} else if (voidRet === "dyn") {
body = [
{ kind: "exprStmt", expr: call, loc },
{ kind: "return", value: dynUndefinedExpr(loc), loc },
];
} else if (voidRet === "strand") {
body = [
{ kind: "exprStmt", expr: call, loc },
strandThrow(
`a void result where the '${lowerer.fmt(toT)}' slot promises '${lowerer.fmt(toT.ret)}' (a thrower typed 'never' never reaches this; a genuinely void function has no result to hand over)`,
`a '${lowerer.fmt(fromT)}' function invoked through a '${lowerer.fmt(toT)}' slot (the parameter types cannot convert — the checker's loose function compatibility admitted the assignment, but the call has no exact lowering)`,
),
];
} else if (strandRet) {
body = [
{ kind: "exprStmt", expr: call, loc },
strandThrow(
`a '${lowerer.fmt(fromT)}' function invoked through a '${lowerer.fmt(toT)}' slot (the result cannot convert to '${lowerer.fmt(toT.ret)}' — the checker's loose function compatibility admitted the assignment, but the call has no exact lowering)`,
),
];
} else if (checkedRet) {
body = [
{ kind: "return", value: { kind: "dynCheck", value: call, type: toT.ret, loc }, loc },
];
} else {
const result = lowerer.coerceToExpected(call, toT.ret);
if (!typeEquals(result.type, toT.ret))
throw new InternalCompilerError("lowerer bug: probed fn-adapter return stopped coercing");
body = [{ kind: "return", value: result, loc }];
const args = fromT.params.map((pt, i) => {
const aRef: IrExpr = { kind: "varRef", localId: `a.${i}`, type: toT.params[i]!, loc };
const partial = partialDynParams.get(i);
if (partial && aRef.type.kind === "union") {
const source = lowerer.unions.get(aRef.type.unionId)!;
const supported =
partial.kind === "union" ? lowerer.unions.get(partial.unionId)!.arms : [partial];
const rejected = new Set(
source.arms.flatMap((arm, tag) =>
supported.some((accepted) => typeEquals(accepted, arm)) ? [] : [tag],
),
);
const narrow =
partial.kind === "union"
? lowerer.unionRetagHelper(aRef.type.unionId, partial.unionId, loc, rejected)
: lowerer.narrowedArmHelper(aRef.type.unionId, partial, loc);
if (!narrow)
throw new InternalCompilerError(
"lowerer bug: partial callable parameter stopped narrowing",
);
return lowerer.coerceToExpected(
{ kind: "call", callee: narrow, args: [aRef], type: partial, loc },
pt,
);
}
const narrowed = narrowedParams.get(i);
const helper =
narrowed && aRef.type.kind === "union" && pt.kind === "union"
? lowerer.unionRetagHelper(aRef.type.unionId, pt.unionId, loc, narrowed)
: null;
const converted: IrExpr = helper
? { kind: "call", callee: helper, args: [aRef], type: pt, loc }
: lowerer.coerceToExpected(aRef, pt);
if (!typeEquals(converted.type, pt))
throw new InternalCompilerError("lowerer bug: probed fn-adapter param stopped coercing");
return converted;
});
const call: IrExpr = {
kind: "callValue",
callee: { kind: "varRef", localId: "f.0", type: fromT, loc },
receiver: { kind: "libCall", fn: "dyn.this", args: [], type: DYN, loc },
args,
type: fromT.ret,
loc,
};
if (toT.ret.kind === "void") {
body = [
{ kind: "exprStmt", expr: call, loc },
{ kind: "return", value: null, loc },
];
} else if (voidRet === "dyn") {
body = [
{ kind: "exprStmt", expr: call, loc },
{ kind: "return", value: dynUndefinedExpr(loc), loc },
];
} else if (voidRet === "strand") {
body = [
{ kind: "exprStmt", expr: call, loc },
strandThrow(
`a void result where the '${lowerer.fmt(toT)}' slot promises '${lowerer.fmt(toT.ret)}' (a thrower typed 'never' never reaches this; a genuinely void function has no result to hand over)`,
),
];
} else if (strandRet) {
body = [
{ kind: "exprStmt", expr: call, loc },
strandThrow(
`a '${lowerer.fmt(fromT)}' function invoked through a '${lowerer.fmt(toT)}' slot (the result cannot convert to '${lowerer.fmt(toT.ret)}' — the checker's loose function compatibility admitted the assignment, but the call has no exact lowering)`,
),
];
} else if (checkedRet) {
body = [
{ kind: "return", value: { kind: "dynCheck", value: call, type: toT.ret, loc }, loc },
];
} else {
const result = lowerer.coerceToExpected(call, toT.ret);
if (!typeEquals(result.type, toT.ret))
throw new InternalCompilerError("lowerer bug: probed fn-adapter return stopped coercing");
body = [{ kind: "return", value: result, loc }];
}
}
} finally {
lowerer.fnStack.pop();
}
lowerer.liftedFns.push(...buildFunctionAdapter(name, fromT, toT, params, [], body, loc));
lowerer.liftedFns.push(
...buildFunctionAdapter(name, fromT, toT, params, context.locals, body, loc),
);
return name;
}
packages/compiler/src/frontend/lowering/containers/array-callbacks.ts
+27 -17
View File
@@ -28,9 +28,12 @@ import {
import { isJsSourceFile, locOf } from "../../program.js";
import {
type SortKeyOrder,
type SortKeyRead,
buildArraySortFn,
buildSortKeyFn,
closureKeyOrder,
sortKeysId,
sortKeysOrder,
} from "../lower-array-sort.js";
import {
arrayIndexPresent,
@@ -1155,7 +1158,7 @@ function findResultRetag(
/** The find-like predicate results whose ToBoolean has a static answer:
* bool passes through; f64/string by value; checked values and
* truthy-answerable unions by their runtime kind or arm. */
function findPredicateOk(lowerer: Lowerer, fnRet: IrType): boolean {
export function findPredicateOk(lowerer: Lowerer, fnRet: IrType): boolean {
return (
fnRet.kind === "bool" ||
fnRet.kind === "f64" ||
@@ -1171,7 +1174,7 @@ function findPredicateOk(lowerer: Lowerer, fnRet: IrType): boolean {
/** ToBoolean of a predicate result inside a synthesized HOF helper — the
* ensureBool subset the find-like path admits (bool through; checked
* values via dynTest; f64/string/truthy-answerable unions via toBool). */
function predicateToBoolean(e: IrExpr): IrExpr {
export function predicateToBoolean(e: IrExpr): IrExpr {
if (e.type.kind === "bool") return e;
if (e.type.kind === "dyn")
return { kind: "dynTest", test: "truthy", value: e, type: BOOL, loc: e.loc };
@@ -2112,8 +2115,10 @@ export function lowerArraySortCall(
elem,
elem.kind === "union" ? lowerer.unions.get(elem.unionId)?.arms : undefined,
);
const keyed = native ? recognizedSortKey(lowerer, fnArg, elem) : null;
const key = `${method}:${typeKey(elem)}:${arity}${native ? ":native" : ""}${keyed ? `:keyed${keyed.order}${keyed.read ? ":field" : ""}` : ""}`;
const keyed = native ? recognizedSortKeys(lowerer, fnArg, elem) : null;
const reads = keyed?.every((k) => k.read !== null) ? keyed.map((k) => k.read!) : null;
const order = keyed ? sortKeysOrder(keyed) : 0;
const key = `${method}:${typeKey(elem)}:${arity}${native ? ":native" : ""}${keyed ? `:keyed${order}${reads ? `:field${reads.length}` : ""}` : ""}`;
let helper = lowerer.arrHofHelpers.get(key);
if (!helper) {
helper = `%arr.${method}.${lowerer.arrHofHelpers.size}`;
@@ -2128,41 +2133,46 @@ export function lowerArraySortCall(
undefinedTag >= 0 ? undefinedTag : null,
loc,
native,
keyed && { order: keyed.order, key: keyed.read !== null },
keyed && { order, key: reads !== null, keys: keyed.length },
),
);
}
const args = [receiver, fnArg];
if (keyed?.read) args.push(sortKeyReader(lowerer, elem, keyed.read, loc));
if (reads) args.push(sortKeyReader(lowerer, elem, reads, loc));
return { kind: "call", callee: helper, args, type: arrT, loc };
}
/** The key order of a capture-free comparator closure the runtime can sort
* by without calling it: over number elements, or over one number field
* of record or class elements. */
function recognizedSortKey(lowerer: Lowerer, fnArg: IrExpr, elem: IrType): SortKeyOrder | null {
/** The keys of a capture-free comparator closure the runtime can sort by
* without calling it: number elements themselves, or number fields of
* record or class elements (one or several). */
function recognizedSortKeys(
lowerer: Lowerer,
fnArg: IrExpr,
elem: IrType,
): readonly SortKeyOrder[] | null {
const keyed = closureKeyOrder(lowerer.liftedFns, fnArg);
if (!keyed) return null;
if (keyed.read === null) return elem.kind === "f64" ? keyed : null;
if (keyed[0]!.read === null) return elem.kind === "f64" ? keyed : null;
return elem.kind === "record" || elem.kind === "object" ? keyed : null;
}
/** A zero-capture closure over the shared `(x) => x.field` key reader. */
/** A zero-capture closure over the shared key reader of these reads
* (buildSortKeyFn): `(x) => x.field`, or `(x, j) => key j` for several. */
function sortKeyReader(
lowerer: Lowerer,
elem: IrType,
read: NonNullable<SortKeyOrder["read"]>,
reads: readonly SortKeyRead[],
loc: SrcLoc,
): IrExpr {
const owner = read.kind === "recordGet" ? read.shapeId : read.className;
const cacheKey = `sortKey:${typeKey(elem)}:${read.kind}:${owner}:${read.field}`;
const cacheKey = `sortKey:${typeKey(elem)}:${sortKeysId(reads.map((read) => ({ order: 1, read })))}`;
let reader = lowerer.arrHofHelpers.get(cacheKey);
if (!reader) {
reader = `%arr.sortKey.${lowerer.arrHofHelpers.size}`;
lowerer.arrHofHelpers.set(cacheKey, reader);
lowerer.liftedFns.push(buildSortKeyFn(reader, elem, read, loc));
lowerer.liftedFns.push(buildSortKeyFn(reader, elem, reads, loc));
}
return { kind: "closure", fnName: reader, captures: [], type: funcOf([elem], F64), loc };
const params = reads.length > 1 ? [elem, F64] : [elem];
return { kind: "closure", fnName: reader, captures: [], type: funcOf(params, F64), loc };
}
/** `d.filter(pred)` on a dyn receiver: the receiver must BE a dyn array
packages/compiler/src/frontend/lowering/containers/bytes.ts
+376 -2
View File
@@ -1,4 +1,4 @@
import { dynUndefinedExpr, varRef } from "../../../ir/build.js";
import { boolLit, dynUndefinedExpr, numLit, varRef } from "../../../ir/build.js";
import { BUF_NUM_METHODS } from "./buffer-numeric-methods.js";
import * as ts from "../../ts7/adapter.js";
import {
@@ -12,6 +12,9 @@ import {
type IrBytesIntrinsicMethod,
type IrExpr,
type IrLibFn,
type IrLocal,
type IrParam,
type IrStmt,
type IrType,
STRING,
type SrcLoc,
@@ -38,6 +41,9 @@ import {
import { lowerCheckedArrayFrom } from "./array-construction.js";
import { lowerToNumberArgument } from "../lower-exprs.js";
import { lowerBytesIteratorValue } from "../iterator-adapters.js";
import { lowerArrayCallback } from "./callback-arguments.js";
import { findPredicateOk, predicateToBoolean } from "./array-callbacks.js";
import { typeKey } from "../../type-mapper.js";
/** TypedArray.prototype.sort and toSorted. The receiver and comparator
* expressions are evaluated before entering the helper (buildBytesSortFn).
@@ -140,7 +146,7 @@ function lowerBytesSortCall(
}
const arity = fnArg.type.params.length;
const keyed = closureKeyOrder(lowerer.liftedFns, fnArg);
const order = keyed !== null && keyed.read === null ? keyed.order : null;
const order = keyed?.length === 1 && keyed[0]!.read === null ? keyed[0]!.order : null;
const helper = helperFor(`${arity}${order === null ? "" : `:keyed${order}`}`, arity, true, order);
return {
kind: "call",
@@ -151,6 +157,371 @@ function lowerBytesSortCall(
};
}
/** The TypedArray.prototype callback methods. */
type BytesCallbackMethod =
| "forEach"
| "map"
| "filter"
| "some"
| "every"
| "find"
| "findIndex"
| "findLast"
| "findLastIndex"
| "reduce"
| "reduceRight";
const BYTES_CALLBACK_METHODS: ReadonlySet<string> = new Set<BytesCallbackMethod>([
"forEach",
"map",
"filter",
"some",
"every",
"find",
"findIndex",
"findLast",
"findLastIndex",
"reduce",
"reduceRight",
]);
/** The typed-array callback methods (forEach, map, filter, some, every,
* find, findIndex, findLast, findLastIndex, reduce, reduceRight). The
* receiver, the callback and reduce's initial value evaluate in order
* before the helper runs. The helper reads the length once (typed arrays
* have no holes) and each element when it visits it, so the callback's
* writes show in later elements. The callback receives the element, its
* index and the typed array (reduce's receive the accumulator first). map
* and filter return a new typed array of the receiver's kind; map stores
* each result with the element conversion of that kind. A thisArg keeps
* the fence, as for arrays. */
function lowerBytesCallbackCall(
lowerer: Lowerer,
call: ts.CallExpression,
access: ts.PropertyAccessExpression,
method: BytesCallbackMethod,
bytesT: IrType & { kind: "bytes" },
): IrExpr {
const loc = locOf(call);
const nArgs = call.arguments.nodes.length;
const reduce = method === "reduce" || method === "reduceRight";
if (nArgs < 1 || nArgs > (reduce ? 2 : 1) || call.arguments.nodes.some(ts.isSpreadElement)) {
lowerer.noLowering(
`.${method} with ${nArgs} arguments on ${BYTES_ELEMENT_NAME[bytesT.elem]}`,
call,
reduce ? undefined : "the thisArg parameter has no lowering — use an arrow function",
);
}
const argNode = call.arguments.nodes[0]!;
const hasInit = reduce && nArgs === 2;
const accT = reduce ? (hasInit ? lowerer.irTypeOf(call) : F64) : null;
if (accT !== null && (accT.kind === "void" || accT.kind === "func"))
lowerer.badType(call, lowerer.typeOf(call));
const receiver = lowerer.lowerExprExpecting(access.expression, bytesT);
const { fnArg, arity } = lowerArrayCallback(
lowerer,
argNode,
accT !== null ? [accT, F64] : [F64],
bytesT,
false,
accT ?? undefined,
);
const fnRet = fnArg.type.ret;
const predicate =
method === "filter" ||
method === "some" ||
method === "every" ||
method === "find" ||
method === "findIndex" ||
method === "findLast" ||
method === "findLastIndex";
if (predicate && !findPredicateOk(lowerer, fnRet))
lowerer.badType(argNode, lowerer.typeOf(argNode));
if (method === "map" && !lowerer.coercibleValue(F64, fnRet))
lowerer.badType(argNode, lowerer.typeOf(argNode));
const resultT: IrType =
accT ??
(method === "forEach"
? VOID
: method === "map" || method === "filter"
? bytesT
: method === "some" || method === "every"
? BOOL
: method === "findIndex" || method === "findLastIndex"
? F64
: lowerer.irTypeOf(call));
const helper = bytesCallbackHelper(lowerer, method, bytesT, resultT, fnRet, arity, hasInit, loc);
const args: IrExpr[] = [receiver, fnArg];
if (hasInit) args.push(lowerer.lowerExprExpecting(call.arguments.nodes[1]!, resultT));
return { kind: "call", callee: helper, args, type: resultT, loc };
}
/** The interned helper of one typed-array callback method shape. */
function bytesCallbackHelper(
lowerer: Lowerer,
method: BytesCallbackMethod,
bytesT: IrType & { kind: "bytes" },
resultT: IrType,
fnRet: IrType,
arity: number,
hasInit: boolean,
loc: SrcLoc,
): string {
const key = `bytes.${method}:${bytesT.elem}:${typeKey(resultT)}:${typeKey(fnRet)}:${arity}:${hasInit ? "init" : ""}`;
const existing = lowerer.arrHofHelpers.get(key);
if (existing) return existing;
const name = `%bytes.${method}.${lowerer.arrHofHelpers.size}`;
lowerer.arrHofHelpers.set(key, name);
const reduce = method === "reduce" || method === "reduceRight";
const backward = method === "reduceRight" || method === "findLast" || method === "findLastIndex";
const fnT = funcOf(
(reduce ? [resultT, F64, F64, bytesT] : [F64, F64, bytesT]).slice(0, arity),
fnRet,
);
const keptT = arrayOf(F64);
const a = (): IrExpr => varRef("a.0", bytesT, loc);
const i = (): IrExpr => varRef("i.0", F64, loc);
const n = (): IrExpr => varRef("n.0", F64, loc);
const acc = (): IrExpr => varRef("acc.0", resultT, loc);
const element = (): IrExpr => varRef("v.0", F64, loc);
const at = (index: IrExpr): IrExpr => ({
kind: "bytesIntrinsic",
method: "get",
receiver: a(),
args: [index],
type: F64,
loc,
});
const callback = (): IrExpr => ({
kind: "callValue",
callee: varRef("f.0", fnT, loc),
args: (reduce ? [acc(), element(), i(), a()] : [element(), i(), a()]).slice(0, arity),
type: fnRet,
loc,
});
const inRange = (): IrExpr =>
backward
? { kind: "bin", op: ">=", left: i(), right: numLit(0, loc), type: BOOL, loc }
: { kind: "bin", op: "<", left: i(), right: n(), type: BOOL, loc };
const advance = (): IrStmt => ({
kind: "assign",
localId: "i.0",
value: {
kind: "bin",
op: backward ? "-" : "+",
left: i(),
right: numLit(1, loc),
type: F64,
loc,
},
loc,
});
const ret = (value: IrExpr | null): IrStmt => ({ kind: "return", value, loc });
// map and filter of a Buffer return a Buffer (Node's species).
const copyBrand = (out: IrExpr): IrStmt => ({
kind: "exprStmt",
expr: { kind: "libCall", fn: "bytes.copyBrand", args: [out, a()], type: VOID, loc },
loc,
});
const locals: IrLocal[] = [
{ id: "a.0", name: "a", type: bytesT, mutable: true },
{ id: "f.0", name: "f", type: fnT, mutable: true },
...(hasInit ? [{ id: "z.0", name: "z", type: resultT, mutable: true } as IrLocal] : []),
...(reduce || method === "map" || method === "filter"
? [{ id: "acc.0", name: "acc", type: resultT, mutable: true } as IrLocal]
: []),
...(method === "filter"
? [{ id: "kept.0", name: "kept", type: keptT, mutable: true } as IrLocal]
: []),
{ id: "n.0", name: "n", type: F64, mutable: false },
{ id: "i.0", name: "i", type: F64, mutable: true },
{ id: "v.0", name: "v", type: F64, mutable: true },
];
const params: IrParam[] = [
{ localId: "a.0", name: "a", type: bytesT },
{ localId: "f.0", name: "f", type: fnT },
...(hasInit ? [{ localId: "z.0", name: "z", type: resultT }] : []),
];
const visit: IrStmt[] = [{ kind: "assign", localId: "v.0", value: at(i()), loc }];
const prologue: IrStmt[] = [];
let epilogue: IrStmt[];
switch (method) {
case "forEach":
visit.push({ kind: "exprStmt", expr: callback(), loc });
epilogue = [ret(null)];
break;
case "map":
prologue.push({
kind: "varDecl",
localId: "acc.0",
init: { kind: "bytesNew", source: n(), type: bytesT, loc },
loc,
});
visit.push({
kind: "bytesSet",
arr: acc(),
index: i(),
value: lowerer.coerceToExpected(callback(), F64),
loc,
});
if (bytesT.elem === "u8") prologue.push(copyBrand(acc()));
epilogue = [ret(acc())];
break;
case "filter":
prologue.push({
kind: "varDecl",
localId: "kept.0",
init: { kind: "arrayLit", elems: [], type: keptT, loc },
loc,
});
visit.push({
kind: "if",
cond: predicateToBoolean(callback()),
then: [
{
kind: "exprStmt",
expr: {
kind: "arrIntrinsic",
method: "push",
receiver: varRef("kept.0", keptT, loc),
args: [element()],
type: F64,
loc,
},
loc,
},
],
else_: null,
loc,
});
epilogue = [
{
kind: "varDecl",
localId: "acc.0",
init: { kind: "bytesNew", source: varRef("kept.0", keptT, loc), type: bytesT, loc },
loc,
},
...(bytesT.elem === "u8" ? [copyBrand(acc())] : []),
ret(acc()),
];
break;
case "some":
case "every": {
const found = method === "some";
const test = predicateToBoolean(callback());
visit.push({
kind: "if",
cond: found ? test : { kind: "unary", op: "!", operand: test, type: BOOL, loc },
then: [ret(boolLit(found, loc))],
else_: null,
loc,
});
epilogue = [ret(boolLit(!found, loc))];
break;
}
case "find":
case "findLast":
visit.push({
kind: "if",
cond: predicateToBoolean(callback()),
then: [ret(lowerer.coerceToExpected(element(), resultT))],
else_: null,
loc,
});
epilogue = [
ret(
lowerer.coerceToExpected(
{ kind: "unitLit", unit: "undefined", type: UNDEFINED_T, loc },
resultT,
),
),
];
break;
case "findIndex":
case "findLastIndex":
visit.push({
kind: "if",
cond: predicateToBoolean(callback()),
then: [ret(i())],
else_: null,
loc,
});
epilogue = [ret(numLit(-1, loc))];
break;
case "reduce":
case "reduceRight":
if (hasInit) {
prologue.push({
kind: "varDecl",
localId: "acc.0",
init: varRef("z.0", resultT, loc),
loc,
});
} else {
prologue.push(
{
kind: "if",
cond: { kind: "unary", op: "!", operand: inRange(), type: BOOL, loc },
then: [
{
kind: "throw",
value: {
kind: "libCall",
fn: "error.new",
args: [
{
kind: "strLit",
value: "Reduce of empty array with no initial value",
type: STRING,
loc,
},
],
type: { kind: "object", className: "%TypeError" },
loc,
},
loc,
},
],
else_: null,
loc,
},
{ kind: "varDecl", localId: "acc.0", init: at(i()), loc },
advance(),
);
}
visit.push({
kind: "assign",
localId: "acc.0",
value: lowerer.coerceToExpected(callback(), resultT),
loc,
});
epilogue = [ret(acc())];
break;
}
const body: IrStmt[] = [
{
kind: "varDecl",
localId: "n.0",
init: { kind: "bytesIntrinsic", method: "length", receiver: a(), args: [], type: F64, loc },
loc,
},
{
kind: "varDecl",
localId: "i.0",
init: backward
? { kind: "bin", op: "-", left: n(), right: numLit(1, loc), type: F64, loc }
: numLit(0, loc),
loc,
},
{ kind: "varDecl", localId: "v.0", init: numLit(0, loc), loc },
...prologue,
{ kind: "for", init: null, cond: inRange(), update: advance(), body: visit, loc },
...epilogue,
];
lowerer.liftedFns.push({ name, params, returnType: resultT, locals, body, loc });
return name;
}
/* ── typed arrays / Buffer ─────────────────────────────────────────────── */
/** The typed-array constructors with a runtime representation, by lib
@@ -466,6 +837,9 @@ export function lowerBytesMethodCall(
if (name === "sort" || name === "toSorted") {
return lowerBytesSortCall(lowerer, call, access, receiverIr);
}
if (BYTES_CALLBACK_METHODS.has(name)) {
return lowerBytesCallbackCall(lowerer, call, access, name as BytesCallbackMethod, receiverIr);
}
// values()/keys()/entries() as iterator objects (for...of heads and
// stored numeric iterators lower before this).
if ((name === "values" || name === "keys" || name === "entries") && nArgs === 0) {

Some files were not shown because too many files have changed in this diff Show More